* RE: Low memory killer problem
From: zhiyuan_zhu @ 2017-05-17 2:47 UTC (permalink / raw)
To: labbott, gregkh; +Cc: vinmenon, linux-mm, skhiani, torvalds, Jet_Li, zzyjsjcom
In-Reply-To: <1f0815e5-5cb7-81a4-24c8-b0608ef2684a@redhat.com>
Thanks Laura,
Indeed, some part of ION memory have registered shrinker.
/proc/meminfo have no account for the memory which ION can be shrinked part, but just have IonTotal and IonInUse.
And another problem is: ION memory is special, maybe hard to reclaim.
So, wating for Google's solution now.
By the way, which E-mail client do you use? :)
Thanks
Zhiyuan zhu
-----Original Message-----
From: Laura Abbott [mailto:labbott@redhat.com]
Sent: Wednesday, May 17, 2017 6:37 AM
To: Zhiyuan Zhu(朱志遠); gregkh@linuxfoundation.org
Cc: vinmenon@codeaurora.org; linux-mm@kvack.org; skhiani@codeaurora.org; torvalds@linux-foundation.org; Jet Li(李發傑)
Subject: Re: Low memory killer problem
On 05/15/2017 08:41 PM, zhiyuan_zhu@htc.com wrote:
> Thanks for your remind,
> I found lowmemorykiller.c have been removed, and ION module still exist since v4.12-rc1.
> I will pay attention to ION module.
>
> But I still have 3 questions,
> Is there any substitute for low-memory-killer after kernel v4.12-rc1 ?
> Can I accounted the ION free to free memory?
> Is there any different from ION free and the normal system memory free?
>
> ION free means: IonTotal - IonInUse - ION reserved memory.
> Thanks a lot.
>
Issues like this are exactly why the LMK was deleted. The problem is the LMK is hooked up as a shrinker so it runs in parallel to any other shrinker. The short answer is yes if you want the LMK to do anything reasonable you probably need to tweak it to account for other memory that may be held in the system (Ion, zswap etc.).
There never seemed to be one universal heuristic that worked for everyone which was part of the reason why most changes exist downstream.
Using some combination of the Ion variables above would work if you experiment. If this sounds like a non-answer, that's because it is.
Thanks,
Laura
> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Monday, May 15, 2017 5:00 PM
> To: Zhiyuan Zhu(朱志遠)
> Cc: vinmenon@codeaurora.org; linux-mm@kvack.org;
> skhiani@codeaurora.org; torvalds@linux-foundation.org; Jet Li(李發傑)
> Subject: Re: Low memory killer problem
>
> On Mon, May 15, 2017 at 08:22:38AM +0000, zhiyuan_zhu@htc.com wrote:
>> Dear Greg,
>>
>> Very sorry my mail history is lost.
>>
>> I found a part of ION memory will be return to system in android
>> platform, But these memorys can’t accounted in low-memory-killer strategy.
>> …
>> And I also found ION memory comes from, kmalloc/vmalloc/alloc pages/reserved memory.
>> I understand reserved memory shouldn't accounted to free memory.
>> But the memory which alloced by kmalloc/vmalloc/alloc pages, can be reclaimed.
>>
>> But the low-memory killer can't accounted this part, Many thanks.
>>
>> Code location,
>> ---> drivers/staging/android/lowmemorykiller.c -> lowmem_scan
>
> That file is gone from the latest kernel release, sorry. So there's not much we can do about this code anymore.
>
> See the mailing list archives for what should be used instead of this code, there is a plan for what to do.
>
> Also note that the ION code has had a lot of reworks lately as well.
>
> good luck!
>
> greg k-h
>
> N r zǧu Ơ{\b 칻\x1c &ޖ) i ^n r ݢj$ $ \x05 ~ '.) ,y m
>
% { j+ צj)Z \x02f \x1d {d $ \x1e /a==
>
^ permalink raw reply
* Re: [Patch v2] mm/vmscan: fix unsequenced modification and access warning
From: Nick Desaulniers @ 2017-05-17 3:01 UTC (permalink / raw)
To: Michal Hocko
Cc: akpm, hannes, mgorman, vbabka, minchan, linux-mm, linux-kernel
In-Reply-To: <20170516082746.GA2481@dhcp22.suse.cz>
> I guess it is worth reporting this to clang bugzilla. Could you take
> care of that Nick?
Done: https://bugs.llvm.org//show_bug.cgi?id=33065
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v4] arm64: fix the overlap between the kernel image and vmalloc address
From: zhong jiang @ 2017-05-17 3:17 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Catalin Marinas, Mark Rutland, linux-mm@kvack.org, Tanxiaojun,
Laura Abbott, linux-arm-kernel@lists.infradead.org
In-Reply-To: <CAKv+Gu8xPKhSq=s2PvNQaJLBy7jEhjnADyjsSkOPc4E36VxX1w@mail.gmail.com>
Hi, Ard
Thank you for reply.
On 2017/5/16 23:03, Ard Biesheuvel wrote:
> On 16 May 2017 at 15:15, zhongjiang <zhongjiang@huawei.com> wrote:
>> Recently, xiaojun report the following issue.
>>
>> [ 4544.984139] Unable to handle kernel paging request at virtual address ffff804392800000
>> [ 4544.991995] pgd = ffff80096745f000
>> [ 4544.995369] [ffff804392800000] *pgd=0000000000000000
>> [ 4545.000297] Internal error: Oops: 96000005 [#1] PREEMPT SMP
>> [ 4545.005815] Modules linked in:
>> [ 4545.008843] CPU: 1 PID: 8976 Comm: cat Not tainted 4.11.0-rc6 #1
>> [ 4545.014790] Hardware name: ARM Juno development board (r1) (DT)
>> [ 4545.020653] task: ffff8009753fdb00 task.stack: ffff80097533c000
>> [ 4545.026520] PC is at __memcpy+0x100/0x180
>> [ 4545.030491] LR is at vread+0x144/0x280
>> [ 4545.034202] pc : [<ffff0000083a1000>] lr : [<ffff0000081c126c>] pstate: 20000145
>> [ 4545.041530] sp : ffff80097533fcb0
>> [ 4545.044811] x29: ffff80097533fcb0 x28: ffff800962d24000
>> [ 4545.050074] x27: 0000000000001000 x26: ffff8009753fdb00
>> [ 4545.055337] x25: ffff000008200000 x24: ffff800977801380
>> [ 4545.060600] x23: ffff8009753fdb00 x22: ffff800962d24000
>> [ 4545.065863] x21: 0000000000001000 x20: ffff000008200000
>> [ 4545.071125] x19: 0000000000001000 x18: 0000ffffefa323c0
>> [ 4545.076387] x17: 0000ffffa9c87440 x16: ffff0000081fdfd0
>> [ 4545.081649] x15: 0000ffffa9d01588 x14: 72a77346b2407be7
>> [ 4545.086911] x13: 5299400690000000 x12: b0000001f9001a79
>> [ 4545.092173] x11: 97fc098d91042260 x10: 0000000000000000
>> [ 4545.097435] x9 : 0000000000000000 x8 : 9110626091260021
>> [ 4545.102698] x7 : 0000000000001000 x6 : ffff800962d24000
>> [ 4545.107960] x5 : ffff8009778013b0 x4 : 0000000000000000
>> [ 4545.113222] x3 : 0400000000000001 x2 : 0000000000000f80
>> [ 4545.118484] x1 : ffff804392800000 x0 : ffff800962d24000
>> [ 4545.123745]
>> [ 4545.125220] Process cat (pid: 8976, stack limit = 0xffff80097533c000)
>> [ 4545.131598] Stack: (0xffff80097533fcb0 to 0xffff800975340000)
>> [ 4545.137289] fca0: ffff80097533fd30 ffff000008270f64
>> [ 4545.145049] fcc0: 000000000000e000 000000003956f000 ffff000008f950d0 ffff80097533feb8
>> [ 4545.152809] fce0: 0000000000002000 ffff8009753fdb00 ffff800962d24000 ffff000008e8d3d8
>> [ 4545.160568] fd00: 0000000000001000 ffff000008200000 0000000000001000 ffff800962d24000
>> [ 4545.168327] fd20: 0000000000001000 ffff000008e884a0 ffff80097533fdb0 ffff00000826340c
>> [ 4545.176086] fd40: ffff800976bf2800 fffffffffffffffb 000000003956d000 ffff80097533feb8
>> [ 4545.183846] fd60: 0000000060000000 0000000000000015 0000000000000124 000000000000003f
>> [ 4545.191605] fd80: ffff000008962000 ffff8009753fdb00 ffff8009753fdb00 ffff8009753fdb00
>> [ 4545.199364] fda0: 0000000300000124 0000000000002000 ffff80097533fdd0 ffff0000081fb83c
>> [ 4545.207123] fdc0: 0000000000010000 ffff80097514f900 ffff80097533fe50 ffff0000081fcb28
>> [ 4545.214883] fde0: 0000000000010000 ffff80097514f900 0000000000000000 0000000000000000
>> [ 4545.222642] fe00: ffff80097533fe30 ffff0000081fca1c ffff80097514f900 0000000000000000
>> [ 4545.230401] fe20: 000000003956d000 ffff80097533feb8 ffff80097533fe50 ffff0000081fcb04
>> [ 4545.238160] fe40: 0000000000010000 ffff80097514f900 ffff80097533fe80 ffff0000081fe014
>> [ 4545.245919] fe60: ffff80097514f900 ffff80097514f900 000000003956d000 0000000000010000
>> [ 4545.253678] fe80: 0000000000000000 ffff000008082f30 0000000000000000 0000800977146000
>> [ 4545.261438] fea0: ffffffffffffffff 0000ffffa9c8745c 0000000000000124 0000000008202000
>> [ 4545.269197] fec0: 0000000000000003 000000003956d000 0000000000010000 0000000000000000
>> [ 4545.276956] fee0: 0000000000011011 0000000000000001 0000000000000011 0000000000000002
>> [ 4545.284715] ff00: 000000000000003f 1f3c201f7372686b 00000000ffffffff 0000000000000030
>> [ 4545.292474] ff20: 0000000000000038 0000000000000000 0000ffffa9bcca94 0000ffffa9d01588
>> [ 4545.300233] ff40: 0000000000000000 0000ffffa9c87440 0000ffffefa323c0 0000000000010000
>> [ 4545.307993] ff60: 000000000041a310 000000003956d000 0000000000000003 000000007fffe000
>> [ 4545.315751] ff80: 00000000004088d0 0000000000010000 0000000000000000 0000000000000000
>> [ 4545.323511] ffa0: 0000000000010000 0000ffffefa32690 0000000000404dcc 0000ffffefa32690
>> [ 4545.331270] ffc0: 0000ffffa9c8745c 0000000060000000 0000000000000003 000000000000003f
>> [ 4545.339029] ffe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 4545.346786] Call trace:
>> [ 4545.349207] Exception stack(0xffff80097533fae0 to 0xffff80097533fc10)
>> [ 4545.355586] fae0: 0000000000001000 0001000000000000 ffff80097533fcb0 ffff0000083a1000
>> [ 4545.363345] fb00: 000000003957c000 ffff80097533fc00 0000000020000145 0000000000000025
>> [ 4545.371105] fb20: ffff800962d24000 ffff000008e8d3d8 0000000000001000 ffff8009753fdb00
>> [ 4545.378864] fb40: 0000000000000000 0000000000000002 ffff80097533fd30 ffff000008082604
>> [ 4545.386623] fb60: 0000000000001000 0001000000000000 ffff80097533fd30 ffff0000083a0a90
>> [ 4545.394382] fb80: ffff800962d24000 ffff804392800000 0000000000000f80 0400000000000001
>> [ 4545.402140] fba0: 0000000000000000 ffff8009778013b0 ffff800962d24000 0000000000001000
>> [ 4545.409899] fbc0: 9110626091260021 0000000000000000 0000000000000000 97fc098d91042260
>> [ 4545.417658] fbe0: b0000001f9001a79 5299400690000000 72a77346b2407be7 0000ffffa9d01588
>> [ 4545.425416] fc00: ffff0000081fdfd0 0000ffffa9c87440
>> [ 4545.430248] [<ffff0000083a1000>] __memcpy+0x100/0x180
>> [ 4545.435253] [<ffff000008270f64>] read_kcore+0x21c/0x3b0
>> [ 4545.440429] [<ffff00000826340c>] proc_reg_read+0x64/0x90
>> [ 4545.445691] [<ffff0000081fb83c>] __vfs_read+0x1c/0x108
>> [ 4545.450779] [<ffff0000081fcb28>] vfs_read+0x80/0x130
>> [ 4545.455696] [<ffff0000081fe014>] SyS_read+0x44/0xa0
>> [ 4545.460528] [<ffff000008082f30>] el0_svc_naked+0x24/0x28
>> [ 4545.465790] Code: d503201f d503201f d503201f d503201f (a8c12027)
>> [ 4545.471852] ---[ end trace 4d1897f94759f461 ]---
>> [ 4545.476435] note: cat[8976] exited with preempt_count 2
>>
>> I find the issue is introduced when applying commit f9040773b7bb
>> ("arm64: move kernel image to base of vmalloc area"). This patch
>> make the kernel image overlap with vmalloc area. It will result in
>> vmalloc area have the huge page table. but the vmalloc_to_page is
>> not realize the change. and the function is public to any arch.
>>
>> I fix it by adding the another kernel image condition in vmalloc_to_page
>> to make it keep the accordance with previous vmalloc mapping.
>>
>> Fixes: f9040773b7bb ("arm64: move kernel image to base of vmalloc area")
>> Reported-by: tan xiaojun <tanxiaojun@huawei.com>
>> Signed-off-by: zhongjiang <zhongjiang@huawei.com>
>> Reviewed-by: Laura Abbott <labbott@redhat.com>
>> ---
>> arch/arm64/mm/mmu.c | 2 +-
>> include/linux/vmalloc.h | 1 +
>> mm/vmalloc.c | 34 +++++++++++++++++++++++++++-------
>> 3 files changed, 29 insertions(+), 8 deletions(-)
>>
>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
>> index 0c429ec..2265c39 100644
>> --- a/arch/arm64/mm/mmu.c
>> +++ b/arch/arm64/mm/mmu.c
>> @@ -509,7 +509,7 @@ static void __init map_kernel_segment(pgd_t *pgd, void *va_start, void *va_end,
>> vma->addr = va_start;
>> vma->phys_addr = pa_start;
>> vma->size = size;
>> - vma->flags = VM_MAP;
>> + vma->flags = VM_KERNEL;
>> vma->caller = __builtin_return_address(0);
>>
>> vm_area_add_early(vma);
>> diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
>> index 0328ce0..c9245af 100644
>> --- a/include/linux/vmalloc.h
>> +++ b/include/linux/vmalloc.h
>> @@ -17,6 +17,7 @@
>> #define VM_ALLOC 0x00000002 /* vmalloc() */
>> #define VM_MAP 0x00000004 /* vmap()ed pages */
>> #define VM_USERMAP 0x00000008 /* suitable for remap_vmalloc_range */
>> +#define VM_KERNEL 0x00000010 /* kernel pages */
> Is it necessary to define this for all architectures? The VM_
> namespace explicitly declares a subrange as having architecture scope
> iirc
It is not necessary, but more general if it is not strictly limited. As other arch
do not care about the flags. if you go against the way. the following whethor let
you satisfy or not.
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 3d9d786..f534df6 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -16,6 +16,13 @@
#define VM_ALLOC 0x00000002 /* vmalloc() */
#define VM_MAP 0x00000004 /* vmap()ed pages */
#define VM_USERMAP 0x00000008 /* suitable for remap_vmalloc_range */
+
+#ifdef CONFIG_ARM64
+#define VM_KERNEL 0x00000010
+#else
+#define VM_KERNEL 0
+#endif
+
how about the above change ?
>> #define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */
>> #define VM_NO_GUARD 0x00000040 /* don't add guard page */
>> #define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */
>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>> index 1dda6d8..b456803 100644
>> --- a/mm/vmalloc.c
>> +++ b/mm/vmalloc.c
>> @@ -1967,11 +1967,27 @@ void *vmalloc_32_user(unsigned long size)
>> EXPORT_SYMBOL(vmalloc_32_user);
>>
>> /*
>> + * VM_KERNEL indicates an address is mapped linearly.The linear mapping may
>> + * use larger pages which vmalloc_to_page cannot handle.
> Please don't overload 'linear mapping' . We often refer to the direct
> kernel mapping mapping of DRAM as 'linear mapping', but in this
> context, it is not clear at all what 'linear' means.
hmm, Exactly clear. but I will drop the comments. It is just a helper function.
It is no necessary to add comments as Laura suggestions.
>
>> + */
>> +static inline struct page *aligned_get_page(char *addr, struct vm_struct *vm)
> This function needs a better name
hmm, straightforward. addr_to_page(). how about this?
Thanks
zhongjiang
>
>> +{
>> + struct page *p = NULL;
>> +
>> + if (vm->flags & VM_KERNEL)
>> + p = virt_to_page(lm_alias(addr));
>> + else
>> + p = vmalloc_to_page(addr);
>> +
>> + return p;
>> +}
>> +
>> +/*
>> * small helper routine , copy contents to buf from addr.
>> * If the page is not present, fill zero.
>> */
>> -
>> -static int aligned_vread(char *buf, char *addr, unsigned long count)
>> +static int aligned_vread(char *buf, char *addr, unsigned long count,
>> + struct vm_struct *vm)
>> {
>> struct page *p;
>> int copied = 0;
>> @@ -1983,7 +1999,7 @@ static int aligned_vread(char *buf, char *addr, unsigned long count)
>> length = PAGE_SIZE - offset;
>> if (length > count)
>> length = count;
>> - p = vmalloc_to_page(addr);
>> + p = aligned_get_page(addr, vm);
>> /*
>> * To do safe access to this _mapped_ area, we need
>> * lock. But adding lock here means that we need to add
>> @@ -2010,7 +2026,8 @@ static int aligned_vread(char *buf, char *addr, unsigned long count)
>> return copied;
>> }
>>
>> -static int aligned_vwrite(char *buf, char *addr, unsigned long count)
>> +static int aligned_vwrite(char *buf, char *addr, unsigned long count,
>> + struct vm_struct *vm)
>> {
>> struct page *p;
>> int copied = 0;
>> @@ -2022,7 +2039,7 @@ static int aligned_vwrite(char *buf, char *addr, unsigned long count)
>> length = PAGE_SIZE - offset;
>> if (length > count)
>> length = count;
>> - p = vmalloc_to_page(addr);
>> + p = aligned_get_page(addr, vm);
>> /*
>> * To do safe access to this _mapped_ area, we need
>> * lock. But adding lock here means that we need to add
>> @@ -2109,7 +2126,7 @@ long vread(char *buf, char *addr, unsigned long count)
>> if (n > count)
>> n = count;
>> if (!(vm->flags & VM_IOREMAP))
>> - aligned_vread(buf, addr, n);
>> + aligned_vread(buf, addr, n, vm);
>> else /* IOREMAP area is treated as memory hole */
>> memset(buf, 0, n);
>> buf += n;
>> @@ -2190,7 +2207,7 @@ long vwrite(char *buf, char *addr, unsigned long count)
>> if (n > count)
>> n = count;
>> if (!(vm->flags & VM_IOREMAP)) {
>> - aligned_vwrite(buf, addr, n);
>> + aligned_vwrite(buf, addr, n, vm);
>> copied++;
>> }
>> buf += n;
>> @@ -2710,6 +2727,9 @@ static int s_show(struct seq_file *m, void *p)
>> if (v->flags & VM_USERMAP)
>> seq_puts(m, " user");
>>
>> + if (v->flags & VM_KERNEL)
>> + seq_puts(m, " kernel");
>> +
>> if (is_vmalloc_addr(v->pages))
>> seq_puts(m, " vpages");
>>
>> --
>> 1.7.12.4
>>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> .
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v3 1/2] mm/hugetlb: Cleanup ARCH_HAS_GIGANTIC_PAGE
From: Aneesh Kumar K.V @ 2017-05-17 4:28 UTC (permalink / raw)
To: akpm, mpe, Anshuman Khandual
Cc: linux-mm, linux-kernel, linuxppc-dev, Aneesh Kumar K.V
This moves the #ifdef in C code to a Kconfig dependency. Also we move the
gigantic_page_supported() function to be arch specific. This gives arch to
conditionally enable runtime allocation of gigantic huge page. Architectures
like ppc64 supports different gigantic huge page size (16G and 1G) based on the
translation mode selected. This provides an opportunity for ppc64 to enable
runtime allocation only w.r.t 1G hugepage.
No functional change in this patch.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
Changes from V2:
* Fix build error with x86
* Update the Kconfig change to match the C #ifdef
arch/arm64/Kconfig | 2 +-
arch/arm64/include/asm/hugetlb.h | 4 ++++
arch/s390/Kconfig | 2 +-
arch/s390/include/asm/hugetlb.h | 3 +++
arch/x86/Kconfig | 2 +-
arch/x86/include/asm/hugetlb.h | 4 ++++
mm/hugetlb.c | 7 ++-----
7 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 3741859765cf..87240dcb6a07 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -11,7 +11,7 @@ config ARM64
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_GCOV_PROFILE_ALL
- select ARCH_HAS_GIGANTIC_PAGE
+ select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
select ARCH_HAS_KCOV
select ARCH_HAS_SET_MEMORY
select ARCH_HAS_SG_CHAIN
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
index bbc1e35aa601..793bd73b0d07 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -83,4 +83,8 @@ extern void huge_ptep_set_wrprotect(struct mm_struct *mm,
extern void huge_ptep_clear_flush(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep);
+#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
+static inline bool gigantic_page_supported(void) { return true; }
+#endif
+
#endif /* __ASM_HUGETLB_H */
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index a2dcef0aacc7..f3637b641d7e 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -67,7 +67,7 @@ config S390
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_GCOV_PROFILE_ALL
- select ARCH_HAS_GIGANTIC_PAGE
+ select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
select ARCH_HAS_KCOV
select ARCH_HAS_SET_MEMORY
select ARCH_HAS_SG_CHAIN
diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h
index cd546a245c68..89057b2cc8fe 100644
--- a/arch/s390/include/asm/hugetlb.h
+++ b/arch/s390/include/asm/hugetlb.h
@@ -112,4 +112,7 @@ static inline pte_t huge_pte_modify(pte_t pte, pgprot_t newprot)
return pte_modify(pte, newprot);
}
+#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
+static inline bool gigantic_page_supported(void) { return true; }
+#endif
#endif /* _ASM_S390_HUGETLB_H */
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index cc98d5a294ee..e39b3b6b7d16 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -22,7 +22,7 @@ config X86_64
def_bool y
depends on 64BIT
# Options that are inherently 64-bit kernel only:
- select ARCH_HAS_GIGANTIC_PAGE
+ select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
select ARCH_SUPPORTS_INT128
select ARCH_USE_CMPXCHG_LOCKREF
select HAVE_ARCH_SOFT_DIRTY
diff --git a/arch/x86/include/asm/hugetlb.h b/arch/x86/include/asm/hugetlb.h
index 3a106165e03a..535af0f2d8ac 100644
--- a/arch/x86/include/asm/hugetlb.h
+++ b/arch/x86/include/asm/hugetlb.h
@@ -85,4 +85,8 @@ static inline void arch_clear_hugepage_flags(struct page *page)
{
}
+#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
+static inline bool gigantic_page_supported(void) { return true; }
+#endif
+
#endif /* _ASM_X86_HUGETLB_H */
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 3d0aab9ee80d..ce090186b992 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1024,9 +1024,7 @@ static int hstate_next_node_to_free(struct hstate *h, nodemask_t *nodes_allowed)
((node = hstate_next_node_to_free(hs, mask)) || 1); \
nr_nodes--)
-#if defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) && \
- ((defined(CONFIG_MEMORY_ISOLATION) && defined(CONFIG_COMPACTION)) || \
- defined(CONFIG_CMA))
+#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
static void destroy_compound_gigantic_page(struct page *page,
unsigned int order)
{
@@ -1158,8 +1156,7 @@ static int alloc_fresh_gigantic_page(struct hstate *h,
return 0;
}
-static inline bool gigantic_page_supported(void) { return true; }
-#else
+#else /* !CONFIG_ARCH_HAS_GIGANTIC_PAGE */
static inline bool gigantic_page_supported(void) { return false; }
static inline void free_gigantic_page(struct page *page, unsigned int order) { }
static inline void destroy_compound_gigantic_page(struct page *page,
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v3 2/2] powerpc/mm/hugetlb: Add support for 1G huge pages
From: Aneesh Kumar K.V @ 2017-05-17 4:28 UTC (permalink / raw)
To: akpm, mpe, Anshuman Khandual
Cc: linux-mm, linux-kernel, linuxppc-dev, Aneesh Kumar K.V
In-Reply-To: <1494995292-4443-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
POWER9 supports hugepages of size 2M and 1G in radix MMU mode. This patch
enables the usage of 1G page size for hugetlbfs. This also update the helper
such we can do 1G page allocation at runtime.
We still don't enable 1G page size on DD1 version. This is to avoid doing
workaround mentioned in commit: 6d3a0379ebdc8 (powerpc/mm: Add
radix__tlb_flush_pte_p9_dd1()
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/book3s/64/hugetlb.h | 10 ++++++++++
arch/powerpc/mm/hugetlbpage.c | 7 +++++--
arch/powerpc/platforms/Kconfig.cputype | 1 +
3 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/hugetlb.h b/arch/powerpc/include/asm/book3s/64/hugetlb.h
index 6666cd366596..5c28bd6f2ae1 100644
--- a/arch/powerpc/include/asm/book3s/64/hugetlb.h
+++ b/arch/powerpc/include/asm/book3s/64/hugetlb.h
@@ -50,4 +50,14 @@ static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma,
else
return entry;
}
+
+#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
+static inline bool gigantic_page_supported(void)
+{
+ if (radix_enabled())
+ return true;
+ return false;
+}
+#endif
+
#endif
diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
index a4f33de4008e..80f6d2ed551a 100644
--- a/arch/powerpc/mm/hugetlbpage.c
+++ b/arch/powerpc/mm/hugetlbpage.c
@@ -763,8 +763,11 @@ static int __init add_huge_page_size(unsigned long long size)
* Hash: 16M and 16G
*/
if (radix_enabled()) {
- if (mmu_psize != MMU_PAGE_2M)
- return -EINVAL;
+ if (mmu_psize != MMU_PAGE_2M) {
+ if (cpu_has_feature(CPU_FTR_POWER9_DD1) ||
+ (mmu_psize != MMU_PAGE_1G))
+ return -EINVAL;
+ }
} else {
if (mmu_psize != MMU_PAGE_16M && mmu_psize != MMU_PAGE_16G)
return -EINVAL;
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 684e886eaae4..b76ef6637016 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -344,6 +344,7 @@ config PPC_STD_MMU_64
config PPC_RADIX_MMU
bool "Radix MMU Support"
depends on PPC_BOOK3S_64
+ select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA
default y
help
Enable support for the Power ISA 3.0 Radix style MMU. Currently this
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [Question] softlock up in handle_mm_fault
From: Yisheng Xie @ 2017-05-17 6:45 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel
Cc: will.deacon, mark.rutland, ard.biesheuvel, mhocko, Hanjun Guo,
Xishi Qiu, liubo95
Hi all,
We met a softlockup problem in handle_mm_fault on platform arm32 with v4.1 kernel.
And from the log it seems do not have any deadlock or loopback.
Does anyone ever met similar problem or any idea about this problem?
Any reply is more than welcome!
Thanks.
Yisheng Xie
-------dmesg--------
[20170512182106]NMI watchdog: BUG: soft lockup - CPU#2 stuck for 410ms! [UMPTB.out:1681]
[20170512182106]Modules linked in: memory_add(O) pramdisk(O) rsm(O) mcss(O) hiuart(O) hii2c(O) himdio(O) hispi(O) npmac(O) hffs(O) hinand(O) nor(O) higmac(O) gic(O) upbcom_ipc(O) bsplogutil(O) Wdt_Hi1380_kernel(O) higpio(O) gpio(O) rtos_snapshot(O) virtualcpu(O) comm(O) NandDrv(O) bsplog(O) rtos_kbox_panic(O) enable_uart_rx(O) uart_suspend(O) double_cluster(O) xt_tcpudp vfat fat usbhid hid usb_device_hisi(O) sd_mod physmap ohci_hcd nfsd nfs_acl exportfs auth_rpcgss oid_registry nfs lockd sunrpc grace nand_ids nand_ecc mtdblock mtd_blkdevs iptable_filter ip_tables ipt_REJECT x_tables nf_reject_ipv4 invalid_Icache(O) ehci_hcd cmdlinepart cfi_probe gen_probe chipreg cfi_cmdset_0002(O) cfi_util mtd cache_ops(O)
[20170512182106]CPU: 2 PID: 1681 Comm: UMPTB.out Tainted: G W O
[20170512182106]Hardware name: Hisilicon A9
[20170512182106]task: c40692c0 ti: c309e000 task.ti: c309e000
[20170512182106]PC is at ptep_set_access_flags+0x0/0x88
[20170512182106]LR is at handle_mm_fault+0x10f0/0x130c
[20170512182106]pc : [<c0207ab4>] lr : [<c01fc5a4>] psr: 60000313
sp : c309fcd8 ip : 00000015 fp : 0000047b
[20170512182106]r10: eeef16e8 r9 : 00000001 r8 : c3b7a200
[20170512182106]r7 : daea475f r6 : ef1cae70 r5 : 8f7ba000 r4 : edf44900
[20170512182106]r3 : daea475f r2 : eeef16e8 r1 : 8f7ba000 r0 : edf44900
[20170512182106]Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[20170512182106]Control: 1ac5387d Table: ae82004a DAC: 55555555
[20170512182106]CPU: 2 PID: 1681 Comm: UMPTB.out Tainted: G W O
[20170512182106]Hardware name: Hisilicon A9
[20170512182106][<c0110410>] (unwind_backtrace) from [<c010b640>] (show_stack+0x18/0x1c)
[20170512182106][<c010b640>] (show_stack) from [<c04a577c>] (dump_stack+0xa4/0xdc)
[20170512182106][<c04a577c>] (dump_stack) from [<c01a220c>] (watchdog_timer_fn+0x220/0x2fc)
[20170512182106][<c01a220c>] (watchdog_timer_fn) from [<c0172a2c>] (hrtimer_run_queues+0x1d0/0x3a8)
[20170512182106][<c0172a2c>] (hrtimer_run_queues) from [<c0171b58>] (run_local_timers+0x8/0x14)
[20170512182106][<c0171b58>] (run_local_timers) from [<c0171b8c>] (update_process_times+0x28/0x5c)
[20170512182106][<c0171b8c>] (update_process_times) from [<c017cfe8>] (tick_periodic+0xac/0xcc)
[20170512182106][<c017cfe8>] (tick_periodic) from [<c017d074>] (tick_handle_periodic+0x24/0x80)
[20170512182106][<c017d074>] (tick_handle_periodic) from [<c010f6f0>] (twd_handler+0x30/0x44)
[20170512182106][<c010f6f0>] (twd_handler) from [<c0165320>] (handle_percpu_devid_irq+0xb4/0x1b0)
[20170512182106][<c0165320>] (handle_percpu_devid_irq) from [<c01614ec>] (generic_handle_irq+0x20/0x30)
[20170512182106][<c01614ec>] (generic_handle_irq) from [<c016178c>] (__handle_domain_irq+0xd8/0x160)
[20170512182106][<c016178c>] (__handle_domain_irq) from [<c01013c0>] (gic_handle_irq+0x40/0x6c)
[20170512182106][<c01013c0>] (gic_handle_irq) from [<c04ab1e8>] (__irq_svc+0x48/0x60)
[20170512182106]Exception stack(0xc309fc90 to 0xc309fcd8)
[20170512182106]fc80: edf44900 8f7ba000 eeef16e8 daea475f
[20170512182106]fca0: edf44900 8f7ba000 ef1cae70 daea475f c3b7a200 00000001 eeef16e8 0000047b
[20170512182106]fcc0: 00000015 c309fcd8 c01fc5a4 c0207ab4 60000313 ffffffff
[20170512182106][<c04ab1e8>] (__irq_svc) from [<c0207ab4>] (ptep_set_access_flags+0x0/0x88)
[20170512182106][<c0207ab4>] (ptep_set_access_flags) from [<ede20000>] (0xede20000)
[20170512182106]=====================SOFTLOCKUP INFO BEGIN=======================
[20170512182106]------------------CPU#2-----------------------------------
[20170512182106][CPU#2] the task [UMPTB.out] is not waiting for a lock,maybe a delay or deadcircle!
[20170512182106]UMPTB.out R running 0 1681 1680 0x00000002
[20170512182106]locked:
[20170512182106]eee5db90 &f->f_pos_lock 2 [<c023eb58>] __fdget_pos+0x38/0x40
[20170512182106]c3c50ab0 &p->lock 2 [<c02448c0>] seq_read+0x28/0x44c
[20170512182106]c3b7a240 &mm->mmap_sem 2 [<c04ab910>] do_page_fault+0xc4/0x364
[20170512182106][<c0110410>] (unwind_backtrace) from [<c010b640>] (show_stack+0x18/0x1c)
[20170512182106][<c010b640>] (show_stack) from [<c01a27f4>] (show_lock_info+0xd0/0x29c)
[20170512182106][<c01a27f4>] (show_lock_info) from [<c01a2240>] (watchdog_timer_fn+0x254/0x2fc)
[20170512182106][<c01a2240>] (watchdog_timer_fn) from [<c0172a2c>] (hrtimer_run_queues+0x1d0/0x3a8)
[20170512182106][<c0172a2c>] (hrtimer_run_queues) from [<c0171b58>] (run_local_timers+0x8/0x14)
[20170512182106][<c0171b58>] (run_local_timers) from [<c0171b8c>] (update_process_times+0x28/0x5c)
[20170512182106][<c0171b8c>] (update_process_times) from [<c017cfe8>] (tick_periodic+0xac/0xcc)
[20170512182106][<c017cfe8>] (tick_periodic) from [<c017d074>] (tick_handle_periodic+0x24/0x80)
[20170512182106][<c017d074>] (tick_handle_periodic) from [<c010f6f0>] (twd_handler+0x30/0x44)
[20170512182106][<c010f6f0>] (twd_handler) from [<c0165320>] (handle_percpu_devid_irq+0xb4/0x1b0)
[20170512182106][<c0165320>] (handle_percpu_devid_irq) from [<c01614ec>] (generic_handle_irq+0x20/0x30)
[20170512182106][<c01614ec>] (generic_handle_irq) from [<c016178c>] (__handle_domain_irq+0xd8/0x160)
[20170512182106][<c016178c>] (__handle_domain_irq) from [<c01013c0>] (gic_handle_irq+0x40/0x6c)
[20170512182106][<c01013c0>] (gic_handle_irq) from [<c04ab1e8>] (__irq_svc+0x48/0x60)
[20170512182106]Exception stack(0xc309fc90 to 0xc309fcd8)
[20170512182106]fc80: edf44900 8f7ba000 eeef16e8 daea475f
[20170512182106]fca0: edf44900 8f7ba000 ef1cae70 daea475f c3b7a200 00000001 eeef16e8 0000047b
[20170512182106]fcc0: 00000015 c309fcd8 c01fc5a4 c0207ab4 60000313 ffffffff
[20170512182106][<c04ab1e8>] (__irq_svc) from [<c0207ab4>] (ptep_set_access_flags+0x0/0x88)
[20170512182106][<c0207ab4>] (ptep_set_access_flags) from [<ede20000>] (0xede20000)
[20170512182106]=====================SOFTLOCKUP INFO END=========================
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH 1/2] drm: replace drm_[cm]alloc* by kvmalloc alternatives
From: Michal Hocko @ 2017-05-17 6:55 UTC (permalink / raw)
To: dri-devel
Cc: linux-kernel, linux-mm, Daniel Vetter, Jani Nikula, Sean Paul,
David Airlie, Michal Hocko
From: Michal Hocko <mhocko@suse.com>
drm_[cm]alloc* has grown their own kvmalloc with vmalloc fallback
implementations. MM has grown kvmalloc* helpers in the meantime. Let's
use those because it a) reduces the code and b) MM has a better idea
how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
with __GFP_NORETRY).
drm_calloc_large needs to get __GFP_ZERO explicitly but it is the same
thing as kvmalloc_array in principle.
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
Hi,
I didn't add Reviewed-by from Chris because the original patch [1]
didn't change drm_calloc_large which I have missed in that posting.
This patch is the same otherwwise.
[1] http://lkml.kernel.org/r/20170516090606.5891-1-mhocko@kernel.org
include/drm/drm_mem_util.h | 31 +++----------------------------
1 file changed, 3 insertions(+), 28 deletions(-)
diff --git a/include/drm/drm_mem_util.h b/include/drm/drm_mem_util.h
index d0f6cf2e5324..a1ddf55fda67 100644
--- a/include/drm/drm_mem_util.h
+++ b/include/drm/drm_mem_util.h
@@ -31,43 +31,18 @@
static __inline__ void *drm_calloc_large(size_t nmemb, size_t size)
{
- if (size != 0 && nmemb > SIZE_MAX / size)
- return NULL;
-
- if (size * nmemb <= PAGE_SIZE)
- return kcalloc(nmemb, size, GFP_KERNEL);
-
- return vzalloc(size * nmemb);
+ return kvmalloc_array(nmemb, size, GFP_KERNEL | __GFP_ZERO);
}
/* Modeled after cairo's malloc_ab, it's like calloc but without the zeroing. */
static __inline__ void *drm_malloc_ab(size_t nmemb, size_t size)
{
- if (size != 0 && nmemb > SIZE_MAX / size)
- return NULL;
-
- if (size * nmemb <= PAGE_SIZE)
- return kmalloc(nmemb * size, GFP_KERNEL);
-
- return vmalloc(size * nmemb);
+ return kvmalloc_array(nmemb, size, GFP_KERNEL);
}
static __inline__ void *drm_malloc_gfp(size_t nmemb, size_t size, gfp_t gfp)
{
- if (size != 0 && nmemb > SIZE_MAX / size)
- return NULL;
-
- if (size * nmemb <= PAGE_SIZE)
- return kmalloc(nmemb * size, gfp);
-
- if (gfp & __GFP_RECLAIMABLE) {
- void *ptr = kmalloc(nmemb * size,
- gfp | __GFP_NOWARN | __GFP_NORETRY);
- if (ptr)
- return ptr;
- }
-
- return __vmalloc(size * nmemb, gfp, PAGE_KERNEL);
+ return kvmalloc_array(nmemb, size, gfp);
}
static __inline void drm_free_large(void *ptr)
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH 2/2] drm: drop drm_[cm]alloc* helpers
From: Michal Hocko @ 2017-05-17 6:55 UTC (permalink / raw)
To: dri-devel
Cc: linux-kernel, linux-mm, Daniel Vetter, Jani Nikula, Sean Paul,
David Airlie, Michal Hocko
In-Reply-To: <20170517065509.18659-1-mhocko@kernel.org>
From: Michal Hocko <mhocko@suse.com>
Now that drm_[cm]alloc* helpers are simple one line wrappers around
kvmalloc_array and drm_free_large is just kvfree alias we can drop
them and replace by their native forms.
This shouldn't introduce any functional change.
Suggested-by: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c | 16 +++----
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 19 ++++----
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +--
drivers/gpu/drm/drm_gem.c | 6 +--
drivers/gpu/drm/etnaviv/etnaviv_gem.c | 12 ++---
drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c | 4 +-
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 12 ++---
drivers/gpu/drm/exynos/exynos_drm_gem.c | 11 +++--
drivers/gpu/drm/i915/i915_debugfs.c | 4 +-
drivers/gpu/drm/i915/i915_gem.c | 4 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 34 +++++++-------
drivers/gpu/drm/i915/i915_gem_gtt.c | 6 +--
drivers/gpu/drm/i915/i915_gem_userptr.c | 8 ++--
drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c | 12 ++---
drivers/gpu/drm/msm/msm_gem.c | 10 ++--
drivers/gpu/drm/radeon/radeon_cs.c | 11 +++--
drivers/gpu/drm/radeon/radeon_gem.c | 2 +-
drivers/gpu/drm/radeon/radeon_ring.c | 4 +-
drivers/gpu/drm/radeon/radeon_vm.c | 4 +-
drivers/gpu/drm/ttm/ttm_tt.c | 13 +++---
drivers/gpu/drm/udl/udl_dmabuf.c | 2 +-
drivers/gpu/drm/udl/udl_gem.c | 2 +-
drivers/gpu/drm/vc4/vc4_gem.c | 15 +++---
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 27 +++++------
include/drm/drmP.h | 1 -
include/drm/drm_mem_util.h | 53 ----------------------
26 files changed, 126 insertions(+), 173 deletions(-)
delete mode 100644 include/drm/drm_mem_util.h
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
index a6649874e6ce..9f0247cdda5e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
@@ -96,7 +96,7 @@ static int amdgpu_bo_list_set(struct amdgpu_device *adev,
int r;
unsigned long total_size = 0;
- array = drm_malloc_ab(num_entries, sizeof(struct amdgpu_bo_list_entry));
+ array = kvmalloc_array(num_entries, sizeof(struct amdgpu_bo_list_entry), GFP_KERNEL);
if (!array)
return -ENOMEM;
memset(array, 0, num_entries * sizeof(struct amdgpu_bo_list_entry));
@@ -148,7 +148,7 @@ static int amdgpu_bo_list_set(struct amdgpu_device *adev,
for (i = 0; i < list->num_entries; ++i)
amdgpu_bo_unref(&list->array[i].robj);
- drm_free_large(list->array);
+ kvfree(list->array);
list->gds_obj = gds_obj;
list->gws_obj = gws_obj;
@@ -163,7 +163,7 @@ static int amdgpu_bo_list_set(struct amdgpu_device *adev,
error_free:
while (i--)
amdgpu_bo_unref(&array[i].robj);
- drm_free_large(array);
+ kvfree(array);
return r;
}
@@ -224,7 +224,7 @@ void amdgpu_bo_list_free(struct amdgpu_bo_list *list)
amdgpu_bo_unref(&list->array[i].robj);
mutex_destroy(&list->lock);
- drm_free_large(list->array);
+ kvfree(list->array);
kfree(list);
}
@@ -244,8 +244,8 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data,
int r;
- info = drm_malloc_ab(args->in.bo_number,
- sizeof(struct drm_amdgpu_bo_list_entry));
+ info = kvmalloc_array(args->in.bo_number,
+ sizeof(struct drm_amdgpu_bo_list_entry), GFP_KERNEL);
if (!info)
return -ENOMEM;
@@ -311,11 +311,11 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data,
memset(args, 0, sizeof(*args));
args->out.list_handle = handle;
- drm_free_large(info);
+ kvfree(info);
return 0;
error_free:
- drm_free_large(info);
+ kvfree(info);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 4e6b9501ab0a..5b3e0f63a115 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -194,7 +194,7 @@ int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, void *data)
size = p->chunks[i].length_dw;
cdata = (void __user *)(uintptr_t)user_chunk.chunk_data;
- p->chunks[i].kdata = drm_malloc_ab(size, sizeof(uint32_t));
+ p->chunks[i].kdata = kvmalloc_array(size, sizeof(uint32_t), GFP_KERNEL);
if (p->chunks[i].kdata == NULL) {
ret = -ENOMEM;
i--;
@@ -247,7 +247,7 @@ int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, void *data)
i = p->nchunks - 1;
free_partial_kdata:
for (; i >= 0; i--)
- drm_free_large(p->chunks[i].kdata);
+ kvfree(p->chunks[i].kdata);
kfree(p->chunks);
p->chunks = NULL;
p->nchunks = 0;
@@ -505,7 +505,7 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
return r;
if (binding_userptr) {
- drm_free_large(lobj->user_pages);
+ kvfree(lobj->user_pages);
lobj->user_pages = NULL;
}
}
@@ -571,7 +571,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
release_pages(e->user_pages,
e->robj->tbo.ttm->num_pages,
false);
- drm_free_large(e->user_pages);
+ kvfree(e->user_pages);
e->user_pages = NULL;
}
@@ -601,8 +601,9 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
list_for_each_entry(e, &need_pages, tv.head) {
struct ttm_tt *ttm = e->robj->tbo.ttm;
- e->user_pages = drm_calloc_large(ttm->num_pages,
- sizeof(struct page*));
+ e->user_pages = kvmalloc_array(ttm->num_pages,
+ sizeof(struct page*),
+ GFP_KERNEL | __GFP_ZERO);
if (!e->user_pages) {
r = -ENOMEM;
DRM_ERROR("calloc failure in %s\n", __func__);
@@ -612,7 +613,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
r = amdgpu_ttm_tt_get_user_pages(ttm, e->user_pages);
if (r) {
DRM_ERROR("amdgpu_ttm_tt_get_user_pages failed.\n");
- drm_free_large(e->user_pages);
+ kvfree(e->user_pages);
e->user_pages = NULL;
goto error_free_pages;
}
@@ -708,7 +709,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
release_pages(e->user_pages,
e->robj->tbo.ttm->num_pages,
false);
- drm_free_large(e->user_pages);
+ kvfree(e->user_pages);
}
}
@@ -761,7 +762,7 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error, bo
amdgpu_bo_list_put(parser->bo_list);
for (i = 0; i < parser->nchunks; i++)
- drm_free_large(parser->chunks[i].kdata);
+ kvfree(parser->chunks[i].kdata);
kfree(parser->chunks);
if (parser->job)
amdgpu_job_free(parser->job);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 07ff3b1514f1..749a6cde7985 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -279,8 +279,9 @@ static int amdgpu_vm_alloc_levels(struct amdgpu_device *adev,
if (!parent->entries) {
unsigned num_entries = amdgpu_vm_num_entries(adev, level);
- parent->entries = drm_calloc_large(num_entries,
- sizeof(struct amdgpu_vm_pt));
+ parent->entries = kvmalloc_array(num_entries,
+ sizeof(struct amdgpu_vm_pt),
+ GFP_KERNEL | __GFP_ZERO);
if (!parent->entries)
return -ENOMEM;
memset(parent->entries, 0 , sizeof(struct amdgpu_vm_pt));
@@ -2198,7 +2199,7 @@ static void amdgpu_vm_free_levels(struct amdgpu_vm_pt *level)
for (i = 0; i <= level->last_entry_used; i++)
amdgpu_vm_free_levels(&level->entries[i]);
- drm_free_large(level->entries);
+ kvfree(level->entries);
}
/**
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index b1e28c944637..8dc11064253d 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -521,7 +521,7 @@ struct page **drm_gem_get_pages(struct drm_gem_object *obj)
npages = obj->size >> PAGE_SHIFT;
- pages = drm_malloc_ab(npages, sizeof(struct page *));
+ pages = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (pages == NULL)
return ERR_PTR(-ENOMEM);
@@ -546,7 +546,7 @@ struct page **drm_gem_get_pages(struct drm_gem_object *obj)
while (i--)
put_page(pages[i]);
- drm_free_large(pages);
+ kvfree(pages);
return ERR_CAST(p);
}
EXPORT_SYMBOL(drm_gem_get_pages);
@@ -582,7 +582,7 @@ void drm_gem_put_pages(struct drm_gem_object *obj, struct page **pages,
put_page(pages[i]);
}
- drm_free_large(pages);
+ kvfree(pages);
}
EXPORT_SYMBOL(drm_gem_put_pages);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
index fd56f92f3469..d45dd9f3b00c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
@@ -748,7 +748,7 @@ static struct page **etnaviv_gem_userptr_do_get_pages(
uintptr_t ptr;
unsigned int flags = 0;
- pvec = drm_malloc_ab(npages, sizeof(struct page *));
+ pvec = kvmallo_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!pvec)
return ERR_PTR(-ENOMEM);
@@ -772,7 +772,7 @@ static struct page **etnaviv_gem_userptr_do_get_pages(
if (ret < 0) {
release_pages(pvec, pinned, 0);
- drm_free_large(pvec);
+ kvfree(pvec);
return ERR_PTR(ret);
}
@@ -823,7 +823,7 @@ static int etnaviv_gem_userptr_get_pages(struct etnaviv_gem_object *etnaviv_obj)
mm = get_task_mm(etnaviv_obj->userptr.task);
pinned = 0;
if (mm == current->mm) {
- pvec = drm_malloc_ab(npages, sizeof(struct page *));
+ pvec = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!pvec) {
mmput(mm);
return -ENOMEM;
@@ -832,7 +832,7 @@ static int etnaviv_gem_userptr_get_pages(struct etnaviv_gem_object *etnaviv_obj)
pinned = __get_user_pages_fast(etnaviv_obj->userptr.ptr, npages,
!etnaviv_obj->userptr.ro, pvec);
if (pinned < 0) {
- drm_free_large(pvec);
+ kvfree(pvec);
mmput(mm);
return pinned;
}
@@ -845,7 +845,7 @@ static int etnaviv_gem_userptr_get_pages(struct etnaviv_gem_object *etnaviv_obj)
}
release_pages(pvec, pinned, 0);
- drm_free_large(pvec);
+ kvfree(pvec);
work = kmalloc(sizeof(*work), GFP_KERNEL);
if (!work) {
@@ -879,7 +879,7 @@ static void etnaviv_gem_userptr_release(struct etnaviv_gem_object *etnaviv_obj)
int npages = etnaviv_obj->base.size >> PAGE_SHIFT;
release_pages(etnaviv_obj->pages, npages, 0);
- drm_free_large(etnaviv_obj->pages);
+ kvfree(etnaviv_obj->pages);
}
put_task_struct(etnaviv_obj->userptr.task);
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
index abed6f781281..e5da4f2300ba 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
@@ -87,7 +87,7 @@ static void etnaviv_gem_prime_release(struct etnaviv_gem_object *etnaviv_obj)
* ours, just free the array we allocated:
*/
if (etnaviv_obj->pages)
- drm_free_large(etnaviv_obj->pages);
+ kvfree(etnaviv_obj->pages);
drm_prime_gem_destroy(&etnaviv_obj->base, etnaviv_obj->sgt);
}
@@ -128,7 +128,7 @@ struct drm_gem_object *etnaviv_gem_prime_import_sg_table(struct drm_device *dev,
npages = size / PAGE_SIZE;
etnaviv_obj->sgt = sgt;
- etnaviv_obj->pages = drm_malloc_ab(npages, sizeof(struct page *));
+ etnaviv_obj->pages = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!etnaviv_obj->pages) {
ret = -ENOMEM;
goto fail;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index de80ee1b71df..ee7069e93eda 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -345,9 +345,9 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
* Copy the command submission and bo array to kernel space in
* one go, and do this outside of any locks.
*/
- bos = drm_malloc_ab(args->nr_bos, sizeof(*bos));
- relocs = drm_malloc_ab(args->nr_relocs, sizeof(*relocs));
- stream = drm_malloc_ab(1, args->stream_size);
+ bos = kvmalloc_array(args->nr_bos, sizeof(*bos), GFP_KERNEL);
+ relocs = kvmalloc_array(args->nr_relocs, sizeof(*relocs), GFP_KERNEL);
+ stream = kvmalloc_array(1, args->stream_size, GFP_KERNEL);
cmdbuf = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc,
ALIGN(args->stream_size, 8) + 8,
args->nr_bos);
@@ -489,11 +489,11 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
if (cmdbuf)
etnaviv_cmdbuf_free(cmdbuf);
if (stream)
- drm_free_large(stream);
+ kvfree(stream);
if (bos)
- drm_free_large(bos);
+ kvfree(bos);
if (relocs)
- drm_free_large(relocs);
+ kvfree(relocs);
return ret;
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.c b/drivers/gpu/drm/exynos/exynos_drm_gem.c
index 55a1579d11b3..c23479be4850 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.c
@@ -59,7 +59,8 @@ static int exynos_drm_alloc_buf(struct exynos_drm_gem *exynos_gem)
nr_pages = exynos_gem->size >> PAGE_SHIFT;
- exynos_gem->pages = drm_calloc_large(nr_pages, sizeof(struct page *));
+ exynos_gem->pages = kvmalloc_array(nr_pages, sizeof(struct page *),
+ GFP_KERNEL | __GFP_ZERO);
if (!exynos_gem->pages) {
DRM_ERROR("failed to allocate pages.\n");
return -ENOMEM;
@@ -101,7 +102,7 @@ static int exynos_drm_alloc_buf(struct exynos_drm_gem *exynos_gem)
dma_free_attrs(to_dma_dev(dev), exynos_gem->size, exynos_gem->cookie,
exynos_gem->dma_addr, exynos_gem->dma_attrs);
err_free:
- drm_free_large(exynos_gem->pages);
+ kvfree(exynos_gem->pages);
return ret;
}
@@ -122,7 +123,7 @@ static void exynos_drm_free_buf(struct exynos_drm_gem *exynos_gem)
(dma_addr_t)exynos_gem->dma_addr,
exynos_gem->dma_attrs);
- drm_free_large(exynos_gem->pages);
+ kvfree(exynos_gem->pages);
}
static int exynos_drm_gem_handle_create(struct drm_gem_object *obj,
@@ -559,7 +560,7 @@ exynos_drm_gem_prime_import_sg_table(struct drm_device *dev,
exynos_gem->dma_addr = sg_dma_address(sgt->sgl);
npages = exynos_gem->size >> PAGE_SHIFT;
- exynos_gem->pages = drm_malloc_ab(npages, sizeof(struct page *));
+ exynos_gem->pages = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!exynos_gem->pages) {
ret = -ENOMEM;
goto err;
@@ -588,7 +589,7 @@ exynos_drm_gem_prime_import_sg_table(struct drm_device *dev,
return &exynos_gem->base;
err_free_large:
- drm_free_large(exynos_gem->pages);
+ kvfree(exynos_gem->pages);
err:
drm_gem_object_release(&exynos_gem->base);
kfree(exynos_gem);
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index bd9abef40c66..c8f3c0cc79fb 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -229,7 +229,7 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)
int ret;
total = READ_ONCE(dev_priv->mm.object_count);
- objects = drm_malloc_ab(total, sizeof(*objects));
+ objects = kvmalloc_array(total, sizeof(*objects), GFP_KERNEL);
if (!objects)
return -ENOMEM;
@@ -274,7 +274,7 @@ static int i915_gem_stolen_list_info(struct seq_file *m, void *data)
mutex_unlock(&dev->struct_mutex);
out:
- drm_free_large(objects);
+ kvfree(objects);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 0c1cbe98c994..aa790a6d38e2 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2556,7 +2556,7 @@ static void *i915_gem_object_map(const struct drm_i915_gem_object *obj,
if (n_pages > ARRAY_SIZE(stack_pages)) {
/* Too big for stack -- allocate temporary array instead */
- pages = drm_malloc_gfp(n_pages, sizeof(*pages), GFP_TEMPORARY);
+ pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_TEMPORARY);
if (!pages)
return NULL;
}
@@ -2578,7 +2578,7 @@ static void *i915_gem_object_map(const struct drm_i915_gem_object *obj,
addr = vmap(pages, n_pages, 0, pgprot);
if (pages != stack_pages)
- drm_free_large(pages);
+ kvfree(pages);
return addr;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index af1965774e7b..04211c970b9f 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1019,11 +1019,11 @@ i915_gem_execbuffer_relocate_slow(struct drm_device *dev,
for (i = 0; i < count; i++)
total += exec[i].relocation_count;
- reloc_offset = drm_malloc_ab(count, sizeof(*reloc_offset));
- reloc = drm_malloc_ab(total, sizeof(*reloc));
+ reloc_offset = kvmalloc_array(count, sizeof(*reloc_offset), GFP_KERNEL);
+ reloc = kvmalloc_array(total, sizeof(*reloc), GFP_KERNEL);
if (reloc == NULL || reloc_offset == NULL) {
- drm_free_large(reloc);
- drm_free_large(reloc_offset);
+ kvfree(reloc);
+ kvfree(reloc_offset);
mutex_lock(&dev->struct_mutex);
return -ENOMEM;
}
@@ -1099,8 +1099,8 @@ i915_gem_execbuffer_relocate_slow(struct drm_device *dev,
*/
err:
- drm_free_large(reloc);
- drm_free_large(reloc_offset);
+ kvfree(reloc);
+ kvfree(reloc_offset);
return ret;
}
@@ -1871,13 +1871,13 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
}
/* Copy in the exec list from userland */
- exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
- exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
+ exec_list = kvmalloc_array(sizeof(*exec_list), args->buffer_count, GFP_KERNEL);
+ exec2_list = kvmalloc_array(sizeof(*exec2_list), args->buffer_count, GFP_KERNEL);
if (exec_list == NULL || exec2_list == NULL) {
DRM_DEBUG("Failed to allocate exec list for %d buffers\n",
args->buffer_count);
- drm_free_large(exec_list);
- drm_free_large(exec2_list);
+ kvfree(exec_list);
+ kvfree(exec2_list);
return -ENOMEM;
}
ret = copy_from_user(exec_list,
@@ -1886,8 +1886,8 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
if (ret != 0) {
DRM_DEBUG("copy %d exec entries failed %d\n",
args->buffer_count, ret);
- drm_free_large(exec_list);
- drm_free_large(exec2_list);
+ kvfree(exec_list);
+ kvfree(exec2_list);
return -EFAULT;
}
@@ -1936,8 +1936,8 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
}
}
- drm_free_large(exec_list);
- drm_free_large(exec2_list);
+ kvfree(exec_list);
+ kvfree(exec2_list);
return ret;
}
@@ -1955,7 +1955,7 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
return -EINVAL;
}
- exec2_list = drm_malloc_gfp(args->buffer_count,
+ exec2_list = kvmalloc_array(args->buffer_count,
sizeof(*exec2_list),
GFP_TEMPORARY);
if (exec2_list == NULL) {
@@ -1969,7 +1969,7 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
if (ret != 0) {
DRM_DEBUG("copy %d exec entries failed %d\n",
args->buffer_count, ret);
- drm_free_large(exec2_list);
+ kvfree(exec2_list);
return -EFAULT;
}
@@ -1996,6 +1996,6 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
}
}
- drm_free_large(exec2_list);
+ kvfree(exec2_list);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index bc3c63e92c16..899427863547 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -3114,7 +3114,7 @@ intel_rotate_pages(struct intel_rotation_info *rot_info,
int ret = -ENOMEM;
/* Allocate a temporary list of source pages for random access. */
- page_addr_list = drm_malloc_gfp(n_pages,
+ page_addr_list = kvmalloc_array(n_pages,
sizeof(dma_addr_t),
GFP_TEMPORARY);
if (!page_addr_list)
@@ -3147,14 +3147,14 @@ intel_rotate_pages(struct intel_rotation_info *rot_info,
DRM_DEBUG_KMS("Created rotated page mapping for object size %zu (%ux%u tiles, %u pages)\n",
obj->base.size, rot_info->plane[0].width, rot_info->plane[0].height, size);
- drm_free_large(page_addr_list);
+ kvfree(page_addr_list);
return st;
err_sg_alloc:
kfree(st);
err_st_alloc:
- drm_free_large(page_addr_list);
+ kvfree(page_addr_list);
DRM_DEBUG_KMS("Failed to create rotated mapping for object size %zu! (%ux%u tiles, %u pages)\n",
obj->base.size, rot_info->plane[0].width, rot_info->plane[0].height, size);
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index 58ccf8b8ca1c..1a0ce1dc68f5 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -507,7 +507,7 @@ __i915_gem_userptr_get_pages_worker(struct work_struct *_work)
ret = -ENOMEM;
pinned = 0;
- pvec = drm_malloc_gfp(npages, sizeof(struct page *), GFP_TEMPORARY);
+ pvec = kvmalloc_array(npages, sizeof(struct page *), GFP_TEMPORARY);
if (pvec != NULL) {
struct mm_struct *mm = obj->userptr.mm->mm;
unsigned int flags = 0;
@@ -555,7 +555,7 @@ __i915_gem_userptr_get_pages_worker(struct work_struct *_work)
mutex_unlock(&obj->mm.lock);
release_pages(pvec, pinned, 0);
- drm_free_large(pvec);
+ kvfree(pvec);
i915_gem_object_put(obj);
put_task_struct(work->task);
@@ -642,7 +642,7 @@ i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
pinned = 0;
if (mm == current->mm) {
- pvec = drm_malloc_gfp(num_pages, sizeof(struct page *),
+ pvec = kvmalloc_array(num_pages, sizeof(struct page *),
GFP_TEMPORARY |
__GFP_NORETRY |
__GFP_NOWARN);
@@ -669,7 +669,7 @@ i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
if (IS_ERR(pages))
release_pages(pvec, pinned, 0);
- drm_free_large(pvec);
+ kvfree(pvec);
return pages;
}
diff --git a/drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c b/drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c
index 19860a372d90..7276194c04f7 100644
--- a/drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c
+++ b/drivers/gpu/drm/i915/selftests/intel_breadcrumbs.c
@@ -117,7 +117,7 @@ static int igt_random_insert_remove(void *arg)
mock_engine_reset(engine);
- waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ waiters = kvmalloc_array(count, sizeof(*waiters), GFP_TEMPORARY);
if (!waiters)
goto out_engines;
@@ -169,7 +169,7 @@ static int igt_random_insert_remove(void *arg)
out_bitmap:
kfree(bitmap);
out_waiters:
- drm_free_large(waiters);
+ kvfree(waiters);
out_engines:
mock_engine_flush(engine);
return err;
@@ -187,7 +187,7 @@ static int igt_insert_complete(void *arg)
mock_engine_reset(engine);
- waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ waiters = kvmalloc_array(count, sizeof(*waiters), GFP_TEMPORARY);
if (!waiters)
goto out_engines;
@@ -254,7 +254,7 @@ static int igt_insert_complete(void *arg)
out_bitmap:
kfree(bitmap);
out_waiters:
- drm_free_large(waiters);
+ kvfree(waiters);
out_engines:
mock_engine_flush(engine);
return err;
@@ -368,7 +368,7 @@ static int igt_wakeup(void *arg)
mock_engine_reset(engine);
- waiters = drm_malloc_gfp(count, sizeof(*waiters), GFP_TEMPORARY);
+ waiters = kvmalloc_array(count, sizeof(*waiters), GFP_TEMPORARY);
if (!waiters)
goto out_engines;
@@ -454,7 +454,7 @@ static int igt_wakeup(void *arg)
put_task_struct(waiters[n].tsk);
}
- drm_free_large(waiters);
+ kvfree(waiters);
out_engines:
mock_engine_flush(engine);
return err;
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 68e509b3b9e4..465dab942afa 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -50,13 +50,13 @@ static struct page **get_pages_vram(struct drm_gem_object *obj,
struct page **p;
int ret, i;
- p = drm_malloc_ab(npages, sizeof(struct page *));
+ p = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!p)
return ERR_PTR(-ENOMEM);
ret = drm_mm_insert_node(&priv->vram.mm, msm_obj->vram_node, npages);
if (ret) {
- drm_free_large(p);
+ kvfree(p);
return ERR_PTR(ret);
}
@@ -127,7 +127,7 @@ static void put_pages(struct drm_gem_object *obj)
drm_gem_put_pages(obj, msm_obj->pages, true, false);
else {
drm_mm_remove_node(msm_obj->vram_node);
- drm_free_large(msm_obj->pages);
+ kvfree(msm_obj->pages);
}
msm_obj->pages = NULL;
@@ -707,7 +707,7 @@ void msm_gem_free_object(struct drm_gem_object *obj)
* ours, just free the array we allocated:
*/
if (msm_obj->pages)
- drm_free_large(msm_obj->pages);
+ kvfree(msm_obj->pages);
drm_prime_gem_destroy(obj, msm_obj->sgt);
} else {
@@ -863,7 +863,7 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev,
msm_obj = to_msm_bo(obj);
msm_obj->sgt = sgt;
- msm_obj->pages = drm_malloc_ab(npages, sizeof(struct page *));
+ msm_obj->pages = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (!msm_obj->pages) {
ret = -ENOMEM;
goto fail;
diff --git a/drivers/gpu/drm/radeon/radeon_cs.c b/drivers/gpu/drm/radeon/radeon_cs.c
index 3ac671f6c8e1..00b22af70f5c 100644
--- a/drivers/gpu/drm/radeon/radeon_cs.c
+++ b/drivers/gpu/drm/radeon/radeon_cs.c
@@ -87,7 +87,8 @@ static int radeon_cs_parser_relocs(struct radeon_cs_parser *p)
p->dma_reloc_idx = 0;
/* FIXME: we assume that each relocs use 4 dwords */
p->nrelocs = chunk->length_dw / 4;
- p->relocs = drm_calloc_large(p->nrelocs, sizeof(struct radeon_bo_list));
+ p->relocs = kvmalloc_array(p->nrelocs, sizeof(struct radeon_bo_list),
+ GFP_KERNEL | __GFP_ZERO);
if (p->relocs == NULL) {
return -ENOMEM;
}
@@ -341,7 +342,7 @@ int radeon_cs_parser_init(struct radeon_cs_parser *p, void *data)
continue;
}
- p->chunks[i].kdata = drm_malloc_ab(size, sizeof(uint32_t));
+ p->chunks[i].kdata = kvmalloc_array(size, sizeof(uint32_t), GFP_KERNEL);
size *= sizeof(uint32_t);
if (p->chunks[i].kdata == NULL) {
return -ENOMEM;
@@ -440,10 +441,10 @@ static void radeon_cs_parser_fini(struct radeon_cs_parser *parser, int error, bo
}
}
kfree(parser->track);
- drm_free_large(parser->relocs);
- drm_free_large(parser->vm_bos);
+ kvfree(parser->relocs);
+ kvfree(parser->vm_bos);
for (i = 0; i < parser->nchunks; i++)
- drm_free_large(parser->chunks[i].kdata);
+ kvfree(parser->chunks[i].kdata);
kfree(parser->chunks);
kfree(parser->chunks_array);
radeon_ib_free(parser->rdev, &parser->ib);
diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index dddb372de2b9..574bf7e6b118 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -587,7 +587,7 @@ static void radeon_gem_va_update_vm(struct radeon_device *rdev,
ttm_eu_backoff_reservation(&ticket, &list);
error_free:
- drm_free_large(vm_bos);
+ kvfree(vm_bos);
if (r && r != -ERESTARTSYS)
DRM_ERROR("Couldn't update BO_VA (%d)\n", r);
diff --git a/drivers/gpu/drm/radeon/radeon_ring.c b/drivers/gpu/drm/radeon/radeon_ring.c
index 8c7872339c2a..84802b201bef 100644
--- a/drivers/gpu/drm/radeon/radeon_ring.c
+++ b/drivers/gpu/drm/radeon/radeon_ring.c
@@ -314,7 +314,7 @@ unsigned radeon_ring_backup(struct radeon_device *rdev, struct radeon_ring *ring
}
/* and then save the content of the ring */
- *data = drm_malloc_ab(size, sizeof(uint32_t));
+ *data = kvmalloc_array(size, sizeof(uint32_t), GFP_KERNEL);
if (!*data) {
mutex_unlock(&rdev->ring_lock);
return 0;
@@ -356,7 +356,7 @@ int radeon_ring_restore(struct radeon_device *rdev, struct radeon_ring *ring,
}
radeon_ring_unlock_commit(rdev, ring, false);
- drm_free_large(data);
+ kvfree(data);
return 0;
}
diff --git a/drivers/gpu/drm/radeon/radeon_vm.c b/drivers/gpu/drm/radeon/radeon_vm.c
index a1358748cea5..5f68245579a3 100644
--- a/drivers/gpu/drm/radeon/radeon_vm.c
+++ b/drivers/gpu/drm/radeon/radeon_vm.c
@@ -132,8 +132,8 @@ struct radeon_bo_list *radeon_vm_get_bos(struct radeon_device *rdev,
struct radeon_bo_list *list;
unsigned i, idx;
- list = drm_malloc_ab(vm->max_pde_used + 2,
- sizeof(struct radeon_bo_list));
+ list = kvmalloc_array(vm->max_pde_used + 2,
+ sizeof(struct radeon_bo_list), GFP_KERNEL);
if (!list)
return NULL;
diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c
index 5260179d788a..8ebc8d3560c3 100644
--- a/drivers/gpu/drm/ttm/ttm_tt.c
+++ b/drivers/gpu/drm/ttm/ttm_tt.c
@@ -39,7 +39,6 @@
#include <linux/slab.h>
#include <linux/export.h>
#include <drm/drm_cache.h>
-#include <drm/drm_mem_util.h>
#include <drm/ttm/ttm_module.h>
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_placement.h>
@@ -53,14 +52,16 @@
*/
static void ttm_tt_alloc_page_directory(struct ttm_tt *ttm)
{
- ttm->pages = drm_calloc_large(ttm->num_pages, sizeof(void*));
+ ttm->pages = kvmalloc_array(ttm->num_pages, sizeof(void*),
+ GFP_KERNEL | __GFP_ZERO);
}
static void ttm_dma_tt_alloc_page_directory(struct ttm_dma_tt *ttm)
{
- ttm->ttm.pages = drm_calloc_large(ttm->ttm.num_pages,
+ ttm->ttm.pages = kvmalloc_array(ttm->ttm.num_pages,
sizeof(*ttm->ttm.pages) +
- sizeof(*ttm->dma_address));
+ sizeof(*ttm->dma_address),
+ GFP_KERNEL | __GFP_ZERO);
ttm->dma_address = (void *) (ttm->ttm.pages + ttm->ttm.num_pages);
}
@@ -208,7 +209,7 @@ EXPORT_SYMBOL(ttm_tt_init);
void ttm_tt_fini(struct ttm_tt *ttm)
{
- drm_free_large(ttm->pages);
+ kvfree(ttm->pages);
ttm->pages = NULL;
}
EXPORT_SYMBOL(ttm_tt_fini);
@@ -243,7 +244,7 @@ void ttm_dma_tt_fini(struct ttm_dma_tt *ttm_dma)
{
struct ttm_tt *ttm = &ttm_dma->ttm;
- drm_free_large(ttm->pages);
+ kvfree(ttm->pages);
ttm->pages = NULL;
ttm_dma->dma_address = NULL;
}
diff --git a/drivers/gpu/drm/udl/udl_dmabuf.c b/drivers/gpu/drm/udl/udl_dmabuf.c
index ed0e636243b2..2e031a894813 100644
--- a/drivers/gpu/drm/udl/udl_dmabuf.c
+++ b/drivers/gpu/drm/udl/udl_dmabuf.c
@@ -228,7 +228,7 @@ static int udl_prime_create(struct drm_device *dev,
return -ENOMEM;
obj->sg = sg;
- obj->pages = drm_malloc_ab(npages, sizeof(struct page *));
+ obj->pages = kvmalloc_array(npages, sizeof(struct page *), GFP_KERNEL);
if (obj->pages == NULL) {
DRM_ERROR("obj pages is NULL %d\n", npages);
return -ENOMEM;
diff --git a/drivers/gpu/drm/udl/udl_gem.c b/drivers/gpu/drm/udl/udl_gem.c
index 775c50e4f02c..db9ceceba30e 100644
--- a/drivers/gpu/drm/udl/udl_gem.c
+++ b/drivers/gpu/drm/udl/udl_gem.c
@@ -146,7 +146,7 @@ int udl_gem_get_pages(struct udl_gem_object *obj)
void udl_gem_put_pages(struct udl_gem_object *obj)
{
if (obj->base.import_attach) {
- drm_free_large(obj->pages);
+ kvfree(obj->pages);
obj->pages = NULL;
return;
}
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index e9c381c42139..bf466674ca9b 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -545,14 +545,15 @@ vc4_cl_lookup_bos(struct drm_device *dev,
return -EINVAL;
}
- exec->bo = drm_calloc_large(exec->bo_count,
- sizeof(struct drm_gem_cma_object *));
+ exec->bo = kvmalloc_array(exec->bo_count,
+ sizeof(struct drm_gem_cma_object *),
+ GFP_KERNEL | __GFP_ZERO);
if (!exec->bo) {
DRM_ERROR("Failed to allocate validated BO pointers\n");
return -ENOMEM;
}
- handles = drm_malloc_ab(exec->bo_count, sizeof(uint32_t));
+ handles = kvmalloc_array(exec->bo_count, sizeof(uint32_t), GFP_KERNEL);
if (!handles) {
ret = -ENOMEM;
DRM_ERROR("Failed to allocate incoming GEM handles\n");
@@ -584,7 +585,7 @@ vc4_cl_lookup_bos(struct drm_device *dev,
spin_unlock(&file_priv->table_lock);
fail:
- drm_free_large(handles);
+ kvfree(handles);
return ret;
}
@@ -622,7 +623,7 @@ vc4_get_bcl(struct drm_device *dev, struct vc4_exec_info *exec)
* read the contents back for validation, and I think the
* bo->vaddr is uncached access.
*/
- temp = drm_malloc_ab(temp_size, 1);
+ temp = kvmalloc_array(temp_size, 1, GFP_KERNEL);
if (!temp) {
DRM_ERROR("Failed to allocate storage for copying "
"in bin/render CLs.\n");
@@ -697,7 +698,7 @@ vc4_get_bcl(struct drm_device *dev, struct vc4_exec_info *exec)
ret = vc4_wait_for_seqno(dev, exec->bin_dep_seqno, ~0ull, true);
fail:
- drm_free_large(temp);
+ kvfree(temp);
return ret;
}
@@ -710,7 +711,7 @@ vc4_complete_exec(struct drm_device *dev, struct vc4_exec_info *exec)
if (exec->bo) {
for (i = 0; i < exec->bo_count; i++)
drm_gem_object_unreference_unlocked(&exec->bo[i]->base);
- drm_free_large(exec->bo);
+ kvfree(exec->bo);
}
while (!list_empty(&exec->unref_list)) {
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index 61f3a963af95..6ed4bfc9b82b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -119,13 +119,14 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
INIT_LIST_HEAD(&validate_list);
if (exbuf->num_bo_handles) {
- bo_handles = drm_malloc_ab(exbuf->num_bo_handles,
- sizeof(uint32_t));
- buflist = drm_calloc_large(exbuf->num_bo_handles,
- sizeof(struct ttm_validate_buffer));
+ bo_handles = kvmalloc_array(exbuf->num_bo_handles,
+ sizeof(uint32_t), GFP_KERNEL);
+ buflist = kvmalloc_array(exbuf->num_bo_handles,
+ sizeof(struct ttm_validate_buffer),
+ GFP_KERNEL | __GFP_ZERO);
if (!bo_handles || !buflist) {
- drm_free_large(bo_handles);
- drm_free_large(buflist);
+ kvfree(bo_handles);
+ kvfree(buflist);
return -ENOMEM;
}
@@ -133,16 +134,16 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
if (copy_from_user(bo_handles, user_bo_handles,
exbuf->num_bo_handles * sizeof(uint32_t))) {
ret = -EFAULT;
- drm_free_large(bo_handles);
- drm_free_large(buflist);
+ kvfree(bo_handles);
+ kvfree(buflist);
return ret;
}
for (i = 0; i < exbuf->num_bo_handles; i++) {
gobj = drm_gem_object_lookup(drm_file, bo_handles[i]);
if (!gobj) {
- drm_free_large(bo_handles);
- drm_free_large(buflist);
+ kvfree(bo_handles);
+ kvfree(buflist);
return -ENOENT;
}
@@ -151,7 +152,7 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
list_add(&buflist[i].head, &validate_list);
}
- drm_free_large(bo_handles);
+ kvfree(bo_handles);
}
ret = virtio_gpu_object_list_validate(&ticket, &validate_list);
@@ -171,7 +172,7 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
/* fence the command bo */
virtio_gpu_unref_list(&validate_list);
- drm_free_large(buflist);
+ kvfree(buflist);
dma_fence_put(&fence->f);
return 0;
@@ -179,7 +180,7 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
ttm_eu_backoff_reservation(&ticket, &validate_list);
out_free:
virtio_gpu_unref_list(&validate_list);
- drm_free_large(buflist);
+ kvfree(buflist);
return ret;
}
diff --git a/include/drm/drmP.h b/include/drm/drmP.h
index e1daa4f343cd..59df08d14b89 100644
--- a/include/drm/drmP.h
+++ b/include/drm/drmP.h
@@ -70,7 +70,6 @@
#include <drm/drm_fourcc.h>
#include <drm/drm_global.h>
#include <drm/drm_hashtab.h>
-#include <drm/drm_mem_util.h>
#include <drm/drm_mm.h>
#include <drm/drm_os_linux.h>
#include <drm/drm_sarea.h>
diff --git a/include/drm/drm_mem_util.h b/include/drm/drm_mem_util.h
deleted file mode 100644
index a1ddf55fda67..000000000000
--- a/include/drm/drm_mem_util.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/*
- * Copyright A(C) 2008 Intel Corporation
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- *
- * Authors:
- * Jesse Barnes <jbarnes@virtuousgeek.org>
- *
- */
-#ifndef _DRM_MEM_UTIL_H_
-#define _DRM_MEM_UTIL_H_
-
-#include <linux/vmalloc.h>
-
-static __inline__ void *drm_calloc_large(size_t nmemb, size_t size)
-{
- return kvmalloc_array(nmemb, size, GFP_KERNEL | __GFP_ZERO);
-}
-
-/* Modeled after cairo's malloc_ab, it's like calloc but without the zeroing. */
-static __inline__ void *drm_malloc_ab(size_t nmemb, size_t size)
-{
- return kvmalloc_array(nmemb, size, GFP_KERNEL);
-}
-
-static __inline__ void *drm_malloc_gfp(size_t nmemb, size_t size, gfp_t gfp)
-{
- return kvmalloc_array(nmemb, size, gfp);
-}
-
-static __inline void drm_free_large(void *ptr)
-{
- kvfree(ptr);
-}
-
-#endif
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [patch V2 15/17] mm/vmscan: Adjust system_state checks
From: Vlastimil Babka @ 2017-05-17 6:56 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: Peter Zijlstra, Ingo Molnar, Steven Rostedt, Mark Rutland,
Greg Kroah-Hartman, Andrew Morton, Johannes Weiner, Mel Gorman,
Michal Hocko, linux-mm
In-Reply-To: <20170516184736.119158930@linutronix.de>
On 05/16/2017 08:42 PM, Thomas Gleixner wrote:
> To enable smp_processor_id() and might_sleep() debug checks earlier, it's
> required to add system states between SYSTEM_BOOTING and SYSTEM_RUNNING.
>
> Adjust the system_state check in kswapd_run() to handle the extra states.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Mel Gorman <mgorman@techsingularity.net>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: linux-mm@kvack.org
Acked-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> mm/vmscan.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -3643,7 +3643,7 @@ int kswapd_run(int nid)
> pgdat->kswapd = kthread_run(kswapd, pgdat, "kswapd%d", nid);
> if (IS_ERR(pgdat->kswapd)) {
> /* failure at boot is fatal */
> - BUG_ON(system_state == SYSTEM_BOOTING);
> + BUG_ON(system_state < SYSTEM_RUNNING);
> pr_err("Failed to start kswapd on node %d\n", nid);
> ret = PTR_ERR(pgdat->kswapd);
> pgdat->kswapd = NULL;
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 06/32] x86/mm: Add Secure Memory Encryption (SME) support
From: Borislav Petkov @ 2017-05-17 7:05 UTC (permalink / raw)
To: Tom Lendacky
Cc: linux-arch, linux-efi, kvm, linux-doc, x86, kexec, linux-kernel,
kasan-dev, linux-mm, iommu, Rik van Riel,
Radim Krčmář, Toshimitsu Kani, Arnd Bergmann,
Jonathan Corbet, Matt Fleming, Michael S. Tsirkin, Joerg Roedel,
Konrad Rzeszutek Wilk, Paolo Bonzini, Larry Woodman,
Brijesh Singh, Ingo Molnar, Andy Lutomirski, H. Peter Anvin,
Andrey Ryabinin, Alexander Potapenko, Dave Young, Thomas Gleixner,
Dmitry Vyukov
In-Reply-To: <6d266f5b-c28d-fe19-24b5-5133532f9eea@amd.com>
On Tue, May 16, 2017 at 02:28:42PM -0500, Tom Lendacky wrote:
> It's most problematic when CONFIG_AMD_MEM_ENCRYPT is not defined since
> we never include an asm/ version from the linux/ path. I could create
> a mem_encrypt.h in include/asm-generic/ that contains the info that
> is in the !CONFIG_AMD_MEM_ENCRYPT path of the linux/ version. Let me
> look into that.
So we need to keep asm/ and linux/ apart. The linux/ stuff is generic,
global, more or less. The asm/ is arch-specific. So they shouldn't be
overlapping wrt definitions, IMHO.
So asm-generic is the proper approach here because then you won't need
the ifndef fun.
Thanks.
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption
From: Joonsoo Kim @ 2017-05-17 7:23 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Andrew Morton, Andrey Ryabinin, Alexander Potapenko, kasan-dev,
linux-mm@kvack.org, LKML, Thomas Gleixner, Ingo Molnar,
H . Peter Anvin, kernel-team
In-Reply-To: <CACT4Y+anOw8=7u-pZ2ceMw0xVnuaO9YKBJAr-2=KOYt_72b2pw@mail.gmail.com>
On Tue, May 16, 2017 at 01:49:10PM -0700, Dmitry Vyukov wrote:
> > Anyway, I have missed inline instrumentation completely.
> >
> > I will attach the fix in the bottom. It doesn't look beautiful
> > since it breaks layer design (some check will be done at report
> > function). However, I think that it's a good trade-off.
>
>
> I can confirm that inline works with that patch.
Thanks for confirming!
>
> I can also confirm that it reduces memory usage. I've booted qemu with
> 2G ram and run some fixed workload. Before:
> 31853 dvyukov 20 0 3043200 765464 21312 S 366.0 4.7 2:39.53
> qemu-system-x86
> 7528 dvyukov 20 0 3043200 732444 21676 S 333.3 4.5 2:23.19
> qemu-system-x86
> After:
> 6192 dvyukov 20 0 3043200 394244 20636 S 17.9 2.4 2:32.95
> qemu-system-x86
> 6265 dvyukov 20 0 3043200 388860 21416 S 399.3 2.4 3:02.88
> qemu-system-x86
> 9005 dvyukov 20 0 3043200 383564 21220 S 397.1 2.3 2:35.33
> qemu-system-x86
>
> However, I see some very significant slowdowns with inline
> instrumentation. I did 3 tests:
> 1. Boot speed, I measured time for a particular message to appear on
> console. Before:
> [ 2.504652] random: crng init done
> [ 2.435861] random: crng init done
> [ 2.537135] random: crng init done
> After:
> [ 7.263402] random: crng init done
> [ 7.263402] random: crng init done
> [ 7.174395] random: crng init done
>
> That's ~3x slowdown.
>
> 2. I've run bench_readv benchmark:
> https://raw.githubusercontent.com/google/sanitizers/master/address-sanitizer/kernel_buildbot/slave/bench_readv.c
> as:
> while true; do time ./bench_readv bench_readv 300000 1; done
>
> Before:
> sys 0m7.299s
> sys 0m7.218s
> sys 0m6.973s
> sys 0m6.892s
> sys 0m7.035s
> sys 0m6.982s
> sys 0m6.921s
> sys 0m6.940s
> sys 0m6.905s
> sys 0m7.006s
>
> After:
> sys 0m8.141s
> sys 0m8.077s
> sys 0m8.067s
> sys 0m8.116s
> sys 0m8.128s
> sys 0m8.115s
> sys 0m8.108s
> sys 0m8.326s
> sys 0m8.529s
> sys 0m8.164s
> sys 0m8.380s
>
> This is ~19% slowdown.
>
> 3. I've run bench_pipes benchmark:
> https://raw.githubusercontent.com/google/sanitizers/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c
> as:
> while true; do time ./bench_pipes 10 10000 1; done
>
> Before:
> sys 0m5.393s
> sys 0m6.178s
> sys 0m5.909s
> sys 0m6.024s
> sys 0m5.874s
> sys 0m5.737s
> sys 0m5.826s
> sys 0m5.664s
> sys 0m5.758s
> sys 0m5.421s
> sys 0m5.444s
> sys 0m5.479s
> sys 0m5.461s
> sys 0m5.417s
>
> After:
> sys 0m8.718s
> sys 0m8.281s
> sys 0m8.268s
> sys 0m8.334s
> sys 0m8.246s
> sys 0m8.267s
> sys 0m8.265s
> sys 0m8.437s
> sys 0m8.228s
> sys 0m8.312s
> sys 0m8.556s
> sys 0m8.680s
>
> This is ~52% slowdown.
>
>
> This does not look acceptable to me. I would ready to pay for this,
> say, 10% of performance. But it seems that this can have up to 2-4x
> slowdown for some workloads.
I found the reasons of above regression. There are two reasons.
1. In my implementation, original shadow to the memory allocated from
memblock is black shadow so it causes to call kasan_report(). It will
pass the check since per page shadow would be zero shadow but it
causes some overhead.
2. Memory used by stackdepot is in a similar situation with #1. It
allocates page and divide it to many objects. Then, use it like as
object. Although there is "KASAN_SANITIZE_stackdepot.o := n" which try
to disable sanitizer, there is a function call (memcmp() in
find_stack()) to other file and sanitizer work for it.
#1 problem can be fixed but more investigation is needed. I will
respin the series after fixing it.
#2 problem also can be fixed. There are two options here. First, uses
private memcmp() for stackdepot and disable sanitizer for it. I think
that this is a right approach since it slowdown the performance in all
KASAN build cases. And, we don't want to sanitize KASAN itself.
Second, I can provide a function to map the actual shadow manually. It
will reduce the case calling kasan_report().
See the attached patch. It implements later approach on #2 problem.
It would reduce performance regression. I have tested your bench_pipes
test with it and found that performance is restored. However, there is
still remaining problem, #1, so I'm not sure that it completely
restore your regression. Could you check that if possible?
Anyway, I think that respin is needed to fix this performance problem
completely.
>
>
> Your use-case is embed devices where you care a lot about both code
> size and memory consumption, right?
Yes.
> I see 2 possible ways forward:
> 1. Enable this new mode only for outline, but keep current scheme for
> inline. Then outline will be "small but slow" type of configuration.
Performance problem is not that bad in OUTLINE build. Therefore,
this is a reasonable option to have.
> 2. Somehow fix slowness (at least in inline mode).
I will try to fix slowness as much as possible. If slowness cannot be
acceptable after such effort, we can choose the direction at that
moment.
>
> > Mapping zero page to non-kernel memory could cause true-negative
> > problem since we cannot flush the TLB in all cpus. We will read zero
> > shadow value value in this case even if actual shadow value is not
> > zero. This is one of the reason that black page is introduced in this
> > patchset.
>
> What does make your current patch work then?
> Say we map a new shadow page, update the page shadow to say that there
> is mapped shadow. Then another CPU loads the page shadow and then
> loads from the newly mapped shadow. If we don't flush TLB, what makes
> the second CPU see the newly mapped shadow?
There is a fix-up processing to see the newly mapped shadow in other
cpus. check_memory_region_slow() exists for that purpose. In stale TLB
case, we will see black shadow and fall in to this function. In this
function, we flush stale TLB and re-check so we can see correct
result.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption
From: Joonsoo Kim @ 2017-05-17 7:25 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Andrew Morton, Andrey Ryabinin, Alexander Potapenko, kasan-dev,
linux-mm@kvack.org, LKML, Thomas Gleixner, Ingo Molnar,
H . Peter Anvin, kernel-team
In-Reply-To: <20170517072315.GA18406@js1304-desktop>
On Wed, May 17, 2017 at 04:23:17PM +0900, Joonsoo Kim wrote:
> > However, I see some very significant slowdowns with inline
> > instrumentation. I did 3 tests:
> > 1. Boot speed, I measured time for a particular message to appear on
> > console. Before:
> > [ 2.504652] random: crng init done
> > [ 2.435861] random: crng init done
> > [ 2.537135] random: crng init done
> > After:
> > [ 7.263402] random: crng init done
> > [ 7.263402] random: crng init done
> > [ 7.174395] random: crng init done
> >
> > That's ~3x slowdown.
> >
> > 2. I've run bench_readv benchmark:
> > https://raw.githubusercontent.com/google/sanitizers/master/address-sanitizer/kernel_buildbot/slave/bench_readv.c
> > as:
> > while true; do time ./bench_readv bench_readv 300000 1; done
> >
> > Before:
> > sys 0m7.299s
> > sys 0m7.218s
> > sys 0m6.973s
> > sys 0m6.892s
> > sys 0m7.035s
> > sys 0m6.982s
> > sys 0m6.921s
> > sys 0m6.940s
> > sys 0m6.905s
> > sys 0m7.006s
> >
> > After:
> > sys 0m8.141s
> > sys 0m8.077s
> > sys 0m8.067s
> > sys 0m8.116s
> > sys 0m8.128s
> > sys 0m8.115s
> > sys 0m8.108s
> > sys 0m8.326s
> > sys 0m8.529s
> > sys 0m8.164s
> > sys 0m8.380s
> >
> > This is ~19% slowdown.
> >
> > 3. I've run bench_pipes benchmark:
> > https://raw.githubusercontent.com/google/sanitizers/master/address-sanitizer/kernel_buildbot/slave/bench_pipes.c
> > as:
> > while true; do time ./bench_pipes 10 10000 1; done
> >
> > Before:
> > sys 0m5.393s
> > sys 0m6.178s
> > sys 0m5.909s
> > sys 0m6.024s
> > sys 0m5.874s
> > sys 0m5.737s
> > sys 0m5.826s
> > sys 0m5.664s
> > sys 0m5.758s
> > sys 0m5.421s
> > sys 0m5.444s
> > sys 0m5.479s
> > sys 0m5.461s
> > sys 0m5.417s
> >
> > After:
> > sys 0m8.718s
> > sys 0m8.281s
> > sys 0m8.268s
> > sys 0m8.334s
> > sys 0m8.246s
> > sys 0m8.267s
> > sys 0m8.265s
> > sys 0m8.437s
> > sys 0m8.228s
> > sys 0m8.312s
> > sys 0m8.556s
> > sys 0m8.680s
> >
> > This is ~52% slowdown.
> >
> >
> > This does not look acceptable to me. I would ready to pay for this,
> > say, 10% of performance. But it seems that this can have up to 2-4x
> > slowdown for some workloads.
>
> I found the reasons of above regression. There are two reasons.
>
> 1. In my implementation, original shadow to the memory allocated from
> memblock is black shadow so it causes to call kasan_report(). It will
> pass the check since per page shadow would be zero shadow but it
> causes some overhead.
>
> 2. Memory used by stackdepot is in a similar situation with #1. It
> allocates page and divide it to many objects. Then, use it like as
> object. Although there is "KASAN_SANITIZE_stackdepot.o := n" which try
> to disable sanitizer, there is a function call (memcmp() in
> find_stack()) to other file and sanitizer work for it.
>
> #1 problem can be fixed but more investigation is needed. I will
> respin the series after fixing it.
>
> #2 problem also can be fixed. There are two options here. First, uses
> private memcmp() for stackdepot and disable sanitizer for it. I think
> that this is a right approach since it slowdown the performance in all
> KASAN build cases. And, we don't want to sanitize KASAN itself.
> Second, I can provide a function to map the actual shadow manually. It
> will reduce the case calling kasan_report().
>
> See the attached patch. It implements later approach on #2 problem.
> It would reduce performance regression. I have tested your bench_pipes
> test with it and found that performance is restored. However, there is
> still remaining problem, #1, so I'm not sure that it completely
> restore your regression. Could you check that if possible?
>
Oops... I missed to attach the patch.
Thanks.
--------------------->8-------------------
^ permalink raw reply
* Re: [PATCH 1/2] drm: replace drm_[cm]alloc* by kvmalloc alternatives
From: Chris Wilson @ 2017-05-17 7:38 UTC (permalink / raw)
To: Michal Hocko
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie, Michal Hocko
In-Reply-To: <20170517065509.18659-1-mhocko@kernel.org>
On Wed, May 17, 2017 at 08:55:08AM +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> drm_[cm]alloc* has grown their own kvmalloc with vmalloc fallback
> implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> use those because it a) reduces the code and b) MM has a better idea
> how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> with __GFP_NORETRY).
>
> drm_calloc_large needs to get __GFP_ZERO explicitly but it is the same
> thing as kvmalloc_array in principle.
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
Just a little surprised that calloc_large users still exist.
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
One more feature request from mm, can we have the
if (size != 0 && n > SIZE_MAX / size)
check exported by itself. It is used by both kvmalloc_array and
kmalloc_array, and in my ioctls I have it open-coded as well to
differentiate between the -EINVAL (for bogus user values) and
genuine -ENOMEM.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Joonsoo Kim @ 2017-05-17 7:44 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, mgorman,
Laura Abbott, Minchan Kim, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K . V, Vlastimil Babka, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <20170516084734.GC2481@dhcp22.suse.cz>
> >
> > Okay. We did a lot of discussion so it's better to summarise it.
> >
> > 1. ZONE_CMA might be a nicer solution than MIGRATETYPE.
> > 2. Additional bit in page flags would cause another kind of
> > maintenance problem so it's better to avoid it as much as possible.
> > 3. Abusing ZONE_MOVABLE looks better than introducing ZONE_CMA since
> > it doesn't need additional bit in page flag.
> > 4. (Not-yet-finished) If ZONE_CMA doesn't need extra bit in page
> > flags with hacky magic and it has no performance regression,
> > ??? (it's okay to use separate zone for CMA?)
>
> As mentioned above. I do not see why we should go over additional hops
> just to have a zone which is not strictly needed. So if there are no
> inherent problems reusing MOVABLE/HIGMEM zone then a separate zone
> sounds like a wrong direction.
>
> But let me repeat. I am _not_ convinced that the migratetype situation
> is all that bad and unfixable. You have mentioned some issues with the
> current approach but none of them seem inherently unfixable. So I would
> still prefer keeping the current way. But I am not going to insist if
> you _really_ believe that the long term maintenance cost will be higher
> than a zone approach and you can reuse MOVABLE/HIGHMEM zones without
> disruptive changes. I can help you with the hotplug part of the MOVABLE
> zone because that is desirable on its own.
Okay. Thanks for sharing your opinion. I will decide the final
direction after some investigation.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] drm: use kvmalloc_array for drm_malloc*
From: Michal Hocko @ 2017-05-17 7:44 UTC (permalink / raw)
To: Chris Wilson
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170516110908.GE26693@nuc-i3427.alporthouse.com>
On Tue 16-05-17 12:09:08, Chris Wilson wrote:
> On Tue, May 16, 2017 at 12:53:52PM +0200, Michal Hocko wrote:
> > On Tue 16-05-17 10:31:19, Chris Wilson wrote:
> > > On Tue, May 16, 2017 at 11:06:06AM +0200, Michal Hocko wrote:
> > > > From: Michal Hocko <mhocko@suse.com>
> > > >
> > > > drm_malloc* has grown their own kmalloc with vmalloc fallback
> > > > implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> > > > use those because it a) reduces the code and b) MM has a better idea
> > > > how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> > > > with __GFP_NORETRY).
> > >
> > > Better? The same idea. The only difference I was reluctant to hand out
> > > large pages for long lived objects. If that's the wisdom of the core mm,
> > > so be it.
> >
> > vmalloc tends to fragment physical memory more os it is preferable to
> > try the physically contiguous request first and only fall back to
> > vmalloc if the first attempt would be too costly or it fails.
>
> Not relevant for the changelog in this patch, but it would be nice to
> have that written in kvmalloc() as to why the scatterring of 4k vmapped
> pages prevents defragmentation when compared to allocating large pages.
Well, it is not as much about defragmentation because both vmapped and
kmalloc allocations are very likely to be unmovable (at least
currently). Theoretically there shouldn't be a problem to make vmapped
pages movable as the ptes can be modified but this is not implemented...
The problem is that vmapped pages are more likely to break up more
larger order blocks. kmalloc will naturally break a single larger block.
> I have vague recollections of seeing the conversation, but a summary as
> to the reason why kvmalloc prefers large pages will be good for future
> reference.
Does the following sound better to you?
diff --git a/mm/util.c b/mm/util.c
index 464df3489903..87499f8119f2 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -357,7 +357,10 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node)
WARN_ON_ONCE((flags & GFP_KERNEL) != GFP_KERNEL);
/*
- * Make sure that larger requests are not too disruptive - no OOM
+ * We want to attempt a large physically contiguous block first because
+ * it is less likely to fragment multiple larger blocks and therefore
+ * contribute to a long term fragmentation less than vmalloc fallback.
+ * However make sure that larger requests are not too disruptive - no OOM
* killer and no allocation failure warnings as we have a fallback
*/
if (size > PAGE_SIZE) {
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH] drm: use kvmalloc_array for drm_malloc*
From: Chris Wilson @ 2017-05-17 7:59 UTC (permalink / raw)
To: Michal Hocko
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170517074453.GC18247@dhcp22.suse.cz>
On Wed, May 17, 2017 at 09:44:53AM +0200, Michal Hocko wrote:
> On Tue 16-05-17 12:09:08, Chris Wilson wrote:
> > On Tue, May 16, 2017 at 12:53:52PM +0200, Michal Hocko wrote:
> > > On Tue 16-05-17 10:31:19, Chris Wilson wrote:
> > > > On Tue, May 16, 2017 at 11:06:06AM +0200, Michal Hocko wrote:
> > > > > From: Michal Hocko <mhocko@suse.com>
> > > > >
> > > > > drm_malloc* has grown their own kmalloc with vmalloc fallback
> > > > > implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> > > > > use those because it a) reduces the code and b) MM has a better idea
> > > > > how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> > > > > with __GFP_NORETRY).
> > > >
> > > > Better? The same idea. The only difference I was reluctant to hand out
> > > > large pages for long lived objects. If that's the wisdom of the core mm,
> > > > so be it.
> > >
> > > vmalloc tends to fragment physical memory more os it is preferable to
> > > try the physically contiguous request first and only fall back to
> > > vmalloc if the first attempt would be too costly or it fails.
> >
> > Not relevant for the changelog in this patch, but it would be nice to
> > have that written in kvmalloc() as to why the scatterring of 4k vmapped
> > pages prevents defragmentation when compared to allocating large pages.
>
> Well, it is not as much about defragmentation because both vmapped and
> kmalloc allocations are very likely to be unmovable (at least
> currently). Theoretically there shouldn't be a problem to make vmapped
> pages movable as the ptes can be modified but this is not implemented...
> The problem is that vmapped pages are more likely to break up more
> larger order blocks. kmalloc will naturally break a single larger block.
>
> > I have vague recollections of seeing the conversation, but a summary as
> > to the reason why kvmalloc prefers large pages will be good for future
> > reference.
>
> Does the following sound better to you?
>
> diff --git a/mm/util.c b/mm/util.c
> index 464df3489903..87499f8119f2 100644
> --- a/mm/util.c
> +++ b/mm/util.c
> @@ -357,7 +357,10 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node)
> WARN_ON_ONCE((flags & GFP_KERNEL) != GFP_KERNEL);
>
> /*
> - * Make sure that larger requests are not too disruptive - no OOM
> + * We want to attempt a large physically contiguous block first because
> + * it is less likely to fragment multiple larger blocks and therefore
> + * contribute to a long term fragmentation less than vmalloc fallback.
> + * However make sure that larger requests are not too disruptive - no OOM
> * killer and no allocation failure warnings as we have a fallback
> */
Hmm, shouldn't we also teach vmalloc to allocate large chunks where
possible - even mixing huge and normal pages? As well as avoiding pinning
the pages and allowing migration.
That comment is helping me to understand why the decison is made to
favour kmalloc over vmalloc, thanks.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] mm: clarify why we want kmalloc before falling backto vmallock
From: Michal Hocko @ 2017-05-17 8:09 UTC (permalink / raw)
To: Andrew Morton; +Cc: Chris Wilson, Vlastimil Babka, linux-mm, LKML, Michal Hocko
From: Michal Hocko <mhocko@suse.com>
While converting drm_[cm]alloc* helpers to kvmalloc* variants Chris
Wilson has wondered why we want to try kmalloc before vmalloc fallback
even for larger allocations requests. Let's clarify that one larger
physically contiguous block is less likely to fragment memory than many
scattered pages which can prevent more large blocks from being created.
Suggested-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
mm/util.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/mm/util.c b/mm/util.c
index 464df3489903..87499f8119f2 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -357,7 +357,10 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node)
WARN_ON_ONCE((flags & GFP_KERNEL) != GFP_KERNEL);
/*
- * Make sure that larger requests are not too disruptive - no OOM
+ * We want to attempt a large physically contiguous block first because
+ * it is less likely to fragment multiple larger blocks and therefore
+ * contribute to a long term fragmentation less than vmalloc fallback.
+ * However make sure that larger requests are not too disruptive - no OOM
* killer and no allocation failure warnings as we have a fallback
*/
if (size > PAGE_SIZE) {
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH] drm: use kvmalloc_array for drm_malloc*
From: Michal Hocko @ 2017-05-17 8:09 UTC (permalink / raw)
To: Chris Wilson
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170517075944.GK26693@nuc-i3427.alporthouse.com>
On Wed 17-05-17 08:59:44, Chris Wilson wrote:
> On Wed, May 17, 2017 at 09:44:53AM +0200, Michal Hocko wrote:
> > On Tue 16-05-17 12:09:08, Chris Wilson wrote:
> > > On Tue, May 16, 2017 at 12:53:52PM +0200, Michal Hocko wrote:
> > > > On Tue 16-05-17 10:31:19, Chris Wilson wrote:
> > > > > On Tue, May 16, 2017 at 11:06:06AM +0200, Michal Hocko wrote:
> > > > > > From: Michal Hocko <mhocko@suse.com>
> > > > > >
> > > > > > drm_malloc* has grown their own kmalloc with vmalloc fallback
> > > > > > implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> > > > > > use those because it a) reduces the code and b) MM has a better idea
> > > > > > how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> > > > > > with __GFP_NORETRY).
> > > > >
> > > > > Better? The same idea. The only difference I was reluctant to hand out
> > > > > large pages for long lived objects. If that's the wisdom of the core mm,
> > > > > so be it.
> > > >
> > > > vmalloc tends to fragment physical memory more os it is preferable to
> > > > try the physically contiguous request first and only fall back to
> > > > vmalloc if the first attempt would be too costly or it fails.
> > >
> > > Not relevant for the changelog in this patch, but it would be nice to
> > > have that written in kvmalloc() as to why the scatterring of 4k vmapped
> > > pages prevents defragmentation when compared to allocating large pages.
> >
> > Well, it is not as much about defragmentation because both vmapped and
> > kmalloc allocations are very likely to be unmovable (at least
> > currently). Theoretically there shouldn't be a problem to make vmapped
> > pages movable as the ptes can be modified but this is not implemented...
> > The problem is that vmapped pages are more likely to break up more
> > larger order blocks. kmalloc will naturally break a single larger block.
> >
> > > I have vague recollections of seeing the conversation, but a summary as
> > > to the reason why kvmalloc prefers large pages will be good for future
> > > reference.
> >
> > Does the following sound better to you?
> >
> > diff --git a/mm/util.c b/mm/util.c
> > index 464df3489903..87499f8119f2 100644
> > --- a/mm/util.c
> > +++ b/mm/util.c
> > @@ -357,7 +357,10 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node)
> > WARN_ON_ONCE((flags & GFP_KERNEL) != GFP_KERNEL);
> >
> > /*
> > - * Make sure that larger requests are not too disruptive - no OOM
> > + * We want to attempt a large physically contiguous block first because
> > + * it is less likely to fragment multiple larger blocks and therefore
> > + * contribute to a long term fragmentation less than vmalloc fallback.
> > + * However make sure that larger requests are not too disruptive - no OOM
> > * killer and no allocation failure warnings as we have a fallback
> > */
>
> Hmm, shouldn't we also teach vmalloc to allocate large chunks where
> possible - even mixing huge and normal pages? As well as avoiding pinning
> the pages and allowing migration.
Yes that would be possible and my vague recollection is that somebody
was working on something like that. Do not have any references, though.
> That comment is helping me to understand why the decison is made to
> favour kmalloc over vmalloc, thanks.
OK, I've sent this clarification to Andrew.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v2 3/6] mm, page_alloc: pass preferred nid instead of zonelist to allocator
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
The main allocator function __alloc_pages_nodemask() takes a zonelist pointer
as one of its parameters. All of its callers directly or indirectly obtain the
zonelist via node_zonelist() using a preferred node id and gfp_mask. We can
make the code a bit simpler by doing the zonelist lookup in
__alloc_pages_nodemask(), passing it a preferred node id instead (gfp_mask is
already another parameter).
There are some code size benefits thanks to removal of inlined node_zonelist():
bloat-o-meter add/remove: 2/2 grow/shrink: 4/36 up/down: 399/-1351 (-952)
This will also make things simpler if we proceed with converting cpusets to
zonelists.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/gfp.h | 11 +++++------
include/linux/mempolicy.h | 6 +++---
mm/hugetlb.c | 15 +++++++++------
mm/memory_hotplug.c | 6 ++----
mm/mempolicy.c | 41 +++++++++++++++++++----------------------
mm/page_alloc.c | 10 +++++-----
6 files changed, 43 insertions(+), 46 deletions(-)
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index 2b1a44f5bdb6..666af3c39d00 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -432,14 +432,13 @@ static inline void arch_alloc_page(struct page *page, int order) { }
#endif
struct page *
-__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
- struct zonelist *zonelist, nodemask_t *nodemask);
+__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
+ nodemask_t *nodemask);
static inline struct page *
-__alloc_pages(gfp_t gfp_mask, unsigned int order,
- struct zonelist *zonelist)
+__alloc_pages(gfp_t gfp_mask, unsigned int order, int preferred_nid)
{
- return __alloc_pages_nodemask(gfp_mask, order, zonelist, NULL);
+ return __alloc_pages_nodemask(gfp_mask, order, preferred_nid, NULL);
}
/*
@@ -452,7 +451,7 @@ __alloc_pages_node(int nid, gfp_t gfp_mask, unsigned int order)
VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
VM_WARN_ON(!node_online(nid));
- return __alloc_pages(gfp_mask, order, node_zonelist(nid, gfp_mask));
+ return __alloc_pages(gfp_mask, order, nid);
}
/*
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index 5f4d8281832b..ecb6cbeede5a 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -146,7 +146,7 @@ extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
enum mpol_rebind_step step);
extern void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new);
-extern struct zonelist *huge_zonelist(struct vm_area_struct *vma,
+extern int huge_node(struct vm_area_struct *vma,
unsigned long addr, gfp_t gfp_flags,
struct mempolicy **mpol, nodemask_t **nodemask);
extern bool init_nodemask_of_mempolicy(nodemask_t *mask);
@@ -269,13 +269,13 @@ static inline void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new)
{
}
-static inline struct zonelist *huge_zonelist(struct vm_area_struct *vma,
+static inline int huge_node(struct vm_area_struct *vma,
unsigned long addr, gfp_t gfp_flags,
struct mempolicy **mpol, nodemask_t **nodemask)
{
*mpol = NULL;
*nodemask = NULL;
- return node_zonelist(0, gfp_flags);
+ return 0;
}
static inline bool init_nodemask_of_mempolicy(nodemask_t *m)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index e5828875f7bb..9f1f399bb913 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -904,6 +904,8 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
struct page *page = NULL;
struct mempolicy *mpol;
nodemask_t *nodemask;
+ gfp_t gfp_mask;
+ int nid;
struct zonelist *zonelist;
struct zone *zone;
struct zoneref *z;
@@ -924,12 +926,13 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
retry_cpuset:
cpuset_mems_cookie = read_mems_allowed_begin();
- zonelist = huge_zonelist(vma, address,
- htlb_alloc_mask(h), &mpol, &nodemask);
+ gfp_mask = htlb_alloc_mask(h);
+ nid = huge_node(vma, address, gfp_mask, &mpol, &nodemask);
+ zonelist = node_zonelist(nid, gfp_mask);
for_each_zone_zonelist_nodemask(zone, z, zonelist,
MAX_NR_ZONES - 1, nodemask) {
- if (cpuset_zone_allowed(zone, htlb_alloc_mask(h))) {
+ if (cpuset_zone_allowed(zone, gfp_mask)) {
page = dequeue_huge_page_node(h, zone_to_nid(zone));
if (page) {
if (avoid_reserve)
@@ -1545,13 +1548,13 @@ static struct page *__hugetlb_alloc_buddy_huge_page(struct hstate *h,
do {
struct page *page;
struct mempolicy *mpol;
- struct zonelist *zl;
+ int nid;
nodemask_t *nodemask;
cpuset_mems_cookie = read_mems_allowed_begin();
- zl = huge_zonelist(vma, addr, gfp, &mpol, &nodemask);
+ nid = huge_node(vma, addr, gfp, &mpol, &nodemask);
mpol_cond_put(mpol);
- page = __alloc_pages_nodemask(gfp, order, zl, nodemask);
+ page = __alloc_pages_nodemask(gfp, order, nid, nodemask);
if (page)
return page;
} while (read_mems_allowed_retry(cpuset_mems_cookie));
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 717c5e301aa8..ba9e09817f37 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1596,11 +1596,9 @@ static struct page *new_node_page(struct page *page, unsigned long private,
gfp_mask |= __GFP_HIGHMEM;
if (!nodes_empty(nmask))
- new_page = __alloc_pages_nodemask(gfp_mask, 0,
- node_zonelist(nid, gfp_mask), &nmask);
+ new_page = __alloc_pages_nodemask(gfp_mask, 0, nid, &nmask);
if (!new_page)
- new_page = __alloc_pages(gfp_mask, 0,
- node_zonelist(nid, gfp_mask));
+ new_page = __alloc_pages(gfp_mask, 0, nid);
return new_page;
}
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index d77177c7283b..c60807625fd5 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1669,9 +1669,9 @@ static nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)
return NULL;
}
-/* Return a zonelist indicated by gfp for node representing a mempolicy */
-static struct zonelist *policy_zonelist(gfp_t gfp, struct mempolicy *policy,
- int nd)
+/* Return the node id preferred by the given mempolicy, or the given id */
+static int policy_node(gfp_t gfp, struct mempolicy *policy,
+ int nd)
{
if (policy->mode == MPOL_PREFERRED && !(policy->flags & MPOL_F_LOCAL))
nd = policy->v.preferred_node;
@@ -1684,7 +1684,7 @@ static struct zonelist *policy_zonelist(gfp_t gfp, struct mempolicy *policy,
WARN_ON_ONCE(policy->mode == MPOL_BIND && (gfp & __GFP_THISNODE));
}
- return node_zonelist(nd, gfp);
+ return nd;
}
/* Do dynamic interleaving for a process */
@@ -1791,38 +1791,37 @@ static inline unsigned interleave_nid(struct mempolicy *pol,
#ifdef CONFIG_HUGETLBFS
/*
- * huge_zonelist(@vma, @addr, @gfp_flags, @mpol)
+ * huge_node(@vma, @addr, @gfp_flags, @mpol)
* @vma: virtual memory area whose policy is sought
* @addr: address in @vma for shared policy lookup and interleave policy
* @gfp_flags: for requested zone
* @mpol: pointer to mempolicy pointer for reference counted mempolicy
* @nodemask: pointer to nodemask pointer for MPOL_BIND nodemask
*
- * Returns a zonelist suitable for a huge page allocation and a pointer
+ * Returns a nid suitable for a huge page allocation and a pointer
* to the struct mempolicy for conditional unref after allocation.
* If the effective policy is 'BIND, returns a pointer to the mempolicy's
* @nodemask for filtering the zonelist.
*
* Must be protected by read_mems_allowed_begin()
*/
-struct zonelist *huge_zonelist(struct vm_area_struct *vma, unsigned long addr,
- gfp_t gfp_flags, struct mempolicy **mpol,
- nodemask_t **nodemask)
+int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags,
+ struct mempolicy **mpol, nodemask_t **nodemask)
{
- struct zonelist *zl;
+ int nid;
*mpol = get_vma_policy(vma, addr);
*nodemask = NULL; /* assume !MPOL_BIND */
if (unlikely((*mpol)->mode == MPOL_INTERLEAVE)) {
- zl = node_zonelist(interleave_nid(*mpol, vma, addr,
- huge_page_shift(hstate_vma(vma))), gfp_flags);
+ nid = interleave_nid(*mpol, vma, addr,
+ huge_page_shift(hstate_vma(vma)));
} else {
- zl = policy_zonelist(gfp_flags, *mpol, numa_node_id());
+ nid = policy_node(gfp_flags, *mpol, numa_node_id());
if ((*mpol)->mode == MPOL_BIND)
*nodemask = &(*mpol)->v.nodes;
}
- return zl;
+ return nid;
}
/*
@@ -1924,12 +1923,10 @@ bool mempolicy_nodemask_intersects(struct task_struct *tsk,
static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,
unsigned nid)
{
- struct zonelist *zl;
struct page *page;
- zl = node_zonelist(nid, gfp);
- page = __alloc_pages(gfp, order, zl);
- if (page && page_zone(page) == zonelist_zone(&zl->_zonerefs[0]))
+ page = __alloc_pages(gfp, order, nid);
+ if (page && page_to_nid(page) == nid)
inc_zone_page_state(page, NUMA_INTERLEAVE_HIT);
return page;
}
@@ -1963,8 +1960,8 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
{
struct mempolicy *pol;
struct page *page;
+ int preferred_nid;
unsigned int cpuset_mems_cookie;
- struct zonelist *zl;
nodemask_t *nmask;
retry_cpuset:
@@ -2007,8 +2004,8 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
}
nmask = policy_nodemask(gfp, pol);
- zl = policy_zonelist(gfp, pol, node);
- page = __alloc_pages_nodemask(gfp, order, zl, nmask);
+ preferred_nid = policy_node(gfp, pol, node);
+ page = __alloc_pages_nodemask(gfp, order, preferred_nid, nmask);
mpol_cond_put(pol);
out:
if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
@@ -2055,7 +2052,7 @@ struct page *alloc_pages_current(gfp_t gfp, unsigned order)
page = alloc_page_interleave(gfp, order, interleave_nodes(pol));
else
page = __alloc_pages_nodemask(gfp, order,
- policy_zonelist(gfp, pol, numa_node_id()),
+ policy_node(gfp, pol, numa_node_id()),
policy_nodemask(gfp, pol));
if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 43aa767c3188..0aceca1076dc 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3962,12 +3962,12 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
}
static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
- struct zonelist *zonelist, nodemask_t *nodemask,
+ int preferred_nid, nodemask_t *nodemask,
struct alloc_context *ac, gfp_t *alloc_mask,
unsigned int *alloc_flags)
{
ac->high_zoneidx = gfp_zone(gfp_mask);
- ac->zonelist = zonelist;
+ ac->zonelist = node_zonelist(preferred_nid, gfp_mask);
ac->nodemask = nodemask;
ac->migratetype = gfpflags_to_migratetype(gfp_mask);
@@ -4012,8 +4012,8 @@ static inline void finalise_ac(gfp_t gfp_mask,
* This is the 'heart' of the zoned buddy allocator.
*/
struct page *
-__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
- struct zonelist *zonelist, nodemask_t *nodemask)
+__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
+ nodemask_t *nodemask)
{
struct page *page;
unsigned int alloc_flags = ALLOC_WMARK_LOW;
@@ -4021,7 +4021,7 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
struct alloc_context ac = { };
gfp_mask &= gfp_allowed_mask;
- if (!prepare_alloc_pages(gfp_mask, order, zonelist, nodemask, &ac, &alloc_mask, &alloc_flags))
+ if (!prepare_alloc_pages(gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags))
return NULL;
finalise_ac(gfp_mask, order, &ac);
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 4/6] mm, mempolicy: simplify rebinding mempolicies when updating cpusets
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Commit c0ff7453bb5c ("cpuset,mm: fix no node to alloc memory when changing
cpuset's mems") has introduced a two-step protocol when rebinding task's
mempolicy due to cpuset update, in order to avoid a parallel allocation seeing
an empty effective nodemask and failing. Later, commit cc9a6c877661 ("cpuset:
mm: reduce large amounts of memory barrier related damage v3") introduced
a seqlock protection and removed the synchronization point between the two
update steps. At that point (or perhaps later), the two-step rebinding became
unnecessary. Currently it only makes sure that the update first adds new nodes
in step 1 and then removes nodes in step 2. Without memory barriers the effects
are questionable, and even then this cannot prevent a parallel zonelist
iteration checking the nodemask at each step to observe all nodes as unusable
for allocation. We now fully rely on the seqlock to prevent premature OOMs and
allocation failures.
We can thus remove the two-step update parts and simplify the code.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/mempolicy.h | 6 +--
include/uapi/linux/mempolicy.h | 8 ----
kernel/cgroup/cpuset.c | 4 +-
mm/mempolicy.c | 102 ++++++++---------------------------------
4 files changed, 21 insertions(+), 99 deletions(-)
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index ecb6cbeede5a..3a58b4be1b0c 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -142,8 +142,7 @@ bool vma_policy_mof(struct vm_area_struct *vma);
extern void numa_default_policy(void);
extern void numa_policy_init(void);
-extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
- enum mpol_rebind_step step);
+extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new);
extern void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new);
extern int huge_node(struct vm_area_struct *vma,
@@ -260,8 +259,7 @@ static inline void numa_default_policy(void)
}
static inline void mpol_rebind_task(struct task_struct *tsk,
- const nodemask_t *new,
- enum mpol_rebind_step step)
+ const nodemask_t *new)
{
}
diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index 9cd8b21dddbe..2a4d89508fec 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -24,13 +24,6 @@ enum {
MPOL_MAX, /* always last member of enum */
};
-enum mpol_rebind_step {
- MPOL_REBIND_ONCE, /* do rebind work at once(not by two step) */
- MPOL_REBIND_STEP1, /* first step(set all the newly nodes) */
- MPOL_REBIND_STEP2, /* second step(clean all the disallowed nodes)*/
- MPOL_REBIND_NSTEP,
-};
-
/* Flags for set_mempolicy */
#define MPOL_F_STATIC_NODES (1 << 15)
#define MPOL_F_RELATIVE_NODES (1 << 14)
@@ -65,7 +58,6 @@ enum mpol_rebind_step {
*/
#define MPOL_F_SHARED (1 << 0) /* identify shared policies */
#define MPOL_F_LOCAL (1 << 1) /* preferred local allocation */
-#define MPOL_F_REBINDING (1 << 2) /* identify policies in rebinding */
#define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */
#define MPOL_F_MORON (1 << 4) /* Migrate On protnone Reference On Node */
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 0f41292be0fb..dfd5b420452d 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1063,9 +1063,7 @@ static void cpuset_change_task_nodemask(struct task_struct *tsk,
}
nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
- mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP1);
-
- mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP2);
+ mpol_rebind_task(tsk, newmems);
tsk->mems_allowed = *newmems;
if (need_loop) {
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index c60807625fd5..047181452040 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -146,22 +146,7 @@ struct mempolicy *get_task_policy(struct task_struct *p)
static const struct mempolicy_operations {
int (*create)(struct mempolicy *pol, const nodemask_t *nodes);
- /*
- * If read-side task has no lock to protect task->mempolicy, write-side
- * task will rebind the task->mempolicy by two step. The first step is
- * setting all the newly nodes, and the second step is cleaning all the
- * disallowed nodes. In this way, we can avoid finding no node to alloc
- * page.
- * If we have a lock to protect task->mempolicy in read-side, we do
- * rebind directly.
- *
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
- */
- void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step);
+ void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes);
} mpol_ops[MPOL_MAX];
static inline int mpol_store_user_nodemask(const struct mempolicy *pol)
@@ -304,19 +289,11 @@ void __mpol_put(struct mempolicy *p)
kmem_cache_free(policy_cache, p);
}
-static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step)
+static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
{
}
-/*
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
- */
-static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step)
+static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
{
nodemask_t tmp;
@@ -325,35 +302,19 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
else if (pol->flags & MPOL_F_RELATIVE_NODES)
mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes);
else {
- /*
- * if step == 1, we use ->w.cpuset_mems_allowed to cache the
- * result
- */
- if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP1) {
- nodes_remap(tmp, pol->v.nodes,
- pol->w.cpuset_mems_allowed, *nodes);
- pol->w.cpuset_mems_allowed = step ? tmp : *nodes;
- } else if (step == MPOL_REBIND_STEP2) {
- tmp = pol->w.cpuset_mems_allowed;
- pol->w.cpuset_mems_allowed = *nodes;
- } else
- BUG();
+ nodes_remap(tmp, pol->v.nodes,pol->w.cpuset_mems_allowed,
+ *nodes);
+ pol->w.cpuset_mems_allowed = tmp;
}
if (nodes_empty(tmp))
tmp = *nodes;
- if (step == MPOL_REBIND_STEP1)
- nodes_or(pol->v.nodes, pol->v.nodes, tmp);
- else if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP2)
- pol->v.nodes = tmp;
- else
- BUG();
+ pol->v.nodes = tmp;
}
static void mpol_rebind_preferred(struct mempolicy *pol,
- const nodemask_t *nodes,
- enum mpol_rebind_step step)
+ const nodemask_t *nodes)
{
nodemask_t tmp;
@@ -379,42 +340,19 @@ static void mpol_rebind_preferred(struct mempolicy *pol,
/*
* mpol_rebind_policy - Migrate a policy to a different set of nodes
*
- * If read-side task has no lock to protect task->mempolicy, write-side
- * task will rebind the task->mempolicy by two step. The first step is
- * setting all the newly nodes, and the second step is cleaning all the
- * disallowed nodes. In this way, we can avoid finding no node to alloc
- * page.
- * If we have a lock to protect task->mempolicy in read-side, we do
- * rebind directly.
- *
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
+ * Per-vma policies are protected by mmap_sem. Allocations using per-task
+ * policies are protected by task->mems_allowed_seq to prevent a premature
+ * OOM/allocation failure due to parallel nodemask modification.
*/
-static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
- enum mpol_rebind_step step)
+static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask)
{
if (!pol)
return;
- if (!mpol_store_user_nodemask(pol) && step == MPOL_REBIND_ONCE &&
+ if (!mpol_store_user_nodemask(pol) &&
nodes_equal(pol->w.cpuset_mems_allowed, *newmask))
return;
- if (step == MPOL_REBIND_STEP1 && (pol->flags & MPOL_F_REBINDING))
- return;
-
- if (step == MPOL_REBIND_STEP2 && !(pol->flags & MPOL_F_REBINDING))
- BUG();
-
- if (step == MPOL_REBIND_STEP1)
- pol->flags |= MPOL_F_REBINDING;
- else if (step == MPOL_REBIND_STEP2)
- pol->flags &= ~MPOL_F_REBINDING;
- else if (step >= MPOL_REBIND_NSTEP)
- BUG();
-
- mpol_ops[pol->mode].rebind(pol, newmask, step);
+ mpol_ops[pol->mode].rebind(pol, newmask);
}
/*
@@ -424,10 +362,9 @@ static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
* Called with task's alloc_lock held.
*/
-void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
- enum mpol_rebind_step step)
+void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new)
{
- mpol_rebind_policy(tsk->mempolicy, new, step);
+ mpol_rebind_policy(tsk->mempolicy, new);
}
/*
@@ -442,7 +379,7 @@ void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new)
down_write(&mm->mmap_sem);
for (vma = mm->mmap; vma; vma = vma->vm_next)
- mpol_rebind_policy(vma->vm_policy, new, MPOL_REBIND_ONCE);
+ mpol_rebind_policy(vma->vm_policy, new);
up_write(&mm->mmap_sem);
}
@@ -2101,10 +2038,7 @@ struct mempolicy *__mpol_dup(struct mempolicy *old)
if (current_cpuset_is_being_rebound()) {
nodemask_t mems = cpuset_mems_allowed(current);
- if (new->flags & MPOL_F_REBINDING)
- mpol_rebind_policy(new, &mems, MPOL_REBIND_STEP2);
- else
- mpol_rebind_policy(new, &mems, MPOL_REBIND_ONCE);
+ mpol_rebind_policy(new, &mems);
}
atomic_set(&new->refcnt, 1);
return new;
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 2/6] mm, mempolicy: stop adjusting current->il_next in mpol_rebind_nodemask()
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
The task->il_next variable stores the next allocation node id for task's
MPOL_INTERLEAVE policy. mpol_rebind_nodemask() updates interleave and
bind mempolicies due to changing cpuset mems. Currently it also tries to
make sure that current->il_next is valid within the updated nodemask. This is
bogus, because 1) we are updating potentially any task's mempolicy, not just
current, and 2) we might be updating a per-vma mempolicy, not task one.
The interleave_nodes() function that uses il_next can cope fine with the value
not being within the currently allowed nodes, so this hasn't manifested as an
actual issue.
We can remove the need for updating il_next completely by changing it to
il_prev and store the node id of the previous interleave allocation instead of
the next id. Then interleave_nodes() can calculate the next id using the
current nodemask and also store it as il_prev, except when querying the next
node via do_get_mempolicy().
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/sched.h | 2 +-
mm/mempolicy.c | 22 +++++++---------------
2 files changed, 8 insertions(+), 16 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 6a97386c785e..b72bbfec01f9 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -884,7 +884,7 @@ struct task_struct {
#ifdef CONFIG_NUMA
/* Protected by alloc_lock: */
struct mempolicy *mempolicy;
- short il_next;
+ short il_prev;
short pref_node_fork;
#endif
#ifdef CONFIG_NUMA_BALANCING
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 37d0b334bfe9..d77177c7283b 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -349,12 +349,6 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
pol->v.nodes = tmp;
else
BUG();
-
- if (!node_isset(current->il_next, tmp)) {
- current->il_next = next_node_in(current->il_next, tmp);
- if (current->il_next >= MAX_NUMNODES)
- current->il_next = numa_node_id();
- }
}
static void mpol_rebind_preferred(struct mempolicy *pol,
@@ -812,9 +806,8 @@ static long do_set_mempolicy(unsigned short mode, unsigned short flags,
}
old = current->mempolicy;
current->mempolicy = new;
- if (new && new->mode == MPOL_INTERLEAVE &&
- nodes_weight(new->v.nodes))
- current->il_next = first_node(new->v.nodes);
+ if (new && new->mode == MPOL_INTERLEAVE)
+ current->il_prev = MAX_NUMNODES-1;
task_unlock(current);
mpol_put(old);
ret = 0;
@@ -916,7 +909,7 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask,
*policy = err;
} else if (pol == current->mempolicy &&
pol->mode == MPOL_INTERLEAVE) {
- *policy = current->il_next;
+ *policy = next_node_in(current->il_prev, pol->v.nodes);
} else {
err = -EINVAL;
goto out;
@@ -1697,14 +1690,13 @@ static struct zonelist *policy_zonelist(gfp_t gfp, struct mempolicy *policy,
/* Do dynamic interleaving for a process */
static unsigned interleave_nodes(struct mempolicy *policy)
{
- unsigned nid, next;
+ unsigned next;
struct task_struct *me = current;
- nid = me->il_next;
- next = next_node_in(nid, policy->v.nodes);
+ next = next_node_in(me->il_prev, policy->v.nodes);
if (next < MAX_NUMNODES)
- me->il_next = next;
- return nid;
+ me->il_prev = next;
+ return next;
}
/*
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 6/6] mm, mempolicy: don't check cpuset seqlock where it doesn't matter
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Two wrappers of __alloc_pages_nodemask() are checking task->mems_allowed_seq
themselves to retry allocation that has raced with a cpuset update. This has
been shown to be ineffective in preventing premature OOM's which can happen in
__alloc_pages_slowpath() long before it returns back to the wrappers to detect
the race at that level. Previous patches have made __alloc_pages_slowpath()
more robust, so we can now simply remove the seqlock checking in the wrappers
to prevent further wrong impression that it can actually help.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/mempolicy.c | 16 ----------------
1 file changed, 16 deletions(-)
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 047181452040..7d8e56214ac0 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1898,12 +1898,9 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
struct mempolicy *pol;
struct page *page;
int preferred_nid;
- unsigned int cpuset_mems_cookie;
nodemask_t *nmask;
-retry_cpuset:
pol = get_vma_policy(vma, addr);
- cpuset_mems_cookie = read_mems_allowed_begin();
if (pol->mode == MPOL_INTERLEAVE) {
unsigned nid;
@@ -1945,8 +1942,6 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
page = __alloc_pages_nodemask(gfp, order, preferred_nid, nmask);
mpol_cond_put(pol);
out:
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
- goto retry_cpuset;
return page;
}
@@ -1964,23 +1959,15 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
* Allocate a page from the kernel page pool. When not in
* interrupt context and apply the current process NUMA policy.
* Returns NULL when no page can be allocated.
- *
- * Don't call cpuset_update_task_memory_state() unless
- * 1) it's ok to take cpuset_sem (can WAIT), and
- * 2) allocating for current task (not interrupt).
*/
struct page *alloc_pages_current(gfp_t gfp, unsigned order)
{
struct mempolicy *pol = &default_policy;
struct page *page;
- unsigned int cpuset_mems_cookie;
if (!in_interrupt() && !(gfp & __GFP_THISNODE))
pol = get_task_policy(current);
-retry_cpuset:
- cpuset_mems_cookie = read_mems_allowed_begin();
-
/*
* No reference counting needed for current->mempolicy
* nor system default_policy
@@ -1992,9 +1979,6 @@ struct page *alloc_pages_current(gfp_t gfp, unsigned order)
policy_node(gfp, pol, numa_node_id()),
policy_nodemask(gfp, pol));
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
- goto retry_cpuset;
-
return page;
}
EXPORT_SYMBOL(alloc_pages_current);
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 1/6] mm, page_alloc: fix more premature OOM due to race with cpuset update
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Commit e47483bca2cc ("mm, page_alloc: fix premature OOM when racing with cpuset
mems update") has fixed known recent regressions found by LTP's cpuset01
testcase. I have however found that by modifying the testcase to use per-vma
mempolicies via bind(2) instead of per-task mempolicies via set_mempolicy(2),
the premature OOM still happens and the issue is much older.
The root of the problem is that the cpuset's mems_allowed and mempolicy's
nodemask can temporarily have no intersection, thus get_page_from_freelist()
cannot find any usable zone. The current semantic for empty intersection is to
ignore mempolicy's nodemask and honour cpuset restrictions. This is checked in
node_zonelist(), but the racy update can happen after we already passed the
check. Such races should be protected by the seqlock task->mems_allowed_seq,
but it doesn't work here, because 1) mpol_rebind_mm() does not happen under
seqlock for write, and doing so would lead to deadlock, as it takes mmap_sem
for write, while the allocation can have mmap_sem for read when it's taking the
seqlock for read. And 2) the seqlock cookie of callers of node_zonelist()
(alloc_pages_vma() and alloc_pages_current()) is different than the one of
__alloc_pages_slowpath(), so there's still a potential race window.
This patch fixes the issue by having __alloc_pages_slowpath() check for empty
intersection of cpuset and ac->nodemask before OOM or allocation failure. If
it's indeed empty, the nodemask is ignored and allocation retried, which mimics
node_zonelist(). This works fine, because almost all callers of
__alloc_pages_nodemask are obtaining the nodemask via node_zonelist(). The only
exception is new_node_page() from hotplug, where the potential violation of
nodemask isn't an issue, as there's already a fallback allocation attempt
without any nodemask. If there's a future caller that needs to have its specific
nodemask honoured over task's cpuset restrictions, we'll have to e.g. add a gfp
flag for that.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/page_alloc.c | 51 ++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 38 insertions(+), 13 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index beb2827fd5de..43aa767c3188 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3661,6 +3661,39 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
return false;
}
+static inline bool
+check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac)
+{
+ /*
+ * It's possible that cpuset's mems_allowed and the nodemask from
+ * mempolicy don't intersect. This should be normally dealt with by
+ * policy_nodemask(), but it's possible to race with cpuset update in
+ * such a way the check therein was true, and then it became false
+ * before we got our cpuset_mems_cookie here.
+ * This assumes that for all allocations, ac->nodemask can come only
+ * from MPOL_BIND mempolicy (whose documented semantics is to be ignored
+ * when it does not intersect with the cpuset restrictions) or the
+ * caller can deal with a violated nodemask.
+ */
+ if (cpusets_enabled() && ac->nodemask &&
+ !cpuset_nodemask_valid_mems_allowed(ac->nodemask)) {
+ ac->nodemask = NULL;
+ return true;
+ }
+
+ /*
+ * When updating a task's mems_allowed or mempolicy nodemask, it is
+ * possible to race with parallel threads in such a way that our
+ * allocation can fail while the mask is being updated. If we are about
+ * to fail, check if the cpuset changed during allocation and if so,
+ * retry.
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ return true;
+
+ return false;
+}
+
static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
struct alloc_context *ac)
@@ -3856,11 +3889,9 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
&compaction_retries))
goto retry;
- /*
- * It's possible we raced with cpuset update so the OOM would be
- * premature (see below the nopage: label for full explanation).
- */
- if (read_mems_allowed_retry(cpuset_mems_cookie))
+
+ /* Deal with possible cpuset update races before we start OOM killing */
+ if (check_retry_cpuset(cpuset_mems_cookie, ac))
goto retry_cpuset;
/* Reclaim has failed us, start killing things */
@@ -3879,14 +3910,8 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
}
nopage:
- /*
- * When updating a task's mems_allowed or mempolicy nodemask, it is
- * possible to race with parallel threads in such a way that our
- * allocation can fail while the mask is being updated. If we are about
- * to fail, check if the cpuset changed during allocation and if so,
- * retry.
- */
- if (read_mems_allowed_retry(cpuset_mems_cookie))
+ /* Deal with possible cpuset update races before we fail */
+ if (check_retry_cpuset(cpuset_mems_cookie, ac))
goto retry_cpuset;
/*
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 0/6] cpuset/mempolicies related fixes and cleanups
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
Changes since RFC v1 [3]:
- Reworked patch 2 after discussion with Christoph Lameter.
- Fix bug in patch 5 spotted by Hillf Danton.
- Rebased to mmotm-2017-05-12-15-53
I would like to stress that this patchset aims to fix issues and cleanup the
code *within the existing documented semantics*, i.e. patch 1 ignores mempolicy
restrictions if the set of allowed nodes has no intersection with set of nodes
allowed by cpuset. I believe discussing potential changes of the semantics can
be better done once we have a baseline with no known bugs of the current
semantics.
===
I've recently summarized the cpuset/mempolicy issues in a LSF/MM proposal [1]
and the discussion itself [2]. I've been trying to rewrite the handling as
proposed, with the idea that changing semantics to make all mempolicies static
wrt cpuset updates (and discarding the relative and default modes) can be tried
on top, as there's a high risk of being rejected/reverted because somebody
might still care about the removed modes.
However I haven't yet figured out how to properly:
1) make mempolicies swappable instead of rebinding in place. I thought mbind()
already works that way and uses refcounting to avoid use-after-free of the old
policy by a parallel allocation, but turns out true refcounting is only done
for shared (shmem) mempolicies, and the actual protection for mbind() comes
from mmap_sem. Extending the refcounting means more overhead in allocator hot
path. Also swapping whole mempolicies means that we have to allocate the new
ones, which can fail, and reverting of the partially done work also means
allocating (note that mbind() doesn't care and will just leave part of the
range updated and part not updated when returning -ENOMEM...).
2) make cpuset's task->mems_allowed also swappable (after converting it from
nodemask to zonelist, which is the easy part) for mostly the same reasons.
The good news is that while trying to do the above, I've at least figured out
how to hopefully close the remaining premature OOM's, and do a buch of cleanups
on top, removing quite some of the code that was also supposed to prevent the
cpuset update races, but doesn't work anymore nowadays. This should fix the
most pressing concerns with this topic and give us a better baseline before
either proceeding with the original proposal, or pushing a change of semantics
that removes the problem 1) above. I'd be then fine with trying to change the
semantic first and rewrite later.
Patchset is based on next-20170411 and has been tested with the LTP cpuset01
stress test.
[1] https://lkml.kernel.org/r/4c44a589-5fd8-08d0-892c-e893bb525b71@suse.cz
[2] https://lwn.net/Articles/717797/
[3] https://marc.info/?l=linux-mm&m=149191957922828&w=2
Vlastimil Babka (6):
mm, page_alloc: fix more premature OOM due to race with cpuset update
mm, mempolicy: stop adjusting current->il_next in
mpol_rebind_nodemask()
mm, page_alloc: pass preferred nid instead of zonelist to allocator
mm, mempolicy: simplify rebinding mempolicies when updating cpusets
mm, cpuset: always use seqlock when changing task's nodemask
mm, mempolicy: don't check cpuset seqlock where it doesn't matter
include/linux/gfp.h | 11 ++-
include/linux/mempolicy.h | 12 ++-
include/linux/sched.h | 2 +-
include/uapi/linux/mempolicy.h | 8 --
kernel/cgroup/cpuset.c | 33 ++------
mm/hugetlb.c | 15 ++--
mm/memory_hotplug.c | 6 +-
mm/mempolicy.c | 181 ++++++++++-------------------------------
mm/page_alloc.c | 61 ++++++++++----
9 files changed, 118 insertions(+), 211 deletions(-)
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v2 5/6] mm, cpuset: always use seqlock when changing task's nodemask
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
When updating task's mems_allowed and rebinding its mempolicy due to cpuset's
mems being changed, we currently only take the seqlock for writing when either
the task has a mempolicy, or the new mems has no intersection with the old
mems. This should be enough to prevent a parallel allocation seeing no
available nodes, but the optimization is IMHO unnecessary (cpuset updates
should not be frequent), and we still potentially risk issues if the
intersection of new and old nodes has limited amount of free/reclaimable
memory. Let's just use the seqlock for all tasks.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
kernel/cgroup/cpuset.c | 29 ++++++++---------------------
1 file changed, 8 insertions(+), 21 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index dfd5b420452d..26a1c360a481 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1038,38 +1038,25 @@ static void cpuset_post_attach(void)
* @tsk: the task to change
* @newmems: new nodes that the task will be set
*
- * In order to avoid seeing no nodes if the old and new nodes are disjoint,
- * we structure updates as setting all new allowed nodes, then clearing newly
- * disallowed ones.
+ * We use the mems_allowed_seq seqlock to safely update both tsk->mems_allowed
+ * and rebind an eventual tasks' mempolicy. If the task is allocating in
+ * parallel, it might temporarily see an empty intersection, which results in
+ * a seqlock check and retry before OOM or allocation failure.
*/
static void cpuset_change_task_nodemask(struct task_struct *tsk,
nodemask_t *newmems)
{
- bool need_loop;
-
task_lock(tsk);
- /*
- * Determine if a loop is necessary if another thread is doing
- * read_mems_allowed_begin(). If at least one node remains unchanged and
- * tsk does not have a mempolicy, then an empty nodemask will not be
- * possible when mems_allowed is larger than a word.
- */
- need_loop = task_has_mempolicy(tsk) ||
- !nodes_intersects(*newmems, tsk->mems_allowed);
- if (need_loop) {
- local_irq_disable();
- write_seqcount_begin(&tsk->mems_allowed_seq);
- }
+ local_irq_disable();
+ write_seqcount_begin(&tsk->mems_allowed_seq);
nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
mpol_rebind_task(tsk, newmems);
tsk->mems_allowed = *newmems;
- if (need_loop) {
- write_seqcount_end(&tsk->mems_allowed_seq);
- local_irq_enable();
- }
+ write_seqcount_end(&tsk->mems_allowed_seq);
+ local_irq_enable();
task_unlock(tsk);
}
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox