* Re: [RFC 1/6] mm, page_alloc: fix more premature OOM due to race with cpuset update
From: Christoph Lameter @ 2017-05-17 13:56 UTC (permalink / raw)
To: Michal Hocko
Cc: Vlastimil Babka, linux-mm, linux-kernel, cgroups, Li Zefan,
Mel Gorman, David Rientjes, Hugh Dickins, Andrea Arcangeli,
Anshuman Khandual, Kirill A. Shutemov, linux-api
In-Reply-To: <20170517092042.GH18247@dhcp22.suse.cz>
On Wed, 17 May 2017, Michal Hocko wrote:
> > We certainly can do that. The failure of the page faults are due to the
> > admin trying to move an application that is not aware of this and is using
> > mempols. That could be an error. Trying to move an application that
> > contains both absolute and relative node numbers is definitely something
> > that is potentiall so screwed up that the kernel should not muck around
> > with such an app.
> >
> > Also user space can determine if the application is using memory policies
> > and can then take appropriate measures (message to the sysadmin to eval
> > tge situation f.e.) or mess aroud with the processes memory policies on
> > its own.
> >
> > So this is certainly a way out of this mess.
>
> So how are you going to distinguish VM_FAULT_OOM from an empty mempolicy
> case in a raceless way?
You dont have to do that if you do not create an empty mempolicy in the
first place. The current kernel code avoids that by first allowing access
to the new set of nodes and removing the old ones from the set when done.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Christoph Lameter @ 2017-05-17 13:54 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Mel Gorman, Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Rik van Riel
In-Reply-To: <1494973607.21847.50.camel@kernel.crashing.org>
On Wed, 17 May 2017, Benjamin Herrenschmidt wrote:
> On Tue, 2017-05-16 at 09:43 +0100, Mel Gorman wrote:
> > I'm not sure what you're asking here. migration is only partially
> > transparent but a move_pages call will be necessary to force pages onto
> > CDM if binding policies are not used so the cost of migration will be
> > invisible. Even if you made it "transparent", the migration cost would
> > be incurred at fault time. If anything, using move_pages would be more
> > predictable as you control when the cost is incurred.
>
> One of the main point of this whole exercise is for applications to not
> have to bother with any of this and now you are bringing all back into
> their lap.
You can provide a library that does it?
> The base idea behind the counters we have on the link is for the HW to
> know when memory is accessed "remotely", so that the device driver can
> make decision about migrating pages into or away from the device,
> especially so that applications don't have to concern themselves with
> memory placement.
Library can enquire about the current placement of the pages and move them
if necessary?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [Qustion] vmalloc area overlap with another allocated vmalloc area
From: Michal Hocko @ 2017-05-17 13:44 UTC (permalink / raw)
To: zhong jiang
Cc: Vlastimil Babka, Joonsoo Kim, Linux Memory Management List, LKML
In-Reply-To: <591C47E5.9010806@huawei.com>
On Wed 17-05-17 20:53:57, zhong jiang wrote:
> +to linux-mm maintainer for any suggestions
>
> Thanks
> zhongjiang
> On 2017/5/16 13:03, zhong jiang wrote:
> > Hi
> >
> > I hit the following issue by runing /proc/vmallocinfo. The kernel is 4.1 stable and
> > 32 bit to be used. after I expand the vamlloc area, the issue is not occur again.
> > it is related to the overflow. but I do not see any problem so far.
Is this a clean 4.1 stable kernel without any additional patches on top?
Are you able to reproduce this? How? Does the same problem happen with
the current Linus tree?
> > cat /proc/vmallocinfo
> > 0xf1580000-0xf1600000 524288 raw_dump_mem_write+0x10c/0x188 phys=8b901000 ioremap
> > 0xf1638000-0xf163a000 8192 mcss_pou_queue_init+0xa0/0x13c [mcss] phys=fc614000 ioremap
> > 0xf528e000-0xf5292000 16384 n_tty_open+0x10/0xd0 pages=3 vmalloc
> > 0xf5000000-0xf9001000 67112960 devm_ioremap+0x38/0x70 phys=40000000 ioremap
> > 0xfe001000-0xfe002000 4096 iotable_init+0x0/0xc phys=20001000 ioremap
> > 0xfe200000-0xfe201000 4096 iotable_init+0x0/0xc phys=1a000000 ioremap
> > 0xff100000-0xff101000 4096 iotable_init+0x0/0xc phys=2000a000 ioremap
> >
> > n_tty_open allocate the vmap area is surrounded by the devm_ioremap ioremap by above info.
> > I do not see also the race in the condition.
> >
> > I have no idea to the issue. Anyone has any suggestions will be appreicated.
> > The related config is attatched.
> >
> > Thanks
> > zhongjiang
>
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] Correct spelling and grammar for notification text
From: Michael DeGuzis @ 2017-05-17 13:38 UTC (permalink / raw)
To: linux-mm; +Cc: trivial, professorkaos64
From: professorkaos64 <mdeguzis@gmail.com>
This patch fixes up some grammar and spelling in the information
block for huge_memory.c.
---
mm/huge_memory.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index a84909cf20d3..af137fc0ca09 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -38,12 +38,12 @@
#include "internal.h"
/*
- * By default transparent hugepage support is disabled in order that avoid
- * to risk increase the memory footprint of applications without a guaranteed
- * benefit. When transparent hugepage support is enabled, is for all mappings,
- * and khugepaged scans all mappings.
+ * By default, transparent hugepage support is disabled in order to avoid
+ * risking an increased memory footprint for applications that are not
+ * guaranteed to benefit from it. When transparent hugepage support is
+ * enabled, it is for all mappings, and khugepaged scans all mappings.
* Defrag is invoked by khugepaged hugepage allocations and by page faults
- * for all hugepage allocations.
+ * for all hugepage allocations.
*/
unsigned long transparent_hugepage_flags __read_mostly =
#ifdef CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [Qustion] vmalloc area overlap with another allocated vmalloc area
From: zhong jiang @ 2017-05-17 12:53 UTC (permalink / raw)
To: Michal Hocko, Vlastimil Babka, Joonsoo Kim
Cc: Linux Memory Management List, LKML
In-Reply-To: <591A8814.1010503@huawei.com>
+to linux-mm maintainer for any suggestions
Thanks
zhongjiang
On 2017/5/16 13:03, zhong jiang wrote:
> Hi
>
> I hit the following issue by runing /proc/vmallocinfo. The kernel is 4.1 stable and
> 32 bit to be used. after I expand the vamlloc area, the issue is not occur again.
> it is related to the overflow. but I do not see any problem so far.
>
> cat /proc/vmallocinfo
> 0xf1580000-0xf1600000 524288 raw_dump_mem_write+0x10c/0x188 phys=8b901000 ioremap
> 0xf1638000-0xf163a000 8192 mcss_pou_queue_init+0xa0/0x13c [mcss] phys=fc614000 ioremap
> 0xf528e000-0xf5292000 16384 n_tty_open+0x10/0xd0 pages=3 vmalloc
> 0xf5000000-0xf9001000 67112960 devm_ioremap+0x38/0x70 phys=40000000 ioremap
> 0xfe001000-0xfe002000 4096 iotable_init+0x0/0xc phys=20001000 ioremap
> 0xfe200000-0xfe201000 4096 iotable_init+0x0/0xc phys=1a000000 ioremap
> 0xff100000-0xff101000 4096 iotable_init+0x0/0xc phys=2000a000 ioremap
>
> n_tty_open allocate the vmap area is surrounded by the devm_ioremap ioremap by above info.
> I do not see also the race in the condition.
>
> I have no idea to the issue. Anyone has any suggestions will be appreicated.
> The related config is attatched.
>
> Thanks
> zhongjiang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: clarify why we want kmalloc before falling backto vmallock
From: Chris Wilson @ 2017-05-17 13:02 UTC (permalink / raw)
To: Michal Hocko; +Cc: Andrew Morton, Vlastimil Babka, linux-mm, LKML, Michal Hocko
In-Reply-To: <20170517080932.21423-1-mhocko@kernel.org>
subject s/vmallock/vmalloc/
On Wed, May 17, 2017 at 10:09:32AM +0200, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> While converting drm_[cm]alloc* helpers to kvmalloc* variants Chris
> Wilson has wondered why we want to try kmalloc before vmalloc fallback
> even for larger allocations requests. Let's clarify that one larger
> physically contiguous block is less likely to fragment memory than many
> scattered pages which can prevent more large blocks from being created.
>
> Suggested-by: Chris Wilson <chris@chris-wilson.co.uk>
> Signed-off-by: Michal Hocko <mhocko@suse.com>
It helped me understand the decisions made by the code, so
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Michal Hocko @ 2017-05-17 12:41 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Mel Gorman, Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <1495014995.3092.20.camel@kernel.crashing.org>
On Wed 17-05-17 19:56:35, Benjamin Herrenschmidt wrote:
> On Wed, 2017-05-17 at 10:15 +0100, Mel Gorman wrote:
[...]
> > Fine -- hot add the memory from the device via a userspace trigger and
> > have the userspace trigger then setup the policies to isolate CDM from
> > general usage.
>
> This is racy though. The memory is hot added, but things can get
> allocated all over it before it has time to adjust the policies. Same
> issue we had with creating a CMA I believe.
memory hotplug is by definition 2 stage. Physical hotadd which just
prepares memory blocks and allocates struct pages and the memory online
phase. You can handle the policy part from the userspace before onlining
te first memblock from your CDM NUMA node.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH 2/2 -v2] drm: drop drm_[cm]alloc* helpers
From: Michal Hocko @ 2017-05-17 12:23 UTC (permalink / raw)
To: dri-devel
Cc: linux-kernel, linux-mm, Daniel Vetter, Jani Nikula, Sean Paul,
David Airlie
In-Reply-To: <20170517065509.18659-2-mhocko@kernel.org>
As it turned out my allyesconfig on x86_64 wasn't sufficient and 0day
build machinery found a failure on arm architecture. It was clearly a
typo. Now I have pushed this to my build battery with cross arch
compilers and it passes so there shouldn't more surprises hopefully.
Here is the v2.
---
^ permalink raw reply
* Re: [PATCH v1 00/11] mm/kasan: support per-page shadow memory to reduce memory consumption
From: Andrey Ryabinin @ 2017-05-17 12:17 UTC (permalink / raw)
To: js1304, Andrew Morton
Cc: Alexander Potapenko, Dmitry Vyukov, kasan-dev, linux-mm,
linux-kernel, Thomas Gleixner, Ingo Molnar, H . Peter Anvin,
kernel-team, Joonsoo Kim
In-Reply-To: <1494897409-14408-1-git-send-email-iamjoonsoo.kim@lge.com>
On 05/16/2017 04:16 AM, js1304@gmail.com wrote:
> From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> Hello, all.
>
> This is an attempt to recude memory consumption of KASAN. Please see
> following description to get the more information.
>
> 1. What is per-page shadow memory
>
> This patch introduces infrastructure to support per-page shadow memory.
> Per-page shadow memory is the same with original shadow memory except
> the granualarity. It's one byte shows the shadow value for the page.
> The purpose of introducing this new shadow memory is to save memory
> consumption.
>
> 2. Problem of current approach
>
> Until now, KASAN needs shadow memory for all the range of the memory
> so the amount of statically allocated memory is so large. It causes
> the problem that KASAN cannot run on the system with hard memory
> constraint. Even if KASAN can run, large memory consumption due to
> KASAN changes behaviour of the workload so we cannot validate
> the moment that we want to check.
>
> 3. How does this patch fix the problem
>
> This patch tries to fix the problem by reducing memory consumption for
> the shadow memory. There are two observations.
>
I think that the best way to deal with your problem is to increase shadow scale size.
You'll need to add tunable to gcc to control shadow size. I expect that gcc has some
places where 8-shadow scale size is hardcoded, but it should be fixable.
The kernel also have some small amount of code written with KASAN_SHADOW_SCALE_SIZE == 8 in mind,
which should be easy to fix.
Note that bigger shadow scale size requires bigger alignment of allocated memory and variables.
However, according to comments in gcc/asan.c gcc already aligns stack and global variables and at
32-bytes boundary.
So we could bump shadow scale up to 32 without increasing current stack consumption.
On a small machine (1Gb) 1/32 of shadow is just 32Mb which is comparable to yours 30Mb, but I expect it to be
much faster. More importantly, this will require only small amount of simple changes in code, which will be
a *lot* more easier to maintain.
I'd start from implementing this on the kernel side only. With KASAN_OUTLINE and disabled
stack instrumentation (--param asan-stack=0) it's doable without any changes in gcc.
...
> base vs patched
>
> MemTotal: 858 MB vs 987 MB
> runtime: 0 MB vs 30MB
> Net Available: 858 MB vs 957 MB
>
> For 4096 MB QEMU system
>
> MemTotal: 3477 MB vs 4000 MB
> runtime: 0 MB vs 50MB
>
> base vs patched (2048 MB QEMU system)
> 204 s vs 224 s
> Net Available: 3477 MB vs 3950 MB
>
> Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/2] drm: drop drm_[cm]alloc* helpers
From: kbuild test robot @ 2017-05-17 11:13 UTC (permalink / raw)
To: Michal Hocko
Cc: kbuild-all, dri-devel, linux-kernel, linux-mm, Daniel Vetter,
Jani Nikula, Sean Paul, David Airlie, Michal Hocko
In-Reply-To: <20170517065509.18659-2-mhocko@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]
Hi Michal,
[auto build test ERROR on drm/drm-next]
[also build test ERROR on v4.12-rc1 next-20170517]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Michal-Hocko/drm-replace-drm_-cm-alloc-by-kvmalloc-alternatives/20170517-150333
base: git://people.freedesktop.org/~airlied/linux.git drm-next
config: arm-allmodconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm
All error/warnings (new ones prefixed by >>):
drivers/gpu//drm/etnaviv/etnaviv_gem.c: In function 'etnaviv_gem_userptr_do_get_pages':
>> drivers/gpu//drm/etnaviv/etnaviv_gem.c:751:9: error: implicit declaration of function 'kvmallo_array' [-Werror=implicit-function-declaration]
pvec = kvmallo_array(npages, sizeof(struct page *), GFP_KERNEL);
^~~~~~~~~~~~~
>> drivers/gpu//drm/etnaviv/etnaviv_gem.c:751:7: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
pvec = kvmallo_array(npages, sizeof(struct page *), GFP_KERNEL);
^
cc1: some warnings being treated as errors
vim +/kvmallo_array +751 drivers/gpu//drm/etnaviv/etnaviv_gem.c
745 {
746 int ret = 0, pinned, npages = etnaviv_obj->base.size >> PAGE_SHIFT;
747 struct page **pvec;
748 uintptr_t ptr;
749 unsigned int flags = 0;
750
> 751 pvec = kvmallo_array(npages, sizeof(struct page *), GFP_KERNEL);
752 if (!pvec)
753 return ERR_PTR(-ENOMEM);
754
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 62372 bytes --]
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Mel Gorman @ 2017-05-17 10:58 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <1495014995.3092.20.camel@kernel.crashing.org>
On Wed, May 17, 2017 at 07:56:35PM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2017-05-17 at 10:15 +0100, Mel Gorman wrote:
> > > We can, via a driver specific hook, since we have specific counters on
> > > the link, so we don't want the autonuma based approach which makes PTEs
> > > inaccessible.
> > >
> >
> > Then poll the driver from a userspace daemon and make placement
> > decisions if automatic NUMA balancings reference-based decisions are
> > unsuitable.
>
> Why a userspace daemon ? I don't get this... the driver will get
> interrupts from the GPU with page lists, it can trigger migrations
> without needing a userspace daemon...
>
Then handle it within the driver. The point is that it still doesn't
need hooks into the core VM at this point.
> > > > To handle it transparently, either the driver needs to do the work in which
> > > > case no special core-kernel support is needed beyond what already exists or
> > > > there is a userspace daemon like numad running in userspace that decides
> > > > when to trigger migrations on a separate process that is using CDM which
> > > > would need to gather information from the driver.
> > >
> > > The driver can handle it, we just need autonuma off the CDM memory (it
> > > can continue operating normally on system memory).
> > >
> >
> > Already suggested that prctl be used to disable automatic numa balancing
> > on a per-task basis. Alternatively, settiing a memory policy will be
> > enough and as the applications are going to need policies anyway, you
> > should be able to get that by default.
>
> I'm not sure we want to disable it for the application vs. disabling it
> for pages that reside on that node,
Then use a memory policy to control which VMAs are exempt. If you do not
wants at all for particular nodes then that would need core VM support
but you'll lose transparency. If you want to flag particular pgdats,
then it'll be adding a check to the task scanner but it would need to be
clearly shown that there is a lot of value in teaching automatic NUMA
balancing this.
> > > > long as the driver hot-adds the CDM memory from a userspace trigger that
> > > > it then responsible for setting up the isolation.
> > >
> > > Yes, I think the NUMA node based approach works fine using a lot of
> > > existing stuff. There are a couple of gaps, which we need to look at
> > > fixing one way or another such as the above, but overall I don't see
> > > the need of some major overhaul, not do I see the need of going down
> > > the path of ZONE_DEVICE.
> > >
> > Your choice, but it also doesn't take away from the fact that special
> > casing in the core does not appear to be required at this point.
>
> Well, yes and no.
>
> If we use the NUMA based approach, then no special casing up to this
> point, the only thing is below, the idea of avoiding "normal"
> allocations for that type of memory.
>
Use cpusets from userspace, and control carefully how and when the memory
is hot-added and what zone it gets added to. We've been through this.
> > Use policies. If the NUMA distance for CDM is set high then even applications
> > that have access to CDM will use every other node before going to CDM.
>
> Yes. That was the original idea. Along with ZONE_MOVABLE to avoid
> kernel allocations completely.
>
Remember that this will include the page table pages which may or may
not be what you want.
> I think Balbir and Anshuman wanted to play with a more fully exclusive
> approach where those allocations are simply not permitted.
>
Use cpusets and control carefully how and when the memory is hot-added
and what zone it gets added to.
> > As
> > you insist on no application awareness, the migration to CDM will have to
> > be controlled by a separate daemon.
>
> Or by the driver itself, I don't think we need a daemon, but that's a
> detail in the grand scheme of things.
>
It also doesn't need core VM hooks or special support.
> > > IE, things should go to the GPU memory if and only if they are either
> > > explicitly put there by the application/driver (the case where
> > > applications do care about manual placement), or the migration case.
> > >
> > > The latter is triggered by the driver, so it's also a case of the
> > > driver allocating the GPU pages and doing a migration to them.
> > >
> > > This is the key thing. Now creating a CMA or using ZONE_MOVABLE can
> > > handle at least keeping kernel allocations off the GPU. However we
> > > would also like to keep random unrelated user memory & page cache off
> > > as well.
> > >
> >
> > Fine -- hot add the memory from the device via a userspace trigger and
> > have the userspace trigger then setup the policies to isolate CDM from
> > general usage.
>
> This is racy though. The memory is hot added, but things can get
> allocated all over it before it has time to adjust the policies. Same
> issue we had with creating a CMA I believe.
>
The race is a non-issue unless for some reason you decide to hot-add the node
when the machine is already heavily loaded and under memory pressure. Do it
near boot time and no CPU-local allocation is going to hit it. In itself,
special casing the core VM is overkill.
If you decide to use ZONE_MOVABLE and take the remote hit penalty of page
tables, then you can also migrate all the pages away after the onlining
and isolation is complete if it's a serious concern in practice.
> Unless we have a way to create a node without actually making it
> available for allocations, so we get a chance to establish policies for
> it, then "online" it ?
>
Conceivably, that could be done although again it's somewhat overkill
as the race only applies if hot-adding CDM under heavy memory pressure
sufficient to overflow to a very remote node.
> Doing these from userspace is a bit nasty since it's expected to all be
> under the control of the GPU driver, but it could be done via a
> combination of GPU driver & udev helpers or a special daemon.
>
Special casing the core VM in multiple places is also nasty as it shoves
all the maintenance overhead into places where most people will not be
able to verify it's still working due to a lack of hardware.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Benjamin Herrenschmidt @ 2017-05-17 9:56 UTC (permalink / raw)
To: Mel Gorman
Cc: Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <20170517091511.gjxx46d2h6gmcqjf@techsingularity.net>
On Wed, 2017-05-17 at 10:15 +0100, Mel Gorman wrote:
> > We can, via a driver specific hook, since we have specific counters on
> > the link, so we don't want the autonuma based approach which makes PTEs
> > inaccessible.
> >
>
> Then poll the driver from a userspace daemon and make placement
> decisions if automatic NUMA balancings reference-based decisions are
> unsuitable.
Why a userspace daemon ? I don't get this... the driver will get
interrupts from the GPU with page lists, it can trigger migrations
without needing a userspace daemon...
> > > To handle it transparently, either the driver needs to do the work in which
> > > case no special core-kernel support is needed beyond what already exists or
> > > there is a userspace daemon like numad running in userspace that decides
> > > when to trigger migrations on a separate process that is using CDM which
> > > would need to gather information from the driver.
> >
> > The driver can handle it, we just need autonuma off the CDM memory (it
> > can continue operating normally on system memory).
> >
>
> Already suggested that prctl be used to disable automatic numa balancing
> on a per-task basis. Alternatively, settiing a memory policy will be
> enough and as the applications are going to need policies anyway, you
> should be able to get that by default.
I'm not sure we want to disable it for the application vs. disabling it
for pages that reside on that node, however, but it could be tricky so
the application first might be a way to get started.
> > > In either case, the existing isolation mechanisms are still sufficient as
> > > long as the driver hot-adds the CDM memory from a userspace trigger that
> > > it then responsible for setting up the isolation.
> >
> > Yes, I think the NUMA node based approach works fine using a lot of
> > existing stuff. There are a couple of gaps, which we need to look at
> > fixing one way or another such as the above, but overall I don't see
> > the need of some major overhaul, not do I see the need of going down
> > the path of ZONE_DEVICE.
> >
> Your choice, but it also doesn't take away from the fact that special
> casing in the core does not appear to be required at this point.
Well, yes and no.
If we use the NUMA based approach, then no special casing up to this
point, the only thing is below, the idea of avoiding "normal"
allocations for that type of memory.
If we use ZONE_DEVICE and the bulk of the HMM infrastructure, then we
get the above, but at the expense of a pile of special casing all over
the place for the "special" kind of struct page created for ZONE_DEVICE
(lacking LRU).
> > > All that aside, this series has nothing to do with the type of magic
> > > you describe and the feedback as iven was "at this point, what you are
> > > looking for does not require special kernel support or heavy wiring into
> > > the core vm".
> > >
> > > > Thus we want to reply on the GPU driver moving the pages around where
> > > > most appropriate (where they are being accessed, either core memory or
> > > > GPU memory) based on inputs from the HW counters monitoring the link.
> > > >
> > >
> > > And if the driver is polling all the accesses, there are still no changes
> > > required to the core vm as long as the driver does the hotplug and allows
> > > userspace to isolate if that is what the applications desire.
> >
> > With one main exception ...
> >
> > We also do want normal allocations to avoid going to the GPU memory.
> >
>
> Use policies. If the NUMA distance for CDM is set high then even applications
> that have access to CDM will use every other node before going to CDM.
Yes. That was the original idea. Along with ZONE_MOVABLE to avoid
kernel allocations completely.
I think Balbir and Anshuman wanted to play with a more fully exclusive
approach where those allocations are simply not permitted.
> As
> you insist on no application awareness, the migration to CDM will have to
> be controlled by a separate daemon.
Or by the driver itself, I don't think we need a daemon, but that's a
detail in the grand scheme of things.
> > IE, things should go to the GPU memory if and only if they are either
> > explicitly put there by the application/driver (the case where
> > applications do care about manual placement), or the migration case.A
> >
> > The latter is triggered by the driver, so it's also a case of the
> > driver allocating the GPU pages and doing a migration to them.
> >
> > This is the key thing. Now creating a CMA or using ZONE_MOVABLE can
> > handle at least keeping kernel allocations off the GPU. However we
> > would also like to keep random unrelated user memory & page cache off
> > as well.
> >
>
> Fine -- hot add the memory from the device via a userspace trigger and
> have the userspace trigger then setup the policies to isolate CDM from
> general usage.
This is racy though. The memory is hot added, but things can get
allocated all over it before it has time to adjust the policies. Same
issue we had with creating a CMA I believe.
I think that's what Balbir was trying to do with the changes to the
core, to be able to create that "don't touche me" NUMA node straight
up.
Unless we have a way to create a node without actually making it
available for allocations, so we get a chance to establish policies for
it, then "online" it ?
Doing these from userspace is a bit nasty since it's expected to all be
under the control of the GPU driver, but it could be done via a
combination of GPU driver & udev helpers or a special daemon.
Cheers,
Ben.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/2] drm: replace drm_[cm]alloc* by kvmalloc alternatives
From: Michal Hocko @ 2017-05-17 9:23 UTC (permalink / raw)
To: Chris Wilson
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170517091241.GL26693@nuc-i3427.alporthouse.com>
On Wed 17-05-17 10:12:41, Chris Wilson wrote:
> On Wed, May 17, 2017 at 11:03:50AM +0200, Michal Hocko wrote:
[...]
> > +static inline bool alloc_array_check(size_t n, size_t size)
> > +{
> > + if (size != 0 && n > SIZE_MAX / size)
> > + return false;
> > + return true;
>
> Just return size == 0 || n <= SIZE_MAX /size ?
>
> Whether or not size being 0 makes for a sane user is another question.
> The guideline is that size is the known constant from sizeof() or
> whatever and n is the variable number to allocate.
>
> But yes, that inline is what I want :)
I will think about this. Maybe it will help to simplify/unify some other
users. Do you have any pointers to save me some grepping...?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 1/6] mm, page_alloc: fix more premature OOM due to race with cpuset update
From: Michal Hocko @ 2017-05-17 9:20 UTC (permalink / raw)
To: Christoph Lameter
Cc: Vlastimil Babka, linux-mm, linux-kernel, cgroups, Li Zefan,
Mel Gorman, David Rientjes, Hugh Dickins, Andrea Arcangeli,
Anshuman Khandual, Kirill A. Shutemov, linux-api
In-Reply-To: <alpine.DEB.2.20.1704301628460.21533@east.gentwo.org>
On Sun 30-04-17 16:33:10, Cristopher Lameter wrote:
> On Wed, 26 Apr 2017, Vlastimil Babka wrote:
>
> > > Such an application typically already has such logic and executes a
> > > binding after discovering its numa node configuration on startup. It would
> > > have to be modified to redo that action when it gets some sort of a signal
> > > from the script telling it that the node config would be changed.
> > >
> > > Having this logic in the application instead of the kernel avoids all the
> > > kernel messes that we keep on trying to deal with and IMHO is much
> > > cleaner.
> >
> > That would be much simpler for us indeed. But we still IMHO can't
> > abruptly start denying page fault allocations for existing applications
> > that don't have the necessary awareness.
>
> We certainly can do that. The failure of the page faults are due to the
> admin trying to move an application that is not aware of this and is using
> mempols. That could be an error. Trying to move an application that
> contains both absolute and relative node numbers is definitely something
> that is potentiall so screwed up that the kernel should not muck around
> with such an app.
>
> Also user space can determine if the application is using memory policies
> and can then take appropriate measures (message to the sysadmin to eval
> tge situation f.e.) or mess aroud with the processes memory policies on
> its own.
>
> So this is certainly a way out of this mess.
So how are you going to distinguish VM_FAULT_OOM from an empty mempolicy
case in a raceless way?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Mel Gorman @ 2017-05-17 9:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <1495011826.3092.18.camel@kernel.crashing.org>
On Wed, May 17, 2017 at 07:03:46PM +1000, Benjamin Herrenschmidt wrote:
> > There is only so much magic that can be applied and if the manual case
> > cannot be handled then the automatic case is problematic. You say that you
> > want kswapd disabled, but have nothing to handle overcommit sanely.
>
> I am not certain we want kswapd disabled, that is definitely more of a
> userspace policy, I agree. It could be in this case that it should
> prioritize different pages but still be able to push out. We *do* have
> age counting etc... just less efficient / higher cost.
>
If you don't want kswapd disabled, then the existing support is
sufficient unless different reclaim policies are required. If so, it
becomes a general problem of NUMA hierarchies where policies for nodes
may differ.
> > You
> > want to disable automatic NUMA balancing yet also be able to automatically
> > detect when data should move from CDM (automatic NUMA balancing by design
> > couldn't move data to CDM without driver support tracking GPU accesses).
>
> We can, via a driver specific hook, since we have specific counters on
> the link, so we don't want the autonuma based approach which makes PTEs
> inaccessible.
>
Then poll the driver from a userspace daemon and make placement
decisions if automatic NUMA balancings reference-based decisions are
unsuitable.
> > To handle it transparently, either the driver needs to do the work in which
> > case no special core-kernel support is needed beyond what already exists or
> > there is a userspace daemon like numad running in userspace that decides
> > when to trigger migrations on a separate process that is using CDM which
> > would need to gather information from the driver.
>
> The driver can handle it, we just need autonuma off the CDM memory (it
> can continue operating normally on system memory).
>
Already suggested that prctl be used to disable automatic numa balancing
on a per-task basis. Alternatively, settiing a memory policy will be
enough and as the applications are going to need policies anyway, you
should be able to get that by default.
> > In either case, the existing isolation mechanisms are still sufficient as
> > long as the driver hot-adds the CDM memory from a userspace trigger that
> > it then responsible for setting up the isolation.
>
> Yes, I think the NUMA node based approach works fine using a lot of
> existing stuff. There are a couple of gaps, which we need to look at
> fixing one way or another such as the above, but overall I don't see
> the need of some major overhaul, not do I see the need of going down
> the path of ZONE_DEVICE.
>
Your choice, but it also doesn't take away from the fact that special
casing in the core does not appear to be required at this point.
> > All that aside, this series has nothing to do with the type of magic
> > you describe and the feedback as iven was "at this point, what you are
> > looking for does not require special kernel support or heavy wiring into
> > the core vm".
> >
> > > Thus we want to reply on the GPU driver moving the pages around where
> > > most appropriate (where they are being accessed, either core memory or
> > > GPU memory) based on inputs from the HW counters monitoring the link.
> > >
> >
> > And if the driver is polling all the accesses, there are still no changes
> > required to the core vm as long as the driver does the hotplug and allows
> > userspace to isolate if that is what the applications desire.
>
> With one main exception ...
>
> We also do want normal allocations to avoid going to the GPU memory.
>
Use policies. If the NUMA distance for CDM is set high then even applications
that have access to CDM will use every other node before going to CDM. As
you insist on no application awareness, the migration to CDM will have to
be controlled by a separate daemon.
> IE, things should go to the GPU memory if and only if they are either
> explicitly put there by the application/driver (the case where
> applications do care about manual placement), or the migration case.
>
> The latter is triggered by the driver, so it's also a case of the
> driver allocating the GPU pages and doing a migration to them.
>
> This is the key thing. Now creating a CMA or using ZONE_MOVABLE can
> handle at least keeping kernel allocations off the GPU. However we
> would also like to keep random unrelated user memory & page cache off
> as well.
>
Fine -- hot add the memory from the device via a userspace trigger and
have the userspace trigger then setup the policies to isolate CDM from
general usage.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/2] drm: replace drm_[cm]alloc* by kvmalloc alternatives
From: Chris Wilson @ 2017-05-17 9:12 UTC (permalink / raw)
To: Michal Hocko
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170517090350.GG18247@dhcp22.suse.cz>
On Wed, May 17, 2017 at 11:03:50AM +0200, Michal Hocko wrote:
> On Wed 17-05-17 08:38:09, Chris Wilson wrote:
> > On Wed, May 17, 2017 at 08:55:08AM +0200, Michal Hocko wrote:
> > > From: Michal Hocko <mhocko@suse.com>
> > >
> > > drm_[cm]alloc* has grown their own kvmalloc with vmalloc fallback
> > > implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> > > use those because it a) reduces the code and b) MM has a better idea
> > > how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> > > with __GFP_NORETRY).
> > >
> > > drm_calloc_large needs to get __GFP_ZERO explicitly but it is the same
> > > thing as kvmalloc_array in principle.
> > >
> > > Signed-off-by: Michal Hocko <mhocko@suse.com>
> >
> > Just a little surprised that calloc_large users still exist.
> >
> > Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
>
> Thanks!
>
> > One more feature request from mm, can we have the
> > if (size != 0 && n > SIZE_MAX / size)
> > check exported by itself.
>
> What do you exactly mean by exporting?
Just make available to others so that little things like choice between
SIZE_MAX and ULONG_MAX are consistent and actually reflect the right
limit (as dictated by kmalloc/kvmalloc/vmalloc...).
> Something like the following?
> I haven't compile tested it outside of mm with different config options.
> Sticking alloc_array_check into mm_types.h is kind of gross but I do not
> have a great idea where to put it. A new header doesn't seem nice.
> ---
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 7cb17c6b97de..f908b14ffc4c 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -534,7 +534,7 @@ static inline void *kvzalloc(size_t size, gfp_t flags)
>
> static inline void *kvmalloc_array(size_t n, size_t size, gfp_t flags)
> {
> - if (size != 0 && n > SIZE_MAX / size)
> + if (!alloc_array_check(n, size))
> return NULL;
>
> return kvmalloc(n * size, flags);
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 45cdb27791a3..d7154b43a0d1 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -601,4 +601,10 @@ typedef struct {
> unsigned long val;
> } swp_entry_t;
>
> +static inline bool alloc_array_check(size_t n, size_t size)
> +{
> + if (size != 0 && n > SIZE_MAX / size)
> + return false;
> + return true;
Just return size == 0 || n <= SIZE_MAX /size ?
Whether or not size being 0 makes for a sane user is another question.
The guideline is that size is the known constant from sizeof() or
whatever and n is the variable number to allocate.
But yes, that inline is what I want :)
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Benjamin Herrenschmidt @ 2017-05-17 9:03 UTC (permalink / raw)
To: Mel Gorman
Cc: Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <20170517082836.whe3hggeew23nwvz@techsingularity.net>
On Wed, 2017-05-17 at 09:28 +0100, Mel Gorman wrote:
> On Wed, May 17, 2017 at 08:26:47AM +1000, Benjamin Herrenschmidt wrote:
> > On Tue, 2017-05-16 at 09:43 +0100, Mel Gorman wrote:
> > > I'm not sure what you're asking here. migration is only partially
> > > transparent but a move_pages call will be necessary to force pages onto
> > > CDM if binding policies are not used so the cost of migration will be
> > > invisible. Even if you made it "transparent", the migration cost would
> > > be incurred at fault time. If anything, using move_pages would be more
> > > predictable as you control when the cost is incurred.
> >
> > One of the main point of this whole exercise is for applications to not
> > have to bother with any of this and now you are bringing all back into
> > their lap.
> >
> > The base idea behind the counters we have on the link is for the HW to
> > know when memory is accessed "remotely", so that the device driver can
> > make decision about migrating pages into or away from the device,
> > especially so that applications don't have to concern themselves with
> > memory placement.
> >
>
> There is only so much magic that can be applied and if the manual case
> cannot be handled then the automatic case is problematic. You say that you
> want kswapd disabled, but have nothing to handle overcommit sanely.
I am not certain we want kswapd disabled, that is definitely more of a
userspace policy, I agree. It could be in this case that it should
prioritize different pages but still be able to push out. We *do* have
age counting etc... just less efficient / higher cost.
> You
> want to disable automatic NUMA balancing yet also be able to automatically
> detect when data should move from CDM (automatic NUMA balancing by design
> couldn't move data to CDM without driver support tracking GPU accesses).
We can, via a driver specific hook, since we have specific counters on
the link, so we don't want the autonuma based approach which makes PTEs
inaccessible.
> To handle it transparently, either the driver needs to do the work in which
> case no special core-kernel support is needed beyond what already exists or
> there is a userspace daemon like numad running in userspace that decides
> when to trigger migrations on a separate process that is using CDM which
> would need to gather information from the driver.
The driver can handle it, we just need autonuma off the CDM memory (it
can continue operating normally on system memory).
> In either case, the existing isolation mechanisms are still sufficient as
> long as the driver hot-adds the CDM memory from a userspace trigger that
> it then responsible for setting up the isolation.
Yes, I think the NUMA node based approach works fine using a lot of
existing stuff. There are a couple of gaps, which we need to look at
fixing one way or another such as the above, but overall I don't see
the need of some major overhaul, not do I see the need of going down
the path of ZONE_DEVICE.
> All that aside, this series has nothing to do with the type of magic
> you describe and the feedback as iven was "at this point, what you are
> looking for does not require special kernel support or heavy wiring into
> the core vm".
>
> > Thus we want to reply on the GPU driver moving the pages around where
> > most appropriate (where they are being accessed, either core memory or
> > GPU memory) based on inputs from the HW counters monitoring the link.
> >
>
> And if the driver is polling all the accesses, there are still no changes
> required to the core vm as long as the driver does the hotplug and allows
> userspace to isolate if that is what the applications desire.
With one main exception ...
We also do want normal allocations to avoid going to the GPU memory.
IE, things should go to the GPU memory if and only if they are either
explicitly put there by the application/driver (the case where
applications do care about manual placement), or the migration case.A
The latter is triggered by the driver, so it's also a case of the
driver allocating the GPU pages and doing a migration to them.
This is the key thing. Now creating a CMA or using ZONE_MOVABLE can
handle at least keeping kernel allocations off the GPU. However we
would also like to keep random unrelated user memory & page cache off
as well.
There are various reasons for that, some related to the fact that the
performance characteristics of that memory (ie latency) could cause
nasty surprises for normal applications, some related to the fact that
this memory is rather unreliable compared to system memory...
Cheers,
Ben.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/2] drm: replace drm_[cm]alloc* by kvmalloc alternatives
From: Michal Hocko @ 2017-05-17 9:03 UTC (permalink / raw)
To: Chris Wilson
Cc: dri-devel, linux-kernel, linux-mm, Daniel Vetter, Jani Nikula,
Sean Paul, David Airlie
In-Reply-To: <20170517073809.GJ26693@nuc-i3427.alporthouse.com>
On Wed 17-05-17 08:38:09, Chris Wilson wrote:
> On Wed, May 17, 2017 at 08:55:08AM +0200, Michal Hocko wrote:
> > From: Michal Hocko <mhocko@suse.com>
> >
> > drm_[cm]alloc* has grown their own kvmalloc with vmalloc fallback
> > implementations. MM has grown kvmalloc* helpers in the meantime. Let's
> > use those because it a) reduces the code and b) MM has a better idea
> > how to implement fallbacks (e.g. do not vmalloc before kmalloc is tried
> > with __GFP_NORETRY).
> >
> > drm_calloc_large needs to get __GFP_ZERO explicitly but it is the same
> > thing as kvmalloc_array in principle.
> >
> > Signed-off-by: Michal Hocko <mhocko@suse.com>
>
> Just a little surprised that calloc_large users still exist.
>
> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Thanks!
> One more feature request from mm, can we have the
> if (size != 0 && n > SIZE_MAX / size)
> check exported by itself.
What do you exactly mean by exporting? Something like the following?
I haven't compile tested it outside of mm with different config options.
Sticking alloc_array_check into mm_types.h is kind of gross but I do not
have a great idea where to put it. A new header doesn't seem nice.
---
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7cb17c6b97de..f908b14ffc4c 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -534,7 +534,7 @@ static inline void *kvzalloc(size_t size, gfp_t flags)
static inline void *kvmalloc_array(size_t n, size_t size, gfp_t flags)
{
- if (size != 0 && n > SIZE_MAX / size)
+ if (!alloc_array_check(n, size))
return NULL;
return kvmalloc(n * size, flags);
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 45cdb27791a3..d7154b43a0d1 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -601,4 +601,10 @@ typedef struct {
unsigned long val;
} swp_entry_t;
+static inline bool alloc_array_check(size_t n, size_t size)
+{
+ if (size != 0 && n > SIZE_MAX / size)
+ return false;
+ return true;
+}
#endif /* _LINUX_MM_TYPES_H */
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 3c37a8c51921..e936ca7c55a1 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -602,7 +602,7 @@ int memcg_update_all_caches(int num_memcgs);
*/
static inline void *kmalloc_array(size_t n, size_t size, gfp_t flags)
{
- if (size != 0 && n > SIZE_MAX / size)
+ if (!alloc_array_check(n, size))
return NULL;
if (__builtin_constant_p(n) && __builtin_constant_p(size))
return kmalloc(n * size, flags);
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [RFC summary] Enable Coherent Device Memory
From: Mel Gorman @ 2017-05-17 8:28 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Balbir Singh, linux-mm, akpm@linux-foundation.org,
Anshuman Khandual, Aneesh Kumar KV, Paul E. McKenney,
Srikar Dronamraju, Haren Myneni, Jérôme Glisse,
Reza Arbab, Vlastimil Babka, Christoph Lameter, Rik van Riel
In-Reply-To: <1494973607.21847.50.camel@kernel.crashing.org>
On Wed, May 17, 2017 at 08:26:47AM +1000, Benjamin Herrenschmidt wrote:
> On Tue, 2017-05-16 at 09:43 +0100, Mel Gorman wrote:
> > I'm not sure what you're asking here. migration is only partially
> > transparent but a move_pages call will be necessary to force pages onto
> > CDM if binding policies are not used so the cost of migration will be
> > invisible. Even if you made it "transparent", the migration cost would
> > be incurred at fault time. If anything, using move_pages would be more
> > predictable as you control when the cost is incurred.
>
> One of the main point of this whole exercise is for applications to not
> have to bother with any of this and now you are bringing all back into
> their lap.
>
> The base idea behind the counters we have on the link is for the HW to
> know when memory is accessed "remotely", so that the device driver can
> make decision about migrating pages into or away from the device,
> especially so that applications don't have to concern themselves with
> memory placement.
>
There is only so much magic that can be applied and if the manual case
cannot be handled then the automatic case is problematic. You say that you
want kswapd disabled, but have nothing to handle overcommit sanely. You
want to disable automatic NUMA balancing yet also be able to automatically
detect when data should move from CDM (automatic NUMA balancing by design
couldn't move data to CDM without driver support tracking GPU accesses).
To handle it transparently, either the driver needs to do the work in which
case no special core-kernel support is needed beyond what already exists or
there is a userspace daemon like numad running in userspace that decides
when to trigger migrations on a separate process that is using CDM which
would need to gather information from the driver.
In either case, the existing isolation mechanisms are still sufficient as
long as the driver hot-adds the CDM memory from a userspace trigger that
it then responsible for setting up the isolation.
All that aside, this series has nothing to do with the type of magic
you describe and the feedback as given was "at this point, what you are
looking for does not require special kernel support or heavy wiring into
the core vm".
> Thus we want to reply on the GPU driver moving the pages around where
> most appropriate (where they are being accessed, either core memory or
> GPU memory) based on inputs from the HW counters monitoring the link.
>
And if the driver is polling all the accesses, there are still no changes
required to the core vm as long as the driver does the hotplug and allows
userspace to isolate if that is what the applications desire.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v2 5/6] mm, cpuset: always use seqlock when changing task's nodemask
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
When updating task's mems_allowed and rebinding its mempolicy due to cpuset's
mems being changed, we currently only take the seqlock for writing when either
the task has a mempolicy, or the new mems has no intersection with the old
mems. This should be enough to prevent a parallel allocation seeing no
available nodes, but the optimization is IMHO unnecessary (cpuset updates
should not be frequent), and we still potentially risk issues if the
intersection of new and old nodes has limited amount of free/reclaimable
memory. Let's just use the seqlock for all tasks.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
kernel/cgroup/cpuset.c | 29 ++++++++---------------------
1 file changed, 8 insertions(+), 21 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index dfd5b420452d..26a1c360a481 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1038,38 +1038,25 @@ static void cpuset_post_attach(void)
* @tsk: the task to change
* @newmems: new nodes that the task will be set
*
- * In order to avoid seeing no nodes if the old and new nodes are disjoint,
- * we structure updates as setting all new allowed nodes, then clearing newly
- * disallowed ones.
+ * We use the mems_allowed_seq seqlock to safely update both tsk->mems_allowed
+ * and rebind an eventual tasks' mempolicy. If the task is allocating in
+ * parallel, it might temporarily see an empty intersection, which results in
+ * a seqlock check and retry before OOM or allocation failure.
*/
static void cpuset_change_task_nodemask(struct task_struct *tsk,
nodemask_t *newmems)
{
- bool need_loop;
-
task_lock(tsk);
- /*
- * Determine if a loop is necessary if another thread is doing
- * read_mems_allowed_begin(). If at least one node remains unchanged and
- * tsk does not have a mempolicy, then an empty nodemask will not be
- * possible when mems_allowed is larger than a word.
- */
- need_loop = task_has_mempolicy(tsk) ||
- !nodes_intersects(*newmems, tsk->mems_allowed);
- if (need_loop) {
- local_irq_disable();
- write_seqcount_begin(&tsk->mems_allowed_seq);
- }
+ local_irq_disable();
+ write_seqcount_begin(&tsk->mems_allowed_seq);
nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
mpol_rebind_task(tsk, newmems);
tsk->mems_allowed = *newmems;
- if (need_loop) {
- write_seqcount_end(&tsk->mems_allowed_seq);
- local_irq_enable();
- }
+ write_seqcount_end(&tsk->mems_allowed_seq);
+ local_irq_enable();
task_unlock(tsk);
}
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 0/6] cpuset/mempolicies related fixes and cleanups
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
Changes since RFC v1 [3]:
- Reworked patch 2 after discussion with Christoph Lameter.
- Fix bug in patch 5 spotted by Hillf Danton.
- Rebased to mmotm-2017-05-12-15-53
I would like to stress that this patchset aims to fix issues and cleanup the
code *within the existing documented semantics*, i.e. patch 1 ignores mempolicy
restrictions if the set of allowed nodes has no intersection with set of nodes
allowed by cpuset. I believe discussing potential changes of the semantics can
be better done once we have a baseline with no known bugs of the current
semantics.
===
I've recently summarized the cpuset/mempolicy issues in a LSF/MM proposal [1]
and the discussion itself [2]. I've been trying to rewrite the handling as
proposed, with the idea that changing semantics to make all mempolicies static
wrt cpuset updates (and discarding the relative and default modes) can be tried
on top, as there's a high risk of being rejected/reverted because somebody
might still care about the removed modes.
However I haven't yet figured out how to properly:
1) make mempolicies swappable instead of rebinding in place. I thought mbind()
already works that way and uses refcounting to avoid use-after-free of the old
policy by a parallel allocation, but turns out true refcounting is only done
for shared (shmem) mempolicies, and the actual protection for mbind() comes
from mmap_sem. Extending the refcounting means more overhead in allocator hot
path. Also swapping whole mempolicies means that we have to allocate the new
ones, which can fail, and reverting of the partially done work also means
allocating (note that mbind() doesn't care and will just leave part of the
range updated and part not updated when returning -ENOMEM...).
2) make cpuset's task->mems_allowed also swappable (after converting it from
nodemask to zonelist, which is the easy part) for mostly the same reasons.
The good news is that while trying to do the above, I've at least figured out
how to hopefully close the remaining premature OOM's, and do a buch of cleanups
on top, removing quite some of the code that was also supposed to prevent the
cpuset update races, but doesn't work anymore nowadays. This should fix the
most pressing concerns with this topic and give us a better baseline before
either proceeding with the original proposal, or pushing a change of semantics
that removes the problem 1) above. I'd be then fine with trying to change the
semantic first and rewrite later.
Patchset is based on next-20170411 and has been tested with the LTP cpuset01
stress test.
[1] https://lkml.kernel.org/r/4c44a589-5fd8-08d0-892c-e893bb525b71@suse.cz
[2] https://lwn.net/Articles/717797/
[3] https://marc.info/?l=linux-mm&m=149191957922828&w=2
Vlastimil Babka (6):
mm, page_alloc: fix more premature OOM due to race with cpuset update
mm, mempolicy: stop adjusting current->il_next in
mpol_rebind_nodemask()
mm, page_alloc: pass preferred nid instead of zonelist to allocator
mm, mempolicy: simplify rebinding mempolicies when updating cpusets
mm, cpuset: always use seqlock when changing task's nodemask
mm, mempolicy: don't check cpuset seqlock where it doesn't matter
include/linux/gfp.h | 11 ++-
include/linux/mempolicy.h | 12 ++-
include/linux/sched.h | 2 +-
include/uapi/linux/mempolicy.h | 8 --
kernel/cgroup/cpuset.c | 33 ++------
mm/hugetlb.c | 15 ++--
mm/memory_hotplug.c | 6 +-
mm/mempolicy.c | 181 ++++++++++-------------------------------
mm/page_alloc.c | 61 ++++++++++----
9 files changed, 118 insertions(+), 211 deletions(-)
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH v2 1/6] mm, page_alloc: fix more premature OOM due to race with cpuset update
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Commit e47483bca2cc ("mm, page_alloc: fix premature OOM when racing with cpuset
mems update") has fixed known recent regressions found by LTP's cpuset01
testcase. I have however found that by modifying the testcase to use per-vma
mempolicies via bind(2) instead of per-task mempolicies via set_mempolicy(2),
the premature OOM still happens and the issue is much older.
The root of the problem is that the cpuset's mems_allowed and mempolicy's
nodemask can temporarily have no intersection, thus get_page_from_freelist()
cannot find any usable zone. The current semantic for empty intersection is to
ignore mempolicy's nodemask and honour cpuset restrictions. This is checked in
node_zonelist(), but the racy update can happen after we already passed the
check. Such races should be protected by the seqlock task->mems_allowed_seq,
but it doesn't work here, because 1) mpol_rebind_mm() does not happen under
seqlock for write, and doing so would lead to deadlock, as it takes mmap_sem
for write, while the allocation can have mmap_sem for read when it's taking the
seqlock for read. And 2) the seqlock cookie of callers of node_zonelist()
(alloc_pages_vma() and alloc_pages_current()) is different than the one of
__alloc_pages_slowpath(), so there's still a potential race window.
This patch fixes the issue by having __alloc_pages_slowpath() check for empty
intersection of cpuset and ac->nodemask before OOM or allocation failure. If
it's indeed empty, the nodemask is ignored and allocation retried, which mimics
node_zonelist(). This works fine, because almost all callers of
__alloc_pages_nodemask are obtaining the nodemask via node_zonelist(). The only
exception is new_node_page() from hotplug, where the potential violation of
nodemask isn't an issue, as there's already a fallback allocation attempt
without any nodemask. If there's a future caller that needs to have its specific
nodemask honoured over task's cpuset restrictions, we'll have to e.g. add a gfp
flag for that.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/page_alloc.c | 51 ++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 38 insertions(+), 13 deletions(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index beb2827fd5de..43aa767c3188 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3661,6 +3661,39 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
return false;
}
+static inline bool
+check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac)
+{
+ /*
+ * It's possible that cpuset's mems_allowed and the nodemask from
+ * mempolicy don't intersect. This should be normally dealt with by
+ * policy_nodemask(), but it's possible to race with cpuset update in
+ * such a way the check therein was true, and then it became false
+ * before we got our cpuset_mems_cookie here.
+ * This assumes that for all allocations, ac->nodemask can come only
+ * from MPOL_BIND mempolicy (whose documented semantics is to be ignored
+ * when it does not intersect with the cpuset restrictions) or the
+ * caller can deal with a violated nodemask.
+ */
+ if (cpusets_enabled() && ac->nodemask &&
+ !cpuset_nodemask_valid_mems_allowed(ac->nodemask)) {
+ ac->nodemask = NULL;
+ return true;
+ }
+
+ /*
+ * When updating a task's mems_allowed or mempolicy nodemask, it is
+ * possible to race with parallel threads in such a way that our
+ * allocation can fail while the mask is being updated. If we are about
+ * to fail, check if the cpuset changed during allocation and if so,
+ * retry.
+ */
+ if (read_mems_allowed_retry(cpuset_mems_cookie))
+ return true;
+
+ return false;
+}
+
static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
struct alloc_context *ac)
@@ -3856,11 +3889,9 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
&compaction_retries))
goto retry;
- /*
- * It's possible we raced with cpuset update so the OOM would be
- * premature (see below the nopage: label for full explanation).
- */
- if (read_mems_allowed_retry(cpuset_mems_cookie))
+
+ /* Deal with possible cpuset update races before we start OOM killing */
+ if (check_retry_cpuset(cpuset_mems_cookie, ac))
goto retry_cpuset;
/* Reclaim has failed us, start killing things */
@@ -3879,14 +3910,8 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
}
nopage:
- /*
- * When updating a task's mems_allowed or mempolicy nodemask, it is
- * possible to race with parallel threads in such a way that our
- * allocation can fail while the mask is being updated. If we are about
- * to fail, check if the cpuset changed during allocation and if so,
- * retry.
- */
- if (read_mems_allowed_retry(cpuset_mems_cookie))
+ /* Deal with possible cpuset update races before we fail */
+ if (check_retry_cpuset(cpuset_mems_cookie, ac))
goto retry_cpuset;
/*
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 6/6] mm, mempolicy: don't check cpuset seqlock where it doesn't matter
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Two wrappers of __alloc_pages_nodemask() are checking task->mems_allowed_seq
themselves to retry allocation that has raced with a cpuset update. This has
been shown to be ineffective in preventing premature OOM's which can happen in
__alloc_pages_slowpath() long before it returns back to the wrappers to detect
the race at that level. Previous patches have made __alloc_pages_slowpath()
more robust, so we can now simply remove the seqlock checking in the wrappers
to prevent further wrong impression that it can actually help.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/mempolicy.c | 16 ----------------
1 file changed, 16 deletions(-)
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 047181452040..7d8e56214ac0 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1898,12 +1898,9 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
struct mempolicy *pol;
struct page *page;
int preferred_nid;
- unsigned int cpuset_mems_cookie;
nodemask_t *nmask;
-retry_cpuset:
pol = get_vma_policy(vma, addr);
- cpuset_mems_cookie = read_mems_allowed_begin();
if (pol->mode == MPOL_INTERLEAVE) {
unsigned nid;
@@ -1945,8 +1942,6 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
page = __alloc_pages_nodemask(gfp, order, preferred_nid, nmask);
mpol_cond_put(pol);
out:
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
- goto retry_cpuset;
return page;
}
@@ -1964,23 +1959,15 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,
* Allocate a page from the kernel page pool. When not in
* interrupt context and apply the current process NUMA policy.
* Returns NULL when no page can be allocated.
- *
- * Don't call cpuset_update_task_memory_state() unless
- * 1) it's ok to take cpuset_sem (can WAIT), and
- * 2) allocating for current task (not interrupt).
*/
struct page *alloc_pages_current(gfp_t gfp, unsigned order)
{
struct mempolicy *pol = &default_policy;
struct page *page;
- unsigned int cpuset_mems_cookie;
if (!in_interrupt() && !(gfp & __GFP_THISNODE))
pol = get_task_policy(current);
-retry_cpuset:
- cpuset_mems_cookie = read_mems_allowed_begin();
-
/*
* No reference counting needed for current->mempolicy
* nor system default_policy
@@ -1992,9 +1979,6 @@ struct page *alloc_pages_current(gfp_t gfp, unsigned order)
policy_node(gfp, pol, numa_node_id()),
policy_nodemask(gfp, pol));
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
- goto retry_cpuset;
-
return page;
}
EXPORT_SYMBOL(alloc_pages_current);
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 2/6] mm, mempolicy: stop adjusting current->il_next in mpol_rebind_nodemask()
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
The task->il_next variable stores the next allocation node id for task's
MPOL_INTERLEAVE policy. mpol_rebind_nodemask() updates interleave and
bind mempolicies due to changing cpuset mems. Currently it also tries to
make sure that current->il_next is valid within the updated nodemask. This is
bogus, because 1) we are updating potentially any task's mempolicy, not just
current, and 2) we might be updating a per-vma mempolicy, not task one.
The interleave_nodes() function that uses il_next can cope fine with the value
not being within the currently allowed nodes, so this hasn't manifested as an
actual issue.
We can remove the need for updating il_next completely by changing it to
il_prev and store the node id of the previous interleave allocation instead of
the next id. Then interleave_nodes() can calculate the next id using the
current nodemask and also store it as il_prev, except when querying the next
node via do_get_mempolicy().
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/sched.h | 2 +-
mm/mempolicy.c | 22 +++++++---------------
2 files changed, 8 insertions(+), 16 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 6a97386c785e..b72bbfec01f9 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -884,7 +884,7 @@ struct task_struct {
#ifdef CONFIG_NUMA
/* Protected by alloc_lock: */
struct mempolicy *mempolicy;
- short il_next;
+ short il_prev;
short pref_node_fork;
#endif
#ifdef CONFIG_NUMA_BALANCING
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 37d0b334bfe9..d77177c7283b 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -349,12 +349,6 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
pol->v.nodes = tmp;
else
BUG();
-
- if (!node_isset(current->il_next, tmp)) {
- current->il_next = next_node_in(current->il_next, tmp);
- if (current->il_next >= MAX_NUMNODES)
- current->il_next = numa_node_id();
- }
}
static void mpol_rebind_preferred(struct mempolicy *pol,
@@ -812,9 +806,8 @@ static long do_set_mempolicy(unsigned short mode, unsigned short flags,
}
old = current->mempolicy;
current->mempolicy = new;
- if (new && new->mode == MPOL_INTERLEAVE &&
- nodes_weight(new->v.nodes))
- current->il_next = first_node(new->v.nodes);
+ if (new && new->mode == MPOL_INTERLEAVE)
+ current->il_prev = MAX_NUMNODES-1;
task_unlock(current);
mpol_put(old);
ret = 0;
@@ -916,7 +909,7 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask,
*policy = err;
} else if (pol == current->mempolicy &&
pol->mode == MPOL_INTERLEAVE) {
- *policy = current->il_next;
+ *policy = next_node_in(current->il_prev, pol->v.nodes);
} else {
err = -EINVAL;
goto out;
@@ -1697,14 +1690,13 @@ static struct zonelist *policy_zonelist(gfp_t gfp, struct mempolicy *policy,
/* Do dynamic interleaving for a process */
static unsigned interleave_nodes(struct mempolicy *policy)
{
- unsigned nid, next;
+ unsigned next;
struct task_struct *me = current;
- nid = me->il_next;
- next = next_node_in(nid, policy->v.nodes);
+ next = next_node_in(me->il_prev, policy->v.nodes);
if (next < MAX_NUMNODES)
- me->il_next = next;
- return nid;
+ me->il_prev = next;
+ return next;
}
/*
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 4/6] mm, mempolicy: simplify rebinding mempolicies when updating cpusets
From: Vlastimil Babka @ 2017-05-17 8:11 UTC (permalink / raw)
To: Andrew Morton
Cc: linux-mm, linux-api, linux-kernel, cgroups, Li Zefan,
Michal Hocko, Mel Gorman, David Rientjes, Christoph Lameter,
Hugh Dickins, Andrea Arcangeli, Anshuman Khandual,
Kirill A. Shutemov, Vlastimil Babka
In-Reply-To: <20170517081140.30654-1-vbabka@suse.cz>
Commit c0ff7453bb5c ("cpuset,mm: fix no node to alloc memory when changing
cpuset's mems") has introduced a two-step protocol when rebinding task's
mempolicy due to cpuset update, in order to avoid a parallel allocation seeing
an empty effective nodemask and failing. Later, commit cc9a6c877661 ("cpuset:
mm: reduce large amounts of memory barrier related damage v3") introduced
a seqlock protection and removed the synchronization point between the two
update steps. At that point (or perhaps later), the two-step rebinding became
unnecessary. Currently it only makes sure that the update first adds new nodes
in step 1 and then removes nodes in step 2. Without memory barriers the effects
are questionable, and even then this cannot prevent a parallel zonelist
iteration checking the nodemask at each step to observe all nodes as unusable
for allocation. We now fully rely on the seqlock to prevent premature OOMs and
allocation failures.
We can thus remove the two-step update parts and simplify the code.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/mempolicy.h | 6 +--
include/uapi/linux/mempolicy.h | 8 ----
kernel/cgroup/cpuset.c | 4 +-
mm/mempolicy.c | 102 ++++++++---------------------------------
4 files changed, 21 insertions(+), 99 deletions(-)
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index ecb6cbeede5a..3a58b4be1b0c 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -142,8 +142,7 @@ bool vma_policy_mof(struct vm_area_struct *vma);
extern void numa_default_policy(void);
extern void numa_policy_init(void);
-extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
- enum mpol_rebind_step step);
+extern void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new);
extern void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new);
extern int huge_node(struct vm_area_struct *vma,
@@ -260,8 +259,7 @@ static inline void numa_default_policy(void)
}
static inline void mpol_rebind_task(struct task_struct *tsk,
- const nodemask_t *new,
- enum mpol_rebind_step step)
+ const nodemask_t *new)
{
}
diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index 9cd8b21dddbe..2a4d89508fec 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -24,13 +24,6 @@ enum {
MPOL_MAX, /* always last member of enum */
};
-enum mpol_rebind_step {
- MPOL_REBIND_ONCE, /* do rebind work at once(not by two step) */
- MPOL_REBIND_STEP1, /* first step(set all the newly nodes) */
- MPOL_REBIND_STEP2, /* second step(clean all the disallowed nodes)*/
- MPOL_REBIND_NSTEP,
-};
-
/* Flags for set_mempolicy */
#define MPOL_F_STATIC_NODES (1 << 15)
#define MPOL_F_RELATIVE_NODES (1 << 14)
@@ -65,7 +58,6 @@ enum mpol_rebind_step {
*/
#define MPOL_F_SHARED (1 << 0) /* identify shared policies */
#define MPOL_F_LOCAL (1 << 1) /* preferred local allocation */
-#define MPOL_F_REBINDING (1 << 2) /* identify policies in rebinding */
#define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */
#define MPOL_F_MORON (1 << 4) /* Migrate On protnone Reference On Node */
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 0f41292be0fb..dfd5b420452d 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1063,9 +1063,7 @@ static void cpuset_change_task_nodemask(struct task_struct *tsk,
}
nodes_or(tsk->mems_allowed, tsk->mems_allowed, *newmems);
- mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP1);
-
- mpol_rebind_task(tsk, newmems, MPOL_REBIND_STEP2);
+ mpol_rebind_task(tsk, newmems);
tsk->mems_allowed = *newmems;
if (need_loop) {
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index c60807625fd5..047181452040 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -146,22 +146,7 @@ struct mempolicy *get_task_policy(struct task_struct *p)
static const struct mempolicy_operations {
int (*create)(struct mempolicy *pol, const nodemask_t *nodes);
- /*
- * If read-side task has no lock to protect task->mempolicy, write-side
- * task will rebind the task->mempolicy by two step. The first step is
- * setting all the newly nodes, and the second step is cleaning all the
- * disallowed nodes. In this way, we can avoid finding no node to alloc
- * page.
- * If we have a lock to protect task->mempolicy in read-side, we do
- * rebind directly.
- *
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
- */
- void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step);
+ void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes);
} mpol_ops[MPOL_MAX];
static inline int mpol_store_user_nodemask(const struct mempolicy *pol)
@@ -304,19 +289,11 @@ void __mpol_put(struct mempolicy *p)
kmem_cache_free(policy_cache, p);
}
-static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step)
+static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
{
}
-/*
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
- */
-static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
- enum mpol_rebind_step step)
+static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
{
nodemask_t tmp;
@@ -325,35 +302,19 @@ static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes,
else if (pol->flags & MPOL_F_RELATIVE_NODES)
mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes);
else {
- /*
- * if step == 1, we use ->w.cpuset_mems_allowed to cache the
- * result
- */
- if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP1) {
- nodes_remap(tmp, pol->v.nodes,
- pol->w.cpuset_mems_allowed, *nodes);
- pol->w.cpuset_mems_allowed = step ? tmp : *nodes;
- } else if (step == MPOL_REBIND_STEP2) {
- tmp = pol->w.cpuset_mems_allowed;
- pol->w.cpuset_mems_allowed = *nodes;
- } else
- BUG();
+ nodes_remap(tmp, pol->v.nodes,pol->w.cpuset_mems_allowed,
+ *nodes);
+ pol->w.cpuset_mems_allowed = tmp;
}
if (nodes_empty(tmp))
tmp = *nodes;
- if (step == MPOL_REBIND_STEP1)
- nodes_or(pol->v.nodes, pol->v.nodes, tmp);
- else if (step == MPOL_REBIND_ONCE || step == MPOL_REBIND_STEP2)
- pol->v.nodes = tmp;
- else
- BUG();
+ pol->v.nodes = tmp;
}
static void mpol_rebind_preferred(struct mempolicy *pol,
- const nodemask_t *nodes,
- enum mpol_rebind_step step)
+ const nodemask_t *nodes)
{
nodemask_t tmp;
@@ -379,42 +340,19 @@ static void mpol_rebind_preferred(struct mempolicy *pol,
/*
* mpol_rebind_policy - Migrate a policy to a different set of nodes
*
- * If read-side task has no lock to protect task->mempolicy, write-side
- * task will rebind the task->mempolicy by two step. The first step is
- * setting all the newly nodes, and the second step is cleaning all the
- * disallowed nodes. In this way, we can avoid finding no node to alloc
- * page.
- * If we have a lock to protect task->mempolicy in read-side, we do
- * rebind directly.
- *
- * step:
- * MPOL_REBIND_ONCE - do rebind work at once
- * MPOL_REBIND_STEP1 - set all the newly nodes
- * MPOL_REBIND_STEP2 - clean all the disallowed nodes
+ * Per-vma policies are protected by mmap_sem. Allocations using per-task
+ * policies are protected by task->mems_allowed_seq to prevent a premature
+ * OOM/allocation failure due to parallel nodemask modification.
*/
-static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
- enum mpol_rebind_step step)
+static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask)
{
if (!pol)
return;
- if (!mpol_store_user_nodemask(pol) && step == MPOL_REBIND_ONCE &&
+ if (!mpol_store_user_nodemask(pol) &&
nodes_equal(pol->w.cpuset_mems_allowed, *newmask))
return;
- if (step == MPOL_REBIND_STEP1 && (pol->flags & MPOL_F_REBINDING))
- return;
-
- if (step == MPOL_REBIND_STEP2 && !(pol->flags & MPOL_F_REBINDING))
- BUG();
-
- if (step == MPOL_REBIND_STEP1)
- pol->flags |= MPOL_F_REBINDING;
- else if (step == MPOL_REBIND_STEP2)
- pol->flags &= ~MPOL_F_REBINDING;
- else if (step >= MPOL_REBIND_NSTEP)
- BUG();
-
- mpol_ops[pol->mode].rebind(pol, newmask, step);
+ mpol_ops[pol->mode].rebind(pol, newmask);
}
/*
@@ -424,10 +362,9 @@ static void mpol_rebind_policy(struct mempolicy *pol, const nodemask_t *newmask,
* Called with task's alloc_lock held.
*/
-void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new,
- enum mpol_rebind_step step)
+void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new)
{
- mpol_rebind_policy(tsk->mempolicy, new, step);
+ mpol_rebind_policy(tsk->mempolicy, new);
}
/*
@@ -442,7 +379,7 @@ void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new)
down_write(&mm->mmap_sem);
for (vma = mm->mmap; vma; vma = vma->vm_next)
- mpol_rebind_policy(vma->vm_policy, new, MPOL_REBIND_ONCE);
+ mpol_rebind_policy(vma->vm_policy, new);
up_write(&mm->mmap_sem);
}
@@ -2101,10 +2038,7 @@ struct mempolicy *__mpol_dup(struct mempolicy *old)
if (current_cpuset_is_being_rebound()) {
nodemask_t mems = cpuset_mems_allowed(current);
- if (new->flags & MPOL_F_REBINDING)
- mpol_rebind_policy(new, &mems, MPOL_REBIND_STEP2);
- else
- mpol_rebind_policy(new, &mems, MPOL_REBIND_ONCE);
+ mpol_rebind_policy(new, &mems);
}
atomic_set(&new->refcnt, 1);
return new;
--
2.12.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox