* Re: [v3 0/9] parallelized "struct page" zeroing
From: David Miller @ 2017-05-12 16:57 UTC (permalink / raw)
To: pasha.tatashin
Cc: mhocko, linux-kernel, sparclinux, linux-mm, linuxppc-dev,
linux-s390, borntraeger, heiko.carstens
In-Reply-To: <65b8a658-76d1-0617-ece8-ff7a3c1c4046@oracle.com>
From: Pasha Tatashin <pasha.tatashin@oracle.com>
Date: Thu, 11 May 2017 16:59:33 -0400
> We should either keep memset() only for deferred struct pages as what
> I have in my patches.
>
> Another option is to add a new function struct_page_clear() which
> would default to memset() and to something else on platforms that
> decide to optimize it.
>
> On SPARC it would call STBIs, and we would do one membar call after
> all "struct pages" are initialized.
No membars will be performed for single individual page struct clear,
the cutoff to use the STBI is larger than that.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [v3 0/9] parallelized "struct page" zeroing
From: David Miller @ 2017-05-12 16:56 UTC (permalink / raw)
To: pasha.tatashin
Cc: mhocko, linux-kernel, sparclinux, linux-mm, linuxppc-dev,
linux-s390, borntraeger, heiko.carstens
In-Reply-To: <9088ad7e-8b3b-8eba-2fdf-7b0e36e4582e@oracle.com>
From: Pasha Tatashin <pasha.tatashin@oracle.com>
Date: Thu, 11 May 2017 16:47:05 -0400
> So, moving memset() into __init_single_page() benefits Intel. I am
> actually surprised why memset() is so slow on intel when it is called
> from memblock. But, hurts SPARC, I guess these membars at the end of
> memset() kills the performance.
Perhaps an x86 expert can chime in, but it might be the case that past
a certain size, the microcode for the enhanced stosb uses non-temporal
stores or something like that.
As for sparc64, yes we can get really killed by the transactional cost
of memset because of the membars.
But I wonder, for a single page struct, if we even use the special
stores and thus eat the membar cost. struct page is only 64 bytes,
and the cutoff in the Niagara4 bzero implementation is "64 + (64 - 8)"
so indeed the initializing stores will not even be used.
So sparc64 will only use initializing stores and do the membars if
at least 2 pages are cleared at a time.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 2/2] mm: swap: move anonymous THP split logic to vmscan
From: Johannes Weiner @ 2017-05-12 16:48 UTC (permalink / raw)
To: Minchan Kim; +Cc: Huang Ying, Andrew Morton, linux-kernel, linux-mm
In-Reply-To: <1494555684-11982-2-git-send-email-minchan@kernel.org>
On Fri, May 12, 2017 at 11:21:24AM +0900, Minchan Kim wrote:
> The add_to_swap aims to allocate swap_space(ie, swap slot and
> swapcache) so if it fails due to lack of space in case of THP
> or something(hdd swap but tries THP swapout) *caller* rather
> than add_to_swap itself should split the THP page and retry it
> with base page which is more natural.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/2] mm: swap: unify swap slot free functions to put_swap_page
From: Johannes Weiner @ 2017-05-12 16:47 UTC (permalink / raw)
To: Minchan Kim; +Cc: Huang Ying, Andrew Morton, linux-kernel, linux-mm
In-Reply-To: <1494555684-11982-1-git-send-email-minchan@kernel.org>
On Fri, May 12, 2017 at 11:21:23AM +0900, Minchan Kim wrote:
> Now, get_swap_page takes struct page and allocates swap space
> according to page size(ie, normal or THP) so it would be more
> cleaner to introduce put_swap_page which is a counter function
> of get_swap_page. Then, it calls right swap slot free function
> depending on page's size.
>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm: per-cgroup memory reclaim stats
From: Johannes Weiner @ 2017-05-12 16:42 UTC (permalink / raw)
To: Balbir Singh
Cc: Roman Gushchin, Tejun Heo, Li Zefan, Michal Hocko,
Vladimir Davydov, cgroups, linux-doc, linux-kernel, linux-mm
In-Reply-To: <1494555922.21563.1.camel@gmail.com>
On Fri, May 12, 2017 at 12:25:22PM +1000, Balbir Singh wrote:
> On Thu, 2017-05-11 at 20:16 +0100, Roman Gushchin wrote:
> > The meaning of each value is the same as for global counters,
> > available using /proc/vmstat.
> >
> > Also, for consistency, rename mem_cgroup_count_vm_event() to
> > count_memcg_event_mm().
> >
>
> I still prefer the mem_cgroup_count_vm_event() name, or memcg_count_vm_event(),
> the namespace upfront makes it easier to parse where to look for the the
> implementation and also grep. In any case the rename should be independent
> patch, but I don't like the name you've proposed.
The memory controller is no longer a tacked-on feature to the VM - the
entire reclaim path is designed around cgroups at this point. The
namespacing is just cumbersome and doesn't add add any value, IMO.
This name is also more consistent with the stats interface, where we
use nodes, zones, memcgs all equally to describe scopes/containers:
inc_node_state(), inc_zone_state(), inc_memcg_state()
> > @@ -357,6 +357,17 @@ static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg)
> > }
> > struct mem_cgroup *mem_cgroup_from_id(unsigned short id);
> >
> > +static inline struct mem_cgroup *lruvec_memcg(struct lruvec *lruvec)
>
> mem_cgroup_from_lruvec()?
This name is consistent with other lruvec accessors such as
lruvec_pgdat() and lruvec_lru_size() etc.
> > @@ -1741,11 +1748,16 @@ shrink_inactive_list(unsigned long nr_to_scan, struct lruvec *lruvec,
> >
> > spin_lock_irq(&pgdat->lru_lock);
> >
> > - if (global_reclaim(sc)) {
> > - if (current_is_kswapd())
> > + if (current_is_kswapd()) {
> > + if (global_reclaim(sc))
> > __count_vm_events(PGSTEAL_KSWAPD, nr_reclaimed);
> > - else
> > + count_memcg_events(lruvec_memcg(lruvec), PGSTEAL_KSWAPD,
> > + nr_reclaimed);
>
> Has the else gone missing? What happens if it's global_reclaim(), do
> we still account the count in memcg?
>
> > + } else {
> > + if (global_reclaim(sc))
> > __count_vm_events(PGSTEAL_DIRECT, nr_reclaimed);
> > + count_memcg_events(lruvec_memcg(lruvec), PGSTEAL_DIRECT,
> > + nr_reclaimed);
>
> It sounds like memcg accumlates both global and memcg reclaim driver
> counts -- is this what we want?
Yes.
Consider a fully containerized system that is using only memory.low
and thus exclusively global reclaim to enforce the partitioning, NOT
artificial limits and limit reclaim. In this case, we still want to
know how much reclaim activity each group is experiencing.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Marcelo Tosatti @ 2017-05-12 16:19 UTC (permalink / raw)
To: Christoph Lameter
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <alpine.DEB.2.20.1705121103120.22831@east.gentwo.org>
On Fri, May 12, 2017 at 11:07:48AM -0500, Christoph Lameter wrote:
> On Fri, 12 May 2017, Marcelo Tosatti wrote:
>
> > In our case, vmstat updates are very rare (CPU is dominated by DPDK).
>
> What is the OS doing on the cores that DPDK runs on? I mean we here can
> clean a processor of all activities and are able to run for a long time
> without any interruptions.
>
> Why would you still let the OS do things on that processor? If activities
> by the OS are required then the existing NOHZ setup already minimizes
> latency to a short burst (and Chris Metcalf's work improves on that).
>
>
> What exactly is the issue you are seeing and want to address? I think we
> have similar aims and as far as I know the current situation is already
> good enough for what you may need. You may just not be aware of how to
> configure this.
I want to disable vmstat worker thread completly from an isolated CPU.
Because it adds overhead to a latency target, target which
the lower the better.
> I doubt that doing inline updates will do much good compared to what we
> already have and what the dataplan mode can do.
Can the dataplan mode disable vmstat worker thread completly on a given
CPU?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Christoph Lameter @ 2017-05-12 16:07 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <20170512154026.GA3556@amt.cnet>
On Fri, 12 May 2017, Marcelo Tosatti wrote:
> In our case, vmstat updates are very rare (CPU is dominated by DPDK).
What is the OS doing on the cores that DPDK runs on? I mean we here can
clean a processor of all activities and are able to run for a long time
without any interruptions.
Why would you still let the OS do things on that processor? If activities
by the OS are required then the existing NOHZ setup already minimizes
latency to a short burst (and Chris Metcalf's work improves on that).
What exactly is the issue you are seeing and want to address? I think we
have similar aims and as far as I know the current situation is already
good enough for what you may need. You may just not be aware of how to
configure this.
I doubt that doing inline updates will do much good compared to what we
already have and what the dataplan mode can do.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Christoph Lameter @ 2017-05-12 16:03 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <20170512154026.GA3556@amt.cnet>
On Fri, 12 May 2017, Marcelo Tosatti wrote:
> OK, i'll check if the patches from Chris work for us and then add
> Tested-by on that.
You may not need those if the quiet_vmstat() function is enough for you.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Marcelo Tosatti @ 2017-05-12 15:40 UTC (permalink / raw)
To: Christoph Lameter
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <alpine.DEB.2.20.1705121002310.22243@east.gentwo.org>
On Fri, May 12, 2017 at 10:11:14AM -0500, Christoph Lameter wrote:
> On Fri, 12 May 2017, Marcelo Tosatti wrote:
>
> > > A bit confused by this one. The vmstat worker is already disabled if there
> > > are no updates. Also the patches by Chris Metcalf on data plane mode add a
> > > prctl to quiet the vmstat workers.
> > >
> > > Why do we need more than this?
> >
> > If there are vmstat statistic updates on a given CPU, and you don't
> > want intervention from the vmstat worker, you change the behaviour of
> > stat data collection to directly write to the global structures (which
> > disables the performance optimization of collecting data in per-cpu
> > counters).
>
> Hmmm.... Ok. That is going to be expensive if you do this for each
> individual vmstat update.
In our case, vmstat updates are very rare (CPU is dominated by DPDK).
> > This way you can disable vmstat worker (because it causes undesired
> > latencies), while allowing vmstatistics to function properly.
>
> Best then to run the vmstat update mechanism when you leave kernel mode to
> get all the updates in one go.
Again, vmstat updates are very rare (CPU is dominated by DPDK).
> > The prctl from Chris Metcalf patchset allows one to disable vmstat
> > worker per CPU? If so, they replace the functionality of the patch
> > "[patch 3/3] MM: allow per-cpu vmstat_worker configuration"
> > of the -v2 series of my patchset, and we can use it instead.
> >
> > Is it integrated already?
>
> The data plane mode patches disables vmstat processing by updating the
> vmstats immediately if necessary and switching off the kworker thread.
OK this is what my patch set is doing.
> So the kworker wont be running until the next time statistics are checked
> by the shepherd task from a remote cpu.
We don't want kworker thread to ever run.
> If the counters have been updated
> then the shepherd task will reenable the kworker. This is already merged
> and has been working for a long time. Data plan mode has not been merged
> yet but the infrastructure in vmstat.c is there because NOHZ needs it too.
OK.
>
> See linux/vmstat.c:quiet_vmstat()
>
> It would be easy to add a /proc file that allows the quieting of the
> vmstat workers for a certain cpu. Just make it call the quiet_vmstat() on
> the right cpu.
>
> This will quiet vmstat down. The shepherd task will check the stats in 2
> second intervals and will then reenable when necessasry.
>
> Note that we already are updating the global structures directly if the
> differential gets too high. Reducing the differential may get you what you
> want.
Yes, we reduce the differential to 1 (== direct updates to global
structures).
OK, i'll check if the patches from Chris work for us and then add
Tested-by on that.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Christoph Lameter @ 2017-05-12 15:11 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <20170512122704.GA30528@amt.cnet>
On Fri, 12 May 2017, Marcelo Tosatti wrote:
> > A bit confused by this one. The vmstat worker is already disabled if there
> > are no updates. Also the patches by Chris Metcalf on data plane mode add a
> > prctl to quiet the vmstat workers.
> >
> > Why do we need more than this?
>
> If there are vmstat statistic updates on a given CPU, and you don't
> want intervention from the vmstat worker, you change the behaviour of
> stat data collection to directly write to the global structures (which
> disables the performance optimization of collecting data in per-cpu
> counters).
Hmmm.... Ok. That is going to be expensive if you do this for each
individual vmstat update.
> This way you can disable vmstat worker (because it causes undesired
> latencies), while allowing vmstatistics to function properly.
Best then to run the vmstat update mechanism when you leave kernel mode to
get all the updates in one go.
> The prctl from Chris Metcalf patchset allows one to disable vmstat
> worker per CPU? If so, they replace the functionality of the patch
> "[patch 3/3] MM: allow per-cpu vmstat_worker configuration"
> of the -v2 series of my patchset, and we can use it instead.
>
> Is it integrated already?
The data plane mode patches disables vmstat processing by updating the
vmstats immediately if necessary and switching off the kworker thread.
So the kworker wont be running until the next time statistics are checked
by the shepherd task from a remote cpu. If the counters have been updated
then the shepherd task will reenable the kworker. This is already merged
and has been working for a long time. Data plan mode has not been merged
yet but the infrastructure in vmstat.c is there because NOHZ needs it too.
See linux/vmstat.c:quiet_vmstat()
It would be easy to add a /proc file that allows the quieting of the
vmstat workers for a certain cpu. Just make it call the quiet_vmstat() on
the right cpu.
This will quiet vmstat down. The shepherd task will check the stats in 2
second intervals and will then reenable when necessasry.
Note that we already are updating the global structures directly if the
differential gets too high. Reducing the differential may get you what you
want.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: 8 Gigabytes and constantly swapping
From: Arthur Marsh @ 2017-05-12 13:39 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm
In-Reply-To: <64c96dd6-651c-eee8-2a30-65e60988d7d8@I-love.SAKURA.ne.jp>
Tetsuo Handa wrote on 12/05/17 20:26:
> On 2017/05/12 17:51, Arthur Marsh wrote:
>> I've been building the Linus git head kernels as the source gets updated and
>> the one built about 3 hours ago managed to get stuck with kswapd0 as the highest
>> consumer of CPU cycles (but still under 1 percent) of processes listed by top for
>> over 15 minutes, after which I hit the power switch and rebooted with a Debian
>> 4.11.0 kernel.
>
> Did the /bin/top process continue showing up-to-dated statistics rather than
> refrain from showing up-to-dated statistics? (I wonder why you had to hit
> the power switch before trying SysRq-m/SysRq-f etc.)
Yes, the /bin/top process continued updating.
The load average reached a high figure (over 14).
kswapd0 was using more CPU cycles than anything else but was still under
1 percent.
There was still a lot of swap space free (about 3 GiB).
buffer/cache dropped below 1 GiB out of 8 GiB RAM.
I would have used alt-sysrq-f but had only recently enabled it in my
kernel builds and didn't think to look it up on my other pc.
>
> If yes, assuming that reading statistics involves memory allocation requests,
> there was no load at at all despite firefox and chromium were running?
Wait time was over 98 percent, load average over 14.
>
> If no, all allocation requests got stuck waiting for memory reclaim?
...
> More description of what happened and how you confirmed that
> the /bin/top process continued working would be helpful.
>
>
It was as if kswapd0 was just left waiting to swap pages in and out
without any other processes getting to complete what they were trying to do.
It does seem to be related to chromium starting up several processes
when opening extra windows/tabs.
Thanks for your help!
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Marcelo Tosatti @ 2017-05-12 12:27 UTC (permalink / raw)
To: Christoph Lameter
Cc: Luiz Capitulino, linux-kernel, linux-mm, Rik van Riel,
Linux RT Users, cmetcalf
In-Reply-To: <alpine.DEB.2.20.1705111035560.2894@east.gentwo.org>
On Thu, May 11, 2017 at 10:37:07AM -0500, Christoph Lameter wrote:
> On Tue, 2 May 2017, Luiz Capitulino wrote:
>
> > Ah, OK. Got this now. I'll give this patch a try. But I think we want
> > to hear from Christoph (who worked on reducing the vmstat interruptions
> > in the past).
>
> A bit confused by this one. The vmstat worker is already disabled if there
> are no updates. Also the patches by Chris Metcalf on data plane mode add a
> prctl to quiet the vmstat workers.
>
> Why do we need more than this?
If there are vmstat statistic updates on a given CPU, and you don't
want intervention from the vmstat worker, you change the behaviour of
stat data collection to directly write to the global structures (which
disables the performance optimization of collecting data in per-cpu
counters).
This way you can disable vmstat worker (because it causes undesired
latencies), while allowing vmstatistics to function properly.
The prctl from Chris Metcalf patchset allows one to disable vmstat
worker per CPU? If so, they replace the functionality of the patch
"[patch 3/3] MM: allow per-cpu vmstat_worker configuration"
of the -v2 series of my patchset, and we can use it instead.
Is it integrated already?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: mm/kasan: zero_p4d_populate() problem?
From: Kirill A. Shutemov @ 2017-05-12 11:04 UTC (permalink / raw)
To: Joonsoo Kim; +Cc: Andrey Ryabinin, linux-mm
In-Reply-To: <20170512055320.GA16929@js1304-desktop>
On Fri, May 12, 2017 at 02:53:22PM +0900, Joonsoo Kim wrote:
> Hello, Kirill.
>
> I found that zero_p4d_populate() in mm/kasan/kasan_init.c of
> next-20170511 doesn't get the benefit of the kasan_zero_pud.
> Do we need to fix it by adding
> "pud_populate(&init_mm, pud, lm_alias(kasan_zero_pud));" when
> alignment requirement is met?
It's not a fix, but optimization. But it makes sense to implement this.
Feel free to send a patch.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: 8 Gigabytes and constantly swapping
From: Tetsuo Handa @ 2017-05-12 10:56 UTC (permalink / raw)
To: Arthur Marsh; +Cc: linux-mm
In-Reply-To: <171e8fa1-3f14-dc18-09b5-48399b250a30@internode.on.net>
On 2017/05/12 17:51, Arthur Marsh wrote:
> I've been building the Linus git head kernels as the source gets updated and
> the one built about 3 hours ago managed to get stuck with kswapd0 as the highest
> consumer of CPU cycles (but still under 1 percent) of processes listed by top for
> over 15 minutes, after which I hit the power switch and rebooted with a Debian
> 4.11.0 kernel.
Did the /bin/top process continue showing up-to-dated statistics rather than
refrain from showing up-to-dated statistics? (I wonder why you had to hit
the power switch before trying SysRq-m/SysRq-f etc.)
If yes, assuming that reading statistics involves memory allocation requests,
there was no load at at all despite firefox and chromium were running?
If no, all allocation requests got stuck waiting for memory reclaim?
>
> The previous kernel built less than 24 hours earlier did not have this problem.
>
> CPU is an Athlon64 (Athlon II X4, 4 cores), RAM is 8GiB, swap is 4GiB, load was
> mainly firefox and chromium. Opening a new window in chromium seemed to help
> trigger the problem.
>
> It's not much information to go on, just wondered if anyone else had experienced
> similar issues?
>
> I'm happy to supply more configuration information and run tests including with
> kernels built with test patches applied.
I don't know but http://lkml.kernel.org/r/20170316100409.GR802@shells.gnugeneration.com
or http://lkml.kernel.org/r/20170502041235.zqmywvj5tiiom3jk@merlins.org ?
More description of what happened and how you confirmed that
the /bin/top process continued working would be helpful.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC summary] Enable Coherent Device Memory
From: Mel Gorman @ 2017-05-12 10:26 UTC (permalink / raw)
To: Balbir Singh
Cc: linux-mm, akpm, khandual, aneesh.kumar, paulmck, srikar, haren,
jglisse, arbab, vbabka, Christoph Lameter, Rik van Riel,
Benjamin Herrenschmidt
In-Reply-To: <1494569882.21563.8.camel@gmail.com>
On Fri, May 12, 2017 at 04:18:02PM +1000, Balbir Singh wrote:
> Why do we need to isolate memory?
> - CDM memory is not meant for normal usage, applications can request for it
> explictly. Oflload their compute to the device where the memory is
> (the offload is via a user space API like CUDA/openCL/...)
It still remains unanswered to a large extent why this cannot be
isolated after the fact via a standard mechanism. It may be easier if
the onlining of CDM memory can be deferred at boot until userspace
helpers can trigger the onlining and isolation.
> How do we isolate the memory - NUMA or HMM-CDM?
> - Since the memory is coherent, NUMA provides the mechanism to isolate to
> a large extent via mempolicy. With NUMA we also get autonuma/kswapd/etc
> running.
This has come up before with respect to autonuma and there appears to be
confusion. autonuma doesn't run on nodes as such. The page table hinting
happens in per-task context but should skip VMAs that are controlled by
a policy. While some care is needed from the application, it's managable
and would perform better than special casing the marking of pages placed
on a CDM-controlled node.
As for kswapd, there isn't a user-controllable method for controlling
this. However, if a device onlining the memory set the watermarks to 0,
it would allow the full CDM memory to be used by the application and kswapd
would never be woken.
KSM is potentially more problematic and initially may have to be disabled
entirely to determine if it actually matters for CDM-aware applications or
not. KSM normally comes into play with virtual machines are involved so it
would have to be decided if CDM is being exposed to guests with pass-thru
or some other mechanism. Initially, just disable it unless the use cases
are known.
> Something we would like to avoid. NUMA gives the application
> a transparent view of memory, in the sense that all mm features work,
> like direct page cache allocation in coherent device memory, limiting
> memory via cgroups if required, etc. With CPUSets, its
> possible for us to isolate allocation. One challenge is that the
> admin on the system may use them differently and applications need to
> be aware of running in the right cpuset to allocate memory from the
> CDM node.
An admin and application has to deal with this complexity regardless.
Particular care would be needed for file-backed data as an application
would have to ensure the data was not already cache resident. For
example, creating a data file and then doing computation on it may be
problematic. Unconditionally, the application is going to have to deal
with migration.
Identifying issues like this are why an end-to-end application that
takes advantage of the feature is important. Otherwise, there is a risk
that APIs are exposed to userspace that are Linux-specific,
device-specific and unusable.
> Putting all applications in the cpuset with the CDM node is
> not the right thing to do, which means the application needs to move itself
> to the right cpuset before requesting for CDM memory. It's not impossible
> to use CPUsets, just hard to configure correctly.
They optionally could also use move_pages.
> - With HMM, we would need a HMM variant HMM-CDM, so that we are not marking
> the pages as unavailable, page cache cannot do directly to coherent memory.
> Audit of mm paths is required. Most of the other things should work.
> User access to HMM-CDM memory behind ZONE_DEVICE is via a device driver.
The main reason why I would prefer HMM-CDM is two-fold. The first is
that using these accelerators still has use cases that are not very well
defined but if an application could use either CDM or HMM transparently
then it may be better overall.
The second reason is because there are technologies like near-memory coming
in the future and there is no infrastructure in place to take advantage like
that. I haven't even heard of plans from developers working with vendors of
such devices on how they intend to support it. Hence, the desired policies
are unknown such as whether the near memory should be isolated or if there
should be policies that promote/demote data between NUMA nodes instead of
reclaim. While I'm not involved in enabling such technology, I worry that
there will be collisiosn in the policies required for CDM and those required
for near-memory but once the API is exposed to userspace, it becomes fixed.
> Do we need to isolate node attributes independent of coherent device memory?
> - Christoph Lameter thought it would be useful to isolate node attributes,
> specifically ksm/autonuma for low latency suff.
Whatever about KSM, I would have suggested that autonuma have a prctl
flag to disable autonuma on a per-task basis. It would be sufficient for
anonymous memory at least. It would have some hazards if a
latency-sensitive application shared file-backed data with a normal
application but latency-sensitive applications generally have to take
care to isolate themselves properly.
> Why do we need migration?
> - Depending on where the memory is being accessed from, we would like to
> migrate pages between system and coherent device memory. HMM provides
> DMA offload capability that is useful in both cases.
That suggests that HMM would be a better idea.
> What is the larger picture - end to end?
> - Applications can allocate memory on the device or in system memory,
> offload the compute via user space API. Migration can be used for performance
> if required since it helps to keep the memory local to the compute.
>
The end-to-end is what matters because there is an expectation that
applications will have to use libraries to control the actual acceleration
and collection of results. The same libraries should be responsible for
doing the migration if necessary. While I accept that bringing up the
library would be inconvenient as supporting tools will be needed for the
application, it's better than quickly exposting CDM devices as NUMA as this
suggests, applying the policies and then finding the same supporting tools
and libraries were needed anyway and the proposed policies did not help.
> Comments from the thread
>
> 1. If we go down the NUMA path, we need to live with the limitations of
> what comes with the cpuless NUMA node
> 2. The changes made to cpusets and mempolicies, make the code more complex
> 3. We need a good end to end story
>
> The comments from the thread were responded to
>
> How do we go about implementing CDM then?
>
> The recommendation from John Hubbard/Mel Gorman and Michal Hocko is to
> use HMM-CDM to solve the problem. Jerome/Balbir and Ben H prefer NUMA-CDM.
> There were suggestions that NUMA might not be ready or is the best approach
> in the long term, but we are yet to identify what changes to NUMA would
> enable it to support NUMA-CDM.
>
Primarily, I would suggest that HMM-CDM be taken as far as possible on the
hope/expectation that an application could transparently use either CDM
(memory visible to both CPU and device) or HMM (special care required)
with a common library API. This may be unworkable ultimately but it's
impossible to know unless someone is fully up to date with exactly how
these devices are to be used by appliications.
If NUMA nodes are still required then the initial path appears to
be controlling the onlining of memory from the device, isolating from
userspace with existing mechanisms and using library awareness to control
the migration. If DMA offloading is required then the device would also
need to control that which may or may not push it towards HMM again.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* mm: page allocation failures in swap_duplicate -> add_swap_count_continuation
From: Christian Borntraeger @ 2017-05-12 9:18 UTC (permalink / raw)
To: linux-mm, Linux Kernel Mailing List
Folks,
recently I have seen page allocation failures during
paging in the paging code:
e.g.
May 05 21:36:53 kernel: Call Trace:
May 05 21:36:53 kernel: ([<0000000000112f62>] show_trace+0x62/0x78)
May 05 21:36:53 kernel: [<0000000000113050>] show_stack+0x68/0xe0
May 05 21:36:53 kernel: [<00000000004fb97e>] dump_stack+0x7e/0xb0
May 05 21:36:53 kernel: [<0000000000299262>] warn_alloc+0xf2/0x190
May 05 21:36:53 kernel: [<000000000029a25a>] __alloc_pages_nodemask+0xeda/0xfe0
May 05 21:36:53 kernel: [<00000000002fa570>] alloc_pages_current+0xb8/0x170
May 05 21:36:53 kernel: [<00000000002f03fc>] add_swap_count_continuation+0x3c/0x280
May 05 21:36:53 kernel: [<00000000002f068c>] swap_duplicate+0x4c/0x80
May 05 21:36:53 kernel: [<00000000002dfbfa>] try_to_unmap_one+0x372/0x578
May 05 21:36:53 kernel: [<000000000030131a>] rmap_walk_ksm+0x14a/0x1d8
May 05 21:36:53 kernel: [<00000000002e0d60>] try_to_unmap+0x140/0x170
May 05 21:36:53 kernel: [<00000000002abc9c>] shrink_page_list+0x944/0xad8
May 05 21:36:53 kernel: [<00000000002ac720>] shrink_inactive_list+0x1e0/0x5b8
May 05 21:36:53 kernel: [<00000000002ad642>] shrink_node_memcg+0x5e2/0x800
May 05 21:36:53 kernel: [<00000000002ad954>] shrink_node+0xf4/0x360
May 05 21:36:53 kernel: [<00000000002aeb00>] kswapd+0x330/0x810
May 05 21:36:53 kernel: [<0000000000189f14>] kthread+0x144/0x168
May 05 21:36:53 kernel: [<00000000008011ea>] kernel_thread_starter+0x6/0xc
May 05 21:36:53 kernel: [<00000000008011e4>] kernel_thread_starter+0x0/0xc
This seems to be new in 4.11 but the relevant code did not seem to have
changed.
Something like this
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 1781308..b2dd53e 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -3039,7 +3039,7 @@ int swap_duplicate(swp_entry_t entry)
int err = 0;
while (!err && __swap_duplicate(entry, 1) == -ENOMEM)
- err = add_swap_count_continuation(entry, GFP_ATOMIC);
+ err = add_swap_count_continuation(entry, GFP_ATOMIC | __GFP_NOWARN);
return err;
}
seems not appropriate, because this code does not know if the caller can
handle returned errors.
Would something like the following (white space damaged cut'n'paste be ok?
(the try_to_unmap_one change looks fine, not sure if copy_one_pte does the
right thing)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 45e91dd..4577494 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -391,7 +391,7 @@ extern swp_entry_t get_swap_page_of_type(int);
extern int get_swap_pages(int n, swp_entry_t swp_entries[]);
extern int add_swap_count_continuation(swp_entry_t, gfp_t);
extern void swap_shmem_alloc(swp_entry_t);
-extern int swap_duplicate(swp_entry_t);
+extern int swap_duplicate(swp_entry_t, gfp_t);
extern int swapcache_prepare(swp_entry_t);
extern void swap_free(swp_entry_t);
extern void swapcache_free(swp_entry_t);
@@ -447,7 +447,7 @@ static inline void swap_shmem_alloc(swp_entry_t swp)
{
}
-static inline int swap_duplicate(swp_entry_t swp)
+int swap_duplicate(swp_entry_t entry, gfp_t gfp_mask)
{
return 0;
}
diff --git a/mm/memory.c b/mm/memory.c
index 235ba51..3ae6f33 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -898,7 +898,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
swp_entry_t entry = pte_to_swp_entry(pte);
if (likely(!non_swap_entry(entry))) {
- if (swap_duplicate(entry) < 0)
+ if (swap_duplicate(entry, __GFP_NOWARN) < 0)
return entry.val;
/* make sure dst_mm is on swapoff's mmlist. */
diff --git a/mm/rmap.c b/mm/rmap.c
index f683801..777feb6 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1433,7 +1433,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
goto discard;
}
- if (swap_duplicate(entry) < 0) {
+ if (swap_duplicate(entry, __GFP_NOWARN) < 0) {
set_pte_at(mm, address, pvmw.pte, pteval);
ret = SWAP_FAIL;
page_vma_mapped_walk_done(&pvmw);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 1781308..1f86268 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -3034,12 +3034,12 @@ void swap_shmem_alloc(swp_entry_t entry)
* if __swap_duplicate() fails for another reason (-EINVAL or -ENOENT), which
* might occur if a page table entry has got corrupted.
*/
-int swap_duplicate(swp_entry_t entry)
+int swap_duplicate(swp_entry_t entry, gfp_t gfp_mask)
{
int err = 0;
while (!err && __swap_duplicate(entry, 1) == -ENOMEM)
- err = add_swap_count_continuation(entry, GFP_ATOMIC);
+ err = add_swap_count_continuation(entry, GFP_ATOMIC | gfp_mask);
return err;
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* 8 Gigabytes and constantly swapping
From: Arthur Marsh @ 2017-05-12 8:51 UTC (permalink / raw)
To: linux-mm
I've been building the Linus git head kernels as the source gets updated
and the one built about 3 hours ago managed to get stuck with kswapd0 as
the highest consumer of CPU cycles (but still under 1 percent) of
processes listed by top for over 15 minutes, after which I hit the power
switch and rebooted with a Debian 4.11.0 kernel.
The previous kernel built less than 24 hours earlier did not have this
problem.
CPU is an Athlon64 (Athlon II X4, 4 cores), RAM is 8GiB, swap is 4GiB,
load was mainly firefox and chromium. Opening a new window in chromium
seemed to help trigger the problem.
It's not much information to go on, just wondered if anyone else had
experienced similar issues?
I'm happy to supply more configuration information and run tests
including with kernels built with test patches applied.
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm/madvise: Dont poison entire HugeTLB page for single page errors
From: Naoya Horiguchi @ 2017-05-12 8:10 UTC (permalink / raw)
To: Anshuman Khandual
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
akpm@linux-foundation.org, aneesh.kumar@linux.vnet.ibm.com
In-Reply-To: <20170420110627.12307-1-khandual@linux.vnet.ibm.com>
On Thu, Apr 20, 2017 at 04:36:27PM +0530, Anshuman Khandual wrote:
> Currently soft_offline_page() migrates the entire HugeTLB page, then
> dequeues it from the active list by making it a dangling HugeTLB page
> which ofcourse can not be used further and marks the entire HugeTLB
> page as poisoned. This might be a costly waste of memory if the error
> involved affects only small section of the entire page.
>
> This changes the behaviour so that only the affected page is marked
> poisoned and then the HugeTLB page is released back to buddy system.
Hi Anshuman,
This is a good catch, and we can solve this issue now because freeing
hwpoisoned page (previously forbidden) is available now.
And I'm thinking that the same issue for hard/soft-offline on free
hugepages can be solved, so I'll submit a patchset which includes
updated version of your patch.
Thanks,
Naoya Horiguchi
>
> Signed-off-by: Anshuman Khandual <khandual@linux.vnet.ibm.com>
> ---
> The number of poisoned pages on the system has reduced as seen from
> dmesg triggered with 'echo m > /proc/sysrq-enter' on powerpc.
>
> include/linux/hugetlb.h | 1 +
> mm/hugetlb.c | 2 +-
> mm/memory-failure.c | 9 ++++-----
> 3 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 7a5917d..f6b80a4 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -470,6 +470,7 @@ static inline pgoff_t basepage_index(struct page *page)
> return __basepage_index(page);
> }
>
> +extern int dissolve_free_huge_page(struct page *page);
> extern int dissolve_free_huge_pages(unsigned long start_pfn,
> unsigned long end_pfn);
> static inline bool hugepage_migration_supported(struct hstate *h)
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 1edfdb8..2fb9ba3 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1444,7 +1444,7 @@ static int free_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed,
> * number of free hugepages would be reduced below the number of reserved
> * hugepages.
> */
> -static int dissolve_free_huge_page(struct page *page)
> +int dissolve_free_huge_page(struct page *page)
> {
> int rc = 0;
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 27f7210..1e377fd 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1597,13 +1597,12 @@ static int soft_offline_huge_page(struct page *page, int flags)
> ret = -EIO;
> } else {
> /* overcommit hugetlb page will be freed to buddy */
> + SetPageHWPoison(page);
> + num_poisoned_pages_inc();
> +
> if (PageHuge(page)) {
> - set_page_hwpoison_huge_page(hpage);
> dequeue_hwpoisoned_huge_page(hpage);
> - num_poisoned_pages_add(1 << compound_order(hpage));
> - } else {
> - SetPageHWPoison(page);
> - num_poisoned_pages_inc();
> + dissolve_free_huge_page(hpage);
> }
> }
> return ret;
> --
> 1.8.5.2
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Michal Hocko @ 2017-05-12 6:38 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, mgorman,
Laura Abbott, Minchan Kim, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K . V, Vlastimil Babka, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <20170512020046.GA5538@js1304-desktop>
On Fri 12-05-17 11:00:48, Joonsoo Kim wrote:
> On Thu, May 11, 2017 at 11:13:04AM +0200, Michal Hocko wrote:
> > On Thu 11-05-17 11:12:43, Joonsoo Kim wrote:
> > > Sorry for the late response. I was on a vacation.
> > >
> > > On Tue, May 02, 2017 at 03:32:29PM +0200, Michal Hocko wrote:
> > > > On Tue 02-05-17 13:01:32, Joonsoo Kim wrote:
> > > > > On Thu, Apr 27, 2017 at 05:06:36PM +0200, Michal Hocko wrote:
> > > > [...]
> > > > > > I see this point and I agree that using a specific zone might be a
> > > > > > _nicer_ solution in the end but you have to consider another aspects as
> > > > > > well. The main one I am worried about is a long term maintainability.
> > > > > > We are really out of page flags and consuming one for a rather specific
> > > > > > usecase is not good. Look at ZONE_DMA. I am pretty sure that almost
> > > > > > no sane HW needs 16MB zone anymore, yet we have hard time to get rid
> > > > > > of it and so we have that memory laying around unused all the time
> > > > > > and blocking one page flag bit. CMA falls into a similar category
> > > > > > AFAIU. I wouldn't be all that surprised if a future HW will not need CMA
> > > > > > allocations in few years, yet we will have to fight to get rid of it
> > > > > > like we do with ZONE_DMA. And not only that. We will also have to fight
> > > > > > finding page flags for other more general usecases in the meantime.
> > > > >
> > > > > This maintenance problem is inherent. This problem exists even if we
> > > > > uses MIGRATETYPE approach. We cannot remove many hooks for CMA if a
> > > > > future HW will not need CMA allocation in few years. The only
> > > > > difference is that one takes single zone bit only for CMA user and the
> > > > > other approach takes many hooks that we need to take care about it all
> > > > > the time.
> > > >
> > > > And I consider this a big difference. Because while hooks are not nice
> > > > they will affect CMA users (in a sense of bugs/performance etc.). While
> > > > an additional bit consumed will affect potential future and more generic
> > > > features.
> > >
> > > Because these hooks are so tricky and are spread on many places,
> > > bugs about these hooks can be made by *non-CMA* user and they hurt
> > > *CMA* user. These hooks could also delay non-CMA user's development speed
> > > since there are many hooks about CMA and understanding how CMA is managed
> > > is rather difficult.
> >
> > Than make those hooks easier to maintain. Seriously this is a
> > non-argument.
>
> I can't understand what you said here.
I wanted to say that you can make those hooks so non-intrusive that
nobody outside of the CMA has to even care that CMA exists.
> With zone approach, someone who
> isn't related to CMA don't need to understand how CMA is managed.
>
> >
> > [...]
> >
> > > > And all this can be isolated to CMA specific hooks with mostly minimum
> > > > impact to most users. I hear you saying that zone approach is more natural
> > > > and I would agree if we wouldn't have to care about the number of zones.
> > >
> > > I attach a solution about one more bit in page flags although I don't
> > > agree with your opinion that additional bit is no-go approach. Just
> > > note that we have already used three bits for zone encoding in some
> > > configuration due to ZONE_DEVICE.
> >
> > I am absolutely not happy about ZONE_DEVICE but there is _no_ other
> > viable solution right now. I know that people behind this are still
> > considering struct page over direct pfn usage but they are not in the
> > same situation as CMA which _can_ work without additional zone.
>
> IIUC, ZONE_DEVICE can reuse the other zone and migratetype.
They are not going to migrate anything or define any allocation fallback
policy because those pages are outside of the page allocator completely.
And that is why a zone approach is a reasonable approach. There are
probably other ways and I will certainly push going that way.
[...]
> > If you _really_ insist on using zone for CMA then reuse ZONE_MOVABLE.
> > I absolutely miss why do you push a specialized zone so hard.
>
> As I said before, there is no fundamental issue to reuse ZONE_MOVABLE.
> I just don't want to reuse it because they have a different
> characteristic. In MM subsystem context, their characteristic is the same.
> However, CMA memory should be used for the device in runtime so more
> allocation guarantee is needed. See the offline_pages() in
> mm/memory_hotplug.c. They can bear in 120 sec to offline the
> memory but CMA memory can't.
This is just an implementation detail. Pinned pages in the CMA ranges
should be easilly checked. Moreover memory hotplug cares only about
hotplugable memory and placing CMA ranges there could be seen as a
configuration bug.
> And, this is a design issue. I don't want to talk about why should we
> pursuit the good design. Originally, ZONE exists to manage different
> type of memory. Migratetype is not for that purpose. Using separate
> zone fits the original purpose. Mixing them would be a bad design and
> we would esaily encounter the unexpected problem in the future.
As I've said earlier. Abusing ZONE_MOVABLE is not ideal either. I would
rather keep the status quo and fix the cluttered code and make it easier
to follow. But if you absolutely insist that a specialized zone is
necessary then ZONE_MOVABLE a) already exists and we do not need to
consume another bit b) most of the CMA zone characteristics overlap
with MOVABLE. So it is the least painful zone to use with the current
restrictions we have.
> > [...]
> > > > No, but I haven't heard any single argument that those bugs are
> > > > impossible to fix with the current approach. They might be harder to fix
> > > > but if I can chose between harder for CMA and harder for other more
> > > > generic HW independent features I will go for the first one. And do not
> > > > take me wrong, I have nothing against CMA as such. It solves a real life
> > > > problem. I just believe it doesn't deserve to consume a new bit in page
> > > > flags because that is just too scarce resource.
> > >
> > > As I mentioned above, I think that maintenance overhead due to CMA
> > > deserves to consume a new bit in page flags. It also provide us
> > > extendability to introduce more zones in the future.
> > >
> > > Anyway, this value-judgement is subjective so I guess that we
> > > cannot agree with each other. To solve your concern,
> > > I make following solution. Please let me know your opinion about this.
> > > This patch can be applied on top of my ZONE_CMA series.
> >
> > I don not think this makes situation any easier or more acceptable for
> > merging.
>
> Please say the reason. This implementation don't use additional bit in
> page flags that you concerned about. And, there is no performance
> regression at least in my simple test.
I really do not want to question your "simple test" but page_zonenum is
used in many performance sensitive paths and proving it doesn't regress
would require testing many different workload. Are you going to do that?
> > But I feel we are looping without much progress. So let me NAK this
> > until it is _proven_ that the current code is unfixable nor ZONE_MOVABLE
> > can be reused
>
> I want to open all the possibilty so could you check that ZONE_MOVABLE
> can be overlapped with other zones? IIRC, your rework doesn't allow
> it.
My rework keeps the status quo, which is based on the assumption that
zones cannot overlap. A longer term plan is that this restriction is
removed. As I've said earlier overlapping zones is an interesting
concept which is definitely worth pursuing.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [RFC summary] Enable Coherent Device Memory
From: Balbir Singh @ 2017-05-12 6:18 UTC (permalink / raw)
To: linux-mm
Cc: akpm, khandual, aneesh.kumar, paulmck, srikar, haren, jglisse,
mgorman, arbab, vbabka, Christoph Lameter, Rik van Riel,
Benjamin Herrenschmidt
Here is a summary of the RFC I posted for coherent device memory
(see https://lwn.net/Articles/720380/)
I did an FAQ in one of the emails, I am extending that to summary form
so that we can move ahead towards decision making
What is coherent device memory?
- Please see the RFC (https://lwn.net/Articles/720380/) and
https://lwn.net/Articles/717601/
Why do we need to isolate memory?
- CDM memory is not meant for normal usage, applications can request for it
explictly. Oflload their compute to the device where the memory is
(the offload is via a user space API like CUDA/openCL/...)
How do we isolate the memory - NUMA or HMM-CDM?
- Since the memory is coherent, NUMA provides the mechanism to isolate to
a large extent via mempolicy. With NUMA we also get autonuma/kswapd/etc
running. Something we would like to avoid. NUMA gives the application
a transparent view of memory, in the sense that all mm features work,
like direct page cache allocation in coherent device memory, limiting
memory via cgroups if required, etc. With CPUSets, its
possible for us to isolate allocation. One challenge is that the
admin on the system may use them differently and applications need to
be aware of running in the right cpuset to allocate memory from the
CDM node. Putting all applications in the cpuset with the CDM node is
not the right thing to do, which means the application needs to move itself
to the right cpuset before requesting for CDM memory. It's not impossible
to use CPUsets, just hard to configure correctly.
- With HMM, we would need a HMM variant HMM-CDM, so that we are not marking
the pages as unavailable, page cache cannot do directly to coherent memory.
Audit of mm paths is required. Most of the other things should work.
User access to HMM-CDM memory behind ZONE_DEVICE is via a device driver.
Do we need to isolate node attributes independent of coherent device memory?
- Christoph Lameter thought it would be useful to isolate node attributes,
specifically ksm/autonuma for low latency suff.
Why do we need migration?
- Depending on where the memory is being accessed from, we would like to
migrate pages between system and coherent device memory. HMM provides
DMA offload capability that is useful in both cases.
What is the larger picture - end to end?
- Applications can allocate memory on the device or in system memory,
offload the compute via user space API. Migration can be used for performance
if required since it helps to keep the memory local to the compute.
Comments from the thread
1. If we go down the NUMA path, we need to live with the limitations of
what comes with the cpuless NUMA node
2. The changes made to cpusets and mempolicies, make the code more complex
3. We need a good end to end story
The comments from the thread were responded to
How do we go about implementing CDM then?
The recommendation from John Hubbard/Mel Gorman and Michal Hocko is to
use HMM-CDM to solve the problem. Jerome/Balbir and Ben H prefer NUMA-CDM.
There were suggestions that NUMA might not be ready or is the best approach
in the long term, but we are yet to identify what changes to NUMA would
enable it to support NUMA-CDM.
The trade-offs and limitations/advantages of both approaches are in the
RFC thread and in the summary above. It seems like the from the discussions
with Michal/Mel/John the direction is to use HMM-CDM for now (both from the
thread and from mm-summit). Can we build consensus on this and move forward?
Are there any objections? Did I miss or misrepresent anything from the threads?
It would be good to get feedback from Andrew Morton and Rik Van Riel as well.
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: Kernel problem
From: Michal Hocko @ 2017-05-12 5:56 UTC (permalink / raw)
To: Randy Dunlap; +Cc: Frank Vosberg, linux-mm@kvack.org
In-Reply-To: <bbde3fc7-fa8c-7872-1099-44a3c293ffba@infradead.org>
On Thu 11-05-17 09:57:25, Randy Dunlap wrote:
[...]
> I'll let someone else comment on the actual warning message:
> Creating hierarchies with use_hierarchy==0 (flat hierarchy) is considered deprecated. If you believe that your setup is correct, we kindly ask you to contact linux-mm@kvack.org and let us know
Well, this warning just says that using not hierarchical memory cgroup
hierarchy is a bad idea and this behavior will not be supported for ever
(or for v2 cgroup for that matter). It should warn users who are using
old kernels to either change their configuration or complain that they
have a valid usecase for such a configuration so that we can think of an
alternative approach. From the original email it is not clear to me
whether this configuration is intentional or not, though.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* mm/kasan: zero_p4d_populate() problem?
From: Joonsoo Kim @ 2017-05-12 5:53 UTC (permalink / raw)
To: kirill.shutemov; +Cc: Andrey Ryabinin, linux-mm
Hello, Kirill.
I found that zero_p4d_populate() in mm/kasan/kasan_init.c of
next-20170511 doesn't get the benefit of the kasan_zero_pud.
Do we need to fix it by adding
"pud_populate(&init_mm, pud, lm_alias(kasan_zero_pud));" when
alignment requirement is met?
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 04/10] x86/mm: Pass flush_tlb_info to flush_tlb_others() etc
From: Andy Lutomirski @ 2017-05-12 3:41 UTC (permalink / raw)
To: Nadav Amit
Cc: Andy Lutomirski, X86 ML, linux-kernel@vger.kernel.org,
Borislav Petkov, Linus Torvalds, Andrew Morton, Mel Gorman,
linux-mm@kvack.org, Rik van Riel, Dave Hansen, Nadav Amit,
Michal Hocko, Sasha Levin
In-Reply-To: <EFEA207A-72E6-4A41-A70A-D91CC9EB8A47@gmail.com>
On Thu, May 11, 2017 at 1:01 PM, Nadav Amit <nadav.amit@gmail.com> wrote:
>
>> On May 7, 2017, at 5:38 AM, Andy Lutomirski <luto@kernel.org> wrote:
>>
>> @@ -243,15 +237,15 @@ static void flush_tlb_func(void *info)
>> return;
>> }
>>
>> - if (f->flush_end == TLB_FLUSH_ALL) {
>> + if (f->end == TLB_FLUSH_ALL) {
>> local_flush_tlb();
>> trace_tlb_flush(TLB_REMOTE_SHOOTDOWN, TLB_FLUSH_ALL);
>> } else {
>> unsigned long addr;
>> unsigned long nr_pages =
>> - (f->flush_end - f->flush_start) / PAGE_SIZE;
>> - addr = f->flush_start;
>> - while (addr < f->flush_end) {
>> + (f->end - f->start) / PAGE_SIZE;
>> + addr = f->start;
>> + while (addr < f->end) {
>> __flush_tlb_single(addr);
>> addr += PAGE_SIZE;
>> }
>> @@ -260,33 +254,27 @@ static void flush_tlb_func(void *info)
>> }
>>
>> void native_flush_tlb_others(const struct cpumask *cpumask,
>> - struct mm_struct *mm, unsigned long start,
>> - unsigned long end)
>> + const struct flush_tlb_info *info)
>> {
>> - struct flush_tlb_info info;
>> -
>> - info.flush_mm = mm;
>> - info.flush_start = start;
>> - info.flush_end = end;
>> -
>> count_vm_tlb_event(NR_TLB_REMOTE_FLUSH);
>> - if (end == TLB_FLUSH_ALL)
>> + if (info->end == TLB_FLUSH_ALL)
>> trace_tlb_flush(TLB_REMOTE_SEND_IPI, TLB_FLUSH_ALL);
>> else
>> trace_tlb_flush(TLB_REMOTE_SEND_IPI,
>> - (end - start) >> PAGE_SHIFT);
>> + (info->end - info->start) >> PAGE_SHIFT);
>
> I know it is stupid, but since you already change the code, can you make
> flush_tlb_func() and native_flush_tlb_others() consistent in the way
> they compute the number of pages? (either ‘>> PAGE_SHIFT’ or ‘/ PAGE_SIZE’)?
I added this to the queue.
>
> On a different topic: I do not like or actually understand why TLBSTATE_OK
> is defined as 1 and not 0. The very least it would generate a better code.
Me neither. The value 0 can happen too, e.g. for init_mm. Maybe I'll
clean that up when I'm all done with this stuff.
>
> Thanks,
> Nadav
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 09/10] x86/mm: Rework lazy TLB to track the actual loaded mm
From: Andy Lutomirski @ 2017-05-12 3:36 UTC (permalink / raw)
To: Ingo Molnar
Cc: Andy Lutomirski, Thomas Gleixner, X86 ML,
linux-kernel@vger.kernel.org, Borislav Petkov, Linus Torvalds,
Andrew Morton, Mel Gorman, linux-mm@kvack.org, Rik van Riel,
Dave Hansen, Nadav Amit, Michal Hocko, Arjan van de Ven
In-Reply-To: <20170511071348.jhgzdgi7blhgenqj@gmail.com>
On Thu, May 11, 2017 at 12:13 AM, Ingo Molnar <mingo@kernel.org> wrote:
> My personal favorite is double underscores prefix, i.e. 'void *__mm', which would
> clearly signal that this is something special. But this does not appear to have
> been picked up overly widely:
Nice bikeshed! I'll use it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 01/10] x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range()
From: Andy Lutomirski @ 2017-05-12 3:35 UTC (permalink / raw)
To: Borislav Petkov
Cc: Andy Lutomirski, X86 ML, linux-kernel@vger.kernel.org,
Linus Torvalds, Andrew Morton, Mel Gorman, linux-mm@kvack.org,
Rik van Riel, Dave Hansen, Nadav Amit, Michal Hocko
In-Reply-To: <20170511174128.rp7dwckpci4gqsxy@pd.tnic>
On Thu, May 11, 2017 at 10:41 AM, Borislav Petkov <bp@suse.de> wrote:
>> +{
>> + flush_tlb_mm_range(vma->vm_mm, a, a + PAGE_SIZE, 0);
>
> VM_NONE);
>
Fixed, although this won't have any effect.
--Andy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox