* Re: [PATCH] vmscan: scan pages until it founds eligible pages
From: Michal Hocko @ 2017-05-02 15:14 UTC (permalink / raw)
To: Minchan Kim
Cc: Andrew Morton, Johannes Weiner, Mel Gorman, kernel-team,
linux-kernel, linux-mm
In-Reply-To: <20170502145150.GA19011@bgram>
On Tue 02-05-17 23:51:50, Minchan Kim wrote:
> Hi Michal,
>
> On Tue, May 02, 2017 at 09:54:32AM +0200, Michal Hocko wrote:
> > On Tue 02-05-17 14:14:52, Minchan Kim wrote:
> > > Oops, forgot to add lkml and linux-mm.
> > > Sorry for that.
> > > Send it again.
> > >
> > > >From 8ddf1c8aa15baf085bc6e8c62ce705459d57ea4c Mon Sep 17 00:00:00 2001
> > > From: Minchan Kim <minchan@kernel.org>
> > > Date: Tue, 2 May 2017 12:34:05 +0900
> > > Subject: [PATCH] vmscan: scan pages until it founds eligible pages
> > >
> > > On Tue, May 02, 2017 at 01:40:38PM +0900, Minchan Kim wrote:
> > > There are premature OOM happening. Although there are a ton of free
> > > swap and anonymous LRU list of elgible zones, OOM happened.
> > >
> > > With investigation, skipping page of isolate_lru_pages makes reclaim
> > > void because it returns zero nr_taken easily so LRU shrinking is
> > > effectively nothing and just increases priority aggressively.
> > > Finally, OOM happens.
> >
> > I am not really sure I understand the problem you are facing. Could you
> > be more specific please? What is your configuration etc...
>
> Sure, KVM guest on x86_64, It has 2G memory and 1G swap and configured
> movablecore=1G to simulate highmem zone.
> Workload is a process consumes 2.2G memory and then random touch the
> address space so it makes lots of swap in/out.
>
> >
> > > balloon invoked oom-killer: gfp_mask=0x17080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO|__GFP_NOTRACK), nodemask=(null), order=0, oom_score_adj=0
> > [...]
> > > Node 0 active_anon:1698864kB inactive_anon:261256kB active_file:208kB inactive_file:184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:532kB dirty:108kB writeback:0kB shmem:172kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
> > > DMA free:7316kB min:32kB low:44kB high:56kB active_anon:8064kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB slab_reclaimable:464kB slab_unreclaimable:40kB kernel_stack:0kB pagetables:24kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> > > lowmem_reserve[]: 0 992 992 1952
> > > DMA32 free:9088kB min:2048kB low:3064kB high:4080kB active_anon:952176kB inactive_anon:0kB active_file:36kB inactive_file:0kB unevictable:0kB writepending:88kB present:1032192kB managed:1019388kB mlocked:0kB slab_reclaimable:13532kB slab_unreclaimable:16460kB kernel_stack:3552kB pagetables:6672kB bounce:0kB free_pcp:56kB local_pcp:24kB free_cma:0kB
> > > lowmem_reserve[]: 0 0 0 959
> >
> > Hmm DMA32 has sufficient free memory to allow this order-0 request.
> > Inactive anon lru is basically empty. Why do not we rotate a really
> > large active anon list? Isn't this the primary problem?
>
> It's a side effect by skipping page logic in isolate_lru_pages
> I mentioned above in changelog.
>
> The problem is a lot of anonymous memory in movable zone(ie, highmem)
> and non-small memory in DMA32 zone.
Such a configuration is questionable on its own. But let't keep this
part alone.
> In heavy memory pressure,
> requesting a page in GFP_KERNEL triggers reclaim. VM knows inactive list
> is low so it tries to deactivate pages. For it, first of all, it tries
> to isolate pages from active list but there are lots of anonymous pages
> from movable zone so skipping logic in isolate_lru_pages works. With
> the result, isolate_lru_pages cannot isolate any eligible pages so
> reclaim trial is effectively void. It continues to meet OOM.
But skipped pages should be rotated and we should eventually hit pages
from the right zone(s). Moreover we should scan the full LRU at priority
0 so why exactly we hit the OOM killer?
Anyway [1] has changed this behavior. Are you seeing the issue with this
patch dropped?
[1] http://www.ozlabs.org/~akpm/mmotm/broken-out/revert-mm-vmscan-account-for-skipped-pages-as-a-partial-scan.patch
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 1/2] mm: Uncharge poisoned pages
From: Laurent Dufour @ 2017-05-02 14:59 UTC (permalink / raw)
To: Michal Hocko
Cc: Andi Kleen, Naoya Horiguchi, linux-kernel, linux-mm, akpm,
Johannes Weiner, Vladimir Davydov
In-Reply-To: <20170428134831.GB26705@dhcp22.suse.cz>
On 28/04/2017 15:48, Michal Hocko wrote:
> On Fri 28-04-17 11:17:34, Laurent Dufour wrote:
>> On 28/04/2017 09:31, Michal Hocko wrote:
>>> [CC Johannes and Vladimir - the patch is
>>> http://lkml.kernel.org/r/1493130472-22843-2-git-send-email-ldufour@linux.vnet.ibm.com]
>>>
>>> On Fri 28-04-17 08:07:55, Michal Hocko wrote:
>>>> On Thu 27-04-17 13:51:23, Andi Kleen wrote:
>>>>> Michal Hocko <mhocko@kernel.org> writes:
>>>>>
>>>>>> On Tue 25-04-17 16:27:51, Laurent Dufour wrote:
>>>>>>> When page are poisoned, they should be uncharged from the root memory
>>>>>>> cgroup.
>>>>>>>
>>>>>>> This is required to avoid a BUG raised when the page is onlined back:
>>>>>>> BUG: Bad page state in process mem-on-off-test pfn:7ae3b
>>>>>>> page:f000000001eb8ec0 count:0 mapcount:0 mapping: (null)
>>>>>>> index:0x1
>>>>>>> flags: 0x3ffff800200000(hwpoison)
>>>>>>
>>>>>> My knowledge of memory poisoning is very rudimentary but aren't those
>>>>>> pages supposed to leak and never come back? In other words isn't the
>>>>>> hoplug code broken because it should leave them alone?
>>>>>
>>>>> Yes that would be the right interpretation. If it was really offlined
>>>>> due to a hardware error the memory will be poisoned and any access
>>>>> could cause a machine check.
>>>>
>>>> OK, thanks for the clarification. Then I am not sure the patch is
>>>> correct. Why do we need to uncharge that page at all?
>>>
>>> Now, I have realized that we actually want to uncharge that page because
>>> it will pin the memcg and we do not want to have that memcg and its
>>> whole hierarchy pinned as well. This used to work before the charge
>>> rework 0a31bc97c80c ("mm: memcontrol: rewrite uncharge API") I guess
>>> because we used to uncharge on page cache removal.
>>>
>>> I do not think the patch is correct, though. memcg_kmem_enabled() will
>>> check whether kmem accounting is enabled and we are talking about page
>>> cache pages here. You should be using mem_cgroup_uncharge instead.
>>
>> Thanks for the review Michal.
>>
>> I was not comfortable either with this patch.
>>
>> I did some tests calling mem_cgroup_uncharge() when isolate_lru_page()
>> succeeds only, so not calling it if isolate_lru_page() failed.
>
> Wait a moment. This cannot possibly work. isolate_lru_page asumes page
> count > 0 and increments the counter so the resulting page count is > 1
> I have only now realized that we have VM_BUG_ON_PAGE(page_count(page), page)
> in uncharge_list().
My mistake, my kernel was not build with CONFIG_DEBUG_VM set.
You're right this cannot work this way.
> This is getting quite hairy. What is the expected page count of the
> hwpoison page? I guess we would need to update the VM_BUG_ON in the
> memcg uncharge code to ignore the page count of hwpoison pages if it can
> be arbitrary.
Based on the experiment I did, page count == 2 when isolate_lru_page()
succeeds, even in the case of a poisoned page. In my case I think this
is because the page is still used by the process which is calling madvise().
I'm wondering if I'm looking at the right place. May be the poisoned
page should remain attach to the memory_cgroup until no one is using it.
In that case this means that something should be done when the page is
off-lined... I've to dig further here.
>
> Before we go any further, is there any documentation about the expected
> behavior and the state of the hwpoison pages? I have a very bad feeling
> that the current behavior is quite arbitrary and "testing driven"
> holes plugging will make it only more messy. So let's start with the
> clear description of what should happen with the hwpoison pages.
I didn't find any documentation about that. The root cause is that a bug
message is displayed when a poisoned page is off-lined, may be this is
in that path that something is missing.
Cheers,
Laurent.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] vmscan: scan pages until it founds eligible pages
From: Minchan Kim @ 2017-05-02 14:51 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, Johannes Weiner, Mel Gorman, kernel-team,
linux-kernel, linux-mm
In-Reply-To: <20170502075432.GC14593@dhcp22.suse.cz>
Hi Michal,
On Tue, May 02, 2017 at 09:54:32AM +0200, Michal Hocko wrote:
> On Tue 02-05-17 14:14:52, Minchan Kim wrote:
> > Oops, forgot to add lkml and linux-mm.
> > Sorry for that.
> > Send it again.
> >
> > >From 8ddf1c8aa15baf085bc6e8c62ce705459d57ea4c Mon Sep 17 00:00:00 2001
> > From: Minchan Kim <minchan@kernel.org>
> > Date: Tue, 2 May 2017 12:34:05 +0900
> > Subject: [PATCH] vmscan: scan pages until it founds eligible pages
> >
> > On Tue, May 02, 2017 at 01:40:38PM +0900, Minchan Kim wrote:
> > There are premature OOM happening. Although there are a ton of free
> > swap and anonymous LRU list of elgible zones, OOM happened.
> >
> > With investigation, skipping page of isolate_lru_pages makes reclaim
> > void because it returns zero nr_taken easily so LRU shrinking is
> > effectively nothing and just increases priority aggressively.
> > Finally, OOM happens.
>
> I am not really sure I understand the problem you are facing. Could you
> be more specific please? What is your configuration etc...
Sure, KVM guest on x86_64, It has 2G memory and 1G swap and configured
movablecore=1G to simulate highmem zone.
Workload is a process consumes 2.2G memory and then random touch the
address space so it makes lots of swap in/out.
>
> > balloon invoked oom-killer: gfp_mask=0x17080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO|__GFP_NOTRACK), nodemask=(null), order=0, oom_score_adj=0
> [...]
> > Node 0 active_anon:1698864kB inactive_anon:261256kB active_file:208kB inactive_file:184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:532kB dirty:108kB writeback:0kB shmem:172kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
> > DMA free:7316kB min:32kB low:44kB high:56kB active_anon:8064kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB slab_reclaimable:464kB slab_unreclaimable:40kB kernel_stack:0kB pagetables:24kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> > lowmem_reserve[]: 0 992 992 1952
> > DMA32 free:9088kB min:2048kB low:3064kB high:4080kB active_anon:952176kB inactive_anon:0kB active_file:36kB inactive_file:0kB unevictable:0kB writepending:88kB present:1032192kB managed:1019388kB mlocked:0kB slab_reclaimable:13532kB slab_unreclaimable:16460kB kernel_stack:3552kB pagetables:6672kB bounce:0kB free_pcp:56kB local_pcp:24kB free_cma:0kB
> > lowmem_reserve[]: 0 0 0 959
>
> Hmm DMA32 has sufficient free memory to allow this order-0 request.
> Inactive anon lru is basically empty. Why do not we rotate a really
> large active anon list? Isn't this the primary problem?
It's a side effect by skipping page logic in isolate_lru_pages
I mentioned above in changelog.
The problem is a lot of anonymous memory in movable zone(ie, highmem)
and non-small memory in DMA32 zone. In heavy memory pressure,
requesting a page in GFP_KERNEL triggers reclaim. VM knows inactive list
is low so it tries to deactivate pages. For it, first of all, it tries
to isolate pages from active list but there are lots of anonymous pages
from movable zone so skipping logic in isolate_lru_pages works. With
the result, isolate_lru_pages cannot isolate any eligible pages so
reclaim trial is effectively void. It continues to meet OOM.
I'm on long vacation from today so understand if my response is slow.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH V2 3/3] mm/slub: wrap kmem_cache->cpu_partial in config CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-02 14:45 UTC (permalink / raw)
To: cl, penberg, rientjes, iamjoonsoo.kim, akpm, willy
Cc: linux-mm, linux-kernel, Wei Yang
In-Reply-To: <20170502144533.10729-1-richard.weiyang@gmail.com>
kmem_cache->cpu_partial is just used when CONFIG_SLUB_CPU_PARTIAL is set,
so wrap it with config CONFIG_SLUB_CPU_PARTIAL will save some space
on 32bit arch.
This patch wrap kmem_cache->cpu_partial in config CONFIG_SLUB_CPU_PARTIAL
and wrap its sysfs too.
Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
---
v2: define slub_cpu_partial() to make code more elegant
---
include/linux/slub_def.h | 13 +++++++++
mm/slub.c | 69 ++++++++++++++++++++++++++----------------------
2 files changed, 51 insertions(+), 31 deletions(-)
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index f882a34bb9aa..d808e8e6293b 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -86,7 +86,9 @@ struct kmem_cache {
int size; /* The size of an object including meta data */
int object_size; /* The size of an object without meta data */
int offset; /* Free pointer offset. */
+#ifdef CONFIG_SLUB_CPU_PARTIAL
int cpu_partial; /* Number of per cpu partial objects to keep around */
+#endif
struct kmem_cache_order_objects oo;
/* Allocation and freeing of slabs */
@@ -130,6 +132,17 @@ struct kmem_cache {
struct kmem_cache_node *node[MAX_NUMNODES];
};
+#ifdef CONFIG_SLUB_CPU_PARTIAL
+#define slub_cpu_partial(s) ((s)->cpu_partial)
+#define slub_set_cpu_partial(s, n) \
+({ \
+ slub_cpu_partial(s) = (n); \
+})
+#else
+#define slub_cpu_partial(s) (0)
+#define slub_set_cpu_partial(s, n)
+#endif // CONFIG_SLUB_CPU_PARTIAL
+
#ifdef CONFIG_SYSFS
#define SLAB_SUPPORTS_SYSFS
void sysfs_slab_release(struct kmem_cache *);
diff --git a/mm/slub.c b/mm/slub.c
index ae6166533261..795112b65c61 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1829,7 +1829,7 @@ static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
stat(s, CPU_PARTIAL_NODE);
}
if (!kmem_cache_has_cpu_partial(s)
- || available > s->cpu_partial / 2)
+ || available > slub_cpu_partial(s) / 2)
break;
}
@@ -3410,6 +3410,39 @@ static void set_min_partial(struct kmem_cache *s, unsigned long min)
s->min_partial = min;
}
+static void set_cpu_partial(struct kmem_cache *s)
+{
+#ifdef CONFIG_SLUB_CPU_PARTIAL
+ /*
+ * cpu_partial determined the maximum number of objects kept in the
+ * per cpu partial lists of a processor.
+ *
+ * Per cpu partial lists mainly contain slabs that just have one
+ * object freed. If they are used for allocation then they can be
+ * filled up again with minimal effort. The slab will never hit the
+ * per node partial lists and therefore no locking will be required.
+ *
+ * This setting also determines
+ *
+ * A) The number of objects from per cpu partial slabs dumped to the
+ * per node list when we reach the limit.
+ * B) The number of objects in cpu partial slabs to extract from the
+ * per node list when we run out of per cpu objects. We only fetch
+ * 50% to keep some capacity around for frees.
+ */
+ if (!kmem_cache_has_cpu_partial(s))
+ s->cpu_partial = 0;
+ else if (s->size >= PAGE_SIZE)
+ s->cpu_partial = 2;
+ else if (s->size >= 1024)
+ s->cpu_partial = 6;
+ else if (s->size >= 256)
+ s->cpu_partial = 13;
+ else
+ s->cpu_partial = 30;
+#endif
+}
+
/*
* calculate_sizes() determines the order and the distribution of data within
* a slab object.
@@ -3568,33 +3601,7 @@ static int kmem_cache_open(struct kmem_cache *s, unsigned long flags)
*/
set_min_partial(s, ilog2(s->size) / 2);
- /*
- * cpu_partial determined the maximum number of objects kept in the
- * per cpu partial lists of a processor.
- *
- * Per cpu partial lists mainly contain slabs that just have one
- * object freed. If they are used for allocation then they can be
- * filled up again with minimal effort. The slab will never hit the
- * per node partial lists and therefore no locking will be required.
- *
- * This setting also determines
- *
- * A) The number of objects from per cpu partial slabs dumped to the
- * per node list when we reach the limit.
- * B) The number of objects in cpu partial slabs to extract from the
- * per node list when we run out of per cpu objects. We only fetch
- * 50% to keep some capacity around for frees.
- */
- if (!kmem_cache_has_cpu_partial(s))
- s->cpu_partial = 0;
- else if (s->size >= PAGE_SIZE)
- s->cpu_partial = 2;
- else if (s->size >= 1024)
- s->cpu_partial = 6;
- else if (s->size >= 256)
- s->cpu_partial = 13;
- else
- s->cpu_partial = 30;
+ set_cpu_partial(s);
#ifdef CONFIG_NUMA
s->remote_node_defrag_ratio = 1000;
@@ -3981,7 +3988,7 @@ void __kmemcg_cache_deactivate(struct kmem_cache *s)
* Disable empty slabs caching. Used to avoid pinning offline
* memory cgroups by kmem pages that can be freed.
*/
- s->cpu_partial = 0;
+ slub_set_cpu_partial(s, 0);
s->min_partial = 0;
/*
@@ -4921,7 +4928,7 @@ SLAB_ATTR(min_partial);
static ssize_t cpu_partial_show(struct kmem_cache *s, char *buf)
{
- return sprintf(buf, "%u\n", s->cpu_partial);
+ return sprintf(buf, "%u\n", slub_cpu_partial(s));
}
static ssize_t cpu_partial_store(struct kmem_cache *s, const char *buf,
@@ -4936,7 +4943,7 @@ static ssize_t cpu_partial_store(struct kmem_cache *s, const char *buf,
if (objects && !kmem_cache_has_cpu_partial(s))
return -EINVAL;
- s->cpu_partial = objects;
+ slub_set_cpu_partial(s, objects);
flush_all(s);
return length;
}
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 2/3] mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
From: Wei Yang @ 2017-05-02 14:45 UTC (permalink / raw)
To: cl, penberg, rientjes, iamjoonsoo.kim, akpm, willy
Cc: linux-mm, linux-kernel, Wei Yang
In-Reply-To: <20170502144533.10729-1-richard.weiyang@gmail.com>
cpu_slab's field partial is used when CONFIG_SLUB_CPU_PARTIAL is set, which
means we can save a pointer's space on each cpu for every slub item.
This patch wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL and wrap its
sysfs too.
Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
---
v2: define slub_percpu_partial() to make code more elegant
---
include/linux/slub_def.h | 19 +++++++++++++++++++
mm/slub.c | 16 +++++++++-------
2 files changed, 28 insertions(+), 7 deletions(-)
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index ec13aab32647..f882a34bb9aa 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -41,12 +41,31 @@ struct kmem_cache_cpu {
void **freelist; /* Pointer to next available object */
unsigned long tid; /* Globally unique transaction id */
struct page *page; /* The slab from which we are allocating */
+#ifdef CONFIG_SLUB_CPU_PARTIAL
struct page *partial; /* Partially allocated frozen slabs */
+#endif
#ifdef CONFIG_SLUB_STATS
unsigned stat[NR_SLUB_STAT_ITEMS];
#endif
};
+#ifdef CONFIG_SLUB_CPU_PARTIAL
+#define slub_percpu_partial(c) ((c)->partial)
+
+#define slub_set_percpu_partial(c, p) \
+({ \
+ slub_percpu_partial(c) = (p)->next; \
+})
+
+#define slub_percpu_partial_read_once(c) READ_ONCE(slub_percpu_partial(c))
+#else
+#define slub_percpu_partial(c) NULL
+
+#define slub_set_percpu_partial(c, p)
+
+#define slub_percpu_partial_read_once(c) NULL
+#endif // CONFIG_SLUB_CPU_PARTIAL
+
/*
* Word size structure that can be atomically updated or read and that
* contains both the order and the number of objects that a slab of the
diff --git a/mm/slub.c b/mm/slub.c
index 7f4bc7027ed5..ae6166533261 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2302,7 +2302,7 @@ static bool has_cpu_slab(int cpu, void *info)
struct kmem_cache *s = info;
struct kmem_cache_cpu *c = per_cpu_ptr(s->cpu_slab, cpu);
- return c->page || c->partial;
+ return c->page || slub_percpu_partial(c);
}
static void flush_all(struct kmem_cache *s)
@@ -2568,9 +2568,9 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
new_slab:
- if (c->partial) {
- page = c->page = c->partial;
- c->partial = page->next;
+ if (slub_percpu_partial(c)) {
+ page = c->page = slub_percpu_partial(c);
+ slub_set_percpu_partial(c, page);
stat(s, CPU_PARTIAL_ALLOC);
c->freelist = NULL;
goto redo;
@@ -4760,7 +4760,7 @@ static ssize_t show_slab_objects(struct kmem_cache *s,
total += x;
nodes[node] += x;
- page = READ_ONCE(c->partial);
+ page = slub_percpu_partial_read_once(c);
if (page) {
node = page_to_nid(page);
if (flags & SO_TOTAL)
@@ -4988,7 +4988,8 @@ static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
int len;
for_each_online_cpu(cpu) {
- struct page *page = per_cpu_ptr(s->cpu_slab, cpu)->partial;
+ struct page *page =
+ slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
if (page) {
pages += page->pages;
@@ -5000,7 +5001,8 @@ static ssize_t slabs_cpu_partial_show(struct kmem_cache *s, char *buf)
#ifdef CONFIG_SMP
for_each_online_cpu(cpu) {
- struct page *page = per_cpu_ptr(s->cpu_slab, cpu) ->partial;
+ struct page *page =
+ slub_percpu_partial(per_cpu_ptr(s->cpu_slab, cpu));
if (page && len < PAGE_SIZE - 20)
len += sprintf(buf + len, " C%d=%d(%d)", cpu,
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 1/3] mm/slub: pack red_left_pad with another int to save a word
From: Wei Yang @ 2017-05-02 14:45 UTC (permalink / raw)
To: cl, penberg, rientjes, iamjoonsoo.kim, akpm, willy
Cc: linux-mm, linux-kernel, Wei Yang
In-Reply-To: <20170502144533.10729-1-richard.weiyang@gmail.com>
On 64bit arch, struct is 8-bytes aligned, so int will occupy a word if it
doesn't sits well.
This patch pack red_left_pad with reserved to save 8 bytes for struct
kmem_cache on a 64bit arch.
Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
---
include/linux/slub_def.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h
index 07ef550c6627..ec13aab32647 100644
--- a/include/linux/slub_def.h
+++ b/include/linux/slub_def.h
@@ -79,9 +79,9 @@ struct kmem_cache {
int inuse; /* Offset to metadata */
int align; /* Alignment */
int reserved; /* Reserved bytes at the end of slabs */
+ int red_left_pad; /* Left redzone padding size */
const char *name; /* Name (only for display!) */
struct list_head list; /* List of slab caches */
- int red_left_pad; /* Left redzone padding size */
#ifdef CONFIG_SYSFS
struct kobject kobj; /* For sysfs */
#endif
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 0/3] try to save some memory for kmem_cache in some cases
From: Wei Yang @ 2017-05-02 14:45 UTC (permalink / raw)
To: cl, penberg, rientjes, iamjoonsoo.kim, akpm, willy
Cc: linux-mm, linux-kernel, Wei Yang
kmem_cache is a frequently used data in kernel. During the code reading, I
found maybe we could save some space in some cases.
1. On 64bit arch, type int will occupy a word if it doesn't sit well.
2. cpu_slab->partial is just used when CONFIG_SLUB_CPU_PARTIAL is set
3. cpu_partial is just used when CONFIG_SLUB_CPU_PARTIAL is set, while just
save some space on 32bit arch.
v2:
define some macro to make the code more elegant
Wei Yang (3):
mm/slub: pack red_left_pad with another int to save a word
mm/slub: wrap cpu_slab->partial in CONFIG_SLUB_CPU_PARTIAL
mm/slub: wrap kmem_cache->cpu_partial in config
CONFIG_SLUB_CPU_PARTIAL
include/linux/slub_def.h | 34 ++++++++++++++++++-
mm/slub.c | 85 ++++++++++++++++++++++++++----------------------
2 files changed, 80 insertions(+), 39 deletions(-)
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Michal Hocko @ 2017-05-02 14:36 UTC (permalink / raw)
To: Balbir Singh
Cc: linux-mm, akpm, khandual, benh, aneesh.kumar, paulmck, srikar,
haren, jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <20170419075242.29929-1-bsingharora@gmail.com>
On Wed 19-04-17 17:52:38, Balbir Singh wrote:
> This is a request for comments on the discussed approaches
> for coherent memory at mm-summit (some of the details are at
> https://lwn.net/Articles/717601/). The latest posted patch
> series is at https://lwn.net/Articles/713035/. I am reposting
> this as RFC, Michal Hocko suggested using HMM for CDM, but
> we believe there are stronger reasons to use the NUMA approach.
> The earlier patches for Coherent Device memory were implemented
> and designed by Anshuman Khandual.
>
> Jerome posted HMM-CDM at https://lwn.net/Articles/713035/.
> The patches do a great deal to enable CDM with HMM, but we
> still believe that HMM with CDM is not a natural way to
> represent coherent device memory and the mm will need
> to be audited and enhanced for it to even work.
>
> With HMM we'll see ZONE_DEVICE pages mapped into
> user space and that would mean a thorough audit of all code
> paths to make sure we are ready for such a use case and enabling
> those use cases, like with HMM CDM patch 1, which changes
> move_pages() and migration paths. I've done a quick
> evaluation to check for features and found limitationd around
> features like migration (page cache
> migration), fault handling to the right location
> (direct page cache allocation in the coherent memory), mlock
> handling, RSS accounting, memcg enforcement for pages not on LRU, etc.
Are those problems not viable to solve?
[...]
> Introduction
>
> CDM device memory is cache coherent with system memory and we would like
> this to show up as a NUMA node, however there are certain algorithms
> that might not be currently suitable for N_COHERENT_MEMORY
>
> 1. AutoNUMA balancing
OK, I can see a reason for that but theoretically the same applies to
cpu less numa nodes in general, no?
> 2. kswapd reclaim
How is the memory reclaim handled then? How are users expected to handle
OOM situation?
> The reason for exposing this device memory as NUMA is to simplify
> the programming model, where memory allocation via malloc() or
> mmap() for example would seamlessly work across both kinds of
> memory. Since we expect the size of device memory to be smaller
> than system RAM, we would like to control the allocation of such
> memory. The proposed mechanism reuses nodemasks and explicit
> specification of the coherent node in the nodemask for allocation
> from device memory. This implementation also allows for kernel
> level allocation via __GFP_THISNODE and existing techniques
> such as page migration to work.
so it basically resembles isol_cpus except for memory, right. I believe
scheduler people are more than unhappy about this interface...
Anyway, I consider CPUless nodes a dirty hack (especially when I see
them mostly used with poorly configured LPARs where no CPUs are left for
a particular memory). Now this is trying to extend this concept even
further to a memory which is not reclaimable by the kernel and requires
an explicit and cooperative memory reclaim from userspace. How is this
going to work? The memory also has a different reliability properties
from RAM which user space doesn't have any clue about from the NUMA
properties exported. Or am I misunderstanding it? That all sounds quite
scary to me.
I very much agree with the last email from Mel and I would really like
to see how would a real application benefit from these nodes.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
From: Luiz Capitulino @ 2017-05-02 14:28 UTC (permalink / raw)
To: Marcelo Tosatti
Cc: linux-kernel, linux-mm, Rik van Riel, Linux RT Users, cl,
cmetcalf
In-Reply-To: <20170425135846.203663532@redhat.com>
On Tue, 25 Apr 2017 10:57:19 -0300
Marcelo Tosatti <mtosatti@redhat.com> wrote:
> The per-CPU vmstat worker is a problem on -RT workloads (because
> ideally the CPU is entirely reserved for the -RT app, without
> interference). The worker transfers accumulated per-CPU
> vmstat counters to global counters.
This is a problem for non-RT too. Any task pinned to an isolated
CPU that doesn't want to be ever interrupted will be interrupted
by the vmstat kworker.
> To resolve the problem, create two tunables:
>
> * Userspace configurable per-CPU vmstat threshold: by default the
> VM code calculates the size of the per-CPU vmstat arrays. This
> tunable allows userspace to configure the values.
>
> * Userspace configurable per-CPU vmstat worker: allow disabling
> the per-CPU vmstat worker.
I have several questions about the tunables:
- What does the vmstat_threshold value mean? What are the implications
of changing this value? What's the difference in choosing 1, 2, 3
or 500?
- If the purpose of having vmstat_threshold is to allow disabling
the vmstat kworker, why can't the kernel pick a value automatically?
- What are the implications of disabling the vmstat kworker? Will vm
stats still be collected someway or will it be completely off for
the CPU?
Also, shouldn't this patch be split into two?
> The patch below contains documentation which describes the tunables
> in more detail.
>
> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
>
> ---
> Documentation/vm/vmstat_thresholds.txt | 38 +++++
> mm/vmstat.c | 248 +++++++++++++++++++++++++++++++--
> 2 files changed, 272 insertions(+), 14 deletions(-)
>
> Index: linux-2.6-git-disable-vmstat-worker/mm/vmstat.c
> ===================================================================
> --- linux-2.6-git-disable-vmstat-worker.orig/mm/vmstat.c 2017-04-25 07:39:13.941019853 -0300
> +++ linux-2.6-git-disable-vmstat-worker/mm/vmstat.c 2017-04-25 10:44:51.581977296 -0300
> @@ -91,8 +91,17 @@
> EXPORT_SYMBOL(vm_zone_stat);
> EXPORT_SYMBOL(vm_node_stat);
>
> +struct vmstat_uparam {
> + atomic_t vmstat_work_enabled;
> + atomic_t user_stat_thresh;
> +};
> +
> +static DEFINE_PER_CPU(struct vmstat_uparam, vmstat_uparam);
> +
> #ifdef CONFIG_SMP
>
> +#define MAX_THRESHOLD 125
> +
> int calculate_pressure_threshold(struct zone *zone)
> {
> int threshold;
> @@ -110,9 +119,9 @@
> threshold = max(1, (int)(watermark_distance / num_online_cpus()));
>
> /*
> - * Maximum threshold is 125
> + * Maximum threshold is MAX_THRESHOLD == 125
> */
> - threshold = min(125, threshold);
> + threshold = min(MAX_THRESHOLD, threshold);
>
> return threshold;
> }
> @@ -188,15 +197,31 @@
> threshold = calculate_normal_threshold(zone);
>
> for_each_online_cpu(cpu) {
> - int pgdat_threshold;
> + int pgdat_threshold, ustat_thresh;
> + struct vmstat_uparam *vup;
>
> - per_cpu_ptr(zone->pageset, cpu)->stat_threshold
> - = threshold;
> + struct per_cpu_nodestat __percpu *pcp;
> + struct per_cpu_pageset *p;
> +
> + p = per_cpu_ptr(zone->pageset, cpu);
> +
> + vup = &per_cpu(vmstat_uparam, cpu);
> + ustat_thresh = atomic_read(&vup->user_stat_thresh);
> +
> + if (ustat_thresh)
> + p->stat_threshold = ustat_thresh;
> + else
> + p->stat_threshold = threshold;
> +
> + pcp = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu);
>
> /* Base nodestat threshold on the largest populated zone. */
> - pgdat_threshold = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu)->stat_threshold;
> - per_cpu_ptr(pgdat->per_cpu_nodestats, cpu)->stat_threshold
> - = max(threshold, pgdat_threshold);
> + pgdat_threshold = pcp->stat_threshold;
> + if (ustat_thresh)
> + pcp->stat_threshold = ustat_thresh;
> + else
> + pcp->stat_threshold = max(threshold,
> + pgdat_threshold);
> }
>
> /*
> @@ -226,9 +251,24 @@
> continue;
>
> threshold = (*calculate_pressure)(zone);
> - for_each_online_cpu(cpu)
> + for_each_online_cpu(cpu) {
> + int t, ustat_thresh;
> + struct vmstat_uparam *vup;
> +
> + vup = &per_cpu(vmstat_uparam, cpu);
> + ustat_thresh = atomic_read(&vup->user_stat_thresh);
> + t = threshold;
> +
> + /*
> + * min because pressure could cause
> + * calculate_pressure'ed value to be smaller.
> + */
> + if (ustat_thresh)
> + t = min(threshold, ustat_thresh);
> +
> per_cpu_ptr(zone->pageset, cpu)->stat_threshold
> - = threshold;
> + = t;
> + }
> }
> }
>
> @@ -1567,6 +1607,9 @@
> long val;
> int err;
> int i;
> + int cpu;
> + struct work_struct __percpu *works;
> + static struct cpumask has_work;
>
> /*
> * The regular update, every sysctl_stat_interval, may come later
> @@ -1580,9 +1623,31 @@
> * transiently negative values, report an error here if any of
> * the stats is negative, so we know to go looking for imbalance.
> */
> - err = schedule_on_each_cpu(refresh_vm_stats);
> - if (err)
> - return err;
> +
> + works = alloc_percpu(struct work_struct);
> + if (!works)
> + return -ENOMEM;
> +
> + cpumask_clear(&has_work);
> + get_online_cpus();
> +
> + for_each_online_cpu(cpu) {
> + struct work_struct *work = per_cpu_ptr(works, cpu);
> + struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
> +
> + if (atomic_read(&vup->vmstat_work_enabled)) {
> + INIT_WORK(work, refresh_vm_stats);
> + schedule_work_on(cpu, work);
> + cpumask_set_cpu(cpu, &has_work);
> + }
> + }
> +
> + for_each_cpu(cpu, &has_work)
> + flush_work(per_cpu_ptr(works, cpu));
> +
> + put_online_cpus();
> + free_percpu(works);
> +
> for (i = 0; i < NR_VM_ZONE_STAT_ITEMS; i++) {
> val = atomic_long_read(&vm_zone_stat[i]);
> if (val < 0) {
> @@ -1674,6 +1739,10 @@
> /* Check processors whose vmstat worker threads have been disabled */
> for_each_online_cpu(cpu) {
> struct delayed_work *dw = &per_cpu(vmstat_work, cpu);
> + struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
> +
> + if (atomic_read(&vup->vmstat_work_enabled) == 0)
> + continue;
>
> if (!delayed_work_pending(dw) && need_update(cpu))
> queue_delayed_work_on(cpu, mm_percpu_wq, dw, 0);
> @@ -1696,6 +1765,135 @@
> round_jiffies_relative(sysctl_stat_interval));
> }
>
> +#ifdef CONFIG_SYSFS
> +
> +static ssize_t vmstat_worker_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + unsigned int cpu = dev->id;
> + struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
> +
> + return sprintf(buf, "%d\n", atomic_read(&vup->vmstat_work_enabled));
> +}
> +
> +static ssize_t vmstat_worker_store(struct device *dev,
> + struct device_attribute *attr,
> + const char *buf, size_t count)
> +{
> + int ret, val;
> + struct vmstat_uparam *vup;
> + unsigned int cpu = dev->id;
> +
> + ret = sscanf(buf, "%d", &val);
> + if (ret != 1 || val > 1 || val < 0)
> + return -EINVAL;
> +
> + preempt_disable();
> +
> + if (cpu_online(cpu)) {
> + vup = &per_cpu(vmstat_uparam, cpu);
> + atomic_set(&vup->vmstat_work_enabled, val);
> + } else
> + count = -EINVAL;
> +
> + preempt_enable();
> +
> + return count;
> +}
> +
> +static ssize_t vmstat_thresh_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + int ret;
> + struct vmstat_uparam *vup;
> + unsigned int cpu = dev->id;
> +
> + preempt_disable();
> +
> + vup = &per_cpu(vmstat_uparam, cpu);
> + ret = sprintf(buf, "%d\n", atomic_read(&vup->user_stat_thresh));
> +
> + preempt_enable();
> +
> + return ret;
> +}
> +
> +static ssize_t vmstat_thresh_store(struct device *dev,
> + struct device_attribute *attr,
> + const char *buf, size_t count)
> +{
> + int ret, val;
> + unsigned int cpu = dev->id;
> + struct vmstat_uparam *vup;
> +
> + ret = sscanf(buf, "%d", &val);
> + if (ret != 1 || val < 1 || val > MAX_THRESHOLD)
> + return -EINVAL;
> +
> + preempt_disable();
> +
> + if (cpu_online(cpu)) {
> + vup = &per_cpu(vmstat_uparam, cpu);
> + atomic_set(&vup->user_stat_thresh, val);
> + } else
> + count = -EINVAL;
> +
> + preempt_enable();
> +
> + return count;
> +}
> +
> +struct device_attribute vmstat_worker_attr =
> + __ATTR(vmstat_worker, 0644, vmstat_worker_show, vmstat_worker_store);
> +
> +struct device_attribute vmstat_threshold_attr =
> + __ATTR(vmstat_threshold, 0644, vmstat_thresh_show, vmstat_thresh_store);
> +
> +static struct attribute *vmstat_attrs[] = {
> + &vmstat_worker_attr.attr,
> + &vmstat_threshold_attr.attr,
> + NULL
> +};
> +
> +static struct attribute_group vmstat_attr_group = {
> + .attrs = vmstat_attrs,
> + .name = "vmstat"
> +};
> +
> +static int vmstat_thresh_cpu_online(unsigned int cpu)
> +{
> + struct device *dev = get_cpu_device(cpu);
> + int ret;
> +
> + ret = sysfs_create_group(&dev->kobj, &vmstat_attr_group);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> +static int vmstat_thresh_cpu_down_prep(unsigned int cpu)
> +{
> + struct device *dev = get_cpu_device(cpu);
> +
> + sysfs_remove_group(&dev->kobj, &vmstat_attr_group);
> + return 0;
> +}
> +
> +static void init_vmstat_sysfs(void)
> +{
> + int cpu;
> +
> + for_each_possible_cpu(cpu) {
> + struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
> +
> + atomic_set(&vup->user_stat_thresh, 0);
> + atomic_set(&vup->vmstat_work_enabled, 1);
> + }
> +}
> +
> +#endif /* CONFIG_SYSFS */
> +
> static void __init init_cpu_node_state(void)
> {
> int node;
> @@ -1723,9 +1921,13 @@
> {
> const struct cpumask *node_cpus;
> int node;
> + struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
>
> node = cpu_to_node(cpu);
>
> + atomic_set(&vup->user_stat_thresh, 0);
> + atomic_set(&vup->vmstat_work_enabled, 1);
> +
> refresh_zone_stat_thresholds();
> node_cpus = cpumask_of_node(node);
> if (cpumask_weight(node_cpus) > 0)
> @@ -1735,7 +1937,7 @@
> return 0;
> }
>
> -#endif
> +#endif /* CONFIG_SMP */
>
> struct workqueue_struct *mm_percpu_wq;
>
> @@ -1772,6 +1974,24 @@
> #endif
> }
>
> +static int __init init_mm_internals_late(void)
> +{
> +#ifdef CONFIG_SYSFS
> + int ret;
> +
> + init_vmstat_sysfs();
> +
> + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "mm/vmstat_thresh:online",
> + vmstat_thresh_cpu_online,
> + vmstat_thresh_cpu_down_prep);
> + if (ret < 0)
> + pr_err("vmstat_thresh: failed to register 'online' hotplug state\n");
> +#endif
> + return 0;
> +}
> +
> +late_initcall(init_mm_internals_late);
> +
> #if defined(CONFIG_DEBUG_FS) && defined(CONFIG_COMPACTION)
>
> /*
> Index: linux-2.6-git-disable-vmstat-worker/Documentation/vm/vmstat_thresholds.txt
> ===================================================================
> --- /dev/null 1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-git-disable-vmstat-worker/Documentation/vm/vmstat_thresholds.txt 2017-04-25 08:46:25.237395070 -0300
> @@ -0,0 +1,38 @@
> +Userspace configurable vmstat thresholds
> +========================================
> +
> +This document describes the tunables to control
> +per-CPU vmstat threshold and per-CPU vmstat worker
> +thread.
> +
> +/sys/devices/system/cpu/cpuN/vmstat/vmstat_threshold:
> +
> +This file contains the per-CPU vmstat threshold.
> +This value is the maximum that a single per-CPU vmstat statistic
> +can accumulate before transferring to the global counters.
> +
> +A value of 0 indicates that the value is set
> +by the in kernel algorithm.
> +
> +A value different than 0 indicates that particular
> +value is used for vmstat_threshold.
> +
> +/sys/devices/system/cpu/cpuN/vmstat/vmstat_worker:
> +
> +Enable/disable the per-CPU vmstat worker.
> +
> +Usage example:
> +=============
> +
> +To disable vmstat_update worker for cpu1:
> +
> +cd /sys/devices/system/cpu/cpu0/vmstat/
> +
> +# echo 1 > vmstat_threshold
> +# echo 0 > vmstat_worker
> +
> +Setting vmstat_threshold to 1 means the per-CPU
> +vmstat statistics will not be out-of-date
> +for CPU 1.
> +
> +
>
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* RE: [PATCH 0/2] mm/memcontrol: fix reclaim bugs in mem_cgroup_iter
From: Christopherson, Sean J @ 2017-05-02 14:20 UTC (permalink / raw)
To: 'Michal Hocko'
Cc: hannes@cmpxchg.org, vdavydov.dev@gmail.com,
cgroups@vger.kernel.org, linux-mm@kvack.org
In-Reply-To: <20170502140357.GL14593@dhcp22.suse.cz>
Michal Hocko <mhocko@kernel.org> wrote:
> On Fri 28-04-17 14:55:45, Sean Christopherson wrote:
> > This patch set contains two bug fixes for mem_cgroup_iter(). The bugs
> > were found by code inspection and were confirmed via synthetic testing
> > that forcefully setup the failing conditions.
>
> I assume that you added some artificial sleeps to make those races more
> probable, right? Or did you manage to hit those issue solely from the
> userspace? I will have a look at those patches. It has been some time
> since I've had it cached. It is pretty subtle code so I would like to
> understand the urgency before I dive into this further.
> --
> Michal Hocko
> SUSE Labs
The code to prove the bugs is completely artificial, it's basically a
unit test for mem_cgroup_iter() that uses a thread barrier to all but
guarantee two threads will call mem_cgroup_iter() simultaneously. I
haven't even attempted to hit this via the actual userspace flow.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free
From: Marc MERLIN @ 2017-05-02 14:15 UTC (permalink / raw)
To: Michal Hocko, Tetsuo Handa
Cc: Linus Torvalds, Vlastimil Babka, linux-mm, LKML, Joonsoo Kim,
Tejun Heo, Greg Kroah-Hartman
In-Reply-To: <20170502074432.GB14593@dhcp22.suse.cz>
On Tue, May 02, 2017 at 09:44:33AM +0200, Michal Hocko wrote:
> On Mon 01-05-17 21:12:35, Marc MERLIN wrote:
> > Howdy,
> >
> > Well, sadly, the problem is more or less back is 4.11.0. The system doesn't really
> > crash but it goes into an infinite loop with
> > [34776.826800] BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 33s!
> > More logs: https://pastebin.com/YqE4riw0
>
> I am seeing a lot of traces where tasks is waiting for an IO. I do not
> see any OOM report there. Why do you believe this is an OOM killer
> issue?
Good question. This is a followup of the problem I had in 4.8.8 until I
got a patch to fix the issue. Then, it used to OOM and later, to pile up
I/O tasks like this.
Now it doesn't OOM anymore, but tasks still pile up.
I temporarily fixed the issue by doing this:
gargamel:~# echo 0 > /proc/sys/vm/dirty_ratio
gargamel:~# echo 0 > /proc/sys/vm/dirty_background_ratio
of course my performance is abysmal now, but I can at least run btrfs
scrub without piling up enough IO to deadlock the system.
On Tue, May 02, 2017 at 07:44:47PM +0900, Tetsuo Handa wrote:
> > Any idea what I should do next?
>
> Maybe you can try collecting list of all in-flight allocations with backtraces
> using kmallocwd patches at
> http://lkml.kernel.org/r/1489578541-81526-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp
> and http://lkml.kernel.org/r/201704272019.JEH26057.SHFOtMLJOOVFQF@I-love.SAKURA.ne.jp
> which also tracks mempool allocations.
> (Well, the
>
> - cond_resched();
> + //cond_resched();
>
> change in the latter patch would not be preferable.)
Thanks. I can give that a shot as soon as my current scrub is done, it
may take another 12 to 24H at this rate.
In the meantimne, as explained above, not allowing any dirty VM has
worked around the problem (Linus pointed out to me in the original
thread that on a lightly loaded 24GB system, even 1 or 2% could still be
a lot of memory for requests to pile up in and cause issues in
degenerative cases like mine).
Now I'm still curious what changed betweeen 4.8.8 + custom patches and 4.11 to cause
this.
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 0/2] mm/memcontrol: fix reclaim bugs in mem_cgroup_iter
From: Michal Hocko @ 2017-05-02 14:03 UTC (permalink / raw)
To: Sean Christopherson; +Cc: hannes, vdavydov.dev, cgroups, linux-mm
In-Reply-To: <1493416547-19212-1-git-send-email-sean.j.christopherson@intel.com>
On Fri 28-04-17 14:55:45, Sean Christopherson wrote:
> This patch set contains two bug fixes for mem_cgroup_iter(). The bugs
> were found by code inspection and were confirmed via synthetic testing
> that forcefully setup the failing conditions.
I assume that you added some artificial sleeps to make those races more
probable, right? Or did you manage to hit those issue solely from the
userspace? I will have a look at those patches. It has been some time
since I've had it cached. It is pretty subtle code so I would like to
understand the urgency before I dive into this further.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] mm, vmalloc: properly track vmalloc users
From: Michal Hocko @ 2017-05-02 13:46 UTC (permalink / raw)
To: Andrew Morton; +Cc: Vlastimil Babka, linux-mm, LKML, Michal Hocko
From: Michal Hocko <mhocko@suse.com>
__vmalloc_node_flags used to be static inline but this has changed by
"mm: introduce kv[mz]alloc helpers" because kvmalloc_node needs to use
it as well and the code is outside of the vmalloc proper. I haven't
realized that changing this will lead to a subtle bug though. The
function is responsible to track the caller as well. This caller is
then printed by /proc/vmallocinfo. If __vmalloc_node_flags is not inline
then we would get only direct users of __vmalloc_node_flags as callers
(e.g. v[mz]alloc) which reduces usefulness of this debugging feature
considerably. It simply doesn't help to see that the given range belongs
to vmalloc as a caller:
0xffffc90002c79000-0xffffc90002c7d000 16384 vmalloc+0x16/0x18 pages=3 vmalloc N0=3
0xffffc90002c81000-0xffffc90002c85000 16384 vmalloc+0x16/0x18 pages=3 vmalloc N1=3
0xffffc90002c8d000-0xffffc90002c91000 16384 vmalloc+0x16/0x18 pages=3 vmalloc N1=3
0xffffc90002c95000-0xffffc90002c99000 16384 vmalloc+0x16/0x18 pages=3 vmalloc N1=3
We really want to catch the _caller_ of the vmalloc function. Fix this
issue by making __vmalloc_node_flags static inline again.
Signed-off-by: Michal Hocko <mhocko@suse.com>
---
Hi Andrew,
this is a follow up fix for mm-introduce-kvalloc-helpers.patch currently
sitting in your mmotm tree. You can either fold this into it or just
keep it on its own which I would consider slightly better for the
reference. Anyway it would be great if it could hit the Linus' tree
along with the original patch.
I am sorry, I should have noticed this earlier.
include/linux/vmalloc.h | 19 +++++++++++++++++++
mm/vmalloc.c | 12 +-----------
2 files changed, 20 insertions(+), 11 deletions(-)
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 46991ad3ddd5..0328ce003992 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -6,6 +6,7 @@
#include <linux/list.h>
#include <linux/llist.h>
#include <asm/page.h> /* pgprot_t */
+#include <asm/pgtable.h> /* PAGE_KERNEL */
#include <linux/rbtree.h>
struct vm_area_struct; /* vma defining user mapping in mm_types.h */
@@ -80,7 +81,25 @@ extern void *__vmalloc_node_range(unsigned long size, unsigned long align,
unsigned long start, unsigned long end, gfp_t gfp_mask,
pgprot_t prot, unsigned long vm_flags, int node,
const void *caller);
+#ifndef CONFIG_MMU
extern void *__vmalloc_node_flags(unsigned long size, int node, gfp_t flags);
+#else
+extern void *__vmalloc_node(unsigned long size, unsigned long align,
+ gfp_t gfp_mask, pgprot_t prot,
+ int node, const void *caller);
+
+/*
+ * We really want to have this inlined due to caller tracking. This
+ * function is used by the highlevel vmalloc apis and so we want to track
+ * their callers and inlining will achieve that.
+ */
+static inline void *__vmalloc_node_flags(unsigned long size,
+ int node, gfp_t flags)
+{
+ return __vmalloc_node(size, 1, flags, PAGE_KERNEL,
+ node, __builtin_return_address(0));
+}
+#endif
extern void vfree(const void *addr);
extern void vfree_atomic(const void *addr);
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 65912eb93a2c..201eb20ba96a 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -1649,9 +1649,6 @@ void *vmap(struct page **pages, unsigned int count,
}
EXPORT_SYMBOL(vmap);
-static void *__vmalloc_node(unsigned long size, unsigned long align,
- gfp_t gfp_mask, pgprot_t prot,
- int node, const void *caller);
static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
pgprot_t prot, int node)
{
@@ -1794,7 +1791,7 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align,
* with mm people.
*
*/
-static void *__vmalloc_node(unsigned long size, unsigned long align,
+void *__vmalloc_node(unsigned long size, unsigned long align,
gfp_t gfp_mask, pgprot_t prot,
int node, const void *caller)
{
@@ -1809,13 +1806,6 @@ void *__vmalloc(unsigned long size, gfp_t gfp_mask, pgprot_t prot)
}
EXPORT_SYMBOL(__vmalloc);
-void *__vmalloc_node_flags(unsigned long size,
- int node, gfp_t flags)
-{
- return __vmalloc_node(size, 1, flags, PAGE_KERNEL,
- node, __builtin_return_address(0));
-}
-
/**
* vmalloc - allocate virtually contiguous memory
* @size: allocation size
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Igor Stoppa @ 2017-05-02 13:41 UTC (permalink / raw)
To: Michal Hocko, Vlastimil Babka
Cc: Joonsoo Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
mgorman, Laura Abbott, Minchan Kim, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K . V, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <20170502130326.GJ14593@dhcp22.suse.cz>
On 02/05/17 16:03, Michal Hocko wrote:
> I can imagine that we could make ZONE_CMA configurable in a way that
> only very well defined use cases would be supported so that we can save
> page flags space. But this alone sounds like a maintainability nightmare
> to me. Especially when I consider ZONE_DMA situation. There is simply
> not an easy way to find out whether my HW really needs DMA zone or
> not. Most probably not but it still is configured and hidden behind
> config ZONE_DMA
> bool "DMA memory allocation support" if EXPERT
> default y
> help
> DMA memory allocation support allows devices with less than 32-bit
> addressing to allocate within the first 16MB of address space.
> Disable if no such devices will be used.
>
> If unsure, say Y.
>
> Are we really ready to add another thing like that? How are distribution
> kernels going to handle that?
In practice there are 2 quite opposite scenarios:
- distros that try to cater to (almost) everyone and are constrained in
what they can leave out
- ad-hoc builds (like Android, but also IoT) where the HW is *very* well
known upfront, because it's probably even impossible to make any change
that doesn't involved a rework station.
So maybe the answer is to not have only EXPERT, but rather DISTRO/CUSTOM
with the implications these can bring.
A generic build would assume to be a DISTRO type, but something else, of
more embedded persuasion, could do otherwise.
ZONE_DMA / ZONE_DMA32 actually seem to be perfect candidates for being
replaced by something else, when unused, as I proposed on Friday:
http://marc.info/?l=linux-mm&m=149337033630993&w=2
It might still be that only some cases would be upstreamable, even after
these changes.
But at least some of those might be useful also for non-Android/ non-IoT
scenarios.
---
igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Michal Hocko @ 2017-05-02 13:32 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Andrew Morton, Rik van Riel, Johannes Weiner, mgorman,
Laura Abbott, Minchan Kim, Marek Szyprowski, Michal Nazarewicz,
Aneesh Kumar K . V, Vlastimil Babka, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <20170502040129.GA27335@js1304-desktop>
On Tue 02-05-17 13:01:32, Joonsoo Kim wrote:
> On Thu, Apr 27, 2017 at 05:06:36PM +0200, Michal Hocko wrote:
[...]
> > I see this point and I agree that using a specific zone might be a
> > _nicer_ solution in the end but you have to consider another aspects as
> > well. The main one I am worried about is a long term maintainability.
> > We are really out of page flags and consuming one for a rather specific
> > usecase is not good. Look at ZONE_DMA. I am pretty sure that almost
> > no sane HW needs 16MB zone anymore, yet we have hard time to get rid
> > of it and so we have that memory laying around unused all the time
> > and blocking one page flag bit. CMA falls into a similar category
> > AFAIU. I wouldn't be all that surprised if a future HW will not need CMA
> > allocations in few years, yet we will have to fight to get rid of it
> > like we do with ZONE_DMA. And not only that. We will also have to fight
> > finding page flags for other more general usecases in the meantime.
>
> This maintenance problem is inherent. This problem exists even if we
> uses MIGRATETYPE approach. We cannot remove many hooks for CMA if a
> future HW will not need CMA allocation in few years. The only
> difference is that one takes single zone bit only for CMA user and the
> other approach takes many hooks that we need to take care about it all
> the time.
And I consider this a big difference. Because while hooks are not nice
they will affect CMA users (in a sense of bugs/performance etc.). While
an additional bit consumed will affect potential future and more generic
features.
[...]
> > I believe that the overhead in the hot path is not such a big deal. We
> > have means to make it 0 when CMA is not used by jumplabels. I assume
> > that the vast majority of systems will not use CMA. Those systems which
> > use CMA should be able to cope with some slight overhead IMHO.
>
> Please don't underestimate number of CMA user. Most of android device
> uses CMA. So, there would be more devices using CMA than the server
> not using CMA. They also have a right to experience the best performance.
This is not a fair comparison, though. Android development model is much
more faster and tend to not care about future maintainability at all. I
do not know about any android device that would run on a clean vanilla
kernel because vendors simply do not care enough (or have time) to put
the code into a proper shape to upstream it. I understand that this
model might work quite well for rapidly changing and moving mobile or
IoT segment but it is not the greatest fit to motivate the core kernel
subsystem development. We are not in the drivers space!
[...]
> > This looks like a nice clean up. Those ifdefs are ugly as hell. One
> > could argue that some of that could be cleaned up by simply adding some
> > helpers (with a jump label to reduce the overhead), though. But is this
> > really strong enough reason to bring the whole zone in? I am not really
> > convinced to be honest.
>
> Please don't underestimate the benefit of this patchset.
> I have said that we need *more* hooks to fix all the problems.
>
> And, please think that this code removal is not only code removal but
> also concept removal. With this removing, we don't need to consider
> ALLOC_CMA for alloc_flags when calling zone_watermark_ok(). There are
> many bugs on it and it still remains. We don't need to consider
> pageblock migratetype when handling pageblock migratetype. We don't
> need to take a great care about calculating the number of CMA
> freepages.
And all this can be isolated to CMA specific hooks with mostly minimum
impact to most users. I hear you saying that zone approach is more natural
and I would agree if we wouldn't have to care about the number of zones.
> > [...]
> >
> > > > Please do _not_ take this as a NAK from me. At least not at this time. I
> > > > am still trying to understand all the consequences but my intuition
> > > > tells me that building on top of highmem like approach will turn out to
> > > > be problematic in future (as we have already seen with the highmem and
> > > > movable zones) so this needs a very prudent consideration.
> > >
> > > I can understand that you are prudent to this issue. However, it takes more
> > > than two years and many people already expressed that ZONE approach is the
> > > way to go.
> >
> > I can see a single Acked-by and one Reviewed-by. It would be much more
> > convincing to see much larger support. Do not take me wrong I am not
> > trying to undermine the feedback so far but we should be clear about one
> > thing. CMA is mostly motivated by the industry which tries to overcome
> > HW limitations which can change in future very easily. I would rather
> > see good enough solution for something like that than a nicer solution
> > which is pushing additional burden on more general usecases.
>
> First of all, current MIGRATETYPE approach isn't good enough to me.
> They caused too many problems and there are many remanining problems.
> It will causes maintenance issue for a long time.
>
> And, although good enough solution can be better than nicer solution
> in some cases, it looks like current situation isn't that case.
> There is a single reason, saving page flag bit, to support good enough
> solution.
>
> I'd like to ask reversly. Is this a enough reason to make CMA user to
> suffer from bugs?
No, but I haven't heard any single argument that those bugs are
impossible to fix with the current approach. They might be harder to fix
but if I can chose between harder for CMA and harder for other more
generic HW independent features I will go for the first one. And do not
take me wrong, I have nothing against CMA as such. It solves a real life
problem. I just believe it doesn't deserve to consume a new bit in page
flags because that is just too scarce resource.
Thanks!
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Jerome Glisse @ 2017-05-02 13:22 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dan Williams, Ingo Molnar, linux-kernel@vger.kernel.org, Linux MM,
Ingo Molnar, Andrew Morton, Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <20170502113746.5ybuix3lnvlk7kxt@node.shutemov.name>
On Tue, May 02, 2017 at 02:37:46PM +0300, Kirill A. Shutemov wrote:
> On Mon, May 01, 2017 at 09:55:48AM -0400, Jerome Glisse wrote:
> > On Mon, May 01, 2017 at 01:23:59PM +0300, Kirill A. Shutemov wrote:
> > > On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
> > > > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
> > > > > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
> > > > > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
> > > > > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
> > > > > > > change ZONE_DEVICE pages to not have an elevated reference count when
> > > > > > > they are created so you can keep the HMM references out of the mm hot
> > > > > > > path?
> > > > > >
> > > > > > 100% sure on that :) I need to callback into driver for 2->1 transition
> > > > > > no way around that. If we change ZONE_DEVICE to not have an elevated
> > > > > > reference count that you need to make a lot more change to mm so that
> > > > > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
> > > > > > to make change to be sure that ZONE_DEVICE page never endup in one of
> > > > > > the path that try to put them back on lru. There is a lot of place that
> > > > > > would need to be updated and it would be highly intrusive and add a
> > > > > > lot of special cases to other hot code path.
> > > > >
> > > > > Could you explain more on where the requirement comes from or point me to
> > > > > where I can read about this.
> > > > >
> > > >
> > > > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> > > > in _any_ vma. So i need to know when a page is freed ie either as result of
> > > > unmap, exit or migration or anything that would free the memory. For zone
> > > > device a page is free once its refcount reach 1 so i need to catch refcount
> > > > transition from 2->1
> > >
> > > What if we would rework zone device to have pages with refcount 0 at
> > > start?
> >
> > That is a _lot_ of work from top of my head because it would need changes
> > to a lot of places and likely more hot code path that simply adding some-
> > thing to put_page() note that i only need something in put_page() i do not
> > need anything in the get page path. Is adding a conditional branch for
> > HMM pages in put_page() that much of a problem ?
>
> Well, it gets inlined everywhere. Removing zone_device code from
> get_page() and put_page() saved non-trivial ~140k in vmlinux for
> allyesconfig.
>
> Re-introducing part this bloat would be unfortunate.
>
> > > > This is the only way i can inform the device that the page is now free. See
> > > >
> > > > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
> > > >
> > > > There is _no_ way around that.
> > >
> > > I'm still not convinced that it's impossible.
> > >
> > > Could you describe lifecycle for pages in case of HMM?
> >
> > Process malloc something, end it over to some function in the program
> > that use the GPU that function call GPU API (OpenCL, CUDA, ...) that
> > trigger a migration to device memory.
> >
> > So in the kernel you get a migration like any existing migration,
> > original page is unmap, if refcount is all ok (no pin) then a device
> > page is allocated and thing are migrated to device memory.
> >
> > What happen after is unknown. Either userspace/kernel driver decide
> > to migrate back to system memory, either there is an munmap, either
> > there is a CPU page fault, ... So from that point on the device page
> > as the exact same life as a regular page.
> >
> > Above i describe the migrate case, but you can also have new memory
> > allocation that directly allocate device memory. For instance if the
> > GPU do a page fault on an address that isn't back by anything then
> > we can directly allocate a device page. No migration involve in that
> > case.
> >
> > HMM pages are like any other pages in most respect. Exception are:
> > - no GUP
>
> Hm. How do you exclude GUP? And why is it required?
Well it is not forbiden it just can not happen simply because as device
memory is not accessible by CPU then the corresponding CPU page table
entry is a special entry and thus GUP trigger a page fault that migrate
thing back to regular memory.
The why is simply because we need to always be able to migrate back to
regular memory as device memory is not accessible by the CPU. So we can
not allow anyone to pin it.
Cheers,
Jerome
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH] mm/nobootmem: return 0 when start_pfn equals end_pfn
From: Wei Yang @ 2017-05-02 13:11 UTC (permalink / raw)
To: akpm; +Cc: linux-mm, linux-kernel, Wei Yang
When start_pfn equals end_pfn, __free_pages_memory() takes no effect and
__free_memory_core() will finally return (end_pfn - start_pfn) = 0.
This patch returns 0 directly when start_pfn equals end_pfn.
Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
---
mm/nobootmem.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/nobootmem.c b/mm/nobootmem.c
index 487dad610731..36454d0f96ee 100644
--- a/mm/nobootmem.c
+++ b/mm/nobootmem.c
@@ -118,7 +118,7 @@ static unsigned long __init __free_memory_core(phys_addr_t start,
unsigned long end_pfn = min_t(unsigned long,
PFN_DOWN(end), max_low_pfn);
- if (start_pfn > end_pfn)
+ if (start_pfn >= end_pfn)
return 0;
__free_pages_memory(start_pfn, end_pfn);
--
2.11.0
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Michal Hocko @ 2017-05-02 13:03 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Joonsoo Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
mgorman, Laura Abbott, Minchan Kim, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K . V, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team
In-Reply-To: <32ac1107-14a3-fdff-ad48-0e246fec704f@suse.cz>
On Tue 02-05-17 10:06:01, Vlastimil Babka wrote:
> On 04/27/2017 05:06 PM, Michal Hocko wrote:
> > On Tue 25-04-17 12:42:57, Joonsoo Kim wrote:
> >> On Mon, Apr 24, 2017 at 03:09:36PM +0200, Michal Hocko wrote:
> >>> On Mon 17-04-17 11:02:12, Joonsoo Kim wrote:
> >>>> On Thu, Apr 13, 2017 at 01:56:15PM +0200, Michal Hocko wrote:
> >>>>> On Wed 12-04-17 10:35:06, Joonsoo Kim wrote:
> > [...]
> >>> not for free. For most common configurations where we have ZONE_DMA,
> >>> ZONE_DMA32, ZONE_NORMAL and ZONE_MOVABLE all the 3 bits are already
> >>> consumed so a new zone will need a new one AFAICS.
> >>
> >> Yes, it requires one more bit for a new zone and it's handled by the patch.
> >
> > I am pretty sure that you are aware that consuming new page flag bits
> > is usually a no-go and something we try to avoid as much as possible
> > because we are in a great shortage there. So there really have to be a
> > _strong_ reason if we go that way. My current understanding that the
> > whole zone concept is more about a more convenient implementation rather
> > than a fundamental change which will solve unsolvable problems with the
> > current approach. More on that below.
>
> I don't see it as such a big issue. It's behind a CONFIG option (so we
> also don't need the jump labels you suggest later) and enabling it
> reduces the number of possible NUMA nodes (not page flags). So either
> you are building a kernel for android phone that needs CMA but will have
> a single NUMA node, or for a large server with many nodes that won't
> have CMA. As long as there won't be large servers that need CMA, we
> should be fine (yes, I know some HW vendors can be very creative, but
> then it's their problem?).
Is this really about Android/UMA systems only? My quick grep seems to disagree
$ git grep CONFIG_CMA=y
arch/arm/configs/exynos_defconfig:CONFIG_CMA=y
arch/arm/configs/imx_v6_v7_defconfig:CONFIG_CMA=y
arch/arm/configs/keystone_defconfig:CONFIG_CMA=y
arch/arm/configs/multi_v7_defconfig:CONFIG_CMA=y
arch/arm/configs/omap2plus_defconfig:CONFIG_CMA=y
arch/arm/configs/tegra_defconfig:CONFIG_CMA=y
arch/arm/configs/vexpress_defconfig:CONFIG_CMA=y
arch/arm64/configs/defconfig:CONFIG_CMA=y
arch/mips/configs/ci20_defconfig:CONFIG_CMA=y
arch/mips/configs/db1xxx_defconfig:CONFIG_CMA=y
arch/s390/configs/default_defconfig:CONFIG_CMA=y
arch/s390/configs/gcov_defconfig:CONFIG_CMA=y
arch/s390/configs/performance_defconfig:CONFIG_CMA=y
arch/s390/defconfig:CONFIG_CMA=y
I am pretty sure s390 and ppc support NUMA and aim at supporting really
large systems.
I can imagine that we could make ZONE_CMA configurable in a way that
only very well defined use cases would be supported so that we can save
page flags space. But this alone sounds like a maintainability nightmare
to me. Especially when I consider ZONE_DMA situation. There is simply
not an easy way to find out whether my HW really needs DMA zone or
not. Most probably not but it still is configured and hidden behind
config ZONE_DMA
bool "DMA memory allocation support" if EXPERT
default y
help
DMA memory allocation support allows devices with less than 32-bit
addressing to allocate within the first 16MB of address space.
Disable if no such devices will be used.
If unsure, say Y.
Are we really ready to add another thing like that? How are distribution
kernels going to handle that?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2] mm, zone_device: replace {get, put}_zone_device_page() with a single reference
From: Kirill A. Shutemov @ 2017-05-02 11:37 UTC (permalink / raw)
To: Jerome Glisse
Cc: Dan Williams, Ingo Molnar, linux-kernel@vger.kernel.org, Linux MM,
Ingo Molnar, Andrew Morton, Logan Gunthorpe, Kirill Shutemov
In-Reply-To: <20170501135545.GA16772@redhat.com>
On Mon, May 01, 2017 at 09:55:48AM -0400, Jerome Glisse wrote:
> On Mon, May 01, 2017 at 01:23:59PM +0300, Kirill A. Shutemov wrote:
> > On Sun, Apr 30, 2017 at 07:14:24PM -0400, Jerome Glisse wrote:
> > > On Sat, Apr 29, 2017 at 01:17:26PM +0300, Kirill A. Shutemov wrote:
> > > > On Fri, Apr 28, 2017 at 03:33:07PM -0400, Jerome Glisse wrote:
> > > > > On Fri, Apr 28, 2017 at 12:22:24PM -0700, Dan Williams wrote:
> > > > > > Are you sure about needing to hook the 2 -> 1 transition? Could we
> > > > > > change ZONE_DEVICE pages to not have an elevated reference count when
> > > > > > they are created so you can keep the HMM references out of the mm hot
> > > > > > path?
> > > > >
> > > > > 100% sure on that :) I need to callback into driver for 2->1 transition
> > > > > no way around that. If we change ZONE_DEVICE to not have an elevated
> > > > > reference count that you need to make a lot more change to mm so that
> > > > > ZONE_DEVICE is never use as fallback for memory allocation. Also need
> > > > > to make change to be sure that ZONE_DEVICE page never endup in one of
> > > > > the path that try to put them back on lru. There is a lot of place that
> > > > > would need to be updated and it would be highly intrusive and add a
> > > > > lot of special cases to other hot code path.
> > > >
> > > > Could you explain more on where the requirement comes from or point me to
> > > > where I can read about this.
> > > >
> > >
> > > HMM ZONE_DEVICE pages are use like other pages (anonymous or file back page)
> > > in _any_ vma. So i need to know when a page is freed ie either as result of
> > > unmap, exit or migration or anything that would free the memory. For zone
> > > device a page is free once its refcount reach 1 so i need to catch refcount
> > > transition from 2->1
> >
> > What if we would rework zone device to have pages with refcount 0 at
> > start?
>
> That is a _lot_ of work from top of my head because it would need changes
> to a lot of places and likely more hot code path that simply adding some-
> thing to put_page() note that i only need something in put_page() i do not
> need anything in the get page path. Is adding a conditional branch for
> HMM pages in put_page() that much of a problem ?
Well, it gets inlined everywhere. Removing zone_device code from
get_page() and put_page() saved non-trivial ~140k in vmlinux for
allyesconfig.
Re-introducing part this bloat would be unfortunate.
> > > This is the only way i can inform the device that the page is now free. See
> > >
> > > https://cgit.freedesktop.org/~glisse/linux/commit/?h=hmm-v21&id=52da8fe1a088b87b5321319add79e43b8372ed7d
> > >
> > > There is _no_ way around that.
> >
> > I'm still not convinced that it's impossible.
> >
> > Could you describe lifecycle for pages in case of HMM?
>
> Process malloc something, end it over to some function in the program
> that use the GPU that function call GPU API (OpenCL, CUDA, ...) that
> trigger a migration to device memory.
>
> So in the kernel you get a migration like any existing migration,
> original page is unmap, if refcount is all ok (no pin) then a device
> page is allocated and thing are migrated to device memory.
>
> What happen after is unknown. Either userspace/kernel driver decide
> to migrate back to system memory, either there is an munmap, either
> there is a CPU page fault, ... So from that point on the device page
> as the exact same life as a regular page.
>
> Above i describe the migrate case, but you can also have new memory
> allocation that directly allocate device memory. For instance if the
> GPU do a page fault on an address that isn't back by anything then
> we can directly allocate a device page. No migration involve in that
> case.
>
> HMM pages are like any other pages in most respect. Exception are:
> - no GUP
Hm. How do you exclude GUP? And why is it required?
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH RFC] hugetlbfs 'noautofill' mount option
From: Anshuman Khandual @ 2017-05-02 10:53 UTC (permalink / raw)
To: Prakash Sangappa, linux-kernel, linux-mm
In-Reply-To: <b0efc671-0d7a-0aef-5646-a635478c31b0@oracle.com>
On 05/01/2017 11:30 PM, Prakash Sangappa wrote:
> Some applications like a database use hugetblfs for performance
> reasons. Files on hugetlbfs filesystem are created and huge pages
> allocated using fallocate() API. Pages are deallocated/freed using
> fallocate() hole punching support that has been added to hugetlbfs.
> These files are mmapped and accessed by many processes as shared memory.
> Such applications keep track of which offsets in the hugetlbfs file have
> pages allocated.
>
> Any access to mapped address over holes in the file, which can occur due
s/mapped/unmapped/ ^ ?
> to bugs in the application, is considered invalid and expect the process
> to simply receive a SIGBUS. However, currently when a hole in the file is
> accessed via the mapped address, kernel/mm attempts to automatically
> allocate a page at page fault time, resulting in implicitly filling the
> hole
But this is expected when you try to control the file allocation from
a mapped address. Any changes while walking past or writing the range
in the memory mapped should reflect exactly in the file on the disk.
Why its not a valid behavior ?
> in the file. This may not be the desired behavior for applications like the
> database that want to explicitly manage page allocations of hugetlbfs
> files.
>
> This patch adds a new hugetlbfs mount option 'noautofill', to indicate that
> pages should not be allocated at page fault time when accessed thru mmapped
> address.
When the page should be allocated for mapping ?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: 4.8.8 kernel trigger OOM killer repeatedly when I have lots of RAM that should be free
From: Tetsuo Handa @ 2017-05-02 10:44 UTC (permalink / raw)
To: Marc MERLIN, Linus Torvalds
Cc: Michal Hocko, Vlastimil Babka, linux-mm, LKML, Joonsoo Kim,
Tejun Heo, Greg Kroah-Hartman
In-Reply-To: <20170502041235.zqmywvj5tiiom3jk@merlins.org>
On 2017/05/02 13:12, Marc MERLIN wrote:
> Well, sadly, the problem is more or less back is 4.11.0. The system doesn't really
> crash but it goes into an infinite loop with
> [34776.826800] BUG: workqueue lockup - pool cpus=6 node=0 flags=0x0 nice=0 stuck for 33s!
Wow, two of workqueues are reaching max active.
[34777.202267] workqueue btrfs-endio-write: flags=0xe
[34777.218313] pwq 16: cpus=0-7 flags=0x4 nice=0 active=8/8
[34777.236548] in-flight: 15168:btrfs_endio_write_helper, 13855:btrfs_endio_write_helper, 3360:btrfs_endio_write_helper, 14241:btrfs_endio_write_helper, 27092:btrfs_endio_write_helper, 15194:btrfs_endio_write_helper, 15169:btrfs_endio_write_helper, 27093:btrfs_endio_write_helper
[34777.316225] delayed: btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper, btrfs_endio_write_helper
[34777.450684] workqueue bcache: flags=0x8
[34779.956462] pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=256/256
[34779.978283] in-flight: 15320:cached_dev_read_done [bcache], 23385:cached_dev_read_done [bcache], 23371:cached_dev_read_done [bcache], 15321:cached_dev_read_done [bcache], 15395:cached_dev_read_done [bcache], 11101:cached_dev_read_done [bcache], 15300:cached_dev_read_done [bcache], 23349:cached_dev_read_done [bcache], 23425:cached_dev_read_done [bcache], 23399:cached_dev_read_done [bcache], 15293:cached_dev_read_done [bcache], 20529:cached_dev_read_done [bcache], 15402:cached_dev_read_done [bcache], 23422:cached_dev_read_done [bcache], 23417:cached_dev_read_done [bcache], 23409:cached_dev_read_done [bcache], 20539:cached_dev_read_done [bcache], 23431:cached_dev_read_done [bcache], 20544:cached_dev_read_done [bcache], 15355:cached_dev_read_done [bcache], 11085:cached_dev_read_done [bcache], 6511:cached_dev_read_done [bcache]
Googling with btrfs_endio_write_helper shows a stuck report with 4.8-rc5, but
seems no response ( https://www.spinics.net/lists/linux-btrfs/msg58633.html ).
> Any idea what I should do next?
Maybe you can try collecting list of all in-flight allocations with backtraces
using kmallocwd patches at
http://lkml.kernel.org/r/1489578541-81526-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp
and http://lkml.kernel.org/r/201704272019.JEH26057.SHFOtMLJOOVFQF@I-love.SAKURA.ne.jp
which also tracks mempool allocations.
(Well, the
- cond_resched();
+ //cond_resched();
change in the latter patch would not be preferable.)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 0/5] {ioctl_}userfaultfd.2: yet another update
From: Mike Rapoport @ 2017-05-02 9:48 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <352eee49-d6d1-3e82-a558-2341484c81f3@gmail.com>
On Mon, May 01, 2017 at 08:34:07PM +0200, Michael Kerrisk (man-pages) wrote:
> Hi Mike,
>
> On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> > Hi Michael,
> >
> > These updates pretty much complete the coverage of 4.11 additions, IMHO.
>
> Thanks for this, but we still await input from Andrea
> on various points.
>
> > Mike Rapoport (5):
> > ioctl_userfaultfd.2: update description of shared memory areas
> > ioctl_userfaultfd.2: UFFDIO_COPY: add ENOENT and ENOSPC description
> > ioctl_userfaultfd.2: add BUGS section
> > userfaultfd.2: add note about asynchronios events delivery
> > userfaultfd.2: update VERSIONS section with 4.11 chanegs
> >
> > man2/ioctl_userfaultfd.2 | 35 +++++++++++++++++++++++++++++++++--
> > man2/userfaultfd.2 | 15 +++++++++++++++
> > 2 files changed, 48 insertions(+), 2 deletions(-)
>
> I've applied all of the above, and done some light editing.
>
> Could you please check my changes in the following commits:
>
> 5191c68806c8ac73fdc89586cde434d2766abb5c
> 265225c1e2311ae26ead116e6c8d2cedd46144fa
Both are Ok
Reviewed-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> Thanks,
>
> Michael
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 4/5] userfaultfd.2: add note about asynchronios events delivery
From: Mike Rapoport @ 2017-05-02 9:46 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <5fb9e169-5d92-2fe8-cc59-5c68cfb6be72@gmail.com>
On Mon, May 01, 2017 at 08:33:45PM +0200, Michael Kerrisk (man-pages) wrote:
> Hi Mike,
>
> On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> > Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
>
> Thanks. Applied. One question below.
>
> > ---
> > man2/userfaultfd.2 | 12 ++++++++++++
> > 1 file changed, 12 insertions(+)
> >
> > diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> > index 8b89162..f177bba 100644
> > --- a/man2/userfaultfd.2
> > +++ b/man2/userfaultfd.2
> > @@ -112,6 +112,18 @@ created for the child process,
> > which allows userfaultfd monitor to perform user-space paging
> > for the child process.
> >
> > +Unlike page faults which have to be synchronous and require
> > +explicit or implicit wakeup,
> > +all other events are delivered asynchronously and
> > +the non-cooperative process resumes execution as
> > +soon as manager executes
> > +.BR read(2).
> > +The userfaultfd manager should carefully synchronize calls
> > +to UFFDIO_COPY with the events processing.
> > +
> > +The current asynchronous model of the event delivery is optimal for
> > +single threaded non-cooperative userfaultfd manager implementations.
>
> The preceding paragraph feels incomplete. It seems like you want to make
> a point with that last sentence, but the point is not explicit. What's
> missing?
I've copied both from Documentation/vm/userfaulftfd.txt, and there we also
talk about possibility of addition of synchronous events delivery and
that makes the paragraph above to seem crippled :)
The major point here is that current events delivery model could be
problematic for multi-threaded monitor. I even suspect that it would be
impossible to ensure synchronization between page faults and non-page
fault events in multi-threaded monitor.
> > +
> > .\" FIXME elaborate about non-cooperating mode, describe its limitations
> > .\" for kernels before 4.11, features added in 4.11
> > .\" and limitations remaining in 4.11
> >
>
> Cheers,
>
> Michael
>
>
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC] dev/mem: "memtester -p 0x6c80000000000 10G" cause crash
From: Xishi Qiu @ 2017-05-02 9:30 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, Mel Gorman, Vlastimil Babka, Joonsoo Kim,
Johannes Weiner, Rik van Riel, Shakeel Butt, Linux MM, LKML,
zhong jiang
In-Reply-To: <20170502091630.GH14593@dhcp22.suse.cz>
On 2017/5/2 17:16, Michal Hocko wrote:
> On Tue 02-05-17 16:52:00, Xishi Qiu wrote:
>> On 2017/5/2 16:43, Michal Hocko wrote:
>>
>>> On Tue 02-05-17 15:59:23, Xishi Qiu wrote:
>>>> Hi, I use "memtester -p 0x6c80000000000 10G" to test physical address 0x6c80000000000
>>>> Because this physical address is invalid, and valid_mmap_phys_addr_range()
>>>> always return 1, so it causes crash.
>>>>
>>>> My question is that should the user assure the physical address is valid?
>>>
>>> We already seem to be checking range_is_allowed(). What is your
>>> CONFIG_STRICT_DEVMEM setting? The code seems to be rather confusing but
>>> my assumption is that you better know what you are doing when mapping
>>> this file.
>>>
>>
>> HI Michal,
>>
>> CONFIG_STRICT_DEVMEM=y, and range_is_allowed() will skip memory, but
>> 0x6c80000000000 is not memory, it is just a invalid address, so it cause
>> crash.
>
> OK, I only now looked at the value. It is beyond addressable limit
> (for 47b address space). None of the checks seems to stop this because
> range_is_allowed() resp. its devmem_is_allowed() will allow it as a
> non RAM (!page_is_ram check). I am not really sure how to fix this or
> whether even we should try to fix this particular problem. As I've said
> /dev/mem is dangerous and you should better know what you are doing when
> accessing it.
>
OK, I know, thank you!
Thanks,
Xishi Qiu
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH man-pages 3/5] ioctl_userfaultfd.2: add BUGS section
From: Mike Rapoport @ 2017-05-02 9:32 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Andrea Arcangeli, linux-kernel, linux-mm, linux-man
In-Reply-To: <345c064d-83fe-3e40-c5cb-5d4b6e5cdff4@gmail.com>
On Mon, May 01, 2017 at 08:33:50PM +0200, Michael Kerrisk (man-pages) wrote:
> Hi Mike,
>
> I've applied this, but have a question.
>
> On 05/01/2017 07:43 AM, Mike Rapoport wrote:
> > The features handshake is not quite convenient.
> > Elaborate about it in the BUGS section.
> >
> > Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> > ---
> > man2/ioctl_userfaultfd.2 | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> > index e12b9de..50316de 100644
> > --- a/man2/ioctl_userfaultfd.2
> > +++ b/man2/ioctl_userfaultfd.2
> > @@ -650,6 +650,15 @@ operations are Linux-specific.
> > .SH EXAMPLE
> > See
> > .BR userfaultfd (2).
> > +.SH BUGS
> > +In order to detect available userfault features and
> > +enable certain subset of those features
>
> I changed "certain" to "some". ("certain subset" here also
> would sound like "some particular subset" of those features.)
> Okay?
Yes, sure.
> > +the usefault file descriptor must be closed after the first
> > +.BR UFFDIO_API
> > +operation that queries features availability and re-opened before
> > +the second
> > +.BR UFFDIO_API
> > +call that actually enables the desired features.
> > .SH SEE ALSO
> > .BR ioctl (2),
> > .BR mmap (2),
>
> Cheers,
>
> Michael
>
>
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox