[RFC] fs/proc/meminfo: introduce Unaccounted statistic

[RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Vlastimil Babka]

The /proc/meminfo virtual file is a collection of various system-wide memory
usage statistics. One of the use cases for looking at the output is to see
whether the kernel might be leaking memory by direct allocations that are not
counted among any of the statistics. This is hovewer not immediately obvious,
because some fields are meant to add up to the MemTotal value, and others not.
Subtle changes also happen over time, e.g. the AnonPages field started
including THP's since commit 3cd14fcd3f12 ("thp: account anon transparent
huge pages into NR_ANON_PAGES") and the Cached field used to include hugetlb
until commit 4165b9b46181 ("hugetlb: do not account hugetlb pages as
NR_FILE_PAGES").

To make kernel memory leaks more obvious, this patch adds an Unaccounted field
whose value is calculated by subtracting a sum of fields that are supposed to
add up to MemTotal without overlap, from the MeTotal value. This should also
help anyone looking at the code to determine these fields easily.

Not-yet-signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: David Rientjes <rientjes@google.com>
---
Hi, I'm wondering if people would find this useful. If you think it is, and
to not make performance worse, I could also make sure in proper submission
that values are not read via global_page_state() multiple times etc...

 fs/proc/meminfo.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
index 8a428498d6b2..3fcd71d4d805 100644
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -48,6 +48,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 	unsigned long committed;
 	long cached;
 	long available;
+	long unaccounted;
 	unsigned long pages[NR_LRU_LISTS];
 	int lru;
 
@@ -65,6 +66,18 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 
 	available = si_mem_available();
 
+	unaccounted = i.totalram - i.freeram
+			- global_node_page_state(NR_ANON_MAPPED)
+			- global_node_page_state(NR_FILE_PAGES)
+			- global_page_state(NR_PAGETABLE)
+			- global_page_state(NR_SLAB_RECLAIMABLE)
+			- global_page_state(NR_SLAB_UNRECLAIMABLE)
+			- (global_page_state(NR_KERNEL_STACK_KB)
+							>> (PAGE_SHIFT - 10))
+			- global_page_state(NR_BOUNCE)
+			- global_node_page_state(NR_WRITEBACK_TEMP)
+			- hugetlb_total_pages();
+
 	show_val_kb(m, "MemTotal:       ", i.totalram);
 	show_val_kb(m, "MemFree:        ", i.freeram);
 	show_val_kb(m, "MemAvailable:   ", available);
@@ -119,6 +132,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 		    global_page_state(NR_PAGETABLE));
 #ifdef CONFIG_QUICKLIST
 	show_val_kb(m, "Quicklists:     ", quicklist_total_size());
+	unaccounted -= quicklist_total_size();
 #endif
 
 	show_val_kb(m, "NFS_Unstable:   ",
@@ -156,6 +170,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
 
 	hugetlb_report_meminfo(m);
 
+	if (unaccounted < 0)
+		unaccounted = 0;
+	show_val_kb(m, "Unaccounted:    ", unaccounted);
+
 	arch_report_meminfo(m);
 
 	return 0;
-- 
2.10.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Michal Hocko]

On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
[...]
> Hi, I'm wondering if people would find this useful. If you think it is, and
> to not make performance worse, I could also make sure in proper submission
> that values are not read via global_page_state() multiple times etc...

I definitely find this information useful and hate to do the math all
the time but on the other hand this is quite fragile and I can imagine
we can easily forget to add something there and provide a misleading
information to the userspace. So I would be worried with a long term
maintainability of this.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Dave Chinner]

On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> [...]
> > Hi, I'm wondering if people would find this useful. If you think it is, and
> > to not make performance worse, I could also make sure in proper submission
> > that values are not read via global_page_state() multiple times etc...
> 
> I definitely find this information useful and hate to do the math all
> the time but on the other hand this is quite fragile and I can imagine
> we can easily forget to add something there and provide a misleading
> information to the userspace. So I would be worried with a long term
> maintainability of this.

This will result in valid memory usage by subsystems like the XFS
buffer cache being reported as "unaccounted". Given this cache
(whose size is shrinker controlled) can grow to gigabytes in size
under various metadata intensive workloads, there's every chance
that such reporting will make users incorrectly think they have a
massive memory leak....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Vlastimil Babka]

On 10/21/2016 12:59 AM, Dave Chinner wrote:
> On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
>> On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
>> [...]
>> > Hi, I'm wondering if people would find this useful. If you think it is, and
>> > to not make performance worse, I could also make sure in proper submission
>> > that values are not read via global_page_state() multiple times etc...
>>
>> I definitely find this information useful and hate to do the math all
>> the time but on the other hand this is quite fragile and I can imagine
>> we can easily forget to add something there and provide a misleading
>> information to the userspace. So I would be worried with a long term
>> maintainability of this.
>
> This will result in valid memory usage by subsystems like the XFS
> buffer cache being reported as "unaccounted". Given this cache
> (whose size is shrinker controlled) can grow to gigabytes in size
> under various metadata intensive workloads, there's every chance
> that such reporting will make users incorrectly think they have a
> massive memory leak....

Is the XFS buffer cache accounted (and visible) somewhere then? I'd say getting 
such large consumers to become visible on the same level as others would be 
another advantage...

And yeah, I can even recall a bug report, where I had to do the calculation 
myself and it looked like a big leak, and it took some effort to connect it to 
xfs buffers. I'd very much welcome for it to be more obvious.

Vlastimil

> Cheers,
>
> Dave.
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Dave Chinner]

On Fri, Oct 21, 2016 at 09:25:10AM +0200, Vlastimil Babka wrote:
> On 10/21/2016 12:59 AM, Dave Chinner wrote:
> >On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> >>On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> >>[...]
> >>> Hi, I'm wondering if people would find this useful. If you think it is, and
> >>> to not make performance worse, I could also make sure in proper submission
> >>> that values are not read via global_page_state() multiple times etc...
> >>
> >>I definitely find this information useful and hate to do the math all
> >>the time but on the other hand this is quite fragile and I can imagine
> >>we can easily forget to add something there and provide a misleading
> >>information to the userspace. So I would be worried with a long term
> >>maintainability of this.
> >
> >This will result in valid memory usage by subsystems like the XFS
> >buffer cache being reported as "unaccounted". Given this cache
> >(whose size is shrinker controlled) can grow to gigabytes in size
> >under various metadata intensive workloads, there's every chance
> >that such reporting will make users incorrectly think they have a
> >massive memory leak....
> 
> Is the XFS buffer cache accounted (and visible) somewhere then? I'd
> say getting such large consumers to become visible on the same level
> as others would be another advantage...

It's handles are visible via the xfs_buf slab cache. By the time
you've got enough memory in the buffer cache for it to be noticed,
the xfs_buf slab is near the top of the list in slabtop.

Of course, because of the crazy way slub names caches, this can be
impossible to find because there isn't a "xfs_buf" slab cache that
shows up in slabtop. It'll end being called something like
"mnt_cache"....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Michal Hocko]

On Fri 21-10-16 09:59:29, Dave Chinner wrote:
> On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> > On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> > [...]
> > > Hi, I'm wondering if people would find this useful. If you think it is, and
> > > to not make performance worse, I could also make sure in proper submission
> > > that values are not read via global_page_state() multiple times etc...
> > 
> > I definitely find this information useful and hate to do the math all
> > the time but on the other hand this is quite fragile and I can imagine
> > we can easily forget to add something there and provide a misleading
> > information to the userspace. So I would be worried with a long term
> > maintainability of this.
> 
> This will result in valid memory usage by subsystems like the XFS
> buffer cache being reported as "unaccounted".

I would argue that the file is more intended for developers than regular
users. Most of those counters simply require a deep knowledge of the MM
subsystem to interpret them correctly (yeah we have seen many reports
that the free memory is too low but we always managed to explain...).

So to me this is more a convenience thing for developers than anything
else. But my worry about maintainability still stands. People who are
adding new counters might easily forget to update this part, so I am not
actually sure it is a good idea long term. Getting a misleading data is
worse than pushing developers to do the math and scratch their heads
about what should be included IMHO.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

[Vlastimil Babka]

On 10/20/2016 03:33 PM, Michal Hocko wrote:
> On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> [...]
>> Hi, I'm wondering if people would find this useful. If you think it is, and
>> to not make performance worse, I could also make sure in proper submission
>> that values are not read via global_page_state() multiple times etc...
>
> I definitely find this information useful and hate to do the math all
> the time but on the other hand this is quite fragile and I can imagine
> we can easily forget to add something there and provide a misleading
> information to the userspace. So I would be worried with a long term
> maintainability of this.

I wouldn't fear this that much. Maybe even on the contrary - if we unknowingly 
change the picture by misacounting something, or changing a counter to become 
subset of another, and Unaccounted starts to give weird values, it will give us 
hint that there's either a problem to fix, or a missed documentation update.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>