Synchronization of the procps tools with /proc/meminfo

Synchronization of the procps tools with /proc/meminfo

[Bart Van Assche]

As known the tools in the procps package (e.g. top and free) obtain
their status information from the Linux kernel by reading a.o.
/proc/meminfo. Both top and free report a.o. the following
information:
* Total amount of physical memory.
* Physical memory in use (reclaimable + unreclaimable).
* Unreclaimable physical memory.
Current versions of the procps tools only take "Buffers" and "Cached"
in account as reclaimable memory and ignore the SReclaimable field
(slab reclaimable, includes a.o. the memory occupied by dentry and
inode structures), one of the more recently added /proc/meminfo field
(the latest procps release (version 3.2.7) dates from June 25, 2006).
I would like to see both top and free modified such that these take
the SReclaimable field in account. The reason is that the numbers
reported by free as "-/+ buffers/cache" are useless on recent kernels
when monitoring a Linux system for memory leaks in kernel and/or
server processes. E.g. when findutils updates its database, a lot of
extra dentry and inodes are cached. The output of "free" shows a
significant increase in the amount of memory used, while only
SReclaimable increased and not the unreclaimable physical memory.

This leads me to the question: if the layout of /proc/meminfo changes,
who communicates these changes to the procps maintainers ? And who
maintains the procps package ? I have tried before to contact Albert
Calahan but without success so far.

See also:
* The procps package -- http://procps.sourceforge.net/
* A previous attempt to inform the procps maintainers:
http://sourceforge.net/mailarchive/forum.php?thread_name=e2e108260802132333w4459ae23o3a5930583f426339%40mail.gmail.com&forum_name=procps-feedback

Bart Van Assche.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Synchronization of the procps tools with /proc/meminfo

[Albert Cahalan]

On Feb 18, 2008 10:55 AM, Bart Van Assche <bart.vanassche@gmail.com> wrote:

> This leads me to the question: if the layout of /proc/meminfo changes,
> who communicates these changes to the procps maintainers ?

Nobody ever informs me. :-(

We thus have abominations like the smaps file. :-(

> And who
> maintains the procps package ? I have tried before to contact Albert
> Calahan but without success so far.

It's in my inbox somewhere. Assuming you used a word
that I will search for, like "procps", I'll get to it. Sorry about
being rather slow lately.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Synchronization of the procps tools with /proc/meminfo

[Bart Van Assche]

On Feb 19, 2008 7:53 PM, Albert Cahalan <acahalan@cs.uml.edu> wrote:
> On Feb 18, 2008 10:55 AM, Bart Van Assche <bart.vanassche@gmail.com> wrote:
>
> > This leads me to the question: if the layout of /proc/meminfo changes,
> > who communicates these changes to the procps maintainers ?
>
> Nobody ever informs me. :-(

That's very unfortunate.

But how should we proceed ? There is not only the SReclaimable field
that was added to /proc/meminfo, there is also the NFS_Unstable field.
I'm not sure whether that last one counts as reclaimable.

Note: Mel Gorman, who's also in CC, is on holiday but will jump in on
this discussion as soon as he's back from holiday.

Bart Van Assche.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Synchronization of the procps tools with /proc/meminfo

[Bart Van Assche]

On Tue, Feb 19, 2008 at 7:53 PM, Albert Cahalan <acahalan@cs.uml.edu> wrote:
> On Feb 18, 2008 10:55 AM, Bart Van Assche <bart.vanassche@gmail.com> wrote:
>
>  > This leads me to the question: if the layout of /proc/meminfo changes,
>  > who communicates these changes to the procps maintainers ?
>
>  Nobody ever informs me. :-(

Albert, can you please review the patch below ?

Thanks,

Bart.


diff -ru orig/procps-3.2.7/proc/library.map procps-3.2.7/proc/library.map
--- orig/procps-3.2.7/proc/library.map	2005-03-14 05:32:40.000000000 +0100
+++ procps-3.2.7/proc/library.map	2008-03-08 10:17:01.000000000 +0100
@@ -18,6 +18,8 @@
   kb_main_free; kb_main_total; kb_main_used; kb_swap_free;
   kb_swap_total; kb_swap_used; kb_main_shared;
   kb_low_total; kb_low_free; kb_high_total; kb_high_free;
+  kb_swap_cached; kb_anon_pages; kb_bounce; kb_nfs_unstable;
+  kb_slab_reclaimable; kb_slab_unreclaimable;
   vm_pgpgin; vm_pgpgout; vm_pswpin; vm_pswpout;
   free_slabinfo; put_slabinfo; get_slabinfo; get_proc_stats;
 local: *;
diff -ru orig/procps-3.2.7/proc/sysinfo.c procps-3.2.7/proc/sysinfo.c
--- orig/procps-3.2.7/proc/sysinfo.c	2006-06-25 08:41:48.000000000 +0200
+++ procps-3.2.7/proc/sysinfo.c	2008-03-08 10:30:14.000000000 +0100
@@ -8,6 +8,8 @@
 // File for parsing top-level /proc entities. */
 //
 // June 2003, Fabian Frederick, disk and slab info
+// Copyright (c) 2003 Fabian Frederick.
+// Copyright (c) 2008 Bart Van Assche.

 #include <stdio.h>
 #include <stdlib.h>
@@ -503,6 +505,11 @@
 unsigned long kb_swap_free;
 unsigned long kb_swap_total;
 /* recently introduced */
+unsigned long kb_anon_pages;
+unsigned long kb_bounce;
+unsigned long kb_nfs_unstable;
+unsigned long kb_slab_reclaimable;
+unsigned long kb_slab_unreclaimable;
 unsigned long kb_high_free;
 unsigned long kb_high_total;
 unsigned long kb_low_free;
@@ -539,6 +546,8 @@
   char *tail;
   static const mem_table_struct mem_table[] = {
   {"Active",       &kb_active},       // important
+  {"AnonPages",    &kb_anon_pages},
+  {"Bounce",       &kb_bounce},
   {"Buffers",      &kb_main_buffers}, // important
   {"Cached",       &kb_main_cached},  // important
   {"Committed_AS", &kb_committed_as},
@@ -556,10 +565,13 @@
   {"MemFree",      &kb_main_free},    // important
   {"MemShared",    &kb_main_shared},  // important, but now gone!
   {"MemTotal",     &kb_main_total},   // important
+  {"NFS_Unstable", &kb_nfs_unstable},
   {"PageTables",   &kb_pagetables},   // kB version of vmstat
nr_page_table_pages
   {"ReverseMaps",  &nr_reversemaps},  // same as vmstat nr_page_table_pages
   {"Slab",         &kb_slab},         // kB version of vmstat nr_slab
   {"SwapCached",   &kb_swap_cached},
+  {"SReclaimable", &kb_slab_reclaimable},
+  {"SUnreclaim",   &kb_slab_unreclaimable},
   {"SwapFree",     &kb_swap_free},    // important
   {"SwapTotal",    &kb_swap_total},   // important
   {"VmallocChunk", &kb_vmalloc_chunk},
@@ -603,6 +615,7 @@
   }
   kb_swap_used = kb_swap_total - kb_swap_free;
   kb_main_used = kb_main_total - kb_main_free;
+  kb_main_cached += kb_slab_reclaimable + kb_swap_cached + kb_nfs_unstable;
 }

 /*****************************************************************/
diff -ru orig/procps-3.2.7/proc/sysinfo.h procps-3.2.7/proc/sysinfo.h
--- orig/procps-3.2.7/proc/sysinfo.h	2006-06-25 08:41:48.000000000 +0200
+++ procps-3.2.7/proc/sysinfo.h	2008-03-08 10:15:41.000000000 +0100
@@ -30,6 +30,11 @@
 extern unsigned long kb_swap_free;
 extern unsigned long kb_swap_total;
 /* recently introduced */
+extern unsigned long kb_anon_pages;
+extern unsigned long kb_bounce;
+extern unsigned long kb_nfs_unstable;
+extern unsigned long kb_slab_reclaimable;
+extern unsigned long kb_slab_unreclaimable;
 extern unsigned long kb_high_free;
 extern unsigned long kb_high_total;
 extern unsigned long kb_low_free;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>