Re: [PATCH] staging: lustre: fix GFP_ATOMIC macro usage

["Dilger, Andreas" <andreas.dilger@intel.com>]

On 2014/01/17, 8:17 AM, "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>
wrote:

>On Fri, Jan 17, 2014 at 05:51:28PM +0300, Dan Carpenter wrote:
>> We will want to get rid of lustre's custom allocator before this gets
>> out of staging.
>> 
>> But one feature that the lustre allocator has which is pretty neat is
>> that it lets you debug how much memory the filesystem is using.  Is
>> there a standard way to find this information?
>
>Create your own mempool/slab/whatever_it's_called and look in the
>debugfs or proc files for the allocator usage, depending on the memory
>allocator the kernel is using.
>
>That's how the rest of the kernel does it, no reason lustre should be
>any different.

The Lustre allocation macros track the memory usage across the whole
filesystem,
not just of a single structure that a mempool/slab/whatever would do.
This is
useful to know for debugging purposes (e.g. user complains about not having
enough RAM for their highly-tuned application, or to check for leaks at
unmount).

It can also log the alloc/free calls and post-process them to find leaks
easily, or find pieces code that is allocating too much memory that are not
using dedicated slabs.  This also works if you encounter a system with a
lot
of allocated memory, enable "free" logging, and then unmount the
filesystem.
The logs will show which structures are being freed (assuming they are not
leaked completely) and point you to whatever is not being shrunk properly.

I don't know if there is any way to track this with regular kmalloc(), and
creating separate slabs for so ever data structure would be ugly.  The
generic
/proc/meminfo data doesn't really tell you what is using all the memory,
and
the size-NNNN slabs give some information, but are used all over the
kernel.

I'm pretty much resigned to losing all of this functionality, but it
definitely
has been very useful for finding problems.

Cheers, Andreas
-- 
Andreas Dilger

Lustre Software Architect
Intel High Performance Data Division