GFP_NOWAIT and GFP_NOMEMALLOC

GFP_NOWAIT and GFP_NOMEMALLOC

[Leonidas .]

Hi list,

I am new here, I have googled/binged enough before posting this message,
in case of redundancy please point me to appropriate links/resources etc.

I want to kmalloc memory while holding spinlocks in process context, here I
can't use GFP_KERNEL flag since it can sleep. Using GFP_ATOMIC guarantees
that allocation will succeed by allocating from emergency pools if needed.
But I dont think, I need to use emergency pool and I want to limit my memory
consumption to ZONE_NORMAL without sleeping, my module is ready to handle
the allocation failure if any.


Something like,

ptr =  kmalloc(..., GFP_KERNEL | GFP_DONT_SLEEP );

if( !ptr ) {

 /*GFP_KERNEL failed, Use GFP_ATOMIC pool*/
 ptr =  kmalloc(..., GFP_ATOMIC );

}
/*Go ahead and do stuff*/

Is there any flag which fetches memory from GFP_KERNEL pool with a
guarantee that
it will not sleep?

Does GFP_NOWAIT | GFP_KERNEL guarantee that?

Is ( GFP_NOWAIT | GFP_KERNEL ) == ( GFP_ATOMIC - dont access emergency pools)?

I am a bit confused about GFP_NOWAIT and GFP_NOMEMALLOC, any pointers will be
helpful.

-Leo.

Re: GFP_NOWAIT and GFP_NOMEMALLOC

[Arjan van de Ven]

On Tue, 29 Sep 2009 16:55:24 +0530
"Leonidas ." <leonidas137@gmail.com> wrote:

> Hi list,
> 
> I am new here, I have googled/binged enough before posting this
> message, in case of redundancy please point me to appropriate
> links/resources etc.
> 
> I want to kmalloc memory while holding spinlocks in process context,
> here I can't use GFP_KERNEL flag since it can sleep. Using GFP_ATOMIC
> guarantees that allocation will succeed by allocating from emergency
> pools if needed. But I dont think, I need to use emergency pool and I
> want to limit my memory consumption to ZONE_NORMAL without sleeping,
> my module is ready to handle the allocation failure if any.

GFP_NOWAIT is what you want.

can you give us a pointer to what your module is about? Maybe there's
better solutions...


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: GFP_NOWAIT and GFP_NOMEMALLOC

[Leonidas .]

On Tue, Sep 29, 2009 at 5:29 PM, Arjan van de Ven <arjan@infradead.org> wrote:
> On Tue, 29 Sep 2009 16:55:24 +0530
> "Leonidas ." <leonidas137@gmail.com> wrote:
>
>> Hi list,
>>
>> I am new here, I have googled/binged enough before posting this
>> message, in case of redundancy please point me to appropriate
>> links/resources etc.
>>
>> I want to kmalloc memory while holding spinlocks in process context,
>> here I can't use GFP_KERNEL flag since it can sleep. Using GFP_ATOMIC
>> guarantees that allocation will succeed by allocating from emergency
>> pools if needed. But I dont think, I need to use emergency pool and I
>> want to limit my memory consumption to ZONE_NORMAL without sleeping,
>> my module is ready to handle the allocation failure if any.
>
> GFP_NOWAIT is what you want.
>
> can you give us a pointer to what your module is about? Maybe there's
> better solutions...
>
>
> --
> Arjan van de Ven        Intel Open Source Technology Centre
> For development, discussion and tips for power savings,
> visit http://www.lesswatts.org
>


The module is going to be a profiling module and the user module which
is under profiling
will call the apis which I am exporting. My apis will collect certain
data from point
it is called and populate internal data strcuctures. My apis can be
called from process
context as well as interrupt context hence I am using spinlocks as
locking mechanism.
The amount of memory I am allocating each time my api is called around
100 bytes,
and in worst case I might end up allocating tens of thousands of
objects of almost same
size. I am aware that in such a scenario I should be using some kind
of memory pool
mechanism and preallocate my buffers to avoid frequent
allocations/deallocations.

But frankly, I am not very familiar with Linux kernel apis yet hence
have resorted to
using a tree data structure for the first cut where I allocate memory as needed
depending on the context I am executing in. To optimize a bit, I do
not free my memory
till some threshold is reached.

Some questions here:

1. GFP_NOWAIT as in kmalloc(size, GFP_KERNEL | GFPNOWAIT)?

or

kmalloc(size, GFP_NOWAIT), I know this one is a dumb one.

2. Any reference for the mempool kind of implementation mentioned above?

3. Any other pointers about design, references etc would be helpful.

-Leo.

Re: GFP_NOWAIT and GFP_NOMEMALLOC

[Arjan van de Ven]

On Tue, 29 Sep 2009 17:46:35 +0530
"Leonidas ." <leonidas137@gmail.com> wrote:
> 
> The module is going to be a profiling module and the user module which
> is under profiling
> will call the apis which I am exporting. My apis will collect certain
> data from point

(I assume you are aware of 'perf' and such)


> But frankly, I am not very familiar with Linux kernel apis yet hence
> have resorted to
> using a tree data structure for the first cut where I allocate memory
> as needed depending on the context I am executing in. To optimize a
> bit, I do not free my memory
> till some threshold is reached.
> 
> Some questions here:
> 
> 1. GFP_NOWAIT as in kmalloc(size, GFP_KERNEL | GFPNOWAIT)?
> 
> or
> 
> kmalloc(size, GFP_NOWAIT), I know this one is a dumb one.

don't do GFP_KERNEL |

because that makes it basically equivalent to GFP_KERNEL, which is a
sleeping version.

> 
> 2. Any reference for the mempool kind of implementation mentioned
> above?

there's mm/mempool.c..

as a general suggestion I would say "look at perf, and see if you can
build on that, or borrow from it"...


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: GFP_NOWAIT and GFP_NOMEMALLOC

[Leonidas .]

On Tue, Sep 29, 2009 at 6:07 PM, Arjan van de Ven <arjan@infradead.org> wrote:
> On Tue, 29 Sep 2009 17:46:35 +0530
> "Leonidas ." <leonidas137@gmail.com> wrote:
>>
>> The module is going to be a profiling module and the user module which
>> is under profiling
>> will call the apis which I am exporting. My apis will collect certain
>> data from point
>
> (I assume you are aware of 'perf' and such)
>
>
>> But frankly, I am not very familiar with Linux kernel apis yet hence
>> have resorted to
>> using a tree data structure for the first cut where I allocate memory
>> as needed depending on the context I am executing in. To optimize a
>> bit, I do not free my memory
>> till some threshold is reached.
>>
>> Some questions here:
>>
>> 1. GFP_NOWAIT as in kmalloc(size, GFP_KERNEL | GFPNOWAIT)?
>>
>> or
>>
>> kmalloc(size, GFP_NOWAIT), I know this one is a dumb one.
>
> don't do GFP_KERNEL |
>
> because that makes it basically equivalent to GFP_KERNEL, which is a
> sleeping version.
>
>>
>> 2. Any reference for the mempool kind of implementation mentioned
>> above?
>
> there's mm/mempool.c..
>
> as a general suggestion I would say "look at perf, and see if you can
> build on that, or borrow from it"...
>
>
> --
> Arjan van de Ven        Intel Open Source Technology Centre
> For development, discussion and tips for power savings,
> visit http://www.lesswatts.org
>

I will have a look at perf. Thanks for the inputs.

-Leo.

Re: GFP_NOWAIT and GFP_NOMEMALLOC

[Pavel Machek]

On Tue 2009-09-29 16:55:24, Leonidas . wrote:
> Hi list,
> 
> I am new here, I have googled/binged enough before posting this message,
> in case of redundancy please point me to appropriate links/resources etc.
> 
> I want to kmalloc memory while holding spinlocks in process context, here I
> can't use GFP_KERNEL flag since it can sleep. Using GFP_ATOMIC guarantees
> that allocation will succeed by allocating from emergency pools if needed.
> But I dont think, I need to use emergency pool and I want to limit my memory
> consumption to ZONE_NORMAL without sleeping, my module is ready to handle
> the allocation failure if any.

Umm. Either preallocate, or use gfp_atomic. It should be ok.

> Something like,
> 
> ptr =  kmalloc(..., GFP_KERNEL | GFP_DONT_SLEEP );
> 
> if( !ptr ) {
> 
>  /*GFP_KERNEL failed, Use GFP_ATOMIC pool*/
>  ptr =  kmalloc(..., GFP_ATOMIC );
> 
> }
> /*Go ahead and do stuff*/

That's pretty stupid, no? Just alloc GFP_ATOMIC directly... oh and
check return values...
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html