How to allocate per-cpu data for online CPUs only (and safely)?

How to allocate per-cpu data for online CPUs only (and safely)?

[Martin Peschke]

I am trying to optimize the memory footprint of some code of mine.
It's been using per-cpu data and alloc_percpu() so far. The latter has 
the disadvantage of getting hold of memory for CPU's which aren't there 
(yet).

I could imagine using CPU-hotplug notifications as triggers for 
additional allocations or for cleaning up unneeded memory. But 
alloc_percpu() appears to conflict with that idea.

I was briefly tempted to derive some code from alloc_percpu() more to my 
liking, until I was scared off by this comment in alloc_percpu():

         /*
          * Cannot use for_each_online_cpu since a cpu may come online
          * and we have no way of figuring out how to fix the array
          * that we have allocated then....
          */

Well, and then there is kernel/profile.c, for example, which boldly 
ignors alloc_percpu()'s qualms and allocates and releases per-cpu data 
as needed.

Is that the way to go?
If so, why alloc_percpu()'s reservations?
Or, does that comment imply that the exploiter isn't expected to take 
care of CPU hotplug events?
Am I missing anything?

Thank you.

Martin

How to allocate per-cpu data for online CPUs only (and safely)?

[Martin Peschke]

I am trying to optimize the memory footprint of some code of mine.
It's been using per-cpu data and alloc_percpu() so far. The latter has
the disadvantage of getting hold of memory for CPU's which aren't there
(yet).

I could imagine using CPU-hotplug notifications as triggers for
additional allocations or for cleaning up unneeded memory. But
alloc_percpu() appears to conflict with that idea.

I was briefly tempted to derive some code from alloc_percpu() more to my
liking, until I was scared off by this comment in alloc_percpu():

         /*
          * Cannot use for_each_online_cpu since a cpu may come online
          * and we have no way of figuring out how to fix the array
          * that we have allocated then....
          */

Well, and then there is kernel/profile.c, for example, which boldly
ignors alloc_percpu()'s qualms and allocates and releases per-cpu data
as needed.

Is that the way to go?
If so, why alloc_percpu()'s reservations?
Or, does that comment imply that the exploiter isn't expected to take
care of CPU hotplug events?
Am I missing anything?

Thank you.

Martin

Re: How to allocate per-cpu data for online CPUs only (and safely)?

[Srivatsa Vaddagiri]

On Tue, Feb 21, 2006 at 11:29:02PM +0100, Martin Peschke wrote:
> I am trying to optimize the memory footprint of some code of mine.
> the disadvantage of getting hold of memory for CPU's which aren't there
> (yet).
> 
> I could imagine using CPU-hotplug notifications as triggers for
> additional allocations or for cleaning up unneeded memory. But
> alloc_percpu() appears to conflict with that idea.
> 
> I was briefly tempted to derive some code from alloc_percpu() more to my
> liking, until I was scared off by this comment in alloc_percpu():
> 
>         /*
>          * Cannot use for_each_online_cpu since a cpu may come online
>          * and we have no way of figuring out how to fix the array
>          * that we have allocated then....
>          */
> 
> Well, and then there is kernel/profile.c, for example, which boldly
> ignors alloc_percpu()'s qualms and allocates and releases per-cpu data
> as needed.
> 
> Is that the way to go?
> If so, why alloc_percpu()'s reservations?

The above comment in alloc_percpu regarding use of for_each_online_cpu
probably refers to the fact that once we handover 'pdata' to the caller
of alloc_percpu, there is no way to go back and fix it (i.e allocate
objects) for CPUs that come online later. This could have been possible if 
alloc_percpu kept track of all per-CPU allocations (chain togther all pdata's 
in a linked list?), but I guess they didn't do it to keep it simple.

I would say the code in kernel/profile.c is fine enough to emulate.

> Or, does that comment imply that the exploiter isn't expected to take
> care of CPU hotplug events?
> Am I missing anything?
> 
> Thank you.
> 
> Martin

-- 
Regards,
vatsa