Re: [PATCHSET] percpu: generalize first chunk allocators and improve lpage NUMA support

[Tejun Heo <tj@kernel.org>]

Hello, Andi.

Andi Kleen wrote:
>> I don't think it will be too complex.  The necessary bits are already
>> there and they are necessary for other stuff too, so...
> 
> Are we looking at a different source base? Here's a random example
> using possible per cpu data I picked in current git: icmp.c

I was talking about percpu allocator proper.  Yeap, the major work
would be in auditing and converting for_each_possible_cpu() users.

> static int __net_init icmp_sk_init(struct net *net)
> {
>         int i, err;
> 
>         net->ipv4.icmp_sk =
>                 kzalloc(nr_cpu_ids * sizeof(struct sock *), GFP_KERNEL);
>         if (net->ipv4.icmp_sk == NULL)
>                 return -ENOMEM;
> 
>         for_each_possible_cpu(i) {
> 		... allocate per cpu socket and some other setup ...
> 	}
> } 
> 
> static void __net_exit icmp_sk_exit(struct net *net)
> {
>         int i;
> 
>         for_each_possible_cpu(i)
>                 inet_ctl_sock_destroy(net->ipv4.icmp_sk[i]);
>         kfree(net->ipv4.icmp_sk);
>         net->ipv4.icmp_sk = NULL;
> }
> 
> You would need to convert that to use a CPU notifier and callbacks
> setting up the sockets. Then make sure there are no races in all of
> this. And get it somehow tested (where is the user base who
> tests cpu hotplug?) 

Maybe it would be better to allocate percpu sockets as proper percpu
variables.  Initialization would still need callback mechanism tho.  I
was thinking about adding @init callback to percpu_alloc(), which
would be much simpler than doing full cpu hotplug callback.

> And there is lots of similar code all over the tree

For static percpu variables, it'll be mostly about converting
for_each_possible_cpu() to for_each_used_cpu() as both allocation and
initialization can be handled by percpu proper.  For dynamic areas,
allocation can be handled by percpu proper but cpus coming online
would need more work to convert.  It'll take some effort but there
aren't too many alloc_percpu() users yet and I don't think it will be
too difficult.  I wouldn't know for sure before I actually try tho.

Thanks.

-- 
tejun