Re: [PATCH] Run PCI driver initialization on local node

Christoph Lameter
2005-07-07T00:23:26Z

On Wed, 6 Jul 2005, Andi Kleen wrote:

> On Wed, Jul 06, 2005 at 09:35:32AM -0700, Christoph Lameter wrote:
> > On Wed, 6 Jul 2005, Andi Kleen wrote:
> >
> > > Instead of adding messy kmalloc_node()s everywhere run the
> > > PCI driver probe on the node local to the device.
> > > Then the normal NUMA aware allocators do the right thing.
> >
> > That depends on the architecture. Some do round robin allocs for periods
> > of time during bootup. I think it is better to explicitly place control
>
> slab will usually do the right thing because it has a forced
> local node policy, but __gfp might not.

GFP allocs may not do the right thing. If you want to do this then it
may be best to set the memory policy to restrict allocations to the node
on which the device resides.

Plus there are CPU less nodes. What happens to those?

> Patching every driver in existence? That sounds like a lot of
> work.

No just patch those that would benefit from it. The existing
dma_alloc_coherent already takes care of many of the placement
issues for driver memory. We would probably need to patch more locations
where higher level control structure allocations are being done.

> The node local placement should be correct for nearly all drivers. I didn't
> see any other fancy placement in your patches. If a driver still wants to do
> fancy placement it is free to overwrite the policy. Having a good
> default is good.

Definitely.