Re: [PATCH net-next-2.6] sched: use xps information for qdisc NUMA affinity

[David Miller <davem@davemloft.net>]

From: Tom Herbert <therbert@google.com>
Date: Tue, 30 Nov 2010 10:31:48 -0800

> On Mon, Nov 29, 2010 at 10:14 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> I was thinking of using XPS tx_queue->cpu mapping to eventually allocate
>> memory with correct NUMA affinities, for qdisc/class stuff for example.
>>
> 
> An interesting idea, but the real question is can this be used for all
> queue related allocations.  This includes those that drivers allocate
> which are probably done in initialization.

Most drivers do, and all drivers ought to, allocate DMA queues and
whatnot when the interface is brought up.

That solves this particular issue.

For example, drivers/net/niu.c does this by calling
niu_alloc_channels() via niu_open().

The only thing we really can't handle currently is the netdev
itself (and the associated driver private).  Jesse Brandeburg
has been reminding me about this over and over :-)

There might be some things we can even do about that part.  For
example, we can put all of the things the driver touches in the
RX and TX fast paths via indirect pointers and therefore be able
to allocate and reallocate those portions as we want long after
device registry.

Doing the core netdev struct itself is too hard because it sits
in so many tables.