Re: [PATCH net-next-2.6] sched: use xps information for qdisc NUMA affinity

[Ben Hutchings <bhutchings@solarflare.com>]

On Tue, 2010-11-30 at 19:39 +0100, Eric Dumazet wrote:
> Le mardi 30 novembre 2010 à 10:31 -0800, Tom Herbert a écrit :
> > On Mon, Nov 29, 2010 at 10:14 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > > I was thinking of using XPS tx_queue->cpu mapping to eventually allocate
> > > memory with correct NUMA affinities, for qdisc/class stuff for example.
> > >
> > 
> > An interesting idea, but the real question is can this be used for all
> > queue related allocations.  This includes those that drivers allocate
> > which are probably done in initialization.
> > 
> 
> This would need a callback to device, to eventually re-allocate its ring
> buffer (or whatever data structures it allocated). Probably worth it,
> considering size of txbd on some NICS (up to one cache line per entry)
> 
> Right now, they tend to allocate their memory on a single NUMA node, so
> it is a problem.

Yes, that's why I proposed an ethtool interface for reconfiguring this.
Although to be honest I haven't yet constructed a case where it made a
difference.  I think the most important objects to be allocated on the
right node are RX buffers, and as long as refill is scheduled on the
same CPU as the IRQ this already happens.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.