Re: [PATCH 3/3] NET: [CORE] Stack changes to add multiqueue hardware support API

[David Miller <davem@davemloft.net>]

From: "Waskiewicz Jr, Peter P" <peter.p.waskiewicz.jr@intel.com>
Date: Tue, 19 Jun 2007 13:01:18 -0700

> I've been thinking about this more today, so please bear with me if I'm
> missing something.  Right now, with how qdisc_restart() is running, we'd
> definitely call netif_subqueue_stopped(dev, skb->queue_mapping) for all
> multi-ring and single-ring devices.  However, with Jamal's and Krishna's
> qdisc_restart() rewrite patch, the checks for netif_queue_stopped() and
> netif_subqueue_stopped() would be pushed into the qdisc's ->dequeue()
> functions.  If that's the case, then the only checks on
> egress_subqueue[x] would be for multi-ring adapters, or if someone was
> silly enough to load sch_{rr|prio} onto a single-ring device with
> multiqueue hardware support compiled in.  Given all of that, I'm not
> sure allocating egress_subqueue[0] at compile time or runtime would make
> any difference either way.  If I'm missing something, please let me know
> - I'd like to reduce any unnecessary pointer dereferences if possible,
> but given the proposed qdisc_restart(), I think the code as-is would be
> ok.

It's not being allocated at "compile time", it's being allocated
linearly into one block of ram in order to avoid pointer
derefs but it's still "dynamic" in that the size isn't known
until the alloc_netdev() call.

We do this trick all over the networking, TCP sockets are 3 or 4
different structures, all allocated into a linear block of
memory so that:

1) only one memory allocation needs to be done for each object
   create, this is not relevant for the net_device case
   except in extreme examples bringing thousands of devices
   up and down which I suppose someone can give a realistic
   example of :-)

2) each part can be accessed as an offset from the other instead
   of a pointer deref which costs a cpu memory read

Please change the allocation strategy as I recommended, thanks.