How do I use multiqueue tx

How do I use multiqueue tx

[Andrew Gallatin]

I've done a patch to myri10ge which enables the use of multiple
transmit queues in our Linux driver using CONFIG_NETDEVICES_MULTIQUEUE
netdev subqueues.  Our current hardware uses multiple queues purely
for increasing host parallelism, not priority/QOS.  I'm looking for a
way, beyond pktgen, to actually use any transmit queue other than 0 to
increase parallelism.

Ideally, we would like to support hashing connections on egress to the
same tx/rx queue pair (and hence MSI-X irq) as our device will hash
the traffic to on ingress.  As far as I can tell, the qdisc layer is
where this hashing currently must happen.  Is this correct?  Or is
there a hook which would allow a driver to supply a hash function to
select the egress queue prior to its transmit function being called?

Otherwise, is my only option to write an entire new qdisc which does
the same sort of hashing as our nic, and force the admin to use tc to
select my custom qdisc?

Thanks,

Drew

Re: How do I use multiqueue tx

[David Miller]

From: Andrew Gallatin <gallatin@myri.com>
Date: Tue, 03 Jun 2008 11:34:18 -0400

> I've done a patch to myri10ge which enables the use of multiple
> transmit queues in our Linux driver using CONFIG_NETDEVICES_MULTIQUEUE
> netdev subqueues.  Our current hardware uses multiple queues purely
> for increasing host parallelism, not priority/QOS.  I'm looking for a
> way, beyond pktgen, to actually use any transmit queue other than 0 to
> increase parallelism.

Even though there are seperate per-TX-queue start/stop states
available to manage, all of the transmit path still uses a
single global lock to manage the packet queue in the generic
device structure.

So you will get zero extra parallelism by using the other queues.

I plan to work on this and have presented my ideas about how to
solve this problem in two presentations I gave in Japan several
months ago, and last week in Berlin.

http://vger.kernel.org/~davem/davem_tokyo08.pdf
http://vger.kernel.org/~davem/davem_berlin08.pdf

Don't even bother trying to get better TX parallelism at the
current time, it'll be so much easier to accomplish after
I do this development which I anticipate should be in a testable
state in about one month.

Re: How do I use multiqueue tx

[Andrew Gallatin]

David Miller wrote:

 > I plan to work on this and have presented my ideas about how to
 > solve this problem in two presentations I gave in Japan several
 > months ago, and last week in Berlin.

Thank you for all the good information.  It is great to hear that
you're planning to work on this.

Thanks,

Drew