Re: [RFC Patch net-next] net_sched: make classifying lockless on ingress

[Jamal Hadi Salim <jhs@mojatatu.com>]

On 12/20/13 21:32, John Fastabend wrote:

> If you only steal the prequeue piece then you don't solve the lock
> contention part so I don't think it helps. At which point I suspect
> you might as well use one of the existing qdiscs not designed for
> multiqueue nics.
>

Indeed.


> Yeah well I imagined I would write a rate limiting qdisc to use
> this infrastructure. Jamal hinted at using a systolic processes
> for this. But I work on this when I have time and have been
> busy the last few months with other things unfortunately.

The main problem is you cant avoid locks once you have sharing across
multiple processors. You could try to improve certain things, but
you'll be doing that at the expense of certain use cases; and for
a general purpose OS, it gets hard.
a) netdev: All qdiscs are attached to a netdev. netdevs are shared
across cpus that is if you want the goodies they come with.
If we can ease that, then we may improve the parallelization.
At one point, in a discussion with Eric, it seemed he was heading
towards a per-netdev-ingress-per-cpu (sort of what  multiqueu does for
transmit). Then you can make certain things like netdev stats loosely 
synchronous and rcu would make a lot of sense.
b) graphs of flows and actions are shareable across netdevs and
cpus. Just choose not to share and you can optimize your use case
(at the expense of missing out the sharing features). IOW, this becomes
a config option.

cheers,
jamal