From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stephen Hemminger Subject: Re: Per-CPU Queueing for QoS Date: Sun, 12 Nov 2017 16:14:31 -0800 Message-ID: <20171112161431.04d45345@xeon-e3> References: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Linux Kernel Network Developers , jianjun.duan@alibaba-inc.com, xiangning.yu@alibaba-inc.com To: Michael Ma Return-path: Received: from mail-pf0-f174.google.com ([209.85.192.174]:54536 "EHLO mail-pf0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751020AbdKMAOd (ORCPT ); Sun, 12 Nov 2017 19:14:33 -0500 Received: by mail-pf0-f174.google.com with SMTP id n89so10616584pfk.11 for ; Sun, 12 Nov 2017 16:14:33 -0800 (PST) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Sun, 12 Nov 2017 13:43:13 -0800 Michael Ma wrote: > Any comments? We plan to implement this as a qdisc and appreciate any early feedback. > > Thanks, > Michael > > > On Nov 9, 2017, at 5:20 PM, Michael Ma wrote: > > > > Currently txq/qdisc selection is based on flow hash so packets from > > the same flow will follow the order when they enter qdisc/txq, which > > avoids out-of-order problem. > > > > To improve the concurrency of QoS algorithm we plan to have multiple > > per-cpu queues for a single TC class and do busy polling from a > > per-class thread to drain these queues. If we can do this frequently > > enough the out-of-order situation in this polling thread should not be > > that bad. > > > > To give more details - in the send path we introduce per-cpu per-class > > queues so that packets from the same class and same core will be > > enqueued to the same place. Then a per-class thread poll the queues > > belonging to its class from all the cpus and aggregate them into > > another per-class queue. This can effectively reduce contention but > > inevitably introduces potential out-of-order issue. > > > > Any concern/suggestion for working towards this direction? In general, there is no meta design discussions in Linux development Several developers have tried to do lockless qdisc and similar things in the past. The devil is in the details, show us the code.