From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99F59C43381 for ; Thu, 28 Feb 2019 17:00:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 71BE7218C3 for ; Thu, 28 Feb 2019 17:00:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732206AbfB1RAr (ORCPT ); Thu, 28 Feb 2019 12:00:47 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58676 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726810AbfB1RAq (ORCPT ); Thu, 28 Feb 2019 12:00:46 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8FCCE316252D; Thu, 28 Feb 2019 17:00:46 +0000 (UTC) Received: from localhost.localdomain (unknown [10.32.181.117]) by smtp.corp.redhat.com (Postfix) with ESMTP id 178C660C4C; Thu, 28 Feb 2019 17:00:44 +0000 (UTC) Message-ID: Subject: Re: [RFC PATCH] packet: rework packet_pick_tx_queue() to use common code selection From: Paolo Abeni To: Willem de Bruijn Cc: Network Development , Saeed Mahameed , Jeff Kirsher , Eric Dumazet , Willem de Bruijn , "David S. Miller" Date: Thu, 28 Feb 2019 18:00:43 +0100 In-Reply-To: References: <397fe96fa8f75483b81cc802191addd1acd01280.1551364536.git.pabeni@redhat.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Thu, 28 Feb 2019 17:00:46 +0000 (UTC) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hi, Thank you for the feedback. On Thu, 2019-02-28 at 11:30 -0500, Willem de Bruijn wrote: > On Thu, Feb 28, 2019 at 10:04 AM Paolo Abeni wrote: > > Currently packet_pick_tx_queue() is the only caller of > > ndo_select_queue() using a fallback argument other than > > __netdev_pick_tx. > > > > Leveraging rx queue, we can obtain a similar queue selection > > behavior using core helpers. After this change, ndo_select_queue() > > is always invoked with __netdev_pick_tx() as fallback. > > We can change ndo_select_queue() signature in a followup patch, > > dropping an indirect call per transmitted packet in some scenarios > > (e.g. TCP syn and XDP generic xmit) > > > > This changes slightly how af packet queue selection happens when > > PACKET_QDISC_BYPASS is set. It's now more similar to plan dev_queue_xmit() > > tacking in account both XPS and TC mapping. > > > > Signed-off-by: Paolo Abeni > > --- > > Note: the main goal here is getting rid of the fallback() indirect > > call in the device drivers implementing ndo_select_queue(). We can > > obtain the same result with the INDIRECT_CALL() harness. Both ways > > we need to export __netdev_pick_tx() but here we avoid the need > > for additional branching. > > --- > > include/linux/netdevice.h | 2 ++ > > net/core/dev.c | 5 +++-- > > net/packet/af_packet.c | 9 +++++++-- > > 3 files changed, 12 insertions(+), 4 deletions(-) > > > > diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h > > index c10b60297d28..29a558bded82 100644 > > --- a/include/linux/netdevice.h > > +++ b/include/linux/netdevice.h > > @@ -2152,6 +2152,8 @@ static inline void netdev_for_each_tx_queue(struct net_device *dev, > > &qdisc_xmit_lock_key); \ > > } > > > > +u16 __netdev_pick_tx(struct net_device *dev, struct sk_buff *skb, > > + struct net_device *sb_dev); > > struct netdev_queue *netdev_pick_tx(struct net_device *dev, > > struct sk_buff *skb, > > struct net_device *sb_dev); > > diff --git a/net/core/dev.c b/net/core/dev.c > > index 2b67f2aa59dd..004d1180671a 100644 > > --- a/net/core/dev.c > > +++ b/net/core/dev.c > > @@ -3704,8 +3704,8 @@ u16 dev_pick_tx_cpu_id(struct net_device *dev, struct sk_buff *skb, > > } > > EXPORT_SYMBOL(dev_pick_tx_cpu_id); > > > > -static u16 __netdev_pick_tx(struct net_device *dev, struct sk_buff *skb, > > - struct net_device *sb_dev) > > +u16 __netdev_pick_tx(struct net_device *dev, struct sk_buff *skb, > > + struct net_device *sb_dev) > > { > > struct sock *sk = skb->sk; > > int queue_index = sk_tx_queue_get(sk); > > @@ -3729,6 +3729,7 @@ static u16 __netdev_pick_tx(struct net_device *dev, struct sk_buff *skb, > > > > return queue_index; > > } > > +EXPORT_SYMBOL(__netdev_pick_tx); > > > > struct netdev_queue *netdev_pick_tx(struct net_device *dev, > > struct sk_buff *skb, > > diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c > > index 8376bc1c1508..1f7ae4a34d27 100644 > > --- a/net/packet/af_packet.c > > +++ b/net/packet/af_packet.c > > @@ -285,14 +285,19 @@ static u16 packet_pick_tx_queue(struct sk_buff *skb) > > { > > struct net_device *dev = skb->dev; > > const struct net_device_ops *ops = dev->netdev_ops; > > + int hint = __packet_pick_tx_queue(dev, skb, NULL); > > > > u16 queue_index; > > > > +#ifdef CONFIG_XPS > > + skb->sender_cpu = hint + 1; > > should this not be raw_smp_processor_id() without the modulo by tx queue count? yes, here sender_cpu is intentionally 'misused' to keep a behavior more similar to the unpatched kernel. I'll drop the modulo in the next iteration. > also, __packet_pick_tx_queue is a one-line wrapper around > dev_pick_tx_cpu_id. If simplifying the entire wrapper can be removed. Yes, ideally that will go with a follow-up patch. dev_pick_tx_cpu_id() is going to loose an argument too, and I'll have to touch this chunk again. I thought that touching it once would reduce noise. > > > > > +#endif > > + skb_record_rx_queue(skb, hint); > > how is this used in the transmit path? __netdev_pick_tx() [no XPS conf in place] -> skb_tx_hash() -> skb_rx_queue_recorded() -> the packet is mapped to (), module tc mappings. > > if (ops->ndo_select_queue) { > > queue_index = ops->ndo_select_queue(dev, skb, NULL, > > - __packet_pick_tx_queue); > > + __netdev_pick_tx); > > queue_index = netdev_cap_txqueue(dev, queue_index); > > } else { > > - queue_index = __packet_pick_tx_queue(dev, skb, NULL); > > + queue_index = __netdev_pick_tx(dev, skb, NULL); > > } > > this now duplicates the core of netdev_pick_tx. I wonder if we can > just call that instead of open coding. > > either by changing to take a struct netdev_queue * (it > only has two callers), or by moving the core into a separate > netdev_pick_tx_core (as __netdev_pick_tx is already used). I tried to minimize the changes in this RFC. I'll go for netdev_pick_tx_core() in the next iteration, if there are no problem with an additional EXPORT_SYMBOL_GPL(). Thanks, Paolo