From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oa1-f99.google.com (mail-oa1-f99.google.com [209.85.160.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF6310785 for ; Thu, 11 Jun 2026 17:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.99 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781198871; cv=none; b=FW6i6KERjtmiRp4/rGCaY+TD7wRb0zGK29W2m+mOYrtnKk5uoYDgTbNxZSI8n3ttt9ADcOEiL1R9kKByA5rjYaSRPFdZqYWhgrUWApQn5VlOtt6IyBgOQ3OdkgQeicpMcnQpdZjuN8aH/SwY8PsrhqSq2c1LPX6+QbW0AFILoGc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781198871; c=relaxed/simple; bh=BUtR5MZqv1EIA3dOo5fZmgYJOLCSX7+idufuoPL6GL8=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=iMVjWACIgdkBra++LEN+VUjExtuyMBTP2w73cw1zKciZt8Of5HTtE8D4Z46jEnzf46SviEo30IMBImKilfZQdu+Oe6TrzpMdXnI+jCMU7KguSUJbYp7+p+9w5ZVxO0m7E/nFRYNC6qMhDE+cPMEFKAHdeLBKvqBmlWv3NAd2ay8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=VR8mZqRz; arc=none smtp.client-ip=209.85.160.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="VR8mZqRz" Received: by mail-oa1-f99.google.com with SMTP id 586e51a60fabf-440ebc72386so85211fac.3 for ; Thu, 11 Jun 2026 10:27:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781198867; x=1781803667; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :dkim-signature:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Beizeq6ASxmWY7LEEw/Tb2oGDEMddZUWMtOPsvA8Iig=; b=YaiDcBqixQ+MByjaYqoyEEgoixR9shZ8l3dCz+7FGka0Y6874hc1S1AjJUORDVBKhU 2PYCaO/6OTA2bhvE3MTzHJFv8KBMfA1QGXCBvNc18nmsP2st3b/RHt1EJ62kSeYubdNO l9LD3y9Xak8UI4vit8h3wKhv7KDbH2s2BOCJ/poQ9az5faZTEtxN1tkkOTOG7ZPwtfpA 4GLMquXq7Pj+2fgrlYRxaBS+4gB/hj/r8iEqjpNnwKg5VnAzZM640w9QVmdfyHy2a52j XRGnTgv7n9GexuanbR86zKSNVx8rAjh/jK/7D5+igr4taSH86SPFGAGzRvQ6HZpauEtE 16CQ== X-Forwarded-Encrypted: i=1; AFNElJ+hStf75bra1bfujOWxFwtDmg0auFSqyF7d9OquvUcM4hN5QCPIiDotGYiLcupBjmNroRKZmoUM3A/YuPQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yw2HEXRIBLF6xntSfcQiYF5EboE5AynB+ohDYKr87M6LTEqwU69 yTtk8ikOpf+DJvgQG0mMRTWwosGXvBSDgVYaTgKrwpiS8s6vgF1UIYoM+17NgvbBy+X77kSdPsQ yHWnDvUhz43eGPlOeBEC2d+JPe4uIIr6ckPMRVt9UGeLgp8ow0WNYbJyqD/xCUVvdya3aSU0A1i 2tZEPyacmKvXQxIyaYYkLFZANM+G3MESQd8ViadBlvzdlVn5wzxvegVWXdniWYuF++JFVmUtOF/ Tmhuhcx4TodvnWiMA== X-Gm-Gg: Acq92OEOBtSKJzlBoCEGj3fbPIe06RV6olY7PFP2gls0FYRd3WyN38r0eYbnAuiOogZ PIenakHsjROsRlAeA06Fovbr5d3ulQX5tbTr7z0F0qYY1+iZWYKRZYmb/GtDWFOz0O3iu3s5HJw phxlE/lAxg7mmVuEmwYximaZjNfpTOUw72+oG+3ZBuL+kIA1BMmb7vLJFhoD9htfKzp7Wp4y6/9 D1RR93Xldo3C9QMBDI732V109GbLxTE7P91GEHMyVtW0HbV1hdRIXTtIE7Uk6IAhQ7xEGgSjmR3 GE33uDHj3FTuSPM1+Izjp/HutKCE3L/LPWf5tk7owzajLAIBPwCt3lcKuookub4WFzpRANbslbk ou9uo36dLzQ6jhIjvSdWIX4IC3tcH/cTTAhmuVTqgmzHtJgzKC6loAoUJqa3Ka0AeQN41LCI1S/ MkW0AYBiHzZooSouoiXo6+uIP2KKveFH3t/aC2yNZ1DKDCOYZiE7B16RXBjw== X-Received: by 2002:a05:6870:c102:b0:43b:50fa:1c23 with SMTP id 586e51a60fabf-44241980d3amr2275415fac.13.1781198866475; Thu, 11 Jun 2026 10:27:46 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-117.dlp.protect.broadcom.com. [144.49.247.117]) by smtp-relay.gmail.com with ESMTPS id 586e51a60fabf-44244d1f160sm226473fac.16.2026.06.11.10.27.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jun 2026 10:27:46 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-vk1-f198.google.com with SMTP id 71dfb90a1353d-5a1912d5c85so35814e0c.0 for ; Thu, 11 Jun 2026 10:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1781198864; x=1781803664; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=Beizeq6ASxmWY7LEEw/Tb2oGDEMddZUWMtOPsvA8Iig=; b=VR8mZqRzkSl/oaXYMoNggxwtKK4IEBuV5l68TziKLOqrHkQvivxtIdH8x/Q2yQa9Uy 9fNIlns1woqTTs0ejfbEx+rT9eH4ZB8QjaVVpw+XKjtJtd56jq2QX2wL37rG2j7Xua/y eDKawldUZxoNXw4lz4Y5JTD0IVkUoZbtj82o8= X-Forwarded-Encrypted: i=1; AFNElJ+qbOG1Nnx46uTADXIwFmYYF7gvi8k0frEhowCh8aP2767QEUMSLf+daThgl5kUqRLUko9ZeTdumFi1lUA=@vger.kernel.org X-Received: by 2002:a05:6122:208a:b0:5ab:2fb:2dfb with SMTP id 71dfb90a1353d-5bb02415945mr2399222e0c.8.1781198864629; Thu, 11 Jun 2026 10:27:44 -0700 (PDT) X-Received: by 2002:a05:6122:208a:b0:5ab:2fb:2dfb with SMTP id 71dfb90a1353d-5bb02415945mr2399209e0c.8.1781198864121; Thu, 11 Jun 2026 10:27:44 -0700 (PDT) Received: from [10.14.7.225] ([192.19.161.248]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-96662efcae2sm1518244241.1.2026.06.11.10.27.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 11 Jun 2026 10:27:42 -0700 (PDT) Message-ID: Date: Thu, 11 Jun 2026 10:27:39 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH net] net: bcmgenet: Use weighted round-robin TX DMA arbitration To: Nicolai Buchwitz , Ovidiu Panait Cc: opendmb@gmail.com, florian.fainelli@broadcom.com, bcm-kernel-feedback-list@broadcom.com, andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org References: <20260610085238.56300-1-ovidiu.panait.rb@renesas.com> <2f7e70123887fcfd8bbf4ce92f73574d@tipi-net.de> Content-Language: en-US From: Justin Chen In-Reply-To: <2f7e70123887fcfd8bbf4ce92f73574d@tipi-net.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e On 6/11/26 1:08 AM, Nicolai Buchwitz wrote: > Hi Ovidiu > > This seems to be a good follow-up to Justin's earlier series [1]. > > On 10.6.2026 10:52, Ovidiu Panait wrote: >> Under heavy network traffic, we observed sporadic TX queue timeouts on >> the >> Raspberry Pi 4. The timeouts can be reproduced by stress testing the TX >> path with multiple concurrent iperf UDP streams: >> >>     iperf3 -c -u -b0 -P16 -t60 >>     NETDEV WATCHDOG: CPU: 0: transmit queue 0 timed out 2044 ms >>     NETDEV WATCHDOG: CPU: 3: transmit queue 0 timed out 2004 ms >> >> Investigation showed that the timeouts are caused by the priority-based >> arbiter. Under heavy load the highest priority queue starves the lower >> priority ones, causing timeouts. The TX strict priority arbiter is not >> suitable for the default use case where all the traffic gets spread >> across all the TX queues. >> >> Therefore, to fix this, switch the TX DMA arbiter to Weighted Round- >> Robin, >> which services all queues, so they do not stall. The weights were chosen >> to follow the existing priority scheme: q0 gets the smallest weight, >> while >> q1-4 get the bulk of the TX bandwidth. >> >> Fixes: 1c1008c793fa ("net: bcmgenet: add main driver file") >> Signed-off-by: Ovidiu Panait >> --- > > AFAIK the existing queue mechanism dates back to the STB QoS use cases > Florian > had in mind when he wrote the driver, so let's hear what he has to say > on this. > > The change itself looks correct to me. My concern is the targeting. This > flips the default policy for every GENET user rather than fixing a specific > defect, and Justin's series already called the persistent timeouts a design > issue rather than a bug. > > So isn't this more net-next material than net with a broad Fixes: tag? > Please also add to the commit message that it drops the existing priority > handling for rings 1-4. > I'm ok with these changes. Internally we no longer require priority queues. It is a legacy use case we no longer have. My idea was to remove the queues entirely and have one big queue. But I figured I would wait for Nicolai's XDP changes so we do not need to remove and re-add queues for XDP. This could be a stop-gap solution until that is done. Justin >>  .../net/ethernet/broadcom/genet/bcmgenet.c    | 23 +++++++------------ >>  1 file changed, 8 insertions(+), 15 deletions(-) >> >> diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/ >> net/ethernet/broadcom/genet/bcmgenet.c >> index 7c11cf916762..ad08c67269be 100644 >> --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c >> +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c >> @@ -40,9 +40,8 @@ >> >>  #include "bcmgenet.h" >> >> -/* Default highest priority queue for multi queue support */ >> -#define GENET_Q1_PRIORITY    0 >> -#define GENET_Q0_PRIORITY    1 >> +#define GENET_Q0_WEIGHT        1 >> +#define GENET_Q1_WEIGHT        4 >> >>  #define GENET_Q0_RX_BD_CNT    \ >>      (TOTAL_DESC - priv->hw_params->rx_queues * priv->hw_params- >> >rx_bds_per_q) >> @@ -2129,13 +2128,6 @@ static netdev_tx_t bcmgenet_xmit(struct sk_buff >> *skb, struct net_device *dev) >>      int i; >> >>      index = skb_get_queue_mapping(skb); >> -    /* Mapping strategy: >> -     * queue_mapping = 0, unclassified, packet xmited through ring 0 >> -     * queue_mapping = 1, goes to ring 1. (highest priority queue) >> -     * queue_mapping = 2, goes to ring 2. >> -     * queue_mapping = 3, goes to ring 3. >> -     * queue_mapping = 4, goes to ring 4. >> -     */ >>      ring = &priv->tx_rings[index]; >>      txq = netdev_get_tx_queue(dev, index); >> >> @@ -2881,8 +2873,9 @@ static int bcmgenet_rdma_disable(struct >> bcmgenet_priv *priv) >> >>  /* Initialize Tx queues >>   * >> - * Queues 1-4 are priority-based, each one has 32 descriptors, >> - * with queue 1 being the highest priority queue. >> + * Queues 1-4 are the priority queues, each one has 32 descriptors. >> + * The weighted round-robin arbiter gives them a larger share of TX >> + * bandwidth than the default queue 0. >>   * >>   * Queue 0 is the default Tx queue with >>   * GENET_Q0_TX_BD_CNT = 256 - 4 * 32 = 128 descriptors. >> @@ -2900,8 +2893,8 @@ static void bcmgenet_init_tx_queues(struct >> net_device *dev) >>      unsigned int start = 0, end = GENET_Q0_TX_BD_CNT; >>      u32 i, ring_mask, dma_priority[3] = {0, 0, 0}; >> >> -    /* Enable strict priority arbiter mode */ >> -    bcmgenet_tdma_writel(priv, DMA_ARBITER_SP, DMA_ARB_CTRL); >> +    /* Enable Weighted Round-Robin arbiter mode */ >> +    bcmgenet_tdma_writel(priv, DMA_ARBITER_WRR, DMA_ARB_CTRL); >> >>      /* Initialize Tx priority queues */ >>      for (i = 0; i <= priv->hw_params->tx_queues; i++) { >> @@ -2909,7 +2902,7 @@ static void bcmgenet_init_tx_queues(struct >> net_device *dev) >>          start = end; >>          end += priv->hw_params->tx_bds_per_q; >>          dma_priority[DMA_PRIO_REG_INDEX(i)] |= >> -            (i ? GENET_Q1_PRIORITY : GENET_Q0_PRIORITY) >> +            (i ? GENET_Q1_WEIGHT : GENET_Q0_WEIGHT) >>              << DMA_PRIO_REG_SHIFT(i); >>      } > > [1] https://lore.kernel.org/netdev/20260406175756.134567-1- > justin.chen@broadcom.com/ > > Thanks > Nicolai