From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FB21314D3E for ; Thu, 27 Nov 2025 20:44:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764276279; cv=none; b=cu0J2sBcs0P8BrAIP6Z2tPrfHq8enqvEl3CNrPmU4vn7VDnDsmpVPrsVBKna7kVa7VLgaOVS+KucXX4uXUygyg08wXsIjytdxy+sIMHjFkAoqXBwfudRcW++C4TxgTdYSccNuhSXfrZK+Gc8QU7NhyWNO48EQ99NiF1xl2Mzkvc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764276279; c=relaxed/simple; bh=s+ujYG85N1S/vvBzu10d4al31eQMM3yatlYRBU38WTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pfIQW3ltSglEdDsWmEJxwRPeSxvAwXV8vcJ40X71FGMvJ84WGPbgFbZ8fVOVCiceOUQdHzJiCuB5l+L6+wKYOqUsrb3L7vFYqAkBN9VedY+1W7p4BuaJpgvXjCdOtTuZNwNOcMPOahOwsGld81i6dy0K3BFQZmv6MO146b6KocU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Z9ym8mhZ; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Z9ym8mhZ" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-42b2e9ac45aso792183f8f.0 for ; Thu, 27 Nov 2025 12:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764276276; x=1764881076; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B5qoAELZ8wHHvykz/cZa6mBE+envmBXhX8Q5+VASiMI=; b=Z9ym8mhZDrOtMBje+/v2PKWH6O/lnU7rf2xVi5fiXPfMmmfyZOrc8poi+q1e6gj7tD MFVCxhvKDhJ56HjohOJjC3BcL4E6lYhWLyuf4VR58MbEAt7Z+R9fOBt7pGeu/UDSq5NI +kA7TBVZDbAPBbA7Mi0CKg6lRut47DB13iGcP9RjM/AppTFvBuIhY+UTp2boOg9QM8dM YMZh0SiJxpIBXbdVc/PRhXSiPV4YnwyOb3AXmbOIIKizS9guIWLGNRztJmeZcURukkLS M6gKEC6GirjPJou50RXjwpEAdJ8HDfUZ/VxyMgVRlKJrIbPgnVFxVKgaCYi7HuCiMxwM ROtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764276276; x=1764881076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=B5qoAELZ8wHHvykz/cZa6mBE+envmBXhX8Q5+VASiMI=; b=bSUK3Is2nUU7XGlbI4zpLpz8Z7e+zo2u6tUOX4/4NohMhzDlaaJ0D34LU01ddzm5DD bavflschbhOodIPDuo+wHUnwT/T17n8HS0Qg//MM+SnUKR6btH8AmSUSyOngkJggDQ5p /JMHqbzgARg1d5I8gu7pInGTZ702nC4TLjkEnxLP8TWg0uEfgYHReRmlXtPLfJQY1VUw HsFdD0coPU/8fxg3GQhNcuPR6FWrADHxaUwE49Ik0EtOhgq4UFl8yrun06pBV0w2uBHT RpnXcprExB16M+bSbCdtSG+lfmaiy1nvactwrOTVqAUgks720b+EQWLdhPabPw/fQzVd 9bUA== X-Forwarded-Encrypted: i=1; AJvYcCWJmEfmUdbTWks/v2gW4S53QKzYYTX3gj35arpF5ss0cJS8FDzOPySuSw+z+Ioz3DhzjYvCj2N7FmY=@vger.kernel.org X-Gm-Message-State: AOJu0Yx3jHaFLT/lErd5lRTszt63ySTvGMwkJUuA16zHEBqK3IvUAm7D tQapIncxjTvSsjmAqvGAxM6PTZGI6UMEWfKcmS7skKyb5SAg1+fwoPSH X-Gm-Gg: ASbGnctaCD/vBL0zt8jH3iSgwjPZax0EwrATTEWsH3+XtP9fvPBf9V38dlRxvjYIsIo sR26VoPIPtvgzDanYn7d7Sxfq0dp6Gk0CFysYKCXSnUzHo/Rn4hrs2mVAFcwaK+i9n1EUaBtaP9 6d0sQjseKVtae4JpI6Q7nywevLtHkSGKSHY0KruBcJqipcO8VFy3S4z3F3MN02Gbaj/S+8lZyez 60mLPDBkyQV2wtrXxaO2dwP9RuaehpkcttLwbZ20EX29dflaIrXumCwjznKlXHX7yFJ8Oc4g0e9 KjXNfc3RAAXzqmB1f2pFdHITklMkGhtOs7lk6XMBQBiqhfKKFPUJnGxgz17kX3LeTnUTGnuqMAS KBnIvPS+x7JHgmNrkLu21gv3sBitqvRlozPbgdcT1aafe4L8ZXauCNo9vqPzdXYaWgkESAgd5cD 8uZA80qExCC/eL4Q== X-Google-Smtp-Source: AGHT+IEyaVmnfDxtl2rFThdCbalOVXR3AbBBiusuPnSeb0Ao82EGUMzRsDZmj1oPm2zoagTXUFdGmQ== X-Received: by 2002:a05:6000:1862:b0:42b:3ee9:4771 with SMTP id ffacd0b85a97d-42e0f22a21dmr13385644f8f.23.1764276275904; Thu, 27 Nov 2025 12:44:35 -0800 (PST) Received: from 127.mynet ([2a01:4b00:bd21:4f00:7cc6:d3ca:494:116c]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-42e1c5d614asm5346105f8f.12.2025.11.27.12.44.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 12:44:34 -0800 (PST) From: Pavel Begunkov To: netdev@vger.kernel.org Cc: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Michael Chan , Pavan Chebbi , Andrew Lunn , Ilias Apalodimas , Stanislav Fomichev , Pavel Begunkov , Jens Axboe , Simon Horman , linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, io-uring@vger.kernel.org, dtatulea@nvidia.com Subject: [PATCH net-next v6 5/8] eth: bnxt: adjust the fill level of agg queues with larger buffers Date: Thu, 27 Nov 2025 20:44:18 +0000 Message-ID: X-Mailer: git-send-email 2.52.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-doc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jakub Kicinski The driver tries to provision more agg buffers than header buffers since multiple agg segments can reuse the same header. The calculation / heuristic tries to provide enough pages for 65k of data for each header (or 4 frags per header if the result is too big). This calculation is currently global to the adapter. If we increase the buffer sizes 8x we don't want 8x the amount of memory sitting on the rings. Luckily we don't have to fill the rings completely, adjust the fill level dynamically in case particular queue has buffers larger than the global size. Signed-off-by: Jakub Kicinski [pavel: rebase on top of agg_size_fac, assert agg_size_fac] Signed-off-by: Pavel Begunkov --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 28 +++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 091da26fdf7c..fc88566779a4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3816,16 +3816,34 @@ static void bnxt_free_rx_rings(struct bnxt *bp) } } +static int bnxt_rx_agg_ring_fill_level(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr) +{ + /* User may have chosen larger than default rx_page_size, + * we keep the ring sizes uniform and also want uniform amount + * of bytes consumed per ring, so cap how much of the rings we fill. + */ + int fill_level = bp->rx_agg_ring_size; + + if (rxr->rx_page_size > BNXT_RX_PAGE_SIZE) + fill_level /= rxr->rx_page_size / BNXT_RX_PAGE_SIZE; + + return fill_level; +} + static int bnxt_alloc_rx_page_pool(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, int numa_node) { - const unsigned int agg_size_fac = PAGE_SIZE / BNXT_RX_PAGE_SIZE; + unsigned int agg_size_fac = rxr->rx_page_size / BNXT_RX_PAGE_SIZE; const unsigned int rx_size_fac = PAGE_SIZE / SZ_4K; struct page_pool_params pp = { 0 }; struct page_pool *pool; - pp.pool_size = bp->rx_agg_ring_size / agg_size_fac; + if (WARN_ON_ONCE(agg_size_fac == 0)) + agg_size_fac = 1; + + pp.pool_size = bnxt_rx_agg_ring_fill_level(bp, rxr) / agg_size_fac; if (BNXT_RX_PAGE_MODE(bp)) pp.pool_size += bp->rx_ring_size / rx_size_fac; @@ -4403,11 +4421,13 @@ static void bnxt_alloc_one_rx_ring_netmem(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, int ring_nr) { + int fill_level, i; u32 prod; - int i; + + fill_level = bnxt_rx_agg_ring_fill_level(bp, rxr); prod = rxr->rx_agg_prod; - for (i = 0; i < bp->rx_agg_ring_size; i++) { + for (i = 0; i < fill_level; i++) { if (bnxt_alloc_rx_netmem(bp, rxr, prod, GFP_KERNEL)) { netdev_warn(bp->dev, "init'ed rx ring %d with %d/%d pages only\n", ring_nr, i, bp->rx_agg_ring_size); -- 2.52.0