From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E43A4314A7E for ; Fri, 7 Nov 2025 23:32:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.15 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762558344; cv=none; b=UAcTAmlabXyygumB7X7TTMgjUueY1gJmTWlB8JquayTSAw+U2srMkRrFAw22SuzpwpH51eQird3wHkeHOO8uo3+BOCUnWDcBfwfdSaGCWiqUJIahSbTjXbgMq870wAt+xU3M+YGrsS5BPVpYWerhYlvQ1bsZkb3hhHmgz1N3XnI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762558344; c=relaxed/simple; bh=A3cflmH1RbhN1nl/QQDNHQ8m7gWHC/i6lO1cSETg5nM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=P3t/ly2sMYPK+OgWgyh0WNMQUpkqjjnJFMF2Bu8VXV4WQtaW5Qx21vVeykVvi5W1Czsv2SZaZV0IecwM3Pc7rEpV4p2KLobezdFVUBW7Hz4TUEeK6qAsMeddMwsQXC7bDlNotVl66a+FoVCbzJFC0UIiL/v7qxJ7ZGDrvSR+y+U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=midlCgB/; arc=none smtp.client-ip=192.198.163.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="midlCgB/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1762558343; x=1794094343; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=A3cflmH1RbhN1nl/QQDNHQ8m7gWHC/i6lO1cSETg5nM=; b=midlCgB/HQOMbIy/Kv7DbPRVa/a+/N5T9US+KPVkj8HndFJ00BCYllqq +gDRusmu5EOm3VT6xWfAXyalNKV3t6SWsCE8v3KXpU5TWkjOtHemDO8x7 jCU4qrRs9/f9CryyOjIAabDG0rt8xOFYuN7Iutev4En7a77kz1hsIcDGc a5+vFWBdoyaR7ZRj/VqRgGn3Zu9laO0vv/wZuySOp/wRahBET77gpALRi FC5EFhOiHJV+WBXZvo1sLMx12WhBV16PXOhoJOLwfx8SAxeB1DaABpKiz QVEl+dFNUa0xUUZkVK3BhhNfQ0TcOQjyPY0mgg5bFgCuW8/XCvSsNUDKm w==; X-CSE-ConnectionGUID: 6vfiFoMgSUe7zISbc5F0HQ== X-CSE-MsgGUID: r87LarX9QHW+/GKJfG4bpA== X-IronPort-AV: E=McAfee;i="6800,10657,11606"; a="64806321" X-IronPort-AV: E=Sophos;i="6.19,288,1754982000"; d="scan'208";a="64806321" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Nov 2025 15:32:19 -0800 X-CSE-ConnectionGUID: CWM0aMPpQCmdks2LzyfJOg== X-CSE-MsgGUID: 3JJ/wRi0TzKbOuvc5p/EWg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.19,288,1754982000"; d="scan'208";a="218815419" Received: from orcnseosdtjek.jf.intel.com (HELO [10.166.28.90]) ([10.166.28.90]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Nov 2025 15:32:20 -0800 From: Jacob Keller Date: Fri, 07 Nov 2025 15:31:46 -0800 Subject: [PATCH iwl-next v3 2/9] ice: use cacheline groups for ice_rx_ring structure Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20251107-jk-refactor-queue-stats-v3-2-771ae1414b2e@intel.com> References: <20251107-jk-refactor-queue-stats-v3-0-771ae1414b2e@intel.com> In-Reply-To: <20251107-jk-refactor-queue-stats-v3-0-771ae1414b2e@intel.com> To: Aleksandr Loktionov , Alexander Lobakin , Tony Nguyen , Przemek Kitszel Cc: Simon Horman , intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, Jacob Keller , Aleksandr Loktionov X-Mailer: b4 0.15-dev-f4b34 X-Developer-Signature: v=1; a=openpgp-sha256; l=6389; i=jacob.e.keller@intel.com; h=from:subject:message-id; bh=A3cflmH1RbhN1nl/QQDNHQ8m7gWHC/i6lO1cSETg5nM=; b=owGbwMvMwCWWNS3WLp9f4wXjabUkhky+xoajmcEuO34XfajexFp6Z8r5CdxF9wx5Qm5d3Ch49 OS0hYslOkpZGMS4GGTFFFkUHEJWXjeeEKb1xlkOZg4rE8gQBi5OAZjIrhJGhonfBS4zFDK4K61q qfqtUPA6+nJUz0ILOVUrqVMsqc96JjMynPtTMC0+PEX8ed/zdeWVE9rtDXOFL5gb2X+do3u2Ybk eMwA= X-Developer-Key: i=jacob.e.keller@intel.com; a=openpgp; fpr=204054A9D73390562AEC431E6A965D3E6F0F28E8 The ice ring structure was reorganized back by commit 65124bbf980c ("ice: Reorganize tx_buf and ring structs"), and later split into a separate ice_rx_ring structure by commit e72bba21355d ("ice: split ice_ring onto Tx/Rx separate structs") The ice_rx_ring structure has comments left over from this prior reorganization indicating which fields belong to which cachelines. Unfortunately, these comments are not all accurate. The intended layout is for x86_64 systems with a 64-byte cache. * Cacheline 1 spans from the start of the struct to the end of the rx_fqes and xdp_buf union. The comments correctly match this. * Cacheline 2 spans from hdr_fqes to the end of hdr_truesize, but the comment indicates it should end xdp and xsk union. * Cacheline 3 spans from the truesize field to the xsk_pool, but the comment wants this to be from the pkt_ctx down to the rcu head field. * Cacheline 4 spans from the rx_hdr_len down to the flags field, but the comment indicates that it starts back at the ice_channel structure pointer. * Cacheline 5 is indicated to cover the xdp_rxq. Because this field is aligned to 64 bytes, this is actually true. However, there is a large 45 byte gap at the end of cacheline 4. The use of comments to indicate cachelines is poor design. In practice, comments like this quickly become outdated as developers add or remove fields, or as other sub-structures change layout and size unexpectedly. The ice_rx_ring structure *is* 5 cachelines (320 bytes), but ends up having quite a lot of empty space at the end just before xdp_rxq. Replace the comments with __cacheline_group_(begin|end)_aligned() annotations. These macros enforce alignment to the start of cachelines, and enforce padding between groups, thus guaranteeing that a following group cannot be part of the same cacheline. Doing this changes the layout by effectively spreading the padding at the tail of cacheline 4 between groups to ensure that the relevant fields are kept as separate cachelines on x86_64 systems. For systems with the expected cache line size of 64 bytes, the structure size does not change. For systems with a larger SMB_CACHE_BYTES size, the structure *will* increase in size a fair bit, however we'll now guarantee that each group is in a separate cacheline. This has an advantage that updates to fields in one group won't trigger cacheline eviction of the other groups. This comes at the expense of extra memory footprint for the rings. If fields get re-arranged, added, or removed, the alignment and padding ensure the relevant fields are kept on separate cache lines. This could result in unexpected changes in the structure size due to padding to keep cachelines separate. To catch such changes during early development, add build time compiler assertions that check the size of each group to ensure it doesn't exceed the size of one cache line. This will produce a compile failure in the event that new fields get added or re-arranged such that any of the cache groups exceed one cacheline. Reviewed-by: Aleksandr Loktionov Signed-off-by: Jacob Keller --- drivers/net/ethernet/intel/ice/ice_txrx.h | 26 +++++++++++++++++++++----- drivers/net/ethernet/intel/ice/ice_main.c | 2 ++ 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h index e440c55d9e9f..8b9247535fcb 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.h +++ b/drivers/net/ethernet/intel/ice/ice_txrx.h @@ -236,7 +236,7 @@ struct ice_tstamp_ring { } ____cacheline_internodealigned_in_smp; struct ice_rx_ring { - /* CL1 - 1st cacheline starts here */ + __cacheline_group_begin_aligned(cl1); void *desc; /* Descriptor ring memory */ struct page_pool *pp; struct net_device *netdev; /* netdev ring maps to */ @@ -253,8 +253,9 @@ struct ice_rx_ring { struct libeth_fqe *rx_fqes; struct xdp_buff **xdp_buf; }; + __cacheline_group_end_aligned(cl1); - /* CL2 - 2nd cacheline starts here */ + __cacheline_group_begin_aligned(cl2); struct libeth_fqe *hdr_fqes; struct page_pool *hdr_pp; @@ -262,8 +263,9 @@ struct ice_rx_ring { struct libeth_xdp_buff_stash xdp; struct libeth_xdp_buff *xsk; }; + __cacheline_group_end_aligned(cl2); - /* CL3 - 3rd cacheline starts here */ + __cacheline_group_begin_aligned(cl3); union { struct ice_pkt_ctx pkt_ctx; struct { @@ -284,7 +286,9 @@ struct ice_rx_ring { struct ice_ring_stats *ring_stats; struct rcu_head rcu; /* to avoid race on free */ - /* CL4 - 4th cacheline starts here */ + __cacheline_group_end_aligned(cl3); + + __cacheline_group_begin_aligned(cl4); struct ice_channel *ch; struct ice_tx_ring *xdp_ring; struct ice_rx_ring *next; /* pointer to next ring in q_vector */ @@ -298,10 +302,22 @@ struct ice_rx_ring { #define ICE_RX_FLAGS_MULTIDEV BIT(3) #define ICE_RX_FLAGS_RING_GCS BIT(4) u8 flags; - /* CL5 - 5th cacheline starts here */ + __cacheline_group_end_aligned(cl4); + + __cacheline_group_begin_aligned(cl5); struct xdp_rxq_info xdp_rxq; + __cacheline_group_end_aligned(cl5); } ____cacheline_internodealigned_in_smp; +static inline void ice_rx_ring_struct_check(void) +{ + CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl1, SMP_CACHE_BYTES); + CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl2, SMP_CACHE_BYTES); + CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl3, SMP_CACHE_BYTES); + CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl4, SMP_CACHE_BYTES); + CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl5, SMP_CACHE_BYTES); +} + struct ice_tx_ring { /* CL1 - 1st cacheline starts here */ struct ice_tx_ring *next; /* pointer to next ring in q_vector */ diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index b16ede1f139d..4731dbaca9de 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -5916,6 +5916,8 @@ static int __init ice_module_init(void) { int status = -ENOMEM; + ice_rx_ring_struct_check(); + pr_info("%s\n", ice_driver_string); pr_info("%s\n", ice_copyright); -- 2.51.0.rc1.197.g6d975e95c9d7