From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 795D9CA0EC4 for ; Tue, 12 Aug 2025 07:55:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 492DE613D6; Tue, 12 Aug 2025 07:55:26 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id nRf5grF7a24R; Tue, 12 Aug 2025 07:55:25 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5923061417 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1754985325; bh=nua+Saoq3cgdpcV2SyDKodrQAeZWBLQm1joHhSepEdg=; h=From:To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=W9KRqBE7txoUZ56sluL6UBzS5sFciBdqQO5K9S2Dc3Ft2UvDpItDuTPe+ExHtuH6l CKxStn3pxCMneLw3vQQQDV0GpDmxzbpIhYn2i+QNDL6NNQToP9yWgVgguo/nWXm6EU vstKBQmjQD29GnBzRU9r1pYK27nv7eyckSXmXjE1NcKnL3Xkr0Tj23omhqh3HaQKTc FWzj8oJtRLWF7D8SaoMvF/S5kxPxmHpxvBDnaBtzZy8KpCJCy4AHKZWAykQMJZEH1Q IRAwOCoCUeO86xfN0SOSWdSokB0XIA5JY2vYXt8WrjDAMqePONrwS7JUDFLM2EE65c X3TpdBd+xpHHg== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp3.osuosl.org (Postfix) with ESMTP id 5923061417; Tue, 12 Aug 2025 07:55:25 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists1.osuosl.org (Postfix) with ESMTP id E27DDB8B for ; Tue, 12 Aug 2025 07:55:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C906061417 for ; Tue, 12 Aug 2025 07:55:23 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id rRM5z_kCfy0Q for ; Tue, 12 Aug 2025 07:55:23 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::630; helo=mail-pl1-x630.google.com; envelope-from=kerneljasonxing@gmail.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org E115C61422 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E115C61422 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by smtp3.osuosl.org (Postfix) with ESMTPS id E115C61422 for ; Tue, 12 Aug 2025 07:55:22 +0000 (UTC) Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-2406fe901c4so32132035ad.1 for ; Tue, 12 Aug 2025 00:55:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754985322; x=1755590122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nua+Saoq3cgdpcV2SyDKodrQAeZWBLQm1joHhSepEdg=; b=qZ3CFcziyMhAMPNqO4w7G1nf2usOucu0Dronr+91+wEWww9ePug3TVKvxOSTS7nHgN Ib2SLguTuRtBupT9xDM0bf12NFz2DPVn2c2+5DXvoJxLZZlSA06BC69KTaL6MvatSAds sgEwvAaSUsim8agDAFzBd/fQWT8v1kFCnC0h7SAmYcXBnWPuyDnp+J6TaHbUt/XA3pun ZUHTB/BC9eAjY6UsDdG5aLpviXU7EqR6nl6XIjrAxti1dDSHl2/SmgyTa+UoWA9OGgwN JVqj16cBbVYYfrnK0Ppw82IhjHauGfrMagl7eey9Us1bVbTHVRgrTaHI+HEOcpDyDJYV nkPA== X-Gm-Message-State: AOJu0YwkIltXt59t/foiV+9Z2oMXqfFmvzArWksSl+mvMic3r2lOePDv BRIv8UaPWirh6CeuaQQL19ZgbMDAF2/izKi1hK6o5QJMweDQ70NQBux2v0JxwRtqdUs= X-Gm-Gg: ASbGnctIscBETNmtg46YrP2PCtxTGqZBWSbmnfc3sZEUGUBfQG2g0tYDd7jpkLjdAWs jG7n0N75dj5hNqj14XXTQaPf7ke7XKiS8DFrPPzBybWV/B9VaDB8mivyXLyikMWCFX3a52o/hTq 0NnKcsXlr9tjbzo3Mr4gEXKiF2XpQb95HSglpgWWo72O/ovSPbPqGLSxvEDZb5Z6zznIDKITYQ5 Opqvp1hpCmE0EKTY+posItbrF/ZoPf4kDldySInbhG7xyxX+tRoqfp+kGC5ncZYK5ojkygFULQu +Doa0prQZxgRb9cjjYKZ6JpZyhC55j14UKCpUFuNL5CLEolxHyXYgl0cN9NsQpHdkH1o02uUOaT am9jF0A/yr5N4POG39yBGrOuwa6Vr7oYftT+3E24WWIsZEUdmJh5a5OEWhgdZffIDdLfe3xBFOt ikE23p X-Google-Smtp-Source: AGHT+IF56R6elMTGuSLc1ld1mjtxGYvTuPLn+GhWA2a1eQT6shTy4jObTSd5VfzNjYgDUxkBjIuwYg== X-Received: by 2002:a17:903:2307:b0:240:a559:be6a with SMTP id d9443c01a7336-242fc33d8b2mr36827355ad.34.1754985322047; Tue, 12 Aug 2025 00:55:22 -0700 (PDT) Received: from KERNELXING-MB0.tencent.com ([43.132.141.21]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b422bac12d4sm24651320a12.32.2025.08.12.00.55.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Aug 2025 00:55:21 -0700 (PDT) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, andrew+netdev@lunn.ch, anthony.l.nguyen@intel.com, przemyslaw.kitszel@intel.com, sdf@fomichev.me, larysa.zaremba@intel.com, maciej.fijalkowski@intel.com Cc: intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, Jason Xing Date: Tue, 12 Aug 2025 15:55:04 +0800 Message-Id: <20250812075504.60498-4-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250812075504.60498-1-kerneljasonxing@gmail.com> References: <20250812075504.60498-1-kerneljasonxing@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754985322; x=1755590122; darn=lists.osuosl.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nua+Saoq3cgdpcV2SyDKodrQAeZWBLQm1joHhSepEdg=; b=DKaVaofoAjJk/wA7HuCx/HqQImpWMrlt3w0g2ynexW7RvLGHbrQDL9uNSVeCQX2xEr kuZKSswjXtEZ2kqpk7PNwl6eJrJw7y8+NP88MNNuvZvjK+Es5CTLJGmw6uv7ZKkxvXrz 0LhdkL0YHVpLVKsQ4NMcTaZtgPGlS21CL5fwe9CR+sPr3knIwIh+yBnQfqGQeSWGkhds CEJ+VQsS5k1XHqKAXhiB+uxKrq812wJ3p4PGqfHQR6Gvw+4GBUdQa076IN3M67HAXMZ9 WxTiZhrXejpGhJEoe2QWOv1yUI9oQ0l8H+F+9IcmagYyiAV4xvaGLABk3/644zAhHt5F QR/g== X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=gmail.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=DKaVaofo Subject: [Intel-wired-lan] [PATCH iwl-net v2 3/3] ixgbe: xsk: support batched xsk Tx interfaces to increase performance X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" From: Jason Xing Like what i40e driver initially did in commit 3106c580fb7cf ("i40e: Use batched xsk Tx interfaces to increase performance"), use the batched xsk feature to transmit packets. Signed-off-by: Jason Xing --- In this version, I still choose use the current implementation. Last time at the first glance, I agreed 'i' is useless but it is not. https://lore.kernel.org/intel-wired-lan/CAL+tcoADu-ZZewsZzGDaL7NugxFTWO_Q+7WsLHs3Mx-XHjJnyg@mail.gmail.com/ --- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 106 +++++++++++++------ 1 file changed, 72 insertions(+), 34 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index f3d3f5c1cdc7..9fe2c4bf8bc5 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -2,12 +2,15 @@ /* Copyright(c) 2018 Intel Corporation. */ #include +#include #include #include #include "ixgbe.h" #include "ixgbe_txrx_common.h" +#define PKTS_PER_BATCH 4 + struct xsk_buff_pool *ixgbe_xsk_pool(struct ixgbe_adapter *adapter, struct ixgbe_ring *ring) { @@ -388,58 +391,93 @@ void ixgbe_xsk_clean_rx_ring(struct ixgbe_ring *rx_ring) } } -static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) +static void ixgbe_set_rs_bit(struct ixgbe_ring *xdp_ring) +{ + u16 ntu = xdp_ring->next_to_use ? xdp_ring->next_to_use - 1 : xdp_ring->count - 1; + union ixgbe_adv_tx_desc *tx_desc; + + tx_desc = IXGBE_TX_DESC(xdp_ring, ntu); + tx_desc->read.cmd_type_len |= cpu_to_le32(IXGBE_TXD_CMD_RS); +} + +static void ixgbe_xmit_pkt(struct ixgbe_ring *xdp_ring, struct xdp_desc *desc, + int i) + { struct xsk_buff_pool *pool = xdp_ring->xsk_pool; union ixgbe_adv_tx_desc *tx_desc = NULL; struct ixgbe_tx_buffer *tx_bi; - struct xdp_desc desc; dma_addr_t dma; u32 cmd_type; - if (!budget) - return true; + dma = xsk_buff_raw_get_dma(pool, desc[i].addr); + xsk_buff_raw_dma_sync_for_device(pool, dma, desc[i].len); - while (likely(budget)) { - if (!netif_carrier_ok(xdp_ring->netdev)) - break; + tx_bi = &xdp_ring->tx_buffer_info[xdp_ring->next_to_use]; + tx_bi->bytecount = desc[i].len; + tx_bi->xdpf = NULL; + tx_bi->gso_segs = 1; - if (!xsk_tx_peek_desc(pool, &desc)) - break; + tx_desc = IXGBE_TX_DESC(xdp_ring, xdp_ring->next_to_use); + tx_desc->read.buffer_addr = cpu_to_le64(dma); - dma = xsk_buff_raw_get_dma(pool, desc.addr); - xsk_buff_raw_dma_sync_for_device(pool, dma, desc.len); + cmd_type = IXGBE_ADVTXD_DTYP_DATA | + IXGBE_ADVTXD_DCMD_DEXT | + IXGBE_ADVTXD_DCMD_IFCS; + cmd_type |= desc[i].len | IXGBE_TXD_CMD_EOP; + tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type); + tx_desc->read.olinfo_status = + cpu_to_le32(desc[i].len << IXGBE_ADVTXD_PAYLEN_SHIFT); - tx_bi = &xdp_ring->tx_buffer_info[xdp_ring->next_to_use]; - tx_bi->bytecount = desc.len; - tx_bi->xdpf = NULL; - tx_bi->gso_segs = 1; + xdp_ring->next_to_use++; +} - tx_desc = IXGBE_TX_DESC(xdp_ring, xdp_ring->next_to_use); - tx_desc->read.buffer_addr = cpu_to_le64(dma); +static void ixgbe_xmit_pkt_batch(struct ixgbe_ring *xdp_ring, struct xdp_desc *desc) +{ + u32 i; - /* put descriptor type bits */ - cmd_type = IXGBE_ADVTXD_DTYP_DATA | - IXGBE_ADVTXD_DCMD_DEXT | - IXGBE_ADVTXD_DCMD_IFCS; - cmd_type |= desc.len | IXGBE_TXD_CMD; - tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type); - tx_desc->read.olinfo_status = - cpu_to_le32(desc.len << IXGBE_ADVTXD_PAYLEN_SHIFT); + unrolled_count(PKTS_PER_BATCH) + for (i = 0; i < PKTS_PER_BATCH; i++) + ixgbe_xmit_pkt(xdp_ring, desc, i); +} - xdp_ring->next_to_use++; - if (xdp_ring->next_to_use == xdp_ring->count) - xdp_ring->next_to_use = 0; +static void ixgbe_fill_tx_hw_ring(struct ixgbe_ring *xdp_ring, + struct xdp_desc *descs, u32 nb_pkts) +{ + u32 batched, leftover, i; + + batched = nb_pkts & ~(PKTS_PER_BATCH - 1); + leftover = nb_pkts & (PKTS_PER_BATCH - 1); + for (i = 0; i < batched; i += PKTS_PER_BATCH) + ixgbe_xmit_pkt_batch(xdp_ring, &descs[i]); + for (i = batched; i < batched + leftover; i++) + ixgbe_xmit_pkt(xdp_ring, &descs[i], 0); +} - budget--; - } +static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) +{ + struct xdp_desc *descs = xdp_ring->xsk_pool->tx_descs; + u32 nb_pkts, nb_processed = 0; - if (tx_desc) { - ixgbe_xdp_ring_update_tail(xdp_ring); - xsk_tx_release(pool); + if (!netif_carrier_ok(xdp_ring->netdev)) + return true; + + nb_pkts = xsk_tx_peek_release_desc_batch(xdp_ring->xsk_pool, budget); + if (!nb_pkts) + return true; + + if (xdp_ring->next_to_use + nb_pkts >= xdp_ring->count) { + nb_processed = xdp_ring->count - xdp_ring->next_to_use; + ixgbe_fill_tx_hw_ring(xdp_ring, descs, nb_processed); + xdp_ring->next_to_use = 0; } - return !!budget; + ixgbe_fill_tx_hw_ring(xdp_ring, &descs[nb_processed], nb_pkts - nb_processed); + + ixgbe_set_rs_bit(xdp_ring); + ixgbe_xdp_ring_update_tail(xdp_ring); + + return nb_pkts < budget; } static void ixgbe_clean_xdp_tx_buffer(struct ixgbe_ring *tx_ring, -- 2.41.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0933F2DEA7E for ; Tue, 12 Aug 2025 07:55:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754985325; cv=none; b=QapCofCoPSZCxAf/aeF8+i2NeX7xafgdDTWe6Ep5V8r5oz+L1/TunU9oXE0v8Kmv1KqeGG9hLjk6lJFaOgQywlCppUABLMXA5KeGinYHCKFlHdod91h4F48vKdUnQjK/f3NU5Qo+hhLuTLeje2xll3vqsy1tY/iC3byFYFS7qsU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754985325; c=relaxed/simple; bh=mrhy9PQiLCJM+jUKKR0YXlw0H0B+XyoLb0acCxub5JA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BqpapDKfXRuPx3G0E6boarsMchpoGhbmFrg/YNHRlyY/HYcHneCQwKzLSmebFz1aS6VcjTCEl87dHYL0THXune6qo8nHcvcPD0iwX3Xsf208PnB+D7N+JyRhN3S62MOgv1/gO+ipbwUTPlVqS81UAAEFnYwCLBIbkJfy1qPStCY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GXL0/bnd; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GXL0/bnd" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-b4233978326so2891316a12.1 for ; Tue, 12 Aug 2025 00:55:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754985322; x=1755590122; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nua+Saoq3cgdpcV2SyDKodrQAeZWBLQm1joHhSepEdg=; b=GXL0/bndnOcyVV6ZZrpl60u8C/ZKu/657t/HlIcfm78bzdiQo1uL9rzYye2rb08BMW 8nrrdX5HDxXckkp2Yeqw08reUwK5t86EmVGj8T+FejKrSFSxSQ+RL3IBzFSDIuPgzbG9 Un2AZFgo0xdtUnelGF38tiFhYg9GabxMecRx0atjoB3ZpqFHGht0IdVFnqppHdKeujvo DvZs6dZ/njlenW0FYdKO7thtR5/pB2n7kOAr6LcEnw8O+u8VARe/H/8e12ouLi+ABhI8 ylvA1AMaLXIq+5toMatqoD4XOihd2lq/10/FgksFyOWXYOLODoJBT9iqUBVTYsVW0/Hg lqDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754985322; x=1755590122; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nua+Saoq3cgdpcV2SyDKodrQAeZWBLQm1joHhSepEdg=; b=QYBEOW8Remtj9A8yHZ0TXPzTP+d3JDAJLkU/ro/gdTRoym+h40APcURoyswDoMH6kn D6yh7qN+xOWrz49QXYiRAQDfj3lrfwHe8/odHClx34LRYI7wtjHB/dtj4tu7Pjl3FQFv DUjwXSGQHkY95i7wpOsfvMxJ4BYAp7SwX9JUFkgTCT48LntuTNJyeCcyKVC51jjjZqCC BpbInlSwB2KguiJGBl4A5D9MsSFWfz/mcinTMdIqnFe8WoxSzgwauWxhK0Juz5OK0qmW so2A1BD9TitvWymPjRiHMCH4QTJ3CDlrl2I89zDlG+6gO0PaKmYDOl8+DkqQ3G4/36c7 hllQ== X-Forwarded-Encrypted: i=1; AJvYcCXyV1gE95PH8XM72yzUPF607pfNF/VanZA6c+y99QBjj/BlEfuhr4shZOYFtoguMcipZOUiBGA=@vger.kernel.org X-Gm-Message-State: AOJu0YyGpRosH8Hsy5Hm9Q0aw49uurDJSYZheEpzlsBJdlq3ZLrrSRwF HkSxBne0j6CCCF+ZugEuiPSAusvfP2us5InUZzhW8kd1e9Zp2n3MGtqJ X-Gm-Gg: ASbGncuWf4SYSSbzPlLUkFj7wNT7gv9PfmGTx33B8bbABicL7nMZvCx416DcN8wujkC ln+RDkXLXvs/hAF5Hb0f/YpA6YYO9ck2DMeztRa1eoR6IQM0M+wCrZftnl79kgZbAEopLZm+HBj IBZaCGnYk/syvZ/8eqKC8j3mNIkp+NKm3sqseXKmiZ4h6K4k8on5kRtWs1zmcw0OLlvh0bI+ISs ELxE0ZCPFesZRzNcad82HCLH7BDV5PJMOhnnlNjbYX4Bl0p7uNaX8DOU8hBwYqqGlIJnHRyKJRZ fEDOtHCmTNN6c7psKnyn+yLcKsKc1wdY/1LlkupFm/seY1kuLpcqaOC4tGruKe0yyebHPC4ylbq VExO9jJEEY/KG2V51z4J7jwstvvV/QlZmayCOiH9j2KJseyXgrImZIWpDhbOnJOWXTM6yQAWN4O 8ESOzS X-Google-Smtp-Source: AGHT+IF56R6elMTGuSLc1ld1mjtxGYvTuPLn+GhWA2a1eQT6shTy4jObTSd5VfzNjYgDUxkBjIuwYg== X-Received: by 2002:a17:903:2307:b0:240:a559:be6a with SMTP id d9443c01a7336-242fc33d8b2mr36827355ad.34.1754985322047; Tue, 12 Aug 2025 00:55:22 -0700 (PDT) Received: from KERNELXING-MB0.tencent.com ([43.132.141.21]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-b422bac12d4sm24651320a12.32.2025.08.12.00.55.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Aug 2025 00:55:21 -0700 (PDT) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, andrew+netdev@lunn.ch, anthony.l.nguyen@intel.com, przemyslaw.kitszel@intel.com, sdf@fomichev.me, larysa.zaremba@intel.com, maciej.fijalkowski@intel.com Cc: intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH iwl-net v2 3/3] ixgbe: xsk: support batched xsk Tx interfaces to increase performance Date: Tue, 12 Aug 2025 15:55:04 +0800 Message-Id: <20250812075504.60498-4-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250812075504.60498-1-kerneljasonxing@gmail.com> References: <20250812075504.60498-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jason Xing Like what i40e driver initially did in commit 3106c580fb7cf ("i40e: Use batched xsk Tx interfaces to increase performance"), use the batched xsk feature to transmit packets. Signed-off-by: Jason Xing --- In this version, I still choose use the current implementation. Last time at the first glance, I agreed 'i' is useless but it is not. https://lore.kernel.org/intel-wired-lan/CAL+tcoADu-ZZewsZzGDaL7NugxFTWO_Q+7WsLHs3Mx-XHjJnyg@mail.gmail.com/ --- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 106 +++++++++++++------ 1 file changed, 72 insertions(+), 34 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index f3d3f5c1cdc7..9fe2c4bf8bc5 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -2,12 +2,15 @@ /* Copyright(c) 2018 Intel Corporation. */ #include +#include #include #include #include "ixgbe.h" #include "ixgbe_txrx_common.h" +#define PKTS_PER_BATCH 4 + struct xsk_buff_pool *ixgbe_xsk_pool(struct ixgbe_adapter *adapter, struct ixgbe_ring *ring) { @@ -388,58 +391,93 @@ void ixgbe_xsk_clean_rx_ring(struct ixgbe_ring *rx_ring) } } -static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) +static void ixgbe_set_rs_bit(struct ixgbe_ring *xdp_ring) +{ + u16 ntu = xdp_ring->next_to_use ? xdp_ring->next_to_use - 1 : xdp_ring->count - 1; + union ixgbe_adv_tx_desc *tx_desc; + + tx_desc = IXGBE_TX_DESC(xdp_ring, ntu); + tx_desc->read.cmd_type_len |= cpu_to_le32(IXGBE_TXD_CMD_RS); +} + +static void ixgbe_xmit_pkt(struct ixgbe_ring *xdp_ring, struct xdp_desc *desc, + int i) + { struct xsk_buff_pool *pool = xdp_ring->xsk_pool; union ixgbe_adv_tx_desc *tx_desc = NULL; struct ixgbe_tx_buffer *tx_bi; - struct xdp_desc desc; dma_addr_t dma; u32 cmd_type; - if (!budget) - return true; + dma = xsk_buff_raw_get_dma(pool, desc[i].addr); + xsk_buff_raw_dma_sync_for_device(pool, dma, desc[i].len); - while (likely(budget)) { - if (!netif_carrier_ok(xdp_ring->netdev)) - break; + tx_bi = &xdp_ring->tx_buffer_info[xdp_ring->next_to_use]; + tx_bi->bytecount = desc[i].len; + tx_bi->xdpf = NULL; + tx_bi->gso_segs = 1; - if (!xsk_tx_peek_desc(pool, &desc)) - break; + tx_desc = IXGBE_TX_DESC(xdp_ring, xdp_ring->next_to_use); + tx_desc->read.buffer_addr = cpu_to_le64(dma); - dma = xsk_buff_raw_get_dma(pool, desc.addr); - xsk_buff_raw_dma_sync_for_device(pool, dma, desc.len); + cmd_type = IXGBE_ADVTXD_DTYP_DATA | + IXGBE_ADVTXD_DCMD_DEXT | + IXGBE_ADVTXD_DCMD_IFCS; + cmd_type |= desc[i].len | IXGBE_TXD_CMD_EOP; + tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type); + tx_desc->read.olinfo_status = + cpu_to_le32(desc[i].len << IXGBE_ADVTXD_PAYLEN_SHIFT); - tx_bi = &xdp_ring->tx_buffer_info[xdp_ring->next_to_use]; - tx_bi->bytecount = desc.len; - tx_bi->xdpf = NULL; - tx_bi->gso_segs = 1; + xdp_ring->next_to_use++; +} - tx_desc = IXGBE_TX_DESC(xdp_ring, xdp_ring->next_to_use); - tx_desc->read.buffer_addr = cpu_to_le64(dma); +static void ixgbe_xmit_pkt_batch(struct ixgbe_ring *xdp_ring, struct xdp_desc *desc) +{ + u32 i; - /* put descriptor type bits */ - cmd_type = IXGBE_ADVTXD_DTYP_DATA | - IXGBE_ADVTXD_DCMD_DEXT | - IXGBE_ADVTXD_DCMD_IFCS; - cmd_type |= desc.len | IXGBE_TXD_CMD; - tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type); - tx_desc->read.olinfo_status = - cpu_to_le32(desc.len << IXGBE_ADVTXD_PAYLEN_SHIFT); + unrolled_count(PKTS_PER_BATCH) + for (i = 0; i < PKTS_PER_BATCH; i++) + ixgbe_xmit_pkt(xdp_ring, desc, i); +} - xdp_ring->next_to_use++; - if (xdp_ring->next_to_use == xdp_ring->count) - xdp_ring->next_to_use = 0; +static void ixgbe_fill_tx_hw_ring(struct ixgbe_ring *xdp_ring, + struct xdp_desc *descs, u32 nb_pkts) +{ + u32 batched, leftover, i; + + batched = nb_pkts & ~(PKTS_PER_BATCH - 1); + leftover = nb_pkts & (PKTS_PER_BATCH - 1); + for (i = 0; i < batched; i += PKTS_PER_BATCH) + ixgbe_xmit_pkt_batch(xdp_ring, &descs[i]); + for (i = batched; i < batched + leftover; i++) + ixgbe_xmit_pkt(xdp_ring, &descs[i], 0); +} - budget--; - } +static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) +{ + struct xdp_desc *descs = xdp_ring->xsk_pool->tx_descs; + u32 nb_pkts, nb_processed = 0; - if (tx_desc) { - ixgbe_xdp_ring_update_tail(xdp_ring); - xsk_tx_release(pool); + if (!netif_carrier_ok(xdp_ring->netdev)) + return true; + + nb_pkts = xsk_tx_peek_release_desc_batch(xdp_ring->xsk_pool, budget); + if (!nb_pkts) + return true; + + if (xdp_ring->next_to_use + nb_pkts >= xdp_ring->count) { + nb_processed = xdp_ring->count - xdp_ring->next_to_use; + ixgbe_fill_tx_hw_ring(xdp_ring, descs, nb_processed); + xdp_ring->next_to_use = 0; } - return !!budget; + ixgbe_fill_tx_hw_ring(xdp_ring, &descs[nb_processed], nb_pkts - nb_processed); + + ixgbe_set_rs_bit(xdp_ring); + ixgbe_xdp_ring_update_tail(xdp_ring); + + return nb_pkts < budget; } static void ixgbe_clean_xdp_tx_buffer(struct ixgbe_ring *tx_ring, -- 2.41.3