From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AFA14A2633 for ; Sat, 3 Oct 2026 21:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.72 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791062597; cv=none; b=Kkx70rWm6QTOTtrD6ouU6D3ARwcTdubzr6naQVzolSgFkZ/UzBTuipMoomjIFuoS2y+w+phe4yczGwO7u+/KUeEwJk8R4sx3ke2Ta5LdL0BKIyrew/qPHvecRPhrU6qf9JmdNPjblIIZKhM8kBZsAfXwNrZzczzqgxFSSyw0C7M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791062597; c=relaxed/simple; bh=3FnAcNdQeBpwauuPUEv8Hb+n/TAjxKWq99qHjkzEXiE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fSRSw8LxylloW+xU8iWsV0QKxudSiocq18C8Ukm44yhdLYoPiu0cpsTLFqBqGjij5stMH98xySnWPThVh+6EODN04glkCGVQZwKjXKFWSHpSue7CFQTHly8ctaOLnJkoGp3woZ7TJ1UYfXtxi/hdshrU8OA/4H+21Ym2XnPhdHM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--lrizzo.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oyvG9vMg; arc=none smtp.client-ip=209.85.218.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--lrizzo.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oyvG9vMg" Received: by mail-ej1-f72.google.com with SMTP id a640c23a62f3a-c29c391bd2fso68306666b.2 for ; Sat, 03 Oct 2026 14:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1791062591; x=1791667391; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ekKJz+HTYRvtVEqBBOEAxTGImVFcYe3lcvJJhW15OKU=; b=oyvG9vMgIw0/OIeAcEiK85a09ZcXIq9rr9EXUvObbXdv1dJIQ+Rds7GKV/LONULmYW 1KsqwpMR0qU8R0xj+SIlkjjmQcYH6W2sbeVYqhb6wtTjpgkZ4CBiPAyTG6YoBL2uOCVJ XvRsDj4e0gLnHs6tVtx9AU0eMxho4gc7aFhdT9FLCdLT06IfTPTXrwTF8DfcWCYIYypw 0G31F/uU46Qli14iAxjZ5RpJPMAnQLrdm0g5PCqztb6Mkaxxs7N8vC9qQnW9qLk6aXzf GXjgxYF3C7NyPOJOrOsyLzsTuaNICWKmQwLXyHir/DPAVgU7rnfpgyKaiRpZ2krK5GY3 fOPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791062591; x=1791667391; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ekKJz+HTYRvtVEqBBOEAxTGImVFcYe3lcvJJhW15OKU=; b=N8T06ZNL+nTWyOrJzm4n3EJaUWamT9+MyOzrbHabyLQW88Uvi+7wY+iNrtm8WooUAl EgDCPDWu/tN0xYHHS1r/9PBQumzuvmAhQwKrzQo75qoJcQOZFhN6ANnDH7Fs2CdkwTSK hGg1JGDl3Av1i0XxCJZOOa2HNpDfZYs16kNPzmu1MZHIeGi4UY2yLpEgvzPZ2xp5ZAvY PcVLS944IbVc4Y3ai+D7eA3MFLc44HEAjSYVCv2JtGr+upgv/bISBwqlMWmQD2ft1DWe cA7v4Tmu695ssNuC2Yt18N2FR4bKvc8yrjL9OJqK+0uZV5RpokllgxRplSnh50EbhuaR pMyw== X-Forwarded-Encrypted: i=1; AKwUvBw0Rq9Kw0SRKleSnl3+DJ5FMRyxzawFAh7sSBenBV9rNk96hsUNkiRnTIA5ma4M5+a5umx9nx4=@vger.kernel.org X-Gm-Message-State: AFq9FYK/vPbmFGrEKXTt/Xpm2Co5TKRlMhujphSM+G/i+fU5yVMeFruu d1wbwTzIdC1NgJZoQr31A0FUApj60srQrsvhFlPhBXWCfTonG8EeQu4BUnNUNPmz7DAoh60YmNU ja5QBAQ== X-Received: from ejbhk6.prod.google.com ([2002:a17:906:c9c6:b0:c2d:efb7:91a3]) (user=lrizzo job=prod-delivery.src-stubby-dispatcher) by 2002:a17:906:4fc9:b0:c2e:3307:b922 with SMTP id a640c23a62f3a-c2e4ade0f4emr633013566b.28.1791062590162; Sat, 03 Oct 2026 14:23:10 -0700 (PDT) Date: Sat, 3 Oct 2026 21:22:37 +0000 In-Reply-To: <20261003212241.3432303-1-lrizzo@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20261003212241.3432303-1-lrizzo@google.com> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog Message-ID: <20261003212241.3432303-19-lrizzo@google.com> Subject: [RFC: DMA_PMD 18/22] net/gve: Use DMA_PMD memory for tx header bounce buffers From: Luigi Rizzo To: Luigi Rizzo , Joerg Roedel , Will Deacon , Robin Murphy , Christoph Hellwig , Marek Szyprowski , Andrew Morton , Vlastimil Babka , David Hildenbrand , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Greg Kroah-Hartman , "Rafael J . Wysocki" , Danilo Krummrich , Jonathan Corbet , Jesper Dangaard Brouer , Ilias Apalodimas , Willem de Bruijn , Kuniyuki Iwashima , Joshua Washington , Harshitha Ramamurthy , Saeed Mahameed , Tariq Toukan , Tony Nguyen , Przemek Kitszel , Alexander Lobakin , Michael Chan , Pavan Chebbi , iommu@lists.linux.dev, netdev@vger.kernel.org, linux-mm@kvack.org, driver-core@lists.linux.dev, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Luigi Rizzo Content-Type: text/plain; charset="UTF-8" When dev->dma_pmd_tx_hdrs is enabled, allocate a MAX_TCP_HEADER bounce buffer per TX slot/packet out of the DMA_PMD arena in raw-addressing mode (both GQI and DQO). When skb_headlen(skb) <= MAX_TCP_HEADER, the linear portion is copied into the pre-mapped bounce buffer instead of calling dma_map_single()/dma_unmap_single() on every packet. Signed-off-by: Luigi Rizzo --- drivers/net/ethernet/google/gve/gve.h | 3 ++ drivers/net/ethernet/google/gve/gve_tx.c | 38 ++++++++++++++--- drivers/net/ethernet/google/gve/gve_tx_dqo.c | 45 +++++++++++++++----- 3 files changed, 69 insertions(+), 17 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 11330404f681b..00e438fe05a50 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "gve_desc.h" @@ -642,6 +643,8 @@ struct gve_tx_ring { }; } dqo; } ____cacheline_aligned; + void *tx_hdr_bufs; + dma_addr_t tx_hdr_bufs_dma; struct netdev_queue *netdev_txq; struct gve_queue_resources *q_resources; /* head and tail pointer idx */ struct device *dev; diff --git a/drivers/net/ethernet/google/gve/gve_tx.c b/drivers/net/ethernet/google/gve/gve_tx.c index 79338fd9bf66e..9610b54ef4792 100644 --- a/drivers/net/ethernet/google/gve/gve_tx.c +++ b/drivers/net/ethernet/google/gve/gve_tx.c @@ -224,6 +224,11 @@ static void gve_tx_free_ring_gqi(struct gve_priv *priv, struct gve_tx_ring *tx, u32 slots; slots = tx->mask + 1; + if (tx->tx_hdr_bufs) { + dma_free_coherent(hdev, slots * MAX_TCP_HEADER, + tx->tx_hdr_bufs, tx->tx_hdr_bufs_dma); + tx->tx_hdr_bufs = NULL; + } dma_free_coherent(hdev, sizeof(*tx->q_resources), tx->q_resources, tx->q_resources_bus); tx->q_resources = NULL; @@ -298,6 +303,11 @@ static int gve_tx_alloc_ring_gqi(struct gve_priv *priv, /* map Tx FIFO */ if (gve_tx_fifo_init(priv, &tx->tx_fifo)) goto abort_with_qpl; + } else if (hdev->dma_pmd_tx_hdrs) { + tx->tx_hdr_bufs = dma_alloc_coherent(hdev, + cfg->ring_size * MAX_TCP_HEADER, + &tx->tx_hdr_bufs_dma, + GFP_KERNEL); } tx->q_resources = @@ -311,6 +321,12 @@ static int gve_tx_alloc_ring_gqi(struct gve_priv *priv, return 0; abort_with_fifo: + if (tx->tx_hdr_bufs) { + dma_free_coherent(hdev, + cfg->ring_size * MAX_TCP_HEADER, + tx->tx_hdr_bufs, tx->tx_hdr_bufs_dma); + tx->tx_hdr_bufs = NULL; + } if (!tx->raw_addressing) gve_tx_fifo_release(priv, &tx->tx_fifo); abort_with_qpl: @@ -423,6 +439,8 @@ static inline int gve_skb_fifo_bytes_required(struct gve_tx_ring *tx, #define MAX_TX_DESC_NEEDED (MAX_SKB_FRAGS + 4) static void gve_tx_unmap_buf(struct device *dev, struct gve_tx_buffer_state *info) { + if (!dma_unmap_len(info, len)) + return; if (info->skb) { dma_unmap_single(dev, dma_unmap_addr(info, dma), dma_unmap_len(info, len), @@ -657,13 +675,21 @@ static int gve_tx_add_skb_no_copy(struct gve_priv *priv, struct gve_tx_ring *tx, info->skb = skb; - addr = dma_map_single(tx->dev, skb->data, len, DMA_TO_DEVICE); - if (unlikely(dma_mapping_error(tx->dev, addr))) { - tx->dma_mapping_error++; - goto drop; + if (tx->tx_hdr_bufs && len <= MAX_TCP_HEADER) { + u32 hdr_off = idx * MAX_TCP_HEADER; + + memcpy(tx->tx_hdr_bufs + hdr_off, skb->data, len); + addr = tx->tx_hdr_bufs_dma + hdr_off; + dma_unmap_len_set(info, len, 0); + } else { + addr = dma_map_single(tx->dev, skb->data, len, DMA_TO_DEVICE); + if (unlikely(dma_mapping_error(tx->dev, addr))) { + tx->dma_mapping_error++; + goto drop; + } + dma_unmap_len_set(info, len, len); + dma_unmap_addr_set(info, dma, addr); } - dma_unmap_len_set(info, len, len); - dma_unmap_addr_set(info, dma, addr); num_descriptors = 1 + shinfo->nr_frags; if (hlen < len) diff --git a/drivers/net/ethernet/google/gve/gve_tx_dqo.c b/drivers/net/ethernet/google/gve/gve_tx_dqo.c index ad99cbb77e291..ac69793492c9f 100644 --- a/drivers/net/ethernet/google/gve/gve_tx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_tx_dqo.c @@ -176,8 +176,9 @@ static void gve_unmap_packet(struct device *dev, return; /* SKB linear portion is guaranteed to be mapped */ - dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), - dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); + if (dma_unmap_len(pkt, len[0])) + dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), + dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); for (i = 1; i < pkt->num_bufs; i++) { netmem_dma_unmap_page_attrs(dev, dma_unmap_addr(pkt, dma[i]), dma_unmap_len(pkt, len[i]), @@ -232,6 +233,13 @@ static void gve_tx_free_ring_dqo(struct gve_priv *priv, struct gve_tx_ring *tx, size_t bytes; u32 qpl_id; + if (tx->tx_hdr_bufs) { + dma_free_coherent(hdev, + tx->dqo.num_pending_packets * MAX_TCP_HEADER, + tx->tx_hdr_bufs, tx->tx_hdr_bufs_dma); + tx->tx_hdr_bufs = NULL; + } + if (tx->q_resources) { dma_free_coherent(hdev, sizeof(*tx->q_resources), tx->q_resources, tx->q_resources_bus); @@ -399,6 +407,12 @@ static int gve_tx_alloc_ring_dqo(struct gve_priv *priv, if (gve_tx_qpl_buf_init(tx)) goto err; + } else if (hdev->dma_pmd_tx_hdrs) { + tx->tx_hdr_bufs = + dma_alloc_coherent(hdev, + tx->dqo.num_pending_packets * MAX_TCP_HEADER, + &tx->tx_hdr_bufs_dma, + GFP_KERNEL); } return 0; @@ -712,12 +726,20 @@ static int gve_tx_add_skb_no_copy_dqo(struct gve_tx_ring *tx, u32 len = skb_headlen(skb); dma_addr_t addr; - addr = dma_map_single(tx->dev, skb->data, len, DMA_TO_DEVICE); - if (unlikely(dma_mapping_error(tx->dev, addr))) - goto err; + if (tx->tx_hdr_bufs && len <= MAX_TCP_HEADER) { + u32 hdr_off = (u32)completion_tag * MAX_TCP_HEADER; - dma_unmap_len_set(pkt, len[pkt->num_bufs], len); - dma_unmap_addr_set(pkt, dma[pkt->num_bufs], addr); + memcpy(tx->tx_hdr_bufs + hdr_off, skb->data, len); + addr = tx->tx_hdr_bufs_dma + hdr_off; + dma_unmap_len_set(pkt, len[pkt->num_bufs], 0); + } else { + addr = dma_map_single(tx->dev, skb->data, len, + DMA_TO_DEVICE); + if (unlikely(dma_mapping_error(tx->dev, addr))) + goto err; + dma_unmap_len_set(pkt, len[pkt->num_bufs], len); + dma_unmap_addr_set(pkt, dma[pkt->num_bufs], addr); + } ++pkt->num_bufs; gve_tx_fill_pkt_desc_dqo(tx, desc_idx, enable_csum, len, addr, @@ -748,10 +770,11 @@ static int gve_tx_add_skb_no_copy_dqo(struct gve_tx_ring *tx, err: for (i = 0; i < pkt->num_bufs; i++) { if (i == 0) { - dma_unmap_single(tx->dev, - dma_unmap_addr(pkt, dma[i]), - dma_unmap_len(pkt, len[i]), - DMA_TO_DEVICE); + if (dma_unmap_len(pkt, len[i])) + dma_unmap_single(tx->dev, + dma_unmap_addr(pkt, dma[i]), + dma_unmap_len(pkt, len[i]), + DMA_TO_DEVICE); } else { dma_unmap_page(tx->dev, dma_unmap_addr(pkt, dma[i]), -- 2.56.0.rc1.315.gc6ed9934b7-goog