* [PATCH net-next 1/3] gve: Decouple header split from RX buffer length
[not found] <20251022182301.1005777-1-joshwash@google.com>
@ 2025-10-22 18:22 ` Joshua Washington
2025-10-22 18:22 ` [PATCH net-next 2/3] gve: Allow ethtool to configure rx_buf_len Joshua Washington
2025-10-22 18:22 ` [PATCH net-next 3/3] gve: Default to max_rx_buffer_size for DQO if device supported Joshua Washington
2 siblings, 0 replies; 8+ messages in thread
From: Joshua Washington @ 2025-10-22 18:22 UTC (permalink / raw)
To: netdev
Cc: Ankit Garg, Harshitha Ramamurthy, Jordan Rhee, Willem de Bruijn,
Joshua Washington, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Praveen Kaligineedi, Ziwei Xiao,
open list
From: Ankit Garg <nktgrg@google.com>
Previously, enabling header split via `gve_set_hsplit_config` also
implicitly changed the RX buffer length to 4K (if supported by the
device). This coupled two settings that should be orthogonal; this patch
removes that side effect.
After this change, `gve_set_hsplit_config` only toggles the header
split configuration. The RX buffer length is no longer affected and
must be configured independently.
Signed-off-by: Ankit Garg <nktgrg@google.com>
Reviewed-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Joshua Washington <joshwash@google.com>
---
drivers/net/ethernet/google/gve/gve.h | 3 ---
drivers/net/ethernet/google/gve/gve_ethtool.c | 2 --
drivers/net/ethernet/google/gve/gve_main.c | 10 ----------
3 files changed, 15 deletions(-)
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index cf95ec25b11a..c237d00c5ab3 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -59,8 +59,6 @@
#define GVE_DEFAULT_RX_BUFFER_SIZE 2048
-#define GVE_MAX_RX_BUFFER_SIZE 4096
-
#define GVE_XDP_RX_BUFFER_SIZE_DQO 4096
#define GVE_DEFAULT_RX_BUFFER_OFFSET 2048
@@ -1249,7 +1247,6 @@ void gve_rx_free_rings_gqi(struct gve_priv *priv,
struct gve_rx_alloc_rings_cfg *cfg);
void gve_rx_start_ring_gqi(struct gve_priv *priv, int idx);
void gve_rx_stop_ring_gqi(struct gve_priv *priv, int idx);
-u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hplit);
bool gve_header_split_supported(const struct gve_priv *priv);
int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
struct gve_rx_alloc_rings_cfg *rx_alloc_cfg);
diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c
index b030a84b678c..db6fc855a511 100644
--- a/drivers/net/ethernet/google/gve/gve_ethtool.c
+++ b/drivers/net/ethernet/google/gve/gve_ethtool.c
@@ -606,8 +606,6 @@ static int gve_set_ringparam(struct net_device *netdev,
} else {
/* Set ring params for the next up */
priv->header_split_enabled = rx_alloc_cfg.enable_header_split;
- priv->rx_cfg.packet_buffer_size =
- rx_alloc_cfg.packet_buffer_size;
priv->tx_desc_cnt = tx_alloc_cfg.ring_size;
priv->rx_desc_cnt = rx_alloc_cfg.ring_size;
}
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 29845e8f3c0d..8d825218965a 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -2041,14 +2041,6 @@ static void gve_tx_timeout(struct net_device *dev, unsigned int txqueue)
priv->tx_timeo_cnt++;
}
-u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hsplit)
-{
- if (enable_hsplit && priv->max_rx_buffer_size >= GVE_MAX_RX_BUFFER_SIZE)
- return GVE_MAX_RX_BUFFER_SIZE;
- else
- return GVE_DEFAULT_RX_BUFFER_SIZE;
-}
-
/* Header split is only supported on DQ RDA queue format. If XDP is enabled,
* header split is not allowed.
*/
@@ -2080,8 +2072,6 @@ int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
return 0;
rx_alloc_cfg->enable_header_split = enable_hdr_split;
- rx_alloc_cfg->packet_buffer_size =
- gve_get_pkt_buf_size(priv, enable_hdr_split);
return 0;
}
--
2.51.1.814.gb8fa24458f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH net-next 2/3] gve: Allow ethtool to configure rx_buf_len
[not found] <20251022182301.1005777-1-joshwash@google.com>
2025-10-22 18:22 ` [PATCH net-next 1/3] gve: Decouple header split from RX buffer length Joshua Washington
@ 2025-10-22 18:22 ` Joshua Washington
2025-10-24 0:14 ` Jakub Kicinski
2025-10-22 18:22 ` [PATCH net-next 3/3] gve: Default to max_rx_buffer_size for DQO if device supported Joshua Washington
2 siblings, 1 reply; 8+ messages in thread
From: Joshua Washington @ 2025-10-22 18:22 UTC (permalink / raw)
To: netdev
Cc: Ankit Garg, Harshitha Ramamurthy, Jordan Rhee, Willem de Bruijn,
Joshua Washington, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Alexei Starovoitov, Daniel Borkmann,
Jesper Dangaard Brouer, John Fastabend, Stanislav Fomichev,
Praveen Kaligineedi, Ziwei Xiao, open list,
open list:XDP (eXpress Data Path):Keyword:(?:b|_)xdp(?:b|_)
From: Ankit Garg <nktgrg@google.com>
Add support for getting and setting the RX buffer length via the
ethtool ring parameters (`ethtool -g`/`-G`). The driver restricts the
allowed buffer length to 2048 (SZ_2K) or 4096 (SZ_4K).
As XDP is only supported when the `rx_buf_len` is 2048, the driver now
enforces this in two places:
1. In `gve_xdp_set`, rejecting XDP programs if the current buffer
length is not 2048.
2. In `gve_set_rx_buf_len_config`, rejecting buffer length changes if
XDP is loaded and the new length is not 2048.
Signed-off-by: Ankit Garg <nktgrg@google.com>
Reviewed-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Joshua Washington <joshwash@google.com>
---
drivers/net/ethernet/google/gve/gve.h | 9 ++++
drivers/net/ethernet/google/gve/gve_ethtool.c | 13 +++++-
drivers/net/ethernet/google/gve/gve_main.c | 45 +++++++++++++++++++
3 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index c237d00c5ab3..a33b44c1eb86 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -1167,6 +1167,12 @@ static inline bool gve_is_gqi(struct gve_priv *priv)
priv->queue_format == GVE_GQI_QPL_FORMAT;
}
+static inline bool gve_is_dqo(struct gve_priv *priv)
+{
+ return priv->queue_format == GVE_DQO_RDA_FORMAT ||
+ priv->queue_format == GVE_DQO_QPL_FORMAT;
+}
+
static inline u32 gve_num_tx_queues(struct gve_priv *priv)
{
return priv->tx_cfg.num_queues + priv->tx_cfg.num_xdp_queues;
@@ -1248,6 +1254,9 @@ void gve_rx_free_rings_gqi(struct gve_priv *priv,
void gve_rx_start_ring_gqi(struct gve_priv *priv, int idx);
void gve_rx_stop_ring_gqi(struct gve_priv *priv, int idx);
bool gve_header_split_supported(const struct gve_priv *priv);
+int gve_set_rx_buf_len_config(struct gve_priv *priv, u32 rx_buf_len,
+ struct netlink_ext_ack *extack,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg);
int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
struct gve_rx_alloc_rings_cfg *rx_alloc_cfg);
/* rx buffer handling */
diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c
index db6fc855a511..52500ae8348e 100644
--- a/drivers/net/ethernet/google/gve/gve_ethtool.c
+++ b/drivers/net/ethernet/google/gve/gve_ethtool.c
@@ -529,6 +529,8 @@ static void gve_get_ringparam(struct net_device *netdev,
cmd->rx_pending = priv->rx_desc_cnt;
cmd->tx_pending = priv->tx_desc_cnt;
+ kernel_cmd->rx_buf_len = priv->rx_cfg.packet_buffer_size;
+
if (!gve_header_split_supported(priv))
kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_UNKNOWN;
else if (priv->header_split_enabled)
@@ -589,6 +591,12 @@ static int gve_set_ringparam(struct net_device *netdev,
int err;
gve_get_curr_alloc_cfgs(priv, &tx_alloc_cfg, &rx_alloc_cfg);
+
+ err = gve_set_rx_buf_len_config(priv, kernel_cmd->rx_buf_len, extack,
+ &rx_alloc_cfg);
+ if (err)
+ return err;
+
err = gve_set_hsplit_config(priv, kernel_cmd->tcp_data_split,
&rx_alloc_cfg);
if (err)
@@ -605,6 +613,8 @@ static int gve_set_ringparam(struct net_device *netdev,
return err;
} else {
/* Set ring params for the next up */
+ priv->rx_cfg.packet_buffer_size =
+ rx_alloc_cfg.packet_buffer_size;
priv->header_split_enabled = rx_alloc_cfg.enable_header_split;
priv->tx_desc_cnt = tx_alloc_cfg.ring_size;
priv->rx_desc_cnt = rx_alloc_cfg.ring_size;
@@ -944,7 +954,8 @@ static int gve_get_ts_info(struct net_device *netdev,
const struct ethtool_ops gve_ethtool_ops = {
.supported_coalesce_params = ETHTOOL_COALESCE_USECS,
- .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT,
+ .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT |
+ ETHTOOL_RING_USE_RX_BUF_LEN,
.get_drvinfo = gve_get_drvinfo,
.get_strings = gve_get_strings,
.get_sset_count = gve_get_sset_count,
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 8d825218965a..8009819c73f2 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -1722,6 +1722,13 @@ static int verify_xdp_configuration(struct net_device *dev)
return -EOPNOTSUPP;
}
+ if (priv->rx_cfg.packet_buffer_size != SZ_2K) {
+ netdev_warn(dev,
+ "XDP is not supported for Rx buf len %d. Set Rx buf len to %d before using XDP.\n",
+ priv->rx_cfg.packet_buffer_size, SZ_2K);
+ return -EOPNOTSUPP;
+ }
+
max_xdp_mtu = priv->rx_cfg.packet_buffer_size - sizeof(struct ethhdr);
if (priv->queue_format == GVE_GQI_QPL_FORMAT)
max_xdp_mtu -= GVE_RX_PAD;
@@ -2050,6 +2057,44 @@ bool gve_header_split_supported(const struct gve_priv *priv)
priv->queue_format == GVE_DQO_RDA_FORMAT && !priv->xdp_prog;
}
+int gve_set_rx_buf_len_config(struct gve_priv *priv, u32 rx_buf_len,
+ struct netlink_ext_ack *extack,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg)
+{
+ u32 old_rx_buf_len = rx_alloc_cfg->packet_buffer_size;
+
+ if (rx_buf_len == old_rx_buf_len)
+ return 0;
+
+ if (!gve_is_dqo(priv)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Modifying Rx buf len is only supported with DQO format");
+ return -EOPNOTSUPP;
+ }
+
+ if (priv->xdp_prog && rx_buf_len != SZ_2K) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Rx buf len can only be 2048 when XDP is on");
+ return -EINVAL;
+ }
+
+ if (rx_buf_len > priv->max_rx_buffer_size) {
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "Rx buf len exceeds the max supported value of %u",
+ priv->max_rx_buffer_size);
+ return -EINVAL;
+ }
+
+ if (rx_buf_len != SZ_2K && rx_buf_len != SZ_4K) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Rx buf len can only be 2048 or 4096");
+ return -EINVAL;
+ }
+ rx_alloc_cfg->packet_buffer_size = rx_buf_len;
+
+ return 0;
+}
+
int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
struct gve_rx_alloc_rings_cfg *rx_alloc_cfg)
{
--
2.51.1.814.gb8fa24458f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH net-next 3/3] gve: Default to max_rx_buffer_size for DQO if device supported
[not found] <20251022182301.1005777-1-joshwash@google.com>
2025-10-22 18:22 ` [PATCH net-next 1/3] gve: Decouple header split from RX buffer length Joshua Washington
2025-10-22 18:22 ` [PATCH net-next 2/3] gve: Allow ethtool to configure rx_buf_len Joshua Washington
@ 2025-10-22 18:22 ` Joshua Washington
2 siblings, 0 replies; 8+ messages in thread
From: Joshua Washington @ 2025-10-22 18:22 UTC (permalink / raw)
To: netdev
Cc: Ankit Garg, Harshitha Ramamurthy, Jordan Rhee, Willem de Bruijn,
Joshua Washington, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Ziwei Xiao, John Fraker,
Dr. David Alan Gilbert, open list
From: Ankit Garg <nktgrg@google.com>
Change the driver's default behavior to prefer the largest available RX
buffer length supported by the device for DQO format, rather than always
using the hardcoded 2K default.
Previously, the driver would initialize with
`GVE_DEFAULT_RX_BUFFER_SIZE` (2K), even if the device advertised support
for a larger length (e.g., 4K).
Performance observations:
- With LRO disabled, we observed >10% improvement in RX single stream
throughput when MTU >=2048.
- With LRO enabled, we observed >10% improvement in RX single stream
throughput when MTU >=1460.
- No regressions were observed.
Signed-off-by: Ankit Garg <nktgrg@google.com>
Reviewed-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Joshua Washington <joshwash@google.com>
---
drivers/net/ethernet/google/gve/gve_adminq.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
index 4f33d094a2ef..b72cc0fa2ba2 100644
--- a/drivers/net/ethernet/google/gve/gve_adminq.c
+++ b/drivers/net/ethernet/google/gve/gve_adminq.c
@@ -987,6 +987,10 @@ static void gve_enable_supported_features(struct gve_priv *priv,
dev_info(&priv->pdev->dev,
"BUFFER SIZES device option enabled with max_rx_buffer_size of %u, header_buf_size of %u.\n",
priv->max_rx_buffer_size, priv->header_buf_size);
+ if (gve_is_dqo(priv) &&
+ priv->max_rx_buffer_size > GVE_DEFAULT_RX_BUFFER_SIZE)
+ priv->rx_cfg.packet_buffer_size =
+ priv->max_rx_buffer_size;
}
/* Read and store ring size ranges given by device */
--
2.51.1.814.gb8fa24458f-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread