From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4099C37FF42 for ; Sat, 11 Jul 2026 09:12:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.48 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783761140; cv=none; b=Wyb5gicW8n1W5nASgTfwzjH5mK/1GqLe4T4Alj5xtOp4HUFo63SJPPUuZpt1v/Jbb7iPA3HUx/3s8lXmVWOVWPmbFMicESgrUImLedduEkXg1D/ReSd06iEGJwYEVXyTjd5JeCd9QcnygszRBf1faMVVEmvnDwQCI8nFSOJMaJ4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783761140; c=relaxed/simple; bh=E6cOOOK8JRaX4aDVdN1M0+1MtzS0eNCG8VJsAdIuLwg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=e1Lsx4sY3JQCwKiRgR0bK5F6/4nEITvon8pnfwfOEL8uxPpAbYncIQ+aEZqHFCBXSGKgx26m34fhYMGZy4mivYe9pv5xdTl0kcTT0mDxHvCm79qy5VEWJ4Fi4AdqiGgEsNWdBsTEMlrFkp3XxWAa/EDW5FK4YJwQweuUWvPJc/k= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=tJ6tL3bn; arc=none smtp.client-ip=209.85.208.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tJ6tL3bn" Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-6986287534eso3080075a12.3 for ; Sat, 11 Jul 2026 02:12:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783761138; x=1784365938; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dvOpz0r7ZGyorI3Hc7NN1bB6/9+WhAP8KvikiSMDSlo=; b=tJ6tL3bnChp+YcRF1+X3EyhkD4QJ9pVyVxIEJHm0a2KxMlMuNtG27CrnWcV7QPdt7H E5ciJMTLWH+c83IR9EL0brFQwm9P7FkcLyEhlqSVTaEdXDcXwclbd8W9VGm08XQUryVs nOB9rN0WTx8Ci4w+rVzcpTBKQxChyCVp605a+Ggc9+PJOg4WZTkZRBXJ/nMKIEWk3NQq 6jY2fuBpddH8aNHky9kJ648zPsHpmWhQZYYGE7cgBIAVIsgfO9dnf+r/MjKyjbv3NGej KTSHgvJ+hlY3gFEAjOGpN11zkvN7FijRNkl5rbL4xQ8K4zTSH/hejrOeoiIrr0RHPKU7 IWQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783761138; x=1784365938; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=dvOpz0r7ZGyorI3Hc7NN1bB6/9+WhAP8KvikiSMDSlo=; b=eyQAGsTA4/qLOmY0uLsRfv8/LWB/OFOkjZgSo+m9Cd0C7302JZDmq8isd6xH6DbR7f Os4iS++7W0zCvlZXwXBwF5SeZnLqPoRLT/ivbtImJ/zi5omPEGKewgOscsT1V/UGHtGC R9bA05Y3rQpfu4B7LvWWoOl0OQFpkJhpUHPjvbxIXbh1Wz7uDjiOCz74JJ+lgjP5OddS WNmikq4DZqPoVU8LhmcJ7FVPMu1iD+d9/fT7wwjIWjRliw6zuDTs0fCiTJtPdiD7H9sq 4VrzjVbrX1p0lLz++kectUBjsYwWahCMdtNNFtfYrCB1ub/n6TRqIW1O60IE7MDK5D+y APtQ== X-Forwarded-Encrypted: i=1; AHgh+RrtXr9QfW9cwPCAMOVcfHhW3o+llsdgY4XIhcJLOM++zcklz1wmDm4Gm4j2Njo9e+dzmA+gTGE=@vger.kernel.org X-Gm-Message-State: AOJu0YyNHDkYbresblrSiRJNcPtmmZ9+LC0cpziRXR4anybCWkonzcvx GrR0suFho59t7XsU40am4+BQqhE04LVNWWFigJVzQPsuOXIjYHikfgI1hJ7QQw== X-Gm-Gg: AfdE7cnz7u2zOiUFkt0eueHWhPjwrv/xkgAvu36wjeLZQAiJqMLE50t/RF9mwNKnY+0 9HhUewSHW43ZIQo5lIayWR6WmkHeO6Swx/b+CVjkvxyauogvSGSb1Bdi6j9kFmP33GD4FFHoFoh NwSs742V7dyZ4mAVB156hspEtjjnUaMgibV3WMDyN/4PYc3NNl05NxalxRak8pCWFRZPSQJf7Mm VPTX65OoBQZ8WQW3h5gik3R9ZOx46rCKEW0YQ5kHnYtolHTSijvRGO4y4R1v2vNRh1uTcHpih5J z7S2jJ5AulNLjGALMEh5Keom58iL2Cyw2p+kaxCRTnAaIn/9L5CO/vz5tRbW0JjgVI2f5i7V6NU 6ro5zpLZZVD4IH/4ay4IHuNvp0jt8Wy3xgY4GL+KcOq/cqtUr/t/VtR0AcX/kJ1ji2rbpigGpzW 30Lyo3/zRmoA3rIbOdQWkzvMzWIpYcaxyNbpk686SpYPM+0e8pqdMQBCiOsrzEjyEudB64BmzxQ C4GRbUybUmn1fn5Qb1pBLCUA+OS05eYWqP85BygPvo3Qe4= X-Received: by 2002:a05:6402:2695:b0:697:d475:9692 with SMTP id 4fb4d7f45d1cf-69c5efaf7a0mr1144254a12.11.1783761137561; Sat, 11 Jul 2026 02:12:17 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-221-54.dab.02.net. [82.132.221.54]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-69c60d47188sm681191a12.27.2026.07.11.02.12.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 11 Jul 2026 02:12:15 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, netdev@vger.kernel.org Subject: [PATCH review-only 03/17] io_uring/zcrx: add RQ iterator Date: Sat, 11 Jul 2026 10:11:26 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add a iterator structure and helper functions for the refill queue processing to avoid polluting io_zcrx_ring_refill() with extra state and logic once it's extended in following patches. Signed-off-by: Pavel Begunkov --- io_uring/zcrx.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c index c4a9a663eba4..45b178afbbc3 100644 --- a/io_uring/zcrx.c +++ b/io_uring/zcrx.c @@ -1088,6 +1088,10 @@ void io_unregister_zcrx(struct io_ring_ctx *ctx) xa_destroy(&ctx->zcrx_ctxs); } +struct zcrx_rq_iter { + int rqes_left; +}; + static inline u32 zcrx_rq_entries(struct zcrx_rq *rq) { u32 entries; @@ -1103,6 +1107,24 @@ static struct io_uring_zcrx_rqe *zcrx_next_rqe(struct zcrx_rq *rq, unsigned mask return &rq->rqes[idx]; } +static inline void zcrx_rq_iter_init(struct zcrx_rq_iter *it, + struct zcrx_rq *rq) +{ + it->rqes_left = min_t(unsigned, zcrx_rq_entries(rq), ZCRX_REFILL_CAP); +} + +static inline bool zcrx_rq_iter_next(struct zcrx_rq_iter *it, + struct zcrx_rq *rq, + struct io_uring_zcrx_rqe **rqe) +{ + it->rqes_left--; + if (unlikely(it->rqes_left < 0)) + return false; + + *rqe = zcrx_next_rqe(rq, rq->nr_entries - 1); + return true; +} + static inline bool io_parse_rqe(struct io_uring_zcrx_rqe *rqe, struct io_zcrx_ifq *ifq, struct net_iov **ret_niov) @@ -1131,17 +1153,15 @@ static unsigned io_zcrx_ring_refill(struct page_pool *pp, netmem_ref *netmems, unsigned to_alloc) { struct zcrx_rq *rq = &ifq->rq; - unsigned int mask = rq->nr_entries - 1; - unsigned int rqes_left; + struct io_uring_zcrx_rqe *rqe; + struct zcrx_rq_iter it; unsigned allocated = 0; guard(spinlock_bh)(&rq->lock); - rqes_left = zcrx_rq_entries(rq); - rqes_left = min_t(unsigned, rqes_left, ZCRX_REFILL_CAP); + zcrx_rq_iter_init(&it, rq); - for (; rqes_left; rqes_left--) { - struct io_uring_zcrx_rqe *rqe = zcrx_next_rqe(rq, mask); + while (zcrx_rq_iter_next(&it, rq, &rqe)) { struct net_iov *niov; netmem_ref netmem; -- 2.54.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB80639989B for ; Sat, 11 Jul 2026 10:40:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783766440; cv=none; b=rVuR3xP/VcrcQjUIuF9FKWl3xFCauQf0MCHtBzpPbuGGvfaqeBBfWFy6cXX2e+X7Nc9osLXnaqPbRTsjMKp7Ylt4YoaVU59DESv6CTIFm5RsVnnsM9s5gTfPNBipljA8MlcaRnbpNy3kzjl5Km0BCinrGuqtbghOZxI1o/j7gBk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783766440; c=relaxed/simple; bh=E6cOOOK8JRaX4aDVdN1M0+1MtzS0eNCG8VJsAdIuLwg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G8/OTKRnaXAsSHkSgdZkBZMSsDesQd3x9gliZ29dboL9X4JMqpYu6FTN0HqJAgAwaM7hDytKY/Aa5aXpjqSRe3LeltJ6Zr+LEN/cVSJpIeykX4WH0kK5bJaQgs0pdj/DirtasidBURZy55JQbVjM64j2aTyR/eQQZRiFLzrVRg4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ghIg4nZR; arc=none smtp.client-ip=209.85.218.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ghIg4nZR" Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-c15c257a488so249840166b.3 for ; Sat, 11 Jul 2026 03:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783766437; x=1784371237; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=dvOpz0r7ZGyorI3Hc7NN1bB6/9+WhAP8KvikiSMDSlo=; b=ghIg4nZRSWfB3jpks7gT83/0l/RxVwSxmKgfxiCZYYLw46eGpI2jHcaU6vNj19s0Or mSjLfJMtFTYh1K45TajB0uwoY7KstWyZd8rtYapWiIdCPMTGUL8ptP24WCS4u/i6Lk7m +lso9rw7fUdp86lOudMl6hUeVAyqHur9NOWsGvjnVJxH2cnWNCstDcWEdJoBXtTIqCDp nDwS9KOJkJx+aQMlYRJ5Uu+nJprf0HVA8Dro0vR6a4frPsFJHjTr6K4vsCzaa9SJHOCN lbcg+eRZN40L4b19HYTPSfezILC3IsXn7uBT+ZfUZF3kGtSs+JvgqDA4HSsJxSlwG5LB kYdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783766437; x=1784371237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=dvOpz0r7ZGyorI3Hc7NN1bB6/9+WhAP8KvikiSMDSlo=; b=SJyjRG3/p81TZn8xYVFtXFdQg5shgAQUEkYb2YsCkNAaTFTIFAbrUNUkj3QPBT5/kl 8hD2tOU3lbsGAyz2PlEpO07NGn89DbnkD63aW4iDdu4OYR+GZanFHcx7u6zL0SGbMTMp hUGOmICyWxZiiaK2ORs0+XWwkzkPku9qdXNoIZuqQcsOS+oa+bJ7z6VODDxxGZgERgT/ ALujia5qpcNPkyJkrWBaX8chIzBasjk4qv1j9fr3GBcaOzpi5jkyt6IU3jyiA731u+bH dmso3P0JzwJDQp7nFfJvbYuWUKzcSlaSIbN7mB+59S/14r0wPPdClFwcMvPBPvLVp2Ge zjRQ== X-Forwarded-Encrypted: i=1; AHgh+RoD9Moxg7B45Q85VQS3kixgTxeBLHdT7ORRJbuG7a4hKGskWu1HwAFBj2CnxqBTS/XehpeMOZ0=@vger.kernel.org X-Gm-Message-State: AOJu0Yyc+mKZlnB22IRAzJlaLk7Q+K/ct1s0chG94fc5W7gukTI+6HjY Mtlk2LM4MeYdFvg+ungE29FggrIh3BeL60NwuRohPLOrb9GzjF1tJ8PaiLe+Sw== X-Gm-Gg: AfdE7clRC5Ay1oFE4o1xux/nttLglHLg4EvlCPvpGwfhejuJQp8U5BJ93KDbDE6H3xd DzPfV+w33i+C9xQegFQoHfYAYA11Mc7QnyfMD1JxHmNii30q+QmlqRkKFLFVycBjm0T1p9+zH2y GIBXrOwfhGI7rDeS3nYe5Arf6zdUwbToAjH4plAiEjTQTFFVcdtGKCyhBzBlhikvSUezQnwmBYj sWK8AiHz7xFSVgFSgxozLcGtWzIiA3vslSPHHHqu4rX3VVow0ZjQJvpL4vOq0byinIo0VvLvrCn +yseJLuZBL717N710oXP7MFTM6sQ6KEWriUhlsM8h4wpBQTKnV+MMW9O+6bfFO2+JP+nXgV/Fdg Rqh4GIhMhWqRzMZMZhF8KRHmyUAQeAgweMNVUAtZRd0aLcoy/KcPBwfl3BhH+5d1u0H0k9n0WCO yW7yKVAVleYRRsZgwZ6tUbjsywCMxeO6W03lKL3FYyof31gjXqtYzn7Q94PO/yAhtyH9Ih+NAwQ 940PnbvPFM3wxwjtRJYnOwuDXLrWGf85Ww1GgOeftTspAELoQ== X-Received: by 2002:a17:907:9604:b0:c12:34ed:e108 with SMTP id a640c23a62f3a-c161f37e257mr76942666b.58.1783766437237; Sat, 11 Jul 2026 03:40:37 -0700 (PDT) Received: from 127.0.0.1localhost (82-132-222-132.dab.02.net. [82.132.222.132]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c15d5de95e6sm483041566b.39.2026.07.11.03.40.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 11 Jul 2026 03:40:36 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com, netdev@vger.kernel.org Subject: [PATCH review-only 03/17] io_uring/zcrx: add RQ iterator Date: Sat, 11 Jul 2026 11:39:56 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID: <20260711103956.2oQcUYe2KaPKxNnTSKkhzujI0K8ZGc496GooBIUTIp0@z> Add a iterator structure and helper functions for the refill queue processing to avoid polluting io_zcrx_ring_refill() with extra state and logic once it's extended in following patches. Signed-off-by: Pavel Begunkov --- io_uring/zcrx.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c index c4a9a663eba4..45b178afbbc3 100644 --- a/io_uring/zcrx.c +++ b/io_uring/zcrx.c @@ -1088,6 +1088,10 @@ void io_unregister_zcrx(struct io_ring_ctx *ctx) xa_destroy(&ctx->zcrx_ctxs); } +struct zcrx_rq_iter { + int rqes_left; +}; + static inline u32 zcrx_rq_entries(struct zcrx_rq *rq) { u32 entries; @@ -1103,6 +1107,24 @@ static struct io_uring_zcrx_rqe *zcrx_next_rqe(struct zcrx_rq *rq, unsigned mask return &rq->rqes[idx]; } +static inline void zcrx_rq_iter_init(struct zcrx_rq_iter *it, + struct zcrx_rq *rq) +{ + it->rqes_left = min_t(unsigned, zcrx_rq_entries(rq), ZCRX_REFILL_CAP); +} + +static inline bool zcrx_rq_iter_next(struct zcrx_rq_iter *it, + struct zcrx_rq *rq, + struct io_uring_zcrx_rqe **rqe) +{ + it->rqes_left--; + if (unlikely(it->rqes_left < 0)) + return false; + + *rqe = zcrx_next_rqe(rq, rq->nr_entries - 1); + return true; +} + static inline bool io_parse_rqe(struct io_uring_zcrx_rqe *rqe, struct io_zcrx_ifq *ifq, struct net_iov **ret_niov) @@ -1131,17 +1153,15 @@ static unsigned io_zcrx_ring_refill(struct page_pool *pp, netmem_ref *netmems, unsigned to_alloc) { struct zcrx_rq *rq = &ifq->rq; - unsigned int mask = rq->nr_entries - 1; - unsigned int rqes_left; + struct io_uring_zcrx_rqe *rqe; + struct zcrx_rq_iter it; unsigned allocated = 0; guard(spinlock_bh)(&rq->lock); - rqes_left = zcrx_rq_entries(rq); - rqes_left = min_t(unsigned, rqes_left, ZCRX_REFILL_CAP); + zcrx_rq_iter_init(&it, rq); - for (; rqes_left; rqes_left--) { - struct io_uring_zcrx_rqe *rqe = zcrx_next_rqe(rq, mask); + while (zcrx_rq_iter_next(&it, rq, &rqe)) { struct net_iov *niov; netmem_ref netmem; -- 2.54.0