* [PATCH v2 08/14] spi: stm32-ospi: Open-code message transfer walk
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609062526.94907-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. stm32_ospi_transfer_one_message() can
consume the following transfer as part of the current operation and then
advance the loop cursor to that entry.
Keep the transfer walk open-coded so the loop step observes that cursor
update and skips the consumed transfer. This preserves the existing
message sequencing semantics and prepares the code for the list iterator
update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/spi/spi-stm32-ospi.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/spi/spi-stm32-ospi.c b/drivers/spi/spi-stm32-ospi.c
index 4461c6e24b9e..4dc2b56b4c20 100644
--- a/drivers/spi/spi-stm32-ospi.c
+++ b/drivers/spi/spi-stm32-ospi.c
@@ -675,7 +675,9 @@ static int stm32_ospi_transfer_one_message(struct spi_controller *ctrl,
gpiod_set_value_cansleep(cs_gpiod, true);
- list_for_each_entry(transfer, &msg->transfers, transfer_list) {
+ for (transfer = list_first_entry(&msg->transfers, typeof(*transfer), transfer_list);
+ !list_entry_is_head(transfer, &msg->transfers, transfer_list);
+ transfer = list_next_entry(transfer, transfer_list)) {
u8 dummy_bytes = 0;
memset(&op, 0, sizeof(op));
--
2.43.0
^ permalink raw reply related
* [PATCH v2 07/14] spi: fsi: Open-code message transfer walk
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609062526.94907-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. fsi_spi_transfer_one_message() can combine
the current transfer with the following transfer and then advance the
cursor to that consumed entry.
Keep the transfer walk open-coded so the loop step observes that cursor
update and skips the consumed transfer. This preserves the existing
message sequencing semantics and prepares the code for the list iterator
update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/spi/spi-fsi.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/spi/spi-fsi.c b/drivers/spi/spi-fsi.c
index f6a75f0184c4..44999f00f5f6 100644
--- a/drivers/spi/spi-fsi.c
+++ b/drivers/spi/spi-fsi.c
@@ -434,7 +434,10 @@ static int fsi_spi_transfer_one_message(struct spi_controller *ctlr,
if (rc)
goto error;
- list_for_each_entry(transfer, &mesg->transfers, transfer_list) {
+ for (transfer = list_first_entry(&mesg->transfers,
+ typeof(*transfer), transfer_list);
+ !list_entry_is_head(transfer, &mesg->transfers, transfer_list);
+ transfer = list_next_entry(transfer, transfer_list)) {
struct fsi_spi_sequence seq;
struct spi_transfer *next = NULL;
--
2.43.0
^ permalink raw reply related
* [PATCH v2 06/14] drm/ttm: Open-code reservation list walk
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609062526.94907-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. ttm_eu_reserve_buffers() may move the
current validation buffer to the duplicates list and then rewinds the
cursor before continuing.
Keep the reservation walk open-coded so the loop step uses the cursor
selected by that duplicate handling. This preserves the existing
traversal semantics and prepares the code for the list iterator update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/gpu/drm/ttm/ttm_execbuf_util.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/ttm/ttm_execbuf_util.c b/drivers/gpu/drm/ttm/ttm_execbuf_util.c
index bc7a83a9fe44..8072f07d5557 100644
--- a/drivers/gpu/drm/ttm/ttm_execbuf_util.c
+++ b/drivers/gpu/drm/ttm/ttm_execbuf_util.c
@@ -86,7 +86,9 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket,
if (ticket)
ww_acquire_init(ticket, &reservation_ww_class);
- list_for_each_entry(entry, list, head) {
+ for (entry = list_first_entry(list, typeof(*entry), head);
+ !list_entry_is_head(entry, list, head);
+ entry = list_next_entry(entry, head)) {
struct ttm_buffer_object *bo = entry->bo;
unsigned int num_fences;
--
2.43.0
^ permalink raw reply related
* [PATCH v2 05/14] drm/i915: Open-code DFS dependency list walk
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609062526.94907-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. __i915_schedule() builds its DFS work list
while walking it by moving newly discovered dependencies to the tail.
Keep the DFS walk open-coded so the next dependency is resolved after any
tail moves performed by the body. This preserves the existing traversal
semantics and prepares the code for the list iterator update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/gpu/drm/i915/i915_scheduler.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index aec1342402ca..da1f60282df8 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -190,7 +190,9 @@ static void __i915_schedule(struct i915_sched_node *node,
* end result is a topological list of requests in reverse order, the
* last element in the list is the request we must execute first.
*/
- list_for_each_entry(dep, &dfs, dfs_link) {
+ for (dep = list_first_entry(&dfs, typeof(*dep), dfs_link);
+ !list_entry_is_head(dep, &dfs, dfs_link);
+ dep = list_next_entry(dep, dfs_link)) {
struct i915_sched_node *node = dep->signaler;
/* If we are already flying, we know we have no signalers */
--
2.43.0
^ permalink raw reply related
* [PATCH v2 04/14] drm/i915/gt: Open-code active timeline walk
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609062526.94907-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. __intel_gt_unset_wedged() drops
timelines->lock while waiting on a fence and then restarts the walk from
the list head after the lock is reacquired.
Keep the loop open-coded so the next timeline is selected after that
restart logic has run. This preserves the existing lock-drop traversal
semantics and prepares the code for the list iterator update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/gpu/drm/i915/gt/intel_reset.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c
index adff482a6c9c..fe0d87e248a7 100644
--- a/drivers/gpu/drm/i915/gt/intel_reset.c
+++ b/drivers/gpu/drm/i915/gt/intel_reset.c
@@ -1077,7 +1077,9 @@ static bool __intel_gt_unset_wedged(struct intel_gt *gt)
* No more can be submitted until we reset the wedged bit.
*/
spin_lock(&timelines->lock);
- list_for_each_entry(tl, &timelines->active_list, link) {
+ for (tl = list_first_entry(&timelines->active_list, typeof(*tl), link);
+ !list_entry_is_head(tl, &timelines->active_list, link);
+ tl = list_next_entry(tl, link)) {
struct dma_fence *fence;
fence = i915_active_fence_get(&tl->last_request);
--
2.43.0
^ permalink raw reply related
* [PATCH v2 03/14] drm/bridge: Open-code bridge chain list walks
From: Kaitao Cheng @ 2026-06-09 6:25 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609061347.93688-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry_from() and
list_for_each_entry_reverse() cache the next or previous element before
entering the loop body. The bridge enable and disable ordering code
adjusts its cursor to skip ranges that have already been handled.
Keep those walks open-coded so the loop step observes the cursor
selected by the body. This preserves the existing bridge ordering
semantics and prepares the code for the list iterator update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/gpu/drm/drm_bridge.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_bridge.c b/drivers/gpu/drm/drm_bridge.c
index d6f512b73389..a538aabc4e0b 100644
--- a/drivers/gpu/drm/drm_bridge.c
+++ b/drivers/gpu/drm/drm_bridge.c
@@ -868,7 +868,8 @@ void drm_atomic_bridge_chain_post_disable(struct drm_bridge *bridge,
encoder = bridge->encoder;
- list_for_each_entry_from(bridge, &encoder->bridge_chain, chain_node) {
+ for (; !list_entry_is_head(bridge, &encoder->bridge_chain, chain_node);
+ bridge = list_next_entry(bridge, chain_node)) {
limit = NULL;
if (!list_is_last(&bridge->chain_node, &encoder->bridge_chain)) {
@@ -962,7 +963,9 @@ void drm_atomic_bridge_chain_pre_enable(struct drm_bridge *bridge,
encoder = bridge->encoder;
- list_for_each_entry_reverse(iter, &encoder->bridge_chain, chain_node) {
+ for (iter = list_last_entry(&encoder->bridge_chain, typeof(*iter), chain_node);
+ !list_entry_is_head(iter, &encoder->bridge_chain, chain_node);
+ iter = list_prev_entry(iter, chain_node)) {
if (iter->pre_enable_prev_first) {
next = iter;
limit = bridge;
--
2.43.0
^ permalink raw reply related
* [PATCH v2 02/14] firewire: core: Open-code topology list walk
From: Kaitao Cheng @ 2026-06-09 6:13 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609061347.93688-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. for_each_fw_node() intentionally appends
newly discovered child nodes to the temporary walk list while the list is
being traversed.
Keep the loop open-coded so the next node is looked up only after
children have been appended. This preserves the current breadth-first
traversal semantics and prepares the code for the list iterator update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/firewire/core-topology.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/firewire/core-topology.c b/drivers/firewire/core-topology.c
index bb2d2db30795..df2ac0dab106 100644
--- a/drivers/firewire/core-topology.c
+++ b/drivers/firewire/core-topology.c
@@ -272,7 +272,9 @@ static void for_each_fw_node(struct fw_card *card, struct fw_node *root,
fw_node_get(root);
list_add_tail(&root->link, &list);
parent = NULL;
- list_for_each_entry(node, &list, link) {
+ for (node = list_first_entry(&list, typeof(*node), link);
+ !list_entry_is_head(node, &list, link);
+ node = list_next_entry(node, link)) {
node->color = card->color;
for (i = 0; i < node->port_count; i++) {
--
2.43.0
^ permalink raw reply related
* [PATCH v2 01/14] drbd: Open-code transfer log list walk
From: Kaitao Cheng @ 2026-06-09 6:13 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaitao Cheng
In-Reply-To: <20260609061347.93688-1-kaitao.cheng@linux.dev>
From: Kaitao Cheng <chengkaitao@kylinos.cn>
A later change will make list_for_each_entry() cache the next element
before entering the loop body. That is the desired behaviour for the
common case, but this transfer log walk temporarily drops
resource->req_lock and revalidates the cursor before continuing.
Keep the loop open-coded so the next request is derived after the body
has completed and after the cursor has been adjusted. This preserves the
existing traversal semantics and prepares the code for the list iterator
update.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
drivers/block/drbd/drbd_debugfs.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/block/drbd/drbd_debugfs.c b/drivers/block/drbd/drbd_debugfs.c
index 12460b584bcb..e90cead90e9d 100644
--- a/drivers/block/drbd/drbd_debugfs.c
+++ b/drivers/block/drbd/drbd_debugfs.c
@@ -308,7 +308,9 @@ static void seq_print_resource_transfer_log_summary(struct seq_file *m,
seq_puts(m, "n\tdevice\tvnr\t" RQ_HDR);
spin_lock_irq(&resource->req_lock);
- list_for_each_entry(req, &connection->transfer_log, tl_requests) {
+ for (req = list_first_entry(&connection->transfer_log, typeof(*req), tl_requests);
+ !list_entry_is_head(req, &connection->transfer_log, tl_requests);
+ req = list_next_entry(req, tl_requests)) {
unsigned int tmp = 0;
unsigned int s;
++count;
--
2.43.0
^ permalink raw reply related
* [PATCH v2 00/14] list: Prepare entry iterators to cache cursor state
From: Kaitao Cheng @ 2026-06-09 6:13 UTC (permalink / raw)
To: Andy Shevchenko, Muchun Song, Philipp Reisner, Lars Ellenberg,
Christoph Böhmwalder, Jens Axboe, Takashi Sakamoto,
Andrzej Hajda, Neil Armstrong, Robert Foss, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Jani Nikula, Joonas Lahtinen, Rodrigo Vivi, Tvrtko Ursulin,
Christian Koenig, Huang Rui, Eddie James, Mark Brown,
Maxime Coquelin, Alexandre Torgue, Laxman Dewangan,
Thierry Reding, Jonathan Hunter, Sowjanya Komatineni,
Davidlohr Bueso, Paul E . McKenney, Josh Triplett, Peter Zijlstra,
Ingo Molnar, Will Deacon, Boqun Feng, Liam Girdwood,
Jaroslav Kysela, Takashi Iwai
Cc: Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Matthew Auld,
Matthew Brost, Waiman Long, drbd-dev, linux-block,
linux1394-devel, dri-devel, intel-gfx, linux-spi, linux-stm32,
linux-arm-kernel, linux-tegra, linux-sound, linux-kernel,
Andrew Morton, Randy Dunlap, Christian Brauner, David Howells,
Luca Ceresoli, Kaitao Cheng, Kaito Cheng
From: Kaito Cheng <chengkaitao@kylinos.cn>
This series prepares for, and then updates, the list_for_each_entry()
family so the common entry iterators cache their next or previous cursor
before the loop body runs.
The first 13 patches open-code loops that intentionally depend on the
old "derive the next entry from the current cursor at the end of the
iteration" behaviour. These loops append work to the list being walked,
restart traversal after dropping a lock, skip an entry consumed by the
current iteration, or otherwise adjust the cursor in the loop body.
The final patch changes include/linux/list.h to keep a private cursor in
the common entry iterators while preserving the public macro interface.
The safe variants remain available when callers need the temporary
cursor explicitly or have stronger mutation requirements.
Changes in v2 (Muchun Song, Andy Shevchenko):
- Drop the list_for_each_entry_mutable*() helpers from v1 and make the
cursor change directly in the existing list_for_each_entry*() helpers.
- Open-code special list walks that rely on updating the loop cursor in
the body, preserving their existing traversal semantics.
Link to v1:
https://lore.kernel.org/all/20260529082149.76764-1-kaitao.cheng@linux.dev/
Kaitao Cheng (14):
drbd: Open-code transfer log list walk
firewire: core: Open-code topology list walk
drm/bridge: Open-code bridge chain list walks
drm/i915/gt: Open-code active timeline walk
drm/i915: Open-code DFS dependency list walk
drm/ttm: Open-code reservation list walk
spi: fsi: Open-code message transfer walk
spi: stm32-ospi: Open-code message transfer walk
spi: stm32-qspi: Open-code message transfer walk
spi: tegra210-quad: Open-code message transfer walk
locking/locktorture: Open-code ww mutex list walk
locking/ww_mutex: Open-code stress reorder list walk
ASoC: dapm: Open-code widget invalidation walk
list: Cache cursors in entry iterators
drivers/block/drbd/drbd_debugfs.c | 4 ++-
drivers/firewire/core-topology.c | 4 ++-
drivers/gpu/drm/drm_bridge.c | 7 ++--
drivers/gpu/drm/i915/gt/intel_reset.c | 4 ++-
drivers/gpu/drm/i915/i915_scheduler.c | 4 ++-
drivers/gpu/drm/ttm/ttm_execbuf_util.c | 4 ++-
drivers/spi/spi-fsi.c | 5 ++-
drivers/spi/spi-stm32-ospi.c | 4 ++-
drivers/spi/spi-stm32-qspi.c | 5 ++-
drivers/spi/spi-tegra210-quad.c | 4 ++-
include/linux/list.h | 46 ++++++++++++++++++++------
kernel/locking/locktorture.c | 4 ++-
kernel/locking/test-ww_mutex.c | 4 ++-
sound/soc/soc-dapm.c | 4 ++-
14 files changed, 78 insertions(+), 25 deletions(-)
--
2.43.0
^ permalink raw reply
* Re: [PATCH] block: clear zone write plugging flag before failing rejected BIOs
From: Jackie Liu @ 2026-06-09 0:36 UTC (permalink / raw)
To: Damien Le Moal, axboe; +Cc: linux-block
In-Reply-To: <33035623-1f1b-4391-9212-e2af5fd9457f@kernel.org>
2026年6月8日 19:42, "Damien Le Moal" <dlemoal@kernel.org mailto:dlemoal@kernel.org?to=%22Damien%20Le%20Moal%22%20%3Cdlemoal%40kernel.org%3E > 写到:
>
> On 2026/06/07 11:18, Jackie Liu wrote:
>
> >
> > From: Jackie Liu <liuyun01@kylinos.cn>
> >
> > Commit fe0418eb9bd6 ("block: Prevent potential deadlocks in zone write plug
> > error recovery") changed blk_zone_wplug_handle_write() to fail BIOs
> > directly when blk_zone_wplug_prepare_bio() rejects them, for example
> > because the write is not aligned to the cached write pointer or the plug
> > needs a write pointer update. However, the BIO is already marked with
> > BIO_ZONE_WRITE_PLUGGING at that point even though it is not issued.
> >
> > Completing such a BIO with bio_io_error() makes bio_endio() call
> > blk_zone_write_plug_bio_endio(), which treats the completion as a failed
> > device write and may poison the cached zone write pointer state by setting
> > BLK_ZONE_WPLUG_NEED_WP_UPDATE.
> >
> Yes, true. But you did not explain clearly why that is a problem. After all, if
> we hit this case, the user issued an unaligned BIO, and so forcing it to do a
> report zones to get everything in sync and the correct write pointer is not a
> bad thing.
>
> If fe0418eb9bd6 change is actually causing you problems, please describe that
> problem clearly. But ideally, I do not want to special case some error
> completions over others and prefer to have a single error path that result in
> the same state for the zone write plugs, regardless of a write error root cause.
Thanks for the review. I agree that the changelog did not describe a
concrete user-visible problem clearly enough.
I was treating NEED_WP_UPDATE on a BIO rejected before submission as stale
state poisoning, because no device write was actually issued. But as you
pointed out, for an invalid/non-sequential write, forcing the user to
resynchronize the write pointer through report zones is consistent with the
current conservative recovery model.
I do not have a concrete regression from fe0418eb9bd6 beyond that extra
recovery requirement, so please drop this patch for now.
Thanks.
Jackie
>
> >
> > Clear BIO_ZONE_WRITE_PLUGGING and drop the zone write plug reference before
> > failing the rejected BIO.
> >
> > Fixes: fe0418eb9bd6 ("block: Prevent potential deadlocks in zone write plug error recovery")
> > Cc: stable@vger.kernel.org # 6.13+
> > Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
> > ---
> > block/blk-zoned.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/block/blk-zoned.c b/block/blk-zoned.c
> > index 6a221c180889..855767d8bfc1 100644
> > --- a/block/blk-zoned.c
> > +++ b/block/blk-zoned.c
> > @@ -1502,7 +1502,9 @@ static bool blk_zone_wplug_handle_write(struct bio *bio, unsigned int nr_segs)
> > goto queue_bio;
> >
> > if (!blk_zone_wplug_prepare_bio(zwplug, bio)) {
> > + bio_clear_flag(bio, BIO_ZONE_WRITE_PLUGGING);
> > spin_unlock_irqrestore(&zwplug->lock, flags);
> > + disk_put_zone_wplug(zwplug);
> > bio_io_error(bio);
> > return true;
> > }
> >
> --
> Damien Le Moal
> Western Digital Research
>
^ permalink raw reply
* [PATCH] brd: normalize non-positive max_part before rounding it up
From: Samuel Moelius @ 2026-06-08 23:59 UTC (permalink / raw)
To: Jens Axboe; +Cc: Samuel Moelius, open list:BLOCK LAYER, open list
`max_part` is an `int` module parameter, but brd only resets zero before
rounding non-divisor values with `1UL << fls(max_part)`.
A negative value such as -1 passes the initial zero check. The modulo
test then reaches the roundup, where `fls(-1)` yields 32. On 64-bit
builds that produces 4294967296, which is then assigned back to `int
max_part` as zero. `brd_alloc()` passes that zero value to
`disk->minors`, and block core warns and rejects the disk.
Normalize non-positive values to the existing one-partition fallback
before the modulo/roundup, and apply the existing `DISK_MAX_PARTS` clamp
before the roundup so it only operates on representable, in-range
values.
Assisted-by: Codex:gpt-5.5-cyber-preview
Signed-off-by: Samuel Moelius <sam.moelius@trailofbits.com>
---
drivers/block/brd.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 00cc8122068f..ed9567f74579 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -371,9 +371,15 @@ static void brd_cleanup(void)
static inline void brd_check_and_reset_par(void)
{
- if (unlikely(!max_part))
+ if (unlikely(max_part <= 0))
max_part = 1;
+ if (max_part > DISK_MAX_PARTS) {
+ pr_info("brd: max_part can't be larger than %d, reset max_part = %d.\n",
+ DISK_MAX_PARTS, DISK_MAX_PARTS);
+ max_part = DISK_MAX_PARTS;
+ }
+
/*
* make sure 'max_part' can be divided exactly by (1U << MINORBITS),
* otherwise, it is possiable to get same dev_t when adding partitions.
@@ -381,11 +387,6 @@ static inline void brd_check_and_reset_par(void)
if ((1U << MINORBITS) % max_part != 0)
max_part = 1UL << fls(max_part);
- if (max_part > DISK_MAX_PARTS) {
- pr_info("brd: max_part can't be larger than %d, reset max_part = %d.\n",
- DISK_MAX_PARTS, DISK_MAX_PARTS);
- max_part = DISK_MAX_PARTS;
- }
}
static int __init brd_init(void)
--
2.43.0
^ permalink raw reply related
* Re: [PATCH 4/4] block: add configurable error injection
From: Bart Van Assche @ 2026-06-08 22:08 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Jonathan Corbet, Damien Le Moal, Hannes Reinecke, Keith Busch,
linux-block, linux-doc, Hannes Reinecke
In-Reply-To: <20260608051416.1205282-5-hch@lst.de>
On 6/7/26 10:14 PM, Christoph Hellwig wrote:
> +Configurable error injection allows injecting specific block layer status codes
> +for ranges of a block device. Errors can be injected unconditionally, or with a
ranges -> sector ranges?
> +static void error_inject_removall(struct gendisk *disk)
> +{
Is a letter "e" perhaps missing from the above function name? (remov ->
remove)
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH 3/4] block: add a str_to_blk_op helper
From: Bart Van Assche @ 2026-06-08 21:57 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Jonathan Corbet, Damien Le Moal, Hannes Reinecke, Keith Busch,
linux-block, linux-doc, Hannes Reinecke
In-Reply-To: <20260608051416.1205282-4-hch@lst.de>
On 6/7/26 10:14 PM, Christoph Hellwig wrote:
> +enum req_op str_to_blk_op(const char *op)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(blk_op_name); i++)
> + if (blk_op_name[i] && !strcmp(blk_op_name[i], op))
> + return (enum req_op)i;
> + return REQ_OP_LAST;
> +}
The above function is similar but not identical to
__sysfs_match_string(). Is __sysfs_match_string() good enough in this
context?
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH 2/4] block: add a "tag" for block status codes
From: Bart Van Assche @ 2026-06-08 21:55 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Jonathan Corbet, Damien Le Moal, Hannes Reinecke, Keith Busch,
linux-block, linux-doc, Hannes Reinecke
In-Reply-To: <20260608051416.1205282-3-hch@lst.de>
On 6/7/26 10:14 PM, Christoph Hellwig wrote:
> +const char *blk_status_to_tag(blk_status_t status)
> +{
> + int idx = (__force int)status;
> +
> + if (WARN_ON_ONCE(idx >= ARRAY_SIZE(blk_errors)))
> + return "<null>";
> + return blk_errors[idx].tag;
> +}
Since designated initializers are used to initialize blk_errors[], it's
probably a good idea to check the value of blk_errors[idx].tag, e.g. as
follows:
return blk_errors[idx].tag ?: "<null>";
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH 1/4] block: add a macro to initialize the status table
From: Bart Van Assche @ 2026-06-08 21:51 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Jonathan Corbet, Damien Le Moal, Hannes Reinecke, Keith Busch,
linux-block, linux-doc, Hannes Reinecke
In-Reply-To: <20260608051416.1205282-2-hch@lst.de>
On 6/7/26 10:14 PM, Christoph Hellwig wrote:
> Prepare for adding a new value to the error table by adding a macro
> to fill it.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
^ permalink raw reply
* Re: [PATCH] block: fix arg type in `blk_mq_update_nr_hw_queues`
From: Bart Van Assche @ 2026-06-08 15:50 UTC (permalink / raw)
To: Andreas Hindborg, Jens Axboe; +Cc: linux-block, linux-kernel
In-Reply-To: <20260608-update-hw-nodes-arg-v1-1-19aba440b32e@kernel.org>
On 6/8/26 1:39 AM, Andreas Hindborg wrote:
> The type of the argument `nr_hw_queues` in the function
> `blk_mq_update_nr_hw_queues` is a signed integer. This is wrong,
> considering the field `nr_hw_queues` of `struct blk_mq_tag_set` is
> unsigned. Thus, change the type of the parameter to unsigned.
Will there ever be storage devices that support more than 2**31 hardware
queues? If not, I think the word "wrong" in the commit message is too
strong.
If this patch does not change the behavior of the code for any practical
use case that would be good to mention.
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH]block: Observing higher CPU utilization during random IO testing
From: Jens Axboe @ 2026-06-08 15:18 UTC (permalink / raw)
To: wenxiong, linux-block; +Cc: tom.leiming, yukuai, wenxiong
In-Reply-To: <76b46c10-201a-4bd1-85a8-49ae6bace572@kernel.dk>
On 6/8/26 9:15 AM, Jens Axboe wrote:
> On 6/4/26 10:27 AM, wenxiong@linux.ibm.com wrote:
>> From: Wen Xiong <wenxiong@linux.ibm.com>
>>
>> Hi All,
>>
>> Our performance team observed the higher CPU utilization in RHEL10 compared
>> to RHEL9.8, observed the similar issue in upstream kernel(v7.1-rc6) as well
>> when running FIO random IO tests. Random IO tests are more CPU intensive
>> than sequential IO tests due to several factors: more context switching,
>> interrupt Handling, cache Inefficiency etc.
>>
>> Given commit 060406c61c7c ("block: add plug while submitting IO")
>> causes performance regression. This patch reverts it.
>>
>> Below is performance comparison with the latest upstream kernel.
>>
>> Iotype qd nj rmix mpstat busy mpstat busy
>> with inner plug without inner plug
>> Randrw 1 20 100 53% 24%
>> Randrw 1 40 100 70% 24%
>> Randrw 1 20 70 40% 24%
>> Randrw 1 40 70 60% 26%
>> Randrw 1 20 0 14% 6%
>> Randrw 1 40 0 20% 7%
>
> I'm fine with the actual change, but the commit title and commit message
> are pretty abysmal. If you were browsing git log and came across
> something that said
>
> "block: Observing higher CPU utilization during random IO testing"
>
> then what would you think the change did? You would have no idea. That
> title is a bug report title, it's not a git commit title. A good git
> commit message should tell you WHY a change is being made, and the title
> should be a short summary of that change.
>
> I'll rewrite these, but please keep this in mind for future submissions.
> Should not be necessary for me to completely rewrite your git commit
> message.
>
> You also identify 060406c61c7c as the commit that this is fixing. This
> goes into a Fixes line before your signed-off-by. And you would
> presumably want this to go into stable as well, yet that's not tagged.
> Please read the documentation on how to submit patches.
Patch also doesn't apply without fuzz. Please submit a v2 that's against
for-7.2/block and heeds the above advice to turn this into something
that's closer to a good patch submission.
--
Jens Axboe
^ permalink raw reply
* Re: [PATCH]block: Observing higher CPU utilization during random IO testing
From: Jens Axboe @ 2026-06-08 15:15 UTC (permalink / raw)
To: wenxiong, linux-block; +Cc: tom.leiming, yukuai, wenxiong
In-Reply-To: <20260604162709.3006327-1-wenxiong@linux.ibm.com>
On 6/4/26 10:27 AM, wenxiong@linux.ibm.com wrote:
> From: Wen Xiong <wenxiong@linux.ibm.com>
>
> Hi All,
>
> Our performance team observed the higher CPU utilization in RHEL10 compared
> to RHEL9.8, observed the similar issue in upstream kernel(v7.1-rc6) as well
> when running FIO random IO tests. Random IO tests are more CPU intensive
> than sequential IO tests due to several factors: more context switching,
> interrupt Handling, cache Inefficiency etc.
>
> Given commit 060406c61c7c ("block: add plug while submitting IO")
> causes performance regression. This patch reverts it.
>
> Below is performance comparison with the latest upstream kernel.
>
> Iotype qd nj rmix mpstat busy mpstat busy
> with inner plug without inner plug
> Randrw 1 20 100 53% 24%
> Randrw 1 40 100 70% 24%
> Randrw 1 20 70 40% 24%
> Randrw 1 40 70 60% 26%
> Randrw 1 20 0 14% 6%
> Randrw 1 40 0 20% 7%
I'm fine with the actual change, but the commit title and commit message
are pretty abysmal. If you were browsing git log and came across
something that said
"block: Observing higher CPU utilization during random IO testing"
then what would you think the change did? You would have no idea. That
title is a bug report title, it's not a git commit title. A good git
commit message should tell you WHY a change is being made, and the title
should be a short summary of that change.
I'll rewrite these, but please keep this in mind for future submissions.
Should not be necessary for me to completely rewrite your git commit
message.
You also identify 060406c61c7c as the commit that this is fixing. This
goes into a Fixes line before your signed-off-by. And you would
presumably want this to go into stable as well, yet that's not tagged.
Please read the documentation on how to submit patches.
--
Jens Axboe
^ permalink raw reply
* Re: [PATCH]block: Observing higher CPU utilization during random IO testing
From: Ming Lei @ 2026-06-08 15:04 UTC (permalink / raw)
To: wenxiong; +Cc: linux-block, axboe, yukuai, wenxiong
In-Reply-To: <20260604162709.3006327-1-wenxiong@linux.ibm.com>
On Thu, Jun 04, 2026 at 12:27:09PM -0400, wenxiong@linux.ibm.com wrote:
> From: Wen Xiong <wenxiong@linux.ibm.com>
>
> Hi All,
>
> Our performance team observed the higher CPU utilization in RHEL10 compared
> to RHEL9.8, observed the similar issue in upstream kernel(v7.1-rc6) as well
> when running FIO random IO tests. Random IO tests are more CPU intensive
> than sequential IO tests due to several factors: more context switching,
> interrupt Handling, cache Inefficiency etc.
>
> Given commit 060406c61c7c ("block: add plug while submitting IO")
> causes performance regression. This patch reverts it.
One thing is that plugging should be applied in outer io code path, also
it doesn't make sense to apply this timestamp optimization for single IO batch.
>
> Below is performance comparison with the latest upstream kernel.
>
> Iotype qd nj rmix mpstat busy mpstat busy
> with inner plug without inner plug
> Randrw 1 20 100 53% 24%
> Randrw 1 40 100 70% 24%
> Randrw 1 20 70 40% 24%
> Randrw 1 40 70 60% 26%
> Randrw 1 20 0 14% 6%
> Randrw 1 40 0 20% 7%
>
> Signed-off-by: Wen Xiong <wenxiong@linux.ibm.com>
> Suggested-by: Ming Lei <tom.leiming@gmail.com>
With commit log update, this patch looks fine:
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Thanks,
Ming
^ permalink raw reply
* [PATCH v3 08/22] iov_iter: Add a segmented queue of bio_vec[]
From: David Howells @ 2026-06-08 14:54 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, Trond Myklebust, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260608145432.681865-1-dhowells@redhat.com>
Add the concept of a segmented queue of bio_vec[] arrays. This allows an
indefinite quantity of elements to be handled and allows things like
network filesystems and crypto drivers to glue bits on the ends without
having to reallocate the array.
The bvecq struct that defines each segment also carries capacity/usage
information along with flags indicating whether the constituent memory
regions need freeing or unpinning and the file position of the first
element in a segment. The bvecq structs are refcounted to allow a queue to
be extracted in batches and split between a number of subrequests.
The bvecq can have the bio_vec[] it manages allocated in with it, but this
is not required. A flag is provided for if this is the case as comparing
->bv to ->__bv is not sufficient to detect this case.
Add an iterator type ITER_BVECQ for it. This is intended to replace
ITER_FOLIOQ (and ITER_XARRAY).
Note that the prev pointer is only really needed for iov_iter_revert() and
could be dispensed with if struct iov_iter contained the head information
as well as the current point.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvecq.h | 56 +++++++
include/linux/iov_iter.h | 69 +++++++-
include/linux/uio.h | 11 ++
lib/iov_iter.c | 322 ++++++++++++++++++++++++++++++++++++-
lib/scatterlist.c | 70 +++++++-
lib/tests/kunit_iov_iter.c | 262 ++++++++++++++++++++++++++++++
6 files changed, 784 insertions(+), 6 deletions(-)
create mode 100644 include/linux/bvecq.h
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 000000000000..15f16f905877
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+ BVECQ_MEM_EXTERNAL, /* Externally retained memory - no freeing */
+ BVECQ_MEM_PAGECACHE, /* Ref'd pagecache pages - must put */
+ BVECQ_MEM_GUP, /* Pinned memory from get_user_pages() - unpin */
+ BVECQ_MEM_ALLOCED, /* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together. The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ *
+ * The file position and discontiguity flag allow non-contiguous data sets to
+ * be chained together, but still teased apart without the need to convert the
+ * info in the bio_vec back into a folio pointer.
+ */
+struct bvecq {
+ struct bvecq *next; /* Next bvec in the list or NULL */
+ struct bvecq *prev; /* Prev bvec in the list or NULL */
+ unsigned long long fpos; /* File position */
+ refcount_t ref;
+ u32 priv; /* Private data */
+ u16 nr_slots; /* Number of elements in bv[] used */
+ u16 max_slots; /* Number of elements allocated in bv[] */
+ enum bvecq_mem mem_type:3; /* What sort of memory and how to free it */
+ bool inline_bv:1; /* T if __bv[] is being used */
+ bool discontig:1; /* T if not contiguous with previous bvecq */
+ struct bio_vec *bv; /* Pointer to array of page fragments */
+ struct bio_vec __bv[]; /* Default array (if ->inline_bv) */
+};
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd398..c19a4c561ab4 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,6 +10,7 @@
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
@@ -141,6 +142,66 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
return progress;
}
+/*
+ * Handle ITER_BVECQ.
+ */
+static __always_inline
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+ iov_step_f step)
+{
+ const struct bvecq *bq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t progress = 0, skip = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bvec;
+ struct page *page;
+ size_t poff, plen;
+ void *base;
+
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ }
+
+ bvec = &bq->bv[slot];
+ page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+ poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+ plen = umin(bvec->bv_len - skip, len);
+
+ while (plen > 0) {
+ size_t part, remain, consumed;
+
+ part = umin(plen, PAGE_SIZE - poff);
+ base = kmap_local_page(page) + poff;
+ remain = step(base, progress, part, priv, priv2);
+ kunmap_local(base);
+
+ consumed = part - remain;
+ progress += consumed;
+ skip += consumed;
+ len -= consumed;
+ if (!len || remain)
+ goto stop;
+ page++;
+ poff = 0;
+ plen -= consumed;
+ }
+
+ skip = 0;
+ slot++;
+ } while (len);
+
+stop:
+ iter->bvecq_slot = slot;
+ iter->bvecq = bq;
+ iter->iov_offset = skip;
+ iter->count -= progress;
+ return progress;
+}
+
/*
* Handle ITER_FOLIOQ.
*/
@@ -306,6 +367,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
@@ -342,8 +405,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
* buffer is presented in segments, which for kernel iteration are broken up by
* physical pages and mapped, with the mapped address being presented.
*
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
- * iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
+ * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
*
* A step functions, @step, must be provided, one for handling mapped kernel
* addresses and the other is given user addresses which have the potential to
@@ -370,6 +433,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
diff --git a/include/linux/uio.h b/include/linux/uio.h
index a9bc5b3067e3..f7cfa6ea8213 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -26,6 +26,7 @@ enum iter_type {
ITER_IOVEC,
ITER_BVEC,
ITER_KVEC,
+ ITER_BVECQ,
ITER_FOLIOQ,
ITER_XARRAY,
ITER_DISCARD,
@@ -68,6 +69,7 @@ struct iov_iter {
const struct iovec *__iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
+ const struct bvecq *bvecq;
const struct folio_queue *folioq;
struct xarray *xarray;
void __user *ubuf;
@@ -77,6 +79,7 @@ struct iov_iter {
};
union {
unsigned long nr_segs;
+ u16 bvecq_slot;
u8 folioq_slot;
loff_t xarray_start;
};
@@ -145,6 +148,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+ return iov_iter_type(i) == ITER_BVECQ;
+}
+
static inline bool iov_iter_is_folioq(const struct iov_iter *i)
{
return iov_iter_type(i) == ITER_FOLIOQ;
@@ -295,6 +303,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
unsigned long nr_segs, size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq,
+ unsigned int first_slot, unsigned int offset, size_t count);
void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
const struct folio_queue *folioq,
unsigned int first_slot, unsigned int offset, size_t count);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index cac7d7364bc2..63fc75c2bc48 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,6 +538,39 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
i->__iov = iov;
}
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ if (!i->count)
+ return;
+ i->count -= by;
+
+ by += i->iov_offset; /* From beginning of current segment. */
+ do {
+ size_t len;
+
+ while (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ }
+
+ len = bq->bv[slot].bv_len;
+
+ if (likely(by < len))
+ break;
+ by -= len;
+ slot++;
+ } while (by);
+
+ i->iov_offset = by;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
{
const struct folio_queue *folioq = i->folioq;
@@ -583,6 +616,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
iov_iter_iovec_advance(i, size);
} else if (iov_iter_is_bvec(i)) {
iov_iter_bvec_advance(i, size);
+ } else if (iov_iter_is_bvecq(i)) {
+ iov_iter_bvecq_advance(i, size);
} else if (iov_iter_is_folioq(i)) {
iov_iter_folioq_advance(i, size);
} else if (iov_iter_is_discard(i)) {
@@ -591,6 +626,32 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
}
EXPORT_SYMBOL(iov_iter_advance);
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ for (;;) {
+ size_t len;
+
+ if (slot == 0) {
+ bq = bq->prev;
+ slot = bq->nr_slots;
+ }
+ slot--;
+
+ len = bq->bv[slot].bv_len;
+ if (unroll <= len) {
+ i->iov_offset = len - unroll;
+ break;
+ }
+ unroll -= len;
+ }
+
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
{
const struct folio_queue *folioq = i->folioq;
@@ -648,6 +709,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
}
unroll -= n;
}
+ } else if (iov_iter_is_bvecq(i)) {
+ i->iov_offset = 0;
+ iov_iter_bvecq_revert(i, unroll);
} else if (iov_iter_is_folioq(i)) {
i->iov_offset = 0;
iov_iter_folioq_revert(i, unroll);
@@ -678,9 +742,24 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
}
+ if (!i->count)
+ return 0;
+ if (unlikely(iov_iter_is_bvecq(i))) {
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+ size_t offset = i->iov_offset;
+
+ while (slot >= bq->nr_slots) {
+ bq = bq->next;
+ if (!bq)
+ return 0;
+ slot = 0;
+ offset = 0;
+ }
+ return umin(i->count, bq->bv[slot].bv_len - offset);
+ }
if (unlikely(iov_iter_is_folioq(i)))
- return !i->count ? 0 :
- umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+ return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
return i->count;
}
EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -717,6 +796,35 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
}
EXPORT_SYMBOL(iov_iter_bvec);
+/**
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers. The pages *must* be prevented
+ * from evaporation, either by the caller.
+ */
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq, unsigned int first_slot,
+ unsigned int offset, size_t count)
+{
+ WARN_ON(direction & ~(READ | WRITE));
+ *i = (struct iov_iter) {
+ .iter_type = ITER_BVECQ,
+ .data_source = direction,
+ .bvecq = bvecq,
+ .bvecq_slot = first_slot,
+ .count = count,
+ .iov_offset = offset,
+ };
+}
+EXPORT_SYMBOL(iov_iter_bvec_queue);
+
/**
* iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
* @i: The iterator to initialise.
@@ -839,6 +947,37 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
return res;
}
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+ const struct bvecq *bq;
+ unsigned long res = 0;
+ unsigned int slot = iter->bvecq_slot;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ if (!size)
+ return res;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ res |= bvec->bv_offset + skip;
+ res |= part;
+
+ size -= part;
+ if (size == 0)
+ return res;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+
+ return res;
+}
+
unsigned long iov_iter_alignment(const struct iov_iter *i)
{
if (likely(iter_is_ubuf(i))) {
@@ -854,6 +993,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return iov_iter_alignment_bvec(i);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_alignment_bvecq(i);
/* With both xarray and folioq types, we're dealing with whole folios. */
if (iov_iter_is_folioq(i))
@@ -1066,6 +1207,36 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
return npages;
}
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+ const struct bvecq *bq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t npages = 0;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+ if (npages >= maxpages)
+ goto out;
+
+ size -= part;
+ if (!size)
+ goto out;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+out:
+ return umin(npages, maxpages);
+}
+
int iov_iter_npages(const struct iov_iter *i, int maxpages)
{
if (unlikely(!i->count))
@@ -1080,6 +1251,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
return iov_npages(i, maxpages);
if (iov_iter_is_bvec(i))
return bvec_npages(i, maxpages);
+ if (iov_iter_is_bvecq(i))
+ return iov_npages_bvecq(i, maxpages);
if (iov_iter_is_folioq(i)) {
unsigned offset = i->iov_offset % PAGE_SIZE;
int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1366,6 +1539,147 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
i->nr_segs = state->nr_segs;
}
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+ size_t maxsize,
+ unsigned int maxpages)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t remain = umin(maxsize, iter->count);
+ size_t count = 0, offset = iter->iov_offset;
+
+ for (;; slot++) {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ while (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(remain > 0);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ }
+
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (unlikely(!bv->bv_page)) {
+ if (blen && count > 0)
+ break;
+ continue;
+ }
+ if (!PAGE_ALIGNED(boff) && count > 0)
+ break;
+
+ boff += offset;
+ blen -= offset;
+ offset = 0;
+ if (!blen)
+ continue;
+
+ blen = umin(blen, remain);
+ remain -= blen;
+ blen += offset_in_page(boff);
+ count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+ if (!PAGE_ALIGNED(blen))
+ break;
+ if (remain <= 0)
+ break;
+ if (count >= maxpages)
+ break;
+ }
+
+ return umin(count, maxpages);
+}
+
+/*
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
+ */
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+ struct page ***pages, size_t maxsize,
+ unsigned int maxpages,
+ iov_iter_extraction_t extraction_flags,
+ size_t *offset0)
+{
+ const struct bvecq *bvecq;
+ struct page **p;
+ unsigned int slot, nr = 0;
+ size_t extracted = 0, offset;
+
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+
+ if (!*pages) {
+ *pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+ if (!*pages)
+ return -ENOMEM;
+ }
+
+ p = *pages;
+
+ /* Now transcribe the page pointers. */
+ extracted = 0;
+ bvecq = iter->bvecq;
+ offset = iter->iov_offset;
+ slot = iter->bvecq_slot;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ while (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ }
+
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (!bv->bv_page)
+ blen = 0;
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - extracted, blen - offset);
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *offset0 = poff;
+
+ p[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
+ slot++;
+ }
+ } while (nr < maxpages && extracted < maxsize);
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
+ return extracted;
+}
+
/*
* Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does
* not get references on the pages, nor does it get a pin on them.
@@ -1708,6 +2022,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
return iov_iter_extract_bvec_pages(i, pages, maxsize,
maxpages, extraction_flags,
offset0);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+ maxpages, extraction_flags,
+ offset0);
if (iov_iter_is_folioq(i))
return iov_iter_extract_folioq_pages(i, pages, maxsize,
maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index b7fe91ef35b8..b92144659543 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,6 +10,7 @@
#include <linux/highmem.h>
#include <linux/kmemleak.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
#include <linux/folio_queue.h>
@@ -1267,6 +1268,68 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
return ret;
}
+/*
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
+ * the scatterlist. The pages are not pinned.
+ */
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
+ ssize_t maxsize,
+ struct sg_table *sgtable,
+ unsigned int sg_max,
+ iov_iter_extraction_t extraction_flags)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ struct scatterlist *sg = sgtable->sgl + sgtable->nents;
+ unsigned int seg = iter->bvecq_slot;
+ ssize_t ret = 0;
+ size_t offset = iter->iov_offset;
+
+ if (seg >= bvecq->nr_slots) {
+ bvecq = bvecq->next;
+ if (WARN_ON_ONCE(!bvecq))
+ return 0;
+ seg = 0;
+ }
+
+ do {
+ const struct bio_vec *bv = &bvecq->bv[seg];
+ size_t blen = bv->bv_len;
+
+ if (!bv->bv_page)
+ blen = 0;
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - ret, blen - offset);
+
+ sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+ sgtable->nents++;
+ sg++;
+ sg_max--;
+ offset += part;
+ ret += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
+ seg++;
+ if (seg >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(ret < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ seg = 0;
+ }
+ }
+ } while (sg_max > 0 && ret < maxsize);
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = seg;
+ iter->iov_offset = offset;
+ iter->count -= ret;
+ return ret;
+}
+
/*
* Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
* the scatterlist. The pages are not pinned.
@@ -1390,8 +1453,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
* addition of @sg_max elements.
*
* The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
+ * aren't pinned; DISCARD-type is not supported.
*
* No end mark is placed on the scatterlist; that's left to the caller.
*
@@ -1423,6 +1486,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
case ITER_KVEC:
return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
+ case ITER_BVECQ:
+ return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+ extraction_flags);
case ITER_FOLIOQ:
return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index f02f7b7aa796..d3e8e22ca9ca 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,6 +12,7 @@
#include <linux/mm.h>
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
#include <linux/scatterlist.h>
#include <linux/minmax.h>
@@ -545,6 +546,185 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
KUNIT_SUCCEED(test);
}
+static void iov_kunit_destroy_bvecq(void *data)
+{
+ struct bvecq *bq, *next;
+
+ for (bq = data; bq; bq = next) {
+ next = bq->next;
+ for (int i = 0; i < bq->nr_slots; i++)
+ if (bq->bv[i].bv_page)
+ put_page(bq->bv[i].bv_page);
+ kfree(bq);
+ }
+}
+
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+ bq->max_slots = max_slots;
+ bq->bv = bq->__bv;
+ bq->inline_bv = true;
+ return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = iov_kunit_alloc_bvecq(test, max_slots);
+ kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+ return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
+ struct iov_iter *iter, int dir,
+ struct bvecq *bq_head,
+ struct page **pages, size_t npages)
+{
+ struct bvecq *bq = bq_head;
+ size_t size = 0;
+
+ for (int i = 0; i < npages; i++) {
+ if (bq->nr_slots >= bq->max_slots) {
+ bq->next = iov_kunit_alloc_bvecq(test, 13);
+ bq->next->prev = bq;
+ bq = bq->next;
+ }
+ bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+ bq->nr_slots++;
+ size += PAGE_SIZE;
+ }
+ iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
+}
+
+/*
+ * Test copying to a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, patt;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ for (i = 0; i < bufsize; i++)
+ scratch[i] = pattern(i);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ memset(buffer, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_to_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ }
+
+ /* Build the expected image in the scratch buffer. */
+ patt = 0;
+ memset(scratch, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++)
+ for (i = pr->from; i < pr->to; i++)
+ scratch[i] = pattern(patt++);
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
+ if (buffer[i] != scratch[i])
+ return;
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
+/*
+ * Test copying from a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, j;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ for (i = 0; i < bufsize; i++)
+ buffer[i] = pattern(i);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ memset(scratch, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_from_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ }
+
+ /* Build the expected image in the main buffer. */
+ i = 0;
+ memset(buffer, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ for (j = pr->from; j < pr->to; j++) {
+ buffer[i++] = pattern(j);
+ if (i >= bufsize)
+ goto stop;
+ }
+ }
+stop:
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
+ if (scratch[i] != buffer[i])
+ return;
+ }
+
+ KUNIT_SUCCEED(test);
+}
+
static void iov_kunit_destroy_xarray(void *data)
{
struct xarray *xarray = data;
@@ -860,6 +1040,85 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
KUNIT_SUCCEED(test);
}
+/*
+ * Test the extraction of ITER_BVECQ-type iterators.
+ */
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **bpages, *pagelist[8], **pages = pagelist;
+ ssize_t len;
+ size_t bufsize, size = 0, npages;
+ int i, from;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ iov_kunit_create_buffer(test, &bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ from = pr->from;
+ size = pr->to - from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, from);
+
+ do {
+ size_t offset0 = LONG_MAX;
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++)
+ pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
+
+ len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
+ ARRAY_SIZE(pagelist), 0, &offset0);
+ KUNIT_EXPECT_GE(test, len, 0);
+ if (len < 0)
+ break;
+ KUNIT_EXPECT_LE(test, len, size);
+ KUNIT_EXPECT_EQ(test, iter.count, size - len);
+ if (len == 0)
+ break;
+ size -= len;
+ KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
+ KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
+ struct page *p;
+ ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
+ int ix;
+
+ KUNIT_ASSERT_GE(test, part, 0);
+ ix = from / PAGE_SIZE;
+ KUNIT_ASSERT_LT(test, ix, npages);
+ p = bpages[ix];
+ KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
+ KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
+ from += part;
+ len -= part;
+ KUNIT_ASSERT_GE(test, len, 0);
+ if (len == 0)
+ break;
+ offset0 = 0;
+ }
+
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ } while (iov_iter_count(&iter) > 0);
+
+ KUNIT_EXPECT_EQ(test, size, 0);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
/*
* Test the extraction of ITER_FOLIOQ-type iterators.
*/
@@ -1219,12 +1478,15 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
KUNIT_CASE(iov_kunit_copy_from_kvec),
KUNIT_CASE(iov_kunit_copy_to_bvec),
KUNIT_CASE(iov_kunit_copy_from_bvec),
+ KUNIT_CASE(iov_kunit_copy_to_bvecq),
+ KUNIT_CASE(iov_kunit_copy_from_bvecq),
KUNIT_CASE(iov_kunit_copy_to_folioq),
KUNIT_CASE(iov_kunit_copy_from_folioq),
KUNIT_CASE(iov_kunit_copy_to_xarray),
KUNIT_CASE(iov_kunit_copy_from_xarray),
KUNIT_CASE(iov_kunit_extract_pages_kvec),
KUNIT_CASE(iov_kunit_extract_pages_bvec),
+ KUNIT_CASE(iov_kunit_extract_pages_bvecq),
KUNIT_CASE(iov_kunit_extract_pages_folioq),
KUNIT_CASE(iov_kunit_extract_pages_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
^ permalink raw reply related
* [PATCH v3 07/22] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
From: David Howells @ 2026-06-08 14:54 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, Trond Myklebust, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260608145432.681865-1-dhowells@redhat.com>
Make iov_iter_get_pages*() wrap iov_iter_extract_pages() for kernel
iterator types (e.g. ITER_BVEC, ITER_FOLIOQ, ITER_XARRAY). The pages
obtained have their refcounts incremented afterwards if they're not slab
pages. ITER_KVEC is left returning -EFAULT.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
lib/iov_iter.c | 164 ++++++-------------------------------------------
1 file changed, 19 insertions(+), 145 deletions(-)
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 243662af1af7..cac7d7364bc2 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -910,118 +910,34 @@ static int want_pages_array(struct page ***res, size_t size,
return count;
}
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
+/*
+ * Wrap iov_iter_extract_pages() and then pin the non-slab pages we got back.
+ * This only works for non-user iterator types as get_pages uses get_user_pages
+ * not pin_user_pages.
+ */
+static ssize_t iter_get_kernel_pages(struct iov_iter *iter,
struct page ***ppages, size_t maxsize,
unsigned maxpages, size_t *_start_offset)
{
- const struct folio_queue *folioq = iter->folioq;
struct page **pages;
- unsigned int slot = iter->folioq_slot;
- size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+ ssize_t ret, done;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(iov_offset != 0))
- return -EIO;
- }
+ ret = iov_iter_extract_pages(iter, ppages, maxsize, maxpages,
+ 0, _start_offset);
+ if (ret <= 0)
+ return ret;
- maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
- if (!maxpages)
- return -ENOMEM;
- *_start_offset = iov_offset & ~PAGE_MASK;
pages = *ppages;
+ for (done = ret + *_start_offset; done > 0; done -= PAGE_SIZE) {
+ struct folio *folio = page_folio(*pages);
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- count -= part;
- iov_offset += part;
- extracted += part;
-
- *pages = folio_page(folio, offset / PAGE_SIZE);
- get_page(*pages);
- pages++;
- maxpages--;
- }
-
- if (maxpages == 0 || extracted >= maxsize)
- break;
-
- if (iov_offset >= fsize) {
- iov_offset = 0;
- slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
- }
- }
-
- iter->count = count;
- iter->iov_offset = iov_offset;
- iter->folioq = folioq;
- iter->folioq_slot = slot;
- return extracted;
-}
-
-static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa,
- pgoff_t index, unsigned int nr_pages)
-{
- XA_STATE(xas, xa, index);
- struct folio *folio;
- unsigned int ret = 0;
-
- rcu_read_lock();
- for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
- if (xas_retry(&xas, folio))
- continue;
-
- /* Has the folio moved or been split? */
- if (unlikely(folio != xas_reload(&xas))) {
- xas_reset(&xas);
- continue;
- }
-
- pages[ret] = folio_file_page(folio, xas.xa_index);
- folio_get(folio);
- if (++ret == nr_pages)
- break;
+ if (!folio_test_slab(folio))
+ folio_get(folio);
+ pages++;
}
- rcu_read_unlock();
return ret;
}
-static ssize_t iter_xarray_get_pages(struct iov_iter *i,
- struct page ***pages, size_t maxsize,
- unsigned maxpages, size_t *_start_offset)
-{
- unsigned nr, offset, count;
- pgoff_t index;
- loff_t pos;
-
- pos = i->xarray_start + i->iov_offset;
- index = pos >> PAGE_SHIFT;
- offset = pos & ~PAGE_MASK;
- *_start_offset = offset;
-
- count = want_pages_array(pages, maxsize, offset, maxpages);
- if (!count)
- return -ENOMEM;
- nr = iter_xarray_populate_pages(*pages, i->xarray, index, count);
- if (nr == 0)
- return 0;
-
- maxsize = min_t(size_t, nr * PAGE_SIZE - offset, maxsize);
- i->iov_offset += maxsize;
- i->count -= maxsize;
- return maxsize;
-}
-
/* must be done on non-empty ITER_UBUF or ITER_IOVEC one */
static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
{
@@ -1044,22 +960,6 @@ static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
BUG(); // if it had been empty, we wouldn't get called
}
-/* must be done on non-empty ITER_BVEC one */
-static struct page *first_bvec_segment(const struct iov_iter *i,
- size_t *size, size_t *start)
-{
- struct page *page;
- size_t skip = i->iov_offset, len;
-
- len = i->bvec->bv_len - skip;
- if (*size > len)
- *size = len;
- skip += i->bvec->bv_offset;
- page = i->bvec->bv_page + skip / PAGE_SIZE;
- *start = skip % PAGE_SIZE;
- return page;
-}
-
static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
unsigned int maxpages, size_t *start)
@@ -1095,36 +995,10 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
iov_iter_advance(i, maxsize);
return maxsize;
}
- if (iov_iter_is_bvec(i)) {
- struct page **p;
- struct page *page;
- page = first_bvec_segment(i, &maxsize, start);
- n = want_pages_array(pages, maxsize, *start, maxpages);
- if (!n)
- return -ENOMEM;
- p = *pages;
- for (int k = 0; k < n; k++) {
- struct folio *folio = page_folio(page + k);
- p[k] = page + k;
- if (!folio_test_slab(folio))
- folio_get(folio);
- }
- maxsize = min_t(size_t, maxsize, n * PAGE_SIZE - *start);
- i->count -= maxsize;
- i->iov_offset += maxsize;
- if (i->iov_offset == i->bvec->bv_len) {
- i->iov_offset = 0;
- i->bvec++;
- i->nr_segs--;
- }
- return maxsize;
- }
- if (iov_iter_is_folioq(i))
- return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
- if (iov_iter_is_xarray(i))
- return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
- return -EFAULT;
+ if (iov_iter_is_kvec(i))
+ return -EFAULT;
+ return iter_get_kernel_pages(i, pages, maxsize, maxpages, start);
}
ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
^ permalink raw reply related
* [PATCH v3 06/22] Add a function to kmap one page of a multipage bio_vec
From: David Howells @ 2026-06-08 14:54 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, Trond Myklebust, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260608145432.681865-1-dhowells@redhat.com>
Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally). The caller is responsible
for calculating how much of the page is then available.
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvec.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index d36dd476feda..f834a862224e 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -299,4 +299,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
^ permalink raw reply related
* Re: [PATCH 4/4] block: add configurable error injection
From: Jens Axboe @ 2026-06-08 14:53 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jonathan Corbet, Damien Le Moal, Hannes Reinecke, Keith Busch,
linux-block, linux-doc, Hannes Reinecke
In-Reply-To: <20260608051416.1205282-5-hch@lst.de>
On 6/7/26 11:14 PM, Christoph Hellwig wrote:
> diff --git a/block/blk.h b/block/blk.h
> index e8b7d5517086..10df23b2cb90 100644
> --- a/block/blk.h
> +++ b/block/blk.h
> @@ -660,6 +660,18 @@ static inline bool should_fail_request(struct block_device *part,
> }
> #endif /* CONFIG_FAIL_MAKE_REQUEST */
>
> +void blk_error_injection_init(struct gendisk *disk);
> +void blk_error_injection_exit(struct gendisk *disk);
> +bool __blk_error_inject(struct bio *bio);
> +static inline bool blk_error_inject(struct bio *bio)
> +{
> + if (!IS_ENABLED(CONFIG_BLK_ERROR_INJECTION))
> + return false;
> + if (!test_bit(GD_ERROR_INJECT, &bio->bi_bdev->bd_disk->state))
> + return false;
> + return __blk_error_inject(bio);
> +}
I really hate this part, that's a pretty deep set of pointer chasings to
figure out if injection is enabled or not, when in practice error
injection is only ever enabled for specific test cases and distros
invariably will set CONFIG_BLK_ERROR_INJECTION because they turn on
every damn thing under the sun.
IOW, that won't fly for the hot path. Maybe a static key would be useful
here?
--
Jens Axboe
^ permalink raw reply
* Re: [PATCH] block: optimize I/O merge hot path with unlikely() hints
From: Jens Axboe @ 2026-06-08 14:34 UTC (permalink / raw)
To: Steven Feng; +Cc: linux-block, linux-kernel
In-Reply-To: <tencent_79B652BD0CC23E093F27914380F161E7E505@qq.com>
On Sat, 06 Jun 2026 10:42:18 +0800, Steven Feng wrote:
> Remove redundant '== false' comparisons and add unlikely() branch
> prediction hints in block I/O merge path functions.
>
> These functions (ll_new_hw_segment, ll_merge_requests_fn, and
> blk_rq_merge_ok) are executed on every I/O request merge attempt,
> making them critical hot paths. Data integrity check failures are
> rare events, so marking these conditions as unlikely() helps the
> CPU optimize the common case by improving branch prediction.
>
> [...]
Applied, thanks!
[1/1] block: optimize I/O merge hot path with unlikely() hints
commit: 7ed4aab1381f3439f45032eb860f89d9da5e45c2
Best regards,
--
Jens Axboe
^ permalink raw reply
* Re: [PATCH next] drivers/block/rbd: Use strscpy() to copy strings into arrays
From: Jens Axboe @ 2026-06-08 14:34 UTC (permalink / raw)
To: Kees Cook, linux-hardening, Arnd Bergmann, ceph-devel,
linux-block, linux-kernel, david.laight.linux
Cc: Ilya Dryomov
In-Reply-To: <20260606202744.5113-5-david.laight.linux@gmail.com>
On Sat, 06 Jun 2026 21:27:44 +0100, david.laight.linux@gmail.com wrote:
> Replacing strcpy() with strscpy() ensures than overflow of the target
> buffer cannot happen.
Applied, thanks!
[1/1] drivers/block/rbd: Use strscpy() to copy strings into arrays
commit: 5ef1b0194b382fafe5023b5b014e4db3b948ee15
Best regards,
--
Jens Axboe
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox