* Re: [PATCH v2 2/5] samples: rust_misc_device: use vertical import style
From: Onur Özkan @ 2026-05-20 2:58 UTC (permalink / raw)
To: Alvin Sun
Cc: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich, rust-for-linux,
linux-block, Onur Özkan
In-Reply-To: <20260520-miscdev-use-format-v2-2-64dc48fc1345@linux.dev>
On Wed, 20 May 2026 10:40:09 +0800
Alvin Sun <alvin.sun@linux.dev> wrote:
> Convert `use` imports to vertical layout for better readability and
> maintainability.
>
> Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
Reviewed-by: Onur Özkan <work@onurozkan.dev>
> ---
> samples/rust/rust_misc_device.rs | 34 ++++++++++++++++++++++++++++------
> 1 file changed, 28 insertions(+), 6 deletions(-)
>
> diff --git a/samples/rust/rust_misc_device.rs b/samples/rust/rust_misc_device.rs
> index 87a1fe63533ae..41e26c825060b 100644
> --- a/samples/rust/rust_misc_device.rs
> +++ b/samples/rust/rust_misc_device.rs
> @@ -97,14 +97,36 @@
>
> use kernel::{
> device::Device,
> - fs::{File, Kiocb},
> - ioctl::{_IO, _IOC_SIZE, _IOR, _IOW},
> - iov::{IovIterDest, IovIterSource},
> - miscdevice::{MiscDevice, MiscDeviceOptions, MiscDeviceRegistration},
> + fs::{
> + File,
> + Kiocb, //
> + },
> + ioctl::{
> + _IO,
> + _IOC_SIZE,
> + _IOR,
> + _IOW, //
> + },
> + iov::{
> + IovIterDest,
> + IovIterSource, //
> + },
> + miscdevice::{
> + MiscDevice,
> + MiscDeviceOptions,
> + MiscDeviceRegistration, //
> + },
> new_mutex,
> prelude::*,
> - sync::{aref::ARef, Mutex},
> - uaccess::{UserSlice, UserSliceReader, UserSliceWriter},
> + sync::{
> + aref::ARef,
> + Mutex, //
> + },
> + uaccess::{
> + UserSlice,
> + UserSliceReader,
> + UserSliceWriter, //
> + },
> };
>
> const RUST_MISC_DEV_HELLO: u32 = _IO('|' as u32, 0x80);
>
> --
> 2.43.0
>
>
^ permalink raw reply
* Re: [PATCH v2 1/5] rust: miscdevice: use vertical import style
From: Onur Özkan @ 2026-05-20 2:57 UTC (permalink / raw)
To: Alvin Sun
Cc: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich, rust-for-linux,
linux-block, Onur Özkan
In-Reply-To: <20260520-miscdev-use-format-v2-1-64dc48fc1345@linux.dev>
On Wed, 20 May 2026 10:40:08 +0800
Alvin Sun <alvin.sun@linux.dev> wrote:
> Convert `use` imports to vertical layout for better readability and
> maintainability.
>
> Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
Reviewed-by: Onur Özkan <work@onurozkan.dev>
> ---
> rust/kernel/miscdevice.rs | 34 ++++++++++++++++++++++++++++------
> 1 file changed, 28 insertions(+), 6 deletions(-)
>
> diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs
> index c3c2052c92069..05a6b6b9770f2 100644
> --- a/rust/kernel/miscdevice.rs
> +++ b/rust/kernel/miscdevice.rs
> @@ -11,16 +11,38 @@
> use crate::{
> bindings,
> device::Device,
> - error::{to_result, Error, Result, VTABLE_DEFAULT_ERROR},
> - ffi::{c_int, c_long, c_uint, c_ulong},
> - fs::{File, Kiocb},
> - iov::{IovIterDest, IovIterSource},
> + error::{
> + to_result,
> + Error,
> + Result,
> + VTABLE_DEFAULT_ERROR, //
> + },
> + ffi::{
> + c_int,
> + c_long,
> + c_uint,
> + c_ulong, //
> + },
> + fs::{
> + File,
> + Kiocb, //
> + },
> + iov::{
> + IovIterDest,
> + IovIterSource, //
> + },
> mm::virt::VmaNew,
> prelude::*,
> seq_file::SeqFile,
> - types::{ForeignOwnable, Opaque},
> + types::{
> + ForeignOwnable,
> + Opaque, //
> + },
> +};
> +use core::{
> + marker::PhantomData,
> + pin::Pin, //
> };
> -use core::{marker::PhantomData, pin::Pin};
>
> /// Options for creating a misc device.
> #[derive(Copy, Clone)]
>
> --
> 2.43.0
>
>
^ permalink raw reply
* [PATCH v2 5/5] rust: block: mq: remove redundant imports and format
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
In-Reply-To: <20260520-miscdev-use-format-v2-0-64dc48fc1345@linux.dev>
Drop `Result`, `Pin`, `pin_data`, `pinned_drop`, `PinInit`, and
`try_pin_init` imports already provided by `kernel::prelude`.
Simplify `error` imports and flatten parameters formatting.
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
rust/kernel/block/mq/gen_disk.rs | 7 +++----
rust/kernel/block/mq/operations.rs | 5 +----
rust/kernel/block/mq/request.rs | 2 +-
rust/kernel/block/mq/tag_set.rs | 22 ++++------------------
4 files changed, 9 insertions(+), 27 deletions(-)
diff --git a/rust/kernel/block/mq/gen_disk.rs b/rust/kernel/block/mq/gen_disk.rs
index 6f599f654f37f..a2c45bde75a7c 100644
--- a/rust/kernel/block/mq/gen_disk.rs
+++ b/rust/kernel/block/mq/gen_disk.rs
@@ -12,9 +12,8 @@
TagSet, //
},
error::{
- self,
from_err_ptr,
- Result, //
+ to_result, //
},
fmt::{
self,
@@ -67,7 +66,7 @@ pub fn rotational(mut self, rotational: bool) -> Self {
/// and that it is a power of two.
pub fn validate_block_size(size: u32) -> Result {
if !(512..=bindings::PAGE_SIZE as u32).contains(&size) || !size.is_power_of_two() {
- Err(error::code::EINVAL)
+ Err(EINVAL)
} else {
Ok(())
}
@@ -177,7 +176,7 @@ pub fn build<T: Operations>(
// operation, so we will not race.
unsafe { bindings::set_capacity(gendisk, self.capacity_sectors) };
- crate::error::to_result(
+ to_result(
// SAFETY: `gendisk` points to a valid and initialized instance of
// `struct gendisk`.
unsafe {
diff --git a/rust/kernel/block/mq/operations.rs b/rust/kernel/block/mq/operations.rs
index 187b0b7791db9..0343069b373c7 100644
--- a/rust/kernel/block/mq/operations.rs
+++ b/rust/kernel/block/mq/operations.rs
@@ -10,10 +10,7 @@
request::RequestDataWrapper,
Request, //
},
- error::{
- from_result,
- Result, //
- },
+ error::from_result,
prelude::*,
sync::{
aref::ARef,
diff --git a/rust/kernel/block/mq/request.rs b/rust/kernel/block/mq/request.rs
index 4e0579660e906..d10fb7627c870 100644
--- a/rust/kernel/block/mq/request.rs
+++ b/rust/kernel/block/mq/request.rs
@@ -7,7 +7,7 @@
use crate::{
bindings,
block::mq::Operations,
- error::Result,
+ prelude::*,
sync::{
aref::{
ARef,
diff --git a/rust/kernel/block/mq/tag_set.rs b/rust/kernel/block/mq/tag_set.rs
index c1fd3e047af50..df3f90bfbb817 100644
--- a/rust/kernel/block/mq/tag_set.rs
+++ b/rust/kernel/block/mq/tag_set.rs
@@ -4,8 +4,6 @@
//!
//! C header: [`include/linux/blk-mq.h`](srctree/include/linux/blk-mq.h)
-use core::pin::Pin;
-
use crate::{
bindings,
block::mq::{
@@ -13,22 +11,14 @@
request::RequestDataWrapper,
Operations, //
},
- error::{
- self,
- Result, //
- },
- prelude::try_pin_init,
+ error::to_result,
+ prelude::*,
types::Opaque, //
};
use core::{
convert::TryInto,
marker::PhantomData, //
};
-use pin_init::{
- pin_data,
- pinned_drop,
- PinInit, //
-};
/// A wrapper for the C `struct blk_mq_tag_set`.
///
@@ -47,11 +37,7 @@ pub struct TagSet<T: Operations> {
impl<T: Operations> TagSet<T> {
/// Try to create a new tag set
- pub fn new(
- nr_hw_queues: u32,
- num_tags: u32,
- num_maps: u32,
- ) -> impl PinInit<Self, error::Error> {
+ pub fn new(nr_hw_queues: u32, num_tags: u32, num_maps: u32) -> impl PinInit<Self, Error> {
let tag_set: bindings::blk_mq_tag_set = pin_init::zeroed();
let tag_set: Result<_> = core::mem::size_of::<RequestDataWrapper>()
.try_into()
@@ -77,7 +63,7 @@ pub fn new(
// SAFETY: we do not move out of `tag_set`.
let tag_set: &mut Opaque<_> = unsafe { Pin::get_unchecked_mut(tag_set) };
// SAFETY: `tag_set` is a reference to an initialized `blk_mq_tag_set`.
- error::to_result( unsafe { bindings::blk_mq_alloc_tag_set(tag_set.get())})
+ to_result( unsafe { bindings::blk_mq_alloc_tag_set(tag_set.get())})
}),
_p: PhantomData,
})
--
2.43.0
^ permalink raw reply related
* [PATCH v2 4/5] rust: block: mq: use vertical import style
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
In-Reply-To: <20260520-miscdev-use-format-v2-0-64dc48fc1345@linux.dev>
Convert `use` imports to vertical layout for better readability and
maintainability.
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
rust/kernel/block/mq/gen_disk.rs | 21 +++++++++++++++++----
rust/kernel/block/mq/operations.rs | 17 +++++++++++++----
rust/kernel/block/mq/request.rs | 12 +++++++++---
rust/kernel/block/mq/tag_set.rs | 24 +++++++++++++++++++-----
4 files changed, 58 insertions(+), 16 deletions(-)
diff --git a/rust/kernel/block/mq/gen_disk.rs b/rust/kernel/block/mq/gen_disk.rs
index 912cb805caf51..6f599f654f37f 100644
--- a/rust/kernel/block/mq/gen_disk.rs
+++ b/rust/kernel/block/mq/gen_disk.rs
@@ -7,14 +7,27 @@
use crate::{
bindings,
- block::mq::{Operations, TagSet},
- error::{self, from_err_ptr, Result},
- fmt::{self, Write},
+ block::mq::{
+ Operations,
+ TagSet, //
+ },
+ error::{
+ self,
+ from_err_ptr,
+ Result, //
+ },
+ fmt::{
+ self,
+ Write, //
+ },
prelude::*,
static_lock_class,
str::NullTerminatedFormatter,
sync::Arc,
- types::{ForeignOwnable, ScopeGuard},
+ types::{
+ ForeignOwnable,
+ ScopeGuard, //
+ },
};
/// A builder for [`GenDisk`].
diff --git a/rust/kernel/block/mq/operations.rs b/rust/kernel/block/mq/operations.rs
index 8ad46129a52c4..187b0b7791db9 100644
--- a/rust/kernel/block/mq/operations.rs
+++ b/rust/kernel/block/mq/operations.rs
@@ -6,11 +6,20 @@
use crate::{
bindings,
- block::mq::{request::RequestDataWrapper, Request},
- error::{from_result, Result},
+ block::mq::{
+ request::RequestDataWrapper,
+ Request, //
+ },
+ error::{
+ from_result,
+ Result, //
+ },
prelude::*,
- sync::{aref::ARef, Refcount},
- types::ForeignOwnable,
+ sync::{
+ aref::ARef,
+ Refcount, //
+ },
+ types::ForeignOwnable, //
};
use core::marker::PhantomData;
diff --git a/rust/kernel/block/mq/request.rs b/rust/kernel/block/mq/request.rs
index ce3e30c81cb5e..4e0579660e906 100644
--- a/rust/kernel/block/mq/request.rs
+++ b/rust/kernel/block/mq/request.rs
@@ -9,13 +9,19 @@
block::mq::Operations,
error::Result,
sync::{
- aref::{ARef, AlwaysRefCounted},
+ aref::{
+ ARef,
+ AlwaysRefCounted, //
+ },
atomic::Relaxed,
- Refcount,
+ Refcount, //
},
types::Opaque,
};
-use core::{marker::PhantomData, ptr::NonNull};
+use core::{
+ marker::PhantomData,
+ ptr::NonNull, //
+};
/// A wrapper around a blk-mq [`struct request`]. This represents an IO request.
///
diff --git a/rust/kernel/block/mq/tag_set.rs b/rust/kernel/block/mq/tag_set.rs
index dae9df408a862..c1fd3e047af50 100644
--- a/rust/kernel/block/mq/tag_set.rs
+++ b/rust/kernel/block/mq/tag_set.rs
@@ -8,13 +8,27 @@
use crate::{
bindings,
- block::mq::{operations::OperationsVTable, request::RequestDataWrapper, Operations},
- error::{self, Result},
+ block::mq::{
+ operations::OperationsVTable,
+ request::RequestDataWrapper,
+ Operations, //
+ },
+ error::{
+ self,
+ Result, //
+ },
prelude::try_pin_init,
- types::Opaque,
+ types::Opaque, //
+};
+use core::{
+ convert::TryInto,
+ marker::PhantomData, //
+};
+use pin_init::{
+ pin_data,
+ pinned_drop,
+ PinInit, //
};
-use core::{convert::TryInto, marker::PhantomData};
-use pin_init::{pin_data, pinned_drop, PinInit};
/// A wrapper for the C `struct blk_mq_tag_set`.
///
--
2.43.0
^ permalink raw reply related
* [PATCH v2 1/5] rust: miscdevice: use vertical import style
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
In-Reply-To: <20260520-miscdev-use-format-v2-0-64dc48fc1345@linux.dev>
Convert `use` imports to vertical layout for better readability and
maintainability.
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
rust/kernel/miscdevice.rs | 34 ++++++++++++++++++++++++++++------
1 file changed, 28 insertions(+), 6 deletions(-)
diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs
index c3c2052c92069..05a6b6b9770f2 100644
--- a/rust/kernel/miscdevice.rs
+++ b/rust/kernel/miscdevice.rs
@@ -11,16 +11,38 @@
use crate::{
bindings,
device::Device,
- error::{to_result, Error, Result, VTABLE_DEFAULT_ERROR},
- ffi::{c_int, c_long, c_uint, c_ulong},
- fs::{File, Kiocb},
- iov::{IovIterDest, IovIterSource},
+ error::{
+ to_result,
+ Error,
+ Result,
+ VTABLE_DEFAULT_ERROR, //
+ },
+ ffi::{
+ c_int,
+ c_long,
+ c_uint,
+ c_ulong, //
+ },
+ fs::{
+ File,
+ Kiocb, //
+ },
+ iov::{
+ IovIterDest,
+ IovIterSource, //
+ },
mm::virt::VmaNew,
prelude::*,
seq_file::SeqFile,
- types::{ForeignOwnable, Opaque},
+ types::{
+ ForeignOwnable,
+ Opaque, //
+ },
+};
+use core::{
+ marker::PhantomData,
+ pin::Pin, //
};
-use core::{marker::PhantomData, pin::Pin};
/// Options for creating a misc device.
#[derive(Copy, Clone)]
--
2.43.0
^ permalink raw reply related
* [PATCH v2 0/5] rust: use vertical import style and remove redundant imports
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
Adopt the vertical import style and drop redundant imports already
re-exported via `kernel::prelude`.
Changes include:
- Convert use statements to vertical import style in miscdevice,
samples, and block/mq
- Remove imports covered by prelude and apply minor formatting
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
Changes in v2:
- Reformat use statements and drop redundant imports in
rust/kernel/block/mq
- Drop redundant imports in rust/kernel/miscdevice
- Link to v1: https://lore.kernel.org/r/20260519-miscdev-use-format-v1-0-11d526ba0edc@linux.dev
---
Alvin Sun (5):
rust: miscdevice: use vertical import style
samples: rust_misc_device: use vertical import style
rust: miscdevice: remove redundant imports
rust: block: mq: use vertical import style
rust: block: mq: remove redundant imports and format
rust/kernel/block/mq/gen_disk.rs | 24 ++++++++++++++++++------
rust/kernel/block/mq/operations.rs | 14 ++++++++++----
rust/kernel/block/mq/request.rs | 14 ++++++++++----
rust/kernel/block/mq/tag_set.rs | 28 ++++++++++++++--------------
rust/kernel/miscdevice.rs | 23 +++++++++++++++++------
samples/rust/rust_misc_device.rs | 34 ++++++++++++++++++++++++++++------
6 files changed, 97 insertions(+), 40 deletions(-)
---
base-commit: 5200f5f493f79f14bbdc349e402a40dfb32f23c8
change-id: 20260519-miscdev-use-format-9ab7e83b1c11
Best regards,
--
Alvin Sun <alvin.sun@linux.dev>
^ permalink raw reply
* [PATCH v2 2/5] samples: rust_misc_device: use vertical import style
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
In-Reply-To: <20260520-miscdev-use-format-v2-0-64dc48fc1345@linux.dev>
Convert `use` imports to vertical layout for better readability and
maintainability.
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
samples/rust/rust_misc_device.rs | 34 ++++++++++++++++++++++++++++------
1 file changed, 28 insertions(+), 6 deletions(-)
diff --git a/samples/rust/rust_misc_device.rs b/samples/rust/rust_misc_device.rs
index 87a1fe63533ae..41e26c825060b 100644
--- a/samples/rust/rust_misc_device.rs
+++ b/samples/rust/rust_misc_device.rs
@@ -97,14 +97,36 @@
use kernel::{
device::Device,
- fs::{File, Kiocb},
- ioctl::{_IO, _IOC_SIZE, _IOR, _IOW},
- iov::{IovIterDest, IovIterSource},
- miscdevice::{MiscDevice, MiscDeviceOptions, MiscDeviceRegistration},
+ fs::{
+ File,
+ Kiocb, //
+ },
+ ioctl::{
+ _IO,
+ _IOC_SIZE,
+ _IOR,
+ _IOW, //
+ },
+ iov::{
+ IovIterDest,
+ IovIterSource, //
+ },
+ miscdevice::{
+ MiscDevice,
+ MiscDeviceOptions,
+ MiscDeviceRegistration, //
+ },
new_mutex,
prelude::*,
- sync::{aref::ARef, Mutex},
- uaccess::{UserSlice, UserSliceReader, UserSliceWriter},
+ sync::{
+ aref::ARef,
+ Mutex, //
+ },
+ uaccess::{
+ UserSlice,
+ UserSliceReader,
+ UserSliceWriter, //
+ },
};
const RUST_MISC_DEV_HELLO: u32 = _IO('|' as u32, 0x80);
--
2.43.0
^ permalink raw reply related
* [PATCH v2 3/5] rust: miscdevice: remove redundant imports
From: Alvin Sun @ 2026-05-20 2:40 UTC (permalink / raw)
To: Arnd Bergmann, Greg Kroah-Hartman, Miguel Ojeda, Boqun Feng,
Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Alice Ryhl, Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-block, Alvin Sun
In-Reply-To: <20260520-miscdev-use-format-v2-0-64dc48fc1345@linux.dev>
Drop `Error`, `Result`, `Pin`, `c_int`, `c_long`, `c_uint`, and
`c_ulong` imports already provided by `kernel::prelude`.
Signed-off-by: Alvin Sun <alvin.sun@linux.dev>
---
rust/kernel/miscdevice.rs | 13 +------------
1 file changed, 1 insertion(+), 12 deletions(-)
diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs
index 05a6b6b9770f2..83ce50def5ac9 100644
--- a/rust/kernel/miscdevice.rs
+++ b/rust/kernel/miscdevice.rs
@@ -13,16 +13,8 @@
device::Device,
error::{
to_result,
- Error,
- Result,
VTABLE_DEFAULT_ERROR, //
},
- ffi::{
- c_int,
- c_long,
- c_uint,
- c_ulong, //
- },
fs::{
File,
Kiocb, //
@@ -39,10 +31,7 @@
Opaque, //
},
};
-use core::{
- marker::PhantomData,
- pin::Pin, //
-};
+use core::marker::PhantomData;
/// Options for creating a misc device.
#[derive(Copy, Clone)]
--
2.43.0
^ permalink raw reply related
* Re: [PATCH V4 0/3] md/nvme: Enable PCI P2PDMA support for RAID0 and NVMe Multipath
From: Chaitanya Kulkarni @ 2026-05-20 0:11 UTC (permalink / raw)
To: axboe@kernel.dk
Cc: song@kernel.org, yukuai@fnnas.com, Chaitanya Kulkarni,
Christoph Hellwig, linan122@huawei.com, kbusch@kernel.org,
sagi@grimberg.me, linux-block@vger.kernel.org,
linux-raid@vger.kernel.org, linux-nvme@lists.infradead.org,
Kiran Modukuri
In-Reply-To: <20260515043535.GB3756@lst.de>
Jens,
On 5/14/26 9:35 PM, Christoph Hellwig wrote:
> Still looks good to me as per the reviews.
>
If there no objection, can we merge this ?
-Chaitanya
^ permalink raw reply
* Announcement: NVMe Live Migration Software Working Group
From: Chaitanya Kulkarni @ 2026-05-19 23:56 UTC (permalink / raw)
To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
linux-pci@vger.kernel.org, kvm@vger.kernel.org
Cc: hch, Keith Busch, Sagi Grimberg, Jens Axboe, bhelgaas@google.com,
Jason Gunthorpe, kevin.tian@intel.com, yi.l.liu@intel.com,
alex@awilliam.me
Hi,
I’m pleased to announce that the NVMe Board of Directors has approved
the formation of the NVMe Live Migration Software Working Group
(NVMe LM SW WG), see [1].
What?
The NVMe LM SW WG will focus on building an end-to-end Linux kernel
solution that follows the NVMe-approved specification and helps ensure
the implementation is aligned with the specification before it is
proposed upstream.
Why?
NVMe Live Migration is one of the more complex features we have
developed over the last four years. The NVMe Board has recognized the
need for a centralized effort around this work. Please bring any related
questions, use cases, or implementation topics to the NVMe LM SW WG.
How?
I am one of the co-chairs of the NVMe LM SW WG and will be coordinating
this effort within the TWG and with the upstream Linux kernel community.
Please reach out to me if your organization is interested in participating.
The prerequisites are that your organization must be a member of the
NVMe TWG and must sign an agreement to participate in the
NVMe LM SW WG. Linux Foundation membership is not required to
participate in this working group.
I look forward to working with you.
_This email is approved by the NVMe Board._
Best regards,
Chaitanya Kulkarni
Co-Chair, NVMe Live Migration Software Working Group
[1] Background
We started this work in August 2022, and it has evolved significantly
over time — from the original SDC 2022 research paper (thanks to
Dr. Stephen Bates) to the NVMe-approved specification.
I've also posted two RFCs to keep the ongoing development aligned with the
kernel community and received valuable feedback from kernel patch
reviews. Thanks to Christoph, Jason, Alex, and others; their input has
helped shape the technical specification.
^ permalink raw reply
* Re: [PATCH] drbd: remove unused drbd_nl_mcgrps[] array
From: Christoph Böhmwalder @ 2026-05-19 20:51 UTC (permalink / raw)
To: Arnd Bergmann, Philipp Reisner, Lars Ellenberg, Jens Axboe,
Jakub Kicinski
Cc: Arnd Bergmann, drbd-dev, linux-block, linux-kernel, drbd-dev
In-Reply-To: <20260519203057.1340528-1-arnd@kernel.org>
On 5/19/26 22:30, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
>
> After the rework, two files have a copy of drbd_nl_mcgrps[], but one
> of them has no references:
>
> drivers/block/drbd/drbd_nl_gen.c:641:42: error: 'drbd_nl_mcgrps' defined but not used [-Werror=unused-const-variable=]
> 641 | static const struct genl_multicast_group drbd_nl_mcgrps[] = {
> | ^~~~~~~~~~~~~~
>
> At the default warning level, -Wunused-const-variables is turned off,
> so this has gone unnoticed.
>
> Remove the extra variable.
>
> Fixes: 8098eeb693c4 ("drbd: replace genl_magic with explicit netlink serialization")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Oops, looks like a copy/paste error on my side.
Thanks for noticing and fixing.
Reviewed-by: Christoph Böhmwalder <christoph.boehmwalder@linbit.com>
^ permalink raw reply
* [PATCH] drbd: remove unused drbd_nl_mcgrps[] array
From: Arnd Bergmann @ 2026-05-19 20:30 UTC (permalink / raw)
To: Philipp Reisner, Lars Ellenberg, Christoph Böhmwalder,
Jens Axboe, Jakub Kicinski
Cc: Arnd Bergmann, drbd-dev, linux-block, linux-kernel
From: Arnd Bergmann <arnd@arndb.de>
After the rework, two files have a copy of drbd_nl_mcgrps[], but one
of them has no references:
drivers/block/drbd/drbd_nl_gen.c:641:42: error: 'drbd_nl_mcgrps' defined but not used [-Werror=unused-const-variable=]
641 | static const struct genl_multicast_group drbd_nl_mcgrps[] = {
| ^~~~~~~~~~~~~~
At the default warning level, -Wunused-const-variables is turned off,
so this has gone unnoticed.
Remove the extra variable.
Fixes: 8098eeb693c4 ("drbd: replace genl_magic with explicit netlink serialization")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
drivers/block/drbd/drbd_nl_gen.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl_gen.c b/drivers/block/drbd/drbd_nl_gen.c
index fb44b948cec8..e133e8415205 100644
--- a/drivers/block/drbd/drbd_nl_gen.c
+++ b/drivers/block/drbd/drbd_nl_gen.c
@@ -638,10 +638,6 @@ const struct genl_split_ops drbd_nl_ops[32] = {
},
};
-static const struct genl_multicast_group drbd_nl_mcgrps[] = {
- [DRBD_NLGRP_EVENTS] = { "events", },
-};
-
static int __drbd_cfg_context_from_attrs(struct drbd_cfg_context *s,
struct nlattr ***ret_nested_attribute_table,
struct genl_info *info)
--
2.39.5
^ permalink raw reply related
* Re: [PATCH] nvme-multipath: set BIO_REMAPPED on bios remapped to per-path namespace disks
From: Keith Busch @ 2026-05-19 19:10 UTC (permalink / raw)
To: hch@lst.de
Cc: Achkinazi, Igor, sagi@grimberg.me, axboe@kernel.dk,
linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, stable@vger.kernel.org
In-Reply-To: <20260519065303.GA7918@lst.de>
On Tue, May 19, 2026 at 08:53:03AM +0200, hch@lst.de wrote:
> On Mon, May 18, 2026 at 01:23:17PM -0600, Keith Busch wrote:
> >
> > Any reason nvme multipath can't call submit_bio_noacct_nocheck()
> > directly instead? If it's safe to skip the eod check here, then it
> > looks safe to skip everything else too.
>
> We really shouldn't expose that, and it doesn't quite to the
> right checks here.
>
> But I think the proper fix is to stop using bio_set_dev entirely.
> We do not want to associate the bio with a new blkg as those are
> basically not going to exist for the underlying devices. I'm also
> not sure we want to allow another round of BPF throttling either.
Simply dropping the bio_set_dev usage doesn't really fix the problem
here. Sure, bio_set_dev clears the BIO_REMAPPED flag, but it's not
guaranteed that flag was set in the first place.
I have an alternate proposal that's part of a larger series. This can
notify the driver of an early error so it can decide how to deal with
the bio:
https://lore.kernel.org/linux-block/20260519172326.3462354-6-kbusch@meta.com/
^ permalink raw reply
* Re: [PATCH v4] bio-integrity-fs: pass data iter to bio_integrity_verify()
From: Jens Axboe @ 2026-05-19 17:55 UTC (permalink / raw)
To: Caleb Sander Mateos
Cc: Anuj Gupta, Christoph Hellwig, linux-block, linux-kernel
In-Reply-To: <20260513182924.1753582-1-csander@purestorage.com>
On Wed, 13 May 2026 12:29:21 -0600, Caleb Sander Mateos wrote:
> bio_integrity_verify() expects the passed struct bvec_iter to be an
> iterator over bio data, not integrity. So construct a separate data
> bvec_iter without the bio_integrity_bytes() conversion and pass it to
> bio_integrity_verify() instead of bip_iter.
Applied, thanks!
[1/1] bio-integrity-fs: pass data iter to bio_integrity_verify()
commit: 431e40042d3599559e588b8946bb28bd440b4f65
Best regards,
--
Jens Axboe
^ permalink raw reply
* Re: [PATCH v4] bio-integrity-fs: pass data iter to bio_integrity_verify()
From: Caleb Sander Mateos @ 2026-05-19 17:30 UTC (permalink / raw)
To: Jens Axboe; +Cc: Anuj Gupta, Christoph Hellwig, linux-block, linux-kernel
In-Reply-To: <20260513182924.1753582-1-csander@purestorage.com>
Hi Jens,
Gentle ping on this fix for 7.1.
Thanks,
Caleb
On Wed, May 13, 2026 at 11:29 AM Caleb Sander Mateos
<csander@purestorage.com> wrote:
>
> bio_integrity_verify() expects the passed struct bvec_iter to be an
> iterator over bio data, not integrity. So construct a separate data
> bvec_iter without the bio_integrity_bytes() conversion and pass it to
> bio_integrity_verify() instead of bip_iter.
>
> Fixes: 0bde8a12b554 ("block: add fs_bio_integrity helpers")
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> ---
> v4: split from series changing ref tag seed units
> v3: https://lore.kernel.org/linux-block/20260417015732.2692434-3-csander@purestorage.com/
>
> block/bio-integrity-fs.c | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/block/bio-integrity-fs.c b/block/bio-integrity-fs.c
> index acb1e5f270d2..0daa42d9ead7 100644
> --- a/block/bio-integrity-fs.c
> +++ b/block/bio-integrity-fs.c
> @@ -53,21 +53,25 @@ EXPORT_SYMBOL_GPL(fs_bio_integrity_generate);
>
> int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size)
> {
> struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk);
> struct bio_integrity_payload *bip = bio_integrity(bio);
> + struct bvec_iter data_iter = {
> + .bi_sector = sector,
> + .bi_size = size,
> + };
>
> /*
> * Reinitialize bip->bip_iter.
> *
> * This is for use in the submitter after the driver is done with the
> * bio. Requires the submitter to remember the sector and the size.
> */
> memset(&bip->bip_iter, 0, sizeof(bip->bip_iter));
> bip->bip_iter.bi_sector = sector;
> bip->bip_iter.bi_size = bio_integrity_bytes(bi, size >> SECTOR_SHIFT);
> - return blk_status_to_errno(bio_integrity_verify(bio, &bip->bip_iter));
> + return blk_status_to_errno(bio_integrity_verify(bio, &data_iter));
> }
>
> static int __init fs_bio_integrity_init(void)
> {
> fs_bio_integrity_cache = kmem_cache_create("fs_bio_integrity",
> --
> 2.54.0
>
^ permalink raw reply
* [PATCH RFC 2/5] block: fix invalid zone append status
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
In-Reply-To: <20260519172326.3462354-1-kbusch@meta.com>
From: Keith Busch <kbusch@kernel.org>
A bio submitted to a zone block device that breaks the limits is invalid
rather than an IO error. Fix up the return code to report it that way,
as the previously used IOERR is considered a path failure.
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
block/blk-core.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 22af5dec112b5..92a802dc8042c 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -604,7 +604,7 @@ static inline blk_status_t blk_check_zone_append(struct request_queue *q,
/* The bio sector must point to the start of a sequential zone */
if (!bdev_is_zone_start(bio->bi_bdev, bio->bi_iter.bi_sector))
- return BLK_STS_IOERR;
+ return BLK_STS_INVAL;
/*
* Not allowed to cross zone boundaries. Otherwise, the BIO will be
@@ -612,11 +612,11 @@ static inline blk_status_t blk_check_zone_append(struct request_queue *q,
* different zones.
*/
if (nr_sectors > q->limits.chunk_sectors)
- return BLK_STS_IOERR;
+ return BLK_STS_INVAL;
/* Make sure the BIO is small enough and will not get split */
if (nr_sectors > q->limits.max_zone_append_sectors)
- return BLK_STS_IOERR;
+ return BLK_STS_INVAL;
bio->bi_opf |= REQ_NOMERGE;
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH RFC 0/5] block: validate bios against queue limits in the entered context
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
From: Keith Busch <kbusch@kernel.org>
The block layer validates bios against various queue limits and device
capacity in submit_bio_noacct(), but these checks run before
bio_queue_enter(). This means they are not serialized against drivers
that update queue limits inside a freeze window. A bio that passes
validation under old limits can enter the queue after the update and
reach the driver with an invalid configuration.
This series moves all limit-dependent validation into
__bio_split_to_limits(), which runs after the queue usage reference has
been acquired. This ensures proper serialization against limit updates.
This changes was motivated by a few recent reports:
https://lore.kernel.org/linux-nvme/MW5PR19MB548483D1FAE4F322E4C97352FD032@MW5PR19MB5484.namprd19.prod.outlook.com/
https://lore.kernel.org/linux-nvme/20260517053635.2282446-1-coshi036@gmail.com/
When an NVMe namespace that is reformatted to use extended metadata that
can't be controller generated/stripped, the driver sets capacity to 0
inside a freeze window because the block layer is not able to form a
viable request for this format. But a bio that passed bio_check_eod()
before the freeze can still reach nvme_setup_rw() after the update,
triggering a WARN that we didn't expect to be possible of reaching.
For NVMe multipath, moving these checks into the entered context
exacerbates a different problem: a bio targeting a path being torn down
(capacity set to 0) would be failed by the block layer before the driver
gets a chance to redirect it to another path. This is a pre-existing
problem, but the initial changes in this series make it easier to hit.
The series addresses this by introducing a new callback to
block_device_operations, called from bio_io_error(), that lets the
driver intercept and redirect failing bios before they are completed.
NVMe multipath uses this to requeue bios back to the head device for
path re-selection when the path is no longer ready.
Note that callers of submit_bio_noacct_nocheck() (bio split, throttle
dispatch) will now hit the validation checks in __bio_split_to_limits()
that they previously bypassed. This is intentional: these checks must
run in the entered context to be properly serialized, and cannot be
skipped so it is a performance cost that can't be avoided.
Keith Busch (5):
blk-mq: fix status for unaligned bio
block: fix invalid zone append status codes
block: validate bio bounds in the queue entered context
block: move bio operation validation into __bio_split_to_limits
block, nvme: add failed_bio callback for multipath bio failover
block/blk-core.c | 144 ----------------------------------
block/blk-mq.c | 3 +-
block/blk.h | 89 ++++++++++++++++++++-
drivers/nvme/host/core.c | 1 +
drivers/nvme/host/multipath.c | 26 ++++++
drivers/nvme/host/nvme.h | 2 +
include/linux/bio.h | 6 --
include/linux/blkdev.h | 16 ++++
8 files changed, 133 insertions(+), 154 deletions(-)
--
2.53.0-Meta
^ permalink raw reply
* [PATCH RFC 3/5] block: validate bio bounds in the queue entered context
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
In-Reply-To: <20260519172326.3462354-1-kbusch@meta.com>
From: Keith Busch <kbusch@kernel.org>
bio_check_eod() in submit_bio_noacct() validates that a bio does not
extend beyond the partition's available sectors. This check runs before
bio_queue_enter(), so it is not serialized against queue limit updates.
A driver that freezes the queue, updates limits, changes the capacity,
and unfreezes can race with a bio that passed the early check under the
old capacity.
Remove bio_check_eod() and replace it with a bounds check in
__bio_split_to_limits(), which runs after the queue usage reference has
been acquired. The check uses partition-aware arithmetic to validate
both partition bounds and disk capacity in a single comparison that
works correctly on the post-remap sector values.
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
block/blk-core.c | 26 --------------------------
block/blk.h | 14 ++++++++++++++
2 files changed, 14 insertions(+), 26 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 92a802dc8042c..c200d0fc44fe7 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -547,30 +547,6 @@ int should_fail_bio(struct bio *bio)
}
ALLOW_ERROR_INJECTION(should_fail_bio, ERRNO);
-/*
- * Check whether this bio extends beyond the end of the device or partition.
- * This may well happen - the kernel calls bread() without checking the size of
- * the device, e.g., when mounting a file system.
- */
-static inline int bio_check_eod(struct bio *bio)
-{
- sector_t maxsector = bdev_nr_sectors(bio->bi_bdev);
- unsigned int nr_sectors = bio_sectors(bio);
-
- if (nr_sectors &&
- (nr_sectors > maxsector ||
- bio->bi_iter.bi_sector > maxsector - nr_sectors)) {
- if (!maxsector)
- return -EIO;
- pr_info_ratelimited("%s: attempt to access beyond end of device\n"
- "%pg: rw=%d, sector=%llu, nr_sectors = %u limit=%llu\n",
- current->comm, bio->bi_bdev, bio->bi_opf,
- bio->bi_iter.bi_sector, nr_sectors, maxsector);
- return -EIO;
- }
- return 0;
-}
-
/*
* Remap block n of partition p to block n+start(p) of the disk.
*/
@@ -802,8 +778,6 @@ void submit_bio_noacct(struct bio *bio)
goto end_io;
bio_check_ro(bio);
if (!bio_flagged(bio, BIO_REMAPPED)) {
- if (unlikely(bio_check_eod(bio)))
- goto end_io;
if (bdev_is_partition(bdev) &&
unlikely(blk_partition_remap(bio)))
goto end_io;
diff --git a/block/blk.h b/block/blk.h
index bf1a80493ff1c..e70acb2d358e3 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -423,6 +423,17 @@ static inline bool bio_may_need_split(struct bio *bio,
static inline struct bio *__bio_split_to_limits(struct bio *bio,
const struct queue_limits *lim, unsigned int *nr_segs)
{
+ if (unlikely(bio_end_sector(bio) > bdev_nr_sectors(bio->bi_bdev) +
+ bio->bi_bdev->bd_start_sect)) {
+ pr_info_ratelimited("%s: attempt to access beyond end of device\n"
+ "%pg: rw=%d, sector=%llu, nr_sectors = %u limit=%llu\n",
+ current->comm, bio->bi_bdev, bio->bi_opf,
+ bio->bi_iter.bi_sector, bio_sectors(bio),
+ bdev_nr_sectors(bio->bi_bdev) +
+ bio->bi_bdev->bd_start_sect);
+ goto ioerr;
+ }
+
switch (bio_op(bio)) {
case REQ_OP_READ:
case REQ_OP_WRITE:
@@ -442,6 +453,9 @@ static inline struct bio *__bio_split_to_limits(struct bio *bio,
*nr_segs = 0;
return bio;
}
+ioerr:
+ bio_io_error(bio);
+ return NULL;
}
/**
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH RFC 5/5] block, nvme: add failed_bio callback for multipath bio failover
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
In-Reply-To: <20260519172326.3462354-1-kbusch@meta.com>
From: Keith Busch <kbusch@kernel.org>
The nvme driver has long utilized a zero capacity to indicate the path
isn't reachable, which creates a race condition with IO dispatch when
paths are being detached on a live system: when the block layer rejects
a bio early due to a capacity check failure, drivers with multipath
support using the original bio have no interception point to redirect
the bio to another path.
We don't want to have to clone the bio just for this condition, so add a
failed_bio callback to block_device_operations, called from
bio_io_error. If the callback returns true, the driver has taken
ownership of the bio and the error completion is skipped.
Implement the callback for NVMe multipath. nvme_failed_bio redirects
failing bios back to the multipath head device's requeue list for
path re-selection, but only when all three conditions are met:
- The bio came through the multipath head (REQ_NVME_MPATH)
- The error is a path-related error (blk_path_error)
- The path is no longer ready (!NVME_NS_READY)
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
drivers/nvme/host/core.c | 1 +
drivers/nvme/host/multipath.c | 26 ++++++++++++++++++++++++++
drivers/nvme/host/nvme.h | 2 ++
include/linux/bio.h | 6 ------
include/linux/blkdev.h | 16 ++++++++++++++++
5 files changed, 45 insertions(+), 6 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c3032d6ad6b1e..ac33b4dd19127 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2650,6 +2650,7 @@ const struct block_device_operations nvme_bdev_ops = {
.get_unique_id = nvme_get_unique_id,
.report_zones = nvme_report_zones,
.pr_ops = &nvme_pr_ops,
+ .failed_bio = nvme_failed_bio,
};
static int nvme_wait_ready(struct nvme_ctrl *ctrl, u32 mask, u32 val,
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 263161cb8ac06..250d0719d32cf 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -134,6 +134,32 @@ void nvme_mpath_start_freeze(struct nvme_subsystem *subsys)
blk_freeze_queue_start(h->disk->queue);
}
+bool nvme_failed_bio(struct bio *bio)
+{
+ unsigned long flags;
+ struct nvme_ns *ns;
+
+ if (!(bio->bi_opf & REQ_NVME_MPATH))
+ return false;
+ if (!blk_path_error(bio->bi_status))
+ return false;
+
+ ns = bio->bi_bdev->bd_disk->queue->queuedata;
+ if (test_bit(NVME_NS_READY, &ns->flags))
+ return false;
+ nvme_mpath_clear_current_path(ns);
+
+ bio->bi_status = BLK_STS_OK;
+ bio_set_dev(bio, ns->head->disk->part0);
+
+ spin_lock_irqsave(&ns->head->requeue_lock, flags);
+ bio_list_add(&ns->head->requeue_list, bio);
+ spin_unlock_irqrestore(&ns->head->requeue_lock, flags);
+
+ kblockd_schedule_work(&ns->head->requeue_work);
+ return true;
+}
+
void nvme_failover_req(struct request *req)
{
struct nvme_ns *ns = req->q->queuedata;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index ccd5e05dac98f..37d4f037b9a8a 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -1028,6 +1028,7 @@ void nvme_mpath_unfreeze(struct nvme_subsystem *subsys);
void nvme_mpath_wait_freeze(struct nvme_subsystem *subsys);
void nvme_mpath_start_freeze(struct nvme_subsystem *subsys);
void nvme_mpath_default_iopolicy(struct nvme_subsystem *subsys);
+bool nvme_failed_bio(struct bio *bio);
void nvme_failover_req(struct request *req);
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl,struct nvme_ns_head *head);
@@ -1079,6 +1080,7 @@ static inline bool nvme_ctrl_use_ana(struct nvme_ctrl *ctrl)
{
return false;
}
+#define nvme_failed_bio NULL
static inline void nvme_failover_req(struct request *req)
{
}
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 1a83a6753d70d..4f01033c32ced 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -371,12 +371,6 @@ void submit_bio(struct bio *bio);
extern void bio_endio(struct bio *);
-static inline void bio_io_error(struct bio *bio)
-{
- bio->bi_status = BLK_STS_IOERR;
- bio_endio(bio);
-}
-
static inline void bio_wouldblock_error(struct bio *bio)
{
bio_set_flag(bio, BIO_QUIET);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 17270a28c66d5..75cbf496e0efa 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1686,8 +1686,24 @@ struct block_device_operations {
* driver.
*/
int (*alternative_gpt_sector)(struct gendisk *disk, sector_t *sector);
+ bool (*failed_bio)(struct bio *bio);
};
+static inline void bio_io_error(struct bio *bio)
+{
+ bio->bi_status = BLK_STS_IOERR;
+
+ if (bio->bi_bdev) {
+ const struct block_device_operations *ops =
+ bio->bi_bdev->bd_disk->fops;
+
+ if (ops->failed_bio && ops->failed_bio(bio))
+ return;
+ }
+
+ bio_endio(bio);
+}
+
#ifdef CONFIG_COMPAT
extern int blkdev_compat_ptr_ioctl(struct block_device *, blk_mode_t,
unsigned int, unsigned long);
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH RFC 1/5] blk-mq: fix status for unaligned bio
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
In-Reply-To: <20260519172326.3462354-1-kbusch@meta.com>
From: Keith Busch <kbusch@kernel.org>
A bio requesting sectors unaligned to the logical format is invalid
rather than an IO error. Fix up the return code because there are some
device mappers that care about distinguishing these kinds of errors.
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
block/blk-mq.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index d0c37daf568f2..881183345a85c 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3182,7 +3182,8 @@ void blk_mq_submit_bio(struct bio *bio)
* have to be done with queue usage counter held.
*/
if (unlikely(bio_unaligned(bio, q))) {
- bio_io_error(bio);
+ bio->bi_status = BLK_STS_INVAL;
+ bio_endio(bio);
goto queue_exit;
}
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH RFC 4/5] block: move bio validation into __bio_split_to_limits
From: Keith Busch @ 2026-05-19 17:23 UTC (permalink / raw)
To: linux-block, linux-nvme
Cc: axboe, hch, tom.leiming, coshi036, Igor.Achkinazi, dlemoal,
Keith Busch
In-Reply-To: <20260519172326.3462354-1-kbusch@meta.com>
From: Keith Busch <kbusch@kernel.org>
The bio checks in submit_bio_noacct() compares queue limits to determine
whether operations like discard, write zeroes, zone append, and atomic
writes are supported and valid. These checks run before
bio_queue_enter(), so they race against any driver that updates queue
limits inside a freeze window.
Move all limit-dependent operation validation from submit_bio_noacct()
into __bio_split_to_limits(), which runs after the queue usage reference
has been acquired. This ensures that all checks are properly serialized
against limit updates.
The non-limit checks (crypto, fault injection, partition remap, and
flush flag handling) remain in submit_bio_noacct() as they do not
depend on queue limits.
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
block/blk-core.c | 118 -----------------------------------------------
block/blk.h | 75 ++++++++++++++++++++++++++++--
2 files changed, 72 insertions(+), 121 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index c200d0fc44fe7..8360c2b5efee5 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -519,25 +519,6 @@ static int __init fail_make_request_debugfs(void)
late_initcall(fail_make_request_debugfs);
#endif /* CONFIG_FAIL_MAKE_REQUEST */
-static inline void bio_check_ro(struct bio *bio)
-{
- if (op_is_write(bio_op(bio)) && bdev_read_only(bio->bi_bdev)) {
- if (op_is_flush(bio->bi_opf) && !bio_sectors(bio))
- return;
-
- if (bdev_test_flag(bio->bi_bdev, BD_RO_WARNED))
- return;
-
- bdev_set_flag(bio->bi_bdev, BD_RO_WARNED);
-
- /*
- * Use ioctl to set underlying disk of raid/dm to read-only
- * will trigger this.
- */
- pr_warn("Trying to write to read-only block-device %pg\n",
- bio->bi_bdev);
- }
-}
int should_fail_bio(struct bio *bio)
{
@@ -566,39 +547,6 @@ static int blk_partition_remap(struct bio *bio)
return 0;
}
-/*
- * Check write append to a zoned block device.
- */
-static inline blk_status_t blk_check_zone_append(struct request_queue *q,
- struct bio *bio)
-{
- int nr_sectors = bio_sectors(bio);
-
- /* Only applicable to zoned block devices */
- if (!bdev_is_zoned(bio->bi_bdev))
- return BLK_STS_NOTSUPP;
-
- /* The bio sector must point to the start of a sequential zone */
- if (!bdev_is_zone_start(bio->bi_bdev, bio->bi_iter.bi_sector))
- return BLK_STS_INVAL;
-
- /*
- * Not allowed to cross zone boundaries. Otherwise, the BIO will be
- * split and could result in non-contiguous sectors being written in
- * different zones.
- */
- if (nr_sectors > q->limits.chunk_sectors)
- return BLK_STS_INVAL;
-
- /* Make sure the BIO is small enough and will not get split */
- if (nr_sectors > q->limits.max_zone_append_sectors)
- return BLK_STS_INVAL;
-
- bio->bi_opf |= REQ_NOMERGE;
-
- return BLK_STS_OK;
-}
-
static void __submit_bio(struct bio *bio)
{
/* If plug is not used, add new plug here to cache nsecs time. */
@@ -731,18 +679,6 @@ void submit_bio_noacct_nocheck(struct bio *bio, bool split)
}
}
-static blk_status_t blk_validate_atomic_write_op_size(struct request_queue *q,
- struct bio *bio)
-{
- if (bio->bi_iter.bi_size > queue_atomic_write_unit_max_bytes(q))
- return BLK_STS_INVAL;
-
- if (bio->bi_iter.bi_size % queue_atomic_write_unit_min_bytes(q))
- return BLK_STS_INVAL;
-
- return BLK_STS_OK;
-}
-
/**
* submit_bio_noacct - re-submit a bio to the block device layer for I/O
* @bio: The bio describing the location in memory and on the device.
@@ -755,7 +691,6 @@ static blk_status_t blk_validate_atomic_write_op_size(struct request_queue *q,
void submit_bio_noacct(struct bio *bio)
{
struct block_device *bdev = bio->bi_bdev;
- struct request_queue *q = bdev_get_queue(bdev);
blk_status_t status = BLK_STS_IOERR;
might_sleep();
@@ -776,7 +711,6 @@ void submit_bio_noacct(struct bio *bio)
if (should_fail_bio(bio))
goto end_io;
- bio_check_ro(bio);
if (!bio_flagged(bio, BIO_REMAPPED)) {
if (bdev_is_partition(bdev) &&
unlikely(blk_partition_remap(bio)))
@@ -800,58 +734,6 @@ void submit_bio_noacct(struct bio *bio)
}
}
- switch (bio_op(bio)) {
- case REQ_OP_READ:
- break;
- case REQ_OP_WRITE:
- if (bio->bi_opf & REQ_ATOMIC) {
- status = blk_validate_atomic_write_op_size(q, bio);
- if (status != BLK_STS_OK)
- goto end_io;
- }
- break;
- case REQ_OP_FLUSH:
- /*
- * REQ_OP_FLUSH can't be submitted through bios, it is only
- * synthetized in struct request by the flush state machine.
- */
- goto not_supported;
- case REQ_OP_DISCARD:
- if (!bdev_max_discard_sectors(bdev))
- goto not_supported;
- break;
- case REQ_OP_SECURE_ERASE:
- if (!bdev_max_secure_erase_sectors(bdev))
- goto not_supported;
- break;
- case REQ_OP_ZONE_APPEND:
- status = blk_check_zone_append(q, bio);
- if (status != BLK_STS_OK)
- goto end_io;
- break;
- case REQ_OP_WRITE_ZEROES:
- if (!q->limits.max_write_zeroes_sectors)
- goto not_supported;
- break;
- case REQ_OP_ZONE_RESET:
- case REQ_OP_ZONE_OPEN:
- case REQ_OP_ZONE_CLOSE:
- case REQ_OP_ZONE_FINISH:
- case REQ_OP_ZONE_RESET_ALL:
- if (!bdev_is_zoned(bio->bi_bdev))
- goto not_supported;
- break;
- case REQ_OP_DRV_IN:
- case REQ_OP_DRV_OUT:
- /*
- * Driver private operations are only used with passthrough
- * requests.
- */
- fallthrough;
- default:
- goto not_supported;
- }
-
if (blk_throtl_bio(bio))
return;
submit_bio_noacct_nocheck(bio, false);
diff --git a/block/blk.h b/block/blk.h
index e70acb2d358e3..d3b897e9b5ee9 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -407,6 +407,22 @@ static inline bool bio_may_need_split(struct bio *bio,
return bv->bv_len + bv->bv_offset > lim->max_fast_segment_size;
}
+static inline void bio_check_ro(struct bio *bio)
+{
+ if (op_is_write(bio_op(bio)) && bdev_read_only(bio->bi_bdev)) {
+ if (op_is_flush(bio->bi_opf) && !bio_sectors(bio))
+ return;
+
+ if (bdev_test_flag(bio->bi_bdev, BD_RO_WARNED))
+ return;
+
+ bdev_set_flag(bio->bi_bdev, BD_RO_WARNED);
+
+ pr_warn("Trying to write to read-only block-device %pg\n",
+ bio->bi_bdev);
+ }
+}
+
/**
* __bio_split_to_limits - split a bio to fit the queue limits
* @bio: bio to be split
@@ -423,6 +439,8 @@ static inline bool bio_may_need_split(struct bio *bio,
static inline struct bio *__bio_split_to_limits(struct bio *bio,
const struct queue_limits *lim, unsigned int *nr_segs)
{
+ bio_check_ro(bio);
+
if (unlikely(bio_end_sector(bio) > bdev_nr_sectors(bio->bi_bdev) +
bio->bi_bdev->bd_start_sect)) {
pr_info_ratelimited("%s: attempt to access beyond end of device\n"
@@ -435,24 +453,75 @@ static inline struct bio *__bio_split_to_limits(struct bio *bio,
}
switch (bio_op(bio)) {
- case REQ_OP_READ:
case REQ_OP_WRITE:
+ if (bio->bi_opf & REQ_ATOMIC) {
+ if (bio->bi_iter.bi_size > lim->atomic_write_unit_max ||
+ bio->bi_iter.bi_size % lim->atomic_write_unit_min)
+ goto invalid;
+ }
+ fallthrough;
+ case REQ_OP_READ:
if (bio_may_need_split(bio, lim))
return bio_split_rw(bio, lim, nr_segs);
*nr_segs = 1;
return bio;
case REQ_OP_ZONE_APPEND:
+ /* Only applicable to zoned block devices */
+ if (!(lim->features & BLK_FEAT_ZONED))
+ goto not_supported;
+
+ /* The bio sector must point to the start of a sequential zone */
+ if (!bdev_is_zone_start(bio->bi_bdev, bio->bi_iter.bi_sector))
+ goto invalid;
+
+ /*
+ * Not allowed to cross zone boundaries. Otherwise, the BIO
+ * will be split and could result in non-contiguous sectors
+ * being written in different zones.
+ */
+ if (bio_sectors(bio) > lim->chunk_sectors)
+ goto invalid;
+
+ /* Make sure the BIO is small enough and will not get split */
+ if (bio_sectors(bio) > lim->max_zone_append_sectors)
+ goto invalid;
+
+ bio->bi_opf |= REQ_NOMERGE;
return bio_split_zone_append(bio, lim, nr_segs);
case REQ_OP_DISCARD:
+ if (!lim->max_discard_sectors)
+ goto not_supported;
+ return bio_split_discard(bio, lim, nr_segs);
case REQ_OP_SECURE_ERASE:
+ if (!lim->max_secure_erase_sectors)
+ goto not_supported;
return bio_split_discard(bio, lim, nr_segs);
case REQ_OP_WRITE_ZEROES:
+ if (!lim->max_write_zeroes_sectors)
+ goto not_supported;
return bio_split_write_zeroes(bio, lim, nr_segs);
- default:
- /* other operations can't be split */
+ case REQ_OP_ZONE_RESET:
+ case REQ_OP_ZONE_OPEN:
+ case REQ_OP_ZONE_CLOSE:
+ case REQ_OP_ZONE_FINISH:
+ case REQ_OP_ZONE_RESET_ALL:
+ if (!(lim->features & BLK_FEAT_ZONED))
+ goto not_supported;
*nr_segs = 0;
return bio;
+ default:
+ WARN_ON_ONCE(1);
+ goto not_supported;
}
+
+invalid:
+ bio->bi_status = BLK_STS_INVAL;
+ bio_endio(bio);
+ return NULL;
+not_supported:
+ bio->bi_status = BLK_STS_NOTSUPP;
+ bio_endio(bio);
+ return NULL;
ioerr:
bio_io_error(bio);
return NULL;
--
2.53.0-Meta
^ permalink raw reply related
* Re: [RFC PATCH v2 0/4] mm/zsmalloc: reduce zs_free() latency on swap release path
From: Nhat Pham @ 2026-05-19 15:22 UTC (permalink / raw)
To: Wenchao Hao
Cc: Yosry Ahmed, Andrew Morton, Chengming Zhou, Jens Axboe,
Johannes Weiner, Minchan Kim, Sergey Senozhatsky, linux-block,
linux-kernel, linux-mm, Barry Song, Xueyuan Chen, Wenchao Hao
In-Reply-To: <CAOptpSMdFwQUxT7bP7s1GWyXvzUQrg3ROtGp3w_zYy2rSkDYRQ@mail.gmail.com>
On Sat, May 9, 2026 at 2:08 AM Wenchao Hao <haowenchao22@gmail.com> wrote:
>
> On Sat, May 9, 2026 at 7:33 AM Nhat Pham <nphamcs@gmail.com> wrote:
> >
> > On Wed, May 6, 2026 at 6:55 AM Wenchao Hao <haowenchao22@gmail.com> wrote:
> > >
> > > On Sat, May 2, 2026 at 3:21 PM Nhat Pham <nphamcs@gmail.com> wrote:
> > > >
> > > >
> > > > Oh man, you are eliminating pool lock here right? This would help my
> > > > other patch series a lot too :)
> > > >
> > > > https://lore.kernel.org/all/CAKEwX=M5YpR0cQrryX_y4pm_BuxyUWZ_8MbhWodwbf1Fe=gzew@mail.gmail.com/
> > > > https://lore.kernel.org/all/CAKEwX=PkFiP+u+ThrzjTKBi+usQf2uuhTZcfB2BNNA8RboOFDQ@mail.gmail.com/
> > > >
> > >
> > > Yes, exactly. With class_idx encoded in the obj value,
> > > zs_free() can determine the correct size_class without
> > > any pool-level lock. The lockless read gives a valid
> > > class_idx because it's invariant across migration (only
> > > PFN changes), and we re-read obj under class->lock to
> > > get a stable PFN.
> > >
> > > >
> > > > /*
> > > > * The pool->lock protects the race with zpage's migration
> > > > * so it's safe to get the page from handle.
> > > > */
> > > > read_lock(&pool->lock);
> > > > obj = handle_to_obj(handle);
> > > > obj_to_zpdesc(obj, &f_zpdesc);
> > > > zspage = get_zspage(f_zpdesc);
> > > > class = zspage_class(pool, zspage);
> > > > spin_lock(&class->lock);
> > > > read_unlock(&pool->lock);
> > > >
> > > > It's basically just this blob right?
> > > >
> > >
> > > Yes, that's the blob being replaced. On the
> > > ZS_OBJ_CLASS_IDX path (64-bit systems), it becomes:
> > >
> > > obj = handle_to_obj(handle);
> > > class = pool->size_class[obj_to_class_idx(obj)];
> > > spin_lock(&class->lock);
> > > obj = handle_to_obj(handle); /* re-read for stable PFN */
> > >
> > > No pool->lock at all. We've also added compile-time
> > > gating (#if BITS_PER_LONG >= 64) since 32-bit systems
> > > lack the spare bits in OBJ_INDEX to fit class_idx. On
> > > 32-bit, it falls back to the original pool->lock path.
> > >
> >
> > BTW, I've tested your idea with a hacky prototype, when I was playing
> > with my vswap series. It absolutely improves free time in the usemem
> > benchmark :) Idea is very promising - I won't scoop your work of
> > course, just letting you know that at least in my use case, it works
> > :) Look forward to seeing it submitted soon!!!
>
> Thanks, Nhat, that's great to hear.
>
> I've split this part out and posted it as its own series:
>
> https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com
>
> Review there would be very welcome.
Huh I think I might have been unsubscribed from linux-mm again -.-
Weird - I wonder if this is because of Gmail shenanigans.
Can you cc me the thread next time just in case?
>
> Also, could you share the details of your usemem setup? I'd like
> to reproduce it locally on the same baseline.
Sure! I left some notes here:
https://lore.kernel.org/all/20260505153854.1612033-1-nphamcs@gmail.com/
But for your convenience, this is the benchmark I ran:
2. Usemem single-threaded: anonymous memory allocation (56GB) on a host
with 32GB RAM, 16 rounds.
I don't put a limit on the cgroup, relying on global pressure (per
Kairui's instructions).
I'm not on my work server right now so I don't have the exact command,
but hopefully that should be enough to show the wins with your patch
series! I wanted to run it for your patch series myself but I do not
have the cycles right now, unfortunately :(
>
> Thanks,
> Wenchao
^ permalink raw reply
* Re: 10x I/O await times in 6.12
From: Matt Fleming @ 2026-05-19 13:31 UTC (permalink / raw)
To: Jens Axboe; +Cc: Yu Kuai, linux-block, linux-kernel, kernel-team, yukuai (C)
In-Reply-To: <aepfV6Gp2uKbHi2N@matt-Precision-5490>
On Thu, Apr 23, 2026 at 07:18:39PM +0100, Matt Fleming wrote:
> On Mon, Apr 21, 2025 at 08:21:13PM +0800, Yu Kuai wrote:
> > If nsecs - plug->cur_ktime is still milliseconds, can you check if
> > the following patch can fix your problem?
> >
> > Thanks,
> > Kuai
> >
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index ae8494d88897..37197502147e 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -1095,7 +1095,9 @@ static inline void blk_account_io_start(struct request
> > *req)
> > return;
> >
> > req->rq_flags |= RQF_IO_STAT;
> > - req->start_time_ns = blk_time_get_ns();
> > +
> > + if (!current->plug)
> > + req->start_time_ns = blk_time_get_ns();
> >
> > /*
> > * All non-passthrough requests are created from a bio with one
> > @@ -2874,6 +2876,7 @@ void blk_mq_flush_plug_list(struct blk_plug *plug,
> > bool from_schedule)
> > {
> > struct request *rq;
> > unsigned int depth;
> > + u64 now;
> >
> > /*
> > * We may have been called recursively midway through handling
> > @@ -2887,6 +2890,10 @@ void blk_mq_flush_plug_list(struct blk_plug *plug,
> > bool from_schedule)
> > depth = plug->rq_count;
> > plug->rq_count = 0;
> >
> > + now = ktime_get_ns();
> > + rq_list_for_each(&plug->mq_list, rq)
> > + rq->start_time_ns = now;
> > +
> > if (!plug->multiple_queues && !plug->has_elevator && !from_schedule)
> > {
> > struct request_queue *q;
> >
>
> It's April again, and this time we're moving to 6.18.
>
> The I/O accounting bug reported at the start of this thread is still an
> issue for us, and we've been carrying the above patch for 12 months.
Hey Jens, any feedback on this patch?
^ permalink raw reply
* Re: [PATCH] block/loop: Fix NULL pointer dereference in lo_rw_aio()
From: Tetsuo Handa @ 2026-05-19 12:37 UTC (permalink / raw)
To: Ming Lei, Hongling Zeng
Cc: Hongling Zeng, axboe, ming.lei, linux-block, linux-kernel
In-Reply-To: <agxJdUf1b0JSDAux@fedora>
On 2026/05/19 20:28, Ming Lei wrote:
>> This means |__loop_clr_fd()|runs while I/O is still active.
>> Regression introduced by:
>> 6050fa4c84cc ("loop: don't hold lo_mutex during __loop_clr_fd()")
>
> Why do you conclude it is caused by above commit?
Some commit in the merge window for 7.1 broke the loop driver.
Even synchronize_rcu() + drain_workqueue(lo->workqueue) is not sufficient.
Please see a thread at https://lkml.kernel.org/r/d43125ff-cc66-49b7-b16d-1b2650c68c23@I-love.SAKURA.ne.jp .
^ permalink raw reply
* [PATCH 4/4] dm crypt: batch all sectors of a bio per crypto request
From: Leonid Ravich @ 2026-05-19 12:00 UTC (permalink / raw)
To: Herbert Xu
Cc: David S . Miller, Mike Snitzer, Mikulas Patocka, Alasdair Kergon,
Ard Biesheuvel, Eric Biggers, Jens Axboe, Horia Geanta,
Gilad Ben-Yossef, linux-crypto, dm-devel, linux-block
In-Reply-To: <20260428101225.24316-1-lravich@amazon.com>
When the underlying skcipher driver advertises support for multiple
data units in a single request (CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT),
configure the cipher with cc->sector_size as data_unit_size and
submit one request per bio instead of one request per sector. This
removes per-sector overhead in the crypto API hot path: request
allocation, callback dispatch, completion handling, and SG setup.
The optimisation is enabled automatically at table load when all
of the following hold:
- the cipher is non-aead (i.e. skcipher);
- tfms_count is 1 (interleaved per-sector keys would break batching);
- the IV mode is plain or plain64 (the only modes whose generator
produces a sequential 64-bit little-endian counter that the cipher
can extend by adding the data-unit index, matching the convention
documented in crypto_skcipher_set_data_unit_size());
- the iv_gen_ops->post() hook is unset (lmk and tcw use it; both are
already excluded by the IV-mode test, but the explicit check makes
the assumption durable against future IV modes);
- dm-integrity is not stacked (no integrity tag or integrity IV);
- the cipher driver advertises multi-data-unit support.
A new CRYPT_MULTI_DATA_UNIT cipher_flag, set once at construction
time, gates the multi-data-unit path. The existing per-sector path
in crypt_convert_block_skcipher() is unchanged; the new
crypt_convert_block_skcipher_multi() is reached from a small dispatch
in crypt_convert() and shares the same backlog/-EBUSY/-EINPROGRESS
flow control with the per-sector path.
Heap-allocated scatterlists are stashed in dm_crypt_request and freed
in crypt_free_req_skcipher() to avoid races between the synchronous-
success free path and async-completion reuse from the request pool.
On -ENOMEM during scatterlist allocation, the bio is requeued via
BLK_STS_DEV_RESOURCE rather than failed, matching the behaviour of
the existing -ENOMEM path for crypto request allocation.
Verified end-to-end with a byte-equivalence test: encrypted output of
plain64 dm-crypt with the multi-data-unit path matches output of the
single-data-unit path bit-for-bit over a 256 MB device.
Signed-off-by: Leonid Ravich <lravich@amazon.com>
---
drivers/md/dm-crypt.c | 248 ++++++++++++++++++++++++++++++++++++++++--
1 file changed, 241 insertions(+), 7 deletions(-)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 5ef43231fe77..b35831d43f0e 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -98,6 +98,14 @@ struct dm_crypt_request {
struct scatterlist sg_in[4];
struct scatterlist sg_out[4];
u64 iv_sector;
+ /*
+ * Heap-allocated scatterlists used by the multi-data-unit path
+ * when one bio is processed in a single skcipher request. NULL
+ * when the inline sg_in[]/sg_out[] arrays above are sufficient
+ * (single-data-unit path). Freed in crypt_free_req_skcipher().
+ */
+ struct scatterlist *sg_in_ext;
+ struct scatterlist *sg_out_ext;
};
struct crypt_config;
@@ -149,6 +157,7 @@ enum cipher_flags {
CRYPT_IV_LARGE_SECTORS, /* Calculate IV from sector_size, not 512B sectors */
CRYPT_ENCRYPT_PREPROCESS, /* Must preprocess data for encryption (elephant) */
CRYPT_KEY_MAC_SIZE_SET, /* The integrity_key_size option was used */
+ CRYPT_MULTI_DATA_UNIT, /* Batch all sectors of a bio per crypto request */
};
/*
@@ -1501,12 +1510,139 @@ static int crypt_convert_block_skcipher(struct crypt_config *cc,
return r;
}
+/*
+ * Multi-data-unit variant of crypt_convert_block_skcipher. Submits all
+ * remaining sectors of the current bio in one skcipher request whose
+ * data_unit_size is cc->sector_size. The cipher walks the IV between
+ * data units (see crypto_skcipher_set_data_unit_size()).
+ *
+ * Returns the same set of values as crypt_convert_block_skcipher:
+ * 0 on synchronous success (full chunk processed),
+ * -EINPROGRESS / -EBUSY on asynchronous dispatch,
+ * -ENOMEM if scatterlist allocation fails (caller maps to
+ * BLK_STS_DEV_RESOURCE so the bio is requeued, not failed),
+ * negative errno otherwise.
+ *
+ * On success the bio iterators have been advanced by the chunk size.
+ */
+static int crypt_convert_block_skcipher_multi(struct crypt_config *cc,
+ struct convert_context *ctx,
+ struct skcipher_request *req,
+ unsigned int *out_processed)
+{
+ const unsigned int sector_size = cc->sector_size;
+ unsigned int total_in = ctx->iter_in.bi_size;
+ unsigned int total_out = ctx->iter_out.bi_size;
+ unsigned int total = min(total_in, total_out);
+ unsigned int n_sectors;
+ unsigned int n_sg_in = 0, n_sg_out = 0;
+ struct dm_crypt_request *dmreq = dmreq_of_req(cc, req);
+ struct scatterlist *sg_in = NULL, *sg_out = NULL;
+ struct bvec_iter iter_in, iter_out;
+ struct bio_vec bv;
+ u8 *iv, *org_iv;
+ int r;
+
+ if (unlikely(total < sector_size))
+ return -EIO;
+ n_sectors = total / sector_size;
+ total = n_sectors * sector_size;
+
+ /*
+ * Walk the bio_vec iterators to count how many SG entries we need
+ * for exactly @total bytes. bi_size of the iterators is at least
+ * @total by construction above.
+ */
+ iter_in = ctx->iter_in;
+ iter_in.bi_size = total;
+ __bio_for_each_segment(bv, ctx->bio_in, iter_in, iter_in)
+ n_sg_in++;
+
+ iter_out = ctx->iter_out;
+ iter_out.bi_size = total;
+ __bio_for_each_segment(bv, ctx->bio_out, iter_out, iter_out)
+ n_sg_out++;
+
+ sg_in = kmalloc_array(n_sg_in, sizeof(*sg_in), GFP_NOIO);
+ sg_out = (ctx->bio_in == ctx->bio_out) ? sg_in :
+ kmalloc_array(n_sg_out, sizeof(*sg_out), GFP_NOIO);
+ if (!sg_in || !sg_out) {
+ kfree(sg_in);
+ if (sg_out != sg_in)
+ kfree(sg_out);
+ return -ENOMEM;
+ }
+
+ sg_init_table(sg_in, n_sg_in);
+ {
+ unsigned int i = 0;
+
+ iter_in = ctx->iter_in;
+ iter_in.bi_size = total;
+ __bio_for_each_segment(bv, ctx->bio_in, iter_in, iter_in)
+ sg_set_page(&sg_in[i++], bv.bv_page, bv.bv_len,
+ bv.bv_offset);
+ }
+
+ if (sg_out != sg_in) {
+ unsigned int i = 0;
+
+ sg_init_table(sg_out, n_sg_out);
+ iter_out = ctx->iter_out;
+ iter_out.bi_size = total;
+ __bio_for_each_segment(bv, ctx->bio_out, iter_out, iter_out)
+ sg_set_page(&sg_out[i++], bv.bv_page, bv.bv_len,
+ bv.bv_offset);
+ }
+
+ /*
+ * Compute the IV for the first data unit. The cipher will derive
+ * IVs for subsequent data units by treating this one as a 128-bit
+ * little-endian counter and adding the data-unit index, which
+ * matches the layout produced by plain and plain64.
+ */
+ dmreq->iv_sector = ctx->cc_sector;
+ if (test_bit(CRYPT_IV_LARGE_SECTORS, &cc->cipher_flags))
+ dmreq->iv_sector >>= cc->sector_shift;
+ dmreq->ctx = ctx;
+
+ iv = iv_of_dmreq(cc, dmreq);
+ org_iv = org_iv_of_dmreq(cc, dmreq);
+ r = cc->iv_gen_ops->generator(cc, org_iv, dmreq);
+ if (r < 0)
+ goto out_free_sg;
+ memcpy(iv, org_iv, cc->iv_size);
+
+ /* Stash the SG arrays for cleanup on completion / free. */
+ dmreq->sg_in_ext = sg_in;
+ dmreq->sg_out_ext = (sg_out == sg_in) ? NULL : sg_out;
+
+ skcipher_request_set_crypt(req, sg_in, sg_out, total, iv);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ *out_processed = total;
+ return r;
+
+out_free_sg:
+ kfree(sg_in);
+ if (sg_out != sg_in)
+ kfree(sg_out);
+ dmreq->sg_in_ext = NULL;
+ dmreq->sg_out_ext = NULL;
+ return r;
+}
+
static void kcryptd_async_done(void *async_req, int error);
static int crypt_alloc_req_skcipher(struct crypt_config *cc,
struct convert_context *ctx)
{
unsigned int key_index = ctx->cc_sector & (cc->tfms_count - 1);
+ struct dm_crypt_request *dmreq;
if (!ctx->r.req) {
ctx->r.req = mempool_alloc(&cc->req_pool, in_interrupt() ? GFP_ATOMIC : GFP_NOIO);
@@ -1516,6 +1652,18 @@ static int crypt_alloc_req_skcipher(struct crypt_config *cc,
skcipher_request_set_tfm(ctx->r.req, cc->cipher_tfm.tfms[key_index]);
+ /*
+ * Initialise the heap-allocated scatterlist pointers so that
+ * crypt_free_req_skcipher() does not read uninitialised memory
+ * for paths that don't take the multi-data-unit branch. The
+ * dmreq trailer lives in the per-bio data area which is not
+ * zeroed by the dm core, and the request is reused from the
+ * mempool across many bios.
+ */
+ dmreq = dmreq_of_req(cc, ctx->r.req);
+ dmreq->sg_in_ext = NULL;
+ dmreq->sg_out_ext = NULL;
+
/*
* Use REQ_MAY_BACKLOG so a cipher driver internally backlogs
* requests if driver request queue is full.
@@ -1562,6 +1710,12 @@ static void crypt_free_req_skcipher(struct crypt_config *cc,
struct skcipher_request *req, struct bio *base_bio)
{
struct dm_crypt_io *io = dm_per_bio_data(base_bio, cc->per_bio_data_size);
+ struct dm_crypt_request *dmreq = dmreq_of_req(cc, req);
+
+ kfree(dmreq->sg_in_ext);
+ dmreq->sg_in_ext = NULL;
+ kfree(dmreq->sg_out_ext);
+ dmreq->sg_out_ext = NULL;
if ((struct skcipher_request *)(io + 1) != req)
mempool_free(req, &cc->req_pool);
@@ -1590,7 +1744,9 @@ static void crypt_free_req(struct crypt_config *cc, void *req, struct bio *base_
static blk_status_t crypt_convert(struct crypt_config *cc,
struct convert_context *ctx, bool atomic, bool reset_pending)
{
- unsigned int sector_step = cc->sector_size >> SECTOR_SHIFT;
+ const unsigned int sector_step = cc->sector_size >> SECTOR_SHIFT;
+ const bool multi_du = test_bit(CRYPT_MULTI_DATA_UNIT, &cc->cipher_flags);
+ unsigned int processed;
int r;
/*
@@ -1611,8 +1767,13 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
atomic_inc(&ctx->cc_pending);
+ processed = cc->sector_size;
if (crypt_integrity_aead(cc))
r = crypt_convert_block_aead(cc, ctx, ctx->r.req_aead, ctx->tag_offset);
+ else if (multi_du)
+ r = crypt_convert_block_skcipher_multi(cc, ctx,
+ ctx->r.req,
+ &processed);
else
r = crypt_convert_block_skcipher(cc, ctx, ctx->r.req, ctx->tag_offset);
@@ -1634,8 +1795,19 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
* exit and continue processing in a workqueue
*/
ctx->r.req = NULL;
- ctx->tag_offset++;
- ctx->cc_sector += sector_step;
+ if (!multi_du) {
+ ctx->tag_offset++;
+ ctx->cc_sector += sector_step;
+ } else {
+ bio_advance_iter(ctx->bio_in,
+ &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out,
+ &ctx->iter_out,
+ processed);
+ ctx->cc_sector +=
+ processed >> SECTOR_SHIFT;
+ }
return BLK_STS_DEV_RESOURCE;
}
} else {
@@ -1649,19 +1821,42 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
*/
case -EINPROGRESS:
ctx->r.req = NULL;
- ctx->tag_offset++;
- ctx->cc_sector += sector_step;
+ if (!multi_du) {
+ ctx->tag_offset++;
+ ctx->cc_sector += sector_step;
+ } else {
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out,
+ processed);
+ ctx->cc_sector += processed >> SECTOR_SHIFT;
+ }
continue;
/*
* The request was already processed (synchronously).
*/
case 0:
atomic_dec(&ctx->cc_pending);
- ctx->cc_sector += sector_step;
- ctx->tag_offset++;
+ if (!multi_du) {
+ ctx->cc_sector += sector_step;
+ ctx->tag_offset++;
+ } else {
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out,
+ processed);
+ ctx->cc_sector += processed >> SECTOR_SHIFT;
+ }
if (!atomic)
cond_resched();
continue;
+ /*
+ * Out of memory for the multi-DU SG arrays — bounce back
+ * to the caller for requeue rather than failing the bio.
+ */
+ case -ENOMEM:
+ atomic_dec(&ctx->cc_pending);
+ return BLK_STS_DEV_RESOURCE;
/*
* There was a data integrity error.
*/
@@ -3142,6 +3337,45 @@ static int crypt_ctr_cipher(struct dm_target *ti, char *cipher_in, char *key)
}
}
+ /*
+ * Enable multi-data-unit batching when the cipher supports it and
+ * the IV layout is one we can derive per-DU from a single starting
+ * IV: plain or plain64 produce a sequential 64-bit little-endian
+ * counter, which matches the convention of
+ * crypto_skcipher_set_data_unit_size(). Restrict to the simple
+ * case (single tfm, no integrity, no per-sector post() callback)
+ * to keep the consumer path small; modes like essiv, lmk, tcw,
+ * eboiv, plain64be, random, null, benbi, and elephant are
+ * deliberately excluded because their generators or post-IV hooks
+ * cannot be re-derived by the cipher between data units.
+ */
+ if (!crypt_integrity_aead(cc) && cc->tfms_count == 1 &&
+ cc->iv_gen_ops &&
+ (cc->iv_gen_ops == &crypt_iv_plain_ops ||
+ cc->iv_gen_ops == &crypt_iv_plain64_ops) &&
+ !cc->iv_gen_ops->post &&
+ !cc->integrity_tag_size && !cc->integrity_iv_size &&
+ crypto_skcipher_supports_multi_data_unit(cc->cipher_tfm.tfms[0])) {
+ ret = crypto_skcipher_set_data_unit_size(cc->cipher_tfm.tfms[0],
+ cc->sector_size);
+ if (!ret) {
+ set_bit(CRYPT_MULTI_DATA_UNIT, &cc->cipher_flags);
+ DMINFO("Using multi-data-unit crypto offload (du=%u)",
+ cc->sector_size);
+ } else {
+ /*
+ * The driver advertised the capability via cra_flags
+ * but rejected the requested data unit size. This is
+ * a driver bug worth seeing in dmesg; fall back to
+ * the per-sector path so the device still activates.
+ */
+ DMWARN_LIMIT("multi-DU offload disabled: %s rejected du=%u (%d)",
+ crypto_skcipher_driver_name(cc->cipher_tfm.tfms[0]),
+ cc->sector_size, ret);
+ ret = 0;
+ }
+ }
+
/* wipe the kernel key payload copy */
if (cc->key_string)
memset(cc->key, 0, cc->key_size * sizeof(u8));
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox