Re: [RFC PATCH v3 4/5] rust: miscdevice: Add `uring_cmd` support

[Caleb Sander Mateos <csander@purestorage.com>]

On Tue, Sep 2, 2025 at 4:18 AM Sidong Yang <sidong.yang@furiosa.ai> wrote:
>
> On Mon, Sep 01, 2025 at 06:12:44PM -0700, Caleb Sander Mateos wrote:
> > On Fri, Aug 22, 2025 at 5:56 AM Sidong Yang <sidong.yang@furiosa.ai> wrote:
> > >
> > > This patch introduces support for `uring_cmd` to the `miscdevice`
> > > framework. This is achieved by adding a new `uring_cmd` method to the
> > > `MiscDevice` trait and wiring it up to the corresponding
> > > `file_operations` entry.
> > >
> > > The `uring_cmd` function provides a mechanism for `io_uring` to issue
> > > commands to a device driver.
> > >
> > > The new `uring_cmd` method takes the device, an `IoUringCmd` object,
> > > and issue flags as arguments. The `IoUringCmd` object is a safe Rust
> > > abstraction around the raw `io_uring_cmd` struct.
> > >
> > > To enable `uring_cmd` for a specific misc device, the `HAS_URING_CMD`
> > > constant must be set to `true` in the `MiscDevice` implementation.
> > >
> > > Signed-off-by: Sidong Yang <sidong.yang@furiosa.ai>
> > > ---
> > >  rust/kernel/miscdevice.rs | 53 ++++++++++++++++++++++++++++++++++++++-
> > >  1 file changed, 52 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/rust/kernel/miscdevice.rs b/rust/kernel/miscdevice.rs
> > > index 6373fe183b27..fcef579218ba 100644
> > > --- a/rust/kernel/miscdevice.rs
> > > +++ b/rust/kernel/miscdevice.rs
> > > @@ -11,9 +11,10 @@
> > >  use crate::{
> > >      bindings,
> > >      device::Device,
> > > -    error::{to_result, Error, Result, VTABLE_DEFAULT_ERROR},
> > > +    error::{from_result, to_result, Error, Result, VTABLE_DEFAULT_ERROR},
> > >      ffi::{c_int, c_long, c_uint, c_ulong},
> > >      fs::File,
> > > +    io_uring::IoUringCmd,
> > >      mm::virt::VmaNew,
> > >      prelude::*,
> > >      seq_file::SeqFile,
> > > @@ -180,6 +181,21 @@ fn show_fdinfo(
> > >      ) {
> > >          build_error!(VTABLE_DEFAULT_ERROR)
> > >      }
> > > +
> > > +    /// Handler for uring_cmd.
> > > +    ///
> > > +    /// This function is invoked when userspace process submits an uring_cmd op
> > > +    /// on io-uring submission queue. The `device` is borrowed instance defined
> > > +    /// by `Ptr`. The `io_uring_cmd` would be used for get arguments cmd_op, sqe,
> > > +    /// cmd_data. The `issue_flags` is the flags includes options for uring_cmd.
> > > +    /// The options are listed in `kernel::io_uring::cmd_flags`.
> > > +    fn uring_cmd(
> > > +        _device: <Self::Ptr as ForeignOwnable>::Borrowed<'_>,
> > > +        _io_uring_cmd: Pin<&mut IoUringCmd>,
> >
> > Passing the IoUringCmd by reference doesn't allow the uring_cmd()
> > implementation to store it beyond the function return. That precludes
> > any asynchronous uring_cmd() implementation, which is kind of the
> > whole point of uring_cmd. I think uring_cmd() needs to transfer
> > ownership of the IoUringCmd so the implementation can complete it
> > asynchronously.
>
> I didn't know that I can take IoUringCmd ownership and calling done().
> In C implementation, is it safe to call `io_uring_cmd_done()` in any context?

It depends on the issue_flags. If IO_URING_F_UNLOCKED is set, it can
be called from any context. But if IO_URING_F_UNLOCKED is not set, it
needs to be called with the io_ring_ctx's uring_lock held. That
generally requires it to either be called during uring_cmd() or from a
task work callback. In either case, issue_flags needs to match what
was passed to uring_cmd() or the task work callback.
You can look at NVMe passthru as an example. It calls
io_uring_cmd_done() from nvme_uring_task_cb(), which is a task work
callback scheduled with io_uring_cmd_do_in_task_lazy() in
nvme_uring_cmd_end_io().

Best,
Caleb