Re: [PATCH 2/5] io_uring: add support for IORING_OP_URING_CMD

Stefan Metzmacher
2021-02-20T03:58:57Z

[-- Attachment #1.1: Type: text/plain, Size: 3249 bytes --]

Hi Jens,

Am 28.01.21 um 03:19 schrieb Jens Axboe:
>>> Assuming that I got that right, that means that the pdu information
>>> doesn't actually go all the way to the end of the sqe, which currently
>>> is just a bunch of padding. Was that intentional, or does this mean
>>> that io_uring_pdu could actually be 8 bytes longer?
>>
>> Also correct. The reason is actually kind of stupid, and I think we
>> should just fix that up. struct io_uring_cmd should fit within the first
>> cacheline of io_kiocb, to avoid bloating that one. But with the members
>> in there, it ends up being 8 bytes too big, if we grab those 8 bytes.
>> What I think we should do is get rid of ->done, and just have drivers
>> call io_uring_cmd_done() instead. We can provide an empty hook for that.
>> Then we can reclaim the 8 bytes, and grow the io_uring_cmd to 56 bytes.
>
> Pushed out that version:
>
> https://git.kernel.dk/cgit/linux-block/log/?h=io_uring-fops.v2
>
> which gives you the full 56 bytes for the payload command.

I think we only have 48 bytes for the payload.

I've rebased and improved your io_uring-fops.v2 on top of your io_uring-worker.v3.

See https://git.samba.org/?p=metze/linux/wip.git;a=shortlog;h=refs/heads/io_uring-fops

I've changed the layout like this:

struct io_uring_sqe {
__u8 opcode; /* type of operation for this sqe */
__u8 flags; /* IOSQE_ flags */
union {
__u16 ioprio; /* ioprio for the request */
__u16 cmd_personality; /* IORING_OP_URING_CMD */
};
__s32 fd; /* file descriptor to do IO on */
union {
__u64 off; /* offset into file */
__u64 addr2;
__u64 cmd_user_data; /* IORING_OP_URING_CMD: data to be passed back at completion time */
};
union {
__u64 addr; /* pointer to buffer or iovecs */
__u64 splice_off_in;
__u64 cmd_pdu_start; /* IORING_OP_URING_CMD: this is the start for the remaining 48 bytes */
};

And then use:

struct io_uring_cmd_pdu {
__u64 data[6]; /* 48 bytes available for free use */
};

So we effectively have this:

struct io_uring_cmd_sqe {
__u8 opcode; /* type of operation for this sqe */
__u8 flags; /* IOSQE_ flags */
__u16 cmd_personality; /* IORING_OP_URING_CMD */
__s32 fd; /* file descriptor to do IO on */
__u64 cmd_user_data; /* IORING_OP_URING_CMD: data to be passed back at completion time */
union {
__u64 cmd_pdu_start; /* IORING_OP_URING_CMD: this is the start for the remaining 48 bytes */
struct io_uring_cmd_pdu cmd_pdu;
};
}

I think it's saner to have a complete block of 48 bytes available for the payload
and move personality and user_data to to top if opcode is IORING_OP_URING_CMD
instead of having a hole that can't be touched.

I also finished the socket glue from struct file -> struct socket -> struct sock

I think it compiles, but I haven't done any tests.

What do you think?
metze

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]