Re: [PATCH RFC 0/2] Support for Multiple Atomicity Mode

[Daniel Gomez <da.gomez@kernel.org>]

On 2026-11-03T11:38:39+02:00, Arthur Simchaev <arthur.simchaev@sandisk.com> wrote:
> On Tue, Sep 29, 2026 at 01:17:10PM +0200, Daniel Gomez wrote:
> > While I think this small feature is worth having (hence this RFC)
> > for users sending atomic write commands that are contiguous, the
> > question I'd also like to discuss really is how to also expose clear
> > MAM semantics to userspace for users who can send larger commands,
> > so they avoid syscall overhead [2] and splitting, and still get the
> > atomic guarantees in the LBA subranges. RWF_ATOMIC currently exposes
> > SAM semantics, but I think MAM may be confusing for users, as it's not
> > simply a larger atomic write but an atomic carrier. Thoughts? What can
> > work best to leverage MAM from userspace without conflicting with SAM?
>
> Hi Daniel,
>
> I agree that RWF_ATOMIC should keep its current meaning: the whole write is
> untorn. Letting it accept larger writes on MAM devices would silently weaken
> that guarantee for existing users.
>
> If we want userspace to send one large write with per-piece guarantees, I would
> keep RWF_ATOMIC unchanged and add a separate opt-in flag that describes the
> guarantee rather than the device mode, for example (names are placeholders):
>
>   pwritev2(fd, iov, 1, pos, RWF_ATOMIC_CHUNKED);
>
>   - statx reports a granule G (e.g. stx_atomic_write_chunk)
>   - the range is split at file offsets that are multiples of G
>   - each piece is untorn; nothing is guaranteed across pieces
>
> If G is tied to atomic_write_unit_max, a G-aligned piece never crosses an atomic
> boundary, because the block layer already requires the boundary to be a power of
> two no smaller than unit_max. A SAM device could then implement the flag with
> one atomic bio per piece, and MAM would only be an optimisation underneath. Open

You mean a "fallthrough" for MAM while making SAM path an iterator?
I was thinking in something easier to start with like simply not
supporting SAM path in this RFW_ATOMIC_CHUNKED flag suggestion, and
return an error like we do today if you send a non atomic conformat io
with RWF_ATOMIC.

The "interesting" thing is (for NVMe MAM) that you can send a large
command and it will be atomic at the chunk boundary anyways. But I guess
users may want to have guarantees of their chunk atomic granularity.

> questions would be the semantics of partial completion and errors, and how
> filesystems keep each piece mapped to one contiguous extent.

Thanks for the suggestion. I plan to work on it after this series.

>
> Before adding a new flag, though: with this series your fio example (io_uring,
> iodepth_batch_submit=4) already sends four 16k RWF_ATOMIC writes as one 64k
> command, so a single large write would only save per-I/O work in the kernel, not
> device commands. I understand real MAM hardware may not be available, but that

HW is available for those who need it. FYI, I'm currently testing this
on a real device and post more details in a new revision.

> cost can be estimated without it, e.g. by comparing host CPU usage of a batch of
> small O_DIRECT writes against one large write of the same total size on any NVMe
> device. Even a relative number (e.g. % CPU per GB written) would show whether
> the remaining overhead justifies new uAPI.

100%.

Thanks Arthur for the review. I talked with Keith at LPC about this so
I was planning to resend it. I will now, take your suggestions and send a
new version.