From: John Garry <john.g.garry@oracle.com>
To: Vitaliy Filippov <vitalifster@gmail.com>
Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org,
linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH] fs: remove power of 2 and length boundary atomic write restrictions
Date: Wed, 7 Jan 2026 15:42:51 +0000 [thread overview]
Message-ID: <02c255b8-2ddb-43bd-9bfd-4946ef065463@oracle.com> (raw)
In-Reply-To: <CAPqjcqqhFWz0eNGJRW-_PoJhdM7f-yxr=pWN2_AfGSP=-VpyMg@mail.gmail.com>
On 07/01/2026 13:05, Vitaliy Filippov wrote:
>> What is the actual usecase you are trying to solve? You mentioned "avoid
>> journaling", which does not explain what you want to achieve.
>>
>> You could arrange your data so that it suits the rules.
>
> I can't. My usecase is a distributed ceph-like SDS based on atomic
> writes. Writes on a virtual block device have arbitrary length &
> offset of course,
Note that the alignment rule is not just for atomic HW boundaries. We
also support atomic writes on stacked devices, where this is relevant -
specifically striped devices, like raid0. Doing an unaligned atomic
write on a striped device may result in trying to issue an atomic write
which straddles 2x separate devices, which would obviously be broken.
> nothing like 2^N, like on a regular block device.
> Atomicity is implemented through journaling (double-write) on disks
> without hardware atomic write support.
>
> Then I found the new atomic write feature and SSDs with support for it
> and implemented a new storage layer which can take advantage of it. My
> new storage layer has write amplification about ~1.0 with atomic
> writes (i.e. almost zero overhead). It's a huge improvement for me -
> the old storage layer has WA from 3 to 4.
>
> And everything was fine until I finally deployed it with enabled
> RWF_ATOMIC (production setups should use safety features) and stumbled
> upon the 2^N restriction... It was a big surprise, I never thought
> that such a limitation could exist. It's absolutely irrational - the
> device doesn't have that limitation and I'm just using the raw device.
This is all described in the man pages.
>
> It's normal and expected in the context of simple file systems like
> ext4 and xfs. But for the raw device... I only discovered it after
> several days of investigation with bpftrace and after reading the
> kernel code. It's really unexpected. I think anyone expects the raw
> NVMe disk to have the same requirements as it's described in the NVMe
> spec.
It seems that you just want to take advantage of the block layer code to
handle submission of an atomic write bio, i.e. reject anything which
cannot be atomically written. In essence, that would be to just set
REQ_ATOMIC. Maybe that could be done as a passthrough command, I'm not sure.
next prev parent reply other threads:[~2026-01-07 15:43 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-12-24 11:53 [PATCH] fs: remove power of 2 and length boundary atomic write restrictions Vitaliy Filippov
2025-12-29 7:15 ` kernel test robot
2025-12-30 7:54 ` John Garry
2025-12-30 9:01 ` Vitaliy Filippov
2026-01-02 17:41 ` John Garry
2026-01-05 18:58 ` Vitaliy Filippov
2026-01-06 9:06 ` John Garry
2026-01-06 10:50 ` Vitaliy Filippov
2026-01-06 11:26 ` John Garry
2026-01-06 13:08 ` Vitaliy Filippov
2026-01-07 10:51 ` John Garry
2026-01-07 13:05 ` Vitaliy Filippov
2026-01-07 15:42 ` John Garry [this message]
2026-01-07 16:21 ` Vitaliy Filippov
2026-01-08 18:18 ` Vitaliy Filippov
2026-01-13 14:25 ` Vitaliy Filippov
2026-01-05 19:29 ` Vitaliy Filippov
2026-01-05 19:44 ` Vitaliy Filippov
2026-01-06 10:55 ` Vitaliy Filippov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=02c255b8-2ddb-43bd-9bfd-4946ef065463@oracle.com \
--to=john.g.garry@oracle.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=vitalifster@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).