Re: [LSF/MM/BPF TOPIC] untorn buffered writes

["Theodore Ts'o" <tytso@mit.edu>]

On Wed, Feb 28, 2024 at 02:11:06PM +0000, Matthew Wilcox wrote:
> I'm not entirely sure that it does become a mess.  If our implementation
> of this ensures that each write ends up in a single folio (even if the
> entire folio is larger than the write), then we will have satisfied the
> semantics of the flag.

What if we do a 32k write which spans two folios?  And what
if the physical pages for those 32k in the buffer cache are not
contiguous?  Are you going to have to join the two 16k folios
together, or maybe two 8k folios and an 16k folio, and relocate pages
to make a contiguous 32k folio when we do a buffered RWF_ATOMIC write
of size 32k?

Folios have to consist of physically contiguous pages, right?  But we
can do send a single 32k write request using scatter-gather even if
the pages are not physically contiguous.  So it would seem to me that
trying to overload the folio size to represent the "atomic write
guarantee" of RWF_ATOMIC seems unwise.

(And yes, the database might not need it to be 32k untorn write, but
what if it sends a 32k write, for example because it's writing a set
of pages to the database journal file?  The RWF_ATOMIC interface
doesn't *know* what is really required, the only thing it knows is the
overly strong guarantees that we set in the definition of that
interface.  Or are we going to make the RWF_ATOMIC interface fail all
writes that aren't exactly 16k?  That seems.... baroque.)

> I think we'd be better off treating RWF_ATOMIC like it's a bs>PS device.
> That takes two somewhat special cases and makes them use the same code
> paths, which probably means fewer bugs as both camps will be testing
> the same code.

But for a bs > PS device, where the logical block size is greater than
the page size, you don't need the RWF_ATOMIC flag at all.  All direct
I/O writes *must* be a multiple of the logical sector size, and
buffered writes, if they are smaller than the block size, *must* be
handled as a read-modify-write, since you can't send writes to the
device smaller than the logical sector size.

This is why I claim that LBS devices and untorn writes are largely
orthogonal; for LBS devices no special API is needed at all, and
certainly not the highly problematic RWF_ATOMIC API that has been
proposed.  (Well, not problematic for Direct I/O, which is what we had
originally focused upon, but highly problematic for buffered I/O.)

	   	   	     	    		    - Ted