Re: [LSF/MM/BPF TOPIC] Implementing the NFS v4.2 WRITE_SAME operation: VFS or NFS ioctl() ?

["Theodore Ts'o" <tytso@mit.edu>]

On Thu, Jan 16, 2025 at 10:45:01AM -0500, Chuck Lever wrote:
> 
> Any database that uses a block size that is larger than the block
> size of the underlying storage media is at risk of a torn write.
> The purpose of WRITE_SAME is to demark the database blocks with
> sentinels on each end of the database block containing a time
> stamp or hash.

There are alternate solutions which various databases to address the
torn write problem:

   * DIF/DIX (although this is super expensive, so this has fallen out
        of favor)
   * In-line checksums in the database block; this approach is fairly
        common for enterprise databases (interestingly, Google's cluster
	file systems, which don't need to support mmap, do this as well)
   * Double-buffered writes using a journal (this is what open source
         databases tend to use)
   * For software-defined cloud block devices (such as Google's
       Persistent Disk, Amazon EBS, etc.) and some NVMe devices,
       aligned writes can be guaranteed up to some write granularity
       (typically up to 32k to 64k, although pretty much all database
       pages today are 16k).  This is actively fielded as
       customer-available products and/or in development in at least
       two first-party cloud database products based on MySQL and/or
       Postgres; and there are some active patches which John Garry
       has been working on so that users can use this technique
       without having to rely on first party cloud product teams
       knowing implementation details of their cloud block devices.
       (This has been discussed in past LSF/MM sessions.)

> If, when read back, the sentinels match, the whole database
> block is good to go. If they do not, then the block is torn
> and recovery is necessary.

Are there some database teams that are actively working on a scheme
based on WRITE SAME?  I have talked to open source developers on the
MySQL and Postgres teams, as well as the first party cloud product
teams at my company and some storage architects at competitor cloud
companies, and no one has mentioned any efforts involving WRITE SAME.
Of course, maybe I simply haven't come across such plans, especially
if they are under some deep, dark NDA.  :-)

However, given that support for WRITE SAME is fairly rare (like
DIF/DIX it's only available if you are willing to pay $$$$ for your
storage, because it's a specialized feature that storage vendors like
to change a lot for), I'm bit surprised that there are database groups
that would be intersted in relying on such a feature, since it tends
not be commonly available.

If there are real-world potential users, go wild, but at least for the
use cases and databases that I'm aware of, the FALLOC_FL_WRITE_ZEROS
and atomic writes patch series (it's really untorn writes but we seem
to have lost that naming battle) is all that we need.

Cheers,

						- Ted