Re: support for RT data checksums

[Christoph Hellwig <hch@lst.de>]

On Wed, Sep 30, 2026 at 05:11:58PM +1000, Dave Chinner wrote:
> My initial thought was that on disk format structures shouldn't be
> defined by the limitations of the OS memory allocation, but <shrug>.
> It kept nagging at me, though.

Well, it would be nice to be able to design without all the real-life
constraints around us, wouldn't it?  I'm trying to strike a balance
between what would be useful (go big) and what is feasible.  If we
want other values, we can always increase the support range for
newer kernels and tools.

> I looked more closely at what XFS_BLI_PREALLOC did to try to
> understand why it existed.  It triggers a max-sized CIL logvec
> structure for the buffer object. For a 32kB buffer logged as a
> single contiguous range, this ends up being about 32kB + a logvec
> header, plus a log iovec, plus a BLF, plus a couple of ophdrs. So
> it's about 32kB + 200-250 bytes.
> 
> That means the shadow buffer for a csum buffer is always considered
> a costly allocation by the MM subsystem.

It ends up using vmalloc exclusively based on tracing for me,
but that might be different on different systems.

> Hence for csum BLI, if a CIL flush happens on a partially filled
> buffer, a good amount of that shadow buffer will go unused. Then we
> allocate another (costly) shadow buffer on the next update. If CIL
> flushes happen frequently enough then we will be repeatedly doing
> costly allocations for shadow buffers that we don't actually use.

Yes.  But if we don't do this we realloc for every few blocks
written, which is a lot more costly.

> Not ideal - I think that means the original "sized for mm fast path"
> intent is really only valid for the read side of the csum
> algorithms as implemented by the patchset.

It is valid for the xfs_buf backing where we actually hit the folio
allocator.  Which is used both for read and write, but obviously
most workloads tend to hit reads a lot harder than writes.

> Ok, we have a solution to this problem. I created ordered buffers
> and one-shot log items to avoid the journalling overhead of static
> inode buffer initialisation back in 2013. The ICREATE log item is
> the one-shot log item that records a buffer should be initialised,
> and the ordered buffer allows the modified buffer to be passed
> through the journal to metadta writeback without it's contents being
> logged.
> 
> Given that csum updates are a small, known size, non-overlapping
> one-shot update to a buffer, they fit the same model that
> ICREAT+ordered implements. Adding a new CSUM log item made up of a
> format header and varible size csum payload region provides the
> equivalent of the ICREAT item for journalled inode buffer
> initialisation.

I initially looked into intent/done based csums, but we still end
with an allocation per log operation, and a memcpy both into that
and into the buffer, while adding a lot of new log items.

One thing I played with for a while until I realized that the simple
buf item actually provides good enough performance is special
xfs_log_vec that is not included in the main log vec / shadow allocation
but points to external memory.  This obviously only works for fixed
size non-overlapping regions, but then isn't too bad.  This is the
prep work for it, which I recently refreshed:

https://git.infradead.org/?p=users/hch/xfs.git;a=shortlog;h=refs/heads/xlog-ophdr

These can work with the buf_item on-disk format, so I'd rather not
prematurely optimize it, as the prototype shows that I can go to that
any time I want.  And eventually I think I'd want to go there, as it
drastically reduces the memory usage if only the format header and
two ophrs need to be allocated ontop of the backing buffer.  But there's
plenty more important things on the plate for now.