Re: Chaining is dead

[Eric Biggers <ebiggers@kernel.org>]

On Tue, Apr 01, 2025 at 11:30:33AM +0800, Herbert Xu wrote:
> On Mon, Mar 31, 2025 at 04:56:30PM +0000, Eric Biggers wrote:
> >
> > For hashing, users would need to allocate an array to hold the digest for every
> > block in the bio or folio.  That would add an additional memory allocation to
> > every I/O.  You said you'd like to fall back to a smaller buffer if the memory
> > allocation fails.  But that's silly; if we have to support that anyway, we might
> > as well do it that way only.  In which case the bio interface is pointless.
> 
> BTW, the existing verity code is hilarious.  In my test, for each
> 256KB bio, it was dividing them up into 4KB pages, and then for each
> one of them, it's looking up the correct hash in the Merkle tree.
> 
> Surely there is some locality here so if you zoom out from 4KB (or
> even 8KB with your multibuffer patch-set) to 256KB you could get all
> the correct hashes in one go, rather than 64 times.

Well, it looks up the bottom-level block before falling back to walking the full
path, and it's cached.  The cache lookup is fast compared to the actual hashing,
or at least traditionally has been.  You're right that it does often look up the
same block repeatedly, and that it's one of the things that needs to be better
optimized now that hashing is so fast that overheads like this become
significant.

- Eric