Re: [PATCHSET v5.6] fstests: fs-verity support for XFS

["Darrick J. Wong" <djwong@kernel.org>]

On Sat, May 11, 2024 at 01:01:46PM +0800, Zorro Lang wrote:
> Hi Darrick,
> 
> Due to only half of this patchset got reviewed, so I'd like to wait for your
> later version. I won't pick up part of this patchset to merge this time, I
> think better to merge it as an integrated patchset.

Christoph and I talked about the future of this patchset at LSF and
there are some file format changes in store, so please hold off on
analyzing this patchset for now.

--D

> Thanks,
> Zorro
> 
> On Mon, Apr 29, 2024 at 08:19:24PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > This patchset adds support for fsverity to XFS.  In keeping with
> > Andrey's original design, XFS stores all fsverity metadata in the
> > extended attribute data.  However, I've made a few changes to the code:
> > First, it now caches merkle tree blocks directly instead of abusing the
> > buffer cache.  This reduces lookup overhead quite a bit, at a cost of
> > needing a new shrinker for cached merkle tree blocks.
> > 
> > To reduce the ondisk footprint further, I also made the verity
> > enablement code detect trailing zeroes whenever fsverity tells us to
> > write a buffer, and elide storing the zeroes.  To further reduce the
> > footprint of sparse files, I also skip writing merkle tree blocks if the
> > block contents are entirely hashes of zeroes.
> > 
> > Next, I implemented more of the tooling around verity, such as debugger
> > support, as much fsck support as I can manage without knowing the
> > internal format of the fsverity information; and added support for
> > xfs_scrub to read fsverity files to validate the consistency of the data
> > against the merkle tree.
> > 
> > Finally, I add the ability for administrators to turn off fsverity,
> > which might help recovering damaged data from an inconsistent file.
> > 
> > From Andrey Albershteyn:
> > 
> > Here's v5 of my patchset of adding fs-verity support to XFS.
> > 
> > This implementation uses extended attributes to store fs-verity
> > metadata. The Merkle tree blocks are stored in the remote extended
> > attributes. The names are offsets into the tree.
> > From Darrick J. Wong:
> > 
> > This v5.3 patchset builds upon v5.2 of Andrey's patchset to implement
> > fsverity for XFS.
> > 
> > The biggest thing that I didn't like in the v5 patchset is the abuse of
> > the data device's buffer cache to store the incore version of the merkle
> > tree blocks.  Not only do verity state flags end up in xfs_buf, but the
> > double-alloc flag wastes memory and doesn't remain internally consistent
> > if the xattrs shift around.
> > 
> > I replaced all of that with a per-inode xarray that indexes incore
> > merkle tree blocks.  For cache hits, this dramatically reduces the
> > amount of work that xfs has to do to feed fsverity.  The per-block
> > overhead is much lower (8 bytes instead of ~300 for xfs_bufs), and we no
> > longer have to entertain layering violations in the buffer cache.  I
> > also added a per-filesystem shrinker so that reclaim can cull cached
> > merkle tree blocks, starting with the leaf tree nodes.
> > 
> > I've also rolled in some changes recommended by the fsverity maintainer,
> > fixed some organization and naming problems in the xfs code, fixed a
> > collision in the xfs_inode iflags, and improved dead merkle tree cleanup
> > per the discussion of the v5 series.  At this point I'm happy enough
> > with this code to start integrating and testing it in my trees, so it's
> > time to send it out a coherent patchset for comments.
> > 
> > For v5.3, I've added bits and pieces of online and offline repair
> > support, reduced the size of partially filled merkle tree blocks by
> > removing trailing zeroes, changed the xattr hash function to better
> > avoid collisions between merkle tree keys, made the fsverity
> > invalidation bitmap unnecessary, and made it so that we can save space
> > on sparse verity files by not storing merkle tree blocks that hash
> > totally zeroed data blocks.
> > 
> > From Andrey Albershteyn:
> > 
> > Here's v5 of my patchset of adding fs-verity support to XFS.
> > 
> > This implementation uses extended attributes to store fs-verity
> > metadata. The Merkle tree blocks are stored in the remote extended
> > attributes. The names are offsets into the tree.
> > 
> > If you're going to start using this code, I strongly recommend pulling
> > from my git trees, which are linked below.
> > 
> > This has been running on the djcloud for months with no problems.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> > 
> > kernel git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=fsverity
> > 
> > xfsprogs git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=fsverity
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fsverity
> > ---
> > Commits in this patchset:
> >  * common/verity: enable fsverity for XFS
> >  * xfs/{021,122}: adapt to fsverity xattrs
> >  * xfs/122: adapt to fsverity
> >  * xfs: test xfs_scrub detection and correction of corrupt fsverity metadata
> >  * xfs: test disabling fsverity
> >  * common/populate: add verity files to populate xfs images
> > ---
> >  common/populate    |   24 +++++++++
> >  common/verity      |   39 ++++++++++++++-
> >  tests/xfs/021      |    3 +
> >  tests/xfs/122.out  |    3 +
> >  tests/xfs/1880     |  135 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/1880.out |   37 ++++++++++++++
> >  tests/xfs/1881     |  111 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/1881.out |   28 +++++++++++
> >  8 files changed, 378 insertions(+), 2 deletions(-)
> >  create mode 100755 tests/xfs/1880
> >  create mode 100644 tests/xfs/1880.out
> >  create mode 100755 tests/xfs/1881
> >  create mode 100644 tests/xfs/1881.out
> > 
> 
>