Re: [PATCH 0/8 v3] xfs: intent whiteouts

[Alli <allison.henderson@oracle.com>]

On Tue, 2022-03-15 at 09:06 +1100, Dave Chinner wrote:
> This is a patchset inspired by the performance regressions that were
> seen from logging 64k xattrs with Allison's delayed attribute
> patchset and trying to work out how to minimise the impact of
> logging xattrs. Most of that is explained in the "xfs: intent item
> whiteouts" patch, so I won't repeat it here.
> 
> The whiteouts massively reduce the journal write overhead of logging
> xattrs - with this patchset I've reduced 2.5GB/s of log traffic (16
> way file create w/64k xattr workload) down to approximately 220MB of
> log traffic, and performance has increased from 9k creates/s to 36k
> creates/s. The workload still writes to disk at 2.5GB/s, but that's
> what writing 35k x 64k xattrs to disk does.
> 
> This is still short of the non-logged attribute mechanism, which
> runs at 40-45k creates a second and 3.5-4GB/s to disk, but it brings
> logged attrs to within roughly 5-15% of non-logged attrs across the
> full range of attribute sizes.
> 
> So, while this patchset was clearly insired and has major positive
> impact on Allison's delayed attribute work, it also applies
> generically to all other intent/intent done pairs that already
> exist. Hence I've created this patchset as a stand-alone patchset
> that isn't dependent on the delayed attributes being committed, nor
> does the delayed attribute patchset need this to function properly.
> IOWs, they can be merged in parallel and then the attribute log item
> implementation be updated to support whiteouts after the fact.
> 
> This patchset is separate to the attr code, though, because
> intent whiteouts are not specific to the attr code. They are a
> generic mechanism that can be applied to all the intent/intent done
> item pairs we already have. This patch set modifies all those
> intents to use whiteouts, and so there is benefits from the patch
> set for all operations that use these intents.
> 
> With respect to the delayed attribute patchset, it can be merged
> without whiteout support and still work correctly with/without this
> patchset in place. Once both intent whiteouts and delayed attrs are
> merged, we can add whiteout support to delayed attributes with only
> a few lines of extra code.
> 
I applied this set to the for-next tag:
01728b44ef1b (tag: xfs-5.18-merge-2), and ran into a few merge
conflicts.  I just went through and fixed them to try and get things
working, but I'm getting some failed assert checks on generic/466:

[  155.219780] XFS: Assertion failed: reg->i_len % sizeof(int32_t) ==
0, file: fs/xfs/xfs_log.c, line: 2536

So maybe still some bugglies to work out.  I'll go over the conflicts
in the review.  Also, I'll push out the branch I have so far. Since
I've already gone through the rebase effort, I may as well share it:
https://github.com/allisonhenderson/xfs_work/tree/whiteouts

Allison

> Changelog:
> 
> Version 3:
> - rebased on 5.17-rc4 + xlog-write-rework
> - no longer dependent on xfs-cil-scalability, so there's some porting
> changes
>   that was needed to remove all the per-cpu CIL dependencies.
> 
> Version 2:
> - not published
> - rebased on 5.15-rc2 + xfs-cil-scalability
> - dropped the kvmalloc changes for CIL shadow buffers as that's a
>   separate perf problem and not something related to intent
>   whiteouts.
> - dropped all the delayed attribute modifications so that the
>   patchset is not dependent on Allison's dev tree.
> - Thanks to Allison for an initial quick review pass - I haven't
>   included those RVB tags because every patch in the series has
>   changed since the original RFC posting.
> 
> RFC:
> - 
> https://lore.kernel.org/linux-xfs/20210909212133.GE2361455@dread.disaster.area/
>