[PATCH 00/45 v3] xfs: consolidated log and optimisation changes

Dave Chinner
2021-03-05T05:18:03Z

Hi folks,

This is a consolidated patchset of all the outstanding patches I've
sent out recently. Previous versions and sub-series descriptions
can be found here:

https://lore.kernel.org/linux-xfs/20210226050158.GW4662@dread.disaster.area/T/#mf72a4f6acc05d117adec3fea5f6fee83432ecfd4
https://lore.kernel.org/linux-xfs/20210223033442.3267258-1-david@fromorbit.com/T/#m80819d7f7940e5f216f3219baf4f98ed35643d13
https://lore.kernel.org/linux-xfs/20210223053212.3287398-1-david@fromorbit.com/T/#mad305a62ab3532493c83b4c615f21fbaf9a87ae0
https://lore.kernel.org/linux-xfs/20210223044636.3280862-1-david@fromorbit.com/T/#m791941d515fd437bf07aa42d35df5ddb124cb80f
https://lore.kernel.org/linux-xfs/20210224063459.3436852-1-david@fromorbit.com/T/#mcb037e1495e6bb4f3289e52d581e53efe2d29765
https://lore.kernel.org/linux-xfs/20210223054748.3292734-1-david@fromorbit.com/T/#md88d8f88657f1e33008f864677921f54b4d64a2f
https://lore.kernel.org/linux-xfs/20210225033725.3558450-1-david@fromorbit.com/T/#mb6a91780514d5abe4d8b04ed1d78a8f8d681f101

The changes are largely just a rebase onto a current TOT kernel, bug
fixes and modifications from review comments. These are itemised in
the change log below. It runs though the fstests auto group fine on
e multiple machines here and performance test numbers are largely
unchanged from previous versions.

I've added a couple of new patches to the end of the series, the
most notable being an update to the delayed logging design document
to include descriptions of transaction types, log space accounting
and how we use relogging to ensure rolling transactions do not
deadlock on log space.

Cheers,

Dave.

Version 3:
- rebase onto 5.12-rc1+
- aggregate many small dependent patchsets in one large one.
- simplify xlog_wait_on_iclog_lsn() back to just a call to xlog_wait_on_iclog()
- remove xfs_blkdev_issue_flush() instead of moving and renaming it.
- pass bio to xfs_flush_bdev_async() so it doesn't need allocation.
- skip cache flush in xfs_flush_bdev_async() if the underlying queue does not
require it.
- fixed whitespace in xfs_flush_bdev_async()
- remove the implicit external log's data device cache flush code and replace it
with an explicit flush in the unmount record write so that it works the same
as the new CIL checkpoint cache pre-flush mechanism. This mechanism now
guarantees metadata vs journal ordering for both internal and external logs.
- updated various commit messages
- fixed incorrect/unintended changes to xfs_log_force() behaviour
- typedef uint64_t xfs_csn_t; and conversion.
- removed stray trace_printk()s that were used for debugging.
- fixed minor formatting details.
- uninlined xlog_prepare_iovec()
- fixed up "lv chain vector and size calculation" commit message to reflect we
are only calculating and passin gin the vector byte count.
- reworked the loop in xlog_write_single() based on Christoph's suggestion. Much
cleaner!
- added patch to pass log ticket down to xlog_sync() so that it accounts the
roundoff to the log ticket rather than directly modifying grant heads. Grant
heads are hot, so every little bit helps.
- added patch to update delayed logging design doc with background material on
how transactions and log space accounting works in XFS.

Version 2:
- fix ticket reservation roundoff to include 2 roundoffs
- removed stale copied comment from roundoff initialisation.
- clarified "separation" to mean "separation for ordering purposes" in commit
message.
- added comment that newly activated, clean, empty iclogs have a LSN of 0 so are
captured by the "iclog lsn < start_lsn" case that avoids needing to wait
before releasing the commit iclog to be written.
- added async cache flush infrastructure
- convert CIL checkpoint push work it issue an unconditional metadata device
cache flush rather than asking the first iclog write to issue it via
REQ_PREFLUSH.
- cleaned up xlog_write() to remove a redundant parameter and prepare the logic
for setting flags on the iclog based on the type of operational data is being
written to the log.
- added XLOG_ICL_NEED_FUA flag to complement the NEED_FLUSH flag, allowing
callers to issue explicit flushes and clear the NEED_FLUSH flag before the
iclog is written without dropping the REQ_FUA requirement in /dev/null...
- added CIL commit-in-start-iclog optimisation that clears the NEED_FLUSH flag
to avoid an unnecessary cache flush when issuing the iclog.
- fixed typo in CIL throttle bugfix comment.
- fixed trailing whitespace in commit message.