[RFC v6 PATCH 00/10] xfs: automatic relogging experiment

Brian Foster
2020-04-06T12:36:41Z

Hi all,

Here's a v6 of the automatic relogging RFC. The primary difference in
this version is a rework of the relog reservation model based on design
discussion on v5. Rather than roll existing transactions and initialize
a fixed size relog transaction, this version acquires worst case relog
reservation directly from the transactions that enable relog of log
items. This facilitates construction of arbitrary combinations of items
in the relog transaction without risk of deadlock due to log reservation
and/or lock ordering. See the associated commit log for further details
on the approach.

Beyond that, there are various other fixes and tweaks from v5. For
example, the log item relog callback is used unconditionally and
abstracts more of the reservation management code, buffer relogging is a
bit more restrictive and reliable, various helpers are refactored,
freeze is partly addressed, etc.

With regard to testing, this version survives a 1+ hour 80xcpu fsstress
workload with random buffer relogging enabled without any notable issues
and without observable reservation leaks. It also survives an fstests
auto run without regression.

Patches 1-5 are preparatory patches and core mechanism. Patch 6 uses
relogging to address the longstanding quotaoff deadlock problem. Patches
7-10 provide buffer relogging support and test code for DEBUG mode
kernels to stress the relog mechanism via random buffer relogging.

Thoughts, reviews, flames appreciated.

Brian

rfcv6:
- Rework relog reservation model.
- Drop unnecessary log ticket t_task fix.
- Use ->iop_relog() callback unconditionally.
- Rudimentary freeze handling for random buffer relogging.
- Various other fixes, tweaks and cleanups.
rfcv5: https://lore.kernel.org/linux-xfs/20200227134321.7238-1-bfoster@redhat.com/
- More fleshed out design to prevent log reservation deadlock and
locking problems.
- Split out core patches between pre-reservation management, relog item
state management and relog mechanism.
- Added experimental buffer relogging capability.
rfcv4: https://lore.kernel.org/linux-xfs/20191205175037.52529-1-bfoster@redhat.com/
- AIL based approach.
rfcv3: https://lore.kernel.org/linux-xfs/20191125185523.47556-1-bfoster@redhat.com/
- CIL based approach.
rfcv2: https://lore.kernel.org/linux-xfs/20191122181927.32870-1-bfoster@redhat.com/
- Different approach based on workqueue and transaction rolling.
rfc: https://lore.kernel.org/linux-xfs/20191024172850.7698-1-bfoster@redhat.com/

Brian Foster (10):
xfs: automatic relogging item management
xfs: create helper for ticket-less log res ungrant
xfs: extra runtime reservation overhead for relog transactions
xfs: relog log reservation stealing and accounting
xfs: automatic log item relog mechanism
xfs: automatically relog the quotaoff start intent
xfs: prevent fs freeze with outstanding relog items
xfs: buffer relogging support prototype
xfs: create an error tag for random relog reservation
xfs: relog random buffers based on errortag

fs/xfs/libxfs/xfs_errortag.h | 4 +-
fs/xfs/libxfs/xfs_shared.h | 1 +
fs/xfs/xfs_buf.c | 4 +
fs/xfs/xfs_buf_item.c | 52 ++++++++++++-
fs/xfs/xfs_dquot_item.c | 26 +++++++
fs/xfs/xfs_error.c | 3 +
fs/xfs/xfs_log.c | 35 +++++++--
fs/xfs/xfs_log.h | 4 +-
fs/xfs/xfs_log_cil.c | 2 +-
fs/xfs/xfs_log_priv.h | 1 +
fs/xfs/xfs_qm_syscalls.c | 12 ++-
fs/xfs/xfs_super.c | 4 +
fs/xfs/xfs_trace.h | 4 +
fs/xfs/xfs_trans.c | 75 ++++++++++++++++--
fs/xfs/xfs_trans.h | 36 ++++++++-
fs/xfs/xfs_trans_ail.c | 146 ++++++++++++++++++++++++++++++++++-
fs/xfs/xfs_trans_buf.c | 73 ++++++++++++++++++
fs/xfs/xfs_trans_priv.h | 20 +++++
18 files changed, 477 insertions(+), 25 deletions(-)

--
2.21.1