All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: zlang@redhat.com, djwong@kernel.org
Cc: fstests@vger.kernel.org, linux-xfs@vger.kernel.org
Subject: [PATCH 03/17] logwrites: warn if we don't think read after discard returns zeroes
Date: Fri, 22 Nov 2024 08:51:14 -0800	[thread overview]
Message-ID: <173229420060.358248.11054238752146807489.stgit@frogsfrogsfrogs> (raw)
In-Reply-To: <173229419991.358248.8516467437316874374.stgit@frogsfrogsfrogs>

From: Darrick J. Wong <djwong@kernel.org>

The logwrites replay program expects that it can issue a DISCARD against
the block device passed to _log_writes_init and that will cause all
subsequent reads to return zeroes.  This is required for correct log
recovery on filesystems such as XFS that skip recovering buffers if
newer ones are found on disk.

Unfortunately, there's no way to discover if a device's discard
implementation actually guarantees zeroes.  There used to be a sysfs
knob keyed to an allowlist, but it is now hardwired to return 0.  So
either we need a magic device that does discard-and-zero, or we need to
do the zeroing ourselves.  The logwrites program does its own zeroing if
there is no discard support, and some tests do their own zeroing.

The only devices we know to work reliably are the software defined ones
that are provided by the kernel itself -- which means dm-thinp.  Warn if
we have a device that supports discard that isn't thinp and the test
fails.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/dmlogwrites |   31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)


diff --git a/common/dmlogwrites b/common/dmlogwrites
index c1c85de9dd43ac..24a8a25ace277f 100644
--- a/common/dmlogwrites
+++ b/common/dmlogwrites
@@ -59,6 +59,35 @@ _require_log_writes_dax_mountopt()
 	fi
 }
 
+_log_writes_check_bdev()
+{
+	local sysfs="/sys/block/$(_short_dev $1)"
+
+	# Some filesystems (e.g. XFS) optimize log recovery by assuming that
+	# they can elide replay of metadata blocks if the block has a higher
+	# log serial number than the transaction being recovered.  This is a
+	# problem if the filesystem log contents can go back in time, which is
+	# what the logwrites replay program does.
+	#
+	# The logwrites replay program begins by erasing the block device's
+	# contents.  This can be done very quickly with DISCARD provided the
+	# device guarantees that all reads after a DISCARD return zeroes, or
+	# very slowly by writing zeroes to the device.  Fast is preferable, but
+	# there's no longer any way to detect that DISCARD actually unmaps
+	# zeroes, so warn the user about this requirement if the test happens
+	# to fail.
+
+	# No discard support means the logwrites will do its own zeroing
+	test "$(cat "$sysfs/queue/discard_max_bytes")" -eq 0 && return
+
+	# dm-thinp guarantees that reads after discards return zeroes
+	dmsetup status "$blkdev" 2>/dev/null | grep -q '^0.* thin ' && return
+
+	echo "HINT: $blkdev doesn't guarantee that reads after DISCARD will return zeroes" >> $seqres.hints
+	echo "      This is required for correct journal replay on some filesystems (e.g. xfs)" >> $seqres.hints
+	echo >> $seqres.hints
+}
+
 # Set up a dm-log-writes device
 #
 # blkdev: the specified target device
@@ -84,6 +113,8 @@ _log_writes_init()
 	LOGWRITES_NAME=logwrites-test
 	LOGWRITES_DMDEV=/dev/mapper/$LOGWRITES_NAME
 	LOGWRITES_TABLE="0 $BLK_DEV_SIZE log-writes $blkdev $LOGWRITES_DEV"
+
+	_log_writes_check_bdev "$blkdev"
 	_dmsetup_create $LOGWRITES_NAME --table "$LOGWRITES_TABLE" || \
 		_fail "failed to create log-writes device"
 }


  parent reply	other threads:[~2024-11-22 16:51 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-22 16:50 [PATCHSET v2] fstests: random fixes for v2024.11.17 Darrick J. Wong
2024-11-22 16:50 ` [PATCH 01/17] generic/757: fix various bugs in this test Darrick J. Wong
2024-11-22 16:50 ` [PATCH 02/17] generic/757: convert to thinp Darrick J. Wong
2024-11-25  5:10   ` Christoph Hellwig
2024-11-22 16:51 ` Darrick J. Wong [this message]
2024-11-25  5:12   ` [PATCH 03/17] logwrites: warn if we don't think read after discard returns zeroes Christoph Hellwig
2024-11-25 17:19     ` Darrick J. Wong
2024-11-22 16:51 ` [PATCH 04/17] logwrites: use BLKZEROOUT if it's available Darrick J. Wong
2024-11-22 16:51 ` [PATCH 05/17] logwrites: only use BLKDISCARD if we know discard zeroes data Darrick J. Wong
2024-11-22 16:52 ` [PATCH 06/17] xfs/113: fix failure to corrupt the entire directory Darrick J. Wong
2024-11-25  5:12   ` Christoph Hellwig
2024-11-22 16:52 ` [PATCH 07/17] xfs/508: fix test for 64k blocksize Darrick J. Wong
2024-11-25  5:13   ` Christoph Hellwig
2024-11-22 16:52 ` [PATCH 08/17] common/rc: capture dmesg when oom kills happen Darrick J. Wong
2024-11-25  5:13   ` Christoph Hellwig
2024-11-22 16:52 ` [PATCH 09/17] generic/562: handle ENOSPC while cloning gracefully Darrick J. Wong
2024-11-25  5:14   ` Christoph Hellwig
2024-11-25  5:16     ` Darrick J. Wong
2024-11-25  5:20       ` Christoph Hellwig
2024-11-26  1:26         ` Darrick J. Wong
2024-11-22 16:53 ` [PATCH 10/17] xfs/163: skip test if we can't shrink due to enospc issues Darrick J. Wong
2024-11-22 16:53 ` [PATCH 11/17] xfs/009: allow logically contiguous preallocations Darrick J. Wong
2024-11-22 16:53 ` [PATCH 12/17] generic/251: use sentinel files to kill the fstrim loop Darrick J. Wong
2024-11-22 16:53 ` [PATCH 13/17] generic/251: constrain runtime via time/load/soak factors Darrick J. Wong
2024-11-25  5:15   ` Christoph Hellwig
2024-11-22 16:54 ` [PATCH 14/17] generic/251: don't copy the fsstress source code Darrick J. Wong
2024-11-25  5:15   ` Christoph Hellwig
2024-11-22 16:54 ` [PATCH 15/17] common/rc: _scratch_mkfs_sized supports extra arguments Darrick J. Wong
2024-11-22 16:54 ` [PATCH 16/17] xfs/157: do not drop necessary mkfs options Darrick J. Wong
2024-11-22 16:54 ` [PATCH 17/17] generic/366: fix directio requirements checking Darrick J. Wong
2024-11-25  5:16   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=173229420060.358248.11054238752146807489.stgit@frogsfrogsfrogs \
    --to=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=zlang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.