public inbox for fstests@vger.kernel.org
 help / color / mirror / Atom feed
From: Alexander Patrakov <patrakov@gmail.com>
To: fstests@vger.kernel.org
Cc: djwong@kernel.org, Alexander Patrakov <patrakov@gmail.com>
Subject: [PATCH v3] _require_sparse_files: rewrite as a direct test instead of a black list
Date: Tue, 19 Dec 2023 04:57:20 +0800	[thread overview]
Message-ID: <20231218205720.3498-1-patrakov@gmail.com> (raw)
In-Reply-To: <CAN_LGv2u5qRq_Zmp+qPGXz-PXdO7Bu33CKn+ycmQX1dzbDRFvw@mail.gmail.com>

_require_sparse_files was implemented as a list of filesystems known not to
support sparse files, and therefore it missed some cases.

However, if sparse files do not work as expected during a test, the risk
is that the test will write out to the disk all the zeros that would
normally be unwritten. This amounts to at least 4 TB for the generic/129
test, and therefore there is a significant media wear-out concern here.

Adding more filesystems to the list of exclusions would not scale and
would not work anyway because CIFS backed by SAMBA is safe, while CIFS
backed by Windows Server 2022 is not (because the specific write
patterns found in generic/014 and generic/129 cause it to ignore the
otherwise-supported request to make a file sparse).

Mitigate this risk by rewriting the check as a small-scale test that
reliably triggers Windows misbehavior. The black list becomes unneeded
because the same test creates and detects non-sparse files on exfat and
hfsplus.

Signed-off-by: Alexander Patrakov <patrakov@gmail.com>
---
 common/rc | 29 +++++++++++++++++++++--------
 1 file changed, 21 insertions(+), 8 deletions(-)

diff --git a/common/rc b/common/rc
index cc92fe06..a9e0ba7e 100644
--- a/common/rc
+++ b/common/rc
@@ -2870,17 +2870,30 @@ _require_fs_space()
 #
 # Check if the filesystem supports sparse files.
 #
-# Unfortunately there is no better way to do this than a manual black list.
+# Filesystems (such as CIFS mounted from a Windows server) that generally
+# support sparse files but are tricked into creating a non-sparse file by one
+# of the tests are treated here as not supporting sparse files. This special
+# treatment is done due to media wear-out concerns -- e.g., generic/129 would
+# write multiple terabytes of zeros if allowed to run on a filesystem that
+# ignores the request to make a file sparse.
 #
 _require_sparse_files()
 {
-    case $FSTYP in
-    hfsplus|exfat)
-        _notrun "Sparse files not supported by this filesystem type: $FSTYP"
-	;;
-    *)
-        ;;
-    esac
+    local testfile="$TEST_DIR/$$.sparsefiletest"
+    rm -f "$testfile"
+
+    # The write size and offset are specifically chosen to trick the Windows
+    # SMB server implementation into dishonoring the request to create a sparse
+    # file, while still fitting into the 64 kb SMB1 maximum request size.
+    # This also creates a non-sparse file on vfat, exfat, and hfsplus.
+    $XFS_IO_PROG -f -c 'pwrite -b 51200 -S 0x61 1638400 51200' "$testfile" >/dev/null
+
+    resulting_file_size_kb=$( du -sk "$testfile" | cut -f 1 )
+    rm -f "$testfile"
+
+    # The threshold of 1 MB allows for filesystems with such large clusters.
+    [ $resulting_file_size_kb -gt 1024 ] && \
+	_notrun "Sparse files are not supported or do not work as expected"
 }
 
 _require_debugfs()
-- 
2.43.0


  reply	other threads:[~2023-12-18 20:57 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-17 15:43 Excessive media wearout caused by generic/129 Alexander E. Patrakov
2023-12-17 17:24 ` [PATCH] generic/129: add a safeguard against media wearout Alexander Patrakov
2023-12-17 18:45 ` Excessive media wearout caused by generic/129 Alexander E. Patrakov
2023-12-17 21:00   ` [PATCH v2] _require_sparse_files: add a safeguard against media wearout Alexander Patrakov
2023-12-18 17:45     ` Darrick J. Wong
2023-12-18 18:27       ` Alexander E. Patrakov
2023-12-18 20:57         ` Alexander Patrakov [this message]
2023-12-18 22:45           ` [PATCH v3] _require_sparse_files: rewrite as a direct test instead of a black list Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231218205720.3498-1-patrakov@gmail.com \
    --to=patrakov@gmail.com \
    --cc=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox