public inbox for fio@vger.kernel.org
 help / color / mirror / Atom feed
From: Ankit Kumar <ankit.kumar@samsung.com>
To: axboe@kernel.dk, vincentfu@gmail.com
Cc: fio@vger.kernel.org, Ankit Kumar <ankit.kumar@samsung.com>
Subject: [PATCH 08/21] verify: header seed check for read only workloads
Date: Thu, 27 Feb 2025 16:17:14 +0530	[thread overview]
Message-ID: <20250227104727.1794982-9-ankit.kumar@samsung.com> (raw)
In-Reply-To: <20250227104727.1794982-1-ankit.kumar@samsung.com>

For read jobs, users should have the option to verify header seeds at a
later point of time. Currently for read jobs header seeds are not
generated

Consider the below mentioned write followed by read workloads. Here fio
should allow header seed verification.

fio --name=test --filesize=16k --rw=randwrite --verify=md5
fio --name=test --filesize=16k --rw=randread --verify=md5 --verify_header_seed=1

However there are other scenarios where header seed verification will
fail. These include:
 * randrepeat is set to false, leading to different seed across runs.
 * randseed is different across write and read workloads.
 * Read workload is changed from sequential to random or vice versa
   across runs.
 * Read workloads run in the same invocation as write, i.e. a write job
   followed by a stonewall read job. Header seed verification will fail
   because random seeds vary between jobs. Refer t/jobs/t0029.fio

If verify_header_seed is explicitly enabled, fio will verify header seed
for the workload.

This reverts part of commit mentioned below
Fixes: def41e55 ("verify: decouple seed generation from buffer fill")

Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
---
 HOWTO.rst | 16 ++++++++++++----
 backend.c | 11 +++++++++++
 fio.1     | 15 +++++++++++----
 3 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/HOWTO.rst b/HOWTO.rst
index 05bdeebb..6d462af3 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -3907,10 +3907,18 @@ Verification
 			:option:`ioengine`\=null, not for much else.
 
 	This option can be used for repeated burn-in tests of a system to make sure
-	that the written data is also correctly read back. If the data direction
-	given is a read or random read, fio will assume that it should verify a
-	previously written file. If the data direction includes any form of write,
-	the verify will be of the newly written data.
+	that the written data is also correctly read back.
+
+	If the data direction given is a read or random read, fio will assume that
+	it should verify a previously written file. In this scenario fio will not
+	verify the block number written in the header. The header seed won't be
+	verified, unless its explicitly requested by setting
+	:option:`verify_header_seed`. Note in this scenario the header seed check
+	will only work if the read invocation exactly matches the original write
+	invocation.
+
+	If the data direction includes any form of write, the verify will be of the
+	newly written data.
 
 	To avoid false verification errors, do not use the norandommap option when
 	verifying data with async I/O engines and I/O depths > 1.  Or use the
diff --git a/backend.c b/backend.c
index f3e5b56a..f5cfffdb 100644
--- a/backend.c
+++ b/backend.c
@@ -1069,6 +1069,17 @@ static void do_io(struct thread_data *td, uint64_t *bytes_done)
 		if (td->o.verify != VERIFY_NONE && io_u->ddir == DDIR_READ &&
 		    ((io_u->flags & IO_U_F_VER_LIST) || !td_rw(td))) {
 
+			/*
+			 * For read only workloads generate the seed. This way
+			 * we can still verify header seed at any later
+			 * invocation.
+			 */
+			if (!td_write(td) && !td->o.verify_pattern_bytes) {
+				io_u->rand_seed = __rand(&td->verify_state);
+				if (sizeof(int) != sizeof(long *))
+					io_u->rand_seed *= __rand(&td->verify_state);
+			}
+
 			if (verify_state_should_stop(td, io_u)) {
 				put_io_u(td, io_u);
 				break;
diff --git a/fio.1 b/fio.1
index 1342a23a..3fbc1657 100644
--- a/fio.1
+++ b/fio.1
@@ -3634,10 +3634,17 @@ Only pretend to verify. Useful for testing internals with
 .RE
 .P
 This option can be used for repeated burn\-in tests of a system to make sure
-that the written data is also correctly read back. If the data direction
-given is a read or random read, fio will assume that it should verify a
-previously written file. If the data direction includes any form of write,
-the verify will be of the newly written data.
+that the written data is also correctly read back.
+.P
+If the data direction given is a read or random read, fio will assume that it
+should verify a previously written file. In this scenario fio will not verify
+the block number written in the header. The header seed won't be verified,
+unless its explicitly requested by setting \fBverify_header_seed\fR option.
+Note in this scenario the header seed check will only work if the read
+invocation exactly matches the original write invocation.
+.P
+If the data direction includes any form of write, the verify will be of the
+newly written data.
 .P
 To avoid false verification errors, do not use the norandommap option when
 verifying data with async I/O engines and I/O depths > 1.  Or use the
-- 
2.25.1


  parent reply	other threads:[~2025-02-27  7:04 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20250227053350epcas5p31b3b5b47590d70804dade59842d87b27@epcas5p3.samsung.com>
2025-02-27 10:47 ` [PATCH 00/21] verify fixes and a new test suite Ankit Kumar
2025-02-27  7:28   ` fiotestbot
2025-02-27 10:47   ` [PATCH 01/21] filesetup: remove unnecessary check Ankit Kumar
2025-02-27 10:47   ` [PATCH 02/21] verify: add missing client/server support for verify_write_sequence Ankit Kumar
2025-02-27 10:47   ` [PATCH 03/21] init: write sequence behavior change for verify_only mode Ankit Kumar
2025-02-27 10:47   ` [PATCH 04/21] fio: add verify_header_seed option Ankit Kumar
2025-02-27 10:47   ` [PATCH 05/21] verify: disable header seed checking instead of overwriting it Ankit Kumar
2025-02-27 10:47   ` [PATCH 06/21] verify: enable header seed check for 100% write jobs Ankit Kumar
2025-02-27 10:47   ` [PATCH 07/21] verify: disable header seed check for verify_only jobs Ankit Kumar
2025-02-27 10:47   ` Ankit Kumar [this message]
2025-02-27 10:47   ` [PATCH 09/21] verify: fix verify issues with norandommap Ankit Kumar
2025-02-27 10:47   ` [PATCH 10/21] verify: disable write sequence checks with norandommap and iodepth > 1 Ankit Kumar
2025-02-27 10:47   ` [PATCH 11/21] backend: fix verify issue during readwrite Ankit Kumar
2025-02-27 10:47   ` [PATCH 12/21] init: fixup verify_offset option Ankit Kumar
2025-02-27 10:47   ` [PATCH 13/21] verify: fix verify issue with offest modifiers Ankit Kumar
2025-02-27 10:47   ` [PATCH 14/21] verify: adjust fio_offset_overlap_risk to include randommap Ankit Kumar
2025-02-27 10:47   ` [PATCH 15/21] t/fiotestcommon: do not require nvmecdev argument for Requirements Ankit Kumar
2025-02-27 10:47   ` [PATCH 16/21] t/fiotestlib: improve JSON decoding Ankit Kumar
2025-02-27 10:47   ` [PATCH 17/21] t/fiotestlib: display stderr size when it is not empty but should be Ankit Kumar
2025-02-27 10:47   ` [PATCH 18/21] t/verify.py: Add verify test script Ankit Kumar
2025-02-27 10:47   ` [PATCH 19/21] t/fiotestcommon: add a success pattern for long tests Ankit Kumar
2025-02-27 10:47   ` [PATCH 20/21] t/run-fio-test: add t/verify.py Ankit Kumar
2025-02-27 10:47   ` [PATCH 21/21] ci: add nightly test for verify Ankit Kumar
2025-02-27 16:54   ` [PATCH 00/21] verify fixes and a new test suite Vincent Fu
2025-03-05 12:36     ` Shinichiro Kawasaki
2025-03-06 19:13       ` Vincent Fu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250227104727.1794982-9-ankit.kumar@samsung.com \
    --to=ankit.kumar@samsung.com \
    --cc=axboe@kernel.dk \
    --cc=fio@vger.kernel.org \
    --cc=vincentfu@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox