From: Ankit Kumar <ankit.kumar@samsung.com>
To: axboe@kernel.dk, vincentfu@gmail.com
Cc: fio@vger.kernel.org, Ankit Kumar <ankit.kumar@samsung.com>
Subject: [PATCH 08/21] verify: header seed check for read only workloads
Date: Thu, 27 Feb 2025 16:17:14 +0530 [thread overview]
Message-ID: <20250227104727.1794982-9-ankit.kumar@samsung.com> (raw)
In-Reply-To: <20250227104727.1794982-1-ankit.kumar@samsung.com>
For read jobs, users should have the option to verify header seeds at a
later point of time. Currently for read jobs header seeds are not
generated
Consider the below mentioned write followed by read workloads. Here fio
should allow header seed verification.
fio --name=test --filesize=16k --rw=randwrite --verify=md5
fio --name=test --filesize=16k --rw=randread --verify=md5 --verify_header_seed=1
However there are other scenarios where header seed verification will
fail. These include:
* randrepeat is set to false, leading to different seed across runs.
* randseed is different across write and read workloads.
* Read workload is changed from sequential to random or vice versa
across runs.
* Read workloads run in the same invocation as write, i.e. a write job
followed by a stonewall read job. Header seed verification will fail
because random seeds vary between jobs. Refer t/jobs/t0029.fio
If verify_header_seed is explicitly enabled, fio will verify header seed
for the workload.
This reverts part of commit mentioned below
Fixes: def41e55 ("verify: decouple seed generation from buffer fill")
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
---
HOWTO.rst | 16 ++++++++++++----
backend.c | 11 +++++++++++
fio.1 | 15 +++++++++++----
3 files changed, 34 insertions(+), 8 deletions(-)
diff --git a/HOWTO.rst b/HOWTO.rst
index 05bdeebb..6d462af3 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -3907,10 +3907,18 @@ Verification
:option:`ioengine`\=null, not for much else.
This option can be used for repeated burn-in tests of a system to make sure
- that the written data is also correctly read back. If the data direction
- given is a read or random read, fio will assume that it should verify a
- previously written file. If the data direction includes any form of write,
- the verify will be of the newly written data.
+ that the written data is also correctly read back.
+
+ If the data direction given is a read or random read, fio will assume that
+ it should verify a previously written file. In this scenario fio will not
+ verify the block number written in the header. The header seed won't be
+ verified, unless its explicitly requested by setting
+ :option:`verify_header_seed`. Note in this scenario the header seed check
+ will only work if the read invocation exactly matches the original write
+ invocation.
+
+ If the data direction includes any form of write, the verify will be of the
+ newly written data.
To avoid false verification errors, do not use the norandommap option when
verifying data with async I/O engines and I/O depths > 1. Or use the
diff --git a/backend.c b/backend.c
index f3e5b56a..f5cfffdb 100644
--- a/backend.c
+++ b/backend.c
@@ -1069,6 +1069,17 @@ static void do_io(struct thread_data *td, uint64_t *bytes_done)
if (td->o.verify != VERIFY_NONE && io_u->ddir == DDIR_READ &&
((io_u->flags & IO_U_F_VER_LIST) || !td_rw(td))) {
+ /*
+ * For read only workloads generate the seed. This way
+ * we can still verify header seed at any later
+ * invocation.
+ */
+ if (!td_write(td) && !td->o.verify_pattern_bytes) {
+ io_u->rand_seed = __rand(&td->verify_state);
+ if (sizeof(int) != sizeof(long *))
+ io_u->rand_seed *= __rand(&td->verify_state);
+ }
+
if (verify_state_should_stop(td, io_u)) {
put_io_u(td, io_u);
break;
diff --git a/fio.1 b/fio.1
index 1342a23a..3fbc1657 100644
--- a/fio.1
+++ b/fio.1
@@ -3634,10 +3634,17 @@ Only pretend to verify. Useful for testing internals with
.RE
.P
This option can be used for repeated burn\-in tests of a system to make sure
-that the written data is also correctly read back. If the data direction
-given is a read or random read, fio will assume that it should verify a
-previously written file. If the data direction includes any form of write,
-the verify will be of the newly written data.
+that the written data is also correctly read back.
+.P
+If the data direction given is a read or random read, fio will assume that it
+should verify a previously written file. In this scenario fio will not verify
+the block number written in the header. The header seed won't be verified,
+unless its explicitly requested by setting \fBverify_header_seed\fR option.
+Note in this scenario the header seed check will only work if the read
+invocation exactly matches the original write invocation.
+.P
+If the data direction includes any form of write, the verify will be of the
+newly written data.
.P
To avoid false verification errors, do not use the norandommap option when
verifying data with async I/O engines and I/O depths > 1. Or use the
--
2.25.1
next prev parent reply other threads:[~2025-02-27 7:04 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20250227053350epcas5p31b3b5b47590d70804dade59842d87b27@epcas5p3.samsung.com>
2025-02-27 10:47 ` [PATCH 00/21] verify fixes and a new test suite Ankit Kumar
2025-02-27 7:28 ` fiotestbot
2025-02-27 10:47 ` [PATCH 01/21] filesetup: remove unnecessary check Ankit Kumar
2025-02-27 10:47 ` [PATCH 02/21] verify: add missing client/server support for verify_write_sequence Ankit Kumar
2025-02-27 10:47 ` [PATCH 03/21] init: write sequence behavior change for verify_only mode Ankit Kumar
2025-02-27 10:47 ` [PATCH 04/21] fio: add verify_header_seed option Ankit Kumar
2025-02-27 10:47 ` [PATCH 05/21] verify: disable header seed checking instead of overwriting it Ankit Kumar
2025-02-27 10:47 ` [PATCH 06/21] verify: enable header seed check for 100% write jobs Ankit Kumar
2025-02-27 10:47 ` [PATCH 07/21] verify: disable header seed check for verify_only jobs Ankit Kumar
2025-02-27 10:47 ` Ankit Kumar [this message]
2025-02-27 10:47 ` [PATCH 09/21] verify: fix verify issues with norandommap Ankit Kumar
2025-02-27 10:47 ` [PATCH 10/21] verify: disable write sequence checks with norandommap and iodepth > 1 Ankit Kumar
2025-02-27 10:47 ` [PATCH 11/21] backend: fix verify issue during readwrite Ankit Kumar
2025-02-27 10:47 ` [PATCH 12/21] init: fixup verify_offset option Ankit Kumar
2025-02-27 10:47 ` [PATCH 13/21] verify: fix verify issue with offest modifiers Ankit Kumar
2025-02-27 10:47 ` [PATCH 14/21] verify: adjust fio_offset_overlap_risk to include randommap Ankit Kumar
2025-02-27 10:47 ` [PATCH 15/21] t/fiotestcommon: do not require nvmecdev argument for Requirements Ankit Kumar
2025-02-27 10:47 ` [PATCH 16/21] t/fiotestlib: improve JSON decoding Ankit Kumar
2025-02-27 10:47 ` [PATCH 17/21] t/fiotestlib: display stderr size when it is not empty but should be Ankit Kumar
2025-02-27 10:47 ` [PATCH 18/21] t/verify.py: Add verify test script Ankit Kumar
2025-02-27 10:47 ` [PATCH 19/21] t/fiotestcommon: add a success pattern for long tests Ankit Kumar
2025-02-27 10:47 ` [PATCH 20/21] t/run-fio-test: add t/verify.py Ankit Kumar
2025-02-27 10:47 ` [PATCH 21/21] ci: add nightly test for verify Ankit Kumar
2025-02-27 16:54 ` [PATCH 00/21] verify fixes and a new test suite Vincent Fu
2025-03-05 12:36 ` Shinichiro Kawasaki
2025-03-06 19:13 ` Vincent Fu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250227104727.1794982-9-ankit.kumar@samsung.com \
--to=ankit.kumar@samsung.com \
--cc=axboe@kernel.dk \
--cc=fio@vger.kernel.org \
--cc=vincentfu@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox