Flexible I/O Tester development
 help / color / mirror / Atom feed
From: "Vaughan" <cxt9401@163.com>
To: 'Sitsofe Wheeler' <sitsofe@gmail.com>
Cc: 'fio' <fio@vger.kernel.org>
Subject: 答复: How to ensure split verification will generate the same configs as write phase?
Date: Wed, 28 Feb 2018 16:44:59 +0800	[thread overview]
Message-ID: <000101d3b070$6b165dd0$41431970$@163.com> (raw)
In-Reply-To: <CALjAwxgYcdTe68JPjo6fJHOYc=nD_Tb8Tyd995D1ONbQx3Lvjg@mail.gmail.com>

> ^^^ Here's a problem - you can't randomly write and then verify sequentially via a different run. In your case you need rw=randread.
Yes. It's my fault to paste the original content of the buggy job file. I understand that rw= option should be paired with or without rand- prefix.

> The  randseed is the same across different invocations of fio unless you say otherwise (see randrepeat - http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-randrepeat
> ). So even if you weren't using a verification state file you will re-generate the same pattern if the rest of the job is the same.
Thank you for the explanation. A deep question about the seed comes:) Though randseed is the same across different invocation of fio, is it different for different sections in one job file?

I made another job file as below:
myjob.fio:
	[global] 
	bs=4k    
	direct=1 
	ioengine=sync
	verify=crc32c  
	filename=/dev/sdf
	io_size=10k

	[write-phase]  
	rw=randwrite
	do_verify=0
	write_iolog=iolog

	[verify-phase]
	stonewall
	rw=randread
	do_verify=1

I think these two sections have the same settings except the date direction. So I expect the later one can verify the write-phase. But it failed.	
run it with --debug, and found the offset generated are not the same, so verify abort with error for the first time, as below: 
	random   24382 off rand 259043585
	io       24382 fill_io_u: io_u 0x25ad740: off=71233536/len=4096/ddir=1//dev/sdf
	random   24382 off rand 3179521932
	io       24382 fill_io_u: io_u 0x25ad740: off=874364928/len=4096/ddir=1//dev/sdf
	random   24382 off rand 3621444214
	io       24382 fill_io_u: io_u 0x25ad740: off=995893248/len=4096/ddir=1//dev/sdf
	random   24388 off rand 1094293889
	io       24388 fill_io_u: io_u 0x25ad500: off=300929024/len=4096/ddir=0//dev/sdf
You can see [24382] is write-phase and [24388] is verify-phase, and the first round of them use different seed I suppose.

I search in the source code, and found the following snippet:
	static int setup_random_seeds(struct thread_data *td)
	{
	...
			for (i = 0; i < FIO_RAND_NR_OFFS; i++) {
					td->rand_seeds[i] = seed * td->thread_number + i;
					seed *= 0x9e370001UL;
			}               

Is td->thread_number different for different sections? I thought they both equal to 1 before.

Later, I realize split verify using two sections in one file is not a formal way? Is it better to just write only one write section, and use 'fio --verify-only jobfile' to do the split verification as the updated one below?

myjob.fio updated:
	[global] 
	bs=4k    
	direct=1 
	ioengine=sync
	verify=crc32c  
	filename=/dev/sdf
	io_size=10k

	[write-phase]  
	rw=randwrite
	do_verify=0

Thank you very much again!

Regards,
Vaughan



  reply	other threads:[~2018-02-28  8:45 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-27 10:45 How to ensure split verification will generate the same configs as write phase? Vaughan
2018-02-27 11:59 ` Sitsofe Wheeler
2018-02-28  8:44   ` Vaughan [this message]
2018-02-28 16:15     ` 答复: " Sitsofe Wheeler
2018-03-09  6:11       ` 答复: " Vaughan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='000101d3b070$6b165dd0$41431970$@163.com' \
    --to=cxt9401@163.com \
    --cc=fio@vger.kernel.org \
    --cc=sitsofe@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox