[PATCH 0/3] three patches

[PATCH 0/3] three patches

[vincentfu]

From: Vincent Fu <vincent.fu@wdc.com>

Jens, please consider this patch series.

The LFSR error message and test script error handling changes I believe
should be uncontroversial.

Regarding the zonemode=strided patch, it will break job files that
depend on the old buggy behavior, but that behavior is so inconsistent
with the documentation that I think this fix is warranted.

Vincent

Vincent Fu (3):
  filesetup: improve LFSR init failure error message
  io_u: move to next zone even if zoneskip is unset
  t/run-fio-tests: improve error handling

 filesetup.c        |  3 +++
 io_u.c             |  3 +--
 t/run-fio-tests.py | 10 ++++++----
 t/strided.py       |  3 +--
 4 files changed, 11 insertions(+), 8 deletions(-)

-- 
2.17.1

[PATCH 1/3] filesetup: improve LFSR init failure error message

[vincentfu]

From: Vincent Fu <vincent.fu@wdc.com>

Especially with small sample spaces, the LFSR random generator
occasionally fails to initialize successfully. When this occurs, the
error message refers to problems allocating a random map. Change the
error message to explicitly mention the LFSR failure.

OLD BEHAVIOR
$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-13-g998b-dirty
Starting 1 process
fio: failed allocating random map. If running a large number of jobs, try the 'norandommap' option or set 'softrandommap'. Or give a larger --alloc-size to fio.

NEW BEHAVIOR
$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-37-g65ee
Starting 1 process
fio: failed initializing LFSR
---
 filesetup.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/filesetup.c b/filesetup.c
index 1d3094c1..7d54c9f1 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -1354,6 +1354,9 @@ bool init_random_map(struct thread_data *td)
 			if (!lfsr_init(&f->lfsr, blocks, seed, 0)) {
 				fio_file_set_lfsr(f);
 				continue;
+			} else {
+				log_err("fio: failed initializing LFSR\n");
+				return false;
 			}
 		} else if (!td->o.norandommap) {
 			f->io_axmap = axmap_new(blocks);
-- 
2.17.1

[PATCH 2/3] io_u: move to next zone even if zoneskip is unset

[vincentfu]

From: Vincent Fu <vincent.fu@wdc.com>

This patch makes fio's behavior under zonemode=strided conform to the
documentation:

    I/O happens in a single zone until zonesize bytes have been
    transferred. After that number of bytes has been transferred
    processing of the next zone starts.

With zonemode=strided, before commit 35f561eb, fio would only move to
the next zone when zoneskip > 0 and zonesize bytes were written. There
would always be zoneskip bytes between the end of one zone and the
beginning of the next zone. If zoneskip was not set or set to 0, all IO
would happen in the first zone.

Commit 35f561eb changed this so that fio would move to the next zone
upon writing zonesize bytes if zoneskip was explicitly set to a value >=
0. This option made it possible for zones to be contiguous. The
documentation was not updated to reflect the new behavior.

I originally intended to submit a patch to update fio's documentation,
but upon further reflection it seems better to change fio's behavior and
have a clean user interface than to change the documentation to note
that zoneskip must be explciitly set in order for fio to move to the
next zone.

This patch also updates t/strided.py to reflect the new behavior.
---
 io_u.c       | 3 +--
 t/strided.py | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/io_u.c b/io_u.c
index 5cbbe85a..b5c31335 100644
--- a/io_u.c
+++ b/io_u.c
@@ -850,8 +850,7 @@ static void setup_strided_zone_mode(struct thread_data *td, struct io_u *io_u)
 	/*
 	 * See if it's time to switch to a new zone
 	 */
-	if (td->zone_bytes >= td->o.zone_size &&
-			fio_option_is_set(&td->o, zone_skip)) {
+	if (td->zone_bytes >= td->o.zone_size) {
 		td->zone_bytes = 0;
 		f->file_offset += td->o.zone_range + td->o.zone_skip;
 
diff --git a/t/strided.py b/t/strided.py
index c159dc0b..aac15d10 100755
--- a/t/strided.py
+++ b/t/strided.py
@@ -22,7 +22,7 @@
 #
 # ===TEST MATRIX===
 #
-# --zonemode=strided, zoneskip >= 0
+# --zonemode=strided, zoneskip unset
 #   w/ randommap and LFSR
 #       zonesize=zonerange  all blocks in zonerange touched
 #       zonesize>zonerange  all blocks touched and roll-over back into zone
@@ -57,7 +57,6 @@ def run_fio(fio, test, index):
                 "--log_offset=1",
                 "--randrepeat=0",
                 "--rw=randread",
-                "--zoneskip=0",
                 "--write_iops_log={0}{1:03d}".format(filename, index),
                 "--output={0}{1:03d}.out".format(filename, index),
                 "--zonerange={zonerange}".format(**test),
-- 
2.17.1

[PATCH 3/3] t/run-fio-tests: improve error handling

[vincentfu]

From: Vincent Fu <vincent.fu@wdc.com>

More gracefully handle errors when running a test. Also change the
EXAMPLE to refer to fio's canonical git repository.
---
 t/run-fio-tests.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/t/run-fio-tests.py b/t/run-fio-tests.py
index 1b8ca0a2..cf8de093 100755
--- a/t/run-fio-tests.py
+++ b/t/run-fio-tests.py
@@ -14,7 +14,7 @@
 #
 #
 # EXAMPLE
-# # git clone [fio-repository]
+# # git clone git://git.kernel.dk/fio.git
 # # cd fio
 # # make -j
 # # python3 t/run-fio-tests.py
@@ -123,6 +123,7 @@ class FioExeTest(FioTest):
         stderr_file = open(self.stderr_file, "w+")
         exticode_file = open(self.exticode_file, "w+")
         try:
+            proc = None
             # Avoid using subprocess.run() here because when a timeout occurs,
             # fio will be stopped with SIGKILL. This does not give fio a
             # chance to clean up and means that child processes may continue
@@ -142,9 +143,10 @@ class FioExeTest(FioTest):
             assert proc.poll()
             self.output['failure'] = 'timeout'
         except Exception:
-            if not proc.poll():
-                proc.terminate()
-                proc.communicate()
+            if proc:
+                if not proc.poll():
+                    proc.terminate()
+                    proc.communicate()
             self.output['failure'] = 'exception'
             self.output['exc_info'] = sys.exc_info()
         finally:
-- 
2.17.1

Re: [PATCH 0/3] three patches

[Jens Axboe]

On 11/11/19 9:30 AM, vincentfu@gmail.com wrote:
> From: Vincent Fu <vincent.fu@wdc.com>
> 
> Jens, please consider this patch series.
> 
> The LFSR error message and test script error handling changes I believe
> should be uncontroversial.
> 
> Regarding the zonemode=strided patch, it will break job files that
> depend on the old buggy behavior, but that behavior is so inconsistent
> with the documentation that I think this fix is warranted.

Agree, and I don't think that's a high risk.

Applied, thanks.

-- 
Jens Axboe