From: Jens Axboe <axboe@kernel.dk>
To: Pete Wyckoff <pw@padd.com>, fio@vger.kernel.org
Subject: Re: two jobs sometimes ignore time_based=1
Date: Mon, 09 Jun 2014 09:56:41 -0600 [thread overview]
Message-ID: <5395D939.5010506@kernel.dk> (raw)
In-Reply-To: <20140603210141.GA29246@padd.com>
On 2014-06-03 15:01, Pete Wyckoff wrote:
> Running fio-2.1.9-1-g0aa0 (with one trivial build patch)
> built on debian sid, deployed on centos 6.5.
>
> No command line args, just this one jobfile:
>
> [global]
> filename=test/mnt/80g
> readwrite=randwrite
> fallocate=none
> blocksize=32k
> ioengine=libaio
> direct=1
> time_based=1
>
> [big-random]
> description=Big random writes, 32k, 80 GB
> filesize=80g
> iodepth=8
> runtime=10
> write_bw_log=big-random
> write_lat_log=big-random
>
> [small-hotspot]
> description=Small hotspot, 32k, 500 MB
> filesize=500m
> iodepth=32
> runtime=10
> write_bw_log=small-hotspot
> write_lat_log=small-hotspot
>
> I want it two run the two jobs concurrently, both exiting after
> 10 seconds regardless of the data volumes written. The filename
> does not exist when I start fio.
>
> Most of the time it just works, and produces output like this:
>
> unix$ rm test/mnt/80g ; ./fio-2.1.9 jobfile
> big-random: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=8
> small-hotspot: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
> fio-2.1.9-1-g0aa0
> Starting 2 processes
> big-random: Laying out IO file(s) (1 file(s) / 81920MB)
> small-hotspot: Laying out IO file(s) (1 file(s) / 500MB)
> Jobs: 2 (f=2): [ww] [30.0% done] [0KB/82560KB/0KB /s] [0/2580/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [40.0% done] [0KB/144.4MB/0KB /s] [0/4618/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [50.0% done] [0KB/135.1MB/0KB /s] [0/4351/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [60.0% done] [0KB/162.3MB/0KB /s] [0/5193/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [70.0% done] [0KB/164.4MB/0KB /s] [0/5259/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [80.0% done] [0KB/165.3MB/0KB /s] [0/5288/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [90.0% done] [0KB/100.8MB/0KB /s] [0/3224/0 iops] [eta 00m:0Jobs: 2 (f=2): [ww] [100.0% done] [0KB/26880KB/0KB /s] [0/840/0 iops] [eta 00m:00s]
> big-random: (groupid=0, jobs=1): err= 0: pid=9572: Tue Jun 3 13:55:49 2014
>
> But sometimes it doesn't stop both jobs at 10 seconds. It seems the
> first one keeps going.
>
> Jobs: 2 (f=2): [ww] [30.0% done] [0KB/158.7MB/0KB /s] [0/5058/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [40.0% done] [0KB/140.6MB/0KB /s] [0/4496/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [50.0% done] [0KB/121.9MB/0KB /s] [0/3900/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [60.0% done] [0KB/165.1MB/0KB /s] [0/5309/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [70.0% done] [0KB/130.3MB/0KB /s] [0/4160/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [80.0% done] [0KB/96480KB/0KB /s] [0/3015/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [90.0% done] [0KB/119.1MB/0KB /s] [0/3839/0 iops] [eta 00m:0
> Jobs: 2 (f=2): [ww] [0.3% done] [0KB/72352KB/0KB /s] [0/2261/0 iops] [eta 55m:26
> Jobs: 1 (f=1): [w_] [57.9% done] [0KB/73536KB/0KB /s] [0/2298/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [63.2% done] [0KB/84384KB/0KB /s] [0/2637/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [68.4% done] [0KB/86208KB/0KB /s] [0/2694/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [73.7% done] [0KB/57446KB/0KB /s] [0/1795/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [78.9% done] [0KB/50432KB/0KB /s] [0/1576/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [84.2% done] [0KB/85600KB/0KB /s] [0/2675/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [89.5% done] [0KB/80480KB/0KB /s] [0/2515/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [94.7% done] [0KB/84192KB/0KB /s] [0/2631/0 iops] [eta 00m:0
> Jobs: 1 (f=1): [w_] [100.0% done] [0KB/85888KB/0KB /s] [0/2684/0 iops] [eta 00m:
> Jobs: 1 (f=1): [w_] [1.1% done] [0KB/53728KB/0KB /s] [0/1679/0 iops] [eta 28m:44
> Jobs: 1 (f=1): [w_] [1.2% done] [0KB/53162KB/0KB /s] [0/1661/0 iops] [eta 28m:35
> Jobs: 1 (f=1): [w_] [1.3% done] [0KB/59908KB/0KB /s] [0/1872/0 iops] [eta 28m:16
> Jobs: 1 (f=1): [w_] [1.3% done] [0KB/37280KB/0KB /s] [0/1165/0 iops] [eta 28m:32
> Jobs: 1 (f=1): [w_] [1.4% done] [0KB/31488KB/0KB /s] [0/984/0 iops] [eta 28m:55s
> Jobs: 1 (f=1): [w_] [1.4% done] [0KB/37888KB/0KB /s] [0/1184/0 iops] [eta 29m:08
> Jobs: 1 (f=1): [w_] [1.5% done] [0KB/51200KB/0KB /s] [0/1600/0 iops] [eta 29m:02
> Jobs: 1 (f=1): [w_] [1.5% done] [0KB/40631KB/0KB /s] [0/1269/0 iops] [eta 29m:10...
>
> There's the first 10 sec chunk with both jobs. Then a second 10
> sec in there with just job#1, then another half hour of only job#1.
> Looking at the ETAs and average throughput, that's about as long
> as it would take to fill the entire 80 GB disk.
>
> Could it be that fio gets confused about time_based=1, and
> tries to run to completion? But only sometimes. This often
> works and I use it for regular testing (at longer runs, like
> 600 sec).
I'll take a look at this, I'm pretty sure it's a recent regression. If
you were so inclined, would be great if you could check if 2.1.8 or
2.1.7 were affected and bisect your way to it...
--
Jens Axboe
next prev parent reply other threads:[~2014-06-09 15:56 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-06-03 21:01 two jobs sometimes ignore time_based=1 Pete Wyckoff
2014-06-09 15:56 ` Jens Axboe [this message]
2014-06-09 19:04 ` Pete Wyckoff
2014-06-09 19:26 ` Jens Axboe
2014-06-09 20:04 ` Pete Wyckoff
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5395D939.5010506@kernel.dk \
--to=axboe@kernel.dk \
--cc=fio@vger.kernel.org \
--cc=pw@padd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.