fio on AIX - fallocate error

fio on AIX - fallocate error

[Carino, Mario]

Hi.
We're trying to run fio 2.1.7 on AIX 6 and we can't seem to eliminate these errors:

fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument

we always get one or the other. It doesn't matter too much what command we run, what options,  etc. We saw a web page which indicates that fio requires an Asynch IO package to be installed and running on AIX. Is that correct? We're trying to find out if we have it installed on our test box. (We're lowly non-root DBA's so getting anything done requires a committee.)

Any ideas?

Thanks

Mario Carino

Re: fio on AIX - fallocate error

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

On 2014-04-09 20:52, Carino, Mario wrote:
> Hi.
> We're trying to run fio 2.1.7 on AIX 6 and we can't seem to eliminate these errors:
>
> fio: posix_fallocate fails: Invalid argument

This one you can ignore, or you can get rid of it by adding:

fallocate=none

to the job file.

> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument

Fio should probably just ignore these flush-cache warnings and continue 
on, it's not really a fatal condition. I have attached a patch for you, 
can you apply that and give it a go?

> we always get one or the other. It doesn't matter too much what command we run,
> what options,  etc. We saw a web page which indicates that fio requires an
> Asynch IO package to be installed and running on AIX. Is that correct? We're
 > trying to find out if we have it installed on our test box.

You'd get a different error if that was your issue.

> (We're lowly non-root
 > DBA's so getting anything done requires a committee.)

Isn't that always the case with AIX? :-)

-- 
Jens Axboe


[-- Attachment #2: fio-make-cache-flush-just-warn.patch --]
[-- Type: text/x-patch, Size: 973 bytes --]

diff --git a/filesetup.c b/filesetup.c
index abea1e60b726..e1e48203f660 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -395,9 +395,6 @@ static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f,
 	dprint(FD_IO, "invalidate cache %s: %llu/%llu\n", f->file_name, off,
 								len);
 
-	/*
-	 * FIXME: add blockdev flushing too
-	 */
 	if (f->mmap_ptr) {
 		ret = posix_madvise(f->mmap_ptr, f->mmap_sz, POSIX_MADV_DONTNEED);
 #ifdef FIO_MADV_FREE
@@ -419,6 +416,11 @@ static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f,
 	} else if (f->filetype == FIO_TYPE_CHAR || f->filetype == FIO_TYPE_PIPE)
 		ret = 0;
 
+	if (ret) {
+		log_info("fio: cache flush failed: %s\n", strerror(errno));
+		ret = 0;
+	}
+
 	if (ret < 0) {
 		td_verror(td, errno, "invalidate_cache");
 		return 1;
@@ -427,7 +429,7 @@ static int __file_invalidate_cache(struct thread_data *td, struct fio_file *f,
 		return 1;
 	}
 
-	return ret;
+	return 0;
 
 }
 

Re: fio on AIX - fallocate error

[Jens Axboe]

On 2014-04-10 08:00, Jens Axboe wrote:
> Fio should probably just ignore these flush-cache warnings and continue
> on, it's not really a fatal condition. I have attached a patch for you,
> can you apply that and give it a go?

I committed a variant of this, since I think it makes sense to do. As a 
user, there's nothing you can do to make it work, if your platform is 
missing the cache invalidation part. And it's not a fatal error 
condition, so fio will just warn and go on.

So you can git pull the update, or download the latest tar ball snapshot 
and give it another go.

-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

Hi Jens,

Well, that's the thing. We can't get rid of the fallocate error regardless of how we set fallocate= in the job file. We've tried 0,1,none,posix and keep (which seems to be invalid now) and we ALWAYS get the fallocate error. Sometimes the job continues to run, but the output file is only created at zero byte length.

Regards.


Mario Carino
AIG
Database Administrator
IT Infrastructure | AIG Property Casualty
5 Wood Hollow Rd, 3rd Floor, Parsippany, NJ 07054
Tel +1 817 490-7145 | Cell +1 551 655-1443 | Direct +1 973 331-8558 (Mon/Wed/Thu)
mario.carino@aig.com | www.aig.com


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 10:00 AM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 2014-04-09 20:52, Carino, Mario wrote:
> Hi.
> We're trying to run fio 2.1.7 on AIX 6 and we can't seem to eliminate these errors:
>
> fio: posix_fallocate fails: Invalid argument

This one you can ignore, or you can get rid of it by adding:

fallocate=none

to the job file.

> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, 
> error=Invalid argument

Fio should probably just ignore these flush-cache warnings and continue on, it's not really a fatal condition. I have attached a patch for you, can you apply that and give it a go?

> we always get one or the other. It doesn't matter too much what 
> command we run, what options,  etc. We saw a web page which indicates 
> that fio requires an Asynch IO package to be installed and running on 
> AIX. Is that correct? We're
 > trying to find out if we have it installed on our test box.

You'd get a different error if that was your issue.

> (We're lowly non-root
 > DBA's so getting anything done requires a committee.)

Isn't that always the case with AIX? :-)

--
Jens Axboe

Re: fio on AIX - fallocate error

[Jens Axboe]

On 04/10/2014 09:30 AM, Carino, Mario wrote:
> Hi Jens,
>
> Well, that's the thing. We can't get rid of the fallocate error
> regardless of how we set fallocate= in the job file. We've tried
 > 0,1,none,posix and keep (which seems to be invalid now) and we ALWAYS
 > get the fallocate error. Sometimes the job continues to run, but the
 > output file is only created at zero byte length.

Can you send me the job file you are using?

-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

Here's the job file:

[4ktest]
fallocate=1
direct=1
rw=randwrite
refill_buffers
norandommap
randrepeat=0
size=2G
bs=4k
iodepth=16
numjobs=16
runtime=60
group_reporting

This job file was created based on a sample command line I was given:

fio --filename=/oradata2/o18g2/IO/fiotest4 --direct=1 --rw=randwrite --refill_buffers --norandommap --randrepeat=0 --size=2G --bs=4k  --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest

I think I understand enough that I need to run a rw=write job first before running the above job, correct?

Mario Carino
AIG
Database Administrator
IT Infrastructure | AIG Property Casualty
5 Wood Hollow Rd, 3rd Floor, Parsippany, NJ 07054
Tel +1 817 490-7145 | Cell +1 551 655-1443 | Direct +1 973 331-8558 (Mon/Wed/Thu)
mario.carino@aig.com | www.aig.com


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 11:34 AM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 04/10/2014 09:30 AM, Carino, Mario wrote:
> Hi Jens,
>
> Well, that's the thing. We can't get rid of the fallocate error 
> regardless of how we set fallocate= in the job file. We've tried
 > 0,1,none,posix and keep (which seems to be invalid now) and we ALWAYS  > get the fallocate error. Sometimes the job continues to run, but the  > output file is only created at zero byte length.

Can you send me the job file you are using?

--
Jens Axboe

Re: fio on AIX - fallocate error

[Jens Axboe]

On 04/10/2014 09:56 AM, Carino, Mario wrote:
> Here's the job file:
>
> [4ktest]
> fallocate=1
> direct=1
> rw=randwrite
> refill_buffers
> norandommap
> randrepeat=0
> size=2G
> bs=4k
> iodepth=16
> numjobs=16
> runtime=60
> group_reporting
>
> This job file was created based on a sample command line I was given:
>
> fio --filename=/oradata2/o18g2/IO/fiotest4 --direct=1 --rw=randwrite --refill_buffers --norandommap --randrepeat=0 --size=2G --bs=4k  --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest
>
> I think I understand enough that I need to run a rw=write job first before running the above job, correct?

No, you should not have to run rw=write first, you can run it as-is and 
fio should setup the files appropriately.

Can you try and delete any leftover 4ktest.* files from the job, and 
then re-run the job and add --debug=file? Then just break it when it 
starts going and send the output in a reply here.

-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

Here's the output. No need to break as it stopped immediately. All of the 4ktest files are zero length.

Command line
./fio  --debug=file  examples/test1.fio

fio: set debug option file
file     48824354 dup files: 0
file     48824354 add file 4ktest.0.0
file     48824354 resize file array to 2 files
file     48824354 file b0002028 "4ktest.0.0" added at 0
4ktest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=16
file     48824354 dup files: 1
file     48824354 add file 4ktest.15.0
file     48824354 resize file array to 2 files
file     48824354 file b00022a8 "4ktest.15.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.14.0
file     48824354 resize file array to 2 files
file     48824354 file b0002528 "4ktest.14.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.13.0
file     48824354 resize file array to 2 files
file     48824354 file b00027a8 "4ktest.13.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.12.0
file     48824354 resize file array to 2 files
file     48824354 file b0002a28 "4ktest.12.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.11.0
file     48824354 resize file array to 2 files
file     48824354 file b0002ca8 "4ktest.11.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.10.0
file     48824354 resize file array to 2 files
file     48824354 file b0002f28 "4ktest.10.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.9.0
file     48824354 resize file array to 2 files
file     48824354 file b00031a8 "4ktest.9.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.8.0
file     48824354 resize file array to 2 files
file     48824354 file b0003428 "4ktest.8.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.7.0
file     48824354 resize file array to 2 files
file     48824354 file b00036a8 "4ktest.7.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.6.0
file     48824354 resize file array to 2 files
file     48824354 file b0003928 "4ktest.6.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.5.0
file     48824354 resize file array to 2 files
file     48824354 file b0003ba8 "4ktest.5.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.4.0
file     48824354 resize file array to 2 files
file     48824354 file b0003e28 "4ktest.4.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.3.0
file     48824354 resize file array to 2 files
file     48824354 file b00040a8 "4ktest.3.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.2.0
file     48824354 resize file array to 2 files
file     48824354 file b0004328 "4ktest.2.0" added at 0
file     48824354 dup files: 1
file     48824354 add file 4ktest.1.0
file     48824354 resize file array to 2 files
file     48824354 file b00045a8 "4ktest.1.0" added at 0
...
fio-2.1.7
Starting 16 processes
file     48824354 setup files
file     48824354 get file size for b0002028/0/b0002148
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.0.0
file     48824354 open file 4ktest.0.0, flags 101
file     48824354 posix_fallocate file 4ktest.0.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00022a8/0/b00023c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.15.0
file     48824354 open file 4ktest.15.0, flags 101
file     48824354 posix_fallocate file 4ktest.15.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0002528/0/b0002648
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.14.0
file     48824354 open file 4ktest.14.0, flags 101
file     48824354 posix_fallocate file 4ktest.14.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00027a8/0/b00028c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.13.0
file     48824354 open file 4ktest.13.0, flags 101
file     48824354 posix_fallocate file 4ktest.13.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0002a28/0/b0002b48
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.12.0
file     48824354 open file 4ktest.12.0, flags 101
file     48824354 posix_fallocate file 4ktest.12.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0002ca8/0/b0002dc8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.11.0
file     48824354 open file 4ktest.11.0, flags 101
file     48824354 posix_fallocate file 4ktest.11.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0002f28/0/b0003048
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.10.0
file     48824354 open file 4ktest.10.0, flags 101
file     48824354 posix_fallocate file 4ktest.10.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00031a8/0/b00032c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.9.0
file     48824354 open file 4ktest.9.0, flags 101
file     48824354 posix_fallocate file 4ktest.9.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0003428/0/b0003548
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.8.0
file     48824354 open file 4ktest.8.0, flags 101
file     48824354 posix_fallocate file 4ktest.8.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00036a8/0/b00037c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.7.0
file     48824354 open file 4ktest.7.0, flags 101
file     48824354 posix_fallocate file 4ktest.7.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0003928/0/b0003a48
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.6.0
file     48824354 open file 4ktest.6.0, flags 101
file     48824354 posix_fallocate file 4ktest.6.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0003ba8/0/b0003cc8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.5.0
file     48824354 open file 4ktest.5.0, flags 101
file     48824354 posix_fallocate file 4ktest.5.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0003e28/0/b0003f48
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.4.0
file     48824354 open file 4ktest.4.0, flags 101
file     48824354 posix_fallocate file 4ktest.4.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00040a8/0/b00041c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.3.0
file     48824354 open file 4ktest.3.0, flags 101
file     48824354 posix_fallocate file 4ktest.3.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b0004328/0/b0004448
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.2.0
file     48824354 open file 4ktest.2.0, flags 101
file     48824354 posix_fallocate file 4ktest.2.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     48824354 setup files
file     48824354 get file size for b00045a8/0/b00046c8
4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
file     48824354 layout unlink 4ktest.1.0
file     48824354 open file 4ktest.1.0, flags 101
file     48824354 posix_fallocate file 4ktest.1.0 size 2147483648
fio: posix_fallocate fails: Invalid argument
fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument


Run status group 0 (all jobs):







Mario Carino


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 10:00 AM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 2014-04-09 20:52, Carino, Mario wrote:
> Hi.
> We're trying to run fio 2.1.7 on AIX 6 and we can't seem to eliminate these errors:
>
> fio: posix_fallocate fails: Invalid argument

This one you can ignore, or you can get rid of it by adding:

fallocate=none

to the job file.

> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, 
> error=Invalid argument

Fio should probably just ignore these flush-cache warnings and continue on, it's not really a fatal condition. I have attached a patch for you, can you apply that and give it a go?

> we always get one or the other. It doesn't matter too much what 
> command we run, what options,  etc. We saw a web page which indicates 
> that fio requires an Asynch IO package to be installed and running on 
> AIX. Is that correct? We're
 > trying to find out if we have it installed on our test box.

You'd get a different error if that was your issue.

> (We're lowly non-root
 > DBA's so getting anything done requires a committee.)

Isn't that always the case with AIX? :-)

--
Jens Axboe

Re: fio on AIX - fallocate error

[Jens Axboe]

On 04/10/2014 10:22 AM, Carino, Mario wrote:
> Here's the output. No need to break as it stopped immediately. All of the 4ktest files are zero length.
>
> Command line
> ./fio  --debug=file  examples/test1.fio
>
> fio: set debug option file
> file     48824354 dup files: 0
> file     48824354 add file 4ktest.0.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002028 "4ktest.0.0" added at 0
> 4ktest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=16
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.15.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00022a8 "4ktest.15.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.14.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002528 "4ktest.14.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.13.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00027a8 "4ktest.13.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.12.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002a28 "4ktest.12.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.11.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002ca8 "4ktest.11.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.10.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002f28 "4ktest.10.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.9.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00031a8 "4ktest.9.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.8.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003428 "4ktest.8.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.7.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00036a8 "4ktest.7.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.6.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003928 "4ktest.6.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.5.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003ba8 "4ktest.5.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.4.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003e28 "4ktest.4.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.3.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00040a8 "4ktest.3.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.2.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0004328 "4ktest.2.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.1.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00045a8 "4ktest.1.0" added at 0
> ...
> fio-2.1.7
> Starting 16 processes
> file     48824354 setup files
> file     48824354 get file size for b0002028/0/b0002148
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.0.0
> file     48824354 open file 4ktest.0.0, flags 101
> file     48824354 posix_fallocate file 4ktest.0.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00022a8/0/b00023c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.15.0
> file     48824354 open file 4ktest.15.0, flags 101
> file     48824354 posix_fallocate file 4ktest.15.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002528/0/b0002648
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.14.0
> file     48824354 open file 4ktest.14.0, flags 101
> file     48824354 posix_fallocate file 4ktest.14.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00027a8/0/b00028c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.13.0
> file     48824354 open file 4ktest.13.0, flags 101
> file     48824354 posix_fallocate file 4ktest.13.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002a28/0/b0002b48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.12.0
> file     48824354 open file 4ktest.12.0, flags 101
> file     48824354 posix_fallocate file 4ktest.12.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002ca8/0/b0002dc8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.11.0
> file     48824354 open file 4ktest.11.0, flags 101
> file     48824354 posix_fallocate file 4ktest.11.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002f28/0/b0003048
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.10.0
> file     48824354 open file 4ktest.10.0, flags 101
> file     48824354 posix_fallocate file 4ktest.10.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00031a8/0/b00032c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.9.0
> file     48824354 open file 4ktest.9.0, flags 101
> file     48824354 posix_fallocate file 4ktest.9.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003428/0/b0003548
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.8.0
> file     48824354 open file 4ktest.8.0, flags 101
> file     48824354 posix_fallocate file 4ktest.8.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00036a8/0/b00037c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.7.0
> file     48824354 open file 4ktest.7.0, flags 101
> file     48824354 posix_fallocate file 4ktest.7.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003928/0/b0003a48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.6.0
> file     48824354 open file 4ktest.6.0, flags 101
> file     48824354 posix_fallocate file 4ktest.6.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003ba8/0/b0003cc8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.5.0
> file     48824354 open file 4ktest.5.0, flags 101
> file     48824354 posix_fallocate file 4ktest.5.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003e28/0/b0003f48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.4.0
> file     48824354 open file 4ktest.4.0, flags 101
> file     48824354 posix_fallocate file 4ktest.4.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00040a8/0/b00041c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.3.0
> file     48824354 open file 4ktest.3.0, flags 101
> file     48824354 posix_fallocate file 4ktest.3.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0004328/0/b0004448
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.2.0
> file     48824354 open file 4ktest.2.0, flags 101
> file     48824354 posix_fallocate file 4ktest.2.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00045a8/0/b00046c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.1.0
> file     48824354 open file 4ktest.1.0, flags 101
> file     48824354 posix_fallocate file 4ktest.1.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
>
>
> Run status group 0 (all jobs):

That's very weird, I wonder why fio exits. Can you try --debug=all 
instead and send that to me? Basically fallocate failing should not 
matter at all, the writes will extend the file. Perhaps sparse files 
don't work in AIX on your file system? In that case, you probably want 
overwrite=1 to have fio lay out the file manually for you.


-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

[-- Attachment #1: Type: text/plain, Size: 11224 bytes --]

Debug output attached

Mario Carino


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 12:35 PM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 04/10/2014 10:22 AM, Carino, Mario wrote:
> Here's the output. No need to break as it stopped immediately. All of the 4ktest files are zero length.
>
> Command line
> ./fio  --debug=file  examples/test1.fio
>
> fio: set debug option file
> file     48824354 dup files: 0
> file     48824354 add file 4ktest.0.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002028 "4ktest.0.0" added at 0
> 4ktest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=16
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.15.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00022a8 "4ktest.15.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.14.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002528 "4ktest.14.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.13.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00027a8 "4ktest.13.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.12.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002a28 "4ktest.12.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.11.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002ca8 "4ktest.11.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.10.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0002f28 "4ktest.10.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.9.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00031a8 "4ktest.9.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.8.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003428 "4ktest.8.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.7.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00036a8 "4ktest.7.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.6.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003928 "4ktest.6.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.5.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003ba8 "4ktest.5.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.4.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0003e28 "4ktest.4.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.3.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00040a8 "4ktest.3.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.2.0
> file     48824354 resize file array to 2 files
> file     48824354 file b0004328 "4ktest.2.0" added at 0
> file     48824354 dup files: 1
> file     48824354 add file 4ktest.1.0
> file     48824354 resize file array to 2 files
> file     48824354 file b00045a8 "4ktest.1.0" added at 0
> ...
> fio-2.1.7
> Starting 16 processes
> file     48824354 setup files
> file     48824354 get file size for b0002028/0/b0002148
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.0.0
> file     48824354 open file 4ktest.0.0, flags 101
> file     48824354 posix_fallocate file 4ktest.0.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00022a8/0/b00023c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.15.0
> file     48824354 open file 4ktest.15.0, flags 101
> file     48824354 posix_fallocate file 4ktest.15.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002528/0/b0002648
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.14.0
> file     48824354 open file 4ktest.14.0, flags 101
> file     48824354 posix_fallocate file 4ktest.14.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00027a8/0/b00028c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.13.0
> file     48824354 open file 4ktest.13.0, flags 101
> file     48824354 posix_fallocate file 4ktest.13.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002a28/0/b0002b48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.12.0
> file     48824354 open file 4ktest.12.0, flags 101
> file     48824354 posix_fallocate file 4ktest.12.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002ca8/0/b0002dc8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.11.0
> file     48824354 open file 4ktest.11.0, flags 101
> file     48824354 posix_fallocate file 4ktest.11.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0002f28/0/b0003048
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.10.0
> file     48824354 open file 4ktest.10.0, flags 101
> file     48824354 posix_fallocate file 4ktest.10.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00031a8/0/b00032c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.9.0
> file     48824354 open file 4ktest.9.0, flags 101
> file     48824354 posix_fallocate file 4ktest.9.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003428/0/b0003548
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.8.0
> file     48824354 open file 4ktest.8.0, flags 101
> file     48824354 posix_fallocate file 4ktest.8.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00036a8/0/b00037c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.7.0
> file     48824354 open file 4ktest.7.0, flags 101
> file     48824354 posix_fallocate file 4ktest.7.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003928/0/b0003a48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.6.0
> file     48824354 open file 4ktest.6.0, flags 101
> file     48824354 posix_fallocate file 4ktest.6.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003ba8/0/b0003cc8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.5.0
> file     48824354 open file 4ktest.5.0, flags 101
> file     48824354 posix_fallocate file 4ktest.5.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0003e28/0/b0003f48
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.4.0
> file     48824354 open file 4ktest.4.0, flags 101
> file     48824354 posix_fallocate file 4ktest.4.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00040a8/0/b00041c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.3.0
> file     48824354 open file 4ktest.3.0, flags 101
> file     48824354 posix_fallocate file 4ktest.3.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b0004328/0/b0004448
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.2.0
> file     48824354 open file 4ktest.2.0, flags 101
> file     48824354 posix_fallocate file 4ktest.2.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
> file     48824354 setup files
> file     48824354 get file size for b00045a8/0/b00046c8
> 4ktest: Laying out IO file(s) (1 file(s) / 2048MB)
> file     48824354 layout unlink 4ktest.1.0
> file     48824354 open file 4ktest.1.0, flags 101
> file     48824354 posix_fallocate file 4ktest.1.0 size 2147483648
> fio: posix_fallocate fails: Invalid argument
> fio: pid=0, err=22/file:filesetup.c:418, func=invalidate_cache, 
> error=Invalid argument
>
>
> Run status group 0 (all jobs):

That's very weird, I wonder why fio exits. Can you try --debug=all instead and send that to me? Basically fallocate failing should not matter at all, the writes will extend the file. Perhaps sparse files don't work in AIX on your file system? In that case, you probably want
overwrite=1 to have fio lay out the file manually for you.


--
Jens Axboe


[-- Attachment #2: fiodebug.out --]
[-- Type: application/octet-stream, Size: 73960 bytes --]

dsadm@dugsmrdatasta01.r1-core.r1.aig.net(/dsProjects/data/work/fio-2.1.7)$ ./fio  --debug=all  examples/test1.fio
fio: set all debug options
parse    23003136 filling default options
parse    23003136 handle_option=filename_format, ptr=$jobname.$jobnum.$filenum
parse    23003136 __handle_option=filename_format, type=5, ptr=$jobname.$jobnum.$filenum
parse    23003136 handle_option=lockfile, ptr=none
parse    23003136 __handle_option=lockfile, type=1, ptr=none
parse    23003136 handle_option=rw, ptr=read
parse    23003136 __handle_option=rw, type=1, ptr=read
parse    23003136 handle_option=rw_sequencer, ptr=sequential
parse    23003136 __handle_option=rw_sequencer, type=1, ptr=sequential
parse    23003136 handle_option=ioengine, ptr=sync
parse    23003136 __handle_option=ioengine, type=5, ptr=sync
parse    23003136 handle_option=iodepth, ptr=1
parse    23003136 __handle_option=iodepth, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=iodepth_batch, ptr=1
parse    23003136 __handle_option=iodepth_batch, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=iodepth_batch_complete, ptr=1
parse    23003136 __handle_option=iodepth_batch_complete, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=fill_device, ptr=0
parse    23003136 __handle_option=fill_device, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=file_append, ptr=0
parse    23003136 __handle_option=file_append, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=offset, ptr=0
parse    23003136 __handle_option=offset, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=offset_increment, ptr=0
parse    23003136 __handle_option=offset_increment, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=number_ios, ptr=0
parse    23003136 __handle_option=number_ios, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=bs, ptr=4k
parse    23003136 __handle_option=bs, type=7, ptr=4k
parse    23003136   ret=0, out=4096
parse    23003136 handle_option=bs_is_seq_rand, ptr=0
parse    23003136 __handle_option=bs_is_seq_rand, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=randrepeat, ptr=1
parse    23003136 __handle_option=randrepeat, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=use_os_rand, ptr=0
parse    23003136 __handle_option=use_os_rand, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=softrandommap, ptr=0
parse    23003136 __handle_option=softrandommap, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=random_generator, ptr=tausworthe
parse    23003136 __handle_option=random_generator, type=1, ptr=tausworthe
parse    23003136 handle_option=random_distribution, ptr=random
parse    23003136 __handle_option=random_distribution, type=1, ptr=random
parse    23003136 handle_option=percentage_random, ptr=100,100,100
parse    23003136 __handle_option=percentage_random, type=7, ptr=100,100,100
parse    23003136   ret=0, out=100
parse    23003136 __handle_option=percentage_random, type=7, ptr=100,100
parse    23003136   ret=0, out=100
parse    23003136 __handle_option=percentage_random, type=7, ptr=100
parse    23003136   ret=0, out=100
parse    23003136 handle_option=allrandrepeat, ptr=0
parse    23003136 __handle_option=allrandrepeat, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=nrfiles, ptr=1
parse    23003136 __handle_option=nrfiles, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=file_service_type, ptr=roundrobin
parse    23003136 __handle_option=file_service_type, type=1, ptr=roundrobin
parse    23003136 handle_option=fallocate, ptr=posix
parse    23003136 __handle_option=fallocate, type=1, ptr=posix
parse    23003136 handle_option=fadvise_hint, ptr=1
parse    23003136 __handle_option=fadvise_hint, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=fsync, ptr=0
parse    23003136 __handle_option=fsync, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=fdatasync, ptr=0
parse    23003136 __handle_option=fdatasync, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=write_barrier, ptr=0
parse    23003136 __handle_option=write_barrier, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=direct, ptr=0
parse    23003136 __handle_option=direct, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=atomic, ptr=0
parse    23003136 __handle_option=atomic, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=buffered, ptr=1
parse    23003136 __handle_option=buffered, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=overwrite, ptr=0
parse    23003136 __handle_option=overwrite, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=loops, ptr=1
parse    23003136 __handle_option=loops, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=numjobs, ptr=1
parse    23003136 __handle_option=numjobs, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=startdelay, ptr=0
parse    23003136 __handle_option=startdelay, type=4, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=runtime, ptr=0
parse    23003136 __handle_option=runtime, type=4, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=mem, ptr=malloc
parse    23003136 __handle_option=mem, type=1, ptr=malloc
parse    23003136 handle_option=iomem_align, ptr=0
parse    23003136 __handle_option=iomem_align, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=verify, ptr=0
parse    23003136 __handle_option=verify, type=1, ptr=0
parse    23003136 handle_option=do_verify, ptr=1
parse    23003136 __handle_option=do_verify, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=verifysort, ptr=1
parse    23003136 __handle_option=verifysort, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=verifysort_nr, ptr=1024
parse    23003136 __handle_option=verifysort_nr, type=7, ptr=1024
parse    23003136   ret=0, out=1024
parse    23003136 handle_option=verify_fatal, ptr=0
parse    23003136 __handle_option=verify_fatal, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=verify_dump, ptr=0
parse    23003136 __handle_option=verify_dump, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=verify_async, ptr=0
parse    23003136 __handle_option=verify_async, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=replay_no_stall, ptr=0
parse    23003136 __handle_option=replay_no_stall, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=zonesize, ptr=0
parse    23003136 __handle_option=zonesize, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=zonerange, ptr=0
parse    23003136 __handle_option=zonerange, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=zoneskip, ptr=0
parse    23003136 __handle_option=zoneskip, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=lockmem, ptr=0
parse    23003136 __handle_option=lockmem, type=3, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=rwmixread, ptr=50
parse    23003136 __handle_option=rwmixread, type=7, ptr=50
parse    23003136   ret=0, out=50
parse    23003136 handle_option=rwmixwrite, ptr=50
parse    23003136 __handle_option=rwmixwrite, type=7, ptr=50
parse    23003136   ret=0, out=50
parse    23003136 handle_option=nice, ptr=0
parse    23003136 __handle_option=nice, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=thinktime, ptr=0
parse    23003136 __handle_option=thinktime, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=thinktime_spin, ptr=0
parse    23003136 __handle_option=thinktime_spin, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=thinktime_blocks, ptr=1
parse    23003136 __handle_option=thinktime_blocks, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=ratecycle, ptr=1000
parse    23003136 __handle_option=ratecycle, type=7, ptr=1000
parse    23003136   ret=0, out=1000
parse    23003136 handle_option=latency_percentile, ptr=100
parse    23003136 __handle_option=latency_percentile, type=9, ptr=100
parse    23003136   out=100.000000
parse    23003136 handle_option=invalidate, ptr=1
parse    23003136 __handle_option=invalidate, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=sync, ptr=0
parse    23003136 __handle_option=sync, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=create_serialize, ptr=1
parse    23003136 __handle_option=create_serialize, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=create_fsync, ptr=1
parse    23003136 __handle_option=create_fsync, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=create_on_open, ptr=0
parse    23003136 __handle_option=create_on_open, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=create_only, ptr=0
parse    23003136 __handle_option=create_only, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=pre_read, ptr=0
parse    23003136 __handle_option=pre_read, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=end_fsync, ptr=0
parse    23003136 __handle_option=end_fsync, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=fsync_on_close, ptr=0
parse    23003136 __handle_option=fsync_on_close, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=unlink, ptr=0
parse    23003136 __handle_option=unlink, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=log_avg_msec, ptr=0
parse    23003136 __handle_option=log_avg_msec, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=bwavgtime, ptr=500
parse    23003136 __handle_option=bwavgtime, type=7, ptr=500
parse    23003136   ret=0, out=500
parse    23003136 handle_option=iopsavgtime, ptr=500
parse    23003136 __handle_option=iopsavgtime, type=7, ptr=500
parse    23003136   ret=0, out=500
parse    23003136 handle_option=scramble_buffers, ptr=1
parse    23003136 __handle_option=scramble_buffers, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=clat_percentiles, ptr=1
parse    23003136 __handle_option=clat_percentiles, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=percentile_list, ptr=1:5:10:20:30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136 __handle_option=percentile_list, type=9, ptr=1:5:10:20:30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=1.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=5:10:20:30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=5.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=10:20:30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=10.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=20:30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=20.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=30:40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=30.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=40:50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=40.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=50:60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=50.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=60:70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=60.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=70:80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=70.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=80:90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=80.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=90:95:99:99.5:99.9:99.95:99.99
parse    23003136   out=90.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=95:99:99.5:99.9:99.95:99.99
parse    23003136   out=95.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=99:99.5:99.9:99.95:99.99
parse    23003136   out=99.000000
parse    23003136 __handle_option=percentile_list, type=9, ptr=99.5:99.9:99.95:99.99
parse    23003136   out=99.500000
parse    23003136 __handle_option=percentile_list, type=9, ptr=99.9:99.95:99.99
parse    23003136   out=99.900000
parse    23003136 __handle_option=percentile_list, type=9, ptr=99.95:99.99
parse    23003136   out=99.950000
parse    23003136 __handle_option=percentile_list, type=9, ptr=99.99
parse    23003136   out=99.990000
parse    23003136 handle_option=gtod_reduce, ptr=0
parse    23003136 __handle_option=gtod_reduce, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=disable_lat, ptr=0
parse    23003136 __handle_option=disable_lat, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=disable_clat, ptr=0
parse    23003136 __handle_option=disable_clat, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=disable_slat, ptr=0
parse    23003136 __handle_option=disable_slat, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=disable_bw_measurement, ptr=0
parse    23003136 __handle_option=disable_bw_measurement, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=unified_rw_reporting, ptr=0
parse    23003136 __handle_option=unified_rw_reporting, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=continue_on_error, ptr=none
parse    23003136 __handle_option=continue_on_error, type=1, ptr=none
parse    23003136 handle_option=error_dump, ptr=0
parse    23003136 __handle_option=error_dump, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=cgroup_nodelete, ptr=0
parse    23003136 __handle_option=cgroup_nodelete, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=kb_base, ptr=1024
parse    23003136 __handle_option=kb_base, type=7, ptr=1024
parse    23003136   ret=0, out=1024
parse    23003136 handle_option=hugepage-size, ptr=0
parse    23003136 __handle_option=hugepage-size, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=flow_id, ptr=0
parse    23003136 __handle_option=flow_id, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=flow, ptr=0
parse    23003136 __handle_option=flow, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=flow_watermark, ptr=1024
parse    23003136 __handle_option=flow_watermark, type=7, ptr=1024
parse    23003136   ret=0, out=1024
parse    23003136 handle_option=flow_sleep, ptr=0
parse    23003136 __handle_option=flow_sleep, type=7, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=load, ptr=1
parse    23003136 __handle_option=load, type=7, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=test-duration, ptr=24h
parse    23003136 __handle_option=test-duration, type=4, ptr=24h
parse    23003136   ret=0, out=86400000000
parse    23003136 handle_option=threads-per-queue, ptr=8
parse    23003136 __handle_option=threads-per-queue, type=7, ptr=8
parse    23003136   ret=0, out=8
parse    23003136 handle_option=read-req-num-512-blocks, ptr=3
parse    23003136 __handle_option=read-req-num-512-blocks, type=7, ptr=3
parse    23003136   ret=0, out=3
parse    23003136 handle_option=large-block-op-kbytes, ptr=128k
parse    23003136 __handle_option=large-block-op-kbytes, type=7, ptr=128k
parse    23003136   ret=0, out=131072
parse    23003136 handle_option=block, ptr=4k
parse    23003136 __handle_option=block, type=7, ptr=4k
parse    23003136   ret=0, out=4096
file     23003136 dup files: 0
parse    23003136 handle_option=direct, ptr=1
parse    23003136 __handle_option=direct, type=8, ptr=1
parse    23003136   ret=0, out=1
parse    23003136 handle_option=rw, ptr=randwrite
parse    23003136 __handle_option=rw, type=1, ptr=randwrite
parse    23003136 handle_option=refill_buffers, ptr=
parse    23003136 __handle_option=refill_buffers, type=10, ptr=
parse    23003136   ret=0, out=1
parse    23003136 handle_option=norandommap, ptr=
parse    23003136 __handle_option=norandommap, type=10, ptr=
parse    23003136   ret=0, out=1
parse    23003136 handle_option=randrepeat, ptr=0
parse    23003136 __handle_option=randrepeat, type=8, ptr=0
parse    23003136   ret=0, out=0
parse    23003136 handle_option=fallocate, ptr=1
parse    23003136 __handle_option=fallocate, type=1, ptr=1
parse    23003136 handle_option=size, ptr=100m
parse    23003136 __handle_option=size, type=3, ptr=100m
parse    23003136   ret=0, out=104857600
parse    23003136 handle_option=group_reporting, ptr=
parse    23003136 __handle_option=group_reporting, type=10, ptr=
parse    23003136   ret=0, out=1
parse    23003136 handle_option=bs, ptr=4k
parse    23003136 __handle_option=bs, type=7, ptr=4k
parse    23003136   ret=0, out=4096
parse    23003136 handle_option=iodepth, ptr=16
parse    23003136 __handle_option=iodepth, type=7, ptr=16
parse    23003136   ret=0, out=16
parse    23003136 handle_option=numjobs, ptr=16
parse    23003136 __handle_option=numjobs, type=7, ptr=16
parse    23003136   ret=0, out=16
parse    23003136 handle_option=runtime, ptr=60
parse    23003136 __handle_option=runtime, type=4, ptr=60
parse    23003136   ret=0, out=60000000
io       23003136 load ioengine sync
file     23003136 add file 4ktest.0.0
file     23003136 resize file array to 2 files
file     23003136 file b0002028 "4ktest.0.0" added at 0
4ktest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=16
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.15.0
file     23003136 resize file array to 2 files
file     23003136 file b00022a8 "4ktest.15.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.14.0
file     23003136 resize file array to 2 files
file     23003136 file b0002528 "4ktest.14.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.13.0
file     23003136 resize file array to 2 files
file     23003136 file b00027a8 "4ktest.13.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.12.0
file     23003136 resize file array to 2 files
file     23003136 file b0002a28 "4ktest.12.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.11.0
file     23003136 resize file array to 2 files
file     23003136 file b0002ca8 "4ktest.11.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.10.0
file     23003136 resize file array to 2 files
file     23003136 file b0002f28 "4ktest.10.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.9.0
file     23003136 resize file array to 2 files
file     23003136 file b00031a8 "4ktest.9.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.8.0
file     23003136 resize file array to 2 files
file     23003136 file b0003428 "4ktest.8.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.7.0
file     23003136 resize file array to 2 files
file     23003136 file b00036a8 "4ktest.7.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.6.0
file     23003136 resize file array to 2 files
file     23003136 file b0003928 "4ktest.6.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.5.0
file     23003136 resize file array to 2 files
file     23003136 file b0003ba8 "4ktest.5.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.4.0
file     23003136 resize file array to 2 files
file     23003136 file b0003e28 "4ktest.4.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.3.0
file     23003136 resize file array to 2 files
file     23003136 file b00040a8 "4ktest.3.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.2.0
file     23003136 resize file array to 2 files
file     23003136 file b0004328 "4ktest.2.0" added at 0
file     23003136 dup files: 1
io       23003136 load ioengine sync
file     23003136 add file 4ktest.1.0
file     23003136 resize file array to 2 files
file     23003136 file b00045a8 "4ktest.1.0" added at 0
...
parse    23003136 free options
fio-2.1.7
mutex    23003136 wait on startup_mutex
mutex    23003136 done waiting on startup_mutex
Starting 16 processes
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0002028/0/b0002148
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00022a8/0/b00023c8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0002528/0/b0002648
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00027a8/0/b00028c8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0002a28/0/b0002b48
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0002ca8/0/b0002dc8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0002f28/0/b0003048
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00031a8/0/b00032c8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0003428/0/b0003548
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00036a8/0/b00037c8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0003928/0/b0003a48
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0003ba8/0/b0003cc8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0003e28/0/b0003f48
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00040a8/0/b00041c8
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b0004328/0/b0004448
process  23003136 pid=0: runstate 4 -> 0
file     23003136 setup files
process  23003136 pid=0: runstate 0 -> 4
file     23003136 get file size for b00045a8/0/b00046c8
process  23003136 pid=0: runstate 4 -> 0
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  59441406 jobs pid=59441406 started
process  59441406 pid=59441406: runstate 1 -> 2
mutex    59441406 up startup_mutex
mutex    59441406 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  14745690 jobs pid=14745690 started
process  14745690 pid=14745690: runstate 1 -> 2
mutex    14745690 up startup_mutex
mutex    14745690 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  30736610 jobs pid=30736610 started
process  30736610 pid=30736610: runstate 1 -> 2
mutex    30736610 up startup_mutex
mutex    30736610 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  24248348 jobs pid=24248348 started
process  24248348 pid=24248348: runstate 1 -> 2
mutex    24248348 up startup_mutex
mutex    24248348 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  23199860 jobs pid=23199860 started
process  23199860 pid=23199860: runstate 1 -> 2
mutex    23199860 up startup_mutex
mutex    23199860 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  50331692 jobs pid=50331692 started
process  50331692 pid=50331692: runstate 1 -> 2
mutex    50331692 up startup_mutex
mutex    50331692 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  59113590 jobs pid=59113590 started
process  59113590 pid=59113590: runstate 1 -> 2
mutex    59113590 up startup_mutex
mutex    59113590 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  23592972 jobs pid=23592972 started
process  23592972 pid=23592972: runstate 1 -> 2
mutex    23592972 up startup_mutex
mutex    23592972 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  61866222 jobs pid=61866222 started
process  61866222 pid=61866222: runstate 1 -> 2
mutex    61866222 up startup_mutex
mutex    61866222 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  21430374 jobs pid=21430374 started
process  21430374 pid=21430374: runstate 1 -> 2
mutex    21430374 up startup_mutex
mutex    21430374 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  27590726 jobs pid=27590726 started
process  27590726 pid=27590726: runstate 1 -> 2
mutex    27590726 up startup_mutex
mutex    27590726 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  35913982 jobs pid=35913982 started
process  35913982 pid=35913982: runstate 1 -> 2
mutex    35913982 up startup_mutex
mutex    35913982 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  49283312 jobs pid=49283312 started
process  49283312 pid=49283312: runstate 1 -> 2
mutex    49283312 up startup_mutex
mutex    49283312 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  44367944 jobs pid=44367944 started
process  44367944 pid=44367944: runstate 1 -> 2
mutex    44367944 up startup_mutex
mutex    44367944 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
mutex    23003136 wait on startup_mutex
process  52363504 jobs pid=52363504 started
process  52363504 pid=52363504: runstate 1 -> 2
mutex    52363504 up startup_mutex
mutex    52363504 wait on td->mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=0: runstate 0 -> 1
process  23003136 will fork
process  40697880 jobs pid=40697880 started
process  40697880 pid=40697880: runstate 1 -> 2
mutex    40697880 up startup_mutex
mutex    40697880 wait on td->mutex
mutex    23003136 wait on startup_mutex
mutex    23003136 done waiting on startup_mutex
process  23003136 pid=59441406: runstate 2 -> 5
process  23003136 pid=14745690: runstate 2 -> 5
process  23003136 pid=30736610: runstate 2 -> 5
process  23003136 pid=24248348: runstate 2 -> 5
process  23003136 pid=23199860: runstate 2 -> 5
process  23003136 pid=50331692: runstate 2 -> 5
process  23003136 pid=59113590: runstate 2 -> 5
process  23003136 pid=23592972: runstate 2 -> 5
process  23003136 pid=61866222: runstate 2 -> 5
process  23003136 pid=21430374: runstate 2 -> 5
process  23003136 pid=27590726: runstate 2 -> 5
process  23003136 pid=35913982: runstate 2 -> 5
mutex    21430374 done waiting on td->mutex
process  23003136 pid=49283312: runstate 2 -> 5
mutex    27590726 done waiting on td->mutex
mutex    59441406 done waiting on td->mutex
process  23003136 pid=44367944: runstate 2 -> 5
mem      21430374 Alloc 73726 for buffers
mutex    50331692 done waiting on td->mutex
mutex    14745690 done waiting on td->mutex
mem      27590726 Alloc 73726 for buffers
mutex    59113590 done waiting on td->mutex
mutex    30736610 done waiting on td->mutex
mem      59441406 Alloc 73726 for buffers
mutex    24248348 done waiting on td->mutex
mutex    23199860 done waiting on td->mutex
mutex    49283312 done waiting on td->mutex
mem      21430374 malloc 73726 30073a18
mutex    44367944 done waiting on td->mutex
mem      14745690 Alloc 73726 for buffers
mutex    35913982 done waiting on td->mutex
mem      50331692 Alloc 73726 for buffers
mem      27590726 malloc 73726 30073a18
mem      59113590 Alloc 73726 for buffers
mutex    23592972 done waiting on td->mutex
mutex    61866222 done waiting on td->mutex
process  23003136 pid=52363504: runstate 2 -> 5
mem      30736610 Alloc 73726 for buffers
mem      59441406 malloc 73726 30073a18
mem      24248348 Alloc 73726 for buffers
mem      23199860 Alloc 73726 for buffers
mem      49283312 Alloc 73726 for buffers
mem      21430374 io_u alloc 30070618, index 0
mem      44367944 Alloc 73726 for buffers
mem      14745690 malloc 73726 30073a18
mem      35913982 Alloc 73726 for buffers
mem      50331692 malloc 73726 30073a18
mem      27590726 io_u alloc 30070618, index 0
mem      59113590 malloc 73726 30073a18
mem      23592972 Alloc 73726 for buffers
mem      61866222 Alloc 73726 for buffers
process  23003136 pid=40697880: runstate 2 -> 5
mutex    52363504 done waiting on td->mutex
mem      30736610 malloc 73726 30073a18
mem      59441406 io_u alloc 30070618, index 0
mem      24248348 malloc 73726 30073a18
mem      23199860 malloc 73726 30073a18
mem      49283312 malloc 73726 30073a18
mem      21430374 io_u 30070618, mem 30074000
mem      44367944 malloc 73726 30073a18
mem      14745690 io_u alloc 30070618, index 0
mem      35913982 malloc 73726 30073a18
mem      50331692 io_u alloc 30070618, index 0
mem      27590726 io_u 30070618, mem 30074000
mem      59113590 io_u alloc 30070618, index 0
mem      23592972 malloc 73726 30073a18
mem      61866222 malloc 73726 30073a18
mem      52363504 Alloc 73726 for buffers
mem      30736610 io_u alloc 30070618, index 0
mem      59441406 io_u 30070618, mem 30074000
mem      24248348 io_u alloc 30070618, index 0
mem      23199860 io_u alloc 30070618, index 0
mem      49283312 io_u alloc 30070618, index 0
mem      21430374 io_u alloc 30070828, index 1
mem      14745690 io_u 30070618, mem 30074000
mem      44367944 io_u alloc 30070618, index 0
mem      50331692 io_u 30070618, mem 30074000
mem      35913982 io_u alloc 30070618, index 0
mem      59113590 io_u 30070618, mem 30074000
mutex    40697880 done waiting on td->mutex
mem      27590726 io_u alloc 30070828, index 1
mem      23592972 io_u alloc 30070618, index 0
mem      61866222 io_u alloc 30070618, index 0
mem      52363504 malloc 73726 30073a18
mem      30736610 io_u 30070618, mem 30074000
mem      24248348 io_u 30070618, mem 30074000
mem      59441406 io_u alloc 30070828, index 1
mem      23199860 io_u 30070618, mem 30074000
mem      49283312 io_u 30070618, mem 30074000
mem      21430374 io_u 30070828, mem 30075000
mem      44367944 io_u 30070618, mem 30074000
mem      14745690 io_u alloc 30070828, index 1
mem      50331692 io_u alloc 30070828, index 1
mem      35913982 io_u 30070618, mem 30074000
mem      59113590 io_u alloc 30070828, index 1
mem      40697880 Alloc 73726 for buffers
mem      27590726 io_u 30070828, mem 30075000
mem      23592972 io_u 30070618, mem 30074000
mem      61866222 io_u 30070618, mem 30074000
mem      52363504 io_u alloc 30070618, index 0
mem      30736610 io_u alloc 30070828, index 1
mem      24248348 io_u alloc 30070828, index 1
mem      59441406 io_u 30070828, mem 30075000
mem      23199860 io_u alloc 30070828, index 1
mem      49283312 io_u alloc 30070828, index 1
mem      21430374 io_u alloc 30070a38, index 2
mem      14745690 io_u 30070828, mem 30075000
mem      44367944 io_u alloc 30070828, index 1
mem      50331692 io_u 30070828, mem 30075000
mem      35913982 io_u alloc 30070828, index 1
mem      59113590 io_u 30070828, mem 30075000
mem      40697880 malloc 73726 30073a18
mem      27590726 io_u alloc 30070a38, index 2
mem      23592972 io_u alloc 30070828, index 1
mem      61866222 io_u alloc 30070828, index 1
mem      52363504 io_u 30070618, mem 30074000
mem      30736610 io_u 30070828, mem 30075000
mem      24248348 io_u 30070828, mem 30075000
mem      59441406 io_u alloc 30070a38, index 2
mem      23199860 io_u 30070828, mem 30075000
mem      49283312 io_u 30070828, mem 30075000
mem      21430374 io_u 30070a38, mem 30076000
mem      14745690 io_u alloc 30070a38, index 2
mem      44367944 io_u 30070828, mem 30075000
mem      50331692 io_u alloc 30070a38, index 2
mem      35913982 io_u 30070828, mem 30075000
mem      59113590 io_u alloc 30070a38, index 2
mem      40697880 io_u alloc 30070618, index 0
mem      27590726 io_u 30070a38, mem 30076000
mem      23592972 io_u 30070828, mem 30075000
mem      61866222 io_u 30070828, mem 30075000
mem      52363504 io_u alloc 30070828, index 1
mem      30736610 io_u alloc 30070a38, index 2
mem      24248348 io_u alloc 30070a38, index 2
mem      59441406 io_u 30070a38, mem 30076000
mem      23199860 io_u alloc 30070a38, index 2
mem      49283312 io_u alloc 30070a38, index 2
mem      21430374 io_u alloc 30070c48, index 3
mem      14745690 io_u 30070a38, mem 30076000
mem      44367944 io_u alloc 30070a38, index 2
mem      50331692 io_u 30070a38, mem 30076000
mem      35913982 io_u alloc 30070a38, index 2
mem      59113590 io_u 30070a38, mem 30076000
mem      40697880 io_u 30070618, mem 30074000
mem      27590726 io_u alloc 30070c48, index 3
mem      23592972 io_u alloc 30070a38, index 2
mem      61866222 io_u alloc 30070a38, index 2
mem      52363504 io_u 30070828, mem 30075000
mem      30736610 io_u 30070a38, mem 30076000
mem      24248348 io_u 30070a38, mem 30076000
mem      59441406 io_u alloc 30070c48, index 3
mem      23199860 io_u 30070a38, mem 30076000
mem      49283312 io_u 30070a38, mem 30076000
mem      21430374 io_u 30070c48, mem 30077000
mem      14745690 io_u alloc 30070c48, index 3
mem      44367944 io_u 30070a38, mem 30076000
mem      50331692 io_u alloc 30070c48, index 3
mem      35913982 io_u 30070a38, mem 30076000
mem      59113590 io_u alloc 30070c48, index 3
mem      40697880 io_u alloc 30070828, index 1
mem      27590726 io_u 30070c48, mem 30077000
mem      23592972 io_u 30070a38, mem 30076000
mem      61866222 io_u 30070a38, mem 30076000
mem      52363504 io_u alloc 30070a38, index 2
mem      30736610 io_u alloc 30070c48, index 3
mem      59441406 io_u 30070c48, mem 30077000
mem      24248348 io_u alloc 30070c48, index 3
mem      23199860 io_u alloc 30070c48, index 3
mem      49283312 io_u alloc 30070c48, index 3
mem      14745690 io_u 30070c48, mem 30077000
mem      21430374 io_u alloc 30070e58, index 4
mem      50331692 io_u 30070c48, mem 30077000
mem      44367944 io_u alloc 30070c48, index 3
mem      35913982 io_u alloc 30070c48, index 3
mem      59113590 io_u 30070c48, mem 30077000
mem      40697880 io_u 30070828, mem 30075000
mem      27590726 io_u alloc 30070e58, index 4
mem      23592972 io_u alloc 30070c48, index 3
mem      61866222 io_u alloc 30070c48, index 3
mem      52363504 io_u 30070a38, mem 30076000
mem      30736610 io_u 30070c48, mem 30077000
mem      59441406 io_u alloc 30070e58, index 4
mem      24248348 io_u 30070c48, mem 30077000
mem      23199860 io_u 30070c48, mem 30077000
mem      49283312 io_u 30070c48, mem 30077000
mem      21430374 io_u 30070e58, mem 30078000
mem      14745690 io_u alloc 30070e58, index 4
mem      50331692 io_u alloc 30070e58, index 4
mem      44367944 io_u 30070c48, mem 30077000
mem      35913982 io_u 30070c48, mem 30077000
mem      59113590 io_u alloc 30070e58, index 4
mem      40697880 io_u alloc 30070a38, index 2
mem      27590726 io_u 30070e58, mem 30078000
mem      23592972 io_u 30070c48, mem 30077000
mem      61866222 io_u 30070c48, mem 30077000
mem      52363504 io_u alloc 30070c48, index 3
mem      30736610 io_u alloc 30070e58, index 4
mem      59441406 io_u 30070e58, mem 30078000
mem      24248348 io_u alloc 30070e58, index 4
mem      23199860 io_u alloc 30070e58, index 4
mem      49283312 io_u alloc 30070e58, index 4
mem      21430374 io_u alloc 30071068, index 5
mem      14745690 io_u 30070e58, mem 30078000
mem      50331692 io_u 30070e58, mem 30078000
mem      44367944 io_u alloc 30070e58, index 4
mem      35913982 io_u alloc 30070e58, index 4
mem      59113590 io_u 30070e58, mem 30078000
mem      40697880 io_u 30070a38, mem 30076000
mem      27590726 io_u alloc 30071068, index 5
mem      23592972 io_u alloc 30070e58, index 4
mem      61866222 io_u alloc 30070e58, index 4
mem      52363504 io_u 30070c48, mem 30077000
mem      30736610 io_u 30070e58, mem 30078000
mem      59441406 io_u alloc 30071068, index 5
mem      24248348 io_u 30070e58, mem 30078000
mem      23199860 io_u 30070e58, mem 30078000
mem      49283312 io_u 30070e58, mem 30078000
mem      21430374 io_u 30071068, mem 30079000
mem      14745690 io_u alloc 30071068, index 5
mem      50331692 io_u alloc 30071068, index 5
mem      44367944 io_u 30070e58, mem 30078000
mem      35913982 io_u 30070e58, mem 30078000
mem      59113590 io_u alloc 30071068, index 5
mem      40697880 io_u alloc 30070c48, index 3
mem      27590726 io_u 30071068, mem 30079000
mem      23592972 io_u 30070e58, mem 30078000
mem      61866222 io_u 30070e58, mem 30078000
mem      52363504 io_u alloc 30070e58, index 4
mem      59441406 io_u 30071068, mem 30079000
mem      30736610 io_u alloc 30071068, index 5
mem      24248348 io_u alloc 30071068, index 5
mem      23199860 io_u alloc 30071068, index 5
mem      49283312 io_u alloc 30071068, index 5
mem      14745690 io_u 30071068, mem 30079000
mem      21430374 io_u alloc 30071278, index 6
mem      50331692 io_u 30071068, mem 30079000
mem      44367944 io_u alloc 30071068, index 5
mem      35913982 io_u alloc 30071068, index 5
mem      59113590 io_u 30071068, mem 30079000
mem      40697880 io_u 30070c48, mem 30077000
mem      27590726 io_u alloc 30071278, index 6
mem      23592972 io_u alloc 30071068, index 5
mem      52363504 io_u 30070e58, mem 30078000
mem      61866222 io_u alloc 30071068, index 5
mem      59441406 io_u alloc 30071278, index 6
mem      30736610 io_u 30071068, mem 30079000
mem      24248348 io_u 30071068, mem 30079000
mem      23199860 io_u 30071068, mem 30079000
mem      49283312 io_u 30071068, mem 30079000
mem      21430374 io_u 30071278, mem 3007a000
mem      14745690 io_u alloc 30071278, index 6
mem      50331692 io_u alloc 30071278, index 6
mem      44367944 io_u 30071068, mem 30079000
mem      35913982 io_u 30071068, mem 30079000
mem      59113590 io_u alloc 30071278, index 6
mem      40697880 io_u alloc 30070e58, index 4
mem      27590726 io_u 30071278, mem 3007a000
mem      23592972 io_u 30071068, mem 30079000
mem      52363504 io_u alloc 30071068, index 5
mem      61866222 io_u 30071068, mem 30079000
mem      59441406 io_u 30071278, mem 3007a000
mem      30736610 io_u alloc 30071278, index 6
mem      24248348 io_u alloc 30071278, index 6
mem      23199860 io_u alloc 30071278, index 6
mem      49283312 io_u alloc 30071278, index 6
mem      14745690 io_u 30071278, mem 3007a000
mem      21430374 io_u alloc 30071648, index 7
mem      50331692 io_u 30071278, mem 3007a000
mem      44367944 io_u alloc 30071278, index 6
mem      35913982 io_u alloc 30071278, index 6
mem      59113590 io_u 30071278, mem 3007a000
mem      40697880 io_u 30070e58, mem 30078000
mem      27590726 io_u alloc 30071648, index 7
mem      23592972 io_u alloc 30071278, index 6
mem      52363504 io_u 30071068, mem 30079000
mem      61866222 io_u alloc 30071278, index 6
mem      59441406 io_u alloc 30071648, index 7
mem      30736610 io_u 30071278, mem 3007a000
mem      24248348 io_u 30071278, mem 3007a000
mem      23199860 io_u 30071278, mem 3007a000
mem      49283312 io_u 30071278, mem 3007a000
mem      21430374 io_u 30071648, mem 3007b000
mem      14745690 io_u alloc 30071648, index 7
mem      44367944 io_u 30071278, mem 3007a000
mem      50331692 io_u alloc 30071648, index 7
mem      35913982 io_u 30071278, mem 3007a000
mem      59113590 io_u alloc 30071648, index 7
mem      40697880 io_u alloc 30071068, index 5
mem      27590726 io_u 30071648, mem 3007b000
mem      23592972 io_u 30071278, mem 3007a000
mem      52363504 io_u alloc 30071278, index 6
mem      61866222 io_u 30071278, mem 3007a000
mem      59441406 io_u 30071648, mem 3007b000
mem      30736610 io_u alloc 30071648, index 7
mem      24248348 io_u alloc 30071648, index 7
mem      23199860 io_u alloc 30071648, index 7
mem      49283312 io_u alloc 30071648, index 7
mem      14745690 io_u 30071648, mem 3007b000
mem      21430374 io_u alloc 30071858, index 8
mem      50331692 io_u 30071648, mem 3007b000
mem      44367944 io_u alloc 30071648, index 7
mem      59113590 io_u 30071648, mem 3007b000
mem      35913982 io_u alloc 30071648, index 7
mem      40697880 io_u 30071068, mem 30079000
mem      27590726 io_u alloc 30071858, index 8
mem      23592972 io_u alloc 30071648, index 7
mem      52363504 io_u 30071278, mem 3007a000
mem      61866222 io_u alloc 30071648, index 7
mem      59441406 io_u alloc 30071858, index 8
mem      30736610 io_u 30071648, mem 3007b000
mem      24248348 io_u 30071648, mem 3007b000
mem      23199860 io_u 30071648, mem 3007b000
mem      49283312 io_u 30071648, mem 3007b000
mem      21430374 io_u 30071858, mem 3007c000
mem      14745690 io_u alloc 30071858, index 8
mem      50331692 io_u alloc 30071858, index 8
mem      44367944 io_u 30071648, mem 3007b000
mem      59113590 io_u alloc 30071858, index 8
mem      35913982 io_u 30071648, mem 3007b000
mem      40697880 io_u alloc 30071278, index 6
mem      27590726 io_u 30071858, mem 3007c000
mem      23592972 io_u 30071648, mem 3007b000
mem      61866222 io_u 30071648, mem 3007b000
mem      52363504 io_u alloc 30071648, index 7
mem      59441406 io_u 30071858, mem 3007c000
mem      30736610 io_u alloc 30071858, index 8
mem      24248348 io_u alloc 30071858, index 8
mem      23199860 io_u alloc 30071858, index 8
mem      49283312 io_u alloc 30071858, index 8
mem      21430374 io_u alloc 30071a68, index 9
mem      14745690 io_u 30071858, mem 3007c000
mem      50331692 io_u 30071858, mem 3007c000
mem      44367944 io_u alloc 30071858, index 8
mem      59113590 io_u 30071858, mem 3007c000
mem      35913982 io_u alloc 30071858, index 8
mem      40697880 io_u 30071278, mem 3007a000
mem      27590726 io_u alloc 30071a68, index 9
mem      23592972 io_u alloc 30071858, index 8
mem      52363504 io_u 30071648, mem 3007b000
mem      61866222 io_u alloc 30071858, index 8
mem      59441406 io_u alloc 30071a68, index 9
mem      30736610 io_u 30071858, mem 3007c000
mem      24248348 io_u 30071858, mem 3007c000
mem      23199860 io_u 30071858, mem 3007c000
mem      49283312 io_u 30071858, mem 3007c000
mem      21430374 io_u 30071a68, mem 3007d000
mem      14745690 io_u alloc 30071a68, index 9
mem      50331692 io_u alloc 30071a68, index 9
mem      44367944 io_u 30071858, mem 3007c000
mem      35913982 io_u 30071858, mem 3007c000
mem      59113590 io_u alloc 30071a68, index 9
mem      40697880 io_u alloc 30071648, index 7
mem      27590726 io_u 30071a68, mem 3007d000
mem      23592972 io_u 30071858, mem 3007c000
mem      52363504 io_u alloc 30071858, index 8
mem      61866222 io_u 30071858, mem 3007c000
mem      59441406 io_u 30071a68, mem 3007d000
mem      30736610 io_u alloc 30071a68, index 9
mem      24248348 io_u alloc 30071a68, index 9
mem      23199860 io_u alloc 30071a68, index 9
mem      49283312 io_u alloc 30071a68, index 9
mem      21430374 io_u alloc 30071c78, index 10
mem      14745690 io_u 30071a68, mem 3007d000
mem      50331692 io_u 30071a68, mem 3007d000
mem      44367944 io_u alloc 30071a68, index 9
mem      35913982 io_u alloc 30071a68, index 9
mem      59113590 io_u 30071a68, mem 3007d000
mem      40697880 io_u 30071648, mem 3007b000
mem      27590726 io_u alloc 30071c78, index 10
mem      23592972 io_u alloc 30071a68, index 9
mem      52363504 io_u 30071858, mem 3007c000
mem      61866222 io_u alloc 30071a68, index 9
mem      59441406 io_u alloc 30071c78, index 10
mem      30736610 io_u 30071a68, mem 3007d000
mem      24248348 io_u 30071a68, mem 3007d000
mem      23199860 io_u 30071a68, mem 3007d000
mem      49283312 io_u 30071a68, mem 3007d000
mem      21430374 io_u 30071c78, mem 3007e000
mem      14745690 io_u alloc 30071c78, index 10
mem      50331692 io_u alloc 30071c78, index 10
mem      44367944 io_u 30071a68, mem 3007d000
mem      35913982 io_u 30071a68, mem 3007d000
mem      59113590 io_u alloc 30071c78, index 10
mem      40697880 io_u alloc 30071858, index 8
mem      27590726 io_u 30071c78, mem 3007e000
mem      23592972 io_u 30071a68, mem 3007d000
mem      61866222 io_u 30071a68, mem 3007d000
mem      52363504 io_u alloc 30071a68, index 9
mem      59441406 io_u 30071c78, mem 3007e000
mem      30736610 io_u alloc 30071c78, index 10
mem      24248348 io_u alloc 30071c78, index 10
mem      23199860 io_u alloc 30071c78, index 10
mem      49283312 io_u alloc 30071c78, index 10
mem      14745690 io_u 30071c78, mem 3007e000
mem      21430374 io_u alloc 30071e08, index 11
mem      50331692 io_u 30071c78, mem 3007e000
mem      44367944 io_u alloc 30071c78, index 10
mem      59113590 io_u 30071c78, mem 3007e000
mem      35913982 io_u alloc 30071c78, index 10
mem      40697880 io_u 30071858, mem 3007c000
mem      27590726 io_u alloc 30071e08, index 11
mem      23592972 io_u alloc 30071c78, index 10
mem      52363504 io_u 30071a68, mem 3007d000
mem      61866222 io_u alloc 30071c78, index 10
mem      59441406 io_u alloc 30071e08, index 11
mem      30736610 io_u 30071c78, mem 3007e000
mem      24248348 io_u 30071c78, mem 3007e000
mem      23199860 io_u 30071c78, mem 3007e000
mem      49283312 io_u 30071c78, mem 3007e000
mem      21430374 io_u 30071e08, mem 3007f000
mem      14745690 io_u alloc 30071e08, index 11
mem      50331692 io_u alloc 30071e08, index 11
mem      44367944 io_u 30071c78, mem 3007e000
mem      35913982 io_u 30071c78, mem 3007e000
mem      59113590 io_u alloc 30071e08, index 11
mem      40697880 io_u alloc 30071a68, index 9
mem      27590726 io_u 30071e08, mem 3007f000
mem      23592972 io_u 30071c78, mem 3007e000
mem      52363504 io_u alloc 30071c78, index 10
mem      61866222 io_u 30071c78, mem 3007e000
mem      59441406 io_u 30071e08, mem 3007f000
mem      30736610 io_u alloc 30071e08, index 11
mem      24248348 io_u alloc 30071e08, index 11
mem      23199860 io_u alloc 30071e08, index 11
mem      21430374 io_u alloc 30072018, index 12
mem      49283312 io_u alloc 30071e08, index 11
mem      14745690 io_u 30071e08, mem 3007f000
mem      50331692 io_u 30071e08, mem 3007f000
mem      44367944 io_u alloc 30071e08, index 11
mem      35913982 io_u alloc 30071e08, index 11
mem      59113590 io_u 30071e08, mem 3007f000
mem      40697880 io_u 30071a68, mem 3007d000
mem      27590726 io_u alloc 30072018, index 12
mem      23592972 io_u alloc 30071e08, index 11
mem      52363504 io_u 30071c78, mem 3007e000
mem      59441406 io_u alloc 30072018, index 12
mem      61866222 io_u alloc 30071e08, index 11
mem      30736610 io_u 30071e08, mem 3007f000
mem      24248348 io_u 30071e08, mem 3007f000
mem      23199860 io_u 30071e08, mem 3007f000
mem      21430374 io_u 30072018, mem 30080000
mem      49283312 io_u 30071e08, mem 3007f000
mem      14745690 io_u alloc 30072018, index 12
mem      50331692 io_u alloc 30072018, index 12
mem      44367944 io_u 30071e08, mem 3007f000
mem      35913982 io_u 30071e08, mem 3007f000
mem      59113590 io_u alloc 30072018, index 12
mem      27590726 io_u 30072018, mem 30080000
mem      40697880 io_u alloc 30071c78, index 10
mem      23592972 io_u 30071e08, mem 3007f000
mem      59441406 io_u 30072018, mem 30080000
mem      61866222 io_u 30071e08, mem 3007f000
mem      52363504 io_u alloc 30071e08, index 11
mem      30736610 io_u alloc 30072018, index 12
mem      24248348 io_u alloc 30072018, index 12
mem      23199860 io_u alloc 30072018, index 12
mem      21430374 io_u alloc 30072228, index 13
mem      49283312 io_u alloc 30072018, index 12
mem      14745690 io_u 30072018, mem 30080000
mem      50331692 io_u 30072018, mem 30080000
mem      44367944 io_u alloc 30072018, index 12
mem      35913982 io_u alloc 30072018, index 12
mem      59113590 io_u 30072018, mem 30080000
mem      40697880 io_u 30071c78, mem 3007e000
mem      27590726 io_u alloc 30072228, index 13
mem      23592972 io_u alloc 30072018, index 12
mem      59441406 io_u alloc 30072228, index 13
mem      52363504 io_u 30071e08, mem 3007f000
mem      61866222 io_u alloc 30072018, index 12
mem      30736610 io_u 30072018, mem 30080000
mem      24248348 io_u 30072018, mem 30080000
mem      23199860 io_u 30072018, mem 30080000
mem      21430374 io_u 30072228, mem 30081000
mem      49283312 io_u 30072018, mem 30080000
mem      14745690 io_u alloc 30072228, index 13
mem      50331692 io_u alloc 30072228, index 13
mem      44367944 io_u 30072018, mem 30080000
mem      35913982 io_u 30072018, mem 30080000
mem      59113590 io_u alloc 30072228, index 13
mem      40697880 io_u alloc 30071e08, index 11
mem      27590726 io_u 30072228, mem 30081000
mem      23592972 io_u 30072018, mem 30080000
mem      59441406 io_u 30072228, mem 30081000
mem      52363504 io_u alloc 30072018, index 12
mem      61866222 io_u 30072018, mem 30080000
mem      30736610 io_u alloc 30072228, index 13
mem      24248348 io_u alloc 30072228, index 13
mem      23199860 io_u alloc 30072228, index 13
mem      21430374 io_u alloc 30085a28, index 14
mem      49283312 io_u alloc 30072228, index 13
mem      14745690 io_u 30072228, mem 30081000
mem      50331692 io_u 30072228, mem 30081000
mem      44367944 io_u alloc 30072228, index 13
mem      59113590 io_u 30072228, mem 30081000
mem      35913982 io_u alloc 30072228, index 13
mem      40697880 io_u 30071e08, mem 3007f000
mem      27590726 io_u alloc 30085a28, index 14
mem      23592972 io_u alloc 30072228, index 13
mem      59441406 io_u alloc 30085a28, index 14
mem      52363504 io_u 30072018, mem 30080000
mem      30736610 io_u 30072228, mem 30081000
mem      61866222 io_u alloc 30072228, index 13
mem      24248348 io_u 30072228, mem 30081000
mem      23199860 io_u 30072228, mem 30081000
mem      21430374 io_u 30085a28, mem 30082000
mem      49283312 io_u 30072228, mem 30081000
mem      14745690 io_u alloc 30085a28, index 14
mem      50331692 io_u alloc 30085a28, index 14
mem      44367944 io_u 30072228, mem 30081000
mem      35913982 io_u 30072228, mem 30081000
mem      59113590 io_u alloc 30085a28, index 14
mem      40697880 io_u alloc 30072018, index 12
mem      27590726 io_u 30085a28, mem 30082000
mem      23592972 io_u 30072228, mem 30081000
mem      59441406 io_u 30085a28, mem 30082000
mem      52363504 io_u alloc 30072228, index 13
mem      30736610 io_u alloc 30085a28, index 14
mem      61866222 io_u 30072228, mem 30081000
mem      24248348 io_u alloc 30085a28, index 14
mem      23199860 io_u alloc 30085a28, index 14
mem      21430374 io_u alloc 30085c38, index 15
mem      14745690 io_u 30085a28, mem 30082000
mem      50331692 io_u 30085a28, mem 30082000
mem      49283312 io_u alloc 30085a28, index 14
mem      44367944 io_u alloc 30085a28, index 14
mem      59113590 io_u 30085a28, mem 30082000
mem      35913982 io_u alloc 30085a28, index 14
mem      40697880 io_u 30072018, mem 30080000
mem      27590726 io_u alloc 30085c38, index 15
mem      23592972 io_u alloc 30085a28, index 14
mem      59441406 io_u alloc 30085c38, index 15
mem      52363504 io_u 30072228, mem 30081000
mem      30736610 io_u 30085a28, mem 30082000
mem      24248348 io_u 30085a28, mem 30082000
mem      61866222 io_u alloc 30085a28, index 14
mem      23199860 io_u 30085a28, mem 30082000
mem      21430374 io_u 30085c38, mem 30083000
mem      14745690 io_u alloc 30085c38, index 15
mem      50331692 io_u alloc 30085c38, index 15
mem      49283312 io_u 30085a28, mem 30082000
mem      44367944 io_u 30085a28, mem 30082000
mem      35913982 io_u 30085a28, mem 30082000
mem      59113590 io_u alloc 30085c38, index 15
mem      27590726 io_u 30085c38, mem 30083000
mem      40697880 io_u alloc 30072228, index 13
mem      23592972 io_u 30085a28, mem 30082000
mem      59441406 io_u 30085c38, mem 30083000
mem      52363504 io_u alloc 30085a28, index 14
mem      30736610 io_u alloc 30085c38, index 15
mem      24248348 io_u alloc 30085c38, index 15
mem      61866222 io_u 30085a28, mem 30082000
mem      23199860 io_u alloc 30085c38, index 15
mem      14745690 io_u 30085c38, mem 30083000
file     21430374 trying file 4ktest.7.0 10
mem      50331692 io_u 30085c38, mem 30083000
mem      49283312 io_u alloc 30085c38, index 15
mem      44367944 io_u alloc 30085c38, index 15
mem      59113590 io_u 30085c38, mem 30083000
mem      35913982 io_u alloc 30085c38, index 15
mem      40697880 io_u 30072228, mem 30081000
mem      23592972 io_u alloc 30085c38, index 15
file     27590726 trying file 4ktest.6.0 10
mem      52363504 io_u 30085a28, mem 30082000
mem      30736610 io_u 30085c38, mem 30083000
mem      24248348 io_u 30085c38, mem 30083000
mem      23199860 io_u 30085c38, mem 30083000
file     59441406 trying file 4ktest.0.0 10
mem      61866222 io_u alloc 30085c38, index 15
file     21430374 fd open 4ktest.7.0
mem      49283312 io_u 30085c38, mem 30083000
mem      44367944 io_u 30085c38, mem 30083000
file     14745690 trying file 4ktest.15.0 10
file     50331692 trying file 4ktest.11.0 10
mem      35913982 io_u 30085c38, mem 30083000
mem      40697880 io_u alloc 30085a28, index 14
file     59113590 trying file 4ktest.10.0 10
mem      23592972 io_u 30085c38, mem 30083000
file     27590726 fd open 4ktest.6.0
mem      52363504 io_u alloc 30085c38, index 15
file     30736610 trying file 4ktest.14.0 10
file     59441406 fd open 4ktest.0.0
file     23199860 trying file 4ktest.12.0 10
file     24248348 trying file 4ktest.13.0 10
mem      61866222 io_u 30085c38, mem 30083000
file     21430374 file not found in hash 4ktest.7.0
file     14745690 fd open 4ktest.15.0
file     49283312 trying file 4ktest.4.0 10
file     50331692 fd open 4ktest.11.0
mem      40697880 io_u 30085a28, mem 30082000
file     44367944 trying file 4ktest.3.0 10
file     59113590 fd open 4ktest.10.0
file     35913982 trying file 4ktest.5.0 10
file     27590726 file not found in hash 4ktest.6.0
mem      52363504 io_u 30085c38, mem 30083000
file     23592972 trying file 4ktest.9.0 10
file     30736610 fd open 4ktest.14.0
file     59441406 file not found in hash 4ktest.0.0
file     23199860 fd open 4ktest.12.0
file     24248348 fd open 4ktest.13.0
file     61866222 trying file 4ktest.8.0 10
file     14745690 file not found in hash 4ktest.15.0
file     49283312 fd open 4ktest.4.0
file     50331692 file not found in hash 4ktest.11.0
mem      40697880 io_u alloc 30085c38, index 15
file     44367944 fd open 4ktest.3.0
file     59113590 file not found in hash 4ktest.10.0
file     35913982 fd open 4ktest.5.0
file     23592972 fd open 4ktest.9.0
file     52363504 trying file 4ktest.2.0 10
file     30736610 file not found in hash 4ktest.14.0
file     23199860 file not found in hash 4ktest.12.0
file     24248348 file not found in hash 4ktest.13.0
file     61866222 fd open 4ktest.8.0
file     49283312 file not found in hash 4ktest.4.0
mem      40697880 io_u 30085c38, mem 30083000
file     44367944 file not found in hash 4ktest.3.0
file     35913982 file not found in hash 4ktest.5.0
file     23592972 file not found in hash 4ktest.9.0
file     52363504 fd open 4ktest.2.0
file     61866222 file not found in hash 4ktest.8.0
file     40697880 trying file 4ktest.1.0 10
file     21430374 get file 4ktest.7.0, ref=0
file     27590726 get file 4ktest.6.0, ref=0
file     52363504 file not found in hash 4ktest.2.0
file     40697880 fd open 4ktest.1.0
file     59441406 get file 4ktest.0.0, ref=0
file     50331692 get file 4ktest.11.0, ref=0
io       21430374 invalidate cache 4ktest.7.0: 0/104857600
io       27590726 invalidate cache 4ktest.6.0: 0/104857600
file     14745690 get file 4ktest.15.0, ref=0
file     59113590 get file 4ktest.10.0, ref=0
file     40697880 file not found in hash 4ktest.1.0
io       59441406 invalidate cache 4ktest.0.0: 0/104857600
io       50331692 invalidate cache 4ktest.11.0: 0/104857600
file     24248348 get file 4ktest.13.0, ref=0
file     23199860 get file 4ktest.12.0, ref=0
file     30736610 get file 4ktest.14.0, ref=0
file     23592972 get file 4ktest.9.0, ref=0
file     44367944 get file 4ktest.3.0, ref=0
file     35913982 get file 4ktest.5.0, ref=0
file     49283312 get file 4ktest.4.0, ref=0
file     61866222 get file 4ktest.8.0, ref=0
io       14745690 invalidate cache 4ktest.15.0: 0/104857600
io       59113590 invalidate cache 4ktest.10.0: 0/104857600
file     27590726 fd close 4ktest.6.0
file     21430374 fd close 4ktest.7.0
io       24248348 invalidate cache 4ktest.13.0: 0/104857600
io       23199860 invalidate cache 4ktest.12.0: 0/104857600
file     59441406 fd close 4ktest.0.0
io       30736610 invalidate cache 4ktest.14.0: 0/104857600
file     50331692 fd close 4ktest.11.0
io       23592972 invalidate cache 4ktest.9.0: 0/104857600
io       44367944 invalidate cache 4ktest.3.0: 0/104857600
io       35913982 invalidate cache 4ktest.5.0: 0/104857600
io       49283312 invalidate cache 4ktest.4.0: 0/104857600
io       61866222 invalidate cache 4ktest.8.0: 0/104857600
file     21430374 error 1 on open of 4ktest.7.0
file     27590726 error 1 on open of 4ktest.6.0
file     59113590 fd close 4ktest.10.0
file     59441406 error 1 on open of 4ktest.0.0
file     24248348 fd close 4ktest.13.0
file     14745690 fd close 4ktest.15.0
file     23199860 fd close 4ktest.12.0
file     50331692 error 1 on open of 4ktest.11.0
file     30736610 fd close 4ktest.14.0
file     21430374 get_next_file_rr: 0
file     27590726 get_next_file_rr: 0
file     59441406 get_next_file_rr: 0
file     59113590 error 1 on open of 4ktest.10.0
file     24248348 error 1 on open of 4ktest.13.0
file     14745690 error 1 on open of 4ktest.15.0
file     23199860 error 1 on open of 4ktest.12.0
file     23592972 fd close 4ktest.9.0
file     50331692 get_next_file_rr: 0
file     44367944 fd close 4ktest.3.0
file     35913982 fd close 4ktest.5.0
file     30736610 error 1 on open of 4ktest.14.0
file     21430374 get_next_file: NULL
file     27590726 get_next_file: NULL
file     49283312 fd close 4ktest.4.0
file     61866222 fd close 4ktest.8.0
file     59441406 get_next_file: NULL
file     59113590 get_next_file_rr: 0
file     24248348 get_next_file_rr: 0
file     14745690 get_next_file_rr: 0
file     23199860 get_next_file_rr: 0
file     23592972 error 1 on open of 4ktest.9.0
file     50331692 get_next_file: NULL
file     44367944 error 1 on open of 4ktest.3.0
file     35913982 error 1 on open of 4ktest.5.0
file     30736610 get_next_file_rr: 0
io       21430374 io_u 30085c38, setting file failed
io       27590726 io_u 30085c38, setting file failed
file     49283312 error 1 on open of 4ktest.4.0
file     61866222 error 1 on open of 4ktest.8.0
io       59441406 io_u 30085c38, setting file failed
file     59113590 get_next_file: NULL
file     24248348 get_next_file: NULL
file     14745690 get_next_file: NULL
file     23199860 get_next_file: NULL
file     23592972 get_next_file_rr: 0
io       50331692 io_u 30085c38, setting file failed
file     44367944 get_next_file_rr: 0
file     35913982 get_next_file_rr: 0
file     30736610 get_next_file: NULL
io       21430374 get_io_u failed
io       27590726 get_io_u failed
file     49283312 get_next_file_rr: 0
file     61866222 get_next_file_rr: 0
io       59441406 get_io_u failed
io       59113590 io_u 30085c38, setting file failed
io       24248348 io_u 30085c38, setting file failed
io       14745690 io_u 30085c38, setting file failed
io       23199860 io_u 30085c38, setting file failed
file     23592972 get_next_file: NULL
io       50331692 get_io_u failed
file     44367944 get_next_file: NULL
file     35913982 get_next_file: NULL
io       30736610 io_u 30085c38, setting file failed
io       21430374 io_u_queued_completed: min=0
io       27590726 io_u_queued_completed: min=0
file     49283312 get_next_file: NULL
file     61866222 get_next_file: NULL
io       59441406 io_u_queued_completed: min=0
io       59113590 get_io_u failed
io       24248348 get_io_u failed
io       14745690 get_io_u failed
io       23199860 get_io_u failed
io       23592972 io_u 30085c38, setting file failed
io       50331692 io_u_queued_completed: min=0
io       44367944 io_u 30085c38, setting file failed
io       35913982 io_u 30085c38, setting file failed
io       30736610 get_io_u failed
io       21430374 getevents: 0
io       27590726 getevents: 0
io       49283312 io_u 30085c38, setting file failed
io       61866222 io_u 30085c38, setting file failed
io       59441406 getevents: 0
io       59113590 io_u_queued_completed: min=0
io       24248348 io_u_queued_completed: min=0
io       14745690 io_u_queued_completed: min=0
io       23199860 io_u_queued_completed: min=0
io       23592972 get_io_u failed
io       50331692 getevents: 0
io       44367944 get_io_u failed
io       35913982 get_io_u failed
io       30736610 io_u_queued_completed: min=0
io       49283312 get_io_u failed
fio: pid=27590726, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
fio: pid=21430374, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
io       61866222 get_io_u failed
io       59113590 getevents: 0
io       24248348 getevents: 0
fio: pid=59441406, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
io       14745690 getevents: 0
io       23199860 getevents: 0
io       23592972 io_u_queued_completed: min=0
io       44367944 io_u_queued_completed: min=0
fio: pid=50331692, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
io       35913982 io_u_queued_completed: min=0
file     52363504 get file 4ktest.2.0, ref=0
file     40697880 get file 4ktest.1.0, ref=0
io       30736610 getevents: 0
file     27590726 close files
io       49283312 io_u_queued_completed: min=0
file     21430374 close files
io       61866222 io_u_queued_completed: min=0
fio: pid=59113590, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
fio: pid=24248348, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     59441406 close files
fio: pid=14745690, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
fio: pid=23199860, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
io       23592972 getevents: 0
io       44367944 getevents: 0
file     50331692 close files
io       35913982 getevents: 0
io       52363504 invalidate cache 4ktest.2.0: 0/104857600
io       40697880 invalidate cache 4ktest.1.0: 0/104857600
fio: pid=30736610, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
io       49283312 getevents: 0
file     27590726 put file 4ktest.6.0, ref=1
io       61866222 getevents: 0
file     21430374 put file 4ktest.7.0, ref=1
file     59113590 close files
file     24248348 close files
file     14745690 close files
file     23199860 close files
file     59441406 put file 4ktest.0.0, ref=1
fio: pid=23592972, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
fio: pid=44367944, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     50331692 put file 4ktest.11.0, ref=1
fio: pid=35913982, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     30736610 close files
file     27590726 fd close 4ktest.6.0
fio: pid=49283312, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     21430374 fd close 4ktest.7.0
fio: pid=61866222, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     59113590 put file 4ktest.10.0, ref=1
file     24248348 put file 4ktest.13.0, ref=1
file     14745690 put file 4ktest.15.0, ref=1
file     23199860 put file 4ktest.12.0, ref=1
file     59441406 fd close 4ktest.0.0
file     23592972 close files
file     40697880 fd close 4ktest.1.0
file     52363504 fd close 4ktest.2.0
file     44367944 close files
file     50331692 fd close 4ktest.11.0
file     35913982 close files
file     30736610 put file 4ktest.14.0, ref=1
mem      27590726 free malloc mem 30073a18
file     49283312 close files
mem      21430374 free malloc mem 30073a18
file     61866222 close files
file     59113590 fd close 4ktest.10.0
file     24248348 fd close 4ktest.13.0
file     14745690 fd close 4ktest.15.0
file     23199860 fd close 4ktest.12.0
file     40697880 error 1 on open of 4ktest.1.0
file     23592972 put file 4ktest.9.0, ref=1
file     52363504 error 1 on open of 4ktest.2.0
file     44367944 put file 4ktest.3.0, ref=1
mem      59441406 free malloc mem 30073a18
file     35913982 put file 4ktest.5.0, ref=1
file     30736610 fd close 4ktest.14.0
io       27590726 close ioengine sync
mem      50331692 free malloc mem 30073a18
file     49283312 put file 4ktest.4.0, ref=1
io       21430374 close ioengine sync
file     61866222 put file 4ktest.8.0, ref=1
mem      24248348 free malloc mem 30073a18
mem      14745690 free malloc mem 30073a18
file     40697880 get_next_file_rr: 0
file     23592972 fd close 4ktest.9.0
mem      59113590 free malloc mem 30073a18
mem      23199860 free malloc mem 30073a18
file     52363504 get_next_file_rr: 0
file     44367944 fd close 4ktest.3.0
io       59441406 close ioengine sync
file     35913982 fd close 4ktest.5.0
io       50331692 close ioengine sync
io       27590726 free ioengine sync
file     49283312 fd close 4ktest.4.0
mem      30736610 free malloc mem 30073a18
io       21430374 free ioengine sync
file     61866222 fd close 4ktest.8.0
io       24248348 close ioengine sync
io       14745690 close ioengine sync
file     40697880 get_next_file: NULL
io       59113590 close ioengine sync
io       23199860 close ioengine sync
file     52363504 get_next_file: NULL
io       59441406 free ioengine sync
mem      23592972 free malloc mem 30073a18
io       50331692 free ioengine sync
mem      44367944 free malloc mem 30073a18
io       30736610 close ioengine sync
process  27590726 pid=27590726: runstate 5 -> 9
mem      49283312 free malloc mem 30073a18
process  21430374 pid=21430374: runstate 5 -> 9
io       24248348 free ioengine sync
mem      35913982 free malloc mem 30073a18
io       14745690 free ioengine sync
mem      61866222 free malloc mem 30073a18
io       40697880 io_u 30085c38, setting file failed
io       59113590 free ioengine sync
io       23199860 free ioengine sync
io       52363504 io_u 30085c38, setting file failed
process  59441406 pid=59441406: runstate 5 -> 9
io       23592972 close ioengine sync
io       44367944 close ioengine sync
process  50331692 pid=50331692: runstate 5 -> 9
io       30736610 free ioengine sync
io       49283312 close ioengine sync
process  24248348 pid=24248348: runstate 5 -> 9
io       35913982 close ioengine sync
process  14745690 pid=14745690: runstate 5 -> 9
process  23003136 pid=21430374: runstate 9 -> 10
io       40697880 get_io_u failed
io       61866222 close ioengine sync
io       52363504 get_io_u failed
process  59113590 pid=59113590: runstate 5 -> 9
process  23199860 pid=23199860: runstate 5 -> 9
io       23592972 free ioengine sync
io       44367944 free ioengine sync
process  30736610 pid=30736610: runstate 5 -> 9
io       49283312 free ioengine sync
io       35913982 free ioengine sync
io       40697880 io_u_queued_completed: min=0
io       61866222 free ioengine sync
io       52363504 io_u_queued_completed: min=0
process  23003136 pid=27590726: runstate 9 -> 10
process  23592972 pid=23592972: runstate 5 -> 9
process  44367944 pid=44367944: runstate 5 -> 9
process  49283312 pid=49283312: runstate 5 -> 9
process  35913982 pid=35913982: runstate 5 -> 9
io       40697880 getevents: 0
process  61866222 pid=61866222: runstate 5 -> 9
io       52363504 getevents: 0
fio: pid=40697880, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     40697880 close files
file     40697880 put file 4ktest.1.0, ref=1
fio: pid=52363504, err=22/file:filesetup.c:418, func=invalidate_cache, error=Invalid argument
file     52363504 close files
file     40697880 fd close 4ktest.1.0
file     52363504 put file 4ktest.2.0, ref=1
file     52363504 fd close 4ktest.2.0
mem      40697880 free malloc mem 30073a18
io       40697880 close ioengine sync
io       40697880 free ioengine sync
process  40697880 pid=40697880: runstate 5 -> 9
mem      52363504 free malloc mem 30073a18
io       52363504 close ioengine sync
io       52363504 free ioengine sync
process  52363504 pid=52363504: runstate 5 -> 9
process  23003136 pid=59441406: runstate 9 -> 10
process  23003136 pid=14745690: runstate 9 -> 10
process  23003136 pid=30736610: runstate 9 -> 10
process  23003136 pid=24248348: runstate 9 -> 10
process  23003136 pid=23199860: runstate 9 -> 10
process  23003136 pid=50331692: runstate 9 -> 10
process  23003136 pid=59113590: runstate 9 -> 10
process  23003136 pid=23592972: runstate 9 -> 10
process  23003136 pid=61866222: runstate 9 -> 10
process  23003136 pid=35913982: runstate 9 -> 10
process  23003136 pid=49283312: runstate 9 -> 10
process  23003136 pid=44367944: runstate 9 -> 10
process  23003136 pid=52363504: runstate 9 -> 10
process  23003136 pid=40697880: runstate 9 -> 10
process  23003136 terminate group_id=-1
process  23003136 setting terminate on 4ktest/59441406
process  23003136 setting terminate on 4ktest/14745690
process  23003136 setting terminate on 4ktest/30736610
process  23003136 setting terminate on 4ktest/24248348
process  23003136 setting terminate on 4ktest/23199860
process  23003136 setting terminate on 4ktest/50331692
process  23003136 setting terminate on 4ktest/59113590
process  23003136 setting terminate on 4ktest/23592972
process  23003136 setting terminate on 4ktest/61866222
process  23003136 setting terminate on 4ktest/21430374
process  23003136 setting terminate on 4ktest/27590726
process  23003136 setting terminate on 4ktest/35913982
process  23003136 setting terminate on 4ktest/49283312
process  23003136 setting terminate on 4ktest/44367944
process  23003136 setting terminate on 4ktest/52363504
process  23003136 setting terminate on 4ktest/40697880


Run status group 0 (all jobs):
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
parse    23003136 free options
io       23003136 ioengine cpuio unregistered
io       23003136 ioengine mmap unregistered
io       23003136 ioengine sync unregistered
io       23003136 ioengine psync unregistered
io       23003136 ioengine vsync unregistered
io       23003136 ioengine pvsync unregistered
io       23003136 ioengine null unregistered
io       23003136 ioengine net unregistered
profile  23003136 unregister profile 'tiobench'
profile  23003136 unregister profile 'act'
io       23003136 ioengine posixaio unregistered

Re: fio on AIX - fallocate error

[Jens Axboe]

On 04/10/2014 10:44 AM, Carino, Mario wrote:
> Debug output attached

Are you running with the previous patch for the fadvise error? It looks 
like you are not, which is why it exits.

-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

I don't know. I didn't download the source code. Where can I get the version with the patch?

Mario Carino


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 12:57 PM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 04/10/2014 10:44 AM, Carino, Mario wrote:
> Debug output attached

Are you running with the previous patch for the fadvise error? It looks like you are not, which is why it exits.

--
Jens Axboe

Re: fio on AIX - fallocate error

[Jens Axboe]

On 04/10/2014 10:59 AM, Carino, Mario wrote:
> I don't know. I didn't download the source code. Where can I get the version with the patch?

http://brick.kernel.dk/snaps/fio-git-latest.tar.gz

-- 
Jens Axboe

RE: fio on AIX - fallocate error

[Carino, Mario]

Hey! I think we're in business now. I'm off for a few days vacation now, so I'll probably pick this up again over the weekend, but my first test looked good.

Thanks so much!

Mario Carino


-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Thursday, April 10, 2014 1:01 PM
To: Carino, Mario; fio@vger.kernel.org
Subject: Re: fio on AIX - fallocate error

On 04/10/2014 10:59 AM, Carino, Mario wrote:
> I don't know. I didn't download the source code. Where can I get the version with the patch?

http://brick.kernel.dk/snaps/fio-git-latest.tar.gz

-- 
Jens Axboe