Re: How to do strict synchronous i/o on Windows?

[Martin Steigerwald <Martin@lichtvoll.de>]

Am Dienstag, 14. August 2012 schrieb Greg Sullivan:
> On 15/08/2012, Martin Steigerwald <Martin@lichtvoll.de> wrote:
> > Am Dienstag, 14. August 2012 schrieb Greg Sullivan:
> >> On 15/08/2012, Martin Steigerwald <Martin@lichtvoll.de> wrote:
> >> > Am Dienstag, 14. August 2012 schrieb Greg Sullivan:
> >> >> On 15 August 2012 03:36, Martin Steigerwald <Martin@lichtvoll.de>
> >> >> wrote:
> >> >> > Hi Greg,
> >> > [�]
> >> >> > Am Dienstag, 14. August 2012 schrieb Greg Sullivan:
> >> >> >
> >> >> >> On Aug 14, 2012 11:06 PM, "Jens Axboe" <axboe@kernel.dk> wrote:
> >> >> >> >
> >> >> >> > On 08/14/2012 08:24 AM, Greg Sullivan wrote:
[�]
> >> >> Is it possible to read from more than file in a single job, in a
> >> >> round-robin fashion? I tried putting more than one file in a single
> >> >> job, but it only opened one file. If you mean to just do random reads
> >> >> in a single file - I've tried that, and the throughput is
> >> >> unrealistically low. I suspect it's because the read-ahead buffer
> >> >> cannot be effective for random accesses.  Of course, reading
> >> >> sequentially from a single file will result in a throughput that is
> >> >> far too high to simulate the application.
> >> >
> >> > Have you tried
> >> >
> >> >        nrfiles=int
> >> >               Number of files to use for this job.  Default: 1.
> >> >
> >> >        openfiles=int
> >> >               Number of files to keep open at the same time.  Default:
> >> >               nrfiles.
> >> >
> >> >        file_service_type=str
> > [�]
> >> > ? (see fio manpage).
> >> >
> >> > It seems to me that all you need is nrfiles. I�d bet that fio
> >> > distributes
> >> > the I/O size given among those files, but AFAIR there is something
> >> > about
> >> > that in fio documentation as well.
> >> >
> >> > Use the doc! ;)
> > [�]
> >> Yes, I have tried all that, and it works, except that it causes disk
> >> queuing, as I stated in my first post. I thought you meant to put all
> >> the files into a single [job name] section of the ini file, to enforce
> >> single threaded io.
> >
> > With just one job running at once?
> >
> > Can you post an example job file?
> >
> > Did you try the sync=1 / direct=1 suggestion from Bruce Chan?
> >
> > I only know the behaviour of fio on Linux where I/O depth of greater
> > than one is only possible with libaio and direct=1. The manpage hints
> > at I/O depth is one for all synchronous I/O engines, so I�d bet that
> > refers to Windows as well.
> >
> > Other than that I have no idea.
[�]
> One INI file, but a seperate [job name] section for each file, yes.
> According to Jens, because each [job name] is a seperate thread, and
> iodepth acts at the thread level, there will still be queuing at the
> device level. If there were a way to do what I want I think Jens would
> have told me, unfortunately.   ;)
> 
> direct io does at least allow me to do cache-less reads though - thankyou.

My suggestion is to use one job with several files.

martin@merkaba:/tmp> cat severalfiles.job 
[global]
size=1G
nrfiles=100

[read]
rw=read

[write]
stonewall
rw=write

(now these are two jobs, but stonewall lets the write job run after the
read one with cache invalidation if not disabled and if supported by OS)

martin@merkaba:/tmp> fio severalfiles.job
read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
write: (g=1): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
2.0.8
Starting 2 processes
read: Laying out IO file(s) (100 file(s) / 1023MB)
write: Laying out IO file(s) (100 file(s) / 1023MB)
Jobs: 1 (f=100)
read: (groupid=0, jobs=1): err= 0: pid=23377
[� lots of fast due to /tmp being a RAM-based filesystem � tmpfs �]


martin@merkaba:/tmp> ls -lh read.1.* | head
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.0
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.1
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.10
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.11
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.12
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.13
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.14
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.15
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.16
-rw-r--r-- 1 martin martin 11M Aug 14 23:15 read.1.17
[� only first ten displayed �]

martin@merkaba:/tmp> find -name "read.1*" 2>/dev/null | wc -l
100

100 files a 11M, due to rounding issues that may nicely add up to
the one GiB.

Raw sizes are:

martin@merkaba:/tmp> ls -l read.1.* | head                  
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.0
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.1
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.10
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.11
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.12
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.13
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.14
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.15
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.16
-rw-r--r-- 1 martin martin 10737418 Aug 14 23:20 read.1.17

Note: When I used filename, fio just created one files regardless of the
nrfiles setting. I would have expected it to use the filename as a prefix.
There might be some way to have it do that.

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7