unable to replay a trace generated w/ fio

unable to replay a trace generated w/ fio

[Alket Memushaj]

Hi all,

I am trying to replay traces with fio (2.1.5) and unable to do so. I
create the trace by adding --write_iolog to a simple job, ie: fio
--name=test --write_iolog=/var/tmp/ex.trace --time_based --runtime=10
--filename=/dev/sdb

I then try to replay this like: fio --name=replay
--read_iolog=/var/tmp/ex.trace but fio exits immediately with counters
set to 0

I would have expected this to run without issue - am I doing anything wrong?

Thanks,
Alket

Re: unable to replay a trace generated w/ fio

[Alket Memushaj]

nevermind - I tried w/ 2.0.3 and it worked so I'll just work with that.

On Sat, May 17, 2014 at 12:52 PM, Alket Memushaj <amemushaj@gmail.com> wrote:
> Hi all,
>
> I am trying to replay traces with fio (2.1.5) and unable to do so. I
> create the trace by adding --write_iolog to a simple job, ie: fio
> --name=test --write_iolog=/var/tmp/ex.trace --time_based --runtime=10
> --filename=/dev/sdb
>
> I then try to replay this like: fio --name=replay
> --read_iolog=/var/tmp/ex.trace but fio exits immediately with counters
> set to 0
>
> I would have expected this to run without issue - am I doing anything wrong?
>
> Thanks,
> Alket

Re: unable to replay a trace generated w/ fio

[Alket Memushaj]

I am not having luck with 2.1.7 either, but anything up to 2.0.11
works. It seems to terminate as soon as it completes reading the trace
file - at the end of which I have /dev/sdb close as that's what's
generated by fio itself when I set write_iolog=<file>

On Sat, May 17, 2014 at 1:10 PM, Alket Memushaj <amemushaj@gmail.com> wrote:
> nevermind - I tried w/ 2.0.3 and it worked so I'll just work with that.
>
> On Sat, May 17, 2014 at 12:52 PM, Alket Memushaj <amemushaj@gmail.com> wrote:
>> Hi all,
>>
>> I am trying to replay traces with fio (2.1.5) and unable to do so. I
>> create the trace by adding --write_iolog to a simple job, ie: fio
>> --name=test --write_iolog=/var/tmp/ex.trace --time_based --runtime=10
>> --filename=/dev/sdb
>>
>> I then try to replay this like: fio --name=replay
>> --read_iolog=/var/tmp/ex.trace but fio exits immediately with counters
>> set to 0
>>
>> I would have expected this to run without issue - am I doing anything wrong?
>>
>> Thanks,
>> Alket

Re: unable to replay a trace generated w/ fio

[Jens Axboe]

On 2014-05-18 04:01, Alket Memushaj wrote:
> I am not having luck with 2.1.7 either, but anything up to 2.0.11
> works. It seems to terminate as soon as it completes reading the trace
> file - at the end of which I have /dev/sdb close as that's what's
> generated by fio itself when I set write_iolog=<file>

I'll take a look, I bet it's an issue from around the time the file 
opening logic was added.

Can you send the exact job you used to create and replay files? Will 
make my life easier.

-- 
Jens Axboe

Re: unable to replay a trace generated w/ fio

[Allen Schade]

Jens Axboe <axboe@...> writes:

> 
> On 2014-05-18 04:01, Alket Memushaj wrote:
> > I am not having luck with 2.1.7 either, but anything up to 2.0.11
> > works. It seems to terminate as soon as it completes reading the trace
> > file - at the end of which I have /dev/sdb close as that's what's
> > generated by fio itself when I set write_iolog=<file>
> 
> I'll take a look, I bet it's an issue from around the time the file 
> opening logic was added.
> 
> Can you send the exact job you used to create and replay files? Will 
> make my life easier.
> 

Hi, I'm am running into this issue too.

Here are simple instructions to replicate: 

[hdc-write]
filename=/dev/hdc
ioengine=sync
iodepth=1
direct=1
thread=1
continue_on_error=1
read_iolog=trace.log


trace.log:
/dev/hdc add
/dev/hdc open
/dev/hdc write 2045557202944 8192
/dev/hdc close

./fio trace.fio 
hdc-write: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.1.7
Starting 1 thread
Segmentation fault (core dumped)

Re: unable to replay a trace generated w/ fio

[Allen Schade]

Sorry I missed a line at the top of the trace file:

fio version 2 iolog
/dev/hdc add
/dev/hdc open
/dev/hdc write 2045557202944 8192
/dev/hdc close

Re: unable to replay a trace generated w/ fio

[Jens Axboe]

On 2014-06-05 13:02, Allen Schade wrote:
> Jens Axboe <axboe@...> writes:
>
>>
>> On 2014-05-18 04:01, Alket Memushaj wrote:
>>> I am not having luck with 2.1.7 either, but anything up to 2.0.11
>>> works. It seems to terminate as soon as it completes reading the trace
>>> file - at the end of which I have /dev/sdb close as that's what's
>>> generated by fio itself when I set write_iolog=<file>
>>
>> I'll take a look, I bet it's an issue from around the time the file
>> opening logic was added.
>>
>> Can you send the exact job you used to create and replay files? Will
>> make my life easier.
>>
>
> Hi, I'm am running into this issue too.
>
> Here are simple instructions to replicate:
>
> [hdc-write]
> filename=/dev/hdc
> ioengine=sync
> iodepth=1
> direct=1
> thread=1
> continue_on_error=1
> read_iolog=trace.log
>
>
> trace.log:
> /dev/hdc add
> /dev/hdc open
> /dev/hdc write 2045557202944 8192
> /dev/hdc close
>
> ./fio trace.fio
> hdc-write: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
> fio-2.1.7
> Starting 1 thread
> Segmentation fault (core dumped)

OK, so the replay bug here is that you added a filename=. Due to a bug 
in the freeing of files logic (fio does do that for iolog loads), we 
don't reset this when freeing files. So fio attempts to close 2 files on 
exit, boom.

This is fixed in -git as of now. Replay of that file still looks weird, 
doesn't look like it actually does that IO. Will debug next...

-- 
Jens Axboe

Re: unable to replay a trace generated w/ fio

[Jens Axboe]

On 2014-06-09 19:53, Jens Axboe wrote:
> On 2014-06-05 13:02, Allen Schade wrote:
>> Jens Axboe <axboe@...> writes:
>>
>>>
>>> On 2014-05-18 04:01, Alket Memushaj wrote:
>>>> I am not having luck with 2.1.7 either, but anything up to 2.0.11
>>>> works. It seems to terminate as soon as it completes reading the trace
>>>> file - at the end of which I have /dev/sdb close as that's what's
>>>> generated by fio itself when I set write_iolog=<file>
>>>
>>> I'll take a look, I bet it's an issue from around the time the file
>>> opening logic was added.
>>>
>>> Can you send the exact job you used to create and replay files? Will
>>> make my life easier.
>>>
>>
>> Hi, I'm am running into this issue too.
>>
>> Here are simple instructions to replicate:
>>
>> [hdc-write]
>> filename=/dev/hdc
>> ioengine=sync
>> iodepth=1
>> direct=1
>> thread=1
>> continue_on_error=1
>> read_iolog=trace.log
>>
>>
>> trace.log:
>> /dev/hdc add
>> /dev/hdc open
>> /dev/hdc write 2045557202944 8192
>> /dev/hdc close
>>
>> ./fio trace.fio
>> hdc-write: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
>> fio-2.1.7
>> Starting 1 thread
>> Segmentation fault (core dumped)
>
> OK, so the replay bug here is that you added a filename=. Due to a bug
> in the freeing of files logic (fio does do that for iolog loads), we
> don't reset this when freeing files. So fio attempts to close 2 files on
> exit, boom.
>
> This is fixed in -git as of now. Replay of that file still looks weird,
> doesn't look like it actually does that IO. Will debug next...

That should be fixed as well now. Can you retry with current -git?

-- 
Jens Axboe