* [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
@ 2011-09-09 5:12 Taisuke Yamada
2011-09-09 8:37 ` Jens Axboe
0 siblings, 1 reply; 9+ messages in thread
From: Taisuke Yamada @ 2011-09-09 5:12 UTC (permalink / raw)
To: fio
[-- Attachment #1: Type: text/plain, Size: 711 bytes --]
Hi.
I'm now trying to run multi-threaded log replay, and noticed
all threads are accessing same sector on a disk. Of course,
this is an expected behavior of "replay".
But since I also wanted to have each thread accessing different
parts of the disk to increase a load, but still following the
access pattern of the original, I added "replay_rebase" option to
shift offset of IO request for each thread.
Althrough there's "1/2" in the subject, this patch is complete
by itself. Next one (2/2) is related, but handles different issue.
Signed-off-by: Taisuke Yamada <tai@rakugaki.org>
---
fio.h | 1 +
log.c | 2 +-
options.c | 8 ++++++++
3 files changed, 10 insertions(+), 1 deletions(-)
[-- Attachment #2: 0001-Added-replay_rebase-option-to-run-multi-threaded-log.patch --]
[-- Type: text/x-patch, Size: 1318 bytes --]
diff --git a/fio.h b/fio.h
index 6eb270d..b28e0ae 100644
--- a/fio.h
+++ b/fio.h
@@ -358,6 +358,7 @@ struct thread_options {
unsigned int gtod_offload;
enum fio_cs clocksource;
unsigned int no_stall;
+ unsigned int replay_rebase;
unsigned int trim_percentage;
unsigned int trim_batch;
unsigned int trim_zero;
diff --git a/log.c b/log.c
index f962864..57fc0f4 100644
--- a/log.c
+++ b/log.c
@@ -129,7 +129,7 @@ int read_iolog_get(struct thread_data *td, struct io_u *io_u)
io_u->ddir = ipo->ddir;
if (ipo->ddir != DDIR_WAIT) {
- io_u->offset = ipo->offset;
+ io_u->offset = ipo->offset + td->o.replay_rebase * td->thread_number;
io_u->buflen = ipo->len;
io_u->file = td->files[ipo->fileno];
get_file(io_u->file);
diff --git a/options.c b/options.c
index 74c24d0..aba14b1 100644
--- a/options.c
+++ b/options.c
@@ -1654,6 +1654,14 @@ static struct fio_option options[FIO_MAX_OPTS] = {
.help = "Replay all I/O onto this device, regardless of trace device",
},
{
+ .name = "replay_rebase",
+ .type = FIO_OPT_INT,
+ .off1 = td_var_offset(replay_rebase),
+ .def = "0",
+ .parent = "read_iolog",
+ .help = "Apply given offset for IO replay. Mainly for parallel replay.",
+ },
+ {
.name = "exec_prerun",
.type = FIO_OPT_STR_STORE,
.off1 = td_var_offset(exec_prerun),
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-09 5:12 [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors Taisuke Yamada
@ 2011-09-09 8:37 ` Jens Axboe
2011-09-15 6:23 ` Taisuke Yamada
0 siblings, 1 reply; 9+ messages in thread
From: Jens Axboe @ 2011-09-09 8:37 UTC (permalink / raw)
To: Taisuke Yamada; +Cc: fio
On 2011-09-09 07:12, Taisuke Yamada wrote:
> Hi.
>
> I'm now trying to run multi-threaded log replay, and noticed
> all threads are accessing same sector on a disk. Of course,
> this is an expected behavior of "replay".
>
> But since I also wanted to have each thread accessing different
> parts of the disk to increase a load, but still following the
> access pattern of the original, I added "replay_rebase" option to
> shift offset of IO request for each thread.
>
> Althrough there's "1/2" in the subject, this patch is complete
> by itself. Next one (2/2) is related, but handles different issue.
Thanks, I can see this being handy. I can apply this, but I would much
prefer if you resend it with the proper updates to HOWTO and fio.1
adding the new option. Otherwise nobody is going to learn about it.
--
Jens Axboe
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
@ 2011-09-12 5:30 Taisuke Yamada
0 siblings, 0 replies; 9+ messages in thread
From: Taisuke Yamada @ 2011-09-12 5:30 UTC (permalink / raw)
To: fio
[-- Attachment #1: Type: text/plain, Size: 771 bytes --]
> But since I also wanted to have each thread accessing different
> parts of the disk to increase a load, but still following the
> access pattern of the original, I added "replay_rebase" option to
> shift offset of IO request for each thread.
>
>> Thanks, I can see this being handy. I can apply this, but I would
>> much prefer if you resend it with the proper updates to HOWTO and
>> fio.1 adding the new option. Otherwise nobody is going to learn about it.
Yes, you're right.
Here's an updated patch with documentation.
Best Regards,
Signed-off-by: Taisuke Yamada <tai@rakugaki.org>
---
HOWTO | 9 +++++++++
fio.1 | 9 +++++++++
fio.h | 1 +
log.c | 2 +-
options.c | 8 ++++++++
5 files changed, 28 insertions(+), 1 deletions(-)
[-- Attachment #2: 0001-Added-replay_rebase-option-to-run-multi-threaded-log.patch --]
[-- Type: text/x-patch, Size: 3277 bytes --]
diff --git a/HOWTO b/HOWTO
index 724b604..132dc4a 100644
--- a/HOWTO
+++ b/HOWTO
@@ -1078,6 +1078,15 @@ replay_redirect=str While replaying I/O patterns using read_iolog the
independent fio invocations. Unfortuantely this also breaks
the strict time ordering between multiple device accesses.
+replay_rebase=int While replaying I/O patterns using read_iolog, the
+ default behavior is to replay the log as-is, even when
+ multiple threads are replaying the same log. This means
+ all threads are accessing the same sector on the device.
+ By specifying this option, each thread will be accessing
+ different sector by adding extra rebase offset. Offset is
+ simply given by "replay_rebase * internal thread number",
+ so IO access pattern itself is preserved.
+
write_bw_log=str If given, write a bandwidth log of the jobs in this job
file. Can be used to store data of the bandwidth of the
jobs in their lifetime. The included fio_generate_plots
diff --git a/fio.1 b/fio.1
index ffc97c9..702e4e3 100644
--- a/fio.1
+++ b/fio.1
@@ -823,6 +823,15 @@ is to replay the IOPS onto the major/minor device that each IOP was recorded
from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
single specified device regardless of the device it was recorded from.
.TP
+.BI replay_rebase \fR=\fPint
+While replaying I/O patterns using read_iolog, the default behavior
+is to replay the log as-is, even when multiple threads are replaying
+the same log. This means all threads are accessing the same sector
+on the device. By specifying this option, each thread will be accessing
+different sector by adding extra rebase offset. Offset is simply given
+by "replay_rebase * internal thread number", so IO access pattern itself
+is preserved.
+.TP
.B write_bw_log \fR=\fPstr
If given, write a bandwidth log of the jobs in this job file. Can be used to
store data of the bandwidth of the jobs in their lifetime. The included
diff --git a/fio.h b/fio.h
index 6eb270d..b28e0ae 100644
--- a/fio.h
+++ b/fio.h
@@ -358,6 +358,7 @@ struct thread_options {
unsigned int gtod_offload;
enum fio_cs clocksource;
unsigned int no_stall;
+ unsigned int replay_rebase;
unsigned int trim_percentage;
unsigned int trim_batch;
unsigned int trim_zero;
diff --git a/log.c b/log.c
index f962864..57fc0f4 100644
--- a/log.c
+++ b/log.c
@@ -129,7 +129,7 @@ int read_iolog_get(struct thread_data *td, struct io_u *io_u)
io_u->ddir = ipo->ddir;
if (ipo->ddir != DDIR_WAIT) {
- io_u->offset = ipo->offset;
+ io_u->offset = ipo->offset + td->o.replay_rebase * td->thread_number;
io_u->buflen = ipo->len;
io_u->file = td->files[ipo->fileno];
get_file(io_u->file);
diff --git a/options.c b/options.c
index 74c24d0..aba14b1 100644
--- a/options.c
+++ b/options.c
@@ -1654,6 +1654,14 @@ static struct fio_option options[FIO_MAX_OPTS] = {
.help = "Replay all I/O onto this device, regardless of trace device",
},
{
+ .name = "replay_rebase",
+ .type = FIO_OPT_INT,
+ .off1 = td_var_offset(replay_rebase),
+ .def = "0",
+ .parent = "read_iolog",
+ .help = "Apply given offset for IO replay. Mainly for parallel replay.",
+ },
+ {
.name = "exec_prerun",
.type = FIO_OPT_STR_STORE,
.off1 = td_var_offset(exec_prerun),
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-09 8:37 ` Jens Axboe
@ 2011-09-15 6:23 ` Taisuke Yamada
2011-09-15 11:10 ` Jens Axboe
2011-09-15 11:14 ` Jens Axboe
0 siblings, 2 replies; 9+ messages in thread
From: Taisuke Yamada @ 2011-09-15 6:23 UTC (permalink / raw)
To: Jens Axboe; +Cc: fio
[-- Attachment #1: Type: text/plain, Size: 860 bytes --]
>> [snip], I added "replay_rebase" option to shift offset of IO request for
>> each thread.
>
> Thanks, I can see this being handy. I can apply this, but I would much
> prefer if you resend it with the proper updates to HOWTO and fio.1
> adding the new option. Otherwise nobody is going to learn about it.
Just in case you have missed (I had to break In-Reply-To chain as git
format-patch did not respect it). I'm re-posting updated patch with docs.
Please apply #1 and #2 if it seems acceptable.
I believe #2 patch is also worth adding to fio, but not sure if I should make
it optional feature (/w doc) or not. By wraparounding offset address, it'll
probably make benchmark result look better than actual for rotational disk
drives. OTOH, it's better than failing to run just because trace-generating
device was larger than replaying device.
Best Regards,
[-- Attachment #2: 0001-Added-replay_rebase-option-to-run-multi-threaded-log.patch --]
[-- Type: text/x-patch, Size: 3727 bytes --]
From 6f47bef547fb4cc7480b44f6ed832a2e5a3b4150 Mon Sep 17 00:00:00 2001
From: Taisuke Yamada <tai@rakugaki.org>
Date: Thu, 8 Sep 2011 21:32:01 +0900
Subject: [PATCH 1/2] Added replay_rebase option to run multi-threaded log
replay on different disk sectors.
---
HOWTO | 9 +++++++++
fio.1 | 9 +++++++++
fio.h | 1 +
log.c | 2 +-
options.c | 8 ++++++++
5 files changed, 28 insertions(+), 1 deletions(-)
diff --git a/HOWTO b/HOWTO
index 724b604..132dc4a 100644
--- a/HOWTO
+++ b/HOWTO
@@ -1078,6 +1078,15 @@ replay_redirect=str While replaying I/O patterns using read_iolog the
independent fio invocations. Unfortuantely this also breaks
the strict time ordering between multiple device accesses.
+replay_rebase=int While replaying I/O patterns using read_iolog, the
+ default behavior is to replay the log as-is, even when
+ multiple threads are replaying the same log. This means
+ all threads are accessing the same sector on the device.
+ By specifying this option, each thread will be accessing
+ different sector by adding extra rebase offset. Offset is
+ simply given by "replay_rebase * internal thread number",
+ so IO access pattern itself is preserved.
+
write_bw_log=str If given, write a bandwidth log of the jobs in this job
file. Can be used to store data of the bandwidth of the
jobs in their lifetime. The included fio_generate_plots
diff --git a/fio.1 b/fio.1
index 7eb2049..057df5c 100644
--- a/fio.1
+++ b/fio.1
@@ -826,6 +826,15 @@ is to replay the IOPS onto the major/minor device that each IOP was recorded
from. Setting \fBreplay_redirect\fR causes all IOPS to be replayed onto the
single specified device regardless of the device it was recorded from.
.TP
+.BI replay_rebase \fR=\fPint
+While replaying I/O patterns using read_iolog, the default behavior
+is to replay the log as-is, even when multiple threads are replaying
+the same log. This means all threads are accessing the same sector
+on the device. By specifying this option, each thread will be accessing
+different sector by adding extra rebase offset. Offset is simply given
+by "replay_rebase * internal thread number", so IO access pattern itself
+is preserved.
+.TP
.B write_bw_log \fR=\fPstr
If given, write a bandwidth log of the jobs in this job file. Can be used to
store data of the bandwidth of the jobs in their lifetime. The included
diff --git a/fio.h b/fio.h
index e93e8f3..e43b57b 100644
--- a/fio.h
+++ b/fio.h
@@ -358,6 +358,7 @@ struct thread_options {
unsigned int gtod_offload;
enum fio_cs clocksource;
unsigned int no_stall;
+ unsigned int replay_rebase;
unsigned int trim_percentage;
unsigned int trim_batch;
unsigned int trim_zero;
diff --git a/log.c b/log.c
index f962864..57fc0f4 100644
--- a/log.c
+++ b/log.c
@@ -129,7 +129,7 @@ int read_iolog_get(struct thread_data *td, struct io_u *io_u)
io_u->ddir = ipo->ddir;
if (ipo->ddir != DDIR_WAIT) {
- io_u->offset = ipo->offset;
+ io_u->offset = ipo->offset + td->o.replay_rebase * td->thread_number;
io_u->buflen = ipo->len;
io_u->file = td->files[ipo->fileno];
get_file(io_u->file);
diff --git a/options.c b/options.c
index 74c24d0..aba14b1 100644
--- a/options.c
+++ b/options.c
@@ -1654,6 +1654,14 @@ static struct fio_option options[FIO_MAX_OPTS] = {
.help = "Replay all I/O onto this device, regardless of trace device",
},
{
+ .name = "replay_rebase",
+ .type = FIO_OPT_INT,
+ .off1 = td_var_offset(replay_rebase),
+ .def = "0",
+ .parent = "read_iolog",
+ .help = "Apply given offset for IO replay. Mainly for parallel replay.",
+ },
+ {
.name = "exec_prerun",
.type = FIO_OPT_STR_STORE,
.off1 = td_var_offset(exec_prerun),
--
1.7.5.4
[-- Attachment #3: 0002-Added-wraparound-IO-support-for-replay-on-smaller-of.patch --]
[-- Type: text/x-patch, Size: 1119 bytes --]
From c8e61c6eac198e97b828cdb40441f855e38daaf2 Mon Sep 17 00:00:00 2001
From: Taisuke Yamada <tai@rakugaki.org>
Date: Thu, 8 Sep 2011 22:28:43 +0900
Subject: [PATCH 2/2] Added wraparound IO support for replay on
smaller/offsetted device.
---
log.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/log.c b/log.c
index 57fc0f4..a636b7e 100644
--- a/log.c
+++ b/log.c
@@ -87,6 +87,7 @@ static int ipo_special(struct thread_data *td, struct io_piece *ipo)
switch (ipo->file_action) {
case FIO_LOG_OPEN_FILE:
+ ret = td_io_get_file_size(td, f); /* for wraparound replay */
ret = td_io_open_file(td, f);
if (!ret)
break;
@@ -132,6 +133,8 @@ int read_iolog_get(struct thread_data *td, struct io_u *io_u)
io_u->offset = ipo->offset + td->o.replay_rebase * td->thread_number;
io_u->buflen = ipo->len;
io_u->file = td->files[ipo->fileno];
+ if (io_u->file->real_file_size > 0)
+ io_u->offset %= io_u->file->real_file_size;
get_file(io_u->file);
dprint(FD_IO, "iolog: get %llu/%lu/%s\n", io_u->offset,
io_u->buflen, io_u->file->file_name);
--
1.7.5.4
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-15 6:23 ` Taisuke Yamada
@ 2011-09-15 11:10 ` Jens Axboe
2011-09-15 11:52 ` Taisuke Yamada
2011-09-15 11:14 ` Jens Axboe
1 sibling, 1 reply; 9+ messages in thread
From: Jens Axboe @ 2011-09-15 11:10 UTC (permalink / raw)
To: Taisuke Yamada; +Cc: fio
On 2011-09-15 08:23, Taisuke Yamada wrote:
>>> [snip], I added "replay_rebase" option to shift offset of IO request for
>>> each thread.
>>
>> Thanks, I can see this being handy. I can apply this, but I would much
>> prefer if you resend it with the proper updates to HOWTO and fio.1
>> adding the new option. Otherwise nobody is going to learn about it.
>
> Just in case you have missed (I had to break In-Reply-To chain as git
> format-patch did not respect it). I'm re-posting updated patch with docs.
> Please apply #1 and #2 if it seems acceptable.
>
> I believe #2 patch is also worth adding to fio, but not sure if I should make
> it optional feature (/w doc) or not. By wraparounding offset address, it'll
> probably make benchmark result look better than actual for rotational disk
> drives. OTOH, it's better than failing to run just because trace-generating
> device was larger than replaying device.
I worry about the 2nd patch, it's a bit of a hack. You could make it an
option and turn it on unconditionally for replay_rebase, or at least
make it conditional on that.
--
Jens Axboe
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-15 6:23 ` Taisuke Yamada
2011-09-15 11:10 ` Jens Axboe
@ 2011-09-15 11:14 ` Jens Axboe
2011-09-15 11:40 ` Taisuke Yamada
1 sibling, 1 reply; 9+ messages in thread
From: Jens Axboe @ 2011-09-15 11:14 UTC (permalink / raw)
To: Taisuke Yamada; +Cc: fio
On 2011-09-15 08:23, Taisuke Yamada wrote:
>>> [snip], I added "replay_rebase" option to shift offset of IO request for
>>> each thread.
>>
>> Thanks, I can see this being handy. I can apply this, but I would much
>> prefer if you resend it with the proper updates to HOWTO and fio.1
>> adding the new option. Otherwise nobody is going to learn about it.
>
> Just in case you have missed (I had to break In-Reply-To chain as git
> format-patch did not respect it). I'm re-posting updated patch with docs.
> Please apply #1 and #2 if it seems acceptable.
On patch 1, I don't know, it seems a bit weird to me. The fact that the
option offset is multiplied by the job number makes sense for ease of
handling job files (since you can just do numjobs=X and be done with
it), but logically it's odd since you have to know that they are
numbered sequentially and do the math to see where it ends up on the
disk.
What is the intended use case for this? I'd much rather see the offset
be absolute, at the cost of a bit more complex job file.
--
Jens Axboe
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-15 11:14 ` Jens Axboe
@ 2011-09-15 11:40 ` Taisuke Yamada
2011-09-15 11:42 ` Jens Axboe
0 siblings, 1 reply; 9+ messages in thread
From: Taisuke Yamada @ 2011-09-15 11:40 UTC (permalink / raw)
To: Jens Axboe; +Cc: fio
> On patch 1, I don't know, it seems a bit weird to me. The fact that the
> option offset is multiplied by the job number makes sense for ease of
> handling job files (since you can just do numjobs=X and be done with
> it), but logically it's odd since you have to know that they are
> numbered sequentially and do the math to see where it ends up on the
> disk.
>
> What is the intended use case for this? I'd much rather see the offset
> be absolute, at the cost of a bit more complex job file.
My intention (and usecase) is just to put more replay I/O load, so
multiplication is purely for ease of use. So yes, I can live perfectly
fine with per-thread(job) absolute offset configuration.
In fact, it may be better as offset can be chosen to access disk more
evenly/randomly.
Do you want me to submit a new patch without multiplication?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-15 11:40 ` Taisuke Yamada
@ 2011-09-15 11:42 ` Jens Axboe
0 siblings, 0 replies; 9+ messages in thread
From: Jens Axboe @ 2011-09-15 11:42 UTC (permalink / raw)
To: Taisuke Yamada; +Cc: fio
On 2011-09-15 13:40, Taisuke Yamada wrote:
>> On patch 1, I don't know, it seems a bit weird to me. The fact that the
>> option offset is multiplied by the job number makes sense for ease of
>> handling job files (since you can just do numjobs=X and be done with
>> it), but logically it's odd since you have to know that they are
>> numbered sequentially and do the math to see where it ends up on the
>> disk.
>>
>> What is the intended use case for this? I'd much rather see the offset
>> be absolute, at the cost of a bit more complex job file.
>
> My intention (and usecase) is just to put more replay I/O load, so
> multiplication is purely for ease of use. So yes, I can live perfectly
> fine with per-thread(job) absolute offset configuration.
> In fact, it may be better as offset can be chosen to access disk more
> evenly/randomly.
>
> Do you want me to submit a new patch without multiplication?
Yeah, please do one without the multiplier and I guess we can put it in.
--
Jens Axboe
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors.
2011-09-15 11:10 ` Jens Axboe
@ 2011-09-15 11:52 ` Taisuke Yamada
0 siblings, 0 replies; 9+ messages in thread
From: Taisuke Yamada @ 2011-09-15 11:52 UTC (permalink / raw)
To: Jens Axboe; +Cc: fio
>> I believe #2 patch is also worth adding to fio, but not sure if I should make
>> it optional feature (/w doc) or not. By wraparounding offset address, it'll
>> probably make benchmark result look better than actual for rotational disk
>> drives. OTOH, it's better than failing to run just because trace-generating
>> device was larger than replaying device.
>
> I worry about the 2nd patch, it's a bit of a hack. You could make it an
> option and turn it on unconditionally for replay_rebase, or at least
> make it conditional on that.
Sure. I'll make it an option ("wraparound=bool"), and submit one as well as #1.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-09-15 11:52 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-09-09 5:12 [PATCH 1/2] Added replay_rebase option to run multi-threaded log replay on different disk sectors Taisuke Yamada
2011-09-09 8:37 ` Jens Axboe
2011-09-15 6:23 ` Taisuke Yamada
2011-09-15 11:10 ` Jens Axboe
2011-09-15 11:52 ` Taisuke Yamada
2011-09-15 11:14 ` Jens Axboe
2011-09-15 11:40 ` Taisuke Yamada
2011-09-15 11:42 ` Jens Axboe
-- strict thread matches above, loose matches on Subject: below --
2011-09-12 5:30 Taisuke Yamada
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.