Group reporting with latency and IOPS

Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

Hi All,

This is neto from Brazil

How are you?

I just need a help to make sure I understood correctly the report (see below --Output). 

Trying to generate 80% Read and 20% Write - 8KB block size

Reads
IOPS: 15330
Latency: (clat) 628usecs - 0.628ms

Writes
IOPS: 3827
Latency: (clat) 577usecs - 0.577ms

Am I doing the correct analysis?

One question is: Is it possible to have global IOPS and latency?

All the best

neto


-- Config file (Windows)

[workload]
bs=8k
ioengine=windowsaio
iodepth=4
size=10g
numjobs=3
direct=1
runtime=30
filename=\\.\PhysicalDrive1
filename=\\.\PhysicalDrive2
filename=\\.\PhysicalDrive3
filename=\\.\PhysicalDrive5
filename=\\.\PhysicalDrive6
filename=\\.\PhysicalDrive7
rw=randrw
rwmixread=80
rwmixwrite=20
thread
group_reporting


-- Output

workload: (groupid=0, jobs=3): err= 0: pid=2096: Mon Jan 28 18:23:32 2013
  read : io=3594.7MB, bw=122645KB/s, iops=15330 , runt= 30008msec
    slat (usec): min=2 , max=2263 , avg= 5.92, stdev= 4.05
    clat (usec): min=128 , max=51478 , avg=628.97, stdev=789.58
     lat (usec): min=134 , max=51482 , avg=634.89, stdev=789.56
    clat percentiles (usec):
     |  1.00th=[  169],  5.00th=[  185], 10.00th=[  195], 20.00th=[  213],
     | 30.00th=[  237], 40.00th=[  306], 50.00th=[  572], 60.00th=[  756],
     | 70.00th=[  796], 80.00th=[  868], 90.00th=[  988], 95.00th=[ 1224],
     | 99.00th=[ 2640], 99.50th=[ 4832], 99.90th=[13376], 99.95th=[15040],
     | 99.99th=[15936]
    bw (KB/s)  : min=17280, max=48528, per=33.37%, avg=40921.04, stdev=6090.31
  write: io=918768KB, bw=30617KB/s, iops=3827 , runt= 30008msec
    slat (usec): min=3 , max=2260 , avg= 7.34, stdev= 6.94
    clat (usec): min=262 , max=19853 , avg=577.21, stdev=629.07
     lat (usec): min=271 , max=19862 , avg=584.54, stdev=629.35
    clat percentiles (usec):
     |  1.00th=[  338],  5.00th=[  378], 10.00th=[  402], 20.00th=[  434],
     | 30.00th=[  458], 40.00th=[  482], 50.00th=[  502], 60.00th=[  532],
     | 70.00th=[  564], 80.00th=[  620], 90.00th=[  732], 95.00th=[  812],
     | 99.00th=[ 1304], 99.50th=[ 2768], 99.90th=[12224], 99.95th=[15168],
     | 99.99th=[15680]
    bw (KB/s)  : min= 4080, max=12368, per=33.36%, avg=10213.68, stdev=1566.17
    lat (usec) : 250=26.81%, 500=21.78%, 750=16.54%, 1000=26.96%
    lat (msec) : 2=6.67%, 4=0.69%, 10=0.37%, 20=0.18%, 50=0.01%
    lat (msec) : 100=0.01%
  cpu          : usr=3.33%, sys=16.67%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=460040/w=114846/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=3594.7MB, aggrb=122644KB/s, minb=122644KB/s, maxb=122644KB/s, mint=3
0008msec, maxt=30008msec
  WRITE: io=918768KB, aggrb=30617KB/s, minb=30617KB/s, maxb=30617KB/s, mint=3000
8msec, maxt=30008msec

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Mon, Jan 28 2013, Neto, Antonio Jose Rodrigues wrote:
> Hi All,
> 
> This is neto from Brazil
> 
> How are you?
> 
> I just need a help to make sure I understood correctly the report (see below --Output). 
> 
> Trying to generate 80% Read and 20% Write - 8KB block size
> 
> Reads
> IOPS: 15330
> Latency: (clat) 628usecs - 0.628ms

Correct. Well 629 really, if rounded properly.

> Writes
> IOPS: 3827
> Latency: (clat) 577usecs - 0.577ms

Correct too.

> Am I doing the correct analysis?

Yep.

> One question is: Is it possible to have global IOPS and latency?

You mean for both read and write combined? Fio does not report that, but
for IOPS you could just add them. For latency, the larger of the two.

-- 
Jens Axboe

Re: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On 1/29/13 8:02 AM, "Jens Axboe" <axboe@kernel.dk> wrote:

>On Mon, Jan 28 2013, Neto, Antonio Jose Rodrigues wrote:
>> Hi All,
>> 
>> This is neto from Brazil
>> 
>> How are you?
>> 
>> I just need a help to make sure I understood correctly the report (see
>>below --Output). 
>> 
>> Trying to generate 80% Read and 20% Write - 8KB block size
>> 
>> Reads
>> IOPS: 15330
>> Latency: (clat) 628usecs - 0.628ms
>
>Correct. Well 629 really, if rounded properly.
>
>> Writes
>> IOPS: 3827
>> Latency: (clat) 577usecs - 0.577ms
>
>Correct too.
>
>> Am I doing the correct analysis?
>
>Yep.
>
>> One question is: Is it possible to have global IOPS and latency?
>
>You mean for both read and write combined? Fio does not report that, but
>for IOPS you could just add them. For latency, the larger of the two.
>
>-- 
>Jens Axboe
>

Hi Jens

This is neto from Brazil

How are you?

Thank you so much for your answer.

How can I have a combined IOPS reported?

Thank you

neto


>

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> atency: (clat) 577usecs - 0.577ms
> >
> >Correct too.
> >
> >> Am I doing the correct analysis?
> >
> >Yep.
> >
> >> One question is: Is it possible to have global IOPS and latency?
> >
> >You mean for both read and write combined? Fio does not report that, but
> >for IOPS you could just add them. For latency, the larger of the two.
> >
> >--=20
> >Jens Axboe
> >
> 
> Hi Jens
> 
> This is neto from Brazil
> 
> How are you?
> 
> Thank you so much for your answer.
> 
> How can I have a combined IOPS reported?

You can't, at least not automatically. You would have to sum them up. It
would not be hard to add at all, though, controlled by an option. I
haven't had any requests for it before now, usually people ask for more
fine grained metrics, not the other way around :-)

-- 
Jens Axboe

RE: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> atency: (clat) 577usecs - 0.577ms
> >
> >Correct too.
> >
> >> Am I doing the correct analysis?
> >
> >Yep.
> >
> >> One question is: Is it possible to have global IOPS and latency?
> >
> >You mean for both read and write combined? Fio does not report that, 
> >but for IOPS you could just add them. For latency, the larger of the two.
> >
> >--=20
> >Jens Axboe
> >
> 
> Hi Jens
> 
> This is neto from Brazil
> 
> How are you?
> 
> Thank you so much for your answer.
> 
> How can I have a combined IOPS reported?

You can't, at least not automatically. You would have to sum them up. It would not be hard to add at all, though, controlled by an option. I haven't had any requests for it before now, usually people ask for more fine grained metrics, not the other way around :-)

--
Jens Axboe

Thank you Jens, I understand. To have an option to group everything could be a good thing especially when you are doing a POC or demonstrating performance for a customer where they are interested in how many IOPS and total latency :-) 

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > atency: (clat) 577usecs - 0.577ms
> > >
> > >Correct too.
> > >
> > >> Am I doing the correct analysis?
> > >
> > >Yep.
> > >
> > >> One question is: Is it possible to have global IOPS and latency?
> > >
> > >You mean for both read and write combined? Fio does not report that, 
> > >but for IOPS you could just add them. For latency, the larger of the two.
> > >
> > >--=20
> > >Jens Axboe
> > >
> > 
> > Hi Jens
> > 
> > This is neto from Brazil
> > 
> > How are you?
> > 
> > Thank you so much for your answer.
> > 
> > How can I have a combined IOPS reported?
> 
> You can't, at least not automatically. You would have to sum them up. It would not be hard to add at all, though, controlled by an option. I haven't had any requests for it before now, usually people ask for more fine grained metrics, not the other way around :-)
> 
> --
> Jens Axboe
> 
> Thank you Jens, I understand. To have an option to group everything
> could be a good thing especially when you are doing a POC or
> demonstrating performance for a customer where they are interested in
> how many IOPS and total latency :-) 

I agree, I can see the use case. I'll add a 'unified_rw_reporting'
option to be able to do that. Most accounting is done collectively, so
it should be pretty simple to do this.

-- 
Jens Axboe

RE: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > atency: (clat) 577usecs - 0.577ms
> > >
> > >Correct too.
> > >
> > >> Am I doing the correct analysis?
> > >
> > >Yep.
> > >
> > >> One question is: Is it possible to have global IOPS and latency?
> > >
> > >You mean for both read and write combined? Fio does not report 
> > >that, but for IOPS you could just add them. For latency, the larger of the two.
> > >
> > >--=20
> > >Jens Axboe
> > >
> > 
> > Hi Jens
> > 
> > This is neto from Brazil
> > 
> > How are you?
> > 
> > Thank you so much for your answer.
> > 
> > How can I have a combined IOPS reported?
> 
> You can't, at least not automatically. You would have to sum them up. 
> It would not be hard to add at all, though, controlled by an option. I 
> haven't had any requests for it before now, usually people ask for 
> more fine grained metrics, not the other way around :-)
> 
> --
> Jens Axboe
> 
> Thank you Jens, I understand. To have an option to group everything 
> could be a good thing especially when you are doing a POC or 
> demonstrating performance for a customer where they are interested in 
> how many IOPS and total latency :-)

I agree, I can see the use case. I'll add a 'unified_rw_reporting'
option to be able to do that. Most accounting is done collectively, so it should be pretty simple to do this.

--
Jens Axboe

Thank you so much! I am starting to use FIO for all my performance POCs here. I hope to have this option as soon as possible ... So I can demonstrate next week.

One question: I am doing a PCO using Windows, and if I download all changes via github how can I compile for Windows? (The POC is on Windows :-)

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > > atency: (clat) 577usecs - 0.577ms
> > > >
> > > >Correct too.
> > > >
> > > >> Am I doing the correct analysis?
> > > >
> > > >Yep.
> > > >
> > > >> One question is: Is it possible to have global IOPS and latency?
> > > >
> > > >You mean for both read and write combined? Fio does not report 
> > > >that, but for IOPS you could just add them. For latency, the larger of the two.
> > > >
> > > >--=20
> > > >Jens Axboe
> > > >
> > > 
> > > Hi Jens
> > > 
> > > This is neto from Brazil
> > > 
> > > How are you?
> > > 
> > > Thank you so much for your answer.
> > > 
> > > How can I have a combined IOPS reported?
> > 
> > You can't, at least not automatically. You would have to sum them up. 
> > It would not be hard to add at all, though, controlled by an option. I 
> > haven't had any requests for it before now, usually people ask for 
> > more fine grained metrics, not the other way around :-)
> > 
> > --
> > Jens Axboe
> > 
> > Thank you Jens, I understand. To have an option to group everything 
> > could be a good thing especially when you are doing a POC or 
> > demonstrating performance for a customer where they are interested in 
> > how many IOPS and total latency :-)
> 
> I agree, I can see the use case. I'll add a 'unified_rw_reporting'
> option to be able to do that. Most accounting is done collectively, so it should be pretty simple to do this.
> 
> --
> Jens Axboe
> 
> Thank you so much! I am starting to use FIO for all my performance
> POCs here. I hope to have this option as soon as possible ... So I can
> demonstrate next week.

The below should have a chance at working, though I haven't tested it
much yet. Will try and find some time to do that shortly.

> One question: I am doing a PCO using Windows, and if I download all
> changes via github how can I compile for Windows? (The POC is on
> Windows :-)

Bruce, correct me if I'm wrong, but I'm pretty sure I saw a "How to
build fio" setup guide from you at some point. If correct, can you post
it here too? Antonio probably isn't the only one.

diff --git a/client.c b/client.c
index 6021e4a..0dc620d 100644
--- a/client.c
+++ b/client.c
@@ -595,6 +595,7 @@ static void convert_ts(struct thread_stat *dst, struct thread_stat *src)
 	dst->groupid	= le32_to_cpu(src->groupid);
 	dst->pid	= le32_to_cpu(src->pid);
 	dst->members	= le32_to_cpu(src->members);
+	dst->unified_rw_rep	= le32_to_cpu(src->unified_rw_rep);
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
 		convert_io_stat(&dst->clat_stat[i], &src->clat_stat[i]);
@@ -667,6 +668,7 @@ static void convert_gs(struct group_run_stats *dst, struct group_run_stats *src)
 
 	dst->kb_base	= le32_to_cpu(src->kb_base);
 	dst->groupid	= le32_to_cpu(src->groupid);
+	dst->unified_rw_rep	= le32_to_cpu(src->unified_rw_rep);
 }
 
 static void handle_ts(struct fio_client *client, struct fio_net_cmd *cmd)
@@ -687,6 +689,7 @@ static void handle_ts(struct fio_client *client, struct fio_net_cmd *cmd)
 
 	client_ts.members++;
 	client_ts.groupid = p->ts.groupid;
+	client_ts.unified_rw_rep = p->ts.unified_rw_rep;
 
 	if (++sum_stat_nr == sum_stat_clients) {
 		strcpy(client_ts.name, "All clients");
diff --git a/eta.c b/eta.c
index fdf55c5..39fe10f 100644
--- a/eta.c
+++ b/eta.c
@@ -226,7 +226,8 @@ static int thread_eta(struct thread_data *td)
 	return eta_sec;
 }
 
-static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,
+static void calc_rate(int unified_rw_rep, unsigned long mtime,
+		      unsigned long long *io_bytes,
 		      unsigned long long *prev_io_bytes, unsigned int *rate)
 {
 	int i;
@@ -235,19 +236,32 @@ static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,
 		unsigned long long diff;
 
 		diff = io_bytes[i] - prev_io_bytes[i];
-		rate[i] = ((1000 * diff) / mtime) / 1024;
+		if (unified_rw_rep) {
+			rate[i] = 0;
+			rate[0] += ((1000 * diff) / mtime) / 1024;
+		} else
+			rate[i] = ((1000 * diff) / mtime) / 1024;
 
 		prev_io_bytes[i] = io_bytes[i];
 	}
 }
 
-static void calc_iops(unsigned long mtime, unsigned long long *io_iops,
+static void calc_iops(int unified_rw_rep, unsigned long mtime,
+		      unsigned long long *io_iops,
 		      unsigned long long *prev_io_iops, unsigned int *iops)
 {
 	int i;
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
-		iops[i] = ((io_iops[i] - prev_io_iops[i]) * 1000) / mtime;
+		unsigned long long diff;
+
+		diff = io_iops[i] - prev_io_iops[i];
+		if (unified_rw_rep) {
+			iops[i] = 0;
+			iops[0] += (diff * 1000) / mtime;
+		} else
+			iops[i] = (diff * 1000) / mtime;
+
 		prev_io_iops[i] = io_iops[i];
 	}
 }
@@ -259,7 +273,7 @@ static void calc_iops(unsigned long mtime, unsigned long long *io_iops,
 int calc_thread_status(struct jobs_eta *je, int force)
 {
 	struct thread_data *td;
-	int i;
+	int i, unified_rw_rep;
 	unsigned long rate_time, disp_time, bw_avg_time, *eta_secs;
 	unsigned long long io_bytes[DDIR_RWDIR_CNT];
 	unsigned long long io_iops[DDIR_RWDIR_CNT];
@@ -293,7 +307,9 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	io_bytes[DDIR_READ] = io_bytes[DDIR_WRITE] = io_bytes[DDIR_TRIM] = 0;
 	io_iops[DDIR_READ] = io_iops[DDIR_WRITE] = io_iops[DDIR_TRIM] = 0;
 	bw_avg_time = ULONG_MAX;
+	unified_rw_rep = 0;
 	for_each_td(td, i) {
+		unified_rw_rep += td->o.unified_rw_rep;
 		if (is_power_of_2(td->o.kb_base))
 			je->is_pow2 = 1;
 		if (td->o.bw_avg_time < bw_avg_time)
@@ -339,9 +355,15 @@ int calc_thread_status(struct jobs_eta *je, int force)
 
 		if (td->runstate > TD_RAMP) {
 			int ddir;
+
 			for (ddir = DDIR_READ; ddir < DDIR_RWDIR_CNT; ddir++) {
-				io_bytes[ddir] += td->io_bytes[ddir];
-				io_iops[ddir] += td->io_blocks[ddir];
+				if (unified_rw_rep) {
+					io_bytes[0] += td->io_bytes[ddir];
+					io_iops[0] += td->io_blocks[ddir];
+				} else {
+					io_bytes[ddir] += td->io_bytes[ddir];
+					io_iops[ddir] += td->io_blocks[ddir];
+				}
 			}
 		}
 	}
@@ -367,7 +389,8 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	rate_time = mtime_since(&rate_prev_time, &now);
 
 	if (write_bw_log && rate_time > bw_avg_time && !in_ramp_time(td)) {
-		calc_rate(rate_time, io_bytes, rate_io_bytes, je->rate);
+		calc_rate(unified_rw_rep, rate_time, io_bytes, rate_io_bytes,
+				je->rate);
 		memcpy(&rate_prev_time, &now, sizeof(now));
 		add_agg_sample(je->rate[DDIR_READ], DDIR_READ, 0);
 		add_agg_sample(je->rate[DDIR_WRITE], DDIR_WRITE, 0);
@@ -382,8 +405,8 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	if (!force && disp_time < 900)
 		return 0;
 
-	calc_rate(disp_time, io_bytes, disp_io_bytes, je->rate);
-	calc_iops(disp_time, io_iops, disp_io_iops, je->iops);
+	calc_rate(unified_rw_rep, disp_time, io_bytes, disp_io_bytes, je->rate);
+	calc_iops(unified_rw_rep, disp_time, io_iops, disp_io_iops, je->iops);
 
 	memcpy(&disp_prev_time, &now, sizeof(now));
 
diff --git a/fio.h b/fio.h
index 9e20299..2fd354a 100644
--- a/fio.h
+++ b/fio.h
@@ -237,6 +237,7 @@ struct thread_options {
 	unsigned int disable_clat;
 	unsigned int disable_slat;
 	unsigned int disable_bw;
+	unsigned int unified_rw_rep;
 	unsigned int gtod_reduce;
 	unsigned int gtod_cpu;
 	unsigned int gtod_offload;
diff --git a/options.c b/options.c
index 1009df3..799e77a 100644
--- a/options.c
+++ b/options.c
@@ -2497,6 +2497,13 @@ static struct fio_option options[FIO_MAX_OPTS] = {
 		.verify	= gtod_cpu_verify,
 	},
 	{
+		.name	= "unified_rw_reporting",
+		.type	= FIO_OPT_BOOL,
+		.off1	= td_var_offset(unified_rw_rep),
+		.help	= "Unify reporting across data direction",
+		.def	= "0",
+	},
+	{
 		.name	= "continue_on_error",
 		.type	= FIO_OPT_STR,
 		.off1	= td_var_offset(continue_on_error),
diff --git a/server.c b/server.c
index 0cc3fad..7ec8531 100644
--- a/server.c
+++ b/server.c
@@ -648,6 +648,7 @@ static void convert_gs(struct group_run_stats *dst, struct group_run_stats *src)
 
 	dst->kb_base	= cpu_to_le32(src->kb_base);
 	dst->groupid	= cpu_to_le32(src->groupid);
+	dst->unified_rw_rep	= cpu_to_le32(src->unified_rw_rep);
 }
 
 /*
@@ -669,8 +670,10 @@ void fio_server_send_ts(struct thread_stat *ts, struct group_run_stats *rs)
 
 	p.ts.error	= cpu_to_le32(ts->error);
 	p.ts.groupid	= cpu_to_le32(ts->groupid);
+	p.ts.unified_rw_rep	= cpu_to_le32(ts->unified_rw_rep);
 	p.ts.pid	= cpu_to_le32(ts->pid);
 	p.ts.members	= cpu_to_le32(ts->members);
+	p.ts.unified_rw_rep	= cpu_to_le32(ts->unified_rw_rep);
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
 		convert_io_stat(&p.ts.clat_stat[i], &ts->clat_stat[i]);
diff --git a/server.h b/server.h
index 3f1bde4..15b802b 100644
--- a/server.h
+++ b/server.h
@@ -36,7 +36,7 @@ struct fio_net_int_cmd {
 };
 
 enum {
-	FIO_SERVER_VER		= 9,
+	FIO_SERVER_VER		= 10,
 
 	FIO_SERVER_MAX_PDU	= 1024,
 
diff --git a/stat.c b/stat.c
index 8e1034b..05a29f5 100644
--- a/stat.c
+++ b/stat.c
@@ -277,9 +277,9 @@ void show_group_stats(struct group_run_stats *rs)
 		p4 = num2str(rs->max_bw[i], 6, rs->kb_base, i2p);
 
 		log_info("%s: io=%sB, aggrb=%sB/s, minb=%sB/s, maxb=%sB/s,"
-			 " mint=%llumsec, maxt=%llumsec\n", ddir_str[i], p1, p2,
-						p3, p4, rs->min_run[i],
-						rs->max_run[i]);
+			 " mint=%llumsec, maxt=%llumsec\n",
+				rs->unified_rw_rep ? "  MIXED" : ddir_str[i],
+				p1, p2, p3, p4, rs->min_run[i], rs->max_run[i]);
 
 		free(p1);
 		free(p2);
@@ -381,8 +381,8 @@ static void show_ddir_status(struct group_run_stats *rs, struct thread_stat *ts,
 	iops_p = num2str(iops, 6, 1, 0);
 
 	log_info("  %s: io=%sB, bw=%sB/s, iops=%s, runt=%6llumsec\n",
-					ddir_str[ddir], io_p, bw_p, iops_p,
-					ts->runtime[ddir]);
+				rs->unified_rw_rep ? "mixed" : ddir_str[ddir],
+				io_p, bw_p, iops_p, ts->runtime[ddir]);
 
 	free(io_p);
 	free(bw_p);
@@ -1062,15 +1062,27 @@ void sum_thread_stats(struct thread_stat *dst, struct thread_stat *src, int nr)
 	int l, k;
 
 	for (l = 0; l < DDIR_RWDIR_CNT; l++) {
-		sum_stat(&dst->clat_stat[l], &src->clat_stat[l], nr);
-		sum_stat(&dst->slat_stat[l], &src->slat_stat[l], nr);
-		sum_stat(&dst->lat_stat[l], &src->lat_stat[l], nr);
-		sum_stat(&dst->bw_stat[l], &src->bw_stat[l], nr);
-
-		dst->io_bytes[l] += src->io_bytes[l];
-
-		if (dst->runtime[l] < src->runtime[l])
-			dst->runtime[l] = src->runtime[l];
+		if (!dst->unified_rw_rep) {
+			sum_stat(&dst->clat_stat[l], &src->clat_stat[l], nr);
+			sum_stat(&dst->slat_stat[l], &src->slat_stat[l], nr);
+			sum_stat(&dst->lat_stat[l], &src->lat_stat[l], nr);
+			sum_stat(&dst->bw_stat[l], &src->bw_stat[l], nr);
+
+			dst->io_bytes[l] += src->io_bytes[l];
+
+			if (dst->runtime[l] < src->runtime[l])
+				dst->runtime[l] = src->runtime[l];
+		} else {
+			sum_stat(&dst->clat_stat[0], &src->clat_stat[l], nr);
+			sum_stat(&dst->slat_stat[0], &src->slat_stat[l], nr);
+			sum_stat(&dst->lat_stat[0], &src->lat_stat[l], nr);
+			sum_stat(&dst->bw_stat[0], &src->bw_stat[l], nr);
+
+			dst->io_bytes[0] += src->io_bytes[l];
+
+			if (dst->runtime[0] < src->runtime[l])
+				dst->runtime[0] = src->runtime[l];
+		}
 	}
 
 	dst->usr_time += src->usr_time;
@@ -1091,14 +1103,24 @@ void sum_thread_stats(struct thread_stat *dst, struct thread_stat *src, int nr)
 		dst->io_u_lat_m[k] += src->io_u_lat_m[k];
 
 	for (k = 0; k < DDIR_RWDIR_CNT; k++) {
-		dst->total_io_u[k] += src->total_io_u[k];
-		dst->short_io_u[k] += src->short_io_u[k];
+		if (!dst->unified_rw_rep) {
+			dst->total_io_u[k] += src->total_io_u[k];
+			dst->short_io_u[k] += src->short_io_u[k];
+		} else {
+			dst->total_io_u[0] += src->total_io_u[k];
+			dst->short_io_u[0] += src->short_io_u[k];
+		}
 	}
 
 	for (k = 0; k < DDIR_RWDIR_CNT; k++) {
 		int m;
-		for (m = 0; m < FIO_IO_U_PLAT_NR; m++)
-			dst->io_u_plat[k][m] += src->io_u_plat[k][m];
+
+		for (m = 0; m < FIO_IO_U_PLAT_NR; m++) {
+			if (!dst->unified_rw_rep)
+				dst->io_u_plat[k][m] += src->io_u_plat[k][m];
+			else
+				dst->io_u_plat[0][m] += src->io_u_plat[k][m];
+		}
 	}
 
 	dst->total_run_time += src->total_run_time;
@@ -1210,6 +1232,7 @@ void show_run_stats(void)
 			ts->pid = td->pid;
 
 			ts->kb_base = td->o.kb_base;
+			ts->unified_rw_rep = td->o.unified_rw_rep;
 		} else if (ts->kb_base != td->o.kb_base && !kb_base_warned) {
 			log_info("fio: kb_base differs for jobs in group, using"
 				 " %u as the base\n", ts->kb_base);
@@ -1239,6 +1262,7 @@ void show_run_stats(void)
 		ts = &threadstats[i];
 		rs = &runstats[ts->groupid];
 		rs->kb_base = ts->kb_base;
+		rs->unified_rw_rep += ts->unified_rw_rep;
 
 		for (j = 0; j < DDIR_RWDIR_CNT; j++) {
 			if (!ts->runtime[j])
diff --git a/stat.h b/stat.h
index 4ca8261..97186c1 100644
--- a/stat.h
+++ b/stat.h
@@ -8,6 +8,7 @@ struct group_run_stats {
 	uint64_t agg[DDIR_RWDIR_CNT];
 	uint32_t kb_base;
 	uint32_t groupid;
+	uint32_t unified_rw_rep;
 };
 
 /*
@@ -120,6 +121,7 @@ struct thread_stat {
 	uint32_t pid;
 	char description[FIO_JOBNAME_SIZE];
 	uint32_t members;
+	uint32_t unified_rw_rep;
 
 	/*
 	 * bandwidth and latency stats

-- 
Jens Axboe

RE: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > > atency: (clat) 577usecs - 0.577ms
> > > >
> > > >Correct too.
> > > >
> > > >> Am I doing the correct analysis?
> > > >
> > > >Yep.
> > > >
> > > >> One question is: Is it possible to have global IOPS and latency?
> > > >
> > > >You mean for both read and write combined? Fio does not report 
> > > >that, but for IOPS you could just add them. For latency, the larger of the two.
> > > >
> > > >--=20
> > > >Jens Axboe
> > > >
> > > 
> > > Hi Jens
> > > 
> > > This is neto from Brazil
> > > 
> > > How are you?
> > > 
> > > Thank you so much for your answer.
> > > 
> > > How can I have a combined IOPS reported?
> > 
> > You can't, at least not automatically. You would have to sum them up. 
> > It would not be hard to add at all, though, controlled by an option. 
> > I haven't had any requests for it before now, usually people ask for 
> > more fine grained metrics, not the other way around :-)
> > 
> > --
> > Jens Axboe
> > 
> > Thank you Jens, I understand. To have an option to group everything 
> > could be a good thing especially when you are doing a POC or 
> > demonstrating performance for a customer where they are interested 
> > in how many IOPS and total latency :-)
> 
> I agree, I can see the use case. I'll add a 'unified_rw_reporting'
> option to be able to do that. Most accounting is done collectively, so it should be pretty simple to do this.
> 
> --
> Jens Axboe
> 
> Thank you so much! I am starting to use FIO for all my performance 
> POCs here. I hope to have this option as soon as possible ... So I can 
> demonstrate next week.

The below should have a chance at working, though I haven't tested it much yet. Will try and find some time to do that shortly.

> One question: I am doing a PCO using Windows, and if I download all 
> changes via github how can I compile for Windows? (The POC is on 
> Windows :-)

Bruce, correct me if I'm wrong, but I'm pretty sure I saw a "How to build fio" setup guide from you at some point. If correct, can you post it here too? Antonio probably isn't the only one.

Jens, sorry to bother you again, but I am trying to apply the I've got this error. Could you please help me how to fix it?

Thank you neto

[root@mickey fio]# patch <a.diff
patching file client.c
Hunk #1 succeeded at 595 with fuzz 2.
patching file eta.c
patch: **** malformed patch at line 38: @@ -235,19 +236,32 @@ static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,






diff --git a/client.c b/client.c
index 6021e4a..0dc620d 100644
--- a/client.c
+++ b/client.c
@@ -595,6 +595,7 @@ static void convert_ts(struct thread_stat *dst, struct thread_stat *src)
 	dst->groupid	= le32_to_cpu(src->groupid);
 	dst->pid	= le32_to_cpu(src->pid);
 	dst->members	= le32_to_cpu(src->members);
+	dst->unified_rw_rep	= le32_to_cpu(src->unified_rw_rep);
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
 		convert_io_stat(&dst->clat_stat[i], &src->clat_stat[i]); @@ -667,6 +668,7 @@ static void convert_gs(struct group_run_stats *dst, struct group_run_stats *src)
 
 	dst->kb_base	= le32_to_cpu(src->kb_base);
 	dst->groupid	= le32_to_cpu(src->groupid);
+	dst->unified_rw_rep	= le32_to_cpu(src->unified_rw_rep);
 }
 
 static void handle_ts(struct fio_client *client, struct fio_net_cmd *cmd) @@ -687,6 +689,7 @@ static void handle_ts(struct fio_client *client, struct fio_net_cmd *cmd)
 
 	client_ts.members++;
 	client_ts.groupid = p->ts.groupid;
+	client_ts.unified_rw_rep = p->ts.unified_rw_rep;
 
 	if (++sum_stat_nr == sum_stat_clients) {
 		strcpy(client_ts.name, "All clients"); diff --git a/eta.c b/eta.c index fdf55c5..39fe10f 100644
--- a/eta.c
+++ b/eta.c
@@ -226,7 +226,8 @@ static int thread_eta(struct thread_data *td)
 	return eta_sec;
 }
 
-static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,
+static void calc_rate(int unified_rw_rep, unsigned long mtime,
+		      unsigned long long *io_bytes,
 		      unsigned long long *prev_io_bytes, unsigned int *rate)  {
 	int i;
@@ -235,19 +236,32 @@ static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,
 		unsigned long long diff;
 
 		diff = io_bytes[i] - prev_io_bytes[i];
-		rate[i] = ((1000 * diff) / mtime) / 1024;
+		if (unified_rw_rep) {
+			rate[i] = 0;
+			rate[0] += ((1000 * diff) / mtime) / 1024;
+		} else
+			rate[i] = ((1000 * diff) / mtime) / 1024;
 
 		prev_io_bytes[i] = io_bytes[i];
 	}
 }
 
-static void calc_iops(unsigned long mtime, unsigned long long *io_iops,
+static void calc_iops(int unified_rw_rep, unsigned long mtime,
+		      unsigned long long *io_iops,
 		      unsigned long long *prev_io_iops, unsigned int *iops)  {
 	int i;
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
-		iops[i] = ((io_iops[i] - prev_io_iops[i]) * 1000) / mtime;
+		unsigned long long diff;
+
+		diff = io_iops[i] - prev_io_iops[i];
+		if (unified_rw_rep) {
+			iops[i] = 0;
+			iops[0] += (diff * 1000) / mtime;
+		} else
+			iops[i] = (diff * 1000) / mtime;
+
 		prev_io_iops[i] = io_iops[i];
 	}
 }
@@ -259,7 +273,7 @@ static void calc_iops(unsigned long mtime, unsigned long long *io_iops,  int calc_thread_status(struct jobs_eta *je, int force)  {
 	struct thread_data *td;
-	int i;
+	int i, unified_rw_rep;
 	unsigned long rate_time, disp_time, bw_avg_time, *eta_secs;
 	unsigned long long io_bytes[DDIR_RWDIR_CNT];
 	unsigned long long io_iops[DDIR_RWDIR_CNT]; @@ -293,7 +307,9 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	io_bytes[DDIR_READ] = io_bytes[DDIR_WRITE] = io_bytes[DDIR_TRIM] = 0;
 	io_iops[DDIR_READ] = io_iops[DDIR_WRITE] = io_iops[DDIR_TRIM] = 0;
 	bw_avg_time = ULONG_MAX;
+	unified_rw_rep = 0;
 	for_each_td(td, i) {
+		unified_rw_rep += td->o.unified_rw_rep;
 		if (is_power_of_2(td->o.kb_base))
 			je->is_pow2 = 1;
 		if (td->o.bw_avg_time < bw_avg_time)
@@ -339,9 +355,15 @@ int calc_thread_status(struct jobs_eta *je, int force)
 
 		if (td->runstate > TD_RAMP) {
 			int ddir;
+
 			for (ddir = DDIR_READ; ddir < DDIR_RWDIR_CNT; ddir++) {
-				io_bytes[ddir] += td->io_bytes[ddir];
-				io_iops[ddir] += td->io_blocks[ddir];
+				if (unified_rw_rep) {
+					io_bytes[0] += td->io_bytes[ddir];
+					io_iops[0] += td->io_blocks[ddir];
+				} else {
+					io_bytes[ddir] += td->io_bytes[ddir];
+					io_iops[ddir] += td->io_blocks[ddir];
+				}
 			}
 		}
 	}
@@ -367,7 +389,8 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	rate_time = mtime_since(&rate_prev_time, &now);
 
 	if (write_bw_log && rate_time > bw_avg_time && !in_ramp_time(td)) {
-		calc_rate(rate_time, io_bytes, rate_io_bytes, je->rate);
+		calc_rate(unified_rw_rep, rate_time, io_bytes, rate_io_bytes,
+				je->rate);
 		memcpy(&rate_prev_time, &now, sizeof(now));
 		add_agg_sample(je->rate[DDIR_READ], DDIR_READ, 0);
 		add_agg_sample(je->rate[DDIR_WRITE], DDIR_WRITE, 0); @@ -382,8 +405,8 @@ int calc_thread_status(struct jobs_eta *je, int force)
 	if (!force && disp_time < 900)
 		return 0;
 
-	calc_rate(disp_time, io_bytes, disp_io_bytes, je->rate);
-	calc_iops(disp_time, io_iops, disp_io_iops, je->iops);
+	calc_rate(unified_rw_rep, disp_time, io_bytes, disp_io_bytes, je->rate);
+	calc_iops(unified_rw_rep, disp_time, io_iops, disp_io_iops, je->iops);
 
 	memcpy(&disp_prev_time, &now, sizeof(now));
 
diff --git a/fio.h b/fio.h
index 9e20299..2fd354a 100644
--- a/fio.h
+++ b/fio.h
@@ -237,6 +237,7 @@ struct thread_options {
 	unsigned int disable_clat;
 	unsigned int disable_slat;
 	unsigned int disable_bw;
+	unsigned int unified_rw_rep;
 	unsigned int gtod_reduce;
 	unsigned int gtod_cpu;
 	unsigned int gtod_offload;
diff --git a/options.c b/options.c
index 1009df3..799e77a 100644
--- a/options.c
+++ b/options.c
@@ -2497,6 +2497,13 @@ static struct fio_option options[FIO_MAX_OPTS] = {
 		.verify	= gtod_cpu_verify,
 	},
 	{
+		.name	= "unified_rw_reporting",
+		.type	= FIO_OPT_BOOL,
+		.off1	= td_var_offset(unified_rw_rep),
+		.help	= "Unify reporting across data direction",
+		.def	= "0",
+	},
+	{
 		.name	= "continue_on_error",
 		.type	= FIO_OPT_STR,
 		.off1	= td_var_offset(continue_on_error),
diff --git a/server.c b/server.c
index 0cc3fad..7ec8531 100644
--- a/server.c
+++ b/server.c
@@ -648,6 +648,7 @@ static void convert_gs(struct group_run_stats *dst, struct group_run_stats *src)
 
 	dst->kb_base	= cpu_to_le32(src->kb_base);
 	dst->groupid	= cpu_to_le32(src->groupid);
+	dst->unified_rw_rep	= cpu_to_le32(src->unified_rw_rep);
 }
 
 /*
@@ -669,8 +670,10 @@ void fio_server_send_ts(struct thread_stat *ts, struct group_run_stats *rs)
 
 	p.ts.error	= cpu_to_le32(ts->error);
 	p.ts.groupid	= cpu_to_le32(ts->groupid);
+	p.ts.unified_rw_rep	= cpu_to_le32(ts->unified_rw_rep);
 	p.ts.pid	= cpu_to_le32(ts->pid);
 	p.ts.members	= cpu_to_le32(ts->members);
+	p.ts.unified_rw_rep	= cpu_to_le32(ts->unified_rw_rep);
 
 	for (i = 0; i < DDIR_RWDIR_CNT; i++) {
 		convert_io_stat(&p.ts.clat_stat[i], &ts->clat_stat[i]); diff --git a/server.h b/server.h index 3f1bde4..15b802b 100644
--- a/server.h
+++ b/server.h
@@ -36,7 +36,7 @@ struct fio_net_int_cmd {  };
 
 enum {
-	FIO_SERVER_VER		= 9,
+	FIO_SERVER_VER		= 10,
 
 	FIO_SERVER_MAX_PDU	= 1024,
 
diff --git a/stat.c b/stat.c
index 8e1034b..05a29f5 100644
--- a/stat.c
+++ b/stat.c
@@ -277,9 +277,9 @@ void show_group_stats(struct group_run_stats *rs)
 		p4 = num2str(rs->max_bw[i], 6, rs->kb_base, i2p);
 
 		log_info("%s: io=%sB, aggrb=%sB/s, minb=%sB/s, maxb=%sB/s,"
-			 " mint=%llumsec, maxt=%llumsec\n", ddir_str[i], p1, p2,
-						p3, p4, rs->min_run[i],
-						rs->max_run[i]);
+			 " mint=%llumsec, maxt=%llumsec\n",
+				rs->unified_rw_rep ? "  MIXED" : ddir_str[i],
+				p1, p2, p3, p4, rs->min_run[i], rs->max_run[i]);
 
 		free(p1);
 		free(p2);
@@ -381,8 +381,8 @@ static void show_ddir_status(struct group_run_stats *rs, struct thread_stat *ts,
 	iops_p = num2str(iops, 6, 1, 0);
 
 	log_info("  %s: io=%sB, bw=%sB/s, iops=%s, runt=%6llumsec\n",
-					ddir_str[ddir], io_p, bw_p, iops_p,
-					ts->runtime[ddir]);
+				rs->unified_rw_rep ? "mixed" : ddir_str[ddir],
+				io_p, bw_p, iops_p, ts->runtime[ddir]);
 
 	free(io_p);
 	free(bw_p);
@@ -1062,15 +1062,27 @@ void sum_thread_stats(struct thread_stat *dst, struct thread_stat *src, int nr)
 	int l, k;
 
 	for (l = 0; l < DDIR_RWDIR_CNT; l++) {
-		sum_stat(&dst->clat_stat[l], &src->clat_stat[l], nr);
-		sum_stat(&dst->slat_stat[l], &src->slat_stat[l], nr);
-		sum_stat(&dst->lat_stat[l], &src->lat_stat[l], nr);
-		sum_stat(&dst->bw_stat[l], &src->bw_stat[l], nr);
-
-		dst->io_bytes[l] += src->io_bytes[l];
-
-		if (dst->runtime[l] < src->runtime[l])
-			dst->runtime[l] = src->runtime[l];
+		if (!dst->unified_rw_rep) {
+			sum_stat(&dst->clat_stat[l], &src->clat_stat[l], nr);
+			sum_stat(&dst->slat_stat[l], &src->slat_stat[l], nr);
+			sum_stat(&dst->lat_stat[l], &src->lat_stat[l], nr);
+			sum_stat(&dst->bw_stat[l], &src->bw_stat[l], nr);
+
+			dst->io_bytes[l] += src->io_bytes[l];
+
+			if (dst->runtime[l] < src->runtime[l])
+				dst->runtime[l] = src->runtime[l];
+		} else {
+			sum_stat(&dst->clat_stat[0], &src->clat_stat[l], nr);
+			sum_stat(&dst->slat_stat[0], &src->slat_stat[l], nr);
+			sum_stat(&dst->lat_stat[0], &src->lat_stat[l], nr);
+			sum_stat(&dst->bw_stat[0], &src->bw_stat[l], nr);
+
+			dst->io_bytes[0] += src->io_bytes[l];
+
+			if (dst->runtime[0] < src->runtime[l])
+				dst->runtime[0] = src->runtime[l];
+		}
 	}
 
 	dst->usr_time += src->usr_time;
@@ -1091,14 +1103,24 @@ void sum_thread_stats(struct thread_stat *dst, struct thread_stat *src, int nr)
 		dst->io_u_lat_m[k] += src->io_u_lat_m[k];
 
 	for (k = 0; k < DDIR_RWDIR_CNT; k++) {
-		dst->total_io_u[k] += src->total_io_u[k];
-		dst->short_io_u[k] += src->short_io_u[k];
+		if (!dst->unified_rw_rep) {
+			dst->total_io_u[k] += src->total_io_u[k];
+			dst->short_io_u[k] += src->short_io_u[k];
+		} else {
+			dst->total_io_u[0] += src->total_io_u[k];
+			dst->short_io_u[0] += src->short_io_u[k];
+		}
 	}
 
 	for (k = 0; k < DDIR_RWDIR_CNT; k++) {
 		int m;
-		for (m = 0; m < FIO_IO_U_PLAT_NR; m++)
-			dst->io_u_plat[k][m] += src->io_u_plat[k][m];
+
+		for (m = 0; m < FIO_IO_U_PLAT_NR; m++) {
+			if (!dst->unified_rw_rep)
+				dst->io_u_plat[k][m] += src->io_u_plat[k][m];
+			else
+				dst->io_u_plat[0][m] += src->io_u_plat[k][m];
+		}
 	}
 
 	dst->total_run_time += src->total_run_time; @@ -1210,6 +1232,7 @@ void show_run_stats(void)
 			ts->pid = td->pid;
 
 			ts->kb_base = td->o.kb_base;
+			ts->unified_rw_rep = td->o.unified_rw_rep;
 		} else if (ts->kb_base != td->o.kb_base && !kb_base_warned) {
 			log_info("fio: kb_base differs for jobs in group, using"
 				 " %u as the base\n", ts->kb_base); @@ -1239,6 +1262,7 @@ void show_run_stats(void)
 		ts = &threadstats[i];
 		rs = &runstats[ts->groupid];
 		rs->kb_base = ts->kb_base;
+		rs->unified_rw_rep += ts->unified_rw_rep;
 
 		for (j = 0; j < DDIR_RWDIR_CNT; j++) {
 			if (!ts->runtime[j])
diff --git a/stat.h b/stat.h
index 4ca8261..97186c1 100644
--- a/stat.h
+++ b/stat.h
@@ -8,6 +8,7 @@ struct group_run_stats {
 	uint64_t agg[DDIR_RWDIR_CNT];
 	uint32_t kb_base;
 	uint32_t groupid;
+	uint32_t unified_rw_rep;
 };
 
 /*
@@ -120,6 +121,7 @@ struct thread_stat {
 	uint32_t pid;
 	char description[FIO_JOBNAME_SIZE];
 	uint32_t members;
+	uint32_t unified_rw_rep;
 
 	/*
 	 * bandwidth and latency stats

--
Jens Axboe

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> Jens, sorry to bother you again, but I am trying to apply the I've got this=
>  error. Could you please help me how to fix it?

I forgot to push a previous change out. Pull again and try applying,
should apply without fuzz.

-- 
Jens Axboe

RE: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> Jens, sorry to bother you again, but I am trying to apply the I've got 
> this=  error. Could you please help me how to fix it?

I forgot to push a previous change out. Pull again and try applying, should apply without fuzz.

I am afraid that I am doing something wrong.... The latest change (pull) was in client.c. I pulled, copied the whole directory but same error. 

[root@mickey fio]# patch <a.diff
patching file client.c
Hunk #1 succeeded at 595 with fuzz 1.
patching file eta.c
patch: **** malformed patch at line 38: @@ -235,19 +236,32 @@ static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,

Re: Group reporting with latency and IOPS

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> > Jens, sorry to bother you again, but I am trying to apply the I've got 
> > this=  error. Could you please help me how to fix it?
> 
> I forgot to push a previous change out. Pull again and try applying, should apply without fuzz.
> 
> I am afraid that I am doing something wrong.... The latest change (pull) was in client.c. I pulled, copied the whole directory but same error. 
> 
> [root@mickey fio]# patch <a.diff
> patching file client.c
> Hunk #1 succeeded at 595 with fuzz 1.
> patching file eta.c
> patch: **** malformed patch at line 38: @@ -235,19 +236,32 @@ static void calc_rate(unsigned long mtime, unsigned long long *io_bytes,

Your mailer or MTA has corrupted the patch. I've attached a zip file for
you, that has a better chance of surviving broken mail setups.

-- 
Jens Axboe


[-- Attachment #2: patch.zip --]
[-- Type: application/x-zip-compressed, Size: 3135 bytes --]

Re: Group reporting with latency and IOPS

[Neto, Antonio Jose Rodrigues]

On 1/29/13 3:32 PM, "Jens Axboe" <axboe@kernel.dk> wrote:

>On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
>> Jens, sorry to bother you again, but I am trying to apply the I've got
>>this=
>>  error. Could you please help me how to fix it?
>
>I forgot to push a previous change out. Pull again and try applying,
>should apply without fuzz.
>
>-- 
>Jens Axboe
>

Hi Jens,

This is neto from Brazil

How are you?

I applied the patch and everything is working. Here is one example:

Thank you very much

neto

-- Output file

workload: (groupid=0, jobs=20): err= 0: pid=2348: Tue Jan 29 22:51:43 2013
  mixed: io=879352KB, bw=168902KB/s, iops=21112 , runt= 30039msec
    slat (usec): min=3 , max=2296 , avg= 6.81, stdev= 4.62
    clat (usec): min=138 , max=248843 , avg=3619.87, stdev=7906.03
     lat (usec): min=145 , max=248863 , avg=3626.68, stdev=7905.91
    clat percentiles (usec):
     |  1.00th=[  350],  5.00th=[  454], 10.00th=[  516], 20.00th=[  724],
     | 30.00th=[  780], 40.00th=[  820], 50.00th=[  868], 60.00th=[  924],
     | 70.00th=[ 1048], 80.00th=[ 3760], 90.00th=[11584], 95.00th=[18304],
     | 99.00th=[39680], 99.50th=[50432], 99.90th=[76288], 99.95th=[90624],
     | 99.99th=[123392]
    bw (KB/s)  : min=  864, max= 9088, per=2.43%, avg=4100.05,
stdev=2630.65
    lat (usec) : 250=0.56%, 500=7.90%, 750=14.84%, 1000=44.16%
    lat (msec) : 2=9.40%, 4=3.46%, 10=7.59%, 20=7.80%, 50=3.77%
    lat (msec) : 100=0.48%, 250=0.03%
  cpu          : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
     issued    : total=r=634207/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  MIXED: io=4954.8MB, aggrb=168902KB/s, minb=168902KB/s, maxb=168902KB/s,
mint=3
0039msec, maxt=30039msec




-- Config File

[workload]
bs=8k
ioengine=windowsaio
iodepth=4
size=1024g
numjobs=20
direct=1
runtime=30
filename=\\.\PhysicalDrive1
filename=\\.\PhysicalDrive2
filename=\\.\PhysicalDrive3
filename=\\.\PhysicalDrive4
filename=\\.\PhysicalDrive5
filename=\\.\PhysicalDrive6
filename=\\.\PhysicalDrive7
filename=\\.\PhysicalDrive8
rw=randrw
rwmixread=80
rwmixwrite=20
thread
unified_rw_reporting=1
group_reporting=1
randrepeat=0

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Wed, Jan 30 2013, Neto, Antonio Jose Rodrigues wrote:
> On 1/29/13 3:32 PM, "Jens Axboe" <axboe@kernel.dk> wrote:
> 
> >On Tue, Jan 29 2013, Neto, Antonio Jose Rodrigues wrote:
> >> Jens, sorry to bother you again, but I am trying to apply the I've got
> >>this=
> >>  error. Could you please help me how to fix it?
> >
> >I forgot to push a previous change out. Pull again and try applying,
> >should apply without fuzz.
> >
> >-- 
> >Jens Axboe
> >
> 
> Hi Jens,
> 
> This is neto from Brazil
> 
> How are you?
> 
> I applied the patch and everything is working. Here is one example:
> 
> Thank you very much
> 
> neto
> 
> -- Output file
> 
> workload: (groupid=0, jobs=20): err= 0: pid=2348: Tue Jan 29 22:51:43 2013
>   mixed: io=879352KB, bw=168902KB/s, iops=21112 , runt= 30039msec
>     slat (usec): min=3 , max=2296 , avg= 6.81, stdev= 4.62
>     clat (usec): min=138 , max=248843 , avg=3619.87, stdev=7906.03
>      lat (usec): min=145 , max=248863 , avg=3626.68, stdev=7905.91
>     clat percentiles (usec):
>      |  1.00th=[  350],  5.00th=[  454], 10.00th=[  516], 20.00th=[  724],
>      | 30.00th=[  780], 40.00th=[  820], 50.00th=[  868], 60.00th=[  924],
>      | 70.00th=[ 1048], 80.00th=[ 3760], 90.00th=[11584], 95.00th=[18304],
>      | 99.00th=[39680], 99.50th=[50432], 99.90th=[76288], 99.95th=[90624],
>      | 99.99th=[123392]
>     bw (KB/s)  : min=  864, max= 9088, per=2.43%, avg=4100.05,
> stdev=2630.65
>     lat (usec) : 250=0.56%, 500=7.90%, 750=14.84%, 1000=44.16%
>     lat (msec) : 2=9.40%, 4=3.46%, 10=7.59%, 20=7.80%, 50=3.77%
>     lat (msec) : 100=0.48%, 250=0.03%
>   cpu          : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0
>   IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >=64=0.0%
>      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >=64=0.0%
>      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >=64=0.0%
>      issued    : total=r=634207/w=0/d=0, short=r=0/w=0/d=0
> 
> Run status group 0 (all jobs):
>   MIXED: io=4954.8MB, aggrb=168902KB/s, minb=168902KB/s, maxb=168902KB/s,
> mint=3
> 0039msec, maxt=30039msec

Excellent. Just need to check whether all cases are handled, then I will
get it committed.

-- 
Jens Axboe

Re: Group reporting with latency and IOPS

[Georg Schönberger]

> On Mon, Jan 28 2013, Neto, Antonio Jose Rodrigues wrote:
> > Hi All,
> > 
> > This is neto from Brazil
> > 
> > How are you?
> > 
> > I just need a help to make sure I understood correctly the report
> > (see below --Output).
> > 
> > Trying to generate 80% Read and 20% Write - 8KB block size
> > 
> > Reads
> > IOPS: 15330
> > Latency: (clat) 628usecs - 0.628ms
> 
> Correct. Well 629 really, if rounded properly.
> 
> > Writes
> > IOPS: 3827
> > Latency: (clat) 577usecs - 0.577ms
> 
> Correct too.
> 
> > Am I doing the correct analysis?
> 
> Yep.
> 
> > One question is: Is it possible to have global IOPS and latency?
> 
> You mean for both read and write combined? Fio does not report that,
> but
> for IOPS you could just add them. For latency, the larger of the two.
> 

Hi Jens,

you say here that the total Latency for a mixed workload is the larger of the two.
Why is it not the sum of the two i.e.
Total Latency Read + Total Latency Write ?

Best Regards, Georg

> --
> Jens Axboe

Re: Group reporting with latency and IOPS

[Jens Axboe]

On Fri, Feb 01 2013, Georg Sch�nberger wrote:
> > On Mon, Jan 28 2013, Neto, Antonio Jose Rodrigues wrote:
> > > Hi All,
> > > 
> > > This is neto from Brazil
> > > 
> > > How are you?
> > > 
> > > I just need a help to make sure I understood correctly the report
> > > (see below --Output).
> > > 
> > > Trying to generate 80% Read and 20% Write - 8KB block size
> > > 
> > > Reads
> > > IOPS: 15330
> > > Latency: (clat) 628usecs - 0.628ms
> > 
> > Correct. Well 629 really, if rounded properly.
> > 
> > > Writes
> > > IOPS: 3827
> > > Latency: (clat) 577usecs - 0.577ms
> > 
> > Correct too.
> > 
> > > Am I doing the correct analysis?
> > 
> > Yep.
> > 
> > > One question is: Is it possible to have global IOPS and latency?
> > 
> > You mean for both read and write combined? Fio does not report that,
> > but
> > for IOPS you could just add them. For latency, the larger of the two.
> > 
> 
> Hi Jens,
> 
> you say here that the total Latency for a mixed workload is the larger of the two.
> Why is it not the sum of the two i.e.
> Total Latency Read + Total Latency Write ?

What I meant that it would be most logical to report the minimum of the
two for minimum latency, and the larger of the two for maximum latency.
That is what would have happened if there were indeed all the same type.
Mean and stddev should be summed appropriately.

For reporting, by definition of setting unified_rw_reporting, there is
not 'latency read' and 'latency write' (or 'latency trim'). There is
just single IO latency.

-- 
Jens Axboe