iodepth and synchronous ioengines ("pitfall")

iodepth and synchronous ioengines ("pitfall")

[Sebastian Kayser]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

Hi,

I just stumbled into a glaring pitfall when playing with iodepth=X for
the first time. Was using ioengine=sync (default), increased iodepth > 1
and wondered briefly why my results didn't change. Thinking about it,
this made perfect sense and the "IO depths" distribution in the result
summary even pointed me to it.

Nevertheless, it might help others to avoid this alltogether if the man
page paragraph on iodepth would include a small heads up / reference to
ioengines. Example patch attached, not quite sure about the wording for
the verify_async aspect.

Sebastian

[-- Attachment #2: Type: message/rfc822, Size: 817 bytes --]

From: Sebastian Kayser <sebastian@skayser.de>
Subject: [PATCH] fio.1: Add ioengines heads up to the iodepth section
Date: Wed, 1 Dec 2010 22:28:47 +0100

---
 fio.1 |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/fio.1 b/fio.1
index b0276c4..ab147f2 100644
--- a/fio.1
+++ b/fio.1
@@ -379,7 +379,9 @@ Loads an external I/O engine object file.  Append the engine filename as
 .RE
 .TP
 .BI iodepth \fR=\fPint
-Number of I/O units to keep in flight against the file.  Default: 1.
+Number of I/O units to keep in flight against the file. Note that increasing
+iodepth beyond 1 will not affect synchronous ioengines (except for small
+degress when verify_async is in use).  Default: 1.
 .TP
 .BI iodepth_batch \fR=\fPint
 Number of I/Os to submit at once.  Default: \fBiodepth\fR.
-- 
1.4.4.4

Re: iodepth and synchronous ioengines ("pitfall")

[Jens Axboe]

On 2010-12-01 22:38, Sebastian Kayser wrote:
> Hi,
> 
> I just stumbled into a glaring pitfall when playing with iodepth=X for
> the first time. Was using ioengine=sync (default), increased iodepth > 1
> and wondered briefly why my results didn't change. Thinking about it,
> this made perfect sense and the "IO depths" distribution in the result
> summary even pointed me to it.
> 
> Nevertheless, it might help others to avoid this alltogether if the man
> page paragraph on iodepth would include a small heads up / reference to
> ioengines. Example patch attached, not quite sure about the wording for
> the verify_async aspect.

It's a good idea. Something else to keep in mind is that even with async
engines, you can run into this issue. Say in Linux and not setting
direct=1, the buffered IO will still be sync. So I think I'll add some
wording as well to have the user keep an eye on the achieved IO depths
and not just assume that it's running with a depth of X for iodepth=X.

-- 
Jens Axboe

Re: iodepth and synchronous ioengines ("pitfall")

[Jens Axboe]

On 2010-12-02 19:59, Jens Axboe wrote:
> On 2010-12-01 22:38, Sebastian Kayser wrote:
>> Hi,
>>
>> I just stumbled into a glaring pitfall when playing with iodepth=X for
>> the first time. Was using ioengine=sync (default), increased iodepth > 1
>> and wondered briefly why my results didn't change. Thinking about it,
>> this made perfect sense and the "IO depths" distribution in the result
>> summary even pointed me to it.
>>
>> Nevertheless, it might help others to avoid this alltogether if the man
>> page paragraph on iodepth would include a small heads up / reference to
>> ioengines. Example patch attached, not quite sure about the wording for
>> the verify_async aspect.
> 
> It's a good idea. Something else to keep in mind is that even with async
> engines, you can run into this issue. Say in Linux and not setting
> direct=1, the buffered IO will still be sync. So I think I'll add some
> wording as well to have the user keep an eye on the achieved IO depths
> and not just assume that it's running with a depth of X for iodepth=X.

http://git.kernel.dk/?p=fio.git;a=commitdiff;h=ee72ca09d4657f6a805c113d9bc43428f4b28a89

-- 
Jens Axboe