Re: No I/O performed by windowsaio/get_io_u: zero buflen

[Jens Axboe <axboe@kernel.dk>]

On 9/17/18 10:33 AM, Smitha Sunder wrote:
> -----Original Message-----
> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Jens Axboe
> Sent: Sunday, September 16, 2018 8:42 PM
> To: smitha sunder <sundersmitha@gmail.com>
> Cc: Sitsofe Wheeler <sitsofe@gmail.com>; fio <fio@vger.kernel.org>
> Subject: Re: No I/O performed by windowsaio/get_io_u: zero buflen
> 
> On 9/16/18 9:34 PM, smitha sunder wrote:
>>
>>
>>> On Sep 16, 2018, at 8:31 PM, Jens Axboe <axboe@kernel.dk> wrote:
>>>
>>>> On 9/16/18 9:13 PM, smitha sunder wrote:
>>>>
>>>>
>>>>>> On Sep 16, 2018, at 8:02 PM, Jens Axboe <axboe@kernel.dk> wrote:
>>>>>>
>>>>>>> On 9/16/18 5:24 PM, smitha sunder wrote:
>>>>>>>> On Sun, Sep 16, 2018 at 3:02 PM, Sitsofe Wheeler <sitsofe@gmail.com> wrote:
>>>>>>>> On Sat, 15 Sep 2018 at 23:14, smitha sunder <sundersmitha@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> I have a 30TB drive and I am running into an issue with random writes.
>>>>>>>> I went through this thread :
>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics
>>>>>>>> .net_lists_fio_msg06294.html&d=DwIDaQ&c=JfeWlBa6VbDyTXraMENjy_b_0yKWuqQ4qY-FPhxK4x8w-TfgRBDyeV4hVQQBEgL2&r=fLVkx46f6GJPZippnZkCtH9m_d1P5HX6rKxJgT7l7Us&m=NaPx5NYD-_SBHGUzJNi4czQueVX-2LvLGD_SEwee41c&s=6QDNQBLcbJUsiXnq84y2HKG98aEmhvabF7uK-yKTbhk&e= that seems to be fixed already.
>>>>>>>
>>>>>>> I think that was something different regarding different 
>>>>>>> blocksizes per direction.
>>>>>>>
>>>>>>>> I see the same issue with random reads as well.
>>>>>>>> So not sure what is the issue here in my case; any help is greatly appreciated.
>>>>>>>>
>>>>>>>>
>>>>>>>> Read Capacity results:
>>>>>>>>  Protection: prot_en=1, p_type=1, p_i_exponent=0 [type 2 
>>>>>>>> protection]  Logical block provisioning: lbpme=1, lbprz=1  Last 
>>>>>>>> logical block address=58781073407 (0xdaf9fffff), Number of 
>>>>>>>> logical blocks=58781073408  Logical block length=512 bytes  
>>>>>>>> Logical blocks per physical block exponent=3 [so physical block
>>>>>>>> length=4096 bytes]
>>>>>>>>  Lowest aligned logical block address=0
>>>>>>>> Hence:
>>>>>>>>  Device size: 30095909584896 bytes, 2.87017e+007 MiB, 30095.9 GB
>>>>>>>>
>>>>>>>>
>>>>>>>> C:\Program Files (x86)\fio>fio --ioengine=windowsaio 
>>>>>>>> --group_reporting
>>>>>>>> --direct=1 --size=100% --bs=4K --thread 
>>>>>>>> --filename=\\.\PhysicalDrive1 --name=precond --rw=randwrite 
>>>>>>>> --iodepth=1 --numjobs=1 --debug=io,random
>>>>>>>
>>>>>>> <snip>
>>>>>>>
>>>>>>>> io       3372  fill: io_u 0A458780:
>>>>>>>> off=0x144365e7d000,len=0x0,ddir=1,file=\\.\PhysicalDrive1
>>>>>>>> io       3372  get_io_u: zero buflen on 0A458780
>>>>>>>> io       3372  get_io_u failed
>>>>>>>> io       3372  drop page cache \\.\PhysicalDrive1
>>>>>>>> random   3372  off rand 17311067694306724737
>>>>>>>
>>>>>>> That offset is crazy big - I'm sure it's bigger than a petabyte 
>>>>>>> so my guess is that something is overflowing. If you use 
>>>>>>> --size=27g does the job go through?
>>>>>>>
>>>>>>> [...]
>>>>>>>
>>>>>>>> I don’t see this issue if I use bs=8K or I use ba=512,8K, etc.
>>>>>>>
>>>>>>> --
>>>>>>> Sitsofe | 
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__sucs.org_-7Es
>>>>>>> its_&d=DwIDaQ&c=JfeWlBa6VbDyTXraMENjy_b_0yKWuqQ4qY-FPhxK4x8w-TfgR
>>>>>>> BDyeV4hVQQBEgL2&r=fLVkx46f6GJPZippnZkCtH9m_d1P5HX6rKxJgT7l7Us&m=N
>>>>>>> aPx5NYD-_SBHGUzJNi4czQueVX-2LvLGD_SEwee41c&s=ZnZsmte_bg5yC3adbwXI
>>>>>>> U4EmuEn97cA9o1YpZxDU4m0&e=
>>>>>> Hi Sitsofe,
>>>>>>
>>>>>> Thanks for the reply!
>>>>>>
>>>>>> Yes; If I use --size=27G or if provide the exact size that the OS 
>>>>>> displays, then the job goes through.
>>>>>
>>>>> Are you running a 32-bit or 64-bit build of fio?
>>>>>
>>>>> --
>>>>> Jens Axboe
>>>>>
>>>> 32 bit.
>>>
>>> I thought so. I see a few 32-bit issues with huge devices. One of 
>>> them is this one:
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__git.kernel.dk_cgi
>>> t_fio_commit_-3Fid-3D604d3f5bd9f2b985568593c23f8292cbc7f4044c&d=DwIDa
>>> Q&c=JfeWlBa6VbDyTXraMENjy_b_0yKWuqQ4qY-FPhxK4x8w-TfgRBDyeV4hVQQBEgL2&
>>> r=fLVkx46f6GJPZippnZkCtH9m_d1P5HX6rKxJgT7l7Us&m=NaPx5NYD-_SBHGUzJNi4c
>>> zQueVX-2LvLGD_SEwee41c&s=AJCs8E23SmJj-z7BbkuP_yxTNDg3g6BZ8FYIcV3EmAY&
>>> e=
>>>
>>> but I'm sure there are others, I'll try and reproduce and get this fixed.
>>>
>>> --
>>> Jens Axboe
>>>
>> I see the same issue even with 64 bit fio build.
> 
> The difference is that an unsigned long on 64-bit windows builds is still 32-bit, whereas it's 64-bit on linux. So the easiest for me is to test on 32-bit linux, which _probably_ then hits the same thing...
> 
> --
> Jens Axboe
> 
> 
> Jens,
> 
> Got the latest build from https://ci.appveyor.com/project/axboe/fio/build/1.0.891/job/awu51avmepn1kvgf/artifacts.
> 
> With size=100%, bs=4K , I no longer see the issue; thanks for the fix!

Great, thanks a lot for reporting and testing!

For what it's worth, I think we got all of them now, puzzled why we haven't
run into this sooner. I guess not a lot of folks run 32-bit builds. I also
verified the random distribution of 32-bit, and it looks fine.

-- 
Jens Axboe