using direct IO with sgp_dd

using direct IO with sgp_dd

[Doron Shoham]

Hi,
I wanted to use direct IO with sgp_dd.

I have done:

echo 1 > /proc/scsi/sg/allow_dio

sgp_dd if=/dev/sg1 of=/dev/null bs=512 bpt=128 thr=8 time=1 count=5M dio=1
time to transfer data was 2.236290 secs, 1200.36 MB/sec
5242880+0 records in
5242880+0 records out
Direct IO requested but incomplete 40960 times

What does it mean "Direct IO requested but incomplete 40960 times"?
And what could be the reason for this?

When using bpt smaller then 128 it doesn't happen.

Thanks,
Doron

Re: using direct IO with sgp_dd

[Bart Van Assche]

On Tue, Mar 11, 2008 at 8:40 AM, Doron Shoham <dorons@voltaire.com> wrote:
>  sgp_dd if=/dev/sg1 of=/dev/null bs=512 bpt=128 thr=8 time=1 count=5M dio=1
>  time to transfer data was 2.236290 secs, 1200.36 MB/sec
>  5242880+0 records in
>  5242880+0 records out
>  Direct IO requested but incomplete 40960 times
>
>  What does it mean "Direct IO requested but incomplete 40960 times"?
>  And what could be the reason for this?

The man page of sgp_dd states that the parameter specified to bs must
be the block size of the physical device. You can query the block size
of a SCSI device with e.g. the scsi_readcap command:

$ scsi_readcap /dev/sda
sg_readcap   /dev/sda
Read Capacity results:
   Last logical block address=625142447 (0x2542eaaf), Number of blocks=625142448
   Logical block length=512 bytes
Hence:
   Device size: 320072933376 bytes, 305245.3 MiB, 320.07 GB

Bart.

Re: using direct IO with sgp_dd

[Doron Shoham]

Bart Van Assche wrote:
> On Tue, Mar 11, 2008 at 8:40 AM, Doron Shoham <dorons@voltaire.com> wrote:
>>  sgp_dd if=/dev/sg1 of=/dev/null bs=512 bpt=128 thr=8 time=1 count=5M dio=1
>>  time to transfer data was 2.236290 secs, 1200.36 MB/sec
>>  5242880+0 records in
>>  5242880+0 records out
>>  Direct IO requested but incomplete 40960 times
>>
>>  What does it mean "Direct IO requested but incomplete 40960 times"?
>>  And what could be the reason for this?
> 
> The man page of sgp_dd states that the parameter specified to bs must
> be the block size of the physical device. You can query the block size
> of a SCSI device with e.g. the scsi_readcap command:
> 
> $ scsi_readcap /dev/sda
> sg_readcap   /dev/sda
> Read Capacity results:
>    Last logical block address=625142447 (0x2542eaaf), Number of blocks=625142448
>    Logical block length=512 bytes
> Hence:
>    Device size: 320072933376 bytes, 305245.3 MiB, 320.07 GB
> 
> Bart.

Hi,
sg_readcap /dev/sdb

Read Capacity results:
   Last block address=3221225471 (0xbfffffff), Number of blocks=3221225472
   Block size=512 bytes
Hence:
   Device size: 1649267441664 bytes, 1572864.0 MiB, 1649.27 GB

The block size is 512 as I used in the sgp_dd command.
The problem is just when using dio=1, without it it works ok.

Thanks,
Doron

Re: using direct IO with sgp_dd

[Douglas Gilbert]

Doron Shoham wrote:
> Bart Van Assche wrote:
>> On Tue, Mar 11, 2008 at 8:40 AM, Doron Shoham <dorons@voltaire.com> wrote:
>>>  sgp_dd if=/dev/sg1 of=/dev/null bs=512 bpt=128 thr=8 time=1 count=5M dio=1
>>>  time to transfer data was 2.236290 secs, 1200.36 MB/sec
>>>  5242880+0 records in
>>>  5242880+0 records out
>>>  Direct IO requested but incomplete 40960 times
>>>
>>>  What does it mean "Direct IO requested but incomplete 40960 times"?
>>>  And what could be the reason for this?
>> The man page of sgp_dd states that the parameter specified to bs must
>> be the block size of the physical device. You can query the block size
>> of a SCSI device with e.g. the scsi_readcap command:
>>
>> $ scsi_readcap /dev/sda
>> sg_readcap   /dev/sda
>> Read Capacity results:
>>    Last logical block address=625142447 (0x2542eaaf), Number of blocks=625142448
>>    Logical block length=512 bytes
>> Hence:
>>    Device size: 320072933376 bytes, 305245.3 MiB, 320.07 GB
>>
>> Bart.
> 
> Hi,
> sg_readcap /dev/sdb
> 
> Read Capacity results:
>    Last block address=3221225471 (0xbfffffff), Number of blocks=3221225472
>    Block size=512 bytes
> Hence:
>    Device size: 1649267441664 bytes, 1572864.0 MiB, 1649.27 GB
> 
> The block size is 512 as I used in the sgp_dd command.
> The problem is just when using dio=1, without it it works ok.

Doron,
Try reducing the bpt value (say to 64 or 32). To see more of
what might be the problem add 'verbose=<n>' to the command
line (and reduce the count to stop being flooded with
debug messages).

Doug Gilbert