Length and alignment in scsi_execute_async and blk_rq_map_user

Length and alignment in scsi_execute_async and blk_rq_map_user

[Brett Dennis]

Hello,

Apologies for posting twice (messed up on the subject line of the first post...)

In kernel 2.6.26, the implementation of __blk_rq_map_user changed and
now includes a call to bio_copy_user:

uaddr = (unsigned long) ubuf;
alignment = queue_dma_alignment(q) | q->dma_pad_mask;
if (!(uaddr & alignment) && !(len & alignment))

     bio = bio_map_user(q, NULL, uaddr, len, reading);

else

     bio = bio_copy_user(q, uaddr, len, reading);


bio_copy_user uses the HBA's scatter-gather pages for the copy.  The
problem I face is that if the data is not aligned properly, or the len
parameter does not divide properly, and the transfer is greater than
the number of scatter-gather pages, the IO operation will fail.  I am
considering increasing the size of my data buffer to meet the
length-alignment requirement and changing the 'len' parameter passed
into blk_rq_map_user to include a larger size than my CDB transfer
length.  This way it will hit the bio_map_user branch instead of the
bio_copy_user branch and I won't have to worry about insufficient
sg-pages.  The tape device should still honor the CDB write length, so
unless there are other limitations imposed by the Linux kernel or SCSI
subsystem, I think this is a workable solution.  Does anyone in this
list know if there are requirements against sending a larger buffer
length than CDB transfer length?

Thanks in advance.

Brett Dennis
IBM tape device driver developer

Re: Length and alignment in scsi_execute_async and blk_rq_map_user

[Boaz Harrosh]

On 06/30/2010 12:23 AM, Brett Dennis wrote:
> Hello,
> 
> Apologies for posting twice (messed up on the subject line of the first post...)
> 
> In kernel 2.6.26, the implementation of __blk_rq_map_user changed and
> now includes a call to bio_copy_user:
> 
> uaddr = (unsigned long) ubuf;
> alignment = queue_dma_alignment(q) | q->dma_pad_mask;
> if (!(uaddr & alignment) && !(len & alignment))
> 
>      bio = bio_map_user(q, NULL, uaddr, len, reading);
> 
> else
> 
>      bio = bio_copy_user(q, uaddr, len, reading);
> 
> 
> bio_copy_user uses the HBA's scatter-gather pages for the copy.  The
> problem I face is that if the data is not aligned properly, or the len
> parameter does not divide properly, and the transfer is greater than
> the number of scatter-gather pages, the IO operation will fail.  I am
> considering increasing the size of my data buffer to meet the
> length-alignment requirement and changing the 'len' parameter passed
> into blk_rq_map_user to include a larger size than my CDB transfer
> length.  This way it will hit the bio_map_user branch instead of the
> bio_copy_user branch and I won't have to worry about insufficient
> sg-pages.  The tape device should still honor the CDB write length, so
> unless there are other limitations imposed by the Linux kernel or SCSI
> subsystem, I think this is a workable solution.  Does anyone in this
> list know if there are requirements against sending a larger buffer
> length than CDB transfer length?
> 

It depends. If you are issuing BLOCK_PC commands then you are in the clear.
If an FS_PC commands then your ULD must take proper care at completion.
What ULD are you using?

> Thanks in advance.
> 
> Brett Dennis
> IBM tape device driver developer

Boaz

Re: Length and alignment in scsi_execute_async and blk_rq_map_user

[Boaz Harrosh]

On 06/30/2010 01:30 PM, Boaz Harrosh wrote:
> On 06/30/2010 12:23 AM, Brett Dennis wrote:
>> Hello,
>>
>> Apologies for posting twice (messed up on the subject line of the first post...)
>>
>> In kernel 2.6.26, the implementation of __blk_rq_map_user changed and
>> now includes a call to bio_copy_user:
>>
>> uaddr = (unsigned long) ubuf;
>> alignment = queue_dma_alignment(q) | q->dma_pad_mask;
>> if (!(uaddr & alignment) && !(len & alignment))
>>
>>      bio = bio_map_user(q, NULL, uaddr, len, reading);
>>
>> else
>>
>>      bio = bio_copy_user(q, uaddr, len, reading);
>>
>>
>> bio_copy_user uses the HBA's scatter-gather pages for the copy.  The
>> problem I face is that if the data is not aligned properly, or the len
>> parameter does not divide properly, and the transfer is greater than
>> the number of scatter-gather pages, the IO operation will fail.  I am
>> considering increasing the size of my data buffer to meet the
>> length-alignment requirement and changing the 'len' parameter passed
>> into blk_rq_map_user to include a larger size than my CDB transfer
>> length.  This way it will hit the bio_map_user branch instead of the
>> bio_copy_user branch and I won't have to worry about insufficient
>> sg-pages.  The tape device should still honor the CDB write length, so
>> unless there are other limitations imposed by the Linux kernel or SCSI
>> subsystem, I think this is a workable solution.  Does anyone in this
>> list know if there are requirements against sending a larger buffer
>> length than CDB transfer length?
>>
> 
> It depends. If you are issuing BLOCK_PC commands then you are in the clear.
> If an FS_PC commands then your ULD must take proper care at completion.
> What ULD are you using?
> 

Sorry I missed the scsi_execute_async() part in the title. So yes it is
BLOCK_PC. I think it should be fine, unless some LLD would break because
LLDs and HW do funny things. I'd try it out. If you have problems with
particular HW, perhaps keep a black-list flag. You might expect some
residual returns from some LLDs.

>> Thanks in advance.
>>
>> Brett Dennis
>> IBM tape device driver developer
> 
> Boaz

Length and alignment in scsi_execute_async and blk_rq_map_user

[Brett Dennis]

Hello,

Question for linux-scsi: In kernel 2.6.26, the implementation of
__blk_rq_map_user changed and now includes a call to bio_copy_user:

uaddr = (unsigned long) ubuf;
alignment = queue_dma_alignment(q) | q->dma_pad_mask;
if (!(uaddr & alignment) && !(len & alignment))

bio = bio_map_user(q, NULL, uaddr, len, reading);

else

bio = bio_copy_user(q, uaddr, len, reading);


bio_copy_user uses the HBA's scatter-gather pages for the copy.  The
problem I face is that if the data size is not aligned properly, or
the len parameter does not divide properly, and the transfer is
greater than the number of scatter-gather pages, the IO operation will
fail.  I am considering increasing the size of my data buffer to meet
the length-alignment requirement and changing the 'len' parameter
passed into blk_rq_map_user to include a larger size than my CDB
transfer length.  The tape device should still honor the CDB write
length, so unless there are other limitations imposed by the Linux
kernel or SCSI subsystem, I think this is a workable solution.  Does
anyone in this list know if there is are requirements against sending
a larger buffer length than CDB transfer length?

Thanks in advance.

Brett Dennis
IBM tape device driver developer