Question: how to map SCSI data DMA address to virtual address?

Question: how to map SCSI data DMA address to virtual address?

[Ju, Seokmann]

Hi,

In the 'scsi_cmnd' structure, there are two entries holding address
information for data to be transferred. One is 'request_buffer' and the
other one is 'buffer'.
In case of 'use_sg' is non-zero, those entries indicates the address of
the scatter-gather table.

Is there way to get virtual address (so that the data could be accessed
by the driver) of the actual data in the case of 'use_sg' is non-zero?

Any comments would be appreciated.


Thank you,

Seokmann

Re: Question: how to map SCSI data DMA address to virtual address?

[Arjan van de Ven]

On Thu, 2006-03-02 at 09:53 -0700, Ju, Seokmann wrote:
> Hi,
> 
> In the 'scsi_cmnd' structure, there are two entries holding address
> information for data to be transferred. One is 'request_buffer' and the
> other one is 'buffer'.
> In case of 'use_sg' is non-zero, those entries indicates the address of
> the scatter-gather table.

use_sg is never non-zero so that's easy

> 
> Is there way to get virtual address (so that the data could be accessed
> by the driver) of the actual data in the case of 'use_sg' is non-zero?

not really; unless you mapped it. The physical address may already been
translated by the iommu... at which point there is no direct mapping to
kernel memory.

Why do you need to do this? It's generally bad for drivers to snoop
data! 

Re: Question: how to map SCSI data DMA address to virtual address?

[linux-os (Dick Johnson)]

On Thu, 2 Mar 2006, Ju, Seokmann wrote:

> Hi,
>
> In the 'scsi_cmnd' structure, there are two entries holding address
> information for data to be transferred. One is 'request_buffer' and the
> other one is 'buffer'.
> In case of 'use_sg' is non-zero, those entries indicates the address of
> the scatter-gather table.
>
> Is there way to get virtual address (so that the data could be accessed
> by the driver) of the actual data in the case of 'use_sg' is non-zero?
>
> Any comments would be appreciated.
>
>
> Thank you,
>
> Seokmann
> -

There is a macro for this purpose. However, for experiments, in
the kernel, you can add PAGE_OFFSET (0xC00000000) to get the virtual
address. The macro is __va(a), its inverse is __pa(a).

Careful. These things can change.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Question: how to map SCSI data DMA address to virtual address?

[Jens Axboe]

On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
> 
> On Thu, 2 Mar 2006, Ju, Seokmann wrote:
> 
> > Hi,
> >
> > In the 'scsi_cmnd' structure, there are two entries holding address
> > information for data to be transferred. One is 'request_buffer' and the
> > other one is 'buffer'.
> > In case of 'use_sg' is non-zero, those entries indicates the address of
> > the scatter-gather table.
> >
> > Is there way to get virtual address (so that the data could be accessed
> > by the driver) of the actual data in the case of 'use_sg' is non-zero?
> >
> > Any comments would be appreciated.
> >
> >
> > Thank you,
> >
> > Seokmann
> > -
> 
> There is a macro for this purpose. However, for experiments, in
> the kernel, you can add PAGE_OFFSET (0xC00000000) to get the virtual
> address. The macro is __va(a), its inverse is __pa(a).
> 
> Careful. These things can change.

Bzzt no, this wont work if an iommu is involved. It also wont work if
the page doesn't have a virtual address mapping (think highmem).

-- 
Jens Axboe

Re: Question: how to map SCSI data DMA address to virtual address?

[linux-os (Dick Johnson)]

On Thu, 2 Mar 2006, Jens Axboe wrote:

> On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
>>
>> On Thu, 2 Mar 2006, Ju, Seokmann wrote:
>>
>>> Hi,
>>>
>>> In the 'scsi_cmnd' structure, there are two entries holding address
>>> information for data to be transferred. One is 'request_buffer' and the
>>> other one is 'buffer'.
>>> In case of 'use_sg' is non-zero, those entries indicates the address of
>>> the scatter-gather table.
>>>
>>> Is there way to get virtual address (so that the data could be accessed
>>> by the driver) of the actual data in the case of 'use_sg' is non-zero?
>>>
>>> Any comments would be appreciated.
>>>
>>>
>>> Thank you,
>>>
>>> Seokmann
>>> -
>>
>> There is a macro for this purpose. However, for experiments, in
>> the kernel, you can add PAGE_OFFSET (0xC00000000) to get the virtual
>> address. The macro is __va(a), its inverse is __pa(a).
>>
>> Careful. These things can change.
>
> Bzzt no, this wont work if an iommu is involved. It also wont work if
> the page doesn't have a virtual address mapping (think highmem).
>
> --
> Jens Axboe
>

Are you going to get DMA-able pages out of high memory? The guy is
doing scatter-table DMA, i.e., linked DMA. I think you need to
build that table with pages from get_dma_page(). If you use
highmem, somebody can program the iommu right out from under
the DMA engine while a DMA is in progress because the CPU is
not involved and can be executing lots of code that can do lots
of bad things.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Question: how to map SCSI data DMA address to virtual address?

[Jens Axboe]

On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
> 
> On Thu, 2 Mar 2006, Jens Axboe wrote:
> 
> > On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
> >>
> >> On Thu, 2 Mar 2006, Ju, Seokmann wrote:
> >>
> >>> Hi,
> >>>
> >>> In the 'scsi_cmnd' structure, there are two entries holding address
> >>> information for data to be transferred. One is 'request_buffer' and the
> >>> other one is 'buffer'.
> >>> In case of 'use_sg' is non-zero, those entries indicates the address of
> >>> the scatter-gather table.
> >>>
> >>> Is there way to get virtual address (so that the data could be accessed
> >>> by the driver) of the actual data in the case of 'use_sg' is non-zero?
> >>>
> >>> Any comments would be appreciated.
> >>>
> >>>
> >>> Thank you,
> >>>
> >>> Seokmann
> >>> -
> >>
> >> There is a macro for this purpose. However, for experiments, in
> >> the kernel, you can add PAGE_OFFSET (0xC00000000) to get the virtual
> >> address. The macro is __va(a), its inverse is __pa(a).
> >>
> >> Careful. These things can change.
> >
> > Bzzt no, this wont work if an iommu is involved. It also wont work if
> > the page doesn't have a virtual address mapping (think highmem).
> >
> > --

> > Jens Axboe
> >
> 
> Are you going to get DMA-able pages out of high memory? The guy is
> doing scatter-table DMA, i.e., linked DMA. I think you need to
> build that table with pages from get_dma_page(). If you use

A highmem page is a regular page, it just doesn't have a kernel virtual
mapping. You can dma to/from these pages just fine.

> highmem, somebody can program the iommu right out from under
> the DMA engine while a DMA is in progress because the CPU is
> not involved and can be executing lots of code that can do lots
> of bad things.

Nonsense. You can't have multiple iommu mappings or dma transactions
against the same highmem page of course, however that is no different
than a low mem page.

-- 
Jens Axboe

Re: Question: how to map SCSI data DMA address to virtual address?

[Christoph Hellwig]

On Thu, Mar 02, 2006 at 09:53:06AM -0700, Ju, Seokmann wrote:
> Hi,
> 
> In the 'scsi_cmnd' structure, there are two entries holding address
> information for data to be transferred. One is 'request_buffer' and the
> other one is 'buffer'.
> In case of 'use_sg' is non-zero, those entries indicates the address of
> the scatter-gather table.
> 
> Is there way to get virtual address (so that the data could be accessed
> by the driver) of the actual data in the case of 'use_sg' is non-zero?

For each sg list entry do something like:

	buffer = kmap_atomic(sg->page, KM_USER0) + sg->offset;
	<access buffer>
	kunmap_atomic(buffer - sg->offset, KM_USER0);

Re: Question: how to map SCSI data DMA address to virtual address?

[James Bottomley]

On Thu, 2006-03-02 at 17:09 +0000, Christoph Hellwig wrote:
> For each sg list entry do something like:
> 
> 	buffer = kmap_atomic(sg->page, KM_USER0) + sg->offset;
> 	<access buffer>
> 	kunmap_atomic(buffer - sg->offset, KM_USER0);

Remember too that the data might not necessarily be valid without a
flush depending on where it has come from (or where it is going).  See
the dma_sync_sg_for_device/cpu.

James

RE: Question: how to map SCSI data DMA address to virtual address?

[Ju, Seokmann]

Hi,

On Thursday, March 02, 2006 11:58 AM Arjan van de Ven wrote:
> Why do you need to do this? It's generally bad for drivers to snoop
> data! 
I understood that is bad. I am trying to make sure the data written to disk drive are identical with the data from upper layer by comparing actual data in the driver.
This is a part of debugging only not for release driver, obviously.
So, is it completely unable to get this done?
Any tricky solution?

Thank you,

> -----Original Message-----
> From: Arjan van de Ven [mailto:arjan@infradead.org] 
> Sent: Thursday, March 02, 2006 11:58 AM
> To: Ju, Seokmann
> Cc: Ju, Seokmann; linux-kernel@vger.kernel.org; 
> linux-scsi@vger.kernel.org
> Subject: Re: Question: how to map SCSI data DMA address to 
> virtual address?
> 
> On Thu, 2006-03-02 at 09:53 -0700, Ju, Seokmann wrote:
> > Hi,
> > 
> > In the 'scsi_cmnd' structure, there are two entries holding address
> > information for data to be transferred. One is 
> 'request_buffer' and the
> > other one is 'buffer'.
> > In case of 'use_sg' is non-zero, those entries indicates 
> the address of
> > the scatter-gather table.
> 
> use_sg is never non-zero so that's easy
> 
> > 
> > Is there way to get virtual address (so that the data could 
> be accessed
> > by the driver) of the actual data in the case of 'use_sg' 
> is non-zero?
> 
> not really; unless you mapped it. The physical address may 
> already been
> translated by the iommu... at which point there is no direct 
> mapping to
> kernel memory.
> 
> Why do you need to do this? It's generally bad for drivers to snoop
> data! 
> 
> 

RE: Question: how to map SCSI data DMA address to virtual address?

[Ju, Seokmann]

Hi,

Thursday, March 02, 2006 12:09 PM Christoph Hellwig wrote:
> For each sg list entry do something like:
> 
> 	buffer = kmap_atomic(sg->page, KM_USER0) + sg->offset;
> 	<access buffer>
> 	kunmap_atomic(buffer - sg->offset, KM_USER0);
Thank you for your comment!
I'll try with this.

Thank you again.

RE: Question: how to map SCSI data DMA address to virtual address?

[Ju, Seokmann]

Hi,

On Thursday, March 02, 2006 3:04 PM Dick Johnson wrote:
> On Thu, 2 Mar 2006, Jens Axboe wrote:
> 
> > On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
> >>
> >> On Thu, 2 Mar 2006, Ju, Seokmann wrote:
> >>
> >>> Hi,
> >>>
> >>> In the 'scsi_cmnd' structure, there are two entries 
> holding address
> >>> information for data to be transferred. One is 
> 'request_buffer' and the
> >>> other one is 'buffer'.
> >>> In case of 'use_sg' is non-zero, those entries indicates 
> the address of
> >>> the scatter-gather table.
> >>>
> >>> Is there way to get virtual address (so that the data 
> could be accessed
> >>> by the driver) of the actual data in the case of 'use_sg' 
> is non-zero?
> >>>
> >>> Any comments would be appreciated.
> >>>
> >>>
> >>> Thank you,
> >>>
> >>> Seokmann
> >>> -
> >>
> >> There is a macro for this purpose. However, for experiments, in
> >> the kernel, you can add PAGE_OFFSET (0xC00000000) to get 
> the virtual
> >> address. The macro is __va(a), its inverse is __pa(a).
> >>
> >> Careful. These things can change.
> >
> > Bzzt no, this wont work if an iommu is involved. It also 
> wont work if
> > the page doesn't have a virtual address mapping (think highmem).
> >
> > --
> 
> > Jens Axboe
> >
> 
> Are you going to get DMA-able pages out of high memory? The guy is
> doing scatter-table DMA, i.e., linked DMA. I think you need to
> build that table with pages from get_dma_page(). If you use
> highmem, somebody can program the iommu right out from under
> the DMA engine while a DMA is in progress because the CPU is
> not involved and can be executing lots of code that can do lots
> of bad things.
Thank you for your valueable comment. One good thing is that the system does 2 GB memory so that highmem won't come into picture. I am implementing the feature with suggestions.

Thank you again,

Seokmann

RE: Question: how to map SCSI data DMA address to virtual address?

[linux-os (Dick Johnson)]

On Thu, 2 Mar 2006, Ju, Seokmann wrote:

> Hi,
>
> On Thursday, March 02, 2006 3:04 PM Dick Johnson wrote:
>> On Thu, 2 Mar 2006, Jens Axboe wrote:
>>
>>> On Thu, Mar 02 2006, linux-os (Dick Johnson) wrote:
>>>>
>>>> On Thu, 2 Mar 2006, Ju, Seokmann wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> In the 'scsi_cmnd' structure, there are two entries
>> holding address
>>>>> information for data to be transferred. One is
>> 'request_buffer' and the
>>>>> other one is 'buffer'.
>>>>> In case of 'use_sg' is non-zero, those entries indicates
>> the address of
>>>>> the scatter-gather table.
>>>>>
>>>>> Is there way to get virtual address (so that the data
>> could be accessed
>>>>> by the driver) of the actual data in the case of 'use_sg'
>> is non-zero?
>>>>>
>>>>> Any comments would be appreciated.
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Seokmann
>>>>> -
>>>>
>>>> There is a macro for this purpose. However, for experiments, in
>>>> the kernel, you can add PAGE_OFFSET (0xC00000000) to get
>> the virtual
>>>> address. The macro is __va(a), its inverse is __pa(a).
>>>>
>>>> Careful. These things can change.
>>>
>>> Bzzt no, this wont work if an iommu is involved. It also
>> wont work if
>>> the page doesn't have a virtual address mapping (think highmem).
>>>
>>> --
>>
>>> Jens Axboe
>>>
>>
>> Are you going to get DMA-able pages out of high memory? The guy is
>> doing scatter-table DMA, i.e., linked DMA. I think you need to
>> build that table with pages from get_dma_page(). If you use
>> highmem, somebody can program the iommu right out from under
>> the DMA engine while a DMA is in progress because the CPU is
>> not involved and can be executing lots of code that can do lots
>> of bad things.

> Thank you for your valueable comment. One good thing is that the system
> does 2 GB memory so that highmem won't come into picture. I am implementing
> the feature with suggestions.
>
> Thank you again,
>
> Seokmann
>

You're welcome. Sometimes the simplest thing is the best thing, especially
for a debugging hack. Of course, Jens may be "more perfectly" correct,
but since you are just checking to see the correct data in your
DMA scatter list, and the memory is "DMA pages", the hack will work.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

RE: Question: how to map SCSI data DMA address to virtual address?

[James Bottomley]

On Thu, 2006-03-02 at 14:04 -0700, Ju, Seokmann wrote:
> Thank you for your valueable comment. One good thing is that the system does 2 GB memory so that highmem won't come into picture. I am implementing the feature with suggestions.

2GB is well into highmem on a traditional x86 with a 3/1 split.

For a private project, you're free to do as you wish subject to the
constraints of the GPL.  However, if you want whatever you're planning
to go upstream someday you need to conform to the requirements of the
Linux API which are as Jens and Christoph have already outlined.

James