[Linux-ia64] RE: DMA memory limitation?

[Linux-ia64] RE: DMA memory limitation?

[Matt_Domsch]

> The important thing is that pci_alloc_consistent and the other PCI DMA
> functions work as advertised on IA64.  If you pass NULL to
> pci_alloc_consistent, IA64 should give you an ISA DMA-able 
> address.  If
> you don't, you get a 32-bit PCI DMA address.  Use of GFP_DMA is a
> arch-specific detail, so don't let me confuse you there.

Until recently, on IA-64, pci_alloc_consistent() given a NULL pci_dev would
fault.  It's been fixed in at least the most recent IA-64 patch.
pci_map_single() and pci_map_sg() still have the same problem, as they
dereference pci_dev w/o checking for NULL first.


-- 
Matt Domsch
Sr. Software Engineer
Dell Linux Solutions
www.dell.com/linux
#2 Linux Server provider with 17% in the US and 14% Worldwide (IDC)!
#3 Unix provider with 18% in the US (Dataquest)!

[Linux-ia64] RE: DMA memory limitation?

[White, Charles]

Is there a patch that fixes the pci_alloc_consistent for older versions of
the IA-64 kernels?  To do 64bit DMA in my driver I need to pass
pci_alloc_consistent pci_dev pointer of NULL.  

I'm interested in getting this to work on the older IA64 kernels that are
currently on distribution CDs. 
		

		From: Matt_Domsch@Dell.com
		To: jgarzik@mandrakesoft.com, sp@scali.no
		Cc: alan@lxorguk.ukuu.org.uk, helgehaf@idb.hist.no, 
			pvvvarma@techmas.hcltech.com,
linux-kernel@vger.kernel.org, 
			davem@redhat.com, linux-ia64@linuxia64.org
		Date: Fri, 6 Jul 2001 10:56:47 -0500 
		Subject: [Linux-ia64] RE: DMA memory limitation?

		> The important thing is that pci_alloc_consistent and the
other PCI DMA
		> functions work as advertised on IA64.  If you pass NULL to
		> pci_alloc_consistent, IA64 should give you an ISA DMA-able

		> address.  If
		> you don't, you get a 32-bit PCI DMA address.  Use of
GFP_DMA is a
		> arch-specific detail, so don't let me confuse you there.

		Until recently, on IA-64, pci_alloc_consistent() given a
NULL pci_dev would
		fault.  It's been fixed in at least the most recent IA-64
patch.
		pci_map_single() and pci_map_sg() still have the same
problem, as they
		dereference pci_dev w/o checking for NULL first.


		-- 
		Matt Domsch
		Sr. Software Engineer
		Dell Linux Solutions
		www.dell.com/linux
		#2 Linux Server provider with 17% in the US and 14%
Worldwide (IDC)!
		#3 Unix provider with 18% in the US (Dataquest)!

		

RE: [Linux-ia64] RE: DMA memory limitation?

[Matt_Domsch]

> Is there a patch that fixes the pci_alloc_consistent for 
> older versions of
> the IA-64 kernels?  To do 64bit DMA in my driver I need to pass
> pci_alloc_consistent pci_dev pointer of NULL.  

You're not going to find such a thing, and it wouldn't make sense even if
you did.  See Jeff Garzik's note again:

>> The important thing is that pci_alloc_consistent and the other PCI DMA
>> functions work as advertised on IA64.  If you pass NULL to
>> pci_alloc_consistent, IA64 should give you an ISA DMA-able
>> address.  If you don't, you get a 32-bit PCI DMA address.

Never have I seen a using NULL pci_dev pointer give you a 64-bit DMA
address.  Per Jeff's comment, it should give you an address with a mask of
24-bits (within ISA space, i.e. <16MB), but since you can't add in any ISA
cards on IA-64, this wouldn't make much sense either.

At present, the DMA address is entirely dependent upon pci_dev->dma_mask,
which by default is a 32-bit mask, and which you can set to be a 64-bit
mask.  If you need a 64-bit DMA address, create a new pci_dev structure, set
the mask to ~0L and pass that in.  (This may cause other problems, I can't
say for certain.)

Thanks,
Matt

--
Matt Domsch
Sr. Software Engineer
Dell Linux Solutions
www.dell.com/linux
#2 Linux Server provider with 17% in the US and 14% Worldwide (IDC)!
#3 Unix provider with 18% in the US (Dataquest)!

RE: [Linux-ia64] RE: DMA memory limitation?

[White, Charles]

Sorry, just to clarify...  
My device has the limitation that the command buffers I send must be in 32
bit space, but the IO can be in 64 bit.  But there is only ONE dma_mask for
the device.  The way I figured out how to do 64 bit DMA (from reading the
information in the DMA-mapping.txt) was to set my DMA mask of the pci_dev
for my device to allow 64 bit DMA, but when I allocated my command buffers(
with pci_alloc_consistent)  send a pci_dev of NULL.  So I would get the
default of memory that was 24bit addressable.  

This Seg faulted in earlier versions of the IA64 port, but works great in
the latest versions.  But now I have been asked if I can get this to work
with the kernels that are released IA64 CDs.  

So, I was wondering if there was already a patch that would fix this one
problem with the some of the older trees.  

		-----Original Message-----
		From:	Matt_Domsch@Dell.com [mailto:Matt_Domsch@Dell.com]
		Sent:	Monday, July 09, 2001 9:47 AM
		To:	White, Charles
		Cc:	linux-ia64@linuxia64.org
		Subject:	RE: [Linux-ia64] RE: DMA memory limitation?

		> Is there a patch that fixes the pci_alloc_consistent for 
		> older versions of
		> the IA-64 kernels?  To do 64bit DMA in my driver I need to
pass
		> pci_alloc_consistent pci_dev pointer of NULL.  

		You're not going to find such a thing, and it wouldn't make
sense even if
		you did.  See Jeff Garzik's note again:

		>> The important thing is that pci_alloc_consistent and the
other PCI DMA
		>> functions work as advertised on IA64.  If you pass NULL
to
		>> pci_alloc_consistent, IA64 should give you an ISA
DMA-able
		>> address.  If you don't, you get a 32-bit PCI DMA address.

		Never have I seen a using NULL pci_dev pointer give you a
64-bit DMA
		address.  Per Jeff's comment, it should give you an address
with a mask of
		24-bits (within ISA space, i.e. <16MB), but since you can't
add in any ISA
		cards on IA-64, this wouldn't make much sense either.

		At present, the DMA address is entirely dependent upon
pci_dev->dma_mask,
		which by default is a 32-bit mask, and which you can set to
be a 64-bit
		mask.  If you need a 64-bit DMA address, create a new
pci_dev structure, set
		the mask to ~0L and pass that in.  (This may cause other
problems, I can't
		say for certain.)

		Thanks,
		Matt

		--
		Matt Domsch
		Sr. Software Engineer
		Dell Linux Solutions
		www.dell.com/linux
		#2 Linux Server provider with 17% in the US and 14%
Worldwide (IDC)!
		#3 Unix provider with 18% in the US (Dataquest)!

		

RE: [Linux-ia64] RE: DMA memory limitation?

[Matt_Domsch]

> My device has the limitation that the command buffers I send 
> must be in 32
> bit space, but the IO can be in 64 bit.  But there is only 
> ONE dma_mask for the device.

Ahh, yes.  The AMI MegaRAID driver/controllers have the same requirements.
megaraid solved it as:

pci_find_device() finds their device, they get a pci_dev back which has a
32-bit dma_mask.
pci_alloc_consistent() the mailbox (this must be in 32-bit space).
if it's a 64-bit-capable controller, set pci_dev->dma_mask = ~0L.
pci_alloc_consistent() the command buffers and scatter-gather lists (these
can be in 64-bit space).

Could something similar work for you?

Thanks,
Matt

--
Matt Domsch
Sr. Software Engineer
Dell Linux Solutions
www.dell.com/linux
#2 Linux Server provider with 17% in the US and 14% Worldwide (IDC)!
#3 Unix provider with 18% in the US (Dataquest)!

RE: [Linux-ia64] RE: DMA memory limitation?

[White, Charles]

Yes, I thought about doing it that way, and it COULD be done, but right now
the driver allocates command buffer for IOCTLs as needed.  

So the pci_dev set to NULL allows to continue to do that... 

		-----Original Message-----
		From:	Matt_Domsch@Dell.com [mailto:Matt_Domsch@Dell.com]
		Sent:	Monday, July 09, 2001 10:20 AM
		To:	White, Charles
		Cc:	linux-ia64@linuxia64.org
		Subject:	RE: [Linux-ia64] RE: DMA memory limitation?

		> My device has the limitation that the command buffers I
send 
		> must be in 32
		> bit space, but the IO can be in 64 bit.  But there is only

		> ONE dma_mask for the device.

		Ahh, yes.  The AMI MegaRAID driver/controllers have the same
requirements.
		megaraid solved it as:

		pci_find_device() finds their device, they get a pci_dev
back which has a
		32-bit dma_mask.
		pci_alloc_consistent() the mailbox (this must be in 32-bit
space).
		if it's a 64-bit-capable controller, set pci_dev->dma_mask ~0L.
		pci_alloc_consistent() the command buffers and
scatter-gather lists (these
		can be in 64-bit space).

		Could something similar work for you?

		Thanks,
		Matt

		--
		Matt Domsch
		Sr. Software Engineer
		Dell Linux Solutions
		www.dell.com/linux
		#2 Linux Server provider with 17% in the US and 14%
Worldwide (IDC)!
		#3 Unix provider with 18% in the US (Dataquest)!
		

Re: [Linux-ia64] RE: DMA memory limitation?

[Jes Sorensen]

>>>>> "Matt" = Matt Domsch <Matt_Domsch@Dell.com> writes:

>>> The important thing is that pci_alloc_consistent and the other PCI
>>> DMA functions work as advertised on IA64.  If you pass NULL to
>>> pci_alloc_consistent, IA64 should give you an ISA DMA-able
>>> address.  If you don't, you get a 32-bit PCI DMA address.

Matt> Never have I seen a using NULL pci_dev pointer give you a 64-bit
Matt> DMA address.  Per Jeff's comment, it should give you an address
Matt> with a mask of 24-bits (within ISA space, i.e. <16MB), but since
Matt> you can't add in any ISA cards on IA-64, this wouldn't make much
Matt> sense either.

Well but we don't have any ISA busses on the ia64 boxes. The only
reason to return a 24 bit mask would be for broken devices that
require a 31 or 29 bit or similar mask, but one should set that via
the dma_mask interface anyway.

Jes