[parisc-linux] ZONE_DMA

[parisc-linux] ZONE_DMA

[Matthew Wilcox]

I notice that right now we're partitioning our memory 50% into DMAable
memory and 50% into normal memory (and 0% into HIGHMEM -- but I believe
this part to be correct).  I can't believe that's really the limit of
memory we can DMA into.

I'm not familiar with the hardware problem here.  Must EISA cards be
restricted to a certain part of the address space?  I can glean from
the source that ISA cards are restricted to a 16MB address space --
are we going to support ISA cards?

I presume that no machine which supports ISA cards has an IOMMU that is
capable of mapping a chunk of higher ram into the 16MB bus address space.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] ZONE_DMA

[Alan Cox]

> I'm not familiar with the hardware problem here.  Must EISA cards be
> restricted to a certain part of the address space?  I can glean from

EISA on PC can hit the full 32bits. DMA is meant to indicate the 16Mb lowest
memory segment that ISA can hit (which may not be at 0 depending on the box)

> the source that ISA cards are restricted to a 16MB address space --
> are we going to support ISA cards?

Some PCI cards have the same limit 8)

> I presume that no machine which supports ISA cards has an IOMMU that is
> capable of mapping a chunk of higher ram into the 16MB bus address space.

Some Alpha's probably can but we don't support that. The pci side API
pci_alloc_consistent does

Re: [parisc-linux] ZONE_DMA

[Matthew Wilcox]

On Mon, Jan 01, 2001 at 06:09:53PM +0000, Alan Cox wrote:
> > I'm not familiar with the hardware problem here.  Must EISA cards be
> > restricted to a certain part of the address space?  I can glean from
> 
> EISA on PC can hit the full 32bits. DMA is meant to indicate the 16Mb lowest
> memory segment that ISA can hit (which may not be at 0 depending on the box)

OK, so people who write EISA drivers are not going to specify GFP_DMA.
Or shouldn't.

> > the source that ISA cards are restricted to a 16MB address space --
> > are we going to support ISA cards?
> 
> Some PCI cards have the same limit 8)

But that's handled through the PCI DMA mapping interface these days.

So how should we be populating our zones?  Put all of our memory in
ZONE_DMA, like ppc, since the only people who'll be specifying GFP_DMA
will have done so incorrectly?  Put the first 16MB of it in there,
since we might support ISA some day?  Put none of it in there on the
grounds we don't support ISA at all?

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] ZONE_DMA

[Alan Cox]

> > EISA on PC can hit the full 32bits. DMA is meant to indicate the 16Mb lowest
> > memory segment that ISA can hit (which may not be at 0 depending on the box)
> 
> OK, so people who write EISA drivers are not going to specify GFP_DMA.
> Or shouldn't.

Or know something about their hardware being crap.

> > Some PCI cards have the same limit 8)
> But that's handled through the PCI DMA mapping interface these days.

Yes

> So how should we be populating our zones?  Put all of our memory in
> ZONE_DMA, like ppc, since the only people who'll be specifying GFP_DMA
> will have done so incorrectly?  Put the first 16MB of it in there,

That is a very dangerous assumption (as some PPC people have discovered when
they DMA stuff into the wrong place)

> since we might support ISA some day?  Put none of it in there on the
> grounds we don't support ISA at all?

I think none is safer. pci_alloc_consistent seems to address all the stuff
we worry about. 

Alan

Re: [parisc-linux] ZONE_DMA

[Matthew Wilcox]

On Mon, Jan 01, 2001 at 06:32:11PM +0000, Alan Cox wrote:
> Or know something about their hardware being crap.

Hm.

> > So how should we be populating our zones?  Put all of our memory in
> > ZONE_DMA, like ppc, since the only people who'll be specifying GFP_DMA
> > will have done so incorrectly?  Put the first 16MB of it in there,
> 
> That is a very dangerous assumption (as some PPC people have discovered when
> they DMA stuff into the wrong place)
> 
> > since we might support ISA some day?  Put none of it in there on the
> > grounds we don't support ISA at all?
> 
> I think none is safer. pci_alloc_consistent seems to address all the stuff
> we worry about. 

But if we have none, then all GFP_DMA allocations will fail.  Which
includes the scsi midlayers.  Ooh, I just noticed that scsi_scan.c
is broken...  expect a patch RSN.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] ZONE_DMA

[Alan Cox]

> But if we have none, then all GFP_DMA allocations will fail.  Which
> includes the scsi midlayers.  Ooh, I just noticed that scsi_scan.c
> is broken...  expect a patch RSN.

SCSI may need fixing, but do you want to do it right 8)

Re: [parisc-linux] ZONE_DMA

[Matthew Wilcox]

On Mon, Jan 01, 2001 at 07:01:19PM +0000, Alan Cox wrote:
> > But if we have none, then all GFP_DMA allocations will fail.  Which
> > includes the scsi midlayers.  Ooh, I just noticed that scsi_scan.c
> > is broken...  expect a patch RSN.
> 
> SCSI may need fixing, but do you want to do it right 8)

I don't think I have time to do that :-)

Here's a patch which makes it slightly better.  Justifications:
 * changed GFP_DMA to GFP_KERNEL | GFP_DMA since GFP_DMA alone is nonsense.
 * got rid of the ?: since it's pure obfuscation in this case.
 * changed &scsi_result0[0] to scsi_result0.  again, sheer obfuscation.

just peering at this little bit makes me realise how large the scsi
problems must be :-)

It compiles.  I promise no more.

Index: drivers/scsi/scsi_scan.c
===================================================================
RCS file: /var/cvs/linux/drivers/scsi/scsi_scan.c,v
retrieving revision 1.1.1.1
diff -u -p -u -r1.1.1.1 scsi_scan.c
--- drivers/scsi/scsi_scan.c	2000/09/19 15:01:34	1.1.1.1
+++ drivers/scsi/scsi_scan.c	2001/01/01 19:09:43
@@ -288,8 +288,11 @@ void scan_scsis(struct Scsi_Host *shpnt,
 		scsi_initialize_queue(SDpnt, shpnt);
 		SDpnt->request_queue.queuedata = (void *) SDpnt;
 		/* Make sure we have something that is valid for DMA purposes */
-		scsi_result = ((!shpnt->unchecked_isa_dma)
-			       ? &scsi_result0[0] : kmalloc(512, GFP_DMA));
+		if (shpnt->unchecked_isa_dma) {
+			scsi_result = kmalloc(512, GFP_KERNEL | GFP_DMA);
+		} else {
+			scsi_result = scsi_result0;
+		}
 	}
 
 	if (scsi_result == NULL) {
@@ -411,7 +414,7 @@ void scan_scsis(struct Scsi_Host *shpnt,
 		}		/* for channel ends */
 	}			/* if/else hardcoded */
 
-      leave:
+ leave:
 
 	{			/* Unchain SRpnt from host_queue */
 		Scsi_Device *prev, *next;
@@ -438,7 +441,7 @@ void scan_scsis(struct Scsi_Host *shpnt,
 	}
 
 	/* If we allocated a buffer so we could do DMA, free it now */
-	if (scsi_result != &scsi_result0[0] && scsi_result != NULL) {
+	if (scsi_result != scsi_result0 && scsi_result != NULL) {
 		kfree(scsi_result);
 	} {
 		Scsi_Device *sdev;

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] ZONE_DMA

[Grant Grundler]

Alan Cox wrote:
> Some Alpha's probably can but we don't support that. The pci side API
> pci_alloc_consistent does

Nit: I think pci_dma_supported() is the interface meant here.

Documentation/DMA-mapping.txt has a good example here on how
pci_dma_supported() supports cripppled PCI devices.

Right now, SBA code (and CCIO too I think) only permit devices
that say they support 32-bit addressing. sba_dma_supported()
code needs to be revisited for two reasons:
o the number of bits the IO MMU actually uses is typically alot less
  - depends on the size of the I/O Pdir. Ie it could support the
  crippled PCI device pretty easily.
o Elroy HW supports 64-bit PCI addressing (Dual Address cycle).

grant

Grant Grundler
Unix Systems Enablement Lab
+1.408.447.7253

Re: [parisc-linux] ZONE_DMA

[Matthew Wilcox]

On Mon, Jan 01, 2001 at 04:59:31PM -0800, Grant Grundler wrote:
> o Elroy HW supports 64-bit PCI addressing (Dual Address cycle).

But, if possible, we want to avoid doing DAC as it cuts down the bandwidth
(and increases latency by a clock).  obviously, if it's a 64-bit device
and a 64-bit controller, there's no problem, but we should try to stick
to 32-bit bus addresses where possible.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] ZONE_DMA

[Grant Grundler]

Matthew Wilcox wrote:
> But, if possible, we want to avoid doing DAC as it cuts down the bandwidth
> (and increases latency by a clock).  obviously, if it's a 64-bit device
> and a 64-bit controller, there's no problem, but we should try to stick
> to 32-bit bus addresses where possible.

That's correct. But then we can bypass the I/O MMU. SBA HW has bits
to support that - but I don't fully understand all the limitations of when
this "mode" can be used. Bypassing the I/O MMU would improve DMA performance
and reduce work CPU has to do to "map" each page of DMA.

Though it's generally a good direction, optimizing the PCI bus isn't
always the right answer.  Ie moving from 32- to 64-bit wide data
transfers is a *much* bigger performance win than DAC.

grant

Grant Grundler
Unix Systems Enablement Lab
+1.408.447.7253