Re: DMA memory limitation?

[Steffen Persvold <sp@scali.no>]

> > GFP_DMA is ISA dma reachable, Forget the IA64, their setup is weird and 
> > should best be ignored until 2.5 as and when they sort it out.

Really ? I don't think I can ignore IA64, there are people who ask for it....

> > > bounce buffers are needed. On Alpha GFP_DMA is not limited at all (I think). Correct me if
> >
> > Alpha has various IOMMU facilities
> >
> > > I'm wrong, but I really think there should be a general way of allocating memory that is
> > > 32bit addressable (something like GFP_32BIT?) so you don't need a lot of #ifdef's in your
> > > code.
> > No ifdefs are needed
> >
> >         GFP_DMA - ISA dma reachable
> >         pci_alloc_* and friends - PCI usable memory
> 
> pci_alloc_* is designed to support ISA.
> 
> Pass pci_dev==NULL to pci_alloc_* for ISA devices, and it allocs GFP_DMA
> for you.
> 

Sure, but the IA64 platforms that are out now doesn't have an IOMMU, so bounce buffers are
used if you don't specify GFP_DMA in your get_free_page.

Now lets say you have a driver with a page allocator. Eventually you want to make some if
the allocated pages available to a 32bit PCI device. These pages has to be consistent (i.e
the driver doesn't have to wait for a PCI flush for the data to be valid, sort of like a
ethernet ring buffer). I could use the pci_alloc_consistent() function
(pci_alloc_consistent() allocates a buffer with GFP_DMA on IA64), but since I already have
the pages, I have to use pci_map_single (or pci_map_sg). Inside pci_map_single on IA64
something called swiotlb buffers (bounce buffers) are used if the device can't support
64bit addressing and the address of the memory to map is above the 4G limit. The swiotlb
buffers are below the 4G limit and therefore reachable by any PCI device. The problem
about these buffers are that the content are not copied to the original location before
you do a pci_sync_* or a pci_unmap_* (they are not consistent) and they are a limited
resource (allocated at boot time). My solution for now was to use :

#if defined(__ia64__)
  int flag = GFP_DMA;
#else
  int flag = 0;
#endif

Maybe IA64 could implement GFP_HIGHMEM (as on i386) so that if no flags were used you were
guaranteed to get 32bit memory ???


Regards
-- 
  Steffen Persvold               Systems Engineer
  Email : mailto:sp@scali.no     Scali AS (http://www.scali.com)
  Tlf   : (+47) 22 62 89 50      Olaf Helsets vei 6
  Fax   : (+47) 22 62 89 51      N-0621 Oslo, Norway