### DMA_ALLOC_COHERENT question

### DMA_ALLOC_COHERENT question

[steve spano]

Hi Folks,

I probably have a simple question and I hope someone can point me in the
right direction.

I have a 2.6.30.2 kernel running with a custom device driver for a
multi-channel sound card via PCI-Express. Interrupts,DMA, everything is
working fine.

We allocate a very small 64KB buffer for our DMA actions using
dma_alloc_coherent.

Tripped across an issue with larger amounts of memory on the motherboard.
This is an x86 ATOM motherboard.

At 256MB of ram, dma_alloc_coherent gets us an address within the first
128MB of ram, near the upper 1/2 of the 256MB actually.

Then at 1GB, we get an address near the upper 1/2 again (512MB)
Then at 2GB, we get an address near the upper 1/2 again (1GB)

This causes a problem on the sound card board because we need to then map
in nearly all 2GB of address space across the PCI bar registers so we can
access our tiny 64KB dma address range sitting up near the 1GB boundary.

This seems like we are doing something not right?

Ideally, we would get a non-cached block of ram that we can DMA to and the
processor can access. This block would ideally always be located near the
base of ram (maybe right after the kernel?). That way we can open a 128MB
window into the system ram from PCI-Express and always be sure to be able
to access our DMA buffer.

Can someone provide some insight in how to do this?
Is there a differnet dma allocation procedure we should use?
Should we just put an "unsigned char []" of 64k in the driver? Can we make
that also non cache?
Or do we have to hack the kernel allocation process?

Im sure there is something simple we can do.

Thanks again

Steve Spano
FLE

Re: ### DMA_ALLOC_COHERENT question

[Clemens Ladisch]

steve spano wrote:
> I have a 2.6.30.2 kernel running with a custom device driver for a
> multi-channel sound card via PCI-Express. Interrupts,DMA, everything is
> working fine.
> 
> We allocate a very small 64KB buffer for our DMA actions using
> dma_alloc_coherent.

Assuming that the sound driver is using the ALSA framework, why don't
you use snd_pcm_lib_malloc_pages like most of the other PCI sound
drivers?

> At 256MB of ram, dma_alloc_coherent gets us an address within the first
> 128MB of ram, near the upper 1/2 of the 256MB actually.
> 
> Then at 1GB, we get an address near the upper 1/2 again (512MB)
> Then at 2GB, we get an address near the upper 1/2 again (1GB)
> 
> This causes a problem on the sound card board because we need to then map
> in nearly all 2GB of address space across the PCI bar registers so we can
> access our tiny 64KB dma address range sitting up near the 1GB boundary.

What have the PCI BARs to do with this?  If your PCIe device wants to do
DMA from/to the computer's main memory, no BARs are involved.

BARs are used to map memory located in the device into the CPU's address
space.

> Ideally, we would get a non-cached block of ram that we can DMA to and the
> processor can access.

"Coherent" means that the CPU cache works correctly in the presence
of DMA.

> This block would ideally always be located near the base of ram (maybe
> right after the kernel?). That way we can open a 128MB window into the
> system ram from PCI-Express and always be sure to be able to access
> our DMA buffer.

What do you mean with "open a window"?  PCIe devices always can access
the entire adress space.

> Should we just put an "unsigned char []" of 64k in the driver?

The memory that modules are loaded into is not DMAable on many
architectures.

> Im sure there is something simple we can do.

Show the source code.  :)


Regards,
Clemens