Re: PCIe device driver question

[Robert Hancock <hancockr@shaw.ca>]

V.Radhakrishnan wrote:
>> My guess there was a bug in your DMA mapping code. I don't think kmap is 
>> what is normally used for this. I think with get_user_pages one usually 
>> takes the returned page pointers to create an SG list and uses 
>> dma_map_sg to create a DMA mapping for them.
> 
> Looking at the actual code, I see that I had used kmap() only to obtain
> kernel virtual addresses for the array of struct pages obtained from
> user space by using get_user_pages.
> 
> Subsequently, I had used dma_map_single() and dma_unmap_single() calls
> for single buffer calls.

I'm suspicious about this usage, I don't know if that will actually 
work. There is a dma_map_page call which is meant for doing a DMA 
mapping on a struct page which should likely be used instead.

> 
> The code didn't have bugs IMHO since it was used for extensive stress
> testing the initial FPGA prototype as well as the final ASIC chip ,
> sometimes running for over 4 days non-stop without breaking.
> 
> However, using Test Access Points on the board and using a Logic
> Analyzer showed that DMA was NOT taking place when RAM > 896 MB was
> used. The hardware gurus said that PCI bus cycles just didn't seem to be
> taking place when RAM > 896 MB was used as the source OR destination
> address.

Are you sure the address being passed to the device was correct in this 
case? There should be nothing magical about 896MB from a hardware point 
of view, and the kernel in general cannot stop you from DMAing anywhere 
you like.

> 
> Perhaps this was a problem in the earlier kernel(s) and has since been
> rectified ? ( I was using 2.6.15 then ... )
> 
> I am just curious since Sanka Piyaratna reported a 'similar' kind of
> situation.
> 
> Regards
> 
> V. Radhakrishnan