From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sh78.surpasshosting.com (sh78.surpasshosting.com [72.29.64.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id C10DFB7113 for ; Wed, 26 Jan 2011 01:31:48 +1100 (EST) Message-ID: <4D3EDEE2.6000809@embedded-sol.com> Date: Tue, 25 Jan 2011 16:32:02 +0200 From: Felix Radensky MIME-Version: 1.0 To: "Ira W. Snyder" Subject: Re: FSL DMA engine transfer to PCI memory References: <4D3DF36A.5050609@embedded-sol.com> <20110124222641.GC26404@ovro.caltech.edu> <4D3E0DBB.60308@embedded-sol.com> <20110125001837.GD26404@ovro.caltech.edu> In-Reply-To: <20110125001837.GD26404@ovro.caltech.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Cc: Scott Wood , "linuxppc-dev@ozlabs.org" List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Hi Ira, On 01/25/2011 02:18 AM, Ira W. Snyder wrote: > On Tue, Jan 25, 2011 at 01:39:39AM +0200, Felix Radensky wrote: >> Hi Ira, Scott >> >> On 01/25/2011 12:26 AM, Ira W. Snyder wrote: >>> On Mon, Jan 24, 2011 at 11:47:22PM +0200, Felix Radensky wrote: >>>> Hi, >>>> >>>> I'm trying to use FSL DMA engine to perform DMA transfer from >>>> memory buffer obtained by kmalloc() to PCI memory. This is on >>>> custom board based on P2020 running linux-2.6.35. The PCI >>>> device is Altera FPGA, connected directly to SoC PCI-E controller. >>>> >>>> 01:00.0 Unassigned class [ff00]: Altera Corporation Unknown device >>>> 0004 (rev 01) >>>> Subsystem: Altera Corporation Unknown device 0004 >>>> Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- >>>> ParErr- Stepping- SERR- FastB2B- >>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>>> >TAbort-SERR->>> Interrupt: pin A routed to IRQ 16 >>>> Region 0: Memory at c0000000 (32-bit, non-prefetchable) >>>> [size=128K] >>>> Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ >>>> Queue=0/0 Enable- >>>> Address: 0000000000000000 Data: 0000 >>>> Capabilities: [78] Power Management version 3 >>>> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA >>>> PME(D0-,D1-,D2-,D3hot-,D3cold-) >>>> Status: D0 PME-Enable- DSel=0 DScale=0 PME- >>>> Capabilities: [80] Express Endpoint IRQ 0 >>>> Device: Supported: MaxPayload 256 bytes, PhantFunc 0, >>>> ExtTag- >>>> Device: Latency L0s<64ns, L1<1us >>>> Device: AtnBtn- AtnInd- PwrInd- >>>> Device: Errors: Correctable- Non-Fatal- Fatal- >>>> Unsupported- >>>> Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ >>>> Device: MaxPayload 128 bytes, MaxReadReq 512 bytes >>>> Link: Supported Speed 2.5Gb/s, Width x1, ASPM L0s, Port 1 >>>> Link: Latency L0s unlimited, L1 unlimited >>>> Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch- >>>> Link: Speed 2.5Gb/s, Width x1 >>>> Capabilities: [100] Virtual Channel >>>> >>>> >>>> I can successfully writel() to PCI memory via address obtained from >>>> pci_ioremap_bar(). >>>> Here's my DMA transfer routine >>>> >>>> static int dma_transfer(struct dma_chan *chan, void *dst, void *src, >>>> size_t len) >>>> { >>>> int rc = 0; >>>> dma_addr_t dma_src; >>>> dma_addr_t dma_dst; >>>> dma_cookie_t cookie; >>>> struct completion cmp; >>>> enum dma_status status; >>>> enum dma_ctrl_flags flags = 0; >>>> struct dma_device *dev = chan->device; >>>> struct dma_async_tx_descriptor *tx = NULL; >>>> unsigned long tmo = msecs_to_jiffies(FPGA_DMA_TIMEOUT_MS); >>>> >>>> dma_src = dma_map_single(dev->dev, src, len, DMA_TO_DEVICE); >>>> if (dma_mapping_error(dev->dev, dma_src)) { >>>> printk(KERN_ERR "Failed to map src for DMA\n"); >>>> return -EIO; >>>> } >>>> >>>> dma_dst = (dma_addr_t)dst; >>>> >>>> flags = DMA_CTRL_ACK | >>>> DMA_COMPL_SRC_UNMAP_SINGLE | >>>> DMA_COMPL_SKIP_DEST_UNMAP | >>>> DMA_PREP_INTERRUPT; >>>> >>>> tx = dev->device_prep_dma_memcpy(chan, dma_dst, dma_src, len, flags); >>>> if (!tx) { >>>> printk(KERN_ERR "%s: Failed to prepare DMA transfer\n", >>>> __FUNCTION__); >>>> dma_unmap_single(dev->dev, dma_src, len, DMA_TO_DEVICE); >>>> return -ENOMEM; >>>> } >>>> >>>> init_completion(&cmp); >>>> tx->callback = dma_callback; >>>> tx->callback_param =&cmp; >>>> cookie = tx->tx_submit(tx); >>>> >>>> if (dma_submit_error(cookie)) { >>>> printk(KERN_ERR "%s: Failed to start DMA transfer\n", >>>> __FUNCTION__); >>>> return -ENOMEM; >>>> } >>>> >>>> dma_async_issue_pending(chan); >>>> >>>> tmo = wait_for_completion_timeout(&cmp, tmo); >>>> status = dma_async_is_tx_complete(chan, cookie, NULL, NULL); >>>> >>>> if (tmo == 0) { >>>> printk(KERN_ERR "%s: Transfer timed out\n", __FUNCTION__); >>>> rc = -ETIMEDOUT; >>>> } else if (status != DMA_SUCCESS) { >>>> printk(KERN_ERR "%s: Transfer failed: status is %s\n", >>>> __FUNCTION__, >>>> status == DMA_ERROR ? "error" : "in progress"); >>>> >>>> dev->device_control(chan, DMA_TERMINATE_ALL, 0); >>>> rc = -EIO; >>>> } >>>> >>>> return rc; >>>> } >>>> >>>> The destination address is PCI memory address returned by >>>> pci_ioremap_bar(). >>>> The transfer silently fails, destination buffer doesn't change >>>> contents, but no >>>> error condition is reported. >>>> >>>> What am I doing wrong ? >>>> >>>> Thanks a lot in advance. >>>> >>> Your destination address is wrong. The device_prep_dma_memcpy() routine >>> works in physical addresses only (dma_addr_t type). Your source address >>> looks fine: you're using the result of dma_map_single(), which returns a >>> physical address. >>> >>> Your destination address should be something that comes from struct >>> pci_dev.resource[x].start + offset if necessary. In your lspci output >>> above, that will be 0xc0000000. >>> >>> Another possible problem: AFAIK you must use the _ONSTACK() variants >>> from include/linux/completion.h for struct completion which are on the >>> stack. >>> >>> Hope it helps, >>> Ira >> Thanks for your help. I'm now passing the result of >> pci_resource_start(pdev, 0) >> as destination address, and destination buffer changes after the >> transfer. But >> the contents of source and destination buffers are different. What >> else could >> be wrong ? >> > After you changed the dst address to pci_resource_start(pdev, 0), I > don't see anything wrong with the code. > > Try using memcpy_toio() to copy some bytes to the FPGA. Also try writing > a single byte at a time (writeb()?) in a loop. This should help > establish that your device is working. > > If you put some pattern in your src buffer (such as 0x0, 0x1, 0x2, ... > 0xff, repeat) does the destination show some pattern after the DMA > completes? (Such as, every 4th byte is correct.) > > Ira memcpy_toio() works fine, the data is written correctly. After DMA, the correct data appears at offsets 0xC, 0x1C, 0x2C, etc. of the destination buffer. I have 12 bytes of junk, 4 bytes of correct data, then again 12 bytes of junk and so on. Felix.