From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4D2E10E4.7040608@domain.hid> Date: Wed, 12 Jan 2011 21:36:52 +0100 From: Gilles Chanteperdrix MIME-Version: 1.0 References: <6FCCA913376DD7488F4139A4D11B8F48016311A5@domain.hid> <4D1FAE66.8080005@domain.hid> <6FCCA913376DD7488F4139A4D11B8F4801673CAB@troe2k1.cs.myharris.net> <4D24E3AC.2000904@domain.hid> <4D24EB92.3050104@domain.hid> <4D24F0B7.7010604@domain.hid> <6FCCA913376DD7488F4139A4D11B8F4801673CD4@domain.hid> <4D25C9D6.4030707@domain.hid> <4D2CE77C.7000109@domain.hid> <6FCCA913376DD7488F4139A4D11B8F4801674163@domain.hid> <4D2DAE16.2060805@domain.hid> <6FCCA913376DD7488F4139A4D11B8F48016741A9@domain.hid> <4D2DD61E.7000006@domain.hid> <6FCCA913376DD7488F4139A4D11B8F48016741F6@domain.hid> In-Reply-To: <6FCCA913376DD7488F4139A4D11B8F48016741F6@domain.hid> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA memory List-Id: Help regarding installation and common use of Xenomai List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Herrera-Bendezu, Luis" Cc: xenomai@xenomai.org Herrera-Bendezu, Luis wrote: >> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] >> Sent: Wednesday, January 12, 2011 11:26 AM >> To: Herrera-Bendezu, Luis >> Cc: xenomai@xenomai.org >> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA memory >> >> Herrera-Bendezu, Luis wrote: >>>> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] >>>> Sent: Wednesday, January 12, 2011 8:35 AM >>>> To: Herrera-Bendezu, Luis >>>> Cc: xenomai@xenomai.org >>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA memory >>>> >>>> Herrera-Bendezu, Luis wrote: >>>>>> -----Original Message----- >>>>>> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] >>>>>> Sent: Tuesday, January 11, 2011 6:28 PM >>>>>> To: Herrera-Bendezu, Luis >>>>>> Cc: xenomai@xenomai.org >>>>>> Subject: Re: [Xenomai-help] [Xenomai -help] User space access to DMA memory >>>>>> >>>>>> Gilles Chanteperdrix wrote: >>>>>>> Herrera-Bendezu, Luis wrote: >>>>>>>> On 01/05/2011 5:29 PM Gilles Chanteperdrix wrote: >>>>>>>>> Steven A. Falco wrote: >>>>>>>>>> On 01/05/2011 04:33 PM, Gilles Chanteperdrix wrote: >>>>>>>>> Ok. Could you try to do the same operation with the native API? You just >>>>>>>>> have to pass H_SHARED | H_DMA | H_NONCACHED as flags to rt_heap_create >>>>>>>>> to get the same effect as pci_dma_alloc_coherent. >>>>>>>>> >>>>>>>>> Just to see if the error lies in RTDM implementation or in Xenomai >>>>>>>>> generic code. >>>>>>>>> >>>>>>>>> >>>>>>> (...) >>>>>>> What about the other thing I asked you to test? >>>>>>> >>>>>> Ping? Any news about this test? >>>>> rt_heap_create() with flags H_SHARED | H_DMA | H_NONCACHED report -EINVAL. >>>>> Documentation indicates that H_NONCACHE is not compatible with H_DMA. >>>> Right, supporting H_NONCACHED | H_DMA would mean that we would have to >>>> use kmalloc/get_free_pages then establish a non-cached mapping in >>>> kernel-space with vm_map_ram. >>>> >>>> Could you try with H_NONCACHED, but without H_DMA? Of course, the >>>> mapping can not be used for DMA, as it will probably not be physically >>>> contiguous, but at least you can try whether accessing in user-space a >>>> non-cached mapping works. >>> It does work but unless an external unit writes to heap memory it would >>> be difficult to verify that the non-cached memory actually works, i.e. >>> access from user space gets expected value(s). >> I am quite confident this works. >> >>>> In any case, we see a defect in the RTDM interface here: we can not ask >>>> the mapping to be mapped non-cacheable, which somewhat defeats the >>>> purpose of pci_alloc_consistent. I do not know enough the powerpc >>>> architecture to know whether this could be the cause of your issue (the >>>> same physical area mapped twice, once cached, once non-cached). However, >>>> on the ARM architecture, for instance, it is bad. >>>> >>> Let me summarize the ideas discussed so far to allocate consistent memory >>> suitable for DMA and corresponding mapping to user space: >>> * rt_heap cannot be created with both H_NONCACHED and H_DMA. But it is >>> automatically mapped to user space with rt_heap_bind(). >>> >>> A solution can be to allocate heap with H_SHARED | H_DMA, somehow >>> get bus address of single block of memory (rt_heap { sba } ?) to >>> used for DMA operations. >> The test with rt_heap was just a test to have a comparison between >> xnheap code and rtdm_mmap. But we may indeed want to fix mmappable >> xnheaps later on. >> >>> * RTDM interface rtdm_mmap/unmap cannot handle non-cacheable memory. >>> >>> A solution can be to allocate GFP_DMA memory with kmalloc(), use >>> rtdm_mmap to map to user space. Programmatically make memory >>> consistent before/after DMA transfer to/from external unit, e.g. >>> dma_sync_single_for_device()/dma_sync_single_for_cpu(). This is >>> similar to what streaming DMA mappings do. >>> >>> * Ideally, it should be possible to take memory allocated from >>> dma_alloc_coherent()/pci_alloc_consistent() and mapped it to >>> user space using rtdm_mmap(). >> To check that the non-cacheable mapping is indeed the problem, here is a >> patch which adds the possibility to add non-cacheable mappings: >> >> diff --git a/ksrc/skins/rtdm/drvlib.c b/ksrc/skins/rtdm/drvlib.c >> index 3495e63..ec0ddae 100644 >> --- a/ksrc/skins/rtdm/drvlib.c >> +++ b/ksrc/skins/rtdm/drvlib.c >> @@ -1791,6 +1791,7 @@ void rtdm_nrtsig_pend(rtdm_nrtsig_t *nrt_sig); >> >> #if defined(CONFIG_XENO_OPT_PERVASIVE) || defined(DOXYGEN_CPP) >> struct rtdm_mmap_data { >> + int noncached; >> void *src_vaddr; >> phys_addr_t src_paddr; >> struct vm_operations_struct *vm_ops; >> @@ -1815,6 +1816,9 @@ static int rtdm_mmap_buffer(struct file *filp, >> struct vm_area_struct *vma) >> maddr = vma->vm_start; >> size = vma->vm_end - vma->vm_start; >> >> + if (mmap_data->noncached) >> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); >> + >> #ifdef CONFIG_MMU >> /* Catch vmalloc memory (vaddr is 0 for I/O mapping) */ >> if ((vaddr >= VMALLOC_START) && (vaddr < VMALLOC_END)) { >> @@ -1975,6 +1979,24 @@ int rtdm_mmap_to_user(rtdm_user_info_t *user_info, >> void *vm_private_data) >> { >> struct rtdm_mmap_data mmap_data = { >> + .noncached = 0, >> + .src_vaddr = src_addr, >> + .src_paddr = 0, >> + .vm_ops = vm_ops, >> + .vm_private_data = vm_private_data >> + }; >> + >> + return rtdm_do_mmap(user_info, &mmap_data, len, prot, pptr); >> +} >> + >> +int rtdm_mmap_noncached_to_user(rtdm_user_info_t *user_info, >> + void *src_addr, size_t len, >> + int prot, void **pptr, >> + struct vm_operations_struct *vm_ops, >> + void *vm_private_data) >> +{ >> + struct rtdm_mmap_data mmap_data = { >> + .noncached = 1, >> .src_vaddr = src_addr, >> .src_paddr = 0, >> .vm_ops = vm_ops, >> @@ -2043,6 +2065,7 @@ int rtdm_iomap_to_user(rtdm_user_info_t *user_info, >> void *vm_private_data) >> { >> struct rtdm_mmap_data mmap_data = { >> + .noncached = 0, >> .src_vaddr = NULL, >> .src_paddr = src_addr, >> .vm_ops = vm_ops, >> >> Simply call rtdm_mmap_noncached_to_user instead of rtdm_mmap_to_user. >> > > Get same kernel panic error as before with this patch. I need to look closer > at what are the memory allocation functions used by pci_ and dma_. Could you try rtdm_iomap_to_user (but beware, pass the physical address returned by pointer by pci_alloc_consistent)? virt_to_page, or __pa will not work with vmalloc/ioremap addresses. And pci_alloc_consistent probably returns a vmalloc mapping due to the fact that the mapping needs to be non-cacheable, which will happen if your powerpc does not support cache snooping, and define CONFIG_NOT_COHERENT_CACHE. -- Gilles.