From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail6.bemta8.messagelabs.com (mail6.bemta8.messagelabs.com [216.82.243.55]) by kanga.kvack.org (Postfix) with ESMTP id 2EA566B0012 for ; Wed, 18 May 2011 11:03:43 -0400 (EDT) Received: by pzk4 with SMTP id 4so992422pzk.14 for ; Wed, 18 May 2011 08:03:41 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20110518144129.GB4296@dumpdata.com> References: <20110518144129.GB4296@dumpdata.com> Date: Wed, 18 May 2011 17:03:41 +0200 Message-ID: Subject: Re: driver mmap implementation for memory allocated with pci_alloc_consistent()? From: Leon Woestenberg Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: linux-pci@vger.kernel.org, Konrad Rzeszutek Wilk Cc: linux-mm@kvack.org Hello, On Wed, May 18, 2011 at 4:41 PM, Konrad Rzeszutek Wilk wrote: > On Wed, May 18, 2011 at 03:02:30PM +0200, Leon Woestenberg wrote: >> >> memory allocated with pci_alloc_consistent() returns the (kernel) >> virtual address and the bus address (which may be different from the >> physical memory address). >> >> What is the correct implementation of the driver mmap (file operation >> method) for such memory? > > You are going to use the physical address from the CPU side. So not > the bus address. Instead use the virtual address and find the > physical address from that. page_to_pfn() does a good job. > pci_alloc_consistent() returns a kernel virtual address. To find the page I think virt_to_page() suits me better, right? #define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) > Then you can call 'vm_insert_page(vma...)' > > Or 'vm_insert_mixed' Thanks, that opens a whole new learning curve experience. Can I call vmalloc_to_page() on memory allocated with pci_alloc_consistent()? If so, then remap_vmalloc_range() looks promising. I could not find PCI driver examples calling vm_insert_page() and I am know I can trip into the different memory type pointers easily. How does your suggestion relate to using the vma ops fault() (formerly known as nopage() to mmap memory allocated by pci_alloc_consistent()? i.e. Such as suggested in http://www.gossamer-threads.com/lists/linux/kernel/702127#702127 > Use 'cscope' on the Linux kernel. Thanks for the suggestion. How would cscope help me find vm_insert_page() given my question? On hind-sight all questions seem to be easy once finding the correct Documentation / source-code in the first place. I usually use http://lxr.linux.no/ and friends. Regards, -- Leon -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with ESMTP id 108AE6B0023 for ; Wed, 18 May 2011 11:41:16 -0400 (EDT) Date: Wed, 18 May 2011 11:40:56 -0400 From: Konrad Rzeszutek Wilk Subject: Re: driver mmap implementation for memory allocated with pci_alloc_consistent()? Message-ID: <20110518154055.GA7037@dumpdata.com> References: <20110518144129.GB4296@dumpdata.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: To: Leon Woestenberg Cc: linux-pci@vger.kernel.org, linux-mm@kvack.org On Wed, May 18, 2011 at 05:03:41PM +0200, Leon Woestenberg wrote: > Hello, > > On Wed, May 18, 2011 at 4:41 PM, Konrad Rzeszutek Wilk > wrote: > > On Wed, May 18, 2011 at 03:02:30PM +0200, Leon Woestenberg wrote: > >> > >> memory allocated with pci_alloc_consistent() returns the (kernel) > >> virtual address and the bus address (which may be different from the > >> physical memory address). > >> > >> What is the correct implementation of the driver mmap (file operation > >> method) for such memory? > > > > You are going to use the physical address from the CPU side. So not > > the bus address. Instead use the virtual address and find the > > physical address from that. page_to_pfn() does a good job. > > > pci_alloc_consistent() returns a kernel virtual address. To find the > page I think virt_to_page() suits me better, right? > > #define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) > > > Then you can call 'vm_insert_page(vma...)' > > > > Or 'vm_insert_mixed' > > Thanks, that opens a whole new learning curve experience. > > Can I call vmalloc_to_page() on memory allocated with > pci_alloc_consistent()? If so, then remap_vmalloc_range() looks > promising. No. That is b/c pci_alloc_consistent allocates pages from .. well, this is a bit complex and varies on the platform. But _mostly_ if your device is 32-bit, it allocates it from ZONE_DMA32. Otherwise it is from other zones. The 'vmalloc' pages are quite different and are usually not exposed to the PCI devices, unless you do some extra jumps (you need to kmap them). > > I could not find PCI driver examples calling vm_insert_page() and I am > know I can trip into the different memory type pointers easily. ttm_bo_vm.c ? fb_defio.c ? > > How does your suggestion relate to using the vma ops fault() (formerly > known as nopage() to mmap memory allocated by pci_alloc_consistent()? You can use the pages that you had allocated via pci_alloc_consistent and stitch them in the userspace vma. > i.e. Such as suggested in > http://www.gossamer-threads.com/lists/linux/kernel/702127#702127 > > > Use 'cscope' on the Linux kernel. > > Thanks for the suggestion. How would cscope help me find > vm_insert_page() given my question? You can find examples of who uses it. > > On hind-sight all questions seem to be easy once finding the correct > Documentation / source-code in the first place. I usually use > http://lxr.linux.no/ and friends. > > > Regards, > -- > Leon -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail6.bemta8.messagelabs.com (mail6.bemta8.messagelabs.com [216.82.243.55]) by kanga.kvack.org (Postfix) with ESMTP id 3E8A36B0012 for ; Wed, 18 May 2011 15:35:08 -0400 (EDT) Received: by pzk4 with SMTP id 4so1139645pzk.14 for ; Wed, 18 May 2011 12:35:06 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20110518154055.GA7037@dumpdata.com> References: <20110518144129.GB4296@dumpdata.com> <20110518154055.GA7037@dumpdata.com> Date: Wed, 18 May 2011 21:35:06 +0200 Message-ID: Subject: Re: driver mmap implementation for memory allocated with pci_alloc_consistent()? From: Leon Woestenberg Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: Konrad Rzeszutek Wilk Cc: linux-pci@vger.kernel.org, linux-mm@kvack.org Hello Konrad, On Wed, May 18, 2011 at 5:40 PM, Konrad Rzeszutek Wilk wrote: > On Wed, May 18, 2011 at 05:03:41PM +0200, Leon Woestenberg wrote: >> On Wed, May 18, 2011 at 4:41 PM, Konrad Rzeszutek Wilk >> wrote: >> > On Wed, May 18, 2011 at 03:02:30PM +0200, Leon Woestenberg wrote: >> >> >> >> memory allocated with pci_alloc_consistent() returns the (kernel) >> >> virtual address and the bus address (which may be different from the >> >> physical memory address). >> >> >> >> What is the correct implementation of the driver mmap (file operation >> >> method) for such memory? >> > >> >> I could not find PCI driver examples calling vm_insert_page() and I am >> know I can trip into the different memory type pointers easily. > > ttm_bo_vm.c ? > fb_defio.c ? > None of which use pci/dma_alloc_consistent(). Obviously, I have no complete understanding of the Linux memory management subsystem, and the info on vm_insert_page() is rather shallow in the case of pci_alloc_consistent(). http://lxr.linux.no/#linux+v2.6.38/mm/memory.c#L1789 1789 update_mmu_cache(vma, addr, pte); /* XXX: why not for insert_page? */ I tried this: static int buffer_mmap(struct file *file, struct vm_area_struct *vma) { ... /* pages must not be cached as this would result in cache line sized accesses to the end point */ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); /* VM_RESERVED: prevent the pages from being swapped out */ vma->vm_flags |= VM_RESERVED; vma->vm_private_data = file->private_data; /* vaddr is the (virtual) address returned by pci_alloc_consistent(); * vsize is the corresponding size */ start = vma->vm_start; /* size is page-aligned */ while (vsize > 0) { printk(KERN_DEBUG "vaddr = %p\n", lro_char->engine->ringbuffer_virt); printk(KERN_DEBUG "start = %p\n", start); struct page *page = virt_to_page(vaddr); printk(KERN_DEBUG "page = %p\n", page); printk(KERN_DEBUG "vm_insert_page(...0x%08lx)\n", (unsigned long)vaddr); /* insert the given page into vma, mapped at the given start address */ err = vm_insert_page(vma, start, page); if (err) { printk(KERN_DEBUG "vm_insert_page()\n"); return err; } start += PAGE_SIZE; vaddr += PAGE_SIZE; vsize -= PAGE_SIZE; } return 0; } which hard crashes my system. Any ideas on a generic function that mmap() pci_alloc_consistent() memory to user space? Thanks, -- Leon -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail203.messagelabs.com (mail203.messagelabs.com [216.82.254.243]) by kanga.kvack.org (Postfix) with ESMTP id 5535D6B0022 for ; Wed, 18 May 2011 19:05:25 -0400 (EDT) Received: by pzk4 with SMTP id 4so1232741pzk.14 for ; Wed, 18 May 2011 16:05:23 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <20110518144129.GB4296@dumpdata.com> <20110518154055.GA7037@dumpdata.com> Date: Thu, 19 May 2011 00:59:18 +0200 Message-ID: Subject: Re: driver mmap implementation for memory allocated with pci_alloc_consistent()? From: Leon Woestenberg Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: linux-pci@vger.kernel.org, linux-mm@kvack.org Hello, On Wed, May 18, 2011 at 9:35 PM, Leon Woestenberg wrote: > On Wed, May 18, 2011 at 5:40 PM, Konrad Rzeszutek Wilk >>> > On Wed, May 18, 2011 at 03:02:30PM +0200, Leon Woestenberg wrote: >>> >> >>> >> What is the correct implementation of the driver mmap (file operation >>> >> method) for such memory? >>> > I have written an implementation based on vm_insert_pfn() and friends, and posted the code in a new thread. It doesn't work yet but I hope some of you kernel experts can look along. Regards, -- Leon -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail203.messagelabs.com (mail203.messagelabs.com [216.82.254.243]) by kanga.kvack.org (Postfix) with ESMTP id 9DF266B0011 for ; Wed, 18 May 2011 21:34:56 -0400 (EDT) Received: by pvc12 with SMTP id 12so1288358pvc.14 for ; Wed, 18 May 2011 18:34:55 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1305767957.2375.117.camel@sli10-conroe> References: <1305767957.2375.117.camel@sli10-conroe> Date: Thu, 19 May 2011 03:34:55 +0200 Message-ID: Subject: Re: driver mmap implementation for memory allocated with pci_alloc_consistent()? From: Leon Woestenberg Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: "linux-pci@vger.kernel.org" , linux-mm@kvack.org Hello, On Thu, May 19, 2011 at 3:19 AM, Shaohua Li wrote: > On Wed, 2011-05-18 at 21:02 +0800, Leon Woestenberg wrote: > why use pci_alloc_consistent? you can allocate pages and mmap it to > userspace. when you want to do dma, you can use pci_map_page to get dma > address for the pages and do whatever. > Thanks for thinking along. I need contiguous memory in this case. But yes, I have just found out that __get_free_pages() with pci_map_single() does work with my mmap() fault() handler. See my other thread with the code posted. I just want to understand how this would work with pci_alloc_consistent(), as that is the generic interface for PCI drivers. Note that the latter provides consistent / coherent mapping, whereas pci_map_single() does not in general. On x86 it probably is the same due to bus-snooping (right?). Regards, -- Leon -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org