Physical address mapping to user space in dom0 problem

Physical address mapping to user space in dom0 problem

[Haw-Yuan Yang]

[-- Attachment #1.1: Type: text/plain, Size: 709 bytes --]

I have a PCI device which can perform bus master DMA. I developed a PCI
device driver which can allocate DMA buffer and provide mmap operation to
map the I/O and DMA buffer to user space.



The PCI device driver works well under Linux 2.6.20. However it brokes under
XEN 3.0.4. It appears that mmap map to the wrong memory.  remap_pfn_range in
mmap function return no error when it is used to map both I/O and DMA
buffer.  If I use io_remap_pfn_range instead of  remap_pfn_range, I/O
mapping works but DMA buffer mapping return -1.



Is this a known problem for XEN 3.0.4 and will be fixed later? Is there any
alternative way to map the DMA address to user space in XEN dom0?



Thanks in advance,



hyang

[-- Attachment #1.2: Type: text/html, Size: 1698 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 221 bytes --]




On 16/3/07 23:27, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:

> Is there any alternative way to map the DMA address to user space in XEN dom0?

Mmap via /dev/mem will work. It uses direct_remap_pfn_range().

 -- Keir


[-- Attachment #1.2: Type: text/html, Size: 688 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Haw-Yuan Yang]

[-- Attachment #1.1: Type: text/plain, Size: 770 bytes --]

It still not working. Here is what I did:

In driver I have an IOCTL option IO_DMA_ALLOC to allocate the DMA buffer by
using pci_alloc_consistent() and return the allocated physical address to
user space.

In user space testing program:

1. Call IOCTL IO_DMA_ALLOC to allocate a 64Kbyte DMA buffer. The physical
address of the allocated buffer is 0x03bb0000.
2. Open /dev/mem and
call mmap(0,64*1024,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0x03bb0000) ;

the return of mmap is -1.


On 3/16/07, Keir Fraser <keir@xensource.com> wrote:
>
>
>
>
> On 16/3/07 23:27, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:
>
> Is there any alternative way to map the DMA address to user space in XEN
> dom0?
>
>
> Mmap via /dev/mem will work. It uses direct_remap_pfn_range().
>
>  -- Keir
>

[-- Attachment #1.2: Type: text/html, Size: 1681 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 591 bytes --]

You want to mmap() the dma_handle, not the CPU pointer.

 -- Keir

On 17/3/07 00:36, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:

> In driver I have an IOCTL option IO_DMA_ALLOC to allocate the DMA buffer by
> using pci_alloc_consistent() and return the allocated physical address to user
> space.
>  
> In user space testing program:
>  
> 1. Call IOCTL IO_DMA_ALLOC to allocate a 64Kbyte DMA buffer. The physical
> address of the allocated buffer is 0x03bb0000.
> 2. Open /dev/mem and call
> mmap(0,64*1024,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0x03bb0000) ;
>  
> the return of mmap is -1.



[-- Attachment #1.2: Type: text/html, Size: 1105 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Haw-Yuan Yang]

[-- Attachment #1.1: Type: text/plain, Size: 923 bytes --]

I knew that pci_alloc_consistent() will return both kernel virtual address
and physical address. If I pass the physical address (dma_handler) to mmap,
it return -1. However if I pass the kernel virtual address to mmap, I got a
pointer point to the worng memory.

hyang


On 3/17/07, Keir Fraser <keir@xensource.com> wrote:
>
> You want to mmap() the dma_handle, not the CPU pointer.
>
>  -- Keir
>
> On 17/3/07 00:36, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:
>
> In driver I have an IOCTL option IO_DMA_ALLOC to allocate the DMA buffer
> by using pci_alloc_consistent() and return the allocated physical address to
> user space.
>
> In user space testing program:
>
> 1. Call IOCTL IO_DMA_ALLOC to allocate a 64Kbyte DMA buffer. The physical
> address of the allocated buffer is 0x03bb0000.
> 2. Open /dev/mem and call
> mmap(0,64*1024,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0x03bb0000) ;
>
> the return of mmap is -1.
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 1856 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 697 bytes --]




On 17/3/07 21:52, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:

> I knew that pci_alloc_consistent() will return both kernel virtual address and
> physical address. If I pass the physical address (dma_handler) to mmap, it
> return -1. However if I pass the kernel virtual address to mmap, I got a
> pointer point to the worng memory.

Oh, I see the problem. You can only mmap() MMIO regions using /dev/mem when
running on Xen. Could your device (which you already implement ioctl() for)
also implement mmap() itself? Then you can simply
remap_pfn_range(virt_to_phys(cpu-ptr-returned-by-pci-alloc-consistent)).
This would be a neater interface to userspace for your device anyway, imo.

 -- Keir


[-- Attachment #1.2: Type: text/html, Size: 1179 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Haw-Yuan Yang]

[-- Attachment #1.1: Type: text/plain, Size: 993 bytes --]

Thanks, It works.

Here is what I have in the mmap handler:

if  the physical address is MMIO
    io_remap_pfn_range(MMIO_address)
else
   remap_pfn_range(virt_to_phys(cpu-ptr-returned-by-pci-alloc-consistent)

hyang


On 3/18/07, Keir Fraser <keir@xensource.com> wrote:
>
>
>
>
> On 17/3/07 21:52, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:
>
> I knew that pci_alloc_consistent() will return both kernel virtual address
> and physical address. If I pass the physical address (dma_handler) to mmap,
> it return -1. However if I pass the kernel virtual address to mmap, I got a
> pointer point to the worng memory.
>
>
> Oh, I see the problem. You can only mmap() MMIO regions using /dev/mem
> when running on Xen. Could your device (which you already implement ioctl()
> for) also implement mmap() itself? Then you can simply
> remap_pfn_range(virt_to_phys(cpu-ptr-returned-by-pci-alloc-consistent)).
> This would be a neater interface to userspace for your device anyway, imo.
>
>  -- Keir
>

[-- Attachment #1.2: Type: text/html, Size: 1950 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Physical address mapping to user space in dom0 problem

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 535 bytes --]




On 18/3/07 20:11, "Haw-Yuan Yang" <hawyuan@gmail.com> wrote:

> Here is what I have in the mmap handler:
>  
> if  the physical address is MMIO
>     io_remap_pfn_range(MMIO_address)
> else
>    remap_pfn_range(virt_to_phys(cpu-ptr-returned-by-pci-alloc-consistent)

Well, perhaps you can simplify your own mmap() handler since you perhaps
know that it will only be used to mmap() your pci_alloc_consistent() memory
block. Hence you may only need to handle the remap_pfn_range() case. But
anyway, you have the idea now.

 -- Keir



[-- Attachment #1.2: Type: text/html, Size: 1100 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel