troubles writing to a mmapped PCI BAR

troubles writing to a mmapped PCI BAR

[Clem Taylor]

I'm working on a Au1550 driver for a PCI based co-processor. The
driver provides mmap() that allows the mapping of a PCI BAR. From
userspace I can happily read from the mmaped region, but writes just
hang the user space program. gdb shows that the user program is
sitting at the write statement. The read() and write() system calls
work just fine as well.

In the driver mmap() does:
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
return io_remap_page_range(vma, vma->vm_start,
    barStart + (vma->vm_pgoff << PAGE_SHIFT),
    vma->vm_end - vma->vm_start, vma->vm_page_prot);

The user space program does:
u32 *mem;
fd = open("/dev/moo-0", O_RDWR);
mem = mmap(NULL, 4*1024*1024. PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
...
fprintf(stderr, "mem[42]=0x%08X\n", mem[42]);
mem[42] = 0xDEADBEEF;

When I run the test, it will print the current value of mem[42], then
hang on the write to mem[42]. /proc/pid/maps shows the mmap with rw-s
permissions:
2abd4000-2afd4000 rw-s 00000000 00:09 134        /dev/moo-0
top shows that the test process is consuming ~100% of the CPU.

I'm really at a loss as to what is happening. I'd imagine that the
userspace program is page faulting (not sure how to verify that) and
the fault handler is not returning. Any ideas what I might be missing?

                       Thanks,
                       Clem

Re: troubles writing to a mmapped PCI BAR

[Clem Taylor]

On 4/21/05, Clem Taylor <clem.taylor@gmail.com> wrote:
> I'm working on a Au1550 driver for a PCI based co-processor. The
> driver provides mmap() that allows the mapping of a PCI BAR. From
> userspace I can happily read from the mmaped region, but writes just
> hang the user space program. gdb shows that the user program is
> sitting at the write statement. The read() and write() system calls
> work just fine as well.

I saw in drivers/video/epson1356fb.c it was doing:
        // FIXME: shouldn't have to do this. If the pages are marked writeable,
        // the TLB fault handlers should set these.
        pgprot_val(vma->vm_page_prot) |= (_PAGE_DIRTY | _PAGE_VALID);

So I tried adding this before my io_remap_page_range() and it seems to
have fixed my problem. My mmap() write to a PCI device is working now.
I tried setting just the _PAGE_DIRTY bit and that seems to be the
trick. Any ideas why I would need do to this or what is going on?

                        --Clem

Re: troubles writing to a mmapped PCI BAR

[Ralf Baechle]

On Fri, Apr 22, 2005 at 04:22:53PM -0400, Clem Taylor wrote:

> So I tried adding this before my io_remap_page_range() and it seems to
> have fixed my problem. My mmap() write to a PCI device is working now.
> I tried setting just the _PAGE_DIRTY bit and that seems to be the
> trick. Any ideas why I would need do to this or what is going on?

Without the process simply won't have access priviledges to that page, see
the TLB chapter of the CPU manual.

(I'm simplifying things alot here.)

  Ralf