troubles writing to a mmapped PCI BAR

[Clem Taylor <clem.taylor@gmail.com>]

I'm working on a Au1550 driver for a PCI based co-processor. The
driver provides mmap() that allows the mapping of a PCI BAR. From
userspace I can happily read from the mmaped region, but writes just
hang the user space program. gdb shows that the user program is
sitting at the write statement. The read() and write() system calls
work just fine as well.

In the driver mmap() does:
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
return io_remap_page_range(vma, vma->vm_start,
    barStart + (vma->vm_pgoff << PAGE_SHIFT),
    vma->vm_end - vma->vm_start, vma->vm_page_prot);

The user space program does:
u32 *mem;
fd = open("/dev/moo-0", O_RDWR);
mem = mmap(NULL, 4*1024*1024. PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
...
fprintf(stderr, "mem[42]=0x%08X\n", mem[42]);
mem[42] = 0xDEADBEEF;

When I run the test, it will print the current value of mem[42], then
hang on the write to mem[42]. /proc/pid/maps shows the mmap with rw-s
permissions:
2abd4000-2afd4000 rw-s 00000000 00:09 134        /dev/moo-0
top shows that the test process is consuming ~100% of the CPU.

I'm really at a loss as to what is happening. I'd imagine that the
userspace program is page faulting (not sure how to verify that) and
the fault handler is not returning. Any ideas what I might be missing?

                       Thanks,
                       Clem