Re: [PATCH 10 of 20] ipath - support for userspace apps using core driver

[Roland Dreier <rdreier@cisco.com>]

 > The first two are windows onto the chip's MMIO space; we create mappings
 > for userspace in our mmap code using io_remap_pfn_range.
 > 
 > - General user-oriented chip registers.
 > 
 > - Chip PIO buffers.
 > 
 > We don't do any funnies with get_page/SetPageReserved (or the new
 > vm_insert_page) on these.  However, we turn on the VM_* flag christmas
 > tree, in a frenzied effort to make the kernel pay no attention:
 > VM_DONTCOPY | VM_DONTEXPAND | VM_IO | VM_SHM | VM_LOCKED.

I don't think you need to do anything beyond io_remap_pfn_range().
Look at the comment inside remap_pfn_range() in mm/memory.c.
You may want VM_DONTCOPY for fork() handling I guess.

 > The next three kinds or memory are allocated with dma_alloc_coherent
 > (first two) or pci_alloc_consistent (last).  We have a nopage handler
 > that knows how to deal with them.  We set the low two bits of
 > vma->vm_private_data to tell the nopage handler what kind of memory it
 > is dealing with.

As a side note, why do you use both dma_alloc_coherent() and
pci_alloc_consistent()?  Presumably all the allocations are for the
same underlying device, so why not just pick one interface (probably
dma_alloc_coherent(), since it lets you specify the GFP mask).

 > Once again, we sprinkle heaps of VM_* flags all over the freshly baked
 > mapping: VM_DONTEXPAND | VM_DONTCOPY | VM_RESERVED | VM_IO | VM_SHM

You probably want VM_RESERVED.  I don't think you want VM_IO (these
pages are in RAM), and there's not much point to VM_SHM, since it's
currently defined as:

#define VM_SHM          0x00000000      /* Means nothing: delete it later */

VM_DONTEXPAND is fine, although you could handle it in your NOPAGE
method too.  And VM_DONTCOPY is just as above -- it might make fork()
work better for you.

 > The nopage handler looks very normal, except it does a get_page on
 > pages marked with IPATH_VM_PIOAVAILREGS, but not on others.  Presumably
 > this is because they've had SetPageReserved set on them.

I think you should always be doing a get_page().  As far as I know,
there's always a put_page() when the userspace mapping is torn down,
and things are going to get pretty bad if a page count goes below 0.

 - R.