Funnies with remap_pfn_range, x86_64, > 4GB RAM, kernels < 2.6.16

["Bryan O'Sullivan" <bos@pathscale.com>]

As I was preparing the ipath driver for submission, I got rid of our
accursed vmops->nopage routine and replaced it with the much simpler use
of remap_pfn_range.

Unfortunately, we've recently been testing this code on machines with
more than 4GB of memory, and found that it is not working reliably on
kernels older than 2.6.16.  As far as I can tell, every prior kernel
back to 2.6.9 is affected.

The symptom occurs when we use remap_pfn_range to map some driver memory
(allocated with dma_alloc_coherent) into userspace, and get the hardware
to DMA into that memory range.  The physical and virtual addresses all
look OK; the DMA from the hardware appears to succeed; but the pages
written to do not show any changes, most of the time (occasionally it
works, but we don't know why).

These problems do not occur with 2.6.16, so this looks like a kernel bug
that got fixed somewhere.  What I'm wondering is (a) does anyone
remember fixing this, because I can't see anything obvious in the myriad
of likely contenders, and (b) has anyone else been faced with this
problem and found a workaround for older kernels?

Thanks,

	<b

-- 
Bryan O'Sullivan <bos@pathscale.com>