Re: Why is mmap()ed reserved memory so slow?

[Ralf Baechle <ralf@linux-mips.org>]

On Fri, Jul 15, 2005 at 06:21:44PM -0400, David Chau wrote:

> I'm working on a driver for the Broadcom 1250, and I am using reserved 
> memory for some data buffers. The board comes with 256 MB of RAM, so I 
> boot Linux with "mem=253M" to reserve some RAM at the top of memory, and 
> then mmap() /dev/mem starting at 253 MB.
> 
> The problem is that accessing this memory is ridiculously slow. A simple 
> benchmark revealed that it takes about 200 cycles to read a 64-bit 
> number.

mmap will create uncached mappings for anything above the highest RAM
address.

> If I mmap() /dev/zero instead, a read takes under 3 cycles.

Because you have a cache hits.  No RAM is that fast.

Above 200 cycles really is how horribly slow RAM is compared to a moderatly
clocked system.

> For those of you who knows how the Linux VM works, could you tell me why 
> the memory access is so slow? It look like it might be invoking the 
> page-fault handler on every read. How can I make memory access faster?

  Ralf