Memory-Mapping with LFS

Memory-Mapping with LFS

[Andreas Baer]

Who is the memory mapping expert? :)

What are the current file size limits for memory mapping via glibc's
mmap() function on linux:

- for a native 32-Bit System not using LFS?
- for a native 32-Bit System using LFS?
- for a native 64-Bit System?

(linux-kernel >2.6, of course)

It would be nice if someone could tell me what I have to consider if I
want to use memory mapping for files. I'm currently a little bit
confused about it (information overflow :)). Personal opinions about
speed (maybe increase or decrease for large files) are also welcome.

--------------------------

The glibc documentation says:
"Since mmapped pages can be stored back to their file when physical
memory is low, it is possible to mmap files orders of magnitude larger
than both the physical memory and swap space. The only limit is address
space. The theoretical limit is 4GB on a 32-bit machine - however, the
actual limit will be smaller since some areas will be reserved for other
purposes. If the LFS interface is used the file size on 32-bit systems
is not limited to 2GB (offsets are signed which reduces the addressable
area of 4GB by half); the full 64-bit are available."

- I doubt that the full 64-Bit (something within Exabyte) are available
in practical use. Right or wrong?


I've also found an old kernel-list e-mail from 2004 that says:
"There is a limit per process in the kernel vm that prevent from
mmapping more than 512GB of data."

- Is this still true for the current kernel?


An example:

Let's presume the following case. I have an 8 GB file, 1 GB physical
memory and I want to use memory mapping for that file using LFS on a
32-Bit machine.

- Is it possible?

If yes, let's presume that I have 2 or more pointers, that are
frequently pointing to completely different places and switch the data
they are pointing to.

- How is it managed (by the kernel)? Through the pages, that are
mentioned in the glibc documentation above? Are these page operations
really faster than normal random file access (lseek etc)?

Re: Memory-Mapping with LFS

[Christoph Lameter]

On Fri, 26 Aug 2005, Andreas Baer wrote:

> What are the current file size limits for memory mapping via glibc's
> mmap() function on linux:

Its 2^wordsize - areas used for other purposes. 1-3GB on i386. A couple of 
petabytes(?) on ia64.

> - I doubt that the full 64-Bit (something within Exabyte) are available
> in practical use. Right or wrong?

You can likely map a 512 M section starting at 6GB into your 
address space.

> I've also found an old kernel-list e-mail from 2004 that says:
> "There is a limit per process in the kernel vm that prevent from
> mmapping more than 512GB of data."
> 
> - Is this still true for the current kernel?

That depends on the architecture. Certainly no problem on Itanium. 

> Let's presume the following case. I have an 8 GB file, 1 GB physical
> memory and I want to use memory mapping for that file using LFS on a
> 32-Bit machine.
> - Is it possible?
Yes. I am not sure why one would have to use LFS. This should work on any 
filesystem.

> If yes, let's presume that I have 2 or more pointers, that are
> frequently pointing to completely different places and switch the data
> they are pointing to.
> 
> - How is it managed (by the kernel)? Through the pages, that are
> mentioned in the glibc documentation above? Are these page operations
> really faster than normal random file access (lseek etc)?

Kernel gets a fault if the page pointed to is not present and reads it in 
from the file. And yes its faster because I/O is aligned and the page does 
not need to be copied from kernel to user space.