mmap()/mlockall()/read()

mmap()/mlockall()/read()

[est]

Benno Senoner discourseth:
> 
> Eric,
> when not mlock()-ing the entire process (or at least the RT relevant data),
> you have no chance to give ANY guarantee thet some pages could
> not be swapped out.

That's exactly my concern..I have no guarantees that I won't run into
N page-faults in a given period and miss my deadlines.  Unfortunately,
locking and mapping aren't that flexible.  I need to use
mlockall(MCL_CURRENT | MCL_FUTURE) to make sure I have no page faults.
Then anything I mmap() will be initially locked as well..even though
it may be larger than my RAM.  A MAP_UNLOCKED option to mmap() would
be a non-portable solution.

> What I'm trying to say: read() doesn't work better than mmap() in terms of
> behaviour during swapping, because the kernel could easily swap out
> your buffer where you read() in the data.

Actually, the last time I tested this extensively (with a 2.0.x kernel
I think), mmap() was much better performing than read()..for a while.
When sequential accessing of a file got to a point around my RAM
limits, the disk activity became insane.  Maybe things are better now.

> I will add a feature to the pagefaulter thread which uses mlock()/munlock if
> root privileges are available, so that even
> Eric is satisfied.
> :-)

Eric is hard to satisfy. :)

> PS: mlockall(MCL_CURRENT|MCL_FUTURE) is a bad idea because when you do the
> mmap() of the large file the process tries to load all into mem,and my scheme
> would not work. Better to use mlockall(MCL_CURRENT) and mlock()/munlock()
> areas on demand.

This is a useful solution in some circumstances.  However, it means I
can't use malloc()/new, may have trouble with shared libraries or
dynamically loaded plugins and have to worry about reserving stack
space.  It's not the way I want to work.

Eric

Re: mmap()/mlockall()/read()

[Benno Senoner]

I agree no all topics,

using mlockall(MCL_CURRENT|MCL_FUTURE) on a low mem box,
and trying to mmap a large file would simply not work.

I would definitively use MAP_UNLOCKED on Linux,
becaue it could give us the best possible performance.
(and maybe provide a fallback solution for other OSes) 

I think the best fallback solution is the following (but not 100% reliable
in the case of syscalls):
mlock(MCL_CURRENT);
mlock() malloec()ed aread
( or is there a better way to do this ?)


but strange,
MAP_UNLOCKED seems to not exist on my 2.2.12 kernel !!
I grepped through the entire kernel source and include files but
nothing !!!!!

in /usr/include/bits/mman.h
there are some flags:
----
/* These are Linux-specific.  */
#ifdef __USE_MISC
# define MAP_GROWSDOWN  0x0100          /* Stack-like segment.  */
# define MAP_DENYWRITE  0x0800          /* ETXTBSY */
# define MAP_EXECUTABLE 0x1000          /* Mark it as an executable.  */
# define MAP_LOCKED     0x2000          /* Lock the mapping.  */
# define MAP_NORESERVE  0x4000          /* Don't check for reservations.  */
#endif
----

no MAP_UNLOCKED present ...
is this a 2.3.x feature ?


> > What I'm trying to say: read() doesn't work better than mmap() in terms of
> > behaviour during swapping, because the kernel could easily swap out
> > your buffer where you read() in the data.
> 
> Actually, the last time I tested this extensively (with a 2.0.x kernel
> I think), mmap() was much better performing than read()..for a while.
> When sequential accessing of a file got to a point around my RAM
> limits, the disk activity became insane.  Maybe things are better now.
> 
> > I will add a feature to the pagefaulter thread which uses mlock()/munlock if
> > root privileges are available, so that even
> > Eric is satisfied.
> > :-)
> 
> Eric is hard to satisfy. :)

You are not the only !
:-)

> 
> > PS: mlockall(MCL_CURRENT|MCL_FUTURE) is a bad idea because when you do the
> > mmap() of the large file the process tries to load all into mem,and my scheme
> > would not work. Better to use mlockall(MCL_CURRENT) and mlock()/munlock()
> > areas on demand.
> 
> This is a useful solution in some circumstances.  However, it means I
> can't use malloc()/new, may have trouble with shared libraries or
> dynamically loaded plugins and have to worry about reserving stack
> space.  It's not the way I want to work.

agreed for shared libs, but if you don't make any strange syscalls ,
I think there are not very much problems.
As for malloc : just mlock() the area after the allocation.

regards,
Benno.

Re: mmap()/mlockall()/read()

[est]

Benno Senoner discourseth:
> 
> I think the best fallback solution is the following (but not 100% reliable
> in the case of syscalls):
> mlock(MCL_CURRENT);
> mlock() malloec()ed aread

Hmm, I remain sceptical that I'll see large apps using this.  Using
other-people's-libraries is common after all.  It would mean a lot of
work that doesn't seem worth the benefit to me.  However, it would be
pleasant if people proved me wrong. :)

> but strange,
> MAP_UNLOCKED seems to not exist on my 2.2.12 kernel !!

It doesn't exist..I made it up! :)

I think it would be A Good Idea, and it looks easy to add.

> > Eric is hard to satisfy. :)
> 
> You are not the only !
> :-)

Thence comes progress.

Eric