Re: mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and ma

Re: mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and ma

[Alan Cox]

> As you suggested I do basically the following:
> - ptr=mmap() at current offset with lenQ2k
> - memcpy(targetbuffer,ptr,len)   (I must use the memcpy since targetbuffer has
> to   be mlocked() since the audio-playing thread can't tolerate pagefaults
> because   it runs in a low-latency cycle.
> - munmap(ptr,len)

You can do 
	ptr=mmap(blah)
	mlock(ptr, ...)
	munmap(ptr, len)

> Does anyone know if there it/will be a way to do unbuffered mmap()  ?
> I think streaming apps would benefit quite a bit from this.

mmap requires buffering - you are sharing the page with the system page
cache. If you mean you want an mgoaway() to go with munmap() thats what
madvise() provides on some other systems but not Linux yet

mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and makes

[Benno Senoner]

> 
> > I'm afraid, if we want that Linux will be a good multimedia OS, this is a
> > _STRONGLY_NEEDED_  feature.
> 
> debatable.  first, you mean "media _producer_ OS", since media clients
> do not generally mess with this kind of bandwidth.  next, you're also 
> assuming that media production apps should be naive about how they do IO.
> for instance, have you tried using mmap, with explicit munmap's on data
> that you're done with?  or played with some of the parameters on the 
> current code that attempts to prevent cache flushing?

I adapted my file streaming app to use mmap() instead of using read() to read
chunks of up  to 512k into a buffer.
As you suggested I do basically the following:
- ptr=mmap() at current offset with lenQ2k
- memcpy(targetbuffer,ptr,len)   (I must use the memcpy since targetbuffer has
to   be mlocked() since the audio-playing thread can't tolerate pagefaults
because   it runs in a low-latency cycle.
- munmap(ptr,len)

I must say it works quite well, and firing up apps, or accessing the KDE menus,
is faster, 
launching my famous "xterm" still produces disk accesses, 
(cached files are still trown away after a while, but not so agressively like
in the read() case )
but it's now faster and the system seems more usable,
especially painting into a GIMP window, since gimp uses the hd quite a lot.
I'm now able to stream about 50 mono audio tracks (44kHz 16bit) from my
IBM 16GB EIDE UDMA disk, while browsing the web, very fun !
Of course if I copy a large file in background, sooner of later, I get a
ring-buffer overrun , because the streaming-app and the cp must share disk
bandwidth.  ( I'm missing SGI's  guaranteed rate I/O of XFS , sigh .. :-)  )  

again , thank you for your suggestion !
It was a bit tricky to mess with all this page aligns, and special cases,
but it pretty fun to code ! ( got nice segfaults at the beginning  :-) )

> 
> in short, cache flushing is well-understood and media production is just
> one minor case where it comes up.  hinting (via O_ flags, madvise, ioctl)
> are the right way to turn up the agressiveness of the existing free-behind
> and pre-fetching mechanisms.

Does anyone know if there it/will be a way to do unbuffered mmap()  ?
I think streaming apps would benefit quite a bit from this.
> 
> regards, mark hahn.

regards,
Benno.

Re: mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and ma

[Benno Senoner]

On Wed, 25 Aug 1999, Alan Cox wrote:
> > As you suggested I do basically the following:
> > - ptr=mmap() at current offset with lenQ2k
> > - memcpy(targetbuffer,ptr,len)   (I must use the memcpy since targetbuffer has
> > to   be mlocked() since the audio-playing thread can't tolerate pagefaults
> > because   it runs in a low-latency cycle.
> > - munmap(ptr,len)
> 
> You can do 
> 	ptr=mmap(blah)
> 	mlock(ptr, ...)
> 	munmap(ptr, len)

will mlock() automatically cause all pages to be read from the file ?
(I don't think)

But since I already do an mlockall() at the beginnig, there is no need of
additional mlock()s

I had an additional idea: mmap()ing the data directly into the circular-buffer
and dirty the pages through writing one word for each page, it might be a
bit faster, because you avoid a memcpy() , but
the problem is that since the audio thread acesses this buffer, I have to
implement some communication to signal the disk thread to unmap the region,
or alternatively keep track of previously mapped regions in the disk thread,
which makes things more complicated.
I'm dealing with a data rate of about 5MB/sec (from disk), therefore the main
bottleneck is the disk not the memcpy() performance.
But I will try to mmap()/munmap() tracking and mmap() directly into the
circular-buffer, let's see if I can lower the CPU load of an additional 1-2%.
:-)

BTW, does anyone know the best method to measure the real % of CPU idle time,
since top is pretty unreliabe.

> 
> > Does anyone know if there it/will be a way to do unbuffered mmap()  ?
> > I think streaming apps would benefit quite a bit from this.
> 
> mmap requires buffering - you are sharing the page with the system page
> cache. If you mean you want an mgoaway() to go with munmap() thats what
> madvise() provides on some other systems but not Linux yet

This will be nice, similar to O_DIRECT performance,
I will use whatever comes first. 
:-) 

regards,
Benno.

Re: mmap() better than read() fro streaming, Was: Re: Streaming disk

[Andrea Arcangeli]

On Wed, 25 Aug 1999, Benno Senoner wrote:

>will mlock() automatically cause all pages to be read from the file ?

Yes.

>But since I already do an mlockall() at the beginnig, there is no need of
>additional mlock()s

Yes (if you use MCL_FUTURE).

Andrea

Re: mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and ma

[Benno Senoner]

On Wed, 25 Aug 1999, Andrea Arcangeli wrote:
> On Wed, 25 Aug 1999, Benno Senoner wrote:
> 
> >will mlock() automatically cause all pages to be read from the file ?
> 
> Yes.
> 
> >But since I already do an mlockall() at the beginnig, there is no need of
> >additional mlock()s
> 
> Yes (if you use MCL_FUTURE).
> 
> Andrea

Yes, I run with mlockall(MCL_CURRENT|MCL_FUTURE)
I can confirm this now:
I just removed my memcpy()s for testing purposes, and the app loads the
data nicely.
Pratically this means that if you enabled MCL_FUTURE,
mmap() will return only after all the mapped segment of the file resides in
memory.
Correct me please if I am wrong.

regards,
Benno.

Re: mmap() better than read() fro streaming, Was: Re: Streaming disk I/O kills file buffering and ma

[Stephen C. Tweedie]

Hi,

Benno Senoner writes:

 > Yes, I run with mlockall(MCL_CURRENT|MCL_FUTURE)
 > I can confirm this now:
 > I just removed my memcpy()s for testing purposes, and the app loads the
 > data nicely.

As it should be.  mlock/mlockall apply to _all_ pages which appear in
your process' address space, regardless of whether they are page cache
or local data pages.

 > Pratically this means that if you enabled MCL_FUTURE, mmap() will
 > return only after all the mapped segment of the file resides in
 > memory.  Correct me please if I am wrong.

Yes.  If MCL_FUTURE is used then all future mmap()s are automatically
marked as locked, and we do a make_pages_present() before returning
from mmap.  This is exactly what mlockall is designed for: memory
access should simply never page fault if you use MCL_CURRENT |
MCL_FUTURE.  This is even true for writable MAP_PRIVATE pages: if you
have full mlockall enabled, then the make_pages_present() will do a
copy-on-write page fault for you.

--Stephen