[RFC] kswapd: Kernel Swapper performance

[RFC] kswapd: Kernel Swapper performance

[Al Boldi]

One thing that has improved in 2.6, wrt 2.4, is swapper performance.  And the 
difference isn't small either: ~5 fold increase in swapin performance.

But swapin performance still lags swapout performance by 50%, which is a bit 
odd, considering swapin to be a read from disk, usually faster, and swapout 
to be a write to disk, usually slower.

Now, this slowdown could be explained by additional seek action, caused by 
different apps paging-in at different times with different locations on 
swap.

Yet, even a single app paging-out consecutive pages, and paging-in the same 
pages in one go, exhibits this upside-down swapout/swapin performance ratio.

Improving this ratio could possibly yield a dramatic improvement in system 
performance under memory load (think tmpfs/swsusp/...).


Thanks!

--
Al

Re: [RFC] kswapd: Kernel Swapper performance

[Rik van Riel]

Al Boldi wrote:
> One thing that has improved in 2.6, wrt 2.4, is swapper performance.  And the 
> difference isn't small either: ~5 fold increase in swapin performance.
> 
> But swapin performance still lags swapout performance by 50%, which is a bit 
> odd, considering swapin to be a read from disk, usually faster, and swapout 
> to be a write to disk, usually slower.

Ahhhhhh, but there's a catch...

You can queue up multiple writes, because the data you want
to write to disk is already in memory.

However, at swapin time you need to read the first bit of
data from disk, after which the program can continue, and
only when the next page fault happens you know what data
to read in next.

Linux does some swapin clustering, but there simply is no
way to know which data will be needed next.

This means reads are serialized and synchronous wrt. program
execution, while writes can overlap and be done asynchronously.

It's a miracle reads are going at 50% of the speed of writes...

> Improving this ratio could possibly yield a dramatic improvement in system 
> performance under memory load (think tmpfs/swsusp/...).

Let me know when you figure out how to look into the future.

Actually, Keir Fraser and Fay Chang came up with a cool trick.

    "Operating System I/O Speculation:
   How Two Invocations Are Faster Than One"

http://www.usenix.org/publications/library/proceedings/usenix03/tech/fraser.html

It is somewhat complex though...

-- 
Who do you trust?
The people with all the right answers?
Or the people with the right questions?

Re: [RFC] kswapd: Kernel Swapper performance

[Al Boldi]

Rik van Riel wrote:
> Al Boldi wrote:
> > One thing that has improved in 2.6, wrt 2.4, is swapper performance. 
> > And the difference isn't small either: ~5 fold increase in swapin
> > performance.
> >
> > But swapin performance still lags swapout performance by 50%, which is a
> > bit odd, considering swapin to be a read from disk, usually faster, and
> > swapout to be a write to disk, usually slower.
>
> Ahhhhhh, but there's a catch...
>
> You can queue up multiple writes, because the data you want
> to write to disk is already in memory.
>
> However, at swapin time you need to read the first bit of
> data from disk, after which the program can continue, and
> only when the next page fault happens you know what data
> to read in next.
>
> Linux does some swapin clustering, but there simply is no
> way to know which data will be needed next.
>
> This means reads are serialized and synchronous wrt. program
> execution, while writes can overlap and be done asynchronously.
>
> It's a miracle reads are going at 50% of the speed of writes...
>
> > Improving this ratio could possibly yield a dramatic improvement in
> > system performance under memory load (think tmpfs/swsusp/...).
>
> Let me know when you figure out how to look into the future.
>
> Actually, Keir Fraser and Fay Chang came up with a cool trick.
>
>     "Operating System I/O Speculation:
>    How Two Invocations Are Faster Than One"
>
> http://www.usenix.org/publications/library/proceedings/usenix03/tech/frase
>r.html
>
> It is somewhat complex though...

Thanks for the link, but I was more thinking about improving consecutive 
swapin rather than random swapin.

Right now, consecutive swapin looks suspiciously slow, and should be at least 
as fast as swapout, if not faster.


Thanks!

--
Al