Standalone swap prefetch patch for mainline

Standalone swap prefetch patch for mainline

[Con Kolivas]

I've had a few requests for a standalone patch implementing swap prefetch for 
mainline.

Here is a patch that is a current rollup that should apply and work for 
vanilla 2.6.21 (ie not a -ck kernel):

http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch

-- 
-ck

Swap prefetch tester

[Con Kolivas]

On Monday 14 May 2007 12:10, Con Kolivas wrote:
> I've had a few requests for a standalone patch implementing swap prefetch
> for mainline.
>
> Here is a patch that is a current rollup that should apply and work for
> vanilla 2.6.21 (ie not a -ck kernel):
>
> http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch

Oh and here is the swap prefetch tester:
http://ck.kolivas.org/patches/swap-prefetch/sp_tester.c

-- 
-ck

Re: [ck] Swap prefetch tester

[Kacper Wysocki]

On 5/14/07, Con Kolivas <kernel@kolivas.org> wrote:
> On Monday 14 May 2007 12:10, Con Kolivas wrote:
> > I've had a few requests for a standalone patch implementing swap prefetch
> > for mainline.
> >
> > Here is a patch that is a current rollup that should apply and work for
> > vanilla 2.6.21 (ie not a -ck kernel):
> >
> > http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch

> Oh and here is the swap prefetch tester:
> http://ck.kolivas.org/patches/swap-prefetch/sp_tester.c

Some results on my 5-year-old dual AMD2400+ with 1GB mem:
***This one done while in X, 2.6.21.1-reiser4-ck1, sp ON
Ram 1034228000  Swap 1959920000
Total ram to be malloced: 1551342000 bytes
Starting first malloc of 775671000 bytes
Starting 1st read of first malloc
Touching this much ram takes 6333 milliseconds
Starting second malloc of 775671000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 103277 milliseconds

***This one done while in X, 2.6.21.1-reiser4-ck-sp38, sp ON
Ram 1034228000  Swap 1959920000
Total ram to be malloced: 1551342000 bytes
Starting first malloc of 775671000 bytes
Starting 1st read of first malloc
Touching this much ram takes 11674 milliseconds
Starting second malloc of 775671000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 24660 milliseconds

...a nice order of magnitude better! Very happy you fixed this Con.
But the time to touch was so different (11674 vs 6333ms) that I
decided to get out of X, and also check the difference when SP is off.
# ./sp_tester; echo 0 > /proc/sys/vm/swap_prefetch ; ./sp_tester


***This one done in the console at runlevel 2, 2.6.21.1-reiser4-ck-sp38, sp ON
Ram 1034228000  Swap 1959920000
Total ram to be malloced: 1551342000 bytes
Starting first malloc of 775671000 bytes
Starting 1st read of first malloc
Touching this much ram takes 5709 milliseconds
Starting second malloc of 775671000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 22243 milliseconds

***This one done in the console at runlevel 2, 2.6.21.1-reiser4-ck-sp38, sp OFF
Ram 1034228000  Swap 1959920000
Total ram to be malloced: 1551342000 bytes
Starting first malloc of 775671000 bytes
Starting 1st read of first malloc
Touching this much ram takes 5816 milliseconds
Starting second malloc of 775671000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 28641 milliseconds

Have I understood correctly that there are more prefetch improvements
in ck2 than just the sp38 patch?

I'll post more results if I ever manage to tear myself from the
computer for 10 minutes when I'm home - it looks like I have a yiff
daemon running at 7% cpu all the time which might have affected the
results.

Speaking of "feel", the net effect of swap prefetch for me seems to be
that I don't notice it - that is, I don't notice my programs behaving
sluggishly when I come back to the computer. Excellent!

-K

Re: [ck] Swap prefetch tester

[Con Kolivas]

On Wednesday 16 May 2007 06:04, Kacper Wysocki wrote:
> ...a nice order of magnitude better! Very happy you fixed this Con.

Great!, thanks for testing.

> Have I understood correctly that there are more prefetch improvements
> in ck2 than just the sp38 patch?

No; 2.6.21-ck2 simply brings it up to sp38. The code in -ck for swap prefetch 
is a little different to suit the SD scheduler and the SCHED_IDLEPRIO patch.
>
> I'll post more results if I ever manage to tear myself from the
> computer for 10 minutes when I'm home - it looks like I have a yiff
> daemon running at 7% cpu all the time which might have affected the
> results.
>
> Speaking of "feel", the net effect of swap prefetch for me seems to be
> that I don't notice it - that is, I don't notice my programs behaving
> sluggishly when I come back to the computer. Excellent!

Excellent!. Thanks very much for your feedback.

-- 
-ck

Re: Standalone swap prefetch patch for mainline

[Brad Campbell]

Con Kolivas wrote:
> I've had a few requests for a standalone patch implementing swap prefetch for 
> mainline.
> 
> Here is a patch that is a current rollup that should apply and work for 
> vanilla 2.6.21 (ie not a -ck kernel):
> 
> http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch
> 

Heh.. 20 mins after I finished scraping the relevant bits from -ck1 and patching my kernel this arrives!

<WARNING - Fanboy rant ahead>

I don't normally get/take the time to pass personal comment, but I have to say Con I've just taken a 
2.6.21 kernel, applied your 0.48 SD scheduler and swap-prefetch from -ck1 (needed to separate them 
out so I could also apply suspend2) and it's like I've bought a new machine.

I'm running on a tiny Vaio with a 4800RPM 1.8" drive, so it's not fast by any stretch of the 
imagination. But the combination of SD and swap-prefetch has given it a new lease on life completely.

Really, you have done a great job with SD. I was so impressed I built a new kernel for my Core Duo 
Mac Mini with the full 2.6.21-ck1 patchset. I wish I'd tried it years ago!

This also prompted me to have a play with CFS, but for my desktop workload on either machine, SD 
just "felt" better (for whatever that may be worth).

+1 for SD
+1 for swap-prefetch

Regards,
Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: Standalone swap prefetch patch for mainline

[Paolo Ciarrocchi]

On 5/15/07, Brad Campbell <brad@wasp.net.au> wrote:
> Con Kolivas wrote:
> > I've had a few requests for a standalone patch implementing swap prefetch for
> > mainline.
> >
> > Here is a patch that is a current rollup that should apply and work for
> > vanilla 2.6.21 (ie not a -ck kernel):
> >
> > http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch
> >
>
> Heh.. 20 mins after I finished scraping the relevant bits from -ck1 and patching my kernel this arrives!
>
> <WARNING - Fanboy rant ahead>
>
> I don't normally get/take the time to pass personal comment, but I have to say Con I've just taken a
> 2.6.21 kernel, applied your 0.48 SD scheduler and swap-prefetch from -ck1 (needed to separate them
> out so I could also apply suspend2) and it's like I've bought a new machine.
>
> I'm running on a tiny Vaio with a 4800RPM 1.8" drive, so it's not fast by any stretch of the
> imagination. But the combination of SD and swap-prefetch has given it a new lease on life completely.
>
> Really, you have done a great job with SD. I was so impressed I built a new kernel for my Core Duo
> Mac Mini with the full 2.6.21-ck1 patchset. I wish I'd tried it years ago!
>
> This also prompted me to have a play with CFS, but for my desktop workload on either machine, SD
> just "felt" better (for whatever that may be worth).

No doubt about the goodnes of the SD scheduler, Con did an amazing
work dnd even if we don't know whether SD will be merged in mailine
(CFS seems to be the best candidate) he demonstrated that there is
plenty of room for improvment in that area.

Brad, is it possible for you to do some more test with the latest
version of both SD and CFS  and post some more detailed feedbacks?
That would help a lot.

> +1 for SD
> +1 for swap-prefetch
>

Regards,

-- 
Paolo

Re: Standalone swap prefetch patch for mainline

[Brad Campbell]

Paolo Ciarrocchi wrote:

> Brad, is it possible for you to do some more test with the latest
> version of both SD and CFS  and post some more detailed feedbacks?
> That would help a lot.

Err.. Ok. I have the latest version of SD (that I know about). I could upgrade CFS, but unlike those 
doing real scientific testing, I'm just getting a feel with my usual usage patterns (thunderbird, 
firefox, openoffice, gimp, x-chat and gnome). I don't have any hard fast metrics I can measure 
against really.

Anything I post would be subjective at best. (And to be honest, once I get a stable and usable 
kernel I tend to stick with it for a month or so as I don't reboot - suspend2 regularly goes 100 
suspend/resume cycles to S3 between reboots for me)

I'm a little more willing to experiment on my desktop box, but I really live from the laptop.

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: Standalone swap prefetch patch for mainline

[Paolo Ciarrocchi]

On 5/15/07, Brad Campbell <brad@wasp.net.au> wrote:
> Paolo Ciarrocchi wrote:
>
> > Brad, is it possible for you to do some more test with the latest
> > version of both SD and CFS  and post some more detailed feedbacks?
> > That would help a lot.
>
> Err.. Ok. I have the latest version of SD (that I know about). I could upgrade CFS, but unlike those
> doing real scientific testing, I'm just getting a feel with my usual usage patterns (thunderbird,
> firefox, openoffice, gimp, x-chat and gnome). I don't have any hard fast metrics I can measure
> against really.

I think that just some words about your feeling while you doing your
standard workload would be sufficient.
Latest CFS is here:
http://people.redhat.com/mingo/cfs-scheduler/
and according to your email you already have latest SD.

> Anything I post would be subjective at best. (And to be honest, once I get a stable and usable
> kernel I tend to stick with it for a month or so as I don't reboot - suspend2 regularly goes 100
> suspend/resume cycles to S3 between reboots for me)
>
> I'm a little more willing to experiment on my desktop box, but I really live from the laptop.

Many thanks in advance.

regards,
-- 
Paolo
"Tutto cio' che merita di essere fatto,merita di essere fatto bene"
Philip Stanhope IV conte di Chesterfield

Re: Standalone swap prefetch patch for mainline

[Con Kolivas]

On Wednesday 16 May 2007 00:45, Brad Campbell wrote:
> Con Kolivas wrote:
> > I've had a few requests for a standalone patch implementing swap prefetch
> > for mainline.
> >
> > Here is a patch that is a current rollup that should apply and work for
> > vanilla 2.6.21 (ie not a -ck kernel):
> >
> > http://ck.kolivas.org/patches/swap-prefetch/2.6.21-swap_prefetch-38.patch
>
> Heh.. 20 mins after I finished scraping the relevant bits from -ck1 and
> patching my kernel this arrives!
>
> <WARNING - Fanboy rant ahead>
>
> I don't normally get/take the time to pass personal comment, but I have to
> say Con I've just taken a 2.6.21 kernel, applied your 0.48 SD scheduler and
> swap-prefetch from -ck1 (needed to separate them out so I could also apply
> suspend2) and it's like I've bought a new machine.
>
> I'm running on a tiny Vaio with a 4800RPM 1.8" drive, so it's not fast by
> any stretch of the imagination. But the combination of SD and swap-prefetch
> has given it a new lease on life completely.
>
> Really, you have done a great job with SD. I was so impressed I built a new
> kernel for my Core Duo Mac Mini with the full 2.6.21-ck1 patchset. I wish
> I'd tried it years ago!
>
> This also prompted me to have a play with CFS, but for my desktop workload
> on either machine, SD just "felt" better (for whatever that may be worth).
>
> +1 for SD
> +1 for swap-prefetch

Great! Thanks for the feedback. I'm pleased that people don't just naturally 
assume CFS will be better and are still willing to test SD. I'm even more 
pleased that you're finding it better than CFS too :)

-- 
-ck