2.6.16-ck1

2.6.16-ck1

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 6101 bytes --]

These are patches designed to improve system responsiveness and interactivity. 
It is configurable to any workload but the default ck patch is aimed at the 
desktop and cks is available with more emphasis on serverspace.

Apply to 2.6.16
http://ck.kolivas.org/patches/2.6/2.6.16/2.6.16-ck1/patch-2.6.16-ck1.bz2

or server version
http://ck.kolivas.org/patches/cks/patch-2.6.16-cks1.bz2

web:
http://kernel.kolivas.org

all patches:
http://ck.kolivas.org/patches/

Split patches available.

* Take note, userspace features have changed in this version *

There are 3 special unprivileged (normal user) scheduling policies available 
as of 2.6.16-ck1. These can be set using the schedtools utility 
(http://freshmeat.net/projects/schedtool/?topic_id=136) which has support for 
all of them. Additionally my toolsched scripts work as transparent wrappers 
for them (http://ck.kolivas.org/apps/toolsched/). Note the policies are 
different to earlier -ck versions:

SCHED_BATCH:
	This is for tasks you explicitly want the cpu scheduler to know are never 
interactive and thus should never receive low latency treatment. Their cpu 
usage is dependant on their nice value. This policy is also supported by 
mainline now which is why there is a change of the naming/numbering scheme.

SCHED_ISO:
	This is for tasks you explicitly want the cpu scheduler to know are low 
latency real-time like tasks but you don't have root privileges for and don't 
wish them to ever starve the machine. They can use up to 80% of the available 
cpu time (on one cpu at any time). This percentage is configurable 
via /proc/sys/kernel/iso_cpu

SCHED_IDLEPRIO:
	This is for tasks you never want to use cpu if *anything* else wants cpu 
time. That is they only ever use spare cpu cycles that would have otherwise 
been idle time on the machine.


Changes since 2.6.15-ck7

Added:
 +sched-implement-smpnice.patch
 +sched-smpnice-apply-review-suggestions.patch
 +sched-smpnice-fix-average-load-per-run-queue-calculations.patch
New improved smpnice implementation

 +sched-store-weighted-load-on-up.patch
 +sched-add-discrete-weighted-cpu-load-function.patch
 +sched-add-above-background-load-function.patch
Updates to smpnice above to be used by swap prefetch

 +sched-generic_optims2.patch
Small naming cleanups / microoptimisations

 +sched-idleprio-1.2.patch
The old SCHED_BATCH policy from 2.6.15-ck7 has been renamed to SCHED_IDLEPRIO 
and small updates to the policy were committed.

 +adaptive-readahead-11.patch
Wu Fengguang's adaptive readahead for improved read throughput without 
thrashing. This version is configurable at build time. As some users have 
reported latency issues with running this code, the config option is most 
welcome, however those latency issues should have been mostly addressed. See 
Documentation/sysctl/vm.txt for more info.


Removed:
 -vmsplit-config_options.patch
This is part of mainline 2.6.16 now

 -2.6.15-dynticks-060227.patch
 -dynticks-disable_smp_config.patch
Dynticks is dead! Long live dynticks! Something killed it off in 2.6.16-rc6 
and I haven't figured out what it is. I don't know if/when I'll be able to 
debug this so for the moment it is dropped.

-patch-2.6.15.6.bz2
Mainline


Modified:
 -sched-staircase13.2.patch
 -sched-staircase13.2_13.3.patch
 -sched-staircase13.3_13.4.patch
 -sched-staircase13.4_13.5.patch
 +sched-staircase14.2.patch
Rolled up and updated to latest version of staircase cpu scheduler. Mostly 
microoptimisations, and intrinsic support for new style of SCHED_BATCH 
policy.

 -schedrange-1.diff
 +schedrange-2.diff
Resync with change in scheduler policies available

 -schedbatch-2.11.diff
Replaced with new SCHED_IDLEPRIO policy

 -sched-iso3.3.patch
 +sched-iso-4.1.patch
Updated SCHED_ISO unprivileged soft real time policy. This now is higher 
priority than any SCHED_NORMAL tasks but lower priority than any true real 
time tasks. The default cpu limit for iso tasks was changed to 80% (Note this 
is set to 0 by default on the cks patchset so server admins would explicitly 
set this only if desired).

 -isobatch_ionice2.diff
 +iso_idleprio_ionice.patch
 -vm-mapped.diff
 +vm-mapped-1.diff
 -vm-background_scan-1.diff
 -mm-highmem_fix_background_scan.patch
 +mm-background_scan.patch
 -mm-prio_dependant_scan.patch
 -mm-batch_prio.patch
 +mm-prio_dependant_scan-1.patch
 +mm-idleprio_prio.patch
Rename/resync/rollups

 -mm-swap_prefetch-28.patch
 -mm-swap_prefetch-tweaks.patch
 +mm-swap_prefetch-30.patch
 +sp-resume1.patch
 +mm-aggresive_swap_prefetch.patch
 +swsusp-post_resume_aggressive_swap_prefetch.patch
Updated to the latest swap prefetch code which will perform swap prefetching 
if low priority tasks are running. This also adds the optional "use once" 
feature of aggressive swap prefetching, and makes swsusp use that setting on 
resume from disk which improves dramatically the immediate interactivity of a 
machine just after resume. See Documentation/sysctl/vm.txt for more info on 
the tunable

 -2615ck7-version.patch
 +2.6.16-ck1-version.patch
Version update


Full patchlist:

sched-implement-smpnice.patch
sched-smpnice-apply-review-suggestions.patch
sched-smpnice-fix-average-load-per-run-queue-calculations.patch
sched-store-weighted-load-on-up.patch
sched-add-discrete-weighted-cpu-load-function.patch
sched-add-above-background-load-function.patch
sched-staircase14.2.patch
sched-generic_optims2.patch
schedrange-2.diff
sched-iso-4.1.patch
sched-idleprio-1.2.patch
defaultcfq.diff
iso_idleprio_ionice.patch
rt_ionice.diff
pdflush-tweaks.patch
hz-default_values.patch
hz-no_default_250.patch
mm-swap_prefetch-30.patch
vm-mapped-1.diff
vm-lots_watermark.diff
mm-background_scan.patch
mm-kswapd_inherit_prio-1.patch
mm-prio_dependant_scan-1.patch
mm-idleprio_prio.patch
sp-resume1.patch
mm-aggresive_swap_prefetch.patch
swsusp-post_resume_aggressive_swap_prefetch.patch
adaptive-readahead-11.patch
2.6.16-ck1-version.patch


Cheers,
Con

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] 2.6.16-ck1

[Hugo Vanwoerkom]

Avuton Olrich wrote:
> On 3/20/06, Con Kolivas <kernel@kolivas.org> wrote:
>   
>> SCHED_IDLEPRIO:
>>         This is for tasks you never want to use cpu if *anything* else wants cpu
>> time. That is they only ever use spare cpu cycles that would have otherwise
>> been idle time on the machine.
>>     
>
> Sounds like something I've wanted for a while, so I could compile
> stuff on the wife's computer, but only when she's not actively using
> it.
> --
> avuton
> --
>   

Or rather: so I could compile stuff on *this* computer while both my 
wife and I are on their separate monitors/kbds, when neither one of us 
is actively using it. ;-)

Hugo







>  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> http://ck.kolivas.org/faqs/replying-to-mailing-list.txt
> ck mailing list - mailto: ck@vds.kolivas.org
> http://vds.kolivas.org/mailman/listinfo/ck


-- 
I think the death is much overrated. 
How could anything so ordinary become a penalty?
John J. Cox

Re: [ck] 2.6.16-ck1

[Rodney Gordon II]

Good job Con, on your patches.. As far as the kernel in general, I'd
like to post some warnings:

Adaptive readahead: I had probs with this before, and I still do.. On
a desktop if you have odd problems (nothing responding for SECONDS,
very slow disk I/O during heavy I/O, etc..) disable it.

The new Yukon2 "sky2" driver: This one really pissed me off. It had me
thinking apache2 AND my linksys router we're on the brink. For some
unknown reason at least for me, in FF it would only half-load some
pages, including ones on localhost AND my router (10.1.1.1) ... I
dunno what the hell is up with this one. I have to stay with the
syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am
back to 2.6.15 at the moment.

nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here
is a current patch that works against the current release:
http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1

All in all, my experience sucked for the first time on this kernel.

Good luck with this new one..
-r

-- 
Rodney "meff" Gordon II               -*-              meff@pobox.com
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource

Re: [ck] 2.6.16-ck1

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1678 bytes --]

Rodney Gordon II writes:

> Good job Con, on your patches.. As far as the kernel in general, I'd
> like to post some warnings:

Thanks.

> Adaptive readahead: I had probs with this before, and I still do.. On
> a desktop if you have odd problems (nothing responding for SECONDS,
> very slow disk I/O during heavy I/O, etc..) disable it.

I was concerned about that myself which is why the only reason I included it 
was because it came in a configurable form where you could choose to enable 
it, and the default was off, and the config option even said suitable to 
_servers_, not desktops.

> The new Yukon2 "sky2" driver: This one really pissed me off. It had me
> thinking apache2 AND my linksys router we're on the brink. For some
> unknown reason at least for me, in FF it would only half-load some
> pages, including ones on localhost AND my router (10.1.1.1) ... I
> dunno what the hell is up with this one. I have to stay with the
> syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am
> back to 2.6.15 at the moment.
> 
> nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here
> is a current patch that works against the current release:
> http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1

Luckily none of these are my fault.

> All in all, my experience sucked for the first time on this kernel.

/me does the "not my fault" look. 

> Good luck with this new one..

Heh. No new one in the works just yet, but I'm actually not planning on 
changing anything. Turn adaptive readahead off, and you're left with out of 
kernel tree, or worse, binary driver problems. 

Cheers,
Con


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] 2.6.16-ck1

[Rodney Gordon II]

On Thu, Mar 23, 2006 at 10:40:17PM +1100, Con Kolivas wrote:
> Rodney Gordon II writes:
> 
> >Good job Con, on your patches.. As far as the kernel in general, I'd
> >like to post some warnings:
> 
> Thanks.
> 
> >Adaptive readahead: I had probs with this before, and I still do.. On
> >a desktop if you have odd problems (nothing responding for SECONDS,
> >very slow disk I/O during heavy I/O, etc..) disable it.
> 
> I was concerned about that myself which is why the only reason I included 
> it was because it came in a configurable form where you could choose to 
> enable it, and the default was off, and the config option even said 
> suitable to _servers_, not desktops.

Yea, just thought I would try it again sighting the improvements in
the ? docs in menuconfig, was just disappointed again.

> >The new Yukon2 "sky2" driver: This one really pissed me off. It had me
> >thinking apache2 AND my linksys router we're on the brink. For some
> >unknown reason at least for me, in FF it would only half-load some
> >pages, including ones on localhost AND my router (10.1.1.1) ... I
> >dunno what the hell is up with this one. I have to stay with the
> >syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am
> >back to 2.6.15 at the moment.
> >
> >nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here
> >is a current patch that works against the current release:
> >http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.8178-2.diff?bug=357992;msg=15;att=1
> 
> Luckily none of these are my fault.

Yep, just warnings in general.

> >All in all, my experience sucked for the first time on this kernel.
> 
> /me does the "not my fault" look. 

LOL, I know, just warnings in general like I have said.

> >Good luck with this new one..
> 
> Heh. No new one in the works just yet, but I'm actually not planning on 
> changing anything. Turn adaptive readahead off, and you're left with out of 
> kernel tree, or worse, binary driver problems. 

Turned it off, disk probs are gone.

Meant to say "Good luck with this new mainline one.."

Your patches are great.. Thus, why I started this post with "Good
job Con, on your patches.." ;)

-r


-- 
Rodney "meff" Gordon II               -*-              meff@pobox.com
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource

Re: [ck] 2.6.16-ck1

[Matt Heler]

Con,

I believe the author of that readahead patch stated that using the following 
below would help in desktop usage :: 

/sbin/blockdev --setra 256 /dev/XXX



On Thursday 23 March 2006 6:40 am, Con Kolivas wrote:
> Rodney Gordon II writes:
> > Good job Con, on your patches.. As far as the kernel in general, I'd
> > like to post some warnings:
>
> Thanks.
>
> > Adaptive readahead: I had probs with this before, and I still do.. On
> > a desktop if you have odd problems (nothing responding for SECONDS,
> > very slow disk I/O during heavy I/O, etc..) disable it.
>
> I was concerned about that myself which is why the only reason I included
> it was because it came in a configurable form where you could choose to
> enable it, and the default was off, and the config option even said
> suitable to _servers_, not desktops.
>
> > The new Yukon2 "sky2" driver: This one really pissed me off. It had me
> > thinking apache2 AND my linksys router we're on the brink. For some
> > unknown reason at least for me, in FF it would only half-load some
> > pages, including ones on localhost AND my router (10.1.1.1) ... I
> > dunno what the hell is up with this one. I have to stay with the
> > syskonnect.com sk98lin patch, which.. doesn't work with 2.6.16 so I am
> > back to 2.6.15 at the moment.
> >
> > nVidia drivers: Broken. I posted a ftbfs bug on the debian bts, here
> > is a current patch that works against the current release:
> > http://bugs.debian.org/cgi-bin/bugreport.cgi/nvidia-kernel-source_1.0.817
> >8-2.diff?bug=357992;msg=15;att=1
>
> Luckily none of these are my fault.
>
> > All in all, my experience sucked for the first time on this kernel.
>
> /me does the "not my fault" look.
>
> > Good luck with this new one..
>
> Heh. No new one in the works just yet, but I'm actually not planning on
> changing anything. Turn adaptive readahead off, and you're left with out of
> kernel tree, or worse, binary driver problems.
>
> Cheers,
> Con

Re: [ck] 2.6.16-ck1

[Wu Fengguang]

Hi Rodney,

Thanks for your testing :)

On Thu, Mar 23, 2006 at 05:31:18AM -0600, Rodney Gordon II wrote:
> Adaptive readahead: I had probs with this before, and I still do.. On
> a desktop if you have odd problems (nothing responding for SECONDS,
> very slow disk I/O during heavy I/O, etc..) disable it.

Your problem on I/O latency with ara can be tracked down with the help
of Ingo's latency tracing patch. It goes like this:

1) download
http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.16.tar.bz2
http://www.vanheusden.com/ara/adaptive-readahead-11-2.6.16-rc6.patch.gz
http://people.redhat.com/mingo/latency-tracing-patches/latency-tracing-v2.6.16.patch

2)
tar jxf linux-2.6.16.tar.bz2
gunzip adaptive-readahead-11-2.6.16-rc6.patch.gz
cd linux-2.6.16
patch -p1 < ../adaptive-readahead-11-2.6.16-rc6.patch
patch -p1 < ../latency-tracing-v2.6.16.patch

3) compile kernel with Adaptive readahead support
4) boot with the new kernel, and run
echo 0 > /proc/sys/kernel/preempt_max_latency
5) feel some latency problems
6) report the content of /proc/latency_trace

Thanks,
Wu

Re: [ck] 2.6.16-ck1

[Rodney Gordon II]

On Thu, Mar 23, 2006 at 09:14:39PM +0800, Wu Fengguang wrote:
> Hi Rodney,
> 
> Thanks for your testing :)
> 
> On Thu, Mar 23, 2006 at 05:31:18AM -0600, Rodney Gordon II wrote:
> > Adaptive readahead: I had probs with this before, and I still do.. On
> > a desktop if you have odd problems (nothing responding for SECONDS,
> > very slow disk I/O during heavy I/O, etc..) disable it.
> 
> Your problem on I/O latency with ara can be tracked down with the help
> of Ingo's latency tracing patch. It goes like this:
> 
> 1) download
> http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.16.tar.bz2
> http://www.vanheusden.com/ara/adaptive-readahead-11-2.6.16-rc6.patch.gz
> http://people.redhat.com/mingo/latency-tracing-patches/latency-tracing-v2.6.16.patch
> 
> 2)
> tar jxf linux-2.6.16.tar.bz2
> gunzip adaptive-readahead-11-2.6.16-rc6.patch.gz
> cd linux-2.6.16
> patch -p1 < ../adaptive-readahead-11-2.6.16-rc6.patch
> patch -p1 < ../latency-tracing-v2.6.16.patch
> 
> 3) compile kernel with Adaptive readahead support
> 4) boot with the new kernel, and run
> echo 0 > /proc/sys/kernel/preempt_max_latency
> 5) feel some latency problems
> 6) report the content of /proc/latency_trace
> 
> Thanks,
> Wu

Will try this as soon as I can get some net drivers working under
2.6.16 ...

-r
-- 
Rodney "meff" Gordon II               -*-              meff@pobox.com
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource

Re: [ck] 2.6.16-ck1

[Stephen Hassard]

On Thu, Mar 23, 2006 at 09:34:19PM +0100, Edouard Gomez wrote:
> Rodney Gordon II wrote:
> >The new Yukon2 "sky2" driver: This one really pissed me off. It had me
> >thinking apache2 AND my linksys router we're on the brink. For some
> I never had the chance to make the sk98lin one working on my box, so I 
> backported sky2 to 2.6.15 quite a few times from netdev-2.6 git tree. 

It might be worth replacing the sky2 driver with the newest bleeding 
edge one over here:

http://developer.osdl.org/shemminger/prototypes/sky2-1.0msi.tar.gz

You might also want to check the developer's blog entry:
http://developer.osdl.org/shemminger/blog/?p=25

---
sky2 1.0?

Looks like a found the root cause of the sky2 hangs on pci-express. I 
copied some code from the SysKonnect driver that reconfigured the 
pci-express max request size. This probably caused receive dma engine to 
fail in face of contention. That will teach me to stop copy/pasting in 
bugs.
---

It might fix some of you issues ..

later,
Steve Hassard

Re: 2.6.16-ck1

[Edouard Gomez]

Stephen Hassard wrote:
> sky2 1.0?
> 
> Looks like a found the root cause of the sky2 hangs on pci-express. I 
> copied some code from the SysKonnect driver that reconfigured the 
> pci-express max request size. This probably caused receive dma engine to 
> fail in face of contention. That will teach me to stop copy/pasting in 
> bugs.
> ---
> 
> It might fix some of you issues ..

I'll backport again from git. The patch i sent was equivalent to 2.6.16 
sky2 module, but i see new patches that came in netdev-2.6.

[PATCH] sky2: more ethtool stats
[PATCH] sky2 version 1.1
[PATCH] sky2: handle all error irqs
[PATCH] sky2: transmit recovery
[PATCH] sky2: whitespace fixes
[PATCH] sky2: add MSI support
[PATCH] sky2: coalescing parameters
[PATCH] sky2: rework of NAPI and IRQ management
[PATCH] sky2: drop broken wake on lan support
[PATCH] sky2: remove support for untested Yukon EC ...
sky2: truncate oversize packets
sky2: force early transmit interruptsdiff to current
sky2: not random enough

-- 
Edouard Gomez

Re: [ck] Re: 2.6.16-ck1

[Rodney Gordon II]

On Fri, Mar 24, 2006 at 07:58:03AM +0100, Edouard Gomez wrote:
> I'll backport again from git. The patch i sent was equivalent to 2.6.16 
> sky2 module, but i see new patches that came in netdev-2.6.
> 
> [PATCH] sky2: more ethtool stats
> [PATCH] sky2 version 1.1
> [PATCH] sky2: handle all error irqs
> [PATCH] sky2: transmit recovery
> [PATCH] sky2: whitespace fixes
> [PATCH] sky2: add MSI support
> [PATCH] sky2: coalescing parameters
> [PATCH] sky2: rework of NAPI and IRQ management
> [PATCH] sky2: drop broken wake on lan support
> [PATCH] sky2: remove support for untested Yukon EC ...
> sky2: truncate oversize packets
> sky2: force early transmit interruptsdiff to current
> sky2: not random enough

For those of us that just want something working now:
Patch up using the latest sk98lin patch generator.. then edit
drivers/net/sk98lin/sky2.c and add this line:

#include "h/skdrv1st.h"
#include "h/skdrv2nd.h"
#include <linux/tcp.h>
#include <linux/ip.h>               <-- ADD THIS

It will build fine and seems to work just fine until syskonnect
releases a new revision

GL

-r

-- 
Rodney "meff" Gordon II               -*-              meff@pobox.com
Systems Administrator / Coder Geek    -*-       Open yourself to OpenSource