2.6.7-ck1

2.6.7-ck1

[Con Kolivas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Patchset update. The focus of this patchset is on system responsiveness with
emphasis on desktops, but the scope of scheduler changes now makes this patch 
suitable to servers as well.

http://kernel.kolivas.org

Includes
Staircase scheduler v7.0 (unchanged resync of v6.E)
Extra scheduling policies for staircase:
 Isochronous scheduling - low latency non-privileged (ie soft real time)
 Batch scheduling - idle scheduling
Autoregulated VM swappiness
Supermount-NG v2.04

Changes:
Staircase scheduler and all policies for scheduler updated
Autoregulated swappiness updated to bias exponentially much more like a real 
world system.

Removed:
Sched domains merged into mainline
Hyperthread nice merged into mainline
CFQ I/O elevator merged into mainline
CDDMA merged into mainline
Bootsplash - dropped; patch getting too old to safely merge
Reiser4 - dropped; patch too old
Grsecurity option - dropped; unknown future and breaks SMP
IO Prio - not part of current CFQ scheduler; await new version


Sysctl options:
Staircase scheduler can be set for different workloads:

echo 0 > /proc/sys/kernel/interactive
will disable interactive tasks from having bursts, thus being even stricter 
about nice levels (suitable for non gui desktop usage or a server)

echo 1 > /proc/sys/kernel/compute
makes round robin intervals much longer, delays task preemption and disables
interactive mode to optimise cpu cache usage - suitable for computational
intensive tasks.

Autoregulated swappiness:
echo 0 > /proc/sys/vm/autoswappiness
disables the autoregulated swappiness thus allowing you to set
swappiness manually as exists in mainline

Using schedtool ( http://freshmeat.net/projects/schedtool/?topic_id=136 ) you 
can enable the different scheduling policies:

schedtool -I -e xmms
or
schedtool -I $pid

will start xmms as an "Isochronous" scheduled task. Alternatively if a task
tries to get real time scheduling but you do not have priviliges for RT 
it will drop the task to SCHED_ISO.

schedtool -B -e setiathome

will start setiathome as a "Batch" scheduled task. This means it will 
not get cpu time unless the cpu is completely idle. Note this is _not_ 
suitable for tasks that are disk intensive as a held semaphore can lead to a 
DoS.

Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFA0C1+ZUg7+tp6mRURAsDmAJ9bgPhhe8On4UeqEhD6FNcvPGqnWACeJG9C
yNJaCuLgi4AMT7gTPA+CHjw=
=DBuT
-----END PGP SIGNATURE-----

Re: 2.6.7-ck1

[Felipe Alfaro Solana]

On Wed, 2004-06-16 at 21:22 +1000, Con Kolivas wrote: 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Patchset update. The focus of this patchset is on system responsiveness with
> emphasis on desktops, but the scope of scheduler changes now makes this patch 
> suitable to servers as well.

I've found some interaction problems between, what I think it's, the
staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
takes more than 1 minute to save the same amount of pages when
suspending to disk.

Re: 2.6.7-ck1

[Con Kolivas]

On Sat, 19 Jun 2004 02:28, Felipe Alfaro Solana wrote:
> On Wed, 2004-06-16 at 21:22 +1000, Con Kolivas wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Patchset update. The focus of this patchset is on system responsiveness
> > with emphasis on desktops, but the scope of scheduler changes now makes
> > this patch suitable to servers as well.
>
> I've found some interaction problems between, what I think it's, the
> staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
> save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
> takes more than 1 minute to save the same amount of pages when
> suspending to disk.

If you're using -ck1 it may even be the autoswappiness. Try disabling that and 
setting a static value for swappiness. If it still exhibits the problem then 
it's probably a bug somewhere in staircase. While the overall design is 
finished (it doesn't really lend itself to tuning), surely there are bugs I 
haven't sorted out even though there are no serious bugs or stability issues 
that have come up. I'm auditing the code as we speak.

Thanks.
Con

Re: 2.6.7-ck1

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1274 bytes --]

Con Kolivas wrote:
> On Sat, 19 Jun 2004 02:28, Felipe Alfaro Solana wrote:
> 
>>On Wed, 2004-06-16 at 21:22 +1000, Con Kolivas wrote:
>>
>>>-----BEGIN PGP SIGNED MESSAGE-----
>>>Hash: SHA1
>>>
>>>Patchset update. The focus of this patchset is on system responsiveness
>>>with emphasis on desktops, but the scope of scheduler changes now makes
>>>this patch suitable to servers as well.
>>
>>I've found some interaction problems between, what I think it's, the
>>staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
>>save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
>>takes more than 1 minute to save the same amount of pages when
>>suspending to disk.
> 
> 
> If you're using -ck1 it may even be the autoswappiness. Try disabling that and 
> setting a static value for swappiness. If it still exhibits the problem then 
> it's probably a bug somewhere in staircase. While the overall design is 
> finished (it doesn't really lend itself to tuning), surely there are bugs I 
> haven't sorted out even though there are no serious bugs or stability issues 
> that have come up. I'm auditing the code as we speak.

You might want to try the attached patch which addresses an issue with 
kernel threads that is going into staircase 7.1

Con

[-- Attachment #2: from-2.6.7-ck1_to_staircase7.1 --]
[-- Type: text/x-troff-man, Size: 426 bytes --]

--- linux-2.6.7-ck2pre/kernel/sched.c	2004-06-19 15:12:15.280924354 +1000
+++ linux-2.6.7-ck1/kernel/sched.c	2004-06-19 14:58:08.000000000 +1000
@@ -334,8 +334,7 @@ static int effective_prio(task_t *p)
 
 	if (used_slice < first_slice)
 		return prio;
-	if (p->mm)
-		prio += 1 + (used_slice - first_slice) / rr;
+	prio += 1 + (used_slice - first_slice) / rr;
 	if (prio > MAX_PRIO - 2)
 		prio = MAX_PRIO - 2;
 	return prio;

Re: 2.6.7-ck1

[Felipe Alfaro Solana]

On Sat, 2004-06-19 at 15:20 +1000, Con Kolivas wrote:
> Con Kolivas wrote:
> >>I've found some interaction problems between, what I think it's, the
> >>staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
> >>save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
> >>takes more than 1 minute to save the same amount of pages when
> >>suspending to disk.
> 
> You might want to try the attached patch which addresses an issue with 
> kernel threads that is going into staircase 7.1

This patch completely fixes my problems with SWSUSP.
Nice job!

Re: 2.6.7-ck1

[Con Kolivas]

On Sat, 19 Jun 2004 04:35, Grzegorz Kulewski wrote:
> Hi Con,
>
> I have two problems with 2.6.7-ck1. My distribution is Gentoo Linux
> unstable with all latest updates. Oh, yes, both 2.6.7-ck1 and 2.6.7-rc3
> I tested have vesafb-tng applied from http://dev.gentoo.org/~spock/, but
> it should not cause any problems because it is very non-intrusive patch I
> think. Maybe you should include this in your patchset?
>
> 1. When booting init script freezes after starting input hotplugging (it
> is udev system). The only way to make it run is to press Ctrl-Alt-SysRQ
> and various keys to display kernel state several times. After that system
> starts normally. I do not know if it is only -ck problem because I had
> no time to test 2.6.7 vanilla, but 2.6.7-rc3 worked fine. (Log included.)

Yes I have a sneaking suspicion it's related to the fact kernel threads are 
fixed priority at the moment in staircase (they dont descend priority like 
normal tasks so act like relatively low priority real time tasks). I'm 
addressing that for the next version so hopefully that will fix it.

> 2. In Gentoo we are often using rsync to sync with newest ebuilds database
> (called Portage). Whole portage is something about 300 MB I think. But the
> part to upgrade is only about 1 MB at most. When I try to rsync the only
> process running on my system is kswapd and all my RAM is used (I have 512
> MB of RAM and 768 MB of swap). System uses 90 MB of swap and top shows
> that only 20 MB are cached. This causes rsync server to terminate
> connection because of timeout. There is nothing big running on my box
> besides GNOME and rsync at this time. Here are some /proc files:

Sounds like you need to disable the autoswappiness for this load and set a 
static value. In fact you may find a very high value useful in speeding this 
load up. The older "linear" design for autoswappiness may have been a more 
correct way to tackle it, although desktop users have enjoyed swapping even 
less using the -as patch. To recreate the older design edit mm/vmscan.c and 
comment out line 724 like so:

-			vm_swappiness = vm_swappiness * vm_swappiness / 100;
+			// vm_swappiness = vm_swappiness * vm_swappiness / 100;

Con

Re: 2.6.7-ck1

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]

Con Kolivas wrote:
> On Sat, 19 Jun 2004 04:35, Grzegorz Kulewski wrote:
> 
>>Hi Con,
>>
>>I have two problems with 2.6.7-ck1. My distribution is Gentoo Linux
>>unstable with all latest updates. Oh, yes, both 2.6.7-ck1 and 2.6.7-rc3
>>I tested have vesafb-tng applied from http://dev.gentoo.org/~spock/, but
>>it should not cause any problems because it is very non-intrusive patch I
>>think. Maybe you should include this in your patchset?
>>
>>1. When booting init script freezes after starting input hotplugging (it
>>is udev system). The only way to make it run is to press Ctrl-Alt-SysRQ
>>and various keys to display kernel state several times. After that system
>>starts normally. I do not know if it is only -ck problem because I had
>>no time to test 2.6.7 vanilla, but 2.6.7-rc3 worked fine. (Log included.)
> 
> 
> Yes I have a sneaking suspicion it's related to the fact kernel threads are 
> fixed priority at the moment in staircase (they dont descend priority like 
> normal tasks so act like relatively low priority real time tasks). I'm 
> addressing that for the next version so hopefully that will fix it.

Here's a diff for -ck1 which brings you up to staircase7.1
Can you try that?

Con

[-- Attachment #2: from-2.6.7-ck1_to_staircase7.1 --]
[-- Type: text/x-troff-man, Size: 426 bytes --]

--- linux-2.6.7-ck2pre/kernel/sched.c	2004-06-19 15:12:15.280924354 +1000
+++ linux-2.6.7-ck1/kernel/sched.c	2004-06-19 14:58:08.000000000 +1000
@@ -334,8 +334,7 @@ static int effective_prio(task_t *p)
 
 	if (used_slice < first_slice)
 		return prio;
-	if (p->mm)
-		prio += 1 + (used_slice - first_slice) / rr;
+	prio += 1 + (used_slice - first_slice) / rr;
 	if (prio > MAX_PRIO - 2)
 		prio = MAX_PRIO - 2;
 	return prio;

Re: 2.6.7-ck1

[Grzegorz Kulewski]

On Sat, 19 Jun 2004, Con Kolivas wrote:

> Con Kolivas wrote:
> > On Sat, 19 Jun 2004 04:35, Grzegorz Kulewski wrote:
> > 
> >>Hi Con,
> >>
> >>I have two problems with 2.6.7-ck1. My distribution is Gentoo Linux
> >>unstable with all latest updates. Oh, yes, both 2.6.7-ck1 and 2.6.7-rc3
> >>I tested have vesafb-tng applied from http://dev.gentoo.org/~spock/, but
> >>it should not cause any problems because it is very non-intrusive patch I
> >>think. Maybe you should include this in your patchset?
> >>
> >>1. When booting init script freezes after starting input hotplugging (it
> >>is udev system). The only way to make it run is to press Ctrl-Alt-SysRQ
> >>and various keys to display kernel state several times. After that system
> >>starts normally. I do not know if it is only -ck problem because I had
> >>no time to test 2.6.7 vanilla, but 2.6.7-rc3 worked fine. (Log included.)
> > 
> > 
> > Yes I have a sneaking suspicion it's related to the fact kernel threads are 
> > fixed priority at the moment in staircase (they dont descend priority like 
> > normal tasks so act like relatively low priority real time tasks). I'm 
> > addressing that for the next version so hopefully that will fix it.
> 
> Here's a diff for -ck1 which brings you up to staircase7.1
> Can you try that?

It does not solve the problem for me, sorry...


Thanks,

Grzegorz Kulewski

Re: 2.6.7-ck1

[Con Kolivas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Grzegorz Kulewski wrote:
|
| On Sat, 19 Jun 2004, Con Kolivas wrote:
|
|
|>Con Kolivas wrote:
|>
|>>On Sat, 19 Jun 2004 04:35, Grzegorz Kulewski wrote:
|>>
|>>
|>>>Hi Con,
|>>>
|>>>I have two problems with 2.6.7-ck1. My distribution is Gentoo Linux
|>>>unstable with all latest updates. Oh, yes, both 2.6.7-ck1 and 2.6.7-rc3
|>>>I tested have vesafb-tng applied from http://dev.gentoo.org/~spock/, but
|>>>it should not cause any problems because it is very non-intrusive
patch I
|>>>think. Maybe you should include this in your patchset?
|>>>
|>>>1. When booting init script freezes after starting input hotplugging (it
|>>>is udev system). The only way to make it run is to press Ctrl-Alt-SysRQ
|>>>and various keys to display kernel state several times. After that
system
|>>>starts normally. I do not know if it is only -ck problem because I had
|>>>no time to test 2.6.7 vanilla, but 2.6.7-rc3 worked fine. (Log
included.)
|>>
|>>
|>>Yes I have a sneaking suspicion it's related to the fact kernel
threads are
|>>fixed priority at the moment in staircase (they dont descend priority
like
|>>normal tasks so act like relatively low priority real time tasks). I'm
|>>addressing that for the next version so hopefully that will fix it.
|>
|>Here's a diff for -ck1 which brings you up to staircase7.1
|>Can you try that?
|
|
| It does not solve the problem for me, sorry...

Ok I'll keep looking. That change was necessary anyway.

Thanks,
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFA1GvtZUg7+tp6mRURAk4MAJ9xJ49Mwyv/Qlb5M4Dm8acsto5kRgCdEtIp
WE4lkMOxIh3HB+IET/oEs9c=
=4ayV
-----END PGP SIGNATURE-----

Re: 2.6.7-ck1

[Pavel Machek]

Hi!

> > Patchset update. The focus of this patchset is on system responsiveness with
> > emphasis on desktops, but the scope of scheduler changes now makes this patch 
> > suitable to servers as well.
> 
> I've found some interaction problems between, what I think it's, the
> staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
> save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
> takes more than 1 minute to save the same amount of pages when
> suspending to disk.

That's probably bug in io subsystem. That happened few times in suse
kernel. Missing unplug?
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: 2.6.7-ck1

[kernel]

Quoting Pavel Machek <pavel@ucw.cz>:

> Hi!
> 
> > > Patchset update. The focus of this patchset is on system responsiveness
> with
> > > emphasis on desktops, but the scope of scheduler changes now makes this
> patch 
> > > suitable to servers as well.
> > 
> > I've found some interaction problems between, what I think it's, the
> > staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
> > save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
> > takes more than 1 minute to save the same amount of pages when
> > suspending to disk.
> 
> That's probably bug in io subsystem. That happened few times in suse
> kernel. Missing unplug?

Thanks. Turned out to be a buggy yield() implementation on my part. Fixed in
later -ck versions.

Cheers,
Con

Re: 2.6.7-ck1

[Felipe Alfaro Solana]

On Tue, 2004-06-29 at 00:46 +0200, Pavel Machek wrote:

> > I've found some interaction problems between, what I think it's, the
> > staircase scheduler and swsusp. With vanilla 2.6.7, swsusp is able to
> > save ~9000 pages to disk in less than 5 seconds, where as 2.6.7-ck1
> > takes more than 1 minute to save the same amount of pages when
> > suspending to disk.
> 
> That's probably bug in io subsystem. That happened few times in suse
> kernel. Missing unplug?

It seems it was a bug in the yield() logic with the staircase scheduler,
as it has been fixed in -ck2 and -ck3. Surely, it was *not* a problemw
with swsusp.

Re: 2.6.7-ck1, cfq ionice?

[Prakash K. Cheemplavam]

So,

I have been using 2.6.7-ck1 for a few days now and must say it is simply 
*great*. Everything is working as it should, but only better. :-) Even 
ut2004 seems to be much smoother using staircase, some people reported 
15% more fps (I haven't measured), but it runs as smooth as ut2003 did 
previously with Nick's scheduler (before the O(1) scheduler was updated 
to its current state in mm).

The only thing left, which is a major pain for me, is disk i/o. Once it 
starts performance goes down, I think even more with staircase than with 
Nick's but this could be due to faster feel of staircase in general...

Example: When I do a emerge rsync in gentoo a tree consisting of nearly 
9000 files gets synced and a cache is built which causes a lot of random 
access on hd. So when I try to use thunderbird mailer at the same time, 
it act like a snail now due to concurrent disk access.

As I understood the cfq ionice part would solve this issue. I never 
tried it, as I think I never had a kernel containing it (and never had 
such a desperate need for it :-). Reading your changelog, it is not 
included anymore in ck1. So should I beg Jens Axboe for a rediff or new 
patch or how to get this piece inside? I think it is the only thing left 
for the next to perfect desktop experience I ever had.

Cheers,

Prakash

Re: 2.6.7-ck1, cfq ionice?

[Prakash K. Cheemplavam]

Prakash K. Cheemplavam wrote:

> Example: When I do a emerge rsync in gentoo a tree consisting of nearly 
> 9000 files gets synced and a cache is built which causes a lot of random 

It should be 90000 files, but well... ;-)

Prakash

Re: 2.6.7-ck1, cfq ionice?

[Con Kolivas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Prakash K. Cheemplavam wrote:
| So,
|
| I have been using 2.6.7-ck1 for a few days now and must say it is simply
| *great*. Everything is working as it should, but only better. :-) Even
| ut2004 seems to be much smoother using staircase, some people reported
| 15% more fps (I haven't measured), but it runs as smooth as ut2003 did
| previously with Nick's scheduler (before the O(1) scheduler was updated
| to its current state in mm).
|
| The only thing left, which is a major pain for me, is disk i/o. Once it
| starts performance goes down, I think even more with staircase than with
| Nick's but this could be due to faster feel of staircase in general...
|
| Example: When I do a emerge rsync in gentoo a tree consisting of nearly
| 9000 files gets synced and a cache is built which causes a lot of random
| access on hd. So when I try to use thunderbird mailer at the same time,
| it act like a snail now due to concurrent disk access.
|
| As I understood the cfq ionice part would solve this issue. I never
| tried it, as I think I never had a kernel containing it (and never had
| such a desperate need for it :-). Reading your changelog, it is not
| included anymore in ck1. So should I beg Jens Axboe for a rediff or new
| patch or how to get this piece inside? I think it is the only thing left
| for the next to perfect desktop experience I ever had.

Hi.

By default -ck1 uses the default I/O scheduler which is anticipatory.
You can set the cfq scheduler (as I recommend and write in my faq) by
adding the boot command:

elevator=cfq

This does not include ionice, though, as the version of ionice that Jens
published was getting quite old and only worked with a much slower
version of the cfq scheduler. I had to drop that version in favour of
the higher throughput cfq scheduler in mainline. See how you go with jus
the normal cfq elevator first.

Hopefully Jens will resync his ionice work with the current cfq
scheduler. As soon as he does I'll include it in -ck ;-)

Cheers,
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFA1MmqZUg7+tp6mRURAi4ZAJ47GL+H1NpgKkbzW/9aA8L85S1xpQCcCljF
ysonA/FE6GXn9BuVHqIEXCQ=
=BDBF
-----END PGP SIGNATURE-----

Re: 2.6.7-ck1, cfq ionice?

[Prakash K. Cheemplavam]

Con Kolivas wrote:
> Prakash K. Cheemplavam wrote:
> | The only thing left, which is a major pain for me, is disk i/o. Once it
> | starts performance goes down, I think even more with staircase than with
> | Nick's but this could be due to faster feel of staircase in general...
> |
> | As I understood the cfq ionice part would solve this issue. I never
> By default -ck1 uses the default I/O scheduler which is anticipatory.
> You can set the cfq scheduler (as I recommend and write in my faq) by
> adding the boot command:
> 
> elevator=cfq

I have forgotten to mention it, but I am already using cfq io scheduler...

> Hopefully Jens will resync his ionice work with the current cfq
> scheduler. As soon as he does I'll include it in -ck ;-)

Great. :-)

bye,

Praaksh

Re: 2.6.7-ck1

[Brian S. Stephan]

Con Kolivas wrote:

> Patchset update. The focus of this patchset is on system responsiveness
> with emphasis on desktops, but the scope of scheduler changes now makes
> this patch suitable to servers as well.
> 
> http://kernel.kolivas.org

Dear Con,
I've been having odd problems with a USB device using your patchset (or
specifically, staircase). I applied the staircase7.1 patch as well. On
boot, an already plugged in device is not identified:

usb 2-2: new low speed USB device using address 3
usbhid: probe of 2-2:1.0 failed with error -5

Furthermore, a USB mouse is also on the bus, but it is identified fine.
Reinserting the other device still doesn't work:

usb 2-2: new low speed USB device using address 3
usbhid: probe of 2-2:1.0 failed with error -5
Adding 377516k swap on /dev/sda2.  Priority:-1 extents:1
XFS mounting filesystem hda1
Ending clean XFS mount for filesystem: hda1
usb 2-2: USB disconnect, address 3
usb 2-2: new low speed USB device using address 4
usbhid: probe of 2-2:1.0 failed with error -5

Everything works fine in 2.6.7:

usb 2-2: new low speed USB device using address 3
drivers/usb/input/hid-core.c: ctrl urb status -32 received
input: USB HID v1.00 Joystick [6666:0667] on usb-0000:00:10.0-2

At first I thought this was just a strict staircase problem, but I then
enabled usbfs and the joystick is attached fine with -ck1, so maybe it's
just a subtle timing issue?

Thought I would make the report and see if it uncovered anything, -ck runs
great. :)

Brian