public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Which kernel is the best for a small linux system?
@ 2006-03-13  0:40 j4K3xBl4sT3r
  2006-03-13  8:00 ` Lexington Luthor
  2006-03-13  8:17 ` Arjan van de Ven
  0 siblings, 2 replies; 40+ messages in thread
From: j4K3xBl4sT3r @ 2006-03-13  0:40 UTC (permalink / raw)
  To: linux-kernel

Hello all,

I've been seeing many Linux versions, with many features, some of them
just for the newest branches (2.4.x and 2.6.x), I would like to know
for which kind of system each kernel is recommended. On the distros
that we see inside the Net there is the 2.4.x series, normally I
update to 2.6.x (in case of my Slackware 10.2, even getting problems
with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
Kernels? If applicable, where can I get (detailed) information about
these issues? I'm new on Kernel managing, started doing my own distros
at less than one month and would like to know it.

Thanks in advanced,
j4k3.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13  0:40 Which kernel is the best for a small linux system? j4K3xBl4sT3r
@ 2006-03-13  8:00 ` Lexington Luthor
  2006-03-15  1:35   ` Grant Coady
  2006-03-13  8:17 ` Arjan van de Ven
  1 sibling, 1 reply; 40+ messages in thread
From: Lexington Luthor @ 2006-03-13  8:00 UTC (permalink / raw)
  To: linux-kernel

j4K3xBl4sT3r wrote:
> Hello all,
> 
> I've been seeing many Linux versions, with many features, some of them
> just for the newest branches (2.4.x and 2.6.x), I would like to know
> for which kind of system each kernel is recommended.

Hi,

I am not a kernel developer, so this is not an official recommendation, 
but exactly what kind of "small" system do you mean?

I build and maintain a Linux distribution geared for memory-constrained 
x86 systems, as old as 386s with 4MB of RAM, and the 2.6 kernel fairs 
very well there. I only do so as a hobby not officially supported, so I 
haven't tested the distribution with a very wide range of workloads, but 
for bootstrapping itself from sources and for general home LAN routing 
work, its great.

You might want to look into patch sets like the 2.6-tiny patches, which 
greatly reduce the memory footprint of the kernel: 
http://www.selenic.com/linux-tiny/

Also, you might want to look into the uclibc or dietlibc libraries, 
which are a much smaller and less memory hungry than glibc (which has 
become a memory pig in recent years).

Regards,
LL


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13  0:40 Which kernel is the best for a small linux system? j4K3xBl4sT3r
  2006-03-13  8:00 ` Lexington Luthor
@ 2006-03-13  8:17 ` Arjan van de Ven
  2006-03-13 18:03   ` Grant Coady
  2006-03-13 18:27   ` Sam Ravnborg
  1 sibling, 2 replies; 40+ messages in thread
From: Arjan van de Ven @ 2006-03-13  8:17 UTC (permalink / raw)
  To: j4K3xBl4sT3r; +Cc: linux-kernel

On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> Hello all,
> 
> I've been seeing many Linux versions, with many features, some of them
> just for the newest branches (2.4.x and 2.6.x), I would like to know
> for which kind of system each kernel is recommended. On the distros
> that we see inside the Net there is the 2.4.x series, normally I
> update to 2.6.x (in case of my Slackware 10.2, even getting problems
> with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> Kernels? If applicable, where can I get (detailed) information about
> these issues? I'm new on Kernel managing, started doing my own distros
> at less than one month and would like to know it.

regardless of the size issue; you should really not start any new
projects based on 2.4 kernels; they are in deep deep maintenance mode
for now, but it's unclear how long they will be (I suppose as long as
people keep sending patches), especially complex security issues should
worry people ;)

2.6 is actively maintained and will be for quite some time :)


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13  8:17 ` Arjan van de Ven
@ 2006-03-13 18:03   ` Grant Coady
  2006-03-13 18:06     ` Arjan van de Ven
  2006-03-13 18:27   ` Sam Ravnborg
  1 sibling, 1 reply; 40+ messages in thread
From: Grant Coady @ 2006-03-13 18:03 UTC (permalink / raw)
  To: Arjan van de Ven; +Cc: j4K3xBl4sT3r, linux-kernel

On Mon, 13 Mar 2006 09:17:47 +0100, Arjan van de Ven <arjan@infradead.org> wrote:

>2.6 is actively maintained and will be for quite some time :)

2.6 is an experimental, unstable and non-trustworthy file muncher.

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:03   ` Grant Coady
@ 2006-03-13 18:06     ` Arjan van de Ven
  2006-03-14  6:21       ` Willy Tarreau
  0 siblings, 1 reply; 40+ messages in thread
From: Arjan van de Ven @ 2006-03-13 18:06 UTC (permalink / raw)
  To: gcoady; +Cc: j4K3xBl4sT3r, linux-kernel

On Tue, 2006-03-14 at 05:03 +1100, Grant Coady wrote:
> On Mon, 13 Mar 2006 09:17:47 +0100, Arjan van de Ven <arjan@infradead.org> wrote:
> 
> >2.6 is actively maintained and will be for quite some time :)
> 
> 2.6 is an experimental, unstable and non-trustworthy file muncher.

that's tripple fud that sounds like a troll ;)
Sorry but it does.

2.6 is very stable for a LOT of people, more so than 2.4 in fact.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13  8:17 ` Arjan van de Ven
  2006-03-13 18:03   ` Grant Coady
@ 2006-03-13 18:27   ` Sam Ravnborg
  2006-03-13 18:41     ` Lee Revell
                       ` (3 more replies)
  1 sibling, 4 replies; 40+ messages in thread
From: Sam Ravnborg @ 2006-03-13 18:27 UTC (permalink / raw)
  To: Arjan van de Ven; +Cc: j4K3xBl4sT3r, linux-kernel

On Mon, Mar 13, 2006 at 09:17:47AM +0100, Arjan van de Ven wrote:
> On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> > Hello all,
> > 
> > I've been seeing many Linux versions, with many features, some of them
> > just for the newest branches (2.4.x and 2.6.x), I would like to know
> > for which kind of system each kernel is recommended. On the distros
> > that we see inside the Net there is the 2.4.x series, normally I
> > update to 2.6.x (in case of my Slackware 10.2, even getting problems
> > with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> > Kernels? If applicable, where can I get (detailed) information about
> > these issues? I'm new on Kernel managing, started doing my own distros
> > at less than one month and would like to know it.
> 
> regardless of the size issue; you should really not start any new
> projects based on 2.4 kernels; they are in deep deep maintenance mode
> for now, but it's unclear how long they will be (I suppose as long as
> people keep sending patches), especially complex security issues should
> worry people ;)
> 
> 2.6 is actively maintained and will be for quite some time :)

Any comments on this:
http://www.denx.de/wiki/Know/Linux24vs26

On another denx.de page I found this summary (so you do not have to
visit the page):
# slow to build: 2.6 takes 30...40% longer to compile
# Big memory footprint in flash: the 2.6 compressed kernel image is
# 30...40% bigger
# Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;
# the available RAM size for applications is 700kB smaller
# Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
# Slow to run: context switches up to 96% slower, local communication
# latencies up to 80% slower, file system latencies up to 76% slower,
# local communication bandwidth less than 50% in some cases.

I'm merely asked because I have been pointed to this page several times
and I do nto have numbers for 2.4 versus 2.6.

Note: denx does support 2.6 now.

I do not concur and recommend 2.6 but wanted to know if anyone had more
insight to share.

	Sam

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:27   ` Sam Ravnborg
@ 2006-03-13 18:41     ` Lee Revell
  2006-03-13 19:01       ` Adrian Bunk
  2006-03-13 19:08     ` Diego Calleja
                       ` (2 subsequent siblings)
  3 siblings, 1 reply; 40+ messages in thread
From: Lee Revell @ 2006-03-13 18:41 UTC (permalink / raw)
  To: Sam Ravnborg; +Cc: Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Mon, 2006-03-13 at 19:27 +0100, Sam Ravnborg wrote:
> # latencies up to 80% slower

This is certainly bullshit, it has not been true since 2.6.7 or so.

Did not visit the page but that list smells like they are selling
something.

Lee


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:41     ` Lee Revell
@ 2006-03-13 19:01       ` Adrian Bunk
  2006-03-14  0:18         ` j4K3xBl4sT3r
  0 siblings, 1 reply; 40+ messages in thread
From: Adrian Bunk @ 2006-03-13 19:01 UTC (permalink / raw)
  To: Lee Revell; +Cc: Sam Ravnborg, Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Mon, Mar 13, 2006 at 01:41:27PM -0500, Lee Revell wrote:
> On Mon, 2006-03-13 at 19:27 +0100, Sam Ravnborg wrote:
> > # latencies up to 80% slower
> 
> This is certainly bullshit, it has not been true since 2.6.7 or so.
> 
> Did not visit the page but that list smells like they are selling
> something.

The might be issues already fixed in 2.6.15 (he tested the then-current 
2.6.11.7) or there might be powerpc specific problems, but after a quick 
look at this page it looks like a serious page.

He also posted the complete lmbench results, dmesg's and .config's at 
his page, and from a first view I'd say he has very well documented 
what and how he measured.

> Lee

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:27   ` Sam Ravnborg
  2006-03-13 18:41     ` Lee Revell
@ 2006-03-13 19:08     ` Diego Calleja
  2006-03-13 22:00     ` Pantelis Antoniou
  2006-03-15 22:57     ` Jan Engelhardt
  3 siblings, 0 replies; 40+ messages in thread
From: Diego Calleja @ 2006-03-13 19:08 UTC (permalink / raw)
  To: Sam Ravnborg; +Cc: arjan, jakexblaster, linux-kernel

El Mon, 13 Mar 2006 19:27:25 +0100,
Sam Ravnborg <sam@ravnborg.org> escribió:

> Any comments on this:
> http://www.denx.de/wiki/Know/Linux24vs26
> 
> On another denx.de page I found this summary (so you do not have to
> visit the page):
> # slow to build: 2.6 takes 30...40% longer to compile
> # Big memory footprint in flash: the 2.6 compressed kernel image is
> # 30...40% bigger
> # Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;

In one of those analysis (2.6 sandpoint kernel) they didn't disable
CONFIG_KALLSYMS (they disabled it on the tqm860l though), that makes the
kernel way too big and should be disabled by embedded systems, I don't
understand. That one at least should be fixed, 2.4 didn't even feature
kallsyms.

Also, they claim that context switches are "on average 55% slower (range:
10...94%)", which may be very well a ppc-only bug (in x86 at least
system calls got much faster). And syscalls being much slower is why most
of the other microbenchmarks look so bad.


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:27   ` Sam Ravnborg
  2006-03-13 18:41     ` Lee Revell
  2006-03-13 19:08     ` Diego Calleja
@ 2006-03-13 22:00     ` Pantelis Antoniou
  2006-03-13 22:01       ` j4K3xBl4sT3r
  2006-03-15 22:57     ` Jan Engelhardt
  3 siblings, 1 reply; 40+ messages in thread
From: Pantelis Antoniou @ 2006-03-13 22:00 UTC (permalink / raw)
  To: Sam Ravnborg
  Cc: Arjan van de Ven, j4K3xBl4sT3r, linux-kernel, Dan Malek,
	Marcelo Tosatti

On Monday 13 March 2006 20:27, Sam Ravnborg wrote:
> On Mon, Mar 13, 2006 at 09:17:47AM +0100, Arjan van de Ven wrote:
> > On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> > > Hello all,
> > > 
> > > I've been seeing many Linux versions, with many features, some of them
> > > just for the newest branches (2.4.x and 2.6.x), I would like to know
> > > for which kind of system each kernel is recommended. On the distros
> > > that we see inside the Net there is the 2.4.x series, normally I
> > > update to 2.6.x (in case of my Slackware 10.2, even getting problems
> > > with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> > > Kernels? If applicable, where can I get (detailed) information about
> > > these issues? I'm new on Kernel managing, started doing my own distros
> > > at less than one month and would like to know it.
> > 
> > regardless of the size issue; you should really not start any new
> > projects based on 2.4 kernels; they are in deep deep maintenance mode
> > for now, but it's unclear how long they will be (I suppose as long as
> > people keep sending patches), especially complex security issues should
> > worry people ;)
> > 
> > 2.6 is actively maintained and will be for quite some time :)
> 
> Any comments on this:
> http://www.denx.de/wiki/Know/Linux24vs26
> 
> On another denx.de page I found this summary (so you do not have to
> visit the page):
> # slow to build: 2.6 takes 30...40% longer to compile
> # Big memory footprint in flash: the 2.6 compressed kernel image is
> # 30...40% bigger
> # Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;
> # the available RAM size for applications is 700kB smaller
> # Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
> # Slow to run: context switches up to 96% slower, local communication
> # latencies up to 80% slower, file system latencies up to 76% slower,
> # local communication bandwidth less than 50% in some cases.
> 
> I'm merely asked because I have been pointed to this page several times
> and I do nto have numbers for 2.4 versus 2.6.
> 
> Note: denx does support 2.6 now.
> 
> I do not concur and recommend 2.6 but wanted to know if anyone had more
> insight to share.
> 
> 	Sam
> -

Hi there.

Since I've been dealing with those platforms quite a lot, let me have
my $0.02.

Yes 2.6 is larger than 2.4 and with small embedded processors with small
caches & a small number of TLBs that footprint is felt quite a lot.

For the 8xx which shows the biggest performance, later kernels offer 
the CONFIG_PIN_TLB option which help quite a bit.

So for anything new I'd recommend 2.6 anyway, the performance delta
is not so great as this test appears to show. I'd like this test to be performed
again against a newer kernel version if possible.

Pantelis

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:00     ` Pantelis Antoniou
@ 2006-03-13 22:01       ` j4K3xBl4sT3r
  2006-03-13 22:20         ` Pantelis Antoniou
  2006-03-13 22:25         ` Lee Revell
  0 siblings, 2 replies; 40+ messages in thread
From: j4K3xBl4sT3r @ 2006-03-13 22:01 UTC (permalink / raw)
  To: linux-kernel

On 3/13/06, Pantelis Antoniou <pantelis@embeddedalley.com> wrote:
> On Monday 13 March 2006 20:27, Sam Ravnborg wrote:
> > On Mon, Mar 13, 2006 at 09:17:47AM +0100, Arjan van de Ven wrote:
> > > On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> > > > Hello all,
> > > >
> > > > I've been seeing many Linux versions, with many features, some of them
> > > > just for the newest branches (2.4.x and 2.6.x), I would like to know
> > > > for which kind of system each kernel is recommended. On the distros
> > > > that we see inside the Net there is the 2.4.x series, normally I
> > > > update to 2.6.x (in case of my Slackware 10.2, even getting problems
> > > > with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> > > > Kernels? If applicable, where can I get (detailed) information about
> > > > these issues? I'm new on Kernel managing, started doing my own distros
> > > > at less than one month and would like to know it.
> > >
> > > regardless of the size issue; you should really not start any new
> > > projects based on 2.4 kernels; they are in deep deep maintenance mode
> > > for now, but it's unclear how long they will be (I suppose as long as
> > > people keep sending patches), especially complex security issues should
> > > worry people ;)
> > >
> > > 2.6 is actively maintained and will be for quite some time :)
> >
> > Any comments on this:
> > http://www.denx.de/wiki/Know/Linux24vs26
> >
> > On another denx.de page I found this summary (so you do not have to
> > visit the page):
> > # slow to build: 2.6 takes 30...40% longer to compile
> > # Big memory footprint in flash: the 2.6 compressed kernel image is
> > # 30...40% bigger
> > # Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;
> > # the available RAM size for applications is 700kB smaller
> > # Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
> > # Slow to run: context switches up to 96% slower, local communication
> > # latencies up to 80% slower, file system latencies up to 76% slower,
> > # local communication bandwidth less than 50% in some cases.
> >
> > I'm merely asked because I have been pointed to this page several times
> > and I do nto have numbers for 2.4 versus 2.6.
> >
> > Note: denx does support 2.6 now.
> >
> > I do not concur and recommend 2.6 but wanted to know if anyone had more
> > insight to share.
> >
> >       Sam
> > -
>
> Hi there.
>
> Since I've been dealing with those platforms quite a lot, let me have
> my $0.02.
>
> Yes 2.6 is larger than 2.4 and with small embedded processors with small
> caches & a small number of TLBs that footprint is felt quite a lot.
>
> For the 8xx which shows the biggest performance, later kernels offer
> the CONFIG_PIN_TLB option which help quite a bit.
>
> So for anything new I'd recommend 2.6 anyway, the performance delta
> is not so great as this test appears to show. I'd like this test to be performed
> again against a newer kernel version if possible.
>
> Pantelis
>

so, in the case of the big footprints, might I use a 2.4.x instead of
2.6.x just to avoid memory leaks and performance loss?

j4k3.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:01       ` j4K3xBl4sT3r
@ 2006-03-13 22:20         ` Pantelis Antoniou
  2006-03-13 22:33           ` j4K3xBl4sT3r
  2006-03-13 22:25         ` Lee Revell
  1 sibling, 1 reply; 40+ messages in thread
From: Pantelis Antoniou @ 2006-03-13 22:20 UTC (permalink / raw)
  To: j4K3xBl4sT3r; +Cc: linux-kernel

On Tuesday 14 March 2006 00:01, j4K3xBl4sT3r wrote:
> On 3/13/06, Pantelis Antoniou <pantelis@embeddedalley.com> wrote:
> > On Monday 13 March 2006 20:27, Sam Ravnborg wrote:
> > > On Mon, Mar 13, 2006 at 09:17:47AM +0100, Arjan van de Ven wrote:
> > > > On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> > > > > Hello all,
> > > > >
> > > > > I've been seeing many Linux versions, with many features, some of them
> > > > > just for the newest branches (2.4.x and 2.6.x), I would like to know
> > > > > for which kind of system each kernel is recommended. On the distros
> > > > > that we see inside the Net there is the 2.4.x series, normally I
> > > > > update to 2.6.x (in case of my Slackware 10.2, even getting problems
> > > > > with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> > > > > Kernels? If applicable, where can I get (detailed) information about
> > > > > these issues? I'm new on Kernel managing, started doing my own distros
> > > > > at less than one month and would like to know it.
> > > >
> > > > regardless of the size issue; you should really not start any new
> > > > projects based on 2.4 kernels; they are in deep deep maintenance mode
> > > > for now, but it's unclear how long they will be (I suppose as long as
> > > > people keep sending patches), especially complex security issues should
> > > > worry people ;)
> > > >
> > > > 2.6 is actively maintained and will be for quite some time :)
> > >
> > > Any comments on this:
> > > http://www.denx.de/wiki/Know/Linux24vs26
> > >
> > > On another denx.de page I found this summary (so you do not have to
> > > visit the page):
> > > # slow to build: 2.6 takes 30...40% longer to compile
> > > # Big memory footprint in flash: the 2.6 compressed kernel image is
> > > # 30...40% bigger
> > > # Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;
> > > # the available RAM size for applications is 700kB smaller
> > > # Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
> > > # Slow to run: context switches up to 96% slower, local communication
> > > # latencies up to 80% slower, file system latencies up to 76% slower,
> > > # local communication bandwidth less than 50% in some cases.
> > >
> > > I'm merely asked because I have been pointed to this page several times
> > > and I do nto have numbers for 2.4 versus 2.6.
> > >
> > > Note: denx does support 2.6 now.
> > >
> > > I do not concur and recommend 2.6 but wanted to know if anyone had more
> > > insight to share.
> > >
> > >       Sam
> > > -
> >
> > Hi there.
> >
> > Since I've been dealing with those platforms quite a lot, let me have
> > my $0.02.
> >
> > Yes 2.6 is larger than 2.4 and with small embedded processors with small
> > caches & a small number of TLBs that footprint is felt quite a lot.
> >
> > For the 8xx which shows the biggest performance, later kernels offer
> > the CONFIG_PIN_TLB option which help quite a bit.
> >
> > So for anything new I'd recommend 2.6 anyway, the performance delta
> > is not so great as this test appears to show. I'd like this test to be performed
> > again against a newer kernel version if possible.
> >
> > Pantelis
> >
> 
> so, in the case of the big footprints, might I use a 2.4.x instead of
> 2.6.x just to avoid memory leaks and performance loss?
> 
> j4k3.
>

What memory leaks? And cut it out with 1337 speak. It stopped
being funny 10 years ago...

Pantelis

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:01       ` j4K3xBl4sT3r
  2006-03-13 22:20         ` Pantelis Antoniou
@ 2006-03-13 22:25         ` Lee Revell
  2006-03-14  3:53           ` Grant Coady
  1 sibling, 1 reply; 40+ messages in thread
From: Lee Revell @ 2006-03-13 22:25 UTC (permalink / raw)
  To: j4K3xBl4sT3r; +Cc: linux-kernel

On Mon, 2006-03-13 at 19:01 -0300, j4K3xBl4sT3r wrote:
> so, in the case of the big footprints, might I use a 2.4.x instead of
> 2.6.x just to avoid memory leaks and performance loss? 

Simply using more memory than a previous version is not a memory leak.

Lee


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:20         ` Pantelis Antoniou
@ 2006-03-13 22:33           ` j4K3xBl4sT3r
  2006-03-16 12:03             ` Pavel Machek
  0 siblings, 1 reply; 40+ messages in thread
From: j4K3xBl4sT3r @ 2006-03-13 22:33 UTC (permalink / raw)
  To: linux-kernel

On 3/13/06, Pantelis Antoniou <pantelis@embeddedalley.com> wrote:
> On Tuesday 14 March 2006 00:01, j4K3xBl4sT3r wrote:
> > On 3/13/06, Pantelis Antoniou <pantelis@embeddedalley.com> wrote:
> > > On Monday 13 March 2006 20:27, Sam Ravnborg wrote:
> > > > On Mon, Mar 13, 2006 at 09:17:47AM +0100, Arjan van de Ven wrote:
> > > > > On Sun, 2006-03-12 at 21:40 -0300, j4K3xBl4sT3r wrote:
> > > > > > Hello all,
> > > > > >
> > > > > > I've been seeing many Linux versions, with many features, some of them
> > > > > > just for the newest branches (2.4.x and 2.6.x), I would like to know
> > > > > > for which kind of system each kernel is recommended. On the distros
> > > > > > that we see inside the Net there is the 2.4.x series, normally I
> > > > > > update to 2.6.x (in case of my Slackware 10.2, even getting problems
> > > > > > with some devices). Is that floppy disks uses only 2.0.x and 2.2.x
> > > > > > Kernels? If applicable, where can I get (detailed) information about
> > > > > > these issues? I'm new on Kernel managing, started doing my own distros
> > > > > > at less than one month and would like to know it.
> > > > >
> > > > > regardless of the size issue; you should really not start any new
> > > > > projects based on 2.4 kernels; they are in deep deep maintenance mode
> > > > > for now, but it's unclear how long they will be (I suppose as long as
> > > > > people keep sending patches), especially complex security issues should
> > > > > worry people ;)
> > > > >
> > > > > 2.6 is actively maintained and will be for quite some time :)
> > > >
> > > > Any comments on this:
> > > > http://www.denx.de/wiki/Know/Linux24vs26
> > > >
> > > > On another denx.de page I found this summary (so you do not have to
> > > > visit the page):
> > > > # slow to build: 2.6 takes 30...40% longer to compile
> > > > # Big memory footprint in flash: the 2.6 compressed kernel image is
> > > > # 30...40% bigger
> > > > # Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM;
> > > > # the available RAM size for applications is 700kB smaller
> > > > # Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
> > > > # Slow to run: context switches up to 96% slower, local communication
> > > > # latencies up to 80% slower, file system latencies up to 76% slower,
> > > > # local communication bandwidth less than 50% in some cases.
> > > >
> > > > I'm merely asked because I have been pointed to this page several times
> > > > and I do nto have numbers for 2.4 versus 2.6.
> > > >
> > > > Note: denx does support 2.6 now.
> > > >
> > > > I do not concur and recommend 2.6 but wanted to know if anyone had more
> > > > insight to share.
> > > >
> > > >       Sam
> > > > -
> > >
> > > Hi there.
> > >
> > > Since I've been dealing with those platforms quite a lot, let me have
> > > my $0.02.
> > >
> > > Yes 2.6 is larger than 2.4 and with small embedded processors with small
> > > caches & a small number of TLBs that footprint is felt quite a lot.
> > >
> > > For the 8xx which shows the biggest performance, later kernels offer
> > > the CONFIG_PIN_TLB option which help quite a bit.
> > >
> > > So for anything new I'd recommend 2.6 anyway, the performance delta
> > > is not so great as this test appears to show. I'd like this test to be performed
> > > again against a newer kernel version if possible.
> > >
> > > Pantelis
> > >
> >
> > so, in the case of the big footprints, might I use a 2.4.x instead of
> > 2.6.x just to avoid memory leaks and performance loss?
> >
> > j4k3.
> >
>
> What memory leaks? And cut it out with 1337 speak. It stopped
> being funny 10 years ago...
>
> Pantelis
>

OffTopic: lol @pantelis, so how would be "memory leak" in leet lang? =p

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 19:01       ` Adrian Bunk
@ 2006-03-14  0:18         ` j4K3xBl4sT3r
  0 siblings, 0 replies; 40+ messages in thread
From: j4K3xBl4sT3r @ 2006-03-14  0:18 UTC (permalink / raw)
  To: linux-kernel

On 3/13/06, Adrian Bunk <bunk@stusta.de> wrote:
> On Mon, Mar 13, 2006 at 01:41:27PM -0500, Lee Revell wrote:
> > On Mon, 2006-03-13 at 19:27 +0100, Sam Ravnborg wrote:
> > > # latencies up to 80% slower
> >
> > This is certainly bullshit, it has not been true since 2.6.7 or so.
> >
> > Did not visit the page but that list smells like they are selling
> > something.
>
> The might be issues already fixed in 2.6.15 (he tested the then-current
> 2.6.11.7) or there might be powerpc specific problems, but after a quick
> look at this page it looks like a serious page.
>
> He also posted the complete lmbench results, dmesg's and .config's at
> his page, and from a first view I'd say he has very well documented
> what and how he measured.
>
> > Lee
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>
>

Ye, I just compiled the lastest kernel, 2.6.15.6, it seems a lot
faster than my old one, 2.6.15.4, with the same configuration, and
faster to compile, even I was compiling dietlibc (that incredible got
only 10MB).

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:25         ` Lee Revell
@ 2006-03-14  3:53           ` Grant Coady
  0 siblings, 0 replies; 40+ messages in thread
From: Grant Coady @ 2006-03-14  3:53 UTC (permalink / raw)
  To: Lee Revell; +Cc: j4K3xBl4sT3r, linux-kernel

On Mon, 13 Mar 2006 17:25:44 -0500, Lee Revell <rlrevell@joe-job.com> wrote:

>On Mon, 2006-03-13 at 19:01 -0300, j4K3xBl4sT3r wrote:
>> so, in the case of the big footprints, might I use a 2.4.x instead of
>> 2.6.x just to avoid memory leaks and performance loss? 
>
>Simply using more memory than a previous version is not a memory leak.

Sure, but there are measured performance loss in some areas 2.6 vs 2.4, 
which seem unimportant to others on lkml.  Interactive ssh console delay 
is the one I notice and have reported a couple times to lkml without 
resolution.  

While 2.4.latest is getting security fixes from Willy Tarreau, I trust 
it more than 2.6.stable to run the firewall box here, which is an old 
pentium/mmx 233 box.  

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:06     ` Arjan van de Ven
@ 2006-03-14  6:21       ` Willy Tarreau
  2006-03-14  7:29         ` Ingo Molnar
  2006-03-14 10:03         ` Grant Coady
  0 siblings, 2 replies; 40+ messages in thread
From: Willy Tarreau @ 2006-03-14  6:21 UTC (permalink / raw)
  To: Arjan van de Ven; +Cc: gcoady, j4K3xBl4sT3r, linux-kernel

On Mon, Mar 13, 2006 at 07:06:52PM +0100, Arjan van de Ven wrote:
> On Tue, 2006-03-14 at 05:03 +1100, Grant Coady wrote:
> > On Mon, 13 Mar 2006 09:17:47 +0100, Arjan van de Ven <arjan@infradead.org> wrote:
> > 
> > >2.6 is actively maintained and will be for quite some time :)
> > 
> > 2.6 is an experimental, unstable and non-trustworthy file muncher.

Uhh? Grant, you are feeling brave to say this in front of so many
kernel developpers ! :-)

> that's tripple fud that sounds like a troll ;)
> Sorry but it does.
> 
> 2.6 is very stable for a LOT of people, more so than 2.4 in fact.

It depends a lot on what people do with it in fact. For instance, it
works better in memory-constrained systems, probably thanks to rmap.
I have one 2.6 running reliably on my web server (hppa) where 2.4
regularly oopsed because of low memory.

I would also recommend it for very small systems with limited
features, thanks to Matt Mackall's tiny patches, mostly merged
in 2.6.16. I've been building boot managers with it which fit
in less than 800 kB including kernel + full initramfs, and
that works amazingly well.

However, network performance has significantly dropped, and the
scheduler is still a big problem. Not only we occasionally see
people complaining about unfair CPU distribution across processes
(may be fixed now), but the scheduler still gives a huge boost to
I/O intensive tasks which do lots of select() with small time-outs,
which makes it practically unusable in network-intensive environments.
I've observed systems on which it was nearly impossible to log in via
SSH because of this, and I could reproduce the problem locally to
create a local DoS where a single user could prevent anybody from
logging in. 2.6.15 has improved a lot on this (pauses have reduced
from 35 seconds to 4 seconds) but it's still not very good.

It's still the major reason why I haven't switched, and why several
people I know regularly jump back to 2.4 when they realize that it's
not their hardware which is slow. On the other side, block I/O seems
to have improved a lot. Slocate takes far less time in 2.6 than in
2.4 and runs smoother.

The last stability concern is about code stability. It's moving
very fast, and whatever version you choose, you'll have a hard
time trying to backport fixes in 1 year. Even for Greg and Chris
it has been a huge work to maintain fixes for both 2.6.14 and
2.6.15. I hope things will stabilize. The only real solution right
now would be to use commercial distros who pay developpers to do
this painful work.

Regards,
Willy


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  6:21       ` Willy Tarreau
@ 2006-03-14  7:29         ` Ingo Molnar
  2006-03-14  8:18           ` Mike Galbraith
  2006-03-14  9:05           ` Mike Galbraith
  2006-03-14 10:03         ` Grant Coady
  1 sibling, 2 replies; 40+ messages in thread
From: Ingo Molnar @ 2006-03-14  7:29 UTC (permalink / raw)
  To: Willy Tarreau
  Cc: Arjan van de Ven, linux-kernel, Andrew Morton, Mike Galbraith


* Willy Tarreau <willy@w.ods.org> wrote:

> scheduler is still a big problem. Not only we occasionally see people 
> complaining about unfair CPU distribution across processes (may be 
> fixed now), but the scheduler still gives a huge boost to I/O 
> intensive tasks which do lots of select() with small time-outs, which 
> makes it practically unusable in network-intensive environments. I've 
> observed systems on which it was nearly impossible to log in via SSH 
> because of this, and I could reproduce the problem locally to create a 
> local DoS where a single user could prevent anybody from logging in.  
> 2.6.15 has improved a lot on this (pauses have reduced from 35 seconds 
> to 4 seconds) but it's still not very good.

i think we've talked about your specific case before, and lets not drop 
it on the floor. IIRC you have some special workload (driven by serial 
lines?) which behaves interactively and which thus gets too much 
boosting.

the passive methods: did you try to mute its impact by renicing it to +5 
or +10? If you know that a workload is not interactive, despite it 
behaving so, you can always prevent it from getting too much attention.  
There's also SCHED_BATCH in 2.6.16-ish kernels.

and there are some active methods as well: you might want to try Mike 
Galbraith's scheduler throttling feature:

 http://lkml.org/lkml/2006/3/3/59
 http://lkml.org/lkml/2006/3/3/63

(which we could try in -mm too perhaps, perhaps Mike has an updated 
patch for 2.6.16-rc6-mm1?)

	Ingo

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  7:29         ` Ingo Molnar
@ 2006-03-14  8:18           ` Mike Galbraith
  2006-03-14  9:05           ` Mike Galbraith
  1 sibling, 0 replies; 40+ messages in thread
From: Mike Galbraith @ 2006-03-14  8:18 UTC (permalink / raw)
  To: Ingo Molnar; +Cc: Willy Tarreau, Arjan van de Ven, linux-kernel, Andrew Morton

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On Tue, 2006-03-14 at 08:29 +0100, Ingo Molnar wrote:

> and there are some active methods as well: you might want to try Mike 
> Galbraith's scheduler throttling feature:
> 
>  http://lkml.org/lkml/2006/3/3/59
>  http://lkml.org/lkml/2006/3/3/63
> 
> (which we could try in -mm too perhaps, perhaps Mike has an updated 
> patch for 2.6.16-rc6-mm1?)

Yeah, the whole thing is in the attached tarball.  The only significant
difference from last posted version is that setting both tunables to 0
causes 100% of the interactivity stuff to be disabled, returning the
scheduler to pure O(1), but with it's ability to starve mostly neutered.

	-Mike

(delicate tummy warning may be appropriate.. dunno;)

[-- Attachment #2: throttle-V20-2.6.16-rc6-mm1.tar.gz --]
[-- Type: application/x-compressed-tar, Size: 7324 bytes --]

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  7:29         ` Ingo Molnar
  2006-03-14  8:18           ` Mike Galbraith
@ 2006-03-14  9:05           ` Mike Galbraith
  2006-03-14 15:54             ` Willy Tarreau
  2006-03-16 15:24             ` Mike Galbraith
  1 sibling, 2 replies; 40+ messages in thread
From: Mike Galbraith @ 2006-03-14  9:05 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Ingo Molnar, Arjan van de Ven, linux-kernel, Andrew Morton

On Tue, 2006-03-14 at 08:29 +0100, Ingo Molnar wrote:
> * Willy Tarreau <willy@w.ods.org> wrote:
> 
> > scheduler is still a big problem. Not only we occasionally see people 
> > complaining about unfair CPU distribution across processes (may be 
> > fixed now), but the scheduler still gives a huge boost to I/O 
> > intensive tasks which do lots of select() with small time-outs, which 
> > makes it practically unusable in network-intensive environments. I've 
> > observed systems on which it was nearly impossible to log in via SSH 
> > because of this, and I could reproduce the problem locally to create a 
> > local DoS where a single user could prevent anybody from logging in.  
> > 2.6.15 has improved a lot on this (pauses have reduced from 35 seconds 
> > to 4 seconds) but it's still not very good.

Hi Willy,

BTW, if you try my stuff, it'd be good to try just the "cleanup" patch
first.  It seems very likely to me that your problem is mostly caused by
the sleep_avg multiplier.  If the first patch cures your woes, try
killing just the multiplier in virgin source.

	-Mike

(oh yeah, the pipe patch is more or less meaningless now, ignore it)


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  6:21       ` Willy Tarreau
  2006-03-14  7:29         ` Ingo Molnar
@ 2006-03-14 10:03         ` Grant Coady
  2006-03-14 22:21           ` Russell King
  1 sibling, 1 reply; 40+ messages in thread
From: Grant Coady @ 2006-03-14 10:03 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Tue, 14 Mar 2006 07:21:44 +0100, Willy Tarreau <willy@w.ods.org> wrote:

>On Mon, Mar 13, 2006 at 07:06:52PM +0100, Arjan van de Ven wrote:
>> On Tue, 2006-03-14 at 05:03 +1100, Grant Coady wrote:
>> > On Mon, 13 Mar 2006 09:17:47 +0100, Arjan van de Ven <arjan@infradead.org> wrote:
>> > 
>> > >2.6 is actively maintained and will be for quite some time :)
>> > 
>> > 2.6 is an experimental, unstable and non-trustworthy file muncher.
>
>Uhh? Grant, you are feeling brave to say this in front of so many
>kernel developpers ! :-)

Hah, I've already offended a maintainer to the point of being 
kill-filed.  So what?  This place is anarchy in action, people 
fight for their ideas -- I been reading lkml since '97...

2.4 series I've used since before Linus slipped the 2.4.0 out 
without fanfare.

2.5 I missed, doing other stuff, take a look at 2.6 for only 
the last year or so, since I think about 2.6.10...

>> that's tripple fud that sounds like a troll ;)
>> Sorry but it does.

That's okay, while I'm no troll, I hold an opinion that a few may share.
>> 
>> 2.6 is very stable for a LOT of people, more so than 2.4 in fact.

Stable in what sense?  Rate of change of code is still accelerating?

2.6.15.5 stable issued with bust NFS 'cos people didn't check their 
bug / security fix patch progress when requested by the stable team?

By stable I mean rate of change of codebase, patch volume per month,  
2.6 is orders of magnitude less stable than 2.4 by that simple measure.

>It depends a lot on what people do with it in fact. For instance, it
>works better in memory-constrained systems, probably thanks to rmap.
>I have one 2.6 running reliably on my web server (hppa) where 2.4
>regularly oopsed because of low memory.

Well, memory is cheap?  Lowest memory box here is 128MB, and I don't 
get my hands on appliance hardware to try the really small stuff.

>However, network performance has significantly dropped, and the
>scheduler is still a big problem. Not only we occasionally see
>people complaining about unfair CPU distribution across processes
>(may be fixed now), but the scheduler still gives a huge boost to
>I/O intensive tasks which do lots of select() with small time-outs,
>which makes it practically unusable in network-intensive environments.

>I've observed systems on which it was nearly impossible to log in via
>SSH because of this, and I could reproduce the problem locally to
>create a local DoS where a single user could prevent anybody from
>logging in. 2.6.15 has improved a lot on this (pauses have reduced
>from 35 seconds to 4 seconds) but it's still not very good.

Yep, but at least running reiserfs3 I can hit reset and not lose 
filesystem when the thing locks itself into thumb-twiddling mode ;)

>It's still the major reason why I haven't switched, and why several
>people I know regularly jump back to 2.4 when they realize that it's
>not their hardware which is slow. On the other side, block I/O seems
>to have improved a lot. Slocate takes far less time in 2.6 than in
>2.4 and runs smoother.

Yes, bulk I/O transfers are faster in 2.6.  I keep going back to 
2.4 when I'm writing simple text files over ssh terminal session.  
2.6 is simply too sluggish on the same hardware.  Thus I stay with 
a dual boot configuration and limit options to the 2.4 kernel set.

No udev, no new features.  Because the overall 2.6 feel is not yet 
there.  Maybe next year?  Who knows.

>The last stability concern is about code stability. It's moving
>very fast, and whatever version you choose, you'll have a hard
>time trying to backport fixes in 1 year. Even for Greg and Chris
>it has been a huge work to maintain fixes for both 2.6.14 and
>2.6.15. I hope things will stabilize. The only real solution right
>now would be to use commercial distros who pay developpers to do
>this painful work.

My distro of choice (Slackware) doesn't, thus I'm terribly annoyed 
at the cowboy developer attitude of 'leave it to the distro to 
stabilise'.  That issue eased a great deal with the 'sucker tree'.

Cheers,
Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  9:05           ` Mike Galbraith
@ 2006-03-14 15:54             ` Willy Tarreau
  2006-03-16 15:24             ` Mike Galbraith
  1 sibling, 0 replies; 40+ messages in thread
From: Willy Tarreau @ 2006-03-14 15:54 UTC (permalink / raw)
  To: Mike Galbraith; +Cc: Ingo Molnar, Arjan van de Ven, linux-kernel, Andrew Morton

On Tue, Mar 14, 2006 at 10:05:30AM +0100, Mike Galbraith wrote:
> On Tue, 2006-03-14 at 08:29 +0100, Ingo Molnar wrote:
> > * Willy Tarreau <willy@w.ods.org> wrote:
> > 
> > > scheduler is still a big problem. Not only we occasionally see people 
> > > complaining about unfair CPU distribution across processes (may be 
> > > fixed now), but the scheduler still gives a huge boost to I/O 
> > > intensive tasks which do lots of select() with small time-outs, which 
> > > makes it practically unusable in network-intensive environments. I've 
> > > observed systems on which it was nearly impossible to log in via SSH 
> > > because of this, and I could reproduce the problem locally to create a 
> > > local DoS where a single user could prevent anybody from logging in.  
> > > 2.6.15 has improved a lot on this (pauses have reduced from 35 seconds 
> > > to 4 seconds) but it's still not very good.
> 
> Hi Willy,
> 
> BTW, if you try my stuff, it'd be good to try just the "cleanup" patch
> first.  It seems very likely to me that your problem is mostly caused by
> the sleep_avg multiplier.  If the first patch cures your woes, try
> killing just the multiplier in virgin source.
> 
> 	-Mike
> 
> (oh yeah, the pipe patch is more or less meaningless now, ignore it)

Hi Mike, Hi Ingo,

thank you both for your insights. I *will* test this, I don't know when
because I'm terribly busy, but I'm really interested.

Ingo, to reply to your question, the typical workload was around 30 Mbps with
1500-2000 sessions/s on a small number of processes (1 to 4*#CPUs). It was
with some kernels around 2.6.8 IIRC. Pauses could be of several hundreds of
milliseconds which was very annoying. But IIRC, if renicing the process(es)
improved SSH responsiveness, it also hurt the service's responsiveness.
The same process running on 2.2, 2.4, solaris 8/10, freebsd and openbsd does
not exhibit the behaviour at all. I've not retried yet with more recent
kernels, I just recently retried the proof of concept I developped at this
date, and all I remember was that 2.6.15+ was really better.

Cheers,
Willy


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14 10:03         ` Grant Coady
@ 2006-03-14 22:21           ` Russell King
  2006-03-15  0:46             ` Grant Coady
  2006-03-15 22:53             ` Jan Engelhardt
  0 siblings, 2 replies; 40+ messages in thread
From: Russell King @ 2006-03-14 22:21 UTC (permalink / raw)
  To: Grant Coady; +Cc: Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Tue, Mar 14, 2006 at 09:03:39PM +1100, Grant Coady wrote:
> By stable I mean rate of change of codebase, patch volume per month,  
> 2.6 is orders of magnitude less stable than 2.4 by that simple measure.

That is no measure of stability.

If, say, I merge a large patch in order to support ARM SMP and Linus
takes that, let's say for the sake of argument that's a 10MB diff.
It doesn't touch anything other than files which are solely built or
used for the ARM architecture.

Are you going to claim that the kernel is, therefore, unstable on
x86?

So, by your very comment above, if all the updates to non-x86
architectures were prevented from happening in mainline, you'd have
a much more stable kernel.

Uh huh.  There's a saying about comparing apples and oranges which
springs to mind here - did you miss that lesson?

(Please do _not_ cc or reply directly to me in this thread - I'll
read replies from the mailing list, thanks.)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14 22:21           ` Russell King
@ 2006-03-15  0:46             ` Grant Coady
       [not found]               ` <20060315080313.GC3166@flint.arm.linux.org.uk>
  2006-03-15 22:53             ` Jan Engelhardt
  1 sibling, 1 reply; 40+ messages in thread
From: Grant Coady @ 2006-03-15  0:46 UTC (permalink / raw)
  To: Russell King; +Cc: Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Tue, 14 Mar 2006 22:21:32 +0000, Russell King <rmk+lkml@arm.linux.org.uk> wrote:

>On Tue, Mar 14, 2006 at 09:03:39PM +1100, Grant Coady wrote:
>> By stable I mean rate of change of codebase, patch volume per month,  
>> 2.6 is orders of magnitude less stable than 2.4 by that simple measure.
>
>That is no measure of stability.

You're welcome to your opinion.

>If, say, I merge a large patch in order to support ARM SMP and Linus
>takes that, let's say for the sake of argument that's a 10MB diff.
>It doesn't touch anything other than files which are solely built or
>used for the ARM architecture.

So what?  You're not one of the people here beholden to pushing a 
distro's agenda for mainstream x86 windoze wannabe desktops.

>So, by your very comment above, if all the updates to non-x86
>architectures were prevented from happening in mainline, you'd have
>a much more stable kernel.

Not at all, you choose whatever interpretation suits your world view.

>(Please do _not_ cc or reply directly to me in this thread - I'll
>read replies from the mailing list, thanks.)

Get real

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13  8:00 ` Lexington Luthor
@ 2006-03-15  1:35   ` Grant Coady
  2006-03-15  1:46     ` Randy.Dunlap
  0 siblings, 1 reply; 40+ messages in thread
From: Grant Coady @ 2006-03-15  1:35 UTC (permalink / raw)
  To: Lexington Luthor; +Cc: linux-kernel

On Mon, 13 Mar 2006 08:00:51 +0000, Lexington Luthor <lexington.luthor@gmail.com> wrote:

>You might want to look into patch sets like the 2.6-tiny patches, which 
>greatly reduce the memory footprint of the kernel: 
>http://www.selenic.com/linux-tiny/

Seems to have stalled since Oct'05?

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15  1:35   ` Grant Coady
@ 2006-03-15  1:46     ` Randy.Dunlap
  0 siblings, 0 replies; 40+ messages in thread
From: Randy.Dunlap @ 2006-03-15  1:46 UTC (permalink / raw)
  To: gcoady; +Cc: lexington.luthor, linux-kernel

On Wed, 15 Mar 2006 12:35:50 +1100 Grant Coady wrote:

> On Mon, 13 Mar 2006 08:00:51 +0000, Lexington Luthor <lexington.luthor@gmail.com> wrote:
> 
> >You might want to look into patch sets like the 2.6-tiny patches, which 
> >greatly reduce the memory footprint of the kernel: 
> >http://www.selenic.com/linux-tiny/
> 
> Seems to have stalled since Oct'05?

maybe not going as strongly as it once was, but parts of it
have been merged into mainline and (parts of) it are also being
used in CELF (www.celinuxforum.org).

---
~Randy

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
       [not found]               ` <20060315080313.GC3166@flint.arm.linux.org.uk>
@ 2006-03-15  9:37                 ` Grant Coady
  0 siblings, 0 replies; 40+ messages in thread
From: Grant Coady @ 2006-03-15  9:37 UTC (permalink / raw)
  To: Russell King; +Cc: linux-kernel

On Wed, 15 Mar 2006 08:03:13 +0000, Russell King <rmk@arm.linux.org.uk> wrote:

>Thanks for following my request - you obviously know precisely how to
>avoid earning the respect of others.

Russell,

I'm not interested in what you think I should be earning, here or 
elsewhere.  You set up a little game and I'm not playing.

You seek criticism when none is offered, repeatedly.  Your problem, 
not mine.  

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14 22:21           ` Russell King
  2006-03-15  0:46             ` Grant Coady
@ 2006-03-15 22:53             ` Jan Engelhardt
  2006-03-15 23:32               ` Grant Coady
  2006-03-16  2:55               ` Valdis.Kletnieks
  1 sibling, 2 replies; 40+ messages in thread
From: Jan Engelhardt @ 2006-03-15 22:53 UTC (permalink / raw)
  Cc: Grant Coady, Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r,
	Linux Kernel Mailing List


>> By stable I mean rate of change of codebase, patch volume per month,  
>> 2.6 is orders of magnitude less stable than 2.4 by that simple measure.
>
>That is no measure of stability.
>

Ack! Let's pick one:

Although the exact numbers of patches per time for a specific 
software manufacturer - let's pick Microsoft as an example - is not known, 
it is usually low (two for this *month* afaics), compared to what hits lkml 
*each day*.

Does that make their software more stable than Linux? I would have my 
doubts about that.



Jan Engelhardt
-- 

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 18:27   ` Sam Ravnborg
                       ` (2 preceding siblings ...)
  2006-03-13 22:00     ` Pantelis Antoniou
@ 2006-03-15 22:57     ` Jan Engelhardt
  2006-03-15 23:10       ` Adrian Bunk
  3 siblings, 1 reply; 40+ messages in thread
From: Jan Engelhardt @ 2006-03-15 22:57 UTC (permalink / raw)
  To: Sam Ravnborg; +Cc: Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

>
>On another denx.de page I found this summary (so you do not have to
>visit the page):
># slow to build: 2.6 takes 30...40% longer to compile

A side effect of all the new optimizations that went into gcc since 2.95,
I would say.


Jan Engelhardt
-- 

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15 22:57     ` Jan Engelhardt
@ 2006-03-15 23:10       ` Adrian Bunk
  2006-03-16 13:39         ` linux-os (Dick Johnson)
  0 siblings, 1 reply; 40+ messages in thread
From: Adrian Bunk @ 2006-03-15 23:10 UTC (permalink / raw)
  To: Jan Engelhardt; +Cc: Sam Ravnborg, Arjan van de Ven, j4K3xBl4sT3r, linux-kernel

On Wed, Mar 15, 2006 at 11:57:12PM +0100, Jan Engelhardt wrote:
> >
> >On another denx.de page I found this summary (so you do not have to
> >visit the page):
> ># slow to build: 2.6 takes 30...40% longer to compile
> 
> A side effect of all the new optimizations that went into gcc since 2.95,
> I would say.

If you would have had a quick look at the results on the webpage you are 
commenting on instead of blindly speculating, you'd have known that your 
statement is bullshit since both the 2.4 and the 2.6 compiles were done 
using gcc 3.3.3.

> Jan Engelhardt

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15 22:53             ` Jan Engelhardt
@ 2006-03-15 23:32               ` Grant Coady
  2006-03-16  4:19                 ` Mike Galbraith
  2006-03-16  2:55               ` Valdis.Kletnieks
  1 sibling, 1 reply; 40+ messages in thread
From: Grant Coady @ 2006-03-15 23:32 UTC (permalink / raw)
  To: Jan Engelhardt
  Cc: Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r,
	Linux Kernel Mailing List

On Wed, 15 Mar 2006 23:53:21 +0100 (MET), Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:

>
>>> By stable I mean rate of change of codebase, patch volume per month,  
>>> 2.6 is orders of magnitude less stable than 2.4 by that simple measure.
>>
>>That is no measure of stability.
>>
>
>Ack! Let's pick one:
>
>Although the exact numbers of patches per time for a specific 
>software manufacturer - let's pick Microsoft as an example - is not known, 
>it is usually low (two for this *month* afaics), compared to what hits lkml 
>*each day*.
>
>Does that make their software more stable than Linux? I would have my 
>doubts about that.

Yeah but MSFT has been very stably broken for many years ;)

Anyway, what I'm trying to put forward is the notion that the high 
patch churn rate in l-k indicates a non-stable, experimental piece 
of work which may one day result in a stable kernel.  But at the 
moment I'll run 24/7 apps on a 2.4.latest box.

That some here choose to bend that point of view into an unintended 
meaning has nothing to do with the simple reality that, for what I 
use linux for, 2.6 is a such sluggish performing kernel that I soon 
revert to 2.4.latest on the one box that runs 24/7 here.

If you cannot accept that, fine, ridicule the testers feedback you 
do not want to hear.

Certainly provides little motivation for testers to provide any 
feedback does it not?  I've had two threads on sluggish terminal 
here performance without resolution.  2.6 feels sluggish, the test 
is simple and repeatable, your ridicule does not change that at all.


2.4.early was a dog of a kernel, I was often bouncing between Linus' 
and ac' branches back then depending on which was working in a 
particular week.  It got better, as will 2.6.  Maybe by 2.6.20+ ?

Grant.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15 22:53             ` Jan Engelhardt
  2006-03-15 23:32               ` Grant Coady
@ 2006-03-16  2:55               ` Valdis.Kletnieks
  1 sibling, 0 replies; 40+ messages in thread
From: Valdis.Kletnieks @ 2006-03-16  2:55 UTC (permalink / raw)
  To: Jan Engelhardt
  Cc: Grant Coady, Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r,
	Linux Kernel Mailing List

[-- Attachment #1: Type: text/plain, Size: 1421 bytes --]

On Wed, 15 Mar 2006 23:53:21 +0100, Jan Engelhardt said:

> Although the exact numbers of patches per time for a specific 
> software manufacturer - let's pick Microsoft as an example - is not known, 
> it is usually low (two for this *month* afaics), compared to what hits lkml 
> *each day*.
> 
> Does that make their software more stable than Linux? I would have my 
> doubts about that.

You have doubts, because it's a totally b0rken metric. ;)

(Incidentally, there is some pretty good evidence in the computer security
community that although Microsoft has *announced* two patches for this month,
that actually there's code tweaks for *other* un-admitted problems as well.
Careful dissection of the patches often finds them poking in parts of the
operating system far removed from where the obvious problem is - so there
could possibly be a dozen or more *actual* fixes in those two patches..)

A better comparison would be the number of things on lkml *per day*,
compared to the number of issues reported *internal to Microsoft* *per day*.

Or do the comparison after trimming out all the lkml code cleanups and fixes
for obscure corner cases that often seem to only be afflicting one or two
users on the entire planet (I know I've reported my share of those types ;)

The main reason the lkml traffic is so high is because we dissect and argue
almost every single line of code in public before it goes in-tree..

[-- Attachment #2: Type: application/pgp-signature, Size: 228 bytes --]

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15 23:32               ` Grant Coady
@ 2006-03-16  4:19                 ` Mike Galbraith
  2006-03-16  8:12                   ` Mike Galbraith
  0 siblings, 1 reply; 40+ messages in thread
From: Mike Galbraith @ 2006-03-16  4:19 UTC (permalink / raw)
  To: gcoady
  Cc: Jan Engelhardt, Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r,
	Linux Kernel Mailing List

On Thu, 2006-03-16 at 10:32 +1100, Grant Coady wrote:
> 
> Certainly provides little motivation for testers to provide any 
> feedback does it not?  I've had two threads on sluggish terminal 
> here performance without resolution.  2.6 feels sluggish, the test 
> is simple and repeatable, your ridicule does not change that at all.

Hmm.  You have a testcase that's both simple _and_ repeatable?  Cool.
What is it?

	-Mike


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-16  4:19                 ` Mike Galbraith
@ 2006-03-16  8:12                   ` Mike Galbraith
  2006-03-16  8:31                     ` Lee Revell
  0 siblings, 1 reply; 40+ messages in thread
From: Mike Galbraith @ 2006-03-16  8:12 UTC (permalink / raw)
  To: gcoady
  Cc: Jan Engelhardt, Willy Tarreau, Arjan van de Ven, j4K3xBl4sT3r,
	Linux Kernel Mailing List

On Thu, 2006-03-16 at 05:19 +0100, Mike Galbraith wrote:
> On Thu, 2006-03-16 at 10:32 +1100, Grant Coady wrote:
> > 
> > Certainly provides little motivation for testers to provide any 
> > feedback does it not?  I've had two threads on sluggish terminal 
> > here performance without resolution.  2.6 feels sluggish, the test 
> > is simple and repeatable, your ridicule does not change that at all.
> 
> Hmm.  You have a testcase that's both simple _and_ repeatable?  Cool.
> What is it?

P.S.  if you're talking about ssh console slowdown thingie, as I write
this I'm ssh'd into my P3/500, and..

[root]:# time w
 09:07:02 up  2:44,  6 users,  load average: 3.74, 4.09, 3.34
USER     TTY        LOGIN@   IDLE   JCPU   PCPU  WHAT
root     tty1      06:23   10:11   0.17s  0.15s  -bash
root     tty2      08:43   23:34  41.09s 41.05s  top d1
root     tty3      08:49   22.00s  0.12s  0.10s  -bash
root     tty4      08:51    3:50   4.05s  3.95s  ab -c 50 -n 10000
http://localhost 81
root     tty5      09:04    1:12   0.06s  0.05s  -bash
root     pts/0     08:38    0.00s  0.18s  0.02s  w

real    0m0.033s
user    0m0.013s
sys     0m0.019s
[root]:#        


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-16  8:12                   ` Mike Galbraith
@ 2006-03-16  8:31                     ` Lee Revell
  2006-03-16  8:55                       ` Mike Galbraith
  0 siblings, 1 reply; 40+ messages in thread
From: Lee Revell @ 2006-03-16  8:31 UTC (permalink / raw)
  To: Mike Galbraith
  Cc: gcoady, Jan Engelhardt, Willy Tarreau, Arjan van de Ven,
	j4K3xBl4sT3r, Linux Kernel Mailing List

On Thu, 2006-03-16 at 09:12 +0100, Mike Galbraith wrote:
> P.S.  if you're talking about ssh console slowdown thingie, as I write
> this I'm ssh'd into my P3/500, and..
> 
> [root]:# time w

> real    0m0.033s
> user    0m0.013s
> sys     0m0.019s
> [root]:#         

I think you left out the result for 2.4.

Lee


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-16  8:31                     ` Lee Revell
@ 2006-03-16  8:55                       ` Mike Galbraith
  2006-03-16  9:20                         ` Mike Galbraith
  0 siblings, 1 reply; 40+ messages in thread
From: Mike Galbraith @ 2006-03-16  8:55 UTC (permalink / raw)
  To: Lee Revell
  Cc: gcoady, Jan Engelhardt, Willy Tarreau, Arjan van de Ven,
	j4K3xBl4sT3r, Linux Kernel Mailing List

On Thu, 2006-03-16 at 03:31 -0500, Lee Revell wrote:
> On Thu, 2006-03-16 at 09:12 +0100, Mike Galbraith wrote:
> > P.S.  if you're talking about ssh console slowdown thingie, as I write
> > this I'm ssh'd into my P3/500, and..
> > 
> > [root]:# time w
> 
> > real    0m0.033s
> > user    0m0.013s
> > sys     0m0.019s
> > [root]:#         
> 
> I think you left out the result for 2.4.

I don't have a 2.4 kernel.  But no matter, I don't think it's going to
get much better than .033s.  Even if it did, .033s sure doesn't feel
sluggish to me.

Running ab from the other box is a bit slower since ssh is using the
same net ab is blasting (only net I have)..

[root]:# time netstat|grep :81|wc -l
   1645

real    0m0.259s
user    0m0.133s
sys     0m0.126s

...but still not what I'd call a slug.

	-Mike     


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-16  8:55                       ` Mike Galbraith
@ 2006-03-16  9:20                         ` Mike Galbraith
  0 siblings, 0 replies; 40+ messages in thread
From: Mike Galbraith @ 2006-03-16  9:20 UTC (permalink / raw)
  To: Lee Revell
  Cc: gcoady, Jan Engelhardt, Willy Tarreau, Arjan van de Ven,
	j4K3xBl4sT3r, Linux Kernel Mailing List

On Thu, 2006-03-16 at 09:55 +0100, Mike Galbraith wrote:
> On Thu, 2006-03-16 at 03:31 -0500, Lee Revell wrote:
> > On Thu, 2006-03-16 at 09:12 +0100, Mike Galbraith wrote:
> > > P.S.  if you're talking about ssh console slowdown thingie, as I write
> > > this I'm ssh'd into my P3/500, and..
> > > 
> > > [root]:# time w
> > 
> > > real    0m0.033s
> > > user    0m0.013s
> > > sys     0m0.019s
> > > [root]:#         
> > 
> > I think you left out the result for 2.4.
> 
> I don't have a 2.4 kernel.  But no matter, I don't think it's going to
> get much better than .033s.  Even if it did, .033s sure doesn't feel
> sluggish to me.
> 
> Running ab from the other box is a bit slower since ssh is using the
> same net ab is blasting (only net I have)..
> 
> [root]:# time netstat|grep :81|wc -l
>    1645
> 
> real    0m0.259s
> user    0m0.133s
> sys     0m0.126s
> 
> ...but still not what I'd call a slug.

Oops.  Wrong kernel.  That was one of my twiddled kernels.  This is
2.6.15 doing the same ab across the net.

[root]:# time netstat|grep :81|wc -l
   1592

real    3m7.502s
user    0m0.119s
sys     0m0.146s

Ouch!!

	-Mike


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-13 22:33           ` j4K3xBl4sT3r
@ 2006-03-16 12:03             ` Pavel Machek
  0 siblings, 0 replies; 40+ messages in thread
From: Pavel Machek @ 2006-03-16 12:03 UTC (permalink / raw)
  To: j4K3xBl4sT3r; +Cc: linux-kernel

> > > so, in the case of the big footprints, might I use a 2.4.x instead of
> > > 2.6.x just to avoid memory leaks and performance loss?
> > >
> > > j4k3.
> > >
> >
> > What memory leaks? And cut it out with 1337 speak. It stopped
> > being funny 10 years ago...
> >
> > Pantelis
> >
> 
> OffTopic: lol @pantelis, so how would be "memory leak" in leet lang? =p

He did not imply memory leak is leet speak. Please fix your From: header.

								Pavel

-- 
23:        M4PStream m4p;

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-15 23:10       ` Adrian Bunk
@ 2006-03-16 13:39         ` linux-os (Dick Johnson)
  0 siblings, 0 replies; 40+ messages in thread
From: linux-os (Dick Johnson) @ 2006-03-16 13:39 UTC (permalink / raw)
  To: Adrian Bunk
  Cc: Jan Engelhardt, Sam Ravnborg, Arjan van de Ven, j4K3xBl4sT3r,
	linux-kernel


On Wed, 15 Mar 2006, Adrian Bunk wrote:

> On Wed, Mar 15, 2006 at 11:57:12PM +0100, Jan Engelhardt wrote:
>>>
>>> On another denx.de page I found this summary (so you do not have to
>>> visit the page):
>>> # slow to build: 2.6 takes 30...40% longer to compile
>>
>> A side effect of all the new optimizations that went into gcc since 2.95,
>> I would say.
>
> If you would have had a quick look at the results on the webpage you are
> commenting on instead of blindly speculating, you'd have known that your
> statement is bullshit since both the 2.4 and the 2.6 compiles were done
> using gcc 3.3.3.
>
>> Jan Engelhardt
>
> cu
> Adrian

There have been no systemic problems with 2.4.26 in small and
embedded systems -- for whatever that's worth. Stuff might not
be "optimum", but networking of all types, and the usual
unistd.h stuff all works fine. It's good for systems you
don't want to have to muck with.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.15.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction, book release in April.
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: Which kernel is the best for a small linux system?
  2006-03-14  9:05           ` Mike Galbraith
  2006-03-14 15:54             ` Willy Tarreau
@ 2006-03-16 15:24             ` Mike Galbraith
  1 sibling, 0 replies; 40+ messages in thread
From: Mike Galbraith @ 2006-03-16 15:24 UTC (permalink / raw)
  To: Willy Tarreau; +Cc: Ingo Molnar, Arjan van de Ven, linux-kernel, Andrew Morton

On Tue, 2006-03-14 at 10:05 +0100, Mike Galbraith wrote:
> On Tue, 2006-03-14 at 08:29 +0100, Ingo Molnar wrote:
> > * Willy Tarreau <willy@w.ods.org> wrote:
> > 
> > > scheduler is still a big problem. Not only we occasionally see people 
> > > complaining about unfair CPU distribution across processes (may be 
> > > fixed now), but the scheduler still gives a huge boost to I/O 
> > > intensive tasks which do lots of select() with small time-outs, which 
> > > makes it practically unusable in network-intensive environments. I've 
> > > observed systems on which it was nearly impossible to log in via SSH 
> > > because of this, and I could reproduce the problem locally to create a 
> > > local DoS where a single user could prevent anybody from logging in.  
> > > 2.6.15 has improved a lot on this (pauses have reduced from 35 seconds 
> > > to 4 seconds) but it's still not very good.
> 
> Hi Willy,
> 
> BTW, if you try my stuff, it'd be good to try just the "cleanup" patch
> first.  It seems very likely to me that your problem is mostly caused by
> the sleep_avg multiplier.  If the first patch cures your woes, try
> killing just the multiplier in virgin source.

Hi again Willy,

Well.  I've done some testing with apache, and I can reproduce the
problem... too darn well in fact.  It's really bad in 2.6.15 here as
well, and while removing the sleep_avg multiplier and the requeue in
schedule() improved things quite a bit, I still have nasty 30s delays
while trying to pipe commands.  These things alone apparently aren't
going to be enough if you want to ssh into a busy server.

	-Mike


^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2006-03-16 15:22 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-03-13  0:40 Which kernel is the best for a small linux system? j4K3xBl4sT3r
2006-03-13  8:00 ` Lexington Luthor
2006-03-15  1:35   ` Grant Coady
2006-03-15  1:46     ` Randy.Dunlap
2006-03-13  8:17 ` Arjan van de Ven
2006-03-13 18:03   ` Grant Coady
2006-03-13 18:06     ` Arjan van de Ven
2006-03-14  6:21       ` Willy Tarreau
2006-03-14  7:29         ` Ingo Molnar
2006-03-14  8:18           ` Mike Galbraith
2006-03-14  9:05           ` Mike Galbraith
2006-03-14 15:54             ` Willy Tarreau
2006-03-16 15:24             ` Mike Galbraith
2006-03-14 10:03         ` Grant Coady
2006-03-14 22:21           ` Russell King
2006-03-15  0:46             ` Grant Coady
     [not found]               ` <20060315080313.GC3166@flint.arm.linux.org.uk>
2006-03-15  9:37                 ` Grant Coady
2006-03-15 22:53             ` Jan Engelhardt
2006-03-15 23:32               ` Grant Coady
2006-03-16  4:19                 ` Mike Galbraith
2006-03-16  8:12                   ` Mike Galbraith
2006-03-16  8:31                     ` Lee Revell
2006-03-16  8:55                       ` Mike Galbraith
2006-03-16  9:20                         ` Mike Galbraith
2006-03-16  2:55               ` Valdis.Kletnieks
2006-03-13 18:27   ` Sam Ravnborg
2006-03-13 18:41     ` Lee Revell
2006-03-13 19:01       ` Adrian Bunk
2006-03-14  0:18         ` j4K3xBl4sT3r
2006-03-13 19:08     ` Diego Calleja
2006-03-13 22:00     ` Pantelis Antoniou
2006-03-13 22:01       ` j4K3xBl4sT3r
2006-03-13 22:20         ` Pantelis Antoniou
2006-03-13 22:33           ` j4K3xBl4sT3r
2006-03-16 12:03             ` Pavel Machek
2006-03-13 22:25         ` Lee Revell
2006-03-14  3:53           ` Grant Coady
2006-03-15 22:57     ` Jan Engelhardt
2006-03-15 23:10       ` Adrian Bunk
2006-03-16 13:39         ` linux-os (Dick Johnson)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox