Wonderful World of Linux 2.6 - Final

Wonderful World of Linux 2.6 - Final

[Joe Pranevich]

Hello,

I haven't even seen the email yet from linux-kernel about the v2.6
release, but if Slashdot says it's so, it *must* be true. I have just
put the finishing touches on my document describing many of the changes
in the new kernel release. If you're interested, please check it out. It
should be pretty accurate.

Check it out:

HTML - http://kniggit.net/wwol26.html
TEXT - http://kniggit.net/wwol26.txt

Happy Holidays,

Joe

Re: Wonderful World of Linux 2.6 - Final

[Andrew Theurer]

On Wednesday 17 December 2003 23:13, Joe Pranevich wrote:
> Hello,
>
> I haven't even seen the email yet from linux-kernel about the v2.6
> release, but if Slashdot says it's so, it *must* be true. I have just
> put the finishing touches on my document describing many of the changes
> in the new kernel release. If you're interested, please check it out. It
> should be pretty accurate.
>
> Check it out:
>
> HTML - http://kniggit.net/wwol26.html
> TEXT - http://kniggit.net/wwol26.txt

Hyperthreading:

"...that the scheduler now knows how to recognize and optimize processor loads 
across both real and virtual processors within a machine."


This is not true.  Ingo's shared runqueue patch did not make it into 2.6, nor 
did Nick's scheduler domain patch.  Workloads with low loads and HT will not 
be scheduled optimally, for example, a kernel compile with -j4 on a 4-way P4, 
with and without HT:

average of 10 kernel compiles with -j4 on 2.6.0-test9:

HT disabled: Elapsed: 145.086s User: 513.808s System: 44.724s CPU: 384.5%
HT enabled: Elapsed: 172.463s User: 633.856s System: 48.003s CPU: 394.8%


-Andrew Theurer

Re: Wonderful World of Linux 2.6 - Final

[Andrew Theurer]

On Thursday 18 December 2003 09:29, Andrew Theurer wrote:
> On Wednesday 17 December 2003 23:13, Joe Pranevich wrote:
> > Hello,
> >
> > I haven't even seen the email yet from linux-kernel about the v2.6
> > release, but if Slashdot says it's so, it *must* be true. I have just
> > put the finishing touches on my document describing many of the changes
> > in the new kernel release. If you're interested, please check it out. It
> > should be pretty accurate.
> >
> > Check it out:
> >
> > HTML - http://kniggit.net/wwol26.html
> > TEXT - http://kniggit.net/wwol26.txt
>
> Hyperthreading:

Damn, I should really read all the emails first...  I'll blame it on the email 
client, yeah, that's it :)  

Anyway, looks like Nick already caught this, so I guess consider the benchmark 
results in the previous email as proof...

Re: Wonderful World of Linux 2.6 - Final

[Andrea Arcangeli]

On Thu, Dec 18, 2003 at 09:29:46AM -0600, Andrew Theurer wrote:
> be scheduled optimally, for example, a kernel compile with -j4 on a 4-way P4, 
> with and without HT:
> 
> average of 10 kernel compiles with -j4 on 2.6.0-test9:
> 
> HT disabled: Elapsed: 145.086s User: 513.808s System: 44.724s CPU: 384.5%
> HT enabled: Elapsed: 172.463s User: 633.856s System: 48.003s CPU: 394.8%

is that 4-way a 4-logical-way or 4-physical-way? If it's a 4-logical
way, this workload is much closer to the best case than the worst case.
I'm guessing a simple -j2 or -j3 should do much worse than that.

Re: Wonderful World of Linux 2.6 - Final

[Andrew Theurer]

On Thursday 18 December 2003 17:52, Andrea Arcangeli wrote:
> On Thu, Dec 18, 2003 at 09:29:46AM -0600, Andrew Theurer wrote:
> > be scheduled optimally, for example, a kernel compile with -j4 on a 4-way
> > P4, with and without HT:
> >
> > average of 10 kernel compiles with -j4 on 2.6.0-test9:
> >
> > HT disabled: Elapsed: 145.086s User: 513.808s System: 44.724s CPU: 384.5%
> > HT enabled: Elapsed: 172.463s User: 633.856s System: 48.003s CPU: 394.8%
>
> is that 4-way a 4-logical-way or 4-physical-way? If it's a 4-logical
> way, this workload is much closer to the best case than the worst case.
> I'm guessing a simple -j2 or -j3 should do much worse than that.

This is 4-way physical/4-way logical (no HT) vs 4-way physical/8-way logical 
(with HT)

Re: Wonderful World of Linux 2.6 - Final

[Andrea Arcangeli]

On Thu, Dec 18, 2003 at 05:58:55PM -0600, Andrew Theurer wrote:
> On Thursday 18 December 2003 17:52, Andrea Arcangeli wrote:
> > On Thu, Dec 18, 2003 at 09:29:46AM -0600, Andrew Theurer wrote:
> > > be scheduled optimally, for example, a kernel compile with -j4 on a 4-way
> > > P4, with and without HT:
> > >
> > > average of 10 kernel compiles with -j4 on 2.6.0-test9:
> > >
> > > HT disabled: Elapsed: 145.086s User: 513.808s System: 44.724s CPU: 384.5%
> > > HT enabled: Elapsed: 172.463s User: 633.856s System: 48.003s CPU: 394.8%
> >
> > is that 4-way a 4-logical-way or 4-physical-way? If it's a 4-logical
> > way, this workload is much closer to the best case than the worst case.
> > I'm guessing a simple -j2 or -j3 should do much worse than that.
> 
> This is 4-way physical/4-way logical (no HT) vs 4-way physical/8-way logical 
> (with HT)

ok then it was closer to the worst case ;)

Re: Wonderful World of Linux 2.6 - Final

[Nick Piggin]

I'll just mention that the "Hyperthreading" section is not entirely
accurate: the process scheduler is blissfully unaware of HT / SMT
presently. It is a must fix item though, and there are a number of
different implementations available to solve this.

I expect something will be merged before too long.

Nick

Re: Wonderful World of Linux 2.6 - Final

[Jeff Garzik]

Nick Piggin wrote:
> I'll just mention that the "Hyperthreading" section is not entirely
> accurate: the process scheduler is blissfully unaware of HT / SMT
> presently. It is a must fix item though, and there are a number of
> different implementations available to solve this.


Are you sure?  I could have sworn Ingo made the scheduler magically 
HT-friendly...

	Jeff

Re: Wonderful World of Linux 2.6 - Final

[Ingo Molnar]

On Thu, 18 Dec 2003, Jeff Garzik wrote:

> Are you sure?  I could have sworn Ingo made the scheduler magically
> HT-friendly...

nope, it's not in 2.6 yet. This area is still under development, with
various approaches being considered.

	Ingo

Re: Wonderful World of Linux 2.6 - Final

[Joe Pranevich]

On Thu, 2003-12-18 at 03:16, Ingo Molnar wrote:
> On Thu, 18 Dec 2003, Jeff Garzik wrote:
> 
> > Are you sure?  I could have sworn Ingo made the scheduler magically
> > HT-friendly...
> 
> nope, it's not in 2.6 yet. This area is still under development, with
> various approaches being considered.

Aaah.... Drat. I could have sworn that it was in and noone caught the
error before.

Oh well. Guess in this case I was over-optimistic.

Until I have the chance to fix it, just imagine that the WWOL document
you are reading is not the 'real' document, but rather a fluke of
quantum science. By some amazing process that can only be properly
performed by semi-sane science professionals in non-laboratory
environments, this copy of WWOL has somehow transcended the dimensional
barrier. It is, in effect, from a parallel universe. In *that* universe,
Ingo completed the necessary changes and merged them into 2.6 quite some
time ago. 

The butterfly effect being what it is, I'm sure there are other changes
in that universe. For example, flamingos may be blue or green. As the
color of Caribbean birds rarely influence kernel development, we should
be otherwise generally safe.

Joe

RE: Wonderful World of Linux 2.6 - Final

[Nakajima, Jun]

I agree with Nick. Ingo's HT-aware scheduler is not in the 2.6 base.
Also the current scheduler does not distinguish idle logical CPU vs.
idle package, having two active processes on the same package, leaving
the other package idle.

	Jun

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> owner@vger.kernel.org] On Behalf Of Jeff Garzik
> Sent: Wednesday, December 17, 2003 11:12 PM
> To: Nick Piggin
> Cc: Joe Pranevich; linux-kernel; mingo@redhat.com
> Subject: Re: Wonderful World of Linux 2.6 - Final
> 
> Nick Piggin wrote:
> > I'll just mention that the "Hyperthreading" section is not entirely
> > accurate: the process scheduler is blissfully unaware of HT / SMT
> > presently. It is a must fix item though, and there are a number of
> > different implementations available to solve this.
> 
> 
> Are you sure?  I could have sworn Ingo made the scheduler magically
> HT-friendly...
> 
> 	Jeff
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe
linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/