Re: Scaling noise

Rob Landley
2003-09-09T14:48:24Z

On Monday 08 September 2003 09:38, Alan Cox wrote:
> On Sad, 2003-09-06 at 16:08, Pavel Machek wrote:
> > Hi!
> >
> > > Maybe this is a better way to get my point across. Think about more
> > > CPUs on the same memory subsystem. I've been trying to make this
> > > scaling point
> >
> > The point of hyperthreading is that more virtual CPUs on same memory
> > subsystem can actually help stuff.
>
> Its a way of exposing asynchronicity keeping the old instruction set.
> Its trying to make better use of the bandwidth available by having
> something else to schedule into stalls. Thats why HT is really good for
> code which is full of polling I/O, badly coded memory accesses but is
> worthless on perfectly tuned hand coded stuff which doesnt stall.

<rant>

I wouldn't call it worthless. "Proof of concept", maybe.

Modern processors (Athlon and P4 both, I believe) have three execution cores,
and so are trying to dispatch three instructions per clock. With
speculation, lookahead, branch prediction, register renaming, instruction
reordering, magic pixie dust, happy thoughts, a tailwind, and 8 zillion other
related things, they can just about do it too, but not even close to 100% of
the time. Extracting three parallel instructions from one instruction stream
is doable, but not fun, and not consistent.

The third core is unavoidably idle some of the time. Trying to keep four
cores bus would be a nightmare. (All the VLIW guys keep trying to unload
this on the compiler. Don't ask me how a compiler is supposed to do branch
prediction and speculative execution. I suppose having to recompile your
binaries for more cores isn't TOO big a problem these days, but the boxed
mainstream desktop apps people wouldn't like it at all.)

Transistor budgets keep going up as manufacturing die sizes shrink, and the
engineers keep wanting to throw transistors at the problem. The first really
easy way to turn transistors into performance are a bigger L1 cache, but
somewhere between 256k and one megabyte per running process you hit some
serious diminishing returns since your working set is in cache and your far
accesses to big datasets (or streaming data) just aren't going to be helped
by more L1 cache.

The other obvious way to turn transistors into performance is to build
execution cores out of them. (Yeah, you can also pipeline yourself to death
to do less per clock for marketing reasons, but there's serious diminishing
returns there too.) With more execution cores, you can (theoretically)
execute more instructions per clock. Except that keeping 3 cores busy out of
one instruction stream is really hard, and 4 would be a nightmare...

Hyperthreading is just a neat hack to keep multiple cores busy. Having
another point of execution to schedule instructions from means you're
guaranteed to keep 1 core busy all the time for each point of execution
(barring memory access latency on "branch to mars" conditions), and with 3
cores and 2 pointes of execution they can fight over the middle core, which
should just about never be idle when the system is loaded.

With hyperthreading (SMT, whatever you wanna call it), the move to 4 execution
cores becomes a no-brainer. (Keeping 2 cores busy from one instruction
stream is relatively trivial), and even 5 (since keeping 3 cores busy is a
solved problem, although it's not busy all the time, but the two threads can
fight for the extra core when they actually have something for it to do...)

And THAT is where SMT starts showing real performance benefits, when you get
to 4 or 5 cores. It's cheaper than SMP on a die because they can share all
sorts of hardware (not the least of which being L1 cache, and you can even
expand L1 cache a bit because you now have the working sets of 2 processes to
stick in it)...

Intel's been desperate for a way to make use of its transistor budget for a
while; manufacturing is what it does better than AMD< not clever processor
design. The original Itanic, case in point, had more than 3 instruction
execution cores in each chip: 3 VLIW, a HP-PA Risc, and a brain-damaged
Pentium (which itself had a couple execution cores)... The long list of
reasons Itanic sucked started with the fact that it had 3 different modes and
whichever one you were in circuitry for the other 2 wouldn't contribute a
darn thing to your performance (although it did not stop there, and in fact
didn't even slow down...)

Of course since power is now the third variable along with price/performance,
sooner or later you'll see chips that individually power down cores as they
go dormant. Possibly even a banked L1 cache; who knows? (It's another
alternative to clocking down the whole chip; power down individual functional
units of the chip. Dunno who might actually do that, or when, but it's nice
to have options...)

</rant>

In brief: hyper threading is cool.

> Its great feature is that HT gets *more* not less useful as the CPU gets
> faster..

Excution point 1 stalls waiting for memory, so execution point 2 gets extra
cores. The classic tale of overlapping processing and I/O, only this time
with the memory bus being the slow device you have to wait for...

Rob