Re: [RFC patch 15/15] LTTng timestamp x86

Linus Torvalds
2008-10-23T15:49:38Z

On Tue, 21 Oct 2008, Bjorn Helgaas wrote:

> On Monday 20 October 2008 04:29:07 pm H. Peter Anvin wrote:
> > Linus Torvalds wrote:
> > >
> > > But it's not going to solve the "hey, I have 512 CPU's, they are all on
> > > different boards, and no, they are _not_ synchronized to one global
> > > clock!".
> > >
> > > That's why I'd suggest making _purely_ local time, and then aiming for
> > > something NTP-like. But maybe there are better solutions out there.
> >
> > At the same time, it would definitely be nice to encourage vendors of
> > large SMP systems to provide a common root crystal (frequency standard)
> > for a single SMP domain. Preferrably a really good one, TCXO or better.
>
> A single root crystal is nice for us software guys. But it often
> also turns into a single point of failure, which the hardware guys
> are always trying to eliminate. So I think multiple crystals are
> inevitable for the really large machines.

They are almost inevitable for another reason too: the interconnect seldom
has a concept of "clock signal" other than for the signalling itself, and
the signal clock is designed for the signal itself and is designed for
signal integrity rather than "stable clock".

Does _any_ common interconnect have integral support for clock
distribution?

And no, nobody is going to add another clock network for just clock
distribution.

So even ignoring redundancy issues, and the fact that people want to
hot-plug things (and yes, that would make a central clock interesting), I
doubt any hw manufacturer really looks at it the way we do.

The best we could hope for is some hardware assist for helping distribute
a common clock. Ie not a real single crystal, but having time-packets in
the interconnect that are used to synchronize nodes whenever there is
communication between them. It's hard to do that in software, because the
overhead is fairly high, but if hardware does at least some of it you
could probably get a damn fine distributed clock source.

But I don't know if any hw people are worried enough about it to do it...

Linus