Re: [RFC PATCH 1/3] Unified trace buffer

[Jeremy Fitzhardinge <jeremy@goop.org>]

Ingo Molnar wrote:
> * Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
>   
>> On Thu, 25 Sep 2008, Ingo Molnar wrote:
>>     
>>> You seem to dismiss that angle by calling my arguments bullshit, but 
>>> i dont know on what basis you dismiss it. Sure, a feature and extra 
>>> complexity _always_ has a robustness cost. If your argument is that 
>>> we should move cpu_clock() to assembly to make it more dependable - 
>>> i'm all for it.
>>>       
>> Umm. cpu_clock() isn't even cross-cpu synchronized, and has actually 
>> thrown away all the information that can make it so, afaik. At least 
>> the comments say "never more than 2 jiffies difference"). You do 
>> realize that if you want to order events across CPU's, we're not 
>> talking about "jiffies" here, we're talking about 50-100 CPU _cycles_.
>>     
>
> Steve got the _worst-case_ cpu_clock() difference down to 60 usecs not 
> so long ago. It might have regressed since then, it's really hard to do 
> it without cross-CPU synchronization.
>
> ( But it's not impossible, as Steve has proven it, because physical time
>   goes on linearly on each CPU so we have a chance to do it: by
>   accurately correlating the GTOD timestamps we get at to-idle/from-idle
>   times to the TSC. )
>
> And note that i'm not only talking about cross-CPU synchronization, i'm 
> also talking about _single CPU_ timestamps. How do you get it right with 
> TSCs via a pure postprocessing method? A very large body of modern CPUs 
> will halt the TSC when they go into idle. (about 70% of the installed 
> base or so)
>
> Note, we absolutely cannot do accurate timings in a pure 
> TSC-post-processing environment: unless you want to trace _every_ 
> to-idle and from-idle event, which can easily be tens of thousands of 
> extra events per seconds.
>
> What we could do perhaps is a hybrid method:
>
>  - save a GTOD+TSC pair at important events, such as to-idle and
>    from-idle, and in the periodic sched_tick(). [ perhaps also save it 
>    when we change cpufreq. ]
>
>  - save the (last_GTOD, _relative_-TSC) pair in the trace entry
>
> with that we have a chance to do good post-processed correlation - at 
> the cost of having 12-16 bytes of timestamp, per trace entry.
>
> Or we could upscale the GTOD to 'TSC time', at go-idle and from-idle. 
> Which is rather complicated with cpufreq - which frequency do we want to 
> upscale to if we have a box with three available frequencies? We could 
> ignore cpufreq altogether - but then there goes dependable tracing on 
> another range of boxes.
>   

The "full timestamp" records should include:

    * absolute tsc
    * absolute monotonic timestamp
    * new tsc freqency

If you then make sure that all the cpufreq/idle/suspend-resume code
emits appropriate records when changing the tsc frequency, then you
should always be able to fully regenerate an absolute timestamp.

If you generate the monotonic timestamp with a good clocksource, then
you should be able to correlate the timestamps between cpus.

Oddly enough, this is identical to the Xen clocksource's use of the tsc ;)

    J