Re: [PATCH v14 20/25] x86/tsc: calibrate tsc only once

[Pavel Tatashin <pasha.tatashin@oracle.com>]

On 07/19/2018 07:01 AM, Thomas Gleixner wrote:
> On Thu, 19 Jul 2018, Peter Zijlstra wrote:
>> On Tue, Jul 17, 2018 at 10:22:06PM -0400, Pavel Tatashin wrote:
>>> During boot tsc is calibrated twice: once in tsc_early_delay_calibrate(),
>>> and the second time in tsc_init().
>>>
>>> Rename tsc_early_delay_calibrate() to tsc_early_init(), and rework it so
>>> the calibration is done only early, and make tsc_init() to use the values
>>> already determined in tsc_early_init().
>>>
>>> Sometimes it is not possible to determine tsc early, as the subsystem that
>>> is required is not yet initialized, in such case try again later in
>>> tsc_init().
>>
>> It might be nice to preserve some of the information tglx dug out during
>> review of all this. Like the various methods of calibrate_*() and their
>> dependencies.
>>
>> And I note that this patch relies on the magic of native_calibrate_cpu()
>> working really early and not exploding in the quick calibration run.
>> This either wants fixing or documenting.
>>
>> I think the initial idea was to only do the fast_calibrate (cpuid, msr
>> and possibly the quick_pit) things early and delay the HPET/PMTIMER
>> magic until later.
> 
> Yes. I really would prefer to have this as an explicit expressed mechanism
> rather than relying on magic variables not being initialized.

What is the best way to achieve this?

I did the following:

1367 static bool __init determine_cpu_tsc_frequencies(bool early)
1368 {
1369         /* Make sure that cpu and tsc are not already calibrated */
1370         WARN_ON(cpu_khz || tsc_khz);
1371                    
1372         if (early) {
1373                 cpu_khz = x86_platform.calibrate_cpu();
1374                 tsc_khz = x86_platform.calibrate_tsc();
1375         } else {
1376                 cpu_khz = hpet_pmtime_calibrate_cpu();
1377         }

 833 /**
 834  * native_calibrate_cpu - calibrate the cpu on boot
 835  */
 836 unsigned long native_calibrate_cpu(void)
 837 {
 838         unsigned long flags, fast_calibrate;
 839 
 840         fast_calibrate = cpu_khz_from_cpuid();
 841         if (fast_calibrate)
 842                 return fast_calibrate;
 843 
 844         fast_calibrate = cpu_khz_from_msr();
 845         if (fast_calibrate)
 846                 return fast_calibrate;
 847 
 848         local_irq_save(flags);
 849         fast_calibrate = quick_pit_calibrate();
 850         local_irq_restore(flags);
 851         if (fast_calibrate)
 852                 return fast_calibrate;
 853 
 854         return hpet_pmtime_calibrate_cpu();
 855 }


And hpet_pmtime_calibrate_cpu() contains all the hpet/pmtime stuff.

However, when cpu_khz = x86_platform.calibrate_cpu() is called the first time, we still call hpet_pmtime_calibrate_cpu() from native_calibrate_cpu(). We cannot simply split native_calibrate_cpu() into two independent functions because it is also called from recalibrate_cpu_khz().

So, the question is how to enforce that the first time we do not call hpet/pmtime?

1. Use a new global variable? Kind of ugly.
2. Use system_state == SYSTEM_BOOTING ? Ugly, and probably not very safe.

Any other suggestion?

Thank you,
Pavel