Re: [PATCH v4: kvm 2/4] Kill the confusing tsc_ref_khz and ref_freq variables.

[Jan Kiszka <jan.kiszka@web.de>]

[-- Attachment #1: Type: text/plain, Size: 4980 bytes --]

Zachary Amsden wrote:
> On 10/09/2009 10:47 AM, Jan Kiszka wrote:
>> Zachary Amsden wrote:
>>   
>>> On 10/09/2009 10:36 AM, Jan Kiszka wrote:
>>>     
>>>> Zachary Amsden wrote:
>>>>
>>>>       
>>>>> On 10/08/2009 01:18 PM, Jan Kiszka wrote:
>>>>>
>>>>>         
>>>>>> Zachary Amsden wrote:
>>>>>>
>>>>>>
>>>>>>           
>>>>>>> They are globals, not clearly protected by any ordering or locking,
>>>>>>> and
>>>>>>> vulnerable to various startup races.
>>>>>>>
>>>>>>> Instead, for variable TSC machines, register the cpufreq notifier
>>>>>>> and
>>>>>>> get
>>>>>>> the TSC frequency directly from the cpufreq machinery.  Not only
>>>>>>> is it
>>>>>>> always right, it is also perfectly accurate, as no error prone
>>>>>>> measurement
>>>>>>> is required.
>>>>>>>
>>>>>>> On such machines, when a new CPU online is brought online, it isn't
>>>>>>> clear what
>>>>>>> frequency it will start with, and it may not correspond to the
>>>>>>> reference, thus
>>>>>>> in hardware_enable we clear the cpu_tsc_khz variable to zero and
>>>>>>> make
>>>>>>> sure
>>>>>>> it is set before running on a VCPU.
>>>>>>>
>>>>>>> Signed-off-by: Zachary Amsden<zamsden@redhat.com>
>>>>>>> ---
>>>>>>>     arch/x86/kvm/x86.c |   26 ++++++++++++++++----------
>>>>>>>     1 files changed, 16 insertions(+), 10 deletions(-)
>>>>>>>
>>>>>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>>>>>> index 15d2ace..de4ce8f 100644
>>>>>>> --- a/arch/x86/kvm/x86.c
>>>>>>> +++ b/arch/x86/kvm/x86.c
>>>>>>> @@ -1326,6 +1326,8 @@ out:
>>>>>>>     void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
>>>>>>>     {
>>>>>>>         kvm_x86_ops->vcpu_load(vcpu, cpu);
>>>>>>> +    if (unlikely(per_cpu(cpu_tsc_khz, cpu) == 0))
>>>>>>> +        per_cpu(cpu_tsc_khz, cpu) = cpufreq_quick_get(cpu);
>>>>>>>         kvm_request_guest_time_update(vcpu);
>>>>>>>     }
>>>>>>>
>>>>>>> @@ -3061,9 +3063,6 @@ static void bounce_off(void *info)
>>>>>>>         /* nothing */
>>>>>>>     }
>>>>>>>
>>>>>>> -static unsigned int  ref_freq;
>>>>>>> -static unsigned long tsc_khz_ref;
>>>>>>> -
>>>>>>>     static int kvmclock_cpufreq_notifier(struct notifier_block *nb,
>>>>>>> unsigned long val,
>>>>>>>                          void *data)
>>>>>>>     {
>>>>>>> @@ -3072,14 +3071,11 @@ static int kvmclock_cpufreq_notifier(struct
>>>>>>> notifier_block *nb, unsigned long va
>>>>>>>         struct kvm_vcpu *vcpu;
>>>>>>>         int i, send_ipi = 0;
>>>>>>>
>>>>>>> -    if (!ref_freq)
>>>>>>> -        ref_freq = freq->old;
>>>>>>> -
>>>>>>>         if (val == CPUFREQ_PRECHANGE&&    freq->old>    freq->new)
>>>>>>>             return 0;
>>>>>>>         if (val == CPUFREQ_POSTCHANGE&&    freq->old<    freq->new)
>>>>>>>             return 0;
>>>>>>> -    per_cpu(cpu_tsc_khz, freq->cpu) = cpufreq_scale(tsc_khz_ref,
>>>>>>> ref_freq, freq->new);
>>>>>>> +    per_cpu(cpu_tsc_khz, freq->cpu) = freq->new;
>>>>>>>
>>>>>>>         spin_lock(&kvm_lock);
>>>>>>>         list_for_each_entry(kvm,&vm_list, vm_list) {
>>>>>>> @@ -3120,12 +3116,14 @@ static void kvm_timer_init(void)
>>>>>>>     {
>>>>>>>         int cpu;
>>>>>>>
>>>>>>> -    for_each_possible_cpu(cpu)
>>>>>>> -        per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
>>>>>>>         if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) {
>>>>>>> -        tsc_khz_ref = tsc_khz;
>>>>>>>            
>>>>>>> cpufreq_register_notifier(&kvmclock_cpufreq_notifier_block,
>>>>>>>                           CPUFREQ_TRANSITION_NOTIFIER);
>>>>>>> +        for_each_online_cpu(cpu)
>>>>>>> +            per_cpu(cpu_tsc_khz, cpu) = cpufreq_get(cpu);
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>> This doesn't build for !CONFIG_CPU_FREQ.
>>>>>>
>>>>>>
>>>>>>            
>>>>> And did it before?
>>>>>
>>>>>          
>>>> Yes, because cpufreq_get, which is undefined without CONFIG_CPU_FREQ,
>>>> did not exist so far. One may argue that this is a deficit of the
>>>> cpufreq API. However, it needs fixing.
>>>>
>>>>        
>>> I'll send a patch today.  I'm going with API deficient and will fix
>>> that, but this means I will also require a fix for kvm-kmod :(
>>>      
>> That won't be tricky, I will queue up a fix.
>>
>> BTW, is the KVM prepared for cpufreq_get returning 0?
>>    
> 
> That will be part of the fix, I think.
> 

That's good, specifically as it looks like cpufreq_get can return 0 even
in the CONFIG_CPU_FREQ case, at least theoretically.

Jan

PS:

diff --git a/external-module-compat-comm.h b/external-module-compat-comm.h
index 47fdc86..de8ab23 100644
--- a/external-module-compat-comm.h
+++ b/external-module-compat-comm.h
@@ -993,3 +993,10 @@ unsigned long kvm_vma_kernel_pagesize(struct vm_area_struct *vma)
 	}					\
 })
 #endif
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,33) && !defined(CONFIG_CPU_FREQ)
+static inline unsigned int cpufreq_get(unsigned int cpu)
+{
+	return 0;
+}
+#endif


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]