Re: [PATCH] Enabling IA32_TSC_ADJUST for guest VM

[Marcelo Tosatti <mtosatti@redhat.com>]

On Tue, Oct 09, 2012 at 04:30:28PM +0200, Avi Kivity wrote:
> On 10/09/2012 04:27 PM, Marcelo Tosatti wrote:
> > On Tue, Oct 09, 2012 at 04:26:32PM +0200, Avi Kivity wrote:
> >> On 10/09/2012 04:24 PM, Marcelo Tosatti wrote:
> >> > On Tue, Oct 09, 2012 at 02:12:18PM +0200, Avi Kivity wrote:
> >> >> On 10/08/2012 07:30 PM, Marcelo Tosatti wrote:
> >> >> > 
> >> >> > From Intel's manual:
> >> >> > 
> >> >> > • If an execution of WRMSR to the IA32_TIME_STAMP_COUNTER MSR adds (or
> >> >> > subtracts) value X from the TSC,
> >> >> > the logical processor also adds (or subtracts) value X from the
> >> >> > IA32_TSC_ADJUST MSR.
> >> >> > 
> >> >> > This is not handled in the patch. 
> >> >> > 
> >> >> > To support migration, it will be necessary to differentiate between
> >> >> > guest initiated and userspace-model initiated msr write. That is, 
> >> >> > only guest initiated TSC writes should affect the value of 
> >> >> > IA32_TSC_ADJUST MSR.
> >> >> > 
> >> >> > Avi, any better idea?
> >> >> > 
> >> >> 
> >> >> I think we need that anyway, since there are some read-only MSRs that
> >> >> need to be configured by the host (nvmx capabilities).  So if we add
> >> >> that feature it will be useful elsewhere.  I don't think it's possible
> >> >> to do it in any other way:
> >> >> 
> >> >> "Local offset value of the IA32_TSC for a
> >> >> logical processor. Reset value is Zero. A
> >> >> write to IA32_TSC will modify the local
> >> >> offset in IA32_TSC_ADJUST and the
> >> >> content of IA32_TSC, but does not affect
> >> >> the internal invariant TSC hardware."
> >> >> 
> >> >> What we want to do is affect the internal invariant TSC hardware, so we
> >> >> can't do that through the normal means.
> >> >> 
> >> >> btw, will tsc writes from userspace (after live migration) cause tsc
> >> >> skew?  If so we should think how to model a guest-wide tsc.
> >> > 
> >> > No because there is an easy shortcut:
> >> > 
> >> >     if (level == KVM_PUT_FULL_STATE) {
> >> >         /*
> >> >          * KVM is yet unable to synchronize TSC values of multiple VCPUs
> >> >          * on
> >> >          * writeback. Until this is fixed, we only write the offset to
> >> >          * SMP
> >> >          * guests after migration, desynchronizing the VCPUs, but
> >> >          * avoiding
> >> >          * huge jump-backs that would occur without any writeback at
> >> >          * all.
> >> >          */
> >> >         if (smp_cpus == 1 || env->tsc != 0) {
> >> >             kvm_msr_entry_set(&msrs[n++], MSR_IA32_TSC, env->tsc);
> >> >         }
> >> >     }
> >> 
> >> Still we write back after migration.  So this needs to be fixed (or I
> >> misunderstood you).
> > 
> > Handled by kvm_write_tsc() in x86.c. Is this what you mean?
> > 
> 
> It will generate a call to ->write_tsc_offset().  Will the values be the
> same for all vcpus?  Note the inputs won't be the same.

Yes:

         * Special case: TSC write with a small delta (1 second) of
         * virtual
         * cycle time against real time is interpreted as an attempt to
         * synchronize the CPU.