Re: [Xen-devel] Re: [PATCH 3/5] x86/pvclock: add vsyscall implementation

[Avi Kivity <avi@redhat.com>]

On 10/10/2009 02:24 AM, Jeremy Fitzhardinge wrote:
> On 10/07/09 03:25, Avi Kivity wrote:
>    
>> def try_pvclock_vtime():
>>    tsc, p0 = rdtscp()
>>    v0 = pvclock[p0].version
>>    tsc, p = rdtscp()
>>    t = pvclock_time(pvclock[p], tsc)
>>    if p != p0 or pvclock[p].version != v0:
>>       raise Exception("Processor or timebased change under our feet")
>>    return t
>>      
> This doesn't quite work.
>
> If we end up migrating some time after the first rdtscp, then the
> accesses to pvclock[] will be cross-cpu.  Since we don't made any strong
> SMP memory ordering guarantees on updating the structure, the snapshot
> isn't guaranteed to be consistent even if we re-check the version at the
> end.
>    

We only hit this if we have a double migration, otherwise we see p != p0.

Most likely all existing implementations do have a write barrier on the 
guest entry path, so if we add a read barrier between the two compares, 
that ensures we're reading from the same cpu again.

> So to use rdtscp we need to either redefine the update of
> pvclock_vcpu_time_info to be SMP-safe, or keep the additional migration
> check.
>    

I think we can update the ABI after verifying all implementations do 
have a write barrier.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.