Re: RE: rdtsc: correctness vs performance on Xen (and KVM?)

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 09/01/09 15:08, Dan Magenheimer wrote:
>>> Just like what is being used to allow apps to get the CPU 
>>>       
>> number on native
>>     
>>> kernels (or the vCPU one on Xen-ified ones): Have a GDT 
>>>       
>> entry the limit of
>>     
>>> which is the number you want, and have the app use the lsl 
>>>       
>> instruction to
>>     
>>> get at it.
>>>       
>> Yes, that's true. Xen could provide such a segment descriptor 
>> in its private
>> area of the GDT. The issue then would be that, in a compound pvclock
>> operation spanning multiple machine instructions, the pCPU 
>> number revealed
>> by the LSL instruction can be stale by the time it is used 
>> later in the
>> compound operation.
>>     
> The algorithm could check the pCPU number before and after
> reading the pvclock data and doing the rdtsc, and if they
> don't match, start again.  (Doesn't the pvclock algorithm
> already do that with some versioning number in the pvclock
> data itself to ensure that the rest of the data didn't
> change while it was being read?)
>   
There's still a race there, if the thread switched PCPU twice during the
operation:

    <running on PCPU A>
    get CPU #
    <switch to PCPU B>
    read tsc
    apply corrections from (from PCPU A)
    <switch to PCPU A>
    check CPU # is the same as we started with: all OK!

note that the <switch to PCPU B> could either be a result of the Xen
scheduler moving the VCPU *or* the Linux scheduler moving the thread to
a different VCPU.  In the former case, Xen could update a version
counter to help detect the discontinuity, but it doesn't really know
about guest scheduling decisions.  I guess the guest kernel could update
the pvclock version counter itself.

> I'm clueless about GDTs and the LSL instrution so would
> need some help prototyping this.
>   

It's what vsyscall already uses.  Your scheme is precisely analogous to
what's already there.

    J