Re: RE: rdtsc: correctness vs performance on Xen (and KVM?)

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 09/04/09 00:19, Jan Beulich wrote:
> struct pvclock_vcpu_time_info has a 'pad0' field afaics.
>   

Ah, yes, I was looking at wall_clock.  We could claim the padding for
"local version", but it would require a 64-bit unpreemptible read, which
is awkward on 32-bit.

>> We only need to worry about Xen preempting a kernel update rather than
>> the other way around.  I think it ends up being very simple:
>>
>> void ctxtsw_update_pvclock(struct pvclock_vcpu_time_info *pvclock)
>> {
>> 	BUG_ON(preemptible());	/* Switching VCPUs would be a disaster */
>>
>> 	/*
>> 	 * We just need to update version; if Xen did it behind our back, then
>> 	 * that's OK with us.  We should never see an update-in-progress because Xen
>> 	 * will always completely update the pvclock structure before rescheduling the
>> 	 * VCPU, so version should always be even.  We don't care if Xen updates the
>> 	 * timing parameters here because we're not in the middle of a clock read.
>> 	 * Usermode might be in the middle of a read, but all it needs to see is version
>> 	 * changing to a new even number, even if this add gets preempted by Xen in
>> 	 * the middle.  There are no cross-PCPU writes going on, so we don't need to
>> 	 * worry about bus-level atomicity.
>> 	 */
>> 	pvclock->version += 2;
>> }
>>     
> No, that won't work as-is, because you can't guarantee the compiler to
> translate this to and add-with-memory-operand. While avoiding a bus
> lock here indeed seems possible (as long as it is clear that user mode will
> never be interested in reading other than the instance of the CPU it's
> currently running on), you won't get away without inline assembly.
>   

I don't think that matters, even if the compiler generates a preemptable
sequence: the end result will always be a changed version number.  Even
if we end up rolling back the version to a smaller number (because Xen
did multiple pvclock updates while it preempted us) nothing will get
confused because nothing observed those intermediate versions.  Xen
itself doesn't care about the version number (its effectively
write-only).  KVM is the same.

    J