Re: write_tsc in a PV domain?

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 08/27/09 20:29, Dan Magenheimer wrote:
>> If an app is sophisticated to do this correctly then it 
>> doesn't need any
>> special assistance from a hypervisor to make the tsc well-behaved.  It
>> should continue to work even in a Xen guest where both the process can
>> skip between VCPUs and the VCPUs can skip between PCPUs.
>>     
> No, I don't think this is true.  An enterprise app that binds processes
> to fixed physical processors on a physical machine can make
> assumptions about the results of rdtsc that aren't valid when
> the vcpus can skip between pcpus. 

You can bind a vcpu to a pcpu or group of pcpus with the right tsc
properties.  At this point you're talking about a specialized
non-portable app with very sensitive dependencies on the system software
and underlying hardware, so requiring some special effort to virtualize
it doesn't seem like a big problem.

>  Further, like Linux itself,
> applications may test assumptions about tsc at startup that are
> assumed to remain valid for the life of the app, which is
> perfectly reasonable on a physical machine and a bad mistake
> in a virtualized environment.
>   

Not really.  An app can't tell whether its initial test happened to be
in a stable period that will be later upset by a power event,
suspend/resume, migration via some other mechanism (like
vserver/containers), etc, etc.  An app making such assumptions will be
very machine and system dependent, and not at all portable.

> True, but any app that tries to run on a NUMA machine without
> being aware of the idiosyncracies of a NUMA machine probably
> has worse problems to deal with than tsc sync.  Further, there
> are many many apps that will likely never ever run on those
> machines.

Who can say?  Effects caused by locality issues will only result in
performance problems rather than outright correctness problems.

>   Are we going to penalize all apps all the time
> because some might run some of the time on a machine where
> tsc is not synced?
>   
They're already penalized.  The population of machines with a tsc which
can be used in the manner you're suggesting is very small, and even then
there are strong caveats.

>> But in this case I'm talking specifically about a Xen PV guest, where
>> the tsc is claimed for use by the Xen clocksource ABI.
>>     
> I just don't understand how you can say that a valid userland
> instruction is "claimed for use" by Xen (or Linux or both).
>   

Apps are free to try and use the tsc in any way they feel like, but it
has never had any guaranteed properties.  Some uses are completely
reasonable (like using it as some entropy to seed an RNG, for example). 
At one point the kernel did disable the tsc for usermode use, but that
was quickly reverted (or perhaps it never made it to mainline) because
its not for the kernel to break backwards compatibility for the sake of
second-guessing usermode.

I think this is getting a bit repetitive.

    J