Re: RE: Live migration fails due to c/s 20627

[Andre Przywara <andre.przywara@amd.com>]

Dan Magenheimer wrote:
>> -----Original Message-----
>> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org]
>> Sent: Tuesday, December 15, 2009 11:26 AM
>> To: Xu, Dongxiao
>> Cc: Dan Magenheimer; Keir Fraser; xen-devel@lists.xensource.com; Kurt
>> Hackel; Dugger, Donald D; Nakajima, Jun; Zhang, Xiantao
>> Subject: Re: Live migration fails due to c/s 20627
>>
>>
>> On 12/15/2009 09:24 AM, Xu, Dongxiao wrote:
>>> If CPU has rdtsc but no rdtscp, then the VM exec control bit in VMCS
>>> won't be turned on. Therefore if rdtscp instruction runs, 
>> it will encounter
>>> invalid op code directly but no VMEXIT.
>>>    
>> Ah, right.  You'd need to make that particular illegal 
>> instruction vmexit.
> 
> Or make ALL illegal instructions vmexit, decode, if rdtscp
> emulate it, else vmenter again.
But is it useful to emulate RDTSCP? I see two use cases for this 
instruction:
1) NUMA aware malloc:
You need to know the current node number _quickly_ to use the right 
bucket to take the memory from. You do not even want using a syscall for 
this, that's why getcpu in Linux is implemented as a vsyscall either 
using RDTSCP or LSL. If you emulate this, this will need a few thousand 
cycles.
2) Making sure TSC values are consistent:
By looking at the core ID you learn whether two consecutive RDTSCPs are 
from the same core and are thus reliable. If you loose a few thousand 
cycles with emulation, than the whole purpose of doing the RDTSCPs is in 
question, as your results would be spoiled due to the overhead.

These two issues are the main reason I refrained from implementing 
RDTSCP virtualization some months ago, as even virtualizing them 
introduces a slight overhead (MSR save/restore). As software seems to 
cope with not having this instruction (and using the perfectly 
virtualized lsl instruction, for instance), I thought the benefit would 
not justify the effort.

Dan, can you summarize the usage of RDTSCP emulation in PV? Honestly I 
got lost in all these threads..


Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632