From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: RE: Live migration fails due to c/s 20627 Date: Wed, 16 Dec 2009 16:41:23 +0100 Message-ID: <4B28FFA3.7060300@amd.com> References: <57c748fc-49ef-4f2f-9c98-92a1dc12116f@default> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <57c748fc-49ef-4f2f-9c98-92a1dc12116f@default> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer Cc: Jeremy Fitzhardinge , xen-devel@lists.xensource.com, kurt.hackel@oracle.com, "Dugger, Donald D" , "Xu, Dongxiao" , Keir Fraser , "Nakajima, Jun" , "Zhang, Xiantao" List-Id: xen-devel@lists.xenproject.org Dan Magenheimer wrote: >> -----Original Message----- >> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org] >> Sent: Tuesday, December 15, 2009 11:26 AM >> To: Xu, Dongxiao >> Cc: Dan Magenheimer; Keir Fraser; xen-devel@lists.xensource.com; Kurt >> Hackel; Dugger, Donald D; Nakajima, Jun; Zhang, Xiantao >> Subject: Re: Live migration fails due to c/s 20627 >> >> >> On 12/15/2009 09:24 AM, Xu, Dongxiao wrote: >>> If CPU has rdtsc but no rdtscp, then the VM exec control bit in VMCS >>> won't be turned on. Therefore if rdtscp instruction runs, >> it will encounter >>> invalid op code directly but no VMEXIT. >>> >> Ah, right. You'd need to make that particular illegal >> instruction vmexit. > > Or make ALL illegal instructions vmexit, decode, if rdtscp > emulate it, else vmenter again. But is it useful to emulate RDTSCP? I see two use cases for this instruction: 1) NUMA aware malloc: You need to know the current node number _quickly_ to use the right bucket to take the memory from. You do not even want using a syscall for this, that's why getcpu in Linux is implemented as a vsyscall either using RDTSCP or LSL. If you emulate this, this will need a few thousand cycles. 2) Making sure TSC values are consistent: By looking at the core ID you learn whether two consecutive RDTSCPs are from the same core and are thus reliable. If you loose a few thousand cycles with emulation, than the whole purpose of doing the RDTSCPs is in question, as your results would be spoiled due to the overhead. These two issues are the main reason I refrained from implementing RDTSCP virtualization some months ago, as even virtualizing them introduces a slight overhead (MSR save/restore). As software seems to cope with not having this instruction (and using the perfectly virtualized lsl instruction, for instance), I thought the benefit would not justify the effort. Dan, can you summarize the usage of RDTSCP emulation in PV? Honestly I got lost in all these threads.. Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448 3567 12 ----to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632