From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: [PATCH v2 00/14] Add VMX TSC scaling support Date: Fri, 11 Dec 2015 10:09:28 +0000 Message-ID: <566AA0D8.6090202@citrix.com> References: <1449435529-12989-1-git-send-email-haozhong.zhang@intel.com> <20151206211434.GA13913@hz-desktop.sh.intel.com> <56654B77.8030205@amazon.de> <20151207101624.GB3547@hz-desktop.sh.intel.com> <5665BC03.5080901@citrix.com> <20151210111324.GP3547@hz-desktop.sh.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , "Zhang, Haozhong" Cc: Keir Fraser , Jan Beulich , "Nakajima, Jun" , "Egger, Christoph" , "xen-devel@lists.xen.org" , Aravind Gopalakrishnan , Suravee Suthikulpanit , Boris Ostrovsky List-Id: xen-devel@lists.xenproject.org On 11/12/15 07:35, Tian, Kevin wrote: >> From: Zhang, Haozhong >> Sent: Thursday, December 10, 2015 7:13 PM >> >> On 12/10/15 10:43, Tian, Kevin wrote: >>>> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] >>>> Sent: Tuesday, December 08, 2015 1:04 AM >>>> >>>> On 07/12/15 10:16, Haozhong Zhang wrote: >>>>> On 12/07/15 10:03, Egger, Christoph wrote: >>>>>> Did you consider nested virtualization? >>>>>> L1 hypervisor may have a different tsc scaling >>>>>> and L2 guest again may have a different tsc scale ratio. >>>>>> >>>>> Oh, I forgot this. I'll check the nested TSC scaling code (mostly >>>>> about nested SVM TSC ratio, because this patch series does not expose >>>>> VMX TSC scaling to L1 guest). >>>>> >>>>> BTW, are there any practical usage scenarios of nested TSC scaling? If >>>>> any, I may also need to consider adding nested virtualization support >>>>> to VMX TSC scaling. >>>> I would say that there are genuine uses for nested TSC scaling. An L1 >>>> hypervisor is going to want to scale for the same reasons as the L0 >>>> hypervisor. >>>> >>>> Having said that, if TSC scaling is correctly hidden from the L1 guests, >>>> I would advise against conflating the two issues together. i.e. getting >>>> nested TSC scaling working is not a prerequisite for accepting this series. >>>> >>> Why is nested TSC scaling even an issue? If L0 hypervisors cross hosts >>> can always guarantee constant TSC frequency through TSC scaling, L1 >>> hypervisor will never see inconstant TSC frequency so why bother to >>> expose TSC scaling at all? >>> >> If exposing TSC scaling to L1, then L0 hypervisor will need to adapt >> the TSC scaling ratio set by L1 hypervisor, which has not been done by >> this patch series. >> >> Consider an example that the host TSC frequency is freq_0 and >> 1. L0 Xen sets TSC scaling ratio to a non-identical one ratio_0. >> >> 2. Then L1 hypervisor will observe a "host" TSC frequency (freq_0 * ratio_0). >> >> 3. If L1 hypervisor sets a TSC scaling ratio ratio_1, it intends to >> provide a guest TSC frequency (freq_0 * ratio_0 * ratio_1). >> >> However, if ratio_1 is directly written into nested VMCS that is >> later applied to the host CPU, then the L2 guest will get an >> incorrect guest TSC frequency (freq_0 * ratio_1). >> >> > My point is why we need expose TSC scaling to L1, if L0 has ensures > constant TSC w/ TSC scaling ratio... Because a set of L1 hypervisors might want to migrate VMs between them. Or a more interesting setup migrating VMs between L0 and L1 hypervisors. (I tried this once and it didn't blow up) Once constant TSC has been advertised to a guest, any future hypervisor it might find itself on must be able to guarantee to provide the same frequency. ~Andrew