From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matt Wilson Subject: Re: [PATCH] xen: schedule: allow dom0 vCPUs to be re-pinned when dom0_vcpus_pin is set Date: Mon, 10 Dec 2012 14:01:10 -0800 Message-ID: <20121210220108.GA11695@u109add4315675089e695.ant.amazon.com> References: <2614dd8be3a01247230c.1354687327@u109add4315675089e695> <50BF2574.6080702@citrix.com> <20121205155906.GA32088@u109add4315675089e695.ant.amazon.com> <50BF839002000078000AE3DF@nat28.tlf.novell.com> <20121205170618.GC32088@u109add4315675089e695.ant.amazon.com> <50C0850402000078000AE77F@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <50C0850402000078000AE77F@nat28.tlf.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: George Dunlap , Andrew Cooper , Keir Fraser , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On Thu, 6 Dec 2012 10:44:04 +0000, Jan Beulich wrote: > On Wed, 5 Dec 2012 09:06:20 -0800, Matt Wilson wrote: > > On Wed, 5 Dec 2012 16:25:36 +0000, Jan Beulich wrote: > > > On Wed, 5 Dec 2012 07:59:08 -0800, Matt Wilson wrote: > > > > > > > > If this is true, the existing is_pinned_vcpu() test is broken: > > > > > > > > #define is_pinned_vcpu(v) ((v)->domain->is_pinned || \ > > > > cpumask_weight((v)->cpu_affinity) == 1) > > > > > > > > It's && not ||. So if someone pins dom0 vCPUs to pCPUs 1:1 after boot, > > > > the MSR traps will suddenly start working. > > > > > > > > See commit: > > > > http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=cc0854dd > > > > > > I don't see what's wrong here. Certain things merely require the > > > pCPU that a vCPU runs on to be stable, which is what the test > > > above is for. > > > > Me either. That said, are you willing to Ack and commit my patch that > > started this thread? > > In no case without Andrew's concerns addressed. Beyond that, > I'd be hesitant to ack it as I'm myself suspecting side effects that > we don't want and/or aren't aware of, and in no case could I > commit it without Keir's ack. Jan, So today if I boot Xen without dom0_vcpus_pin set, dom0's vCPUs will be allowed to run on any pCPU. Xen will block attempts to write certain MSRs (MSR_AMD64_NB_CFG, MSR_FAM10H_MMIO_CONF_BASE and MSR_IA32_ENERGY_PERF_BIAS). The VCPUOP_get_physid subop of the vcpu_op hypercall will not return the initial APIC ID or ACPI ID for dom0. Also today, if I run "xl vcpu-pin 0 0", suddenly those MSR writes and the VCPUOP_get_physid hypercall will start working for vCPU 0. For what it's worth, only legacy XenoLinux-derived kernels appear to use this hypercall during SMP boot. Upstream Linux does not. I think that the real risk is in the XenoLinux SMP booting code on AMD processors where sometimes initial APIC ID != ACPI ID. If the CPU pinning changes, the ACPI ID to APIC ID mapping will be wrong. This broke PowerNow! when it ran in dom0. But PowerNow! is handled by the hypervisor now. So what's the real danger here? Andrew, your thoughts? Thanks, Matt