From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Bader Subject: Re: Xen HVM regression on certain Intel CPUs Date: Wed, 27 Mar 2013 17:24:47 +0100 Message-ID: <51531D4F.5080900@canonical.com> References: <51530F9F.10805@canonical.com> <515315EC.4030803@canonical.com> <20130327160427.GB6688@phenom.dumpdata.com> <515319AF.1050900@zytor.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2770241483610022566==" Return-path: In-Reply-To: <515319AF.1050900@zytor.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "H. Peter Anvin" Cc: wei.y.yang@intel.com, "xen-devel@lists.xensource.com" , haitao.shan@intel.com, xin.li@intel.com, Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============2770241483610022566== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enig6F693F3AD35782BEAB27D391" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig6F693F3AD35782BEAB27D391 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 27.03.2013 17:09, H. Peter Anvin wrote: > On 03/27/2013 09:04 AM, Konrad Rzeszutek Wilk wrote: >>>> >>>> From the Xen debugging console it was possible to gather a bit more = data which >>>> pointed to a failure very close to setting CR4 in startup_32. On thi= s particular >>>> hardware the saved CR4 (about to be set) was 0x1407f0. >>>> >>>> This would set two flags that somehow feel dangerous: PGE (page glob= al enable) >>>> and SMEP (supervisor mode execution protection). SMEP turns out to b= e the main >>>> offender and the following change allows the APs to start: >>>> >>>> --- a/arch/x86/realmode/rm/trampoline_64.S >>>> +++ b/arch/x86/realmode/rm/trampoline_64.S >>>> @@ -93,7 +93,9 @@ ENTRY(startup_32) >>>> movl %edx, %fs >>>> movl %edx, %gs >>>> >>>> - movl pa_tr_cr4, %eax >>>> + movl $X86_CR4_SMEP, %eax >>>> + notl %eax >>>> + andl pa_tr_cr4, %eax >>>> movl %eax, %cr4 # Enable PAE mode >>>> >>>> # Setup trampoline 4 level pagetables >>>> >>>> Now I am not completely convinced that this is really the way to go.= Likely the >>>> Xen hypervisor should not start up the guest with CR4 on the BP cont= aining those >>>> flags. But maybe it still makes sense to mask some dangerous ones of= f in the >>>> realmode code (btw, it seemed that masking the assignments in arch_s= etup or >>>> setup_realmode did not work). >>>> >>>> And finally I am wondering why the SMEP flag in CR4 is set anyway. M= y >>>> understanding would be that this should only be done if cpuid[7].ebx= has bit7 >>>> set. And this does not seem to be the case at least on the one box I= was doing >>>> the bisection on. >>> >>> Seems that I was relying on the wrong source of information when chec= king SMEP >>> support. The cpuid command seems at fail. But /proc/cpuinfo reports i= t. So that >>> at least explains where that comes from... sorry for that. >> >> OK, so if you boot Xen with smep=3D1 (which disables SMEP, kind of cou= nterintuive flag) >> that would work fine? >> >> CC-ing the Intel folks who added this in. >> >=20 > If it is present in /proc/cpuinfo and not in cpuid it means the kernel > thinks it has SMEP but the CPU doesn't... an obvious case of fail. > However, *where the hell* does the bit come from in the first place? I did not yet have time to track down all sources but I thought that /proc/cpuinfo is in some way assembled from whatever cpuid info the kerne= l has. I am more suspicious of the cpuid command I was using. Let me check for x= 86info. >=20 > That is what we need to track down. >=20 > When you say Xen HVM, am I correct in assuming that neither CPUID nor > CR4 operations in the main kernel are run through paravirt_ops? Not paravirt ops likely but the hypervisor traps access. At least cpuid f= rom within a hvm guest I expect to be filtered. So when checking things I wen= t to bare-metal. Will fetch more info and get back. -Stefan >=20 > -hpa >=20 >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >=20 --------------enig6F693F3AD35782BEAB27D391 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJRUx1PAAoJEOhnXe7L7s6j7/8QANYiWlzvmzaXQtTFnb+sBM2O uFX2w28b5J5iKIkLBo7/vSqdk3h6xWXYDUJ9yhBXdfMInkKMbNY7MoINZvNOc0hM yrxeZ+jahv7uCMxxtqvdsu2y8qvtlEjvTuTLQEMt8D8dObG/TzQ1wIuw7D7bT1z6 zc4tNiFtY7foh9n1gH17mIQZWkN6erFJIPyZ1jZgJUzXG/gU6WAHZ02614GRnAfd JLlP3uWEflRFxI/wEHmDAm6c+z0VRwb2CFM9rJUPkSDWGzxnJGCr0CG4eaWEU1oP O35Y46zjtWbPvMWEuGLWl+OgBqZV7n7tB7CxsZuZuW9DIPOxW8TP/qgyEw0clmSC XFRq+ICKO97x3vwqH4vaZM50A1jdcasMA0Nnl43xmKfspThXXbflB8X4VE2Zyz4e yGXi8DA3WCzOBuejzxTVO6HlpJDmqGMX1fi6lsDXkVr57FqzN7wnH4fVFvdxkiV0 GeRGNjrE40Ee6xSKC5GaGwvlR2D5rQrI8MmHp62oJZa0MPVo5/Mn+JLAyvSzbEJP PW6iL+x+qh+E7Ay0s5DAKXAgzB1hs1GeB6SNrPzthGMK7l6F/dAIoGds2a5iyl0o MlSO1/YFadPgLTGggpZ0V5AzPGsixVKyEsXv/pU92ssVYHFudRxu8e3kxR6yz3uR DRP52x/lsaGqx5nyvOdk =b0tH -----END PGP SIGNATURE----- --------------enig6F693F3AD35782BEAB27D391-- --===============2770241483610022566== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============2770241483610022566==--