From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Bader Subject: Re: Xen HVM regression on certain Intel CPUs Date: Wed, 27 Mar 2013 16:53:16 +0100 Message-ID: <515315EC.4030803@canonical.com> References: <51530F9F.10805@canonical.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7926014175920836288==" Return-path: In-Reply-To: <51530F9F.10805@canonical.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "xen-devel@lists.xensource.com" Cc: Konrad Rzeszutek Wilk , "H. Peter Anvin" List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============7926014175920836288== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enig8C6FFA0B96C5FF93CFBBEDA6" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8C6FFA0B96C5FF93CFBBEDA6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 27.03.2013 16:26, Stefan Bader wrote: > Recently I ran some experiments on newer hardware and realized that whe= n booting > any kernel newer or equal to v3.5 (Xen version 4.2.1) in 64bit mode wou= ld fail > to bring up any APs (message about CPU Stuck). I was able to normally b= isect > into a range of realmode changes and then manually drill down to the fo= llowing > commit: >=20 > commit cda846f101fb1396b6924f1d9b68ac3d42de5403 > Author: Jarkko Sakkinen > Date: Tue May 8 21:22:46 2012 +0300 >=20 > x86, realmode: read cr4 and EFER from kernel for 64-bit trampoline >=20 > This patch changes 64-bit trampoline so that CR4 and > EFER are provided by the kernel instead of using fixed > values. >=20 > From the Xen debugging console it was possible to gather a bit more dat= a which > pointed to a failure very close to setting CR4 in startup_32. On this p= articular > hardware the saved CR4 (about to be set) was 0x1407f0. >=20 > This would set two flags that somehow feel dangerous: PGE (page global = enable) > and SMEP (supervisor mode execution protection). SMEP turns out to be t= he main > offender and the following change allows the APs to start: >=20 > --- a/arch/x86/realmode/rm/trampoline_64.S > +++ b/arch/x86/realmode/rm/trampoline_64.S > @@ -93,7 +93,9 @@ ENTRY(startup_32) > movl %edx, %fs > movl %edx, %gs >=20 > - movl pa_tr_cr4, %eax > + movl $X86_CR4_SMEP, %eax > + notl %eax > + andl pa_tr_cr4, %eax > movl %eax, %cr4 # Enable PAE mode >=20 > # Setup trampoline 4 level pagetables >=20 > Now I am not completely convinced that this is really the way to go. Li= kely the > Xen hypervisor should not start up the guest with CR4 on the BP contain= ing those > flags. But maybe it still makes sense to mask some dangerous ones off i= n the > realmode code (btw, it seemed that masking the assignments in arch_setu= p or > setup_realmode did not work). >=20 > And finally I am wondering why the SMEP flag in CR4 is set anyway. My > understanding would be that this should only be done if cpuid[7].ebx ha= s bit7 > set. And this does not seem to be the case at least on the one box I wa= s doing > the bisection on. Seems that I was relying on the wrong source of information when checking= SMEP support. The cpuid command seems at fail. But /proc/cpuinfo reports it. S= o that at least explains where that comes from... sorry for that. >=20 > -Stefan >=20 >=20 >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >=20 --------------enig8C6FFA0B96C5FF93CFBBEDA6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJRUxXsAAoJEOhnXe7L7s6jXTQQAMj+R6lXcxJk4CT/BpsdH7qs Z+nNt50ClHRkhS9hdCJY33ZsoIjVlRm0s9YspPWMG7F5O9IAiG+Ghj0bKICJWs7H SNf3kBk9l7RKInrKvOn9lmrjO3Zdz0W9/pnZ0v+V7PLwN6itHXcX7Hbudni6WpNV YcbsmJ3w5P10FU1id8Pj3INDaX95PGNj28hM3+N+xIO1UT8wKWCJSRc6z4tORINs gzF08ne9gmPcHLW7R6dLjDCnhIdJtQBjPOdP2hu7Gm9YtLrC8P4gGLuhVynnQdzg f5aBAF40hbTveBqa5NbdYT8EtC/ZAH87jUoLQKSmjqCNqA1kS1EbgO6m0xPWEvWB Tgmvluo9tdCvvgG+YK4BrRasXNeZOJooQ9uPwK5+ij2E+bx0psNaUY/1kxCTYqZg Gx0h583x54DlIsxcbqLrf5oU2A9tBEW7xzCQkdiYxCAbbneaQA2vINGVJIXSrSpb uTMHYyuLgbK35NlBr3CeOkKXcZRT4Ih8+6Jjn4PEwYLL8M7yEB/35iBTP3cORZPY wldiJroHgPCIAV7JnyUzZkf+MO5kS/o0vXQ3jz8WMB+neN3i9CBRzIcbFBrHV+Kt fBj1mkhoXGQMe40sinw4LLoHonFMjDhq3IKonRs3DVBHe+p7N1ijBwOSrHWckTww nKPAIIj/6ZLmqGY2vfin =HWOg -----END PGP SIGNATURE----- --------------enig8C6FFA0B96C5FF93CFBBEDA6-- --===============7926014175920836288== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============7926014175920836288==--