From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Bader Subject: Xen HVM regression on certain Intel CPUs Date: Wed, 27 Mar 2013 16:26:23 +0100 Message-ID: <51530F9F.10805@canonical.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0212043240814042128==" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "xen-devel@lists.xensource.com" Cc: "H. Peter Anvin" , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============0212043240814042128== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enigA82EED91597B97642375863F" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA82EED91597B97642375863F Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable Recently I ran some experiments on newer hardware and realized that when = booting any kernel newer or equal to v3.5 (Xen version 4.2.1) in 64bit mode would= fail to bring up any APs (message about CPU Stuck). I was able to normally bis= ect into a range of realmode changes and then manually drill down to the foll= owing commit: commit cda846f101fb1396b6924f1d9b68ac3d42de5403 Author: Jarkko Sakkinen Date: Tue May 8 21:22:46 2012 +0300 x86, realmode: read cr4 and EFER from kernel for 64-bit trampoline This patch changes 64-bit trampoline so that CR4 and EFER are provided by the kernel instead of using fixed values. =46rom the Xen debugging console it was possible to gather a bit more dat= a which pointed to a failure very close to setting CR4 in startup_32. On this par= ticular hardware the saved CR4 (about to be set) was 0x1407f0. This would set two flags that somehow feel dangerous: PGE (page global en= able) and SMEP (supervisor mode execution protection). SMEP turns out to be the= main offender and the following change allows the APs to start: --- a/arch/x86/realmode/rm/trampoline_64.S +++ b/arch/x86/realmode/rm/trampoline_64.S @@ -93,7 +93,9 @@ ENTRY(startup_32) movl %edx, %fs movl %edx, %gs - movl pa_tr_cr4, %eax + movl $X86_CR4_SMEP, %eax + notl %eax + andl pa_tr_cr4, %eax movl %eax, %cr4 # Enable PAE mode # Setup trampoline 4 level pagetables Now I am not completely convinced that this is really the way to go. Like= ly the Xen hypervisor should not start up the guest with CR4 on the BP containin= g those flags. But maybe it still makes sense to mask some dangerous ones off in = the realmode code (btw, it seemed that masking the assignments in arch_setup = or setup_realmode did not work). And finally I am wondering why the SMEP flag in CR4 is set anyway. My understanding would be that this should only be done if cpuid[7].ebx has = bit7 set. And this does not seem to be the case at least on the one box I was = doing the bisection on. -Stefan --------------enigA82EED91597B97642375863F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJRUw+gAAoJEOhnXe7L7s6jlnIP/2hN7YtS/lfRe2NX1dTW9atx Uj6XeIkuS79qSWU0RycEE1btBvUrNm+j/61w8fUynjUNeyUXWeHoNpQEHdfADgsz PUQW+oiOW710zJtJqXB8ARWmuznd6ALK4vQD9hmOSbY79WGV7Ven3A2nQEcAI6l8 95bttWdaePwimsPvPJpg5ii2FyGPQhDRhcvW7oshppYKvTP1Ccq09rwnZxeDY8Ef OBfmIkxLfTcvuXg2c7uXCOQ+OO48qfSI5mnvItvT0fQUtxwA9Nml70v9hA91oBgT acr8zMcLGbgXkkelu73/gI/PEVKG2XpWJso7moR11HBVbxR9lrLSKxtpVdf/luuz il6p276FC6oUj3p4tiuo/Dh1T1WIUhHRyaDhD05p53OKh0+UY4JWgWJugiIvOk1H YntzeYkAVHzmz9bWvOAGCwkB4+5EoinOVTaCtPtm6vqp34Y5WwzbgGakRXNCU8pi aRSv6WiZNW2vptu1RqDyYX1LnHeLzyBJQs5lQhLTzcrcqqfaFXHhabQ1M4zxuIMW WhUxwdfExvDoztegAd1Yo+C9uA3WJI2ZxKs/VDifwtgjGgloDsILlUrZIC53vRDy i4kVl7POrfM+emc0HdGByeLhFLFhn+rmIKB3NXyRPRiyhf8zAVGZIoALc8dQunho 9JmuP4LJAUbr3V9VlPLR =TySx -----END PGP SIGNATURE----- --------------enigA82EED91597B97642375863F-- --===============0212043240814042128== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0212043240814042128==--