From: Xen.org security team <security@xen.org>
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
xen-users@lists.xen.org, oss-security@lists.openwall.com
Cc: "Xen.org security team" <security-team-members@xen.org>
Subject: Xen Security Advisory 263 (CVE-2018-3639) - Speculative Store Bypass
Date: Mon, 21 May 2018 21:00:33 +0000 [thread overview]
Message-ID: <E1fKruz-00065r-CD@xenbits.xenproject.org> (raw)
[-- Attachment #1: Type: text/plain, Size: 14156 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Xen Security Advisory CVE-2018-3639 / XSA-263
Speculative Store Bypass
ISSUE DESCRIPTION
=================
Contemporary high performance processors may use a technique commonly
known as Memory Disambiguation, whereby speculative execution may
proceed past unresolved stores. This opens a speculative sidechannel in
which loads from an address which have had a recent store can observe
and operate on the older, stale, value.
For more details, see:
https://bugs.chromium.org/p/project-zero/issues/detail?id=1528
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00115.html
https://www.amd.com/securityupdates
IMPACT
======
An attacker who can locate or create a suitable code gadget in a
different privilege context may be able to infer the content of
arbitrary memory accessible to that other privilege context.
At the time of writing, there are no known vulnerable gadgets in the
compiled hypervisor code. Xen has no interfaces which allow JIT code
to be provided. Therefore we believe that the hypervisor itself is
not vulnerable. Additionally, we do not think there is a viable
information leak by one Xen guest against another non-cooperating
guest.
However, in most configurations, within-guest information leak is
possible. Mitigation for this generally depends on guest changes (for
which you must consult your OS vendor) *and* on hypervisor support,
provided in this advisory.
VULNERABLE SYSTEMS
==================
Systems running all versions of Xen are affected.
Processors from all vendors are affected to different extents.
Further communication will be made for Arm. See
https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability
for more details.
MITIGATION
==========
This issue can be mitigated with a combination of software and firmware
changes.
RESOLUTION
==========
This is a hardware bug. The primary mitigation in Xen context is
modification of guests, especially JITs in guests, to avoid generating
vulnerable code. Such modifications do not require support from Xen.
Alternatively, the following patches provide some workarounds:
On AMD hardware, for Fam15h processors and later, the patches offer a
host-wide global control for whether Memory Disambiguation is enabled
(default) or disabled. Controls are not virtualised for guests. When
the global control is set to disabled (`spec-ctrl=ssbd' on the
hypervisor command line), the vulnerability is eliminated without the
need for other guest or hypervisor changes.
On Intel hardware, a microcode update is required in order to work
around the problem by disabling memory disambiguation. Consult your
hardware vendor or your dom0 OS distributor for the firmware/microcode
update. With the microcode update in place, the patches offer a
host-wide control (which would eliminate the vulnerability on the
whole system without guest changes), and virtualised controls for
guests to use (which addresses the issue in a guest-specific manner).
Consult your guest operating system vendors, for further information
and advice.
(Additionally, host firmware may be vulnerable and may require updates
for that reason. Consult your hardware vendor.)
xsa263-unstable/*.patch xen-unstable
xsa263-4.10/*.patch Xen 4.10.x
xsa263-4.9/*.patch Xen 4.9.x
xsa263-4.8/*.patch Xen 4.8.x
xsa263-4.7/*.patch Xen 4.7.x
xsa263-4.6/*.patch Xen 4.6.x
$ sha256sum xsa263* xsa263*/*
0751367b3e92580514297392292e2705c817f75a3553463feaee7d6ed769f12b xsa263.meta
2143d7801db550b693abb8e1fd16bee186a92e79ae33bfe9bef613334dffa7f3 xsa263-unstable/0001-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
ef9c36d50dfdf34fa65aa5195d24af09a86f117ba8ed3655dad017d44668cd6b xsa263-unstable/0002-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
bc44d297e2ae51deefd18bfa1990ac5081aa0dcfd45f5ce3452b917aff7f0915 xsa263-unstable/0003-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
92451b6d7e0e98f96fad7de78fab8496cbbc18447fb8044a1dece8a8d5d44562 xsa263-4.6/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch
201adebebce630db211d369d92534e33411f92ad8a809f5c778f31c3cfe8a716 xsa263-4.6/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch
7e487e6927e9d0acbbde65a1126f8f7f020007ebd2d5c41f9b4b3c56df2a7db2 xsa263-4.6/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch
2cb64b3ca057f7e37eafe8afa69eed58aa307333bd905549d685d3f0887974d8 xsa263-4.6/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch
e3d4f0280a7c1ef901f4cf3686b237139dfdfdb161b858e310b559c534c8ee3a xsa263-4.6/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch
988b261e4feb3706c21b8587b06db8fd36707707c8f79edfaaa9a8354c839eea xsa263-4.6/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch
c8ad66195aba972a1d20d8fd304f79b9a2f1f9bc71b154cd340ef2a9525f678b xsa263-4.6/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch
67f4a26aa92161ab46e369d9ef55f76581d7657520cf363384e6c11c50e03104 xsa263-4.6/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch
a72acc1142641f8a7b4d0ed44fa85bcaf766f0ad240749ab7631698639db859c xsa263-4.6/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch
d66d809b3255fd039ecb8a5500370d3e6ef17992298ed68de9a785df26796272 xsa263-4.6/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch
b6779decba081261c4f600aab4d732d170f2d1dae8b4e694c905dc6c588ac54c xsa263-4.6/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
79e9eca02880bdb30f17dd7d90b5cbfb8d743abc637335b5920edc91bdb01f33 xsa263-4.6/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
700a98e8f3251664349abd76ba28365017f6e7be75cea3008c7eb110e911ca58 xsa263-4.6/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
1805a7394ddee85c94adf3e8513a41b3f269e3d0fbb2a26bb1ae8d6fbbef7148 xsa263-4.7/0001-x86-Fix-x86-further-CPUID-handling-adjustments.patch
082e868801a1435bd8e68013aa799433f3c2688ea77a19644dcc3af4fe36c2e2 xsa263-4.7/0002-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch
1136be258338ae27b7439635ae745a4cb0a3b5a595fbff4c3d1e75a9d1258889 xsa263-4.7/0003-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch
dc6e236b5d846f108ffba44bd10146caf78f7b843f51f98f514d1b07683a4a87 xsa263-4.7/0004-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch
7a92156ade9f658bce7f34b0ecb797f3087a92bd41048b492bf09cea8fc4c40d xsa263-4.7/0005-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch
0785e01b3060e12e72f489dc7851cdbc35f346610a66137a815b4d0b4dc7f1ce xsa263-4.7/0006-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch
0e2b652d6edc9b769ab1c7fffc405ae3794332ff1aea89654b9dadeea117f519 xsa263-4.7/0007-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch
335be11dc9fc805851c67336ed2809b828b9e37a0789e0ea41c56a35a4212b10 xsa263-4.7/0008-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch
3e0481c5d52154653abbace60b7c3821fe761f51854e4e8a4a20edbf0e1f9ec9 xsa263-4.7/0009-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch
d17b526ec320683283ce0547b9fe8c127d43ae7de0c246aa7ec114e75edf2f45 xsa263-4.7/0010-x86-cpuid-Improvements-to-guest-policies-for-specula.patch
59f7911ea86e2128525eb1f0064a611a4bb4feb44cdd91a47b3038759ec79a0f xsa263-4.7/0011-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch
a7825bfa4f96128b3671544da90dfcc25e711cc01f243e1703edd10c60c1faab xsa263-4.7/0012-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
7deae902ebdaada1293dee6549d8cd0408e2c98827c59a5628a0829b17fa7c61 xsa263-4.7/0013-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
b38cf27b16d81d4038a2428acc8a08eb45dd0dcd42df1a691187d3691f56869b xsa263-4.7/0014-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
92c35fc3324bf32dbee3b1a34f008b8212b5bce32cb87109fec4f129d15de90e xsa263-4.8/0001-x86-Fix-x86-further-CPUID-handling-adjustments.patch
535e3fe4e9c27a683ffead7b2fde65b1ffb3231d451dc33c96f3ddc88408f5ef xsa263-4.8/0002-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch
8babb75ee944188a8662aff6fa6e753206a6ab4fc51451b02ee953c963cd8f0f xsa263-4.8/0003-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch
c4712e00101b4b3f1d49810e47790a80661b9c29ae1d6afc41736b1d65d69eeb xsa263-4.8/0004-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch
101c46174e1f977473a7e32a1e7c89728427cd0831aeaf276c318d2a17c7eee5 xsa263-4.8/0005-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch
e74a0049fb9da19203c9187cfd869203149b7c99480e857916ae95ebdc683132 xsa263-4.8/0006-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch
569f8a8e1e0d9ad066b650d2a46a45a156719d5846112da248853263d3002dd2 xsa263-4.8/0007-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch
9c79be6dc3ef9d281c5b676f7d03e44a9308204149a76ade48e76e63aabfa820 xsa263-4.8/0008-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch
8b5ec69f7ebdda4dd0d8c81cd4c11d75417f7201c03dc79348b35987da3fb209 xsa263-4.8/0009-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch
1b25684bd0c15caff188eb2a3c38f8f72320a63e9de437cfa1ecb759a89e6d54 xsa263-4.8/0010-x86-cpuid-Improvements-to-guest-policies-for-specula.patch
ad8d1552930d3c48dca7d26efde668cf3341cfdef717be6a6b71fc9c1cc1b667 xsa263-4.8/0011-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch
1394f75feeff17fb1d7b4eb6b6d9f18a123886d9178fca060f7b4f69618a5edc xsa263-4.8/0012-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
f63f5a060985c55bc4002d6f949f91c39dcf8fe339efc7133d21debea269bb16 xsa263-4.8/0013-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
a67c16cee885a529182f8f12941077551eb8398f9d8c2f870bea11409b98de8d xsa263-4.8/0014-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
491b2bf3cd6cf0da0cfca0d3a9329a735652d3eb9a89819e45ab328ed1bfd2b9 xsa263-4.9/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch
c7e04d812284592aa7c97ef240ca5603e8998d6b2be3861f1ed40f6ad767f111 xsa263-4.9/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch
85f98f5119c5384779254633bc3cdedc639093e7e9dfa45e3ebc165f656619a9 xsa263-4.9/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch
9927b85433720e9acfc52316c85bc4a5fb9eab240d3acc358a675b3a38aa9f09 xsa263-4.9/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch
8714566a3aa99d0fe3a105637f4c87b1ac451437bd827ed85e74e5f4ba9aadc5 xsa263-4.9/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch
317ba3977dab37070c793f5fa3bec6e45cf4f6dfda8fa99a3bd43a769da8ee18 xsa263-4.9/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch
2f717ed9174d47d419d218ffb82c4d3ba8af7fbc602fdece526bb013c6e5376b xsa263-4.9/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch
265ad11325438064487c9349dc29de9cdebc14503a17234945eb36dd66050255 xsa263-4.9/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch
2174e399924c7a2fcee49466bf87a217d2c61563f7ffd1d7f76e3bbea32fead4 xsa263-4.9/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch
a9fe0cbfcd3d77da2e25b410a4dbad115f11368e61854ebcd3306c15ce71e5ef xsa263-4.9/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch
37cac5bf8e9cd66716f3cb2e40fe5c80726edc4aab50863d6cb28e72ed1286ac xsa263-4.9/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
c559af676460fe3de383ba8df5f004a19a7031e97bb69e36a0c3c550887de404 xsa263-4.9/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
f6aec439eb0f00191cb43b910cbf4c4291e4bc7311bdf9d55c839e8fd1075357 xsa263-4.9/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
740f2819e346dadeeae25e04b40dc6cc447c7123a8fe0cd1064a1efa8301b4c4 xsa263-4.10/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch
46e42c295b4947721fe47085e8a37523756c0b9383da90c22aa0682ad04599b6 xsa263-4.10/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch
ed38748b15ad70869c5bf2b1eea91ddbb0a9e1f63f66614259320258e1f523e6 xsa263-4.10/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch
18a5cadfbb1cb7045c6efe4a526eb31aacbef995b57342afe544aede972e890f xsa263-4.10/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch
68ffb02ec8aa5f4c677e4debaf65685f94aae0661f74b9dee3eef16cd73a6024 xsa263-4.10/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch
85843f6c987ce465e8c87900e0ae368bf7cb7d8da64371b7e44df8907527b27c xsa263-4.10/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch
d5da58489adf87228391436dfd1f01c47b312bfef93f207f988e6b50d137d5d9 xsa263-4.10/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch
4c6a28b834b79a0c2bfcde8c9fc2246c7ff9b0c90b6d3bac221a53c51afc6c99 xsa263-4.10/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch
414c5d2f888a77077e168c4fca6f75be4de42a28a5927eb9f5f9a5a5c0bdf723 xsa263-4.10/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch
6fc2508d7f9183faf83eb1931e415663d32d8df12d3ab021d73616e2de54caa6 xsa263-4.10/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch
3f83546f98d98853ba7c6afd0eb23fe9a7cbceba0f8acea32e9fe466d86bf56c xsa263-4.10/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch
93e7b786d1224f672cc79c753e83eda607e02c8c9414ec926c532a92c0e6c70a xsa263-4.10/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch
d15756d90fc911a6f5ba28c2b84b5d3a6f3b864e85b1cf4c0fe82f8482af981d xsa263-4.10/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch
$
NOTE REGARDING LACK OF EMBARGO
==============================
We understand that despite an attempt to organise predisclosure, the
discoverers ultimately did not authorise a predisclosure.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCAAGBQJbAvkgAAoJEIP+FMlX6CvZJSYIAJuxYhP2DoGxRvBbmmx/70NN
v1EFr+bBH51kMby79UNYKMR5SnZ5yHgjO0f76t9KnQlN3LUkOs9zLEfiyPebmI5I
fTACRG8/I8IRDUzzlWlYuEwLDqXBzJn5gvrMIXVz7/K45yOdVDih8GudI0kfDmT+
z3MulyzkYr6+epGfQIxJuo/qfmfBbLJR6RX6QEtF/Y2NEpdLV3zxuFdx3QN2A5Am
s2ephgBJmhMHj+s9GvyUWkry22QaZLVnamCKK5NBOKUuu3Ha8yCVf5u51+9LoXL7
UWVqbQiQChJMr1L5ij/IcHLgf6gDHxCrCvfRwF7hXcRl0EimMQlZNkQwMy/hrgM=
=wPqD
-----END PGP SIGNATURE-----
[-- Attachment #2: xsa263.meta --]
[-- Type: application/octet-stream, Size: 1576 bytes --]
{
"XSA": 263,
"SupportedVersions": [
"master",
"4.10",
"4.9",
"4.8",
"4.7",
"4.6"
],
"Trees": [
"xen"
],
"Recipes": {
"4.10": {
"Recipes": {
"xen": {
"StableRef": "a0355180b660b149f8054b9facdd9cac8ec86a95",
"Prereqs": [],
"Patches": [
"xsa263-4.10/*.patch"
]
}
}
},
"4.6": {
"Recipes": {
"xen": {
"StableRef": "12b9fca6046741ffcda9eb3320f47093ed5d9ef0",
"Prereqs": [],
"Patches": [
"xsa263-4.6/*.patch"
]
}
}
},
"4.7": {
"Recipes": {
"xen": {
"StableRef": "ce22cc35df523db025983f303c201d9cef6179db",
"Prereqs": [],
"Patches": [
"xsa263-4.7/*.patch"
]
}
}
},
"4.8": {
"Recipes": {
"xen": {
"StableRef": "197e605e03a1017e2b4fb57859456da8f9cea468",
"Prereqs": [],
"Patches": [
"xsa263-4.8/*.patch"
]
}
}
},
"4.9": {
"Recipes": {
"xen": {
"StableRef": "74fa9552c1e3ef79bd4db0a67fc538bbd61b7561",
"Prereqs": [],
"Patches": [
"xsa263-4.9/*.patch"
]
}
}
},
"master": {
"Recipes": {
"xen": {
"StableRef": "f097a3a84221b0ad2848a1368ac9932180739642",
"Prereqs": [],
"Patches": [
"xsa263-unstable/*.patch"
]
}
}
}
}
}
[-- Attachment #3: xsa263-unstable/0001-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4400 bytes --]
From dec86a0c3cc6ceb947fcc8baad2c0f5539483b1d Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index b6b1530..da570b4 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1757,7 +1757,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1801,6 +1801,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index fc9677f..458a3fe 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -9,6 +9,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -594,6 +595,25 @@ static void init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
/* MFENCE stops RDTSC speculation */
if (!cpu_has_lfence_dispatch)
__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 3373369..20b215e 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool __read_mostly opt_ibpb = true;
+bool __read_mostly opt_ssbd = false;
bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -180,6 +181,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 993b958..91bed1b 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern bool opt_ssbd;
extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #4: xsa263-unstable/0002-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 10231 bytes --]
From 6f41a038c9471cbb118213329afa764099da323f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
tools/misc/xen-cpuid.c | 1 +
xen/arch/x86/cpuid.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/msr-index.h | 2 ++
xen/include/public/arch-x86/cpufeatureset.h | 1 +
xen/tools/gen-cpuid.py | 17 +++++++++++++----
8 files changed, 42 insertions(+), 12 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index da570b4..8712a83 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -489,9 +489,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1782,7 +1783,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1804,7 +1805,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 3a21f4e..7b0f594 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -205,6 +205,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
{"arch-caps", 0x00000007, 0, CPUID_REG_EDX, 29, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"lahfsahf", 0x80000001, NA, CPUID_REG_ECX, 0, 1},
{"cmplegacy", 0x80000001, NA, CPUID_REG_ECX, 1, 1},
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index adc7fce..e116339 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -144,6 +144,7 @@ static const char *str_7d0[32] =
[26] = "ibrsb", [27] = "stibp",
/* 28 */ [29] = "arch_caps",
+ /* 30 */ [31] = "ssbd",
};
static struct {
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 827b6c5..4b8d330 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -43,6 +43,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 20b215e..7db6e51 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -208,26 +208,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
printk(" Compiled-in support: INDIRECT_THUNK\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -496,6 +501,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 6d94d65..8fbccc8 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -38,6 +38,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -46,6 +47,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 8da5783..7acf822 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -245,6 +245,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 6359afb..3fecae8 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -261,10 +261,19 @@ def crunch_numbers(state):
AVX512BW, AVX512VL, AVX512VBMI, AVX512_4VNNIW,
AVX512_4FMAPS, AVX512_VPOPCNTDQ],
- # Single Thread Indirect Branch Predictors enumerates a new bit in the
- # MSR enumerated by Indirect Branch Restricted Speculation/Indirect
- # Branch Prediction Barrier enumeration.
- IBRSB: [STIBP],
+ # The features:
+ # * Single Thread Indirect Branch Predictors
+ # * Speculative Store Bypass Disable
+ #
+ # enumerate new bits in MSR_SPEC_CTRL, which is enumerated by Indirect
+ # Branch Restricted Speculation/Indirect Branch Prediction Barrier.
+ #
+ # In practice, these features also enumerate the presense of
+ # MSR_SPEC_CTRL. However, no real hardware will exist with SSBD but
+ # not IBRSB, and we pass this MSR directly to guests. Treating them
+ # as dependent features simplifies Xen's logic, and prevents the guest
+ # from seeing implausible configurations.
+ IBRSB: [STIBP, SSBD],
}
deep_features = tuple(sorted(deps.keys()))
--
2.1.4
[-- Attachment #5: xsa263-unstable/0003-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 2709 bytes --]
From 1769f77d9f0acea18de8ae740fce86dee92d4020 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr(), and offer SSBD to guests by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/msr.c | 8 ++++++--
xen/include/public/arch-x86/cpufeatureset.h | 2 +-
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index da9aa59..1e12ccb 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -197,6 +197,8 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
switch ( msr )
{
+ uint64_t rsvd;
+
case MSR_INTEL_PLATFORM_INFO:
case MSR_ARCH_CAPABILITIES:
/* Read-only */
@@ -232,8 +234,10 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
* Note: SPEC_CTRL_STIBP is specified as safe to use (i.e. ignored)
* when STIBP isn't enumerated in hardware.
*/
+ rsvd = ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (cp->feat.ssbd ? SPEC_CTRL_SSBD : 0));
- if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( val & rsvd )
goto gp_fault; /* Rsvd bit set? */
vp->spec_ctrl.raw = val;
@@ -252,12 +256,12 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
case MSR_INTEL_MISC_FEATURES_ENABLES:
{
- uint64_t rsvd = ~0ull;
bool old_cpuid_faulting = vp->misc_features_enables.cpuid_faulting;
if ( !vp->misc_features_enables.available )
goto gp_fault;
+ rsvd = ~0ull;
if ( dp->plaform_info.cpuid_faulting )
rsvd &= ~MSR_MISC_FEATURES_CPUID_FAULTING;
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 7acf822..c721c12 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -245,7 +245,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
-XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
+XEN_CPUFEATURE(SSBD, 9*32+31) /*A MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
--
2.1.4
[-- Attachment #6: xsa263-4.6/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch --]
[-- Type: application/octet-stream, Size: 3819 bytes --]
From 8ce1ac5a5b0703b441b6c2e1a3496f3fd6f9d245 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 12:21:00 +0100
Subject: [PATCH] x86/spec_ctrl: Read MSR_ARCH_CAPABILITIES only once
Make it available from the beginning of init_speculation_mitigations(), and
pass it into appropriate functions. Fix an RSBA typo while moving the
affected comment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d6c65187252a6c1810fd24c4d46f812840de8d3c)
---
xen/arch/x86/spec_ctrl.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1621c17..89c6f7d 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -81,18 +81,15 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
-static void __init print_details(enum ind_thunk thunk)
+static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
unsigned int _7d0 = 0, e8b = 0, tmp;
- uint64_t caps = 0;
/* Collect diagnostics about available mitigations. */
if ( boot_cpu_data.cpuid_level >= 7 )
cpuid_count(7, 0, &tmp, &tmp, &tmp, &_7d0);
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- if ( _7d0 & cpufeat_mask(X86_FEATURE_ARCH_CAPS) )
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
@@ -124,7 +121,7 @@ static void __init print_details(enum ind_thunk thunk)
}
/* Calculate whether Retpoline is known-safe on this CPU. */
-static bool_t __init __maybe_unused retpoline_safe(void)
+static bool_t __init __maybe_unused retpoline_safe(uint64_t caps)
{
unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
@@ -135,19 +132,12 @@ static bool_t __init __maybe_unused retpoline_safe(void)
boot_cpu_data.x86 != 6 )
return 0;
- if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
- {
- uint64_t caps;
-
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
-
- /*
- * RBSA may be set by a hypervisor to indicate that we may move to a
- * processor which isn't retpoline-safe.
- */
- if ( caps & ARCH_CAPS_RSBA )
- return 0;
- }
+ /*
+ * RSBA may be set by a hypervisor to indicate that we may move to a
+ * processor which isn't retpoline-safe.
+ */
+ if ( caps & ARCH_CAPS_RSBA )
+ return 0;
switch ( boot_cpu_data.x86_model )
{
@@ -217,6 +207,10 @@ void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
bool_t ibrs = 0;
+ uint64_t caps = 0;
+
+ if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
+ rdmsrl(MSR_ARCH_CAPABILITIES, caps);
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -244,7 +238,7 @@ void __init init_speculation_mitigations(void)
* On Intel hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe() )
+ else if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else
#endif
@@ -328,7 +322,7 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_bti_ist_info has been calculated. */
init_shadow_spec_ctrl_state();
- print_details(thunk);
+ print_details(thunk, caps);
}
static void __init __maybe_unused build_assertions(void)
--
2.1.4
[-- Attachment #7: xsa263-4.6/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch --]
[-- Type: application/octet-stream, Size: 5109 bytes --]
From 46bd7d0db401af4703414dcabdedbdaaee0da10b Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Express Xen's choice of MSR_SPEC_CTRL value as
a variable
At the moment, we have two different encodings of Xen's MSR_SPEC_CTRL value,
which is a side effect of how the Spectre series developed. One encoding is
via an alias with the bottom bit of bti_ist_info, and can encode IBRS or not,
but not other configurations such as STIBP.
Break Xen's value out into a separate variable (in the top of stack block for
XPTI reasons) and use this instead of bti_ist_info in the IST path.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 66dfae0f32bfbc899c2f3446d5ee57068cb7f957)
---
xen/arch/x86/spec_ctrl.c | 8 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/include/asm-x86/current.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 2 ++
xen/include/asm-x86/spec_ctrl_asm.h | 8 ++------
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 89c6f7d..d6c8656 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,7 @@ static int8_t __initdata opt_ibrs = -1;
static bool_t __initdata opt_rsb_native = 1;
static bool_t __initdata opt_rsb_vmexit = 1;
bool_t __read_mostly opt_ibpb = 1;
+uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_bti_ist_info;
static int __init parse_bti(const char *s)
@@ -282,11 +283,14 @@ void __init init_speculation_mitigations(void)
* guests.
*/
if ( ibrs )
+ {
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
__set_bit(X86_FEATURE_XEN_IBRS_SET, boot_cpu_data.x86_capability);
+ }
else
__set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_WRMSR | ibrs;
+ default_bti_ist_info |= BTI_IST_WRMSR;
}
/*
@@ -327,8 +331,6 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
- /* The optimised assembly relies on this alias. */
- BUILD_BUG_ON(BTI_IST_IBRS != SPEC_CTRL_IBRS);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 96c5eb4..1293cff 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -140,6 +140,7 @@ void __dummy__(void)
OFFSET(CPUINFO_xen_cr3, struct cpu_info, xen_cr3);
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
+ OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index 461d5f3..e1eef3d 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -57,6 +57,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
+ uint8_t xen_spec_ctrl;
bool_t use_shadow_spec_ctrl;
uint8_t bti_ist_info;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 39823af..3d103f9 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_bti_ist_info;
static inline void init_shadow_spec_ctrl_state(void)
@@ -34,6 +35,7 @@ static inline void init_shadow_spec_ctrl_state(void)
struct cpu_info *info = get_cpu_info();
info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->xen_spec_ctrl = default_xen_spec_ctrl;
info->bti_ist_info = default_bti_ist_info;
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index f225485..e2ac57a 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -21,7 +21,6 @@
#define __X86_SPEC_CTRL_ASM_H__
/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_IBRS (1 << 0)
#define BTI_IST_WRMSR (1 << 1)
#define BTI_IST_RSB (1 << 2)
@@ -285,12 +284,9 @@
setz %dl
and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
- /*
- * Load Xen's intended value. SPEC_CTRL_IBRS vs 0 is encoded in the
- * bottom bit of bti_ist_info, via a deliberate alias with BTI_IST_IBRS.
- */
+ /* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
- and $BTI_IST_IBRS, %eax
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
xor %edx, %edx
wrmsr
--
2.1.4
[-- Attachment #8: xsa263-4.6/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch --]
[-- Type: application/octet-stream, Size: 12684 bytes --]
From 86b5e84a5ef3ee9cb8642520d0510a6290e9ef0d Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Merge bti_ist_info and use_shadow_spec_ctrl
into spec_ctrl_flags
All 3 bits of information here are control flags for the entry/exit code
behaviour. Treat them as such, rather than having two different variables.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 5262ba2e7799001402dfe139ff944e035dfff928)
---
xen/arch/x86/acpi/power.c | 4 +--
xen/arch/x86/spec_ctrl.c | 10 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 3 +--
xen/include/asm-x86/current.h | 3 +--
xen/include/asm-x86/nops.h | 3 ++-
xen/include/asm-x86/spec_ctrl.h | 10 ++++----
xen/include/asm-x86/spec_ctrl_asm.h | 50 ++++++++++++++++++++-----------------
7 files changed, 44 insertions(+), 39 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 87cc09e..c1643e2 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -178,7 +178,7 @@ static int enter_state(u32 state)
ci = get_cpu_info();
spec_ctrl_enter_idle(ci);
/* Avoid NMI/#MC using MSR_SPEC_CTRL until we've reloaded microcode. */
- ci->bti_ist_info = 0;
+ ci->spec_ctrl_flags &= ~SCF_ist_wrmsr;
ACPI_FLUSH_CPU_CACHE();
@@ -220,7 +220,7 @@ static int enter_state(u32 state)
microcode_resume_cpu(0);
/* Re-enabled default NMI/#MC use of MSR_SPEC_CTRL. */
- ci->bti_ist_info = default_bti_ist_info;
+ ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
spec_ctrl_exit_idle(ci);
done:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index d6c8656..5387eea 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -39,7 +39,7 @@ static bool_t __initdata opt_rsb_native = 1;
static bool_t __initdata opt_rsb_vmexit = 1;
bool_t __read_mostly opt_ibpb = 1;
uint8_t __read_mostly default_xen_spec_ctrl;
-uint8_t __read_mostly default_bti_ist_info;
+uint8_t __read_mostly default_spec_ctrl_flags;
static int __init parse_bti(const char *s)
{
@@ -290,7 +290,7 @@ void __init init_speculation_mitigations(void)
else
__set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_WRMSR;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
/*
@@ -309,7 +309,7 @@ void __init init_speculation_mitigations(void)
if ( opt_rsb_native )
{
__set_bit(X86_FEATURE_RSB_NATIVE, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_RSB;
+ default_spec_ctrl_flags |= SCF_ist_rsb;
}
/*
@@ -323,7 +323,7 @@ void __init init_speculation_mitigations(void)
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
opt_ibpb = 0;
- /* (Re)init BSP state now that default_bti_ist_info has been calculated. */
+ /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
print_details(thunk, caps);
@@ -331,6 +331,8 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
+ /* The optimised assembly relies on this alias. */
+ BUILD_BUG_ON(SCF_use_shadow != 1);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 1293cff..85a8aec 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -141,8 +141,7 @@ void __dummy__(void)
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
- OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
- OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
+ OFFSET(CPUINFO_spec_ctrl_flags, struct cpu_info, spec_ctrl_flags);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
BLANK();
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index e1eef3d..04bc760 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -58,8 +58,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
uint8_t xen_spec_ctrl;
- bool_t use_shadow_spec_ctrl;
- uint8_t bti_ist_info;
+ uint8_t spec_ctrl_flags;
unsigned long __pad;
/* get_stack_bottom() must be 16-byte aligned */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index 0a5d680..90a94be 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -61,10 +61,11 @@
#define ASM_NOP7 _ASM_MK_NOP(K8_NOP7)
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
-#define ASM_NOP21 ASM_NOP8; ASM_NOP8; ASM_NOP5
+#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP23 ASM_NOP8; ASM_NOP8; ASM_NOP7
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 3d103f9..5801f4d 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -28,15 +28,15 @@ void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
extern uint8_t default_xen_spec_ctrl;
-extern uint8_t default_bti_ist_info;
+extern uint8_t default_spec_ctrl_flags;
static inline void init_shadow_spec_ctrl_state(void)
{
struct cpu_info *info = get_cpu_info();
- info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->shadow_spec_ctrl = 0;
info->xen_spec_ctrl = default_xen_spec_ctrl;
- info->bti_ist_info = default_bti_ist_info;
+ info->spec_ctrl_flags = default_spec_ctrl_flags;
}
/* WARNING! `ret`, `call *`, `jmp *` not safe after this call. */
@@ -50,7 +50,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
*/
info->shadow_spec_ctrl = val;
barrier();
- info->use_shadow_spec_ctrl = 1;
+ info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
@@ -65,7 +65,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
* Disable shadowing before updating the MSR. There are no SMP issues
* here; only local processor ordering concerns.
*/
- info->use_shadow_spec_ctrl = 0;
+ info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index e2ac57a..9e68c07 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -20,9 +20,10 @@
#ifndef __X86_SPEC_CTRL_ASM_H__
#define __X86_SPEC_CTRL_ASM_H__
-/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_WRMSR (1 << 1)
-#define BTI_IST_RSB (1 << 2)
+/* Encoding of cpuinfo.spec_ctrl_flags */
+#define SCF_use_shadow (1 << 0)
+#define SCF_ist_wrmsr (1 << 1)
+#define SCF_ist_rsb (1 << 2)
#ifdef __ASSEMBLY__
#include <asm/msr-index.h>
@@ -49,20 +50,20 @@
* after VMEXIT. The VMEXIT-specific code reads MSR_SPEC_CTRL and updates
* current before loading Xen's MSR_SPEC_CTRL setting.
*
- * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and
- * use_shadow_spec_ctrl boolean per cpu. The synchronous use is:
+ * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and a use_shadow
+ * boolean in the per cpu spec_ctrl_flags. The synchronous use is:
*
* 1) Store guest value in shadow_spec_ctrl
- * 2) Set use_shadow_spec_ctrl boolean
+ * 2) Set the use_shadow boolean
* 3) Load guest value into MSR_SPEC_CTRL
* 4) Exit to guest
* 5) Entry from guest
- * 6) Clear use_shadow_spec_ctrl boolean
+ * 6) Clear the use_shadow boolean
* 7) Load Xen's value into MSR_SPEC_CTRL
*
* The asynchronous use for interrupts/exceptions is:
* - Set/clear IBRS on entry to Xen
- * - On exit to Xen, check use_shadow_spec_ctrl
+ * - On exit to Xen, check use_shadow
* - If set, load shadow_spec_ctrl
*
* Therefore, an interrupt/exception which hits the synchronous path between
@@ -133,7 +134,7 @@
xor %edx, %edx
/* Clear SPEC_CTRL shadowing *before* loading Xen's value. */
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
mov $\ibrs_val, %eax
@@ -159,12 +160,14 @@
* block so calculate the position directly.
*/
.if \maybexen
+ xor %eax, %eax
/* Branchless `if ( !xen ) clear_shadowing` */
testb $3, UREGS_cs(%rsp)
- setz %al
- and %al, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %al
+ not %eax
+ and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
.else
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
.endif
/* Load Xen's intended value. */
@@ -183,8 +186,8 @@
*/
xor %edx, %edx
- cmpb %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%rbx)
- je .L\@_skip
+ testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
+ jz .L\@_skip
mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax
mov $MSR_SPEC_CTRL, %ecx
@@ -205,7 +208,7 @@
mov %eax, CPUINFO_shadow_spec_ctrl(%rsp)
/* Set SPEC_CTRL shadowing *before* loading the guest value. */
- movb $1, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ orb $SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
mov $MSR_SPEC_CTRL, %ecx
xor %edx, %edx
@@ -228,7 +231,7 @@
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP21), \
+ ALTERNATIVE_2 __stringify(ASM_NOP22), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -239,7 +242,7 @@
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
+ ALTERNATIVE_2 __stringify(ASM_NOP36), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -267,22 +270,23 @@
* This is logical merge of DO_OVERWRITE_RSB and DO_SPEC_CTRL_ENTRY
* maybexen=1, but with conditionals rather than alternatives.
*/
- movzbl STACK_CPUINFO_FIELD(bti_ist_info)(%r14), %eax
+ movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %eax
- testb $BTI_IST_RSB, %al
+ test $SCF_ist_rsb, %al
jz .L\@_skip_rsb
DO_OVERWRITE_RSB tmp=rdx /* Clobbers %rcx/%rdx */
.L\@_skip_rsb:
- testb $BTI_IST_WRMSR, %al
+ test $SCF_ist_wrmsr, %al
jz .L\@_skip_wrmsr
xor %edx, %edx
testb $3, UREGS_cs(%rsp)
- setz %dl
- and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %dl
+ not %edx
+ and %dl, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
/* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
@@ -309,7 +313,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
* Requires %rbx=stack_end
* Clobbers %rax, %rcx, %rdx
*/
- testb $BTI_IST_WRMSR, STACK_CPUINFO_FIELD(bti_ist_info)(%rbx)
+ testb $SCF_ist_wrmsr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
jz .L\@_skip
DO_SPEC_CTRL_EXIT_TO_XEN
--
2.1.4
[-- Attachment #9: xsa263-4.6/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch --]
[-- Type: application/octet-stream, Size: 11868 bytes --]
From 6a922ce2c11e142c9e1e6eae7337ffa0a14a68f6 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Fold the XEN_IBRS_{SET,CLEAR} ALTERNATIVES
together
Currently, the SPEC_CTRL_{ENTRY,EXIT}_* macros encode Xen's choice of
MSR_SPEC_CTRL as an immediate constant, and chooses between IBRS or not by
doubling up the entire alternative block.
There is now a variable holding Xen's choice of value, so use that and
simplify the alternatives.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit af949407eaba7af71067f23d5866cd0bf1f1144d)
---
xen/arch/x86/cpu/common.c | 8 ++-----
xen/arch/x86/spec_ctrl.c | 12 +++++-----
xen/include/asm-x86/cpufeature.h | 3 +--
xen/include/asm-x86/nops.h | 6 ++---
xen/include/asm-x86/spec_ctrl.h | 14 +++++++-----
xen/include/asm-x86/spec_ctrl_asm.h | 45 +++++++++++++------------------------
6 files changed, 35 insertions(+), 53 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 0d9f525..3da0979 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -359,13 +359,9 @@ void __cpuinit identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_IND_THUNK_JMP,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_IND_THUNK_JMP, c->x86_capability);
- if (test_bit(X86_FEATURE_XEN_IBRS_SET,
+ if (test_bit(X86_FEATURE_SC_MSR,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_XEN_IBRS_SET, c->x86_capability);
- if (test_bit(X86_FEATURE_XEN_IBRS_CLEAR,
- boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_XEN_IBRS_CLEAR,
- c->x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
if (test_bit(X86_FEATURE_RSB_NATIVE,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_RSB_NATIVE, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 5387eea..4fcbba2 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -114,8 +114,9 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_XEN_IBRS_SET) ? " IBRS+" :
- boot_cpu_has(X86_FEATURE_XEN_IBRS_CLEAR) ? " IBRS-" : "",
+ boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
+ " IBRS-" : "",
opt_ibpb ? " IBPB" : "",
boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
@@ -282,13 +283,10 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
+ __set_bit(X86_FEATURE_SC_MSR, boot_cpu_data.x86_capability);
+
if ( ibrs )
- {
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- __set_bit(X86_FEATURE_XEN_IBRS_SET, boot_cpu_data.x86_capability);
- }
- else
- __set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 82ad43f..ed4f18c 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -65,8 +65,7 @@
#define X86_FEATURE_IND_THUNK_LFENCE (3*32+ 1) /* Use IND_THUNK_LFENCE */
#define X86_FEATURE_IND_THUNK_JMP (3*32+ 2) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB (3*32+ 3) /* IBRSB || IBPB */
-#define X86_FEATURE_XEN_IBRS_SET (3*32+ 4) /* IBRSB && IRBS set in Xen */
-#define X86_FEATURE_XEN_IBRS_CLEAR (3*32+ 5) /* IBRSB && IBRS clear in Xen */
+#define X86_FEATURE_SC_MSR (3*32+ 4) /* MSR_SPEC_CTRL used by Xen */
#define X86_FEATURE_RSB_NATIVE (3*32+ 6) /* RSB overwrite needed for native */
#define X86_FEATURE_RSB_VMEXIT (3*32+ 7) /* RSB overwrite needed for vmexit */
#define X86_FEATURE_CONSTANT_TSC (3*32+ 8) /* TSC ticks at a constant rate */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index 90a94be..04463b4 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -61,11 +61,11 @@
#define ASM_NOP7 _ASM_MK_NOP(K8_NOP7)
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
-#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP23 ASM_NOP8; ASM_NOP8; ASM_NOP7
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
+#define ASM_NOP25 ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP39 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5801f4d..6c11562 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,14 +52,16 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
- :: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
+ :: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
+ "i" (X86_FEATURE_SC_MSR)
+ : "memory" );
}
/* WARNING! `ret`, `call *`, `jmp *` not safe before this call. */
static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
{
- uint32_t val = SPEC_CTRL_IBRS;
+ uint32_t val = info->xen_spec_ctrl;
/*
* Disable shadowing before updating the MSR. There are no SMP issues
@@ -67,8 +69,10 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
- :: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
+ :: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
+ "i" (X86_FEATURE_SC_MSR)
+ : "memory" );
}
#endif /* !__X86_SPEC_CTRL_H__ */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 9e68c07..ab47508 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -117,7 +117,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -137,11 +137,11 @@
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
- mov $\ibrs_val, %eax
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
wrmsr
.endm
-.macro DO_SPEC_CTRL_ENTRY maybexen:req ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY maybexen:req
/*
* Requires %rsp=regs (also cpuinfo if !maybexen)
* Requires %r14=stack_end (if maybexen)
@@ -166,12 +166,12 @@
setnz %al
not %eax
and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
.else
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
.endif
- /* Load Xen's intended value. */
- mov $\ibrs_val, %eax
wrmsr
.endm
@@ -219,47 +219,32 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP33), \
+ DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP22), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP25), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP36), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP39), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
- ALTERNATIVE_2 __stringify(ASM_NOP23), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP23), \
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
/* Use when exiting to guest context. */
#define SPEC_CTRL_EXIT_TO_GUEST \
- ALTERNATIVE_2 __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
--
2.1.4
[-- Attachment #10: xsa263-4.6/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch --]
[-- Type: application/octet-stream, Size: 13321 bytes --]
From 3b602557e94afe1a6175e8a86e470fc439e1bd0c Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 30 Apr 2018 14:20:23 +0100
Subject: [PATCH] x86/spec_ctrl: Rename bits of infrastructure to avoid NATIVE
and VMEXIT
In hindsight, using NATIVE and VMEXIT as naming terminology was not clever.
A future change wants to split SPEC_CTRL_EXIT_TO_GUEST into PV and HVM
specific implementations, and using VMEXIT as a term is completely wrong.
Take the opportunity to fix some stale documentation in spec_ctrl_asm.h. The
IST helpers were missing from the large comment block, and since
SPEC_CTRL_ENTRY_FROM_INTR_IST was introduced, we've gained a new piece of
functionality which currently depends on the fine grain control, which exists
in lieu of livepatching. Note this in the comment.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d9822b8a38114e96e4516dc998f4055249364d5d)
---
xen/arch/x86/cpu/common.c | 8 ++++----
xen/arch/x86/hvm/svm/entry.S | 4 ++--
xen/arch/x86/hvm/vmx/entry.S | 4 ++--
xen/arch/x86/spec_ctrl.c | 20 ++++++++++----------
xen/arch/x86/x86_64/compat/entry.S | 2 +-
xen/arch/x86/x86_64/entry.S | 2 +-
xen/include/asm-x86/cpufeature.h | 4 ++--
xen/include/asm-x86/spec_ctrl_asm.h | 36 +++++++++++++++++++++++++-----------
8 files changed, 47 insertions(+), 33 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 3da0979..1ba1622 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -362,12 +362,12 @@ void __cpuinit identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_SC_MSR,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
- if (test_bit(X86_FEATURE_RSB_NATIVE,
+ if (test_bit(X86_FEATURE_SC_RSB_PV,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_RSB_NATIVE, c->x86_capability);
- if (test_bit(X86_FEATURE_RSB_VMEXIT,
+ __set_bit(X86_FEATURE_SC_RSB_PV, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_RSB_HVM,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_RSB_VMEXIT, c->x86_capability);
+ __set_bit(X86_FEATURE_SC_RSB_HVM, c->x86_capability);
/* AND the already accumulated flags with these */
for ( i = 0 ; i < NCAPINTS ; i++ )
diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S
index 706bdd3..6426452 100644
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -81,7 +81,7 @@ UNLIKELY_END(svm_trace)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
pop %r15
pop %r14
@@ -106,7 +106,7 @@ UNLIKELY_END(svm_trace)
GET_CURRENT(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov VCPU_svm_vmcb(%rbx),%rcx
diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S
index d43ae26..32e0f87 100644
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -37,7 +37,7 @@ ENTRY(vmx_asm_vmexit_handler)
movb $1,VCPU_vmx_launched(%rbx)
mov %rax,VCPU_hvm_guest_cr2(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov %rsp,%rdi
@@ -72,7 +72,7 @@ UNLIKELY_END(realmode)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
mov VCPU_hvm_guest_cr2(%rbx),%rax
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 4fcbba2..91e1848 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -35,8 +35,8 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool_t __initdata opt_rsb_native = 1;
-static bool_t __initdata opt_rsb_vmexit = 1;
+static bool_t __initdata opt_rsb_pv = 1;
+static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -69,9 +69,9 @@ static int __init parse_bti(const char *s)
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
else if ( (val = parse_boolean("rsb_native", s, ss)) >= 0 )
- opt_rsb_native = val;
+ opt_rsb_pv = val;
else if ( (val = parse_boolean("rsb_vmexit", s, ss)) >= 0 )
- opt_rsb_vmexit = val;
+ opt_rsb_hvm = val;
else
rc = -EINVAL;
@@ -118,8 +118,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
}
/* Calculate whether Retpoline is known-safe on this CPU. */
@@ -304,9 +304,9 @@ void __init init_speculation_mitigations(void)
* If a processors speculates to 32bit PV guest kernel mappings, it is
* speculating in 64bit supervisor mode, and can leak data.
*/
- if ( opt_rsb_native )
+ if ( opt_rsb_pv )
{
- __set_bit(X86_FEATURE_RSB_NATIVE, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_RSB_PV, boot_cpu_data.x86_capability);
default_spec_ctrl_flags |= SCF_ist_rsb;
}
@@ -314,8 +314,8 @@ void __init init_speculation_mitigations(void)
* HVM guests can always poison the RSB to point at Xen supervisor
* mappings.
*/
- if ( opt_rsb_vmexit )
- __set_bit(X86_FEATURE_RSB_VMEXIT, boot_cpu_data.x86_capability);
+ if ( opt_rsb_hvm )
+ __set_bit(X86_FEATURE_SC_RSB_HVM, boot_cpu_data.x86_capability);
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index c211e9a..6a48fc5 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -235,7 +235,7 @@ ENTRY(compat_restore_all_guest)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL adj=8 compat=1
.Lft0: iretq
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 7c8211a..8cecfd4 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -71,7 +71,7 @@ restore_all_guest:
mov %r15d, %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL
testw $TRAP_syscall,4(%rsp)
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index ed4f18c..9c8bca9 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -66,8 +66,8 @@
#define X86_FEATURE_IND_THUNK_JMP (3*32+ 2) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB (3*32+ 3) /* IBRSB || IBPB */
#define X86_FEATURE_SC_MSR (3*32+ 4) /* MSR_SPEC_CTRL used by Xen */
-#define X86_FEATURE_RSB_NATIVE (3*32+ 6) /* RSB overwrite needed for native */
-#define X86_FEATURE_RSB_VMEXIT (3*32+ 7) /* RSB overwrite needed for vmexit */
+#define X86_FEATURE_SC_RSB_PV (3*32+ 6) /* RSB overwrite needed for PV */
+#define X86_FEATURE_SC_RSB_HVM (3*32+ 7) /* RSB overwrite needed for HVM */
#define X86_FEATURE_CONSTANT_TSC (3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_NONSTOP_TSC (3*32+ 9) /* TSC does not stop in C states */
#define X86_FEATURE_ARAT (3*32+ 10) /* Always running APIC timer */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index ab47508..be5cba3 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -72,11 +72,14 @@
*
* The following ASM fragments implement this algorithm. See their local
* comments for further details.
- * - SPEC_CTRL_ENTRY_FROM_VMEXIT
+ * - SPEC_CTRL_ENTRY_FROM_HVM
* - SPEC_CTRL_ENTRY_FROM_PV
* - SPEC_CTRL_ENTRY_FROM_INTR
+ * - SPEC_CTRL_ENTRY_FROM_INTR_IST
+ * - SPEC_CTRL_EXIT_TO_XEN_IST
* - SPEC_CTRL_EXIT_TO_XEN
- * - SPEC_CTRL_EXIT_TO_GUEST
+ * - SPEC_CTRL_EXIT_TO_PV
+ * - SPEC_CTRL_EXIT_TO_HVM
*/
.macro DO_OVERWRITE_RSB tmp=rax
@@ -117,7 +120,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
+.macro DO_SPEC_CTRL_ENTRY_FROM_HVM
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -216,23 +219,23 @@
.endm
/* Use after a VMEXIT from an HVM guest. */
-#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
+#define SPEC_CTRL_ENTRY_FROM_HVM \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP33), \
- DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP39), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
@@ -241,12 +244,22 @@
ALTERNATIVE __stringify(ASM_NOP23), \
DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
-/* Use when exiting to guest context. */
-#define SPEC_CTRL_EXIT_TO_GUEST \
+/* Use when exiting to PV guest context. */
+#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
-/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
+/* Use when exiting to HVM guest context. */
+#define SPEC_CTRL_EXIT_TO_HVM \
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+
+/*
+ * Use in IST interrupt/exception context. May interrupt Xen or PV context.
+ * Fine grain control of SCF_ist_wrmsr is needed for safety in the S3 resume
+ * path to avoid using MSR_SPEC_CTRL before the microcode introducing it has
+ * been reloaded.
+ */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
/*
* Requires %rsp=regs, %r14=stack_end
@@ -293,6 +306,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
UNLIKELY_END(\@_serialise)
.endm
+/* Use when exiting to Xen in IST context. */
.macro SPEC_CTRL_EXIT_TO_XEN_IST
/*
* Requires %rbx=stack_end
--
2.1.4
[-- Attachment #11: xsa263-4.6/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch --]
[-- Type: application/octet-stream, Size: 3711 bytes --]
From 0609578b617af3f4a4da8e1c2f5cab1ab20e48e5 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 May 2018 14:06:16 +0100
Subject: [PATCH] x86/spec_ctrl: Elide MSR_SPEC_CTRL handling in idle context
when possible
If Xen is virtualising MSR_SPEC_CTRL handling for guests, but using 0 as its
own MSR_SPEC_CTRL value, spec_ctrl_{enter,exit}_idle() need not write to the
MSR.
Requested-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 94df6e8588e35cc2028ccb3fd2921c6e6360605e)
---
xen/arch/x86/cpu/common.c | 3 +++
xen/arch/x86/spec_ctrl.c | 4 ++++
xen/include/asm-x86/cpufeature.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 4 ++--
4 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 1ba1622..15e831a 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -368,6 +368,9 @@ void __cpuinit identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_SC_RSB_HVM,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_SC_RSB_HVM, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_MSR_IDLE,
+ boot_cpu_data.x86_capability))
+ __set_bit(X86_FEATURE_SC_MSR_IDLE, c->x86_capability);
/* AND the already accumulated flags with these */
for ( i = 0 ; i < NCAPINTS ; i++ )
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 91e1848..778f8e5 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -324,6 +324,10 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
+ /* If Xen is using any MSR_SPEC_CTRL settings, adjust the idle path. */
+ if ( default_xen_spec_ctrl )
+ __set_bit(X86_FEATURE_SC_MSR_IDLE, boot_cpu_data.x86_capability);
+
print_details(thunk, caps);
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 9c8bca9..d140982 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -76,6 +76,7 @@
#define X86_FEATURE_XTOPOLOGY (3*32+13) /* cpu topology enum extensions */
#define X86_FEATURE_CPUID_FAULTING (3*32+14) /* cpuid faulting */
#define X86_FEATURE_CLFLUSH_MONITOR (3*32+15) /* clflush reqd with monitor */
+#define X86_FEATURE_SC_MSR_IDLE (3*32+16) /* SC_MSR && default_xen_spec_ctrl */
/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
#define X86_FEATURE_XMM3 (4*32+ 0) /* Streaming SIMD Extensions-3 */
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 6c11562..ec943e1 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -54,7 +54,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_SC_MSR)
+ "i" (X86_FEATURE_SC_MSR_IDLE)
: "memory" );
}
@@ -71,7 +71,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_SC_MSR)
+ "i" (X86_FEATURE_SC_MSR_IDLE)
: "memory" );
}
--
2.1.4
[-- Attachment #12: xsa263-4.6/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch --]
[-- Type: application/octet-stream, Size: 7267 bytes --]
From 74d7f96bc51d7b84afad2351b753000012ec606a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Split X86_FEATURE_SC_MSR into PV and HVM
variants
In order to separately control whether MSR_SPEC_CTRL is virtualised for PV and
HVM guests, split the feature used to control runtime alternatives into two.
Xen will use MSR_SPEC_CTRL itself if either of these features are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit fa9eb09d446a1279f5e861e6b84fa8675dabf148)
---
xen/arch/x86/cpu/common.c | 7 +++++--
xen/arch/x86/spec_ctrl.c | 6 ++++--
xen/include/asm-x86/cpufeature.h | 5 +++--
xen/include/asm-x86/spec_ctrl_asm.h | 12 ++++++------
4 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 15e831a..0e8fd2e 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -359,9 +359,12 @@ void __cpuinit identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_IND_THUNK_JMP,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_IND_THUNK_JMP, c->x86_capability);
- if (test_bit(X86_FEATURE_SC_MSR,
+ if (test_bit(X86_FEATURE_SC_MSR_PV,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_PV, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_MSR_HVM,
+ boot_cpu_data.x86_capability))
+ __set_bit(X86_FEATURE_SC_MSR_HVM, c->x86_capability);
if (test_bit(X86_FEATURE_SC_RSB_PV,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_SC_RSB_PV, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 778f8e5..fece105 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -114,7 +114,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
@@ -283,7 +284,8 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
- __set_bit(X86_FEATURE_SC_MSR, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
if ( ibrs )
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index d140982..3aaa6c8 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -65,7 +65,8 @@
#define X86_FEATURE_IND_THUNK_LFENCE (3*32+ 1) /* Use IND_THUNK_LFENCE */
#define X86_FEATURE_IND_THUNK_JMP (3*32+ 2) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB (3*32+ 3) /* IBRSB || IBPB */
-#define X86_FEATURE_SC_MSR (3*32+ 4) /* MSR_SPEC_CTRL used by Xen */
+#define X86_FEATURE_SC_MSR_PV (3*32+ 4) /* MSR_SPEC_CTRL used by Xen for PV */
+#define X86_FEATURE_SC_MSR_HVM (3*32+ 5) /* MSR_SPEC_CTRL used by Xen for HVM */
#define X86_FEATURE_SC_RSB_PV (3*32+ 6) /* RSB overwrite needed for PV */
#define X86_FEATURE_SC_RSB_HVM (3*32+ 7) /* RSB overwrite needed for HVM */
#define X86_FEATURE_CONSTANT_TSC (3*32+ 8) /* TSC ticks at a constant rate */
@@ -76,7 +77,7 @@
#define X86_FEATURE_XTOPOLOGY (3*32+13) /* cpu topology enum extensions */
#define X86_FEATURE_CPUID_FAULTING (3*32+14) /* cpuid faulting */
#define X86_FEATURE_CLFLUSH_MONITOR (3*32+15) /* clflush reqd with monitor */
-#define X86_FEATURE_SC_MSR_IDLE (3*32+16) /* SC_MSR && default_xen_spec_ctrl */
+#define X86_FEATURE_SC_MSR_IDLE (3*32+16) /* (SC_MSR_PV || SC_MSR_HVM) && default_xen_spec_ctrl */
/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
#define X86_FEATURE_XMM3 (4*32+ 0) /* Streaming SIMD Extensions-3 */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index be5cba3..30077d7 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -223,36 +223,36 @@
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP33), \
- DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR_HVM
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR_PV
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP39), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR_PV
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
ALTERNATIVE __stringify(ASM_NOP23), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV
/* Use when exiting to PV guest context. */
#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV
/* Use when exiting to HVM guest context. */
#define SPEC_CTRL_EXIT_TO_HVM \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_HVM
/*
* Use in IST interrupt/exception context. May interrupt Xen or PV context.
--
2.1.4
[-- Attachment #13: xsa263-4.6/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch --]
[-- Type: application/octet-stream, Size: 4804 bytes --]
From e9bdf629557bfe5cc095628413ab9e759d37a7e8 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 9 May 2018 13:59:56 +0100
Subject: [PATCH] x86/spec_ctrl: Explicitly set Xen's default MSR_SPEC_CTRL
value
With the impending ability to disable MSR_SPEC_CTRL handling on a
per-guest-type basis, the first exit-from-guest may not have the side effect
of loading Xen's choice of value. Explicitly set Xen's default during the BSP
and AP boot paths.
For the BSP however, delay setting a non-zero MSR_SPEC_CTRL default until
after dom0 has been constructed when safe to do so. Oracle report that this
speeds up boots of some hardware by 50s.
"when safe to do so" is based on whether we are virtualised. A native boot
won't have any other code running in a position to mount an attack.
Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb8c12020307b39a89273d7699e89000451987ab)
---
xen/arch/x86/setup.c | 7 +++++++
xen/arch/x86/smpboot.c | 8 ++++++++
xen/arch/x86/spec_ctrl.c | 32 ++++++++++++++++++++++++++++++++
xen/include/asm-x86/spec_ctrl.h | 2 ++
4 files changed, 49 insertions(+)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 606a57a..c3adee5 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1538,6 +1538,13 @@ void __init noreturn __start_xen(unsigned long mbi_p)
setup_io_bitmap(dom0);
+ if ( bsp_delay_spec_ctrl )
+ {
+ get_cpu_info()->spec_ctrl_flags &= ~SCF_use_shadow;
+ barrier();
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+ }
+
/* Jump to the 1:1 virtual mappings of cpu0_stack. */
asm volatile ("mov %[stk], %%rsp; jmp %c[fn]" ::
[stk] "g" (__va(__pa(get_stack_bottom()))),
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index b69d63a..7f57dcf 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -379,6 +379,14 @@ void start_secondary(void *unused)
else
microcode_resume_cpu(cpu);
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. Note: MSR_SPEC_CTRL may only become available
+ * after loading microcode.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+
smp_callin();
setup_secondary_APIC_clock();
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index fece105..844a22f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,8 @@ static int8_t __initdata opt_ibrs = -1;
static bool_t __initdata opt_rsb_pv = 1;
static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
+
+bool_t __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -331,6 +333,36 @@ void __init init_speculation_mitigations(void)
__set_bit(X86_FEATURE_SC_MSR_IDLE, boot_cpu_data.x86_capability);
print_details(thunk, caps);
+
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. For performance reasons, when safe to do so, we
+ * delay applying non-zero settings until after dom0 has been constructed.
+ *
+ * "when safe to do so" is based on whether we are virtualised. A native
+ * boot won't have any other code running in a position to mount an
+ * attack.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ {
+ bsp_delay_spec_ctrl = !cpu_has_hypervisor && default_xen_spec_ctrl;
+
+ /*
+ * If delaying MSR_SPEC_CTRL setup, use the same mechanism as
+ * spec_ctrl_enter_idle(), by using a shadow value of zero.
+ */
+ if ( bsp_delay_spec_ctrl )
+ {
+ struct cpu_info *info = get_cpu_info();
+
+ info->shadow_spec_ctrl = 0;
+ barrier();
+ info->spec_ctrl_flags |= SCF_use_shadow;
+ barrier();
+ }
+
+ wrmsrl(MSR_SPEC_CTRL, bsp_delay_spec_ctrl ? 0 : default_xen_spec_ctrl);
+ }
}
static void __init __maybe_unused build_assertions(void)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index ec943e1..d36f0e9 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,8 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+
+extern bool_t bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_spec_ctrl_flags;
--
2.1.4
[-- Attachment #14: xsa263-4.6/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch --]
[-- Type: application/octet-stream, Size: 2474 bytes --]
From 73d4d1e5399fee57c4c903bc3068729c5d12f935 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 1 May 2018 11:59:03 +0100
Subject: [PATCH] x86/cpuid: Improvements to guest policies for speculative
sidechannel features
If Xen isn't virtualising MSR_SPEC_CTRL for guests, IBRSB shouldn't be
advertised. It is not currently possible to express this via the existing
command line options, but such an ability will be introduced.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb06b308ec71b23f37a44f5e2351fe2cae0306e9)
---
xen/arch/x86/hvm/hvm.c | 3 +++
xen/arch/x86/traps.c | 6 +++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 78f44c5..67f75b9 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4624,6 +4624,9 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
if ( count == 0 )
{
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_HVM) )
+ *edx &= ~cpufeat_mask(X86_FEATURE_IBRSB);
+
/*
* Override STIBP to match IBRS. Guests can safely use STIBP
* functionality on non-HT hardware, but can't necesserily protect
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index c23f4c0..75d41b1 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -879,6 +879,9 @@ void pv_cpuid(struct cpu_user_regs *regs)
case 0x00000007:
if ( regs->_ecx == 0 )
{
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ d &= ~cpufeat_mask(X86_FEATURE_IBRSB);
+
/*
* Override STIBP to match IBRS. Guests can safely use STIBP
* functionality on non-HT hardware, but can't necesserily protect
@@ -966,7 +969,8 @@ void pv_cpuid(struct cpu_user_regs *regs)
cpufeat_mask(X86_FEATURE_ADX) |
cpufeat_mask(X86_FEATURE_FSGSBASE));
- d &= cpufeat_mask(X86_FEATURE_IBRSB);
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ d &= ~cpufeat_mask(X86_FEATURE_IBRSB);
/* Override STIBP to match IBRS (see above). */
if ( d & cpufeat_mask(X86_FEATURE_IBRSB) )
--
2.1.4
[-- Attachment #15: xsa263-4.6/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch --]
[-- Type: application/octet-stream, Size: 14607 bytes --]
From 9fd05ec3b5c50ef70096968f15f4198b63d8869f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:52:55 +0100
Subject: [PATCH] x86/spec_ctrl: Introduce a new `spec-ctrl=` command line
argument to replace `bti=`
In hindsight, the options for `bti=` aren't as flexible or useful as expected
(including several options which don't appear to behave as intended).
Changing the behaviour of an existing option is problematic for compatibility,
so introduce a new `spec-ctrl=` in the hopes that we can do better.
One common way of deploying Xen is with a single PV dom0 and all domUs being
HVM domains. In such a setup, an administrator who has weighed up the risks
may wish to forgo protection against malicious PV domains, to reduce the
overall performance hit. To cater for this usecase, `spec-ctrl=no-pv` will
disable all speculative protection for PV domains, while leaving all
speculative protection for HVM domains intact.
For coding clarity as much as anything else, the suboptions are grouped by
logical area; those which affect the alternatives blocks, and those which
affect Xen's in-hypervisor settings. See the xen-command-line.markdown for
full details of the new options.
While changing the command line options, take the time to change how the data
is reported to the user. The three DEBUG printks are upgraded to unilateral,
as they are all relevant pieces of information, and the old "mitigations:"
line is split in the two logical areas described above.
Sample output from booting with `spec-ctrl=no-pv` looks like:
(XEN) Speculative mitigation facilities:
(XEN) Hardware features: IBRS/IBPB STIBP IBPB
(XEN) Compiled-in support: INDIRECT_THUNK
(XEN) Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS-, Other: IBPB
(XEN) Support for VMs: PV: None, HVM: MSR_SPEC_CTRL RSB
(XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 3352afc26c497d26ecb70527db3cb29daf7b1422)
---
docs/misc/xen-command-line.markdown | 49 +++++++++++
xen/arch/x86/spec_ctrl.c | 159 ++++++++++++++++++++++++++++++------
xen/include/asm-x86/cpufeature.h | 1 +
3 files changed, 186 insertions(+), 23 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index dea3e54..903eab7 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -247,6 +247,9 @@ the NMI watchdog is also enabled.
### bti (x86)
> `= List of [ thunk=retpoline|lfence|jmp, ibrs=<bool>, ibpb=<bool>, rsb_{vmexit,native}=<bool> ]`
+**WARNING: This command line option is deprecated, and superseded by
+_spec-ctrl=_ - using both options in combination is undefined.**
+
Branch Target Injection controls. By default, Xen will pick the most
appropriate BTI mitigations based on compiled in support, loaded microcode,
and hardware details.
@@ -1394,6 +1397,52 @@ enforces the maximum theoretically necessary timeout of 670ms. Any number
is being interpreted as a custom timeout in milliseconds. Zero or boolean
false disable the quirk workaround, which is also the default.
+### spec-ctrl (x86)
+> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+
+Controls for speculative execution sidechannel mitigations. By default, Xen
+will pick the most appropriate mitigations based on compiled in support,
+loaded microcode, and hardware details, and will virtualise appropriate
+mitigations for guests to use.
+
+**WARNING: Any use of this option may interfere with heuristics. Use with
+extreme care.**
+
+An overall boolean value, `spec-ctrl=no`, can be specified to turn off all
+mitigations, including pieces of infrastructure used to virtualise certain
+mitigation features for guests. Alternatively, a slightly more restricted
+`spec-ctrl=no-xen` can be used to turn off all of Xen's mitigations, while
+leaving the virtualisation support in place for guests to use. Use of a
+positive boolean value for either of these options is invalid.
+
+The booleans `pv=`, `hvm=`, `msr-sc=` and `rsb=` offer fine grained control
+over the alternative blocks used by Xen. These impact Xen's ability to
+protect itself, and Xen's ability to virtualise support for guests to use.
+
+* `pv=` and `hvm=` offer control over all suboptions for PV and HVM guests
+ respectively.
+* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
+ on entry and exit. These blocks are necessary to virtualise support for
+ guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
+ Return Address Stack on entry to Xen.
+
+If Xen was compiled with INDIRECT\_THUNK support, `bti-thunk=` can be used to
+select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
+locations. The default thunk is `retpoline` (generally preferred for Intel
+hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
+overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+
+On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
+`ibrs=` option can be used to force or prevent Xen using the feature itself.
+If Xen is not using IBRS itself, functionality is still set up so IBRS can be
+virtualised for guests.
+
+On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
+option can be used to force (the default) or prevent Xen from issuing branch
+prediction barriers on vcpu context switches.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 844a22f..afb6e5a 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -26,6 +26,13 @@
#include <asm/spec_ctrl.h>
#include <asm/spec_ctrl_asm.h>
+/* Cmdline controls for Xen's alternative blocks. */
+static bool_t __initdata opt_msr_sc_pv = 1;
+static bool_t __initdata opt_msr_sc_hvm = 1;
+static bool_t __initdata opt_rsb_pv = 1;
+static bool_t __initdata opt_rsb_hvm = 1;
+
+/* Cmdline controls for Xen's speculative settings. */
static enum ind_thunk {
THUNK_DEFAULT, /* Decide which thunk to use at boot time. */
THUNK_NONE, /* Missing compiler support for thunks. */
@@ -35,8 +42,6 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool_t __initdata opt_rsb_pv = 1;
-static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
bool_t __initdata bsp_delay_spec_ctrl;
@@ -84,8 +89,95 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
+static int __init parse_spec_ctrl(const char *s)
+{
+ const char *ss;
+ int val, rc = 0;
+
+ do {
+ ss = strchr(s, ',');
+ if ( !ss )
+ ss = strchr(s, '\0');
+
+ /* Global and Xen-wide disable. */
+ val = parse_bool(s);
+ if ( !val )
+ {
+ opt_msr_sc_pv = 0;
+ opt_msr_sc_hvm = 0;
+
+ disable_common:
+ opt_rsb_pv = 0;
+ opt_rsb_hvm = 0;
+
+ opt_thunk = THUNK_JMP;
+ opt_ibrs = 0;
+ opt_ibpb = 0;
+ }
+ else if ( val > 0 )
+ rc = -EINVAL;
+ else if ( (val = parse_boolean("xen", s, ss)) >= 0 )
+ {
+ if ( !val )
+ goto disable_common;
+
+ rc = -EINVAL;
+ }
+
+ /* Xen's alternative blocks. */
+ else if ( (val = parse_boolean("pv", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_rsb_pv = val;
+ }
+ else if ( (val = parse_boolean("hvm", s, ss)) >= 0 )
+ {
+ opt_msr_sc_hvm = val;
+ opt_rsb_hvm = val;
+ }
+ else if ( (val = parse_boolean("msr-sc", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_msr_sc_hvm = val;
+ }
+ else if ( (val = parse_boolean("rsb", s, ss)) >= 0 )
+ {
+ opt_rsb_pv = val;
+ opt_rsb_hvm = val;
+ }
+
+ /* Xen's speculative sidechannel mitigation settings. */
+ else if ( !strncmp(s, "bti-thunk=", 10) )
+ {
+ s += 10;
+
+ if ( !strncmp(s, "retpoline", ss - s) )
+ opt_thunk = THUNK_RETPOLINE;
+ else if ( !strncmp(s, "lfence", ss - s) )
+ opt_thunk = THUNK_LFENCE;
+ else if ( !strncmp(s, "jmp", ss - s) )
+ opt_thunk = THUNK_JMP;
+ else
+ rc = -EINVAL;
+ }
+ else if ( (val = parse_boolean("ibrs", s, ss)) >= 0 )
+ opt_ibrs = val;
+ else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
+ opt_ibpb = val;
+ else
+ rc = -EINVAL;
+
+ s = ss + 1;
+ } while ( *ss );
+
+ return rc;
+}
+custom_param("spec-ctrl", parse_spec_ctrl);
+
static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
+ bool_t use_spec_ctrl = (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM));
unsigned int _7d0 = 0, e8b = 0, tmp;
/* Collect diagnostics about available mitigations. */
@@ -94,10 +186,10 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
+ printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(XENLOG_DEBUG " Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
@@ -110,19 +202,29 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk(XENLOG_DEBUG " Compiled-in support: INDIRECT_THUNK\n");
#endif
- printk(XENLOG_INFO
- "BTI mitigations: Thunk %s, Others:%s%s%s%s\n",
+ /* Settings for Xen's protection, irrespective of guests. */
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
+ !use_spec_ctrl ? "No" :
+ (default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ opt_ibpb ? " IBPB" : "");
+
+ /*
+ * Alternatives blocks for protecting against and/or virtualising
+ * mitigation support for guests.
+ */
+ printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n",
(boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
- boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
- default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
- " IBRS-" : "",
- opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "",
+ (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ||
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "");
}
/* Calculate whether Retpoline is known-safe on this CPU. */
@@ -211,7 +313,7 @@ static bool_t __init __maybe_unused retpoline_safe(uint64_t caps)
void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
- bool_t ibrs = 0;
+ bool_t use_spec_ctrl = 0, ibrs = 0;
uint64_t caps = 0;
if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
@@ -279,20 +381,31 @@ void __init init_speculation_mitigations(void)
else if ( thunk == THUNK_JMP )
__set_bit(X86_FEATURE_IND_THUNK_JMP, boot_cpu_data.x86_capability);
+ /*
+ * If we are on hardware supporting MSR_SPEC_CTRL, see about setting up
+ * the alternatives blocks so we can virtualise support for guests.
+ */
if ( boot_cpu_has(X86_FEATURE_IBRSB) )
{
- /*
- * Even if we've chosen to not have IBRS set in Xen context, we still
- * need the IBRS entry/exit logic to virtualise IBRS support for
- * guests.
- */
- __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
- __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
+ if ( opt_msr_sc_pv )
+ {
+ use_spec_ctrl = 1;
+ __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
+ }
- if ( ibrs )
- default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
+ if ( opt_msr_sc_hvm )
+ {
+ use_spec_ctrl = 1;
+ __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
+ }
+
+ if ( use_spec_ctrl )
+ {
+ if ( ibrs )
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ }
}
/*
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 3aaa6c8..22429db 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -235,6 +235,7 @@
#define cpu_has_svm boot_cpu_has(X86_FEATURE_SVM)
#define cpu_has_vmx boot_cpu_has(X86_FEATURE_VMXE)
+#define cpu_has_hypervisor boot_cpu_has(X86_FEATURE_HYPERVISOR)
#define cpu_has_cpuid_faulting boot_cpu_has(X86_FEATURE_CPUID_FAULTING)
#define cpu_has_lfence_dispatch boot_cpu_has(X86_FEATURE_LFENCE_DISPATCH)
--
2.1.4
[-- Attachment #16: xsa263-4.6/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4325 bytes --]
From 460f73851142a2a625ae51750154579e6992fe63 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 903eab7..e2f4ae3 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1399,7 +1399,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1443,6 +1443,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 9d27e2a..efd943a 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -10,6 +10,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -496,6 +497,25 @@ static void __devinit init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
switch(c->x86)
{
case 0xf ... 0x17:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index afb6e5a..8df606c 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool_t __read_mostly opt_ibpb = 1;
+bool_t __read_mostly opt_ssbd;
bool_t __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -164,6 +165,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index d36f0e9..dd084d2 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+extern bool_t opt_ssbd;
extern bool_t bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #17: xsa263-4.6/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 8427 bytes --]
From 4b53a786d4df2d99a0ce2d294aeaeb3e7c136a86 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
xen/arch/x86/setup.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/cpufeature.h | 1 +
xen/include/asm-x86/msr-index.h | 2 ++
6 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index e2f4ae3..3d54baf 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -440,9 +440,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1424,7 +1425,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1446,7 +1447,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 17e9f0f..6a28fec 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -160,6 +160,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"fpu", 0x00000001, NA, CPUID_REG_EDX, 0, 1},
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"topoext", 0x80000001, NA, CPUID_REG_ECX, 22, 1},
{"tbm", 0x80000001, NA, CPUID_REG_ECX, 21, 1},
{"nodeid", 0x80000001, NA, CPUID_REG_ECX, 19, 1},
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index c3adee5..f3831e0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -135,6 +135,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 8df606c..13f72ce 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -192,13 +192,15 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
#ifdef CONFIG_INDIRECT_THUNK
@@ -206,13 +208,16 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
#endif
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -411,6 +416,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 22429db..20c6d62 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -171,6 +171,7 @@
#define X86_FEATURE_IBRSB (9*32+26) /* IBRS and IBPB support (used by Intel) */
#define X86_FEATURE_STIBP (9*32+27) /* STIBP */
#define X86_FEATURE_ARCH_CAPS (9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+#define X86_FEATURE_SSBD (9*32+31) /* MSR_SPEC_CTRL.SSBD available */
/* An alias of a feature we know is always going to be present. */
#define X86_FEATURE_ALWAYS X86_FEATURE_LM
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index e5845a7..c21801c 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -35,6 +35,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -43,6 +44,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
--
2.1.4
[-- Attachment #18: xsa263-4.6/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 2503 bytes --]
From f7134d8cbfb8fa42f3e4970f75ab0b0a7a11596e Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr().
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/domctl.c | 3 ++-
xen/arch/x86/hvm/hvm.c | 3 ++-
xen/arch/x86/traps.c | 3 ++-
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 90fe100..0357d62 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1136,7 +1136,8 @@ long arch_do_domctl(
* ignored) when STIBP isn't enumerated in hardware.
*/
- if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (boot_cpu_has(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0)) )
break;
v->arch.spec_ctrl = msr.value;
continue;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 67f75b9..b5b72d0 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4997,7 +4997,8 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
goto gp_fault; /* Rsvd bit set? */
v->arch.spec_ctrl = msr_content;
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 75d41b1..eeac17b 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2744,7 +2744,8 @@ static int emulate_privileged_op(struct cpu_user_regs *regs)
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
goto fail; /* Rsvd bit set? */
v->arch.spec_ctrl = eax;
--
2.1.4
[-- Attachment #19: xsa263-4.7/0001-x86-Fix-x86-further-CPUID-handling-adjustments.patch --]
[-- Type: application/octet-stream, Size: 3405 bytes --]
From 39cbbe4a52b6e2ba54114a5710bc23a3de2a497c Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 15 May 2018 16:37:59 +0100
Subject: [PATCH] x86: Fix "x86: further CPUID handling adjustments"
c/s f9616884e (a backport of c/s 0d703a701 "x86/feature: Definitions for
Indirect Branch Controls") missed a CPUID adjustment when calculating the raw
featureset. This impacts host administrator diagnostics.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
c/s 62b187969 "x86: further CPUID handling adjustments" make some adjustments.
However, it breaks levelling of guests, making it impossible for the toolstack
to hide STIBP or IBPB from guests on hardware with up-to-date microcode.
The dom0 issue referenced in the commit message was fixed by the hunk
adjusting the zeroing alone. STIBP and IBPB don't need (and indeed, must not
be for levelling purposes) OR'd into the leaf.
One final item which was missed in backport was the need to ignore the
toolstack choice of STIBP, and set it equal to IBRSB. This needs doing after
the mask has been applied.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/hvm/hvm.c | 8 +++++---
xen/arch/x86/traps.c | 8 +++++---
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index ff1c6fa..0a1d4a9 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3496,10 +3496,13 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
special_features[FEATURESET_7b0]);
*ecx &= hvm_featureset[FEATURESET_7c0];
-
- *edx |= cpufeat_mask(X86_FEATURE_STIBP);
*edx &= hvm_featureset[FEATURESET_7d0];
+ /* Force STIBP equal to IBRSB */
+ *edx &= ~cpufeat_mask(X86_FEATURE_STIBP);
+ if ( *edx & cpufeat_mask(X86_FEATURE_IBRSB) )
+ *edx |= cpufeat_mask(X86_FEATURE_STIBP);
+
/* Don't expose HAP-only features to non-hap guests. */
if ( !hap_enabled(d) )
{
@@ -3657,7 +3660,6 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
hvm_cpuid(0x80000001, NULL, NULL, NULL, &_edx);
*eax |= (_edx & cpufeat_mask(X86_FEATURE_LM) ? vaddr_bits : 32) << 8;
- *ebx |= cpufeat_mask(X86_FEATURE_IBPB);
*ebx &= hvm_featureset[FEATURESET_e8b];
break;
}
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 0f34b21..da26749 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1088,10 +1088,13 @@ void pv_cpuid(struct cpu_user_regs *regs)
special_features[FEATURESET_7b0]);
c &= pv_featureset[FEATURESET_7c0];
-
- d |= cpufeat_mask(X86_FEATURE_STIBP);
d &= pv_featureset[FEATURESET_7d0];
+ /* Force STIBP equal to IBRSB */
+ d &= ~cpufeat_mask(X86_FEATURE_STIBP);
+ if ( d & cpufeat_mask(X86_FEATURE_IBRSB) )
+ d |= cpufeat_mask(X86_FEATURE_STIBP);
+
if ( !is_pvh_domain(currd) )
{
/*
@@ -1188,7 +1191,6 @@ void pv_cpuid(struct cpu_user_regs *regs)
case 0x80000008:
a = paddr_bits | (vaddr_bits << 8);
- b |= cpufeat_mask(X86_FEATURE_IBPB);
b &= pv_featureset[FEATURESET_e8b];
break;
--
2.1.4
[-- Attachment #20: xsa263-4.7/0002-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch --]
[-- Type: application/octet-stream, Size: 3860 bytes --]
From 1c7e62d3e008af6deac600dab7699c74ccb0e994 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 12:21:00 +0100
Subject: [PATCH] x86/spec_ctrl: Read MSR_ARCH_CAPABILITIES only once
Make it available from the beginning of init_speculation_mitigations(), and
pass it into appropriate functions. Fix an RSBA typo while moving the
affected comment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d6c65187252a6c1810fd24c4d46f812840de8d3c)
---
xen/arch/x86/spec_ctrl.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index bb9ce47..15d7741 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -81,18 +81,15 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
-static void __init print_details(enum ind_thunk thunk)
+static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
unsigned int _7d0 = 0, e8b = 0, tmp;
- uint64_t caps = 0;
/* Collect diagnostics about available mitigations. */
if ( boot_cpu_data.cpuid_level >= 7 )
cpuid_count(7, 0, &tmp, &tmp, &tmp, &_7d0);
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- if ( _7d0 & cpufeat_mask(X86_FEATURE_ARCH_CAPS) )
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
@@ -125,7 +122,7 @@ static void __init print_details(enum ind_thunk thunk)
}
/* Calculate whether Retpoline is known-safe on this CPU. */
-static bool_t __init retpoline_safe(void)
+static bool_t __init retpoline_safe(uint64_t caps)
{
unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
@@ -136,19 +133,12 @@ static bool_t __init retpoline_safe(void)
boot_cpu_data.x86 != 6 )
return 0;
- if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
- {
- uint64_t caps;
-
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
-
- /*
- * RBSA may be set by a hypervisor to indicate that we may move to a
- * processor which isn't retpoline-safe.
- */
- if ( caps & ARCH_CAPS_RSBA )
- return 0;
- }
+ /*
+ * RSBA may be set by a hypervisor to indicate that we may move to a
+ * processor which isn't retpoline-safe.
+ */
+ if ( caps & ARCH_CAPS_RSBA )
+ return 0;
switch ( boot_cpu_data.x86_model )
{
@@ -218,6 +208,10 @@ void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
bool_t ibrs = 0;
+ uint64_t caps = 0;
+
+ if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
+ rdmsrl(MSR_ARCH_CAPABILITIES, caps);
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -246,7 +240,7 @@ void __init init_speculation_mitigations(void)
* On Intel hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe() )
+ else if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = 1;
@@ -331,7 +325,7 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_bti_ist_info has been calculated. */
init_shadow_spec_ctrl_state();
- print_details(thunk);
+ print_details(thunk, caps);
}
static void __init __maybe_unused build_assertions(void)
--
2.1.4
[-- Attachment #21: xsa263-4.7/0003-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch --]
[-- Type: application/octet-stream, Size: 5109 bytes --]
From 270448fdae987d12c43a9fe0eb0e2f5531654d1f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Express Xen's choice of MSR_SPEC_CTRL value as
a variable
At the moment, we have two different encodings of Xen's MSR_SPEC_CTRL value,
which is a side effect of how the Spectre series developed. One encoding is
via an alias with the bottom bit of bti_ist_info, and can encode IBRS or not,
but not other configurations such as STIBP.
Break Xen's value out into a separate variable (in the top of stack block for
XPTI reasons) and use this instead of bti_ist_info in the IST path.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 66dfae0f32bfbc899c2f3446d5ee57068cb7f957)
---
xen/arch/x86/spec_ctrl.c | 8 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/include/asm-x86/current.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 2 ++
xen/include/asm-x86/spec_ctrl_asm.h | 8 ++------
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 15d7741..0070898 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,7 @@ static int8_t __initdata opt_ibrs = -1;
static bool_t __initdata opt_rsb_native = 1;
static bool_t __initdata opt_rsb_vmexit = 1;
bool_t __read_mostly opt_ibpb = 1;
+uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_bti_ist_info;
static int __init parse_bti(const char *s)
@@ -285,11 +286,14 @@ void __init init_speculation_mitigations(void)
* guests.
*/
if ( ibrs )
+ {
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
__set_bit(X86_FEATURE_XEN_IBRS_SET, boot_cpu_data.x86_capability);
+ }
else
__set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_WRMSR | ibrs;
+ default_bti_ist_info |= BTI_IST_WRMSR;
}
/*
@@ -330,8 +334,6 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
- /* The optimised assembly relies on this alias. */
- BUILD_BUG_ON(BTI_IST_IBRS != SPEC_CTRL_IBRS);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 46012ef..1317b65 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -142,6 +142,7 @@ void __dummy__(void)
OFFSET(CPUINFO_xen_cr3, struct cpu_info, xen_cr3);
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
+ OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index c26c60a..2057b55 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -57,6 +57,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
+ uint8_t xen_spec_ctrl;
bool_t use_shadow_spec_ctrl;
uint8_t bti_ist_info;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 68c756f..ac8574f 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_bti_ist_info;
static inline void init_shadow_spec_ctrl_state(void)
@@ -34,6 +35,7 @@ static inline void init_shadow_spec_ctrl_state(void)
struct cpu_info *info = get_cpu_info();
info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->xen_spec_ctrl = default_xen_spec_ctrl;
info->bti_ist_info = default_bti_ist_info;
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 69cf3cc..9c16945 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -21,7 +21,6 @@
#define __X86_SPEC_CTRL_ASM_H__
/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_IBRS (1 << 0)
#define BTI_IST_WRMSR (1 << 1)
#define BTI_IST_RSB (1 << 2)
@@ -285,12 +284,9 @@
setz %dl
and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
- /*
- * Load Xen's intended value. SPEC_CTRL_IBRS vs 0 is encoded in the
- * bottom bit of bti_ist_info, via a deliberate alias with BTI_IST_IBRS.
- */
+ /* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
- and $BTI_IST_IBRS, %eax
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
xor %edx, %edx
wrmsr
--
2.1.4
[-- Attachment #22: xsa263-4.7/0004-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch --]
[-- Type: application/octet-stream, Size: 13253 bytes --]
From f7aa936369dcd5de184ef7b0811aaa3742a477d4 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Merge bti_ist_info and use_shadow_spec_ctrl
into spec_ctrl_flags
All 3 bits of information here are control flags for the entry/exit code
behaviour. Treat them as such, rather than having two different variables.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 5262ba2e7799001402dfe139ff944e035dfff928)
---
xen/arch/x86/acpi/power.c | 4 +--
xen/arch/x86/spec_ctrl.c | 10 ++++---
xen/arch/x86/x86_64/asm-offsets.c | 3 +--
xen/include/asm-x86/current.h | 3 +--
xen/include/asm-x86/nops.h | 5 ++--
xen/include/asm-x86/spec_ctrl.h | 10 +++----
xen/include/asm-x86/spec_ctrl_asm.h | 52 ++++++++++++++++++++-----------------
7 files changed, 45 insertions(+), 42 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index f106a5e..c125063 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -178,7 +178,7 @@ static int enter_state(u32 state)
ci = get_cpu_info();
spec_ctrl_enter_idle(ci);
/* Avoid NMI/#MC using MSR_SPEC_CTRL until we've reloaded microcode. */
- ci->bti_ist_info = 0;
+ ci->spec_ctrl_flags &= ~SCF_ist_wrmsr;
ACPI_FLUSH_CPU_CACHE();
@@ -219,7 +219,7 @@ static int enter_state(u32 state)
microcode_resume_cpu(0);
/* Re-enabled default NMI/#MC use of MSR_SPEC_CTRL. */
- ci->bti_ist_info = default_bti_ist_info;
+ ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
spec_ctrl_exit_idle(ci);
done:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 0070898..cfdb709 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -39,7 +39,7 @@ static bool_t __initdata opt_rsb_native = 1;
static bool_t __initdata opt_rsb_vmexit = 1;
bool_t __read_mostly opt_ibpb = 1;
uint8_t __read_mostly default_xen_spec_ctrl;
-uint8_t __read_mostly default_bti_ist_info;
+uint8_t __read_mostly default_spec_ctrl_flags;
static int __init parse_bti(const char *s)
{
@@ -293,7 +293,7 @@ void __init init_speculation_mitigations(void)
else
__set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_WRMSR;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
/*
@@ -312,7 +312,7 @@ void __init init_speculation_mitigations(void)
if ( opt_rsb_native )
{
__set_bit(X86_FEATURE_RSB_NATIVE, boot_cpu_data.x86_capability);
- default_bti_ist_info |= BTI_IST_RSB;
+ default_spec_ctrl_flags |= SCF_ist_rsb;
}
/*
@@ -326,7 +326,7 @@ void __init init_speculation_mitigations(void)
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
opt_ibpb = 0;
- /* (Re)init BSP state now that default_bti_ist_info has been calculated. */
+ /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
print_details(thunk, caps);
@@ -334,6 +334,8 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
+ /* The optimised assembly relies on this alias. */
+ BUILD_BUG_ON(SCF_use_shadow != 1);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 1317b65..b1fc806 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -143,8 +143,7 @@ void __dummy__(void)
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
- OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
- OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
+ OFFSET(CPUINFO_spec_ctrl_flags, struct cpu_info, spec_ctrl_flags);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
BLANK();
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index 2057b55..43aac0b 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -58,8 +58,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
uint8_t xen_spec_ctrl;
- bool_t use_shadow_spec_ctrl;
- uint8_t bti_ist_info;
+ uint8_t spec_ctrl_flags;
unsigned long __pad;
/* get_stack_bottom() must be 16-byte aligned */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index f00bd16..cab2bad 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -64,10 +64,9 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP21 ASM_NOP8; ASM_NOP8; ASM_NOP5
+#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP29 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP5
-#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index ac8574f..4dc4dfa 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -28,15 +28,15 @@ void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
extern uint8_t default_xen_spec_ctrl;
-extern uint8_t default_bti_ist_info;
+extern uint8_t default_spec_ctrl_flags;
static inline void init_shadow_spec_ctrl_state(void)
{
struct cpu_info *info = get_cpu_info();
- info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->shadow_spec_ctrl = 0;
info->xen_spec_ctrl = default_xen_spec_ctrl;
- info->bti_ist_info = default_bti_ist_info;
+ info->spec_ctrl_flags = default_spec_ctrl_flags;
}
/* WARNING! `ret`, `call *`, `jmp *` not safe after this call. */
@@ -50,7 +50,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
*/
info->shadow_spec_ctrl = val;
barrier();
- info->use_shadow_spec_ctrl = 1;
+ info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
* Disable shadowing before updating the MSR. There are no SMP issues
* here; only local processor ordering concerns.
*/
- info->use_shadow_spec_ctrl = 0;
+ info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 9c16945..582403a 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -20,9 +20,10 @@
#ifndef __X86_SPEC_CTRL_ASM_H__
#define __X86_SPEC_CTRL_ASM_H__
-/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_WRMSR (1 << 1)
-#define BTI_IST_RSB (1 << 2)
+/* Encoding of cpuinfo.spec_ctrl_flags */
+#define SCF_use_shadow (1 << 0)
+#define SCF_ist_wrmsr (1 << 1)
+#define SCF_ist_rsb (1 << 2)
#ifdef __ASSEMBLY__
#include <asm/msr-index.h>
@@ -49,20 +50,20 @@
* after VMEXIT. The VMEXIT-specific code reads MSR_SPEC_CTRL and updates
* current before loading Xen's MSR_SPEC_CTRL setting.
*
- * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and
- * use_shadow_spec_ctrl boolean per cpu. The synchronous use is:
+ * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and a use_shadow
+ * boolean in the per cpu spec_ctrl_flags. The synchronous use is:
*
* 1) Store guest value in shadow_spec_ctrl
- * 2) Set use_shadow_spec_ctrl boolean
+ * 2) Set the use_shadow boolean
* 3) Load guest value into MSR_SPEC_CTRL
* 4) Exit to guest
* 5) Entry from guest
- * 6) Clear use_shadow_spec_ctrl boolean
+ * 6) Clear the use_shadow boolean
* 7) Load Xen's value into MSR_SPEC_CTRL
*
* The asynchronous use for interrupts/exceptions is:
* - Set/clear IBRS on entry to Xen
- * - On exit to Xen, check use_shadow_spec_ctrl
+ * - On exit to Xen, check use_shadow
* - If set, load shadow_spec_ctrl
*
* Therefore, an interrupt/exception which hits the synchronous path between
@@ -133,7 +134,7 @@
xor %edx, %edx
/* Clear SPEC_CTRL shadowing *before* loading Xen's value. */
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
mov $\ibrs_val, %eax
@@ -159,12 +160,14 @@
* block so calculate the position directly.
*/
.if \maybexen
+ xor %eax, %eax
/* Branchless `if ( !xen ) clear_shadowing` */
testb $3, UREGS_cs(%rsp)
- setz %al
- and %al, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %al
+ not %eax
+ and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
.else
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
.endif
/* Load Xen's intended value. */
@@ -183,8 +186,8 @@
*/
xor %edx, %edx
- cmpb %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%rbx)
- je .L\@_skip
+ testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
+ jz .L\@_skip
mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax
mov $MSR_SPEC_CTRL, %ecx
@@ -205,7 +208,7 @@
mov %eax, CPUINFO_shadow_spec_ctrl(%rsp)
/* Set SPEC_CTRL shadowing *before* loading the guest value. */
- movb $1, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ orb $SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
mov $MSR_SPEC_CTRL, %ecx
xor %edx, %edx
@@ -216,7 +219,7 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -228,7 +231,7 @@
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP21), \
+ ALTERNATIVE_2 __stringify(ASM_NOP22), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -239,7 +242,7 @@
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP29), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -267,22 +270,23 @@
* This is logical merge of DO_OVERWRITE_RSB and DO_SPEC_CTRL_ENTRY
* maybexen=1, but with conditionals rather than alternatives.
*/
- movzbl STACK_CPUINFO_FIELD(bti_ist_info)(%r14), %eax
+ movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %eax
- testb $BTI_IST_RSB, %al
+ test $SCF_ist_rsb, %al
jz .L\@_skip_rsb
DO_OVERWRITE_RSB tmp=rdx /* Clobbers %rcx/%rdx */
.L\@_skip_rsb:
- testb $BTI_IST_WRMSR, %al
+ test $SCF_ist_wrmsr, %al
jz .L\@_skip_wrmsr
xor %edx, %edx
testb $3, UREGS_cs(%rsp)
- setz %dl
- and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %dl
+ not %edx
+ and %dl, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
/* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
@@ -309,7 +313,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
* Requires %rbx=stack_end
* Clobbers %rax, %rcx, %rdx
*/
- testb $BTI_IST_WRMSR, STACK_CPUINFO_FIELD(bti_ist_info)(%rbx)
+ testb $SCF_ist_wrmsr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
jz .L\@_skip
DO_SPEC_CTRL_EXIT_TO_XEN
--
2.1.4
[-- Attachment #23: xsa263-4.7/0005-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch --]
[-- Type: application/octet-stream, Size: 11412 bytes --]
From 25ab5524cba628a2a494cb369ed9fbcf2e81ba34 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Fold the XEN_IBRS_{SET,CLEAR} ALTERNATIVES
together
Currently, the SPEC_CTRL_{ENTRY,EXIT}_* macros encode Xen's choice of
MSR_SPEC_CTRL as an immediate constant, and chooses between IBRS or not by
doubling up the entire alternative block.
There is now a variable holding Xen's choice of value, so use that and
simplify the alternatives.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit af949407eaba7af71067f23d5866cd0bf1f1144d)
---
xen/arch/x86/cpu/common.c | 8 ++-----
xen/arch/x86/spec_ctrl.c | 12 +++++-----
xen/include/asm-x86/cpufeature.h | 3 +--
xen/include/asm-x86/nops.h | 3 ++-
xen/include/asm-x86/spec_ctrl.h | 6 ++---
xen/include/asm-x86/spec_ctrl_asm.h | 45 +++++++++++++------------------------
6 files changed, 28 insertions(+), 49 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 43007c5..335d464 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -417,13 +417,9 @@ void identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_IND_THUNK_JMP,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_IND_THUNK_JMP, c->x86_capability);
- if (test_bit(X86_FEATURE_XEN_IBRS_SET,
+ if (test_bit(X86_FEATURE_SC_MSR,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_XEN_IBRS_SET, c->x86_capability);
- if (test_bit(X86_FEATURE_XEN_IBRS_CLEAR,
- boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_XEN_IBRS_CLEAR,
- c->x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
if (test_bit(X86_FEATURE_RSB_NATIVE,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_RSB_NATIVE, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index cfdb709..fc35fe8 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,8 +112,9 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_XEN_IBRS_SET) ? " IBRS+" :
- boot_cpu_has(X86_FEATURE_XEN_IBRS_CLEAR) ? " IBRS-" : "",
+ boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
+ " IBRS-" : "",
opt_ibpb ? " IBPB" : "",
boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
@@ -285,13 +286,10 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
+ __set_bit(X86_FEATURE_SC_MSR, boot_cpu_data.x86_capability);
+
if ( ibrs )
- {
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- __set_bit(X86_FEATURE_XEN_IBRS_SET, boot_cpu_data.x86_capability);
- }
- else
- __set_bit(X86_FEATURE_XEN_IBRS_CLEAR, boot_cpu_data.x86_capability);
default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 6a18755..66efc5b 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -30,8 +30,7 @@
#define X86_FEATURE_IND_THUNK_LFENCE ((FSCAPINTS+0)*32+ 11) /* Use IND_THUNK_LFENCE */
#define X86_FEATURE_IND_THUNK_JMP ((FSCAPINTS+0)*32+ 12) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB ((FSCAPINTS+0)*32+ 13) /* IBRSB || IBPB */
-#define X86_FEATURE_XEN_IBRS_SET ((FSCAPINTS+0)*32+ 14) /* IBRSB && IRBS set in Xen */
-#define X86_FEATURE_XEN_IBRS_CLEAR ((FSCAPINTS+0)*32+ 15) /* IBRSB && IBRS clear in Xen */
+#define X86_FEATURE_SC_MSR ((FSCAPINTS+0)*32+ 14) /* MSR_SPEC_CTRL used by Xen */
#define X86_FEATURE_RSB_NATIVE ((FSCAPINTS+0)*32+ 16) /* RSB overwrite needed for native */
#define X86_FEATURE_RSB_VMEXIT ((FSCAPINTS+0)*32+ 17) /* RSB overwrite needed for vmexit */
#define X86_FEATURE_NO_XPTI ((FSCAPINTS+0)*32+ 18) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index cab2bad..ad32c2e 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -64,9 +64,10 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP25 ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 4dc4dfa..6c11562 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -54,14 +54,14 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_XEN_IBRS_SET)
+ "i" (X86_FEATURE_SC_MSR)
: "memory" );
}
/* WARNING! `ret`, `call *`, `jmp *` not safe before this call. */
static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
{
- uint32_t val = SPEC_CTRL_IBRS;
+ uint32_t val = info->xen_spec_ctrl;
/*
* Disable shadowing before updating the MSR. There are no SMP issues
@@ -71,7 +71,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_XEN_IBRS_SET)
+ "i" (X86_FEATURE_SC_MSR)
: "memory" );
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 582403a..941aeb7 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -117,7 +117,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -137,11 +137,11 @@
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
- mov $\ibrs_val, %eax
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
wrmsr
.endm
-.macro DO_SPEC_CTRL_ENTRY maybexen:req ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY maybexen:req
/*
* Requires %rsp=regs (also cpuinfo if !maybexen)
* Requires %r14=stack_end (if maybexen)
@@ -166,12 +166,12 @@
setnz %al
not %eax
and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
.else
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
.endif
- /* Load Xen's intended value. */
- mov $\ibrs_val, %eax
wrmsr
.endm
@@ -219,47 +219,32 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP36), \
+ DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP22), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP25), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP33), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
- ALTERNATIVE_2 __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP17), \
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
/* Use when exiting to guest context. */
#define SPEC_CTRL_EXIT_TO_GUEST \
- ALTERNATIVE_2 __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
--
2.1.4
[-- Attachment #24: xsa263-4.7/0006-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch --]
[-- Type: application/octet-stream, Size: 13347 bytes --]
From c71d5690d40f12cd75cf5dea1e245a59b29cb5d5 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 30 Apr 2018 14:20:23 +0100
Subject: [PATCH] x86/spec_ctrl: Rename bits of infrastructure to avoid NATIVE
and VMEXIT
In hindsight, using NATIVE and VMEXIT as naming terminology was not clever.
A future change wants to split SPEC_CTRL_EXIT_TO_GUEST into PV and HVM
specific implementations, and using VMEXIT as a term is completely wrong.
Take the opportunity to fix some stale documentation in spec_ctrl_asm.h. The
IST helpers were missing from the large comment block, and since
SPEC_CTRL_ENTRY_FROM_INTR_IST was introduced, we've gained a new piece of
functionality which currently depends on the fine grain control, which exists
in lieu of livepatching. Note this in the comment.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d9822b8a38114e96e4516dc998f4055249364d5d)
---
xen/arch/x86/cpu/common.c | 8 ++++----
xen/arch/x86/hvm/svm/entry.S | 4 ++--
xen/arch/x86/hvm/vmx/entry.S | 4 ++--
xen/arch/x86/spec_ctrl.c | 20 ++++++++++----------
xen/arch/x86/x86_64/compat/entry.S | 2 +-
xen/arch/x86/x86_64/entry.S | 2 +-
xen/include/asm-x86/cpufeature.h | 4 ++--
xen/include/asm-x86/spec_ctrl_asm.h | 36 +++++++++++++++++++++++++-----------
8 files changed, 47 insertions(+), 33 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 335d464..d6dff4c 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -420,12 +420,12 @@ void identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_SC_MSR,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
- if (test_bit(X86_FEATURE_RSB_NATIVE,
+ if (test_bit(X86_FEATURE_SC_RSB_PV,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_RSB_NATIVE, c->x86_capability);
- if (test_bit(X86_FEATURE_RSB_VMEXIT,
+ __set_bit(X86_FEATURE_SC_RSB_PV, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_RSB_HVM,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_RSB_VMEXIT, c->x86_capability);
+ __set_bit(X86_FEATURE_SC_RSB_HVM, c->x86_capability);
if (test_bit(X86_FEATURE_NO_XPTI,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_NO_XPTI, c->x86_capability);
diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S
index 289e946..d824bcd 100644
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -81,7 +81,7 @@ UNLIKELY_END(svm_trace)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
pop %r15
pop %r14
@@ -106,7 +106,7 @@ UNLIKELY_END(svm_trace)
GET_CURRENT(bx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov VCPU_svm_vmcb(%rbx),%rcx
diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S
index 7aa0e85..f1528e8 100644
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -37,7 +37,7 @@ ENTRY(vmx_asm_vmexit_handler)
movb $1,VCPU_vmx_launched(%rbx)
mov %rax,VCPU_hvm_guest_cr2(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov %rsp,%rdi
@@ -72,7 +72,7 @@ UNLIKELY_END(realmode)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
mov VCPU_hvm_guest_cr2(%rbx),%rax
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index fc35fe8..a67daa2 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -35,8 +35,8 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool_t __initdata opt_rsb_native = 1;
-static bool_t __initdata opt_rsb_vmexit = 1;
+static bool_t __initdata opt_rsb_pv = 1;
+static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -69,9 +69,9 @@ static int __init parse_bti(const char *s)
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
else if ( (val = parse_boolean("rsb_native", s, ss)) >= 0 )
- opt_rsb_native = val;
+ opt_rsb_pv = val;
else if ( (val = parse_boolean("rsb_vmexit", s, ss)) >= 0 )
- opt_rsb_vmexit = val;
+ opt_rsb_hvm = val;
else
rc = -EINVAL;
@@ -116,8 +116,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -307,9 +307,9 @@ void __init init_speculation_mitigations(void)
* If a processors speculates to 32bit PV guest kernel mappings, it is
* speculating in 64bit supervisor mode, and can leak data.
*/
- if ( opt_rsb_native )
+ if ( opt_rsb_pv )
{
- __set_bit(X86_FEATURE_RSB_NATIVE, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_RSB_PV, boot_cpu_data.x86_capability);
default_spec_ctrl_flags |= SCF_ist_rsb;
}
@@ -317,8 +317,8 @@ void __init init_speculation_mitigations(void)
* HVM guests can always poison the RSB to point at Xen supervisor
* mappings.
*/
- if ( opt_rsb_vmexit )
- __set_bit(X86_FEATURE_RSB_VMEXIT, boot_cpu_data.x86_capability);
+ if ( opt_rsb_hvm )
+ __set_bit(X86_FEATURE_SC_RSB_HVM, boot_cpu_data.x86_capability);
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index 40f4400..3865225 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -237,7 +237,7 @@ ENTRY(compat_restore_all_guest)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL adj=8 compat=1
.Lft0: iretq
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index df265ec..1c4f014 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -78,7 +78,7 @@ restore_all_guest:
mov %r15d, %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL
testw $TRAP_syscall,4(%rsp)
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 66efc5b..e472176 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -31,8 +31,8 @@
#define X86_FEATURE_IND_THUNK_JMP ((FSCAPINTS+0)*32+ 12) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB ((FSCAPINTS+0)*32+ 13) /* IBRSB || IBPB */
#define X86_FEATURE_SC_MSR ((FSCAPINTS+0)*32+ 14) /* MSR_SPEC_CTRL used by Xen */
-#define X86_FEATURE_RSB_NATIVE ((FSCAPINTS+0)*32+ 16) /* RSB overwrite needed for native */
-#define X86_FEATURE_RSB_VMEXIT ((FSCAPINTS+0)*32+ 17) /* RSB overwrite needed for vmexit */
+#define X86_FEATURE_SC_RSB_PV ((FSCAPINTS+0)*32+ 16) /* RSB overwrite needed for PV */
+#define X86_FEATURE_SC_RSB_HVM ((FSCAPINTS+0)*32+ 17) /* RSB overwrite needed for HVM */
#define X86_FEATURE_NO_XPTI ((FSCAPINTS+0)*32+ 18) /* XPTI mitigation not in use */
#define cpufeat_word(idx) ((idx) / 32)
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 941aeb7..b330e20 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -72,11 +72,14 @@
*
* The following ASM fragments implement this algorithm. See their local
* comments for further details.
- * - SPEC_CTRL_ENTRY_FROM_VMEXIT
+ * - SPEC_CTRL_ENTRY_FROM_HVM
* - SPEC_CTRL_ENTRY_FROM_PV
* - SPEC_CTRL_ENTRY_FROM_INTR
+ * - SPEC_CTRL_ENTRY_FROM_INTR_IST
+ * - SPEC_CTRL_EXIT_TO_XEN_IST
* - SPEC_CTRL_EXIT_TO_XEN
- * - SPEC_CTRL_EXIT_TO_GUEST
+ * - SPEC_CTRL_EXIT_TO_PV
+ * - SPEC_CTRL_EXIT_TO_HVM
*/
.macro DO_OVERWRITE_RSB tmp=rax
@@ -117,7 +120,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
+.macro DO_SPEC_CTRL_ENTRY_FROM_HVM
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -216,23 +219,23 @@
.endm
/* Use after a VMEXIT from an HVM guest. */
-#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
+#define SPEC_CTRL_ENTRY_FROM_HVM \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
@@ -241,12 +244,22 @@
ALTERNATIVE __stringify(ASM_NOP17), \
DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
-/* Use when exiting to guest context. */
-#define SPEC_CTRL_EXIT_TO_GUEST \
+/* Use when exiting to PV guest context. */
+#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
-/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
+/* Use when exiting to HVM guest context. */
+#define SPEC_CTRL_EXIT_TO_HVM \
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+
+/*
+ * Use in IST interrupt/exception context. May interrupt Xen or PV context.
+ * Fine grain control of SCF_ist_wrmsr is needed for safety in the S3 resume
+ * path to avoid using MSR_SPEC_CTRL before the microcode introducing it has
+ * been reloaded.
+ */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
/*
* Requires %rsp=regs, %r14=stack_end
@@ -293,6 +306,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
UNLIKELY_END(\@_serialise)
.endm
+/* Use when exiting to Xen in IST context. */
.macro SPEC_CTRL_EXIT_TO_XEN_IST
/*
* Requires %rbx=stack_end
--
2.1.4
[-- Attachment #25: xsa263-4.7/0007-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch --]
[-- Type: application/octet-stream, Size: 3684 bytes --]
From 2e69f13af14b9df89bec7bd99fc32b9547ece72d Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 May 2018 14:06:16 +0100
Subject: [PATCH] x86/spec_ctrl: Elide MSR_SPEC_CTRL handling in idle context
when possible
If Xen is virtualising MSR_SPEC_CTRL handling for guests, but using 0 as its
own MSR_SPEC_CTRL value, spec_ctrl_{enter,exit}_idle() need not write to the
MSR.
Requested-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 94df6e8588e35cc2028ccb3fd2921c6e6360605e)
---
xen/arch/x86/cpu/common.c | 3 +++
xen/arch/x86/spec_ctrl.c | 4 ++++
xen/include/asm-x86/cpufeature.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 4 ++--
4 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index d6dff4c..e1c479a 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -429,6 +429,9 @@ void identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_NO_XPTI,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_NO_XPTI, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_MSR_IDLE,
+ boot_cpu_data.x86_capability))
+ __set_bit(X86_FEATURE_SC_MSR_IDLE, c->x86_capability);
/* AND the already accumulated flags with these */
for ( i = 0 ; i < NCAPINTS ; i++ )
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index a67daa2..f5dd14e 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -327,6 +327,10 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
+ /* If Xen is using any MSR_SPEC_CTRL settings, adjust the idle path. */
+ if ( default_xen_spec_ctrl )
+ __set_bit(X86_FEATURE_SC_MSR_IDLE, boot_cpu_data.x86_capability);
+
print_details(thunk, caps);
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index e472176..2466f5b 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -34,6 +34,7 @@
#define X86_FEATURE_SC_RSB_PV ((FSCAPINTS+0)*32+ 16) /* RSB overwrite needed for PV */
#define X86_FEATURE_SC_RSB_HVM ((FSCAPINTS+0)*32+ 17) /* RSB overwrite needed for HVM */
#define X86_FEATURE_NO_XPTI ((FSCAPINTS+0)*32+ 18) /* XPTI mitigation not in use */
+#define X86_FEATURE_SC_MSR_IDLE ((FSCAPINTS+0)*32+ 19) /* SC_MSR && default_xen_spec_ctrl */
#define cpufeat_word(idx) ((idx) / 32)
#define cpufeat_bit(idx) ((idx) % 32)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 6c11562..ec943e1 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -54,7 +54,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_SC_MSR)
+ "i" (X86_FEATURE_SC_MSR_IDLE)
: "memory" );
}
@@ -71,7 +71,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", %c3)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0),
- "i" (X86_FEATURE_SC_MSR)
+ "i" (X86_FEATURE_SC_MSR_IDLE)
: "memory" );
}
--
2.1.4
[-- Attachment #26: xsa263-4.7/0008-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch --]
[-- Type: application/octet-stream, Size: 7050 bytes --]
From 77d5b370bb9d3db4fc969884929a4eec88c4a8bd Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Split X86_FEATURE_SC_MSR into PV and HVM
variants
In order to separately control whether MSR_SPEC_CTRL is virtualised for PV and
HVM guests, split the feature used to control runtime alternatives into two.
Xen will use MSR_SPEC_CTRL itself if either of these features are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit fa9eb09d446a1279f5e861e6b84fa8675dabf148)
---
xen/arch/x86/cpu/common.c | 7 +++++--
xen/arch/x86/spec_ctrl.c | 6 ++++--
xen/include/asm-x86/cpufeature.h | 5 +++--
xen/include/asm-x86/spec_ctrl_asm.h | 12 ++++++------
4 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index e1c479a..147c83d 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -417,9 +417,12 @@ void identify_cpu(struct cpuinfo_x86 *c)
if (test_bit(X86_FEATURE_IND_THUNK_JMP,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_IND_THUNK_JMP, c->x86_capability);
- if (test_bit(X86_FEATURE_SC_MSR,
+ if (test_bit(X86_FEATURE_SC_MSR_PV,
boot_cpu_data.x86_capability))
- __set_bit(X86_FEATURE_SC_MSR, c->x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_PV, c->x86_capability);
+ if (test_bit(X86_FEATURE_SC_MSR_HVM,
+ boot_cpu_data.x86_capability))
+ __set_bit(X86_FEATURE_SC_MSR_HVM, c->x86_capability);
if (test_bit(X86_FEATURE_SC_RSB_PV,
boot_cpu_data.x86_capability))
__set_bit(X86_FEATURE_SC_RSB_PV, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index f5dd14e..f31fa6b 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,7 +112,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
@@ -286,7 +287,8 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
- __set_bit(X86_FEATURE_SC_MSR, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
+ __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
if ( ibrs )
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 2466f5b..d6b9951 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -30,11 +30,12 @@
#define X86_FEATURE_IND_THUNK_LFENCE ((FSCAPINTS+0)*32+ 11) /* Use IND_THUNK_LFENCE */
#define X86_FEATURE_IND_THUNK_JMP ((FSCAPINTS+0)*32+ 12) /* Use IND_THUNK_JMP */
#define X86_FEATURE_XEN_IBPB ((FSCAPINTS+0)*32+ 13) /* IBRSB || IBPB */
-#define X86_FEATURE_SC_MSR ((FSCAPINTS+0)*32+ 14) /* MSR_SPEC_CTRL used by Xen */
+#define X86_FEATURE_SC_MSR_PV ((FSCAPINTS+0)*32+ 14) /* MSR_SPEC_CTRL used by Xen for PV */
+#define X86_FEATURE_SC_MSR_HVM ((FSCAPINTS+0)*32+ 15) /* MSR_SPEC_CTRL used by Xen for HVM */
#define X86_FEATURE_SC_RSB_PV ((FSCAPINTS+0)*32+ 16) /* RSB overwrite needed for PV */
#define X86_FEATURE_SC_RSB_HVM ((FSCAPINTS+0)*32+ 17) /* RSB overwrite needed for HVM */
#define X86_FEATURE_NO_XPTI ((FSCAPINTS+0)*32+ 18) /* XPTI mitigation not in use */
-#define X86_FEATURE_SC_MSR_IDLE ((FSCAPINTS+0)*32+ 19) /* SC_MSR && default_xen_spec_ctrl */
+#define X86_FEATURE_SC_MSR_IDLE ((FSCAPINTS+0)*32+ 19) /* (SC_MSR_PV || SC_MSR_HVM) && default_xen_spec_ctrl */
#define cpufeat_word(idx) ((idx) / 32)
#define cpufeat_bit(idx) ((idx) % 32)
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index b330e20..4d864eb 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -223,36 +223,36 @@
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR_HVM
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR_PV
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR_PV
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
ALTERNATIVE __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV
/* Use when exiting to PV guest context. */
#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV
/* Use when exiting to HVM guest context. */
#define SPEC_CTRL_EXIT_TO_HVM \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_HVM
/*
* Use in IST interrupt/exception context. May interrupt Xen or PV context.
--
2.1.4
[-- Attachment #27: xsa263-4.7/0009-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch --]
[-- Type: application/octet-stream, Size: 4805 bytes --]
From 9df9a4ee5129cdb1b185abff94f9c2b7b0e820f9 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 9 May 2018 13:59:56 +0100
Subject: [PATCH] x86/spec_ctrl: Explicitly set Xen's default MSR_SPEC_CTRL
value
With the impending ability to disable MSR_SPEC_CTRL handling on a
per-guest-type basis, the first exit-from-guest may not have the side effect
of loading Xen's choice of value. Explicitly set Xen's default during the BSP
and AP boot paths.
For the BSP however, delay setting a non-zero MSR_SPEC_CTRL default until
after dom0 has been constructed when safe to do so. Oracle report that this
speeds up boots of some hardware by 50s.
"when safe to do so" is based on whether we are virtualised. A native boot
won't have any other code running in a position to mount an attack.
Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb8c12020307b39a89273d7699e89000451987ab)
---
xen/arch/x86/setup.c | 7 +++++++
xen/arch/x86/smpboot.c | 8 ++++++++
xen/arch/x86/spec_ctrl.c | 32 ++++++++++++++++++++++++++++++++
xen/include/asm-x86/spec_ctrl.h | 2 ++
4 files changed, 49 insertions(+)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index e04a1e4..0ad69d0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1621,6 +1621,13 @@ void __init noreturn __start_xen(unsigned long mbi_p)
setup_io_bitmap(dom0);
+ if ( bsp_delay_spec_ctrl )
+ {
+ get_cpu_info()->spec_ctrl_flags &= ~SCF_use_shadow;
+ barrier();
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+ }
+
/* Jump to the 1:1 virtual mappings of cpu0_stack. */
asm volatile ("mov %[stk], %%rsp; jmp %c[fn]" ::
[stk] "g" (__va(__pa(get_stack_bottom()))),
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 51c24fa..f6abecd 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -344,6 +344,14 @@ void start_secondary(void *unused)
else
microcode_resume_cpu(cpu);
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. Note: MSR_SPEC_CTRL may only become available
+ * after loading microcode.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+
smp_callin();
setup_secondary_APIC_clock();
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index f31fa6b..c3e940f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,8 @@ static int8_t __initdata opt_ibrs = -1;
static bool_t __initdata opt_rsb_pv = 1;
static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
+
+bool_t __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -334,6 +336,36 @@ void __init init_speculation_mitigations(void)
__set_bit(X86_FEATURE_SC_MSR_IDLE, boot_cpu_data.x86_capability);
print_details(thunk, caps);
+
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. For performance reasons, when safe to do so, we
+ * delay applying non-zero settings until after dom0 has been constructed.
+ *
+ * "when safe to do so" is based on whether we are virtualised. A native
+ * boot won't have any other code running in a position to mount an
+ * attack.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ {
+ bsp_delay_spec_ctrl = !cpu_has_hypervisor && default_xen_spec_ctrl;
+
+ /*
+ * If delaying MSR_SPEC_CTRL setup, use the same mechanism as
+ * spec_ctrl_enter_idle(), by using a shadow value of zero.
+ */
+ if ( bsp_delay_spec_ctrl )
+ {
+ struct cpu_info *info = get_cpu_info();
+
+ info->shadow_spec_ctrl = 0;
+ barrier();
+ info->spec_ctrl_flags |= SCF_use_shadow;
+ barrier();
+ }
+
+ wrmsrl(MSR_SPEC_CTRL, bsp_delay_spec_ctrl ? 0 : default_xen_spec_ctrl);
+ }
}
static void __init __maybe_unused build_assertions(void)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index ec943e1..d36f0e9 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,8 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+
+extern bool_t bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_spec_ctrl_flags;
--
2.1.4
[-- Attachment #28: xsa263-4.7/0010-x86-cpuid-Improvements-to-guest-policies-for-specula.patch --]
[-- Type: application/octet-stream, Size: 5476 bytes --]
From f9949a56cc6d7f223233f5a8a95cd115de6f1449 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 1 May 2018 11:59:03 +0100
Subject: [PATCH] x86/cpuid: Improvements to guest policies for speculative
sidechannel features
If Xen isn't virtualising MSR_SPEC_CTRL for guests, IBRSB shouldn't be
advertised. It is not currently possible to express this via the existing
command line options, but such an ability will be introduced.
Another useful option in some usecases is to offer IBPB without IBRS. When a
guest kernel is known to be compatible (uses retpoline and knows about the AMD
IBPB feature bit), an administrator with pre-Skylake hardware may wish to hide
IBRS. This allows the VM to have full protection, without Xen or the VM
needing to touch MSR_SPEC_CTRL, which can reduce the overhead of Spectre
mitigations.
Break the logic common to both PV and HVM CPUID calculations into a common
helper, to avoid duplication.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb06b308ec71b23f37a44f5e2351fe2cae0306e9)
---
xen/arch/x86/cpuid.c | 60 ++++++++++++++++++++++++++++++++--------------------
1 file changed, 37 insertions(+), 23 deletions(-)
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index fffcecd..bade364 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -136,6 +136,28 @@ static void __init calculate_raw_featureset(void)
&tmp, &tmp);
}
+static void __init guest_common_feature_adjustments(uint32_t *fs)
+{
+ /* Unconditionally claim to be able to set the hypervisor bit. */
+ __set_bit(X86_FEATURE_HYPERVISOR, fs);
+
+ /*
+ * If IBRS is offered to the guest, unconditionally offer STIBP. It is a
+ * nop on non-HT hardware, and has this behaviour to make heterogeneous
+ * setups easier to manage.
+ */
+ if ( test_bit(X86_FEATURE_IBRSB, fs) )
+ __set_bit(X86_FEATURE_STIBP, fs);
+
+ /*
+ * On hardware which supports IBRS/IBPB, we can offer IBPB independently
+ * of IBRS by using the AMD feature bit. An administrator may wish for
+ * performance reasons to offer IBPB without IBRS.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ __set_bit(X86_FEATURE_IBPB, fs);
+}
+
static void __init calculate_pv_featureset(void)
{
unsigned int i;
@@ -143,9 +165,6 @@ static void __init calculate_pv_featureset(void)
for ( i = 0; i < FSCAPINTS; ++i )
pv_featureset[i] = host_featureset[i] & pv_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, pv_featureset);
-
/*
* Allow the toolstack to set HTT, X2APIC and CMP_LEGACY. These bits
* affect how to interpret topology information in other cpuid leaves.
@@ -154,15 +173,14 @@ static void __init calculate_pv_featureset(void)
__set_bit(X86_FEATURE_X2APIC, pv_featureset);
__set_bit(X86_FEATURE_CMP_LEGACY, pv_featureset);
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, pv_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, pv_featureset);
+ /*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for PV guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ __clear_bit(X86_FEATURE_IBRSB, pv_featureset);
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, pv_featureset);
- }
+ guest_common_feature_adjustments(pv_featureset);
sanitise_featureset(pv_featureset);
}
@@ -181,9 +199,6 @@ static void __init calculate_hvm_featureset(void)
for ( i = 0; i < FSCAPINTS; ++i )
hvm_featureset[i] = host_featureset[i] & hvm_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, hvm_featureset);
-
/*
* Allow the toolstack to set HTT, X2APIC and CMP_LEGACY. These bits
* affect how to interpret topology information in other cpuid leaves.
@@ -208,6 +223,13 @@ static void __init calculate_hvm_featureset(void)
__set_bit(X86_FEATURE_SEP, hvm_featureset);
/*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for HVM guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_HVM) )
+ __clear_bit(X86_FEATURE_IBRSB, hvm_featureset);
+
+ /*
* With VT-x, some features are only supported by Xen if dedicated
* hardware support is also available.
*/
@@ -220,15 +242,7 @@ static void __init calculate_hvm_featureset(void)
__clear_bit(X86_FEATURE_XSAVES, hvm_featureset);
}
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, hvm_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, hvm_featureset);
-
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, hvm_featureset);
- }
+ guest_common_feature_adjustments(hvm_featureset);
sanitise_featureset(hvm_featureset);
}
--
2.1.4
[-- Attachment #29: xsa263-4.7/0011-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch --]
[-- Type: application/octet-stream, Size: 14196 bytes --]
From cbb12e58536a082e5f1843afe1d42e9aa60d4b2c Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:52:55 +0100
Subject: [PATCH] x86/spec_ctrl: Introduce a new `spec-ctrl=` command line
argument to replace `bti=`
In hindsight, the options for `bti=` aren't as flexible or useful as expected
(including several options which don't appear to behave as intended).
Changing the behaviour of an existing option is problematic for compatibility,
so introduce a new `spec-ctrl=` in the hopes that we can do better.
One common way of deploying Xen is with a single PV dom0 and all domUs being
HVM domains. In such a setup, an administrator who has weighed up the risks
may wish to forgo protection against malicious PV domains, to reduce the
overall performance hit. To cater for this usecase, `spec-ctrl=no-pv` will
disable all speculative protection for PV domains, while leaving all
speculative protection for HVM domains intact.
For coding clarity as much as anything else, the suboptions are grouped by
logical area; those which affect the alternatives blocks, and those which
affect Xen's in-hypervisor settings. See the xen-command-line.markdown for
full details of the new options.
While changing the command line options, take the time to change how the data
is reported to the user. The three DEBUG printks are upgraded to unilateral,
as they are all relevant pieces of information, and the old "mitigations:"
line is split in the two logical areas described above.
Sample output from booting with `spec-ctrl=no-pv` looks like:
(XEN) Speculative mitigation facilities:
(XEN) Hardware features: IBRS/IBPB STIBP IBPB
(XEN) Compiled-in support: INDIRECT_THUNK
(XEN) Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS-, Other: IBPB
(XEN) Support for VMs: PV: None, HVM: MSR_SPEC_CTRL RSB
(XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 3352afc26c497d26ecb70527db3cb29daf7b1422)
---
docs/misc/xen-command-line.markdown | 49 +++++++++++
xen/arch/x86/spec_ctrl.c | 160 ++++++++++++++++++++++++++++++------
2 files changed, 186 insertions(+), 23 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 2f61118..456979d 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -247,6 +247,9 @@ the NMI watchdog is also enabled.
### bti (x86)
> `= List of [ thunk=retpoline|lfence|jmp, ibrs=<bool>, ibpb=<bool>, rsb_{vmexit,native}=<bool> ]`
+**WARNING: This command line option is deprecated, and superseded by
+_spec-ctrl=_ - using both options in combination is undefined.**
+
Branch Target Injection controls. By default, Xen will pick the most
appropriate BTI mitigations based on compiled in support, loaded microcode,
and hardware details.
@@ -1443,6 +1446,52 @@ enforces the maximum theoretically necessary timeout of 670ms. Any number
is being interpreted as a custom timeout in milliseconds. Zero or boolean
false disable the quirk workaround, which is also the default.
+### spec-ctrl (x86)
+> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+
+Controls for speculative execution sidechannel mitigations. By default, Xen
+will pick the most appropriate mitigations based on compiled in support,
+loaded microcode, and hardware details, and will virtualise appropriate
+mitigations for guests to use.
+
+**WARNING: Any use of this option may interfere with heuristics. Use with
+extreme care.**
+
+An overall boolean value, `spec-ctrl=no`, can be specified to turn off all
+mitigations, including pieces of infrastructure used to virtualise certain
+mitigation features for guests. Alternatively, a slightly more restricted
+`spec-ctrl=no-xen` can be used to turn off all of Xen's mitigations, while
+leaving the virtualisation support in place for guests to use. Use of a
+positive boolean value for either of these options is invalid.
+
+The booleans `pv=`, `hvm=`, `msr-sc=` and `rsb=` offer fine grained control
+over the alternative blocks used by Xen. These impact Xen's ability to
+protect itself, and Xen's ability to virtualise support for guests to use.
+
+* `pv=` and `hvm=` offer control over all suboptions for PV and HVM guests
+ respectively.
+* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
+ on entry and exit. These blocks are necessary to virtualise support for
+ guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
+ Return Address Stack on entry to Xen.
+
+If Xen was compiled with INDIRECT\_THUNK support, `bti-thunk=` can be used to
+select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
+locations. The default thunk is `retpoline` (generally preferred for Intel
+hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
+overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+
+On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
+`ibrs=` option can be used to force or prevent Xen using the feature itself.
+If Xen is not using IBRS itself, functionality is still set up so IBRS can be
+virtualised for guests.
+
+On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
+option can be used to force (the default) or prevent Xen from issuing branch
+prediction barriers on vcpu context switches.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index c3e940f..347cd14 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -26,6 +26,13 @@
#include <asm/spec_ctrl.h>
#include <asm/spec_ctrl_asm.h>
+/* Cmdline controls for Xen's alternative blocks. */
+static bool_t __initdata opt_msr_sc_pv = 1;
+static bool_t __initdata opt_msr_sc_hvm = 1;
+static bool_t __initdata opt_rsb_pv = 1;
+static bool_t __initdata opt_rsb_hvm = 1;
+
+/* Cmdline controls for Xen's speculative settings. */
static enum ind_thunk {
THUNK_DEFAULT, /* Decide which thunk to use at boot time. */
THUNK_NONE, /* Missing compiler support for thunks. */
@@ -35,8 +42,6 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool_t __initdata opt_rsb_pv = 1;
-static bool_t __initdata opt_rsb_hvm = 1;
bool_t __read_mostly opt_ibpb = 1;
bool_t __initdata bsp_delay_spec_ctrl;
@@ -84,8 +89,95 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
+static int __init parse_spec_ctrl(const char *s)
+{
+ const char *ss;
+ int val, rc = 0;
+
+ do {
+ ss = strchr(s, ',');
+ if ( !ss )
+ ss = strchr(s, '\0');
+
+ /* Global and Xen-wide disable. */
+ val = parse_bool(s);
+ if ( !val )
+ {
+ opt_msr_sc_pv = 0;
+ opt_msr_sc_hvm = 0;
+
+ disable_common:
+ opt_rsb_pv = 0;
+ opt_rsb_hvm = 0;
+
+ opt_thunk = THUNK_JMP;
+ opt_ibrs = 0;
+ opt_ibpb = 0;
+ }
+ else if ( val > 0 )
+ rc = -EINVAL;
+ else if ( (val = parse_boolean("xen", s, ss)) >= 0 )
+ {
+ if ( !val )
+ goto disable_common;
+
+ rc = -EINVAL;
+ }
+
+ /* Xen's alternative blocks. */
+ else if ( (val = parse_boolean("pv", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_rsb_pv = val;
+ }
+ else if ( (val = parse_boolean("hvm", s, ss)) >= 0 )
+ {
+ opt_msr_sc_hvm = val;
+ opt_rsb_hvm = val;
+ }
+ else if ( (val = parse_boolean("msr-sc", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_msr_sc_hvm = val;
+ }
+ else if ( (val = parse_boolean("rsb", s, ss)) >= 0 )
+ {
+ opt_rsb_pv = val;
+ opt_rsb_hvm = val;
+ }
+
+ /* Xen's speculative sidechannel mitigation settings. */
+ else if ( !strncmp(s, "bti-thunk=", 10) )
+ {
+ s += 10;
+
+ if ( !strncmp(s, "retpoline", ss - s) )
+ opt_thunk = THUNK_RETPOLINE;
+ else if ( !strncmp(s, "lfence", ss - s) )
+ opt_thunk = THUNK_LFENCE;
+ else if ( !strncmp(s, "jmp", ss - s) )
+ opt_thunk = THUNK_JMP;
+ else
+ rc = -EINVAL;
+ }
+ else if ( (val = parse_boolean("ibrs", s, ss)) >= 0 )
+ opt_ibrs = val;
+ else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
+ opt_ibpb = val;
+ else
+ rc = -EINVAL;
+
+ s = ss + 1;
+ } while ( *ss );
+
+ return rc;
+}
+custom_param("spec-ctrl", parse_spec_ctrl);
+
static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
+ bool_t use_spec_ctrl = (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM));
unsigned int _7d0 = 0, e8b = 0, tmp;
/* Collect diagnostics about available mitigations. */
@@ -94,10 +186,10 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
+ printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(XENLOG_DEBUG " Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
@@ -107,20 +199,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
- printk(XENLOG_DEBUG " Compiled-in support: INDIRECT_THUNK\n");
+ printk(" Compiled-in support: INDIRECT_THUNK\n");
- printk("BTI mitigations: Thunk %s, Others:%s%s%s%s\n",
+ /* Settings for Xen's protection, irrespective of guests. */
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
+ !use_spec_ctrl ? "No" :
+ (default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ opt_ibpb ? " IBPB" : "");
+
+ /*
+ * Alternatives blocks for protecting against and/or virtualising
+ * mitigation support for guests.
+ */
+ printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n",
(boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
- boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
- default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
- " IBRS-" : "",
- opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "",
+ (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ||
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -212,7 +315,7 @@ static bool_t __init retpoline_safe(uint64_t caps)
void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
- bool_t ibrs = 0;
+ bool_t use_spec_ctrl = 0, ibrs = 0;
uint64_t caps = 0;
if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
@@ -282,20 +385,31 @@ void __init init_speculation_mitigations(void)
else if ( thunk == THUNK_JMP )
__set_bit(X86_FEATURE_IND_THUNK_JMP, boot_cpu_data.x86_capability);
+ /*
+ * If we are on hardware supporting MSR_SPEC_CTRL, see about setting up
+ * the alternatives blocks so we can virtualise support for guests.
+ */
if ( boot_cpu_has(X86_FEATURE_IBRSB) )
{
- /*
- * Even if we've chosen to not have IBRS set in Xen context, we still
- * need the IBRS entry/exit logic to virtualise IBRS support for
- * guests.
- */
- __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
- __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
+ if ( opt_msr_sc_pv )
+ {
+ use_spec_ctrl = 1;
+ __set_bit(X86_FEATURE_SC_MSR_PV, boot_cpu_data.x86_capability);
+ }
- if ( ibrs )
- default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
+ if ( opt_msr_sc_hvm )
+ {
+ use_spec_ctrl = 1;
+ __set_bit(X86_FEATURE_SC_MSR_HVM, boot_cpu_data.x86_capability);
+ }
+
+ if ( use_spec_ctrl )
+ {
+ if ( ibrs )
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ }
}
/*
--
2.1.4
[-- Attachment #30: xsa263-4.7/0012-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4315 bytes --]
From 60ac0e418713a10a87a4c56853c34d4fc4c96b3a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 456979d..0cc2f86 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1448,7 +1448,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1492,6 +1492,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 4a1310d..5fc2b1d 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -10,6 +10,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -591,6 +592,25 @@ static void init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
switch(c->x86)
{
case 0xf ... 0x17:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 347cd14..8480c39 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool_t __read_mostly opt_ibpb = 1;
+bool_t __read_mostly opt_ssbd;
bool_t __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -164,6 +165,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index d36f0e9..dd084d2 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool_t opt_ibpb;
+extern bool_t opt_ssbd;
extern bool_t bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #31: xsa263-4.7/0013-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 10292 bytes --]
From 4af65cba99372d4636a064ded4836e025c960525 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
tools/misc/xen-cpuid.c | 3 +--
xen/arch/x86/cpuid.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/msr-index.h | 2 ++
xen/include/public/arch-x86/cpufeatureset.h | 1 +
xen/tools/gen-cpuid.py | 17 +++++++++++++----
8 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 0cc2f86..b4e009c 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -442,9 +442,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1473,7 +1474,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1495,7 +1496,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 041b64a..5194288 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -161,6 +161,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
{"arch-caps", 0x00000007, 0, CPUID_REG_EDX, 29, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"topoext", 0x80000001, NA, CPUID_REG_ECX, 22, 1},
{"tbm", 0x80000001, NA, CPUID_REG_ECX, 21, 1},
{"nodeid", 0x80000001, NA, CPUID_REG_ECX, 19, 1},
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 06a9c7f..faac91d 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -153,8 +153,7 @@ static const char *str_7d0[32] =
[26] = "ibrsb", [27] = "stibp",
[28] = "REZ", [29] = "arch_caps",
-
- [30 ... 31] = "REZ",
+ [30] = "REZ", [31] = "ssbd",
};
static struct {
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index bade364..35b7746 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -42,6 +42,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 8480c39..b017486 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -192,26 +192,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
printk(" Compiled-in support: INDIRECT_THUNK\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -415,6 +420,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index b2c6ae8..c79ce7e 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -35,6 +35,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -43,6 +44,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 7714108..f711658 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -230,6 +230,7 @@ XEN_CPUFEATURE(IBPB, 8*32+12) /*A IBPB support only (no IBRS, used by
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index fde39db..bf9113f 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -245,10 +245,19 @@ def crunch_numbers(state):
# standard 3DNow in the earlier K6 processors.
_3DNOW: [_3DNOWEXT],
- # Single Thread Indirect Branch Predictors enumerates a new bit in the
- # MSR enumerated by Indirect Branch Restricted Speculation/Indirect
- # Branch Prediction Barrier enumeration.
- IBRSB: [STIBP],
+ # The features:
+ # * Single Thread Indirect Branch Predictors
+ # * Speculative Store Bypass Disable
+ #
+ # enumerate new bits in MSR_SPEC_CTRL, which is enumerated by Indirect
+ # Branch Restricted Speculation/Indirect Branch Prediction Barrier.
+ #
+ # In practice, these features also enumerate the presense of
+ # MSR_SPEC_CTRL. However, no real hardware will exist with SSBD but
+ # not IBRSB, and we pass this MSR directly to guests. Treating them
+ # as dependent features simplifies Xen's logic, and prevents the guest
+ # from seeing implausible configurations.
+ IBRSB: [STIBP, SSBD],
}
deep_features = tuple(sorted(deps.keys()))
--
2.1.4
[-- Attachment #32: xsa263-4.7/0014-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 3390 bytes --]
From 0e4ba02bd12d5c201b82fb3165876de5ad2daf01 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr(), and offer SSBD to guests by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/domctl.c | 3 ++-
xen/arch/x86/hvm/hvm.c | 3 ++-
xen/arch/x86/traps.c | 3 ++-
xen/include/public/arch-x86/cpufeatureset.h | 2 +-
4 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 6893387..49ca8dd 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1365,7 +1365,8 @@ long arch_do_domctl(
* ignored) when STIBP isn't enumerated in hardware.
*/
- if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (boot_cpu_has(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0)) )
break;
v->arch.spec_ctrl = msr.value;
continue;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 0a1d4a9..0981181 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3996,7 +3996,8 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
goto gp_fault; /* Rsvd bit set? */
v->arch.spec_ctrl = msr_content;
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index da26749..b236c97 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2908,7 +2908,8 @@ static int emulate_privileged_op(struct cpu_user_regs *regs)
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
goto fail; /* Rsvd bit set? */
v->arch.spec_ctrl = eax;
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index f711658..3d57339 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -230,7 +230,7 @@ XEN_CPUFEATURE(IBPB, 8*32+12) /*A IBPB support only (no IBRS, used by
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
-XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
+XEN_CPUFEATURE(SSBD, 9*32+31) /*A MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
--
2.1.4
[-- Attachment #33: xsa263-4.8/0001-x86-Fix-x86-further-CPUID-handling-adjustments.patch --]
[-- Type: application/octet-stream, Size: 3405 bytes --]
From ff4a2a18be8965b2182c6e58e09cd6fa143ca47f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 15 May 2018 16:37:59 +0100
Subject: [PATCH] x86: Fix "x86: further CPUID handling adjustments"
c/s f9616884e (a backport of c/s 0d703a701 "x86/feature: Definitions for
Indirect Branch Controls") missed a CPUID adjustment when calculating the raw
featureset. This impacts host administrator diagnostics.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
c/s 62b187969 "x86: further CPUID handling adjustments" make some adjustments.
However, it breaks levelling of guests, making it impossible for the toolstack
to hide STIBP or IBPB from guests on hardware with up-to-date microcode.
The dom0 issue referenced in the commit message was fixed by the hunk
adjusting the zeroing alone. STIBP and IBPB don't need (and indeed, must not
be for levelling purposes) OR'd into the leaf.
One final item which was missed in backport was the need to ignore the
toolstack choice of STIBP, and set it equal to IBRSB. This needs doing after
the mask has been applied.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/hvm/hvm.c | 8 +++++---
xen/arch/x86/traps.c | 8 +++++---
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4ffa30c..7c88023 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3586,10 +3586,13 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
special_features[FEATURESET_7b0]);
*ecx &= hvm_featureset[FEATURESET_7c0];
-
- *edx |= cpufeat_mask(X86_FEATURE_STIBP);
*edx &= hvm_featureset[FEATURESET_7d0];
+ /* Force STIBP equal to IBRSB */
+ *edx &= ~cpufeat_mask(X86_FEATURE_STIBP);
+ if ( *edx & cpufeat_mask(X86_FEATURE_IBRSB) )
+ *edx |= cpufeat_mask(X86_FEATURE_STIBP);
+
/* Don't expose HAP-only features to non-hap guests. */
if ( !hap_enabled(d) )
{
@@ -3761,7 +3764,6 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
hvm_cpuid(0x80000001, NULL, NULL, NULL, &_edx);
*eax |= (_edx & cpufeat_mask(X86_FEATURE_LM) ? vaddr_bits : 32) << 8;
- *ebx |= cpufeat_mask(X86_FEATURE_IBPB);
*ebx &= hvm_featureset[FEATURESET_e8b];
break;
}
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 508c18e..4a0ad5d 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1155,10 +1155,13 @@ void pv_cpuid(struct cpu_user_regs *regs)
special_features[FEATURESET_7b0]);
c &= pv_featureset[FEATURESET_7c0];
-
- d |= cpufeat_mask(X86_FEATURE_STIBP);
d &= pv_featureset[FEATURESET_7d0];
+ /* Force STIBP equal to IBRSB */
+ d &= ~cpufeat_mask(X86_FEATURE_STIBP);
+ if ( d & cpufeat_mask(X86_FEATURE_IBRSB) )
+ d |= cpufeat_mask(X86_FEATURE_STIBP);
+
if ( !is_pvh_domain(currd) )
{
/*
@@ -1271,7 +1274,6 @@ void pv_cpuid(struct cpu_user_regs *regs)
case 0x80000008:
a = paddr_bits | (vaddr_bits << 8);
- b |= cpufeat_mask(X86_FEATURE_IBPB);
b &= pv_featureset[FEATURESET_e8b];
break;
--
2.1.4
[-- Attachment #34: xsa263-4.8/0002-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch --]
[-- Type: application/octet-stream, Size: 3871 bytes --]
From 9b6481b420c0c56e08614f3975348e444570ed5c Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 12:21:00 +0100
Subject: [PATCH] x86/spec_ctrl: Read MSR_ARCH_CAPABILITIES only once
Make it available from the beginning of init_speculation_mitigations(), and
pass it into appropriate functions. Fix an RSBA typo while moving the
affected comment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d6c65187252a6c1810fd24c4d46f812840de8d3c)
---
xen/arch/x86/spec_ctrl.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index fa67a0f..dc90743 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -81,18 +81,15 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
-static void __init print_details(enum ind_thunk thunk)
+static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
unsigned int _7d0 = 0, e8b = 0, tmp;
- uint64_t caps = 0;
/* Collect diagnostics about available mitigations. */
if ( boot_cpu_data.cpuid_level >= 7 )
cpuid_count(7, 0, &tmp, &tmp, &tmp, &_7d0);
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- if ( _7d0 & cpufeat_mask(X86_FEATURE_ARCH_CAPS) )
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
@@ -125,7 +122,7 @@ static void __init print_details(enum ind_thunk thunk)
}
/* Calculate whether Retpoline is known-safe on this CPU. */
-static bool __init retpoline_safe(void)
+static bool __init retpoline_safe(uint64_t caps)
{
unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
@@ -136,19 +133,12 @@ static bool __init retpoline_safe(void)
boot_cpu_data.x86 != 6 )
return false;
- if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
- {
- uint64_t caps;
-
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
-
- /*
- * RBSA may be set by a hypervisor to indicate that we may move to a
- * processor which isn't retpoline-safe.
- */
- if ( caps & ARCH_CAPS_RSBA )
- return false;
- }
+ /*
+ * RSBA may be set by a hypervisor to indicate that we may move to a
+ * processor which isn't retpoline-safe.
+ */
+ if ( caps & ARCH_CAPS_RSBA )
+ return false;
switch ( boot_cpu_data.x86_model )
{
@@ -218,6 +208,10 @@ void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
bool ibrs = false;
+ uint64_t caps = 0;
+
+ if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
+ rdmsrl(MSR_ARCH_CAPABILITIES, caps);
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -246,7 +240,7 @@ void __init init_speculation_mitigations(void)
* On Intel hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe() )
+ else if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = true;
@@ -331,7 +325,7 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_bti_ist_info has been calculated. */
init_shadow_spec_ctrl_state();
- print_details(thunk);
+ print_details(thunk, caps);
}
static void __init __maybe_unused build_assertions(void)
--
2.1.4
[-- Attachment #35: xsa263-4.8/0003-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch --]
[-- Type: application/octet-stream, Size: 5070 bytes --]
From bd0af2ab4bf7648e7c84c83a7a0af76e21ea81b4 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Express Xen's choice of MSR_SPEC_CTRL value as
a variable
At the moment, we have two different encodings of Xen's MSR_SPEC_CTRL value,
which is a side effect of how the Spectre series developed. One encoding is
via an alias with the bottom bit of bti_ist_info, and can encode IBRS or not,
but not other configurations such as STIBP.
Break Xen's value out into a separate variable (in the top of stack block for
XPTI reasons) and use this instead of bti_ist_info in the IST path.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 66dfae0f32bfbc899c2f3446d5ee57068cb7f957)
---
xen/arch/x86/spec_ctrl.c | 8 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/include/asm-x86/current.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 2 ++
xen/include/asm-x86/spec_ctrl_asm.h | 8 ++------
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index dc90743..1143521 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,7 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
+uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_bti_ist_info;
static int __init parse_bti(const char *s)
@@ -285,11 +286,14 @@ void __init init_speculation_mitigations(void)
* guests.
*/
if ( ibrs )
+ {
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
+ }
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR | ibrs;
+ default_bti_ist_info |= BTI_IST_WRMSR;
}
/*
@@ -330,8 +334,6 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
- /* The optimised assembly relies on this alias. */
- BUILD_BUG_ON(BTI_IST_IBRS != SPEC_CTRL_IBRS);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 6d7fad8..e0aff2c 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -142,6 +142,7 @@ void __dummy__(void)
OFFSET(CPUINFO_xen_cr3, struct cpu_info, xen_cr3);
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
+ OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index 5f8f687..fc0a2fb 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -57,6 +57,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
+ uint8_t xen_spec_ctrl;
bool use_shadow_spec_ctrl;
uint8_t bti_ist_info;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5ab4ff3..5e4fc84 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_bti_ist_info;
static inline void init_shadow_spec_ctrl_state(void)
@@ -34,6 +35,7 @@ static inline void init_shadow_spec_ctrl_state(void)
struct cpu_info *info = get_cpu_info();
info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->xen_spec_ctrl = default_xen_spec_ctrl;
info->bti_ist_info = default_bti_ist_info;
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 69cf3cc..9c16945 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -21,7 +21,6 @@
#define __X86_SPEC_CTRL_ASM_H__
/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_IBRS (1 << 0)
#define BTI_IST_WRMSR (1 << 1)
#define BTI_IST_RSB (1 << 2)
@@ -285,12 +284,9 @@
setz %dl
and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
- /*
- * Load Xen's intended value. SPEC_CTRL_IBRS vs 0 is encoded in the
- * bottom bit of bti_ist_info, via a deliberate alias with BTI_IST_IBRS.
- */
+ /* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
- and $BTI_IST_IBRS, %eax
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
xor %edx, %edx
wrmsr
--
2.1.4
[-- Attachment #36: xsa263-4.8/0004-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch --]
[-- Type: application/octet-stream, Size: 13293 bytes --]
From ca6b6c4327e1885db8422927c47f6733abfac19f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Merge bti_ist_info and use_shadow_spec_ctrl
into spec_ctrl_flags
All 3 bits of information here are control flags for the entry/exit code
behaviour. Treat them as such, rather than having two different variables.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 5262ba2e7799001402dfe139ff944e035dfff928)
---
xen/arch/x86/acpi/power.c | 4 +--
xen/arch/x86/spec_ctrl.c | 10 ++++---
xen/arch/x86/x86_64/asm-offsets.c | 3 +--
xen/include/asm-x86/current.h | 3 +--
xen/include/asm-x86/nops.h | 5 ++--
xen/include/asm-x86/spec_ctrl.h | 10 +++----
xen/include/asm-x86/spec_ctrl_asm.h | 52 ++++++++++++++++++++-----------------
7 files changed, 45 insertions(+), 42 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 6fc32e8..733379e 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -216,7 +216,7 @@ static int enter_state(u32 state)
ci = get_cpu_info();
spec_ctrl_enter_idle(ci);
/* Avoid NMI/#MC using MSR_SPEC_CTRL until we've reloaded microcode. */
- ci->bti_ist_info = 0;
+ ci->spec_ctrl_flags &= ~SCF_ist_wrmsr;
ACPI_FLUSH_CPU_CACHE();
@@ -257,7 +257,7 @@ static int enter_state(u32 state)
microcode_resume_cpu(0);
/* Re-enabled default NMI/#MC use of MSR_SPEC_CTRL. */
- ci->bti_ist_info = default_bti_ist_info;
+ ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
spec_ctrl_exit_idle(ci);
done:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1143521..2d69910 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -39,7 +39,7 @@ static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
-uint8_t __read_mostly default_bti_ist_info;
+uint8_t __read_mostly default_spec_ctrl_flags;
static int __init parse_bti(const char *s)
{
@@ -293,7 +293,7 @@ void __init init_speculation_mitigations(void)
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
/*
@@ -312,7 +312,7 @@ void __init init_speculation_mitigations(void)
if ( opt_rsb_native )
{
setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
- default_bti_ist_info |= BTI_IST_RSB;
+ default_spec_ctrl_flags |= SCF_ist_rsb;
}
/*
@@ -326,7 +326,7 @@ void __init init_speculation_mitigations(void)
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
opt_ibpb = false;
- /* (Re)init BSP state now that default_bti_ist_info has been calculated. */
+ /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
print_details(thunk, caps);
@@ -334,6 +334,8 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
+ /* The optimised assembly relies on this alias. */
+ BUILD_BUG_ON(SCF_use_shadow != 1);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index e0aff2c..d939a13 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -143,8 +143,7 @@ void __dummy__(void)
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
- OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
- OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
+ OFFSET(CPUINFO_spec_ctrl_flags, struct cpu_info, spec_ctrl_flags);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
BLANK();
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index fc0a2fb..43aac0b 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -58,8 +58,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
uint8_t xen_spec_ctrl;
- bool use_shadow_spec_ctrl;
- uint8_t bti_ist_info;
+ uint8_t spec_ctrl_flags;
unsigned long __pad;
/* get_stack_bottom() must be 16-byte aligned */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index f00bd16..cab2bad 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -64,10 +64,9 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP21 ASM_NOP8; ASM_NOP8; ASM_NOP5
+#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP29 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP5
-#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5e4fc84..059e291 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -28,15 +28,15 @@ void init_speculation_mitigations(void);
extern bool opt_ibpb;
extern uint8_t default_xen_spec_ctrl;
-extern uint8_t default_bti_ist_info;
+extern uint8_t default_spec_ctrl_flags;
static inline void init_shadow_spec_ctrl_state(void)
{
struct cpu_info *info = get_cpu_info();
- info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->shadow_spec_ctrl = 0;
info->xen_spec_ctrl = default_xen_spec_ctrl;
- info->bti_ist_info = default_bti_ist_info;
+ info->spec_ctrl_flags = default_spec_ctrl_flags;
}
/* WARNING! `ret`, `call *`, `jmp *` not safe after this call. */
@@ -50,7 +50,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
*/
info->shadow_spec_ctrl = val;
barrier();
- info->use_shadow_spec_ctrl = true;
+ info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
@@ -65,7 +65,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
* Disable shadowing before updating the MSR. There are no SMP issues
* here; only local processor ordering concerns.
*/
- info->use_shadow_spec_ctrl = false;
+ info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 9c16945..582403a 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -20,9 +20,10 @@
#ifndef __X86_SPEC_CTRL_ASM_H__
#define __X86_SPEC_CTRL_ASM_H__
-/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_WRMSR (1 << 1)
-#define BTI_IST_RSB (1 << 2)
+/* Encoding of cpuinfo.spec_ctrl_flags */
+#define SCF_use_shadow (1 << 0)
+#define SCF_ist_wrmsr (1 << 1)
+#define SCF_ist_rsb (1 << 2)
#ifdef __ASSEMBLY__
#include <asm/msr-index.h>
@@ -49,20 +50,20 @@
* after VMEXIT. The VMEXIT-specific code reads MSR_SPEC_CTRL and updates
* current before loading Xen's MSR_SPEC_CTRL setting.
*
- * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and
- * use_shadow_spec_ctrl boolean per cpu. The synchronous use is:
+ * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and a use_shadow
+ * boolean in the per cpu spec_ctrl_flags. The synchronous use is:
*
* 1) Store guest value in shadow_spec_ctrl
- * 2) Set use_shadow_spec_ctrl boolean
+ * 2) Set the use_shadow boolean
* 3) Load guest value into MSR_SPEC_CTRL
* 4) Exit to guest
* 5) Entry from guest
- * 6) Clear use_shadow_spec_ctrl boolean
+ * 6) Clear the use_shadow boolean
* 7) Load Xen's value into MSR_SPEC_CTRL
*
* The asynchronous use for interrupts/exceptions is:
* - Set/clear IBRS on entry to Xen
- * - On exit to Xen, check use_shadow_spec_ctrl
+ * - On exit to Xen, check use_shadow
* - If set, load shadow_spec_ctrl
*
* Therefore, an interrupt/exception which hits the synchronous path between
@@ -133,7 +134,7 @@
xor %edx, %edx
/* Clear SPEC_CTRL shadowing *before* loading Xen's value. */
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
mov $\ibrs_val, %eax
@@ -159,12 +160,14 @@
* block so calculate the position directly.
*/
.if \maybexen
+ xor %eax, %eax
/* Branchless `if ( !xen ) clear_shadowing` */
testb $3, UREGS_cs(%rsp)
- setz %al
- and %al, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %al
+ not %eax
+ and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
.else
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
.endif
/* Load Xen's intended value. */
@@ -183,8 +186,8 @@
*/
xor %edx, %edx
- cmpb %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%rbx)
- je .L\@_skip
+ testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
+ jz .L\@_skip
mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax
mov $MSR_SPEC_CTRL, %ecx
@@ -205,7 +208,7 @@
mov %eax, CPUINFO_shadow_spec_ctrl(%rsp)
/* Set SPEC_CTRL shadowing *before* loading the guest value. */
- movb $1, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ orb $SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
mov $MSR_SPEC_CTRL, %ecx
xor %edx, %edx
@@ -216,7 +219,7 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -228,7 +231,7 @@
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP21), \
+ ALTERNATIVE_2 __stringify(ASM_NOP22), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -239,7 +242,7 @@
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP29), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -267,22 +270,23 @@
* This is logical merge of DO_OVERWRITE_RSB and DO_SPEC_CTRL_ENTRY
* maybexen=1, but with conditionals rather than alternatives.
*/
- movzbl STACK_CPUINFO_FIELD(bti_ist_info)(%r14), %eax
+ movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %eax
- testb $BTI_IST_RSB, %al
+ test $SCF_ist_rsb, %al
jz .L\@_skip_rsb
DO_OVERWRITE_RSB tmp=rdx /* Clobbers %rcx/%rdx */
.L\@_skip_rsb:
- testb $BTI_IST_WRMSR, %al
+ test $SCF_ist_wrmsr, %al
jz .L\@_skip_wrmsr
xor %edx, %edx
testb $3, UREGS_cs(%rsp)
- setz %dl
- and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %dl
+ not %edx
+ and %dl, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
/* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
@@ -309,7 +313,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
* Requires %rbx=stack_end
* Clobbers %rax, %rcx, %rdx
*/
- testb $BTI_IST_WRMSR, STACK_CPUINFO_FIELD(bti_ist_info)(%rbx)
+ testb $SCF_ist_wrmsr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
jz .L\@_skip
DO_SPEC_CTRL_EXIT_TO_XEN
--
2.1.4
[-- Attachment #37: xsa263-4.8/0005-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch --]
[-- Type: application/octet-stream, Size: 10501 bytes --]
From 5c56dda28af710abec8bb9663897075f17d5dab9 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Fold the XEN_IBRS_{SET,CLEAR} ALTERNATIVES
together
Currently, the SPEC_CTRL_{ENTRY,EXIT}_* macros encode Xen's choice of
MSR_SPEC_CTRL as an immediate constant, and chooses between IBRS or not by
doubling up the entire alternative block.
There is now a variable holding Xen's choice of value, so use that and
simplify the alternatives.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit af949407eaba7af71067f23d5866cd0bf1f1144d)
---
xen/arch/x86/spec_ctrl.c | 12 +++++-----
xen/include/asm-x86/cpufeature.h | 3 +--
xen/include/asm-x86/nops.h | 3 ++-
xen/include/asm-x86/spec_ctrl.h | 6 ++---
xen/include/asm-x86/spec_ctrl_asm.h | 45 +++++++++++++------------------------
5 files changed, 26 insertions(+), 43 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 2d69910..b62cfcc 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,8 +112,9 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_XEN_IBRS_SET) ? " IBRS+" :
- boot_cpu_has(X86_FEATURE_XEN_IBRS_CLEAR) ? " IBRS-" : "",
+ boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
+ " IBRS-" : "",
opt_ibpb ? " IBPB" : "",
boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
@@ -285,13 +286,10 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+
if ( ibrs )
- {
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
- }
- else
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 16cc730..2d06625 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -25,8 +25,7 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+14) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
-XEN_CPUFEATURE(XEN_IBRS_SET, (FSCAPINTS+0)*32+18) /* IBRSB && IRBS set in Xen */
-XEN_CPUFEATURE(XEN_IBRS_CLEAR, (FSCAPINTS+0)*32+19) /* IBRSB && IBRS clear in Xen */
+XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for native */
XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for vmexit */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index cab2bad..ad32c2e 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -64,9 +64,10 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP25 ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 059e291..7d7c42e 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,14 +52,14 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
/* WARNING! `ret`, `call *`, `jmp *` not safe before this call. */
static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
{
- uint32_t val = SPEC_CTRL_IBRS;
+ uint32_t val = info->xen_spec_ctrl;
/*
* Disable shadowing before updating the MSR. There are no SMP issues
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 582403a..941aeb7 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -117,7 +117,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -137,11 +137,11 @@
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
- mov $\ibrs_val, %eax
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
wrmsr
.endm
-.macro DO_SPEC_CTRL_ENTRY maybexen:req ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY maybexen:req
/*
* Requires %rsp=regs (also cpuinfo if !maybexen)
* Requires %r14=stack_end (if maybexen)
@@ -166,12 +166,12 @@
setnz %al
not %eax
and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
.else
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
.endif
- /* Load Xen's intended value. */
- mov $\ibrs_val, %eax
wrmsr
.endm
@@ -219,47 +219,32 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP36), \
+ DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP22), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP25), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP33), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
- ALTERNATIVE_2 __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP17), \
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
/* Use when exiting to guest context. */
#define SPEC_CTRL_EXIT_TO_GUEST \
- ALTERNATIVE_2 __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
--
2.1.4
[-- Attachment #38: xsa263-4.8/0006-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch --]
[-- Type: application/octet-stream, Size: 12340 bytes --]
From e2ee82715193869fd19b73eedea7440d01401e7b Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 30 Apr 2018 14:20:23 +0100
Subject: [PATCH] x86/spec_ctrl: Rename bits of infrastructure to avoid NATIVE
and VMEXIT
In hindsight, using NATIVE and VMEXIT as naming terminology was not clever.
A future change wants to split SPEC_CTRL_EXIT_TO_GUEST into PV and HVM
specific implementations, and using VMEXIT as a term is completely wrong.
Take the opportunity to fix some stale documentation in spec_ctrl_asm.h. The
IST helpers were missing from the large comment block, and since
SPEC_CTRL_ENTRY_FROM_INTR_IST was introduced, we've gained a new piece of
functionality which currently depends on the fine grain control, which exists
in lieu of livepatching. Note this in the comment.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d9822b8a38114e96e4516dc998f4055249364d5d)
---
xen/arch/x86/hvm/svm/entry.S | 4 ++--
xen/arch/x86/hvm/vmx/entry.S | 4 ++--
xen/arch/x86/spec_ctrl.c | 20 ++++++++++----------
xen/arch/x86/x86_64/compat/entry.S | 2 +-
xen/arch/x86/x86_64/entry.S | 2 +-
xen/include/asm-x86/cpufeature.h | 4 ++--
xen/include/asm-x86/spec_ctrl_asm.h | 36 +++++++++++++++++++++++++-----------
7 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S
index 289e946..d824bcd 100644
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -81,7 +81,7 @@ UNLIKELY_END(svm_trace)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
pop %r15
pop %r14
@@ -106,7 +106,7 @@ UNLIKELY_END(svm_trace)
GET_CURRENT(bx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov VCPU_svm_vmcb(%rbx),%rcx
diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S
index 7aa0e85..f1528e8 100644
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -37,7 +37,7 @@ ENTRY(vmx_asm_vmexit_handler)
movb $1,VCPU_vmx_launched(%rbx)
mov %rax,VCPU_hvm_guest_cr2(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov %rsp,%rdi
@@ -72,7 +72,7 @@ UNLIKELY_END(realmode)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
mov VCPU_hvm_guest_cr2(%rbx),%rax
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index b62cfcc..015a9e2 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -35,8 +35,8 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_native = true;
-static bool __initdata opt_rsb_vmexit = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -69,9 +69,9 @@ static int __init parse_bti(const char *s)
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
else if ( (val = parse_boolean("rsb_native", s, ss)) >= 0 )
- opt_rsb_native = val;
+ opt_rsb_pv = val;
else if ( (val = parse_boolean("rsb_vmexit", s, ss)) >= 0 )
- opt_rsb_vmexit = val;
+ opt_rsb_hvm = val;
else
rc = -EINVAL;
@@ -116,8 +116,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -307,9 +307,9 @@ void __init init_speculation_mitigations(void)
* If a processors speculates to 32bit PV guest kernel mappings, it is
* speculating in 64bit supervisor mode, and can leak data.
*/
- if ( opt_rsb_native )
+ if ( opt_rsb_pv )
{
- setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_PV);
default_spec_ctrl_flags |= SCF_ist_rsb;
}
@@ -317,8 +317,8 @@ void __init init_speculation_mitigations(void)
* HVM guests can always poison the RSB to point at Xen supervisor
* mappings.
*/
- if ( opt_rsb_vmexit )
- setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
+ if ( opt_rsb_hvm )
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_HVM);
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index b3ba857..6299547 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -171,7 +171,7 @@ ENTRY(compat_restore_all_guest)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL adj=8 compat=1
.Lft0: iretq
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index f61dd25..797f1b2 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -194,7 +194,7 @@ restore_all_guest:
mov %r15d, %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL
testw $TRAP_syscall,4(%rsp)
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 2d06625..846b5c3 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -26,8 +26,8 @@ XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
-XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for native */
-XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for vmexit */
+XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
+XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
#define NCAPINTS (FSCAPINTS + 1) /* N 32-bit words worth of info */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 941aeb7..b330e20 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -72,11 +72,14 @@
*
* The following ASM fragments implement this algorithm. See their local
* comments for further details.
- * - SPEC_CTRL_ENTRY_FROM_VMEXIT
+ * - SPEC_CTRL_ENTRY_FROM_HVM
* - SPEC_CTRL_ENTRY_FROM_PV
* - SPEC_CTRL_ENTRY_FROM_INTR
+ * - SPEC_CTRL_ENTRY_FROM_INTR_IST
+ * - SPEC_CTRL_EXIT_TO_XEN_IST
* - SPEC_CTRL_EXIT_TO_XEN
- * - SPEC_CTRL_EXIT_TO_GUEST
+ * - SPEC_CTRL_EXIT_TO_PV
+ * - SPEC_CTRL_EXIT_TO_HVM
*/
.macro DO_OVERWRITE_RSB tmp=rax
@@ -117,7 +120,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
+.macro DO_SPEC_CTRL_ENTRY_FROM_HVM
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -216,23 +219,23 @@
.endm
/* Use after a VMEXIT from an HVM guest. */
-#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
+#define SPEC_CTRL_ENTRY_FROM_HVM \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
@@ -241,12 +244,22 @@
ALTERNATIVE __stringify(ASM_NOP17), \
DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
-/* Use when exiting to guest context. */
-#define SPEC_CTRL_EXIT_TO_GUEST \
+/* Use when exiting to PV guest context. */
+#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
-/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
+/* Use when exiting to HVM guest context. */
+#define SPEC_CTRL_EXIT_TO_HVM \
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+
+/*
+ * Use in IST interrupt/exception context. May interrupt Xen or PV context.
+ * Fine grain control of SCF_ist_wrmsr is needed for safety in the S3 resume
+ * path to avoid using MSR_SPEC_CTRL before the microcode introducing it has
+ * been reloaded.
+ */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
/*
* Requires %rsp=regs, %r14=stack_end
@@ -293,6 +306,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
UNLIKELY_END(\@_serialise)
.endm
+/* Use when exiting to Xen in IST context. */
.macro SPEC_CTRL_EXIT_TO_XEN_IST
/*
* Requires %rbx=stack_end
--
2.1.4
[-- Attachment #39: xsa263-4.8/0007-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch --]
[-- Type: application/octet-stream, Size: 3149 bytes --]
From 1b5982d3b8380150870f4d4287c01a59c5a0963a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 May 2018 14:06:16 +0100
Subject: [PATCH] x86/spec_ctrl: Elide MSR_SPEC_CTRL handling in idle context
when possible
If Xen is virtualising MSR_SPEC_CTRL handling for guests, but using 0 as its
own MSR_SPEC_CTRL value, spec_ctrl_{enter,exit}_idle() need not write to the
MSR.
Requested-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 94df6e8588e35cc2028ccb3fd2921c6e6360605e)
---
xen/arch/x86/spec_ctrl.c | 4 ++++
xen/include/asm-x86/cpufeature.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 4 ++--
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 015a9e2..55ef79f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -327,6 +327,10 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
+ /* If Xen is using any MSR_SPEC_CTRL settings, adjust the idle path. */
+ if ( default_xen_spec_ctrl )
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
+
print_details(thunk, caps);
}
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 846b5c3..a043dd6 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -29,6 +29,7 @@ XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xe
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* SC_MSR && default_xen_spec_ctrl */
#define NCAPINTS (FSCAPINTS + 1) /* N 32-bit words worth of info */
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 7d7c42e..77f92ba 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,7 +52,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
--
2.1.4
[-- Attachment #40: xsa263-4.8/0008-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch --]
[-- Type: application/octet-stream, Size: 6105 bytes --]
From 5a67eb584c5bee8101fbef322f9753227a50e84b Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Split X86_FEATURE_SC_MSR into PV and HVM
variants
In order to separately control whether MSR_SPEC_CTRL is virtualised for PV and
HVM guests, split the feature used to control runtime alternatives into two.
Xen will use MSR_SPEC_CTRL itself if either of these features are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit fa9eb09d446a1279f5e861e6b84fa8675dabf148)
---
xen/arch/x86/spec_ctrl.c | 6 ++++--
xen/include/asm-x86/cpufeature.h | 5 +++--
xen/include/asm-x86/spec_ctrl_asm.h | 12 ++++++------
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 55ef79f..a940308 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,7 +112,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
@@ -286,7 +287,8 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
- setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
if ( ibrs )
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index a043dd6..b4505a4 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -25,11 +25,12 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+14) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
-XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
+XEN_CPUFEATURE(SC_MSR_PV, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen for PV */
+XEN_CPUFEATURE(SC_MSR_HVM, (FSCAPINTS+0)*32+19) /* MSR_SPEC_CTRL used by Xen for HVM */
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
-XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* SC_MSR && default_xen_spec_ctrl */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* (SC_MSR_PV || SC_MSR_HVM) && default_xen_spec_ctrl */
#define NCAPINTS (FSCAPINTS + 1) /* N 32-bit words worth of info */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index b330e20..4d864eb 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -223,36 +223,36 @@
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR_HVM
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR_PV
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR_PV
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
ALTERNATIVE __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV
/* Use when exiting to PV guest context. */
#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV
/* Use when exiting to HVM guest context. */
#define SPEC_CTRL_EXIT_TO_HVM \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_HVM
/*
* Use in IST interrupt/exception context. May interrupt Xen or PV context.
--
2.1.4
[-- Attachment #41: xsa263-4.8/0009-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch --]
[-- Type: application/octet-stream, Size: 4771 bytes --]
From b970276dca2ba3f6fc2013cd90dfd09a0cfb5692 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 9 May 2018 13:59:56 +0100
Subject: [PATCH] x86/spec_ctrl: Explicitly set Xen's default MSR_SPEC_CTRL
value
With the impending ability to disable MSR_SPEC_CTRL handling on a
per-guest-type basis, the first exit-from-guest may not have the side effect
of loading Xen's choice of value. Explicitly set Xen's default during the BSP
and AP boot paths.
For the BSP however, delay setting a non-zero MSR_SPEC_CTRL default until
after dom0 has been constructed when safe to do so. Oracle report that this
speeds up boots of some hardware by 50s.
"when safe to do so" is based on whether we are virtualised. A native boot
won't have any other code running in a position to mount an attack.
Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb8c12020307b39a89273d7699e89000451987ab)
---
xen/arch/x86/setup.c | 7 +++++++
xen/arch/x86/smpboot.c | 8 ++++++++
xen/arch/x86/spec_ctrl.c | 32 ++++++++++++++++++++++++++++++++
xen/include/asm-x86/spec_ctrl.h | 2 ++
4 files changed, 49 insertions(+)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index cbdc041..651d14e 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1676,6 +1676,13 @@ void __init noreturn __start_xen(unsigned long mbi_p)
setup_io_bitmap(dom0);
+ if ( bsp_delay_spec_ctrl )
+ {
+ get_cpu_info()->spec_ctrl_flags &= ~SCF_use_shadow;
+ barrier();
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+ }
+
/* Jump to the 1:1 virtual mappings of cpu0_stack. */
asm volatile ("mov %[stk], %%rsp; jmp %c[fn]" ::
[stk] "g" (__va(__pa(get_stack_bottom()))),
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index c9f39b1..88b0c44 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -342,6 +342,14 @@ void start_secondary(void *unused)
else
microcode_resume_cpu(cpu);
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. Note: MSR_SPEC_CTRL may only become available
+ * after loading microcode.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+
smp_callin();
init_percpu_time();
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index a940308..3adec1a 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,8 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_pv = true;
static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
+
+bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -334,6 +336,36 @@ void __init init_speculation_mitigations(void)
setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
print_details(thunk, caps);
+
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. For performance reasons, when safe to do so, we
+ * delay applying non-zero settings until after dom0 has been constructed.
+ *
+ * "when safe to do so" is based on whether we are virtualised. A native
+ * boot won't have any other code running in a position to mount an
+ * attack.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ {
+ bsp_delay_spec_ctrl = !cpu_has_hypervisor && default_xen_spec_ctrl;
+
+ /*
+ * If delaying MSR_SPEC_CTRL setup, use the same mechanism as
+ * spec_ctrl_enter_idle(), by using a shadow value of zero.
+ */
+ if ( bsp_delay_spec_ctrl )
+ {
+ struct cpu_info *info = get_cpu_info();
+
+ info->shadow_spec_ctrl = 0;
+ barrier();
+ info->spec_ctrl_flags |= SCF_use_shadow;
+ barrier();
+ }
+
+ wrmsrl(MSR_SPEC_CTRL, bsp_delay_spec_ctrl ? 0 : default_xen_spec_ctrl);
+ }
}
static void __init __maybe_unused build_assertions(void)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 77f92ba..c6a38f4 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,8 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+
+extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_spec_ctrl_flags;
--
2.1.4
[-- Attachment #42: xsa263-4.8/0010-x86-cpuid-Improvements-to-guest-policies-for-specula.patch --]
[-- Type: application/octet-stream, Size: 5476 bytes --]
From 63288a25ad52cdb12ab5eac70fdfd30e85bd84aa Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 1 May 2018 11:59:03 +0100
Subject: [PATCH] x86/cpuid: Improvements to guest policies for speculative
sidechannel features
If Xen isn't virtualising MSR_SPEC_CTRL for guests, IBRSB shouldn't be
advertised. It is not currently possible to express this via the existing
command line options, but such an ability will be introduced.
Another useful option in some usecases is to offer IBPB without IBRS. When a
guest kernel is known to be compatible (uses retpoline and knows about the AMD
IBPB feature bit), an administrator with pre-Skylake hardware may wish to hide
IBRS. This allows the VM to have full protection, without Xen or the VM
needing to touch MSR_SPEC_CTRL, which can reduce the overhead of Spectre
mitigations.
Break the logic common to both PV and HVM CPUID calculations into a common
helper, to avoid duplication.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb06b308ec71b23f37a44f5e2351fe2cae0306e9)
---
xen/arch/x86/cpuid.c | 60 ++++++++++++++++++++++++++++++++--------------------
1 file changed, 37 insertions(+), 23 deletions(-)
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index fffcecd..bade364 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -136,6 +136,28 @@ static void __init calculate_raw_featureset(void)
&tmp, &tmp);
}
+static void __init guest_common_feature_adjustments(uint32_t *fs)
+{
+ /* Unconditionally claim to be able to set the hypervisor bit. */
+ __set_bit(X86_FEATURE_HYPERVISOR, fs);
+
+ /*
+ * If IBRS is offered to the guest, unconditionally offer STIBP. It is a
+ * nop on non-HT hardware, and has this behaviour to make heterogeneous
+ * setups easier to manage.
+ */
+ if ( test_bit(X86_FEATURE_IBRSB, fs) )
+ __set_bit(X86_FEATURE_STIBP, fs);
+
+ /*
+ * On hardware which supports IBRS/IBPB, we can offer IBPB independently
+ * of IBRS by using the AMD feature bit. An administrator may wish for
+ * performance reasons to offer IBPB without IBRS.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ __set_bit(X86_FEATURE_IBPB, fs);
+}
+
static void __init calculate_pv_featureset(void)
{
unsigned int i;
@@ -143,9 +165,6 @@ static void __init calculate_pv_featureset(void)
for ( i = 0; i < FSCAPINTS; ++i )
pv_featureset[i] = host_featureset[i] & pv_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, pv_featureset);
-
/*
* Allow the toolstack to set HTT, X2APIC and CMP_LEGACY. These bits
* affect how to interpret topology information in other cpuid leaves.
@@ -154,15 +173,14 @@ static void __init calculate_pv_featureset(void)
__set_bit(X86_FEATURE_X2APIC, pv_featureset);
__set_bit(X86_FEATURE_CMP_LEGACY, pv_featureset);
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, pv_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, pv_featureset);
+ /*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for PV guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ __clear_bit(X86_FEATURE_IBRSB, pv_featureset);
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, pv_featureset);
- }
+ guest_common_feature_adjustments(pv_featureset);
sanitise_featureset(pv_featureset);
}
@@ -181,9 +199,6 @@ static void __init calculate_hvm_featureset(void)
for ( i = 0; i < FSCAPINTS; ++i )
hvm_featureset[i] = host_featureset[i] & hvm_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, hvm_featureset);
-
/*
* Allow the toolstack to set HTT, X2APIC and CMP_LEGACY. These bits
* affect how to interpret topology information in other cpuid leaves.
@@ -208,6 +223,13 @@ static void __init calculate_hvm_featureset(void)
__set_bit(X86_FEATURE_SEP, hvm_featureset);
/*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for HVM guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_HVM) )
+ __clear_bit(X86_FEATURE_IBRSB, hvm_featureset);
+
+ /*
* With VT-x, some features are only supported by Xen if dedicated
* hardware support is also available.
*/
@@ -220,15 +242,7 @@ static void __init calculate_hvm_featureset(void)
__clear_bit(X86_FEATURE_XSAVES, hvm_featureset);
}
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, hvm_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, hvm_featureset);
-
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, hvm_featureset);
- }
+ guest_common_feature_adjustments(hvm_featureset);
sanitise_featureset(hvm_featureset);
}
--
2.1.4
[-- Attachment #43: xsa263-4.8/0011-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch --]
[-- Type: application/octet-stream, Size: 14129 bytes --]
From 4f635420eeea10a703f88b474edf11819ef28c3a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:52:55 +0100
Subject: [PATCH] x86/spec_ctrl: Introduce a new `spec-ctrl=` command line
argument to replace `bti=`
In hindsight, the options for `bti=` aren't as flexible or useful as expected
(including several options which don't appear to behave as intended).
Changing the behaviour of an existing option is problematic for compatibility,
so introduce a new `spec-ctrl=` in the hopes that we can do better.
One common way of deploying Xen is with a single PV dom0 and all domUs being
HVM domains. In such a setup, an administrator who has weighed up the risks
may wish to forgo protection against malicious PV domains, to reduce the
overall performance hit. To cater for this usecase, `spec-ctrl=no-pv` will
disable all speculative protection for PV domains, while leaving all
speculative protection for HVM domains intact.
For coding clarity as much as anything else, the suboptions are grouped by
logical area; those which affect the alternatives blocks, and those which
affect Xen's in-hypervisor settings. See the xen-command-line.markdown for
full details of the new options.
While changing the command line options, take the time to change how the data
is reported to the user. The three DEBUG printks are upgraded to unilateral,
as they are all relevant pieces of information, and the old "mitigations:"
line is split in the two logical areas described above.
Sample output from booting with `spec-ctrl=no-pv` looks like:
(XEN) Speculative mitigation facilities:
(XEN) Hardware features: IBRS/IBPB STIBP IBPB
(XEN) Compiled-in support: INDIRECT_THUNK
(XEN) Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS-, Other: IBPB
(XEN) Support for VMs: PV: None, HVM: MSR_SPEC_CTRL RSB
(XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 3352afc26c497d26ecb70527db3cb29daf7b1422)
---
docs/misc/xen-command-line.markdown | 49 +++++++++++
xen/arch/x86/spec_ctrl.c | 160 ++++++++++++++++++++++++++++++------
2 files changed, 186 insertions(+), 23 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 7ad9717..0f3edaf 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -255,6 +255,9 @@ the NMI watchdog is also enabled.
### bti (x86)
> `= List of [ thunk=retpoline|lfence|jmp, ibrs=<bool>, ibpb=<bool>, rsb_{vmexit,native}=<bool> ]`
+**WARNING: This command line option is deprecated, and superseded by
+_spec-ctrl=_ - using both options in combination is undefined.**
+
Branch Target Injection controls. By default, Xen will pick the most
appropriate BTI mitigations based on compiled in support, loaded microcode,
and hardware details.
@@ -1515,6 +1518,52 @@ enforces the maximum theoretically necessary timeout of 670ms. Any number
is being interpreted as a custom timeout in milliseconds. Zero or boolean
false disable the quirk workaround, which is also the default.
+### spec-ctrl (x86)
+> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+
+Controls for speculative execution sidechannel mitigations. By default, Xen
+will pick the most appropriate mitigations based on compiled in support,
+loaded microcode, and hardware details, and will virtualise appropriate
+mitigations for guests to use.
+
+**WARNING: Any use of this option may interfere with heuristics. Use with
+extreme care.**
+
+An overall boolean value, `spec-ctrl=no`, can be specified to turn off all
+mitigations, including pieces of infrastructure used to virtualise certain
+mitigation features for guests. Alternatively, a slightly more restricted
+`spec-ctrl=no-xen` can be used to turn off all of Xen's mitigations, while
+leaving the virtualisation support in place for guests to use. Use of a
+positive boolean value for either of these options is invalid.
+
+The booleans `pv=`, `hvm=`, `msr-sc=` and `rsb=` offer fine grained control
+over the alternative blocks used by Xen. These impact Xen's ability to
+protect itself, and Xen's ability to virtualise support for guests to use.
+
+* `pv=` and `hvm=` offer control over all suboptions for PV and HVM guests
+ respectively.
+* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
+ on entry and exit. These blocks are necessary to virtualise support for
+ guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
+ Return Address Stack on entry to Xen.
+
+If Xen was compiled with INDIRECT\_THUNK support, `bti-thunk=` can be used to
+select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
+locations. The default thunk is `retpoline` (generally preferred for Intel
+hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
+overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+
+On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
+`ibrs=` option can be used to force or prevent Xen using the feature itself.
+If Xen is not using IBRS itself, functionality is still set up so IBRS can be
+virtualised for guests.
+
+On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
+option can be used to force (the default) or prevent Xen from issuing branch
+prediction barriers on vcpu context switches.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 3adec1a..1a59b54 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -26,6 +26,13 @@
#include <asm/spec_ctrl.h>
#include <asm/spec_ctrl_asm.h>
+/* Cmdline controls for Xen's alternative blocks. */
+static bool __initdata opt_msr_sc_pv = true;
+static bool __initdata opt_msr_sc_hvm = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
+
+/* Cmdline controls for Xen's speculative settings. */
static enum ind_thunk {
THUNK_DEFAULT, /* Decide which thunk to use at boot time. */
THUNK_NONE, /* Missing compiler support for thunks. */
@@ -35,8 +42,6 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_pv = true;
-static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
bool __initdata bsp_delay_spec_ctrl;
@@ -84,8 +89,95 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
+static int __init parse_spec_ctrl(const char *s)
+{
+ const char *ss;
+ int val, rc = 0;
+
+ do {
+ ss = strchr(s, ',');
+ if ( !ss )
+ ss = strchr(s, '\0');
+
+ /* Global and Xen-wide disable. */
+ val = parse_bool(s);
+ if ( !val )
+ {
+ opt_msr_sc_pv = false;
+ opt_msr_sc_hvm = false;
+
+ disable_common:
+ opt_rsb_pv = false;
+ opt_rsb_hvm = false;
+
+ opt_thunk = THUNK_JMP;
+ opt_ibrs = 0;
+ opt_ibpb = false;
+ }
+ else if ( val > 0 )
+ rc = -EINVAL;
+ else if ( (val = parse_boolean("xen", s, ss)) >= 0 )
+ {
+ if ( !val )
+ goto disable_common;
+
+ rc = -EINVAL;
+ }
+
+ /* Xen's alternative blocks. */
+ else if ( (val = parse_boolean("pv", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_rsb_pv = val;
+ }
+ else if ( (val = parse_boolean("hvm", s, ss)) >= 0 )
+ {
+ opt_msr_sc_hvm = val;
+ opt_rsb_hvm = val;
+ }
+ else if ( (val = parse_boolean("msr-sc", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_msr_sc_hvm = val;
+ }
+ else if ( (val = parse_boolean("rsb", s, ss)) >= 0 )
+ {
+ opt_rsb_pv = val;
+ opt_rsb_hvm = val;
+ }
+
+ /* Xen's speculative sidechannel mitigation settings. */
+ else if ( !strncmp(s, "bti-thunk=", 10) )
+ {
+ s += 10;
+
+ if ( !strncmp(s, "retpoline", ss - s) )
+ opt_thunk = THUNK_RETPOLINE;
+ else if ( !strncmp(s, "lfence", ss - s) )
+ opt_thunk = THUNK_LFENCE;
+ else if ( !strncmp(s, "jmp", ss - s) )
+ opt_thunk = THUNK_JMP;
+ else
+ rc = -EINVAL;
+ }
+ else if ( (val = parse_boolean("ibrs", s, ss)) >= 0 )
+ opt_ibrs = val;
+ else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
+ opt_ibpb = val;
+ else
+ rc = -EINVAL;
+
+ s = ss + 1;
+ } while ( *ss );
+
+ return rc;
+}
+custom_param("spec-ctrl", parse_spec_ctrl);
+
static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
+ bool use_spec_ctrl = (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM));
unsigned int _7d0 = 0, e8b = 0, tmp;
/* Collect diagnostics about available mitigations. */
@@ -94,10 +186,10 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
+ printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(XENLOG_DEBUG " Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
@@ -107,20 +199,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
- printk(XENLOG_DEBUG " Compiled-in support: INDIRECT_THUNK\n");
+ printk(" Compiled-in support: INDIRECT_THUNK\n");
- printk("BTI mitigations: Thunk %s, Others:%s%s%s%s\n",
+ /* Settings for Xen's protection, irrespective of guests. */
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
+ !use_spec_ctrl ? "No" :
+ (default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ opt_ibpb ? " IBPB" : "");
+
+ /*
+ * Alternatives blocks for protecting against and/or virtualising
+ * mitigation support for guests.
+ */
+ printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n",
(boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
- boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
- default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
- " IBRS-" : "",
- opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "",
+ (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ||
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -212,7 +315,7 @@ static bool __init retpoline_safe(uint64_t caps)
void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
- bool ibrs = false;
+ bool use_spec_ctrl = false, ibrs = false;
uint64_t caps = 0;
if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
@@ -282,20 +385,31 @@ void __init init_speculation_mitigations(void)
else if ( thunk == THUNK_JMP )
setup_force_cpu_cap(X86_FEATURE_IND_THUNK_JMP);
+ /*
+ * If we are on hardware supporting MSR_SPEC_CTRL, see about setting up
+ * the alternatives blocks so we can virtualise support for guests.
+ */
if ( boot_cpu_has(X86_FEATURE_IBRSB) )
{
- /*
- * Even if we've chosen to not have IBRS set in Xen context, we still
- * need the IBRS entry/exit logic to virtualise IBRS support for
- * guests.
- */
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ if ( opt_msr_sc_pv )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ }
- if ( ibrs )
- default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
+ if ( opt_msr_sc_hvm )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ }
+
+ if ( use_spec_ctrl )
+ {
+ if ( ibrs )
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ }
}
/*
--
2.1.4
[-- Attachment #44: xsa263-4.8/0012-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4402 bytes --]
From 3e21da4dad90b64d9e0de6fd9ac6b290eb60bb2f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 0f3edaf..619db15 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1520,7 +1520,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1564,6 +1564,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 5c1bb13..b04cf6d 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -10,6 +10,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -591,6 +592,25 @@ static void init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
/* MFENCE stops RDTSC speculation */
if (!cpu_has_lfence_dispatch)
__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1a59b54..0fb628b 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool __read_mostly opt_ibpb = true;
+bool __read_mostly opt_ssbd = false;
bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -164,6 +165,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index c6a38f4..4678a40 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern bool opt_ssbd;
extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #45: xsa263-4.8/0013-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 10329 bytes --]
From cf8af740da47fd195029e525f555e0ce6f4eead4 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
tools/misc/xen-cpuid.c | 3 +--
xen/arch/x86/cpuid.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/msr-index.h | 2 ++
xen/include/public/arch-x86/cpufeatureset.h | 1 +
xen/tools/gen-cpuid.py | 17 +++++++++++++----
8 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 619db15..1eaef52 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -456,9 +456,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1545,7 +1546,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1567,7 +1568,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 3c00bb5..b426898 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -161,6 +161,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
{"arch-caps", 0x00000007, 0, CPUID_REG_EDX, 29, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"topoext", 0x80000001, NA, CPUID_REG_ECX, 22, 1},
{"tbm", 0x80000001, NA, CPUID_REG_ECX, 21, 1},
{"nodeid", 0x80000001, NA, CPUID_REG_ECX, 19, 1},
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 06a9c7f..faac91d 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -153,8 +153,7 @@ static const char *str_7d0[32] =
[26] = "ibrsb", [27] = "stibp",
[28] = "REZ", [29] = "arch_caps",
-
- [30 ... 31] = "REZ",
+ [30] = "REZ", [31] = "ssbd",
};
static struct {
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index bade364..35b7746 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -42,6 +42,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 0fb628b..18515eb 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -192,26 +192,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
printk(" Compiled-in support: INDIRECT_THUNK\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -415,6 +420,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index e06a8fa..9969504 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -35,6 +35,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -43,6 +44,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index ef358cf..93645bd 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -239,6 +239,7 @@ XEN_CPUFEATURE(IBPB, 8*32+12) /*A IBPB support only (no IBRS, used by
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 9b2cb6f..0240e55 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -255,10 +255,19 @@ def crunch_numbers(state):
AVX512F: [AVX512DQ, AVX512IFMA, AVX512PF, AVX512ER, AVX512CD,
AVX512BW, AVX512VL, AVX512VBMI],
- # Single Thread Indirect Branch Predictors enumerates a new bit in the
- # MSR enumerated by Indirect Branch Restricted Speculation/Indirect
- # Branch Prediction Barrier enumeration.
- IBRSB: [STIBP],
+ # The features:
+ # * Single Thread Indirect Branch Predictors
+ # * Speculative Store Bypass Disable
+ #
+ # enumerate new bits in MSR_SPEC_CTRL, which is enumerated by Indirect
+ # Branch Restricted Speculation/Indirect Branch Prediction Barrier.
+ #
+ # In practice, these features also enumerate the presense of
+ # MSR_SPEC_CTRL. However, no real hardware will exist with SSBD but
+ # not IBRSB, and we pass this MSR directly to guests. Treating them
+ # as dependent features simplifies Xen's logic, and prevents the guest
+ # from seeing implausible configurations.
+ IBRSB: [STIBP, SSBD],
}
deep_features = tuple(sorted(deps.keys()))
--
2.1.4
[-- Attachment #46: xsa263-4.8/0014-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 3337 bytes --]
From 88849195b6e07e2b8cc5b556f588db42b9351456 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr(), and offer SSBD to guests by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/domctl.c | 3 ++-
xen/arch/x86/hvm/hvm.c | 3 ++-
xen/arch/x86/traps.c | 3 ++-
xen/include/public/arch-x86/cpufeatureset.h | 2 +-
4 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 659dc9f..ad5d20b 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1389,7 +1389,8 @@ long arch_do_domctl(
* ignored) when STIBP isn't enumerated in hardware.
*/
- if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (boot_cpu_has(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0)) )
break;
v->arch.spec_ctrl = msr.value;
continue;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 7c88023..6a95ae9 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4147,7 +4147,8 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
goto gp_fault; /* Rsvd bit set? */
v->arch.spec_ctrl = msr_content;
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 4a0ad5d..f8c85e3 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2752,7 +2752,8 @@ static int priv_op_write_msr(unsigned int reg, uint64_t val,
* when STIBP isn't enumerated in hardware.
*/
- if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ ((edx & cpufeat_mask(X86_FEATURE_SSBD) ? SPEC_CTRL_SSBD : 0))) )
break; /* Rsvd bit set? */
curr->arch.spec_ctrl = val;
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 93645bd..70a17f7 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -239,7 +239,7 @@ XEN_CPUFEATURE(IBPB, 8*32+12) /*A IBPB support only (no IBRS, used by
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
-XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
+XEN_CPUFEATURE(SSBD, 9*32+31) /*A MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
--
2.1.4
[-- Attachment #47: xsa263-4.9/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch --]
[-- Type: application/octet-stream, Size: 3871 bytes --]
From 79570591b03f3a52ed7c7d923dc943a6b9912f63 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 12:21:00 +0100
Subject: [PATCH] x86/spec_ctrl: Read MSR_ARCH_CAPABILITIES only once
Make it available from the beginning of init_speculation_mitigations(), and
pass it into appropriate functions. Fix an RSBA typo while moving the
affected comment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d6c65187252a6c1810fd24c4d46f812840de8d3c)
---
xen/arch/x86/spec_ctrl.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index fa67a0f..dc90743 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -81,18 +81,15 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
-static void __init print_details(enum ind_thunk thunk)
+static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
unsigned int _7d0 = 0, e8b = 0, tmp;
- uint64_t caps = 0;
/* Collect diagnostics about available mitigations. */
if ( boot_cpu_data.cpuid_level >= 7 )
cpuid_count(7, 0, &tmp, &tmp, &tmp, &_7d0);
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- if ( _7d0 & cpufeat_mask(X86_FEATURE_ARCH_CAPS) )
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
@@ -125,7 +122,7 @@ static void __init print_details(enum ind_thunk thunk)
}
/* Calculate whether Retpoline is known-safe on this CPU. */
-static bool __init retpoline_safe(void)
+static bool __init retpoline_safe(uint64_t caps)
{
unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
@@ -136,19 +133,12 @@ static bool __init retpoline_safe(void)
boot_cpu_data.x86 != 6 )
return false;
- if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
- {
- uint64_t caps;
-
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
-
- /*
- * RBSA may be set by a hypervisor to indicate that we may move to a
- * processor which isn't retpoline-safe.
- */
- if ( caps & ARCH_CAPS_RSBA )
- return false;
- }
+ /*
+ * RSBA may be set by a hypervisor to indicate that we may move to a
+ * processor which isn't retpoline-safe.
+ */
+ if ( caps & ARCH_CAPS_RSBA )
+ return false;
switch ( boot_cpu_data.x86_model )
{
@@ -218,6 +208,10 @@ void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
bool ibrs = false;
+ uint64_t caps = 0;
+
+ if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
+ rdmsrl(MSR_ARCH_CAPABILITIES, caps);
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -246,7 +240,7 @@ void __init init_speculation_mitigations(void)
* On Intel hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe() )
+ else if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = true;
@@ -331,7 +325,7 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_bti_ist_info has been calculated. */
init_shadow_spec_ctrl_state();
- print_details(thunk);
+ print_details(thunk, caps);
}
static void __init __maybe_unused build_assertions(void)
--
2.1.4
[-- Attachment #48: xsa263-4.9/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch --]
[-- Type: application/octet-stream, Size: 5070 bytes --]
From c7bf6074ea8c3496894d517371ea6d9d3bd90c98 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Express Xen's choice of MSR_SPEC_CTRL value as
a variable
At the moment, we have two different encodings of Xen's MSR_SPEC_CTRL value,
which is a side effect of how the Spectre series developed. One encoding is
via an alias with the bottom bit of bti_ist_info, and can encode IBRS or not,
but not other configurations such as STIBP.
Break Xen's value out into a separate variable (in the top of stack block for
XPTI reasons) and use this instead of bti_ist_info in the IST path.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 66dfae0f32bfbc899c2f3446d5ee57068cb7f957)
---
xen/arch/x86/spec_ctrl.c | 8 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/include/asm-x86/current.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 2 ++
xen/include/asm-x86/spec_ctrl_asm.h | 8 ++------
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index dc90743..1143521 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,7 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
+uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_bti_ist_info;
static int __init parse_bti(const char *s)
@@ -285,11 +286,14 @@ void __init init_speculation_mitigations(void)
* guests.
*/
if ( ibrs )
+ {
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
+ }
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR | ibrs;
+ default_bti_ist_info |= BTI_IST_WRMSR;
}
/*
@@ -330,8 +334,6 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
- /* The optimised assembly relies on this alias. */
- BUILD_BUG_ON(BTI_IST_IBRS != SPEC_CTRL_IBRS);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index d66dbf0..6dd0476 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -142,6 +142,7 @@ void __dummy__(void)
OFFSET(CPUINFO_xen_cr3, struct cpu_info, xen_cr3);
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
+ OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index 4678a0f..d10b13c 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -56,6 +56,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
+ uint8_t xen_spec_ctrl;
bool use_shadow_spec_ctrl;
uint8_t bti_ist_info;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5ab4ff3..5e4fc84 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_bti_ist_info;
static inline void init_shadow_spec_ctrl_state(void)
@@ -34,6 +35,7 @@ static inline void init_shadow_spec_ctrl_state(void)
struct cpu_info *info = get_cpu_info();
info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->xen_spec_ctrl = default_xen_spec_ctrl;
info->bti_ist_info = default_bti_ist_info;
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 69cf3cc..9c16945 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -21,7 +21,6 @@
#define __X86_SPEC_CTRL_ASM_H__
/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_IBRS (1 << 0)
#define BTI_IST_WRMSR (1 << 1)
#define BTI_IST_RSB (1 << 2)
@@ -285,12 +284,9 @@
setz %dl
and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
- /*
- * Load Xen's intended value. SPEC_CTRL_IBRS vs 0 is encoded in the
- * bottom bit of bti_ist_info, via a deliberate alias with BTI_IST_IBRS.
- */
+ /* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
- and $BTI_IST_IBRS, %eax
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
xor %edx, %edx
wrmsr
--
2.1.4
[-- Attachment #49: xsa263-4.9/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch --]
[-- Type: application/octet-stream, Size: 13293 bytes --]
From e57b5ea6fa2b2d93fde033cec11d6ec6d524ade3 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Merge bti_ist_info and use_shadow_spec_ctrl
into spec_ctrl_flags
All 3 bits of information here are control flags for the entry/exit code
behaviour. Treat them as such, rather than having two different variables.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 5262ba2e7799001402dfe139ff944e035dfff928)
---
xen/arch/x86/acpi/power.c | 4 +--
xen/arch/x86/spec_ctrl.c | 10 ++++---
xen/arch/x86/x86_64/asm-offsets.c | 3 +--
xen/include/asm-x86/current.h | 3 +--
xen/include/asm-x86/nops.h | 5 ++--
xen/include/asm-x86/spec_ctrl.h | 10 +++----
xen/include/asm-x86/spec_ctrl_asm.h | 52 ++++++++++++++++++++-----------------
7 files changed, 45 insertions(+), 42 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index f7085d3..f3480aa 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -215,7 +215,7 @@ static int enter_state(u32 state)
ci = get_cpu_info();
spec_ctrl_enter_idle(ci);
/* Avoid NMI/#MC using MSR_SPEC_CTRL until we've reloaded microcode. */
- ci->bti_ist_info = 0;
+ ci->spec_ctrl_flags &= ~SCF_ist_wrmsr;
ACPI_FLUSH_CPU_CACHE();
@@ -256,7 +256,7 @@ static int enter_state(u32 state)
microcode_resume_cpu(0);
/* Re-enabled default NMI/#MC use of MSR_SPEC_CTRL. */
- ci->bti_ist_info = default_bti_ist_info;
+ ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
spec_ctrl_exit_idle(ci);
done:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1143521..2d69910 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -39,7 +39,7 @@ static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
-uint8_t __read_mostly default_bti_ist_info;
+uint8_t __read_mostly default_spec_ctrl_flags;
static int __init parse_bti(const char *s)
{
@@ -293,7 +293,7 @@ void __init init_speculation_mitigations(void)
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
/*
@@ -312,7 +312,7 @@ void __init init_speculation_mitigations(void)
if ( opt_rsb_native )
{
setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
- default_bti_ist_info |= BTI_IST_RSB;
+ default_spec_ctrl_flags |= SCF_ist_rsb;
}
/*
@@ -326,7 +326,7 @@ void __init init_speculation_mitigations(void)
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
opt_ibpb = false;
- /* (Re)init BSP state now that default_bti_ist_info has been calculated. */
+ /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
print_details(thunk, caps);
@@ -334,6 +334,8 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
+ /* The optimised assembly relies on this alias. */
+ BUILD_BUG_ON(SCF_use_shadow != 1);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 6dd0476..cc97d75 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -143,8 +143,7 @@ void __dummy__(void)
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
- OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
- OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
+ OFFSET(CPUINFO_spec_ctrl_flags, struct cpu_info, spec_ctrl_flags);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
BLANK();
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index d10b13c..7afff0e 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -57,8 +57,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
uint8_t xen_spec_ctrl;
- bool use_shadow_spec_ctrl;
- uint8_t bti_ist_info;
+ uint8_t spec_ctrl_flags;
unsigned long __pad;
/* get_stack_bottom() must be 16-byte aligned */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index 37f9819..b744895 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -62,10 +62,9 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP21 ASM_NOP8; ASM_NOP8; ASM_NOP5
+#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP29 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP5
-#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5e4fc84..059e291 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -28,15 +28,15 @@ void init_speculation_mitigations(void);
extern bool opt_ibpb;
extern uint8_t default_xen_spec_ctrl;
-extern uint8_t default_bti_ist_info;
+extern uint8_t default_spec_ctrl_flags;
static inline void init_shadow_spec_ctrl_state(void)
{
struct cpu_info *info = get_cpu_info();
- info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->shadow_spec_ctrl = 0;
info->xen_spec_ctrl = default_xen_spec_ctrl;
- info->bti_ist_info = default_bti_ist_info;
+ info->spec_ctrl_flags = default_spec_ctrl_flags;
}
/* WARNING! `ret`, `call *`, `jmp *` not safe after this call. */
@@ -50,7 +50,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
*/
info->shadow_spec_ctrl = val;
barrier();
- info->use_shadow_spec_ctrl = true;
+ info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
@@ -65,7 +65,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
* Disable shadowing before updating the MSR. There are no SMP issues
* here; only local processor ordering concerns.
*/
- info->use_shadow_spec_ctrl = false;
+ info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 9c16945..582403a 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -20,9 +20,10 @@
#ifndef __X86_SPEC_CTRL_ASM_H__
#define __X86_SPEC_CTRL_ASM_H__
-/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_WRMSR (1 << 1)
-#define BTI_IST_RSB (1 << 2)
+/* Encoding of cpuinfo.spec_ctrl_flags */
+#define SCF_use_shadow (1 << 0)
+#define SCF_ist_wrmsr (1 << 1)
+#define SCF_ist_rsb (1 << 2)
#ifdef __ASSEMBLY__
#include <asm/msr-index.h>
@@ -49,20 +50,20 @@
* after VMEXIT. The VMEXIT-specific code reads MSR_SPEC_CTRL and updates
* current before loading Xen's MSR_SPEC_CTRL setting.
*
- * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and
- * use_shadow_spec_ctrl boolean per cpu. The synchronous use is:
+ * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and a use_shadow
+ * boolean in the per cpu spec_ctrl_flags. The synchronous use is:
*
* 1) Store guest value in shadow_spec_ctrl
- * 2) Set use_shadow_spec_ctrl boolean
+ * 2) Set the use_shadow boolean
* 3) Load guest value into MSR_SPEC_CTRL
* 4) Exit to guest
* 5) Entry from guest
- * 6) Clear use_shadow_spec_ctrl boolean
+ * 6) Clear the use_shadow boolean
* 7) Load Xen's value into MSR_SPEC_CTRL
*
* The asynchronous use for interrupts/exceptions is:
* - Set/clear IBRS on entry to Xen
- * - On exit to Xen, check use_shadow_spec_ctrl
+ * - On exit to Xen, check use_shadow
* - If set, load shadow_spec_ctrl
*
* Therefore, an interrupt/exception which hits the synchronous path between
@@ -133,7 +134,7 @@
xor %edx, %edx
/* Clear SPEC_CTRL shadowing *before* loading Xen's value. */
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
mov $\ibrs_val, %eax
@@ -159,12 +160,14 @@
* block so calculate the position directly.
*/
.if \maybexen
+ xor %eax, %eax
/* Branchless `if ( !xen ) clear_shadowing` */
testb $3, UREGS_cs(%rsp)
- setz %al
- and %al, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %al
+ not %eax
+ and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
.else
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
.endif
/* Load Xen's intended value. */
@@ -183,8 +186,8 @@
*/
xor %edx, %edx
- cmpb %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%rbx)
- je .L\@_skip
+ testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
+ jz .L\@_skip
mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax
mov $MSR_SPEC_CTRL, %ecx
@@ -205,7 +208,7 @@
mov %eax, CPUINFO_shadow_spec_ctrl(%rsp)
/* Set SPEC_CTRL shadowing *before* loading the guest value. */
- movb $1, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ orb $SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
mov $MSR_SPEC_CTRL, %ecx
xor %edx, %edx
@@ -216,7 +219,7 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -228,7 +231,7 @@
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP21), \
+ ALTERNATIVE_2 __stringify(ASM_NOP22), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -239,7 +242,7 @@
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP29), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -267,22 +270,23 @@
* This is logical merge of DO_OVERWRITE_RSB and DO_SPEC_CTRL_ENTRY
* maybexen=1, but with conditionals rather than alternatives.
*/
- movzbl STACK_CPUINFO_FIELD(bti_ist_info)(%r14), %eax
+ movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %eax
- testb $BTI_IST_RSB, %al
+ test $SCF_ist_rsb, %al
jz .L\@_skip_rsb
DO_OVERWRITE_RSB tmp=rdx /* Clobbers %rcx/%rdx */
.L\@_skip_rsb:
- testb $BTI_IST_WRMSR, %al
+ test $SCF_ist_wrmsr, %al
jz .L\@_skip_wrmsr
xor %edx, %edx
testb $3, UREGS_cs(%rsp)
- setz %dl
- and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %dl
+ not %edx
+ and %dl, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
/* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
@@ -309,7 +313,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
* Requires %rbx=stack_end
* Clobbers %rax, %rcx, %rdx
*/
- testb $BTI_IST_WRMSR, STACK_CPUINFO_FIELD(bti_ist_info)(%rbx)
+ testb $SCF_ist_wrmsr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
jz .L\@_skip
DO_SPEC_CTRL_EXIT_TO_XEN
--
2.1.4
[-- Attachment #50: xsa263-4.9/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch --]
[-- Type: application/octet-stream, Size: 10505 bytes --]
From df0421d0368ba545f37b07c08b13591540227dcc Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Fold the XEN_IBRS_{SET,CLEAR} ALTERNATIVES
together
Currently, the SPEC_CTRL_{ENTRY,EXIT}_* macros encode Xen's choice of
MSR_SPEC_CTRL as an immediate constant, and chooses between IBRS or not by
doubling up the entire alternative block.
There is now a variable holding Xen's choice of value, so use that and
simplify the alternatives.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit af949407eaba7af71067f23d5866cd0bf1f1144d)
---
xen/arch/x86/spec_ctrl.c | 12 +++++-----
xen/include/asm-x86/cpufeatures.h | 3 +--
xen/include/asm-x86/nops.h | 3 ++-
xen/include/asm-x86/spec_ctrl.h | 6 ++---
xen/include/asm-x86/spec_ctrl_asm.h | 45 +++++++++++++------------------------
5 files changed, 26 insertions(+), 43 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 2d69910..b62cfcc 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,8 +112,9 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_XEN_IBRS_SET) ? " IBRS+" :
- boot_cpu_has(X86_FEATURE_XEN_IBRS_CLEAR) ? " IBRS-" : "",
+ boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
+ " IBRS-" : "",
opt_ibpb ? " IBPB" : "",
boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
@@ -285,13 +286,10 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+
if ( ibrs )
- {
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
- }
- else
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index 84d5c5b..6119bab 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -28,8 +28,7 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+14) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
-XEN_CPUFEATURE(XEN_IBRS_SET, (FSCAPINTS+0)*32+18) /* IBRSB && IRBS set in Xen */
-XEN_CPUFEATURE(XEN_IBRS_CLEAR, (FSCAPINTS+0)*32+19) /* IBRSB && IBRS clear in Xen */
+XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for native */
XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for vmexit */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index b744895..913e9f0 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -62,9 +62,10 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP25 ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 059e291..7d7c42e 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,14 +52,14 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
/* WARNING! `ret`, `call *`, `jmp *` not safe before this call. */
static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
{
- uint32_t val = SPEC_CTRL_IBRS;
+ uint32_t val = info->xen_spec_ctrl;
/*
* Disable shadowing before updating the MSR. There are no SMP issues
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 582403a..941aeb7 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -117,7 +117,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -137,11 +137,11 @@
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
- mov $\ibrs_val, %eax
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
wrmsr
.endm
-.macro DO_SPEC_CTRL_ENTRY maybexen:req ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY maybexen:req
/*
* Requires %rsp=regs (also cpuinfo if !maybexen)
* Requires %r14=stack_end (if maybexen)
@@ -166,12 +166,12 @@
setnz %al
not %eax
and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
.else
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
.endif
- /* Load Xen's intended value. */
- mov $\ibrs_val, %eax
wrmsr
.endm
@@ -219,47 +219,32 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP36), \
+ DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP22), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP25), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP33), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
- ALTERNATIVE_2 __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP17), \
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
/* Use when exiting to guest context. */
#define SPEC_CTRL_EXIT_TO_GUEST \
- ALTERNATIVE_2 __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
--
2.1.4
[-- Attachment #51: xsa263-4.9/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch --]
[-- Type: application/octet-stream, Size: 12273 bytes --]
From e00632c06f088bfe4bd110686faa4a7e01a5667b Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 30 Apr 2018 14:20:23 +0100
Subject: [PATCH] x86/spec_ctrl: Rename bits of infrastructure to avoid NATIVE
and VMEXIT
In hindsight, using NATIVE and VMEXIT as naming terminology was not clever.
A future change wants to split SPEC_CTRL_EXIT_TO_GUEST into PV and HVM
specific implementations, and using VMEXIT as a term is completely wrong.
Take the opportunity to fix some stale documentation in spec_ctrl_asm.h. The
IST helpers were missing from the large comment block, and since
SPEC_CTRL_ENTRY_FROM_INTR_IST was introduced, we've gained a new piece of
functionality which currently depends on the fine grain control, which exists
in lieu of livepatching. Note this in the comment.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d9822b8a38114e96e4516dc998f4055249364d5d)
---
xen/arch/x86/hvm/svm/entry.S | 4 ++--
xen/arch/x86/hvm/vmx/entry.S | 4 ++--
xen/arch/x86/spec_ctrl.c | 20 ++++++++++----------
xen/arch/x86/x86_64/compat/entry.S | 2 +-
xen/arch/x86/x86_64/entry.S | 2 +-
xen/include/asm-x86/cpufeatures.h | 4 ++--
xen/include/asm-x86/spec_ctrl_asm.h | 36 +++++++++++++++++++++++++-----------
7 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S
index 7c91595..d0e9171 100644
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -80,7 +80,7 @@ UNLIKELY_END(svm_trace)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
pop %r15
pop %r14
@@ -105,7 +105,7 @@ UNLIKELY_END(svm_trace)
GET_CURRENT(bx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov VCPU_svm_vmcb(%rbx),%rcx
diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S
index f823850..bdcd3ca 100644
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -36,7 +36,7 @@ ENTRY(vmx_asm_vmexit_handler)
movb $1,VCPU_vmx_launched(%rbx)
mov %rax,VCPU_hvm_guest_cr2(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov %rsp,%rdi
@@ -71,7 +71,7 @@ UNLIKELY_END(realmode)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
mov VCPU_hvm_guest_cr2(%rbx),%rax
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index b62cfcc..015a9e2 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -35,8 +35,8 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_native = true;
-static bool __initdata opt_rsb_vmexit = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -69,9 +69,9 @@ static int __init parse_bti(const char *s)
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
else if ( (val = parse_boolean("rsb_native", s, ss)) >= 0 )
- opt_rsb_native = val;
+ opt_rsb_pv = val;
else if ( (val = parse_boolean("rsb_vmexit", s, ss)) >= 0 )
- opt_rsb_vmexit = val;
+ opt_rsb_hvm = val;
else
rc = -EINVAL;
@@ -116,8 +116,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -307,9 +307,9 @@ void __init init_speculation_mitigations(void)
* If a processors speculates to 32bit PV guest kernel mappings, it is
* speculating in 64bit supervisor mode, and can leak data.
*/
- if ( opt_rsb_native )
+ if ( opt_rsb_pv )
{
- setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_PV);
default_spec_ctrl_flags |= SCF_ist_rsb;
}
@@ -317,8 +317,8 @@ void __init init_speculation_mitigations(void)
* HVM guests can always poison the RSB to point at Xen supervisor
* mappings.
*/
- if ( opt_rsb_vmexit )
- setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
+ if ( opt_rsb_hvm )
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_HVM);
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index c538643..63cd51f 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -163,7 +163,7 @@ ENTRY(compat_restore_all_guest)
mov VCPU_arch_spec_ctrl(%rbx), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL adj=8 compat=1
.Lft0: iretq
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 7004f52..cdf5090 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -193,7 +193,7 @@ restore_all_guest:
mov %r15d, %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL
testw $TRAP_syscall,4(%rsp)
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index 6119bab..1353fe5 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -29,6 +29,6 @@ XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
-XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for native */
-XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for vmexit */
+XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
+XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 941aeb7..b330e20 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -72,11 +72,14 @@
*
* The following ASM fragments implement this algorithm. See their local
* comments for further details.
- * - SPEC_CTRL_ENTRY_FROM_VMEXIT
+ * - SPEC_CTRL_ENTRY_FROM_HVM
* - SPEC_CTRL_ENTRY_FROM_PV
* - SPEC_CTRL_ENTRY_FROM_INTR
+ * - SPEC_CTRL_ENTRY_FROM_INTR_IST
+ * - SPEC_CTRL_EXIT_TO_XEN_IST
* - SPEC_CTRL_EXIT_TO_XEN
- * - SPEC_CTRL_EXIT_TO_GUEST
+ * - SPEC_CTRL_EXIT_TO_PV
+ * - SPEC_CTRL_EXIT_TO_HVM
*/
.macro DO_OVERWRITE_RSB tmp=rax
@@ -117,7 +120,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
+.macro DO_SPEC_CTRL_ENTRY_FROM_HVM
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -216,23 +219,23 @@
.endm
/* Use after a VMEXIT from an HVM guest. */
-#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
+#define SPEC_CTRL_ENTRY_FROM_HVM \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
@@ -241,12 +244,22 @@
ALTERNATIVE __stringify(ASM_NOP17), \
DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
-/* Use when exiting to guest context. */
-#define SPEC_CTRL_EXIT_TO_GUEST \
+/* Use when exiting to PV guest context. */
+#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
-/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
+/* Use when exiting to HVM guest context. */
+#define SPEC_CTRL_EXIT_TO_HVM \
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+
+/*
+ * Use in IST interrupt/exception context. May interrupt Xen or PV context.
+ * Fine grain control of SCF_ist_wrmsr is needed for safety in the S3 resume
+ * path to avoid using MSR_SPEC_CTRL before the microcode introducing it has
+ * been reloaded.
+ */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
/*
* Requires %rsp=regs, %r14=stack_end
@@ -293,6 +306,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
UNLIKELY_END(\@_serialise)
.endm
+/* Use when exiting to Xen in IST context. */
.macro SPEC_CTRL_EXIT_TO_XEN_IST
/*
* Requires %rbx=stack_end
--
2.1.4
[-- Attachment #52: xsa263-4.9/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch --]
[-- Type: application/octet-stream, Size: 3083 bytes --]
From 12dec36f81cdd8aecc5388f983884cbb4e437ce8 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 May 2018 14:06:16 +0100
Subject: [PATCH] x86/spec_ctrl: Elide MSR_SPEC_CTRL handling in idle context
when possible
If Xen is virtualising MSR_SPEC_CTRL handling for guests, but using 0 as its
own MSR_SPEC_CTRL value, spec_ctrl_{enter,exit}_idle() need not write to the
MSR.
Requested-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 94df6e8588e35cc2028ccb3fd2921c6e6360605e)
---
xen/arch/x86/spec_ctrl.c | 4 ++++
xen/include/asm-x86/cpufeatures.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 4 ++--
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 015a9e2..55ef79f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -327,6 +327,10 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
+ /* If Xen is using any MSR_SPEC_CTRL settings, adjust the idle path. */
+ if ( default_xen_spec_ctrl )
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
+
print_details(thunk, caps);
}
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index 1353fe5..f419c36 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -32,3 +32,4 @@ XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xe
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* SC_MSR && default_xen_spec_ctrl */
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 7d7c42e..77f92ba 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,7 +52,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
--
2.1.4
[-- Attachment #53: xsa263-4.9/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch --]
[-- Type: application/octet-stream, Size: 6034 bytes --]
From 161f6c16d82ed912f6b656c90572ca265a4f0f78 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Split X86_FEATURE_SC_MSR into PV and HVM
variants
In order to separately control whether MSR_SPEC_CTRL is virtualised for PV and
HVM guests, split the feature used to control runtime alternatives into two.
Xen will use MSR_SPEC_CTRL itself if either of these features are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit fa9eb09d446a1279f5e861e6b84fa8675dabf148)
---
xen/arch/x86/spec_ctrl.c | 6 ++++--
xen/include/asm-x86/cpufeatures.h | 5 +++--
xen/include/asm-x86/spec_ctrl_asm.h | 12 ++++++------
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 55ef79f..a940308 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,7 +112,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
@@ -286,7 +287,8 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
- setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
if ( ibrs )
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index f419c36..f568265 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -28,8 +28,9 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+14) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+15) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+16) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+17) /* IBRSB || IBPB */
-XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen */
+XEN_CPUFEATURE(SC_MSR_PV, (FSCAPINTS+0)*32+18) /* MSR_SPEC_CTRL used by Xen for PV */
+XEN_CPUFEATURE(SC_MSR_HVM, (FSCAPINTS+0)*32+19) /* MSR_SPEC_CTRL used by Xen for HVM */
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+20) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+21) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+22) /* XPTI mitigation not in use */
-XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* SC_MSR && default_xen_spec_ctrl */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+23) /* (SC_MSR_PV || SC_MSR_HVM) && default_xen_spec_ctrl */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index b330e20..4d864eb 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -223,36 +223,36 @@
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR_HVM
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR_PV
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR_PV
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
ALTERNATIVE __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV
/* Use when exiting to PV guest context. */
#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV
/* Use when exiting to HVM guest context. */
#define SPEC_CTRL_EXIT_TO_HVM \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_HVM
/*
* Use in IST interrupt/exception context. May interrupt Xen or PV context.
--
2.1.4
[-- Attachment #54: xsa263-4.9/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch --]
[-- Type: application/octet-stream, Size: 4771 bytes --]
From be34661999a9090bd0dfbf2e47af3c12889a5ccf Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 9 May 2018 13:59:56 +0100
Subject: [PATCH] x86/spec_ctrl: Explicitly set Xen's default MSR_SPEC_CTRL
value
With the impending ability to disable MSR_SPEC_CTRL handling on a
per-guest-type basis, the first exit-from-guest may not have the side effect
of loading Xen's choice of value. Explicitly set Xen's default during the BSP
and AP boot paths.
For the BSP however, delay setting a non-zero MSR_SPEC_CTRL default until
after dom0 has been constructed when safe to do so. Oracle report that this
speeds up boots of some hardware by 50s.
"when safe to do so" is based on whether we are virtualised. A native boot
won't have any other code running in a position to mount an attack.
Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb8c12020307b39a89273d7699e89000451987ab)
---
xen/arch/x86/setup.c | 7 +++++++
xen/arch/x86/smpboot.c | 8 ++++++++
xen/arch/x86/spec_ctrl.c | 32 ++++++++++++++++++++++++++++++++
xen/include/asm-x86/spec_ctrl.h | 2 ++
4 files changed, 49 insertions(+)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 29cbe42..bae9ca0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1687,6 +1687,13 @@ void __init noreturn __start_xen(unsigned long mbi_p)
setup_io_bitmap(dom0);
+ if ( bsp_delay_spec_ctrl )
+ {
+ get_cpu_info()->spec_ctrl_flags &= ~SCF_use_shadow;
+ barrier();
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+ }
+
/* Jump to the 1:1 virtual mappings of cpu0_stack. */
asm volatile ("mov %[stk], %%rsp; jmp %c[fn]" ::
[stk] "g" (__va(__pa(get_stack_bottom()))),
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 3d5faa2..f4a1588 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -344,6 +344,14 @@ void start_secondary(void *unused)
else
microcode_resume_cpu(cpu);
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. Note: MSR_SPEC_CTRL may only become available
+ * after loading microcode.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+
smp_callin();
init_percpu_time();
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index a940308..3adec1a 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,8 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_pv = true;
static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
+
+bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -334,6 +336,36 @@ void __init init_speculation_mitigations(void)
setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
print_details(thunk, caps);
+
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. For performance reasons, when safe to do so, we
+ * delay applying non-zero settings until after dom0 has been constructed.
+ *
+ * "when safe to do so" is based on whether we are virtualised. A native
+ * boot won't have any other code running in a position to mount an
+ * attack.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ {
+ bsp_delay_spec_ctrl = !cpu_has_hypervisor && default_xen_spec_ctrl;
+
+ /*
+ * If delaying MSR_SPEC_CTRL setup, use the same mechanism as
+ * spec_ctrl_enter_idle(), by using a shadow value of zero.
+ */
+ if ( bsp_delay_spec_ctrl )
+ {
+ struct cpu_info *info = get_cpu_info();
+
+ info->shadow_spec_ctrl = 0;
+ barrier();
+ info->spec_ctrl_flags |= SCF_use_shadow;
+ barrier();
+ }
+
+ wrmsrl(MSR_SPEC_CTRL, bsp_delay_spec_ctrl ? 0 : default_xen_spec_ctrl);
+ }
}
static void __init __maybe_unused build_assertions(void)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 77f92ba..c6a38f4 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,8 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+
+extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_spec_ctrl_flags;
--
2.1.4
[-- Attachment #55: xsa263-4.9/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch --]
[-- Type: application/octet-stream, Size: 5183 bytes --]
From eea942504afa2b91a3934b8f712758277a0adf00 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 1 May 2018 11:59:03 +0100
Subject: [PATCH] x86/cpuid: Improvements to guest policies for speculative
sidechannel features
If Xen isn't virtualising MSR_SPEC_CTRL for guests, IBRSB shouldn't be
advertised. It is not currently possible to express this via the existing
command line options, but such an ability will be introduced.
Another useful option in some usecases is to offer IBPB without IBRS. When a
guest kernel is known to be compatible (uses retpoline and knows about the AMD
IBPB feature bit), an administrator with pre-Skylake hardware may wish to hide
IBRS. This allows the VM to have full protection, without Xen or the VM
needing to touch MSR_SPEC_CTRL, which can reduce the overhead of Spectre
mitigations.
Break the logic common to both PV and HVM CPUID calculations into a common
helper, to avoid duplication.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb06b308ec71b23f37a44f5e2351fe2cae0306e9)
---
xen/arch/x86/cpuid.c | 60 ++++++++++++++++++++++++++++++++--------------------
1 file changed, 37 insertions(+), 23 deletions(-)
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 7f7f6be..ebc1638 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -374,6 +374,28 @@ static void __init calculate_host_policy(void)
}
}
+static void __init guest_common_feature_adjustments(uint32_t *fs)
+{
+ /* Unconditionally claim to be able to set the hypervisor bit. */
+ __set_bit(X86_FEATURE_HYPERVISOR, fs);
+
+ /*
+ * If IBRS is offered to the guest, unconditionally offer STIBP. It is a
+ * nop on non-HT hardware, and has this behaviour to make heterogeneous
+ * setups easier to manage.
+ */
+ if ( test_bit(X86_FEATURE_IBRSB, fs) )
+ __set_bit(X86_FEATURE_STIBP, fs);
+
+ /*
+ * On hardware which supports IBRS/IBPB, we can offer IBPB independently
+ * of IBRS by using the AMD feature bit. An administrator may wish for
+ * performance reasons to offer IBPB without IBRS.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ __set_bit(X86_FEATURE_IBPB, fs);
+}
+
static void __init calculate_pv_max_policy(void)
{
struct cpuid_policy *p = &pv_max_policy;
@@ -386,18 +408,14 @@ static void __init calculate_pv_max_policy(void)
for ( i = 0; i < ARRAY_SIZE(pv_featureset); ++i )
pv_featureset[i] &= pv_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, pv_featureset);
-
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, pv_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, pv_featureset);
+ /*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for PV guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ __clear_bit(X86_FEATURE_IBRSB, pv_featureset);
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, pv_featureset);
- }
+ guest_common_feature_adjustments(pv_featureset);
sanitise_featureset(pv_featureset);
cpuid_featureset_to_policy(pv_featureset, p);
@@ -425,9 +443,6 @@ static void __init calculate_hvm_max_policy(void)
for ( i = 0; i < ARRAY_SIZE(hvm_featureset); ++i )
hvm_featureset[i] &= hvm_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, hvm_featureset);
-
/*
* Xen can provide an APIC emulation to HVM guests even if the host's APIC
* isn't enabled.
@@ -443,6 +458,13 @@ static void __init calculate_hvm_max_policy(void)
__set_bit(X86_FEATURE_SEP, hvm_featureset);
/*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for HVM guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_HVM) )
+ __clear_bit(X86_FEATURE_IBRSB, hvm_featureset);
+
+ /*
* With VT-x, some features are only supported by Xen if dedicated
* hardware support is also available.
*/
@@ -455,15 +477,7 @@ static void __init calculate_hvm_max_policy(void)
__clear_bit(X86_FEATURE_XSAVES, hvm_featureset);
}
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, hvm_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, hvm_featureset);
-
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, hvm_featureset);
- }
+ guest_common_feature_adjustments(hvm_featureset);
sanitise_featureset(hvm_featureset);
cpuid_featureset_to_policy(hvm_featureset, p);
--
2.1.4
[-- Attachment #56: xsa263-4.9/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch --]
[-- Type: application/octet-stream, Size: 14129 bytes --]
From 128146fcee77e9f27262dda89620f20f7386cd04 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:52:55 +0100
Subject: [PATCH] x86/spec_ctrl: Introduce a new `spec-ctrl=` command line
argument to replace `bti=`
In hindsight, the options for `bti=` aren't as flexible or useful as expected
(including several options which don't appear to behave as intended).
Changing the behaviour of an existing option is problematic for compatibility,
so introduce a new `spec-ctrl=` in the hopes that we can do better.
One common way of deploying Xen is with a single PV dom0 and all domUs being
HVM domains. In such a setup, an administrator who has weighed up the risks
may wish to forgo protection against malicious PV domains, to reduce the
overall performance hit. To cater for this usecase, `spec-ctrl=no-pv` will
disable all speculative protection for PV domains, while leaving all
speculative protection for HVM domains intact.
For coding clarity as much as anything else, the suboptions are grouped by
logical area; those which affect the alternatives blocks, and those which
affect Xen's in-hypervisor settings. See the xen-command-line.markdown for
full details of the new options.
While changing the command line options, take the time to change how the data
is reported to the user. The three DEBUG printks are upgraded to unilateral,
as they are all relevant pieces of information, and the old "mitigations:"
line is split in the two logical areas described above.
Sample output from booting with `spec-ctrl=no-pv` looks like:
(XEN) Speculative mitigation facilities:
(XEN) Hardware features: IBRS/IBPB STIBP IBPB
(XEN) Compiled-in support: INDIRECT_THUNK
(XEN) Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS-, Other: IBPB
(XEN) Support for VMs: PV: None, HVM: MSR_SPEC_CTRL RSB
(XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 3352afc26c497d26ecb70527db3cb29daf7b1422)
---
docs/misc/xen-command-line.markdown | 49 +++++++++++
xen/arch/x86/spec_ctrl.c | 160 ++++++++++++++++++++++++++++++------
2 files changed, 186 insertions(+), 23 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index dbea91d..cf88419 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -255,6 +255,9 @@ the NMI watchdog is also enabled.
### bti (x86)
> `= List of [ thunk=retpoline|lfence|jmp, ibrs=<bool>, ibpb=<bool>, rsb_{vmexit,native}=<bool> ]`
+**WARNING: This command line option is deprecated, and superseded by
+_spec-ctrl=_ - using both options in combination is undefined.**
+
Branch Target Injection controls. By default, Xen will pick the most
appropriate BTI mitigations based on compiled in support, loaded microcode,
and hardware details.
@@ -1606,6 +1609,52 @@ enforces the maximum theoretically necessary timeout of 670ms. Any number
is being interpreted as a custom timeout in milliseconds. Zero or boolean
false disable the quirk workaround, which is also the default.
+### spec-ctrl (x86)
+> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+
+Controls for speculative execution sidechannel mitigations. By default, Xen
+will pick the most appropriate mitigations based on compiled in support,
+loaded microcode, and hardware details, and will virtualise appropriate
+mitigations for guests to use.
+
+**WARNING: Any use of this option may interfere with heuristics. Use with
+extreme care.**
+
+An overall boolean value, `spec-ctrl=no`, can be specified to turn off all
+mitigations, including pieces of infrastructure used to virtualise certain
+mitigation features for guests. Alternatively, a slightly more restricted
+`spec-ctrl=no-xen` can be used to turn off all of Xen's mitigations, while
+leaving the virtualisation support in place for guests to use. Use of a
+positive boolean value for either of these options is invalid.
+
+The booleans `pv=`, `hvm=`, `msr-sc=` and `rsb=` offer fine grained control
+over the alternative blocks used by Xen. These impact Xen's ability to
+protect itself, and Xen's ability to virtualise support for guests to use.
+
+* `pv=` and `hvm=` offer control over all suboptions for PV and HVM guests
+ respectively.
+* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
+ on entry and exit. These blocks are necessary to virtualise support for
+ guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
+ Return Address Stack on entry to Xen.
+
+If Xen was compiled with INDIRECT\_THUNK support, `bti-thunk=` can be used to
+select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
+locations. The default thunk is `retpoline` (generally preferred for Intel
+hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
+overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+
+On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
+`ibrs=` option can be used to force or prevent Xen using the feature itself.
+If Xen is not using IBRS itself, functionality is still set up so IBRS can be
+virtualised for guests.
+
+On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
+option can be used to force (the default) or prevent Xen from issuing branch
+prediction barriers on vcpu context switches.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 3adec1a..1a59b54 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -26,6 +26,13 @@
#include <asm/spec_ctrl.h>
#include <asm/spec_ctrl_asm.h>
+/* Cmdline controls for Xen's alternative blocks. */
+static bool __initdata opt_msr_sc_pv = true;
+static bool __initdata opt_msr_sc_hvm = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
+
+/* Cmdline controls for Xen's speculative settings. */
static enum ind_thunk {
THUNK_DEFAULT, /* Decide which thunk to use at boot time. */
THUNK_NONE, /* Missing compiler support for thunks. */
@@ -35,8 +42,6 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_pv = true;
-static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
bool __initdata bsp_delay_spec_ctrl;
@@ -84,8 +89,95 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
+static int __init parse_spec_ctrl(const char *s)
+{
+ const char *ss;
+ int val, rc = 0;
+
+ do {
+ ss = strchr(s, ',');
+ if ( !ss )
+ ss = strchr(s, '\0');
+
+ /* Global and Xen-wide disable. */
+ val = parse_bool(s);
+ if ( !val )
+ {
+ opt_msr_sc_pv = false;
+ opt_msr_sc_hvm = false;
+
+ disable_common:
+ opt_rsb_pv = false;
+ opt_rsb_hvm = false;
+
+ opt_thunk = THUNK_JMP;
+ opt_ibrs = 0;
+ opt_ibpb = false;
+ }
+ else if ( val > 0 )
+ rc = -EINVAL;
+ else if ( (val = parse_boolean("xen", s, ss)) >= 0 )
+ {
+ if ( !val )
+ goto disable_common;
+
+ rc = -EINVAL;
+ }
+
+ /* Xen's alternative blocks. */
+ else if ( (val = parse_boolean("pv", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_rsb_pv = val;
+ }
+ else if ( (val = parse_boolean("hvm", s, ss)) >= 0 )
+ {
+ opt_msr_sc_hvm = val;
+ opt_rsb_hvm = val;
+ }
+ else if ( (val = parse_boolean("msr-sc", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_msr_sc_hvm = val;
+ }
+ else if ( (val = parse_boolean("rsb", s, ss)) >= 0 )
+ {
+ opt_rsb_pv = val;
+ opt_rsb_hvm = val;
+ }
+
+ /* Xen's speculative sidechannel mitigation settings. */
+ else if ( !strncmp(s, "bti-thunk=", 10) )
+ {
+ s += 10;
+
+ if ( !strncmp(s, "retpoline", ss - s) )
+ opt_thunk = THUNK_RETPOLINE;
+ else if ( !strncmp(s, "lfence", ss - s) )
+ opt_thunk = THUNK_LFENCE;
+ else if ( !strncmp(s, "jmp", ss - s) )
+ opt_thunk = THUNK_JMP;
+ else
+ rc = -EINVAL;
+ }
+ else if ( (val = parse_boolean("ibrs", s, ss)) >= 0 )
+ opt_ibrs = val;
+ else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
+ opt_ibpb = val;
+ else
+ rc = -EINVAL;
+
+ s = ss + 1;
+ } while ( *ss );
+
+ return rc;
+}
+custom_param("spec-ctrl", parse_spec_ctrl);
+
static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
+ bool use_spec_ctrl = (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM));
unsigned int _7d0 = 0, e8b = 0, tmp;
/* Collect diagnostics about available mitigations. */
@@ -94,10 +186,10 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
+ printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(XENLOG_DEBUG " Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
@@ -107,20 +199,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
- printk(XENLOG_DEBUG " Compiled-in support: INDIRECT_THUNK\n");
+ printk(" Compiled-in support: INDIRECT_THUNK\n");
- printk("BTI mitigations: Thunk %s, Others:%s%s%s%s\n",
+ /* Settings for Xen's protection, irrespective of guests. */
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
+ !use_spec_ctrl ? "No" :
+ (default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ opt_ibpb ? " IBPB" : "");
+
+ /*
+ * Alternatives blocks for protecting against and/or virtualising
+ * mitigation support for guests.
+ */
+ printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n",
(boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
- boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
- default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
- " IBRS-" : "",
- opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "",
+ (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ||
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -212,7 +315,7 @@ static bool __init retpoline_safe(uint64_t caps)
void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
- bool ibrs = false;
+ bool use_spec_ctrl = false, ibrs = false;
uint64_t caps = 0;
if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
@@ -282,20 +385,31 @@ void __init init_speculation_mitigations(void)
else if ( thunk == THUNK_JMP )
setup_force_cpu_cap(X86_FEATURE_IND_THUNK_JMP);
+ /*
+ * If we are on hardware supporting MSR_SPEC_CTRL, see about setting up
+ * the alternatives blocks so we can virtualise support for guests.
+ */
if ( boot_cpu_has(X86_FEATURE_IBRSB) )
{
- /*
- * Even if we've chosen to not have IBRS set in Xen context, we still
- * need the IBRS entry/exit logic to virtualise IBRS support for
- * guests.
- */
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ if ( opt_msr_sc_pv )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ }
- if ( ibrs )
- default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
+ if ( opt_msr_sc_hvm )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ }
+
+ if ( use_spec_ctrl )
+ {
+ if ( ibrs )
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ }
}
/*
--
2.1.4
[-- Attachment #57: xsa263-4.9/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4400 bytes --]
From 21c05fe06fb62aea7f29d4d216b8ab623754ae98 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index cf88419..e2b363f 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1611,7 +1611,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1655,6 +1655,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index 40c0bac..e4fea60 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -9,6 +9,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -590,6 +591,25 @@ static void init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
/* MFENCE stops RDTSC speculation */
if (!cpu_has_lfence_dispatch)
__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1a59b54..0fb628b 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool __read_mostly opt_ibpb = true;
+bool __read_mostly opt_ssbd = false;
bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -164,6 +165,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index c6a38f4..4678a40 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern bool opt_ssbd;
extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #58: xsa263-4.9/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 10325 bytes --]
From 55325566b3a14167501fd0cb045d6343b17b6946 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
tools/misc/xen-cpuid.c | 3 +--
xen/arch/x86/cpuid.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/msr-index.h | 2 ++
xen/include/public/arch-x86/cpufeatureset.h | 1 +
xen/tools/gen-cpuid.py | 17 +++++++++++++----
8 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index e2b363f..4b8e4b6 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -456,9 +456,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1636,7 +1637,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1658,7 +1659,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 3c00bb5..b426898 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -161,6 +161,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
{"arch-caps", 0x00000007, 0, CPUID_REG_EDX, 29, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"topoext", 0x80000001, NA, CPUID_REG_ECX, 22, 1},
{"tbm", 0x80000001, NA, CPUID_REG_ECX, 21, 1},
{"nodeid", 0x80000001, NA, CPUID_REG_ECX, 19, 1},
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 24800fd..9739265 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -161,8 +161,7 @@ static const char *str_7d0[32] =
[26] = "ibrsb", [27] = "stibp",
[28] = "REZ", [29] = "arch_caps",
-
- [30 ... 31] = "REZ",
+ [30] = "REZ", [31] = "ssbd",
};
static struct {
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index ebc1638..83348b5 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -43,6 +43,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 0fb628b..18515eb 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -192,26 +192,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
printk(" Compiled-in support: INDIRECT_THUNK\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -415,6 +420,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 9b0679e..4024ef5 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -38,6 +38,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -46,6 +47,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index f4b4c0f..43f42b6 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -244,6 +244,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 613b909..65526ff 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -257,10 +257,19 @@ def crunch_numbers(state):
AVX512BW, AVX512VL, AVX512VBMI, AVX512_4VNNIW,
AVX512_4FMAPS, AVX512_VPOPCNTDQ],
- # Single Thread Indirect Branch Predictors enumerates a new bit in the
- # MSR enumerated by Indirect Branch Restricted Speculation/Indirect
- # Branch Prediction Barrier enumeration.
- IBRSB: [STIBP],
+ # The features:
+ # * Single Thread Indirect Branch Predictors
+ # * Speculative Store Bypass Disable
+ #
+ # enumerate new bits in MSR_SPEC_CTRL, which is enumerated by Indirect
+ # Branch Restricted Speculation/Indirect Branch Prediction Barrier.
+ #
+ # In practice, these features also enumerate the presense of
+ # MSR_SPEC_CTRL. However, no real hardware will exist with SSBD but
+ # not IBRSB, and we pass this MSR directly to guests. Treating them
+ # as dependent features simplifies Xen's logic, and prevents the guest
+ # from seeing implausible configurations.
+ IBRSB: [STIBP, SSBD],
}
deep_features = tuple(sorted(deps.keys()))
--
2.1.4
[-- Attachment #59: xsa263-4.9/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 3307 bytes --]
From 76da4cf6ec259ffa43b83af75e19cdf289e5731e Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr(), and offer SSBD to guests by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/domctl.c | 3 ++-
xen/arch/x86/hvm/hvm.c | 3 ++-
xen/arch/x86/traps.c | 3 ++-
xen/include/public/arch-x86/cpufeatureset.h | 2 +-
4 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 1874949..47b8835 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1408,7 +1408,8 @@ long arch_do_domctl(
* ignored) when STIBP isn't enumerated in hardware.
*/
- if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr.value & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (d->arch.cpuid->feat.ssbd ? SPEC_CTRL_SSBD : 0)) )
break;
v->arch.spec_ctrl = msr.value;
continue;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 1a47ed9..de47c20 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3619,7 +3619,8 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,
* when STIBP isn't enumerated in hardware.
*/
- if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( msr_content & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (d->arch.cpuid->feat.ssbd ? SPEC_CTRL_SSBD : 0)) )
goto gp_fault; /* Rsvd bit set? */
v->arch.spec_ctrl = msr_content;
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index b6add03..93b909c 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2858,7 +2858,8 @@ static int priv_op_write_msr(unsigned int reg, uint64_t val,
* when STIBP isn't enumerated in hardware.
*/
- if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (currd->arch.cpuid->feat.ssbd ? SPEC_CTRL_SSBD : 0)) )
break; /* Rsvd bit set? */
curr->arch.spec_ctrl = val;
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 43f42b6..f2baea4 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -244,7 +244,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
-XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
+XEN_CPUFEATURE(SSBD, 9*32+31) /*A MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
--
2.1.4
[-- Attachment #60: xsa263-4.10/0001-x86-spec_ctrl-Read-MSR_ARCH_CAPABILITIES-only-once.patch --]
[-- Type: application/octet-stream, Size: 3871 bytes --]
From 13fafdf5c97d3bc2a8851c4d1796feac0f82d498 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 12:21:00 +0100
Subject: [PATCH] x86/spec_ctrl: Read MSR_ARCH_CAPABILITIES only once
Make it available from the beginning of init_speculation_mitigations(), and
pass it into appropriate functions. Fix an RSBA typo while moving the
affected comment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d6c65187252a6c1810fd24c4d46f812840de8d3c)
---
xen/arch/x86/spec_ctrl.c | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index fa67a0f..dc90743 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -81,18 +81,15 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
-static void __init print_details(enum ind_thunk thunk)
+static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
unsigned int _7d0 = 0, e8b = 0, tmp;
- uint64_t caps = 0;
/* Collect diagnostics about available mitigations. */
if ( boot_cpu_data.cpuid_level >= 7 )
cpuid_count(7, 0, &tmp, &tmp, &tmp, &_7d0);
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- if ( _7d0 & cpufeat_mask(X86_FEATURE_ARCH_CAPS) )
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
@@ -125,7 +122,7 @@ static void __init print_details(enum ind_thunk thunk)
}
/* Calculate whether Retpoline is known-safe on this CPU. */
-static bool __init retpoline_safe(void)
+static bool __init retpoline_safe(uint64_t caps)
{
unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
@@ -136,19 +133,12 @@ static bool __init retpoline_safe(void)
boot_cpu_data.x86 != 6 )
return false;
- if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
- {
- uint64_t caps;
-
- rdmsrl(MSR_ARCH_CAPABILITIES, caps);
-
- /*
- * RBSA may be set by a hypervisor to indicate that we may move to a
- * processor which isn't retpoline-safe.
- */
- if ( caps & ARCH_CAPS_RSBA )
- return false;
- }
+ /*
+ * RSBA may be set by a hypervisor to indicate that we may move to a
+ * processor which isn't retpoline-safe.
+ */
+ if ( caps & ARCH_CAPS_RSBA )
+ return false;
switch ( boot_cpu_data.x86_model )
{
@@ -218,6 +208,10 @@ void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
bool ibrs = false;
+ uint64_t caps = 0;
+
+ if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
+ rdmsrl(MSR_ARCH_CAPABILITIES, caps);
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -246,7 +240,7 @@ void __init init_speculation_mitigations(void)
* On Intel hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe() )
+ else if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = true;
@@ -331,7 +325,7 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_bti_ist_info has been calculated. */
init_shadow_spec_ctrl_state();
- print_details(thunk);
+ print_details(thunk, caps);
}
static void __init __maybe_unused build_assertions(void)
--
2.1.4
[-- Attachment #61: xsa263-4.10/0002-x86-spec_ctrl-Express-Xen-s-choice-of-MSR_SPEC_CTRL-.patch --]
[-- Type: application/octet-stream, Size: 5070 bytes --]
From d7b345e4ca136a995bfaaf2ee20901ee20e63570 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Express Xen's choice of MSR_SPEC_CTRL value as
a variable
At the moment, we have two different encodings of Xen's MSR_SPEC_CTRL value,
which is a side effect of how the Spectre series developed. One encoding is
via an alias with the bottom bit of bti_ist_info, and can encode IBRS or not,
but not other configurations such as STIBP.
Break Xen's value out into a separate variable (in the top of stack block for
XPTI reasons) and use this instead of bti_ist_info in the IST path.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 66dfae0f32bfbc899c2f3446d5ee57068cb7f957)
---
xen/arch/x86/spec_ctrl.c | 8 +++++---
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/include/asm-x86/current.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 2 ++
xen/include/asm-x86/spec_ctrl_asm.h | 8 ++------
5 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index dc90743..1143521 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,7 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
+uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_bti_ist_info;
static int __init parse_bti(const char *s)
@@ -285,11 +286,14 @@ void __init init_speculation_mitigations(void)
* guests.
*/
if ( ibrs )
+ {
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
+ }
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR | ibrs;
+ default_bti_ist_info |= BTI_IST_WRMSR;
}
/*
@@ -330,8 +334,6 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
- /* The optimised assembly relies on this alias. */
- BUILD_BUG_ON(BTI_IST_IBRS != SPEC_CTRL_IBRS);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 13478d4..0726147 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -142,6 +142,7 @@ void __dummy__(void)
OFFSET(CPUINFO_xen_cr3, struct cpu_info, xen_cr3);
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
+ OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index 4678a0f..d10b13c 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -56,6 +56,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
+ uint8_t xen_spec_ctrl;
bool use_shadow_spec_ctrl;
uint8_t bti_ist_info;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5ab4ff3..5e4fc84 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_bti_ist_info;
static inline void init_shadow_spec_ctrl_state(void)
@@ -34,6 +35,7 @@ static inline void init_shadow_spec_ctrl_state(void)
struct cpu_info *info = get_cpu_info();
info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->xen_spec_ctrl = default_xen_spec_ctrl;
info->bti_ist_info = default_bti_ist_info;
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 1f2b6f3..697da13 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -21,7 +21,6 @@
#define __X86_SPEC_CTRL_ASM_H__
/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_IBRS (1 << 0)
#define BTI_IST_WRMSR (1 << 1)
#define BTI_IST_RSB (1 << 2)
@@ -286,12 +285,9 @@
setz %dl
and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
- /*
- * Load Xen's intended value. SPEC_CTRL_IBRS vs 0 is encoded in the
- * bottom bit of bti_ist_info, via a deliberate alias with BTI_IST_IBRS.
- */
+ /* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
- and $BTI_IST_IBRS, %eax
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
xor %edx, %edx
wrmsr
--
2.1.4
[-- Attachment #62: xsa263-4.10/0003-x86-spec_ctrl-Merge-bti_ist_info-and-use_shadow_spec.patch --]
[-- Type: application/octet-stream, Size: 13293 bytes --]
From a0c2f734b4c683cb407e10ff943671c413480287 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Merge bti_ist_info and use_shadow_spec_ctrl
into spec_ctrl_flags
All 3 bits of information here are control flags for the entry/exit code
behaviour. Treat them as such, rather than having two different variables.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 5262ba2e7799001402dfe139ff944e035dfff928)
---
xen/arch/x86/acpi/power.c | 4 +--
xen/arch/x86/spec_ctrl.c | 10 ++++---
xen/arch/x86/x86_64/asm-offsets.c | 3 +--
xen/include/asm-x86/current.h | 3 +--
xen/include/asm-x86/nops.h | 5 ++--
xen/include/asm-x86/spec_ctrl.h | 10 +++----
xen/include/asm-x86/spec_ctrl_asm.h | 52 ++++++++++++++++++++-----------------
7 files changed, 45 insertions(+), 42 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index f7085d3..f3480aa 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -215,7 +215,7 @@ static int enter_state(u32 state)
ci = get_cpu_info();
spec_ctrl_enter_idle(ci);
/* Avoid NMI/#MC using MSR_SPEC_CTRL until we've reloaded microcode. */
- ci->bti_ist_info = 0;
+ ci->spec_ctrl_flags &= ~SCF_ist_wrmsr;
ACPI_FLUSH_CPU_CACHE();
@@ -256,7 +256,7 @@ static int enter_state(u32 state)
microcode_resume_cpu(0);
/* Re-enabled default NMI/#MC use of MSR_SPEC_CTRL. */
- ci->bti_ist_info = default_bti_ist_info;
+ ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
spec_ctrl_exit_idle(ci);
done:
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1143521..2d69910 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -39,7 +39,7 @@ static bool __initdata opt_rsb_native = true;
static bool __initdata opt_rsb_vmexit = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
-uint8_t __read_mostly default_bti_ist_info;
+uint8_t __read_mostly default_spec_ctrl_flags;
static int __init parse_bti(const char *s)
{
@@ -293,7 +293,7 @@ void __init init_speculation_mitigations(void)
else
setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
- default_bti_ist_info |= BTI_IST_WRMSR;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
/*
@@ -312,7 +312,7 @@ void __init init_speculation_mitigations(void)
if ( opt_rsb_native )
{
setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
- default_bti_ist_info |= BTI_IST_RSB;
+ default_spec_ctrl_flags |= SCF_ist_rsb;
}
/*
@@ -326,7 +326,7 @@ void __init init_speculation_mitigations(void)
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
opt_ibpb = false;
- /* (Re)init BSP state now that default_bti_ist_info has been calculated. */
+ /* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
print_details(thunk, caps);
@@ -334,6 +334,8 @@ void __init init_speculation_mitigations(void)
static void __init __maybe_unused build_assertions(void)
{
+ /* The optimised assembly relies on this alias. */
+ BUILD_BUG_ON(SCF_use_shadow != 1);
}
/*
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 0726147..97242e5 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -143,8 +143,7 @@ void __dummy__(void)
OFFSET(CPUINFO_pv_cr3, struct cpu_info, pv_cr3);
OFFSET(CPUINFO_shadow_spec_ctrl, struct cpu_info, shadow_spec_ctrl);
OFFSET(CPUINFO_xen_spec_ctrl, struct cpu_info, xen_spec_ctrl);
- OFFSET(CPUINFO_use_shadow_spec_ctrl, struct cpu_info, use_shadow_spec_ctrl);
- OFFSET(CPUINFO_bti_ist_info, struct cpu_info, bti_ist_info);
+ OFFSET(CPUINFO_spec_ctrl_flags, struct cpu_info, spec_ctrl_flags);
DEFINE(CPUINFO_sizeof, sizeof(struct cpu_info));
BLANK();
diff --git a/xen/include/asm-x86/current.h b/xen/include/asm-x86/current.h
index d10b13c..7afff0e 100644
--- a/xen/include/asm-x86/current.h
+++ b/xen/include/asm-x86/current.h
@@ -57,8 +57,7 @@ struct cpu_info {
/* See asm-x86/spec_ctrl_asm.h for usage. */
unsigned int shadow_spec_ctrl;
uint8_t xen_spec_ctrl;
- bool use_shadow_spec_ctrl;
- uint8_t bti_ist_info;
+ uint8_t spec_ctrl_flags;
unsigned long __pad;
/* get_stack_bottom() must be 16-byte aligned */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index 37f9819..b744895 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -62,10 +62,9 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP21 ASM_NOP8; ASM_NOP8; ASM_NOP5
+#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
-#define ASM_NOP29 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP5
-#define ASM_NOP32 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5e4fc84..059e291 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -28,15 +28,15 @@ void init_speculation_mitigations(void);
extern bool opt_ibpb;
extern uint8_t default_xen_spec_ctrl;
-extern uint8_t default_bti_ist_info;
+extern uint8_t default_spec_ctrl_flags;
static inline void init_shadow_spec_ctrl_state(void)
{
struct cpu_info *info = get_cpu_info();
- info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+ info->shadow_spec_ctrl = 0;
info->xen_spec_ctrl = default_xen_spec_ctrl;
- info->bti_ist_info = default_bti_ist_info;
+ info->spec_ctrl_flags = default_spec_ctrl_flags;
}
/* WARNING! `ret`, `call *`, `jmp *` not safe after this call. */
@@ -50,7 +50,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
*/
info->shadow_spec_ctrl = val;
barrier();
- info->use_shadow_spec_ctrl = true;
+ info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
@@ -65,7 +65,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
* Disable shadowing before updating the MSR. There are no SMP issues
* here; only local processor ordering concerns.
*/
- info->use_shadow_spec_ctrl = false;
+ info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 697da13..39fb4f8 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -20,9 +20,10 @@
#ifndef __X86_SPEC_CTRL_ASM_H__
#define __X86_SPEC_CTRL_ASM_H__
-/* Encoding of the bottom bits in cpuinfo.bti_ist_info */
-#define BTI_IST_WRMSR (1 << 1)
-#define BTI_IST_RSB (1 << 2)
+/* Encoding of cpuinfo.spec_ctrl_flags */
+#define SCF_use_shadow (1 << 0)
+#define SCF_ist_wrmsr (1 << 1)
+#define SCF_ist_rsb (1 << 2)
#ifdef __ASSEMBLY__
#include <asm/msr-index.h>
@@ -49,20 +50,20 @@
* after VMEXIT. The VMEXIT-specific code reads MSR_SPEC_CTRL and updates
* current before loading Xen's MSR_SPEC_CTRL setting.
*
- * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and
- * use_shadow_spec_ctrl boolean per cpu. The synchronous use is:
+ * Factor 2 is harder. We maintain a shadow_spec_ctrl value, and a use_shadow
+ * boolean in the per cpu spec_ctrl_flags. The synchronous use is:
*
* 1) Store guest value in shadow_spec_ctrl
- * 2) Set use_shadow_spec_ctrl boolean
+ * 2) Set the use_shadow boolean
* 3) Load guest value into MSR_SPEC_CTRL
* 4) Exit to guest
* 5) Entry from guest
- * 6) Clear use_shadow_spec_ctrl boolean
+ * 6) Clear the use_shadow boolean
* 7) Load Xen's value into MSR_SPEC_CTRL
*
* The asynchronous use for interrupts/exceptions is:
* - Set/clear IBRS on entry to Xen
- * - On exit to Xen, check use_shadow_spec_ctrl
+ * - On exit to Xen, check use_shadow
* - If set, load shadow_spec_ctrl
*
* Therefore, an interrupt/exception which hits the synchronous path between
@@ -134,7 +135,7 @@
xor %edx, %edx
/* Clear SPEC_CTRL shadowing *before* loading Xen's value. */
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
mov $\ibrs_val, %eax
@@ -160,12 +161,14 @@
* block so calculate the position directly.
*/
.if \maybexen
+ xor %eax, %eax
/* Branchless `if ( !xen ) clear_shadowing` */
testb $3, UREGS_cs(%rsp)
- setz %al
- and %al, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %al
+ not %eax
+ and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
.else
- movb %dl, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
.endif
/* Load Xen's intended value. */
@@ -184,8 +187,8 @@
*/
xor %edx, %edx
- cmpb %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%rbx)
- je .L\@_skip
+ testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
+ jz .L\@_skip
mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax
mov $MSR_SPEC_CTRL, %ecx
@@ -206,7 +209,7 @@
mov %eax, CPUINFO_shadow_spec_ctrl(%rsp)
/* Set SPEC_CTRL shadowing *before* loading the guest value. */
- movb $1, CPUINFO_use_shadow_spec_ctrl(%rsp)
+ orb $SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
mov $MSR_SPEC_CTRL, %ecx
xor %edx, %edx
@@ -217,7 +220,7 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP32), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -229,7 +232,7 @@
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP21), \
+ ALTERNATIVE_2 __stringify(ASM_NOP22), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -240,7 +243,7 @@
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP29), \
+ ALTERNATIVE_2 __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
ibrs_val=SPEC_CTRL_IBRS), \
X86_FEATURE_XEN_IBRS_SET, \
@@ -268,22 +271,23 @@
* This is logical merge of DO_OVERWRITE_RSB and DO_SPEC_CTRL_ENTRY
* maybexen=1, but with conditionals rather than alternatives.
*/
- movzbl STACK_CPUINFO_FIELD(bti_ist_info)(%r14), %eax
+ movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %eax
- testb $BTI_IST_RSB, %al
+ test $SCF_ist_rsb, %al
jz .L\@_skip_rsb
DO_OVERWRITE_RSB tmp=rdx /* Clobbers %rcx/%rdx */
.L\@_skip_rsb:
- testb $BTI_IST_WRMSR, %al
+ test $SCF_ist_wrmsr, %al
jz .L\@_skip_wrmsr
xor %edx, %edx
testb $3, UREGS_cs(%rsp)
- setz %dl
- and %dl, STACK_CPUINFO_FIELD(use_shadow_spec_ctrl)(%r14)
+ setnz %dl
+ not %edx
+ and %dl, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
/* Load Xen's intended value. */
mov $MSR_SPEC_CTRL, %ecx
@@ -310,7 +314,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
* Requires %rbx=stack_end
* Clobbers %rax, %rcx, %rdx
*/
- testb $BTI_IST_WRMSR, STACK_CPUINFO_FIELD(bti_ist_info)(%rbx)
+ testb $SCF_ist_wrmsr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx)
jz .L\@_skip
DO_SPEC_CTRL_EXIT_TO_XEN
--
2.1.4
[-- Attachment #63: xsa263-4.10/0004-x86-spec_ctrl-Fold-the-XEN_IBRS_-SET-CLEAR-ALTERNATI.patch --]
[-- Type: application/octet-stream, Size: 10505 bytes --]
From 0b1aded85866f48cdede20c54d30cf593f8a83f7 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Fold the XEN_IBRS_{SET,CLEAR} ALTERNATIVES
together
Currently, the SPEC_CTRL_{ENTRY,EXIT}_* macros encode Xen's choice of
MSR_SPEC_CTRL as an immediate constant, and chooses between IBRS or not by
doubling up the entire alternative block.
There is now a variable holding Xen's choice of value, so use that and
simplify the alternatives.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit af949407eaba7af71067f23d5866cd0bf1f1144d)
---
xen/arch/x86/spec_ctrl.c | 12 +++++-----
xen/include/asm-x86/cpufeatures.h | 3 +--
xen/include/asm-x86/nops.h | 3 ++-
xen/include/asm-x86/spec_ctrl.h | 6 ++---
xen/include/asm-x86/spec_ctrl_asm.h | 45 +++++++++++++------------------------
5 files changed, 26 insertions(+), 43 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 2d69910..b62cfcc 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,8 +112,9 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_XEN_IBRS_SET) ? " IBRS+" :
- boot_cpu_has(X86_FEATURE_XEN_IBRS_CLEAR) ? " IBRS-" : "",
+ boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
+ " IBRS-" : "",
opt_ibpb ? " IBPB" : "",
boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
@@ -285,13 +286,10 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+
if ( ibrs )
- {
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_SET);
- }
- else
- setup_force_cpu_cap(X86_FEATURE_XEN_IBRS_CLEAR);
default_spec_ctrl_flags |= SCF_ist_wrmsr;
}
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index c9b1a48..ca58b0e 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -26,8 +26,7 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+12) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+13) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+14) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+15) /* IBRSB || IBPB */
-XEN_CPUFEATURE(XEN_IBRS_SET, (FSCAPINTS+0)*32+16) /* IBRSB && IRBS set in Xen */
-XEN_CPUFEATURE(XEN_IBRS_CLEAR, (FSCAPINTS+0)*32+17) /* IBRSB && IBRS clear in Xen */
+XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+16) /* MSR_SPEC_CTRL used by Xen */
XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+18) /* RSB overwrite needed for native */
XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+19) /* RSB overwrite needed for vmexit */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+20) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/nops.h b/xen/include/asm-x86/nops.h
index b744895..913e9f0 100644
--- a/xen/include/asm-x86/nops.h
+++ b/xen/include/asm-x86/nops.h
@@ -62,9 +62,10 @@
#define ASM_NOP8 _ASM_MK_NOP(K8_NOP8)
#define ASM_NOP17 ASM_NOP8; ASM_NOP7; ASM_NOP2
-#define ASM_NOP22 ASM_NOP8; ASM_NOP8; ASM_NOP6
#define ASM_NOP24 ASM_NOP8; ASM_NOP8; ASM_NOP8
+#define ASM_NOP25 ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
#define ASM_NOP33 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP7; ASM_NOP2
+#define ASM_NOP36 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP4
#define ASM_NOP40 ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8; ASM_NOP8
#define ASM_NOP_MAX 8
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 059e291..7d7c42e 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,14 +52,14 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
/* WARNING! `ret`, `call *`, `jmp *` not safe before this call. */
static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
{
- uint32_t val = SPEC_CTRL_IBRS;
+ uint32_t val = info->xen_spec_ctrl;
/*
* Disable shadowing before updating the MSR. There are no SMP issues
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 39fb4f8..17dd2cc 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -117,7 +117,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -138,11 +138,11 @@
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
/* Load Xen's intended value. */
- mov $\ibrs_val, %eax
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
wrmsr
.endm
-.macro DO_SPEC_CTRL_ENTRY maybexen:req ibrs_val:req
+.macro DO_SPEC_CTRL_ENTRY maybexen:req
/*
* Requires %rsp=regs (also cpuinfo if !maybexen)
* Requires %r14=stack_end (if maybexen)
@@ -167,12 +167,12 @@
setnz %al
not %eax
and %al, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14)
+ movzbl STACK_CPUINFO_FIELD(xen_spec_ctrl)(%r14), %eax
.else
andb $~SCF_use_shadow, CPUINFO_spec_ctrl_flags(%rsp)
+ movzbl CPUINFO_xen_spec_ctrl(%rsp), %eax
.endif
- /* Load Xen's intended value. */
- mov $\ibrs_val, %eax
wrmsr
.endm
@@ -220,47 +220,32 @@
#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY_FROM_VMEXIT \
- ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP36), \
+ DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP22), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP25), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
- ALTERNATIVE_2 __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 \
- ibrs_val=SPEC_CTRL_IBRS), \
- X86_FEATURE_XEN_IBRS_SET, \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1 ibrs_val=0), \
- X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP33), \
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
- ALTERNATIVE_2 __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP17), \
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
/* Use when exiting to guest context. */
#define SPEC_CTRL_EXIT_TO_GUEST \
- ALTERNATIVE_2 __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_SET, \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_XEN_IBRS_CLEAR
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
--
2.1.4
[-- Attachment #64: xsa263-4.10/0005-x86-spec_ctrl-Rename-bits-of-infrastructure-to-avoid.patch --]
[-- Type: application/octet-stream, Size: 12279 bytes --]
From 5cc3611de7d09140e55caa2c2d120ad326fff937 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 30 Apr 2018 14:20:23 +0100
Subject: [PATCH] x86/spec_ctrl: Rename bits of infrastructure to avoid NATIVE
and VMEXIT
In hindsight, using NATIVE and VMEXIT as naming terminology was not clever.
A future change wants to split SPEC_CTRL_EXIT_TO_GUEST into PV and HVM
specific implementations, and using VMEXIT as a term is completely wrong.
Take the opportunity to fix some stale documentation in spec_ctrl_asm.h. The
IST helpers were missing from the large comment block, and since
SPEC_CTRL_ENTRY_FROM_INTR_IST was introduced, we've gained a new piece of
functionality which currently depends on the fine grain control, which exists
in lieu of livepatching. Note this in the comment.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit d9822b8a38114e96e4516dc998f4055249364d5d)
---
xen/arch/x86/hvm/svm/entry.S | 4 ++--
xen/arch/x86/hvm/vmx/entry.S | 4 ++--
xen/arch/x86/spec_ctrl.c | 20 ++++++++++----------
xen/arch/x86/x86_64/compat/entry.S | 2 +-
xen/arch/x86/x86_64/entry.S | 2 +-
xen/include/asm-x86/cpufeatures.h | 4 ++--
xen/include/asm-x86/spec_ctrl_asm.h | 36 +++++++++++++++++++++++++-----------
7 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S
index bf092fe..5e7c080 100644
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -83,7 +83,7 @@ UNLIKELY_END(svm_trace)
mov VCPUMSR_spec_ctrl_raw(%rax), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
pop %r15
pop %r14
@@ -108,7 +108,7 @@ UNLIKELY_END(svm_trace)
GET_CURRENT(bx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov VCPU_svm_vmcb(%rbx),%rcx
diff --git a/xen/arch/x86/hvm/vmx/entry.S b/xen/arch/x86/hvm/vmx/entry.S
index e750544..aa2f103 100644
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -38,7 +38,7 @@ ENTRY(vmx_asm_vmexit_handler)
movb $1,VCPU_vmx_launched(%rbx)
mov %rax,VCPU_hvm_guest_cr2(%rbx)
- SPEC_CTRL_ENTRY_FROM_VMEXIT /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
+ SPEC_CTRL_ENTRY_FROM_HVM /* Req: b=curr %rsp=regs/cpuinfo, Clob: acd */
/* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
mov %rsp,%rdi
@@ -76,7 +76,7 @@ UNLIKELY_END(realmode)
mov VCPUMSR_spec_ctrl_raw(%rax), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_HVM /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
mov VCPU_hvm_guest_cr2(%rbx),%rax
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index b62cfcc..015a9e2 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -35,8 +35,8 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_native = true;
-static bool __initdata opt_rsb_vmexit = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -69,9 +69,9 @@ static int __init parse_bti(const char *s)
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
else if ( (val = parse_boolean("rsb_native", s, ss)) >= 0 )
- opt_rsb_native = val;
+ opt_rsb_pv = val;
else if ( (val = parse_boolean("rsb_vmexit", s, ss)) >= 0 )
- opt_rsb_vmexit = val;
+ opt_rsb_hvm = val;
else
rc = -EINVAL;
@@ -116,8 +116,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_RSB_NATIVE) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_RSB_VMEXIT) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -307,9 +307,9 @@ void __init init_speculation_mitigations(void)
* If a processors speculates to 32bit PV guest kernel mappings, it is
* speculating in 64bit supervisor mode, and can leak data.
*/
- if ( opt_rsb_native )
+ if ( opt_rsb_pv )
{
- setup_force_cpu_cap(X86_FEATURE_RSB_NATIVE);
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_PV);
default_spec_ctrl_flags |= SCF_ist_rsb;
}
@@ -317,8 +317,8 @@ void __init init_speculation_mitigations(void)
* HVM guests can always poison the RSB to point at Xen supervisor
* mappings.
*/
- if ( opt_rsb_vmexit )
- setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
+ if ( opt_rsb_hvm )
+ setup_force_cpu_cap(X86_FEATURE_SC_RSB_HVM);
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index a47cb9d..6a27d98 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -166,7 +166,7 @@ ENTRY(compat_restore_all_guest)
mov VCPUMSR_spec_ctrl_raw(%rax), %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL adj=8 compat=1
.Lft0: iretq
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 41d3ec2..0a0763a 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -196,7 +196,7 @@ restore_all_guest:
mov %r15d, %eax
/* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */
- SPEC_CTRL_EXIT_TO_GUEST /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
+ SPEC_CTRL_EXIT_TO_PV /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
RESTORE_ALL
testw $TRAP_syscall,4(%rsp)
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index ca58b0e..f9aa5d7 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -27,6 +27,6 @@ XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+13) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+14) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+15) /* IBRSB || IBPB */
XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+16) /* MSR_SPEC_CTRL used by Xen */
-XEN_CPUFEATURE(RSB_NATIVE, (FSCAPINTS+0)*32+18) /* RSB overwrite needed for native */
-XEN_CPUFEATURE(RSB_VMEXIT, (FSCAPINTS+0)*32+19) /* RSB overwrite needed for vmexit */
+XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+18) /* RSB overwrite needed for PV */
+XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+19) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+20) /* XPTI mitigation not in use */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 17dd2cc..3d156ed 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -72,11 +72,14 @@
*
* The following ASM fragments implement this algorithm. See their local
* comments for further details.
- * - SPEC_CTRL_ENTRY_FROM_VMEXIT
+ * - SPEC_CTRL_ENTRY_FROM_HVM
* - SPEC_CTRL_ENTRY_FROM_PV
* - SPEC_CTRL_ENTRY_FROM_INTR
+ * - SPEC_CTRL_ENTRY_FROM_INTR_IST
+ * - SPEC_CTRL_EXIT_TO_XEN_IST
* - SPEC_CTRL_EXIT_TO_XEN
- * - SPEC_CTRL_EXIT_TO_GUEST
+ * - SPEC_CTRL_EXIT_TO_PV
+ * - SPEC_CTRL_EXIT_TO_HVM
*/
.macro DO_OVERWRITE_RSB tmp=rax
@@ -117,7 +120,7 @@
mov %\tmp, %rsp /* Restore old %rsp */
.endm
-.macro DO_SPEC_CTRL_ENTRY_FROM_VMEXIT
+.macro DO_SPEC_CTRL_ENTRY_FROM_HVM
/*
* Requires %rbx=current, %rsp=regs/cpuinfo
* Clobbers %rax, %rcx, %rdx
@@ -217,23 +220,23 @@
.endm
/* Use after a VMEXIT from an HVM guest. */
-#define SPEC_CTRL_ENTRY_FROM_VMEXIT \
+#define SPEC_CTRL_ENTRY_FROM_HVM \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_VMEXIT; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_VMEXIT, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
- DO_OVERWRITE_RSB, X86_FEATURE_RSB_NATIVE; \
+ DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
__stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
@@ -242,12 +245,22 @@
ALTERNATIVE __stringify(ASM_NOP17), \
DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
-/* Use when exiting to guest context. */
-#define SPEC_CTRL_EXIT_TO_GUEST \
+/* Use when exiting to PV guest context. */
+#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
-/* TODO: Drop these when the alternatives infrastructure is NMI/#MC safe. */
+/* Use when exiting to HVM guest context. */
+#define SPEC_CTRL_EXIT_TO_HVM \
+ ALTERNATIVE __stringify(ASM_NOP24), \
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+
+/*
+ * Use in IST interrupt/exception context. May interrupt Xen or PV context.
+ * Fine grain control of SCF_ist_wrmsr is needed for safety in the S3 resume
+ * path to avoid using MSR_SPEC_CTRL before the microcode introducing it has
+ * been reloaded.
+ */
.macro SPEC_CTRL_ENTRY_FROM_INTR_IST
/*
* Requires %rsp=regs, %r14=stack_end
@@ -294,6 +307,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise):
UNLIKELY_END(\@_serialise)
.endm
+/* Use when exiting to Xen in IST context. */
.macro SPEC_CTRL_EXIT_TO_XEN_IST
/*
* Requires %rbx=stack_end
--
2.1.4
[-- Attachment #65: xsa263-4.10/0006-x86-spec_ctrl-Elide-MSR_SPEC_CTRL-handling-in-idle-c.patch --]
[-- Type: application/octet-stream, Size: 3083 bytes --]
From 811fcf5137abdcd5b9ea7e5212098adb5bedae0f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 May 2018 14:06:16 +0100
Subject: [PATCH] x86/spec_ctrl: Elide MSR_SPEC_CTRL handling in idle context
when possible
If Xen is virtualising MSR_SPEC_CTRL handling for guests, but using 0 as its
own MSR_SPEC_CTRL value, spec_ctrl_{enter,exit}_idle() need not write to the
MSR.
Requested-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 94df6e8588e35cc2028ccb3fd2921c6e6360605e)
---
xen/arch/x86/spec_ctrl.c | 4 ++++
xen/include/asm-x86/cpufeatures.h | 1 +
xen/include/asm-x86/spec_ctrl.h | 4 ++--
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 015a9e2..55ef79f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -327,6 +327,10 @@ void __init init_speculation_mitigations(void)
/* (Re)init BSP state now that default_spec_ctrl_flags has been calculated. */
init_shadow_spec_ctrl_state();
+ /* If Xen is using any MSR_SPEC_CTRL settings, adjust the idle path. */
+ if ( default_xen_spec_ctrl )
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
+
print_details(thunk, caps);
}
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index f9aa5d7..32b7f04 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -30,3 +30,4 @@ XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+16) /* MSR_SPEC_CTRL used by Xe
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+18) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+19) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+20) /* XPTI mitigation not in use */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+21) /* SC_MSR && default_xen_spec_ctrl */
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 7d7c42e..77f92ba 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -52,7 +52,7 @@ static always_inline void spec_ctrl_enter_idle(struct cpu_info *info)
barrier();
info->spec_ctrl_flags |= SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
@@ -67,7 +67,7 @@ static always_inline void spec_ctrl_exit_idle(struct cpu_info *info)
*/
info->spec_ctrl_flags &= ~SCF_use_shadow;
barrier();
- asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR)
+ asm volatile ( ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_SC_MSR_IDLE)
:: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory" );
}
--
2.1.4
[-- Attachment #66: xsa263-4.10/0007-x86-spec_ctrl-Split-X86_FEATURE_SC_MSR-into-PV-and-H.patch --]
[-- Type: application/octet-stream, Size: 6034 bytes --]
From 2acc4cba7eb2559bafdd4d8238466ad81322a35a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 17 Apr 2018 14:15:04 +0100
Subject: [PATCH] x86/spec_ctrl: Split X86_FEATURE_SC_MSR into PV and HVM
variants
In order to separately control whether MSR_SPEC_CTRL is virtualised for PV and
HVM guests, split the feature used to control runtime alternatives into two.
Xen will use MSR_SPEC_CTRL itself if either of these features are active.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit fa9eb09d446a1279f5e861e6b84fa8675dabf148)
---
xen/arch/x86/spec_ctrl.c | 6 ++++--
xen/include/asm-x86/cpufeatures.h | 5 +++--
xen/include/asm-x86/spec_ctrl_asm.h | 12 ++++++------
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 55ef79f..a940308 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -112,7 +112,8 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- boot_cpu_has(X86_FEATURE_SC_MSR) ?
+ (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
" IBRS-" : "",
opt_ibpb ? " IBPB" : "",
@@ -286,7 +287,8 @@ void __init init_speculation_mitigations(void)
* need the IBRS entry/exit logic to virtualise IBRS support for
* guests.
*/
- setup_force_cpu_cap(X86_FEATURE_SC_MSR);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
if ( ibrs )
default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
diff --git a/xen/include/asm-x86/cpufeatures.h b/xen/include/asm-x86/cpufeatures.h
index 32b7f04..b90aa2d 100644
--- a/xen/include/asm-x86/cpufeatures.h
+++ b/xen/include/asm-x86/cpufeatures.h
@@ -26,8 +26,9 @@ XEN_CPUFEATURE(LFENCE_DISPATCH, (FSCAPINTS+0)*32+12) /* lfence set as Dispatch S
XEN_CPUFEATURE(IND_THUNK_LFENCE,(FSCAPINTS+0)*32+13) /* Use IND_THUNK_LFENCE */
XEN_CPUFEATURE(IND_THUNK_JMP, (FSCAPINTS+0)*32+14) /* Use IND_THUNK_JMP */
XEN_CPUFEATURE(XEN_IBPB, (FSCAPINTS+0)*32+15) /* IBRSB || IBPB */
-XEN_CPUFEATURE(SC_MSR, (FSCAPINTS+0)*32+16) /* MSR_SPEC_CTRL used by Xen */
+XEN_CPUFEATURE(SC_MSR_PV, (FSCAPINTS+0)*32+16) /* MSR_SPEC_CTRL used by Xen for PV */
+XEN_CPUFEATURE(SC_MSR_HVM, (FSCAPINTS+0)*32+17) /* MSR_SPEC_CTRL used by Xen for HVM */
XEN_CPUFEATURE(SC_RSB_PV, (FSCAPINTS+0)*32+18) /* RSB overwrite needed for PV */
XEN_CPUFEATURE(SC_RSB_HVM, (FSCAPINTS+0)*32+19) /* RSB overwrite needed for HVM */
XEN_CPUFEATURE(NO_XPTI, (FSCAPINTS+0)*32+20) /* XPTI mitigation not in use */
-XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+21) /* SC_MSR && default_xen_spec_ctrl */
+XEN_CPUFEATURE(SC_MSR_IDLE, (FSCAPINTS+0)*32+21) /* (SC_MSR_PV || SC_MSR_HVM) && default_xen_spec_ctrl */
diff --git a/xen/include/asm-x86/spec_ctrl_asm.h b/xen/include/asm-x86/spec_ctrl_asm.h
index 3d156ed..c659f3f 100644
--- a/xen/include/asm-x86/spec_ctrl_asm.h
+++ b/xen/include/asm-x86/spec_ctrl_asm.h
@@ -224,36 +224,36 @@
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_HVM; \
ALTERNATIVE __stringify(ASM_NOP36), \
- DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_ENTRY_FROM_HVM, X86_FEATURE_SC_MSR_HVM
/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */
#define SPEC_CTRL_ENTRY_FROM_PV \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP25), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), X86_FEATURE_SC_MSR_PV
/* Use in interrupt/exception context. May interrupt Xen or PV context. */
#define SPEC_CTRL_ENTRY_FROM_INTR \
ALTERNATIVE __stringify(ASM_NOP40), \
DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \
ALTERNATIVE __stringify(ASM_NOP33), \
- __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR
+ __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), X86_FEATURE_SC_MSR_PV
/* Use when exiting to Xen context. */
#define SPEC_CTRL_EXIT_TO_XEN \
ALTERNATIVE __stringify(ASM_NOP17), \
- DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV
/* Use when exiting to PV guest context. */
#define SPEC_CTRL_EXIT_TO_PV \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV
/* Use when exiting to HVM guest context. */
#define SPEC_CTRL_EXIT_TO_HVM \
ALTERNATIVE __stringify(ASM_NOP24), \
- DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR
+ DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_HVM
/*
* Use in IST interrupt/exception context. May interrupt Xen or PV context.
--
2.1.4
[-- Attachment #67: xsa263-4.10/0008-x86-spec_ctrl-Explicitly-set-Xen-s-default-MSR_SPEC_.patch --]
[-- Type: application/octet-stream, Size: 4781 bytes --]
From 5b223f41d59887ea5d13e2406597ff472ba6f2fc Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 9 May 2018 13:59:56 +0100
Subject: [PATCH] x86/spec_ctrl: Explicitly set Xen's default MSR_SPEC_CTRL
value
With the impending ability to disable MSR_SPEC_CTRL handling on a
per-guest-type basis, the first exit-from-guest may not have the side effect
of loading Xen's choice of value. Explicitly set Xen's default during the BSP
and AP boot paths.
For the BSP however, delay setting a non-zero MSR_SPEC_CTRL default until
after dom0 has been constructed when safe to do so. Oracle report that this
speeds up boots of some hardware by 50s.
"when safe to do so" is based on whether we are virtualised. A native boot
won't have any other code running in a position to mount an attack.
Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb8c12020307b39a89273d7699e89000451987ab)
---
xen/arch/x86/setup.c | 7 +++++++
xen/arch/x86/smpboot.c | 8 ++++++++
xen/arch/x86/spec_ctrl.c | 32 ++++++++++++++++++++++++++++++++
xen/include/asm-x86/spec_ctrl.h | 2 ++
4 files changed, 49 insertions(+)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 482fe11..1995c4c 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1746,6 +1746,13 @@ void __init noreturn __start_xen(unsigned long mbi_p)
setup_io_bitmap(dom0);
+ if ( bsp_delay_spec_ctrl )
+ {
+ get_cpu_info()->spec_ctrl_flags &= ~SCF_use_shadow;
+ barrier();
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+ }
+
/* Jump to the 1:1 virtual mappings of cpu0_stack. */
asm volatile ("mov %[stk], %%rsp; jmp %c[fn]" ::
[stk] "g" (__va(__pa(get_stack_bottom()))),
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index f81fc2c..ee8b183 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -351,6 +351,14 @@ void start_secondary(void *unused)
else
microcode_resume_cpu(cpu);
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. Note: MSR_SPEC_CTRL may only become available
+ * after loading microcode.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ wrmsrl(MSR_SPEC_CTRL, default_xen_spec_ctrl);
+
if ( xen_guest )
hypervisor_ap_setup();
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index a940308..3adec1a 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -38,6 +38,8 @@ static int8_t __initdata opt_ibrs = -1;
static bool __initdata opt_rsb_pv = true;
static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
+
+bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
uint8_t __read_mostly default_spec_ctrl_flags;
@@ -334,6 +336,36 @@ void __init init_speculation_mitigations(void)
setup_force_cpu_cap(X86_FEATURE_SC_MSR_IDLE);
print_details(thunk, caps);
+
+ /*
+ * If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
+ * any firmware settings. For performance reasons, when safe to do so, we
+ * delay applying non-zero settings until after dom0 has been constructed.
+ *
+ * "when safe to do so" is based on whether we are virtualised. A native
+ * boot won't have any other code running in a position to mount an
+ * attack.
+ */
+ if ( boot_cpu_has(X86_FEATURE_IBRSB) )
+ {
+ bsp_delay_spec_ctrl = !cpu_has_hypervisor && default_xen_spec_ctrl;
+
+ /*
+ * If delaying MSR_SPEC_CTRL setup, use the same mechanism as
+ * spec_ctrl_enter_idle(), by using a shadow value of zero.
+ */
+ if ( bsp_delay_spec_ctrl )
+ {
+ struct cpu_info *info = get_cpu_info();
+
+ info->shadow_spec_ctrl = 0;
+ barrier();
+ info->spec_ctrl_flags |= SCF_use_shadow;
+ barrier();
+ }
+
+ wrmsrl(MSR_SPEC_CTRL, bsp_delay_spec_ctrl ? 0 : default_xen_spec_ctrl);
+ }
}
static void __init __maybe_unused build_assertions(void)
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 77f92ba..c6a38f4 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,8 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+
+extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
extern uint8_t default_spec_ctrl_flags;
--
2.1.4
[-- Attachment #68: xsa263-4.10/0009-x86-cpuid-Improvements-to-guest-policies-for-specula.patch --]
[-- Type: application/octet-stream, Size: 5186 bytes --]
From bce7a2145abc3c7e5bfd7e2168714d194124a3ab Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Tue, 1 May 2018 11:59:03 +0100
Subject: [PATCH] x86/cpuid: Improvements to guest policies for speculative
sidechannel features
If Xen isn't virtualising MSR_SPEC_CTRL for guests, IBRSB shouldn't be
advertised. It is not currently possible to express this via the existing
command line options, but such an ability will be introduced.
Another useful option in some usecases is to offer IBPB without IBRS. When a
guest kernel is known to be compatible (uses retpoline and knows about the AMD
IBPB feature bit), an administrator with pre-Skylake hardware may wish to hide
IBRS. This allows the VM to have full protection, without Xen or the VM
needing to touch MSR_SPEC_CTRL, which can reduce the overhead of Spectre
mitigations.
Break the logic common to both PV and HVM CPUID calculations into a common
helper, to avoid duplication.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit cb06b308ec71b23f37a44f5e2351fe2cae0306e9)
---
xen/arch/x86/cpuid.c | 60 ++++++++++++++++++++++++++++++++--------------------
1 file changed, 37 insertions(+), 23 deletions(-)
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index b3c9ac6..b45b145 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -368,6 +368,28 @@ static void __init calculate_host_policy(void)
}
}
+static void __init guest_common_feature_adjustments(uint32_t *fs)
+{
+ /* Unconditionally claim to be able to set the hypervisor bit. */
+ __set_bit(X86_FEATURE_HYPERVISOR, fs);
+
+ /*
+ * If IBRS is offered to the guest, unconditionally offer STIBP. It is a
+ * nop on non-HT hardware, and has this behaviour to make heterogeneous
+ * setups easier to manage.
+ */
+ if ( test_bit(X86_FEATURE_IBRSB, fs) )
+ __set_bit(X86_FEATURE_STIBP, fs);
+
+ /*
+ * On hardware which supports IBRS/IBPB, we can offer IBPB independently
+ * of IBRS by using the AMD feature bit. An administrator may wish for
+ * performance reasons to offer IBPB without IBRS.
+ */
+ if ( host_cpuid_policy.feat.ibrsb )
+ __set_bit(X86_FEATURE_IBPB, fs);
+}
+
static void __init calculate_pv_max_policy(void)
{
struct cpuid_policy *p = &pv_max_cpuid_policy;
@@ -380,18 +402,14 @@ static void __init calculate_pv_max_policy(void)
for ( i = 0; i < ARRAY_SIZE(pv_featureset); ++i )
pv_featureset[i] &= pv_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, pv_featureset);
-
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, pv_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, pv_featureset);
+ /*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for PV guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_PV) )
+ __clear_bit(X86_FEATURE_IBRSB, pv_featureset);
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, pv_featureset);
- }
+ guest_common_feature_adjustments(pv_featureset);
sanitise_featureset(pv_featureset);
cpuid_featureset_to_policy(pv_featureset, p);
@@ -419,9 +437,6 @@ static void __init calculate_hvm_max_policy(void)
for ( i = 0; i < ARRAY_SIZE(hvm_featureset); ++i )
hvm_featureset[i] &= hvm_featuremask[i];
- /* Unconditionally claim to be able to set the hypervisor bit. */
- __set_bit(X86_FEATURE_HYPERVISOR, hvm_featureset);
-
/*
* Xen can provide an APIC emulation to HVM guests even if the host's APIC
* isn't enabled.
@@ -438,6 +453,13 @@ static void __init calculate_hvm_max_policy(void)
__set_bit(X86_FEATURE_SEP, hvm_featureset);
/*
+ * If Xen isn't virtualising MSR_SPEC_CTRL for HVM guests because of
+ * administrator choice, hide the feature.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SC_MSR_HVM) )
+ __clear_bit(X86_FEATURE_IBRSB, hvm_featureset);
+
+ /*
* With VT-x, some features are only supported by Xen if dedicated
* hardware support is also available.
*/
@@ -450,15 +472,7 @@ static void __init calculate_hvm_max_policy(void)
__clear_bit(X86_FEATURE_XSAVES, hvm_featureset);
}
- /* On hardware with IBRS/IBPB support, there are further adjustments. */
- if ( test_bit(X86_FEATURE_IBRSB, hvm_featureset) )
- {
- /* Offer STIBP unconditionally. It is a nop on non-HT hardware. */
- __set_bit(X86_FEATURE_STIBP, hvm_featureset);
-
- /* AMD's IBPB is a subset of IBRS/IBPB. */
- __set_bit(X86_FEATURE_IBPB, hvm_featureset);
- }
+ guest_common_feature_adjustments(hvm_featureset);
sanitise_featureset(hvm_featureset);
cpuid_featureset_to_policy(hvm_featureset, p);
--
2.1.4
[-- Attachment #69: xsa263-4.10/0010-x86-spec_ctrl-Introduce-a-new-spec-ctrl-command-line.patch --]
[-- Type: application/octet-stream, Size: 14133 bytes --]
From 952ff9f5590e37952d7dd3d89e16a47a238ab079 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:52:55 +0100
Subject: [PATCH] x86/spec_ctrl: Introduce a new `spec-ctrl=` command line
argument to replace `bti=`
In hindsight, the options for `bti=` aren't as flexible or useful as expected
(including several options which don't appear to behave as intended).
Changing the behaviour of an existing option is problematic for compatibility,
so introduce a new `spec-ctrl=` in the hopes that we can do better.
One common way of deploying Xen is with a single PV dom0 and all domUs being
HVM domains. In such a setup, an administrator who has weighed up the risks
may wish to forgo protection against malicious PV domains, to reduce the
overall performance hit. To cater for this usecase, `spec-ctrl=no-pv` will
disable all speculative protection for PV domains, while leaving all
speculative protection for HVM domains intact.
For coding clarity as much as anything else, the suboptions are grouped by
logical area; those which affect the alternatives blocks, and those which
affect Xen's in-hypervisor settings. See the xen-command-line.markdown for
full details of the new options.
While changing the command line options, take the time to change how the data
is reported to the user. The three DEBUG printks are upgraded to unilateral,
as they are all relevant pieces of information, and the old "mitigations:"
line is split in the two logical areas described above.
Sample output from booting with `spec-ctrl=no-pv` looks like:
(XEN) Speculative mitigation facilities:
(XEN) Hardware features: IBRS/IBPB STIBP IBPB
(XEN) Compiled-in support: INDIRECT_THUNK
(XEN) Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS-, Other: IBPB
(XEN) Support for VMs: PV: None, HVM: MSR_SPEC_CTRL RSB
(XEN) XPTI (64-bit PV only): Dom0 enabled, DomU enabled
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 3352afc26c497d26ecb70527db3cb29daf7b1422)
---
docs/misc/xen-command-line.markdown | 49 +++++++++++
xen/arch/x86/spec_ctrl.c | 160 ++++++++++++++++++++++++++++++------
2 files changed, 186 insertions(+), 23 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 6c673ee..43a6ddb 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -248,6 +248,9 @@ the NMI watchdog is also enabled.
### bti (x86)
> `= List of [ thunk=retpoline|lfence|jmp, ibrs=<bool>, ibpb=<bool>, rsb_{vmexit,native}=<bool> ]`
+**WARNING: This command line option is deprecated, and superseded by
+_spec-ctrl=_ - using both options in combination is undefined.**
+
Branch Target Injection controls. By default, Xen will pick the most
appropriate BTI mitigations based on compiled in support, loaded microcode,
and hardware details.
@@ -1698,6 +1701,52 @@ enforces the maximum theoretically necessary timeout of 670ms. Any number
is being interpreted as a custom timeout in milliseconds. Zero or boolean
false disable the quirk workaround, which is also the default.
+### spec-ctrl (x86)
+> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+
+Controls for speculative execution sidechannel mitigations. By default, Xen
+will pick the most appropriate mitigations based on compiled in support,
+loaded microcode, and hardware details, and will virtualise appropriate
+mitigations for guests to use.
+
+**WARNING: Any use of this option may interfere with heuristics. Use with
+extreme care.**
+
+An overall boolean value, `spec-ctrl=no`, can be specified to turn off all
+mitigations, including pieces of infrastructure used to virtualise certain
+mitigation features for guests. Alternatively, a slightly more restricted
+`spec-ctrl=no-xen` can be used to turn off all of Xen's mitigations, while
+leaving the virtualisation support in place for guests to use. Use of a
+positive boolean value for either of these options is invalid.
+
+The booleans `pv=`, `hvm=`, `msr-sc=` and `rsb=` offer fine grained control
+over the alternative blocks used by Xen. These impact Xen's ability to
+protect itself, and Xen's ability to virtualise support for guests to use.
+
+* `pv=` and `hvm=` offer control over all suboptions for PV and HVM guests
+ respectively.
+* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
+ on entry and exit. These blocks are necessary to virtualise support for
+ guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
+ Return Address Stack on entry to Xen.
+
+If Xen was compiled with INDIRECT\_THUNK support, `bti-thunk=` can be used to
+select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
+locations. The default thunk is `retpoline` (generally preferred for Intel
+hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
+overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+
+On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
+`ibrs=` option can be used to force or prevent Xen using the feature itself.
+If Xen is not using IBRS itself, functionality is still set up so IBRS can be
+virtualised for guests.
+
+On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
+option can be used to force (the default) or prevent Xen from issuing branch
+prediction barriers on vcpu context switches.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 3adec1a..4f9282f 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -26,6 +26,13 @@
#include <asm/spec_ctrl.h>
#include <asm/spec_ctrl_asm.h>
+/* Cmdline controls for Xen's alternative blocks. */
+static bool __initdata opt_msr_sc_pv = true;
+static bool __initdata opt_msr_sc_hvm = true;
+static bool __initdata opt_rsb_pv = true;
+static bool __initdata opt_rsb_hvm = true;
+
+/* Cmdline controls for Xen's speculative settings. */
static enum ind_thunk {
THUNK_DEFAULT, /* Decide which thunk to use at boot time. */
THUNK_NONE, /* Missing compiler support for thunks. */
@@ -35,8 +42,6 @@ static enum ind_thunk {
THUNK_JMP,
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
-static bool __initdata opt_rsb_pv = true;
-static bool __initdata opt_rsb_hvm = true;
bool __read_mostly opt_ibpb = true;
bool __initdata bsp_delay_spec_ctrl;
@@ -84,8 +89,95 @@ static int __init parse_bti(const char *s)
}
custom_param("bti", parse_bti);
+static int __init parse_spec_ctrl(const char *s)
+{
+ const char *ss;
+ int val, rc = 0;
+
+ do {
+ ss = strchr(s, ',');
+ if ( !ss )
+ ss = strchr(s, '\0');
+
+ /* Global and Xen-wide disable. */
+ val = parse_bool(s, ss);
+ if ( !val )
+ {
+ opt_msr_sc_pv = false;
+ opt_msr_sc_hvm = false;
+
+ disable_common:
+ opt_rsb_pv = false;
+ opt_rsb_hvm = false;
+
+ opt_thunk = THUNK_JMP;
+ opt_ibrs = 0;
+ opt_ibpb = false;
+ }
+ else if ( val > 0 )
+ rc = -EINVAL;
+ else if ( (val = parse_boolean("xen", s, ss)) >= 0 )
+ {
+ if ( !val )
+ goto disable_common;
+
+ rc = -EINVAL;
+ }
+
+ /* Xen's alternative blocks. */
+ else if ( (val = parse_boolean("pv", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_rsb_pv = val;
+ }
+ else if ( (val = parse_boolean("hvm", s, ss)) >= 0 )
+ {
+ opt_msr_sc_hvm = val;
+ opt_rsb_hvm = val;
+ }
+ else if ( (val = parse_boolean("msr-sc", s, ss)) >= 0 )
+ {
+ opt_msr_sc_pv = val;
+ opt_msr_sc_hvm = val;
+ }
+ else if ( (val = parse_boolean("rsb", s, ss)) >= 0 )
+ {
+ opt_rsb_pv = val;
+ opt_rsb_hvm = val;
+ }
+
+ /* Xen's speculative sidechannel mitigation settings. */
+ else if ( !strncmp(s, "bti-thunk=", 10) )
+ {
+ s += 10;
+
+ if ( !strncmp(s, "retpoline", ss - s) )
+ opt_thunk = THUNK_RETPOLINE;
+ else if ( !strncmp(s, "lfence", ss - s) )
+ opt_thunk = THUNK_LFENCE;
+ else if ( !strncmp(s, "jmp", ss - s) )
+ opt_thunk = THUNK_JMP;
+ else
+ rc = -EINVAL;
+ }
+ else if ( (val = parse_boolean("ibrs", s, ss)) >= 0 )
+ opt_ibrs = val;
+ else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
+ opt_ibpb = val;
+ else
+ rc = -EINVAL;
+
+ s = ss + 1;
+ } while ( *ss );
+
+ return rc;
+}
+custom_param("spec-ctrl", parse_spec_ctrl);
+
static void __init print_details(enum ind_thunk thunk, uint64_t caps)
{
+ bool use_spec_ctrl = (boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM));
unsigned int _7d0 = 0, e8b = 0, tmp;
/* Collect diagnostics about available mitigations. */
@@ -94,10 +186,10 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
if ( boot_cpu_data.extended_cpuid_level >= 0x80000008 )
cpuid(0x80000008, &tmp, &e8b, &tmp, &tmp);
- printk(XENLOG_DEBUG "Speculative mitigation facilities:\n");
+ printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(XENLOG_DEBUG " Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
@@ -107,20 +199,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
- printk(XENLOG_DEBUG " Compiled-in support: INDIRECT_THUNK\n");
+ printk(" Compiled-in support: INDIRECT_THUNK\n");
- printk("BTI mitigations: Thunk %s, Others:%s%s%s%s\n",
+ /* Settings for Xen's protection, irrespective of guests. */
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
+ !use_spec_ctrl ? "No" :
+ (default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ opt_ibpb ? " IBPB" : "");
+
+ /*
+ * Alternatives blocks for protecting against and/or virtualising
+ * mitigation support for guests.
+ */
+ printk(" Support for VMs: PV:%s%s%s, HVM:%s%s%s\n",
(boot_cpu_has(X86_FEATURE_SC_MSR_PV) ||
- boot_cpu_has(X86_FEATURE_SC_MSR_HVM)) ?
- default_xen_spec_ctrl & SPEC_CTRL_IBRS ? " IBRS+" :
- " IBRS-" : "",
- opt_ibpb ? " IBPB" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB_NATIVE" : "",
- boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB_VMEXIT" : "");
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_PV) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_PV) ? " RSB" : "",
+ (boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ||
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM)) ? "" : " None",
+ boot_cpu_has(X86_FEATURE_SC_MSR_HVM) ? " MSR_SPEC_CTRL" : "",
+ boot_cpu_has(X86_FEATURE_SC_RSB_HVM) ? " RSB" : "");
printk("XPTI: %s\n",
boot_cpu_has(X86_FEATURE_NO_XPTI) ? "disabled" : "enabled");
@@ -212,7 +315,7 @@ static bool __init retpoline_safe(uint64_t caps)
void __init init_speculation_mitigations(void)
{
enum ind_thunk thunk = THUNK_DEFAULT;
- bool ibrs = false;
+ bool use_spec_ctrl = false, ibrs = false;
uint64_t caps = 0;
if ( boot_cpu_has(X86_FEATURE_ARCH_CAPS) )
@@ -282,20 +385,31 @@ void __init init_speculation_mitigations(void)
else if ( thunk == THUNK_JMP )
setup_force_cpu_cap(X86_FEATURE_IND_THUNK_JMP);
+ /*
+ * If we are on hardware supporting MSR_SPEC_CTRL, see about setting up
+ * the alternatives blocks so we can virtualise support for guests.
+ */
if ( boot_cpu_has(X86_FEATURE_IBRSB) )
{
- /*
- * Even if we've chosen to not have IBRS set in Xen context, we still
- * need the IBRS entry/exit logic to virtualise IBRS support for
- * guests.
- */
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
- setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ if ( opt_msr_sc_pv )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_PV);
+ }
- if ( ibrs )
- default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
+ if ( opt_msr_sc_hvm )
+ {
+ use_spec_ctrl = true;
+ setup_force_cpu_cap(X86_FEATURE_SC_MSR_HVM);
+ }
+
+ if ( use_spec_ctrl )
+ {
+ if ( ibrs )
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS;
- default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ default_spec_ctrl_flags |= SCF_ist_wrmsr;
+ }
}
/*
--
2.1.4
[-- Attachment #70: xsa263-4.10/0011-x86-AMD-Mitigations-for-GPZ-SP4-Speculative-Store-By.patch --]
[-- Type: application/octet-stream, Size: 4400 bytes --]
From 918320daf34931cd5c1c0d9c439ce853f6575970 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Thu, 26 Apr 2018 10:56:28 +0100
Subject: [PATCH] x86/AMD: Mitigations for GPZ SP4 - Speculative Store Bypass
AMD processors will execute loads and stores with the same base register in
program order, which is typically how a compiler emits code.
Therefore, by default no mitigating actions are taken, despite there being
corner cases which are vulnerable to the issue.
For performance testing, or for users with particularly sensitive workloads,
the `spec-ctrl=ssbd` command line option is available to force Xen to disable
Memory Disambiguation on applicable hardware.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 7 ++++++-
xen/arch/x86/cpu/amd.c | 20 ++++++++++++++++++++
xen/arch/x86/spec_ctrl.c | 3 +++
xen/include/asm-x86/spec_ctrl.h | 1 +
4 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 43a6ddb..4e0e580 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1703,7 +1703,7 @@ false disable the quirk workaround, which is also the default.
### spec-ctrl (x86)
> `= List of [ <bool>, xen=<bool>, {pv,hvm,msr-sc,rsb}=<bool>,
-> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb}=<bool> ]`
+> bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -1747,6 +1747,11 @@ On hardware supporting IBPB (Indirect Branch Prediction Barrier), the `ibpb=`
option can be used to force (the default) or prevent Xen from issuing branch
prediction barriers on vcpu context switches.
+On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
+option can be used to force or prevent Xen using the feature itself. On AMD
+hardware, this is a global option applied at boot, and not virtualised for
+guest use.
+
### sync\_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index fc9677f..458a3fe 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -9,6 +9,7 @@
#include <asm/amd.h>
#include <asm/hvm/support.h>
#include <asm/setup.h> /* amd_init_cpu */
+#include <asm/spec_ctrl.h>
#include <asm/acpi.h>
#include <asm/apic.h>
@@ -594,6 +595,25 @@ static void init_amd(struct cpuinfo_x86 *c)
c->x86_capability);
}
+ /*
+ * If the user has explicitly chosen to disable Memory Disambiguation
+ * to mitigiate Speculative Store Bypass, poke the appropriate MSR.
+ */
+ if (opt_ssbd) {
+ int bit = -1;
+
+ switch (c->x86) {
+ case 0x15: bit = 54; break;
+ case 0x16: bit = 33; break;
+ case 0x17: bit = 10; break;
+ }
+
+ if (bit >= 0 && !rdmsr_safe(MSR_AMD64_LS_CFG, value)) {
+ value |= 1ull << bit;
+ wrmsr_safe(MSR_AMD64_LS_CFG, value);
+ }
+ }
+
/* MFENCE stops RDTSC speculation */
if (!cpu_has_lfence_dispatch)
__set_bit(X86_FEATURE_MFENCE_RDTSC, c->x86_capability);
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 4f9282f..e326056 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -43,6 +43,7 @@ static enum ind_thunk {
} opt_thunk __initdata = THUNK_DEFAULT;
static int8_t __initdata opt_ibrs = -1;
bool __read_mostly opt_ibpb = true;
+bool __read_mostly opt_ssbd = false;
bool __initdata bsp_delay_spec_ctrl;
uint8_t __read_mostly default_xen_spec_ctrl;
@@ -164,6 +165,8 @@ static int __init parse_spec_ctrl(const char *s)
opt_ibrs = val;
else if ( (val = parse_boolean("ibpb", s, ss)) >= 0 )
opt_ibpb = val;
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ opt_ssbd = val;
else
rc = -EINVAL;
diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index c6a38f4..4678a40 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -27,6 +27,7 @@
void init_speculation_mitigations(void);
extern bool opt_ibpb;
+extern bool opt_ssbd;
extern bool bsp_delay_spec_ctrl;
extern uint8_t default_xen_spec_ctrl;
--
2.1.4
[-- Attachment #71: xsa263-4.10/0012-x86-Intel-Mitigations-for-GPZ-SP4-Speculative-Store-.patch --]
[-- Type: application/octet-stream, Size: 10261 bytes --]
From db6adc8e55dd43a1b4bb20e06a69475c503cb934 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Wed, 28 Mar 2018 15:21:39 +0100
Subject: [PATCH] x86/Intel: Mitigations for GPZ SP4 - Speculative Store Bypass
To combat GPZ SP4 "Speculative Store Bypass", Intel have extended their
speculative sidechannel mitigations specification as follows:
* A feature bit to indicate that Speculative Store Bypass Disable is
supported.
* A new bit in MSR_SPEC_CTRL which, when set, disables memory disambiguation
in the pipeline.
* A new bit in MSR_ARCH_CAPABILITIES, which will be set in future hardware,
indicating that the hardware is not susceptible to Speculative Store Bypass
sidechannels.
For contemporary processors, this interface will be implemented via a
microcode update.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
docs/misc/xen-command-line.markdown | 12 +++++++-----
tools/libxl/libxl_cpuid.c | 1 +
tools/misc/xen-cpuid.c | 3 +--
xen/arch/x86/cpuid.c | 5 +++++
xen/arch/x86/spec_ctrl.c | 15 ++++++++++++---
xen/include/asm-x86/msr-index.h | 2 ++
xen/include/public/arch-x86/cpufeatureset.h | 1 +
xen/tools/gen-cpuid.py | 17 +++++++++++++----
8 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 4e0e580..107889d 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -496,9 +496,10 @@ accounting for hardware capabilities as enumerated via CPUID.
Currently accepted:
-The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb` are used by
-default if avaiable. They can be ignored, e.g. `no-ibrsb`, at which point Xen
-won't use them itself, and won't offer them to guests.
+The Speculation Control hardware features `ibrsb`, `stibp`, `ibpb`, `ssbd` are
+used by default if available and applicable. They can be ignored,
+e.g. `no-ibrsb`, at which point Xen won't use them itself, and won't offer
+them to guests.
### cpuid\_mask\_cpu (AMD only)
> `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b`
@@ -1728,7 +1729,7 @@ protect itself, and Xen's ability to virtualise support for guests to use.
respectively.
* `msr-sc=` offers control over Xen's support for manipulating MSR\_SPEC\_CTRL
on entry and exit. These blocks are necessary to virtualise support for
- guests and if disabled, guests will be unable to use IBRS/STIBP/etc.
+ guests and if disabled, guests will be unable to use IBRS/STIBP/SSBD/etc.
* `rsb=` offers control over whether to overwrite the Return Stack Buffer /
Return Address Stack on entry to Xen.
@@ -1750,7 +1751,8 @@ prediction barriers on vcpu context switches.
On hardware supporting SSBD (Speculative Store Bypass Disable), the `ssbd=`
option can be used to force or prevent Xen using the feature itself. On AMD
hardware, this is a global option applied at boot, and not virtualised for
-guest use.
+guest use. On Intel hardware, the feature is virtualised for guests,
+independently of Xen's choice of setting.
### sync\_console
> `= <boolean>`
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 3a21f4e..7b0f594 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -205,6 +205,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
{"ibrsb", 0x00000007, 0, CPUID_REG_EDX, 26, 1},
{"stibp", 0x00000007, 0, CPUID_REG_EDX, 27, 1},
{"arch-caps", 0x00000007, 0, CPUID_REG_EDX, 29, 1},
+ {"ssbd", 0x00000007, 0, CPUID_REG_EDX, 31, 1},
{"lahfsahf", 0x80000001, NA, CPUID_REG_ECX, 0, 1},
{"cmplegacy", 0x80000001, NA, CPUID_REG_ECX, 1, 1},
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index b1a46c6..2483a81 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -166,8 +166,7 @@ static const char *str_7d0[32] =
[26] = "ibrsb", [27] = "stibp",
[28] = "REZ", [29] = "arch_caps",
-
- [30 ... 31] = "REZ",
+ [30] = "REZ", [31] = "ssbd",
};
static struct {
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index b45b145..6a710b7 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -43,6 +43,11 @@ static int __init parse_xen_cpuid(const char *s)
if ( !val )
setup_clear_cpu_cap(X86_FEATURE_STIBP);
}
+ else if ( (val = parse_boolean("ssbd", s, ss)) >= 0 )
+ {
+ if ( !val )
+ setup_clear_cpu_cap(X86_FEATURE_SSBD);
+ }
else
rc = -EINVAL;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index e326056..89e3825 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -192,26 +192,31 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
printk("Speculative mitigation facilities:\n");
/* Hardware features which pertain to speculative mitigations. */
- printk(" Hardware features:%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s\n",
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBRS/IBPB" : "",
(_7d0 & cpufeat_mask(X86_FEATURE_STIBP)) ? " STIBP" : "",
+ (_7d0 & cpufeat_mask(X86_FEATURE_SSBD)) ? " SSBD" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ? " IBPB" : "",
(caps & ARCH_CAPABILITIES_IBRS_ALL) ? " IBRS_ALL" : "",
(caps & ARCH_CAPABILITIES_RDCL_NO) ? " RDCL_NO" : "",
- (caps & ARCH_CAPS_RSBA) ? " RSBA" : "");
+ (caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
+ (caps & ARCH_CAPS_SSB_NO) ? " SSB_NO" : "");
/* Compiled-in support which pertains to BTI mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
printk(" Compiled-in support: INDIRECT_THUNK\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s, Other:%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s, Other:%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
!use_spec_ctrl ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
+ !use_spec_ctrl || !boot_cpu_has(X86_FEATURE_SSBD)
+ ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
opt_ibpb ? " IBPB" : "");
/*
@@ -415,6 +420,10 @@ void __init init_speculation_mitigations(void)
}
}
+ /* If we have SSBD available, see whether we should use it. */
+ if ( boot_cpu_has(X86_FEATURE_SSBD) && use_spec_ctrl && opt_ssbd )
+ default_xen_spec_ctrl |= SPEC_CTRL_SSBD;
+
/*
* PV guests can poison the RSB to any virtual address from which
* they can execute a call instruction. This is necessarily outside
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 68fae91..93d6f4e 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -38,6 +38,7 @@
#define MSR_SPEC_CTRL 0x00000048
#define SPEC_CTRL_IBRS (_AC(1, ULL) << 0)
#define SPEC_CTRL_STIBP (_AC(1, ULL) << 1)
+#define SPEC_CTRL_SSBD (_AC(1, ULL) << 2)
#define MSR_PRED_CMD 0x00000049
#define PRED_CMD_IBPB (_AC(1, ULL) << 0)
@@ -46,6 +47,7 @@
#define ARCH_CAPABILITIES_RDCL_NO (_AC(1, ULL) << 0)
#define ARCH_CAPABILITIES_IBRS_ALL (_AC(1, ULL) << 1)
#define ARCH_CAPS_RSBA (_AC(1, ULL) << 2)
+#define ARCH_CAPS_SSB_NO (_AC(1, ULL) << 4)
/* Intel MSRs. Some also available on other CPUs */
#define MSR_IA32_PERFCTR0 0x000000c1
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 8da5783..7acf822 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -245,6 +245,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
diff --git a/xen/tools/gen-cpuid.py b/xen/tools/gen-cpuid.py
index 613b909..65526ff 100755
--- a/xen/tools/gen-cpuid.py
+++ b/xen/tools/gen-cpuid.py
@@ -257,10 +257,19 @@ def crunch_numbers(state):
AVX512BW, AVX512VL, AVX512VBMI, AVX512_4VNNIW,
AVX512_4FMAPS, AVX512_VPOPCNTDQ],
- # Single Thread Indirect Branch Predictors enumerates a new bit in the
- # MSR enumerated by Indirect Branch Restricted Speculation/Indirect
- # Branch Prediction Barrier enumeration.
- IBRSB: [STIBP],
+ # The features:
+ # * Single Thread Indirect Branch Predictors
+ # * Speculative Store Bypass Disable
+ #
+ # enumerate new bits in MSR_SPEC_CTRL, which is enumerated by Indirect
+ # Branch Restricted Speculation/Indirect Branch Prediction Barrier.
+ #
+ # In practice, these features also enumerate the presense of
+ # MSR_SPEC_CTRL. However, no real hardware will exist with SSBD but
+ # not IBRSB, and we pass this MSR directly to guests. Treating them
+ # as dependent features simplifies Xen's logic, and prevents the guest
+ # from seeing implausible configurations.
+ IBRSB: [STIBP, SSBD],
}
deep_features = tuple(sorted(deps.keys()))
--
2.1.4
[-- Attachment #72: xsa263-4.10/0013-x86-msr-Virtualise-MSR_SPEC_CTRL.SSBD-for-guests-to-.patch --]
[-- Type: application/octet-stream, Size: 2709 bytes --]
From 02d0027a89dc49875a41e939498936874a32360f Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Fri, 13 Apr 2018 15:42:34 +0000
Subject: [PATCH] x86/msr: Virtualise MSR_SPEC_CTRL.SSBD for guests to use
Almost all infrastructure is already in place. Update the reserved bits
calculation in guest_wrmsr(), and offer SSBD to guests by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/msr.c | 8 ++++++--
xen/include/public/arch-x86/cpufeatureset.h | 2 +-
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 48d061d..21219c4 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -178,6 +178,8 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
switch ( msr )
{
+ uint64_t rsvd;
+
case MSR_INTEL_PLATFORM_INFO:
case MSR_ARCH_CAPABILITIES:
/* Read-only */
@@ -213,8 +215,10 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
* Note: SPEC_CTRL_STIBP is specified as safe to use (i.e. ignored)
* when STIBP isn't enumerated in hardware.
*/
+ rsvd = ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP |
+ (cp->feat.ssbd ? SPEC_CTRL_SSBD : 0));
- if ( val & ~(SPEC_CTRL_IBRS | SPEC_CTRL_STIBP) )
+ if ( val & rsvd )
goto gp_fault; /* Rsvd bit set? */
vp->spec_ctrl.raw = val;
@@ -233,12 +237,12 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
case MSR_INTEL_MISC_FEATURES_ENABLES:
{
- uint64_t rsvd = ~0ull;
bool old_cpuid_faulting = vp->misc_features_enables.cpuid_faulting;
if ( !vp->misc_features_enables.available )
goto gp_fault;
+ rsvd = ~0ull;
if ( dp->plaform_info.cpuid_faulting )
rsvd &= ~MSR_MISC_FEATURES_CPUID_FAULTING;
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 7acf822..c721c12 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -245,7 +245,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single
XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */
XEN_CPUFEATURE(STIBP, 9*32+27) /*A! STIBP */
XEN_CPUFEATURE(ARCH_CAPS, 9*32+29) /* IA32_ARCH_CAPABILITIES MSR */
-XEN_CPUFEATURE(SSBD, 9*32+31) /* MSR_SPEC_CTRL.SSBD available */
+XEN_CPUFEATURE(SSBD, 9*32+31) /*A MSR_SPEC_CTRL.SSBD available */
#endif /* XEN_CPUFEATURE */
--
2.1.4
[-- Attachment #73: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
reply other threads:[~2018-05-21 21:00 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E1fKruz-00065r-CD@xenbits.xenproject.org \
--to=security@xen.org \
--cc=oss-security@lists.openwall.com \
--cc=security-team-members@xen.org \
--cc=xen-announce@lists.xen.org \
--cc=xen-devel@lists.xen.org \
--cc=xen-users@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).