[MODERATED] [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[MODERATED] [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[konrad.wilk]

mdd stands for "Memory Disambiguation Disable". It is used to disable
a common industry performance optimization known as "Speculative
Store Bypass" in which loads from addresses to which a recent store
has occurred may (speculatively) see an older value. Intel refers
to this feature as "Memory Disambiguation", which is part of their
"Smart Memory Access" capability in Haswell and later processors.

Some processors have an implementation bug that enables a cache
side-channel attack against such speculatively read values. An
attacker can create exploit code that allows them to read memory
outside of a sandbox environment (for example, malicious JavaScript
in a web page), or to perform more complex attacks against code
running within the same privilege level, e.g. via the stack.

We provide two sets of knobs. The first are x86 specific:

 nomdd
 mdd=[off,auto,on]

We also support cross-architecture versions of these:

 nossbd
 ssbd=[off,auto,on]

By default Memory Disambiguation is enabled hence the parameters are:

 - auto - if possible will disable Memory Disambiguation (MD) at boot
	  if CPU is vulnerable to speculation attacks against MD
 - on   - disable Memory Disambiguation
 - off  - enable Memory Disambiguation

And as mentioned - some CPUs may have an Memory Disambiguation
implementation which is not vulnerable to speculation attack.
For those, the 'auto' (default) option will pick the right choice.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

---
v1.3:
 - Wrong array was used when figuring mdd_v4= arguments
 - If broken microcode found, also disable MD feature.
 - s/mdd_v4/mdd/
 - s/kernel/userpsace/
 - Fix up the commit description
v1.4:
 - Incorporate various feedback and suggestions from Jon Masters
 - Change mdd parameters to allow only big hammer on/off for now
 - Introduce "ssbd" industry standard term for cross-architecture
 - Use mdd_ and MDD_ prefixes to clearly spell out that "enable"
   means we are actually disabling a CPU processor feature (MD)
v2:
 - s/mdd/ssb/ in the x86 code to match with the 'ssb' part

Conflicts:
	arch/x86/kernel/cpu/bugs.c
---
 Documentation/admin-guide/kernel-parameters.txt |  42 ++++++++++
 arch/x86/include/asm/cpufeatures.h              |   1 +
 arch/x86/include/asm/nospec-branch.h            |   6 ++
 arch/x86/kernel/cpu/bugs.c                      | 101 ++++++++++++++++++++++++
 arch/x86/kernel/cpu/intel.c                     |   1 +
 5 files changed, 151 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 1d1d53f85ddd..7a8f7680234e 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2177,6 +2177,22 @@
 			Format: <first>,<last>
 			Specifies range of consoles to be captured by the MDA.
 
+	mdd=		[X86] Control Memory Disambiguation (MD) Disable mitigation
+			(Speculative Store Bypass) vulnerability.
+
+			Certain CPUs are vulnerable to an exploit against a
+			performance optimization known as "Memory Disambiguation"
+			or by the industry term "Speculative Store Bypass". This
+			parameter controls whether the MD optimization is used.
+
+			on - unconditionally disable Memory Disambiguation
+			off - unconditionally enable Memory Disambiguation
+			auto - kernel detects whether your CPU model contains a
+			       vulnerable implementation of Memory Disambiguation
+			       and picks the most appropriate mitigation available
+
+			Not specifying this option is equivalent to mdd=auto.
+
 	mem=nn[KMG]	[KNL,BOOT] Force usage of a specific amount of memory
 			Amount of memory to be used when the kernel is not able
 			to see the whole system memory or for test.
@@ -2639,6 +2655,8 @@
 
 	nohugeiomap	[KNL,x86] Disable kernel huge I/O mappings.
 
+	nomdd		[X86] Disable all mitigations for the Memory Disambiguation vulnerability
+
 	nosmt		[KNL,S390] Disable symmetric multithreading (SMT).
 			Equivalent to smt=1.
 
@@ -2647,6 +2665,8 @@
 			allow data leaks with this option, which is equivalent
 			to spectre_v2=off.
 
+	nossbd		[HW] Disable all mitigations for the Speculative Store Bypass vulnerability
+
 	noxsave		[BUGS=X86] Disables x86 extended register state save
 			and restore using xsave. The kernel will fallback to
 			enabling legacy floating-point and sse state.
@@ -4019,6 +4039,28 @@
 			expediting.  Set to zero to disable automatic
 			expediting.
 
+	ssbd		[HW] Control Speculative Store Bypass (SSB) Disable mitigation
+			(Speculative Store Bypass vulnerability)
+
+			Certain CPUs are vulnerable to an exploit against a
+			a common industry wide performance optimization known
+			as "Speculative Store Bypass" in which recent stores
+			to the same memory location may not be observed by
+			later loads. This can be used in cache side channel
+			attacks to read memory to which the attacker does
+			not directly have access (e.g. sandboxed code).
+
+			This parameter controls whether the Speculative Store
+			Bypass optimization is used.
+
+			on - unconditionally disable Speculative Store Bypass
+			off - unconditionally enable Speculative Store Bypass
+			auto - kernel detects whether your CPU model contains a
+			       vulnerable implementation of Speculative Store
+			       Bypass and picks the most appropriate mitigation
+
+			Not specifying this option is equivalent to ssbd=auto
+
 	stack_guard_gap=	[MM]
 			override the default stack gap protection. The value
 			is in page units and it defines how many pages prior
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index f0ea33c95dc1..418ba6aee0d5 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -333,6 +333,7 @@
 #define X86_FEATURE_SPEC_CTRL		(18*32+26) /* "" Speculation Control (IBRS + IBPB) */
 #define X86_FEATURE_INTEL_STIBP		(18*32+27) /* "" Single Thread Indirect Branch Predictors */
 #define X86_FEATURE_ARCH_CAPABILITIES	(18*32+29) /* IA32_ARCH_CAPABILITIES MSR (Intel) */
+#define X86_FEATURE_MDD			(18*32+31) /* Memory Disambiguation Disable */
 
 /*
  * BUG word(s)
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index f8cc8e0c0c9c..96e5d61e035a 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -224,6 +224,12 @@ extern u64 set_spec_ctrl(u64 val);
 extern u64 clear_spec_ctrl(u64 val);
 extern bool need_spec_ctrl_acc(void);
 
+/* The Speculative Store Bypass disable variants */
+enum ssb_mitigation {
+	SSB_NONE,
+	SSB_DISABLE,
+};
+
 extern char __indirect_thunk_start[];
 extern char __indirect_thunk_end[];
 
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 2838802346fc..b8513a93f4b8 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -27,6 +27,7 @@
 #include <asm/intel-family.h>
 
 static void __init spectre_v2_select_mitigation(void);
+static void __init ssb_select_mitigation(void);
 static void __init spec_ctrl_save_msr(void);
 
 void __init check_bugs(void)
@@ -43,6 +44,12 @@ void __init check_bugs(void)
 	/* Select the proper spectre mitigation before patching alternatives */
 	spectre_v2_select_mitigation();
 
+	/*
+	 * Select the proper mitigation for any exposure to Memory Disambiguation
+	 * vulnerability (otherwise known as Speculative Store Bypass)
+	 */
+	ssb_select_mitigation();
+
 #ifdef CONFIG_X86_32
 	/*
 	 * Check whether we are able to run this kernel safely on SMP.
@@ -344,6 +351,97 @@ static void __init spectre_v2_select_mitigation(void)
 	}
 }
 
+#undef pr_fmt
+#define pr_fmt(fmt)	"SSB: " fmt
+
+static enum ssb_mitigation ssb_mode = SSB_NONE;
+
+/* The kernel command line selection */
+enum ssb_mitigation_cmd {
+	SSB_CMD_NONE,
+	SSB_CMD_AUTO,
+	SSB_CMD_ON,
+};
+
+static const char *ssb_strings[] = {
+	[SSB_NONE]			= "Vulnerable",
+	[SSB_DISABLE]			= "Mitigation: Memory Disambiguation (MD) disabled"
+};
+
+static const struct {
+	const char *option;
+	enum ssb_mitigation_cmd cmd;
+} ssb_mitigation_options[] = {
+	{ "auto",		SSB_CMD_AUTO },	/* Platform decides */
+	{ "on",			SSB_CMD_ON },	/* Disable SSB */
+	{ "off",		SSB_CMD_NONE },	/* Don't touch SSB */
+};
+
+static enum ssb_mitigation_cmd __init ssb_parse_cmdline(void)
+{
+	char arg[20];
+	int ret, i;
+	enum ssb_mitigation_cmd cmd = SSB_CMD_AUTO;
+
+	/*
+	 * On Intel it is exposed in CPUID as Memory Disambiguation Disable
+	 * bit.
+	 */
+	if (!boot_cpu_has(X86_FEATURE_MDD))
+                return SSB_CMD_NONE;
+
+	if (cmdline_find_option_bool(boot_command_line, "nomdd") ||
+	    cmdline_find_option_bool(boot_command_line, "nossbd"))
+		return SSB_CMD_NONE;
+	else {
+		ret = cmdline_find_option(boot_command_line, "mdd", arg, sizeof(arg));
+		if (ret < 0)
+			ret = cmdline_find_option(boot_command_line, "ssbd", arg, sizeof(arg));
+		if (ret < 0)
+			return SSB_CMD_AUTO;
+
+		for (i = 0; i < ARRAY_SIZE(ssb_mitigation_options); i++) {
+			if (!match_option(arg, ret, ssb_mitigation_options[i].option))
+				continue;
+
+			cmd = ssb_mitigation_options[i].cmd;
+			break;
+		}
+
+		if (i >= ARRAY_SIZE(ssb_mitigation_options)) {
+			pr_err("unknown option (%s). Switching to AUTO select\n", arg);
+			return SSB_CMD_AUTO;
+		}
+	}
+
+	return cmd;
+}
+
+static void __init ssb_select_mitigation(void)
+{
+	enum ssb_mitigation_cmd cmd = ssb_parse_cmdline();
+	enum ssb_mitigation mode = SSB_NONE;
+
+	if (!boot_cpu_has_bug(X86_BUG_CPU_SSB) &&
+	    (cmd == SSB_CMD_NONE || cmd == SSB_CMD_AUTO))
+		return;
+
+	switch (cmd) {
+	case SSB_CMD_AUTO:
+	case SSB_CMD_ON:
+		mode = SSB_DISABLE;
+		break;
+	case SSB_CMD_NONE:
+		break;
+	}
+
+	ssb_mode = mode;
+	pr_info("%s\n", ssb_strings[mode]);
+
+	if (mode == SSB_NONE)
+		setup_clear_cpu_cap(X86_FEATURE_MDD);
+}
+
 #undef pr_fmt
 
 #ifdef CONFIG_SYSFS
@@ -370,6 +468,9 @@ ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr,
 			       boot_cpu_has(X86_FEATURE_USE_IBRS_FW) ? ", IBRS_FW" : "",
 			       spectre_v2_module_string());
 
+	case X86_BUG_CPU_SSB:
+		return sprintf(buf, "%s\n", ssb_strings[ssb_mode]);
+
 	default:
 		break;
 	}
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index c3af167d0a70..16ee38d8977a 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -189,6 +189,7 @@ static void early_init_intel(struct cpuinfo_x86 *c)
 		setup_clear_cpu_cap(X86_FEATURE_STIBP);
 		setup_clear_cpu_cap(X86_FEATURE_SPEC_CTRL);
 		setup_clear_cpu_cap(X86_FEATURE_INTEL_STIBP);
+		setup_clear_cpu_cap(X86_FEATURE_MDD);
 	}
 
 	/*
-- 
2.14.3

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Jon Masters]

[-- Attachment #1: Type: text/plain, Size: 2605 bytes --]

On 04/19/2018 10:25 PM, speck for konrad.wilk_at_oracle.com wrote:

> +	nossbd		[HW] Disable all mitigations for the Speculative Store Bypass vulnerability
> +

> +	ssbd		[HW] Control Speculative Store Bypass (SSB) Disable mitigation
> +			(Speculative Store Bypass vulnerability)
> +
> +			Certain CPUs are vulnerable to an exploit against a
> +			a common industry wide performance optimization known
> +			as "Speculative Store Bypass" in which recent stores
> +			to the same memory location may not be observed by
> +			later loads. This can be used in cache side channel
> +			attacks to read memory to which the attacker does
> +			not directly have access (e.g. sandboxed code).
> +
> +			This parameter controls whether the Speculative Store
> +			Bypass optimization is used.
> +
> +			on - unconditionally disable Speculative Store Bypass
> +			off - unconditionally enable Speculative Store Bypass
> +			auto - kernel detects whether your CPU model contains a
> +			       vulnerable implementation of Speculative Store
> +			       Bypass and picks the most appropriate mitigation
> +
> +			Not specifying this option is equivalent to ssbd=auto
> +

I've pinged the Arm, and IBM p and z teams working on mitigations with a
copy of the proposed text above and a description of expected behavior.
They're going to let us know if there are any problems with this plan.

In the case of Arm, they'll do a new SMC (Secure Monitor Call) into ATF
(Arm Trusted Firmware) that will handle the chicken bits. The cpu errata
mechanics on Arm will then allow for uarch specific optimizations. There
are a few of these likely, and I've spoken with various vendors on that.

In the case of IBM p, it's the new "stf-flush" behavior (which they can
then alternative out based on this param similar to how they handle
"pti" and "nopti" even though on POWER they actually previously
mitigated meltdown using a new nop encoding instruction[0]).

In the case of IBM z, it'll probably be similar to p but done using a
new facility. There's already alternatives patching for that too and
they used similar for the branch predictor control knobs prior to
implementing expolines for Spectre-v2 already.

Jon.

[0] That does an L1D$ flush on every kernel/hypervisor exit. Yea, this
isn't the first time we've seen the need for such an instruction in the
industry. We (at Red Hat) actually also asked Intel to implement an L1D$
flush instruction last year when pondering original meltdown mitigation.

-- 
Computer Architect | Sent from my Fedora powered laptop

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Peter Zijlstra]

On Thu, Apr 19, 2018 at 10:25:46PM -0400, speck for konrad.wilk_at_oracle.com wrote:
> We provide two sets of knobs. The first are x86 specific:
> 
>  nomdd
>  mdd=[off,auto,on]
> 
> We also support cross-architecture versions of these:
> 
>  nossbd
>  ssbd=[off,auto,on]
> 

So why does x86 have to be different? Can't it simply use the generic
named ones? I didn't see a rationale for that.

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Jon Masters]

[-- Attachment #1: Type: text/plain, Size: 974 bytes --]

(your mail was unreadable by enigmail on Thunderbird for some reason,
manually extracted using GPG, excuse any copy+paste errors below)

On 04/20/2018 07:21 AM, speck for Peter Zijlstra wrote:
On Thu, Apr 19, 2018 at 10:25:46PM -0400, speck for konrad.wilk_at_oracle.c=
om wrote:
>> We provide two sets of knobs. The first are x86 specific:
>>
>>  nomdd
>>  mdd=3D[off,auto,on]
>>
>> We also support cross-architecture versions of these:
>>
>>  nossbd
>>  ssbd=3D[off,auto,on]

> So why does x86 have to be different? Can't it simply use the generic
> named ones? I didn't see a rationale for that.

x86 doesn't have to be different. I originally proposed keeping also
"mdd" just to keep everyone happy on the x86 side in case they cared.
But "ssbd" is a nice industry neutral term and also works for x86. It
also will fit with the language that Google will use in their text.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Konrad Rzeszutek Wilk]

On Fri, Apr 20, 2018 at 11:33:05AM -0400, speck for Jon Masters wrote:
> (your mail was unreadable by enigmail on Thunderbird for some reason,
> manually extracted using GPG, excuse any copy+paste errors below)

Use mutt :-)
> 
> On 04/20/2018 07:21 AM, speck for Peter Zijlstra wrote:
> On Thu, Apr 19, 2018 at 10:25:46PM -0400, speck for konrad.wilk_at_oracle.c=
> om wrote:
> >> We provide two sets of knobs. The first are x86 specific:
> >>
> >>  nomdd
> >>  mdd=3D[off,auto,on]
> >>
> >> We also support cross-architecture versions of these:
> >>
> >>  nossbd
> >>  ssbd=3D[off,auto,on]
> 
> > So why does x86 have to be different? Can't it simply use the generic
> > named ones? I didn't see a rationale for that.
> 
> x86 doesn't have to be different. I originally proposed keeping also
> "mdd" just to keep everyone happy on the x86 side in case they cared.
> But "ssbd" is a nice industry neutral term and also works for x86. It
> also will fit with the language that Google will use in their text.

Ugh, another web-page. ssbdattack.com?

Let me rip out the mdd from the documentation and the code.

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Jon Masters]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

On 04/20/2018 11:58 AM, speck for Konrad Rzeszutek Wilk wrote:
> On Fri, Apr 20, 2018 at 11:33:05AM -0400, speck for Jon Masters wrote:
>> (your mail was unreadable by enigmail on Thunderbird for some reason,
>> manually extracted using GPG, excuse any copy+paste errors below)
> 
> Use mutt :-)

;) I used to, and before that pine, but $reasons...

>> x86 doesn't have to be different. I originally proposed keeping also
>> "mdd" just to keep everyone happy on the x86 side in case they cared.
>> But "ssbd" is a nice industry neutral term and also works for x86. It
>> also will fit with the language that Google will use in their text.
> 
> Ugh, another web-page. ssbdattack.com?

Nope. Not in this case. GPZ didn't name the last ones (Graz did). We've
been tracking all domain registrations from various research orgs for
the past 6+ months and have also been told there isn't an external (non
Jann) researcher involved in v4 at this time. GPZ don't plan to blog,
just unlock the bug, but of course the media will do the rest. When I
went to Graz a couple months ago, I did some extensive fishing to see
what they were currently researching, e.g. to see if they found this
one, and was pleasantly surprised to see them mostly researching
elsewhere. For example, AMD Zen is getting heavily torn down.

(there is a catchy name for a variant of L1TF targeting SGX, but I think
you heard that one in another context already)

> Let me rip out the mdd from the documentation and the code.

:)

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Borislav Petkov]

On Thu, Apr 19, 2018 at 10:25:46PM -0400, speck for konrad.wilk_at_oracle.com wrote:
> x86/ssb: Provide boot parameters to select which SSB/MD mitigation to use
> 
> mdd stands for "Memory Disambiguation Disable". It is used to disable
> a common industry performance optimization known as "Speculative
> Store Bypass" in which loads from addresses to which a recent store
> has occurred may (speculatively) see an older value. Intel refers
> to this feature as "Memory Disambiguation", which is part of their
> "Smart Memory Access" capability in Haswell and later processors.

...

> +static enum ssb_mitigation_cmd __init ssb_parse_cmdline(void)
> +{
> +	char arg[20];
> +	int ret, i;
> +	enum ssb_mitigation_cmd cmd = SSB_CMD_AUTO;
> +
> +	/*
> +	 * On Intel it is exposed in CPUID as Memory Disambiguation Disable
> +	 * bit.
> +	 */
> +	if (!boot_cpu_has(X86_FEATURE_MDD))
> +                return SSB_CMD_NONE;

Some whitespace damage?

ERROR: code indent should use tabs where possible
#247: FILE: arch/x86/kernel/cpu/bugs.c:391:
+                return SSB_CMD_NONE;$

WARNING: please, no spaces at the start of a line
#247: FILE: arch/x86/kernel/cpu/bugs.c:391:
+                return SSB_CMD_NONE;$


-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

[MODERATED] Re: [patch 06/11] [PATCH v2 06/10] Linux Patch #6

[Borislav Petkov]

On Thu, Apr 19, 2018 at 10:25:46PM -0400, speck for konrad.wilk_at_oracle.com wrote:
> +static void __init ssb_select_mitigation(void)
> +{
> +	enum ssb_mitigation_cmd cmd = ssb_parse_cmdline();
> +	enum ssb_mitigation mode = SSB_NONE;
> +
> +	if (!boot_cpu_has_bug(X86_BUG_CPU_SSB) &&
> +	    (cmd == SSB_CMD_NONE || cmd == SSB_CMD_AUTO))
> +		return;
> +
> +	switch (cmd) {
> +	case SSB_CMD_AUTO:
> +	case SSB_CMD_ON:
> +		mode = SSB_DISABLE;
> +		break;
> +	case SSB_CMD_NONE:
> +		break;
> +	}
> +
> +	ssb_mode = mode;
> +	pr_info("%s\n", ssb_strings[mode]);
> +
> +	if (mode == SSB_NONE)
> +		setup_clear_cpu_cap(X86_FEATURE_MDD);

So this logic doesn't work here: the fact that we disable the mitigation
doesn't mean that the MDD feature is not there anymore - it is simply
disabled. So you can't clear that bit.

You need to define a synthetic X86_FEATURE_USE_MDD bit like the spectre
stuff does define X86_FEATURE_USE_IBPB and X86_FEATURE_USE_IBRS_FW, for
example, and test it whenever you want do to MDD.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
--