[PATCH 1/2] x86: Track minimum microcode revision globally

[PATCH 1/2] x86: Track minimum microcode revision globally

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

For bug workarounds depending on microcode revisions we need to
know the minimum microcode revision shared by all CPUs.

This patch adds infrastructure to track this.

Every time microcode is updated we update a global variable for
the minimum microcode revision of all the CPUs in the system.
At boot time we use the lowest available microcode (and warn
if  they are inconsistent)

At CPU hotplug or S3 resume time there is a short race window
where something might run on the CPUs but before the microcode
update notifier runs. For the current workarounds that need this
this is acceptable and shouldn't be a problem.

Only tested on Intel CPUs, but should work for AMD too.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 arch/x86/include/asm/processor.h  |    3 ++
 arch/x86/kernel/cpu/amd.c         |    2 +
 arch/x86/kernel/cpu/common.c      |   41 +++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/cpu/cpu.h         |    3 ++
 arch/x86/kernel/cpu/intel.c       |    2 +
 arch/x86/kernel/microcode_amd.c   |    1 +
 arch/x86/kernel/microcode_intel.c |    1 +
 7 files changed, 53 insertions(+), 0 deletions(-)

diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 39bc577..0c66d53 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -984,4 +984,7 @@ bool set_pm_idle_to_default(void);
 
 void stop_this_cpu(void *dummy);
 
+extern int min_microcode;
+void update_min_microcode(struct cpuinfo_x86 *c);
+
 #endif /* _ASM_X86_PROCESSOR_H */
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 146bb62..e3f9937 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -684,6 +684,8 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c)
 	}
 
 	rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
+	boot_update_min_microcode(c);
 }
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 6b9333b..5357bb6 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -44,6 +44,9 @@
 
 #include "cpu.h"
 
+/* Global minimum microcode version */
+int min_microcode;
+
 /* all of these masks are initialized in setup_cpu_local_masks() */
 cpumask_var_t cpu_initialized_mask;
 cpumask_var_t cpu_callout_mask;
@@ -1168,6 +1171,44 @@ static void dbg_restore_debug_regs(void)
 #define dbg_restore_debug_regs()
 #endif /* ! CONFIG_KGDB */
 
+
+/*
+ * Track the minimum global microcode version.  On early bootup assume
+ * the BIOS set all CPUs to the same revision. If that's not the case
+ * some code may be already running assuming the newer revision, but
+ * there's not much we can do about that (but it's unlikely to be
+ * problem in early bootup)
+ */
+__cpuinit void boot_update_min_microcode(struct cpuinfo_x86 *c)
+{	
+	static int boot_min_microcode;
+
+	if (!boot_min_microcode) {
+		boot_min_microcode = c->microcode;
+		min_microcode = c->microcode;
+	} else if (c->microcode < boot_min_microcode) {
+		pr_warn("CPU %d has lower microcode revision %x at boot than boot CPU (%x)\n",
+			smp_processor_id(), 
+			c->microcode,
+			boot_min_microcode);
+		min_microcode = c->microcode;
+	}
+}
+
+void update_min_microcode(struct cpuinfo_x86 *c)
+{
+	int i;
+
+	for_each_online_cpu (i)
+		if (cpu_data(i).microcode < c->microcode)
+			return;
+	if (min_microcode != c->microcode) {
+		min_microcode = c->microcode;
+		pr_info("Minimum microcode revision updated to %x\n", c->microcode);
+	}
+}
+EXPORT_SYMBOL_GPL(update_min_microcode);
+
 /*
  * cpu_init() initializes state that is per-CPU. Some data is already
  * initialized (naturally) in the bootstrap process, such as the GDT
diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
index 8bacc78..7f92040 100644
--- a/arch/x86/kernel/cpu/cpu.h
+++ b/arch/x86/kernel/cpu/cpu.h
@@ -34,4 +34,7 @@ extern const struct cpu_dev *const __x86_cpu_dev_start[],
 
 extern void get_cpu_cap(struct cpuinfo_x86 *c);
 extern void cpu_detect_cache_sizes(struct cpuinfo_x86 *c);
+
+extern void boot_update_min_microcode(struct cpuinfo_x86 *c);
+
 #endif /* ARCH_X86_CPU_H */
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 3e6ff6c..d48a13c 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -54,6 +54,8 @@ static void __cpuinit early_init_intel(struct cpuinfo_x86 *c)
 		/* Required by the SDM */
 		sync_core();
 		rdmsr(MSR_IA32_UCODE_REV, lower_word, c->microcode);
+
+		boot_update_min_microcode(c);
 	}
 
 	/*
diff --git a/arch/x86/kernel/microcode_amd.c b/arch/x86/kernel/microcode_amd.c
index 8a2ce8f..b589c7a 100644
--- a/arch/x86/kernel/microcode_amd.c
+++ b/arch/x86/kernel/microcode_amd.c
@@ -215,6 +215,7 @@ static int apply_microcode_amd(int cpu)
 	pr_info("CPU%d: new patch_level=0x%08x\n", cpu, rev);
 	uci->cpu_sig.rev = rev;
 	c->microcode = rev;
+	update_min_microcode(c);
 
 	return 0;
 }
diff --git a/arch/x86/kernel/microcode_intel.c b/arch/x86/kernel/microcode_intel.c
index 0327e2b..50afbb9 100644
--- a/arch/x86/kernel/microcode_intel.c
+++ b/arch/x86/kernel/microcode_intel.c
@@ -326,6 +326,7 @@ static int apply_microcode(int cpu)
 	uci->cpu_sig.rev = val[1];
 	c->microcode = val[1];
 
+	update_min_microcode(c);
 	return 0;
 }
 
-- 
1.7.7.6

[PATCH 2/2] perf/x86: check ucode before disabling PEBS on SandyBridge

[Andi Kleen]

From: Stephane Eranian <eranian@google.com>

[AK: Updated version of Stephane's patch, now using the new global
tracking microcode number and with the correct microcode revision
for SandyBridge-E*. Also use pr_warn_once and checkpatch fixes.]

This patch checks the microcode version before disabling
PEBS on SandyBridge model 42 (desktop, mobile), and 45 (SNB-EP).
PEBS was disabled for both models due to an erratum.

A workaround is implemented by micro-code 0x28. This patch checks
the microcode version and disables PEBS support if version < 0x28.
The check is done each time a PEBS event is created and NOT at boot
time because the micro-code update may only be done after the kernel
has booted.

Go to downloadcenter.intel.com to download microcode updates.
Need microcode update 6/6/2012 or later.

Signed-off-by: Stephane Eranian <eranian@google.com>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 arch/x86/kernel/cpu/perf_event_intel.c |   38 +++++++++++++++++++++++--------
 1 files changed, 28 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/cpu/perf_event_intel.c b/arch/x86/kernel/cpu/perf_event_intel.c
index 187c294..e659641 100644
--- a/arch/x86/kernel/cpu/perf_event_intel.c
+++ b/arch/x86/kernel/cpu/perf_event_intel.c
@@ -13,6 +13,7 @@
 
 #include <asm/hardirq.h>
 #include <asm/apic.h>
+#include <asm/processor.h>
 
 #include "perf_event.h"
 
@@ -1392,6 +1393,25 @@ static void intel_pebs_aliases_snb(struct perf_event *event)
 	}
 }
 
+static int check_pebs_quirks(void)
+{
+	int model = cpu_data(smp_processor_id()).x86_model;
+
+	/* do not have PEBS to begin with */
+	if (!x86_pmu.pebs)
+		return 0;
+
+	/*
+	 * check ucode version for SNB, SNB-EP
+	 */
+	if ((model == 42 && min_microcode < 0x28) ||
+		(model == 45 && min_microcode < 0x618)) {
+		pr_warn_once("SandyBridge PEBS unavailable due to CPU erratum, update microcode\n");
+		return -ENOTSUPP;
+	}
+	return 0;
+}
+
 static int intel_pmu_hw_config(struct perf_event *event)
 {
 	int ret = x86_pmu_hw_config(event);
@@ -1399,8 +1419,14 @@ static int intel_pmu_hw_config(struct perf_event *event)
 	if (ret)
 		return ret;
 
-	if (event->attr.precise_ip && x86_pmu.pebs_aliases)
-		x86_pmu.pebs_aliases(event);
+	if (event->attr.precise_ip) {
+
+		if (check_pebs_quirks())
+			return -ENOTSUPP;
+
+		if (x86_pmu.pebs_aliases)
+			x86_pmu.pebs_aliases(event);
+	}
 
 	if (intel_pmu_needs_lbr_smpl(event)) {
 		ret = intel_pmu_setup_lbr_filter(event);
@@ -1712,13 +1738,6 @@ static __init void intel_clovertown_quirk(void)
 	x86_pmu.pebs_constraints = NULL;
 }
 
-static __init void intel_sandybridge_quirk(void)
-{
-	printk(KERN_WARNING "PEBS disabled due to CPU errata.\n");
-	x86_pmu.pebs = 0;
-	x86_pmu.pebs_constraints = NULL;
-}
-
 static const struct { int id; char *name; } intel_arch_events_map[] __initconst = {
 	{ PERF_COUNT_HW_CPU_CYCLES, "cpu cycles" },
 	{ PERF_COUNT_HW_INSTRUCTIONS, "instructions" },
@@ -1910,7 +1929,6 @@ __init int intel_pmu_init(void)
 
 	case 42: /* SandyBridge */
 	case 45: /* SandyBridge, "Romely-EP" */
-		x86_add_quirk(intel_sandybridge_quirk);
 	case 58: /* IvyBridge */
 		memcpy(hw_cache_event_ids, snb_hw_cache_event_ids,
 		       sizeof(hw_cache_event_ids));
-- 
1.7.7.6

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[H. Peter Anvin]

On 06/12/2012 01:33 PM, Andi Kleen wrote:
>  
> +/* Global minimum microcode version */
> +int min_microcode;
> +

Any reason not to use boot_cpu_data.microcode for this, much like we do
with the CPUID common subset?

	-hpa

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[Andi Kleen]

On 6/12/2012 1:36 PM, H. Peter Anvin wrote:
> On 06/12/2012 01:33 PM, Andi Kleen wrote:
>>
>> +/* Global minimum microcode version */
>> +int min_microcode;
>> +
> Any reason not to use boot_cpu_data.microcode for this, much like we do
> with the CPUID common subset?
No reason. I can switch to that.

-Andi

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[Borislav Petkov]

On Tue, Jun 12, 2012 at 01:33:53PM -0700, Andi Kleen wrote:
> From: Andi Kleen <ak@linux.intel.com>
> 
> For bug workarounds depending on microcode revisions we need to
> know the minimum microcode revision shared by all CPUs.
> 
> This patch adds infrastructure to track this.
> 
> Every time microcode is updated we update a global variable for
> the minimum microcode revision of all the CPUs in the system.
> At boot time we use the lowest available microcode (and warn
> if  they are inconsistent)
> 
> At CPU hotplug or S3 resume time there is a short race window
> where something might run on the CPUs but before the microcode
> update notifier runs. For the current workarounds that need this
> this is acceptable and shouldn't be a problem.
> 
> Only tested on Intel CPUs, but should work for AMD too.
> 
> Signed-off-by: Andi Kleen <ak@linux.intel.com>
> ---
>  arch/x86/include/asm/processor.h  |    3 ++
>  arch/x86/kernel/cpu/amd.c         |    2 +
>  arch/x86/kernel/cpu/common.c      |   41 +++++++++++++++++++++++++++++++++++++
>  arch/x86/kernel/cpu/cpu.h         |    3 ++
>  arch/x86/kernel/cpu/intel.c       |    2 +
>  arch/x86/kernel/microcode_amd.c   |    1 +
>  arch/x86/kernel/microcode_intel.c |    1 +
>  7 files changed, 53 insertions(+), 0 deletions(-)

[ … ]

> +/*
> + * Track the minimum global microcode version.  On early bootup assume
> + * the BIOS set all CPUs to the same revision. If that's not the case
> + * some code may be already running assuming the newer revision, but
> + * there's not much we can do about that (but it's unlikely to be
> + * problem in early bootup)
> + */
> +__cpuinit void boot_update_min_microcode(struct cpuinfo_x86 *c)
> +{	
> +	static int boot_min_microcode;
> +
> +	if (!boot_min_microcode) {
> +		boot_min_microcode = c->microcode;
> +		min_microcode = c->microcode;
> +	} else if (c->microcode < boot_min_microcode) {
> +		pr_warn("CPU %d has lower microcode revision %x at boot than boot CPU (%x)\n",
> +			smp_processor_id(), 
> +			c->microcode,
> +			boot_min_microcode);

What do we need that for? Mixed-ucode versions?

-- 
Regards/Gruss,
    Boris.

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[H. Peter Anvin]

On 06/12/2012 01:49 PM, Borislav Petkov wrote:
> 
> What do we need that for? Mixed-ucode versions?
> 

Different CPUs in a multisocket system may have different compatible
ucode versions.  Of course, I am not even sure if it makes any sense at
all to compare microcode versions without qualifying them with F:M:S.

	-hpa

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[Borislav Petkov]

On Tue, Jun 12, 2012 at 01:52:14PM -0700, H. Peter Anvin wrote:
> Different CPUs in a multisocket system may have different compatible
> ucode versions.

Ah, so you need the different ucode for the different steppings of the
cpus on the board, right? Assuming ucode versions are incremented per
F:M:S tuple, of course, then you can have different compatible microcode
versions.

> Of course, I am not even sure if it makes any sense at all to compare
> microcode versions without qualifying them with F:M:S.

Yep.

-- 
Regards/Gruss,
    Boris.

Re: [PATCH 1/2] x86: Track minimum microcode revision globally

[Andi Kleen]

On Tue, Jun 12, 2012 at 10:59:56PM +0200, Borislav Petkov wrote:
> On Tue, Jun 12, 2012 at 01:52:14PM -0700, H. Peter Anvin wrote:
> > Different CPUs in a multisocket system may have different compatible
> > ucode versions.
> 
> Ah, so you need the different ucode for the different steppings of the
> cpus on the board, right? Assuming ucode versions are incremented per
> F:M:S tuple, of course, then you can have different compatible microcode
> versions.

I believe just for different models. Different models in a single system
may be possible, but are quite unsupported. The code doesn't
handle that, but I don't really worry about that.

> > Of course, I am not even sure if it makes any sense at all to compare
> > microcode versions without qualifying them with F:M:S.

It doesn't.

But I think this model handles 99.999% of all sane configs, don't
really want to complicate it more.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.