PROPOSAL: Microcode loading under x86 - various options, discussion, etc

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 4440 bytes --]

Hey,

Out of the patches that are out of tree the ones that are still missing are:
 - microcode loader.

I dug around the "old" implementation of microcode_xen and looked at some old
Red Hat bugs to get an idea of its pedigree. What I am not sure about, and
I would appreciate some feedback on that, is whether:

 - One should not do a microcode once the CPU has gone in VT mode. That is it
   has some HVM guests running? (or PVH)? Is that some bogus out-of-date
   information that was relevant for the first generation CPUs?


a)Anyhow, barring that I looked at how the baremetal version of the microcode
  update works to see if the hypervisor could trap on the MSR writes and continue
  on with the update.

  The Intel and AMD Linux drivers seem to follow the same pattern:

  1). Find out what the current microcode version is. That on
   Intel) is via cpuid (0x00000001) and potential RDMSR on MSR_IA32_PLATFORM_ID
   AMD) is via cpuid (0x00000001)

   (both of them read the eax register value)
  2). Apply if neccessary:
   Intel) does an wrmslr on MSR_IA32_UCODE_WRITE (with payload), then wrmslr
       to MSR_IA32_UCODE_REV (with 0), do cpuid (to flush the pipeline + L1)  and then
       rdmsrl MSR_IA32_UCODE_REV to double check.

   AMD) is via doing rdmsr on MSR_AMD64_PATCH_LEVEL (to check) then follow it via
       wrmslr to MSR_AMD64_PATCH_LOADER and rdmsr of MSR_AMD64_PATCH_LEVEL.


  Great, except that the "blob" that is provided via these MSR is just an virtual
  address. No size, nothing. Just 'here it is', and the CPU has to figure out the
  size and whether the blob is correct by itself.

  That means implementing this in the hypervisor to do continuation of the microcode
  loading is not an option. 

b) The microcode_xen driver is not an upstream option either - I don't remember the
  details of it, but I do recall Boris Petkov being unhappy about it.

c) Anyhow, thinking about kexec solved I wrote a little tool (see attached) that sure
  enough allows me to update the microcode. But this is not really an option - unless
  we add some code in /etc/init.d/xencommons to use this program (with more logic in it)
  and load the latest microcode.

d) The other option is to use the hypervisor loading logic that Jan developed - it
  works, but it requires changes in dracut (or mkinitrd) to append all of the
  firmwares (for a specific platform - you can't mix Intel and AMD) and add it to
  the stanze. This does it for me:

	cat /lib/firmware/ucode-intel/* > /srv/tftpboot/lab/tst035/microcode.bin

   And then this extra piece of stanze makes it work:

   	KERNEL mboot.c32
   	APPEND xen.gz   ucode=2 --- vmlinuz  --- initramfs.cpio.gz --- microcode.bin


e) A variation of this - is to piggyback on the early-microcode code work done
  by Intel (and AMD), where they construct an cpio image with microcodes and append it to
  the initrd and scan for a known signature during the boot. The nice thing is that it
  is generic (can have both AMD and Intel blobs) - Linux does it an Xen can do it too.
  (See Documentation/x86/early-microcode.txt). Problem is I can't find any
  tools (dracut, mkinitrd, etc) that implement it. The tools (dracut) would probably do:


	mkdir initrd                                                                       
	cd initrd                                                                          
	mkdir kernel                                                                       
	mkdir kernel/x86                                                                   
	mkdir kernel/x86/microcode                                                         
	cp /srv/tftpboot/lab/tst035/microcode.bin kernel/x86/microcode/GenuineIntel.bin
	cp /srv/tftpboot/lab/tst035/amd-microcode.bin kernel/x86/microcode/AuthenticAMD.bin
	find .|cpio -oc >../ucode.cpio                                                     
	cd ..                                                                              
	cat ucode.cpio /boot/initrd-3.5.0.img >/boot/initrd-3.5.0.ucode.img

  (lifted from said file).

  Anyhow, the neat thing about e) is that once the tools have this, we can just piggyback
  on it by scanning for the signature and then be able to load the microcode.


I am leaning towards e) b/c it would allow us to:
 - automatically during bootup find the microcode
 - one extra blob for AMD and Intel platforms.
 - generic - as Linux OS can use it as well.

Thoughts?

[-- Attachment #2: 0001-misc-xenmicrocode-Upload-lib-firmware-some-blob-to-t.patch --]
[-- Type: text/plain, Size: 5463 bytes --]

>From 23ae50ba25109d228bb70e2cdb334dc0611bd1d0 Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Wed, 3 Jul 2013 16:16:36 -0400
Subject: [PATCH] misc/xenmicrocode: Upload /lib/firmware/<some blob> to the hypervisor.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 tools/libxc/xc_misc.c     |   19 ++++++++++++
 tools/libxc/xenctrl.h     |    2 +
 tools/misc/Makefile       |    7 +++-
 tools/misc/xenmicrocode.c |   70 +++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 96 insertions(+), 2 deletions(-)
 create mode 100644 tools/misc/xenmicrocode.c

diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
index 3c5d64a..2f3487d 100644
--- a/tools/libxc/xc_misc.c
+++ b/tools/libxc/xc_misc.c
@@ -211,6 +211,25 @@ int xc_mca_op(xc_interface *xch, struct xen_mc *mc)
     xc_hypercall_bounce_post(xch, mc);
     return ret;
 }
+int xc_platform_op(xc_interface *xch, struct xen_platform_op *op)
+{
+    int ret = 0;
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BOUNCE(op, sizeof(*op), XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
+
+    if ( xc_hypercall_bounce_pre(xch, op) )
+    {
+        PERROR("Could not bounce xen_platform_op memory buffer");
+        return -1;
+    }
+    op->interface_version = XENPF_INTERFACE_VERSION;
+
+    hypercall.op = __HYPERVISOR_platform_op;
+    hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(op);
+    ret = do_xen_hypercall(xch, &hypercall);
+    xc_hypercall_bounce_post(xch, op);
+    return ret;
+}
 #endif
 
 int xc_perfc_reset(xc_interface *xch)
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index b7741ca..42a2828 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -53,6 +53,7 @@
 #include <xen/foreign/x86_32.h>
 #include <xen/foreign/x86_64.h>
 #include <xen/arch-x86/xen-mca.h>
+#include <xen/platform.h>
 #endif
 
 #ifdef __ia64__
@@ -1780,6 +1781,7 @@ int xc_cpuid_apply_policy(xc_interface *xch,
 void xc_cpuid_to_str(const unsigned int *regs,
                      char **strs);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
+int xc_platform_op(xc_interface *xch, struct xen_platform_op *platform_op);
 #endif
 
 struct xc_px_val {
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index 22e60fd..b578f6c 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -10,7 +10,7 @@ CFLAGS += $(CFLAGS_libxenstore)
 HDRS     = $(wildcard *.h)
 
 TARGETS-y := xenperf xenpm xen-tmem-list-parse gtraceview gtracestat xenlockprof xenwatchdogd
-TARGETS-$(CONFIG_X86) += xen-detect xen-hvmctx xen-hvmcrash xen-lowmemd
+TARGETS-$(CONFIG_X86) += xen-detect xen-hvmctx xen-hvmcrash xen-lowmemd xenmicrocode
 TARGETS-$(CONFIG_MIGRATE) += xen-hptool
 TARGETS := $(TARGETS-y)
 
@@ -23,7 +23,7 @@ INSTALL_BIN-$(CONFIG_X86) += xen-detect
 INSTALL_BIN := $(INSTALL_BIN-y)
 
 INSTALL_SBIN-y := xm xen-bugtool xen-python-path xend xenperf xsview xenpm xen-tmem-list-parse gtraceview gtracestat xenlockprof xenwatchdogd xen-ringwatch
-INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx xen-hvmcrash xen-lowmemd
+INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx xen-hvmcrash xen-lowmemd xenmicrocode
 INSTALL_SBIN-$(CONFIG_MIGRATE) += xen-hptool
 INSTALL_SBIN := $(INSTALL_SBIN-y)
 
@@ -67,6 +67,9 @@ xenperf: xenperf.o
 xenpm: xenpm.o
 	$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
 
+xenmicrocode: xenmicrocode.o
+	$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
+
 gtracestat: gtracestat.o
 	$(CC) $(LDFLAGS) -o $@ $< $(APPEND_LDFLAGS)
 
diff --git a/tools/misc/xenmicrocode.c b/tools/misc/xenmicrocode.c
new file mode 100644
index 0000000..80bb743
--- /dev/null
+++ b/tools/misc/xenmicrocode.c
@@ -0,0 +1,70 @@
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <errno.h>
+#include <string.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <xenctrl.h>
+
+int main(int argc, char *argv[])
+{
+	int fd = 0;
+	unsigned char *fbuf;
+	int len;
+    xc_interface *xc_handle;
+    char *filename;
+    struct stat buf;
+    DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
+    struct xen_platform_op op;
+
+    filename = argv[1];
+	fd = open(filename, O_RDONLY);
+	if (fd <= 0) {
+		printf("Could not open; err: %d(%s)\n", errno, strerror(errno));
+		return errno;
+	}
+	if (stat(filename, &buf) != 0) {
+		printf("Could not open; err: %d(%s)\n", errno, strerror(errno));
+		return errno;
+	}
+
+	printf("%s: %ld\n", filename, buf.st_size);
+	len = buf.st_size;
+	fbuf = mmap(0, len, PROT_READ, MAP_PRIVATE, fd, 0);
+    if ( (xc_handle = xc_interface_open(0,0,0)) == 0 )
+    {
+        fprintf(stderr, "Error opening xc interface: %d (%s)\n",
+                errno, strerror(errno));
+        return 1;
+    }
+	if (fbuf == MAP_FAILED) {
+		printf("Could not map: error: %d(%s)\n", errno,
+		       strerror(errno));
+		return errno;
+	}
+
+    uc = xc_hypercall_buffer_alloc(xc_handle, uc, len);
+    memcpy(uc, fbuf, len);
+
+    set_xen_guest_handle(op.u.microcode.data, uc);
+    op.cmd = XENPF_microcode_update;
+    op.interface_version = XENPF_INTERFACE_VERSION;
+    op.u.microcode.length = len;
+    xc_platform_op(xc_handle, &op);
+
+    xc_hypercall_buffer_free(xc_handle, uc);
+    xc_interface_close(xc_handle);
+
+	if (munmap(fbuf, len)) {
+		printf("Could not unmap: %d(%s)\n", errno, strerror(errno));
+		return errno;
+	}
+	close(fd);
+	return 0;
+}
-- 
1.7.3.4


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel