Re: How many patches are missing in upstream Linux?

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Mar 11, 2014 at 07:35:55PM +0100, Atom2 wrote:
> 
> 
> Am 11.03.14 18:53, schrieb Konrad Rzeszutek Wilk:
> >On Tue, Mar 11, 2014 at 06:38:52PM +0100, Atom2 wrote:
> >>
> >>
> >>Am 10.03.14 21:55, schrieb Jeremy Fitzhardinge:
> >>[snip]
> >>>>The maintainer is being <insert your own opinion here>:
> >>>>  - runtime microcode. What I had been told was to use the 'early
> >>>>    microcode' mechanism - which is now implemented and Xen can also scan
> >>>>    the initramfs to extract the microcode payload and apply it.
> >>>
> >>>I've never got that to work, but ucode=-1 with a microcode.dat multiboot
> >>>modules works pretty well.
> >>>
> >>>>    But it misses the important part of server longevity in that the
> >>>>    host might be running for years and the microcode might need to be
> >>>>    updated during that time. bpetkov wasn't too thrilled about the
> >>>>    runtime microcode and I hadn't come back to this yet to figure out
> >>>>    what are his exact technical misgivings.
> >>>
> >>>I think, in the end, he (and Ingo) were advocating just doing a full
> >>>emulation of the Intel/AMD update mechanism in the set/getmsr PV
> >>>callbacks. Which is doable but... well, not pretty. Maybe a new attempt
> >>>with get a new reception.
> >>
> >>I can add my experience from the perspective of a user trying to get
> >>early microcode loading to work with my system:
> >>
> >>First of all I can confirm that it sort of works - but as Jeremy has
> >>pointed out, there seem to be issues with including it into the
> >>initramfs. After many unsuccessful tries I also only got it to work
> >>when I used a separate entry in multiboot (grub2) which referred to
> >>the blob for uploading the payload to the CPU.
> >>
> >>As soon as I had the microcode integrated into the initramfs as
> >>described at
> >>http://lists.xen.org/archives/html/xen-devel/2013-09/msg02902.html
> >>the system would no longer boot with a message stating that it could
> >>not find the root filesystem - I assume that was at the time when
> >>the initramfs was about to be unpacked and loaded (if I recall
> >>correctly there were no messages from the initramfs on screen).
> >>Therefore I couldn't verify whether the microcode was actually
> >>lodaded before that.
> >
> >Hm, that would imply that Linux couldn't skip past the 'non-gzip'
> >payload of the initramfs. But it does work for me so I am wondering
> >what I am doing differently.
> >
> >Could you send an email with your .config and perhaps your serial log
> >if you still have it?
> .config on its way to your email address; if you require the serial
> log I would need to set that up and that would probably take a bit
> of time.

I think I do need that.
I took a peek at it and tweaked my kernel to have similar
options - that is disable the CONFIG_MICROCODE_EARLY in the kernel.
Have similar initramfs.

And this is what I boot with:
[konrad@build-external tst031]$ cpio -it -F initramfs.cpio.gz 
.
kernel
kernel/x86
kernel/x86/microcode
kernel/x86/microcode/GenuineIntel.bin
kernel/x86/microcode/AuthenticAMD.bin
1037 blocks
[konrad@build-external tst031]$ file initramfs.cpio.gz 
initramfs.cpio.gz: ASCII cpio archive (SVR4 with no CRC)

And Linux has no problems booting with it. Neither does
Xen + Linux with 'ucode=scan' or without that option.

Xen sees it:
(XEN)  Init. ramdisk: ffffffff823f9000->ffffffff87858606
Linux sees it:
[    0.000000] RAMDISK: [mem 0x023f9000-0x07858fff]
then unpacks it:
[   13.261414] Unpacking initramfs...
[   15.735168] Freeing initrd memory: 86400K (ffff8800023f9000 - ffff880007859000)

so not sure what to make out of it.

Is the ramdisk of yours a special filesystem? (Can you have squash/etc
filesystems on ramdisks?)
> >
> >
> >>
> >>My feeling at that time was that the unpacking of the initramfs had
> >>failed and it was probably more a linux issue rather than a XEN
> >>issue - albeit the concatenated file which per the link above should
> >>be created by
> >>
> >>"cat ucode.cpio /boot/initrd-3.5.0.img >/boot/initrd-3.5.0.ucode.img"
> >>
> >>does not resemble a valid cpio archive any longer (you also can't
> >>work with it on the command line - e.g. list its content). I assume
> >>that though the kernel expects a valid cpio format file and
> >>therefore is unable to unpack and subsequently fails.
> >
> >It should skip the bits that it does not understand.
> Also on the CLI? I did another test right now:
> 
> cat /boot/microcode.bin /boot/initrd-3.11.7-hardened-r1.gz >
> /tmp/initrd-3.11.7-hardened-r1.gz
> 
> Now the initrd is prepended by the microcode. But when I do
> 
> zcat /tmp/initrd-3.11.7-hardened-r1.gz | cpio -itv
> 
> I get errors output from both gzip and cpio as follows:
> 
> gzip: /tmp/initrd-3.11.7-hardened-r1.gz: not in gzip format
> cpio: premature end of archive
> 
> And to me that makes sense - how would gzip know how to deal with a
> compressed image file prepended by an uncompressed part without
> knowing the internal structure (i.e. at least its length). And cpio
> is not able to unpack because it doesn't get valid input from gzip
> through the pipe.
> 
> Unless you say that the initrd can't be compressed - that would on
> the face of it probably make sense, but even then a test with cpio
> failed with errors.

What about itself without the 'z' in? So cat /tmp/init.. | cpio -itv?
You should see some files.

> >>
> >>But in any case, with the additional file in multiboot it works
> >>flawlessly. The only catch is that the file distributed by Intel
> >>(the microcode.dat which is a text file) has to be converted inot a
> >>binary file (I also stripped it down to only contain the parts
> >>necessary for my CPU, but I don't know whether that's strictly
> >>necessary). And it must _NOT_ be a cpio archive in that case.
> >
> >You should be able to do 'cat /lib/firmware/intel_ucode/* > /boot/microcode.blob'
> >and that would do it too.
> I was not aware of that and had a hard time finding a solution.
> Initially I thought the microcode.dat from Intel's website could be
> used as is.
> >
> >>
> >>Searching google there are a few tools available to convert the
> >>Intel distributed microcode.dat to binary format and reduce it to
> >>only the parts required for a specific CPU.
> >>
> >>Regards Atom2
> I actually downloaded the source of a tool called iucode_tool-1.0.1
> (don't ask me from where) and compiled it. There are two advantages
> to that approach:
> Firstly you can build from the Intel website's microcode.dat and you
> don't have to wait until your distro (in my case gentoo which are
> usually anyways pretty quick) makes the latest code available
> through their regular distribution system.
> Secondly iucode_tool  also provides an option to only extract the
> code required for a specific processor (defaults to the one in the
> PC) and thus makes the file much smaller (in my case it's only
> 10,240 bytes as opposed to 576,512 bytes compared to the cat
> /lib/firmware/... above).

Right.
> 
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xen.org
> >http://lists.xen.org/xen-devel
> >