Re: How many patches are missing in upstream Linux?

[Atom2 <ariel.atom2@web2web.at>]

Am 11.03.14 18:53, schrieb Konrad Rzeszutek Wilk:
> On Tue, Mar 11, 2014 at 06:38:52PM +0100, Atom2 wrote:
>>
>>
>> Am 10.03.14 21:55, schrieb Jeremy Fitzhardinge:
>> [snip]
>>>> The maintainer is being <insert your own opinion here>:
>>>>   - runtime microcode. What I had been told was to use the 'early
>>>>     microcode' mechanism - which is now implemented and Xen can also scan
>>>>     the initramfs to extract the microcode payload and apply it.
>>>
>>> I've never got that to work, but ucode=-1 with a microcode.dat multiboot
>>> modules works pretty well.
>>>
>>>>     But it misses the important part of server longevity in that the
>>>>     host might be running for years and the microcode might need to be
>>>>     updated during that time. bpetkov wasn't too thrilled about the
>>>>     runtime microcode and I hadn't come back to this yet to figure out
>>>>     what are his exact technical misgivings.
>>>
>>> I think, in the end, he (and Ingo) were advocating just doing a full
>>> emulation of the Intel/AMD update mechanism in the set/getmsr PV
>>> callbacks. Which is doable but... well, not pretty. Maybe a new attempt
>>> with get a new reception.
>>
>> I can add my experience from the perspective of a user trying to get
>> early microcode loading to work with my system:
>>
>> First of all I can confirm that it sort of works - but as Jeremy has
>> pointed out, there seem to be issues with including it into the
>> initramfs. After many unsuccessful tries I also only got it to work
>> when I used a separate entry in multiboot (grub2) which referred to
>> the blob for uploading the payload to the CPU.
>>
>> As soon as I had the microcode integrated into the initramfs as
>> described at
>> http://lists.xen.org/archives/html/xen-devel/2013-09/msg02902.html
>> the system would no longer boot with a message stating that it could
>> not find the root filesystem - I assume that was at the time when
>> the initramfs was about to be unpacked and loaded (if I recall
>> correctly there were no messages from the initramfs on screen).
>> Therefore I couldn't verify whether the microcode was actually
>> lodaded before that.
>
> Hm, that would imply that Linux couldn't skip past the 'non-gzip'
> payload of the initramfs. But it does work for me so I am wondering
> what I am doing differently.
>
> Could you send an email with your .config and perhaps your serial log
> if you still have it?
.config on its way to your email address; if you require the serial log 
I would need to set that up and that would probably take a bit of time.
>
>
>>
>> My feeling at that time was that the unpacking of the initramfs had
>> failed and it was probably more a linux issue rather than a XEN
>> issue - albeit the concatenated file which per the link above should
>> be created by
>>
>> "cat ucode.cpio /boot/initrd-3.5.0.img >/boot/initrd-3.5.0.ucode.img"
>>
>> does not resemble a valid cpio archive any longer (you also can't
>> work with it on the command line - e.g. list its content). I assume
>> that though the kernel expects a valid cpio format file and
>> therefore is unable to unpack and subsequently fails.
>
> It should skip the bits that it does not understand.
Also on the CLI? I did another test right now:

cat /boot/microcode.bin /boot/initrd-3.11.7-hardened-r1.gz > 
/tmp/initrd-3.11.7-hardened-r1.gz

Now the initrd is prepended by the microcode. But when I do

zcat /tmp/initrd-3.11.7-hardened-r1.gz | cpio -itv

I get errors output from both gzip and cpio as follows:

gzip: /tmp/initrd-3.11.7-hardened-r1.gz: not in gzip format
cpio: premature end of archive

And to me that makes sense - how would gzip know how to deal with a 
compressed image file prepended by an uncompressed part without knowing 
the internal structure (i.e. at least its length). And cpio is not able 
to unpack because it doesn't get valid input from gzip through the pipe.

Unless you say that the initrd can't be compressed - that would on the 
face of it probably make sense, but even then a test with cpio failed 
with errors.
>>
>> But in any case, with the additional file in multiboot it works
>> flawlessly. The only catch is that the file distributed by Intel
>> (the microcode.dat which is a text file) has to be converted inot a
>> binary file (I also stripped it down to only contain the parts
>> necessary for my CPU, but I don't know whether that's strictly
>> necessary). And it must _NOT_ be a cpio archive in that case.
>
> You should be able to do 'cat /lib/firmware/intel_ucode/* > /boot/microcode.blob'
> and that would do it too.
I was not aware of that and had a hard time finding a solution. 
Initially I thought the microcode.dat from Intel's website could be used 
as is.
>
>>
>> Searching google there are a few tools available to convert the
>> Intel distributed microcode.dat to binary format and reduce it to
>> only the parts required for a specific CPU.
>>
>> Regards Atom2
I actually downloaded the source of a tool called iucode_tool-1.0.1 
(don't ask me from where) and compiled it. There are two advantages to 
that approach:
Firstly you can build from the Intel website's microcode.dat and you 
don't have to wait until your distro (in my case gentoo which are 
usually anyways pretty quick) makes the latest code available through 
their regular distribution system.
Secondly iucode_tool  also provides an option to only extract the code 
required for a specific processor (defaults to the one in the PC) and 
thus makes the file much smaller (in my case it's only 10,240 bytes as 
opposed to 576,512 bytes compared to the cat /lib/firmware/... above).

>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>