From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: How many patches are missing in upstream Linux? Date: Tue, 11 Mar 2014 16:33:29 -0400 Message-ID: <20140311203329.GA7719@phenom.dumpdata.com> References: <53185E4D0200007800121725@nat28.tlf.novell.com> <20140306192610.GI9852@localhost.localdomain> <531E26DD.5030807@goop.org> <531F4A2C.7020209@web2web.at> <20140311175352.GC15088@phenom.dumpdata.com> <531F578B.40707@web2web.at> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WNTMt-0004Di-4N for xen-devel@lists.xenproject.org; Tue, 11 Mar 2014 20:33:43 +0000 Content-Disposition: inline In-Reply-To: <531F578B.40707@web2web.at> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Atom2 Cc: Jeremy Fitzhardinge , daniel.kiper@oracle.com, david.vrabel@citrix.com, Jan Beulich , Yang Z Zhang , "xen-devel@lists.xenproject.org" , boris.ostrovsky@oracle.com List-Id: xen-devel@lists.xenproject.org On Tue, Mar 11, 2014 at 07:35:55PM +0100, Atom2 wrote: > > > Am 11.03.14 18:53, schrieb Konrad Rzeszutek Wilk: > >On Tue, Mar 11, 2014 at 06:38:52PM +0100, Atom2 wrote: > >> > >> > >>Am 10.03.14 21:55, schrieb Jeremy Fitzhardinge: > >>[snip] > >>>>The maintainer is being : > >>>> - runtime microcode. What I had been told was to use the 'early > >>>> microcode' mechanism - which is now implemented and Xen can also scan > >>>> the initramfs to extract the microcode payload and apply it. > >>> > >>>I've never got that to work, but ucode=-1 with a microcode.dat multiboot > >>>modules works pretty well. > >>> > >>>> But it misses the important part of server longevity in that the > >>>> host might be running for years and the microcode might need to be > >>>> updated during that time. bpetkov wasn't too thrilled about the > >>>> runtime microcode and I hadn't come back to this yet to figure out > >>>> what are his exact technical misgivings. > >>> > >>>I think, in the end, he (and Ingo) were advocating just doing a full > >>>emulation of the Intel/AMD update mechanism in the set/getmsr PV > >>>callbacks. Which is doable but... well, not pretty. Maybe a new attempt > >>>with get a new reception. > >> > >>I can add my experience from the perspective of a user trying to get > >>early microcode loading to work with my system: > >> > >>First of all I can confirm that it sort of works - but as Jeremy has > >>pointed out, there seem to be issues with including it into the > >>initramfs. After many unsuccessful tries I also only got it to work > >>when I used a separate entry in multiboot (grub2) which referred to > >>the blob for uploading the payload to the CPU. > >> > >>As soon as I had the microcode integrated into the initramfs as > >>described at > >>http://lists.xen.org/archives/html/xen-devel/2013-09/msg02902.html > >>the system would no longer boot with a message stating that it could > >>not find the root filesystem - I assume that was at the time when > >>the initramfs was about to be unpacked and loaded (if I recall > >>correctly there were no messages from the initramfs on screen). > >>Therefore I couldn't verify whether the microcode was actually > >>lodaded before that. > > > >Hm, that would imply that Linux couldn't skip past the 'non-gzip' > >payload of the initramfs. But it does work for me so I am wondering > >what I am doing differently. > > > >Could you send an email with your .config and perhaps your serial log > >if you still have it? > .config on its way to your email address; if you require the serial > log I would need to set that up and that would probably take a bit > of time. I think I do need that. I took a peek at it and tweaked my kernel to have similar options - that is disable the CONFIG_MICROCODE_EARLY in the kernel. Have similar initramfs. And this is what I boot with: [konrad@build-external tst031]$ cpio -it -F initramfs.cpio.gz . kernel kernel/x86 kernel/x86/microcode kernel/x86/microcode/GenuineIntel.bin kernel/x86/microcode/AuthenticAMD.bin 1037 blocks [konrad@build-external tst031]$ file initramfs.cpio.gz initramfs.cpio.gz: ASCII cpio archive (SVR4 with no CRC) And Linux has no problems booting with it. Neither does Xen + Linux with 'ucode=scan' or without that option. Xen sees it: (XEN) Init. ramdisk: ffffffff823f9000->ffffffff87858606 Linux sees it: [ 0.000000] RAMDISK: [mem 0x023f9000-0x07858fff] then unpacks it: [ 13.261414] Unpacking initramfs... [ 15.735168] Freeing initrd memory: 86400K (ffff8800023f9000 - ffff880007859000) so not sure what to make out of it. Is the ramdisk of yours a special filesystem? (Can you have squash/etc filesystems on ramdisks?) > > > > > >> > >>My feeling at that time was that the unpacking of the initramfs had > >>failed and it was probably more a linux issue rather than a XEN > >>issue - albeit the concatenated file which per the link above should > >>be created by > >> > >>"cat ucode.cpio /boot/initrd-3.5.0.img >/boot/initrd-3.5.0.ucode.img" > >> > >>does not resemble a valid cpio archive any longer (you also can't > >>work with it on the command line - e.g. list its content). I assume > >>that though the kernel expects a valid cpio format file and > >>therefore is unable to unpack and subsequently fails. > > > >It should skip the bits that it does not understand. > Also on the CLI? I did another test right now: > > cat /boot/microcode.bin /boot/initrd-3.11.7-hardened-r1.gz > > /tmp/initrd-3.11.7-hardened-r1.gz > > Now the initrd is prepended by the microcode. But when I do > > zcat /tmp/initrd-3.11.7-hardened-r1.gz | cpio -itv > > I get errors output from both gzip and cpio as follows: > > gzip: /tmp/initrd-3.11.7-hardened-r1.gz: not in gzip format > cpio: premature end of archive > > And to me that makes sense - how would gzip know how to deal with a > compressed image file prepended by an uncompressed part without > knowing the internal structure (i.e. at least its length). And cpio > is not able to unpack because it doesn't get valid input from gzip > through the pipe. > > Unless you say that the initrd can't be compressed - that would on > the face of it probably make sense, but even then a test with cpio > failed with errors. What about itself without the 'z' in? So cat /tmp/init.. | cpio -itv? You should see some files. > >> > >>But in any case, with the additional file in multiboot it works > >>flawlessly. The only catch is that the file distributed by Intel > >>(the microcode.dat which is a text file) has to be converted inot a > >>binary file (I also stripped it down to only contain the parts > >>necessary for my CPU, but I don't know whether that's strictly > >>necessary). And it must _NOT_ be a cpio archive in that case. > > > >You should be able to do 'cat /lib/firmware/intel_ucode/* > /boot/microcode.blob' > >and that would do it too. > I was not aware of that and had a hard time finding a solution. > Initially I thought the microcode.dat from Intel's website could be > used as is. > > > >> > >>Searching google there are a few tools available to convert the > >>Intel distributed microcode.dat to binary format and reduce it to > >>only the parts required for a specific CPU. > >> > >>Regards Atom2 > I actually downloaded the source of a tool called iucode_tool-1.0.1 > (don't ask me from where) and compiled it. There are two advantages > to that approach: > Firstly you can build from the Intel website's microcode.dat and you > don't have to wait until your distro (in my case gentoo which are > usually anyways pretty quick) makes the latest code available > through their regular distribution system. > Secondly iucode_tool also provides an option to only extract the > code required for a specific processor (defaults to the one in the > PC) and thus makes the file much smaller (in my case it's only > 10,240 bytes as opposed to 576,512 bytes compared to the cat > /lib/firmware/... above). Right. > > > > >_______________________________________________ > >Xen-devel mailing list > >Xen-devel@lists.xen.org > >http://lists.xen.org/xen-devel > >