Re: Using memdisk with grub2 and a gzip-compressed ISO

[Andrei Borzenkov <arvidjaar@gmail.com>]

В Thu, 23 Apr 2015 10:23:09 -0400
David Shaw <dshaw@jabberwocky.com> пишет:

> Hello,
> 
> Pardon the cross-post to two lists, but this problem seems to lie in a somewhat gray area between grub2 and memdisk.  Basically, the problem is this: it used to be possible to boot a compressed ISO image via memdisk and grub legacy, but this no longer works with grub2.
> 
> Here's the setup: Two machines, both with 4GiB of RAM.  The only difference between the two is that one is using grub legacy (0.97) and the other is using grub2 (2.02).  I am using memdisk from syslinux 4.02 (though for completeness I also tried 4.05 and 6.03, with the same results).  The ISO image itself is 3388047 bytes gzip compressed and 9394176 bytes uncompressed.
> 
> The grub legacy configuration:
> 
>    title boot ISO image
>            kernel /memdisk iso
>            initrd /my-image.iso.gz
> 
> The grub2 configuration:
> 
>    menuentry 'boot ISO image' {
>      linux16 /memdisk iso
>      initrd16 /my-image.iso.gz
>    }
> 
> If anyone would like to see it, the sample gzip-compressed ISO image is\x01 at http://www.jabberwocky.com/my-image.iso.gz
> 
> Now to the problem: When booting the box with grub 0.97, it works.  The ISO boots and the right things happen.  Here's the output from memdisk:
> 
>   Ramdisk at 0x37cb4000, length 0x0033b28f
>   gzip image: decompressed addr 0xbf5fa800, len 0x008f5800: ok
> 
> When booting the box with grub2 2.02, it does not work.  The error is:
> 
>   Ramdisk at 0x37979000, length 0x0033b290
>   gzip image: decompressed addr 0xbfff7000, len 0x00008f58: failed
>   Decompression error: output buffer overrun
> 
> I'm not sure if this is related to the problem, but note the length in the "Ramdisk" line from grub legacy is one byte shorter than the length from grub2.  Also note that the length given in the "gzip image" line is shifted one byte to the left in the grub legacy version (i.e. it's exactly 0x100 times larger).
> 
> 1) I have tried this with memdisk from syslinux 4.02, 4.05, and 6.03 with the same results each time.
> 2) Changing the level of compression (i.e. gzip -1 instead of gzip -9) does not make a difference.
> 3) This works fine with an uncompressed image with grub2.  This also works fine with a zip-compressed image with grub2.  It only seems to fail with a gzip-compressed image with grub2.
> 
> Andrei Borzenkov kindly analyzed the image and suggested I contact both the syslinux and grub groups as he has a notion of what went wrong.  Andrei, can you fill in anything I missed?
> 

What happens is as following. grub initrd command supports both legacy
initrd and initramfs. Initramfs is concatenation of (compressed) cpio
archives. For uncompressed cpio archive start of archive and end of
trailer are required to be aligned on 4 bytes boundary according to
Documentation/early-userspace/buffer-format.txt. Kernel actually checks
at least start alignment.

Kernel is pretty liberal in what it gets. It will ignore any amount of
zero padding between individual archives (and at the beginning as well)
and treats initrd size only as upper boundary - it will decompress
stream starting from the beginning, not assuming anything about total
compressed stream size. This allows grub to simply pad files on 4 bytes
from both ends without actually knowing anything about content.

Here where the problem happens. When loaded by grub initrd gets extra
zero byte padding, thus shifting total size by one byte, because
syslinux apparently assumes initrd size is always exact and interprets
last 4 bytes as size.

I'd actually expect that if syslinux is using Linux compatible
protocol to load initrd, it accepts at least the same data as Linux
would. May be it could relax requirement for exact size for initrd?
Especially as it apparently happens only with some compression
algorithms.