From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1YlKTV-0005zB-QA for mharc-grub-devel@gnu.org; Thu, 23 Apr 2015 12:59:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40072) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YlKTT-0005xb-2O for grub-devel@gnu.org; Thu, 23 Apr 2015 12:59:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YlKTP-0000DL-Ow for grub-devel@gnu.org; Thu, 23 Apr 2015 12:59:39 -0400 Received: from mail-lb0-x236.google.com ([2a00:1450:4010:c04::236]:34420) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YlKTP-0000D9-CC for grub-devel@gnu.org; Thu, 23 Apr 2015 12:59:35 -0400 Received: by lbcga7 with SMTP id ga7so17988165lbc.1 for ; Thu, 23 Apr 2015 09:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=DIfy63gGIg7Agd5+/j7DdviKiQjABmlSRdJ+fobfwWQ=; b=R7eoanbI+LWIfMTE0OLbcLp9Z/rKO3xPq/XVYsLwNYren0PZfRcJaHwTIwH2RIXaph ou+WrSY/cuH14Pat90KNnJ0zPDVEtXLZ6yauWOEfunG/idBv8sJ2IQxzfocSEA52SWuv siCGmF9Eqy24gAlOg25MmYm+Ys9DwUeOi/lAfxzDLO1+N9tGRGwbcIddMw9IbED4Ryv0 T5+9fqCU3IBTwDQv3xh+nLQ6pX44t1OCzMejp5gnGwsEqLf5fBvSIMGCPU+2e0633/et 5HidJ5LhLCwroD3PYUc8C9X1xe75a4DmaoNTl3fsZTs6rwkN7y9zXmHBOyVaqM00A706 NG6Q== X-Received: by 10.112.144.69 with SMTP id sk5mr3271323lbb.6.1429808374465; Thu, 23 Apr 2015 09:59:34 -0700 (PDT) Received: from opensuse.site (ppp91-76-14-38.pppoe.mtu-net.ru. [91.76.14.38]) by mx.google.com with ESMTPSA id w10sm1986225laz.6.2015.04.23.09.59.33 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2015 09:59:33 -0700 (PDT) Date: Thu, 23 Apr 2015 19:59:31 +0300 From: Andrei Borzenkov To: David Shaw Subject: Re: Using memdisk with grub2 and a gzip-compressed ISO Message-ID: <20150423195931.1eb53b70@opensuse.site> In-Reply-To: References: X-Mailer: Claws Mail 3.11.0 (GTK+ 2.24.27; x86_64-suse-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:4010:c04::236 Cc: grub-devel@gnu.org, syslinux@zytor.com X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: The development of GNU GRUB List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2015 16:59:40 -0000 =D0=92 Thu, 23 Apr 2015 10:23:09 -0400 David Shaw =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > Hello, >=20 > Pardon the cross-post to two lists, but this problem seems to lie in a so= mewhat gray area between grub2 and memdisk. Basically, the problem is this= : it used to be possible to boot a compressed ISO image via memdisk and gru= b legacy, but this no longer works with grub2. >=20 > Here's the setup: Two machines, both with 4GiB of RAM. The only differen= ce between the two is that one is using grub legacy (0.97) and the other is= using grub2 (2.02). I am using memdisk from syslinux 4.02 (though for com= pleteness I also tried 4.05 and 6.03, with the same results). The ISO imag= e itself is 3388047 bytes gzip compressed and 9394176 bytes uncompressed. >=20 > The grub legacy configuration: >=20 > title boot ISO image > kernel /memdisk iso > initrd /my-image.iso.gz >=20 > The grub2 configuration: >=20 > menuentry 'boot ISO image' { > linux16 /memdisk iso > initrd16 /my-image.iso.gz > } >=20 > If anyone would like to see it, the sample gzip-compressed ISO image is= =01 at http://www.jabberwocky.com/my-image.iso.gz >=20 > Now to the problem: When booting the box with grub 0.97, it works. The I= SO boots and the right things happen. Here's the output from memdisk: >=20 > Ramdisk at 0x37cb4000, length 0x0033b28f > gzip image: decompressed addr 0xbf5fa800, len 0x008f5800: ok >=20 > When booting the box with grub2 2.02, it does not work. The error is: >=20 > Ramdisk at 0x37979000, length 0x0033b290 > gzip image: decompressed addr 0xbfff7000, len 0x00008f58: failed > Decompression error: output buffer overrun >=20 > I'm not sure if this is related to the problem, but note the length in th= e "Ramdisk" line from grub legacy is one byte shorter than the length from = grub2. Also note that the length given in the "gzip image" line is shifted= one byte to the left in the grub legacy version (i.e. it's exactly 0x100 t= imes larger). >=20 > 1) I have tried this with memdisk from syslinux 4.02, 4.05, and 6.03 with= the same results each time. > 2) Changing the level of compression (i.e. gzip -1 instead of gzip -9) do= es not make a difference. > 3) This works fine with an uncompressed image with grub2. This also work= s fine with a zip-compressed image with grub2. It only seems to fail with = a gzip-compressed image with grub2. >=20 > Andrei Borzenkov kindly analyzed the image and suggested I contact both t= he syslinux and grub groups as he has a notion of what went wrong. Andrei,= can you fill in anything I missed? >=20 What happens is as following. grub initrd command supports both legacy initrd and initramfs. Initramfs is concatenation of (compressed) cpio archives. For uncompressed cpio archive start of archive and end of trailer are required to be aligned on 4 bytes boundary according to Documentation/early-userspace/buffer-format.txt. Kernel actually checks at least start alignment. Kernel is pretty liberal in what it gets. It will ignore any amount of zero padding between individual archives (and at the beginning as well) and treats initrd size only as upper boundary - it will decompress stream starting from the beginning, not assuming anything about total compressed stream size. This allows grub to simply pad files on 4 bytes from both ends without actually knowing anything about content. Here where the problem happens. When loaded by grub initrd gets extra zero byte padding, thus shifting total size by one byte, because syslinux apparently assumes initrd size is always exact and interprets last 4 bytes as size. I'd actually expect that if syslinux is using Linux compatible protocol to load initrd, it accepts at least the same data as Linux would. May be it could relax requirement for exact size for initrd? Especially as it apparently happens only with some compression algorithms.