Re: [Bugme-new] [Bug 14848] New: Kernel fails to boot if compressed with bzip2 or lzma

Re: [Bugme-new] [Bug 14848] New: Kernel fails to boot if compressed with bzip2 or lzma

[Andrew Morton]

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Sun, 20 Dec 2009 01:46:05 GMT
bugzilla-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=14848
> 
>            Summary: Kernel fails to boot if compressed with bzip2 or lzma
>            Product: Other
>            Version: 2.5
>     Kernel Version: 2.6.33-rc1
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: Other
>         AssignedTo: other_other@kernel-bugs.osdl.org
>         ReportedBy: sd@sedf.de
>         Regression: Yes
> 
> 
> The kernel fails to boot if compressed with bzip2 or lzma, gzip works.
> 
> Commit 4a2ff67c88211026afcbdbc190c13f705dae1b59 does:
> 
> 
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -208,7 +208,7 @@ cmd_gzip = (cat $(filter-out FORCE,$^) | gzip -f -9 > $@)
> || \
> 
>  # Bzip2 and LZMA do not include size in file... so we have to fake that;
>  # append the size as a 32-bit littleendian number as gzip does.
> -size_append = /bin/echo -ne $(shell                                    \
> +size_append = printf $(shell                                           \
>  dec_size=0;                                                            \
>  for F in $1; do                                                               
> \
>         fsize=$$(stat -c "%s" $$F);                                     \
> 
> 
> This is wrong, because it calls the shell's builtin printf (instead
> /usr/bin/printf) which does not support the necessary arguments.
> 
> At my machine (debian) "/bin/echo -ne" works.
> 

Please send a tested, signed-off patch as per
Documentation/SubmittingPatches, thanks.

[PATCH] kbuild: really fix bzImage build with non-bash sh

[Jonathan Nieder]

In an x86 build with CONFIG_KERNEL_LZMA enabled and dash as sh,
arch/x86/boot/compressed/vmlinux.bin.lzma ends with
'\xf0\x7d\x39\x00' (16 bytes) instead of the 4 bytes intended and
the resulting vmlinuz fails to boot.  This improves on the
previous behavior, in which the file contained the characters
'-ne ' as well, but not by much.

Previous commits replaced "echo -ne" first with "/bin/echo -ne",
then "printf" in the hope of improving portability, but none of
these commands is guaranteed to support hexadecimal escapes on
POSIX systems.  So use the shell to convert from hexadecimal to
octal.

With this change, an LZMA-compressed kernel built with dash as sh
boots correctly again.

Reported-by: sd@sedf.de
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alek Du <alek.du@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
Andrew Morton wrote:

> switched to email.

Thanks.

sd@sedf.de reported through <http://bugzilla.kernel.org/show_bug.cgi?id=14848>:

>> The kernel fails to boot if compressed with bzip2 or lzma, gzip works.
>> 
>> Commit 4a2ff67c88211026afcbdbc190c13f705dae1b59 does:

Thanks for tracking it down.  The change below works here.

 scripts/Makefile.lib |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index cd815ac..eabedbb 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -219,8 +219,13 @@ for F in $1; do								\
 	fsize=$$(stat -c "%s" $$F);					\
 	dec_size=$$(expr $$dec_size + $$fsize);				\
 done;									\
-printf "%08x" $$dec_size |						\
-	sed 's/\(..\)\(..\)\(..\)\(..\)/\\\\x\4\\\\x\3\\\\x\2\\\\x\1/g'	\
+printf "%08x\n" $$dec_size |						\
+	sed 's/\(..\)/\1 /g' | {					\
+		read ch0 ch1 ch2 ch3;					\
+		for ch in $$ch3 $$ch2 $$ch1 $$ch0; do			\
+			printf '%s%03o' '\\' $$((0x$$ch)); 		\
+		done;							\
+	}								\
 )
 
 quiet_cmd_bzip2 = BZIP2   $@
-- 
1.6.5.7

Re: [PATCH] kbuild: really fix bzImage build with non-bash sh

[Michal Marek]

On 28.12.2009 20:38, Jonathan Nieder wrote:
> In an x86 build with CONFIG_KERNEL_LZMA enabled and dash as sh,
> arch/x86/boot/compressed/vmlinux.bin.lzma ends with
> '\xf0\x7d\x39\x00' (16 bytes) instead of the 4 bytes intended and
> the resulting vmlinuz fails to boot.  This improves on the
> previous behavior, in which the file contained the characters
> '-ne ' as well, but not by much.
> 
> Previous commits replaced "echo -ne" first with "/bin/echo -ne",
> then "printf" in the hope of improving portability, but none of
> these commands is guaranteed to support hexadecimal escapes on
> POSIX systems.  So use the shell to convert from hexadecimal to
> octal.
> 
> With this change, an LZMA-compressed kernel built with dash as sh
> boots correctly again.
> 
> Reported-by: sd@sedf.de
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> Cc: Michal Marek <mmarek@suse.cz>
> Cc: Michael Tokarev <mjt@tls.msk.ru>
> Cc: Alek Du <alek.du@intel.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> ---
> Andrew Morton wrote:
> 
>> switched to email.
> 
> Thanks.
> 
> sd@sedf.de reported through <http://bugzilla.kernel.org/show_bug.cgi?id=14848>:
> 
>>> The kernel fails to boot if compressed with bzip2 or lzma, gzip works.
>>>
>>> Commit 4a2ff67c88211026afcbdbc190c13f705dae1b59 does:
> 
> Thanks for tracking it down.  The change below works here.

Worked for me as well, so let's hope this finally fixes the issue with
/bin/sh -> dash. I added it to for-linus and will send a pull request soon.

Thanks!
Michal

> 
>  scripts/Makefile.lib |    9 +++++++--
>  1 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index cd815ac..eabedbb 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -219,8 +219,13 @@ for F in $1; do								\
>  	fsize=$$(stat -c "%s" $$F);					\
>  	dec_size=$$(expr $$dec_size + $$fsize);				\
>  done;									\
> -printf "%08x" $$dec_size |						\
> -	sed 's/\(..\)\(..\)\(..\)\(..\)/\\\\x\4\\\\x\3\\\\x\2\\\\x\1/g'	\
> +printf "%08x\n" $$dec_size |						\
> +	sed 's/\(..\)/\1 /g' | {					\
> +		read ch0 ch1 ch2 ch3;					\
> +		for ch in $$ch3 $$ch2 $$ch1 $$ch0; do			\
> +			printf '%s%03o' '\\' $$((0x$$ch)); 		\
> +		done;							\
> +	}								\
>  )
>  
>  quiet_cmd_bzip2 = BZIP2   $@