[PATCH v3] ARM: support XZ compressed kernels

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

Wire up support for the XZ decompressor

Signed-off-by: Imre Kaloz <kaloz@openwrt.org>
---
New in v3: refreshed against current git

New in v2: added missing .gitignore modification

 arch/arm/Kconfig                        |    1 +
 arch/arm/boot/compressed/.gitignore     |    2 ++
 arch/arm/boot/compressed/Makefile       |   15 ++++++++++++---
 arch/arm/boot/compressed/decompress.c   |    4 ++++
 arch/arm/boot/compressed/piggy.xzkern.S |    6 ++++++
 5 files changed, 25 insertions(+), 3 deletions(-)
 create mode 100644 arch/arm/boot/compressed/piggy.xzkern.S

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 2bda424..866e652 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -20,6 +20,7 @@ config ARM
 	select HAVE_KERNEL_GZIP
 	select HAVE_KERNEL_LZO
 	select HAVE_KERNEL_LZMA
+	select HAVE_KERNEL_XZ
 	select HAVE_IRQ_WORK
 	select HAVE_PERF_EVENTS
 	select PERF_USE_VMALLOC
diff --git a/arch/arm/boot/compressed/.gitignore b/arch/arm/boot/compressed/.gitignore
index e0936a1..d0d441c 100644
--- a/arch/arm/boot/compressed/.gitignore
+++ b/arch/arm/boot/compressed/.gitignore
@@ -1,8 +1,10 @@
+ashldi3.S
 font.c
 lib1funcs.S
 piggy.gzip
 piggy.lzo
 piggy.lzma
+piggy.xzkern
 vmlinux
 vmlinux.lds
 
diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 21f56ff..0b9a6d1 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -92,6 +92,7 @@ SEDFLAGS	= s/TEXT_START/$(ZTEXTADDR)/;s/BSS_START/$(ZBSSADDR)/
 suffix_$(CONFIG_KERNEL_GZIP) = gzip
 suffix_$(CONFIG_KERNEL_LZO)  = lzo
 suffix_$(CONFIG_KERNEL_LZMA) = lzma
+suffix_$(CONFIG_KERNEL_XZ)   = xzkern
 
 # Borrowed libfdt files for the ATAG compatibility mode
 
@@ -112,10 +113,12 @@ endif
 
 targets       := vmlinux vmlinux.lds \
 		 piggy.$(suffix_y) piggy.$(suffix_y).o \
-		 lib1funcs.o lib1funcs.S font.o font.c head.o misc.o $(OBJS)
+		 lib1funcs.o lib1funcs.S ashldi3.o ashldi3.S \
+		 font.o font.c head.o misc.o $(OBJS)
 
 # Make sure files are removed during clean
-extra-y       += piggy.gzip piggy.lzo piggy.lzma lib1funcs.S $(libfdt) $(libfdt_hdrs)
+extra-y       += piggy.gzip piggy.lzo piggy.lzma piggy.xzkern \
+		 lib1funcs.S ashldi3.S $(libfdt) $(libfdt_hdrs)
 
 ifeq ($(CONFIG_FUNCTION_TRACER),y)
 ORIG_CFLAGS := $(KBUILD_CFLAGS)
@@ -150,6 +153,12 @@ lib1funcs = $(obj)/lib1funcs.o
 $(obj)/lib1funcs.S: $(srctree)/arch/$(SRCARCH)/lib/lib1funcs.S
 	$(call cmd,shipped)
 
+# For __aeabi_llsl
+ashldi3 = $(obj)/ashldi3.o
+
+$(obj)/ashldi3.S: $(srctree)/arch/$(SRCARCH)/lib/ashldi3.S
+	$(call cmd,shipped)
+
 # We need to prevent any GOTOFF relocs being used with references
 # to symbols in the .bss section since we cannot relocate them
 # independently from the rest at run time.  This can be achieved by
@@ -171,7 +180,7 @@ if [ $(words $(ZRELADDR)) -gt 1 -a "$(CONFIG_AUTO_ZRELADDR)" = "" ]; then \
 fi
 
 $(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.$(suffix_y).o \
-	 	$(addprefix $(obj)/, $(OBJS)) $(lib1funcs) FORCE
+	 	$(addprefix $(obj)/, $(OBJS)) $(lib1funcs) $(ashldi3) FORCE
 	@$(check_for_multiple_zreladdr)
 	$(call if_changed,ld)
 	@$(check_for_bad_syms)
diff --git a/arch/arm/boot/compressed/decompress.c b/arch/arm/boot/compressed/decompress.c
index 07be5a2..0ecd8b4 100644
--- a/arch/arm/boot/compressed/decompress.c
+++ b/arch/arm/boot/compressed/decompress.c
@@ -44,6 +44,10 @@ extern void error(char *);
 #include "../../../../lib/decompress_unlzma.c"
 #endif
 
+#ifdef CONFIG_KERNEL_XZ
+#include "../../../../lib/decompress_unxz.c"
+#endif
+
 int do_decompress(u8 *input, int len, u8 *output, void (*error)(char *x))
 {
 	return decompress(input, len, NULL, NULL, output, NULL, error);
diff --git a/arch/arm/boot/compressed/piggy.xzkern.S b/arch/arm/boot/compressed/piggy.xzkern.S
new file mode 100644
index 0000000..5703f30
--- /dev/null
+++ b/arch/arm/boot/compressed/piggy.xzkern.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.xzkern"
+	.globl	input_data_end
+input_data_end:
-- 
1.7.1

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

On Mon, 07 Nov 2011 21:03:36 +0100, Imre Kaloz <kaloz@openwrt.org> wrote:

> Wire up support for the XZ decompressor
>
> Signed-off-by: Imre Kaloz <kaloz@openwrt.org>

ping?


Imre

[PATCH v3] ARM: support XZ compressed kernels

[Nicolas Pitre]

On Fri, 11 Nov 2011, Imre Kaloz wrote:

> On Mon, 07 Nov 2011 21:03:36 +0100, Imre Kaloz <kaloz@openwrt.org> wrote:
> 
> > Wire up support for the XZ decompressor
> > 
> > Signed-off-by: Imre Kaloz <kaloz@openwrt.org>
> 
> ping?

Acked-by: Nicolas Pitre <nico@linaro.org>


Nicolas

[PATCH v3] ARM: support XZ compressed kernels

[Linus Walleij]

On Mon, Nov 7, 2011 at 9:03 PM, Imre Kaloz <kaloz@openwrt.org> wrote:

> Wire up support for the XZ decompressor
>
> Signed-off-by: Imre Kaloz <kaloz@openwrt.org>

Looks useful.
Acked-by: Linus Walleij <linus.walleij@linaro.org>

Put it in Russell's patch tracker.
Linus Walleij

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

On Sat, 12 Nov 2011 11:50:02 +0100, Linus Walleij  
<linus.walleij@linaro.org> wrote:

> On Mon, Nov 7, 2011 at 9:03 PM, Imre Kaloz <kaloz@openwrt.org> wrote:
>
>> Wire up support for the XZ decompressor
>>
>> Signed-off-by: Imre Kaloz <kaloz@openwrt.org>
>
> Looks useful.
> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>
> Put it in Russell's patch tracker.
> Linus Walleij

A previous version was there, but Russell applied it then dropped it,  
hence I didn't want to add it again.

Russell, would you like me to add it there?


Imre

[PATCH v3] ARM: support XZ compressed kernels

[Veli-Pekka Peltola]

Hi,

On 11/13/2011 12:56 PM, Imre Kaloz wrote:
> On Sat, 12 Nov 2011 11:50:02 +0100, Linus Walleij
> <linus.walleij@linaro.org> wrote:
>
>> On Mon, Nov 7, 2011 at 9:03 PM, Imre Kaloz <kaloz@openwrt.org> wrote:
>>
>>> Wire up support for the XZ decompressor
>>>
>>> Signed-off-by: Imre Kaloz <kaloz@openwrt.org>
>>
>> Looks useful.
>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>
>> Put it in Russell's patch tracker.
>> Linus Walleij
>
> A previous version was there, but Russell applied it then dropped it,
> hence I didn't want to add it again.
>
> Russell, would you like me to add it there?
>
>
> Imre

It has been quite silent related to this patch. What is current status?

I have been using XZ in our development kernels nine months now without 
any problems on AT91 platform.

Acked-by: Veli-Pekka Peltola <veli-pekka.peltola@bluegiga.com>

--
Veli-Pekka Peltola

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

Hi,

On Thu, 19 Jan 2012 12:40:24 +0100, Veli-Pekka Peltola  
<veli-pekka.peltola@bluegiga.com> wrote:

> On 11/13/2011 12:56 PM, Imre Kaloz wrote:
>> On Sat, 12 Nov 2011 11:50:02 +0100, Linus Walleij
>> <linus.walleij@linaro.org> wrote:
>>
>>> On Mon, Nov 7, 2011 at 9:03 PM, Imre Kaloz <kaloz@openwrt.org> wrote:
>>>
>>>> Wire up support for the XZ decompressor
>>>>
>>>> Signed-off-by: Imre Kaloz <kaloz@openwrt.org>
>>>
>>> Looks useful.
>>> Acked-by: Linus Walleij <linus.walleij@linaro.org>
>>>
>>> Put it in Russell's patch tracker.
>>> Linus Walleij
>>
>> A previous version was there, but Russell applied it then dropped it,
>> hence I didn't want to add it again.
>>
>> Russell, would you like me to add it there?
>>
>>
>> Imre
>
> It has been quite silent related to this patch. What is current status?
>
> I have been using XZ in our development kernels nine months now without  
> any problems on AT91 platform.
>
> Acked-by: Veli-Pekka Peltola <veli-pekka.peltola@bluegiga.com>
>
> --
> Veli-Pekka Peltola


Recent changes introduced an error, but Nico helped me clean that up. Will  
submit the final patch to the patch system this weekend.

Thanks for testing :)


Imre

[PATCH v3] ARM: support XZ compressed kernels

[Veli-Pekka Peltola]

Hi Imre,

On 01/19/2012 01:43 PM, Imre Kaloz wrote:
> Recent changes introduced an error, but Nico helped me clean that up.
> Will submit the final patch to the patch system this weekend.

Just a reminder: I was unable to find this from the patch tracking 
system. Any updates? Can I help somehow?

--
Veli-Pekka Peltola

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

On Wed, 25 Jan 2012 18:00:46 +0100, Veli-Pekka Peltola  
<veli-pekka.peltola@bluegiga.com> wrote:

Hi,

> On 01/19/2012 01:43 PM, Imre Kaloz wrote:
>> Recent changes introduced an error, but Nico helped me clean that up.
>> Will submit the final patch to the patch system this weekend.
>
> Just a reminder: I was unable to find this from the patch tracking  
> system. Any updates? Can I help somehow?

It has been added as patch 7001/2 [1]. Let's hope Russell could send it  
upstream soon, given it's not intrusive and floats around for months now :)


Imre

[1] http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=7001/2

[PATCH v3] ARM: support XZ compressed kernels

[Russell King - ARM Linux]

On Wed, Jan 25, 2012 at 07:00:46PM +0200, Veli-Pekka Peltola wrote:
> Hi Imre,
>
> On 01/19/2012 01:43 PM, Imre Kaloz wrote:
>> Recent changes introduced an error, but Nico helped me clean that up.
>> Will submit the final patch to the patch system this weekend.
>
> Just a reminder: I was unable to find this from the patch tracking  
> system. Any updates? Can I help somehow?

Well, this is where things got to last time around:

http://lists.arm.linux.org.uk/lurker/message/20110722.200731.32f090d9.en.html

and then everything stopped.  No further feedback, nothing.  So I've
been totally ignoring XZ stuff because I've been believing that the
issue hasn't been fixed.

It seems that it has now been fixed, back in July.  Nice to have been
told about that development, etc.

So yes, as the problem has been resolved, we can now move forward on
it.  I'm not sure it's -rc material though, so I'll queue it for the
next merge window.  We've waited six months already since the 'fix'
went in, so plainly another three won't hurt one bit.

[PATCH v3] ARM: support XZ compressed kernels

[Imre Kaloz]

Hi Russell,

On Thu, 26 Jan 2012 21:26:30 +0100, Russell King - ARM Linux  
<linux@arm.linux.org.uk> wrote:

>> Just a reminder: I was unable to find this from the patch tracking
>> system. Any updates? Can I help somehow?
>
> Well, this is where things got to last time around:
>
> http://lists.arm.linux.org.uk/lurker/message/20110722.200731.32f090d9.en.html
>
> and then everything stopped.  No further feedback, nothing.  So I've
> been totally ignoring XZ stuff because I've been believing that the
> issue hasn't been fixed.
>
> It seems that it has now been fixed, back in July.  Nice to have been
> told about that development, etc.

As far as I know in the private mails that part has been cleared up with  
Lasse - you were CC'd all the time.

> So yes, as the problem has been resolved, we can now move forward on
> it.  I'm not sure it's -rc material though, so I'll queue it for the
> next merge window.  We've waited six months already since the 'fix'
> went in, so plainly another three won't hurt one bit.

When I've noticed v2 didn't get upstream without that change, I did a v3,  
and got no response for that from you [1]. Than the string.c related  
changes broke that, hence this (v4) version. If possible, please try to  
get this into 3.3, as I think there's no chance it could break anything.


Imre


[1]  
http://lists.arm.linux.org.uk/lurker/message/20111107.200336.feeeb3c5.en.html

[PATCH v3] ARM: support XZ compressed kernels

[Nicolas Pitre]

On Thu, 26 Jan 2012, Imre Kaloz wrote:

> Hi Russell,
> 
> On Thu, 26 Jan 2012 21:26:30 +0100, Russell King - ARM Linux
> > So yes, as the problem has been resolved, we can now move forward on
> > it.  I'm not sure it's -rc material though, so I'll queue it for the
> > next merge window.  We've waited six months already since the 'fix'
> > went in, so plainly another three won't hurt one bit.
> 
> When I've noticed v2 didn't get upstream without that change, I did a v3, and
> got no response for that from you [1]. Than the string.c related changes broke
> that, hence this (v4) version. If possible, please try to get this into 3.3,
> as I think there's no chance it could break anything.

That doesn't matter if this may or may not break anything.  The merge 
window for v3.3 is over, and only bug fixes are acceptable now 
until the next merge window opens.


Nicolas