AArch64 kernel image decompression

AArch64 kernel image decompression

[Josh Cartwright]

Hey Will-

Scanning the Documentation/arm64/booting.txt document, I came across
this, listing an optional step a bootloader might take:

"The AArch64 kernel does not currently provide a decompressor and
therefore requires decompression (gzip, etc.) to be performed by the
boot loader if a compressed Image target (e.g. Image.gz) is used."

Do you envision that, if a decompressor is desired, it will always be
the responsibility of a bootloader to implement it?

In other words, would you accept patches adding support for building
zImages (and others) for AArch64?

Thanks,
   Josh

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

AArch64 kernel image decompression

[Will Deacon]

On Fri, Jan 10, 2014 at 05:04:36PM +0000, Josh Cartwright wrote:
> Hey Will-

Hi Josh,

> Scanning the Documentation/arm64/booting.txt document, I came across
> this, listing an optional step a bootloader might take:
> 
> "The AArch64 kernel does not currently provide a decompressor and
> therefore requires decompression (gzip, etc.) to be performed by the
> boot loader if a compressed Image target (e.g. Image.gz) is used."
> 
> Do you envision that, if a decompressor is desired, it will always be
> the responsibility of a bootloader to implement it?

That's certainly our current stance, yes. It also simplifies the booting
code and things like EFI stub, so I'd be reluctant to consider adding a
decompressor to the arm64 kernel without a compelling use-case.

> In other words, would you accept patches adding support for building
> zImages (and others) for AArch64?

Are these patches actually needed, or are you trying to find a project
involving the arm64 kernel? If the latter, I'm sure we can think of plenty
of other things you can work on :)

Will

AArch64 kernel image decompression

[bhupesh.sharma at freescale.com]

Hi Will,

> -----Original Message-----
> From: linux-arm-kernel [mailto:linux-arm-kernel-
> bounces at lists.infradead.org] On Behalf Of Will Deacon
> Sent: Monday, January 13, 2014 4:48 PM
> To: Josh Cartwright
> Cc: Catalin Marinas; linux-arm-kernel at lists.infradead.org
> Subject: Re: AArch64 kernel image decompression
> 
> On Fri, Jan 10, 2014 at 05:04:36PM +0000, Josh Cartwright wrote:
> > Hey Will-
> 
> Hi Josh,
> 
> > Scanning the Documentation/arm64/booting.txt document, I came across
> > this, listing an optional step a bootloader might take:
> >
> > "The AArch64 kernel does not currently provide a decompressor and
> > therefore requires decompression (gzip, etc.) to be performed by the
> > boot loader if a compressed Image target (e.g. Image.gz) is used."
> >
> > Do you envision that, if a decompressor is desired, it will always be
> > the responsibility of a bootloader to implement it?
> 
> That's certainly our current stance, yes. It also simplifies the booting
> code and things like EFI stub, so I'd be reluctant to consider adding a
> decompressor to the arm64 kernel without a compelling use-case.
> 
> > In other words, would you accept patches adding support for building
> > zImages (and others) for AArch64?
> 
> Are these patches actually needed, or are you trying to find a project
> involving the arm64 kernel? If the latter, I'm sure we can think of
> plenty of other things you can work on :)
> 

With the u-boot bootloader also having AArch64 support now (see [1]), it would
make sense to add the uImage and decompressor support to the AArch64 Makefile in 
Linux kernel as well, so that u-boot can also boot uImages for AArch64 systems
out-off-box.

[1]. http://git.denx.de/?p=u-boot/u-boot-arm.git;a=shortlog;h=refs/heads/aarch64

Regards,
Bhupesh

AArch64 kernel image decompression

[Arnd Bergmann]

On Monday 13 January 2014 11:29:20 bhupesh.sharma at freescale.com wrote:
> 
> With the u-boot bootloader also having AArch64 support now (see [1]), it would
> make sense to add the uImage and decompressor support to the AArch64 Makefile in 
> Linux kernel as well, so that u-boot can also boot uImages for AArch64 systems
> out-off-box.
> 
> [1]. http://git.denx.de/?p=u-boot/u-boot-arm.git;a=shortlog;h=refs/heads/aarch64

u-boot should never have to use a uImage on arm64, since it already
supports booting a vmlinux file.

A more sensible thing to add would be vmlinux.gz support in u-boot,
if that doesn't exist already.

	Arnd

AArch64 kernel image decompression

[bhupesh.sharma at freescale.com]

Hi Arnd,

> -----Original Message-----
> From: Arnd Bergmann [mailto:arnd at arndb.de]
> Sent: Monday, January 13, 2014 5:13 PM
> To: linux-arm-kernel at lists.infradead.org
> Cc: Sharma Bhupesh-B45370; 'Will Deacon'; 'Josh Cartwright'; 'Catalin
> Marinas'
> Subject: Re: AArch64 kernel image decompression
> 
> On Monday 13 January 2014 11:29:20 bhupesh.sharma at freescale.com wrote:
> >
> > With the u-boot bootloader also having AArch64 support now (see [1]),
> > it would make sense to add the uImage and decompressor support to the
> > AArch64 Makefile in Linux kernel as well, so that u-boot can also boot
> > uImages for AArch64 systems out-off-box.
> >
> > [1].
> > http://git.denx.de/?p=u-boot/u-boot-arm.git;a=shortlog;h=refs/heads/aa
> > rch64
> 
> u-boot should never have to use a uImage on arm64, since it already
> supports booting a vmlinux file.
> 
> A more sensible thing to add would be vmlinux.gz support in u-boot, if
> that doesn't exist already.
> 

AFAIK, u-boot requires a u-boot specific header (see [1]) on top of any image it wants to
boot - whether it's a uncompressed vmlinux or a compressed vmlinux.gz

In the ARM32 code base we had the Makefile rule to generate a uImage automatically (see [2]), with ARM64
now, I am forced to compress the vmlinux and put a u-boot header on top of it using 'mkimage' and then
feed the 'uImage' thus generated to the u-boot.

So, it would be good if the ARM64 Makefile can mimic the ARM32 one in this aspect - now that we have
a working u-boot capable of booting vanilla linux on an ARMv8 foundation model.

[1]. http://www.denx.de/wiki/DULG/UBootImages
[2]. https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/arch/arm/Makefile?id=refs/tags/next-20140110#n330

Regards,
Bhupesh

AArch64 kernel image decompression

[Olof Johansson]

On Mon, Jan 13, 2014 at 7:52 PM, bhupesh.sharma at freescale.com
<bhupesh.sharma@freescale.com> wrote:

> AFAIK, u-boot requires a u-boot specific header (see [1]) on top of any image it wants to
> boot - whether it's a uncompressed vmlinux or a compressed vmlinux.gz
>
> In the ARM32 code base we had the Makefile rule to generate a uImage automatically (see [2]), with ARM64
> now, I am forced to compress the vmlinux and put a u-boot header on top of it using 'mkimage' and then
> feed the 'uImage' thus generated to the u-boot.
>
> So, it would be good if the ARM64 Makefile can mimic the ARM32 one in this aspect - now that we have
> a working u-boot capable of booting vanilla linux on an ARMv8 foundation model.

No, there should be no uImage target on arm64. Newer u-boots can
already boot an ARM zImage (through the bootz command), and should be
enhanced as needed to boot a vmlinux if needed.

Bringing in uImage as a target on ARM might have been a mistake, since
now people want to bring in new targets for whatever boot format they
happen to want. So it's a good idea to keep arm64 from going down the
same path and stick to the native, raw, formats.


-Olof

AArch64 kernel image decompression

[Arnd Bergmann]

On Tuesday 14 January 2014, Olof Johansson wrote:
> On Mon, Jan 13, 2014 at 7:52 PM, bhupesh.sharma at freescale.com
> <bhupesh.sharma@freescale.com> wrote:
> 
> > AFAIK, u-boot requires a u-boot specific header (see [1]) on top of any image it wants to
> > boot - whether it's a uncompressed vmlinux or a compressed vmlinux.gz
> >
> > In the ARM32 code base we had the Makefile rule to generate a uImage automatically (see [2]), with ARM64
> > now, I am forced to compress the vmlinux and put a u-boot header on top of it using 'mkimage' and then
> > feed the 'uImage' thus generated to the u-boot.
> >
> > So, it would be good if the ARM64 Makefile can mimic the ARM32 one in this aspect - now that we have
> > a working u-boot capable of booting vanilla linux on an ARMv8 foundation model.
>
> No, there should be no uImage target on arm64. Newer u-boots can
> already boot an ARM zImage (through the bootz command), and should be
> enhanced as needed to boot a vmlinux if needed.

Right. We've been trying to deprecate the uImage target on arm32 for a while,
but it's hard when all the documentation points to it. Hopefully not having
it on arm64 will teach people about the fact that it's not really needed
anyway.

Note that uImage is fundamentally incompatible with multiplatform kernels,
since it hardcodes the load address in the image file.

I thought we could already boot both zImage and vmlinux on arm32. I have
to correct myself about saying we should boot a vmlinux on arm64 though:
the documented interface is booting Image, which includes a Linux-defined
header, or Image.gz.
 
> Bringing in uImage as a target on ARM might have been a mistake, since
> now people want to bring in new targets for whatever boot format they
> happen to want. So it's a good idea to keep arm64 from going down the
> same path and stick to the native, raw, formats.

Yes. Not sure if there was much choice about uImage at the time arm32
u-boot was done, and multiplatform wasn't on the radar back then, but
we should have made the switch much earlier.

	Arnd

AArch64 kernel image decompression

[Russell King - ARM Linux]

On Tue, Jan 14, 2014 at 07:14:51AM +0100, Arnd Bergmann wrote:
> I thought we could already boot both zImage and vmlinux on arm32.

We can now do with zImage, but it requires a u-boot which supports the
bootz command.  All the platforms rigged up to my test system with the
exception of the LDP3430 all support bootz, and use it, and I notice
that new platforms such as the now released cubox-i and hummingboard
support bootz right from the start, since it's the only sensible way
to boot a multiplatform kernel with a dt blob.

-- 
FTTC broadband for 0.8mile line: 5.8Mbps down 500kbps up.  Estimation
in database were 13.1 to 19Mbit for a good line, about 7.5+ for a bad.
Estimate before purchase was "up to 13.2Mbit".

AArch64 kernel image decompression

[Stephen Warren]

On 01/13/2014 11:14 PM, Arnd Bergmann wrote:
...
> Note that uImage is fundamentally incompatible with multiplatform kernels,
> since it hardcodes the load address in the image file.

FWIW, with "mkimage -T kernel_noload", that's no longer true.

AArch64 kernel image decompression

[Marek Vasut]

On Monday, January 13, 2014 at 12:42:41 PM, Arnd Bergmann wrote:
> On Monday 13 January 2014 11:29:20 bhupesh.sharma at freescale.com wrote:
> > With the u-boot bootloader also having AArch64 support now (see [1]), it
> > would make sense to add the uImage and decompressor support to the
> > AArch64 Makefile in Linux kernel as well, so that u-boot can also boot
> > uImages for AArch64 systems out-off-box.
> > 
> > [1].
> > http://git.denx.de/?p=u-boot/u-boot-arm.git;a=shortlog;h=refs/heads/aarc
> > h64
> 
> u-boot should never have to use a uImage on arm64, since it already
> supports booting a vmlinux file.

Just a side-note. The benefit of uImage (resp. please use fitImage, as it's more 
modern) is that it is protected by checksum against possible corruption. 
Besides, U-Boot can do the decompression of uImage payload, so there's no need 
to have decompressor in arm64 kernel.

Best regards,
Marek Vasut

AArch64 kernel image decompression

[bhupesh.sharma at freescale.com]

> -----Original Message-----
> From: Marek Vasut [mailto:marex at denx.de]
> Sent: Tuesday, January 14, 2014 6:21 PM
> To: linux-arm-kernel at lists.infradead.org
> Cc: Arnd Bergmann; Sharma Bhupesh-B45370; 'Catalin Marinas'; 'Josh
> Cartwright'; 'Will Deacon'; Wolfgang Denk
> Subject: Re: AArch64 kernel image decompression
> 
> On Monday, January 13, 2014 at 12:42:41 PM, Arnd Bergmann wrote:
> > On Monday 13 January 2014 11:29:20 bhupesh.sharma at freescale.com wrote:
> > > With the u-boot bootloader also having AArch64 support now (see
> > > [1]), it would make sense to add the uImage and decompressor support
> > > to the
> > > AArch64 Makefile in Linux kernel as well, so that u-boot can also
> > > boot uImages for AArch64 systems out-off-box.
> > >
> > > [1].
> > > http://git.denx.de/?p=u-boot/u-boot-arm.git;a=shortlog;h=refs/heads/
> > > aarc
> > > h64
> >
> > u-boot should never have to use a uImage on arm64, since it already
> > supports booting a vmlinux file.
> 
> Just a side-note. The benefit of uImage (resp. please use fitImage, as
> it's more
> modern) is that it is protected by checksum against possible corruption.
> Besides, U-Boot can do the decompression of uImage payload, so there's no
> need to have decompressor in arm64 kernel.
> 

I agree with Marek. Usually for non-cryptographic-secure (key based cryptography) boot
method the CRC protection is usually the best possible protection against Image
corruption. CRC check is usually used in a number of field-deployed solutions to ensure
linux image sanity. 

Regards,
Bhupesh

AArch64 kernel image decompression

[Wolfgang Denk]

Dear Bhupesh,

In message <819346bb83b04348a266643afd5bd6f6@BN1PR03MB220.namprd03.prod.outlook.com> you wrote:
>
> I agree with Marek. Usually for non-cryptographic-secure (key based cryptography) boot
> method the CRC protection is usually the best possible protection against Image
> corruption. CRC check is usually used in a number of field-deployed solutions to ensure
> linux image sanity. 

The FIT image approach as suggested by Marek also allows for secure
boot methods (like cryptographically signed images); this is used in
production for several systems, including the Google Chromebook.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Each honest calling, each walk of life, has its own  elite,  its  own
aristocracy based on excellence of performance. - James Bryant Conant

AArch64 kernel image decompression

[Olof Johansson]

On Tue, Jan 14, 2014 at 5:47 AM, Wolfgang Denk <wd@denx.de> wrote:
> Dear Bhupesh,
>
> In message <819346bb83b04348a266643afd5bd6f6@BN1PR03MB220.namprd03.prod.outlook.com> you wrote:
>>
>> I agree with Marek. Usually for non-cryptographic-secure (key based cryptography) boot
>> method the CRC protection is usually the best possible protection against Image
>> corruption. CRC check is usually used in a number of field-deployed solutions to ensure
>> linux image sanity.
>
> The FIT image approach as suggested by Marek also allows for secure
> boot methods (like cryptographically signed images); this is used in
> production for several systems, including the Google Chromebook.

This can very easily be misunderstood so I would like to clarify:
Chromebooks do not use the fitImage version of signatures and
verification, it is done independently from a separate library that
wraps either a zImage (x86) or a fitImage (ARM).


-Olof

AArch64 kernel image decompression

[Wolfgang Denk]

Dear Olof,

In message <CAOesGMj0HPWD=Mavt1ut+Oze4KQZHjhHmcKTcdKELaF=Or8XsA@mail.gmail.com> you wrote:
>
> > The FIT image approach as suggested by Marek also allows for secure
> > boot methods (like cryptographically signed images); this is used in
> > production for several systems, including the Google Chromebook.
> 
> This can very easily be misunderstood so I would like to clarify:
> Chromebooks do not use the fitImage version of signatures and
> verification, it is done independently from a separate library that
> wraps either a zImage (x86) or a fitImage (ARM).

This is not what I'm talking about.  What I mean is the support for
Verified Boot (see [1]) and FIT Signature Verification (see [2]) which
is done with plain FIT images, without additional wrapping.

[1] http://git.denx.de/?p=u-boot.git;a=blob;f=doc/uImage.FIT/verified-boot.txt
[2] http://git.denx.de/?p=u-boot.git;a=blob;f=doc/uImage.FIT/signature.txt

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Don't hit a man when he's down - kick him; it's easier.

AArch64 kernel image decompression

[Olof Johansson]

Hi,

On Tue, Jan 14, 2014 at 10:53 AM, Wolfgang Denk <wd@denx.de> wrote:
> Dear Olof,
>
> In message <CAOesGMj0HPWD=Mavt1ut+Oze4KQZHjhHmcKTcdKELaF=Or8XsA@mail.gmail.com> you wrote:
>>
>> > The FIT image approach as suggested by Marek also allows for secure
>> > boot methods (like cryptographically signed images); this is used in
>> > production for several systems, including the Google Chromebook.
>>
>> This can very easily be misunderstood so I would like to clarify:
>> Chromebooks do not use the fitImage version of signatures and
>> verification, it is done independently from a separate library that
>> wraps either a zImage (x86) or a fitImage (ARM).
>
> This is not what I'm talking about.  What I mean is the support for
> Verified Boot (see [1]) and FIT Signature Verification (see [2]) which
> is done with plain FIT images, without additional wrapping.
>
> [1] http://git.denx.de/?p=u-boot.git;a=blob;f=doc/uImage.FIT/verified-boot.txt
> [2] http://git.denx.de/?p=u-boot.git;a=blob;f=doc/uImage.FIT/signature.txt

The incorrect part of your statement was the claim that it was used
for secure boot on ARM Chromebooks. It is not -- the u-boot
implementation is completely different from what is used there.


-Olof

AArch64 kernel image decompression

[Josh Cartwright]

On Mon, Jan 13, 2014 at 11:17:33AM +0000, Will Deacon wrote:
> On Fri, Jan 10, 2014 at 05:04:36PM +0000, Josh Cartwright wrote:
> > Hey Will-
>
> Hi Josh,
>
> > Scanning the Documentation/arm64/booting.txt document, I came across
> > this, listing an optional step a bootloader might take:
> >
> > "The AArch64 kernel does not currently provide a decompressor and
> > therefore requires decompression (gzip, etc.) to be performed by the
> > boot loader if a compressed Image target (e.g. Image.gz) is used."
> >
> > Do you envision that, if a decompressor is desired, it will always be
> > the responsibility of a bootloader to implement it?
>
> That's certainly our current stance, yes. It also simplifies the booting
> code and things like EFI stub, so I'd be reluctant to consider adding a
> decompressor to the arm64 kernel without a compelling use-case.

Okay, thanks for the clarification.

> > In other words, would you accept patches adding support for building
> > zImages (and others) for AArch64?
> 
> Are these patches actually needed, or are you trying to find a project
> involving the arm64 kernel?

Really, I'm just trying to get a better handle on the difference in
booting requirements for 32-bit and 64-bit ARM kernels.  I haven't yet
determined whether compression is even necessary for our usecase.

If we did find it necessary and you had said "yes, we'd like in-kernel
decompression, but we haven't gotten around to it yet", then we'd likely
go down the path of implementing it.  Anyway, I'm probably getting too
far ahead of myself. :)

Thanks,
   Josh

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation