Re: [PATCH 2/2] arm: fix kernel image size

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

Hi,

* Russell King <rmk-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160617 12:52]:
> If we want to assume that the compressed image will expand by a maximum
> of 4x, we actually need to reserve 5x the space, since we need to keep
> a copy of the compessed image around while decompressing.

Looks like 5x is not enough with omap2plus_defconfig at least..
This one needs to be set to 6x for that. Maybe we should just make
it go all the way to 11 :)

Otherwise we get this without initrd:

kernel: 0xb6c2d008 kernel_size: 0x3a4418
MEMORY RANGES
0000000080000000-00000000bfdfffff (0)
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6c2d008
segment[0].bufsz = 0x3a4418
segment[0].mem   = 0x80008000
segment[0].memsz = 0x3a5000
segment[1].buf   = 0xadf80
segment[1].bufsz = 0xe000
segment[1].mem   = 0x8123f000
segment[1].memsz = 0xe000
[   32.886077] kexec_core: Starting new kernel
[   32.890502] Disabling non-boot CPUs ...
[   32.926696] CPU1: shutdown
[   32.991882] Bye!
Warning: Neither atags nor dtb found

And we get this with initrd:

kernel: 0xb6bf4008 kernel_size: 0x3a4418
MEMORY RANGES
0000000080000000-00000000fedfffff (0)
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 3
segment[0].buf   = 0xb6bf4008
segment[0].bufsz = 0x3a4418
segment[0].mem   = 0x80008000
segment[0].memsz = 0x3a5000
segment[1].buf   = 0xb6686008
segment[1].bufsz = 0x56dba2
segment[1].mem   = 0x8123e000
segment[1].memsz = 0x56e000
segment[2].buf   = 0xadf80
segment[2].bufsz = 0xd080
segment[2].mem   = 0x817ac000
segment[2].memsz = 0xe000
...
INITRD: 0x8123e000+0x0056dba2 overlaps in-use memory region - disabling initrd

Regards,

Tony

Re: [PATCH 2/2] arm: fix kernel image size

[Russell King - ARM Linux]

On Tue, Jun 21, 2016 at 12:43:21AM -0700, Tony Lindgren wrote:
> Hi,
> 
> * Russell King <rmk-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160617 12:52]:
> > If we want to assume that the compressed image will expand by a maximum
> > of 4x, we actually need to reserve 5x the space, since we need to keep
> > a copy of the compessed image around while decompressing.
> 
> Looks like 5x is not enough with omap2plus_defconfig at least..
> This one needs to be set to 6x for that. Maybe we should just make
> it go all the way to 11 :)

That's not really what this commit is about - it's about making the
code consistent with the comments in the file.  I could have instead
changed the comment saying "4x" to "3x" instead.

There isn't really an answer which will always work for this problem,
as I've already detailed in a previous thread discussing the issue.
Adding a zImage header don't fix the problem.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

* Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 02:50]:
> On Tue, Jun 21, 2016 at 12:43:21AM -0700, Tony Lindgren wrote:
> > Hi,
> > 
> > * Russell King <rmk-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160617 12:52]:
> > > If we want to assume that the compressed image will expand by a maximum
> > > of 4x, we actually need to reserve 5x the space, since we need to keep
> > > a copy of the compessed image around while decompressing.
> > 
> > Looks like 5x is not enough with omap2plus_defconfig at least..
> > This one needs to be set to 6x for that. Maybe we should just make
> > it go all the way to 11 :)
> 
> That's not really what this commit is about - it's about making the
> code consistent with the comments in the file.  I could have instead
> changed the comment saying "4x" to "3x" instead.
> 
> There isn't really an answer which will always work for this problem,
> as I've already detailed in a previous thread discussing the issue.
> Adding a zImage header don't fix the problem.

Well how about we change both comments and code to use a value
that works?

Tony

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

* Tony Lindgren <tony-4v6yS6AI5VpBDgjK7y7TUQ@public.gmane.org> [160621 03:41]:
> * Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 02:50]:
> > 
> > There isn't really an answer which will always work for this problem,
> > as I've already detailed in a previous thread discussing the issue.
> > Adding a zImage header don't fix the problem.
> 
> Well how about we change both comments and code to use a value
> that works?

Maybe zImage size + MAX_RODATA_SZ + 4x zImage size?

Then the MAX_RODATA_SZ could be 2 or 4 MB or whatever we
think is sufficient to kick the can until we have a better
solution.

Regards,

Tony

Re: [PATCH 2/2] arm: fix kernel image size

[Russell King - ARM Linux]

On Tue, Jun 21, 2016 at 03:57:20AM -0700, Tony Lindgren wrote:
> * Tony Lindgren <tony-4v6yS6AI5VpBDgjK7y7TUQ@public.gmane.org> [160621 03:41]:
> > * Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 02:50]:
> > > 
> > > There isn't really an answer which will always work for this problem,
> > > as I've already detailed in a previous thread discussing the issue.
> > > Adding a zImage header don't fix the problem.
> > 
> > Well how about we change both comments and code to use a value
> > that works?
> 
> Maybe zImage size + MAX_RODATA_SZ + 4x zImage size?
> 
> Then the MAX_RODATA_SZ could be 2 or 4 MB or whatever we
> think is sufficient to kick the can until we have a better
> solution.

Maybe if you give some details about your failing case - you omitted any
details about the uncompressed image size...

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

* Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 08:46]:
> On Tue, Jun 21, 2016 at 03:57:20AM -0700, Tony Lindgren wrote:
> > 
> > Maybe zImage size + MAX_RODATA_SZ + 4x zImage size?
> > 
> > Then the MAX_RODATA_SZ could be 2 or 4 MB or whatever we
> > think is sufficient to kick the can until we have a better
> > solution.
> 
> Maybe if you give some details about your failing case - you omitted any
> details about the uncompressed image size...

Looks like we currently have this:

Memory: 2033088K/2095100K available (7376K kernel code, 784K rwdata,
2312K rodata, 1024K init, 8117K bss, 45628K reserved,
16384K cma-reserved, 1292284K highmem)
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
    lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0a762a4   (10681 kB)
      .init : 0xc0b00000 - 0xc0c00000   (1024 kB)
      .data : 0xc0c00000 - 0xc0cc411c   ( 785 kB)
       .bss : 0xc0cc6000 - 0xc14b37f8   (8118 kB)

Regards,

Tony

Re: [PATCH 2/2] arm: fix kernel image size

[Russell King - ARM Linux]

On Tue, Jun 21, 2016 at 09:55:25AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 08:46]:
> > On Tue, Jun 21, 2016 at 03:57:20AM -0700, Tony Lindgren wrote:
> > > 
> > > Maybe zImage size + MAX_RODATA_SZ + 4x zImage size?
> > > 
> > > Then the MAX_RODATA_SZ could be 2 or 4 MB or whatever we
> > > think is sufficient to kick the can until we have a better
> > > solution.
> > 
> > Maybe if you give some details about your failing case - you omitted any
> > details about the uncompressed image size...
> 
> Looks like we currently have this:
> 
> Memory: 2033088K/2095100K available (7376K kernel code, 784K rwdata,
> 2312K rodata, 1024K init, 8117K bss, 45628K reserved,
> 16384K cma-reserved, 1292284K highmem)
> Virtual kernel memory layout:
>     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
>     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
>     vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
>     lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
>     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
>     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
>       .text : 0xc0008000 - 0xc0a762a4   (10681 kB)
>       .init : 0xc0b00000 - 0xc0c00000   (1024 kB)
>       .data : 0xc0c00000 - 0xc0cc411c   ( 785 kB)
>        .bss : 0xc0cc6000 - 0xc14b37f8   (8118 kB)

So that's 20.7MB, and the zImage was 3.6MB.  You're getting an
expansion ratio of 5.7x.

To fix that, we'd need to up it to 7x, but the problem with upping
it in this way is that it increases the requirements for the
crashdump region too.  We can play with this number, but there will
always be cases where it doesn't work - either because the ratio is
too big or too small.

By way of illustration, "zImage size + MAX_RODATA_SZ + 4x zImage size"
doesn't even work for this case, since you need about 25MB.  Your
calculation comes out at 22MB, which is 3MB short.  Not only that, but
it introduces (from what I can see) an irrelevant fudge factor of
"MAX_RODATA_SZ" which has no basis in what's really going on here, and
I'm left wondering what "MAX_RODATA_SZ" is actually referring to.

So... I'm of the opinion that we shouldn't play with it - but instead
try to come up with a solution which *doesn't* involve teaching kexec
a whole load of internals about how the ARM kernel booting happens.

What's more worrying to me at the moment, though, is that the kexec code
only tries to find memory for the actual "kernel" size, not the actual
space required to decompress the kernel.  It could find a chunk of
memory large enough to fit the kernel image to be loaded, but is not big
enough to allow its decompression.  kexec is really quite a mess on
ARM. :(

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

* Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 14:54]:
> So that's 20.7MB, and the zImage was 3.6MB.  You're getting an
> expansion ratio of 5.7x.
> 
> To fix that, we'd need to up it to 7x, but the problem with upping
> it in this way is that it increases the requirements for the
> crashdump region too.  We can play with this number, but there will
> always be cases where it doesn't work - either because the ratio is
> too big or too small.

OK

> By way of illustration, "zImage size + MAX_RODATA_SZ + 4x zImage size"
> doesn't even work for this case, since you need about 25MB.  Your
> calculation comes out at 22MB, which is 3MB short.  Not only that, but
> it introduces (from what I can see) an irrelevant fudge factor of
> "MAX_RODATA_SZ" which has no basis in what's really going on here, and
> I'm left wondering what "MAX_RODATA_SZ" is actually referring to.

Well I thinking the section alignment split might give us some
known size for rodata, but looking at it more it's only with
DEBUG_ALIGN_RODATA. If we can't make any assumptions about the
size of the rodata, then no point adding it.

> So... I'm of the opinion that we shouldn't play with it - but instead
> try to come up with a solution which *doesn't* involve teaching kexec
> a whole load of internals about how the ARM kernel booting happens.
> 
> What's more worrying to me at the moment, though, is that the kexec code
> only tries to find memory for the actual "kernel" size, not the actual
> space required to decompress the kernel.  It could find a chunk of
> memory large enough to fit the kernel image to be loaded, but is not big
> enough to allow its decompression.  kexec is really quite a mess on
> ARM. :(

How about we check the size of RAM available, and if there is plenty
of RAM we use a safe compression ratio of 8. If RAM is a problem,
we could make the compression ratio smaller and warn about it. And
we could also allow passing the compression ratio to kexec as an
option.

I forgot if we still have some 128MB limits too..

Regards,

Tony

Re: [PATCH 2/2] arm: fix kernel image size

[Russell King - ARM Linux]

On Wed, Jun 22, 2016 at 12:36:16AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160621 14:54]:
> > So that's 20.7MB, and the zImage was 3.6MB.  You're getting an
> > expansion ratio of 5.7x.
> > 
> > To fix that, we'd need to up it to 7x, but the problem with upping
> > it in this way is that it increases the requirements for the
> > crashdump region too.  We can play with this number, but there will
> > always be cases where it doesn't work - either because the ratio is
> > too big or too small.
> 
> OK
> 
> > By way of illustration, "zImage size + MAX_RODATA_SZ + 4x zImage size"
> > doesn't even work for this case, since you need about 25MB.  Your
> > calculation comes out at 22MB, which is 3MB short.  Not only that, but
> > it introduces (from what I can see) an irrelevant fudge factor of
> > "MAX_RODATA_SZ" which has no basis in what's really going on here, and
> > I'm left wondering what "MAX_RODATA_SZ" is actually referring to.
> 
> Well I thinking the section alignment split might give us some
> known size for rodata, but looking at it more it's only with
> DEBUG_ALIGN_RODATA. If we can't make any assumptions about the
> size of the rodata, then no point adding it.

I think you're just getting completely confused about what the problem
is, due to the naming of the config option.

It's got nothing to do with the size of the read-only data.  Really.

When CONFIG_DEBUG_RODATA is enabled, we align the text, read-only data,
data, and so on to 1MB boundaries so that we can change the permissions
of the sections to enforce the properties of the various ELF segments.
This padding massively inflates the size of the run-time kernel image,
and can result in megabytes of wastage (if we're as little as one byte
into 1MB, we have to round up to the next megabyte.)

The padding, being a string of zeros, is very compressable, and this
increases the compression ratio.

The BSS is not included in the compressed image, but needs to be taken
account of when avoiding overwriting the DTB.

> How about we check the size of RAM available, and if there is plenty
> of RAM we use a safe compression ratio of 8. If RAM is a problem,
> we could make the compression ratio smaller and warn about it. And
> we could also allow passing the compression ratio to kexec as an
> option.

You do know that you can already pass the "size" of the kernel to be
reserved to kexec.  --image-size.  It's not documented in --help, but
it is there (which is another annoying thing about ARM kexec...)

> I forgot if we still have some 128MB limits too..

We do.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [PATCH 2/2] arm: fix kernel image size

[Tony Lindgren]

* Russell King - ARM Linux <linux-I+IVW8TIWO2tmTQ+vhA3Yw@public.gmane.org> [160622 01:32]:
> When CONFIG_DEBUG_RODATA is enabled, we align the text, read-only data,
> data, and so on to 1MB boundaries so that we can change the permissions
> of the sections to enforce the properties of the various ELF segments.
> This padding massively inflates the size of the run-time kernel image,
> and can result in megabytes of wastage (if we're as little as one byte
> into 1MB, we have to round up to the next megabyte.)

Yes understood, and looks like we cannot make any assumptions about
what the alignment might be.

> The padding, being a string of zeros, is very compressable, and this
> increases the compression ratio.
> 
> The BSS is not included in the compressed image, but needs to be taken
> account of when avoiding overwriting the DTB.
> 
> > How about we check the size of RAM available, and if there is plenty
> > of RAM we use a safe compression ratio of 8. If RAM is a problem,
> > we could make the compression ratio smaller and warn about it. And
> > we could also allow passing the compression ratio to kexec as an
> > option.
> 
> You do know that you can already pass the "size" of the kernel to be
> reserved to kexec.  --image-size.  It's not documented in --help, but
> it is there (which is another annoying thing about ARM kexec...)

Oh ok that's nice.

Regards,

Tony