[PATCH] s390/boot: Add 'alloc' to info.bin .vmlinux.info section flags

[PATCH] s390/boot: Add 'alloc' to info.bin .vmlinux.info section flags

[Nathan Chancellor]

When attempting to boot a kernel compiled with OBJCOPY=llvm-objcopy,
there is a crash right at boot:

  Out of memory allocating 6d7800 bytes 8 aligned in range 0:20000000
  Reserved memory ranges:
  0000000000000000 a394c3c30d90cdaf DECOMPRESSOR
  Usable online memory ranges (info source: sclp read info [3]):
  0000000000000000 0000000020000000
  Usable online memory total: 20000000 Reserved: a394c3c30d90cdaf Free: 0
  Call Trace:
  (sp:0000000000033e90 [<0000000000012fbc>] physmem_alloc_top_down+0x5c/0x104)
   sp:0000000000033f00 [<0000000000011d56>] startup_kernel+0x3a6/0x77c
   sp:0000000000033f60 [<00000000000100f4>] startup_normal+0xd4/0xd4

GNU objcopy does not have any issues. Looking at differences between the
object files in each build reveals info.bin does not get properly
populated with llvm-objcopy, which results in an empty .vmlinux.info
section.

  $ file {gnu,llvm}-objcopy/arch/s390/boot/info.bin
  gnu-objcopy/arch/s390/boot/info.bin:  data
  llvm-objcopy/arch/s390/boot/info.bin: empty

  $ llvm-readelf --section-headers {gnu,llvm}-objcopy/arch/s390/boot/vmlinux | rg 'File:|\.vmlinux\.info|\.decompressor\.syms'
  File: gnu-objcopy/arch/s390/boot/vmlinux
    [12] .vmlinux.info     PROGBITS        0000000000034000 035000 000078 00  WA  0   0  1
    [13] .decompressor.syms PROGBITS       0000000000034078 035078 000b00 00  WA  0   0  1
  File: llvm-objcopy/arch/s390/boot/vmlinux
    [12] .vmlinux.info     PROGBITS        0000000000034000 035000 000000 00  WA  0   0  1
    [13] .decompressor.syms PROGBITS       0000000000034000 035000 000b00 00  WA  0   0  1

Ulrich points out that llvm-objcopy only copies sections marked as alloc
with a binary output target, whereas the .vmlinux.info section is only
marked as load. Add 'alloc' in addition to 'load', so that both objcopy
implementations work properly:

  $ file {gnu,llvm}-objcopy/arch/s390/boot/info.bin
  gnu-objcopy/arch/s390/boot/info.bin:  data
  llvm-objcopy/arch/s390/boot/info.bin: data

  $ llvm-readelf --section-headers {gnu,llvm}-objcopy/arch/s390/boot/vmlinux | rg 'File:|\.vmlinux\.info|\.decompressor\.syms'
  File: gnu-objcopy/arch/s390/boot/vmlinux
    [12] .vmlinux.info     PROGBITS        0000000000034000 035000 000078 00  WA  0   0  1
    [13] .decompressor.syms PROGBITS       0000000000034078 035078 000b00 00  WA  0   0  1
  File: llvm-objcopy/arch/s390/boot/vmlinux
    [12] .vmlinux.info     PROGBITS        0000000000034000 035000 000078 00  WA  0   0  1
    [13] .decompressor.syms PROGBITS       0000000000034078 035078 000b00 00  WA  0   0  1

Closes: https://github.com/ClangBuiltLinux/linux/issues/1996
Link: https://github.com/llvm/llvm-project/commit/3c02cb7492fc78fb678264cebf57ff88e478e14f
Suggested-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
No fixes tag because I have not tested to see if there are other issues
on older kernel versions yet (so I don't think prioritizing a backport
is necessary yet) but I think commit 5a2e1853d689 ("s390: avoid vmlinux
segments overlap") introduced this.
---
 arch/s390/boot/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index ace0bda1ad24..d40135efdec4 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -94,7 +94,7 @@ OBJCOPYFLAGS_syms.o := -I binary -O elf64-s390 -B s390:64-bit --rename-section .
 $(obj)/syms.o: $(obj)/syms.bin FORCE
 	$(call if_changed,objcopy)
 
-OBJCOPYFLAGS_info.bin := -O binary --only-section=.vmlinux.info --set-section-flags .vmlinux.info=load
+OBJCOPYFLAGS_info.bin := -O binary --only-section=.vmlinux.info --set-section-flags .vmlinux.info=alloc,load
 $(obj)/info.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 

---
base-commit: 03325e9b64c48313527f0373c4688d393b1edaf3
change-id: 20240216-s390-fix-boot-with-llvm-objcopy-019a7638e5bf

Best regards,
-- 
Nathan Chancellor <nathan@kernel.org>

Re: [PATCH] s390/boot: Add 'alloc' to info.bin .vmlinux.info section flags

[Heiko Carstens]

On Fri, Feb 16, 2024 at 12:55:53PM -0700, Nathan Chancellor wrote:
> When attempting to boot a kernel compiled with OBJCOPY=llvm-objcopy,
> there is a crash right at boot:
> 
>   Out of memory allocating 6d7800 bytes 8 aligned in range 0:20000000
>   Reserved memory ranges:
>   0000000000000000 a394c3c30d90cdaf DECOMPRESSOR
>   Usable online memory ranges (info source: sclp read info [3]):
>   0000000000000000 0000000020000000
>   Usable online memory total: 20000000 Reserved: a394c3c30d90cdaf Free: 0
>   Call Trace:
>   (sp:0000000000033e90 [<0000000000012fbc>] physmem_alloc_top_down+0x5c/0x104)
>    sp:0000000000033f00 [<0000000000011d56>] startup_kernel+0x3a6/0x77c
>    sp:0000000000033f60 [<00000000000100f4>] startup_normal+0xd4/0xd4
> 
> GNU objcopy does not have any issues. Looking at differences between the
> object files in each build reveals info.bin does not get properly
> populated with llvm-objcopy, which results in an empty .vmlinux.info
> section.
...
> Closes: https://github.com/ClangBuiltLinux/linux/issues/1996
> Link: https://github.com/llvm/llvm-project/commit/3c02cb7492fc78fb678264cebf57ff88e478e14f
> Suggested-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

Thanks a lot, applied!

However when building the kernel with "LLVM=1" I can see
something else which looks like an llvm-objdump bug to me:

$make LLVM=1 bzImage
...
  SECTCMP .boot.data
llvm-objdump: warning: section '.boot.data' mentioned in a -j/--section option, but not found in any input file

This works without warning with GNU objcopy, and actually the output
is also different:

$ objdump -v
GNU objdump (GNU Binutils for Ubuntu) 2.41

$ objdump -t -j .boot.data arch/s390/boot/vmlinux

arch/s390/boot/vmlinux:     file format elf64-s390

SYMBOL TABLE:
0000000000020000 l     O .boot.data     0000000000003000 sclp_info_sccb
00000000000240e0 l     O .boot.data     0000000000000001 sclp_info_sccb_valid
0000000000023010 g     O .boot.data     0000000000000008 ident_map_size
0000000000023018 g     O .boot.data     00000000000010c8 physmem_info
00000000000250e2 g       .boot.data     0000000000000000 __boot_data_end
0000000000020000 g       .boot.data     0000000000000000 __boot_data_start
00000000000240e2 g     O .boot.data     0000000000001000 early_command_line
0000000000023008 g     O .boot.data     0000000000000008 early_ipl_comp_list_size
0000000000023000 g     O .boot.data     0000000000000008 early_ipl_comp_list_addr

While with llvm-copy:

$ llvm-objdump --version
LLVM (http://llvm.org/):
  LLVM version 19.0.0git

$ llvm-objdump -t -j .boot.data arch/s390/boot/vmlinux

arch/s390/boot/vmlinux: file format elf64-s390

SYMBOL TABLE:
0000000000000200 l       .head.text     0000000000000000 ipl_start
0000000000010020 l       .head.text     00000000000000d4 startup_normal
00000000000101b0 l       .head.text     00000000000000b2 startup_kdump
0000000000010280 l       .head.text     000000000000005a startup_pgm_check_handler
000000000001025c l       .head.text     0000000000000000 startup_kdump_relocated
0000000000000000 l    df *ABS*  0000000000000000 als.c
000000000001e040 l     O .rodata        0000000000000018 als
000000000001f6f0 l     O .data  0000000000000050 print_missing_facilities.als_str
0000000000011800 l     F .text  00000000000000e2 print_machine_type
...
0000000000020000 l     O .boot.data     0000000000003000 sclp_info_sccb
00000000000240e0 l     O .boot.data     0000000000000001 sclp_info_sccb_valid
... and so on (everything is dumped)
llvm-objdump: warning: section '.boot.data' mentioned in a -j/--section option, but not found in any input file

So somehow llvmdump's "-j/--section" option doesn not seem to work.

Re: [PATCH] s390/boot: Add 'alloc' to info.bin .vmlinux.info section flags

[Nathan Chancellor]

Hi Heiko,

On Mon, Feb 19, 2024 at 12:32:48PM +0100, Heiko Carstens wrote:
> On Fri, Feb 16, 2024 at 12:55:53PM -0700, Nathan Chancellor wrote:
> > When attempting to boot a kernel compiled with OBJCOPY=llvm-objcopy,
> > there is a crash right at boot:
> > 
> >   Out of memory allocating 6d7800 bytes 8 aligned in range 0:20000000
> >   Reserved memory ranges:
> >   0000000000000000 a394c3c30d90cdaf DECOMPRESSOR
> >   Usable online memory ranges (info source: sclp read info [3]):
> >   0000000000000000 0000000020000000
> >   Usable online memory total: 20000000 Reserved: a394c3c30d90cdaf Free: 0
> >   Call Trace:
> >   (sp:0000000000033e90 [<0000000000012fbc>] physmem_alloc_top_down+0x5c/0x104)
> >    sp:0000000000033f00 [<0000000000011d56>] startup_kernel+0x3a6/0x77c
> >    sp:0000000000033f60 [<00000000000100f4>] startup_normal+0xd4/0xd4
> > 
> > GNU objcopy does not have any issues. Looking at differences between the
> > object files in each build reveals info.bin does not get properly
> > populated with llvm-objcopy, which results in an empty .vmlinux.info
> > section.
> ...
> > Closes: https://github.com/ClangBuiltLinux/linux/issues/1996
> > Link: https://github.com/llvm/llvm-project/commit/3c02cb7492fc78fb678264cebf57ff88e478e14f
> > Suggested-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> 
> Thanks a lot, applied!
> 
> However when building the kernel with "LLVM=1" I can see

Thanks for testing, we are so close!

> something else which looks like an llvm-objdump bug to me:
> 
> $make LLVM=1 bzImage
> ...
>   SECTCMP .boot.data
> llvm-objdump: warning: section '.boot.data' mentioned in a -j/--section option, but not found in any input file
> 
> This works without warning with GNU objcopy, and actually the output
> is also different:
> 
> $ objdump -v
> GNU objdump (GNU Binutils for Ubuntu) 2.41
> 
> $ objdump -t -j .boot.data arch/s390/boot/vmlinux
> 
> arch/s390/boot/vmlinux:     file format elf64-s390
> 
> SYMBOL TABLE:
> 0000000000020000 l     O .boot.data     0000000000003000 sclp_info_sccb
> 00000000000240e0 l     O .boot.data     0000000000000001 sclp_info_sccb_valid
> 0000000000023010 g     O .boot.data     0000000000000008 ident_map_size
> 0000000000023018 g     O .boot.data     00000000000010c8 physmem_info
> 00000000000250e2 g       .boot.data     0000000000000000 __boot_data_end
> 0000000000020000 g       .boot.data     0000000000000000 __boot_data_start
> 00000000000240e2 g     O .boot.data     0000000000001000 early_command_line
> 0000000000023008 g     O .boot.data     0000000000000008 early_ipl_comp_list_size
> 0000000000023000 g     O .boot.data     0000000000000008 early_ipl_comp_list_addr
> 
> While with llvm-copy:
> 
> $ llvm-objdump --version
> LLVM (http://llvm.org/):
>   LLVM version 19.0.0git
> 
> $ llvm-objdump -t -j .boot.data arch/s390/boot/vmlinux
> 
> arch/s390/boot/vmlinux: file format elf64-s390
> 
> SYMBOL TABLE:
> 0000000000000200 l       .head.text     0000000000000000 ipl_start
> 0000000000010020 l       .head.text     00000000000000d4 startup_normal
> 00000000000101b0 l       .head.text     00000000000000b2 startup_kdump
> 0000000000010280 l       .head.text     000000000000005a startup_pgm_check_handler
> 000000000001025c l       .head.text     0000000000000000 startup_kdump_relocated
> 0000000000000000 l    df *ABS*  0000000000000000 als.c
> 000000000001e040 l     O .rodata        0000000000000018 als
> 000000000001f6f0 l     O .data  0000000000000050 print_missing_facilities.als_str
> 0000000000011800 l     F .text  00000000000000e2 print_machine_type
> ...
> 0000000000020000 l     O .boot.data     0000000000003000 sclp_info_sccb
> 00000000000240e0 l     O .boot.data     0000000000000001 sclp_info_sccb_valid
> ... and so on (everything is dumped)
> llvm-objdump: warning: section '.boot.data' mentioned in a -j/--section option, but not found in any input file
> 
> So somehow llvmdump's "-j/--section" option doesn not seem to work.

Re-reading Jordan's response to my initial report about this a couple of
years ago at https://github.com/ClangBuiltLinux/linux/issues/859, I
think I understand the issue now. '-j' / '--section' does work but not
for '-t'.  It works for '-h' and a couple of other flags:

https://github.com/llvm/llvm-project/blob/61a96e5afadd034e7d13126f0e43731bbad7ad89/llvm/test/tools/llvm-objdump/section-filter.test

and because '-t' does not work with '-j', there is a warning because the
'-j' argument values went unhandled:

https://github.com/llvm/llvm-project/blob/61a96e5afadd034e7d13126f0e43731bbad7ad89/llvm/tools/llvm-objdump/llvm-objdump.cpp#L485
https://github.com/llvm/llvm-project/blob/61a96e5afadd034e7d13126f0e43731bbad7ad89/llvm/tools/llvm-objdump/llvm-objdump.cpp#L3672

which is what Jordan's upstream report (https://llvm.org/pr50085) was
about fixing. However, it may not be too hard to fix '-t' to work with
'-j', I intend to take a look at it at some point this week.

However, since I am more of a kernel hacker than I am an LLVM one, I
came up with a potential solution in arch/s390/boot/Makefile, which is
basically just filtering the symbol table manually with grep... it
appears to produce stable results based on a small test Makefile I have
to make sure the output looks sane before running through sha256sum. I'd
be happy to send this as a formal patch if you'd accept it for full
LLVM=1 compatibility with LLVM 19.0.0+ and Linux 6.9+.

Cheers,
Nathan

diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index d40135efdec4..be3655825b4c 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -56,9 +56,9 @@ clean-files += vmlinux.map
 
 quiet_cmd_section_cmp = SECTCMP $*
 define cmd_section_cmp
-	s1=`$(OBJDUMP) -t -j "$*" "$<" | sort | \
+	s1=`$(OBJDUMP) -t "$<" | grep "\s$*\s\+" | sort | \
 		sed -n "/0000000000000000/! s/.*\s$*\s\+//p" | sha256sum`; \
-	s2=`$(OBJDUMP) -t -j "$*" "$(word 2,$^)" | sort | \
+	s2=`$(OBJDUMP) -t "$(word 2,$^)" | grep "\s$*\s\+" | sort | \
 		sed -n "/0000000000000000/! s/.*\s$*\s\+//p" | sha256sum`; \
 	if [ "$$s1" != "$$s2" ]; then \
 		echo "error: section $* differs between $< and $(word 2,$^)" >&2; \

Re: [PATCH] s390/boot: Add 'alloc' to info.bin .vmlinux.info section flags

[Heiko Carstens]

Hi Nathan,

> > [...]
> > So somehow llvmdump's "-j/--section" option doesn not seem to work.
> 
> However, since I am more of a kernel hacker than I am an LLVM one, I
> came up with a potential solution in arch/s390/boot/Makefile, which is
> basically just filtering the symbol table manually with grep... it
> appears to produce stable results based on a small test Makefile I have
> to make sure the output looks sane before running through sha256sum. I'd
> be happy to send this as a formal patch if you'd accept it for full
> LLVM=1 compatibility with LLVM 19.0.0+ and Linux 6.9+.

Ok, please send a formal patch for this approach.