From: Joe Lawrence <joe.lawrence@redhat.com>
To: Sumanth Korikkar <sumanthk@linux.ibm.com>
Cc: linux-s390@vger.kernel.org, hca@linux.ibm.com,
jpoimboe@kernel.org, gor@linux.ibm.com, iii@linux.ibm.com,
agordeev@linux.ibm.com
Subject: Re: [PATCH v2 0/4] s390: compile relocatable kernel with/without fPIE
Date: Tue, 18 Jun 2024 16:37:16 -0400 [thread overview]
Message-ID: <ZnHv/HmiYHoQRkUU@redhat.com> (raw)
In-Reply-To: <20240219132734.22881-1-sumanthk@linux.ibm.com>
On Mon, Feb 19, 2024 at 02:27:30PM +0100, Sumanth Korikkar wrote:
> Hi All,
>
> This is a rebased version of Josh's patch series with a few fixups.
> https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/log/?h=s390
>
> This introduces the capability to compile the s390 relocatable kernel
> with and without the -fPIE option.
>
> When utilizing the kpatch functionality, it is advisable to compile the
> kernel without the -fPIE option. This is particularly important if the
> kernel is built with the -ffunction-sections and -fdata-sections flags.
> The linker imposes a restriction on the number of sections (limited to
> 64k), necessitating the omission of -fPIE.
>
> [1] https://gcc.gnu.org/pipermail/gcc-patches/2023-June/622872.html
> [2] https://gcc.gnu.org/pipermail/gcc-patches/2023-August/625986.html
>
> Gcc recently implemented an optimization [1] for loading symbols without
> explicit alignment, aligning with the IBM Z ELF ABI. This ABI mandates
> symbols to reside on a 2-byte boundary, enabling the use of the larl
> instruction. However, kernel linker scripts may still generate unaligned
> symbols. To address this, a new -munaligned-symbols option has been
> introduced [2] in recent gcc versions. This option has to be used with
> future gcc versions.
>
> Older Clang lacks support for handling unaligned symbols generated
> by kernel linker scripts when the kernel is built without -fPIE. However,
> future versions of Clang will include support for the -munaligned-symbols
> option. When the support is unavailable, compile the kernel with -fPIE
> to maintain the existing behavior.
>
> Patch 1 filters out -munaligned-symbol flag for vdso code. This is beneficial
> when compiling kernel with -fno-PIE and -munaligned-symbols combination.
>
> Patch 2 introduces the 'relocs' tool, which reads the vmlinux file and
> generates a vmlinux.relocs_64 section, containing offsets for all
> R_390_64 relocations.
>
> Patch 3 enables the compilation of a relocatable kernel with or without
> the -fPIE option. It allows for building the relocatable kernel without
> -fPIE. However, if compiler cannot handle unaligned symbols, the kernel
> is built with -fPIE.
>
> Patch 4 handles orphan .rela sections when kernel is built with
> -fno-PIE.
>
> kpatch tools changes:
> * -mno-pic-data-is-text-relative prevents relative addressing between
> code and data. This is needed to avoid relocation error when klp text
> and data are too far apart. kpatch already includes this flag.
> However, with these changes, ARCH_KFLAGS+="-fPIC" should be added to
> s390 kpatch tools, As -mno-pic-data-is-text-relative can be used only
> with -fPIC. The corresponding pull request will be sent to kpatch
> tools.
Hi Sumanth,
I noticed interesting compiler differences when adding -fPIC build
option and not. The difference in resulting output can confuse
kpatch-build when it tries to verify that its reference build (with the
mentioned options, plus --ffunction-sections and -fdata-sections),
doesn't line up closely enough with the original vmlinux source (sans
all these options).
I don't think a kpatch-build PR was ever opened to add "-fPIC", but the
compiler now warns that its required. Have you ever seen optimization /
build output changes when adding this option and if so, were there
additional kpatch-build implications?
Here is a quick example that I stumbled on while investigating the
kpatch-build shadow-newpid.patch integration test that modifies
kernel/fork.c. I couldn't reproduce with the s390 defconfig, but it
shows up when building with RHEL-10 config. Reproducer steps and
disassembly examples follows.
Regards,
-- Joe
-->8-- -->8-- -->8-- -->8-- -->8-- -->8-- -->8-- -->8-- -->8-- -->8--
# Setup - v6.9 with a RHEL-10 config
$ git clone --depth=1 --branch v6.9 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
$ cd linux
$ curl -o .config https://file.rdu.redhat.com/~jolawren/kernel-s390x-rhel.config
$ make olddefconfig
# Build and disassemble kernel/fork.c :: mmput()
$ make -s -j$(nproc) kernel/fork.o
$ objdump -Dr -j.text --disassemble=mmput kernel/fork.o
kernel/fork.o: file format elf64-s390
Disassembly of section .text:
0000000000001090 <mmput>:
1090: c0 04 00 00 00 00 jgnop 1090 <mmput>
1096: a7 18 ff ff lhi %r1,-1
109a: eb 01 21 3c 00 f8 laa %r0,%r1,316(%r2)
10a0: 07 e0 bnor %r0
10a2: ec 08 00 06 01 7e cije %r0,1,10ae <mmput+0x1e>
10a8: c0 f4 00 00 00 00 jg 10a8 <mmput+0x18>
10aa: R_390_PC32DBL __s390_indirect_jump_r14+0x2
10ae: c0 f4 00 00 00 00 jg 10ae <mmput+0x1e>
10b0: R_390_PLT32DBL __mmput+0x2
# Build with -fPIC and disassemble kernel/fork.c :: mmput()
$ make -s clean
$ KCFLAGS="-fPIC -mno-pic-data-is-text-relative -fno-section-anchors" make -s -j$(nproc) kernel/fork.o
$ objdump -Dr -j.text --disassemble=mmput kernel/fork.o
kernel/fork.o: file format elf64-s390
Disassembly of section .text:
0000000000002430 <mmput>:
2430: c0 04 00 00 00 00 jgnop 2430 <mmput>
2436: a7 18 ff ff lhi %r1,-1
243a: eb 01 21 3c 00 f8 laa %r0,%r1,316(%r2)
2440: 07 e0 bnor %r0
2442: ec 08 00 06 01 7e cije %r0,1,244e <mmput+0x1e>
2448: c0 f4 00 00 00 00 jg 2448 <mmput+0x18>
244a: R_390_PC32DBL __s390_indirect_jump_r14+0x2
244e: eb 9f f0 60 00 24 stmg %r9,%r15,96(%r15)
2454: b9 04 00 ef lgr %r14,%r15
2458: e3 f0 ff c0 ff 71 lay %r15,-64(%r15)
245e: e3 e0 f0 98 00 24 stg %r14,152(%r15)
2464: b9 04 00 b2 lgr %r11,%r2
2468: c0 e5 00 00 00 00 brasl %r14,2468 <mmput+0x38>
246a: R_390_PLT32DBL uprobe_clear_state+0x2
246e: b9 04 00 2b lgr %r2,%r11
2472: c0 e5 00 00 00 00 brasl %r14,2472 <mmput+0x42>
2474: R_390_PLT32DBL exit_aio+0x2
2478: e3 20 b4 f8 00 04 lg %r2,1272(%r11)
247e: a7 20 00 01 tmlh %r2,1
2482: a7 74 00 6f jne 2560 <mmput+0x130>
2486: e3 30 b4 f8 00 04 lg %r3,1272(%r11)
248c: a7 30 00 02 tmlh %r3,2
2490: a7 74 00 61 jne 2552 <mmput+0x122>
2494: b9 04 00 2b lgr %r2,%r11
2498: 41 90 b1 88 la %r9,392(%r11)
249c: c0 e5 00 00 00 00 brasl %r14,249c <mmput+0x6c>
249e: R_390_PLT32DBL exit_mmap+0x2
24a2: b9 04 00 2b lgr %r2,%r11
24a6: c0 e5 00 00 00 00 brasl %r14,24a6 <mmput+0x76>
24a8: R_390_PLT32DBL mm_put_huge_zero_page+0x2
24ac: a7 39 00 00 lghi %r3,0
24b0: b9 04 00 2b lgr %r2,%r11
24b4: c0 e5 00 00 00 00 brasl %r14,24b4 <mmput+0x84>
24b6: R_390_PLT32DBL set_mm_exe_file+0x2
24ba: e3 40 b1 88 00 04 lg %r4,392(%r11)
24c0: ec 94 00 2f 80 64 cgrje %r9,%r4,251e <mmput+0xee>
24c6: c4 a8 00 00 00 00 lgrl %r10,24c6 <mmput+0x96>
24c8: R_390_GOTENT mmlist_lock+0x2
24cc: 58 10 03 ac l %r1,940
24d0: a7 58 00 00 lhi %r5,0
24d4: ba 51 a0 00 cs %r5,%r1,0(%r10)
24d8: ec 56 00 53 00 7e cijne %r5,0,257e <mmput+0x14e>
24de: b9 04 00 29 lgr %r2,%r9
24e2: c0 e5 00 00 00 00 brasl %r14,24e2 <mmput+0xb2>
24e4: R_390_PLT32DBL __list_del_entry_valid_or_report+0x2
24e8: ec 28 00 0f 00 7c cgije %r2,0,2506 <mmput+0xd6>
24ee: e3 e0 b1 90 00 04 lg %r14,400(%r11)
24f4: e3 20 b1 88 00 04 lg %r2,392(%r11)
24fa: e3 e0 20 08 00 24 stg %r14,8(%r2)
2500: e3 20 e0 00 00 24 stg %r2,0(%r14)
2506: e5 48 b1 88 01 00 mvghi 392(%r11),256
250c: e5 48 b1 90 01 22 mvghi 400(%r11),290
2512: a7 08 00 00 lhi %r0,0
2516: 47 00 00 00 nop 0
251a: 40 00 a0 02 sth %r0,2(%r10)
251e: e3 30 b4 68 00 02 ltg %r3,1128(%r11)
2524: a7 84 00 08 je 2534 <mmput+0x104>
2528: e3 20 30 10 00 04 lg %r2,16(%r3)
252e: c0 e5 00 00 00 00 brasl %r14,252e <mmput+0xfe>
2530: R_390_PLT32DBL module_put+0x2
2534: a7 48 ff ff lhi %r4,-1
2538: eb 94 b0 00 00 f8 laa %r9,%r4,0(%r11)
253e: 07 e0 bnor %r0
2540: ec 98 00 17 01 7e cije %r9,1,256e <mmput+0x13e>
2546: eb 9f f0 a0 00 04 lmg %r9,%r15,160(%r15)
254c: c0 f4 00 00 00 00 jg 254c <mmput+0x11c>
254e: R_390_PC32DBL __s390_indirect_jump_r14+0x2
2552: b9 04 00 2b lgr %r2,%r11
2556: c0 e5 00 00 00 00 brasl %r14,2556 <mmput+0x126>
2558: R_390_PLT32DBL __khugepaged_exit+0x2
255c: a7 f4 ff 9c j 2494 <mmput+0x64>
2560: b9 04 00 2b lgr %r2,%r11
2564: c0 e5 00 00 00 00 brasl %r14,2564 <mmput+0x134>
2566: R_390_PLT32DBL __ksm_exit+0x2
256a: a7 f4 ff 8e j 2486 <mmput+0x56>
256e: b9 04 00 2b lgr %r2,%r11
2572: eb 9f f0 a0 00 04 lmg %r9,%r15,160(%r15)
2578: c0 f4 00 00 00 00 jg 2578 <mmput+0x148>
257a: R_390_PLT32DBL __mmdrop+0x2
257e: b9 04 00 2a lgr %r2,%r10
2582: c0 e5 00 00 00 00 brasl %r14,2582 <mmput+0x152>
2584: R_390_PLT32DBL arch_spin_lock_wait+0x2
2588: a7 f4 ff ab j 24de <mmput+0xae>
# GCC information
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/s390x-redhat-linux/14/lto-wrapper
Target: s390x-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --without-isl --enable-gnu-indirect-function --with-long-double-128 --with-arch=z14 --with-tune=z15 --enable-decimal-float --build=s390x-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1 --enable-host-pie --enable-host-bind-now
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.1.1 20240507 (Red Hat 14.1.1-1) (GCC)
next prev parent reply other threads:[~2024-06-18 20:37 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-19 13:27 [PATCH v2 0/4] s390: compile relocatable kernel with/without fPIE Sumanth Korikkar
2024-02-19 13:27 ` [PATCH v2 1/4] s390/vdso64: filter out munaligned-symbols flag for vdso Sumanth Korikkar
2024-02-19 13:27 ` [PATCH v2 2/4] s390: Add relocs tool Sumanth Korikkar
2024-02-19 16:11 ` Heiko Carstens
2024-02-19 13:27 ` [PATCH v2 3/4] s390: Compile relocatable kernel without -fPIE Sumanth Korikkar
2024-02-19 13:27 ` [PATCH v2 4/4] s390/kernel: vmlinux.lds.S: handle orphan .rela sections Sumanth Korikkar
2024-02-19 16:12 ` [PATCH v2 0/4] s390: compile relocatable kernel with/without fPIE Heiko Carstens
2024-06-18 20:37 ` Joe Lawrence [this message]
2024-06-19 17:01 ` Sumanth Korikkar
2024-06-19 18:23 ` Joe Lawrence
2024-06-21 7:14 ` Sumanth Korikkar
2024-06-21 11:32 ` Joe Lawrence
2024-06-21 16:59 ` Joe Lawrence
2024-06-24 9:59 ` Sumanth Korikkar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZnHv/HmiYHoQRkUU@redhat.com \
--to=joe.lawrence@redhat.com \
--cc=agordeev@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iii@linux.ibm.com \
--cc=jpoimboe@kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=sumanthk@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.