* [PATCH 1/3] arm64: module: Keep executable init sections in the core text region
2026-11-19 16:58 [PATCH 0/3] arm64: Combine module .init.text with .text Ard Biesheuvel
@ 2026-11-19 16:58 ` Ard Biesheuvel
2026-11-19 16:58 ` [PATCH 2/3] arm64: Re-enable kernel BTI for compatible linkers Ard Biesheuvel
2026-11-19 16:58 ` [PATCH 3/3] arm64: module: Revert separate init PLT and init ftrace trampoline Ard Biesheuvel
2 siblings, 0 replies; 4+ messages in thread
From: Ard Biesheuvel @ 2026-11-19 16:58 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, will, Ard Biesheuvel, Mark Brown, Josh Poimboeuf,
Nick Desaulniers
From: Ard Biesheuvel <ardb@kernel.org>
Since commit 3e35d303ab7d ("arm64: module: rework module VA range
selection"), the core and init text regions of a module may end up more
than 128 MiB apart, i.e., out of direct branching range of each other.
This is why the module loader maintains a separate .init.plt section for
branches emitted into .init.text, and a separate ftrace trampoline for
ftrace callsites in .init.text.
__init code typically makes up only a small fraction of a module's code,
so instead, keep .init.text in the core text region, by overriding
module_init_section(). Do the same for .exit.text in
module_exit_section(), which is only consulted when CONFIG_MODULE_UNLOAD
is disabled, in which case .exit sections are moved into the init region
too.
On arm64 allmodconfig (10604 modules, disregarding modules with 'test' or
'torture' in their name), this grows the aggregate size of the modules'
core text regions by 2.6 MB (+0.58%), or 243 bytes per module on
average. As this only costs an additional page at runtime if a module's
core text region happens to end within that many bytes of a page
boundary, this is expected to affect ~5.5% of modules. In practice, 527
modules (5%) grow by at least one page, of which 16 by more than one,
adding up to 567 pages of 4 KiB. Given that allmodconfig enables KASAN,
KCOV, UBSAN and GCOV instrumentation, which inflates __init code by
1.5-2x, these figures should be taken as an upper bound.
On a distro-style configuration without sanitizers (1023 modules), the
average growth is 104 bytes per module, or ~10 KiB per 100 loaded
modules, which is expected to cost ~2.5 additional pages of 4 KiB (or
~0.6 pages of 16 KiB).
This permits the separate init PLT and init ftrace trampoline to be
removed in a subsequent patch.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/kernel/module.c | 33 ++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 24adb581af0e..39e7a50733a9 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -25,6 +25,39 @@
#include <asm/sections.h>
#include <asm/text-patching.h>
+/*
+ * The module loader allocates the core and init text regions of a module
+ * independently, and so they may end up out of direct branching range of each
+ * other. This complicates the handling of PLTs and ftrace trampolines, which
+ * need to be emitted within range of the branch instructions that use them.
+ *
+ * So keep executable .init sections in the core text region instead. This
+ * costs a bit of memory, as __init code will no longer be freed after the
+ * module's init routine returns, but this is usually a small fraction of the
+ * overall size of the module. Non-executable .init sections are not affected.
+ */
+bool module_init_section(const char *name)
+{
+ if (strstarts(name, ".init.text"))
+ return false;
+
+ return strstarts(name, ".init");
+}
+
+/*
+ * Without CONFIG_MODULE_UNLOAD, the module loader places .exit sections into
+ * the init region, as they will never be used. Keep .exit.text in the core
+ * text region in this case too, so that branches emitted into it are always
+ * guaranteed to be in range of the core PLT.
+ */
+bool module_exit_section(const char *name)
+{
+ if (strstarts(name, ".exit.text"))
+ return false;
+
+ return strstarts(name, ".exit");
+}
+
enum aarch64_reloc_op {
RELOC_OP_NONE,
RELOC_OP_ABS,
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 3/3] arm64: module: Revert separate init PLT and init ftrace trampoline
2026-11-19 16:58 [PATCH 0/3] arm64: Combine module .init.text with .text Ard Biesheuvel
2026-11-19 16:58 ` [PATCH 1/3] arm64: module: Keep executable init sections in the core text region Ard Biesheuvel
2026-11-19 16:58 ` [PATCH 2/3] arm64: Re-enable kernel BTI for compatible linkers Ard Biesheuvel
@ 2026-11-19 16:58 ` Ard Biesheuvel
2 siblings, 0 replies; 4+ messages in thread
From: Ard Biesheuvel @ 2026-11-19 16:58 UTC (permalink / raw)
To: linux-arm-kernel
Cc: catalin.marinas, will, Ard Biesheuvel, Mark Brown, Josh Poimboeuf,
Nick Desaulniers
From: Ard Biesheuvel <ardb@kernel.org>
This reverts commit a7ed7b9d0ebb ("arm64: ftrace: fix unreachable PLT for
ftrace_caller in init_module with CONFIG_DYNAMIC_FTRACE") and commit
24af6c4e4e0f ("arm64: module: split core and init PLT sections").
Now that executable init sections are kept in the core text region, all
branch instructions that may need a PLT entry, as well as all ftrace
callsites, live in the core text region, and so a single PLT section and
a single ftrace trampoline suffice.
This also means that branches to symbols defined in another section of
the same module no longer need a PLT slot, as all executable sections are
within direct branching range of each other. So only reserve PLT slots
for branches to external symbols, i.e., SHN_UNDEF symbols as before
24af6c4e4e0f, but also SHN_LIVEPATCH symbols (and other reserved section
indexes), as these are resolved against vmlinux or other modules.
On arm64 allmodconfig (10604 modules, disregarding modules with 'test' or
'torture' in their name), this drops 2126 reserved but never used PLT
slots for intra-module branches (about 25 KB in total).
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
arch/arm64/include/asm/module.h | 11 +--
arch/arm64/include/asm/module.lds.h | 2 -
arch/arm64/kernel/ftrace.c | 13 +--
arch/arm64/kernel/module-plts.c | 98 +++++++-------------
arch/arm64/kernel/module.c | 13 +--
5 files changed, 40 insertions(+), 97 deletions(-)
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index fb9b88eebeb1..9d34f915d59b 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -7,24 +7,17 @@
#include <asm-generic/module.h>
-struct mod_plt_sec {
+struct mod_arch_specific {
int plt_shndx;
int plt_num_entries;
int plt_max_entries;
-};
-
-struct mod_arch_specific {
- struct mod_plt_sec core;
- struct mod_plt_sec init;
/* for CONFIG_DYNAMIC_FTRACE */
struct plt_entry *ftrace_trampolines;
- struct plt_entry *init_ftrace_trampolines;
};
u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
- void *loc, const Elf64_Rela *rela,
- Elf64_Sym *sym);
+ const Elf64_Rela *rela, Elf64_Sym *sym);
u64 module_emit_veneer_for_adrp(struct module *mod, Elf64_Shdr *sechdrs,
void *loc, u64 val);
diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h
index 603f92a8a73e..983e768203a1 100644
--- a/arch/arm64/include/asm/module.lds.h
+++ b/arch/arm64/include/asm/module.lds.h
@@ -1,8 +1,6 @@
SECTIONS {
.plt 0 : { BYTE(0) }
- .init.plt 0 : { BYTE(0) }
.text.ftrace_trampoline 0 : { BYTE(0) }
- .init.text.ftrace_trampoline 0 : { BYTE(0) }
#ifdef CONFIG_UNWIND_TABLES
/*
diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c
index e1a3c0b3a051..389675690293 100644
--- a/arch/arm64/kernel/ftrace.c
+++ b/arch/arm64/kernel/ftrace.c
@@ -258,17 +258,10 @@ int ftrace_update_ftrace_func(ftrace_func_t func)
return ftrace_modify_code(pc, 0, new, false);
}
-static struct plt_entry *get_ftrace_plt(struct module *mod, unsigned long addr)
+static struct plt_entry *get_ftrace_plt(struct module *mod)
{
#ifdef CONFIG_MODULES
- struct plt_entry *plt = NULL;
-
- if (within_module_mem_type(addr, mod, MOD_INIT_TEXT))
- plt = mod->arch.init_ftrace_trampolines;
- else if (within_module_mem_type(addr, mod, MOD_TEXT))
- plt = mod->arch.ftrace_trampolines;
- else
- return NULL;
+ struct plt_entry *plt = mod->arch.ftrace_trampolines;
return &plt[FTRACE_PLT_IDX];
#else
@@ -339,7 +332,7 @@ static bool ftrace_find_callable_addr(struct dyn_ftrace *rec,
if (WARN_ON(!mod))
return false;
- plt = get_ftrace_plt(mod, pc);
+ plt = get_ftrace_plt(mod);
if (!plt) {
pr_err("ftrace: no module PLT for %ps\n", (void *)*addr);
return false;
diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c
index 7afd370da9f4..0a0257b6a5e8 100644
--- a/arch/arm64/kernel/module-plts.c
+++ b/arch/arm64/kernel/module-plts.c
@@ -67,11 +67,9 @@ static bool plt_entries_equal(const struct plt_entry *a,
}
u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
- void *loc, const Elf64_Rela *rela,
- Elf64_Sym *sym)
+ const Elf64_Rela *rela, Elf64_Sym *sym)
{
- struct mod_plt_sec *pltsec = !within_module_init((unsigned long)loc, mod) ?
- &mod->arch.core : &mod->arch.init;
+ struct mod_arch_specific *pltsec = &mod->arch;
struct plt_entry *plt = (struct plt_entry *)sechdrs[pltsec->plt_shndx].sh_addr;
int i = pltsec->plt_num_entries;
int j = i - 1;
@@ -101,8 +99,7 @@ u64 module_emit_plt_entry(struct module *mod, Elf64_Shdr *sechdrs,
u64 module_emit_veneer_for_adrp(struct module *mod, Elf64_Shdr *sechdrs,
void *loc, u64 val)
{
- struct mod_plt_sec *pltsec = !within_module_init((unsigned long)loc, mod) ?
- &mod->arch.core : &mod->arch.init;
+ struct mod_arch_specific *pltsec = &mod->arch;
struct plt_entry *plt = (struct plt_entry *)sechdrs[pltsec->plt_shndx].sh_addr;
int i = pltsec->plt_num_entries++;
u32 br;
@@ -154,8 +151,21 @@ static bool duplicate_rel(const Elf64_Rela *rela, int num)
return num > 0 && cmp_rela(rela + num, rela + num - 1) == 0;
}
+/*
+ * We only have to consider branch targets that resolve to symbols that are
+ * not defined in the module itself. This is not simply a heuristic, it is a
+ * fundamental limitation, since the PLT itself is part of the module's core
+ * text region, and needs to be within 128 MB as well, so the module's code can
+ * never grow beyond that limit. Note that SHN_LIVEPATCH symbols are resolved
+ * against vmlinux or other modules, so these need to be taken into account.
+ */
+static bool branch_target_is_external(const Elf64_Sym *s)
+{
+ return s->st_shndx == SHN_UNDEF || s->st_shndx >= SHN_LORESERVE;
+}
+
static unsigned int count_plts(Elf64_Sym *syms, Elf64_Rela *rela, int num,
- Elf64_Word dstidx, Elf_Shdr *dstsec)
+ Elf_Shdr *dstsec)
{
unsigned int ret = 0;
Elf64_Sym *s;
@@ -167,19 +177,8 @@ static unsigned int count_plts(Elf64_Sym *syms, Elf64_Rela *rela, int num,
switch (ELF64_R_TYPE(rela[i].r_info)) {
case R_AARCH64_JUMP26:
case R_AARCH64_CALL26:
- /*
- * We only have to consider branch targets that resolve
- * to symbols that are defined in a different section.
- * This is not simply a heuristic, it is a fundamental
- * limitation, since there is no guaranteed way to emit
- * PLT entries sufficiently close to the branch if the
- * section size exceeds the range of a branch
- * instruction. So ignore relocations against defined
- * symbols if they live in the same section as the
- * relocation target.
- */
s = syms + ELF64_R_SYM(rela[i].r_info);
- if (s->st_shndx == dstidx)
+ if (!branch_target_is_external(s))
break;
/*
@@ -246,13 +245,11 @@ static unsigned int count_plts(Elf64_Sym *syms, Elf64_Rela *rela, int num,
return ret;
}
-static bool branch_rela_needs_plt(Elf64_Sym *syms, Elf64_Rela *rela,
- Elf64_Word dstidx)
+static bool branch_rela_needs_plt(Elf64_Sym *syms, Elf64_Rela *rela)
{
-
Elf64_Sym *s = syms + ELF64_R_SYM(rela->r_info);
- if (s->st_shndx == dstidx)
+ if (!branch_target_is_external(s))
return false;
return ELF64_R_TYPE(rela->r_info) == R_AARCH64_JUMP26 ||
@@ -261,14 +258,14 @@ static bool branch_rela_needs_plt(Elf64_Sym *syms, Elf64_Rela *rela,
/* Group branch PLT relas at the front end of the array. */
static int partition_branch_plt_relas(Elf64_Sym *syms, Elf64_Rela *rela,
- int numrels, Elf64_Word dstidx)
+ int numrels)
{
int i = 0, j = numrels - 1;
while (i < j) {
- if (branch_rela_needs_plt(syms, &rela[i], dstidx))
+ if (branch_rela_needs_plt(syms, &rela[i]))
i++;
- else if (branch_rela_needs_plt(syms, &rela[j], dstidx))
+ else if (branch_rela_needs_plt(syms, &rela[j]))
swap(rela[i], rela[j]);
else
j--;
@@ -280,10 +277,9 @@ static int partition_branch_plt_relas(Elf64_Sym *syms, Elf64_Rela *rela,
int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
char *secstrings, struct module *mod)
{
- unsigned long core_plts = 0;
- unsigned long init_plts = 0;
+ unsigned long plts = 0;
Elf64_Sym *syms = NULL;
- Elf_Shdr *pltsec, *tramp = NULL, *init_tramp = NULL;
+ Elf_Shdr *pltsec, *tramp = NULL;
int i;
/*
@@ -292,21 +288,16 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
*/
for (i = 0; i < ehdr->e_shnum; i++) {
if (!strcmp(secstrings + sechdrs[i].sh_name, ".plt"))
- mod->arch.core.plt_shndx = i;
- else if (!strcmp(secstrings + sechdrs[i].sh_name, ".init.plt"))
- mod->arch.init.plt_shndx = i;
+ mod->arch.plt_shndx = i;
else if (!strcmp(secstrings + sechdrs[i].sh_name,
".text.ftrace_trampoline"))
tramp = sechdrs + i;
- else if (!strcmp(secstrings + sechdrs[i].sh_name,
- ".init.text.ftrace_trampoline"))
- init_tramp = sechdrs + i;
else if (sechdrs[i].sh_type == SHT_SYMTAB)
syms = (Elf64_Sym *)sechdrs[i].sh_addr;
}
- if (!mod->arch.core.plt_shndx || !mod->arch.init.plt_shndx) {
- pr_err("%s: module PLT section(s) missing\n", mod->name);
+ if (!mod->arch.plt_shndx) {
+ pr_err("%s: module PLT section missing\n", mod->name);
return -ENOEXEC;
}
if (!syms) {
@@ -330,34 +321,20 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
* sort branch relocations requiring a PLT by type, symbol index
* and addend
*/
- nents = partition_branch_plt_relas(syms, rels, numrels,
- sechdrs[i].sh_info);
+ nents = partition_branch_plt_relas(syms, rels, numrels);
if (nents)
sort(rels, nents, sizeof(Elf64_Rela), cmp_rela, NULL);
- if (!module_init_layout_section(secstrings + dstsec->sh_name))
- core_plts += count_plts(syms, rels, numrels,
- sechdrs[i].sh_info, dstsec);
- else
- init_plts += count_plts(syms, rels, numrels,
- sechdrs[i].sh_info, dstsec);
+ plts += count_plts(syms, rels, numrels, dstsec);
}
- pltsec = sechdrs + mod->arch.core.plt_shndx;
- pltsec->sh_type = SHT_NOBITS;
- pltsec->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
- pltsec->sh_addralign = L1_CACHE_BYTES;
- pltsec->sh_size = (core_plts + 1) * sizeof(struct plt_entry);
- mod->arch.core.plt_num_entries = 0;
- mod->arch.core.plt_max_entries = core_plts;
-
- pltsec = sechdrs + mod->arch.init.plt_shndx;
+ pltsec = sechdrs + mod->arch.plt_shndx;
pltsec->sh_type = SHT_NOBITS;
pltsec->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
pltsec->sh_addralign = L1_CACHE_BYTES;
- pltsec->sh_size = (init_plts + 1) * sizeof(struct plt_entry);
- mod->arch.init.plt_num_entries = 0;
- mod->arch.init.plt_max_entries = init_plts;
+ pltsec->sh_size = (plts + 1) * sizeof(struct plt_entry);
+ mod->arch.plt_num_entries = 0;
+ mod->arch.plt_max_entries = plts;
if (tramp) {
tramp->sh_type = SHT_NOBITS;
@@ -366,12 +343,5 @@ int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
tramp->sh_size = NR_FTRACE_PLTS * sizeof(struct plt_entry);
}
- if (init_tramp) {
- init_tramp->sh_type = SHT_NOBITS;
- init_tramp->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
- init_tramp->sh_addralign = __alignof__(struct plt_entry);
- init_tramp->sh_size = NR_FTRACE_PLTS * sizeof(struct plt_entry);
- }
-
return 0;
}
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 39e7a50733a9..e0cbd499e4a9 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -450,7 +450,7 @@ int apply_relocate_add(Elf64_Shdr *sechdrs,
ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 26,
AARCH64_INSN_IMM_26, me);
if (ovf == -ERANGE) {
- val = module_emit_plt_entry(me, sechdrs, loc, &rel[i], sym);
+ val = module_emit_plt_entry(me, sechdrs, &rel[i], sym);
if (!val)
return -ENOEXEC;
ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2,
@@ -499,17 +499,6 @@ static int module_init_ftrace_plt(const Elf_Ehdr *hdr,
__init_plt(&plts[FTRACE_PLT_IDX], FTRACE_ADDR);
mod->arch.ftrace_trampolines = plts;
-
- s = find_section(hdr, sechdrs, ".init.text.ftrace_trampoline");
- if (!s)
- return -ENOEXEC;
-
- plts = (void *)s->sh_addr;
-
- __init_plt(&plts[FTRACE_PLT_IDX], FTRACE_ADDR);
-
- mod->arch.init_ftrace_trampolines = plts;
-
#endif
return 0;
}
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply related [flat|nested] 4+ messages in thread