* Re: [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Peter Zijlstra @ 2026-04-23 16:29 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <gpq7mfal5gydlrqsm5mza5hzx5aa2rq7yk6olozlzotdnl7e24@ljzzzwwsputr>
On Thu, Apr 23, 2026 at 09:23:12AM -0700, Josh Poimboeuf wrote:
> > > Sorry, how do you get 64 here?
> >
> > DEBUG_FORCE_FUNCTION_ALIGNMENT_64B=y
>
> Ok, so in that case it would be 5-byte cfi symbol and 59-byte NOP gap.
> Or a 64-byte pfx for the !CFI case.
Just so.
^ permalink raw reply
* Re: [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Josh Poimboeuf @ 2026-04-23 16:23 UTC (permalink / raw)
To: Peter Zijlstra
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <20260423151925.GG1064669@noisy.programming.kicks-ass.net>
On Thu, Apr 23, 2026 at 05:19:25PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 23, 2026 at 08:16:08AM -0700, Josh Poimboeuf wrote:
> > On Thu, Apr 23, 2026 at 10:47:58AM +0200, Peter Zijlstra wrote:
> > > On Wed, Apr 22, 2026 at 09:04:13PM -0700, Josh Poimboeuf wrote:
> > > > PREFIX_SYMBOLS has a !CFI dependency because the compiler already
> > > > generates __cfi_ prefix symbols for kCFI builds, so objtool-generated
> > > > __pfx_ symbols were considered redundant.
> > > >
> > > > However, the __cfi_ symbols only cover the 5-byte kCFI type hash. With
> > > > FUNCTION_CALL_PADDING, there are also 11 bytes of NOP padding between
> > > > the hash and the function entry which have no symbol to claim them.
> > >
> > > If you force the function alignment to 64 bytes, the prefix will also be
> > > 64bytes, rather than the normal 16.
> >
> > Sorry, how do you get 64 here?
>
> DEBUG_FORCE_FUNCTION_ALIGNMENT_64B=y
Ok, so in that case it would be 5-byte cfi symbol and 59-byte NOP gap.
Or a 64-byte pfx for the !CFI case.
> > > > The NOPs can be rewritten with call depth tracking thunks at runtime.
> > > > Without a symbol, unwinders and other tools that symbolize code
> > > > locations misattribute those bytes.
> > > >
> > > > Remove the !CFI guard so objtool creates __pfx_ symbols for all
> > > > CALL_PADDING configs, covering the full padding area regardless of
> > > > whether there's also a __cfi_ symbol.
> > >
> > > Egads, that a ton of symbols :/ Does it not make sense to 'fix' up the
> > > __cfi_ symbols to cover the whole prefix?
> >
> > Yeah, I suppose that would be better, via objtool I presume.
>
> Yup.
--
Josh
^ permalink raw reply
* Re: [PATCH 47/48] objtool: Improve and simplify prefix symbol detection
From: Peter Zijlstra @ 2026-04-23 15:21 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <be5iv2gj7mfmizs6v3dsygnv2c2tn7aniabz3fopgrbnnnfdnj@433avj4qybyp>
On Thu, Apr 23, 2026 at 08:19:57AM -0700, Josh Poimboeuf wrote:
> On Thu, Apr 23, 2026 at 10:55:20AM +0200, Peter Zijlstra wrote:
> > On Wed, Apr 22, 2026 at 09:04:15PM -0700, Josh Poimboeuf wrote:
> > > Only create prefix symbols for functions that have
> > > __patchable_function_entries entries, since those are the only functions
> > > where prefix NOPs are intentional.
> >
> > __CFI_TYPE() as used in SYM_TYPED_ENTRY() will also generate the NOPs
> > but will not have __patchable_function_entries, because ASM not
> > compiler.
>
> Hm, but those already have __cfi_ symbols, no?
Yes, but you said those were 'short' -- but fair, I did not check if the
asm stub generated symbols of the correct length.
^ permalink raw reply
* Re: [PATCH 47/48] objtool: Improve and simplify prefix symbol detection
From: Josh Poimboeuf @ 2026-04-23 15:19 UTC (permalink / raw)
To: Peter Zijlstra
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <20260423085520.GZ3126523@noisy.programming.kicks-ass.net>
On Thu, Apr 23, 2026 at 10:55:20AM +0200, Peter Zijlstra wrote:
> On Wed, Apr 22, 2026 at 09:04:15PM -0700, Josh Poimboeuf wrote:
> > Only create prefix symbols for functions that have
> > __patchable_function_entries entries, since those are the only functions
> > where prefix NOPs are intentional.
>
> __CFI_TYPE() as used in SYM_TYPED_ENTRY() will also generate the NOPs
> but will not have __patchable_function_entries, because ASM not
> compiler.
Hm, but those already have __cfi_ symbols, no?
--
Josh
^ permalink raw reply
* Re: [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Peter Zijlstra @ 2026-04-23 15:19 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <n6xqf563o4moyl3sqp37ymakjhyvbxfinghi5k5lygeocak6ns@ugrn3b7csjot>
On Thu, Apr 23, 2026 at 08:16:08AM -0700, Josh Poimboeuf wrote:
> On Thu, Apr 23, 2026 at 10:47:58AM +0200, Peter Zijlstra wrote:
> > On Wed, Apr 22, 2026 at 09:04:13PM -0700, Josh Poimboeuf wrote:
> > > PREFIX_SYMBOLS has a !CFI dependency because the compiler already
> > > generates __cfi_ prefix symbols for kCFI builds, so objtool-generated
> > > __pfx_ symbols were considered redundant.
> > >
> > > However, the __cfi_ symbols only cover the 5-byte kCFI type hash. With
> > > FUNCTION_CALL_PADDING, there are also 11 bytes of NOP padding between
> > > the hash and the function entry which have no symbol to claim them.
> >
> > If you force the function alignment to 64 bytes, the prefix will also be
> > 64bytes, rather than the normal 16.
>
> Sorry, how do you get 64 here?
DEBUG_FORCE_FUNCTION_ALIGNMENT_64B=y
> > > The NOPs can be rewritten with call depth tracking thunks at runtime.
> > > Without a symbol, unwinders and other tools that symbolize code
> > > locations misattribute those bytes.
> > >
> > > Remove the !CFI guard so objtool creates __pfx_ symbols for all
> > > CALL_PADDING configs, covering the full padding area regardless of
> > > whether there's also a __cfi_ symbol.
> >
> > Egads, that a ton of symbols :/ Does it not make sense to 'fix' up the
> > __cfi_ symbols to cover the whole prefix?
>
> Yeah, I suppose that would be better, via objtool I presume.
Yup.
^ permalink raw reply
* Re: [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Josh Poimboeuf @ 2026-04-23 15:16 UTC (permalink / raw)
To: Peter Zijlstra
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <20260423084758.GY3126523@noisy.programming.kicks-ass.net>
On Thu, Apr 23, 2026 at 10:47:58AM +0200, Peter Zijlstra wrote:
> On Wed, Apr 22, 2026 at 09:04:13PM -0700, Josh Poimboeuf wrote:
> > PREFIX_SYMBOLS has a !CFI dependency because the compiler already
> > generates __cfi_ prefix symbols for kCFI builds, so objtool-generated
> > __pfx_ symbols were considered redundant.
> >
> > However, the __cfi_ symbols only cover the 5-byte kCFI type hash. With
> > FUNCTION_CALL_PADDING, there are also 11 bytes of NOP padding between
> > the hash and the function entry which have no symbol to claim them.
>
> If you force the function alignment to 64 bytes, the prefix will also be
> 64bytes, rather than the normal 16.
Sorry, how do you get 64 here?
> > The NOPs can be rewritten with call depth tracking thunks at runtime.
> > Without a symbol, unwinders and other tools that symbolize code
> > locations misattribute those bytes.
> >
> > Remove the !CFI guard so objtool creates __pfx_ symbols for all
> > CALL_PADDING configs, covering the full padding area regardless of
> > whether there's also a __cfi_ symbol.
>
> Egads, that a ton of symbols :/ Does it not make sense to 'fix' up the
> __cfi_ symbols to cover the whole prefix?
Yeah, I suppose that would be better, via objtool I presume.
--
Josh
^ permalink raw reply
* Re: [PATCH 43/48] objtool: Add insn_sym() helper
From: Josh Poimboeuf @ 2026-04-23 15:14 UTC (permalink / raw)
To: Peter Zijlstra
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <20260423084537.GX3126523@noisy.programming.kicks-ass.net>
On Thu, Apr 23, 2026 at 10:45:37AM +0200, Peter Zijlstra wrote:
> On Wed, Apr 22, 2026 at 09:04:11PM -0700, Josh Poimboeuf wrote:
> > Alternative replacement instructions awkwardly have insn->sym set to the
> > function they get patched to rather than the symbol (or rather lack
> > thereof) they belong to in the file.
> >
> > This makes it difficult to know where a given instruction actually
> > lives.
> >
> > Add a new insn_sym() helper which preserves the existing semantic of
> > insn->sym. Rename insn->sym to insn->_sym, which contains the actual
> > ELF binary symbol (or NULL, for alternative replacements) an instruction
> > lives in.
> >
> > The private insn->_sym value will be needed for a subsequent patch.
> >
>
> > +/*
> > + * Return the symbol associated with an instruction. For alternative
> > + * replacements, return the symbol of the original code being replaced rather
> > + * than NULL. insn->_sym reflects the actual location in the ELF file.
> > + */
> > +static inline struct symbol *insn_sym(struct instruction *insn)
> > +{
> > + struct symbol *sym = insn->_sym;
> > +
> > + if (!sym && insn->alt_group && insn->alt_group->orig_group)
> > + sym = insn->alt_group->orig_group->first_insn->_sym;
> > +
> > + return sym;
> > +}
>
> That is a bit of a deref fest -- this does not affect performance
> negatively?
Ha, the "deref fest" only happens for alternative replacement
instructions, so presumably not?
--
Josh
^ permalink raw reply
* Re: [PATCH 32/48] objtool: Add is_cold_func() helper
From: Peter Zijlstra @ 2026-04-23 15:14 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <qp7srvaafzmgsh334jeidseax2zgvt2j65iugsru5co3wrm6ka@opizry3c2m6d>
On Thu, Apr 23, 2026 at 08:12:07AM -0700, Josh Poimboeuf wrote:
> On Thu, Apr 23, 2026 at 10:38:49AM +0200, Peter Zijlstra wrote:
> > On Wed, Apr 22, 2026 at 09:04:00PM -0700, Josh Poimboeuf wrote:
> >
> > > diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> > > index 00c2389f345f..8a6e1338af97 100644
> > > --- a/tools/objtool/elf.c
> > > +++ b/tools/objtool/elf.c
> > > @@ -586,8 +586,11 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> > > if (strstarts(sym->name, ".klp.sym"))
> > > sym->klp = 1;
> > >
> > > + sym->pfunc = sym->cfunc = sym;
> > > +
> > > if (!sym->klp && !is_sec_sym(sym) && strstr(sym->name, ".cold")) {
> > > - sym->cold = 1;
> > > + /* Tell read_symbols() this is a cold subfunction */
> > > + sym->pfunc = NULL;
> > >
> > > /*
> > > * Clang doesn't mark cold subfunctions as STT_FUNC, which
> > > @@ -596,8 +599,6 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> > > sym->type = STT_FUNC;
> > > }
> > >
> > > - sym->pfunc = sym->cfunc = sym;
> > > -
> > > return 0;
> > > }
> >
> > So now the cold subfunction has a NULL parent-function and a
> > child-function that points to the parent?
> >
> > I'm confused.
>
> It's a bit clunky. As the comment implies, 'sym->pfunc = NULL' is a
> signal to it caller read_symbols() that this is a .cold function. Then,
> after all the symbols have been added, read_symbols() goes and finds the
> parent.
>
> I think I did it this way because klp-diff.c calls elf_add_symbol() (via
> elf_create_symbol()) and later needs to call is_cold_func() on it. In
> that case, even though the parent isn't set, it still works because
> is_cold_func() returns true for sym->pfunc != sym;
I'm thinking this needs more comments if it stays like this. Is most
confusing.
^ permalink raw reply
* Re: [PATCH 32/48] objtool: Add is_cold_func() helper
From: Josh Poimboeuf @ 2026-04-23 15:12 UTC (permalink / raw)
To: Peter Zijlstra
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <20260423083849.GV3126523@noisy.programming.kicks-ass.net>
On Thu, Apr 23, 2026 at 10:38:49AM +0200, Peter Zijlstra wrote:
> On Wed, Apr 22, 2026 at 09:04:00PM -0700, Josh Poimboeuf wrote:
>
> > diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> > index 00c2389f345f..8a6e1338af97 100644
> > --- a/tools/objtool/elf.c
> > +++ b/tools/objtool/elf.c
> > @@ -586,8 +586,11 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> > if (strstarts(sym->name, ".klp.sym"))
> > sym->klp = 1;
> >
> > + sym->pfunc = sym->cfunc = sym;
> > +
> > if (!sym->klp && !is_sec_sym(sym) && strstr(sym->name, ".cold")) {
> > - sym->cold = 1;
> > + /* Tell read_symbols() this is a cold subfunction */
> > + sym->pfunc = NULL;
> >
> > /*
> > * Clang doesn't mark cold subfunctions as STT_FUNC, which
> > @@ -596,8 +599,6 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> > sym->type = STT_FUNC;
> > }
> >
> > - sym->pfunc = sym->cfunc = sym;
> > -
> > return 0;
> > }
>
> So now the cold subfunction has a NULL parent-function and a
> child-function that points to the parent?
>
> I'm confused.
It's a bit clunky. As the comment implies, 'sym->pfunc = NULL' is a
signal to it caller read_symbols() that this is a .cold function. Then,
after all the symbols have been added, read_symbols() goes and finds the
parent.
I think I did it this way because klp-diff.c calls elf_add_symbol() (via
elf_create_symbol()) and later needs to call is_cold_func() on it. In
that case, even though the parent isn't set, it still works because
is_cold_func() returns true for sym->pfunc != sym;
--
Josh
^ permalink raw reply
* Re: [PATCH v2 2/2] module/kallsyms: sort function symbols and use binary search
From: Petr Pavlu @ 2026-04-23 14:00 UTC (permalink / raw)
To: Stanislaw Gruszka
Cc: linux-modules, Sami Tolvanen, Luis Chamberlain, linux-kernel,
linux-trace-kernel, live-patching, Daniel Gomez, Aaron Tomlin,
Steven Rostedt, Masami Hiramatsu, Jordan Rome, Viktor Malik
In-Reply-To: <20260327110005.16499-2-stf_xl@wp.pl>
On 3/27/26 12:00 PM, Stanislaw Gruszka wrote:
> Module symbol lookup via find_kallsyms_symbol() performs a linear scan
> over the entire symtab when resolving an address. The number of symbols
> in module symtabs has grown over the years, largely due to additional
> metadata in non-standard sections, making this lookup very slow.
>
> Improve this by separating function symbols during module load, placing
> them at the beginning of the symtab, sorting them by address, and using
> binary search when resolving addresses in module text.
>
> This also should improve times for linear symbol name lookups, as valid
> function symbols are now located at the beginning of the symtab.
>
> The cost of sorting is small relative to module load time. In repeated
> module load tests [1], depending on .config options, this change
> increases load time between 2% and 4%. With cold caches, the difference
> is not measurable, as memory access latency dominates.
>
> The sorting theoretically could be done in compile time, but much more
> complicated as we would have to simulate kernel addresses resolution
> for symbols, and then correct relocation entries. That would be risky
> if get out of sync.
>
> The improvement can be observed when listing ftrace filter functions.
>
> Before:
>
> root@nano:~# time cat /sys/kernel/tracing/available_filter_functions | wc -l
> 74908
>
> real 0m1.315s
> user 0m0.000s
> sys 0m1.312s
>
> After:
>
> root@nano:~# time cat /sys/kernel/tracing/available_filter_functions | wc -l
> 74911
>
> real 0m0.167s
> user 0m0.004s
> sys 0m0.175s
>
> (there are three more symbols introduced by the patch)
>
> For livepatch modules, the symtab layout is preserved and the existing
> linear search is used. For this case, it should be possible to keep
> the original ELF symtab instead of copying it 1:1, but that is outside
> the scope of this patch.
>
> Link: https://gist.github.com/sgruszka/09f3fb1dad53a97b1aad96e1927ab117 [1]
> Signed-off-by: Stanislaw Gruszka <stf_xl@wp.pl>
Sorry for the delay reviewing this patch.
> ---
> v1 -> v2:
> - fix searching data symbols for CONFIG_KALLSYMS_ALL
> - use kallsyms_symbol_value() in elf_sym_cmp()
>
> include/linux/module.h | 1 +
> kernel/module/internal.h | 1 +
> kernel/module/kallsyms.c | 171 +++++++++++++++++++++++++++++----------
> 3 files changed, 130 insertions(+), 43 deletions(-)
>
> diff --git a/include/linux/module.h b/include/linux/module.h
> index ac254525014c..67c053afa882 100644
> --- a/include/linux/module.h
> +++ b/include/linux/module.h
> @@ -379,6 +379,7 @@ struct module_memory {
> struct mod_kallsyms {
> Elf_Sym *symtab;
> unsigned int num_symtab;
> + unsigned int num_func_syms;
> char *strtab;
> char *typetab;
> };
> diff --git a/kernel/module/internal.h b/kernel/module/internal.h
> index 618202578b42..6a4d498619b1 100644
> --- a/kernel/module/internal.h
> +++ b/kernel/module/internal.h
> @@ -73,6 +73,7 @@ struct load_info {
> bool sig_ok;
> #ifdef CONFIG_KALLSYMS
> unsigned long mod_kallsyms_init_off;
> + unsigned long num_func_syms;
> #endif
> #ifdef CONFIG_MODULE_DECOMPRESS
> #ifdef CONFIG_MODULE_STATS
> diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c
> index f23126d804b2..d69e99e67707 100644
> --- a/kernel/module/kallsyms.c
> +++ b/kernel/module/kallsyms.c
> @@ -10,6 +10,7 @@
> #include <linux/kallsyms.h>
> #include <linux/buildid.h>
> #include <linux/bsearch.h>
> +#include <linux/sort.h>
> #include "internal.h"
>
> /* Lookup exported symbol in given range of kernel_symbols */
> @@ -103,6 +104,95 @@ static bool is_core_symbol(const Elf_Sym *src, const Elf_Shdr *sechdrs,
> return true;
> }
>
> +static inline bool is_func_symbol(const Elf_Sym *sym)
> +{
> + return sym->st_shndx != SHN_UNDEF && sym->st_size != 0 &&
> + ELF_ST_TYPE(sym->st_info) == STT_FUNC;
> +}
> +
> +static unsigned int bsearch_func_symbol(struct mod_kallsyms *kallsyms,
> + unsigned long addr,
> + unsigned long *bestval,
> + unsigned long *nextval)
> +
> +{
> + unsigned int mid, low = 1, high = kallsyms->num_func_syms + 1;
> + unsigned int best = 0;
> + unsigned long thisval;
> +
> + while (low < high) {
> + mid = low + (high - low) / 2;
> + thisval = kallsyms_symbol_value(&kallsyms->symtab[mid]);
> +
> + if (thisval <= addr) {
> + *bestval = thisval;
> + best = mid;
> + low = mid + 1;
If thisval == addr, the search moves to the right and finds the last
symbol with the same address. I believe it should do the opposite and
return the first symbol to match the behavior of
search_kallsyms_symbol().
> + } else {
> + *nextval = thisval;
> + high = mid;
> + }
> + }
> +
> + return best;
> +}
> +
> +static const char *kallsyms_symbol_name(struct mod_kallsyms *kallsyms,
> + unsigned int symnum)
> +{
> + return kallsyms->strtab + kallsyms->symtab[symnum].st_name;
> +}
> +
> +static unsigned int search_kallsyms_symbol(struct mod_kallsyms *kallsyms,
> + unsigned long addr,
> + unsigned long *bestval,
> + unsigned long *nextval)
> +{
> + unsigned int i, best = 0;
> +
> + /*
> + * Scan for closest preceding symbol and next symbol. (ELF starts
> + * real symbols at 1). Skip the initial function symbols range
> + * if num_func_syms is non-zero, those are handled separately for
> + * the core TEXT segment lookup.
> + */
> + for (i = 1 + kallsyms->num_func_syms; i < kallsyms->num_symtab; i++) {
> + const Elf_Sym *sym = &kallsyms->symtab[i];
> + unsigned long thisval = kallsyms_symbol_value(sym);
> +
> + if (sym->st_shndx == SHN_UNDEF)
> + continue;
> +
> + /*
> + * We ignore unnamed symbols: they're uninformative
> + * and inserted at a whim.
> + */
> + if (*kallsyms_symbol_name(kallsyms, i) == '\0' ||
> + is_mapping_symbol(kallsyms_symbol_name(kallsyms, i)))
> + continue;
> +
> + if (thisval <= addr && thisval > *bestval) {
> + best = i;
> + *bestval = thisval;
> + }
> + if (thisval > addr && thisval < *nextval)
> + *nextval = thisval;
> + }
> +
> + return best;
> +}
> +
> +static int elf_sym_cmp(const void *a, const void *b)
> +{
> + unsigned long val_a = kallsyms_symbol_value((const Elf_Sym *)a);
> + unsigned long val_b = kallsyms_symbol_value((const Elf_Sym *)b);
> +
> + if (val_a < val_b)
> + return -1;
> +
> + return val_a > val_b;
Does this comparison function and the sort() call result in stable
sorting? If val_a and val_b are the same, the sorting should preserve
the original order.
> +}
> +
> /*
> * We only allocate and copy the strings needed by the parts of symtab
> * we keep. This is simple, but has the effect of making multiple
> @@ -115,9 +205,10 @@ void layout_symtab(struct module *mod, struct load_info *info)
> Elf_Shdr *symsect = info->sechdrs + info->index.sym;
> Elf_Shdr *strsect = info->sechdrs + info->index.str;
> const Elf_Sym *src;
> - unsigned int i, nsrc, ndst, strtab_size = 0;
> + unsigned int i, nsrc, ndst, nfunc, strtab_size = 0;
> struct module_memory *mod_mem_data = &mod->mem[MOD_DATA];
> struct module_memory *mod_mem_init_data = &mod->mem[MOD_INIT_DATA];
> + bool is_lp_mod = is_livepatch_module(mod);
>
> /* Put symbol section at end of init part of module. */
> symsect->sh_flags |= SHF_ALLOC;
> @@ -129,12 +220,14 @@ void layout_symtab(struct module *mod, struct load_info *info)
> nsrc = symsect->sh_size / sizeof(*src);
>
> /* Compute total space required for the core symbols' strtab. */
> - for (ndst = i = 0; i < nsrc; i++) {
> - if (i == 0 || is_livepatch_module(mod) ||
> + for (ndst = nfunc = i = 0; i < nsrc; i++) {
> + if (i == 0 || is_lp_mod ||
> is_core_symbol(src + i, info->sechdrs, info->hdr->e_shnum,
> info->index.pcpu)) {
> strtab_size += strlen(&info->strtab[src[i].st_name]) + 1;
> ndst++;
> + if (!is_lp_mod && is_func_symbol(src + i))
> + nfunc++;
> }
> }
>
> @@ -156,6 +249,7 @@ void layout_symtab(struct module *mod, struct load_info *info)
> mod_mem_init_data->size = ALIGN(mod_mem_init_data->size,
> __alignof__(struct mod_kallsyms));
> info->mod_kallsyms_init_off = mod_mem_init_data->size;
> + info->num_func_syms = nfunc;
>
> mod_mem_init_data->size += sizeof(struct mod_kallsyms);
> info->init_typeoffs = mod_mem_init_data->size;
> @@ -169,7 +263,7 @@ void layout_symtab(struct module *mod, struct load_info *info)
> */
> void add_kallsyms(struct module *mod, const struct load_info *info)
> {
> - unsigned int i, ndst;
> + unsigned int i, di, nfunc, ndst;
> const Elf_Sym *src;
> Elf_Sym *dst;
> char *s;
> @@ -178,6 +272,7 @@ void add_kallsyms(struct module *mod, const struct load_info *info)
> void *data_base = mod->mem[MOD_DATA].base;
> void *init_data_base = mod->mem[MOD_INIT_DATA].base;
> struct mod_kallsyms *kallsyms;
> + bool is_lp_mod = is_livepatch_module(mod);
>
> kallsyms = init_data_base + info->mod_kallsyms_init_off;
This code is followed by the initialization of kallsyms:
kallsyms->symtab = (void *)symsec->sh_addr;
kallsyms->num_symtab = symsec->sh_size / sizeof(Elf_Sym);
/* Make sure we get permanent strtab: don't use info->strtab. */
kallsyms->strtab = (void *)info->sechdrs[info->index.str].sh_addr;
kallsyms->typetab = init_data_base + info->init_typeoffs;
I suggest adding 'kallsyms->num_func_syms = 0;' after the initialization
of kallsyms->num_symtab.
>
> @@ -194,19 +289,28 @@ void add_kallsyms(struct module *mod, const struct load_info *info)
> mod->core_kallsyms.symtab = dst = data_base + info->symoffs;
> mod->core_kallsyms.strtab = s = data_base + info->stroffs;
> mod->core_kallsyms.typetab = data_base + info->core_typeoffs;
> +
> strtab_size = info->core_typeoffs - info->stroffs;
> src = kallsyms->symtab;
> - for (ndst = i = 0; i < kallsyms->num_symtab; i++) {
> + ndst = info->num_func_syms + 1;
> +
> + for (nfunc = i = 0; i < kallsyms->num_symtab; i++) {
> kallsyms->typetab[i] = elf_type(src + i, info);
> - if (i == 0 || is_livepatch_module(mod) ||
> + if (i == 0 || is_lp_mod ||
> is_core_symbol(src + i, info->sechdrs, info->hdr->e_shnum,
> info->index.pcpu)) {
> ssize_t ret;
>
> - mod->core_kallsyms.typetab[ndst] =
> - kallsyms->typetab[i];
> - dst[ndst] = src[i];
> - dst[ndst++].st_name = s - mod->core_kallsyms.strtab;
> + if (i == 0)
> + di = 0;
> + else if (!is_lp_mod && is_func_symbol(src + i))
> + di = 1 + nfunc++;
> + else
> + di = ndst++;
> +
> + mod->core_kallsyms.typetab[di] = kallsyms->typetab[i];
> + dst[di] = src[i];
> + dst[di].st_name = s - mod->core_kallsyms.strtab;
> ret = strscpy(s, &kallsyms->strtab[src[i].st_name],
> strtab_size);
> if (ret < 0)
> @@ -216,9 +320,13 @@ void add_kallsyms(struct module *mod, const struct load_info *info)
> }
> }
>
> + WARN_ON_ONCE(nfunc != info->num_func_syms);
> + sort(dst + 1, nfunc, sizeof(Elf_Sym), elf_sym_cmp, NULL);
> +
The code sorts mod->core_kallsyms.symtab but mod->core_kallsyms.typetab
is not reordered accordingly.
> /* Set up to point into init section. */
> rcu_assign_pointer(mod->kallsyms, kallsyms);
> mod->core_kallsyms.num_symtab = ndst;
> + mod->core_kallsyms.num_func_syms = nfunc;
> }
>
> #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID)
> @@ -241,11 +349,6 @@ void init_build_id(struct module *mod, const struct load_info *info)
> }
> #endif
>
> -static const char *kallsyms_symbol_name(struct mod_kallsyms *kallsyms, unsigned int symnum)
> -{
> - return kallsyms->strtab + kallsyms->symtab[symnum].st_name;
> -}
> -
> /*
> * Given a module and address, find the corresponding symbol and return its name
> * while providing its size and offset if needed.
> @@ -255,7 +358,10 @@ static const char *find_kallsyms_symbol(struct module *mod,
> unsigned long *size,
> unsigned long *offset)
> {
> - unsigned int i, best = 0;
> + unsigned int (*search)(struct mod_kallsyms *kallsyms,
> + unsigned long addr, unsigned long *bestval,
> + unsigned long *nextval);
> + unsigned int best;
> unsigned long nextval, bestval;
> struct mod_kallsyms *kallsyms = rcu_dereference(mod->kallsyms);
> struct module_memory *mod_mem = NULL;
> @@ -266,6 +372,11 @@ static const char *find_kallsyms_symbol(struct module *mod,
> continue;
> #endif
> if (within_module_mem_type(addr, mod, type)) {
> + if (type == MOD_TEXT && kallsyms->num_func_syms > 0)
> + search = bsearch_func_symbol;
I'm not sure if it is ok to limit the search only to function symbols
when the address lies in MOD_TEXT. The text can theoretically contain
non-function symbols. Could this optimization be adjusted to sort all
MOD_TEXT symbols (excluding anonymous and mapping symbols) and move them
to the front of the symbol table?
> + else
> + search = search_kallsyms_symbol;
> +
> mod_mem = &mod->mem[type];
> break;
> }
> @@ -278,33 +389,7 @@ static const char *find_kallsyms_symbol(struct module *mod,
> nextval = (unsigned long)mod_mem->base + mod_mem->size;
> bestval = (unsigned long)mod_mem->base - 1;
>
> - /*
> - * Scan for closest preceding symbol, and next symbol. (ELF
> - * starts real symbols at 1).
> - */
> - for (i = 1; i < kallsyms->num_symtab; i++) {
> - const Elf_Sym *sym = &kallsyms->symtab[i];
> - unsigned long thisval = kallsyms_symbol_value(sym);
> -
> - if (sym->st_shndx == SHN_UNDEF)
> - continue;
> -
> - /*
> - * We ignore unnamed symbols: they're uninformative
> - * and inserted at a whim.
> - */
> - if (*kallsyms_symbol_name(kallsyms, i) == '\0' ||
> - is_mapping_symbol(kallsyms_symbol_name(kallsyms, i)))
> - continue;
> -
> - if (thisval <= addr && thisval > bestval) {
> - best = i;
> - bestval = thisval;
> - }
> - if (thisval > addr && thisval < nextval)
> - nextval = thisval;
> - }
> -
> + best = search(kallsyms, addr, &bestval, &nextval);
> if (!best)
> return NULL;
>
--
Thanks,
Petr
^ permalink raw reply
* Re: [PATCH 47/48] objtool: Improve and simplify prefix symbol detection
From: Peter Zijlstra @ 2026-04-23 8:55 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <45d385f7112ccb71f991a8524e3f9f48b37c1fd9.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:04:15PM -0700, Josh Poimboeuf wrote:
> Only create prefix symbols for functions that have
> __patchable_function_entries entries, since those are the only functions
> where prefix NOPs are intentional.
__CFI_TYPE() as used in SYM_TYPED_ENTRY() will also generate the NOPs
but will not have __patchable_function_entries, because ASM not
compiler.
^ permalink raw reply
* Re: [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Peter Zijlstra @ 2026-04-23 8:47 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <70107aab81b01f8a2360f052ff550a9e97c30f79.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:04:13PM -0700, Josh Poimboeuf wrote:
> PREFIX_SYMBOLS has a !CFI dependency because the compiler already
> generates __cfi_ prefix symbols for kCFI builds, so objtool-generated
> __pfx_ symbols were considered redundant.
>
> However, the __cfi_ symbols only cover the 5-byte kCFI type hash. With
> FUNCTION_CALL_PADDING, there are also 11 bytes of NOP padding between
> the hash and the function entry which have no symbol to claim them.
If you force the function alignment to 64 bytes, the prefix will also be
64bytes, rather than the normal 16.
> The NOPs can be rewritten with call depth tracking thunks at runtime.
> Without a symbol, unwinders and other tools that symbolize code
> locations misattribute those bytes.
>
> Remove the !CFI guard so objtool creates __pfx_ symbols for all
> CALL_PADDING configs, covering the full padding area regardless of
> whether there's also a __cfi_ symbol.
Egads, that a ton of symbols :/ Does it not make sense to 'fix' up the
__cfi_ symbols to cover the whole prefix?
^ permalink raw reply
* Re: [PATCH 43/48] objtool: Add insn_sym() helper
From: Peter Zijlstra @ 2026-04-23 8:45 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <d0762e0bd04d4d93940c212d2b8080bdced0cb29.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:04:11PM -0700, Josh Poimboeuf wrote:
> Alternative replacement instructions awkwardly have insn->sym set to the
> function they get patched to rather than the symbol (or rather lack
> thereof) they belong to in the file.
>
> This makes it difficult to know where a given instruction actually
> lives.
>
> Add a new insn_sym() helper which preserves the existing semantic of
> insn->sym. Rename insn->sym to insn->_sym, which contains the actual
> ELF binary symbol (or NULL, for alternative replacements) an instruction
> lives in.
>
> The private insn->_sym value will be needed for a subsequent patch.
>
> +/*
> + * Return the symbol associated with an instruction. For alternative
> + * replacements, return the symbol of the original code being replaced rather
> + * than NULL. insn->_sym reflects the actual location in the ELF file.
> + */
> +static inline struct symbol *insn_sym(struct instruction *insn)
> +{
> + struct symbol *sym = insn->_sym;
> +
> + if (!sym && insn->alt_group && insn->alt_group->orig_group)
> + sym = insn->alt_group->orig_group->first_insn->_sym;
> +
> + return sym;
> +}
That is a bit of a deref fest -- this does not affect performance
negatively?
^ permalink raw reply
* Re: [PATCH 34/48] objtool: Consolidate file decoding into decode_file()
From: Peter Zijlstra @ 2026-04-23 8:41 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <1b6557569cbdf893b832c37ba16dafaf69f9c3f6.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:04:02PM -0700, Josh Poimboeuf wrote:
> decode_sections() relies on CFI and cfi_hash initialization done
> separately in check(), making it unusable outside of check().
>
> Consolidate the initialization into decode_sections() and rename it to
> decode_file(), and make it global along with free_insns() and
> insn_reloc() for use by other objtool components -- namely, the checksum
> code which will be moving to another file.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
^ permalink raw reply
* Re: [PATCH 32/48] objtool: Add is_cold_func() helper
From: Peter Zijlstra @ 2026-04-23 8:38 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <8eea11ea7d0efc5fcd2e57a10c4285fe998f0cec.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:04:00PM -0700, Josh Poimboeuf wrote:
> diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> index 00c2389f345f..8a6e1338af97 100644
> --- a/tools/objtool/elf.c
> +++ b/tools/objtool/elf.c
> @@ -586,8 +586,11 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> if (strstarts(sym->name, ".klp.sym"))
> sym->klp = 1;
>
> + sym->pfunc = sym->cfunc = sym;
> +
> if (!sym->klp && !is_sec_sym(sym) && strstr(sym->name, ".cold")) {
> - sym->cold = 1;
> + /* Tell read_symbols() this is a cold subfunction */
> + sym->pfunc = NULL;
>
> /*
> * Clang doesn't mark cold subfunctions as STT_FUNC, which
> @@ -596,8 +599,6 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
> sym->type = STT_FUNC;
> }
>
> - sym->pfunc = sym->cfunc = sym;
> -
> return 0;
> }
So now the cold subfunction has a NULL parent-function and a
child-function that points to the parent?
I'm confused.
^ permalink raw reply
* Re: [PATCH 31/48] objtool: Add is_alias_sym() helper
From: Peter Zijlstra @ 2026-04-23 8:35 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <360097539ed947aea82ce5392548a898a346ffa7.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:03:59PM -0700, Josh Poimboeuf wrote:
> Improve readability with a new is_alias_sym() helper.
>
> No functional changes intended.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> tools/objtool/check.c | 6 +++---
> tools/objtool/include/objtool/elf.h | 5 +++++
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index 54ceac857979..4c18d6e7f6c3 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -491,7 +491,7 @@ static int decode_instructions(struct objtool_file *file)
> return -1;
> }
>
> - if (func->embedded_insn || func->alias != func)
> + if (func->embedded_insn || is_alias_sym(func))
> continue;
>
> if (!find_insn(file, sec, func->offset)) {
> @@ -2229,7 +2229,7 @@ static int add_jump_table_alts(struct objtool_file *file)
> return 0;
>
> for_each_sym(file->elf, func) {
> - if (!is_func_sym(func) || func->alias != func)
> + if (!is_func_sym(func) || is_alias_sym(func))
> continue;
>
> mark_func_jump_tables(file, func);
> @@ -4527,7 +4527,7 @@ static int validate_symbol(struct objtool_file *file, struct section *sec,
> return 1;
> }
>
> - if (sym->pfunc != sym || sym->alias != sym)
> + if (sym->pfunc != sym || is_alias_sym(sym))
> return 0;
>
> insn = find_insn(file, sec, sym->offset);
> diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
> index cd5844c7b4e2..3abe4cbc584c 100644
> --- a/tools/objtool/include/objtool/elf.h
> +++ b/tools/objtool/include/objtool/elf.h
> @@ -279,6 +279,11 @@ static inline bool is_local_sym(struct symbol *sym)
> return sym->bind == STB_LOCAL;
> }
>
> +static inline bool is_alias_sym(struct symbol *sym)
> +{
> + return sym->alias != sym;
> +}
> +
> static inline bool is_prefix_func(struct symbol *sym)
> {
> return sym->prefix;
> --
> 2.53.0
>
^ permalink raw reply
* Re: [PATCH 26/48] objtool/klp: Don't set sym->file for section symbols
From: Peter Zijlstra @ 2026-04-23 8:34 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <a051f7f3c6adb479cabe0b4e1f08552f1412583e.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:03:54PM -0700, Josh Poimboeuf wrote:
> Section symbols aren't grouped after their corresponding FILE symbols.
> Their sym->file should really be NULL rather than whatever random FILE
> happened to be last.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> tools/objtool/elf.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> index c4cb371e72b2..00c2389f345f 100644
> --- a/tools/objtool/elf.c
> +++ b/tools/objtool/elf.c
> @@ -680,7 +680,7 @@ static int read_symbols(struct elf *elf)
>
> if (is_file_sym(sym))
> file = sym;
> - else if (sym->bind == STB_LOCAL)
> + else if (sym->bind == STB_LOCAL && !is_sec_sym(sym))
> sym->file = file;
> }
>
> --
> 2.53.0
>
^ permalink raw reply
* Re: [PATCH 17/48] objtool: Fix reloc hash collision in find_reloc_by_dest_range()
From: Peter Zijlstra @ 2026-04-23 8:32 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <09dab1995c4ba6ca29dd70b0a7472f1a2975fefd.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:03:45PM -0700, Josh Poimboeuf wrote:
> In find_reloc_by_dest_range(), hash collisions can cause a high-offset
> relocation to appear when probing a low-offset hash bucket.
>
> Only return early when the best match found so far genuinely belongs to
> the current bucket (its offset is within the bucket's stride range).
> Otherwise, continue scanning later buckets which may contain
> lower-offset matches.
Maybe mention (and or add a comment to the function) that in case of
multiple matches in the given range, it will return the lowest address
one.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Fixes: 74b873e49d92 ("objtool: Optimize find_rela_by_dest_range()")
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
> ---
> tools/objtool/elf.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> index a5486e172e5c..c4cb371e72b2 100644
> --- a/tools/objtool/elf.c
> +++ b/tools/objtool/elf.c
> @@ -370,11 +370,11 @@ struct reloc *find_reloc_by_dest_range(const struct elf *elf, struct section *se
> r = reloc;
> }
> }
> - if (r)
> + if (r && (reloc_offset(r) & OFFSET_STRIDE_MASK) == o)
> return r;
> }
>
> - return NULL;
> + return r;
> }
>
> struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *sec, unsigned long offset)
> --
> 2.53.0
>
^ permalink raw reply
* Re: [PATCH 05/48] objtool: Move mark_rodata() to elf.c
From: Peter Zijlstra @ 2026-04-23 8:19 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Joe Lawrence, Song Liu,
Miroslav Benes, Petr Mladek
In-Reply-To: <c32b3d8d0770c93f8c0d8e4a989f2f43c29e9a5f.1776916871.git.jpoimboe@kernel.org>
On Wed, Apr 22, 2026 at 09:03:33PM -0700, Josh Poimboeuf wrote:
> Move the sec->rodata marking from check.c to elf.c so it's set during
> ELF reading rather than during the check pipeline. This makes the
> rodata flag available to all objtool users, including klp-diff which
> reads ELF files directly without running check().
>
> Add an is_rodata_sec() helper to elf.h for consistency with
> is_text_sec() and is_string_sec().
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> tools/objtool/check.c | 11 +++--------
> tools/objtool/elf.c | 13 +++++++++++++
> tools/objtool/include/objtool/elf.h | 5 +++++
> 3 files changed, 21 insertions(+), 8 deletions(-)
>
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index 9b11cf3193b9..5722d4568401 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -2566,7 +2566,6 @@ static int classify_symbols(struct objtool_file *file)
> static void mark_rodata(struct objtool_file *file)
> {
> struct section *sec;
> - bool found = false;
>
> /*
> * Search for the following rodata sections, each of which can
> @@ -2579,15 +2578,11 @@ static void mark_rodata(struct objtool_file *file)
> * .rodata.str1.* sections are ignored; they don't contain jump tables.
> */
> for_each_sec(file->elf, sec) {
> - if ((!strncmp(sec->name, ".rodata", 7) &&
> - !strstr(sec->name, ".str1.")) ||
> - !strncmp(sec->name, ".data.rel.ro", 12)) {
> - sec->rodata = true;
> - found = true;
> + if (is_rodata_sec(sec)) {
> + file->rodata = true;
> + return;
> }
> }
> -
> - file->rodata = found;
> }
>
> static void mark_holes(struct objtool_file *file)
> diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
> index f3df2bde119f..ac9da81a7a2f 100644
> --- a/tools/objtool/elf.c
> +++ b/tools/objtool/elf.c
> @@ -1172,6 +1172,17 @@ static int read_relocs(struct elf *elf)
> return 0;
> }
>
> +static void mark_rodata(struct elf *elf)
> +{
> + struct section *sec;
> +
> + for_each_sec(elf, sec) {
> + if ((strstarts(sec->name, ".rodata") && !strstr(sec->name, ".str1.")) ||
> + strstarts(sec->name, ".data.rel.ro"))
> + sec->rodata = true;
> + }
> +}
> +
> struct elf *elf_open_read(const char *name, int flags)
> {
> struct elf *elf;
> @@ -1222,6 +1233,8 @@ struct elf *elf_open_read(const char *name, int flags)
> if (read_sections(elf))
> goto err;
>
> + mark_rodata(elf);
> +
> if (read_symbols(elf))
> goto err;
>
> diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
> index 25573e5af76e..c61bd57767f9 100644
> --- a/tools/objtool/include/objtool/elf.h
> +++ b/tools/objtool/include/objtool/elf.h
> @@ -296,6 +296,11 @@ static inline bool is_text_sec(struct section *sec)
> return sec->sh.sh_flags & SHF_EXECINSTR;
> }
>
> +static inline bool is_rodata_sec(struct section *sec)
> +{
> + return sec->rodata;
> +}
> +
> static inline bool sec_changed(struct section *sec)
> {
> return sec->_changed;
> --
> 2.53.0
>
^ permalink raw reply
* [PATCH 48/48] objtool/klp: Cache dont_correlate() result
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
Cache the dont_correlate() result once per symbol at the start of
correlate_symbols(). This reduces klp diff time on an arm64 LTO
vmlinux.o from 2m51s to 35s.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/elf.h | 1 +
tools/objtool/klp-diff.c | 29 +++++++++++++++++------------
2 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index 21441bd72971..fb4fec7d8a6e 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -95,6 +95,7 @@ struct symbol {
u8 changed : 1;
u8 included : 1;
u8 klp : 1;
+ u8 dont_correlate : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 420d05633aba..a848015b477c 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -566,7 +566,7 @@ static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
/* Count orig candidates */
for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
(!maybe_same_file(sym1, sym2)))
continue;
@@ -592,7 +592,7 @@ static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
/* Count patched candidates */
for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2))
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate)
continue;
/* Level 1 */
@@ -728,7 +728,7 @@ static struct symbol *find_twin_suffixed(struct elf *elf, struct symbol *sym1)
return NULL;
for_each_sym_by_name(elf, name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2))
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate)
continue;
count++;
match = sym2;
@@ -762,7 +762,7 @@ static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
struct symbol *sym2, *match = NULL;
for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
!maybe_same_file(sym1, sym2))
continue;
if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
@@ -774,7 +774,7 @@ static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
}
for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
!maybe_same_file(sym1, sym2))
continue;
if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
@@ -806,6 +806,11 @@ static int correlate_symbols(struct elfs *e)
struct symbol *sym1, *sym2;
bool progress;
+ for_each_sym(e->orig, sym1)
+ sym1->dont_correlate = dont_correlate(sym1);
+ for_each_sym(e->patched, sym2)
+ sym2->dont_correlate = dont_correlate(sym2);
+
/* Correlate FILE symbols */
file1_sym = first_file_symbol(e->orig);
file2_sym = first_file_symbol(e->patched);
@@ -846,7 +851,7 @@ static int correlate_symbols(struct elfs *e)
do {
progress = false;
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin(e, sym1);
if (!sym2)
@@ -860,7 +865,7 @@ static int correlate_symbols(struct elfs *e)
return -1;
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin_suffixed(e->patched, sym1);
if (!sym2)
@@ -872,7 +877,7 @@ static int correlate_symbols(struct elfs *e)
} while (progress);
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin_positional(e, sym1);
if (!sym2)
@@ -882,7 +887,7 @@ static int correlate_symbols(struct elfs *e)
}
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
WARN("no correlation: %s", sym1->name);
}
@@ -1102,7 +1107,7 @@ static int mark_changed_functions(struct elfs *e)
/* Find changed functions */
for_each_sym(e->orig, orig_sym) {
- if (dont_correlate(orig_sym))
+ if (orig_sym->dont_correlate)
continue;
patched_sym = orig_sym->twin;
@@ -1123,7 +1128,7 @@ static int mark_changed_functions(struct elfs *e)
/* Find added functions and print them */
for_each_sym(e->patched, patched_sym) {
- if (!is_func_sym(patched_sym) || dont_correlate(patched_sym))
+ if (!is_func_sym(patched_sym) || patched_sym->dont_correlate)
continue;
if (!patched_sym->twin) {
@@ -1263,7 +1268,7 @@ static bool klp_reloc_needed(struct reloc *patched_reloc)
struct export *export;
/* no external symbol to reference */
- if (dont_correlate(patched_sym))
+ if (patched_sym->dont_correlate)
return false;
/* For included functions, a regular reloc will do. */
--
2.53.0
^ permalink raw reply related
* [PATCH 47/48] objtool: Improve and simplify prefix symbol detection
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
Only create prefix symbols for functions that have
__patchable_function_entries entries, since those are the only functions
where prefix NOPs are intentional.
This both simplifies the detection and makes it more accurate.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 90 ++++++++++---------------------------------
1 file changed, 21 insertions(+), 69 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 410061aeed26..d438c5a4444b 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -4271,17 +4271,6 @@ static bool ignore_unreachable_insn(struct objtool_file *file, struct instructio
* For FineIBT or kCFI, a certain number of bytes preceding the function may be
* NOPs. Those NOPs may be rewritten at runtime and executed, so give them a
* proper function name: __pfx_<func>.
- *
- * The NOPs may not exist for the following cases:
- *
- * - compiler cloned functions (*.cold, *.part0, etc)
- * - asm functions created with inline asm or without SYM_FUNC_START()
- *
- * Also, the function may already have a prefix from a previous objtool run
- * (livepatch extracted functions, or manually running objtool multiple times).
- *
- * So return 0 if the NOPs are missing or the function already has a prefix
- * symbol.
*/
static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
{
@@ -4289,10 +4278,6 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
char name[SYM_NAME_LEN];
struct cfi_state *cfi;
- if (!is_func_sym(func) || is_prefix_func(func) || is_cold_func(func) ||
- func->static_call_tramp)
- return 0;
-
if ((strlen(func->name) + sizeof("__pfx_") > SYM_NAME_LEN)) {
WARN("%s: symbol name too long, can't create __pfx_ symbol",
func->name);
@@ -4302,59 +4287,21 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
if (snprintf_check(name, SYM_NAME_LEN, "__pfx_%s", func->name))
return -1;
- if (file->klp) {
- struct symbol *pfx;
-
- pfx = find_symbol_by_offset(func->sec, func->offset - opts.prefix);
- if (pfx && is_prefix_func(pfx) && !strcmp(pfx->name, name))
- return 0;
- }
-
- insn = find_insn(file, func->sec, func->offset);
- if (!insn) {
- WARN("%s: can't find starting instruction", func->name);
+ if (!elf_create_symbol(file->elf, name, func->sec,
+ GELF_ST_BIND(func->sym.st_info),
+ GELF_ST_TYPE(func->sym.st_info),
+ func->offset - opts.prefix, opts.prefix))
return -1;
- }
-
- for (prev = prev_insn_same_sec(file, insn);
- prev;
- prev = prev_insn_same_sec(file, prev)) {
- u64 offset;
-
- if (prev->type != INSN_NOP)
- return 0;
-
- offset = func->offset - prev->offset;
-
- if (offset > opts.prefix)
- return 0;
-
- if (offset < opts.prefix)
- continue;
-
- if (!elf_create_symbol(file->elf, name, func->sec,
- GELF_ST_BIND(func->sym.st_info),
- GELF_ST_TYPE(func->sym.st_info),
- prev->offset, opts.prefix))
- return -1;
-
- break;
- }
-
- if (!prev)
- return 0;
-
- if (!insn->cfi) {
- /*
- * This can happen if stack validation isn't enabled or the
- * function is annotated with STACK_FRAME_NON_STANDARD.
- */
- return 0;
- }
/* Propagate insn->cfi to the prefix code */
+ insn = find_insn(file, func->sec, func->offset);
+ if (!insn || !insn->cfi)
+ return 0;
+
cfi = cfi_hash_find_or_add(insn->cfi);
- for (; prev != insn; prev = next_insn_same_sec(file, prev))
+ for (prev = find_insn(file, func->sec, func->offset - opts.prefix);
+ prev && prev != insn;
+ prev = next_insn_same_sec(file, prev))
prev->cfi = cfi;
return 0;
@@ -4362,15 +4309,20 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
static int create_prefix_symbols(struct objtool_file *file)
{
- struct section *sec;
+ struct section *pfe_sec;
struct symbol *func;
+ struct reloc *reloc;
- for_each_sec(file->elf, sec) {
- if (!is_text_sec(sec))
+ for_each_sec(file->elf, pfe_sec) {
+ if (strcmp(pfe_sec->name, "__patchable_function_entries"))
+ continue;
+ if (!pfe_sec->rsec)
continue;
- sec_for_each_sym(sec, func) {
- if (create_prefix_symbol(file, func))
+ for_each_reloc(pfe_sec->rsec, reloc) {
+ func = find_func_by_offset(reloc->sym->sec,
+ reloc->sym->offset + reloc_addend(reloc) + opts.prefix);
+ if (func && create_prefix_symbol(file, func))
return -1;
}
}
--
2.53.0
^ permalink raw reply related
* [PATCH 46/48] objtool/klp: Make function prefix handling more generic
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
The way x86 klp-diff handles function prefix symbols is a bit awkward,
Also, x86 is the only arch which needs the __pfx_/cfi_ prefix symbols,
so this approach isn't extensible to other arches. And while other
arches *do* use __patchable_function_entries (PFEs), they use them in
completely different ways.
In preparation for supporting other arches, use a more generic approach
that will work for all arches with prefixed areas and/or PFE sections.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/elf.h | 15 +--
tools/objtool/klp-diff.c | 187 ++++++++++++++++++++++++----
2 files changed, 161 insertions(+), 41 deletions(-)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index ba13dd67cf26..21441bd72971 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -119,6 +119,7 @@ struct elf {
struct list_head sections;
struct list_head symbols;
unsigned long num_relocs;
+ int pfe_offset;
int symbol_bits;
int symbol_name_bits;
@@ -532,20 +533,6 @@ static inline void set_sym_next_reloc(struct reloc *reloc, struct reloc *next)
reloc && reloc_offset(reloc) < sym->offset + sym->len; \
reloc = rsec_next_reloc(sym->sec->rsec, reloc))
-static inline struct symbol *get_func_prefix(struct symbol *func)
-{
- struct symbol *prev;
-
- if (!is_func_sym(func))
- return NULL;
-
- prev = sec_prev_sym(func);
- if (prev && is_prefix_func(prev))
- return prev;
-
- return NULL;
-}
-
#define OFFSET_STRIDE_BITS 4
#define OFFSET_STRIDE (1UL << OFFSET_STRIDE_BITS)
#define OFFSET_STRIDE_MASK (~(OFFSET_STRIDE - 1))
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index acb76aefd04f..420d05633aba 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -213,6 +213,88 @@ static int read_sym_checksums(struct elf *elf)
return 0;
}
+/*
+ * Detect the offset from the function entry point to its
+ * __patchable_function_entries (PFE) relocation target.
+ *
+ * offset < 0 (before function entry):
+ *
+ * CONFIG_FINEIBT (x86)
+ * CONFIG_MITIGATION_CALL_DEPTH_TRACKING (x86)
+ */
+static int read_pfe_offset(struct elf *elf)
+{
+ bool has_pfe = false;
+ struct section *sec;
+
+ for_each_sec(elf, sec) {
+ struct reloc *reloc;
+
+ if (strcmp(sec->name, "__patchable_function_entries"))
+ continue;
+ if (!sec->rsec)
+ continue;
+
+ has_pfe = true;
+
+ for_each_reloc(sec->rsec, reloc) {
+ unsigned long target = reloc->sym->offset + reloc_addend(reloc);
+ struct symbol *func;
+
+ func = find_func_containing(reloc->sym->sec, target);
+ if (func) {
+ if (is_prefix_func(func))
+ elf->pfe_offset = target - (func->offset + func->len);
+ else
+ elf->pfe_offset = target - func->offset;
+ return 0;
+ }
+ }
+ }
+
+ if (has_pfe) {
+ ERROR("can't find __patchable_function_entries offset");
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Detect the size of the area before a function's entry point. This prefix
+ * area is used for CFI type hashes, call thunks, or ftrace call ops.
+ *
+ * __pfx_ prefix function (x86):
+ *
+ * CONFIG_MITIGATION_CALL_DEPTH_TRACKING
+ *
+ * __cfi_ prefix function (x86):
+ *
+ * CONFIG_CFI
+ */
+static unsigned long func_pfx_size(struct elf *elf, struct symbol *func)
+{
+ struct symbol *pfx;
+
+ /* x86 __pfx_ and/or __cfi_ */
+ if (func->offset) {
+ pfx = find_func_containing(func->sec, func->offset - 1);
+ if (pfx && pfx->prefix) {
+ struct symbol *pfx2;
+
+ /* FineIBT has both */
+ if (pfx->offset) {
+ pfx2 = find_func_containing(func->sec, pfx->offset - 1);
+ if (pfx2 && pfx2->prefix)
+ pfx = pfx2;
+ }
+
+ return func->offset - pfx->offset;
+ }
+ }
+ return 0;
+}
+
static struct symbol *first_file_symbol(struct elf *elf)
{
struct symbol *sym;
@@ -302,6 +384,7 @@ static bool is_special_section(struct section *sec)
"__ex_table",
"__jump_table",
"__mcount_loc",
+ "__patchable_function_entries",
/*
* Extract .static_call_sites here to inherit non-module
@@ -872,7 +955,7 @@ static struct symbol *__clone_symbol(struct elf *elf, struct symbol *patched_sym
bool data_too)
{
struct section *out_sec = NULL;
- unsigned long offset = 0;
+ unsigned long offset = 0, pfx_size = 0;
struct symbol *out_sym;
if (data_too && !is_undef_sym(patched_sym)) {
@@ -901,20 +984,26 @@ static struct symbol *__clone_symbol(struct elf *elf, struct symbol *patched_sym
offset = ALIGN(sec_size(out_sec), out_sec->sh.sh_addralign);
if (patched_sym->len || is_sec_sym(patched_sym)) {
- void *data = NULL;
size_t size;
+ void *data = NULL;
+
+ /* Clone function prefix area */
+ if (is_func_sym(patched_sym))
+ pfx_size = func_pfx_size(elf, patched_sym);
/* bss doesn't have data */
if (patched_sym->sec->data && patched_sym->sec->data->d_buf)
- data = patched_sym->sec->data->d_buf + patched_sym->offset;
+ data = patched_sym->sec->data->d_buf + patched_sym->offset - pfx_size;
if (is_sec_sym(patched_sym))
size = sec_size(patched_sym->sec);
else
- size = patched_sym->len;
+ size = patched_sym->len + pfx_size;
if (!elf_add_data(elf, out_sec, data, size))
return NULL;
+
+ offset += pfx_size;
}
}
@@ -924,6 +1013,23 @@ static struct symbol *__clone_symbol(struct elf *elf, struct symbol *patched_sym
if (!out_sym)
return NULL;
+ /*
+ * The copied prefixed area may have had a __cfi_ symbol which needs to
+ * be copied. During the module link, objtool collates these in a
+ * .cfi_sites section for FineIBT.
+ */
+ if (pfx_size && is_func_sym(patched_sym)) {
+ struct symbol *cfi_sym;
+
+ cfi_sym = find_func_containing(patched_sym->sec, patched_sym->offset - pfx_size);
+ if (cfi_sym && strstarts(cfi_sym->name, "__cfi_")) {
+ if (!elf_create_symbol(elf, cfi_sym->name, out_sec,
+ cfi_sym->bind, cfi_sym->type,
+ offset - pfx_size, cfi_sym->len))
+ return NULL;
+ }
+ }
+
sym_created:
patched_sym->clone = out_sym;
out_sym->clone = patched_sym;
@@ -960,20 +1066,11 @@ static const char *sym_bind(struct symbol *sym)
static struct symbol *clone_symbol(struct elfs *e, struct symbol *patched_sym,
bool data_too)
{
- struct symbol *pfx;
-
if (patched_sym->clone)
return patched_sym->clone;
dbg_clone("%s%s", patched_sym->name, data_too ? " [+DATA]" : "");
- /* Make sure the prefix gets cloned first */
- if (is_func_sym(patched_sym) && data_too) {
- pfx = get_func_prefix(patched_sym);
- if (pfx)
- clone_symbol(e, pfx, true);
- }
-
if (!__clone_symbol(e->out, patched_sym, data_too))
return NULL;
@@ -985,15 +1082,8 @@ static struct symbol *clone_symbol(struct elfs *e, struct symbol *patched_sym,
static void mark_included_function(struct symbol *func)
{
- struct symbol *pfx;
-
func->included = 1;
- /* Include prefix function */
- pfx = get_func_prefix(func);
- if (pfx)
- pfx->included = 1;
-
/* Make sure .cold parent+child always stay together */
if (func->cfunc && func->cfunc != func)
func->cfunc->included = 1;
@@ -1222,17 +1312,37 @@ static int convert_reloc_sym_to_secsym(struct elf *elf, struct reloc *reloc)
return 0;
}
+/*
+ * __patchable_function_entries relocs point to the patchable entry NOPs,
+ * which are at 'pfe_offset' bytes from the function symbol.
+ *
+ * Some entries (e.g., removed weak functions, syscall -ENOSYS stubs) don't
+ * have a corresponding function symbol. Skip those with a return value of 1.
+ */
+static int convert_pfe_reloc(struct elf *elf, struct reloc *reloc)
+{
+ struct symbol *func;
+
+ func = find_func_by_offset(reloc->sym->sec,
+ reloc->sym->offset +
+ reloc_addend(reloc) - elf->pfe_offset);
+ if (!func)
+ return 1;
+
+ reloc->sym = func;
+ set_reloc_sym(elf, reloc, func->idx);
+ set_reloc_addend(elf, reloc, elf->pfe_offset);
+ return 0;
+}
+
static int convert_reloc_secsym_to_sym(struct elf *elf, struct reloc *reloc)
{
struct symbol *sym = reloc->sym;
struct section *sec = sym->sec;
- /* If the symbol has a dedicated section, it's easy to find */
- sym = find_symbol_by_offset(sec, 0);
- if (sym && sym->len == sec_size(sec))
- goto found_sym;
+ if (!strcmp(reloc->sec->name, ".rela__patchable_function_entries"))
+ return convert_pfe_reloc(elf, reloc);
- /* No dedicated section; find the symbol manually */
sym = find_symbol_containing(sec, arch_adjusted_addend(reloc));
if (!sym) {
/*
@@ -1249,7 +1359,6 @@ static int convert_reloc_secsym_to_sym(struct elf *elf, struct reloc *reloc)
return -1;
}
-found_sym:
reloc->sym = sym;
set_reloc_sym(elf, reloc, sym->idx);
set_reloc_addend(elf, reloc, reloc_addend(reloc) - sym->offset);
@@ -1802,6 +1911,9 @@ static int validate_special_section_klp_reloc(struct elfs *e, struct symbol *sym
static int clone_special_section(struct elfs *e, struct section *patched_sec)
{
+ bool is_pfe = !strcmp(patched_sec->name, "__patchable_function_entries");
+ struct section *out_sec = NULL;
+ struct reloc *patched_reloc;
struct symbol *patched_sym;
/*
@@ -1809,6 +1921,7 @@ static int clone_special_section(struct elfs *e, struct section *patched_sec)
* reference included functions.
*/
sec_for_each_sym(patched_sec, patched_sym) {
+ struct symbol *out_sym;
int ret;
if (!is_object_sym(patched_sym))
@@ -1823,8 +1936,23 @@ static int clone_special_section(struct elfs *e, struct section *patched_sec)
if (ret > 0)
continue;
- if (!clone_symbol(e, patched_sym, true))
+ out_sym = clone_symbol(e, patched_sym, true);
+ if (!out_sym)
return -1;
+
+ if (!is_pfe || (out_sec && out_sec->sh.sh_link))
+ continue;
+
+ /*
+ * For reasons, the patched object has multiple PFE sections,
+ * but we only need to create one combined section for the
+ * output. Link the single PFE ouput section to a random text
+ * section to satisfy the linker for SHF_LINK_ORDER.
+ */
+ out_sec = out_sym->sec;
+ patched_reloc = find_reloc_by_dest(e->patched, patched_sec,
+ patched_sym->offset);
+ out_sec->sh.sh_link = patched_reloc->sym->clone->sec->idx;
}
return 0;
@@ -2121,6 +2249,9 @@ int cmd_klp_diff(int argc, const char **argv)
if (read_sym_checksums(e.patched))
return -1;
+ if (read_pfe_offset(e.patched))
+ return -1;
+
if (correlate_symbols(&e))
return -1;
@@ -2134,6 +2265,8 @@ int cmd_klp_diff(int argc, const char **argv)
if (!e.out)
return -1;
+ e.out->pfe_offset = e.patched->pfe_offset;
+
/*
* Special section fake symbols are needed so that individual special
* section entries can be extracted by clone_special_sections().
--
2.53.0
^ permalink raw reply related
* [PATCH 45/48] x86/Kconfig: Enable CONFIG_PREFIX_SYMBOLS for FineIBT
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
PREFIX_SYMBOLS has a !CFI dependency because the compiler already
generates __cfi_ prefix symbols for kCFI builds, so objtool-generated
__pfx_ symbols were considered redundant.
However, the __cfi_ symbols only cover the 5-byte kCFI type hash. With
FUNCTION_CALL_PADDING, there are also 11 bytes of NOP padding between
the hash and the function entry which have no symbol to claim them.
The NOPs can be rewritten with call depth tracking thunks at runtime.
Without a symbol, unwinders and other tools that symbolize code
locations misattribute those bytes.
Remove the !CFI guard so objtool creates __pfx_ symbols for all
CALL_PADDING configs, covering the full padding area regardless of
whether there's also a __cfi_ symbol.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
arch/x86/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f3f7cb01d69d..493b0038ac8d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2439,7 +2439,7 @@ config CALL_THUNKS
config PREFIX_SYMBOLS
def_bool y
- depends on CALL_PADDING && !CFI
+ depends on CALL_PADDING
menuconfig CPU_MITIGATIONS
bool "Mitigations for CPU vulnerabilities"
--
2.53.0
^ permalink raw reply related
* [PATCH 44/48] objtool/klp: Fix position-dependent checksums for non-relocated jumps/calls
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
When computing klp checksums, instructions with non-relocated jump/call
destination offsets are problematic because the offset values can change
when surrounding code has moved, causing the function to be incorrectly
marked as changed.
Specifically, that includes jumps from alternatives to the end of the
alternative, which from objtool's perspective are jumps to the end of
the alternative instruction block in the original function.
Note that 'jump_dest' jumps don't include sibling calls (those use
call_dest), nor do they include jumps to/from .cold sub functions (those
are cross-section and need a reloc).
Fix it by hashing the opcode bytes (excluding the immediate operand)
along with a position-independent representation of the destination.
For calls, use the function name, and for jumps, use the destination's
offset within its function.
[Note the "9 bit hole" comment was wrong: it has been 8 bits since
commit 70589843b36f ("objtool: Add option to trace function validation")
added the 'trace' field. Adding the 4-bit 'immediate_len' field now
leaves a 4-bit hole.]
Fixes: 0d83da43b1e1 ("objtool/klp: Add --checksum option to generate per-function checksums")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/arch/x86/decode.c | 17 ++++++++-
tools/objtool/include/objtool/arch.h | 3 ++
tools/objtool/include/objtool/check.h | 3 +-
tools/objtool/klp-checksum.c | 53 ++++++++++++++++++++++++---
4 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index 350b8ee6e776..1b387d5a195b 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -805,14 +805,27 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
break;
}
- if (ins.immediate.nbytes)
+ if (ins.immediate.nbytes) {
insn->immediate = ins.immediate.value;
- else if (ins.displacement.nbytes)
+ insn->immediate_len = ins.immediate.nbytes;
+ } else if (ins.displacement.nbytes) {
insn->immediate = ins.displacement.value;
+ insn->immediate_len = ins.displacement.nbytes;
+ }
return 0;
}
+size_t arch_jump_opcode_bytes(struct objtool_file *file, struct instruction *insn,
+ unsigned char *buf)
+{
+ size_t len;
+
+ len = insn->len - insn->immediate_len;
+ memcpy(buf, insn->sec->data->d_buf + insn->offset, len);
+ return len;
+}
+
void arch_initial_func_cfi_state(struct cfi_init_state *state)
{
int i;
diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/objtool/arch.h
index 8866158975fc..96d828a8401f 100644
--- a/tools/objtool/include/objtool/arch.h
+++ b/tools/objtool/include/objtool/arch.h
@@ -79,6 +79,9 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
unsigned long offset, unsigned int maxlen,
struct instruction *insn);
+size_t arch_jump_opcode_bytes(struct objtool_file *file, struct instruction *insn,
+ unsigned char *buf);
+
bool arch_callee_saved_reg(unsigned char reg);
unsigned long arch_jump_destination(struct instruction *insn);
diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h
index 0c53476528a8..bc99355e66e9 100644
--- a/tools/objtool/include/objtool/check.h
+++ b/tools/objtool/include/objtool/check.h
@@ -68,6 +68,7 @@ struct instruction {
s8 instr;
u32 idx : INSN_CHUNK_BITS,
+ immediate_len : 4,
dead_end : 1,
ignore_alts : 1,
hint : 1,
@@ -81,7 +82,7 @@ struct instruction {
hole : 1,
fake : 1,
trace : 1;
- /* 9 bit hole */
+ /* 4 bit hole */
struct alt_group *alt_group;
struct instruction *jump_dest;
diff --git a/tools/objtool/klp-checksum.c b/tools/objtool/klp-checksum.c
index adfd02447a45..4100b4dfba86 100644
--- a/tools/objtool/klp-checksum.c
+++ b/tools/objtool/klp-checksum.c
@@ -66,17 +66,58 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
if (insn->fake)
return;
- __checksum_update_insn(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
-
if (!reloc) {
struct symbol *call_dest = insn_call_dest(insn);
+ struct instruction *jump_dest = insn->jump_dest;
- if (call_dest)
- __checksum_update_insn(func, insn, call_dest->demangled_name,
- strlen(call_dest->demangled_name));
- goto alts;
+ /*
+ * For a jump/call non-relocated dest offset embedded in the
+ * instruction, the offset may vary due to changes in
+ * surrounding code. Just hash the opcode and a
+ * position-independent representation of the destination.
+ */
+
+ if (call_dest || jump_dest) {
+ unsigned char buf[16];
+ size_t len;
+
+ len = arch_jump_opcode_bytes(file, insn, buf);
+ __checksum_update_insn(func, insn, buf, len);
+
+ if (call_dest) {
+ __checksum_update_insn(func, insn, call_dest->demangled_name,
+ strlen(call_dest->demangled_name));
+
+ } else if (jump_dest) {
+ struct symbol *dest_sym;
+ unsigned long offset;
+
+ /*
+ * use insn->_sym instead of insn_sym() here.
+ * For alternative replacements, the latter
+ * would give the function of the code being
+ * replaced.
+ */
+ dest_sym = jump_dest->_sym;
+ if (!dest_sym)
+ goto alts;
+
+ __checksum_update_insn(func, insn, dest_sym->demangled_name,
+ strlen(dest_sym->demangled_name));
+
+ offset = jump_dest->offset - dest_sym->offset;
+ __checksum_update_insn(func, insn, &offset, sizeof(offset));
+ }
+
+ goto alts;
+ }
}
+ __checksum_update_insn(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
+
+ if (!reloc)
+ goto alts;
+
sym = reloc->sym;
offset = arch_insn_adjusted_addend(insn, reloc);
--
2.53.0
^ permalink raw reply related
* [PATCH 43/48] objtool: Add insn_sym() helper
From: Josh Poimboeuf @ 2026-04-23 4:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1776916871.git.jpoimboe@kernel.org>
Alternative replacement instructions awkwardly have insn->sym set to the
function they get patched to rather than the symbol (or rather lack
thereof) they belong to in the file.
This makes it difficult to know where a given instruction actually
lives.
Add a new insn_sym() helper which preserves the existing semantic of
insn->sym. Rename insn->sym to insn->_sym, which contains the actual
ELF binary symbol (or NULL, for alternative replacements) an instruction
lives in.
The private insn->_sym value will be needed for a subsequent patch.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 31 ++++++++++++---------------
tools/objtool/disas.c | 22 +++++++++----------
tools/objtool/include/objtool/check.h | 19 ++++++++++++++--
tools/objtool/include/objtool/warn.h | 6 +++---
tools/objtool/trace.c | 8 +++----
5 files changed, 48 insertions(+), 38 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index ae047be919c5..410061aeed26 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -495,7 +495,7 @@ static int decode_instructions(struct objtool_file *file)
}
sym_for_each_insn(file, func, insn) {
- insn->sym = func;
+ insn->_sym = func;
if (is_func_sym(func) &&
insn->type == INSN_ENDBR &&
list_empty(&insn->call_node)) {
@@ -859,15 +859,14 @@ static int create_ibt_endbr_seal_sections(struct objtool_file *file)
list_for_each_entry(insn, &file->endbr_list, call_node) {
int *site = (int *)sec->data->d_buf + idx;
- struct symbol *sym = insn->sym;
+ struct symbol *func = insn_func(insn);
*site = 0;
- if (opts.module && sym && is_func_sym(sym) &&
- insn->offset == sym->offset &&
- (!strcmp(sym->name, "init_module") ||
- !strcmp(sym->name, "cleanup_module"))) {
+ if (opts.module && func && insn->offset == func->offset &&
+ (!strcmp(func->name, "init_module") ||
+ !strcmp(func->name, "cleanup_module"))) {
ERROR("%s(): Magic init_module() function name is deprecated, use module_init(fn) instead",
- sym->name);
+ func->name);
return -1;
}
@@ -1581,7 +1580,7 @@ static int add_jump_destinations(struct objtool_file *file)
}
if (!dest_sym || is_sec_sym(dest_sym)) {
- dest_sym = dest_insn->sym;
+ dest_sym = insn_sym(dest_insn);
if (!dest_sym)
goto set_jump_dest;
}
@@ -1597,7 +1596,7 @@ static int add_jump_destinations(struct objtool_file *file)
continue;
}
- if (!insn->sym || insn->sym->pfunc == dest_sym->pfunc)
+ if (!insn_sym(insn) || insn_sym(insn)->pfunc == dest_sym->pfunc)
goto set_jump_dest;
/*
@@ -1770,7 +1769,6 @@ static int handle_group_alt(struct objtool_file *file,
nop->offset = special_alt->new_off + special_alt->new_len;
nop->len = special_alt->orig_len - special_alt->new_len;
nop->type = INSN_NOP;
- nop->sym = orig_insn->sym;
nop->alt_group = new_alt_group;
nop->fake = 1;
}
@@ -1789,7 +1787,6 @@ static int handle_group_alt(struct objtool_file *file,
last_new_insn = insn;
- insn->sym = orig_insn->sym;
insn->alt_group = new_alt_group;
/*
@@ -2432,12 +2429,12 @@ static int __annotate_late(struct objtool_file *file, int type, struct instructi
break;
case ANNOTYPE_NOCFI:
- sym = insn->sym;
+ sym = insn_sym(insn);
if (!sym) {
ERROR_INSN(insn, "dodgy NOCFI annotation");
return -1;
}
- insn->sym->nocfi = 1;
+ insn_sym(insn)->nocfi = 1;
break;
default:
@@ -2538,7 +2535,7 @@ static void mark_holes(struct objtool_file *file)
* favour of a regular symbol, but leaves the code in place.
*/
for_each_insn(file, insn) {
- if (insn->sym || !find_symbol_hole_containing(insn->sec, insn->offset)) {
+ if (insn_sym(insn) || !find_symbol_hole_containing(insn->sec, insn->offset)) {
in_hole = false;
continue;
}
@@ -2982,7 +2979,7 @@ static int update_cfi_state(struct instruction *insn,
}
if (op->dest.reg == CFI_BP && op->src.reg == CFI_SP &&
- insn->sym->frame_pointer) {
+ insn_sym(insn)->frame_pointer) {
/* addi.d fp,sp,imm on LoongArch */
if (cfa->base == CFI_SP && cfa->offset == op->src.offset) {
cfa->base = CFI_BP;
@@ -2994,7 +2991,7 @@ static int update_cfi_state(struct instruction *insn,
if (op->dest.reg == CFI_SP && op->src.reg == CFI_BP) {
/* addi.d sp,fp,imm on LoongArch */
if (cfa->base == CFI_BP && cfa->offset == 0) {
- if (insn->sym->frame_pointer) {
+ if (insn_sym(insn)->frame_pointer) {
cfa->base = CFI_SP;
cfa->offset = -op->src.offset;
}
@@ -4171,7 +4168,7 @@ static int validate_retpoline(struct objtool_file *file)
* broken.
*/
list_for_each_entry(insn, &file->retpoline_call_list, call_node) {
- struct symbol *sym = insn->sym;
+ struct symbol *sym = insn_sym(insn);
if (sym && (is_notype_sym(sym) ||
is_func_sym(sym)) && !sym->nocfi) {
diff --git a/tools/objtool/disas.c b/tools/objtool/disas.c
index 59090234af19..e6a54a83605c 100644
--- a/tools/objtool/disas.c
+++ b/tools/objtool/disas.c
@@ -210,7 +210,7 @@ static bool disas_print_addr_alt(bfd_vma addr, struct disassemble_info *dinfo)
offset = addr - alt_group->first_insn->offset;
addr = orig_first_insn->offset + offset;
- sym = orig_first_insn->sym;
+ sym = insn_sym(orig_first_insn);
disas_print_addr_sym(orig_first_insn->sec, sym, addr, dinfo);
@@ -222,15 +222,13 @@ static void disas_print_addr_noreloc(bfd_vma addr,
{
struct disas_context *dctx = dinfo->application_data;
struct instruction *insn = dctx->insn;
- struct symbol *sym = NULL;
+ struct symbol *sym = insn_sym(insn);
if (disas_print_addr_alt(addr, dinfo))
return;
- if (insn->sym && addr >= insn->sym->offset &&
- addr < insn->sym->offset + insn->sym->len) {
- sym = insn->sym;
- }
+ if (sym && (addr < sym->offset || addr >= sym->offset + sym->len))
+ sym = NULL;
disas_print_addr_sym(insn->sec, sym, addr, dinfo);
}
@@ -291,9 +289,9 @@ static void disas_print_address(bfd_vma addr, struct disassemble_info *dinfo)
* up. So check it first.
*/
jump_dest = insn->jump_dest;
- if (jump_dest && jump_dest->sym && jump_dest->offset == addr) {
+ if (jump_dest && insn_sym(jump_dest) && jump_dest->offset == addr) {
if (!disas_print_addr_alt(addr, dinfo))
- disas_print_addr_sym(jump_dest->sec, jump_dest->sym,
+ disas_print_addr_sym(jump_dest->sec, insn_sym(jump_dest),
addr, dinfo);
return;
}
@@ -768,8 +766,8 @@ static int disas_alt_jump(struct disas_alt *dalt)
if (orig_insn->len == 5)
suffix[0] = 'q';
str = strfmt("jmp%-3s %lx <%s+0x%lx>", suffix,
- dest_insn->offset, dest_insn->sym->name,
- dest_insn->offset - dest_insn->sym->offset);
+ dest_insn->offset, insn_sym(dest_insn)->name,
+ dest_insn->offset - insn_sym(dest_insn)->offset);
nops = 0;
} else {
str = strfmt("nop%d", orig_insn->len);
@@ -794,8 +792,8 @@ static int disas_alt_extable(struct disas_alt *dalt)
alt_insn = dalt->alt->insn;
str = strfmt("resume at 0x%lx <%s+0x%lx>",
- alt_insn->offset, alt_insn->sym->name,
- alt_insn->offset - alt_insn->sym->offset);
+ alt_insn->offset, insn_sym(alt_insn)->name,
+ alt_insn->offset - insn_sym(alt_insn)->offset);
if (!str)
return -1;
diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h
index eea64728d39b..0c53476528a8 100644
--- a/tools/objtool/include/objtool/check.h
+++ b/tools/objtool/include/objtool/check.h
@@ -94,14 +94,29 @@ struct instruction {
};
};
struct alternative *alts;
- struct symbol *sym;
+ struct symbol *_sym;
struct stack_op *stack_ops;
struct cfi_state *cfi;
};
+/*
+ * Return the symbol associated with an instruction. For alternative
+ * replacements, return the symbol of the original code being replaced rather
+ * than NULL. insn->_sym reflects the actual location in the ELF file.
+ */
+static inline struct symbol *insn_sym(struct instruction *insn)
+{
+ struct symbol *sym = insn->_sym;
+
+ if (!sym && insn->alt_group && insn->alt_group->orig_group)
+ sym = insn->alt_group->orig_group->first_insn->_sym;
+
+ return sym;
+}
+
static inline struct symbol *insn_func(struct instruction *insn)
{
- struct symbol *sym = insn->sym;
+ struct symbol *sym = insn_sym(insn);
if (sym && sym->type != STT_FUNC)
sym = NULL;
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index a9936d60980c..870e147f3a56 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -77,13 +77,13 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define WARN_INSN(insn, format, ...) \
({ \
struct instruction *_insn = (insn); \
- if (!_insn->sym || !_insn->sym->warned) { \
+ if (!insn_sym(_insn) || !insn_sym(_insn)->warned) { \
WARN_FUNC(_insn->sec, _insn->offset, format, \
##__VA_ARGS__); \
BT_INSN(_insn, ""); \
} \
- if (_insn->sym) \
- _insn->sym->warned = 1; \
+ if (insn_sym(_insn)) \
+ insn_sym(_insn)->warned = 1; \
})
#define BT_INSN(insn, format, ...) \
diff --git a/tools/objtool/trace.c b/tools/objtool/trace.c
index 5dec44dab781..61c6aa302bc3 100644
--- a/tools/objtool/trace.c
+++ b/tools/objtool/trace.c
@@ -169,8 +169,8 @@ void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
*/
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s for instruction at 0x%lx <%s+0x%lx>",
alt_name,
- orig_insn->offset, orig_insn->sym->name,
- orig_insn->offset - orig_insn->sym->offset);
+ orig_insn->offset, insn_sym(orig_insn)->name,
+ orig_insn->offset - insn_sym(orig_insn)->offset);
} else {
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s", alt_name);
}
@@ -185,8 +185,8 @@ void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
if (orig_insn->type == INSN_NOP) {
suffix[0] = (orig_insn->len == 5) ? 'q' : '\0';
TRACE_ADDR(orig_insn, "jmp%-3s %lx <%s+0x%lx>", suffix,
- alt_insn->offset, alt_insn->sym->name,
- alt_insn->offset - alt_insn->sym->offset);
+ alt_insn->offset, insn_sym(alt_insn)->name,
+ alt_insn->offset - insn_sym(alt_insn)->offset);
} else {
TRACE_ADDR(orig_insn, "nop%d", orig_insn->len);
trace_depth--;
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox