* Re: [PATCH v2 48/53] objtool: Add insn_sym() helper
From: Song Liu @ 2026-05-01 12:11 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <52633c62366f87d9b78ebd77873a08b9ac6d31c8.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Alternative replacement instructions awkwardly have insn->sym set to the
> function they get patched to rather than the symbol (or rather lack
> thereof) they belong to in the file.
>
> This makes it difficult to know where a given instruction actually
> lives.
>
> Add a new insn_sym() helper which preserves the existing semantic of
> insn->sym. Rename insn->sym to insn->_sym, which contains the actual
> ELF binary symbol (or NULL, for alternative replacements) an instruction
> lives in.
>
> The private insn->_sym value will be needed for a subsequent patch.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 46/53] objtool/klp: Rewrite symbol correlation algorithm
From: Song Liu @ 2026-05-01 12:07 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <27fcb5a17cc7b6821d8b1c4b9812ebb5b4ee6a5c.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Rewrite the symbol correlation code, using a tiered list of
> deterministic strategies in a loop. For duplicately named symbols, each
> tier applies a filter with the goal of finding a 1:1 deterministic
> correlation between the original and patched version of the symbol.
>
> The three matching strategies are:
>
> find_twin(): A funnel of progressively tighter filters. Candidates
> with the same demangled name are counted at four levels: name, scope
> (local-vs-global), file (strict file association), and checksum
> (unchanged functions). The widest level that yields a 1:1 match wins,
> narrower levels are only tried when the wider level is ambiguous.
>
> find_twin_suffixed(): Uses already-correlated LLVM symbol pairs to map
> .llvm.<hash> suffixes from orig to patched. Because all promoted
> symbols from the same TU share the same hash, one correlated pair
> seeds the mapping for the entire TU.
>
> find_twin_positional(): Last resort, matches symbols by position among
> same-named candidates, similar to livepatch sympos. Used for data
> objects like __quirk variables where no deterministic filter can
> distinguish the candidates.
>
> Overall this works much better than the existing algorithm, particularly
> with LTO kernels.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Thanks for improving the correlation algorithm and adding detailed
comments about all these scenarios!
^ permalink raw reply
* Re: [PATCH v2 45/53] objtool/klp: Calculate object checksums
From: Song Liu @ 2026-05-01 10:53 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <88ebcc903a3c534f2c7d35b95f62d845105d40af.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Start checksumming data objects in preparation for revamping the
> correlation algorithm.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 44/53] klp-build: Validate short-circuit prerequisites
From: Song Liu @ 2026-05-01 10:49 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <ec40bb551a583c7a7c329199e41d21a0f086775c.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> The --short-circuit option implicitly requires that certain directories
> are already in klp-tmp. Enforce that to prevent confusing errors.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 38/53] objtool: Add is_cold_func() helper
From: Song Liu @ 2026-05-01 10:43 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <a84513224f38c7c7ca2cf2a4930f87d43a76908b.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:11 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Add an is_cold_func() helper. No functional changes intended.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 32/53] klp-build: Remove redundant SRC and OBJ variables
From: Song Liu @ 2026-05-01 10:42 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <63b0d7848597ad6011e1f56c8fdd53593d09a992.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> SRC and OBJ are both set to $(pwd) and are always identical. The script
> already enforces that klp-build runs from the kernel root directory, and
> builds are done in-place, making these variables unnecessary.
>
> Suggested-by: Song Liu <song@kernel.org>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 24/53] klp-build: Fix checksum comparison for changed offsets
From: Song Liu @ 2026-05-01 10:41 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <6392d4f0c8837ccc0498a1c79a2d9534dacfce82.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> The klp-build -f/--show-first-changed feature uses diff to compare
> checksum log lines between original and patched objects. However, diff
> compares entire lines, including the offset field. When a function is
> at a different section offset, the offset field differs even though the
> instruction checksum is identical, causing the wrong instruction to be
> printed.
>
> Only compare the checksum field when looking for the first changed
> instruction. Also print both the original and patched offsets when they
> differ.
>
> Fixes: 78be9facfb5e ("livepatch/klp-build: Add --show-first-changed option to show function divergence")
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 23/53] klp-build: Fix hang on out-of-date .config
From: Song Liu @ 2026-05-01 10:39 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <5e2a75b8ce5120bbbec6c8e992f1d3c772b8e5d5.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:10 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> If .config is out of date with the kernel source, 'make syncconfig'
> hangs while waiting for user input on new config options. Detect the
> mismatch and return an error.
>
> Fixes: 6f93f7b06810 ("livepatch/klp-build: Fix inconsistent kernel version")
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 21/53] objtool/klp: Fix reloc corruption in convert_reloc_sym_to_secsym()
From: Song Liu @ 2026-05-01 10:38 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <9b419d82a20dbc54be4a59cfec04ab13987a2e6c.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:10 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Use the section symbol's index instead of the old symbol's index when
> updating the ELF relocation entry in convert_reloc_sym_to_secsym().
>
> Found by Sashiko review.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 20/53] objtool/klp: Don't correlate .rodata.cst* constant pool objects
From: Song Liu @ 2026-05-01 10:37 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <80d6f8df4db610a6c9f68031dc0153f04814f2fa.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:08 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Clang aggregates UBSAN type descriptors into shared anonymous
> .data..L__unnamed_* sections. This data is used by UBSAN trap handlers.
>
> When a changed function has an UBSAN bounds check, klp-diff clones the
> entire UBSAN data section associated with the TU. Relocations within
> the cloned section that reference named rodata objects in .rodata.cst*
> (like 'exponent', 'pirq_ali_set.irqmap') become KLP relocations because
> those objects now get correlated.
>
> That results in a .klp.rela.vmlinux..data section which can easily have
> thousands of KLP relocs, most of which are completely superfluous, used
> by functions which aren't cloned to the patch module.
>
> The .rodata.cst* sections are SHF_MERGE constant pool sections
> containing small fixed-size data (lookup tables, bitmasks) that is only
> read by value. Pointer identity is never relevant for these objects, so
> correlating them is unnecessary.
>
> Exclude .rodata.cst* objects from correlation so they get cloned as
> local data instead of generating KLP relocations.
>
> It might be possible to someday treat UBSAN data sections as special
> sections, and only extract the few needed entries. But this works for
> now.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
> ---
> tools/objtool/klp-diff.c | 17 ++++++++++++++++-
> 1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
> index bf37c652188b..ca87bcb9afa3 100644
> --- a/tools/objtool/klp-diff.c
> +++ b/tools/objtool/klp-diff.c
> @@ -372,6 +372,21 @@ static bool is_initcall_sym(struct symbol *sym)
> strstarts(sym->name, "__initstub__");
> }
>
> +/*
> + * Some .rodata is anonymous and can't be correlated due to there being no
> + * symbol names.
> + *
> + * The .rodata.cst* sections aren't technically anonymous, they're SHF_MERGE
> + * constant pool sections containing small fixed-size data (lookup tables,
> + * bitmasks) which are only read by value, so pointer equivalence isn't needed.
> + * They are typically referenced by UBSAN data sections.
> + */
> +static bool is_anonymous_rodata(struct symbol *sym)
> +{
> + return is_rodata_sec(sym->sec) &&
> + (!is_object_sym(sym) || strstarts(sym->sec->name, ".rodata.cst"));
> +}
> +
> /*
> * These symbols should never be correlated, so their local patched versions
> * are used instead of linking to the originals.
> @@ -386,7 +401,7 @@ static bool dont_correlate(struct symbol *sym)
> is_uncorrelated_static_local(sym) ||
> is_local_label(sym) ||
> is_string_sec(sym->sec) ||
> - (is_rodata_sec(sym->sec) && !is_object_sym(sym)) ||
^^^^
This line was added in 19/53. Maybe we can merge 19 and 20?
Thanks,
Song
> + is_anonymous_rodata(sym) ||
> is_initcall_sym(sym) ||
> is_addressable_sym(sym) ||
> is_special_section(sym->sec) ||
> --
> 2.53.0
>
^ permalink raw reply
* Re: [PATCH v2 19/53] objtool/klp: Fix pointer comparisons for rodata objects
From: Song Liu @ 2026-05-01 10:35 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <07de8098fd8981321baab0ff552f65aa2cfc31ec.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:08 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> klp-diff treats all rodata as uncorrelated, so any reference to it uses
> a duplicated copy rather than using a KLP reloc.
>
> For the contents of the data itself, a duplicated copy is fine.
> However, pointer comparisons (e.g., f->f_op == &foo_ops) are broken.
>
> Fix it by correlating non-anonymous rodata objects.
>
> Also, use a new find_symbol_containing_inclusive() helper for matching
> the end of a symbol so bounds calculations don't get broken, for the
> case where an array or other symbol's ending address is used as part of
> a bounds calculation.
>
> While these are really two distinct changes, they need to be done in the
> same patch so as to avoid introducing bisection regressions.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 18/53] objtool/klp: Simplify reloc symbol conversion
From: Song Liu @ 2026-05-01 10:31 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <9572b2e15500e5ed8dcbaac78c966557d3000d85.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:09 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Inline section_reference_needed() and is_reloc_allowed() into
> convert_reloc_sym() and remove the redundant is_reloc_allowed() check in
> clone_reloc().
>
> Move the is_sec_sym() checks into the convert callees so they become
> no-ops when the reloc is already in the right format. This allows
> convert_reloc_sym() to unconditionally dispatch to the right converter
> based on section type.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 09/53] objtool: Replace iterator callback with for_each_sym_by_mangled_name()
From: Song Liu @ 2026-05-01 10:28 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <cb95eae9cc63ca04f881c69c93eed6bac0c751fe.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:08 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Convert the callback-based iterate_sym_by_demangled_name() with a new
> for_each_sym_by_demangled_name() macro. This eliminates the callback
> struct/function and makes the code more compact and readable.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 07/53] objtool/klp: Improve local label check
From: Song Liu @ 2026-05-01 10:27 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <54175a1446aae952ad4d886eec3f64fcbdbeb375.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:08 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Clang emits various .L-prefixed local symbols beyond .Ltmp*, such as
> .L__const.* for local constant data. These are assembler-local labels
> not present in kallsyms, so they can never be resolved at module load
> time.
>
> Broaden the check from .Ltmp* to all .L* symbols so they get cloned into
> the patch module instead.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* Re: [PATCH v2 03/53] objtool/klp: Don't correlate __ADDRESSABLE() symbols
From: Song Liu @ 2026-05-01 10:26 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: x86, linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Miroslav Benes, Petr Mladek
In-Reply-To: <8e69c287bd6c33cec1535228e2239b33f4602bc6.1777575752.git.jpoimboe@kernel.org>
On Fri, May 1, 2026 at 5:08 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> Symbols created by __ADDRESSABLE() are only used to convince the
> toolchain not to optimize out the referenced symbol.
>
> Reviewed-by: Miroslav Benes <mbenes@suse.cz>
> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Acked-by: Song Liu <song@kernel.org>
^ permalink raw reply
* [PATCH v2 53/53] objtool/klp: Cache dont_correlate() result
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Cache the dont_correlate() result once per symbol at the start of
correlate_symbols(). This reduces klp diff time on an arm64 LTO
vmlinux.o from 2m51s to 35s.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/elf.h | 1 +
tools/objtool/klp-diff.c | 29 +++++++++++++++++------------
2 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index fccf72cbd343..d9c44df9cc76 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -96,6 +96,7 @@ struct symbol {
u8 changed : 1;
u8 included : 1;
u8 klp : 1;
+ u8 dont_correlate : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index ed3bf1c55001..f8787d7d1454 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -524,7 +524,7 @@ static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
/* Count orig candidates */
for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
(!maybe_same_file(sym1, sym2)))
continue;
@@ -550,7 +550,7 @@ static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
/* Count patched candidates */
for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
!maybe_same_file(sym1, sym2))
continue;
@@ -693,7 +693,7 @@ static struct symbol *find_twin_suffixed(struct elf *elf, struct symbol *sym1)
return NULL;
for_each_sym_by_name(elf, name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2))
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate)
continue;
count++;
match = sym2;
@@ -733,7 +733,7 @@ static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
struct symbol *sym2, *match = NULL;
for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
!maybe_same_file(sym1, sym2))
continue;
if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
@@ -745,7 +745,7 @@ static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
}
for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
- if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ if (sym2->twin || sym1->type != sym2->type || sym2->dont_correlate ||
!maybe_same_file(sym1, sym2))
continue;
if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
@@ -777,6 +777,11 @@ static int correlate_symbols(struct elfs *e)
struct symbol *sym1, *sym2;
bool progress;
+ for_each_sym(e->orig, sym1)
+ sym1->dont_correlate = dont_correlate(sym1);
+ for_each_sym(e->patched, sym2)
+ sym2->dont_correlate = dont_correlate(sym2);
+
/* Correlate FILE symbols */
file1_sym = first_file_symbol(e->orig);
file2_sym = first_file_symbol(e->patched);
@@ -817,7 +822,7 @@ static int correlate_symbols(struct elfs *e)
do {
progress = false;
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin(e, sym1);
if (!sym2)
@@ -831,7 +836,7 @@ static int correlate_symbols(struct elfs *e)
return -1;
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin_suffixed(e->patched, sym1);
if (!sym2)
@@ -843,7 +848,7 @@ static int correlate_symbols(struct elfs *e)
} while (progress);
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
sym2 = find_twin_positional(e, sym1);
if (!sym2)
@@ -853,7 +858,7 @@ static int correlate_symbols(struct elfs *e)
}
for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1))
+ if (sym1->twin || sym1->dont_correlate)
continue;
WARN("no correlation: %s", sym1->name);
}
@@ -1066,7 +1071,7 @@ static int mark_changed_functions(struct elfs *e)
/* Find changed functions */
for_each_sym(e->orig, orig_sym) {
- if (dont_correlate(orig_sym))
+ if (orig_sym->dont_correlate)
continue;
patched_sym = orig_sym->twin;
@@ -1087,7 +1092,7 @@ static int mark_changed_functions(struct elfs *e)
/* Find added functions and print them */
for_each_sym(e->patched, patched_sym) {
- if (!is_func_sym(patched_sym) || dont_correlate(patched_sym))
+ if (!is_func_sym(patched_sym) || patched_sym->dont_correlate)
continue;
if (!patched_sym->twin) {
@@ -1193,7 +1198,7 @@ static bool klp_reloc_needed(struct reloc *patched_reloc)
struct export *export;
/* no external symbol to reference */
- if (dont_correlate(patched_sym))
+ if (patched_sym->dont_correlate)
return false;
/* For included functions, a regular reloc will do. */
--
2.53.0
^ permalink raw reply related
* [PATCH v2 52/53] objtool: Improve and simplify prefix symbol detection
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Only create prefix symbols for functions that have
__patchable_function_entries entries, since those are the only C
functions where prefix NOPs are intentional.
This both simplifies the detection and makes it more accurate.
Note that assembly functions using SYM_TYPED_FUNC_START() can also have
prefixed NOPs, but that macro already creates their __cfi_ symbols.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 90 ++++++++++---------------------------------
1 file changed, 21 insertions(+), 69 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 0d9b859b006e..1635c87a4ac8 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -4296,17 +4296,6 @@ static bool ignore_unreachable_insn(struct objtool_file *file, struct instructio
* For FineIBT or kCFI, a certain number of bytes preceding the function may be
* NOPs. Those NOPs may be rewritten at runtime and executed, so give them a
* proper function name: __pfx_<func>.
- *
- * The NOPs may not exist for the following cases:
- *
- * - compiler cloned functions (*.cold, *.part0, etc)
- * - asm functions created with inline asm or without SYM_FUNC_START()
- *
- * Also, the function may already have a prefix from a previous objtool run
- * (livepatch extracted functions, or manually running objtool multiple times).
- *
- * So return 0 if the NOPs are missing or the function already has a prefix
- * symbol.
*/
static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
{
@@ -4314,10 +4303,6 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
char name[SYM_NAME_LEN];
struct cfi_state *cfi;
- if (!is_func_sym(func) || is_prefix_func(func) || is_cold_func(func) ||
- func->static_call_tramp)
- return 0;
-
if ((strlen(func->name) + sizeof("__pfx_") > SYM_NAME_LEN)) {
WARN("%s: symbol name too long, can't create __pfx_ symbol",
func->name);
@@ -4327,59 +4312,21 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
if (snprintf_check(name, SYM_NAME_LEN, "__pfx_%s", func->name))
return -1;
- if (file->klp) {
- struct symbol *pfx;
-
- pfx = find_symbol_by_offset(func->sec, func->offset - opts.prefix);
- if (pfx && is_prefix_func(pfx) && !strcmp(pfx->name, name))
- return 0;
- }
-
- insn = find_insn(file, func->sec, func->offset);
- if (!insn) {
- WARN("%s: can't find starting instruction", func->name);
+ if (!elf_create_symbol(file->elf, name, func->sec,
+ GELF_ST_BIND(func->sym.st_info),
+ GELF_ST_TYPE(func->sym.st_info),
+ func->offset - opts.prefix, opts.prefix))
return -1;
- }
-
- for (prev = prev_insn_same_sec(file, insn);
- prev;
- prev = prev_insn_same_sec(file, prev)) {
- u64 offset;
-
- if (prev->type != INSN_NOP)
- return 0;
-
- offset = func->offset - prev->offset;
-
- if (offset > opts.prefix)
- return 0;
-
- if (offset < opts.prefix)
- continue;
-
- if (!elf_create_symbol(file->elf, name, func->sec,
- GELF_ST_BIND(func->sym.st_info),
- GELF_ST_TYPE(func->sym.st_info),
- prev->offset, opts.prefix))
- return -1;
-
- break;
- }
-
- if (!prev)
- return 0;
-
- if (!insn->cfi) {
- /*
- * This can happen if stack validation isn't enabled or the
- * function is annotated with STACK_FRAME_NON_STANDARD.
- */
- return 0;
- }
/* Propagate insn->cfi to the prefix code */
+ insn = find_insn(file, func->sec, func->offset);
+ if (!insn || !insn->cfi)
+ return 0;
+
cfi = cfi_hash_find_or_add(insn->cfi);
- for (; prev != insn; prev = next_insn_same_sec(file, prev))
+ for (prev = find_insn(file, func->sec, func->offset - opts.prefix);
+ prev && prev != insn;
+ prev = next_insn_same_sec(file, prev))
prev->cfi = cfi;
return 0;
@@ -4387,15 +4334,20 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
static int create_prefix_symbols(struct objtool_file *file)
{
- struct section *sec;
+ struct section *pfe_sec;
struct symbol *func;
+ struct reloc *reloc;
- for_each_sec(file->elf, sec) {
- if (!is_text_sec(sec))
+ for_each_sec(file->elf, pfe_sec) {
+ if (strcmp(pfe_sec->name, "__patchable_function_entries"))
+ continue;
+ if (!pfe_sec->rsec)
continue;
- sec_for_each_sym(sec, func) {
- if (create_prefix_symbol(file, func))
+ for_each_reloc(pfe_sec->rsec, reloc) {
+ func = find_func_by_offset(reloc->sym->sec,
+ reloc->sym->offset + reloc_addend(reloc) + opts.prefix);
+ if (func && create_prefix_symbol(file, func))
return -1;
}
}
--
2.53.0
^ permalink raw reply related
* [PATCH v2 51/53] objtool/klp: Fix kCFI prefix finding/cloning
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
With CFI+CALL_PADDING, Clang places .Ltmp labels at the start of the NOP
padding (offset 5) between the __cfi_ prefix and the function entry
point. get_func_prefix() only checks the immediately previous symbol,
so the intervening .Ltmp label causes it to miss the __cfi_ prefix
symbol.
This results in klp-diff not cloning the kCFI type hash into the
livepatch module, causing a CFI failure at module load when calling
callback functions through indirect calls:
CFI failure at __klp_enable_patch+0xab/0x140
(target: pre_patch_callback+0x0/0x80 [livepatch_combined];
expected type: 0xde073954)
Instead of walking backward through the section's symbol list, just use
find_func_containing() for the byte before the function. This works now
that __cfi_ symbols are being grown by objtool to fill the padding.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/elf.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index 305183f30a33..fccf72cbd343 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -539,10 +539,10 @@ static inline struct symbol *get_func_prefix(struct symbol *func)
{
struct symbol *prev;
- if (!is_func_sym(func))
+ if (!is_func_sym(func) || !func->offset)
return NULL;
- prev = sec_prev_sym(func);
+ prev = find_func_containing(func->sec, func->offset - 1);
if (prev && is_prefix_func(prev))
return prev;
--
2.53.0
^ permalink raw reply related
* [PATCH v2 50/53] objtool: Grow __cfi_* prefix symbols for all CFI+CALL_PADDING
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
For all CONFIG_CFI+CONFIG_CALL_PADDING configs, for C functions, the
__cfi_ symbols only cover the 5-byte kCFI type hash. After that there
also N bytes of NOP padding between the hash and the function entry
which aren't associated with any symbol.
The NOPs can be replaced with actual code at runtime. Without a symbol,
unwinders and tooling have no way of knowing where those bytes belong.
Grow the existing __cfi_* symbols to fill that gap.
Note that assembly functions with SYM_TYPED_FUNC_START() aren't affected
by this issue, their __cfi_ symbols also cover the padding.
Also, CONFIG_PREFIX_SYMBOLS has no reason to exist: CONFIG_CALL_PADDING
is what causes the compiler to emit NOP padding before function entry
(via -fpatchable-function-entry), so it's the right condition for
creating prefix symbols.
Remove CONFIG_PREFIX_SYMBOLS, as it's no longer needed. Simplify the
LONGEST_SYM_KUNIT_TEST dependency accordingly. Rework objtool's
arguments a bit to handle the variety of prefix/cfi-related cases.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
arch/x86/Kconfig | 4 --
lib/Kconfig.debug | 2 +-
scripts/Makefile.lib | 7 +++-
tools/objtool/builtin-check.c | 15 +++++++-
tools/objtool/check.c | 49 ++++++++++++++++++++-----
tools/objtool/elf.c | 20 ++++++++++
tools/objtool/include/objtool/builtin.h | 7 ++--
tools/objtool/include/objtool/elf.h | 1 +
8 files changed, 84 insertions(+), 21 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f3f7cb01d69d..3eb3c48d764a 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2437,10 +2437,6 @@ config CALL_THUNKS
def_bool n
select CALL_PADDING
-config PREFIX_SYMBOLS
- def_bool y
- depends on CALL_PADDING && !CFI
-
menuconfig CPU_MITIGATIONS
bool "Mitigations for CPU vulnerabilities"
default y
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 8ff5adcfe1e0..4f7496b3268d 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -3070,7 +3070,7 @@ config FORTIFY_KUNIT_TEST
config LONGEST_SYM_KUNIT_TEST
tristate "Test the longest symbol possible" if !KUNIT_ALL_TESTS
depends on KUNIT && KPROBES
- depends on !PREFIX_SYMBOLS && !CFI && !GCOV_KERNEL
+ depends on !CALL_PADDING && !CFI && !GCOV_KERNEL
default KUNIT_ALL_TESTS
help
Tests the longest symbol possible
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 0718e39cedda..7e216d82e988 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -187,7 +187,11 @@ objtool-args-$(CONFIG_HAVE_JUMP_LABEL_HACK) += --hacks=jump_label
objtool-args-$(CONFIG_HAVE_NOINSTR_HACK) += --hacks=noinstr
objtool-args-$(CONFIG_MITIGATION_CALL_DEPTH_TRACKING) += --hacks=skylake
objtool-args-$(CONFIG_X86_KERNEL_IBT) += --ibt
-objtool-args-$(CONFIG_FINEIBT) += --cfi
+objtool-args-$(CONFIG_CALL_PADDING) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES)
+ifdef CONFIG_CALL_PADDING
+objtool-args-$(CONFIG_CFI) += --cfi
+objtool-args-$(CONFIG_FINEIBT) += --fineibt
+endif
objtool-args-$(CONFIG_FTRACE_MCOUNT_USE_OBJTOOL) += --mcount
ifdef CONFIG_FTRACE_MCOUNT_USE_OBJTOOL
objtool-args-$(CONFIG_HAVE_OBJTOOL_NOP_MCOUNT) += --mnop
@@ -200,7 +204,6 @@ objtool-args-$(CONFIG_STACK_VALIDATION) += --stackval
objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) += --static-call
objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) += --uaccess
objtool-args-$(or $(CONFIG_GCOV_KERNEL),$(CONFIG_KCOV)) += --no-unreachable
-objtool-args-$(CONFIG_PREFIX_SYMBOLS) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES)
objtool-args-$(CONFIG_OBJTOOL_WERROR) += --werror
objtool-args = $(objtool-args-y) \
diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c
index ec7f10a5ef19..118c3de2f293 100644
--- a/tools/objtool/builtin-check.c
+++ b/tools/objtool/builtin-check.c
@@ -73,7 +73,6 @@ static int parse_hacks(const struct option *opt, const char *str, int unset)
static const struct option check_options[] = {
OPT_GROUP("Actions:"),
- OPT_BOOLEAN(0, "cfi", &opts.cfi, "annotate kernel control flow integrity (kCFI) function preambles"),
OPT_STRING_OPTARG('d', "disas", &opts.disas, "function-pattern", "disassemble functions", "*"),
OPT_CALLBACK_OPTARG('h', "hacks", NULL, NULL, "jump_label,noinstr,skylake", "patch toolchain bugs/limitations", parse_hacks),
OPT_BOOLEAN('i', "ibt", &opts.ibt, "validate and annotate IBT"),
@@ -84,7 +83,7 @@ static const struct option check_options[] = {
OPT_BOOLEAN('r', "retpoline", &opts.retpoline, "validate and annotate retpoline usage"),
OPT_BOOLEAN(0, "rethunk", &opts.rethunk, "validate and annotate rethunk usage"),
OPT_BOOLEAN(0, "unret", &opts.unret, "validate entry unret placement"),
- OPT_INTEGER(0, "prefix", &opts.prefix, "generate prefix symbols"),
+ OPT_INTEGER(0, "prefix", &opts.prefix, "generate or grow prefix symbols for N-byte function padding"),
OPT_BOOLEAN('l', "sls", &opts.sls, "validate straight-line-speculation mitigations"),
OPT_BOOLEAN('s', "stackval", &opts.stackval, "validate frame pointer rules"),
OPT_BOOLEAN('t', "static-call", &opts.static_call, "annotate static calls"),
@@ -92,6 +91,8 @@ static const struct option check_options[] = {
OPT_CALLBACK_OPTARG(0, "dump", NULL, NULL, "orc", "dump metadata", parse_dump),
OPT_GROUP("Options:"),
+ OPT_BOOLEAN(0, "cfi", &opts.cfi, "grow kCFI preamble symbols (use with --prefix)"),
+ OPT_BOOLEAN(0, "fineibt", &opts.fineibt, "create .cfi_sites section for FineIBT"),
OPT_BOOLEAN(0, "backtrace", &opts.backtrace, "unwind on error"),
OPT_BOOLEAN(0, "backup", &opts.backup, "create backup (.orig) file on warning/error"),
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
@@ -163,6 +164,16 @@ static bool opts_valid(void)
return false;
}
+ if (opts.cfi && !opts.prefix) {
+ ERROR("--cfi requires --prefix");
+ return false;
+ }
+
+ if (opts.fineibt && !opts.cfi) {
+ ERROR("--fineibt requires --cfi");
+ return false;
+ }
+
if (opts.disas ||
opts.hack_jump_label ||
opts.hack_noinstr ||
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 410061aeed26..0d9b859b006e 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -881,6 +881,31 @@ static int create_ibt_endbr_seal_sections(struct objtool_file *file)
return 0;
}
+/*
+* Grow __cfi_ symbols to fill the NOP gap between the 'mov <hash>, %rax' and
+* the start of the function.
+*/
+static int grow_cfi_symbols(struct objtool_file *file)
+{
+ struct symbol *sym;
+
+ for_each_sym(file->elf, sym) {
+ if (!is_func_sym(sym) || !strstarts(sym->name, "__cfi_") ||
+ sym->len != 5)
+ continue;
+
+ if (!find_func_by_offset(sym->sec, sym->offset + sym->len + opts.prefix))
+ continue;
+
+ sym->len += opts.prefix;
+ sym->sym.st_size = sym->len;
+ if (elf_write_symbol(file->elf, sym))
+ return -1;
+ }
+
+ return 0;
+}
+
static int create_cfi_sections(struct objtool_file *file)
{
struct section *sec;
@@ -4903,12 +4928,6 @@ int check(struct objtool_file *file)
goto out;
}
- if (opts.cfi) {
- ret = create_cfi_sections(file);
- if (ret)
- goto out;
- }
-
if (opts.rethunk) {
ret = create_return_sites_sections(file);
if (ret)
@@ -4928,9 +4947,21 @@ int check(struct objtool_file *file)
}
if (opts.prefix) {
- ret = create_prefix_symbols(file);
- if (ret)
- goto out;
+ if (!opts.cfi) {
+ ret = create_prefix_symbols(file);
+ if (ret)
+ goto out;
+ } else {
+ ret = grow_cfi_symbols(file);
+ if (ret)
+ goto out;
+
+ if (opts.fineibt) {
+ ret = create_cfi_sections(file);
+ if (ret)
+ goto out;
+ }
+ }
}
if (opts.ibt) {
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index d9cee8d5d9e8..33c95a74a51b 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -997,6 +997,26 @@ struct symbol *elf_create_symbol(struct elf *elf, const char *name,
return sym;
}
+int elf_write_symbol(struct elf *elf, struct symbol *sym)
+{
+ struct section *symtab, *symtab_shndx;
+
+ symtab = find_section_by_name(elf, ".symtab");
+ if (!symtab) {
+ ERROR("no .symtab");
+ return -1;
+ }
+
+ symtab_shndx = find_section_by_name(elf, ".symtab_shndx");
+
+ if (elf_update_symbol(elf, symtab, symtab_shndx, sym))
+ return -1;
+
+ mark_sec_changed(elf, symtab, true);
+
+ return 0;
+}
+
struct symbol *elf_create_section_symbol(struct elf *elf, struct section *sec)
{
struct symbol *sym = calloc(1, sizeof(*sym));
diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/include/objtool/builtin.h
index b9e229ed4dc0..e844e9c82b7b 100644
--- a/tools/objtool/include/objtool/builtin.h
+++ b/tools/objtool/include/objtool/builtin.h
@@ -9,8 +9,8 @@
struct opts {
/* actions: */
- bool cfi;
bool checksum;
+ const char *disas;
bool dump_orc;
bool hack_jump_label;
bool hack_noinstr;
@@ -20,6 +20,7 @@ struct opts {
bool noabs;
bool noinstr;
bool orc;
+ int prefix;
bool retpoline;
bool rethunk;
bool unret;
@@ -27,14 +28,14 @@ struct opts {
bool stackval;
bool static_call;
bool uaccess;
- int prefix;
- const char *disas;
/* options: */
bool backtrace;
bool backup;
+ bool cfi;
const char *debug_checksum;
bool dryrun;
+ bool fineibt;
bool link;
bool mnop;
bool module;
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index e452784df702..305183f30a33 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -199,6 +199,7 @@ struct reloc *elf_init_reloc_data_sym(struct elf *elf, struct section *sec,
struct symbol *sym,
s64 addend);
+int elf_write_symbol(struct elf *elf, struct symbol *sym);
int elf_write_insn(struct elf *elf, struct section *sec, unsigned long offset,
unsigned int len, const char *insn);
--
2.53.0
^ permalink raw reply related
* [PATCH v2 49/53] objtool/klp: Fix position-dependent checksums for non-relocated jumps/calls
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
When computing klp checksums, instructions with non-relocated jump/call
destination offsets are problematic because the offset values can change
when surrounding code has moved, causing the function to be incorrectly
marked as changed.
Specifically, that includes jumps from alternatives to the end of the
alternative, which from objtool's perspective are jumps to the end of
the alternative instruction block in the original function.
Note that 'jump_dest' jumps don't include sibling calls (those use
call_dest), nor do they include jumps to/from .cold sub functions (those
are cross-section and need a reloc).
Fix it by hashing the opcode bytes (excluding the immediate operand)
along with a position-independent representation of the destination.
For calls, use the function name, and for jumps, use the destination's
offset within its function.
[Note the "9 bit hole" comment was wrong: it has been 8 bits since
commit 70589843b36f ("objtool: Add option to trace function validation")
added the 'trace' field. Adding the 4-bit 'immediate_len' field now
leaves a 4-bit hole.]
Fixes: 0d83da43b1e1 ("objtool/klp: Add --checksum option to generate per-function checksums")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/arch/x86/decode.c | 17 ++++++++-
tools/objtool/include/objtool/arch.h | 3 ++
tools/objtool/include/objtool/check.h | 3 +-
tools/objtool/klp-checksum.c | 53 ++++++++++++++++++++++++---
4 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index 350b8ee6e776..1b387d5a195b 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -805,14 +805,27 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
break;
}
- if (ins.immediate.nbytes)
+ if (ins.immediate.nbytes) {
insn->immediate = ins.immediate.value;
- else if (ins.displacement.nbytes)
+ insn->immediate_len = ins.immediate.nbytes;
+ } else if (ins.displacement.nbytes) {
insn->immediate = ins.displacement.value;
+ insn->immediate_len = ins.displacement.nbytes;
+ }
return 0;
}
+size_t arch_jump_opcode_bytes(struct objtool_file *file, struct instruction *insn,
+ unsigned char *buf)
+{
+ size_t len;
+
+ len = insn->len - insn->immediate_len;
+ memcpy(buf, insn->sec->data->d_buf + insn->offset, len);
+ return len;
+}
+
void arch_initial_func_cfi_state(struct cfi_init_state *state)
{
int i;
diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/objtool/arch.h
index 8866158975fc..96d828a8401f 100644
--- a/tools/objtool/include/objtool/arch.h
+++ b/tools/objtool/include/objtool/arch.h
@@ -79,6 +79,9 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
unsigned long offset, unsigned int maxlen,
struct instruction *insn);
+size_t arch_jump_opcode_bytes(struct objtool_file *file, struct instruction *insn,
+ unsigned char *buf);
+
bool arch_callee_saved_reg(unsigned char reg);
unsigned long arch_jump_destination(struct instruction *insn);
diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h
index fe08205d8eb1..063f5985fecd 100644
--- a/tools/objtool/include/objtool/check.h
+++ b/tools/objtool/include/objtool/check.h
@@ -68,6 +68,7 @@ struct instruction {
s8 instr;
u32 idx : INSN_CHUNK_BITS,
+ immediate_len : 4,
dead_end : 1,
ignore_alts : 1,
hint : 1,
@@ -81,7 +82,7 @@ struct instruction {
hole : 1,
fake : 1,
trace : 1;
- /* 9 bit hole */
+ /* 4 bit hole */
struct alt_group *alt_group;
struct instruction *jump_dest;
diff --git a/tools/objtool/klp-checksum.c b/tools/objtool/klp-checksum.c
index 19653dbe109d..b8e47f28997e 100644
--- a/tools/objtool/klp-checksum.c
+++ b/tools/objtool/klp-checksum.c
@@ -66,17 +66,58 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
if (insn->fake)
return;
- __checksum_update_insn(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
-
if (!reloc) {
struct symbol *call_dest = insn_call_dest(insn);
+ struct instruction *jump_dest = insn->jump_dest;
- if (call_dest)
- __checksum_update_insn(func, insn, call_dest->demangled_name,
- strlen(call_dest->demangled_name));
- goto alts;
+ /*
+ * For a jump/call non-relocated dest offset embedded in the
+ * instruction, the offset may vary due to changes in
+ * surrounding code. Just hash the opcode and a
+ * position-independent representation of the destination.
+ */
+
+ if (call_dest || jump_dest) {
+ unsigned char buf[16];
+ size_t len;
+
+ len = arch_jump_opcode_bytes(file, insn, buf);
+ __checksum_update_insn(func, insn, buf, len);
+
+ if (call_dest) {
+ __checksum_update_insn(func, insn, call_dest->demangled_name,
+ strlen(call_dest->demangled_name));
+
+ } else if (jump_dest) {
+ struct symbol *dest_sym;
+ unsigned long offset;
+
+ /*
+ * use insn->_sym instead of insn_sym() here.
+ * For alternative replacements, the latter
+ * would give the function of the code being
+ * replaced.
+ */
+ dest_sym = jump_dest->_sym;
+ if (!dest_sym)
+ goto alts;
+
+ __checksum_update_insn(func, insn, dest_sym->demangled_name,
+ strlen(dest_sym->demangled_name));
+
+ offset = jump_dest->offset - dest_sym->offset;
+ __checksum_update_insn(func, insn, &offset, sizeof(offset));
+ }
+
+ goto alts;
+ }
}
+ __checksum_update_insn(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
+
+ if (!reloc)
+ goto alts;
+
sym = reloc->sym;
offset = arch_insn_adjusted_addend(insn, reloc);
--
2.53.0
^ permalink raw reply related
* [PATCH v2 48/53] objtool: Add insn_sym() helper
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Alternative replacement instructions awkwardly have insn->sym set to the
function they get patched to rather than the symbol (or rather lack
thereof) they belong to in the file.
This makes it difficult to know where a given instruction actually
lives.
Add a new insn_sym() helper which preserves the existing semantic of
insn->sym. Rename insn->sym to insn->_sym, which contains the actual
ELF binary symbol (or NULL, for alternative replacements) an instruction
lives in.
The private insn->_sym value will be needed for a subsequent patch.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 31 ++++++++++++---------------
tools/objtool/disas.c | 22 +++++++++----------
tools/objtool/include/objtool/check.h | 20 +++++++++++++++--
tools/objtool/include/objtool/warn.h | 6 +++---
tools/objtool/trace.c | 8 +++----
5 files changed, 49 insertions(+), 38 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index ae047be919c5..410061aeed26 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -495,7 +495,7 @@ static int decode_instructions(struct objtool_file *file)
}
sym_for_each_insn(file, func, insn) {
- insn->sym = func;
+ insn->_sym = func;
if (is_func_sym(func) &&
insn->type == INSN_ENDBR &&
list_empty(&insn->call_node)) {
@@ -859,15 +859,14 @@ static int create_ibt_endbr_seal_sections(struct objtool_file *file)
list_for_each_entry(insn, &file->endbr_list, call_node) {
int *site = (int *)sec->data->d_buf + idx;
- struct symbol *sym = insn->sym;
+ struct symbol *func = insn_func(insn);
*site = 0;
- if (opts.module && sym && is_func_sym(sym) &&
- insn->offset == sym->offset &&
- (!strcmp(sym->name, "init_module") ||
- !strcmp(sym->name, "cleanup_module"))) {
+ if (opts.module && func && insn->offset == func->offset &&
+ (!strcmp(func->name, "init_module") ||
+ !strcmp(func->name, "cleanup_module"))) {
ERROR("%s(): Magic init_module() function name is deprecated, use module_init(fn) instead",
- sym->name);
+ func->name);
return -1;
}
@@ -1581,7 +1580,7 @@ static int add_jump_destinations(struct objtool_file *file)
}
if (!dest_sym || is_sec_sym(dest_sym)) {
- dest_sym = dest_insn->sym;
+ dest_sym = insn_sym(dest_insn);
if (!dest_sym)
goto set_jump_dest;
}
@@ -1597,7 +1596,7 @@ static int add_jump_destinations(struct objtool_file *file)
continue;
}
- if (!insn->sym || insn->sym->pfunc == dest_sym->pfunc)
+ if (!insn_sym(insn) || insn_sym(insn)->pfunc == dest_sym->pfunc)
goto set_jump_dest;
/*
@@ -1770,7 +1769,6 @@ static int handle_group_alt(struct objtool_file *file,
nop->offset = special_alt->new_off + special_alt->new_len;
nop->len = special_alt->orig_len - special_alt->new_len;
nop->type = INSN_NOP;
- nop->sym = orig_insn->sym;
nop->alt_group = new_alt_group;
nop->fake = 1;
}
@@ -1789,7 +1787,6 @@ static int handle_group_alt(struct objtool_file *file,
last_new_insn = insn;
- insn->sym = orig_insn->sym;
insn->alt_group = new_alt_group;
/*
@@ -2432,12 +2429,12 @@ static int __annotate_late(struct objtool_file *file, int type, struct instructi
break;
case ANNOTYPE_NOCFI:
- sym = insn->sym;
+ sym = insn_sym(insn);
if (!sym) {
ERROR_INSN(insn, "dodgy NOCFI annotation");
return -1;
}
- insn->sym->nocfi = 1;
+ insn_sym(insn)->nocfi = 1;
break;
default:
@@ -2538,7 +2535,7 @@ static void mark_holes(struct objtool_file *file)
* favour of a regular symbol, but leaves the code in place.
*/
for_each_insn(file, insn) {
- if (insn->sym || !find_symbol_hole_containing(insn->sec, insn->offset)) {
+ if (insn_sym(insn) || !find_symbol_hole_containing(insn->sec, insn->offset)) {
in_hole = false;
continue;
}
@@ -2982,7 +2979,7 @@ static int update_cfi_state(struct instruction *insn,
}
if (op->dest.reg == CFI_BP && op->src.reg == CFI_SP &&
- insn->sym->frame_pointer) {
+ insn_sym(insn)->frame_pointer) {
/* addi.d fp,sp,imm on LoongArch */
if (cfa->base == CFI_SP && cfa->offset == op->src.offset) {
cfa->base = CFI_BP;
@@ -2994,7 +2991,7 @@ static int update_cfi_state(struct instruction *insn,
if (op->dest.reg == CFI_SP && op->src.reg == CFI_BP) {
/* addi.d sp,fp,imm on LoongArch */
if (cfa->base == CFI_BP && cfa->offset == 0) {
- if (insn->sym->frame_pointer) {
+ if (insn_sym(insn)->frame_pointer) {
cfa->base = CFI_SP;
cfa->offset = -op->src.offset;
}
@@ -4171,7 +4168,7 @@ static int validate_retpoline(struct objtool_file *file)
* broken.
*/
list_for_each_entry(insn, &file->retpoline_call_list, call_node) {
- struct symbol *sym = insn->sym;
+ struct symbol *sym = insn_sym(insn);
if (sym && (is_notype_sym(sym) ||
is_func_sym(sym)) && !sym->nocfi) {
diff --git a/tools/objtool/disas.c b/tools/objtool/disas.c
index 59090234af19..e6a54a83605c 100644
--- a/tools/objtool/disas.c
+++ b/tools/objtool/disas.c
@@ -210,7 +210,7 @@ static bool disas_print_addr_alt(bfd_vma addr, struct disassemble_info *dinfo)
offset = addr - alt_group->first_insn->offset;
addr = orig_first_insn->offset + offset;
- sym = orig_first_insn->sym;
+ sym = insn_sym(orig_first_insn);
disas_print_addr_sym(orig_first_insn->sec, sym, addr, dinfo);
@@ -222,15 +222,13 @@ static void disas_print_addr_noreloc(bfd_vma addr,
{
struct disas_context *dctx = dinfo->application_data;
struct instruction *insn = dctx->insn;
- struct symbol *sym = NULL;
+ struct symbol *sym = insn_sym(insn);
if (disas_print_addr_alt(addr, dinfo))
return;
- if (insn->sym && addr >= insn->sym->offset &&
- addr < insn->sym->offset + insn->sym->len) {
- sym = insn->sym;
- }
+ if (sym && (addr < sym->offset || addr >= sym->offset + sym->len))
+ sym = NULL;
disas_print_addr_sym(insn->sec, sym, addr, dinfo);
}
@@ -291,9 +289,9 @@ static void disas_print_address(bfd_vma addr, struct disassemble_info *dinfo)
* up. So check it first.
*/
jump_dest = insn->jump_dest;
- if (jump_dest && jump_dest->sym && jump_dest->offset == addr) {
+ if (jump_dest && insn_sym(jump_dest) && jump_dest->offset == addr) {
if (!disas_print_addr_alt(addr, dinfo))
- disas_print_addr_sym(jump_dest->sec, jump_dest->sym,
+ disas_print_addr_sym(jump_dest->sec, insn_sym(jump_dest),
addr, dinfo);
return;
}
@@ -768,8 +766,8 @@ static int disas_alt_jump(struct disas_alt *dalt)
if (orig_insn->len == 5)
suffix[0] = 'q';
str = strfmt("jmp%-3s %lx <%s+0x%lx>", suffix,
- dest_insn->offset, dest_insn->sym->name,
- dest_insn->offset - dest_insn->sym->offset);
+ dest_insn->offset, insn_sym(dest_insn)->name,
+ dest_insn->offset - insn_sym(dest_insn)->offset);
nops = 0;
} else {
str = strfmt("nop%d", orig_insn->len);
@@ -794,8 +792,8 @@ static int disas_alt_extable(struct disas_alt *dalt)
alt_insn = dalt->alt->insn;
str = strfmt("resume at 0x%lx <%s+0x%lx>",
- alt_insn->offset, alt_insn->sym->name,
- alt_insn->offset - alt_insn->sym->offset);
+ alt_insn->offset, insn_sym(alt_insn)->name,
+ alt_insn->offset - insn_sym(alt_insn)->offset);
if (!str)
return -1;
diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/objtool/check.h
index eea64728d39b..fe08205d8eb1 100644
--- a/tools/objtool/include/objtool/check.h
+++ b/tools/objtool/include/objtool/check.h
@@ -94,14 +94,30 @@ struct instruction {
};
};
struct alternative *alts;
- struct symbol *sym;
+ struct symbol *_sym;
struct stack_op *stack_ops;
struct cfi_state *cfi;
};
+/*
+ * Return the symbol associated with an instruction. For alternative
+ * replacements, return the symbol of the original code being replaced rather
+ * than NULL. insn->_sym reflects the actual location in the ELF file.
+ */
+static inline struct symbol *insn_sym(struct instruction *insn)
+{
+ struct symbol *sym = insn->_sym;
+
+ if ((!sym || !is_func_sym(sym)) &&
+ insn->alt_group && insn->alt_group->orig_group)
+ sym = insn->alt_group->orig_group->first_insn->_sym;
+
+ return sym;
+}
+
static inline struct symbol *insn_func(struct instruction *insn)
{
- struct symbol *sym = insn->sym;
+ struct symbol *sym = insn_sym(insn);
if (sym && sym->type != STT_FUNC)
sym = NULL;
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index a9936d60980c..870e147f3a56 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -77,13 +77,13 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define WARN_INSN(insn, format, ...) \
({ \
struct instruction *_insn = (insn); \
- if (!_insn->sym || !_insn->sym->warned) { \
+ if (!insn_sym(_insn) || !insn_sym(_insn)->warned) { \
WARN_FUNC(_insn->sec, _insn->offset, format, \
##__VA_ARGS__); \
BT_INSN(_insn, ""); \
} \
- if (_insn->sym) \
- _insn->sym->warned = 1; \
+ if (insn_sym(_insn)) \
+ insn_sym(_insn)->warned = 1; \
})
#define BT_INSN(insn, format, ...) \
diff --git a/tools/objtool/trace.c b/tools/objtool/trace.c
index 5dec44dab781..61c6aa302bc3 100644
--- a/tools/objtool/trace.c
+++ b/tools/objtool/trace.c
@@ -169,8 +169,8 @@ void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
*/
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s for instruction at 0x%lx <%s+0x%lx>",
alt_name,
- orig_insn->offset, orig_insn->sym->name,
- orig_insn->offset - orig_insn->sym->offset);
+ orig_insn->offset, insn_sym(orig_insn)->name,
+ orig_insn->offset - insn_sym(orig_insn)->offset);
} else {
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s", alt_name);
}
@@ -185,8 +185,8 @@ void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
if (orig_insn->type == INSN_NOP) {
suffix[0] = (orig_insn->len == 5) ? 'q' : '\0';
TRACE_ADDR(orig_insn, "jmp%-3s %lx <%s+0x%lx>", suffix,
- alt_insn->offset, alt_insn->sym->name,
- alt_insn->offset - alt_insn->sym->offset);
+ alt_insn->offset, insn_sym(alt_insn)->name,
+ alt_insn->offset - insn_sym(alt_insn)->offset);
} else {
TRACE_ADDR(orig_insn, "nop%d", orig_insn->len);
trace_depth--;
--
2.53.0
^ permalink raw reply related
* [PATCH v2 46/53] objtool/klp: Rewrite symbol correlation algorithm
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Rewrite the symbol correlation code, using a tiered list of
deterministic strategies in a loop. For duplicately named symbols, each
tier applies a filter with the goal of finding a 1:1 deterministic
correlation between the original and patched version of the symbol.
The three matching strategies are:
find_twin(): A funnel of progressively tighter filters. Candidates
with the same demangled name are counted at four levels: name, scope
(local-vs-global), file (strict file association), and checksum
(unchanged functions). The widest level that yields a 1:1 match wins,
narrower levels are only tried when the wider level is ambiguous.
find_twin_suffixed(): Uses already-correlated LLVM symbol pairs to map
.llvm.<hash> suffixes from orig to patched. Because all promoted
symbols from the same TU share the same hash, one correlated pair
seeds the mapping for the entire TU.
find_twin_positional(): Last resort, matches symbols by position among
same-named candidates, similar to livepatch sympos. Used for data
objects like __quirk variables where no deterministic filter can
distinguish the candidates.
Overall this works much better than the existing algorithm, particularly
with LTO kernels.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/klp-diff.c | 521 +++++++++++++++++++++++++++++----------
1 file changed, 385 insertions(+), 136 deletions(-)
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 5f13d759e02f..a9c993298b82 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -408,78 +408,358 @@ static bool dont_correlate(struct symbol *sym)
is_special_section_aux(sym->sec);
}
-/*
- * When there is no full name match, try match demangled_name. This would
- * match original foo.llvm.123 to patched foo.llvm.456.
- *
- * Note that, in very rare cases, it is possible to have multiple
- * foo.llvm.<hash> in the same kernel. When this happens, report error and
- * fail the diff.
- */
-static int find_global_symbol_by_demangled_name(struct elf *elf, struct symbol *sym,
- struct symbol **out_sym)
+static const char *llvm_suffix(const char *name)
{
- struct symbol *sym2, *result = NULL;
- int count = 0;
+ return strstr(name, ".llvm.");
+}
- for_each_sym_by_demangled_name(elf, sym->demangled_name, sym2) {
- if (is_local_sym(sym2) || sym2->twin)
+static bool is_llvm_sym(struct symbol *sym)
+{
+ return llvm_suffix(sym->name);
+}
+
+/*
+ * Determine if two symbols have compatible source file origins:
+ *
+ * - If both symbols are local, only return true if they belong to the same
+ * ELF file symbol.
+ *
+ * - If both symbols are global, always return true, as globals don't have
+ * file associations.
+ *
+ * - If they have different scopes, also return true, as the patch might have
+ * changed the symbol's scope.
+ *
+ * Works for both same-ELF (direct pointer compare) and cross-ELF
+ * (compare via file->twin) cases.
+ */
+static bool maybe_same_file(struct symbol *sym1, struct symbol *sym2)
+{
+ if (!sym1->file || !sym2->file)
+ return true;
+ if (sym1->file == sym2->file)
+ return true;
+ return sym1->file->twin == sym2->file;
+}
+
+/*
+ * Similar to maybe_same_file(), but strict: no scope changes allowed.
+ *
+ * Works for both same-ELF (direct pointer compare) and cross-ELF
+ * (compare via file->twin) cases.
+ */
+static bool same_file(struct symbol *sym1, struct symbol *sym2)
+{
+ if (llvm_suffix(sym1->name) && llvm_suffix(sym2->name))
+ return true;
+ if (!sym1->file && !sym2->file)
+ return true;
+ if (!sym1->file || !sym2->file)
+ return false;
+ if (sym1->file == sym2->file)
+ return true;
+ return sym1->file->twin == sym2->file;
+}
+
+/*
+ * Is it a local symbol, or at least was it local in the translation unit
+ * before LLVM promoted it?
+ */
+static bool is_tu_local_sym(struct symbol *sym)
+{
+ return is_local_sym(sym) || is_llvm_sym(sym);
+}
+
+/*
+ * Try to find sym1's twin in patched using deterministic matching.
+ *
+ * Multiple symbols can share a demangled name (e.g., static functions in
+ * different TUs). This function counts same-named candidates through a
+ * funnel of progressively tighter filters. Each level is a strict subset
+ * of the previous one.
+ *
+ * The widest level that yields a 1:1 match wins. Narrower levels are only
+ * needed when the wider level is ambiguous (count > 1).
+ *
+ * Candidates are pre-filtered by maybe_same_file(), which narrows most
+ * local symbols to their own TU. For example, 19 different static
+ * type_show() functions across vmlinux.o each see only one candidate after
+ * pre-filtering, so they match immediately at Level 1.
+ *
+ * Level 1 (name): Works when the demangled name is unique after
+ * pre-filtering. Handles most symbols: unique globals like copy_signal(),
+ * or per-TU locals like pcspkr_probe().
+ *
+ * Level 2 (scope): Filters by local-vs-global (TU-local-vs-not). Example:
+ * parse_header() exists as both a static and a global function. Level 1
+ * sees both (same demangled name), but Level 2 separates them by scope.
+ *
+ * Level 3 (file): Strict file matching via same_file(), which rejects scope
+ * changes. Example: LLVM-promoted foo.llvm.12345 (global, no FILE symbol)
+ * vs genuine local foo (has FILE symbol). Both are TU-local so Level 2
+ * can't distinguish them, but same_file() rejects the pair because one has
+ * a file association and the other doesn't.
+ *
+ * Level 4 (checksum): Distinguishes by function checksum. Example:
+ * usb_devnode.llvm.AAA and usb_devnode.llvm.BBB are two LLVM-promoted
+ * functions from different TUs with the same demangled name. After a TU
+ * change, the .llvm. hashes change but the functions themselves may be
+ * unchanged. Level 4 matches each to the patched candidate with the
+ * same checksum.
+ */
+static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
+{
+ struct symbol *name_last = NULL, *scope_last = NULL,
+ *file_last = NULL, *csum_last = NULL;
+ unsigned int name_orig = 0, name_patched = 0;
+ unsigned int scope_orig = 0, scope_patched = 0;
+ unsigned int file_orig = 0, file_patched = 0;
+ unsigned int csum_orig = 0, csum_patched = 0;
+ struct symbol *sym2, *match = NULL;
+
+ /* Count orig candidates */
+ for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
+ if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ (!maybe_same_file(sym1, sym2)))
continue;
- count++;
- result = sym2;
+ /* Level 1: name match (widest filter) */
+ name_orig++;
+
+ /* Level 2: scope (scope changes allowed) */
+ if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2))
+ continue;
+ scope_orig++;
+
+ /* Level 3: file (scope changes disallowed) */
+ if (!same_file(sym1, sym2))
+ continue;
+ file_orig++;
+
+ /* Level 4: checksum (unchanged symbols) */
+ if (sym1->len != sym2->len || !sym1->csum.checksum ||
+ sym1->csum.checksum != sym2->csum.checksum)
+ continue;
+ csum_orig++;
}
- if (count > 1) {
- ERROR("Multiple (%d) correlation candidates for %s", count, sym->name);
- return -1;
+ /* Count patched candidates */
+ for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
+ if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ !maybe_same_file(sym1, sym2))
+ continue;
+
+ /* Level 1 */
+ name_patched++;
+ name_last = sym2;
+
+ /* Level 2 */
+ if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2))
+ continue;
+ scope_patched++;
+ scope_last = sym2;
+
+ /* Level 3 */
+ if (!same_file(sym1, sym2))
+ continue;
+ file_patched++;
+ file_last = sym2;
+
+ /* Level 4 */
+ if (sym1->len != sym2->len || !sym1->csum.checksum ||
+ sym1->csum.checksum != sym2->csum.checksum)
+ continue;
+ csum_patched++;
+ csum_last = sym2;
+ }
+
+ /* Return the widest level that yields a unique (1:1) match */
+ if (name_orig == 1 && name_patched == 1)
+ match = name_last;
+ else if (scope_orig == 1 && scope_patched == 1)
+ match = scope_last;
+ else if (file_orig == 1 && file_patched == 1)
+ match = file_last;
+ else if (csum_orig == 1 && csum_patched == 1)
+ match = csum_last;
+
+ return match;
+}
+
+struct llvm_suffix_pair {
+ struct hlist_node hash;
+ const char *orig;
+ const char *patched;
+};
+
+static DECLARE_HASHTABLE(suffix_map, 7);
+
+/*
+ * Build a mapping of known orig-to-patched LLVM suffixes based on
+ * already-correlated symbol pairs. All promoted symbols from the same TU
+ * share the same .llvm.<hash> suffix, so one correlated pair seeds the map
+ * for the entire TU.
+ */
+static int update_suffix_map(struct elf *elf)
+{
+ struct llvm_suffix_pair *entry;
+ struct symbol *sym;
+
+ for_each_sym(elf, sym) {
+ const char *s1, *s2;
+ bool found;
+
+ if (!sym->twin)
+ continue;
+
+ s1 = llvm_suffix(sym->name);
+ s2 = llvm_suffix(sym->twin->name);
+
+ if (!s1 || !s2)
+ continue;
+
+ found = false;
+ hash_for_each_possible(suffix_map, entry, hash, str_hash(s1)) {
+ if (!strcmp(entry->orig, s1)) {
+ found = true;
+ break;
+ }
+ }
+ if (found)
+ continue;
+
+ entry = calloc(1, sizeof(*entry));
+ if (!entry) {
+ ERROR_GLIBC("calloc");
+ return -1;
+ }
+
+ entry->orig = s1;
+ entry->patched = s2;
+ hash_add(suffix_map, &entry->hash, str_hash(s1));
}
- *out_sym = result;
return 0;
}
/*
- * For each symbol in the original kernel, find its corresponding "twin" in the
- * patched kernel.
+ * Match by translating the symbol's .llvm.<hash> suffix through the suffix
+ * map to find the corresponding hash suffix for the patched object.
+ *
+ * Example: In the original kernel, TU drivers/base/core.c contains
+ * foo.llvm.12345 and bar.llvm.12345 (same TU, same hash). After patching,
+ * they become foo.llvm.67890 and bar.llvm.67890. If foo was already
+ * correlated by find_twin() (e.g., unique by name), the suffix map records
+ * .llvm.12345 -> .llvm.67890. When processing bar.llvm.12345, this
+ * function looks up .llvm.12345, gets .llvm.67890, constructs the name
+ * bar.llvm.67890, and finds the match.
+ */
+static struct symbol *find_twin_suffixed(struct elf *elf, struct symbol *sym1)
+{
+ const char *suffix, *patched_suffix = NULL;
+ struct symbol *sym2, *match = NULL;
+ char name[SYM_NAME_LEN];
+ struct llvm_suffix_pair *entry;
+ int count = 0;
+
+ suffix = llvm_suffix(sym1->name);
+ if (!suffix)
+ return NULL;
+
+ hash_for_each_possible(suffix_map, entry, hash, str_hash(suffix)) {
+ if (!strcmp(entry->orig, suffix)) {
+ patched_suffix = entry->patched;
+ break;
+ }
+ }
+ if (!patched_suffix)
+ return NULL;
+
+ if (snprintf_check(name, SYM_NAME_LEN, "%s%s",
+ sym1->demangled_name, patched_suffix))
+ return NULL;
+
+ for_each_sym_by_name(elf, name, sym2) {
+ if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2))
+ continue;
+ count++;
+ match = sym2;
+ }
+
+ if (count == 1)
+ return match;
+
+ return NULL;
+}
+
+/*
+ * Last-resort positional matching.
+ *
+ * Finds a symbol with the same position in the symbol table among
+ * same-demangled-name candidates, similar to livepatch sympos. Note that
+ * LLVM-promoted symbols are globals, which come after locals in the symbol
+ * table, so we have to be careful not to compare different scopes.
+ *
+ * Example: arch/x86/events/intel/core.c defines many __quirk variables via
+ * X86_MATCH_*() macros. In the symbol table they appear as __quirk.90,
+ * __quirk.97, __quirk.101, etc., all with demangled name __quirk, same
+ * scope, and same FILE symbol. No deterministic filter can distinguish
+ * them, so they're matched by position: the 1st __quirk in orig matches the
+ * 1st in patched, the 2nd matches the 2nd, etc.
+ *
+ * This is less deterministic than the other strategies, so it's done last.
+ */
+static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
+{
+ unsigned int idx_orig = 0, idx_patched = 0;
+ unsigned int sym1_pos = 0;
+ struct symbol *sym2, *match = NULL;
+
+ for_each_sym_by_demangled_name(e->orig, sym1->demangled_name, sym2) {
+ if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ !maybe_same_file(sym1, sym2))
+ continue;
+ if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
+ is_llvm_sym(sym1) != is_llvm_sym(sym2))
+ continue;
+ if (sym1 == sym2)
+ sym1_pos = idx_orig;
+ idx_orig++;
+ }
+
+ for_each_sym_by_demangled_name(e->patched, sym1->demangled_name, sym2) {
+ if (sym2->twin || sym1->type != sym2->type || dont_correlate(sym2) ||
+ !maybe_same_file(sym1, sym2))
+ continue;
+ if (is_tu_local_sym(sym1) != is_tu_local_sym(sym2) ||
+ is_llvm_sym(sym1) != is_llvm_sym(sym2))
+ continue;
+ if (idx_patched == sym1_pos)
+ match = sym2;
+ idx_patched++;
+ }
+
+ if (idx_orig != idx_patched)
+ return NULL;
+
+ return match;
+}
+
+/*
+ * Correlate symbols between the orig and patched objects. This is a
+ * prerequisite for detecting changed functions, as well as for properly
+ * translating relocations so they point to the correct symbol.
*/
static int correlate_symbols(struct elfs *e)
{
struct symbol *file1_sym, *file2_sym;
struct symbol *sym1, *sym2;
+ bool progress;
+ /* Correlate FILE symbols */
file1_sym = first_file_symbol(e->orig);
file2_sym = first_file_symbol(e->patched);
- /*
- * Correlate any locals before the first FILE symbol. This has been
- * seen when LTO inexplicably strips the initramfs_data.o FILE symbol
- * due to the file only containing data and no code.
- */
- for_each_sym(e->orig, sym1) {
- if (sym1 == file1_sym || !is_local_sym(sym1))
- break;
-
- if (dont_correlate(sym1))
- continue;
-
- for_each_sym(e->patched, sym2) {
- if (sym2 == file2_sym || !is_local_sym(sym2))
- break;
-
- if (sym2->twin || dont_correlate(sym2))
- continue;
-
- if (strcmp(sym1->demangled_name, sym2->demangled_name))
- continue;
-
- sym1->twin = sym2;
- sym2->twin = sym1;
- break;
- }
- }
-
- /* Correlate locals after the first FILE symbol */
for (; ; file1_sym = next_file_symbol(e->orig, file1_sym),
file2_sym = next_file_symbol(e->patched, file2_sym)) {
@@ -503,92 +783,52 @@ static int correlate_symbols(struct elfs *e)
file1_sym->twin = file2_sym;
file2_sym->twin = file1_sym;
-
- sym1 = file1_sym;
-
- for_each_sym_continue(e->orig, sym1) {
- if (is_file_sym(sym1) || !is_local_sym(sym1))
- break;
-
- if (dont_correlate(sym1))
- continue;
-
- sym2 = file2_sym;
- for_each_sym_continue(e->patched, sym2) {
- if (is_file_sym(sym2) || !is_local_sym(sym2))
- break;
-
- if (sym2->twin || dont_correlate(sym2))
- continue;
-
- if (strcmp(sym1->demangled_name, sym2->demangled_name))
- continue;
-
- sym1->twin = sym2;
- sym2->twin = sym1;
- break;
- }
- }
}
- /* Correlate globals */
- for_each_sym(e->orig, sym1) {
- if (sym1->bind == STB_LOCAL)
- continue;
-
- sym2 = find_global_symbol_by_name(e->patched, sym1->name);
- if (sym2 && !sym2->twin) {
- sym1->twin = sym2;
- sym2->twin = sym1;
- }
- }
/*
- * Correlate globals with demangled_name.
- * A separate loop is needed because we want to finish all the
- * full name correlations first.
+ * Correlate in two phases: loop deterministic levels until no more
+ * progress, then use positional fallback for the rest. This prevents
+ * the nondeterministic positional matching from stealing symbols that
+ * have deterministic matches.
*/
+ hash_init(suffix_map);
+ do {
+ progress = false;
+ for_each_sym(e->orig, sym1) {
+ if (sym1->twin || dont_correlate(sym1))
+ continue;
+ sym2 = find_twin(e, sym1);
+ if (!sym2)
+ continue;
+ sym1->twin = sym2;
+ sym2->twin = sym1;
+ progress = true;
+ }
+
+ if (update_suffix_map(e->orig))
+ return -1;
+
+ for_each_sym(e->orig, sym1) {
+ if (sym1->twin || dont_correlate(sym1))
+ continue;
+ sym2 = find_twin_suffixed(e->patched, sym1);
+ if (!sym2)
+ continue;
+ sym1->twin = sym2;
+ sym2->twin = sym1;
+ progress = true;
+ }
+ } while (progress);
+
for_each_sym(e->orig, sym1) {
- if (sym1->bind == STB_LOCAL || sym1->twin)
+ if (sym1->twin || dont_correlate(sym1))
continue;
-
- if (find_global_symbol_by_demangled_name(e->patched, sym1, &sym2))
- return -1;
-
- if (sym2 && !sym2->twin) {
- sym1->twin = sym2;
- sym2->twin = sym1;
- }
- }
-
- /* Correlate original locals with patched globals */
- for_each_sym(e->orig, sym1) {
- if (sym1->twin || dont_correlate(sym1) || !is_local_sym(sym1))
+ sym2 = find_twin_positional(e, sym1);
+ if (!sym2)
continue;
-
- sym2 = find_global_symbol_by_name(e->patched, sym1->name);
- if (!sym2 && find_global_symbol_by_demangled_name(e->patched, sym1, &sym2))
- return -1;
-
- if (sym2 && !sym2->twin) {
- sym1->twin = sym2;
- sym2->twin = sym1;
- }
- }
-
- /* Correlate original globals with patched locals */
- for_each_sym(e->patched, sym2) {
- if (sym2->twin || dont_correlate(sym2) || !is_local_sym(sym2))
- continue;
-
- sym1 = find_global_symbol_by_name(e->orig, sym2->name);
- if (!sym1 && find_global_symbol_by_demangled_name(e->orig, sym2, &sym1))
- return -1;
-
- if (sym1 && !sym1->twin) {
- sym2->twin = sym1;
- sym1->twin = sym2;
- }
+ sym1->twin = sym2;
+ sym2->twin = sym1;
}
for_each_sym(e->orig, sym1) {
@@ -800,19 +1040,24 @@ static void mark_included_function(struct symbol *func)
*/
static int mark_changed_functions(struct elfs *e)
{
- struct symbol *sym_orig, *patched_sym;
+ struct symbol *orig_sym, *patched_sym;
bool changed = false;
/* Find changed functions */
- for_each_sym(e->orig, sym_orig) {
- if (!is_func_sym(sym_orig) || dont_correlate(sym_orig))
+ for_each_sym(e->orig, orig_sym) {
+ if (dont_correlate(orig_sym))
continue;
- patched_sym = sym_orig->twin;
+ patched_sym = orig_sym->twin;
if (!patched_sym)
continue;
- if (sym_orig->csum.checksum != patched_sym->csum.checksum) {
+ if (orig_sym->csum.checksum != patched_sym->csum.checksum) {
+ if (!is_func_sym(orig_sym)) {
+ ERROR("changed data: %s", orig_sym->name);
+ return -1;
+ }
+
patched_sym->changed = 1;
mark_included_function(patched_sym);
changed = true;
@@ -837,7 +1082,7 @@ static int mark_changed_functions(struct elfs *e)
printf("%s: changed function: %s\n", objname, patched_sym->name);
}
- return !changed ? -1 : 0;
+ return !changed ? 1 : 0;
}
static int clone_included_functions(struct elfs *e)
@@ -1870,6 +2115,7 @@ static int copy_import_ns(struct elfs *e)
int cmd_klp_diff(int argc, const char **argv)
{
struct elfs e = {0};
+ int ret;
argc = parse_options(argc, argv, klp_diff_options, klp_diff_usage, 0);
if (argc != 3)
@@ -1896,7 +2142,10 @@ int cmd_klp_diff(int argc, const char **argv)
if (correlate_symbols(&e))
return -1;
- if (mark_changed_functions(&e))
+ ret = mark_changed_functions(&e);
+ if (ret < 0)
+ return -1;
+ if (ret > 0)
return 0;
e.out = elf_create_file(&e.orig->ehdr, argv[2]);
--
2.53.0
^ permalink raw reply related
* [PATCH v2 47/53] objtool/klp: Add correlation debugging output
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Add debugging messages to show how duplicate symbols get correlated, and
split the --debug feature into --debug-correlate and --debug-clone.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/warn.h | 16 +++++++----
tools/objtool/klp-diff.c | 42 ++++++++++++++++++++++------
tools/objtool/objtool.c | 3 --
3 files changed, 45 insertions(+), 16 deletions(-)
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index 595ee8009667..a9936d60980c 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -109,7 +109,7 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define ERROR_FUNC(sec, offset, format, ...) __WARN_FUNC(ERROR_STR, sec, offset, format, ##__VA_ARGS__)
#define ERROR_INSN(insn, format, ...) ERROR_FUNC(insn->sec, insn->offset, format, ##__VA_ARGS__)
-extern bool debug;
+extern bool debug, debug_correlate, debug_clone;
extern int indent;
static inline void unindent(int *unused) { indent--; }
@@ -148,15 +148,21 @@ static inline void unindent(int *unused) { indent--; }
(unsigned long long)checksum); \
})
-#define __dbg_indent(format, ...) \
+#define dbg_correlate(args...) \
({ \
- if (unlikely(debug)) \
+ if (unlikely(debug_correlate)) \
+ __dbg(args); \
+})
+
+#define __dbg_clone(format, ...) \
+({ \
+ if (unlikely(debug_clone)) \
__dbg("%*s" format, indent * 8, "", ##__VA_ARGS__); \
})
-#define dbg_indent(args...) \
+#define dbg_clone(args...) \
int __cleanup(unindent) __dummy_##__COUNTER__; \
- __dbg_indent(args); \
+ __dbg_clone(args); \
indent++
#endif /* _WARN_H */
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index a9c993298b82..ed3bf1c55001 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -33,6 +33,9 @@ struct export {
char *mod, *sym;
};
+bool debug, debug_correlate, debug_clone;
+int indent;
+
static const char * const klp_diff_usage[] = {
"objtool klp diff [<options>] <in1.o> <in2.o> <out.o>",
NULL,
@@ -40,7 +43,9 @@ static const char * const klp_diff_usage[] = {
static const struct option klp_diff_options[] = {
OPT_GROUP("Options:"),
- OPT_BOOLEAN('d', "debug", &debug, "enable debug output"),
+ OPT_BOOLEAN('d', "debug", &debug, "enable all debug output"),
+ OPT_BOOLEAN(0, "debug-correlate", &debug_correlate, "enable correlation debug output"),
+ OPT_BOOLEAN(0, "debug-clone", &debug_clone, "enable cloning debug output"),
OPT_END(),
};
@@ -583,6 +588,14 @@ static struct symbol *find_twin(struct elfs *e, struct symbol *sym1)
else if (csum_orig == 1 && csum_patched == 1)
match = csum_last;
+ if (!match)
+ return NULL;
+
+ if (name_orig != 1 || name_patched != 1)
+ dbg_correlate("find_twin(): %s%s -> %s%s",
+ sym1->name, is_func_sym(sym1) ? "()" : "",
+ match->name, is_func_sym(match) ? "()" : "");
+
return match;
}
@@ -686,10 +699,14 @@ static struct symbol *find_twin_suffixed(struct elf *elf, struct symbol *sym1)
match = sym2;
}
- if (count == 1)
- return match;
+ if (count != 1)
+ return NULL;
- return NULL;
+ dbg_correlate("find_suffixed_twin(): %s%s -> %s%s",
+ sym1->name, is_func_sym(sym1) ? "()" : "",
+ match->name, is_func_sym(match) ? "()" : "");
+
+ return match;
}
/*
@@ -742,6 +759,10 @@ static struct symbol *find_twin_positional(struct elfs *e, struct symbol *sym1)
if (idx_orig != idx_patched)
return NULL;
+ dbg_correlate("find_twin_positional(): %s%s -> %s%s",
+ sym1->name, is_func_sym(sym1) ? "()" : "",
+ match->name, is_func_sym(match) ? "()" : "");
+
return match;
}
@@ -998,7 +1019,7 @@ static struct symbol *clone_symbol(struct elfs *e, struct symbol *patched_sym,
if (patched_sym->clone)
return patched_sym->clone;
- dbg_indent("%s%s", patched_sym->name, data_too ? " [+DATA]" : "");
+ dbg_clone("%s%s", patched_sym->name, data_too ? " [+DATA]" : "");
/* Make sure the prefix gets cloned first */
if (is_func_sym(patched_sym) && data_too) {
@@ -1375,7 +1396,7 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
klp_sym = find_symbol_by_name(e->out, sym_name);
if (!klp_sym) {
- __dbg_indent("%s", sym_name);
+ __dbg_clone("%s", sym_name);
/* STB_WEAK: avoid modpost undefined symbol warnings */
klp_sym = elf_create_symbol(e->out, sym_name, NULL,
@@ -1426,7 +1447,7 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
}
#define dbg_clone_reloc(sec, offset, patched_sym, addend, export, klp) \
- dbg_indent("%s+0x%lx: %s%s0x%lx [%s%s%s%s%s%s]", \
+ dbg_clone("%s+0x%lx: %s%s0x%lx [%s%s%s%s%s%s]", \
sec->name, offset, patched_sym->name, \
addend >= 0 ? "+" : "-", labs(addend), \
sym_type(patched_sym), \
@@ -1481,7 +1502,7 @@ static int clone_reloc(struct elfs *e, struct reloc *patched_reloc,
if (is_string_sec(patched_sym->sec)) {
const char *str = patched_sym->sec->data->d_buf + addend;
- __dbg_indent("\"%s\"", escape_str(str));
+ __dbg_clone("\"%s\"", escape_str(str));
addend = elf_add_string(e->out, out_sym->sec, str);
if (addend == -1)
@@ -2121,6 +2142,11 @@ int cmd_klp_diff(int argc, const char **argv)
if (argc != 3)
usage_with_options(klp_diff_usage, klp_diff_options);
+ if (debug) {
+ debug_correlate = true;
+ debug_clone = true;
+ }
+
objname = argv[0];
e.orig = elf_open_read(argv[0], O_RDONLY);
diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c
index 1c3622117c33..a4e139dee7e9 100644
--- a/tools/objtool/objtool.c
+++ b/tools/objtool/objtool.c
@@ -16,9 +16,6 @@
#include <objtool/objtool.h>
#include <objtool/warn.h>
-bool debug;
-int indent;
-
static struct objtool_file file;
struct objtool_file *objtool_open_read(const char *filename)
--
2.53.0
^ permalink raw reply related
* [PATCH v2 45/53] objtool/klp: Calculate object checksums
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
Start checksumming data objects in preparation for revamping the
correlation algorithm.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/checksum.h | 44 ++++++++----
tools/objtool/include/objtool/warn.h | 29 ++++----
tools/objtool/klp-checksum.c | 89 +++++++++++++++++++-----
tools/objtool/klp-diff.c | 2 +-
4 files changed, 117 insertions(+), 47 deletions(-)
diff --git a/tools/objtool/include/objtool/checksum.h b/tools/objtool/include/objtool/checksum.h
index be4eb7dfe6f2..d46293f54716 100644
--- a/tools/objtool/include/objtool/checksum.h
+++ b/tools/objtool/include/objtool/checksum.h
@@ -6,28 +6,44 @@
#ifdef BUILD_KLP
-static inline void checksum_init(struct symbol *func)
+static inline void checksum_init(struct symbol *sym)
{
- if (func && !func->csum.state) {
- func->csum.state = XXH3_createState();
- XXH3_64bits_reset(func->csum.state);
+ if (sym && !sym->csum.state) {
+ sym->csum.state = XXH3_createState();
+ XXH3_64bits_reset(sym->csum.state);
}
}
-static inline void checksum_update(struct symbol *func,
- struct instruction *insn,
- const void *data, size_t size)
+static inline void __checksum_update(struct symbol *sym, const void *data,
+ size_t size)
{
- XXH3_64bits_update(func->csum.state, data, size);
- dbg_checksum(func, insn, XXH3_64bits_digest(func->csum.state));
+ XXH3_64bits_update(sym->csum.state, data, size);
}
-static inline void checksum_finish(struct symbol *func)
+static inline void __checksum_update_insn(struct symbol *sym,
+ struct instruction *insn,
+ const void *data, size_t size)
{
- if (func && func->csum.state) {
- func->csum.checksum = XXH3_64bits_digest(func->csum.state);
- XXH3_freeState(func->csum.state);
- func->csum.state = NULL;
+ __checksum_update(sym, data, size);
+ dbg_checksum_insn(sym, insn, XXH3_64bits_digest(sym->csum.state));
+}
+
+static inline void __checksum_update_object(struct symbol *sym,
+ unsigned long offset,
+ const char *what, const void *data,
+ size_t size)
+{
+ __checksum_update(sym, &offset, sizeof(offset));
+ __checksum_update(sym, data, size);
+ dbg_checksum_object(sym, offset, what, XXH3_64bits_digest(sym->csum.state));
+}
+
+static inline void checksum_finish(struct symbol *sym)
+{
+ if (sym && sym->csum.state) {
+ sym->csum.checksum = XXH3_64bits_digest(sym->csum.state);
+ XXH3_freeState(sym->csum.state);
+ sym->csum.state = NULL;
}
}
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index fa8b7d292e83..595ee8009667 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -130,10 +130,22 @@ static inline void unindent(int *unused) { indent--; }
objname ? ": " : "", \
##__VA_ARGS__)
-#define dbg(args...) \
+#define dbg_checksum_insn(func, insn, checksum) \
({ \
- if (unlikely(debug)) \
- __dbg(args); \
+ if (unlikely(func->debug_checksum)) { \
+ char *insn_off = offstr(insn->sec, insn->offset); \
+ __dbg("checksum: %s(): %s %016llx", \
+ func->name, insn_off, (unsigned long long)checksum);\
+ free(insn_off); \
+ } \
+})
+
+#define dbg_checksum_object(sym, offset, what, checksum) \
+({ \
+ if (unlikely(sym->debug_checksum)) \
+ __dbg("checksum: %s+0x%lx: %s %016llx", \
+ sym->name, offset, what, \
+ (unsigned long long)checksum); \
})
#define __dbg_indent(format, ...) \
@@ -147,15 +159,4 @@ static inline void unindent(int *unused) { indent--; }
__dbg_indent(args); \
indent++
-#define dbg_checksum(func, insn, checksum) \
-({ \
- if (unlikely(insn->sym && insn->sym->pfunc && \
- insn->sym->pfunc->debug_checksum)) { \
- char *insn_off = offstr(insn->sec, insn->offset); \
- __dbg("checksum: %s %s %016llx", \
- func->name, insn_off, (unsigned long long)checksum);\
- free(insn_off); \
- } \
-})
-
#endif /* _WARN_H */
diff --git a/tools/objtool/klp-checksum.c b/tools/objtool/klp-checksum.c
index e4a910f3211c..19653dbe109d 100644
--- a/tools/objtool/klp-checksum.c
+++ b/tools/objtool/klp-checksum.c
@@ -35,7 +35,7 @@ static int checksum_debug_init(struct objtool_file *file)
*comma = '\0';
for_each_sym_by_name(file->elf, s, sym) {
- if (!is_func_sym(sym))
+ if (!is_func_sym(sym) && !is_object_sym(sym))
continue;
sym->debug_checksum = 1;
found = true;
@@ -66,14 +66,14 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
if (insn->fake)
return;
- checksum_update(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
+ __checksum_update_insn(func, insn, insn->sec->data->d_buf + insn->offset, insn->len);
if (!reloc) {
struct symbol *call_dest = insn_call_dest(insn);
if (call_dest)
- checksum_update(func, insn, call_dest->demangled_name,
- strlen(call_dest->demangled_name));
+ __checksum_update_insn(func, insn, call_dest->demangled_name,
+ strlen(call_dest->demangled_name));
goto alts;
}
@@ -84,7 +84,7 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
char *str;
str = sym->sec->data->d_buf + sym->offset + offset;
- checksum_update(func, insn, str, strlen(str));
+ __checksum_update_insn(func, insn, str, strlen(str));
goto alts;
}
@@ -96,8 +96,9 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
offset -= sym->offset;
}
- checksum_update(func, insn, sym->demangled_name, strlen(sym->demangled_name));
- checksum_update(func, insn, &offset, sizeof(offset));
+ __checksum_update_insn(func, insn, sym->demangled_name,
+ strlen(sym->demangled_name));
+ __checksum_update_insn(func, insn, &offset, sizeof(offset));
alts:
for (alt = insn->alts; alt; alt = alt->next) {
@@ -108,12 +109,13 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
break;
in_alt = true;
- checksum_update(func, insn, &alt->type, sizeof(alt->type));
+ __checksum_update_insn(func, insn, &alt->type,
+ sizeof(alt->type));
if (alt_group && alt_group->orig_group) {
struct instruction *alt_insn;
- checksum_update(func, insn, &alt_group->feature, sizeof(alt_group->feature));
+ __checksum_update_insn(func, insn, &alt_group->feature,sizeof(alt_group->feature));
for (alt_insn = alt->insn; alt_insn; alt_insn = next_insn_same_sec(file, alt_insn)) {
checksum_update_insn(file, func, alt_insn);
@@ -128,31 +130,82 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
}
}
+static void checksum_update_object(struct objtool_file *file, struct symbol *sym)
+{
+ struct reloc *reloc;
+
+ __checksum_update_object(sym, 0, "len", &sym->len, sizeof(sym->len));
+
+ if (sym->sec->data->d_buf)
+ __checksum_update_object(sym, 0, "data",
+ sym->sec->data->d_buf + sym->offset,
+ sym->len);
+
+ sym_for_each_reloc(file->elf, sym, reloc) {
+ unsigned long sym_offset = reloc_offset(reloc) - sym->offset;
+ struct symbol *target = reloc->sym;
+ s64 offset;
+
+ offset = reloc_addend(reloc);
+
+ if (is_string_sec(target->sec)) {
+ char *str;
+
+ str = target->sec->data->d_buf + target->offset + offset;
+ __checksum_update_object(sym, sym_offset,
+ "reloc string", str, strlen(str));
+ continue;
+ }
+
+ if (is_sec_sym(target)) {
+ target = find_symbol_containing(reloc->sym->sec, offset);
+ if (!target)
+ continue;
+
+ offset -= target->offset;
+ }
+
+ __checksum_update_object(sym, sym_offset, "reloc name",
+ target->demangled_name,
+ strlen(target->demangled_name));
+ __checksum_update_object(sym, sym_offset, "reloc addend",
+ &offset, sizeof(offset));
+ }
+}
+
int calculate_checksums(struct objtool_file *file)
{
struct instruction *insn;
- struct symbol *func;
+ struct symbol *sym;
if (checksum_debug_init(file))
return -1;
- for_each_sym(file->elf, func) {
+ for_each_sym(file->elf, sym) {
+
/*
* Skip cold subfunctions and aliases: they share the
* parent's checksum via func_for_each_insn() which
* follows func->cfunc into the cold subfunction.
*/
- if (!is_func_sym(func) || is_cold_func(func) ||
- is_alias_sym(func) || !func->len)
+ if (is_cold_func(sym) || is_alias_sym(sym) || !sym->len ||
+ !sym->sec || !sym->sec->data)
continue;
- checksum_init(func);
+ if (is_func_sym(sym)) {
+ checksum_init(sym);
+ func_for_each_insn(file, sym, insn)
+ checksum_update_insn(file, sym, insn);
+ checksum_finish(sym);
- func_for_each_insn(file, func, insn)
- checksum_update_insn(file, func, insn);
+ } else if (is_object_sym(sym)) {
+ checksum_init(sym);
+ checksum_update_object(file, sym);
+ checksum_finish(sym);
+ }
- checksum_finish(func);
}
+
return 0;
}
@@ -213,7 +266,7 @@ int cmd_klp_checksum(int argc, const char **argv)
int ret;
const struct option options[] = {
- OPT_STRING(0, "debug-checksum", &opts.debug_checksum, "funcs", "enable checksum debug output"),
+ OPT_STRING(0, "debug-checksum", &opts.debug_checksum, "syms", "enable checksum debug output"),
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
OPT_END(),
};
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 95359ad336bb..5f13d759e02f 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -201,7 +201,7 @@ static int read_sym_checksums(struct elf *elf)
return -1;
}
- if (is_func_sym(sym))
+ if (is_func_sym(sym) || is_object_sym(sym))
sym->csum.checksum = sym_checksum->checksum;
}
--
2.53.0
^ permalink raw reply related
* [PATCH v2 44/53] klp-build: Validate short-circuit prerequisites
From: Josh Poimboeuf @ 2026-05-01 4:08 UTC (permalink / raw)
To: x86
Cc: linux-kernel, live-patching, Peter Zijlstra, Joe Lawrence,
Song Liu, Miroslav Benes, Petr Mladek
In-Reply-To: <cover.1777575752.git.jpoimboe@kernel.org>
The --short-circuit option implicitly requires that certain directories
are already in klp-tmp. Enforce that to prevent confusing errors.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/livepatch/klp-build | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/scripts/livepatch/klp-build b/scripts/livepatch/klp-build
index f8a80ad0f829..3adb2a7fd9c1 100755
--- a/scripts/livepatch/klp-build
+++ b/scripts/livepatch/klp-build
@@ -437,6 +437,20 @@ do_init() {
# builds in pwd.
[[ ! "$PWD" -ef "$SCRIPT_DIR/../.." ]] && die "please run from the kernel root directory"
+ if (( SHORT_CIRCUIT >= 2 )); then
+ [[ -f "$ORIG_DIR/.complete" ]] || die "-S $SHORT_CIRCUIT requires completed $ORIG_DIR"
+ fi
+ if (( SHORT_CIRCUIT >= 3 )); then
+ [[ -f "$PATCHED_DIR/.complete" ]] || die "-S $SHORT_CIRCUIT requires completed $PATCHED_DIR"
+ fi
+ if (( SHORT_CIRCUIT >= 4 )); then
+ [[ -f "$ORIG_CSUM_DIR/.complete" ]] || die "-S $SHORT_CIRCUIT requires completed $ORIG_CSUM_DIR"
+ [[ -f "$PATCHED_CSUM_DIR/.complete" ]] || die "-S $SHORT_CIRCUIT requires completed $PATCHED_CSUM_DIR"
+ fi
+ if (( SHORT_CIRCUIT >= 5 )); then
+ [[ -f "$DIFF_DIR/.complete" ]] || die "-S $SHORT_CIRCUIT requires completed $DIFF_DIR"
+ fi
+
(( SHORT_CIRCUIT <= 1 )) && rm -rf "$TMP_DIR"
mkdir -p "$TMP_DIR"
@@ -593,6 +607,7 @@ copy_orig_objects() {
mv -f "$TMP_DIR/build.log" "$ORIG_DIR"
touch "$TIMESTAMP"
+ touch "$ORIG_DIR/.complete"
}
# Copy all changed objects to $PATCHED_DIR
@@ -631,6 +646,7 @@ copy_patched_objects() {
(( found == 0 )) && die "no changes detected"
mv -f "$TMP_DIR/build.log" "$PATCHED_DIR"
+ touch "$PATCHED_DIR/.complete"
}
# Copy .o files to a separate directory and run "objtool klp checksum" on each
@@ -712,6 +728,8 @@ diff_objects() {
die "objtool klp diff failed"
)
done
+
+ touch "$DIFF_DIR/.complete"
}
# For each changed object, run "objtool klp checksum" with --debug-checksum to
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox