* Re: [PATCH] powerpc64/bpf: support direct_call on livepatch function
From: Naveen N Rao @ 2025-10-06 7:52 UTC (permalink / raw)
To: Hari Bathini
Cc: Madhavan Srinivasan, linuxppc-dev, Christophe Leroy,
Michael Ellerman, Nicholas Piggin, bpf, Alexei Starovoitov,
Daniel Borkmann, Andrii Nakryiko, Song Liu, Jiri Olsa,
Viktor Malik, live-patching, Josh Poimboeuf, Joe Lawrence,
Jiri Kosina, linux-trace-kernel, Steven Rostedt, Masami Hiramatsu,
Mark Rutland, Shung-Hsi Yu
In-Reply-To: <20251002192755.86441-1-hbathini@linux.ibm.com>
On Fri, Oct 03, 2025 at 12:57:54AM +0530, Hari Bathini wrote:
> Today, livepatch takes precedence over direct_call. Instead, save the
> state and make direct_call before handling livepatch.
If we call into the BPF trampoline first and if we have
BPF_TRAMP_F_CALL_ORIG set, does this result in the BPF trampoline
calling the new copy of the live-patched function or the old one?
- Naveen
^ permalink raw reply
* Re: [PATCH] powerpc64/bpf: support direct_call on livepatch function
From: kernel test robot @ 2025-10-03 11:21 UTC (permalink / raw)
To: Hari Bathini, Madhavan Srinivasan
Cc: oe-kbuild-all, linuxppc-dev, Christophe Leroy, Naveen N. Rao,
Michael Ellerman, Nicholas Piggin, bpf, Alexei Starovoitov,
Daniel Borkmann, Andrii Nakryiko, Song Liu, Jiri Olsa,
Viktor Malik, live-patching, Josh Poimboeuf, Joe Lawrence,
Jiri Kosina, linux-trace-kernel, Steven Rostedt, Masami Hiramatsu,
Mark Rutland, Shung-Hsi Yu
In-Reply-To: <20251002192755.86441-1-hbathini@linux.ibm.com>
Hi Hari,
kernel test robot noticed the following build errors:
[auto build test ERROR on powerpc/next]
[also build test ERROR on powerpc/fixes trace/for-next bpf-next/net bpf-next/master bpf/master linus/master v6.17 next-20251002]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Hari-Bathini/powerpc64-bpf-support-direct_call-on-livepatch-function/20251003-033243
base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
patch link: https://lore.kernel.org/r/20251002192755.86441-1-hbathini%40linux.ibm.com
patch subject: [PATCH] powerpc64/bpf: support direct_call on livepatch function
config: powerpc64-randconfig-001-20251003 (https://download.01.org/0day-ci/archive/20251003/202510031817.t50YvoeN-lkp@intel.com/config)
compiler: powerpc64-linux-gcc (GCC) 12.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251003/202510031817.t50YvoeN-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510031817.t50YvoeN-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/powerpc/net/bpf_jit_comp.c:24:
arch/powerpc/net/bpf_jit_comp.c: In function '__arch_prepare_bpf_trampoline':
>> include/linux/stddef.h:16:33: error: 'struct thread_info' has no member named 'livepatch_sp'
16 | #define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
| ^~~~~~~~~~~~~~~~~~
arch/powerpc/net/bpf_jit.h:29:34: note: in definition of macro 'PLANT_INSTR'
29 | do { if (d) { (d)[idx] = instr; } idx++; } while (0)
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:857:17: note: in expansion of macro 'EMIT'
857 | EMIT(PPC_RAW_ADDI(_R12, _R12, offsetof(struct thread_info, livepatch_sp)));
| ^~~~
arch/powerpc/include/asm/ppc-opcode.h:501:85: note: in expansion of macro 'IMM_L'
501 | #define PPC_RAW_ADDI(d, a, i) (0x38000000 | ___PPC_RT(d) | ___PPC_RA(a) | IMM_L(i))
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:857:22: note: in expansion of macro 'PPC_RAW_ADDI'
857 | EMIT(PPC_RAW_ADDI(_R12, _R12, offsetof(struct thread_info, livepatch_sp)));
| ^~~~~~~~~~~~
arch/powerpc/net/bpf_jit_comp.c:857:47: note: in expansion of macro 'offsetof'
857 | EMIT(PPC_RAW_ADDI(_R12, _R12, offsetof(struct thread_info, livepatch_sp)));
| ^~~~~~~~
>> arch/powerpc/net/bpf_jit_comp.c:860:48: error: 'LIVEPATCH_STACK_FRAME_SIZE' undeclared (first use in this function)
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
arch/powerpc/net/bpf_jit.h:29:34: note: in definition of macro 'PLANT_INSTR'
29 | do { if (d) { (d)[idx] = instr; } idx++; } while (0)
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:860:17: note: in expansion of macro 'EMIT'
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~
arch/powerpc/include/asm/ppc-opcode.h:501:85: note: in expansion of macro 'IMM_L'
501 | #define PPC_RAW_ADDI(d, a, i) (0x38000000 | ___PPC_RT(d) | ___PPC_RA(a) | IMM_L(i))
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:860:22: note: in expansion of macro 'PPC_RAW_ADDI'
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~~~~~~~~~
arch/powerpc/net/bpf_jit_comp.c:860:48: note: each undeclared identifier is reported only once for each function it appears in
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
arch/powerpc/net/bpf_jit.h:29:34: note: in definition of macro 'PLANT_INSTR'
29 | do { if (d) { (d)[idx] = instr; } idx++; } while (0)
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:860:17: note: in expansion of macro 'EMIT'
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~
arch/powerpc/include/asm/ppc-opcode.h:501:85: note: in expansion of macro 'IMM_L'
501 | #define PPC_RAW_ADDI(d, a, i) (0x38000000 | ___PPC_RT(d) | ___PPC_RA(a) | IMM_L(i))
| ^~~~~
arch/powerpc/net/bpf_jit_comp.c:860:22: note: in expansion of macro 'PPC_RAW_ADDI'
860 | EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
| ^~~~~~~~~~~~
vim +16 include/linux/stddef.h
6e218287432472 Richard Knutsson 2006-09-30 14
^1da177e4c3f41 Linus Torvalds 2005-04-16 15 #undef offsetof
14e83077d55ff4 Rasmus Villemoes 2022-03-23 @16 #define offsetof(TYPE, MEMBER) __builtin_offsetof(TYPE, MEMBER)
3876488444e712 Denys Vlasenko 2015-03-09 17
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [BUG?] ppc64le: fentry BPF not triggered after live patch (v6.14)
From: Hari Bathini @ 2025-10-02 19:45 UTC (permalink / raw)
To: Song Liu, Naveen N Rao
Cc: Jiri Olsa, Steven Rostedt, Shung-Hsi Yu, bpf, Michael Ellerman,
Mark Rutland, Daniel Borkmann, Masahiro Yamada, Nicholas Piggin,
Alexei Starovoitov, Masami Hiramatsu, Andrii Nakryiko,
Christophe Leroy, Vishal Chourasia, Mahesh J Salgaonkar,
Miroslav Benes, Michal Suchánek, linux-kernel, linuxppc-dev,
linux-trace-kernel, live-patching
In-Reply-To: <81b222ec-7635-411f-b72f-804284295edf@linux.ibm.com>
On 07/04/25 1:52 pm, Hari Bathini wrote:
>
>
> On 04/04/25 12:03 am, Song Liu wrote:
>> On Thu, Apr 3, 2025 at 8:30 AM Naveen N Rao <naveen@kernel.org> wrote:
>> [...]
>>>
>>> We haven't addressed this particular interaction in the powerpc support
>>> for ftrace direct and BPF trampolines. Right now, live patching takes
>>> priority so we call the livepatch'ed function and skip further ftrace
>>> direct calls.
>>>
>>> I'm curious if this works on arm64 with which we share support for
>>> DYNAMIC_FTRACE_WITH_CALL_OPS.
>>
>> We still need to land [1] for arm64 to support livepatch. In a quick test
>> with [1], livepatch and bpf trampoline works together. I haven't looked
>> into the arm64 JIT code, so I am not sure whether all the corner cases
>> are properly handled.
>>
>> [1] https://lore.kernel.org/live-patching/20250320171559.3423224-1-
>> song@kernel.org/
>
> Thanks for checking this on arm64, Song.
> As Naveen pointed out, with out of line trampoline
> on ppc64le, there are a few things to sort out with
> regard to livepatch & BPF Trampoline interaction. Will
> try and take a stab at it soon.
Sorry, couldn't get to it sooner.
Posted the change that fixes livepatch & BPF trampoline
interaction in powerpc upstream now. Please take a look:
https://lore.kernel.org/all/20251002192755.86441-1-hbathini@linux.ibm.com/
- Hari
^ permalink raw reply
* [PATCH] powerpc64/bpf: support direct_call on livepatch function
From: Hari Bathini @ 2025-10-02 19:27 UTC (permalink / raw)
To: Madhavan Srinivasan
Cc: linuxppc-dev, Christophe Leroy, Naveen N. Rao, Michael Ellerman,
Nicholas Piggin, bpf, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Song Liu, Jiri Olsa, Viktor Malik, live-patching,
Josh Poimboeuf, Joe Lawrence, Jiri Kosina, linux-trace-kernel,
Steven Rostedt, Masami Hiramatsu, Mark Rutland, Shung-Hsi Yu
Today, livepatch takes precedence over direct_call. Instead, save the
state and make direct_call before handling livepatch. This change
inadvertly skips livepatch stack restore, when an attached fmod_ret
program fails. To handle this scenario, set cr0.eq bit to indicate
livepatch is active while making the direct_call, save the expected
livepatch stack state on the trampoline stack and restore it, if and
when required, during do_fexit in the trampoline code.
Reported-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Closes: https://lore.kernel.org/all/rwmwrvvtg3pd7qrnt3of6dideioohwhsplancoc2gdrjran7bg@j5tqng6loymr/
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
---
arch/powerpc/include/asm/livepatch.h | 15 +++++
arch/powerpc/kernel/trace/ftrace_entry.S | 74 ++++++++++++++++++++----
arch/powerpc/net/bpf_jit_comp.c | 71 ++++++++++++++++++++++-
3 files changed, 149 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/include/asm/livepatch.h b/arch/powerpc/include/asm/livepatch.h
index d044a1fd4f44..356c1eb46f5d 100644
--- a/arch/powerpc/include/asm/livepatch.h
+++ b/arch/powerpc/include/asm/livepatch.h
@@ -7,6 +7,20 @@
#ifndef _ASM_POWERPC_LIVEPATCH_H
#define _ASM_POWERPC_LIVEPATCH_H
+#ifdef CONFIG_LIVEPATCH_64
+#define LIVEPATCH_STACK_MAGIC_OFFSET 8
+#define LIVEPATCH_STACK_LR_OFFSET 16
+#define LIVEPATCH_STACK_TOC_OFFSET 24
+
+#if defined(CONFIG_PPC_FTRACE_OUT_OF_LINE) && defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS)
+#define LIVEPATCH_STACK_FRAME_SIZE 32 /* Allocate 4 x 8 bytes (to save new NIP as well) */
+#define LIVEPATCH_STACK_NIP_OFFSET 32
+#else
+#define LIVEPATCH_STACK_FRAME_SIZE 24 /* Allocate 3 x 8 bytes */
+#endif
+#endif
+
+#ifndef __ASSEMBLY__
#include <linux/sched.h>
#include <linux/sched/task_stack.h>
@@ -20,4 +34,5 @@ static inline void klp_init_thread_info(struct task_struct *p)
static inline void klp_init_thread_info(struct task_struct *p) { }
#endif
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_LIVEPATCH_H */
diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S
index 6599fe3c6234..b98f12f378b1 100644
--- a/arch/powerpc/kernel/trace/ftrace_entry.S
+++ b/arch/powerpc/kernel/trace/ftrace_entry.S
@@ -8,6 +8,7 @@
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ftrace.h>
+#include <asm/livepatch.h>
#include <asm/ppc-opcode.h>
#include <asm/thread_info.h>
#include <asm/bug.h>
@@ -244,6 +245,8 @@
/* jump after _mcount site */
#ifdef CONFIG_PPC_FTRACE_OUT_OF_LINE
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
+ /* For direct_call, set cr0.eq bit only if livepatch is active */
+ crclr 4*cr0+eq
bnectr cr1
#endif
/*
@@ -306,10 +309,14 @@ ftrace_no_trace:
mtctr r12
REST_GPRS(11, 12, r1)
addi r1, r1, SWITCH_FRAME_SIZE+STACK_FRAME_MIN_SIZE
+ /* For direct_call, set cr0.eq bit only if livepatch is active */
+ crclr 4*cr0+eq
bctr
.Lftrace_direct_call:
mtctr r12
addi r1, r1, SWITCH_FRAME_SIZE+STACK_FRAME_MIN_SIZE
+ /* For direct_call, set cr0.eq bit only if livepatch is active */
+ crclr 4*cr0+eq
bctr
SYM_FUNC_START(ftrace_stub_direct_tramp)
blr
@@ -340,25 +347,72 @@ SYM_FUNC_END(ftrace_stub_direct_tramp)
livepatch_handler:
ld r12, PACA_THREAD_INFO(r13)
- /* Allocate 3 x 8 bytes */
ld r11, TI_livepatch_sp(r12)
- addi r11, r11, 24
+ /* Allocate stack to save LR, TOC & optionally NIP (in case of direct_call) */
+ addi r11, r11, LIVEPATCH_STACK_FRAME_SIZE
std r11, TI_livepatch_sp(r12)
/* Store stack end marker */
lis r12, STACK_END_MAGIC@h
ori r12, r12, STACK_END_MAGIC@l
- std r12, -8(r11)
+ std r12, -LIVEPATCH_STACK_MAGIC_OFFSET(r11)
/* Save toc & real LR on livepatch stack */
- std r2, -24(r11)
+ std r2, -LIVEPATCH_STACK_TOC_OFFSET(r11)
#ifndef CONFIG_PPC_FTRACE_OUT_OF_LINE
mflr r12
- std r12, -16(r11)
+ std r12, -LIVEPATCH_STACK_LR_OFFSET(r11)
mfctr r12
#else
- std r0, -16(r11)
+ std r0, -LIVEPATCH_STACK_LR_OFFSET(r11)
mflr r12
+
+ /* Also, save new NIP on livepatch stack before the direct_call */
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
+ std r12, -LIVEPATCH_STACK_NIP_OFFSET(r11)
+
+ /* For direct_call, set cr0.eq bit to indicate livepatch is active */
+ crset 4*cr0+eq
+ /* Jump to the direct_call */
+ bnectrl cr1
+
+ /*
+ * The address to jump after direct call is deduced based on ftrace OOL stub sequence.
+ * The seemingly insignificant couple of instructions below is to mimic that here to
+ * jump back to the livepatch handler code below.
+ */
+ nop
+ b 1f
+
+ /*
+ * Restore the state for livepatching from the livepatch stack.
+ * Before that, check if livepatch stack is intact. Use r0 for it.
+ */
+1: mtctr r0
+ ld r12, PACA_THREAD_INFO(r13)
+ ld r11, TI_livepatch_sp(r12)
+ lis r0, STACK_END_MAGIC@h
+ ori r0, r0, STACK_END_MAGIC@l
+ ld r12, -LIVEPATCH_STACK_MAGIC_OFFSET(r11)
+1: tdne r12, r0
+ EMIT_BUG_ENTRY 1b, __FILE__, __LINE__ - 1, 0
+ mfctr r0
+
+ /*
+ * A change in r0 implies the direct_call is not done yet. The direct_call
+ * will take care of calling the original LR. Update r0 in livepatch stack
+ * with the new LR in the direct_call.
+ */
+ ld r12, -LIVEPATCH_STACK_LR_OFFSET(r11)
+ cmpd r12, r0
+ beq 1f
+ mflr r0
+ std r0, -LIVEPATCH_STACK_LR_OFFSET(r11)
+
+ /* Put new NIP back in r12 to proceed with livepatch handling */
+1: ld r12, -LIVEPATCH_STACK_NIP_OFFSET(r11)
+#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */
+
/* Put ctr in r12 for global entry and branch there */
mtctr r12
#endif
@@ -377,18 +431,18 @@ livepatch_handler:
/* Check stack marker hasn't been trashed */
lis r2, STACK_END_MAGIC@h
ori r2, r2, STACK_END_MAGIC@l
- ld r12, -8(r11)
+ ld r12, -LIVEPATCH_STACK_MAGIC_OFFSET(r11)
1: tdne r12, r2
EMIT_BUG_ENTRY 1b, __FILE__, __LINE__ - 1, 0
/* Restore LR & toc from livepatch stack */
- ld r12, -16(r11)
+ ld r12, -LIVEPATCH_STACK_LR_OFFSET(r11)
mtlr r12
- ld r2, -24(r11)
+ ld r2, -LIVEPATCH_STACK_TOC_OFFSET(r11)
/* Pop livepatch stack frame */
ld r12, PACA_THREAD_INFO(r13)
- subi r11, r11, 24
+ subi r11, r11, LIVEPATCH_STACK_FRAME_SIZE
std r11, TI_livepatch_sp(r12)
/* Return to original caller of live patched function */
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 88ad5ba7b87f..cc86867d85cd 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -19,6 +19,7 @@
#include <asm/kprobes.h>
#include <asm/text-patching.h>
+#include <asm/livepatch.h>
#include "bpf_jit.h"
@@ -678,14 +679,16 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
struct bpf_tramp_links *tlinks,
void *func_addr)
{
- int regs_off, nregs_off, ip_off, run_ctx_off, retval_off, nvr_off, alt_lr_off, r4_off = 0;
int i, ret, nr_regs, bpf_frame_size = 0, bpf_dummy_frame_size = 0, func_frame_offset;
struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN];
+ int regs_off, nregs_off, ip_off, run_ctx_off, retval_off, nvr_off;
struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY];
struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT];
+ int alt_lr_off, r4_off = 0, livepatch_sp_off = 0;
struct codegen_context codegen_ctx, *ctx;
u32 *image = (u32 *)rw_image;
ppc_inst_t branch_insn;
+ bool handle_lp = false;
u32 *branches = NULL;
bool save_ret;
@@ -716,6 +719,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
* dummy frame for unwind [ back chain 1 ] --
* [ padding ] align stack frame
* r4_off [ r4 (tailcallcnt) ] optional - 32-bit powerpc
+ * [ *current.TI.lp_sp ]
+ * livepatch_sp_off [ current.TI.lp_sp ] optional - livepatch stack info
* alt_lr_off [ real lr (ool stub)] optional - actual lr
* [ r26 ]
* nvr_off [ r25 ] nvr save area
@@ -780,10 +785,20 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
nvr_off = bpf_frame_size;
bpf_frame_size += 2 * SZL;
+
/* Optional save area for actual LR in case of ool ftrace */
if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) {
alt_lr_off = bpf_frame_size;
bpf_frame_size += SZL;
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS)) {
+ handle_lp = (func_ptr_is_kernel_text(func_addr) && fmod_ret->nr_links &&
+ (flags & BPF_TRAMP_F_CALL_ORIG));
+ }
+ }
+
+ if (handle_lp) {
+ livepatch_sp_off = bpf_frame_size;
+ bpf_frame_size += 2 * SZL;
}
if (IS_ENABLED(CONFIG_PPC32)) {
@@ -822,6 +837,30 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
if (IS_ENABLED(CONFIG_PPC32) && nr_regs < 2)
EMIT(PPC_RAW_STL(_R4, _R1, r4_off));
+ /* Save expected livepatch stack state on the trampoline stack */
+ if (handle_lp) {
+ /*
+ * The caller is expected to set cr0.eq bit, if livepatch was active on it.
+ *
+ * If livepatch is active, save address & the expected value of
+ * livepatch stack pointer on the trampoline stack.
+ * Else, set both of them to 0.
+ */
+ PPC_BCC_SHORT(COND_EQ, (ctx->idx + 5) * 4);
+ EMIT(PPC_RAW_LI(_R12, 0));
+ EMIT(PPC_RAW_STL(_R12, _R1, livepatch_sp_off));
+ EMIT(PPC_RAW_STL(_R12, _R1, livepatch_sp_off + SZL));
+ PPC_JMP((ctx->idx + 7) * 4);
+
+ EMIT(PPC_RAW_LL(_R12, _R13, offsetof(struct paca_struct, __current) +
+ offsetof(struct task_struct, thread_info)));
+ EMIT(PPC_RAW_ADDI(_R12, _R12, offsetof(struct thread_info, livepatch_sp)));
+ EMIT(PPC_RAW_STL(_R12, _R1, livepatch_sp_off));
+ EMIT(PPC_RAW_LL(_R12, _R12, 0));
+ EMIT(PPC_RAW_ADDI(_R12, _R12, -LIVEPATCH_STACK_FRAME_SIZE));
+ EMIT(PPC_RAW_STL(_R12, _R1, livepatch_sp_off + SZL));
+ }
+
bpf_trampoline_save_args(image, ctx, func_frame_offset, nr_regs, regs_off);
/* Save our return address */
@@ -932,6 +971,36 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
image[branches[i]] = ppc_inst_val(branch_insn);
}
+ /*
+ * Restore livepatch stack state if livepatch was active & an attached
+ * fmod_ret program failed.
+ */
+ if (handle_lp) {
+ EMIT(PPC_RAW_LL(_R12, _R1, livepatch_sp_off + SZL));
+ EMIT(PPC_RAW_CMPLI(_R12, 0));
+
+ /*
+ * If expected value (_R12) of livepatch stack pointer saved on the
+ * trampoline stack is 0, livepatch was not active. Skip the rest.
+ */
+ PPC_BCC_SHORT(COND_EQ, (ctx->idx + 7) * 4);
+
+ EMIT(PPC_RAW_LL(_R25, _R1, livepatch_sp_off));
+ EMIT(PPC_RAW_LL(_R25, _R25, 0));
+
+ /*
+ * If the expected value (_R12) of livepatch stack pointer saved on the
+ * trampoline stack is not the same as actual value (_R25), it implies
+ * fmod_ret program failed and skipped calling the traced/livepatch'ed
+ * function. The livepatch'ed function did not get a chance to tear down
+ * the livepatch stack it setup. Take care of that here in do_fexit.
+ */
+ EMIT(PPC_RAW_CMPD(_R12, _R25));
+ PPC_BCC_SHORT(COND_EQ, (ctx->idx + 3) * 4);
+ EMIT(PPC_RAW_LL(_R25, _R1, livepatch_sp_off));
+ EMIT(PPC_RAW_STL(_R12, _R25, 0));
+ }
+
for (i = 0; i < fexit->nr_links; i++)
if (invoke_bpf_prog(image, ro_image, ctx, fexit->links[i], regs_off, retval_off,
run_ctx_off, false)) {
--
2.51.0
^ permalink raw reply related
* Re: [PATCH v2 0/6] unwind, arm64: add sframe unwinder for kernel
From: Puranjay Mohan @ 2025-09-29 19:55 UTC (permalink / raw)
To: Song Liu
Cc: Dylan Hatch, Josh Poimboeuf, Steven Rostedt, Indu Bhagat,
Peter Zijlstra, Will Deacon, Catalin Marinas, Jiri Kosina,
Roman Gushchin, Weinan Liu, Mark Rutland, Ian Rogers,
linux-toolchains, linux-kernel, live-patching, joe.lawrence,
Puranjay Mohan
In-Reply-To: <CAPhsuW5zUEeM3DAw-3OVNS9KmM2vG9B1GaR9KEKS_KFQo-VG9Q@mail.gmail.com>
On Mon, Sep 29, 2025 at 9:46 PM Song Liu <song@kernel.org> wrote:
>
> On Thu, Sep 4, 2025 at 3:39 PM Dylan Hatch <dylanbhatch@google.com> wrote:
> >
> > This patchset implements a generic kernel sframe-based [1] unwinder.
> > The main goal is to support reliable stacktraces on arm64.
> >
> > On x86 orc unwinder provides reliable stacktraces. But arm64 misses the
> > required support from objtool: it cannot generate orc unwind tables for
> > arm64.
> >
> > Currently, there's already a sframe unwinder proposed for userspace: [2].
> > Since the sframe unwind table algorithm is similar, these two proposals
> > could integrate common functionality in the future.
> >
> > Currently, only GCC supports sframe.
> >
> > These patches are based on v6.17-rc4 and are available on github [3].
> >
> > Ref:
> > [1]: https://sourceware.org/binutils/docs/sframe-spec.html
> > [2]: https://lore.kernel.org/lkml/cover.1730150953.git.jpoimboe@kernel.org/
> > [3]: https://github.com/dylanbhatch/linux/tree/sframe-v2
>
> I run the following test on this sframe-v2 branch:
>
> bpftrace -e 'kprobe:security_file_open {printf("%s",
> kstack);@count+=1; if (@count > 1) {exit();}}'
>
> security_file_open+0
> bpf_prog_eaca355a0dcdca7f_kprobe_security_file_open_1+16641632@./bpftrace.bpf.o:0
> path_openat+1892
> do_filp_open+132
> do_open_execat+84
> alloc_bprm+44
> do_execveat_common.isra.0+116
> __arm64_sys_execve+72
> invoke_syscall+76
> el0_svc_common.constprop.0+68
> do_el0_svc+32
> el0_svc+56
> el0t_64_sync_handler+152
> el0t_64_sync+388
>
> This looks wrong. The right call trace should be:
>
> do_filp_open
> => path_openat
> => vfs_open
> => do_dentry_open
> => security_file_open
> => bpf_prog_eaca355a0dcdca7f_...
>
> I am not sure whether this is just a problem with the bpf program,
> or also with something else.
I will try to debug this more but am just curious about BPF's
interactions with sframe.
The sframe data for bpf programs doesn't exist, so we would need to
add that support
and that wouldn't be trivial, given the BPF programs are JITed.
Thanks,
Puranjay
^ permalink raw reply
* Re: [PATCH v2 0/6] unwind, arm64: add sframe unwinder for kernel
From: Song Liu @ 2025-09-29 19:46 UTC (permalink / raw)
To: Dylan Hatch
Cc: Josh Poimboeuf, Steven Rostedt, Indu Bhagat, Peter Zijlstra,
Will Deacon, Catalin Marinas, Jiri Kosina, Roman Gushchin,
Weinan Liu, Mark Rutland, Ian Rogers, linux-toolchains,
linux-kernel, live-patching, joe.lawrence, Puranjay Mohan
In-Reply-To: <20250904223850.884188-1-dylanbhatch@google.com>
On Thu, Sep 4, 2025 at 3:39 PM Dylan Hatch <dylanbhatch@google.com> wrote:
>
> This patchset implements a generic kernel sframe-based [1] unwinder.
> The main goal is to support reliable stacktraces on arm64.
>
> On x86 orc unwinder provides reliable stacktraces. But arm64 misses the
> required support from objtool: it cannot generate orc unwind tables for
> arm64.
>
> Currently, there's already a sframe unwinder proposed for userspace: [2].
> Since the sframe unwind table algorithm is similar, these two proposals
> could integrate common functionality in the future.
>
> Currently, only GCC supports sframe.
>
> These patches are based on v6.17-rc4 and are available on github [3].
>
> Ref:
> [1]: https://sourceware.org/binutils/docs/sframe-spec.html
> [2]: https://lore.kernel.org/lkml/cover.1730150953.git.jpoimboe@kernel.org/
> [3]: https://github.com/dylanbhatch/linux/tree/sframe-v2
I run the following test on this sframe-v2 branch:
bpftrace -e 'kprobe:security_file_open {printf("%s",
kstack);@count+=1; if (@count > 1) {exit();}}'
security_file_open+0
bpf_prog_eaca355a0dcdca7f_kprobe_security_file_open_1+16641632@./bpftrace.bpf.o:0
path_openat+1892
do_filp_open+132
do_open_execat+84
alloc_bprm+44
do_execveat_common.isra.0+116
__arm64_sys_execve+72
invoke_syscall+76
el0_svc_common.constprop.0+68
do_el0_svc+32
el0_svc+56
el0t_64_sync_handler+152
el0t_64_sync+388
This looks wrong. The right call trace should be:
do_filp_open
=> path_openat
=> vfs_open
=> do_dentry_open
=> security_file_open
=> bpf_prog_eaca355a0dcdca7f_...
I am not sure whether this is just a problem with the bpf program,
or also with something else.
Thanks,
Song
^ permalink raw reply
* Re: [PATCH v2 0/3] powerpc/ftrace: Fix livepatch module OOL ftrace corruption
From: Madhavan Srinivasan @ 2025-09-22 5:44 UTC (permalink / raw)
To: linuxppc-dev, live-patching, Joe Lawrence
Cc: Michael Ellerman, Nicholas Piggin, Christophe Leroy, Naveen N Rao
In-Reply-To: <20250912142740.3581368-1-joe.lawrence@redhat.com>
On Fri, 12 Sep 2025 10:27:37 -0400, Joe Lawrence wrote:
> This patch series fixes a couple of bugs in the powerpc64 out-of-line
> (OOL) ftrace support for modules, and follows up with a patch to
> simplify the module .stubs allocation code. An analysis of the module
> stub area corruption that prompted this work can be found in the v1
> thread [1].
>
> The first two patches fix bugs introduced by commit eec37961a56a
> ("powerpc64/ftrace: Move ftrace sequence out of line"). The first,
> suggested by Naveen, ensures that a NOP'd ftrace call site has its
> ftrace_ops record updated correctly. The second patch corrects a loop in
> setup_ftrace_ool_stubs() to ensure all required stubs are reserved, not
> just the first. Together, these bugs lead to potential corruption of the
> OOL ftrace stubs area for livepatch modules.
>
> [...]
Applied to powerpc/next.
[1/3] powerpc/ftrace: ensure ftrace record ops are always set for NOPs
https://git.kernel.org/powerpc/c/5337609a314828aa2474ac359db615f475c4a4d2
[2/3] powerpc64/modules: correctly iterate over stubs in setup_ftrace_ool_stubs
https://git.kernel.org/powerpc/c/f6b4df37ebfeb47e50e27780500d2d06b4d211bd
[3/3] powerpc64/modules: replace stub allocation sentinel with an explicit counter
https://git.kernel.org/powerpc/c/b137312fbf2dd1edc39acf7e8e6e8ac0a6ad72c0
Thanks
^ permalink raw reply
* Re: [PATCH v4 00/63] objtool,livepatch: klp-build livepatch module generation
From: Josh Poimboeuf @ 2025-09-18 16:32 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
On Wed, Sep 17, 2025 at 09:03:08AM -0700, Josh Poimboeuf wrote:
> Changes since v3 (https://lore.kernel.org/cover.1750980516.git.jpoimboe@kernel.org):
>
> - Get rid of the SHF_MERGE+SHF_WRITE toolchain shenanigans in favor of
> simple .discard.annotate_data annotations
> - Fix potential double free in elf_create_reloc()
> - Sync interval_tree_generic.h (Peter)
> - Refactor prefix symbol creation error handling
> - Rebase on tip/master and fix new issue (--checksum getting added with --noabs)
>
> (v3..v4 diff below)
git branch:
git://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git klp-build-v4
--
Josh
^ permalink raw reply
* [PATCH v4.1 12/63] interval_tree: Fix ITSTATIC usage for *_subtree_search()
From: Josh Poimboeuf @ 2025-09-18 16:30 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <1e0b4913420f199b3c6d2c536b849e5d79911d21.1758067942.git.jpoimboe@kernel.org>
For consistency with the other function templates, change
_subtree_search_*() to use the user-supplied ITSTATIC rather than the
hard-coded 'static'.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
v4.1: Fixed a couple more INTERVAL_TREE_DEFINE usages.
drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h | 4 ++++
include/linux/interval_tree.h | 4 ++++
include/linux/interval_tree_generic.h | 2 +-
include/linux/mm.h | 2 ++
lib/interval_tree.c | 1 +
tools/include/linux/interval_tree_generic.h | 2 +-
6 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h
index 1d7fc3226bcad..cfb42a8f5768b 100644
--- a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h
+++ b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h
@@ -53,6 +53,10 @@ extern void
usnic_uiom_interval_tree_remove(struct usnic_uiom_interval_node *node,
struct rb_root_cached *root);
extern struct usnic_uiom_interval_node *
+usnic_uiom_interval_tree_subtree_search(struct usnic_uiom_interval_node *node,
+ unsigned long start,
+ unsigned long last);
+extern struct usnic_uiom_interval_node *
usnic_uiom_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start,
unsigned long last);
diff --git a/include/linux/interval_tree.h b/include/linux/interval_tree.h
index 2b8026a399064..9d5791e9f737a 100644
--- a/include/linux/interval_tree.h
+++ b/include/linux/interval_tree.h
@@ -19,6 +19,10 @@ extern void
interval_tree_remove(struct interval_tree_node *node,
struct rb_root_cached *root);
+extern struct interval_tree_node *
+interval_tree_subtree_search(struct interval_tree_node *node,
+ unsigned long start, unsigned long last);
+
extern struct interval_tree_node *
interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);
diff --git a/include/linux/interval_tree_generic.h b/include/linux/interval_tree_generic.h
index 1b400f26f63d6..c5a2fed49eb0d 100644
--- a/include/linux/interval_tree_generic.h
+++ b/include/linux/interval_tree_generic.h
@@ -77,7 +77,7 @@ ITSTATIC void ITPREFIX ## _remove(ITSTRUCT *node, \
* Cond2: start <= ITLAST(node) \
*/ \
\
-static ITSTRUCT * \
+ITSTATIC ITSTRUCT * \
ITPREFIX ## _subtree_search(ITSTRUCT *node, ITTYPE start, ITTYPE last) \
{ \
while (true) { \
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 1ae97a0b8ec75..69baa9a1e2cb4 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3265,6 +3265,8 @@ void vma_interval_tree_insert_after(struct vm_area_struct *node,
struct rb_root_cached *root);
void vma_interval_tree_remove(struct vm_area_struct *node,
struct rb_root_cached *root);
+struct vm_area_struct *vma_interval_tree_subtree_search(struct vm_area_struct *node,
+ unsigned long start, unsigned long last);
struct vm_area_struct *vma_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);
struct vm_area_struct *vma_interval_tree_iter_next(struct vm_area_struct *node,
diff --git a/lib/interval_tree.c b/lib/interval_tree.c
index 324766e9bf637..9ceb084b6b4ef 100644
--- a/lib/interval_tree.c
+++ b/lib/interval_tree.c
@@ -13,6 +13,7 @@ INTERVAL_TREE_DEFINE(struct interval_tree_node, rb,
EXPORT_SYMBOL_GPL(interval_tree_insert);
EXPORT_SYMBOL_GPL(interval_tree_remove);
+EXPORT_SYMBOL_GPL(interval_tree_subtree_search);
EXPORT_SYMBOL_GPL(interval_tree_iter_first);
EXPORT_SYMBOL_GPL(interval_tree_iter_next);
diff --git a/tools/include/linux/interval_tree_generic.h b/tools/include/linux/interval_tree_generic.h
index 1b400f26f63d6..c5a2fed49eb0d 100644
--- a/tools/include/linux/interval_tree_generic.h
+++ b/tools/include/linux/interval_tree_generic.h
@@ -77,7 +77,7 @@ ITSTATIC void ITPREFIX ## _remove(ITSTRUCT *node, \
* Cond2: start <= ITLAST(node) \
*/ \
\
-static ITSTRUCT * \
+ITSTATIC ITSTRUCT * \
ITPREFIX ## _subtree_search(ITSTRUCT *node, ITTYPE start, ITTYPE last) \
{ \
while (true) { \
--
2.50.0
^ permalink raw reply related
* Re: [PATCH v2] LoongArch: Fix unreliable stack for live patching
From: Huacai Chen @ 2025-09-18 9:14 UTC (permalink / raw)
To: Tiezhu Yang; +Cc: Xi Zhang, live-patching, loongarch, linux-kernel
In-Reply-To: <20250916093509.17306-1-yangtiezhu@loongson.cn>
Applied, thanks.
Huacai
On Tue, Sep 16, 2025 at 5:35 PM Tiezhu Yang <yangtiezhu@loongson.cn> wrote:
>
> When testing the kernel live patching with "modprobe livepatch-sample",
> there is a timeout over 15 seconds from "starting patching transition"
> to "patching complete", dmesg shows "unreliable stack" for user tasks
> in debug mode, here is one of the messages:
>
> livepatch: klp_try_switch_task: bash:1193 has an unreliable stack
>
> The "unreliable stack" is because it can not unwind from do_syscall()
> to its previous frame handle_syscall(), it should use fp to find the
> original stack top due to secondary stack in do_syscall(), but fp is
> not used for some other functions, then fp can not be restored by the
> next frame of do_syscall(), so it is necessary to save fp if task is
> not current to get the stack top of do_syscall().
>
> Here are the call chains:
>
> klp_enable_patch()
> klp_try_complete_transition()
> klp_try_switch_task()
> klp_check_and_switch_task()
> klp_check_stack()
> stack_trace_save_tsk_reliable()
> arch_stack_walk_reliable()
>
> When executing "rmmod livepatch-sample", there exists the similar issue.
> With this patch, it takes a short time for patching and unpatching.
>
> Before:
>
> # modprobe livepatch-sample
> # dmesg -T | tail -3
> [Sat Sep 6 11:00:20 2025] livepatch: 'livepatch_sample': starting patching transition
> [Sat Sep 6 11:00:35 2025] livepatch: signaling remaining tasks
> [Sat Sep 6 11:00:36 2025] livepatch: 'livepatch_sample': patching complete
>
> # echo 0 > /sys/kernel/livepatch/livepatch_sample/enabled
> # rmmod livepatch_sample
> rmmod: ERROR: Module livepatch_sample is in use
> # rmmod livepatch_sample
> # dmesg -T | tail -3
> [Sat Sep 6 11:06:05 2025] livepatch: 'livepatch_sample': starting unpatching transition
> [Sat Sep 6 11:06:20 2025] livepatch: signaling remaining tasks
> [Sat Sep 6 11:06:21 2025] livepatch: 'livepatch_sample': unpatching complete
>
> After:
>
> # modprobe livepatch-sample
> # dmesg -T | tail -2
> [Tue Sep 16 16:19:30 2025] livepatch: 'livepatch_sample': starting patching transition
> [Tue Sep 16 16:19:31 2025] livepatch: 'livepatch_sample': patching complete
>
> # echo 0 > /sys/kernel/livepatch/livepatch_sample/enabled
> # rmmod livepatch_sample
> # dmesg -T | tail -2
> [Tue Sep 16 16:19:36 2025] livepatch: 'livepatch_sample': starting unpatching transition
> [Tue Sep 16 16:19:37 2025] livepatch: 'livepatch_sample': unpatching complete
>
> Cc: stable@vger.kernel.org # v6.9+
> Fixes: 199cc14cb4f1 ("LoongArch: Add kernel livepatching support")
> Reported-by: Xi Zhang <zhangxi@kylinos.cn>
> Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
> ---
> arch/loongarch/kernel/stacktrace.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/loongarch/kernel/stacktrace.c b/arch/loongarch/kernel/stacktrace.c
> index 9a038d1070d7..387dc4d3c486 100644
> --- a/arch/loongarch/kernel/stacktrace.c
> +++ b/arch/loongarch/kernel/stacktrace.c
> @@ -51,12 +51,13 @@ int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry,
> if (task == current) {
> regs->regs[3] = (unsigned long)__builtin_frame_address(0);
> regs->csr_era = (unsigned long)__builtin_return_address(0);
> + regs->regs[22] = 0;
> } else {
> regs->regs[3] = thread_saved_fp(task);
> regs->csr_era = thread_saved_ra(task);
> + regs->regs[22] = task->thread.reg22;
> }
> regs->regs[1] = 0;
> - regs->regs[22] = 0;
>
> for (unwind_start(&state, task, regs);
> !unwind_done(&state) && !unwind_error(&state); unwind_next_frame(&state)) {
> --
> 2.42.0
>
^ permalink raw reply
* Re: [PATCH v2 6/6] unwind: arm64: Add reliable stacktrace with sframe unwinder.
From: Josh Poimboeuf @ 2025-09-17 23:41 UTC (permalink / raw)
To: Dylan Hatch
Cc: Steven Rostedt, Indu Bhagat, Peter Zijlstra, Will Deacon,
Catalin Marinas, Jiri Kosina, Roman Gushchin, Weinan Liu,
Mark Rutland, Ian Rogers, linux-toolchains, linux-kernel,
live-patching, joe.lawrence, Puranjay Mohan, Song Liu,
Prasanna Kumar T S M
In-Reply-To: <20250904223850.884188-7-dylanbhatch@google.com>
On Thu, Sep 04, 2025 at 10:38:50PM +0000, Dylan Hatch wrote:
> +noinline notrace int arch_stack_walk_reliable(
> + stack_trace_consume_fn consume_entry,
> + void *cookie, struct task_struct *task)
> +{
> + struct kunwind_reliable_consume_entry_data data = {
> + .consume_entry = consume_entry,
> + .cookie = cookie,
> + .unreliable = false,
> + };
> +
> + kunwind_stack_walk(arch_kunwind_reliable_consume_entry, &data, task, NULL);
> +
> + if (data.unreliable)
> + return -EINVAL;
As far I can tell, the *only* error condition being checked is if it
(successfully) fell back to frame pointers.
What if there was some bad or missing sframe data? Or some unexpected
condition on the stack?
Also, does the exception handling code have correct cfi/sframe metadata?
In order for it to be "reliable", we need to know the unwind reached the
end of the stack (e.g., the task pt_regs frame, from entry-from-user).
--
Josh
^ permalink raw reply
* [PATCH v4 63/63] livepatch: Introduce source code helpers for livepatch modules
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add some helper macros which can be used by livepatch source .patch
files to register callbacks, convert static calls to regular calls where
needed, and patch syscalls.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
include/linux/livepatch_helpers.h | 77 +++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
create mode 100644 include/linux/livepatch_helpers.h
diff --git a/include/linux/livepatch_helpers.h b/include/linux/livepatch_helpers.h
new file mode 100644
index 0000000000000..99d68d0773fa8
--- /dev/null
+++ b/include/linux/livepatch_helpers.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_LIVEPATCH_HELPERS_H
+#define _LINUX_LIVEPATCH_HELPERS_H
+
+/*
+ * Interfaces for use by livepatch patches
+ */
+
+#include <linux/syscalls.h>
+#include <linux/livepatch.h>
+
+#ifdef MODULE
+#define KLP_OBJNAME __KBUILD_MODNAME
+#else
+#define KLP_OBJNAME vmlinux
+#endif
+
+/* Livepatch callback registration */
+
+#define KLP_CALLBACK_PTRS ".discard.klp_callback_ptrs"
+
+#define KLP_PRE_PATCH_CALLBACK(func) \
+ klp_pre_patch_t __used __section(KLP_CALLBACK_PTRS) \
+ __PASTE(__KLP_PRE_PATCH_PREFIX, KLP_OBJNAME) = func
+
+#define KLP_POST_PATCH_CALLBACK(func) \
+ klp_post_patch_t __used __section(KLP_CALLBACK_PTRS) \
+ __PASTE(__KLP_POST_PATCH_PREFIX, KLP_OBJNAME) = func
+
+#define KLP_PRE_UNPATCH_CALLBACK(func) \
+ klp_pre_unpatch_t __used __section(KLP_CALLBACK_PTRS) \
+ __PASTE(__KLP_PRE_UNPATCH_PREFIX, KLP_OBJNAME) = func
+
+#define KLP_POST_UNPATCH_CALLBACK(func) \
+ klp_post_unpatch_t __used __section(KLP_CALLBACK_PTRS) \
+ __PASTE(__KLP_POST_UNPATCH_PREFIX, KLP_OBJNAME) = func
+
+/*
+ * Replace static_call() usage with this macro when create-diff-object
+ * recommends it due to the original static call key living in a module.
+ *
+ * This converts the static call to a regular indirect call.
+ */
+#define KLP_STATIC_CALL(name) \
+ ((typeof(STATIC_CALL_TRAMP(name))*)(STATIC_CALL_KEY(name).func))
+
+/* Syscall patching */
+
+#define KLP_SYSCALL_DEFINE1(name, ...) KLP_SYSCALL_DEFINEx(1, _##name, __VA_ARGS__)
+#define KLP_SYSCALL_DEFINE2(name, ...) KLP_SYSCALL_DEFINEx(2, _##name, __VA_ARGS__)
+#define KLP_SYSCALL_DEFINE3(name, ...) KLP_SYSCALL_DEFINEx(3, _##name, __VA_ARGS__)
+#define KLP_SYSCALL_DEFINE4(name, ...) KLP_SYSCALL_DEFINEx(4, _##name, __VA_ARGS__)
+#define KLP_SYSCALL_DEFINE5(name, ...) KLP_SYSCALL_DEFINEx(5, _##name, __VA_ARGS__)
+#define KLP_SYSCALL_DEFINE6(name, ...) KLP_SYSCALL_DEFINEx(6, _##name, __VA_ARGS__)
+
+#define KLP_SYSCALL_DEFINEx(x, sname, ...) \
+ __KLP_SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
+
+#ifdef CONFIG_X86_64
+// TODO move this to arch/x86/include/asm/syscall_wrapper.h and share code
+#define __KLP_SYSCALL_DEFINEx(x, name, ...) \
+ static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)); \
+ static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\
+ __X64_SYS_STUBx(x, name, __VA_ARGS__) \
+ __IA32_SYS_STUBx(x, name, __VA_ARGS__) \
+ static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)) \
+ { \
+ long ret = __klp_do_sys##name(__MAP(x,__SC_CAST,__VA_ARGS__));\
+ __MAP(x,__SC_TEST,__VA_ARGS__); \
+ __PROTECT(x, ret,__MAP(x,__SC_ARGS,__VA_ARGS__)); \
+ return ret; \
+ } \
+ static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__))
+
+#endif
+
+#endif /* _LINUX_LIVEPATCH_HELPERS_H */
--
2.50.0
^ permalink raw reply related
* [PATCH v4 62/63] livepatch/klp-build: Add --show-first-changed option to show function divergence
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a --show-first-changed option to identify where changed functions
begin to diverge:
- Parse 'objtool klp diff' output to find changed functions.
- Run objtool again on each object with --debug-checksum=<funcs>.
- Diff the per-instruction checksum debug output to locate the first
differing instruction.
This can be useful for quickly determining where and why a function
changed.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/livepatch/klp-build | 82 +++++++++++++++++++++++++++++++++++--
1 file changed, 78 insertions(+), 4 deletions(-)
diff --git a/scripts/livepatch/klp-build b/scripts/livepatch/klp-build
index 28ee259ce5f6e..881e052e7faef 100755
--- a/scripts/livepatch/klp-build
+++ b/scripts/livepatch/klp-build
@@ -20,7 +20,7 @@ set -o nounset
# This helps keep execution in pipes so pipefail+errexit can catch errors.
shopt -s lastpipe
-unset DEBUG_CLONE SKIP_CLEANUP XTRACE
+unset DEBUG_CLONE DIFF_CHECKSUM SKIP_CLEANUP XTRACE
REPLACE=1
SHORT_CIRCUIT=0
@@ -114,6 +114,7 @@ Usage: $SCRIPT [OPTIONS] PATCH_FILE(s)
Generate a livepatch module.
Options:
+ -f, --show-first-changed Show address of first changed instruction
-j, --jobs=<jobs> Build jobs to run simultaneously [default: $JOBS]
-o, --output=<file.ko> Output file [default: livepatch-<patch-name>.ko]
--no-replace Disable livepatch atomic replace
@@ -141,8 +142,8 @@ process_args() {
local long
local args
- short="hj:o:vdS:T"
- long="help,jobs:,output:,no-replace,verbose,debug,short-circuit:,keep-tmp"
+ short="hfj:o:vdS:T"
+ long="help,show-first-changed,jobs:,output:,no-replace,verbose,debug,short-circuit:,keep-tmp"
args=$(getopt --options "$short" --longoptions "$long" -- "$@") || {
echo; usage; exit
@@ -155,6 +156,10 @@ process_args() {
usage
exit 0
;;
+ -f | --show-first-changed)
+ DIFF_CHECKSUM=1
+ shift
+ ;;
-j | --jobs)
JOBS="$2"
shift 2
@@ -618,6 +623,7 @@ diff_objects() {
local orig_file="$rel_file"
local patched_file="$PATCHED_DIR/$rel_file"
local out_file="$DIFF_DIR/$rel_file"
+ local filter=()
local cmd=()
mkdir -p "$(dirname "$out_file")"
@@ -630,16 +636,80 @@ diff_objects() {
cmd+=("$patched_file")
cmd+=("$out_file")
+ if [[ -v DIFF_CHECKSUM ]]; then
+ filter=("grep0")
+ filter+=("-Ev")
+ filter+=("DEBUG: .*checksum: ")
+ else
+ filter=("cat")
+ fi
+
(
cd "$ORIG_DIR"
"${cmd[@]}" \
1> >(tee -a "$log") \
- 2> >(tee -a "$log" >&2) || \
+ 2> >(tee -a "$log" | "${filter[@]}" >&2) || \
die "objtool klp diff failed"
)
done
}
+# For each changed object, run objtool with --debug-checksum to get the
+# per-instruction checksums, and then diff those to find the first changed
+# instruction for each function.
+diff_checksums() {
+ local orig_log="$ORIG_DIR/checksum.log"
+ local patched_log="$PATCHED_DIR/checksum.log"
+ local -A funcs
+ local cmd=()
+ local line
+ local file
+ local func
+
+ gawk '/\.o: changed function: / {
+ sub(/:$/, "", $1)
+ print $1, $NF
+ }' "$KLP_DIFF_LOG" | mapfile -t lines
+
+ for line in "${lines[@]}"; do
+ read -r file func <<< "$line"
+ if [[ ! -v funcs["$file"] ]]; then
+ funcs["$file"]="$func"
+ else
+ funcs["$file"]+=" $func"
+ fi
+ done
+
+ cmd=("$SRC/tools/objtool/objtool")
+ cmd+=("--checksum")
+ cmd+=("--link")
+ cmd+=("--dry-run")
+
+ for file in "${!funcs[@]}"; do
+ local opt="--debug-checksum=${funcs[$file]// /,}"
+
+ (
+ cd "$ORIG_DIR"
+ "${cmd[@]}" "$opt" "$file" &> "$orig_log" || \
+ ( cat "$orig_log" >&2; die "objtool --debug-checksum failed" )
+
+ cd "$PATCHED_DIR"
+ "${cmd[@]}" "$opt" "$file" &> "$patched_log" || \
+ ( cat "$patched_log" >&2; die "objtool --debug-checksum failed" )
+ )
+
+ for func in ${funcs[$file]}; do
+ diff <( grep0 -E "^DEBUG: .*checksum: $func " "$orig_log" | sed "s|$ORIG_DIR/||") \
+ <( grep0 -E "^DEBUG: .*checksum: $func " "$patched_log" | sed "s|$PATCHED_DIR/||") \
+ | gawk '/^< DEBUG: / {
+ gsub(/:/, "")
+ printf "%s: %s: %s\n", $3, $5, $6
+ exit
+ }' || true
+ done
+ done
+}
+
# Build and post-process livepatch module in $KMOD_DIR
build_patch_module() {
local makefile="$KMOD_DIR/Kbuild"
@@ -743,6 +813,10 @@ fi
if (( SHORT_CIRCUIT <= 3 )); then
status "Diffing objects"
diff_objects
+ if [[ -v DIFF_CHECKSUM ]]; then
+ status "Finding first changed instructions"
+ diff_checksums
+ fi
fi
if (( SHORT_CIRCUIT <= 4 )); then
--
2.50.0
^ permalink raw reply related
* [PATCH v4 61/63] livepatch/klp-build: Add --debug option to show cloning decisions
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a --debug option which gets passed to "objtool klp diff" to enable
debug output related to cloning decisions.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/livepatch/klp-build | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/scripts/livepatch/klp-build b/scripts/livepatch/klp-build
index 01ed0b66bfaff..28ee259ce5f6e 100755
--- a/scripts/livepatch/klp-build
+++ b/scripts/livepatch/klp-build
@@ -20,7 +20,7 @@ set -o nounset
# This helps keep execution in pipes so pipefail+errexit can catch errors.
shopt -s lastpipe
-unset SKIP_CLEANUP XTRACE
+unset DEBUG_CLONE SKIP_CLEANUP XTRACE
REPLACE=1
SHORT_CIRCUIT=0
@@ -120,6 +120,7 @@ Options:
-v, --verbose Pass V=1 to kernel/module builds
Advanced Options:
+ -d, --debug Show symbol/reloc cloning decisions
-S, --short-circuit=STEP Start at build step (requires prior --keep-tmp)
1|orig Build original kernel (default)
2|patched Build patched kernel
@@ -140,8 +141,8 @@ process_args() {
local long
local args
- short="hj:o:vS:T"
- long="help,jobs:,output:,no-replace,verbose,short-circuit:,keep-tmp"
+ short="hj:o:vdS:T"
+ long="help,jobs:,output:,no-replace,verbose,debug,short-circuit:,keep-tmp"
args=$(getopt --options "$short" --longoptions "$long" -- "$@") || {
echo; usage; exit
@@ -174,6 +175,11 @@ process_args() {
VERBOSE="V=1"
shift
;;
+ -d | --debug)
+ DEBUG_CLONE=1
+ keep_tmp=1
+ shift
+ ;;
-S | --short-circuit)
[[ ! -d "$TMP_DIR" ]] && die "--short-circuit requires preserved klp-tmp dir"
keep_tmp=1
@@ -596,6 +602,7 @@ copy_patched_objects() {
diff_objects() {
local log="$KLP_DIFF_LOG"
local files=()
+ local opts=()
rm -rf "$DIFF_DIR"
mkdir -p "$DIFF_DIR"
@@ -603,6 +610,8 @@ diff_objects() {
find "$PATCHED_DIR" -type f -name "*.o" | mapfile -t files
[[ ${#files[@]} -eq 0 ]] && die "no changes detected"
+ [[ -v DEBUG_CLONE ]] && opts=("--debug")
+
# Diff all changed objects
for file in "${files[@]}"; do
local rel_file="${file#"$PATCHED_DIR"/}"
@@ -616,6 +625,7 @@ diff_objects() {
cmd=("$SRC/tools/objtool/objtool")
cmd+=("klp")
cmd+=("diff")
+ (( ${#opts[@]} > 0 )) && cmd+=("${opts[@]}")
cmd+=("$orig_file")
cmd+=("$patched_file")
cmd+=("$out_file")
--
2.50.0
^ permalink raw reply related
* [PATCH v4 60/63] livepatch/klp-build: Introduce klp-build script for generating livepatch modules
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a klp-build script which automates the generation of a livepatch
module from a source .patch file by performing the following steps:
- Builds an original kernel with -function-sections and
-fdata-sections, plus objtool function checksumming.
- Applies the .patch file and rebuilds the kernel using the same
options.
- Runs 'objtool klp diff' to detect changed functions and generate
intermediate binary diff objects.
- Builds a kernel module which links the diff objects with some
livepatch module init code (scripts/livepatch/init.c).
- Finalizes the livepatch module (aka work around linker wreckage)
using 'objtool klp post-link'.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/Makefile.lib | 1 +
scripts/livepatch/fix-patch-lines | 2 +-
scripts/livepatch/klp-build | 743 ++++++++++++++++++++++++++++++
tools/objtool/klp-diff.c | 6 +-
4 files changed, 749 insertions(+), 3 deletions(-)
create mode 100755 scripts/livepatch/klp-build
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 28a1c08e3b221..f4b33919ec371 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -173,6 +173,7 @@ ifdef CONFIG_OBJTOOL
objtool := $(objtree)/tools/objtool/objtool
+objtool-args-$(CONFIG_KLP_BUILD) += --checksum
objtool-args-$(CONFIG_HAVE_JUMP_LABEL_HACK) += --hacks=jump_label
objtool-args-$(CONFIG_HAVE_NOINSTR_HACK) += --hacks=noinstr
objtool-args-$(CONFIG_MITIGATION_CALL_DEPTH_TRACKING) += --hacks=skylake
diff --git a/scripts/livepatch/fix-patch-lines b/scripts/livepatch/fix-patch-lines
index 73c5e3dea46e1..fa7d4f6592e6b 100755
--- a/scripts/livepatch/fix-patch-lines
+++ b/scripts/livepatch/fix-patch-lines
@@ -23,7 +23,7 @@ BEGIN {
in_hunk = 1
- # for @@ -1,3 +1,4 @@:
+ # @@ -1,3 +1,4 @@:
# 1: line number in old file
# 3: how many lines the hunk covers in old file
# 1: line number in new file
diff --git a/scripts/livepatch/klp-build b/scripts/livepatch/klp-build
new file mode 100755
index 0000000000000..01ed0b66bfaff
--- /dev/null
+++ b/scripts/livepatch/klp-build
@@ -0,0 +1,743 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Build a livepatch module
+
+# shellcheck disable=SC1090,SC2155
+
+if (( BASH_VERSINFO[0] < 4 || \
+ (BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] < 4) )); then
+ echo "error: this script requires bash 4.4+" >&2
+ exit 1
+fi
+
+set -o errexit
+set -o errtrace
+set -o pipefail
+set -o nounset
+
+# Allow doing 'cmd | mapfile -t array' instead of 'mapfile -t array < <(cmd)'.
+# This helps keep execution in pipes so pipefail+errexit can catch errors.
+shopt -s lastpipe
+
+unset SKIP_CLEANUP XTRACE
+
+REPLACE=1
+SHORT_CIRCUIT=0
+JOBS="$(getconf _NPROCESSORS_ONLN)"
+VERBOSE="-s"
+shopt -o xtrace | grep -q 'on' && XTRACE=1
+
+# Avoid removing the previous $TMP_DIR until args have been fully processed.
+KEEP_TMP=1
+
+SCRIPT="$(basename "$0")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+FIX_PATCH_LINES="$SCRIPT_DIR/fix-patch-lines"
+
+SRC="$(pwd)"
+OBJ="$(pwd)"
+
+CONFIG="$OBJ/.config"
+TMP_DIR="$OBJ/klp-tmp"
+
+ORIG_DIR="$TMP_DIR/orig"
+PATCHED_DIR="$TMP_DIR/patched"
+DIFF_DIR="$TMP_DIR/diff"
+KMOD_DIR="$TMP_DIR/kmod"
+
+STASH_DIR="$TMP_DIR/stash"
+TIMESTAMP="$TMP_DIR/timestamp"
+PATCH_TMP_DIR="$TMP_DIR/tmp"
+
+KLP_DIFF_LOG="$DIFF_DIR/diff.log"
+
+grep0() {
+ command grep "$@" || true
+}
+
+status() {
+ echo "$*"
+}
+
+warn() {
+ echo "error: $SCRIPT: $*" >&2
+}
+
+die() {
+ warn "$@"
+ exit 1
+}
+
+declare -a STASHED_FILES
+
+stash_file() {
+ local file="$1"
+ local rel_file="${file#"$SRC"/}"
+
+ [[ ! -e "$file" ]] && die "no file to stash: $file"
+
+ mkdir -p "$STASH_DIR/$(dirname "$rel_file")"
+ cp -f "$file" "$STASH_DIR/$rel_file"
+
+ STASHED_FILES+=("$rel_file")
+}
+
+restore_files() {
+ local file
+
+ for file in "${STASHED_FILES[@]}"; do
+ mv -f "$STASH_DIR/$file" "$SRC/$file" || warn "can't restore file: $file"
+ done
+
+ STASHED_FILES=()
+}
+
+cleanup() {
+ set +o nounset
+ revert_patches "--recount"
+ restore_files
+ [[ "$KEEP_TMP" -eq 0 ]] && rm -rf "$TMP_DIR"
+ return 0
+}
+
+trap_err() {
+ warn "line ${BASH_LINENO[0]}: '$BASH_COMMAND'"
+}
+
+trap cleanup EXIT INT TERM HUP
+trap trap_err ERR
+
+__usage() {
+ cat <<EOF
+Usage: $SCRIPT [OPTIONS] PATCH_FILE(s)
+Generate a livepatch module.
+
+Options:
+ -j, --jobs=<jobs> Build jobs to run simultaneously [default: $JOBS]
+ -o, --output=<file.ko> Output file [default: livepatch-<patch-name>.ko]
+ --no-replace Disable livepatch atomic replace
+ -v, --verbose Pass V=1 to kernel/module builds
+
+Advanced Options:
+ -S, --short-circuit=STEP Start at build step (requires prior --keep-tmp)
+ 1|orig Build original kernel (default)
+ 2|patched Build patched kernel
+ 3|diff Diff objects
+ 4|kmod Build patch module
+ -T, --keep-tmp Preserve tmp dir on exit
+
+EOF
+}
+
+usage() {
+ __usage >&2
+}
+
+process_args() {
+ local keep_tmp=0
+ local short
+ local long
+ local args
+
+ short="hj:o:vS:T"
+ long="help,jobs:,output:,no-replace,verbose,short-circuit:,keep-tmp"
+
+ args=$(getopt --options "$short" --longoptions "$long" -- "$@") || {
+ echo; usage; exit
+ }
+ eval set -- "$args"
+
+ while true; do
+ case "$1" in
+ -h | --help)
+ usage
+ exit 0
+ ;;
+ -j | --jobs)
+ JOBS="$2"
+ shift 2
+ ;;
+ -o | --output)
+ [[ "$2" != *.ko ]] && die "output filename should end with .ko"
+ OUTFILE="$2"
+ NAME="$(basename "$OUTFILE")"
+ NAME="${NAME%.ko}"
+ NAME="$(module_name_string "$NAME")"
+ shift 2
+ ;;
+ --no-replace)
+ REPLACE=0
+ shift
+ ;;
+ -v | --verbose)
+ VERBOSE="V=1"
+ shift
+ ;;
+ -S | --short-circuit)
+ [[ ! -d "$TMP_DIR" ]] && die "--short-circuit requires preserved klp-tmp dir"
+ keep_tmp=1
+ case "$2" in
+ 1 | orig) SHORT_CIRCUIT=1; ;;
+ 2 | patched) SHORT_CIRCUIT=2; ;;
+ 3 | diff) SHORT_CIRCUIT=3; ;;
+ 4 | mod) SHORT_CIRCUIT=4; ;;
+ *) die "invalid short-circuit step '$2'" ;;
+ esac
+ shift 2
+ ;;
+ -T | --keep-tmp)
+ keep_tmp=1
+ shift
+ ;;
+ --)
+ shift
+ break
+ ;;
+ *)
+ usage
+ exit 1
+ ;;
+ esac
+ done
+
+ if [[ $# -eq 0 ]]; then
+ usage
+ exit 1
+ fi
+
+ KEEP_TMP="$keep_tmp"
+ PATCHES=("$@")
+}
+
+# temporarily disable xtrace for especially verbose code
+xtrace_save() {
+ [[ -v XTRACE ]] && set +x
+ return 0
+}
+
+xtrace_restore() {
+ [[ -v XTRACE ]] && set -x
+ return 0
+}
+
+validate_config() {
+ xtrace_save "reading .config"
+ source "$CONFIG" || die "no .config file in $(dirname "$CONFIG")"
+ xtrace_restore
+
+ [[ -v CONFIG_LIVEPATCH ]] || \
+ die "CONFIG_LIVEPATCH not enabled"
+
+ [[ -v CONFIG_KLP_BUILD ]] || \
+ die "CONFIG_KLP_BUILD not enabled"
+
+ [[ -v CONFIG_GCC_PLUGIN_LATENT_ENTROPY ]] && \
+ die "kernel option 'CONFIG_GCC_PLUGIN_LATENT_ENTROPY' not supported"
+
+ [[ -v CONFIG_GCC_PLUGIN_RANDSTRUCT ]] && \
+ die "kernel option 'CONFIG_GCC_PLUGIN_RANDSTRUCT' not supported"
+
+ return 0
+}
+
+# Only allow alphanumerics and '_' and '-' in the module name. Everything else
+# is replaced with '-'. Also truncate to 55 chars so the full name + NUL
+# terminator fits in the kernel's 56-byte module name array.
+module_name_string() {
+ echo "${1//[^a-zA-Z0-9_-]/-}" | cut -c 1-55
+}
+
+# If the module name wasn't specified on the cmdline with --output, give it a
+# name based on the patch name.
+set_module_name() {
+ [[ -v NAME ]] && return 0
+
+ if [[ "${#PATCHES[@]}" -eq 1 ]]; then
+ NAME="$(basename "${PATCHES[0]}")"
+ NAME="${NAME%.*}"
+ else
+ NAME="patch"
+ fi
+
+ NAME="livepatch-$NAME"
+ NAME="$(module_name_string "$NAME")"
+
+ OUTFILE="$NAME.ko"
+}
+
+# Hardcode the value printed by the localversion script to prevent patch
+# application from appending it with '+' due to a dirty git working tree.
+set_kernelversion() {
+ local file="$SRC/scripts/setlocalversion"
+ local localversion
+
+ stash_file "$file"
+
+ localversion="$(cd "$SRC" && make --no-print-directory kernelversion)"
+ localversion="$(cd "$SRC" && KERNELVERSION="$localversion" ./scripts/setlocalversion)"
+ [[ -z "$localversion" ]] && die "setlocalversion failed"
+
+ sed -i "2i echo $localversion; exit 0" scripts/setlocalversion
+}
+
+get_patch_files() {
+ local patch="$1"
+
+ grep0 -E '^(--- |\+\+\+ )' "$patch" \
+ | gawk '{print $2}' \
+ | sed 's|^[^/]*/||' \
+ | sort -u
+}
+
+# Make sure git re-stats the changed files
+git_refresh() {
+ local patch="$1"
+ local files=()
+
+ [[ ! -e "$SRC/.git" ]] && return
+
+ get_patch_files "$patch" | mapfile -t files
+
+ (
+ cd "$SRC"
+ git update-index -q --refresh -- "${files[@]}"
+ )
+}
+
+check_unsupported_patches() {
+ local patch
+
+ for patch in "${PATCHES[@]}"; do
+ local files=()
+
+ get_patch_files "$patch" | mapfile -t files
+
+ for file in "${files[@]}"; do
+ case "$file" in
+ lib/*|*.S)
+ die "unsupported patch to $file"
+ ;;
+ esac
+ done
+ done
+}
+
+apply_patch() {
+ local patch="$1"
+ shift
+ local extra_args=("$@")
+
+ [[ ! -f "$patch" ]] && die "$patch doesn't exist"
+
+ (
+ cd "$SRC"
+
+ # The sed strips the version signature from 'git format-patch',
+ # otherwise 'git apply --recount' warns.
+ sed -n '/^-- /q;p' "$patch" |
+ git apply "${extra_args[@]}"
+ )
+
+ APPLIED_PATCHES+=("$patch")
+}
+
+revert_patch() {
+ local patch="$1"
+ shift
+ local extra_args=("$@")
+ local tmp=()
+
+ (
+ cd "$SRC"
+
+ sed -n '/^-- /q;p' "$patch" |
+ git apply --reverse "${extra_args[@]}"
+ )
+ git_refresh "$patch"
+
+ for p in "${APPLIED_PATCHES[@]}"; do
+ [[ "$p" == "$patch" ]] && continue
+ tmp+=("$p")
+ done
+
+ APPLIED_PATCHES=("${tmp[@]}")
+}
+
+apply_patches() {
+ local patch
+
+ for patch in "${PATCHES[@]}"; do
+ apply_patch "$patch"
+ done
+}
+
+revert_patches() {
+ local extra_args=("$@")
+ local patches=("${APPLIED_PATCHES[@]}")
+
+ for (( i=${#patches[@]}-1 ; i>=0 ; i-- )) ; do
+ revert_patch "${patches[$i]}" "${extra_args[@]}"
+ done
+
+ APPLIED_PATCHES=()
+}
+
+validate_patches() {
+ check_unsupported_patches
+ apply_patches
+ revert_patches
+}
+
+do_init() {
+ # We're not yet smart enough to handle anything other than in-tree
+ # builds in pwd.
+ [[ ! "$SRC" -ef "$SCRIPT_DIR/../.." ]] && die "please run from the kernel root directory"
+ [[ ! "$OBJ" -ef "$SCRIPT_DIR/../.." ]] && die "please run from the kernel root directory"
+
+ (( SHORT_CIRCUIT <= 1 )) && rm -rf "$TMP_DIR"
+ mkdir -p "$TMP_DIR"
+
+ APPLIED_PATCHES=()
+
+ [[ -x "$FIX_PATCH_LINES" ]] || die "can't find fix-patch-lines"
+
+ validate_config
+ set_module_name
+ set_kernelversion
+}
+
+# Refresh the patch hunk headers, specifically the line numbers and counts.
+refresh_patch() {
+ local patch="$1"
+ local tmpdir="$PATCH_TMP_DIR"
+ local files=()
+
+ rm -rf "$tmpdir"
+ mkdir -p "$tmpdir/a"
+ mkdir -p "$tmpdir/b"
+
+ # Get all source files affected by the patch
+ get_patch_files "$patch" | mapfile -t files
+
+ # Copy orig source files to 'a'
+ ( cd "$SRC" && echo "${files[@]}" | xargs cp --parents --target-directory="$tmpdir/a" )
+
+ # Copy patched source files to 'b'
+ apply_patch "$patch" --recount
+ ( cd "$SRC" && echo "${files[@]}" | xargs cp --parents --target-directory="$tmpdir/b" )
+ revert_patch "$patch" --recount
+
+ # Diff 'a' and 'b' to make a clean patch
+ ( cd "$tmpdir" && git diff --no-index --no-prefix a b > "$patch" ) || true
+}
+
+# Copy the patches to a temporary directory, fix their lines so as not to
+# affect the __LINE__ macro for otherwise unchanged functions further down the
+# file, and update $PATCHES to point to the fixed patches.
+fix_patches() {
+ local idx
+ local i
+
+ rm -f "$TMP_DIR"/*.patch
+
+ idx=0001
+ for i in "${!PATCHES[@]}"; do
+ local old_patch="${PATCHES[$i]}"
+ local tmp_patch="$TMP_DIR/tmp.patch"
+ local patch="${PATCHES[$i]}"
+ local new_patch
+
+ new_patch="$TMP_DIR/$idx-fixed-$(basename "$patch")"
+
+ cp -f "$old_patch" "$tmp_patch"
+ refresh_patch "$tmp_patch"
+ "$FIX_PATCH_LINES" "$tmp_patch" > "$new_patch"
+ refresh_patch "$new_patch"
+
+ PATCHES[i]="$new_patch"
+
+ rm -f "$tmp_patch"
+ idx=$(printf "%04d" $(( 10#$idx + 1 )))
+ done
+}
+
+clean_kernel() {
+ local cmd=()
+
+ cmd=("make")
+ cmd+=("--silent")
+ cmd+=("-j$JOBS")
+ cmd+=("clean")
+
+ (
+ cd "$SRC"
+ "${cmd[@]}"
+ )
+}
+
+build_kernel() {
+ local log="$TMP_DIR/build.log"
+ local cmd=()
+
+ cmd=("make")
+
+ # When a patch to a kernel module references a newly created unexported
+ # symbol which lives in vmlinux or another kernel module, the patched
+ # kernel build fails with the following error:
+ #
+ # ERROR: modpost: "klp_string" [fs/xfs/xfs.ko] undefined!
+ #
+ # The undefined symbols are working as designed in that case. They get
+ # resolved later when the livepatch module build link pulls all the
+ # disparate objects together into the same kernel module.
+ #
+ # It would be good to have a way to tell modpost to skip checking for
+ # undefined symbols altogether. For now, just convert the error to a
+ # warning with KBUILD_MODPOST_WARN, and grep out the warning to avoid
+ # confusing the user.
+ #
+ cmd+=("KBUILD_MODPOST_WARN=1")
+
+ cmd+=("$VERBOSE")
+ cmd+=("-j$JOBS")
+ cmd+=("KCFLAGS=-ffunction-sections -fdata-sections")
+ cmd+=("vmlinux")
+ cmd+=("modules")
+
+ (
+ cd "$SRC"
+ "${cmd[@]}" \
+ 1> >(tee -a "$log") \
+ 2> >(tee -a "$log" | grep0 -v "modpost.*undefined!" >&2)
+ )
+}
+
+find_objects() {
+ local opts=("$@")
+
+ # Find root-level vmlinux.o and non-root-level .ko files,
+ # excluding klp-tmp/ and .git/
+ find "$OBJ" \( -path "$TMP_DIR" -o -path "$OBJ/.git" -o -regex "$OBJ/[^/][^/]*\.ko" \) -prune -o \
+ -type f "${opts[@]}" \
+ \( -name "*.ko" -o -path "$OBJ/vmlinux.o" \) \
+ -printf '%P\n'
+}
+
+# Copy all .o archives to $ORIG_DIR
+copy_orig_objects() {
+ local files=()
+
+ rm -rf "$ORIG_DIR"
+ mkdir -p "$ORIG_DIR"
+
+ find_objects | mapfile -t files
+
+ xtrace_save "copying orig objects"
+ for _file in "${files[@]}"; do
+ local rel_file="${_file/.ko/.o}"
+ local file="$OBJ/$rel_file"
+ local file_dir="$(dirname "$file")"
+ local orig_file="$ORIG_DIR/$rel_file"
+ local orig_dir="$(dirname "$orig_file")"
+ local cmd_file="$file_dir/.$(basename "$file").cmd"
+
+ [[ ! -f "$file" ]] && die "missing $(basename "$file") for $_file"
+
+ mkdir -p "$orig_dir"
+ cp -f "$file" "$orig_dir"
+ [[ -e "$cmd_file" ]] && cp -f "$cmd_file" "$orig_dir"
+ done
+ xtrace_restore
+
+ mv -f "$TMP_DIR/build.log" "$ORIG_DIR"
+ touch "$TIMESTAMP"
+}
+
+# Copy all changed objects to $PATCHED_DIR
+copy_patched_objects() {
+ local files=()
+ local opts=()
+ local found=0
+
+ rm -rf "$PATCHED_DIR"
+ mkdir -p "$PATCHED_DIR"
+
+ # Note this doesn't work with some configs, thus the 'cmp' below.
+ opts=("-newer")
+ opts+=("$TIMESTAMP")
+
+ find_objects "${opts[@]}" | mapfile -t files
+
+ xtrace_save "copying changed objects"
+ for _file in "${files[@]}"; do
+ local rel_file="${_file/.ko/.o}"
+ local file="$OBJ/$rel_file"
+ local orig_file="$ORIG_DIR/$rel_file"
+ local patched_file="$PATCHED_DIR/$rel_file"
+ local patched_dir="$(dirname "$patched_file")"
+
+ [[ ! -f "$file" ]] && die "missing $(basename "$file") for $_file"
+
+ cmp -s "$orig_file" "$file" && continue
+
+ mkdir -p "$patched_dir"
+ cp -f "$file" "$patched_dir"
+ found=1
+ done
+ xtrace_restore
+
+ (( found == 0 )) && die "no changes detected"
+
+ mv -f "$TMP_DIR/build.log" "$PATCHED_DIR"
+}
+
+# Diff changed objects, writing output object to $DIFF_DIR
+diff_objects() {
+ local log="$KLP_DIFF_LOG"
+ local files=()
+
+ rm -rf "$DIFF_DIR"
+ mkdir -p "$DIFF_DIR"
+
+ find "$PATCHED_DIR" -type f -name "*.o" | mapfile -t files
+ [[ ${#files[@]} -eq 0 ]] && die "no changes detected"
+
+ # Diff all changed objects
+ for file in "${files[@]}"; do
+ local rel_file="${file#"$PATCHED_DIR"/}"
+ local orig_file="$rel_file"
+ local patched_file="$PATCHED_DIR/$rel_file"
+ local out_file="$DIFF_DIR/$rel_file"
+ local cmd=()
+
+ mkdir -p "$(dirname "$out_file")"
+
+ cmd=("$SRC/tools/objtool/objtool")
+ cmd+=("klp")
+ cmd+=("diff")
+ cmd+=("$orig_file")
+ cmd+=("$patched_file")
+ cmd+=("$out_file")
+
+ (
+ cd "$ORIG_DIR"
+ "${cmd[@]}" \
+ 1> >(tee -a "$log") \
+ 2> >(tee -a "$log" >&2) || \
+ die "objtool klp diff failed"
+ )
+ done
+}
+
+# Build and post-process livepatch module in $KMOD_DIR
+build_patch_module() {
+ local makefile="$KMOD_DIR/Kbuild"
+ local log="$KMOD_DIR/build.log"
+ local kmod_file
+ local cflags=()
+ local files=()
+ local cmd=()
+
+ rm -rf "$KMOD_DIR"
+ mkdir -p "$KMOD_DIR"
+
+ cp -f "$SRC/scripts/livepatch/init.c" "$KMOD_DIR"
+
+ echo "obj-m := $NAME.o" > "$makefile"
+ echo -n "$NAME-y := init.o" >> "$makefile"
+
+ find "$DIFF_DIR" -type f -name "*.o" | mapfile -t files
+ [[ ${#files[@]} -eq 0 ]] && die "no changes detected"
+
+ for file in "${files[@]}"; do
+ local rel_file="${file#"$DIFF_DIR"/}"
+ local orig_file="$ORIG_DIR/$rel_file"
+ local orig_dir="$(dirname "$orig_file")"
+ local kmod_file="$KMOD_DIR/$rel_file"
+ local kmod_dir="$(dirname "$kmod_file")"
+ local cmd_file="$orig_dir/.$(basename "$file").cmd"
+
+ mkdir -p "$kmod_dir"
+ cp -f "$file" "$kmod_dir"
+ [[ -e "$cmd_file" ]] && cp -f "$cmd_file" "$kmod_dir"
+
+ # Tell kbuild this is a prebuilt object
+ cp -f "$file" "${kmod_file}_shipped"
+
+ echo -n " $rel_file" >> "$makefile"
+ done
+
+ echo >> "$makefile"
+
+ cflags=("-ffunction-sections")
+ cflags+=("-fdata-sections")
+ [[ $REPLACE -eq 0 ]] && cflags+=("-DKLP_NO_REPLACE")
+
+ cmd=("make")
+ cmd+=("$VERBOSE")
+ cmd+=("-j$JOBS")
+ cmd+=("--directory=.")
+ cmd+=("M=$KMOD_DIR")
+ cmd+=("KCFLAGS=${cflags[*]}")
+
+ # Build a "normal" kernel module with init.c and the diffed objects
+ (
+ cd "$SRC"
+ "${cmd[@]}" \
+ 1> >(tee -a "$log") \
+ 2> >(tee -a "$log" >&2)
+ )
+
+ kmod_file="$KMOD_DIR/$NAME.ko"
+
+ # Save off the intermediate binary for debugging
+ cp -f "$kmod_file" "$kmod_file.orig"
+
+ # Work around issue where slight .config change makes corrupt BTF
+ objcopy --remove-section=.BTF "$kmod_file"
+
+ # Fix (and work around) linker wreckage for klp syms / relocs
+ "$SRC/tools/objtool/objtool" klp post-link "$kmod_file" || die "objtool klp post-link failed"
+
+ cp -f "$kmod_file" "$OUTFILE"
+}
+
+
+################################################################################
+
+process_args "$@"
+do_init
+
+if (( SHORT_CIRCUIT <= 1 )); then
+ status "Validating patch(es)"
+ validate_patches
+ status "Building original kernel"
+ clean_kernel
+ build_kernel
+ status "Copying original object files"
+ copy_orig_objects
+fi
+
+if (( SHORT_CIRCUIT <= 2 )); then
+ status "Fixing patch(es)"
+ fix_patches
+ apply_patches
+ status "Building patched kernel"
+ build_kernel
+ revert_patches
+ status "Copying patched object files"
+ copy_patched_objects
+fi
+
+if (( SHORT_CIRCUIT <= 3 )); then
+ status "Diffing objects"
+ diff_objects
+fi
+
+if (( SHORT_CIRCUIT <= 4 )); then
+ status "Building patch module: $OUTFILE"
+ build_patch_module
+fi
+
+status "SUCCESS"
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 817d44394a78c..4d1f9e9977eb9 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -241,10 +241,12 @@ static struct symbol *next_file_symbol(struct elf *elf, struct symbol *sym)
static bool is_uncorrelated_static_local(struct symbol *sym)
{
static const char * const vars[] = {
- "__key.",
- "__warned.",
"__already_done.",
"__func__.",
+ "__key.",
+ "__warned.",
+ "_entry.",
+ "_entry_ptr.",
"_rs.",
"descriptor.",
"CSWTCH.",
--
2.50.0
^ permalink raw reply related
* [PATCH v4 59/63] livepatch/klp-build: Add stub init code for livepatch modules
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a module initialization stub which can be linked with binary diff
objects to produce a livepatch module.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/livepatch/init.c | 108 +++++++++++++++++++++++++++++++++++++++
1 file changed, 108 insertions(+)
create mode 100644 scripts/livepatch/init.c
diff --git a/scripts/livepatch/init.c b/scripts/livepatch/init.c
new file mode 100644
index 0000000000000..2274d8f5a4826
--- /dev/null
+++ b/scripts/livepatch/init.c
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Init code for a livepatch kernel module
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/livepatch.h>
+
+extern struct klp_object_ext __start_klp_objects[];
+extern struct klp_object_ext __stop_klp_objects[];
+
+static struct klp_patch *patch;
+
+static int __init livepatch_mod_init(void)
+{
+ struct klp_object *objs;
+ unsigned int nr_objs;
+ int ret;
+
+ nr_objs = __stop_klp_objects - __start_klp_objects;
+
+ if (!nr_objs) {
+ pr_err("nothing to patch!\n");
+ ret = -EINVAL;
+ goto err;
+ }
+
+ patch = kzalloc(sizeof(*patch), GFP_KERNEL);
+ if (!patch) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ objs = kzalloc(sizeof(struct klp_object) * (nr_objs + 1), GFP_KERNEL);
+ if (!objs) {
+ ret = -ENOMEM;
+ goto err_free_patch;
+ }
+
+ for (int i = 0; i < nr_objs; i++) {
+ struct klp_object_ext *obj_ext = __start_klp_objects + i;
+ struct klp_func_ext *funcs_ext = obj_ext->funcs;
+ unsigned int nr_funcs = obj_ext->nr_funcs;
+ struct klp_func *funcs = objs[i].funcs;
+ struct klp_object *obj = objs + i;
+
+ funcs = kzalloc(sizeof(struct klp_func) * (nr_funcs + 1), GFP_KERNEL);
+ if (!funcs) {
+ ret = -ENOMEM;
+ for (int j = 0; j < i; j++)
+ kfree(objs[i].funcs);
+ goto err_free_objs;
+ }
+
+ for (int j = 0; j < nr_funcs; j++) {
+ funcs[j].old_name = funcs_ext[j].old_name;
+ funcs[j].new_func = funcs_ext[j].new_func;
+ funcs[j].old_sympos = funcs_ext[j].sympos;
+ }
+
+ obj->name = obj_ext->name;
+ obj->funcs = funcs;
+
+ memcpy(&obj->callbacks, &obj_ext->callbacks, sizeof(struct klp_callbacks));
+ }
+
+ patch->mod = THIS_MODULE;
+ patch->objs = objs;
+
+ /* TODO patch->states */
+
+#ifdef KLP_NO_REPLACE
+ patch->replace = false;
+#else
+ patch->replace = true;
+#endif
+
+ return klp_enable_patch(patch);
+
+err_free_objs:
+ kfree(objs);
+err_free_patch:
+ kfree(patch);
+err:
+ return ret;
+}
+
+static void __exit livepatch_mod_exit(void)
+{
+ unsigned int nr_objs;
+
+ nr_objs = __stop_klp_objects - __start_klp_objects;
+
+ for (int i = 0; i < nr_objs; i++)
+ kfree(patch->objs[i].funcs);
+
+ kfree(patch->objs);
+ kfree(patch);
+}
+
+module_init(livepatch_mod_init);
+module_exit(livepatch_mod_exit);
+MODULE_LICENSE("GPL");
+MODULE_INFO(livepatch, "Y");
+MODULE_DESCRIPTION("Livepatch module");
--
2.50.0
^ permalink raw reply related
* [PATCH v4 58/63] livepatch/klp-build: Introduce fix-patch-lines script to avoid __LINE__ diff noise
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
The __LINE__ macro creates challenges for binary diffing. When a .patch
file adds or removes lines, it shifts the line numbers for all code
below it.
This can cause the code generation of functions using __LINE__ to change
due to the line number constant being embedded in a MOV instruction,
despite there being no semantic difference.
Avoid such false positives by adding a fix-patch-lines script which can
be used to insert a #line directive in each patch hunk affecting the
line numbering. This script will be used by klp-build, which will be
introduced in a subsequent patch.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
MAINTAINERS | 1 +
scripts/livepatch/fix-patch-lines | 79 +++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+)
create mode 100755 scripts/livepatch/fix-patch-lines
diff --git a/MAINTAINERS b/MAINTAINERS
index 605390cdfb75e..485042b545b3e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14225,6 +14225,7 @@ F: include/linux/livepatch*.h
F: kernel/livepatch/
F: kernel/module/livepatch.c
F: samples/livepatch/
+F: scripts/livepatch/
F: tools/testing/selftests/livepatch/
LLC (802.2)
diff --git a/scripts/livepatch/fix-patch-lines b/scripts/livepatch/fix-patch-lines
new file mode 100755
index 0000000000000..73c5e3dea46e1
--- /dev/null
+++ b/scripts/livepatch/fix-patch-lines
@@ -0,0 +1,79 @@
+#!/usr/bin/awk -f
+# SPDX-License-Identifier: GPL-2.0
+#
+# Use #line directives to preserve original __LINE__ numbers across patches to
+# avoid unwanted compilation changes.
+
+BEGIN {
+ in_hunk = 0
+ skip = 0
+}
+
+/^--- / {
+ skip = $2 !~ /\.(c|h)$/
+ print
+ next
+}
+
+/^@@/ {
+ if (skip) {
+ print
+ next
+ }
+
+ in_hunk = 1
+
+ # for @@ -1,3 +1,4 @@:
+ # 1: line number in old file
+ # 3: how many lines the hunk covers in old file
+ # 1: line number in new file
+ # 4: how many lines the hunk covers in new file
+
+ match($0, /^@@ -([0-9]+)(,([0-9]+))? \+([0-9]+)(,([0-9]+))? @@/, m)
+
+ # Set 'cur' to the old file's line number at the start of the hunk. It
+ # gets incremented for every context line and every line removal, so
+ # that it always represents the old file's current line number.
+ cur = m[1]
+
+ # last = last line number of current hunk
+ last = cur + (m[3] ? m[3] : 1) - 1
+
+ need_line_directive = 0
+
+ print
+ next
+}
+
+{
+ if (skip || !in_hunk || $0 ~ /^\\ No newline at end of file/) {
+ print
+ next
+ }
+
+ # change line
+ if ($0 ~ /^[+-]/) {
+ # inject #line after this group of changes
+ need_line_directive = 1
+
+ if ($0 ~ /^-/)
+ cur++
+
+ print
+ next
+ }
+
+ # If this is the first context line after a group of changes, inject
+ # the #line directive to force the compiler to correct the line
+ # numbering to match the original file.
+ if (need_line_directive) {
+ print "+#line " cur
+ need_line_directive = 0
+ }
+
+ if (cur == last)
+ in_hunk = 0
+
+ cur++
+ print
+}
--
2.50.0
^ permalink raw reply related
* [PATCH v4 57/63] kbuild,objtool: Defer objtool validation step for CONFIG_KLP_BUILD
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
In preparation for klp-build, defer objtool validation for
CONFIG_KLP_BUILD kernels until the final pre-link archive (e.g.,
vmlinux.o, module-foo.o) is built. This will simplify the process of
generating livepatch modules.
Delayed objtool is generally preferred anyway, and is already standard
for IBT and LTO. Eventually the per-translation-unit mode will be
phased out.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
scripts/Makefile.lib | 2 +-
scripts/link-vmlinux.sh | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 15fee73e92895..28a1c08e3b221 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -197,7 +197,7 @@ objtool-args = $(objtool-args-y) \
$(if $(delay-objtool), --link) \
$(if $(part-of-module), --module)
-delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT))
+delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT),$(CONFIG_KLP_BUILD))
cmd_objtool = $(if $(objtool-enabled), ; $(objtool) $(objtool-args) $@)
cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(objtool))' ; } >> $(dot-target).cmd)
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 51367c2bfc21e..59f8752362926 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -60,7 +60,8 @@ vmlinux_link()
# skip output file argument
shift
- if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT; then
+ if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT ||
+ is_enabled CONFIG_KLP_BUILD; then
# Use vmlinux.o instead of performing the slow LTO link again.
objs=vmlinux.o
libs=
--
2.50.0
^ permalink raw reply related
* [PATCH v4 56/63] livepatch: Add CONFIG_KLP_BUILD
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
In preparation for introducing klp-build, add a new CONFIG_KLP_BUILD
option. The initial version will only be supported on x86-64.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
arch/x86/Kconfig | 1 +
kernel/livepatch/Kconfig | 12 ++++++++++++
2 files changed, 13 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 19b1dbc8d5d26..986d31587e999 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -260,6 +260,7 @@ config X86
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_KRETPROBES
select HAVE_RETHOOK
+ select HAVE_KLP_BUILD if X86_64
select HAVE_LIVEPATCH if X86_64
select HAVE_MIXED_BREAKPOINTS_REGS
select HAVE_MOD_ARCH_SPECIFIC
diff --git a/kernel/livepatch/Kconfig b/kernel/livepatch/Kconfig
index 53d51ed619a3d..4c0a9c18d0b25 100644
--- a/kernel/livepatch/Kconfig
+++ b/kernel/livepatch/Kconfig
@@ -18,3 +18,15 @@ config LIVEPATCH
module uses the interface provided by this option to register
a patch, causing calls to patched functions to be redirected
to new function code contained in the patch module.
+
+config HAVE_KLP_BUILD
+ bool
+ help
+ Arch supports klp-build
+
+config KLP_BUILD
+ def_bool y
+ depends on LIVEPATCH && HAVE_KLP_BUILD
+ select OBJTOOL
+ help
+ Enable klp-build support
--
2.50.0
^ permalink raw reply related
* [PATCH v4 55/63] objtool: Add base objtool support for livepatch modules
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
In preparation for klp-build, enable "classic" objtool to work on
livepatch modules:
- Avoid duplicate symbol/section warnings for prefix symbols and the
.static_call_sites and __mcount_loc sections which may have already
been extracted by klp diff.
- Add __klp_funcs to the IBT function pointer section whitelist.
- Prevent KLP symbols from getting incorrectly classified as cold
subfunctions.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 52 ++++++++++++++++++++++---
tools/objtool/elf.c | 5 ++-
tools/objtool/include/objtool/elf.h | 1 +
tools/objtool/include/objtool/objtool.h | 2 +-
4 files changed, 53 insertions(+), 7 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 6a9ce090a2cde..ba591a325d52e 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3,6 +3,7 @@
* Copyright (C) 2015-2017 Josh Poimboeuf <jpoimboe@redhat.com>
*/
+#define _GNU_SOURCE /* memmem() */
#include <string.h>
#include <stdlib.h>
#include <inttypes.h>
@@ -611,6 +612,20 @@ static int init_pv_ops(struct objtool_file *file)
return 0;
}
+static bool is_livepatch_module(struct objtool_file *file)
+{
+ struct section *sec;
+
+ if (!opts.module)
+ return false;
+
+ sec = find_section_by_name(file->elf, ".modinfo");
+ if (!sec)
+ return false;
+
+ return memmem(sec->data->d_buf, sec_size(sec), "\0livepatch=Y", 12);
+}
+
static int create_static_call_sections(struct objtool_file *file)
{
struct static_call_site *site;
@@ -622,7 +637,14 @@ static int create_static_call_sections(struct objtool_file *file)
sec = find_section_by_name(file->elf, ".static_call_sites");
if (sec) {
- WARN("file already has .static_call_sites section, skipping");
+ /*
+ * Livepatch modules may have already extracted the static call
+ * site entries to take advantage of vmlinux static call
+ * privileges.
+ */
+ if (!file->klp)
+ WARN("file already has .static_call_sites section, skipping");
+
return 0;
}
@@ -666,7 +688,7 @@ static int create_static_call_sections(struct objtool_file *file)
key_sym = find_symbol_by_name(file->elf, tmp);
if (!key_sym) {
- if (!opts.module) {
+ if (!opts.module || file->klp) {
ERROR("static_call: can't find static_call_key symbol: %s", tmp);
return -1;
}
@@ -885,7 +907,13 @@ static int create_mcount_loc_sections(struct objtool_file *file)
sec = find_section_by_name(file->elf, "__mcount_loc");
if (sec) {
- WARN("file already has __mcount_loc section, skipping");
+ /*
+ * Livepatch modules have already extracted their __mcount_loc
+ * entries to cover the !CONFIG_FTRACE_MCOUNT_USE_OBJTOOL case.
+ */
+ if (!file->klp)
+ WARN("file already has __mcount_loc section, skipping");
+
return 0;
}
@@ -2579,6 +2607,8 @@ static bool validate_branch_enabled(void)
static int decode_sections(struct objtool_file *file)
{
+ file->klp = is_livepatch_module(file);
+
mark_rodata(file);
if (init_pv_ops(file))
@@ -4254,6 +4284,9 @@ static bool ignore_unreachable_insn(struct objtool_file *file, struct instructio
* - compiler cloned functions (*.cold, *.part0, etc)
* - asm functions created with inline asm or without SYM_FUNC_START()
*
+ * Also, the function may already have a prefix from a previous objtool run
+ * (livepatch extracted functions, or manually running objtool multiple times).
+ *
* So return 0 if the NOPs are missing or the function already has a prefix
* symbol.
*/
@@ -4276,6 +4309,14 @@ static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
if (snprintf_check(name, SYM_NAME_LEN, "__pfx_%s", func->name))
return -1;
+ if (file->klp) {
+ struct symbol *pfx;
+
+ pfx = find_symbol_by_offset(func->sec, func->offset - opts.prefix);
+ if (pfx && is_prefix_func(pfx) && !strcmp(pfx->name, name))
+ return 0;
+ }
+
insn = find_insn(file, func->sec, func->offset);
if (!insn) {
WARN("%s: can't find starting instruction", func->name);
@@ -4628,6 +4669,7 @@ static int validate_ibt(struct objtool_file *file)
!strncmp(sec->name, ".debug", 6) ||
!strcmp(sec->name, ".altinstructions") ||
!strcmp(sec->name, ".ibt_endbr_seal") ||
+ !strcmp(sec->name, ".kcfi_traps") ||
!strcmp(sec->name, ".orc_unwind_ip") ||
!strcmp(sec->name, ".retpoline_sites") ||
!strcmp(sec->name, ".smp_locks") ||
@@ -4637,12 +4679,12 @@ static int validate_ibt(struct objtool_file *file)
!strcmp(sec->name, "__bug_table") ||
!strcmp(sec->name, "__ex_table") ||
!strcmp(sec->name, "__jump_table") ||
+ !strcmp(sec->name, "__klp_funcs") ||
!strcmp(sec->name, "__mcount_loc") ||
- !strcmp(sec->name, ".kcfi_traps") ||
!strcmp(sec->name, ".llvm.call-graph-profile") ||
!strcmp(sec->name, ".llvm_bb_addr_map") ||
!strcmp(sec->name, "__tracepoints") ||
- strstr(sec->name, "__patchable_function_entries"))
+ !strcmp(sec->name, "__patchable_function_entries"))
continue;
for_each_reloc(sec->rsec, reloc)
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index 4bb7ce994b6a7..5feeefc7fc8f8 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -499,7 +499,10 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
strstarts(sym->name, "__pi___cfi_")))
sym->prefix = 1;
- if (is_func_sym(sym) && strstr(sym->name, ".cold"))
+ if (strstarts(sym->name, ".klp.sym"))
+ sym->klp = 1;
+
+ if (!sym->klp && is_func_sym(sym) && strstr(sym->name, ".cold"))
sym->cold = 1;
sym->pfunc = sym->cfunc = sym;
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index 60f844e43c5a2..21d8b825fd8f0 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -88,6 +88,7 @@ struct symbol {
u8 debug_checksum : 1;
u8 changed : 1;
u8 included : 1;
+ u8 klp : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/include/objtool/objtool.h
index 7f70b41d1b8db..f7051bbe0bcb2 100644
--- a/tools/objtool/include/objtool/objtool.h
+++ b/tools/objtool/include/objtool/objtool.h
@@ -28,7 +28,7 @@ struct objtool_file {
struct list_head mcount_loc_list;
struct list_head endbr_list;
struct list_head call_list;
- bool ignore_unreachables, hints, rodata;
+ bool ignore_unreachables, hints, rodata, klp;
unsigned int nr_endbr;
unsigned int nr_endbr_int;
--
2.50.0
^ permalink raw reply related
* [PATCH v4 54/63] objtool: Refactor prefix symbol creation code
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
The prefix symbol creation code currently ignores all errors, presumably
because some functions don't have the leading NOPs.
Shuffle the code around a bit, improve the error handling and document
why some errors are ignored.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/check.c | 59 ++++++++++++++++++++++-------
tools/objtool/elf.c | 17 ---------
tools/objtool/include/objtool/elf.h | 2 -
3 files changed, 46 insertions(+), 32 deletions(-)
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 6eaae3c007486..6a9ce090a2cde 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -15,6 +15,7 @@
#include <objtool/special.h>
#include <objtool/warn.h>
#include <objtool/checksum.h>
+#include <objtool/util.h>
#include <linux/objtool_types.h>
#include <linux/hashtable.h>
@@ -4243,37 +4244,71 @@ static bool ignore_unreachable_insn(struct objtool_file *file, struct instructio
return false;
}
-static int add_prefix_symbol(struct objtool_file *file, struct symbol *func)
+/*
+ * For FineIBT or kCFI, a certain number of bytes preceding the function may be
+ * NOPs. Those NOPs may be rewritten at runtime and executed, so give them a
+ * proper function name: __pfx_<func>.
+ *
+ * The NOPs may not exist for the following cases:
+ *
+ * - compiler cloned functions (*.cold, *.part0, etc)
+ * - asm functions created with inline asm or without SYM_FUNC_START()
+ *
+ * So return 0 if the NOPs are missing or the function already has a prefix
+ * symbol.
+ */
+static int create_prefix_symbol(struct objtool_file *file, struct symbol *func)
{
struct instruction *insn, *prev;
+ char name[SYM_NAME_LEN];
struct cfi_state *cfi;
- insn = find_insn(file, func->sec, func->offset);
- if (!insn)
+ if (!is_func_sym(func) || is_prefix_func(func) ||
+ func->cold || func->static_call_tramp)
+ return 0;
+
+ if ((strlen(func->name) + sizeof("__pfx_") > SYM_NAME_LEN)) {
+ WARN("%s: symbol name too long, can't create __pfx_ symbol",
+ func->name);
+ return 0;
+ }
+
+ if (snprintf_check(name, SYM_NAME_LEN, "__pfx_%s", func->name))
return -1;
+ insn = find_insn(file, func->sec, func->offset);
+ if (!insn) {
+ WARN("%s: can't find starting instruction", func->name);
+ return -1;
+ }
+
for (prev = prev_insn_same_sec(file, insn);
prev;
prev = prev_insn_same_sec(file, prev)) {
u64 offset;
if (prev->type != INSN_NOP)
- return -1;
+ return 0;
offset = func->offset - prev->offset;
if (offset > opts.prefix)
- return -1;
+ return 0;
if (offset < opts.prefix)
continue;
- elf_create_prefix_symbol(file->elf, func, opts.prefix);
+ if (!elf_create_symbol(file->elf, name, func->sec,
+ GELF_ST_BIND(func->sym.st_info),
+ GELF_ST_TYPE(func->sym.st_info),
+ prev->offset, opts.prefix))
+ return -1;
+
break;
}
if (!prev)
- return -1;
+ return 0;
if (!insn->cfi) {
/*
@@ -4291,7 +4326,7 @@ static int add_prefix_symbol(struct objtool_file *file, struct symbol *func)
return 0;
}
-static int add_prefix_symbols(struct objtool_file *file)
+static int create_prefix_symbols(struct objtool_file *file)
{
struct section *sec;
struct symbol *func;
@@ -4301,10 +4336,8 @@ static int add_prefix_symbols(struct objtool_file *file)
continue;
sec_for_each_sym(sec, func) {
- if (!is_func_sym(func))
- continue;
-
- add_prefix_symbol(file, func);
+ if (create_prefix_symbol(file, func))
+ return -1;
}
}
@@ -4931,7 +4964,7 @@ int check(struct objtool_file *file)
}
if (opts.prefix) {
- ret = add_prefix_symbols(file);
+ ret = create_prefix_symbols(file);
if (ret)
goto out;
}
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index e1daae0630be9..4bb7ce994b6a7 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -942,23 +942,6 @@ struct symbol *elf_create_section_symbol(struct elf *elf, struct section *sec)
return sym;
}
-struct symbol *
-elf_create_prefix_symbol(struct elf *elf, struct symbol *orig, size_t size)
-{
- size_t namelen = strlen(orig->name) + sizeof("__pfx_");
- char name[SYM_NAME_LEN];
- unsigned long offset;
-
- snprintf(name, namelen, "__pfx_%s", orig->name);
-
- offset = orig->sym.st_value - size;
-
- return elf_create_symbol(elf, name, orig->sec,
- GELF_ST_BIND(orig->sym.st_info),
- GELF_ST_TYPE(orig->sym.st_info),
- offset, size);
-}
-
struct reloc *elf_init_reloc(struct elf *elf, struct section *rsec,
unsigned int reloc_idx, unsigned long offset,
struct symbol *sym, s64 addend, unsigned int type)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index e2cd817fca522..60f844e43c5a2 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -148,8 +148,6 @@ struct symbol *elf_create_symbol(struct elf *elf, const char *name,
unsigned int type, unsigned long offset,
size_t size);
struct symbol *elf_create_section_symbol(struct elf *elf, struct section *sec);
-struct symbol *elf_create_prefix_symbol(struct elf *elf, struct symbol *orig,
- size_t size);
void *elf_add_data(struct elf *elf, struct section *sec, const void *data,
size_t size);
--
2.50.0
^ permalink raw reply related
* [PATCH v4 53/63] objtool/klp: Add post-link subcommand to finalize livepatch modules
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Livepatch needs some ELF magic which linkers don't like:
- Two relocation sections (.rela*, .klp.rela*) for the same text
section.
- Use of SHN_LIVEPATCH to mark livepatch symbols.
Unfortunately linkers tend to mangle such things. To work around that,
klp diff generates a linker-compliant intermediate binary which encodes
the relevant KLP section/reloc/symbol metadata.
After module linking, the .ko then needs to be converted to an actual
livepatch module. Introduce a new klp post-link subcommand to do so.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/Build | 2 +-
tools/objtool/builtin-klp.c | 1 +
tools/objtool/include/objtool/klp.h | 4 +
tools/objtool/klp-post-link.c | 168 ++++++++++++++++++++++++++++
4 files changed, 174 insertions(+), 1 deletion(-)
create mode 100644 tools/objtool/klp-post-link.c
diff --git a/tools/objtool/Build b/tools/objtool/Build
index 0b01657671d7f..8cd71b9a5eef1 100644
--- a/tools/objtool/Build
+++ b/tools/objtool/Build
@@ -9,7 +9,7 @@ objtool-y += elf.o
objtool-y += objtool.o
objtool-$(BUILD_ORC) += orc_gen.o orc_dump.o
-objtool-$(BUILD_KLP) += builtin-klp.o klp-diff.o
+objtool-$(BUILD_KLP) += builtin-klp.o klp-diff.o klp-post-link.o
objtool-y += libstring.o
objtool-y += libctype.o
diff --git a/tools/objtool/builtin-klp.c b/tools/objtool/builtin-klp.c
index 9b13dd1182af1..56d5a5b92f725 100644
--- a/tools/objtool/builtin-klp.c
+++ b/tools/objtool/builtin-klp.c
@@ -14,6 +14,7 @@ struct subcmd {
static struct subcmd subcmds[] = {
{ "diff", "Generate binary diff of two object files", cmd_klp_diff, },
+ { "post-link", "Finalize klp symbols/relocs after module linking", cmd_klp_post_link, },
};
static void cmd_klp_usage(void)
diff --git a/tools/objtool/include/objtool/klp.h b/tools/objtool/include/objtool/klp.h
index 07928fac059b5..ad830a7ce55b6 100644
--- a/tools/objtool/include/objtool/klp.h
+++ b/tools/objtool/include/objtool/klp.h
@@ -2,6 +2,9 @@
#ifndef _OBJTOOL_KLP_H
#define _OBJTOOL_KLP_H
+#define SHF_RELA_LIVEPATCH 0x00100000
+#define SHN_LIVEPATCH 0xff20
+
/*
* __klp_objects and __klp_funcs are created by klp diff and used by the patch
* module init code to build the klp_patch, klp_object and klp_func structs
@@ -27,5 +30,6 @@ struct klp_reloc {
};
int cmd_klp_diff(int argc, const char **argv);
+int cmd_klp_post_link(int argc, const char **argv);
#endif /* _OBJTOOL_KLP_H */
diff --git a/tools/objtool/klp-post-link.c b/tools/objtool/klp-post-link.c
new file mode 100644
index 0000000000000..c013e39957b11
--- /dev/null
+++ b/tools/objtool/klp-post-link.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Read the intermediate KLP reloc/symbol representations created by klp diff
+ * and convert them to the proper format required by livepatch. This needs to
+ * run last to avoid linker wreckage. Linkers don't tend to handle the "two
+ * rela sections for a single base section" case very well, nor do they like
+ * SHN_LIVEPATCH.
+ *
+ * This is the final tool in the livepatch module generation pipeline:
+ *
+ * kernel builds -> objtool klp diff -> module link -> objtool klp post-link
+ */
+
+#include <fcntl.h>
+#include <gelf.h>
+#include <objtool/objtool.h>
+#include <objtool/warn.h>
+#include <objtool/klp.h>
+#include <objtool/util.h>
+#include <linux/livepatch_external.h>
+
+static int fix_klp_relocs(struct elf *elf)
+{
+ struct section *symtab, *klp_relocs;
+
+ klp_relocs = find_section_by_name(elf, KLP_RELOCS_SEC);
+ if (!klp_relocs)
+ return 0;
+
+ symtab = find_section_by_name(elf, ".symtab");
+ if (!symtab) {
+ ERROR("missing .symtab");
+ return -1;
+ }
+
+ for (int i = 0; i < sec_size(klp_relocs) / sizeof(struct klp_reloc); i++) {
+ struct klp_reloc *klp_reloc;
+ unsigned long klp_reloc_off;
+ struct section *sec, *tmp, *klp_rsec;
+ unsigned long offset;
+ struct reloc *reloc;
+ char sym_modname[64];
+ char rsec_name[SEC_NAME_LEN];
+ u64 addend;
+ struct symbol *sym, *klp_sym;
+
+ klp_reloc_off = i * sizeof(*klp_reloc);
+ klp_reloc = klp_relocs->data->d_buf + klp_reloc_off;
+
+ /*
+ * Read __klp_relocs[i]:
+ */
+
+ /* klp_reloc.sec_offset */
+ reloc = find_reloc_by_dest(elf, klp_relocs,
+ klp_reloc_off + offsetof(struct klp_reloc, offset));
+ if (!reloc) {
+ ERROR("malformed " KLP_RELOCS_SEC " section");
+ return -1;
+ }
+
+ sec = reloc->sym->sec;
+ offset = reloc_addend(reloc);
+
+ /* klp_reloc.sym */
+ reloc = find_reloc_by_dest(elf, klp_relocs,
+ klp_reloc_off + offsetof(struct klp_reloc, sym));
+ if (!reloc) {
+ ERROR("malformed " KLP_RELOCS_SEC " section");
+ return -1;
+ }
+
+ klp_sym = reloc->sym;
+ addend = reloc_addend(reloc);
+
+ /* symbol format: .klp.sym.modname.sym_name,sympos */
+ if (sscanf(klp_sym->name + strlen(KLP_SYM_PREFIX), "%55[^.]", sym_modname) != 1)
+ ERROR("can't find modname in klp symbol '%s'", klp_sym->name);
+
+ /*
+ * Create the KLP rela:
+ */
+
+ /* section format: .klp.rela.sec_objname.section_name */
+ if (snprintf_check(rsec_name, SEC_NAME_LEN,
+ KLP_RELOC_SEC_PREFIX "%s.%s",
+ sym_modname, sec->name))
+ return -1;
+
+ klp_rsec = find_section_by_name(elf, rsec_name);
+ if (!klp_rsec) {
+ klp_rsec = elf_create_section(elf, rsec_name, 0,
+ elf_rela_size(elf),
+ SHT_RELA, elf_addr_size(elf),
+ SHF_ALLOC | SHF_INFO_LINK | SHF_RELA_LIVEPATCH);
+ if (!klp_rsec)
+ return -1;
+
+ klp_rsec->sh.sh_link = symtab->idx;
+ klp_rsec->sh.sh_info = sec->idx;
+ klp_rsec->base = sec;
+ }
+
+ tmp = sec->rsec;
+ sec->rsec = klp_rsec;
+ if (!elf_create_reloc(elf, sec, offset, klp_sym, addend, klp_reloc->type))
+ return -1;
+ sec->rsec = tmp;
+
+ /*
+ * Fix up the corresponding KLP symbol:
+ */
+
+ klp_sym->sym.st_shndx = SHN_LIVEPATCH;
+ if (!gelf_update_sym(symtab->data, klp_sym->idx, &klp_sym->sym)) {
+ ERROR_ELF("gelf_update_sym");
+ return -1;
+ }
+
+ /*
+ * Disable the original non-KLP reloc by converting it to R_*_NONE:
+ */
+
+ reloc = find_reloc_by_dest(elf, sec, offset);
+ sym = reloc->sym;
+ sym->sym.st_shndx = SHN_LIVEPATCH;
+ set_reloc_type(elf, reloc, 0);
+ if (!gelf_update_sym(symtab->data, sym->idx, &sym->sym)) {
+ ERROR_ELF("gelf_update_sym");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * This runs on the livepatch module after all other linking has been done. It
+ * converts the intermediate __klp_relocs section into proper KLP relocs to be
+ * processed by livepatch. This needs to run last to avoid linker wreckage.
+ * Linkers don't tend to handle the "two rela sections for a single base
+ * section" case very well, nor do they appreciate SHN_LIVEPATCH.
+ */
+int cmd_klp_post_link(int argc, const char **argv)
+{
+ struct elf *elf;
+
+ argc--;
+ argv++;
+
+ if (argc != 1) {
+ fprintf(stderr, "%d\n", argc);
+ fprintf(stderr, "usage: objtool link <file.ko>\n");
+ return -1;
+ }
+
+ elf = elf_open_read(argv[0], O_RDWR);
+ if (!elf)
+ return -1;
+
+ if (fix_klp_relocs(elf))
+ return -1;
+
+ if (elf_write(elf))
+ return -1;
+
+ return elf_close(elf);
+}
--
2.50.0
^ permalink raw reply related
* [PATCH v4 52/63] objtool/klp: Add --debug option to show cloning decisions
From: Josh Poimboeuf @ 2025-09-17 16:04 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a --debug option to klp diff which prints cloning decisions and an
indented dependency tree for all cloned symbols and relocations. This
helps visualize which symbols and relocations were included and why.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/include/objtool/warn.h | 21 ++++++++
tools/objtool/klp-diff.c | 75 ++++++++++++++++++++++++++++
tools/objtool/objtool.c | 3 ++
3 files changed, 99 insertions(+)
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index 29173a1368d79..e88322d97573e 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -102,6 +102,10 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define ERROR_FUNC(sec, offset, format, ...) __WARN_FUNC(ERROR_STR, sec, offset, format, ##__VA_ARGS__)
#define ERROR_INSN(insn, format, ...) WARN_FUNC(insn->sec, insn->offset, format, ##__VA_ARGS__)
+extern bool debug;
+extern int indent;
+
+static inline void unindent(int *unused) { indent--; }
#define __dbg(format, ...) \
fprintf(stderr, \
@@ -110,6 +114,23 @@ static inline char *offstr(struct section *sec, unsigned long offset)
objname ? ": " : "", \
##__VA_ARGS__)
+#define dbg(args...) \
+({ \
+ if (unlikely(debug)) \
+ __dbg(args); \
+})
+
+#define __dbg_indent(format, ...) \
+({ \
+ if (unlikely(debug)) \
+ __dbg("%*s" format, indent * 8, "", ##__VA_ARGS__); \
+})
+
+#define dbg_indent(args...) \
+ int __attribute__((cleanup(unindent))) __dummy_##__COUNTER__; \
+ __dbg_indent(args); \
+ indent++
+
#define dbg_checksum(func, insn, checksum) \
({ \
if (unlikely(insn->sym && insn->sym->pfunc && \
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 0d69b621a26cf..817d44394a78c 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -38,6 +38,8 @@ static const char * const klp_diff_usage[] = {
};
static const struct option klp_diff_options[] = {
+ OPT_GROUP("Options:"),
+ OPT_BOOLEAN('d', "debug", &debug, "enable debug output"),
OPT_END(),
};
@@ -48,6 +50,38 @@ static inline u32 str_hash(const char *str)
return jhash(str, strlen(str), 0);
}
+static char *escape_str(const char *orig)
+{
+ size_t len = 0;
+ const char *a;
+ char *b, *new;
+
+ for (a = orig; *a; a++) {
+ switch (*a) {
+ case '\001': len += 5; break;
+ case '\n':
+ case '\t': len += 2; break;
+ default: len++;
+ }
+ }
+
+ new = malloc(len + 1);
+ if (!new)
+ return NULL;
+
+ for (a = orig, b = new; *a; a++) {
+ switch (*a) {
+ case '\001': memcpy(b, "<SOH>", 5); b += 5; break;
+ case '\n': *b++ = '\\'; *b++ = 'n'; break;
+ case '\t': *b++ = '\\'; *b++ = 't'; break;
+ default: *b++ = *a;
+ }
+ }
+
+ *b = '\0';
+ return new;
+}
+
static int read_exports(void)
{
const char *symvers = "Module.symvers";
@@ -528,6 +562,28 @@ static struct symbol *__clone_symbol(struct elf *elf, struct symbol *patched_sym
return out_sym;
}
+static const char *sym_type(struct symbol *sym)
+{
+ switch (sym->type) {
+ case STT_NOTYPE: return "NOTYPE";
+ case STT_OBJECT: return "OBJECT";
+ case STT_FUNC: return "FUNC";
+ case STT_SECTION: return "SECTION";
+ case STT_FILE: return "FILE";
+ default: return "UNKNOWN";
+ }
+}
+
+static const char *sym_bind(struct symbol *sym)
+{
+ switch (sym->bind) {
+ case STB_LOCAL: return "LOCAL";
+ case STB_GLOBAL: return "GLOBAL";
+ case STB_WEAK: return "WEAK";
+ default: return "UNKNOWN";
+ }
+}
+
/*
* Copy a symbol to the output object, optionally including its data and
* relocations.
@@ -540,6 +596,8 @@ static struct symbol *clone_symbol(struct elfs *e, struct symbol *patched_sym,
if (patched_sym->clone)
return patched_sym->clone;
+ dbg_indent("%s%s", patched_sym->name, data_too ? " [+DATA]" : "");
+
/* Make sure the prefix gets cloned first */
if (is_func_sym(patched_sym) && data_too) {
pfx = get_func_prefix(patched_sym);
@@ -902,6 +960,8 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
klp_sym = find_symbol_by_name(e->out, sym_name);
if (!klp_sym) {
+ __dbg_indent("%s", sym_name);
+
/* STB_WEAK: avoid modpost undefined symbol warnings */
klp_sym = elf_create_symbol(e->out, sym_name, NULL,
STB_WEAK, patched_sym->type, 0, 0);
@@ -950,6 +1010,17 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
return 0;
}
+#define dbg_clone_reloc(sec, offset, patched_sym, addend, export, klp) \
+ dbg_indent("%s+0x%lx: %s%s0x%lx [%s%s%s%s%s%s]", \
+ sec->name, offset, patched_sym->name, \
+ addend >= 0 ? "+" : "-", labs(addend), \
+ sym_type(patched_sym), \
+ patched_sym->type == STT_SECTION ? "" : " ", \
+ patched_sym->type == STT_SECTION ? "" : sym_bind(patched_sym), \
+ is_undef_sym(patched_sym) ? " UNDEF" : "", \
+ export ? " EXPORTED" : "", \
+ klp ? " KLP" : "")
+
/* Copy a reloc and its symbol to the output object */
static int clone_reloc(struct elfs *e, struct reloc *patched_reloc,
struct section *sec, unsigned long offset)
@@ -969,6 +1040,8 @@ static int clone_reloc(struct elfs *e, struct reloc *patched_reloc,
klp = klp_reloc_needed(patched_reloc);
+ dbg_clone_reloc(sec, offset, patched_sym, addend, export, klp);
+
if (klp) {
if (clone_reloc_klp(e, patched_reloc, sec, offset, export))
return -1;
@@ -1000,6 +1073,8 @@ static int clone_reloc(struct elfs *e, struct reloc *patched_reloc,
if (is_string_sec(patched_sym->sec)) {
const char *str = patched_sym->sec->data->d_buf + addend;
+ __dbg_indent("\"%s\"", escape_str(str));
+
addend = elf_add_string(e->out, out_sym->sec, str);
if (addend == -1)
return -1;
diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c
index c8f611c1320d1..3c26ed561c7ef 100644
--- a/tools/objtool/objtool.c
+++ b/tools/objtool/objtool.c
@@ -16,6 +16,9 @@
#include <objtool/objtool.h>
#include <objtool/warn.h>
+bool debug;
+int indent;
+
static struct objtool_file file;
struct objtool_file *objtool_open_read(const char *filename)
--
2.50.0
^ permalink raw reply related
* [PATCH v4 51/63] objtool/klp: Introduce klp diff subcommand for diffing object files
From: Josh Poimboeuf @ 2025-09-17 16:03 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a new klp diff subcommand which performs a binary diff between two
object files and extracts changed functions into a new object which can
then be linked into a livepatch module.
This builds on concepts from the longstanding out-of-tree kpatch [1]
project which began in 2012 and has been used for many years to generate
livepatch modules for production kernels. However, this is a complete
rewrite which incorporates hard-earned lessons from 12+ years of
maintaining kpatch.
Key improvements compared to kpatch-build:
- Integrated with objtool: Leverages objtool's existing control-flow
graph analysis to help detect changed functions.
- Works on vmlinux.o: Supports late-linked objects, making it
compatible with LTO, IBT, and similar.
- Simplified code base: ~3k fewer lines of code.
- Upstream: No more out-of-tree #ifdef hacks, far less cruft.
- Cleaner internals: Vastly simplified logic for symbol/section/reloc
inclusion and special section extraction.
- Robust __LINE__ macro handling: Avoids false positive binary diffs
caused by the __LINE__ macro by introducing a fix-patch-lines script
(coming in a later patch) which injects #line directives into the
source .patch to preserve the original line numbers at compile time.
Note the end result of this subcommand is not yet functionally complete.
Livepatch needs some ELF magic which linkers don't like:
- Two relocation sections (.rela*, .klp.rela*) for the same text
section.
- Use of SHN_LIVEPATCH to mark livepatch symbols.
Unfortunately linkers tend to mangle such things. To work around that,
klp diff generates a linker-compliant intermediate binary which encodes
the relevant KLP section/reloc/symbol metadata.
After module linking, a klp post-link step (coming soon) will clean up
the mess and convert the linked .ko into a fully compliant livepatch
module.
Note this subcommand requires the diffed binaries to have been compiled
with -ffunction-sections and -fdata-sections, and processed with
'objtool --checksum'. Those constraints will be handled by a klp-build
script introduced in a later patch.
Without '-ffunction-sections -fdata-sections', reliable object diffing
would be infeasible due to toolchain limitations:
- For intra-file+intra-section references, the compiler might
occasionally generated hard-coded instruction offsets instead of
relocations.
- Section-symbol-based references can be ambiguous:
- Overlapping or zero-length symbols create ambiguity as to which
symbol is being referenced.
- A reference to the end of a symbol (e.g., checking array bounds)
can be misinterpreted as a reference to the next symbol, or vice
versa.
A potential future alternative to '-ffunction-sections -fdata-sections'
would be to introduce a toolchain option that forces symbol-based
(non-section) relocations.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
MAINTAINERS | 2 +-
include/linux/livepatch.h | 25 +-
include/linux/livepatch_external.h | 76 +
kernel/livepatch/core.c | 4 +-
scripts/module.lds.S | 10 +-
tools/include/linux/livepatch_external.h | 76 +
tools/include/linux/string.h | 14 +
tools/objtool/Build | 4 +-
tools/objtool/Makefile | 3 +-
tools/objtool/arch/x86/decode.c | 40 +
tools/objtool/builtin-klp.c | 52 +
tools/objtool/check.c | 14 -
tools/objtool/elf.c | 21 +-
tools/objtool/include/objtool/arch.h | 1 +
tools/objtool/include/objtool/builtin.h | 2 +
tools/objtool/include/objtool/elf.h | 56 +-
tools/objtool/include/objtool/klp.h | 31 +
tools/objtool/include/objtool/objtool.h | 2 +
tools/objtool/include/objtool/util.h | 19 +
tools/objtool/klp-diff.c | 1646 ++++++++++++++++++++++
tools/objtool/objtool.c | 41 +-
tools/objtool/sync-check.sh | 1 +
tools/objtool/weak.c | 7 +
23 files changed, 2088 insertions(+), 59 deletions(-)
create mode 100644 include/linux/livepatch_external.h
create mode 100644 tools/include/linux/livepatch_external.h
create mode 100644 tools/objtool/builtin-klp.c
create mode 100644 tools/objtool/include/objtool/klp.h
create mode 100644 tools/objtool/include/objtool/util.h
create mode 100644 tools/objtool/klp-diff.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 4756cf7b88ae4..605390cdfb75e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14221,7 +14221,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/livepatching/livepatching.g
F: Documentation/ABI/testing/sysfs-kernel-livepatch
F: Documentation/livepatch/
F: arch/powerpc/include/asm/livepatch.h
-F: include/linux/livepatch.h
+F: include/linux/livepatch*.h
F: kernel/livepatch/
F: kernel/module/livepatch.c
F: samples/livepatch/
diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h
index 51a258c24ff52..772919e8096aa 100644
--- a/include/linux/livepatch.h
+++ b/include/linux/livepatch.h
@@ -13,6 +13,7 @@
#include <linux/ftrace.h>
#include <linux/completion.h>
#include <linux/list.h>
+#include <linux/livepatch_external.h>
#include <linux/livepatch_sched.h>
#if IS_ENABLED(CONFIG_LIVEPATCH)
@@ -77,30 +78,6 @@ struct klp_func {
bool transition;
};
-struct klp_object;
-
-/**
- * struct klp_callbacks - pre/post live-(un)patch callback structure
- * @pre_patch: executed before code patching
- * @post_patch: executed after code patching
- * @pre_unpatch: executed before code unpatching
- * @post_unpatch: executed after code unpatching
- * @post_unpatch_enabled: flag indicating if post-unpatch callback
- * should run
- *
- * All callbacks are optional. Only the pre-patch callback, if provided,
- * will be unconditionally executed. If the parent klp_object fails to
- * patch for any reason, including a non-zero error status returned from
- * the pre-patch callback, no further callbacks will be executed.
- */
-struct klp_callbacks {
- int (*pre_patch)(struct klp_object *obj);
- void (*post_patch)(struct klp_object *obj);
- void (*pre_unpatch)(struct klp_object *obj);
- void (*post_unpatch)(struct klp_object *obj);
- bool post_unpatch_enabled;
-};
-
/**
* struct klp_object - kernel object structure for live patching
* @name: module name (or NULL for vmlinux)
diff --git a/include/linux/livepatch_external.h b/include/linux/livepatch_external.h
new file mode 100644
index 0000000000000..138af19b0f5cd
--- /dev/null
+++ b/include/linux/livepatch_external.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * External livepatch interfaces for patch creation tooling
+ */
+
+#ifndef _LINUX_LIVEPATCH_EXTERNAL_H_
+#define _LINUX_LIVEPATCH_EXTERNAL_H_
+
+#include <linux/types.h>
+
+#define KLP_RELOC_SEC_PREFIX ".klp.rela."
+#define KLP_SYM_PREFIX ".klp.sym."
+
+#define __KLP_PRE_PATCH_PREFIX __klp_pre_patch_callback_
+#define __KLP_POST_PATCH_PREFIX __klp_post_patch_callback_
+#define __KLP_PRE_UNPATCH_PREFIX __klp_pre_unpatch_callback_
+#define __KLP_POST_UNPATCH_PREFIX __klp_post_unpatch_callback_
+
+#define KLP_PRE_PATCH_PREFIX __stringify(__KLP_PRE_PATCH_PREFIX)
+#define KLP_POST_PATCH_PREFIX __stringify(__KLP_POST_PATCH_PREFIX)
+#define KLP_PRE_UNPATCH_PREFIX __stringify(__KLP_PRE_UNPATCH_PREFIX)
+#define KLP_POST_UNPATCH_PREFIX __stringify(__KLP_POST_UNPATCH_PREFIX)
+
+struct klp_object;
+
+typedef int (*klp_pre_patch_t)(struct klp_object *obj);
+typedef void (*klp_post_patch_t)(struct klp_object *obj);
+typedef void (*klp_pre_unpatch_t)(struct klp_object *obj);
+typedef void (*klp_post_unpatch_t)(struct klp_object *obj);
+
+/**
+ * struct klp_callbacks - pre/post live-(un)patch callback structure
+ * @pre_patch: executed before code patching
+ * @post_patch: executed after code patching
+ * @pre_unpatch: executed before code unpatching
+ * @post_unpatch: executed after code unpatching
+ * @post_unpatch_enabled: flag indicating if post-unpatch callback
+ * should run
+ *
+ * All callbacks are optional. Only the pre-patch callback, if provided,
+ * will be unconditionally executed. If the parent klp_object fails to
+ * patch for any reason, including a non-zero error status returned from
+ * the pre-patch callback, no further callbacks will be executed.
+ */
+struct klp_callbacks {
+ klp_pre_patch_t pre_patch;
+ klp_post_patch_t post_patch;
+ klp_pre_unpatch_t pre_unpatch;
+ klp_post_unpatch_t post_unpatch;
+ bool post_unpatch_enabled;
+};
+
+/*
+ * 'struct klp_{func,object}_ext' are compact "external" representations of
+ * 'struct klp_{func,object}'. They are used by objtool for livepatch
+ * generation. The structs are then read by the livepatch module and converted
+ * to the real structs before calling klp_enable_patch().
+ *
+ * TODO make these the official API for klp_enable_patch(). That should
+ * simplify livepatch's interface as well as its data structure lifetime
+ * management.
+ */
+struct klp_func_ext {
+ const char *old_name;
+ void *new_func;
+ unsigned long sympos;
+};
+
+struct klp_object_ext {
+ const char *name;
+ struct klp_func_ext *funcs;
+ struct klp_callbacks callbacks;
+ unsigned int nr_funcs;
+};
+
+#endif /* _LINUX_LIVEPATCH_EXTERNAL_H_ */
diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index 7e443c2cf7d48..0044a81250139 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -224,7 +224,7 @@ static int klp_resolve_symbols(Elf_Shdr *sechdrs, const char *strtab,
/* Format: .klp.sym.sym_objname.sym_name,sympos */
cnt = sscanf(strtab + sym->st_name,
- ".klp.sym.%55[^.].%511[^,],%lu",
+ KLP_SYM_PREFIX "%55[^.].%511[^,],%lu",
sym_objname, sym_name, &sympos);
if (cnt != 3) {
pr_err("symbol %s has an incorrectly formatted name\n",
@@ -303,7 +303,7 @@ static int klp_write_section_relocs(struct module *pmod, Elf_Shdr *sechdrs,
* See comment in klp_resolve_symbols() for an explanation
* of the selected field width value.
*/
- cnt = sscanf(shstrtab + sec->sh_name, ".klp.rela.%55[^.]",
+ cnt = sscanf(shstrtab + sec->sh_name, KLP_RELOC_SEC_PREFIX "%55[^.]",
sec_objname);
if (cnt != 1) {
pr_err("section %s has an incorrectly formatted name\n",
diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index 2632c6cb8ebe7..3037d5e5527c0 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -34,8 +34,16 @@ SECTIONS {
__patchable_function_entries : { *(__patchable_function_entries) }
+ __klp_funcs 0: ALIGN(8) { KEEP(*(__klp_funcs)) }
+
+ __klp_objects 0: ALIGN(8) {
+ __start_klp_objects = .;
+ KEEP(*(__klp_objects))
+ __stop_klp_objects = .;
+ }
+
#ifdef CONFIG_ARCH_USES_CFI_TRAPS
- __kcfi_traps : { KEEP(*(.kcfi_traps)) }
+ __kcfi_traps : { KEEP(*(.kcfi_traps)) }
#endif
.text : {
diff --git a/tools/include/linux/livepatch_external.h b/tools/include/linux/livepatch_external.h
new file mode 100644
index 0000000000000..138af19b0f5cd
--- /dev/null
+++ b/tools/include/linux/livepatch_external.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * External livepatch interfaces for patch creation tooling
+ */
+
+#ifndef _LINUX_LIVEPATCH_EXTERNAL_H_
+#define _LINUX_LIVEPATCH_EXTERNAL_H_
+
+#include <linux/types.h>
+
+#define KLP_RELOC_SEC_PREFIX ".klp.rela."
+#define KLP_SYM_PREFIX ".klp.sym."
+
+#define __KLP_PRE_PATCH_PREFIX __klp_pre_patch_callback_
+#define __KLP_POST_PATCH_PREFIX __klp_post_patch_callback_
+#define __KLP_PRE_UNPATCH_PREFIX __klp_pre_unpatch_callback_
+#define __KLP_POST_UNPATCH_PREFIX __klp_post_unpatch_callback_
+
+#define KLP_PRE_PATCH_PREFIX __stringify(__KLP_PRE_PATCH_PREFIX)
+#define KLP_POST_PATCH_PREFIX __stringify(__KLP_POST_PATCH_PREFIX)
+#define KLP_PRE_UNPATCH_PREFIX __stringify(__KLP_PRE_UNPATCH_PREFIX)
+#define KLP_POST_UNPATCH_PREFIX __stringify(__KLP_POST_UNPATCH_PREFIX)
+
+struct klp_object;
+
+typedef int (*klp_pre_patch_t)(struct klp_object *obj);
+typedef void (*klp_post_patch_t)(struct klp_object *obj);
+typedef void (*klp_pre_unpatch_t)(struct klp_object *obj);
+typedef void (*klp_post_unpatch_t)(struct klp_object *obj);
+
+/**
+ * struct klp_callbacks - pre/post live-(un)patch callback structure
+ * @pre_patch: executed before code patching
+ * @post_patch: executed after code patching
+ * @pre_unpatch: executed before code unpatching
+ * @post_unpatch: executed after code unpatching
+ * @post_unpatch_enabled: flag indicating if post-unpatch callback
+ * should run
+ *
+ * All callbacks are optional. Only the pre-patch callback, if provided,
+ * will be unconditionally executed. If the parent klp_object fails to
+ * patch for any reason, including a non-zero error status returned from
+ * the pre-patch callback, no further callbacks will be executed.
+ */
+struct klp_callbacks {
+ klp_pre_patch_t pre_patch;
+ klp_post_patch_t post_patch;
+ klp_pre_unpatch_t pre_unpatch;
+ klp_post_unpatch_t post_unpatch;
+ bool post_unpatch_enabled;
+};
+
+/*
+ * 'struct klp_{func,object}_ext' are compact "external" representations of
+ * 'struct klp_{func,object}'. They are used by objtool for livepatch
+ * generation. The structs are then read by the livepatch module and converted
+ * to the real structs before calling klp_enable_patch().
+ *
+ * TODO make these the official API for klp_enable_patch(). That should
+ * simplify livepatch's interface as well as its data structure lifetime
+ * management.
+ */
+struct klp_func_ext {
+ const char *old_name;
+ void *new_func;
+ unsigned long sympos;
+};
+
+struct klp_object_ext {
+ const char *name;
+ struct klp_func_ext *funcs;
+ struct klp_callbacks callbacks;
+ unsigned int nr_funcs;
+};
+
+#endif /* _LINUX_LIVEPATCH_EXTERNAL_H_ */
diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
index 8499f509f03e7..51ad3cf4fa822 100644
--- a/tools/include/linux/string.h
+++ b/tools/include/linux/string.h
@@ -44,6 +44,20 @@ static inline bool strstarts(const char *str, const char *prefix)
return strncmp(str, prefix, strlen(prefix)) == 0;
}
+/*
+ * Checks if a string ends with another.
+ */
+static inline bool str_ends_with(const char *str, const char *substr)
+{
+ size_t len = strlen(str);
+ size_t sublen = strlen(substr);
+
+ if (sublen > len)
+ return false;
+
+ return !strcmp(str + len - sublen, substr);
+}
+
extern char * __must_check skip_spaces(const char *);
extern char *strim(char *);
diff --git a/tools/objtool/Build b/tools/objtool/Build
index a3cdf8af6635a..0b01657671d7f 100644
--- a/tools/objtool/Build
+++ b/tools/objtool/Build
@@ -8,8 +8,8 @@ objtool-y += builtin-check.o
objtool-y += elf.o
objtool-y += objtool.o
-objtool-$(BUILD_ORC) += orc_gen.o
-objtool-$(BUILD_ORC) += orc_dump.o
+objtool-$(BUILD_ORC) += orc_gen.o orc_dump.o
+objtool-$(BUILD_KLP) += builtin-klp.o klp-diff.o
objtool-y += libstring.o
objtool-y += libctype.o
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 958761c05b7c3..48928c9bebef1 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -15,13 +15,14 @@ ifeq ($(ARCH_HAS_KLP),y)
HAVE_XXHASH = $(shell echo "int main() {}" | \
$(HOSTCC) -xc - -o /dev/null -lxxhash 2> /dev/null && echo y || echo n)
ifeq ($(HAVE_XXHASH),y)
+ BUILD_KLP := y
LIBXXHASH_CFLAGS := $(shell $(HOSTPKG_CONFIG) libxxhash --cflags 2>/dev/null) \
-DBUILD_KLP
LIBXXHASH_LIBS := $(shell $(HOSTPKG_CONFIG) libxxhash --libs 2>/dev/null || echo -lxxhash)
endif
endif
-export BUILD_ORC
+export BUILD_ORC BUILD_KLP
ifeq ($(srctree),)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index b2c320f701f94..5c72beeaa3a71 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -88,6 +88,46 @@ s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc)
return phys_to_virt(addend);
}
+static void scan_for_insn(struct section *sec, unsigned long offset,
+ unsigned long *insn_off, unsigned int *insn_len)
+{
+ unsigned long o = 0;
+ struct insn insn;
+
+ while (1) {
+
+ insn_decode(&insn, sec->data->d_buf + o, sec_size(sec) - o,
+ INSN_MODE_64);
+
+ if (o + insn.length > offset) {
+ *insn_off = o;
+ *insn_len = insn.length;
+ return;
+ }
+
+ o += insn.length;
+ }
+}
+
+u64 arch_adjusted_addend(struct reloc *reloc)
+{
+ unsigned int type = reloc_type(reloc);
+ s64 addend = reloc_addend(reloc);
+ unsigned long insn_off;
+ unsigned int insn_len;
+
+ if (type == R_X86_64_PLT32)
+ return addend + 4;
+
+ if (type != R_X86_64_PC32 || !is_text_sec(reloc->sec->base))
+ return addend;
+
+ scan_for_insn(reloc->sec->base, reloc_offset(reloc),
+ &insn_off, &insn_len);
+
+ return addend + insn_off + insn_len - reloc_offset(reloc);
+}
+
unsigned long arch_jump_destination(struct instruction *insn)
{
return insn->offset + insn->len + insn->immediate;
diff --git a/tools/objtool/builtin-klp.c b/tools/objtool/builtin-klp.c
new file mode 100644
index 0000000000000..9b13dd1182af1
--- /dev/null
+++ b/tools/objtool/builtin-klp.c
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <subcmd/parse-options.h>
+#include <string.h>
+#include <stdlib.h>
+#include <objtool/builtin.h>
+#include <objtool/objtool.h>
+#include <objtool/klp.h>
+
+struct subcmd {
+ const char *name;
+ const char *description;
+ int (*fn)(int, const char **);
+};
+
+static struct subcmd subcmds[] = {
+ { "diff", "Generate binary diff of two object files", cmd_klp_diff, },
+};
+
+static void cmd_klp_usage(void)
+{
+ fprintf(stderr, "usage: objtool klp <subcommand> [<options>]\n\n");
+ fprintf(stderr, "Subcommands:\n");
+
+ for (int i = 0; i < ARRAY_SIZE(subcmds); i++) {
+ struct subcmd *cmd = &subcmds[i];
+
+ fprintf(stderr, " %s\t%s\n", cmd->name, cmd->description);
+ }
+
+ exit(1);
+}
+
+int cmd_klp(int argc, const char **argv)
+{
+ argc--;
+ argv++;
+
+ if (!argc)
+ cmd_klp_usage();
+
+ if (argc) {
+ for (int i = 0; i < ARRAY_SIZE(subcmds); i++) {
+ struct subcmd *cmd = &subcmds[i];
+
+ if (!strcmp(cmd->name, argv[0]))
+ return cmd->fn(argc, argv);
+ }
+ }
+
+ cmd_klp_usage();
+ return 0;
+}
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 5eb0c92aa9fe5..6eaae3c007486 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -185,20 +185,6 @@ static bool is_sibling_call(struct instruction *insn)
return (is_static_jump(insn) && insn_call_dest(insn));
}
-/*
- * Checks if a string ends with another.
- */
-static bool str_ends_with(const char *s, const char *sub)
-{
- const int slen = strlen(s);
- const int sublen = strlen(sub);
-
- if (sublen > slen)
- return 0;
-
- return !memcmp(s + slen - sublen, sub, sublen);
-}
-
/*
* Checks if a function is a Rust "noreturn" one.
*/
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index 0119b3b4c5540..e1daae0630be9 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -288,6 +288,18 @@ struct symbol *find_symbol_by_name(const struct elf *elf, const char *name)
return NULL;
}
+struct symbol *find_global_symbol_by_name(const struct elf *elf, const char *name)
+{
+ struct symbol *sym;
+
+ elf_hash_for_each_possible(symbol_name, sym, name_hash, str_hash(name)) {
+ if (!strcmp(sym->name, name) && !is_local_sym(sym))
+ return sym;
+ }
+
+ return NULL;
+}
+
struct reloc *find_reloc_by_dest_range(const struct elf *elf, struct section *sec,
unsigned long offset, unsigned int len)
{
@@ -475,6 +487,8 @@ static int elf_add_symbol(struct elf *elf, struct symbol *sym)
else
entry = &sym->sec->symbol_list;
list_add(&sym->list, entry);
+
+ list_add_tail(&sym->global_list, &elf->symbols);
elf_hash_add(symbol, &sym->hash, sym->idx);
elf_hash_add(symbol_name, &sym->name_hash, str_hash(sym->name));
@@ -531,6 +545,9 @@ static int read_symbols(struct elf *elf)
ERROR_GLIBC("calloc");
return -1;
}
+
+ INIT_LIST_HEAD(&elf->symbols);
+
for (i = 0; i < symbols_nr; i++) {
sym = &elf->symbol_data[i];
@@ -639,7 +656,7 @@ static int mark_group_syms(struct elf *elf)
return -1;
}
- list_for_each_entry(sec, &elf->sections, list) {
+ for_each_sec(elf, sec) {
if (sec->sh.sh_type == SHT_GROUP &&
sec->sh.sh_link == symtab->idx) {
sym = find_symbol_by_index(elf, sec->sh.sh_info);
@@ -1224,6 +1241,8 @@ struct elf *elf_create_file(GElf_Ehdr *ehdr, const char *name)
return NULL;
}
+ INIT_LIST_HEAD(&elf->symbols);
+
if (!elf_alloc_hash(section, 1000) ||
!elf_alloc_hash(section_name, 1000) ||
!elf_alloc_hash(symbol, 10000) ||
diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/objtool/arch.h
index a4502947307a4..d89f8b5ec14e3 100644
--- a/tools/objtool/include/objtool/arch.h
+++ b/tools/objtool/include/objtool/arch.h
@@ -84,6 +84,7 @@ bool arch_callee_saved_reg(unsigned char reg);
unsigned long arch_jump_destination(struct instruction *insn);
s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc);
+u64 arch_adjusted_addend(struct reloc *reloc);
const char *arch_nop_insn(int len);
const char *arch_ret_insn(int len);
diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/include/objtool/builtin.h
index cee9fc0318777..bb0b25eb08ba4 100644
--- a/tools/objtool/include/objtool/builtin.h
+++ b/tools/objtool/include/objtool/builtin.h
@@ -53,4 +53,6 @@ int objtool_run(int argc, const char **argv);
int make_backup(void);
+int cmd_klp(int argc, const char **argv);
+
#endif /* _BUILTIN_H */
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index a1f1762f89c49..e2cd817fca522 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -18,6 +18,7 @@
#include <objtool/checksum_types.h>
#include <arch/elf.h>
+#define SEC_NAME_LEN 1024
#define SYM_NAME_LEN 512
#define bswap_if_needed(elf, val) __bswap_if_needed(&elf->ehdr, val)
@@ -53,10 +54,12 @@ struct section {
bool _changed, text, rodata, noinstr, init, truncate;
struct reloc *relocs;
unsigned long nr_alloc_relocs;
+ struct section *twin;
};
struct symbol {
struct list_head list;
+ struct list_head global_list;
struct rb_node node;
struct elf_hash_node hash;
struct elf_hash_node name_hash;
@@ -83,10 +86,13 @@ struct symbol {
u8 cold : 1;
u8 prefix : 1;
u8 debug_checksum : 1;
+ u8 changed : 1;
+ u8 included : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
struct checksum csum;
+ struct symbol *twin, *clone;
};
struct reloc {
@@ -104,6 +110,7 @@ struct elf {
const char *name, *tmp_name;
unsigned int num_files;
struct list_head sections;
+ struct list_head symbols;
unsigned long num_relocs;
int symbol_bits;
@@ -179,6 +186,7 @@ struct section *find_section_by_name(const struct elf *elf, const char *name);
struct symbol *find_func_by_offset(struct section *sec, unsigned long offset);
struct symbol *find_symbol_by_offset(struct section *sec, unsigned long offset);
struct symbol *find_symbol_by_name(const struct elf *elf, const char *name);
+struct symbol *find_global_symbol_by_name(const struct elf *elf, const char *name);
struct symbol *find_symbol_containing(const struct section *sec, unsigned long offset);
int find_symbol_hole_containing(const struct section *sec, unsigned long offset);
struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *sec, unsigned long offset);
@@ -448,22 +456,48 @@ static inline void set_sym_next_reloc(struct reloc *reloc, struct reloc *next)
#define sec_for_each_sym(sec, sym) \
list_for_each_entry(sym, &sec->symbol_list, list)
+#define sec_prev_sym(sym) \
+ sym->sec && sym->list.prev != &sym->sec->symbol_list ? \
+ list_prev_entry(sym, list) : NULL
+
#define for_each_sym(elf, sym) \
- for (struct section *__sec, *__fake = (struct section *)1; \
- __fake; __fake = NULL) \
- for_each_sec(elf, __sec) \
- sec_for_each_sym(__sec, sym)
+ list_for_each_entry(sym, &elf->symbols, global_list)
+
+#define for_each_sym_continue(elf, sym) \
+ list_for_each_entry_continue(sym, &elf->symbols, global_list)
+
+#define rsec_next_reloc(rsec, reloc) \
+ reloc_idx(reloc) < sec_num_entries(rsec) - 1 ? reloc + 1 : NULL
#define for_each_reloc(rsec, reloc) \
- for (int __i = 0, __fake = 1; __fake; __fake = 0) \
- for (reloc = rsec->relocs; \
- __i < sec_num_entries(rsec); \
- __i++, reloc++)
+ for (reloc = rsec->relocs; reloc; reloc = rsec_next_reloc(rsec, reloc))
#define for_each_reloc_from(rsec, reloc) \
- for (int __i = reloc_idx(reloc); \
- __i < sec_num_entries(rsec); \
- __i++, reloc++)
+ for (; reloc; reloc = rsec_next_reloc(rsec, reloc))
+
+#define for_each_reloc_continue(rsec, reloc) \
+ for (reloc = rsec_next_reloc(rsec, reloc); reloc; \
+ reloc = rsec_next_reloc(rsec, reloc))
+
+#define sym_for_each_reloc(elf, sym, reloc) \
+ for (reloc = find_reloc_by_dest_range(elf, sym->sec, \
+ sym->offset, sym->len); \
+ reloc && reloc_offset(reloc) < sym->offset + sym->len; \
+ reloc = rsec_next_reloc(sym->sec->rsec, reloc))
+
+static inline struct symbol *get_func_prefix(struct symbol *func)
+{
+ struct symbol *prev;
+
+ if (!is_func_sym(func))
+ return NULL;
+
+ prev = sec_prev_sym(func);
+ if (prev && is_prefix_func(prev))
+ return prev;
+
+ return NULL;
+}
#define OFFSET_STRIDE_BITS 4
#define OFFSET_STRIDE (1UL << OFFSET_STRIDE_BITS)
diff --git a/tools/objtool/include/objtool/klp.h b/tools/objtool/include/objtool/klp.h
new file mode 100644
index 0000000000000..07928fac059b5
--- /dev/null
+++ b/tools/objtool/include/objtool/klp.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _OBJTOOL_KLP_H
+#define _OBJTOOL_KLP_H
+
+/*
+ * __klp_objects and __klp_funcs are created by klp diff and used by the patch
+ * module init code to build the klp_patch, klp_object and klp_func structs
+ * needed by the livepatch API.
+ */
+#define KLP_OBJECTS_SEC "__klp_objects"
+#define KLP_FUNCS_SEC "__klp_funcs"
+
+/*
+ * __klp_relocs is an intermediate section which are created by klp diff and
+ * converted into KLP symbols/relas by "objtool klp post-link". This is needed
+ * to work around the linker, which doesn't preserve SHN_LIVEPATCH or
+ * SHF_RELA_LIVEPATCH, nor does it support having two RELA sections for a
+ * single PROGBITS section.
+ */
+#define KLP_RELOCS_SEC "__klp_relocs"
+#define KLP_STRINGS_SEC ".rodata.klp.str1.1"
+
+struct klp_reloc {
+ void *offset;
+ void *sym;
+ u32 type;
+};
+
+int cmd_klp_diff(int argc, const char **argv);
+
+#endif /* _OBJTOOL_KLP_H */
diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/include/objtool/objtool.h
index c0dc86a78ff65..7f70b41d1b8db 100644
--- a/tools/objtool/include/objtool/objtool.h
+++ b/tools/objtool/include/objtool/objtool.h
@@ -39,6 +39,8 @@ struct objtool_file {
struct pv_state *pv_ops;
};
+char *top_level_dir(const char *file);
+
struct objtool_file *objtool_open_read(const char *_objname);
int objtool_pv_add(struct objtool_file *file, int idx, struct symbol *func);
diff --git a/tools/objtool/include/objtool/util.h b/tools/objtool/include/objtool/util.h
new file mode 100644
index 0000000000000..a0180b312f732
--- /dev/null
+++ b/tools/objtool/include/objtool/util.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _UTIL_H
+#define _UTIL_H
+
+#include <objtool/warn.h>
+
+#define snprintf_check(str, size, format, args...) \
+({ \
+ int __ret = snprintf(str, size, format, args); \
+ if (__ret < 0) \
+ ERROR_GLIBC("snprintf"); \
+ else if (__ret >= size) \
+ ERROR("snprintf() failed for '" format "'", args); \
+ else \
+ __ret = 0; \
+ __ret; \
+})
+
+#endif /* _UTIL_H */
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
new file mode 100644
index 0000000000000..0d69b621a26cf
--- /dev/null
+++ b/tools/objtool/klp-diff.c
@@ -0,0 +1,1646 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#define _GNU_SOURCE /* memmem() */
+#include <subcmd/parse-options.h>
+#include <stdlib.h>
+#include <string.h>
+#include <libgen.h>
+#include <stdio.h>
+#include <ctype.h>
+
+#include <objtool/objtool.h>
+#include <objtool/warn.h>
+#include <objtool/arch.h>
+#include <objtool/klp.h>
+#include <objtool/util.h>
+#include <arch/special.h>
+
+#include <linux/objtool_types.h>
+#include <linux/livepatch_external.h>
+#include <linux/stringify.h>
+#include <linux/string.h>
+#include <linux/jhash.h>
+
+#define sizeof_field(TYPE, MEMBER) sizeof((((TYPE *)0)->MEMBER))
+
+struct elfs {
+ struct elf *orig, *patched, *out;
+ const char *modname;
+};
+
+struct export {
+ struct hlist_node hash;
+ char *mod, *sym;
+};
+
+static const char * const klp_diff_usage[] = {
+ "objtool klp diff [<options>] <in1.o> <in2.o> <out.o>",
+ NULL,
+};
+
+static const struct option klp_diff_options[] = {
+ OPT_END(),
+};
+
+static DEFINE_HASHTABLE(exports, 15);
+
+static inline u32 str_hash(const char *str)
+{
+ return jhash(str, strlen(str), 0);
+}
+
+static int read_exports(void)
+{
+ const char *symvers = "Module.symvers";
+ char line[1024], *path = NULL;
+ unsigned int line_num = 1;
+ FILE *file;
+
+ file = fopen(symvers, "r");
+ if (!file) {
+ path = top_level_dir(symvers);
+ if (!path) {
+ ERROR("can't open '%s', \"objtool diff\" should be run from the kernel tree", symvers);
+ return -1;
+ }
+
+ file = fopen(path, "r");
+ if (!file) {
+ ERROR_GLIBC("fopen");
+ return -1;
+ }
+ }
+
+ while (fgets(line, 1024, file)) {
+ char *sym, *mod, *type;
+ struct export *export;
+
+ sym = strchr(line, '\t');
+ if (!sym) {
+ ERROR("malformed Module.symvers (sym) at line %d", line_num);
+ return -1;
+ }
+
+ *sym++ = '\0';
+
+ mod = strchr(sym, '\t');
+ if (!mod) {
+ ERROR("malformed Module.symvers (mod) at line %d", line_num);
+ return -1;
+ }
+
+ *mod++ = '\0';
+
+ type = strchr(mod, '\t');
+ if (!type) {
+ ERROR("malformed Module.symvers (type) at line %d", line_num);
+ return -1;
+ }
+
+ *type++ = '\0';
+
+ if (*sym == '\0' || *mod == '\0') {
+ ERROR("malformed Module.symvers at line %d", line_num);
+ return -1;
+ }
+
+ export = calloc(1, sizeof(*export));
+ if (!export) {
+ ERROR_GLIBC("calloc");
+ return -1;
+ }
+
+ export->mod = strdup(mod);
+ if (!export->mod) {
+ ERROR_GLIBC("strdup");
+ return -1;
+ }
+
+ export->sym = strdup(sym);
+ if (!export->sym) {
+ ERROR_GLIBC("strdup");
+ return -1;
+ }
+
+ hash_add(exports, &export->hash, str_hash(sym));
+ }
+
+ free(path);
+ fclose(file);
+
+ return 0;
+}
+
+static int read_sym_checksums(struct elf *elf)
+{
+ struct section *sec;
+
+ sec = find_section_by_name(elf, ".discard.sym_checksum");
+ if (!sec) {
+ ERROR("'%s' missing .discard.sym_checksum section, file not processed by 'objtool --checksum'?",
+ elf->name);
+ return -1;
+ }
+
+ if (!sec->rsec) {
+ ERROR("missing reloc section for .discard.sym_checksum");
+ return -1;
+ }
+
+ if (sec_size(sec) % sizeof(struct sym_checksum)) {
+ ERROR("struct sym_checksum size mismatch");
+ return -1;
+ }
+
+ for (int i = 0; i < sec_size(sec) / sizeof(struct sym_checksum); i++) {
+ struct sym_checksum *sym_checksum;
+ struct reloc *reloc;
+ struct symbol *sym;
+
+ sym_checksum = (struct sym_checksum *)sec->data->d_buf + i;
+
+ reloc = find_reloc_by_dest(elf, sec, i * sizeof(*sym_checksum));
+ if (!reloc) {
+ ERROR("can't find reloc for sym_checksum[%d]", i);
+ return -1;
+ }
+
+ sym = reloc->sym;
+
+ if (is_sec_sym(sym)) {
+ ERROR("not sure how to handle section %s", sym->name);
+ return -1;
+ }
+
+ if (is_func_sym(sym))
+ sym->csum.checksum = sym_checksum->checksum;
+ }
+
+ return 0;
+}
+
+static struct symbol *first_file_symbol(struct elf *elf)
+{
+ struct symbol *sym;
+
+ for_each_sym(elf, sym) {
+ if (is_file_sym(sym))
+ return sym;
+ }
+
+ return NULL;
+}
+
+static struct symbol *next_file_symbol(struct elf *elf, struct symbol *sym)
+{
+ for_each_sym_continue(elf, sym) {
+ if (is_file_sym(sym))
+ return sym;
+ }
+
+ return NULL;
+}
+
+/*
+ * Certain static local variables should never be correlated. They will be
+ * used in place rather than referencing the originals.
+ */
+static bool is_uncorrelated_static_local(struct symbol *sym)
+{
+ static const char * const vars[] = {
+ "__key.",
+ "__warned.",
+ "__already_done.",
+ "__func__.",
+ "_rs.",
+ "descriptor.",
+ "CSWTCH.",
+ };
+
+ if (!is_object_sym(sym) || !is_local_sym(sym))
+ return false;
+
+ if (!strcmp(sym->sec->name, ".data.once"))
+ return true;
+
+ for (int i = 0; i < ARRAY_SIZE(vars); i++) {
+ if (strstarts(sym->name, vars[i]))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Clang emits several useless .Ltmp_* code labels.
+ */
+static bool is_clang_tmp_label(struct symbol *sym)
+{
+ return sym->type == STT_NOTYPE &&
+ is_text_sec(sym->sec) &&
+ strstarts(sym->name, ".Ltmp") &&
+ isdigit(sym->name[5]);
+}
+
+static bool is_special_section(struct section *sec)
+{
+ static const char * const specials[] = {
+ ".altinstructions",
+ ".smp_locks",
+ "__bug_table",
+ "__ex_table",
+ "__jump_table",
+ "__mcount_loc",
+
+ /*
+ * Extract .static_call_sites here to inherit non-module
+ * preferential treatment. The later static call processing
+ * during klp module build will be skipped when it sees this
+ * section already exists.
+ */
+ ".static_call_sites",
+ };
+
+ static const char * const non_special_discards[] = {
+ ".discard.addressable",
+ ".discard.sym_checksum",
+ };
+
+ if (is_text_sec(sec))
+ return false;
+
+ for (int i = 0; i < ARRAY_SIZE(specials); i++) {
+ if (!strcmp(sec->name, specials[i]))
+ return true;
+ }
+
+ /* Most .discard data sections are special */
+ for (int i = 0; i < ARRAY_SIZE(non_special_discards); i++) {
+ if (!strcmp(sec->name, non_special_discards[i]))
+ return false;
+ }
+
+ return strstarts(sec->name, ".discard.");
+}
+
+/*
+ * These sections are referenced by special sections but aren't considered
+ * special sections themselves.
+ */
+static bool is_special_section_aux(struct section *sec)
+{
+ static const char * const specials_aux[] = {
+ ".altinstr_replacement",
+ ".altinstr_aux",
+ };
+
+ for (int i = 0; i < ARRAY_SIZE(specials_aux); i++) {
+ if (!strcmp(sec->name, specials_aux[i]))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * These symbols should never be correlated, so their local patched versions
+ * are used instead of linking to the originals.
+ */
+static bool dont_correlate(struct symbol *sym)
+{
+ return is_file_sym(sym) ||
+ is_null_sym(sym) ||
+ is_sec_sym(sym) ||
+ is_prefix_func(sym) ||
+ is_uncorrelated_static_local(sym) ||
+ is_clang_tmp_label(sym) ||
+ is_string_sec(sym->sec) ||
+ is_special_section(sym->sec) ||
+ is_special_section_aux(sym->sec) ||
+ strstarts(sym->name, "__initcall__");
+}
+
+/*
+ * For each symbol in the original kernel, find its corresponding "twin" in the
+ * patched kernel.
+ */
+static int correlate_symbols(struct elfs *e)
+{
+ struct symbol *file1_sym, *file2_sym;
+ struct symbol *sym1, *sym2;
+
+ /* Correlate locals */
+ for (file1_sym = first_file_symbol(e->orig),
+ file2_sym = first_file_symbol(e->patched); ;
+ file1_sym = next_file_symbol(e->orig, file1_sym),
+ file2_sym = next_file_symbol(e->patched, file2_sym)) {
+
+ if (!file1_sym && file2_sym) {
+ ERROR("FILE symbol mismatch: NULL != %s", file2_sym->name);
+ return -1;
+ }
+
+ if (file1_sym && !file2_sym) {
+ ERROR("FILE symbol mismatch: %s != NULL", file1_sym->name);
+ return -1;
+ }
+
+ if (!file1_sym)
+ break;
+
+ if (strcmp(file1_sym->name, file2_sym->name)) {
+ ERROR("FILE symbol mismatch: %s != %s", file1_sym->name, file2_sym->name);
+ return -1;
+ }
+
+ file1_sym->twin = file2_sym;
+ file2_sym->twin = file1_sym;
+
+ sym1 = file1_sym;
+
+ for_each_sym_continue(e->orig, sym1) {
+ if (is_file_sym(sym1) || !is_local_sym(sym1))
+ break;
+
+ if (dont_correlate(sym1))
+ continue;
+
+ sym2 = file2_sym;
+ for_each_sym_continue(e->patched, sym2) {
+ if (is_file_sym(sym2) || !is_local_sym(sym2))
+ break;
+
+ if (sym2->twin || dont_correlate(sym2))
+ continue;
+
+ if (strcmp(sym1->demangled_name, sym2->demangled_name))
+ continue;
+
+ sym1->twin = sym2;
+ sym2->twin = sym1;
+ break;
+ }
+ }
+ }
+
+ /* Correlate globals */
+ for_each_sym(e->orig, sym1) {
+ if (sym1->bind == STB_LOCAL)
+ continue;
+
+ sym2 = find_global_symbol_by_name(e->patched, sym1->name);
+
+ if (sym2 && !sym2->twin && !strcmp(sym1->name, sym2->name)) {
+ sym1->twin = sym2;
+ sym2->twin = sym1;
+ }
+ }
+
+ for_each_sym(e->orig, sym1) {
+ if (sym1->twin || dont_correlate(sym1))
+ continue;
+ WARN("no correlation: %s", sym1->name);
+ }
+
+ return 0;
+}
+
+/* "sympos" is used by livepatch to disambiguate duplicate symbol names */
+static unsigned long find_sympos(struct elf *elf, struct symbol *sym)
+{
+ bool vmlinux = str_ends_with(objname, "vmlinux.o");
+ unsigned long sympos = 0, nr_matches = 0;
+ bool has_dup = false;
+ struct symbol *s;
+
+ if (sym->bind != STB_LOCAL)
+ return 0;
+
+ if (vmlinux && sym->type == STT_FUNC) {
+ /*
+ * HACK: Unfortunately, symbol ordering can differ between
+ * vmlinux.o and vmlinux due to the linker script emitting
+ * .text.unlikely* before .text*. Count .text.unlikely* first.
+ *
+ * TODO: Disambiguate symbols more reliably (checksums?)
+ */
+ for_each_sym(elf, s) {
+ if (strstarts(s->sec->name, ".text.unlikely") &&
+ !strcmp(s->name, sym->name)) {
+ nr_matches++;
+ if (s == sym)
+ sympos = nr_matches;
+ else
+ has_dup = true;
+ }
+ }
+ for_each_sym(elf, s) {
+ if (!strstarts(s->sec->name, ".text.unlikely") &&
+ !strcmp(s->name, sym->name)) {
+ nr_matches++;
+ if (s == sym)
+ sympos = nr_matches;
+ else
+ has_dup = true;
+ }
+ }
+ } else {
+ for_each_sym(elf, s) {
+ if (!strcmp(s->name, sym->name)) {
+ nr_matches++;
+ if (s == sym)
+ sympos = nr_matches;
+ else
+ has_dup = true;
+ }
+ }
+ }
+
+ if (!sympos) {
+ ERROR("can't find sympos for %s", sym->name);
+ return ULONG_MAX;
+ }
+
+ return has_dup ? sympos : 0;
+}
+
+static int clone_sym_relocs(struct elfs *e, struct symbol *patched_sym);
+
+static struct symbol *__clone_symbol(struct elf *elf, struct symbol *patched_sym,
+ bool data_too)
+{
+ struct section *out_sec = NULL;
+ unsigned long offset = 0;
+ struct symbol *out_sym;
+
+ if (data_too && !is_undef_sym(patched_sym)) {
+ struct section *patched_sec = patched_sym->sec;
+
+ out_sec = find_section_by_name(elf, patched_sec->name);
+ if (!out_sec) {
+ out_sec = elf_create_section(elf, patched_sec->name, 0,
+ patched_sec->sh.sh_entsize,
+ patched_sec->sh.sh_type,
+ patched_sec->sh.sh_addralign,
+ patched_sec->sh.sh_flags);
+ if (!out_sec)
+ return NULL;
+ }
+
+ if (is_string_sec(patched_sym->sec)) {
+ out_sym = elf_create_section_symbol(elf, out_sec);
+ if (!out_sym)
+ return NULL;
+
+ goto sym_created;
+ }
+
+ if (!is_sec_sym(patched_sym))
+ offset = sec_size(out_sec);
+
+ if (patched_sym->len || is_sec_sym(patched_sym)) {
+ void *data = NULL;
+ size_t size;
+
+ /* bss doesn't have data */
+ if (patched_sym->sec->data->d_buf)
+ data = patched_sym->sec->data->d_buf + patched_sym->offset;
+
+ if (is_sec_sym(patched_sym))
+ size = sec_size(patched_sym->sec);
+ else
+ size = patched_sym->len;
+
+ if (!elf_add_data(elf, out_sec, data, size))
+ return NULL;
+ }
+ }
+
+ out_sym = elf_create_symbol(elf, patched_sym->name, out_sec,
+ patched_sym->bind, patched_sym->type,
+ offset, patched_sym->len);
+ if (!out_sym)
+ return NULL;
+
+sym_created:
+ patched_sym->clone = out_sym;
+ out_sym->clone = patched_sym;
+
+ return out_sym;
+}
+
+/*
+ * Copy a symbol to the output object, optionally including its data and
+ * relocations.
+ */
+static struct symbol *clone_symbol(struct elfs *e, struct symbol *patched_sym,
+ bool data_too)
+{
+ struct symbol *pfx;
+
+ if (patched_sym->clone)
+ return patched_sym->clone;
+
+ /* Make sure the prefix gets cloned first */
+ if (is_func_sym(patched_sym) && data_too) {
+ pfx = get_func_prefix(patched_sym);
+ if (pfx)
+ clone_symbol(e, pfx, true);
+ }
+
+ if (!__clone_symbol(e->out, patched_sym, data_too))
+ return NULL;
+
+ if (data_too && clone_sym_relocs(e, patched_sym))
+ return NULL;
+
+ return patched_sym->clone;
+}
+
+static void mark_included_function(struct symbol *func)
+{
+ struct symbol *pfx;
+
+ func->included = 1;
+
+ /* Include prefix function */
+ pfx = get_func_prefix(func);
+ if (pfx)
+ pfx->included = 1;
+
+ /* Make sure .cold parent+child always stay together */
+ if (func->cfunc && func->cfunc != func)
+ func->cfunc->included = 1;
+ if (func->pfunc && func->pfunc != func)
+ func->pfunc->included = 1;
+}
+
+/*
+ * Copy all changed functions (and their dependencies) from the patched object
+ * to the output object.
+ */
+static int mark_changed_functions(struct elfs *e)
+{
+ struct symbol *sym_orig, *patched_sym;
+ bool changed = false;
+
+ /* Find changed functions */
+ for_each_sym(e->orig, sym_orig) {
+ if (!is_func_sym(sym_orig) || is_prefix_func(sym_orig))
+ continue;
+
+ patched_sym = sym_orig->twin;
+ if (!patched_sym)
+ continue;
+
+ if (sym_orig->csum.checksum != patched_sym->csum.checksum) {
+ patched_sym->changed = 1;
+ mark_included_function(patched_sym);
+ changed = true;
+ }
+ }
+
+ /* Find added functions and print them */
+ for_each_sym(e->patched, patched_sym) {
+ if (!is_func_sym(patched_sym) || is_prefix_func(patched_sym))
+ continue;
+
+ if (!patched_sym->twin) {
+ printf("%s: new function: %s\n", objname, patched_sym->name);
+ mark_included_function(patched_sym);
+ changed = true;
+ }
+ }
+
+ /* Print changed functions */
+ for_each_sym(e->patched, patched_sym) {
+ if (patched_sym->changed)
+ printf("%s: changed function: %s\n", objname, patched_sym->name);
+ }
+
+ return !changed ? -1 : 0;
+}
+
+static int clone_included_functions(struct elfs *e)
+{
+ struct symbol *patched_sym;
+
+ for_each_sym(e->patched, patched_sym) {
+ if (patched_sym->included) {
+ if (!clone_symbol(e, patched_sym, true))
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Determine whether a relocation should reference the section rather than the
+ * underlying symbol.
+ */
+static bool section_reference_needed(struct section *sec)
+{
+ /*
+ * String symbols are zero-length and uncorrelated. It's easier to
+ * deal with them as section symbols.
+ */
+ if (is_string_sec(sec))
+ return true;
+
+ /*
+ * .rodata has mostly anonymous data so there's no way to determine the
+ * length of a needed reference. just copy the whole section if needed.
+ */
+ if (strstarts(sec->name, ".rodata"))
+ return true;
+
+ /* UBSAN anonymous data */
+ if (strstarts(sec->name, ".data..Lubsan") || /* GCC */
+ strstarts(sec->name, ".data..L__unnamed_")) /* Clang */
+ return true;
+
+ return false;
+}
+
+static bool is_reloc_allowed(struct reloc *reloc)
+{
+ return section_reference_needed(reloc->sym->sec) == is_sec_sym(reloc->sym);
+}
+
+static struct export *find_export(struct symbol *sym)
+{
+ struct export *export;
+
+ hash_for_each_possible(exports, export, hash, str_hash(sym->name)) {
+ if (!strcmp(export->sym, sym->name))
+ return export;
+ }
+
+ return NULL;
+}
+
+static const char *__find_modname(struct elfs *e)
+{
+ struct section *sec;
+ char *name;
+
+ sec = find_section_by_name(e->orig, ".modinfo");
+ if (!sec) {
+ ERROR("missing .modinfo section");
+ return NULL;
+ }
+
+ name = memmem(sec->data->d_buf, sec_size(sec), "\0name=", 6);
+ if (name)
+ return name + 6;
+
+ name = strdup(e->orig->name);
+ if (!name) {
+ ERROR_GLIBC("strdup");
+ return NULL;
+ }
+
+ for (char *c = name; *c; c++) {
+ if (*c == '/')
+ name = c + 1;
+ else if (*c == '-')
+ *c = '_';
+ else if (*c == '.') {
+ *c = '\0';
+ break;
+ }
+ }
+
+ return name;
+}
+
+/* Get the object's module name as defined by the kernel (and klp_object) */
+static const char *find_modname(struct elfs *e)
+{
+ const char *modname;
+
+ if (e->modname)
+ return e->modname;
+
+ modname = __find_modname(e);
+ e->modname = modname;
+ return modname;
+}
+
+/*
+ * Copying a function from its native compiled environment to a kernel module
+ * removes its natural access to local functions/variables and unexported
+ * globals. References to such symbols need to be converted to KLP relocs so
+ * the kernel arch relocation code knows to apply them and where to find the
+ * symbols. Particularly, duplicate static symbols need to be disambiguated.
+ */
+static bool klp_reloc_needed(struct reloc *patched_reloc)
+{
+ struct symbol *patched_sym = patched_reloc->sym;
+ struct export *export;
+
+ /* no external symbol to reference */
+ if (dont_correlate(patched_sym))
+ return false;
+
+ /* For included functions, a regular reloc will do. */
+ if (patched_sym->included)
+ return false;
+
+ /*
+ * If exported by a module, it has to be a klp reloc. Thanks to the
+ * clusterfunk that is late module patching, the patch module is
+ * allowed to be loaded before any modules it depends on.
+ *
+ * If exported by vmlinux, a normal reloc will do.
+ */
+ export = find_export(patched_sym);
+ if (export)
+ return strcmp(export->mod, "vmlinux");
+
+ if (!patched_sym->twin) {
+ /*
+ * Presumably the symbol and its reference were added by the
+ * patch. The symbol could be defined in this .o or in another
+ * .o in the patch module.
+ *
+ * This check needs to be *after* the export check due to the
+ * possibility of the patch adding a new UNDEF reference to an
+ * exported symbol.
+ */
+ return false;
+ }
+
+ /* Unexported symbol which lives in the original vmlinux or module. */
+ return true;
+}
+
+static int convert_reloc_sym_to_secsym(struct elf *elf, struct reloc *reloc)
+{
+ struct symbol *sym = reloc->sym;
+ struct section *sec = sym->sec;
+
+ if (!sec->sym && !elf_create_section_symbol(elf, sec))
+ return -1;
+
+ reloc->sym = sec->sym;
+ set_reloc_sym(elf, reloc, sym->idx);
+ set_reloc_addend(elf, reloc, sym->offset + reloc_addend(reloc));
+ return 0;
+}
+
+static int convert_reloc_secsym_to_sym(struct elf *elf, struct reloc *reloc)
+{
+ struct symbol *sym = reloc->sym;
+ struct section *sec = sym->sec;
+
+ /* If the symbol has a dedicated section, it's easy to find */
+ sym = find_symbol_by_offset(sec, 0);
+ if (sym && sym->len == sec_size(sec))
+ goto found_sym;
+
+ /* No dedicated section; find the symbol manually */
+ sym = find_symbol_containing(sec, arch_adjusted_addend(reloc));
+ if (!sym) {
+ /*
+ * This can happen for special section references to weak code
+ * whose symbol has been stripped by the linker.
+ */
+ return -1;
+ }
+
+found_sym:
+ reloc->sym = sym;
+ set_reloc_sym(elf, reloc, sym->idx);
+ set_reloc_addend(elf, reloc, reloc_addend(reloc) - sym->offset);
+ return 0;
+}
+
+/*
+ * Convert a relocation symbol reference to the needed format: either a section
+ * symbol or the underlying symbol itself.
+ */
+static int convert_reloc_sym(struct elf *elf, struct reloc *reloc)
+{
+ if (is_reloc_allowed(reloc))
+ return 0;
+
+ if (section_reference_needed(reloc->sym->sec))
+ return convert_reloc_sym_to_secsym(elf, reloc);
+ else
+ return convert_reloc_secsym_to_sym(elf, reloc);
+}
+
+/*
+ * Convert a regular relocation to a klp relocation (sort of).
+ */
+static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
+ struct section *sec, unsigned long offset,
+ struct export *export)
+{
+ struct symbol *patched_sym = patched_reloc->sym;
+ s64 addend = reloc_addend(patched_reloc);
+ const char *sym_modname, *sym_orig_name;
+ static struct section *klp_relocs;
+ struct symbol *sym, *klp_sym;
+ unsigned long klp_reloc_off;
+ char sym_name[SYM_NAME_LEN];
+ struct klp_reloc klp_reloc;
+ unsigned long sympos;
+
+ if (!patched_sym->twin) {
+ ERROR("unexpected klp reloc for new symbol %s", patched_sym->name);
+ return -1;
+ }
+
+ /*
+ * Keep the original reloc intact for now to avoid breaking objtool run
+ * which relies on proper relocations for many of its features. This
+ * will be disabled later by "objtool klp post-link".
+ *
+ * Convert it to UNDEF (and WEAK to avoid modpost warnings).
+ */
+
+ sym = patched_sym->clone;
+ if (!sym) {
+ /* STB_WEAK: avoid modpost undefined symbol warnings */
+ sym = elf_create_symbol(e->out, patched_sym->name, NULL,
+ STB_WEAK, patched_sym->type, 0, 0);
+ if (!sym)
+ return -1;
+
+ patched_sym->clone = sym;
+ sym->clone = patched_sym;
+ }
+
+ if (!elf_create_reloc(e->out, sec, offset, sym, addend, reloc_type(patched_reloc)))
+ return -1;
+
+ /*
+ * Create the KLP symbol.
+ */
+
+ if (export) {
+ sym_modname = export->mod;
+ sym_orig_name = export->sym;
+ sympos = 0;
+ } else {
+ sym_modname = find_modname(e);
+ if (!sym_modname)
+ return -1;
+
+ sym_orig_name = patched_sym->twin->name;
+ sympos = find_sympos(e->orig, patched_sym->twin);
+ if (sympos == ULONG_MAX)
+ return -1;
+ }
+
+ /* symbol format: .klp.sym.modname.sym_name,sympos */
+ if (snprintf_check(sym_name, SYM_NAME_LEN, KLP_SYM_PREFIX "%s.%s,%ld",
+ sym_modname, sym_orig_name, sympos))
+ return -1;
+
+ klp_sym = find_symbol_by_name(e->out, sym_name);
+ if (!klp_sym) {
+ /* STB_WEAK: avoid modpost undefined symbol warnings */
+ klp_sym = elf_create_symbol(e->out, sym_name, NULL,
+ STB_WEAK, patched_sym->type, 0, 0);
+ if (!klp_sym)
+ return -1;
+ }
+
+ /*
+ * Create the __klp_relocs entry. This will be converted to an actual
+ * KLP rela by "objtool klp post-link".
+ *
+ * This intermediate step is necessary to prevent corruption by the
+ * linker, which doesn't know how to properly handle two rela sections
+ * applying to the same base section.
+ */
+
+ if (!klp_relocs) {
+ klp_relocs = elf_create_section(e->out, KLP_RELOCS_SEC, 0,
+ 0, SHT_PROGBITS, 8, SHF_ALLOC);
+ if (!klp_relocs)
+ return -1;
+ }
+
+ klp_reloc_off = sec_size(klp_relocs);
+ memset(&klp_reloc, 0, sizeof(klp_reloc));
+
+ klp_reloc.type = reloc_type(patched_reloc);
+ if (!elf_add_data(e->out, klp_relocs, &klp_reloc, sizeof(klp_reloc)))
+ return -1;
+
+ /* klp_reloc.offset */
+ if (!sec->sym && !elf_create_section_symbol(e->out, sec))
+ return -1;
+
+ if (!elf_create_reloc(e->out, klp_relocs,
+ klp_reloc_off + offsetof(struct klp_reloc, offset),
+ sec->sym, offset, R_ABS64))
+ return -1;
+
+ /* klp_reloc.sym */
+ if (!elf_create_reloc(e->out, klp_relocs,
+ klp_reloc_off + offsetof(struct klp_reloc, sym),
+ klp_sym, addend, R_ABS64))
+ return -1;
+
+ return 0;
+}
+
+/* Copy a reloc and its symbol to the output object */
+static int clone_reloc(struct elfs *e, struct reloc *patched_reloc,
+ struct section *sec, unsigned long offset)
+{
+ struct symbol *patched_sym = patched_reloc->sym;
+ struct export *export = find_export(patched_sym);
+ long addend = reloc_addend(patched_reloc);
+ struct symbol *out_sym;
+ bool klp;
+
+ if (!is_reloc_allowed(patched_reloc)) {
+ ERROR_FUNC(patched_reloc->sec->base, reloc_offset(patched_reloc),
+ "missing symbol for reference to %s+%ld",
+ patched_sym->name, addend);
+ return -1;
+ }
+
+ klp = klp_reloc_needed(patched_reloc);
+
+ if (klp) {
+ if (clone_reloc_klp(e, patched_reloc, sec, offset, export))
+ return -1;
+
+ return 0;
+ }
+
+ /*
+ * Why !export sets 'data_too':
+ *
+ * Unexported non-klp symbols need to live in the patch module,
+ * otherwise there will be unresolved symbols. Notably, this includes:
+ *
+ * - New functions/data
+ * - String sections
+ * - Special section entries
+ * - Uncorrelated static local variables
+ * - UBSAN sections
+ */
+ out_sym = clone_symbol(e, patched_sym, patched_sym->included || !export);
+ if (!out_sym)
+ return -1;
+
+ /*
+ * For strings, all references use section symbols, thanks to
+ * section_reference_needed(). clone_symbol() has cloned an empty
+ * version of the string section. Now copy the string itself.
+ */
+ if (is_string_sec(patched_sym->sec)) {
+ const char *str = patched_sym->sec->data->d_buf + addend;
+
+ addend = elf_add_string(e->out, out_sym->sec, str);
+ if (addend == -1)
+ return -1;
+ }
+
+ if (!elf_create_reloc(e->out, sec, offset, out_sym, addend,
+ reloc_type(patched_reloc)))
+ return -1;
+
+ return 0;
+}
+
+/* Copy all relocs needed for a symbol's contents */
+static int clone_sym_relocs(struct elfs *e, struct symbol *patched_sym)
+{
+ struct section *patched_rsec = patched_sym->sec->rsec;
+ struct reloc *patched_reloc;
+ unsigned long start, end;
+ struct symbol *out_sym;
+
+ out_sym = patched_sym->clone;
+ if (!out_sym) {
+ ERROR("no clone for %s", patched_sym->name);
+ return -1;
+ }
+
+ if (!patched_rsec)
+ return 0;
+
+ if (!is_sec_sym(patched_sym) && !patched_sym->len)
+ return 0;
+
+ if (is_string_sec(patched_sym->sec))
+ return 0;
+
+ if (is_sec_sym(patched_sym)) {
+ start = 0;
+ end = sec_size(patched_sym->sec);
+ } else {
+ start = patched_sym->offset;
+ end = start + patched_sym->len;
+ }
+
+ for_each_reloc(patched_rsec, patched_reloc) {
+ unsigned long offset;
+
+ if (reloc_offset(patched_reloc) < start ||
+ reloc_offset(patched_reloc) >= end)
+ continue;
+
+ /*
+ * Skip any reloc referencing .altinstr_aux. Its code is
+ * always patched by alternatives. See ALTERNATIVE_TERNARY().
+ */
+ if (patched_reloc->sym->sec &&
+ !strcmp(patched_reloc->sym->sec->name, ".altinstr_aux"))
+ continue;
+
+ if (convert_reloc_sym(e->patched, patched_reloc)) {
+ ERROR_FUNC(patched_rsec->base, reloc_offset(patched_reloc),
+ "failed to convert reloc sym '%s' to its proper format",
+ patched_reloc->sym->name);
+ return -1;
+ }
+
+ offset = out_sym->offset + (reloc_offset(patched_reloc) - patched_sym->offset);
+
+ if (clone_reloc(e, patched_reloc, out_sym->sec, offset))
+ return -1;
+ }
+ return 0;
+
+}
+
+static int create_fake_symbol(struct elf *elf, struct section *sec,
+ unsigned long offset, size_t size)
+{
+ char name[SYM_NAME_LEN];
+ unsigned int type;
+ static int ctr;
+ char *c;
+
+ if (snprintf_check(name, SYM_NAME_LEN, "%s_%d", sec->name, ctr++))
+ return -1;
+
+ for (c = name; *c; c++)
+ if (*c == '.')
+ *c = '_';
+
+ /*
+ * STT_NOTYPE: Prevent objtool from validating .altinstr_replacement
+ * while still allowing objdump to disassemble it.
+ */
+ type = is_text_sec(sec) ? STT_NOTYPE : STT_OBJECT;
+ return elf_create_symbol(elf, name, sec, STB_LOCAL, type, offset, size) ? 0 : -1;
+}
+
+/*
+ * Special sections (alternatives, etc) are basically arrays of structs.
+ * For all the special sections, create a symbol for each struct entry. This
+ * is a bit cumbersome, but it makes the extracting of the individual entries
+ * much more straightforward.
+ *
+ * There are three ways to identify the entry sizes for a special section:
+ *
+ * 1) ELF section header sh_entsize: Ideally this would be used almost
+ * everywhere. But unfortunately the toolchains make it difficult. The
+ * assembler .[push]section directive syntax only takes entsize when
+ * combined with SHF_MERGE. But Clang disallows combining SHF_MERGE with
+ * SHF_WRITE. And some special sections do need to be writable.
+ *
+ * Another place this wouldn't work is .altinstr_replacement, whose entries
+ * don't have a fixed size.
+ *
+ * 2) ANNOTATE_DATA_SPECIAL: This is a lightweight objtool annotation which
+ * points to the beginning of each entry. The size of the entry is then
+ * inferred by the location of the subsequent annotation (or end of
+ * section).
+ *
+ * 3) Simple array of pointers: If the special section is just a basic array of
+ * pointers, the entry size can be inferred by the number of relocations.
+ * No annotations needed.
+ *
+ * Note I also tried to create per-entry symbols at the time of creation, in
+ * the original [inline] asm. Unfortunately, creating uniquely named symbols
+ * is trickier than one might think, especially with Clang inline asm. I
+ * eventually just gave up trying to make that work, in favor of using
+ * ANNOTATE_DATA_SPECIAL and creating the symbols here after the fact.
+ */
+static int create_fake_symbols(struct elf *elf)
+{
+ struct section *sec;
+ struct reloc *reloc;
+
+ /*
+ * 1) Make symbols for all the ANNOTATE_DATA_SPECIAL entries:
+ */
+
+ sec = find_section_by_name(elf, ".discard.annotate_data");
+ if (!sec || !sec->rsec)
+ return 0;
+
+ for_each_reloc(sec->rsec, reloc) {
+ unsigned long offset, size;
+ struct reloc *next_reloc;
+
+ if (annotype(elf, sec, reloc) != ANNOTYPE_DATA_SPECIAL)
+ continue;
+
+ offset = reloc_addend(reloc);
+
+ size = 0;
+ next_reloc = reloc;
+ for_each_reloc_continue(sec->rsec, next_reloc) {
+ if (annotype(elf, sec, next_reloc) != ANNOTYPE_DATA_SPECIAL ||
+ next_reloc->sym->sec != reloc->sym->sec)
+ continue;
+
+ size = reloc_addend(next_reloc) - offset;
+ break;
+ }
+
+ if (!size)
+ size = sec_size(reloc->sym->sec) - offset;
+
+ if (create_fake_symbol(elf, reloc->sym->sec, offset, size))
+ return -1;
+ }
+
+ /*
+ * 2) Make symbols for sh_entsize, and simple arrays of pointers:
+ */
+
+ for_each_sec(elf, sec) {
+ unsigned int entry_size;
+ unsigned long offset;
+
+ if (!is_special_section(sec) || find_symbol_by_offset(sec, 0))
+ continue;
+
+ if (!sec->rsec) {
+ ERROR("%s: missing special section relocations", sec->name);
+ return -1;
+ }
+
+ entry_size = sec->sh.sh_entsize;
+ if (!entry_size) {
+ entry_size = arch_reloc_size(sec->rsec->relocs);
+ if (sec_size(sec) != entry_size * sec_num_entries(sec->rsec)) {
+ ERROR("%s: missing special section entsize or annotations", sec->name);
+ return -1;
+ }
+ }
+
+ for (offset = 0; offset < sec_size(sec); offset += entry_size) {
+ if (create_fake_symbol(elf, sec, offset, entry_size))
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/* Keep a special section entry if it references an included function */
+static bool should_keep_special_sym(struct elf *elf, struct symbol *sym)
+{
+ struct reloc *reloc;
+
+ if (is_sec_sym(sym) || !sym->sec->rsec)
+ return false;
+
+ sym_for_each_reloc(elf, sym, reloc) {
+ if (convert_reloc_sym(elf, reloc))
+ continue;
+
+ if (is_func_sym(reloc->sym) && reloc->sym->included)
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Klp relocations aren't allowed for __jump_table and .static_call_sites if
+ * the referenced symbol lives in a kernel module, because such klp relocs may
+ * be applied after static branch/call init, resulting in code corruption.
+ *
+ * Validate a special section entry to avoid that. Note that an inert
+ * tracepoint is harmless enough, in that case just skip the entry and print a
+ * warning. Otherwise, return an error.
+ *
+ * This is only a temporary limitation which will be fixed when livepatch adds
+ * support for submodules: fully self-contained modules which are embedded in
+ * the top-level livepatch module's data and which can be loaded on demand when
+ * their corresponding to-be-patched module gets loaded. Then klp relocs can
+ * be retired.
+ *
+ * Return:
+ * -1: error: validation failed
+ * 1: warning: tracepoint skipped
+ * 0: success
+ */
+static int validate_special_section_klp_reloc(struct elfs *e, struct symbol *sym)
+{
+ bool static_branch = !strcmp(sym->sec->name, "__jump_table");
+ bool static_call = !strcmp(sym->sec->name, ".static_call_sites");
+ struct symbol *code_sym = NULL;
+ unsigned long code_offset = 0;
+ struct reloc *reloc;
+ int ret = 0;
+
+ if (!static_branch && !static_call)
+ return 0;
+
+ sym_for_each_reloc(e->patched, sym, reloc) {
+ const char *sym_modname;
+ struct export *export;
+
+ /* Static branch/call keys are always STT_OBJECT */
+ if (reloc->sym->type != STT_OBJECT) {
+
+ /* Save code location which can be printed below */
+ if (reloc->sym->type == STT_FUNC && !code_sym) {
+ code_sym = reloc->sym;
+ code_offset = reloc_addend(reloc);
+ }
+
+ continue;
+ }
+
+ if (!klp_reloc_needed(reloc))
+ continue;
+
+ export = find_export(reloc->sym);
+ if (export) {
+ sym_modname = export->mod;
+ } else {
+ sym_modname = find_modname(e);
+ if (!sym_modname)
+ return -1;
+ }
+
+ /* vmlinux keys are ok */
+ if (!strcmp(sym_modname, "vmlinux"))
+ continue;
+
+ if (static_branch) {
+ if (strstarts(reloc->sym->name, "__tracepoint_")) {
+ WARN("%s: disabling unsupported tracepoint %s",
+ code_sym->name, reloc->sym->name + 13);
+ ret = 1;
+ continue;
+ }
+
+ ERROR("%s+0x%lx: unsupported static branch key %s. Use static_key_enabled() instead",
+ code_sym->name, code_offset, reloc->sym->name);
+ return -1;
+ }
+
+ /* static call */
+ if (strstarts(reloc->sym->name, "__SCK__tp_func_")) {
+ ret = 1;
+ continue;
+ }
+
+ ERROR("%s()+0x%lx: unsupported static call key %s. Use KLP_STATIC_CALL() instead",
+ code_sym->name, code_offset, reloc->sym->name);
+ return -1;
+ }
+
+ return ret;
+}
+
+static int clone_special_section(struct elfs *e, struct section *patched_sec)
+{
+ struct symbol *patched_sym;
+
+ /*
+ * Extract all special section symbols (and their dependencies) which
+ * reference included functions.
+ */
+ sec_for_each_sym(patched_sec, patched_sym) {
+ int ret;
+
+ if (!is_object_sym(patched_sym))
+ continue;
+
+ if (!should_keep_special_sym(e->patched, patched_sym))
+ continue;
+
+ ret = validate_special_section_klp_reloc(e, patched_sym);
+ if (ret < 0)
+ return -1;
+ if (ret > 0)
+ continue;
+
+ if (!clone_symbol(e, patched_sym, true))
+ return -1;
+ }
+
+ return 0;
+}
+
+/* Extract only the needed bits from special sections */
+static int clone_special_sections(struct elfs *e)
+{
+ struct section *patched_sec;
+
+ if (create_fake_symbols(e->patched))
+ return -1;
+
+ for_each_sec(e->patched, patched_sec) {
+ if (is_special_section(patched_sec)) {
+ if (clone_special_section(e, patched_sec))
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Create __klp_objects and __klp_funcs sections which are intermediate
+ * sections provided as input to the patch module's init code for building the
+ * klp_patch, klp_object and klp_func structs for the livepatch API.
+ */
+static int create_klp_sections(struct elfs *e)
+{
+ size_t obj_size = sizeof(struct klp_object_ext);
+ size_t func_size = sizeof(struct klp_func_ext);
+ struct section *obj_sec, *funcs_sec, *str_sec;
+ struct symbol *funcs_sym, *str_sym, *sym;
+ char sym_name[SYM_NAME_LEN];
+ unsigned int nr_funcs = 0;
+ const char *modname;
+ void *obj_data;
+ s64 addend;
+
+ obj_sec = elf_create_section_pair(e->out, KLP_OBJECTS_SEC, obj_size, 0, 0);
+ if (!obj_sec)
+ return -1;
+
+ funcs_sec = elf_create_section_pair(e->out, KLP_FUNCS_SEC, func_size, 0, 0);
+ if (!funcs_sec)
+ return -1;
+
+ funcs_sym = elf_create_section_symbol(e->out, funcs_sec);
+ if (!funcs_sym)
+ return -1;
+
+ str_sec = elf_create_section(e->out, KLP_STRINGS_SEC, 0, 0,
+ SHT_PROGBITS, 1,
+ SHF_ALLOC | SHF_STRINGS | SHF_MERGE);
+ if (!str_sec)
+ return -1;
+
+ if (elf_add_string(e->out, str_sec, "") == -1)
+ return -1;
+
+ str_sym = elf_create_section_symbol(e->out, str_sec);
+ if (!str_sym)
+ return -1;
+
+ /* allocate klp_object_ext */
+ obj_data = elf_add_data(e->out, obj_sec, NULL, obj_size);
+ if (!obj_data)
+ return -1;
+
+ modname = find_modname(e);
+ if (!modname)
+ return -1;
+
+ /* klp_object_ext.name */
+ if (strcmp(modname, "vmlinux")) {
+ addend = elf_add_string(e->out, str_sec, modname);
+ if (addend == -1)
+ return -1;
+
+ if (!elf_create_reloc(e->out, obj_sec,
+ offsetof(struct klp_object_ext, name),
+ str_sym, addend, R_ABS64))
+ return -1;
+ }
+
+ /* klp_object_ext.funcs */
+ if (!elf_create_reloc(e->out, obj_sec, offsetof(struct klp_object_ext, funcs),
+ funcs_sym, 0, R_ABS64))
+ return -1;
+
+ for_each_sym(e->out, sym) {
+ unsigned long offset = nr_funcs * func_size;
+ unsigned long sympos;
+ void *func_data;
+
+ if (!is_func_sym(sym) || sym->cold || !sym->clone || !sym->clone->changed)
+ continue;
+
+ /* allocate klp_func_ext */
+ func_data = elf_add_data(e->out, funcs_sec, NULL, func_size);
+ if (!func_data)
+ return -1;
+
+ /* klp_func_ext.old_name */
+ addend = elf_add_string(e->out, str_sec, sym->clone->twin->name);
+ if (addend == -1)
+ return -1;
+
+ if (!elf_create_reloc(e->out, funcs_sec,
+ offset + offsetof(struct klp_func_ext, old_name),
+ str_sym, addend, R_ABS64))
+ return -1;
+
+ /* klp_func_ext.new_func */
+ if (!elf_create_reloc(e->out, funcs_sec,
+ offset + offsetof(struct klp_func_ext, new_func),
+ sym, 0, R_ABS64))
+ return -1;
+
+ /* klp_func_ext.sympos */
+ BUILD_BUG_ON(sizeof(sympos) != sizeof_field(struct klp_func_ext, sympos));
+ sympos = find_sympos(e->orig, sym->clone->twin);
+ if (sympos == ULONG_MAX)
+ return -1;
+ memcpy(func_data + offsetof(struct klp_func_ext, sympos), &sympos,
+ sizeof_field(struct klp_func_ext, sympos));
+
+ nr_funcs++;
+ }
+
+ /* klp_object_ext.nr_funcs */
+ BUILD_BUG_ON(sizeof(nr_funcs) != sizeof_field(struct klp_object_ext, nr_funcs));
+ memcpy(obj_data + offsetof(struct klp_object_ext, nr_funcs), &nr_funcs,
+ sizeof_field(struct klp_object_ext, nr_funcs));
+
+ /*
+ * Find callback pointers created by KLP_PRE_PATCH_CALLBACK() and
+ * friends, and add them to the klp object.
+ */
+
+ if (snprintf_check(sym_name, SYM_NAME_LEN, KLP_PRE_PATCH_PREFIX "%s", modname))
+ return -1;
+
+ sym = find_symbol_by_name(e->out, sym_name);
+ if (sym) {
+ struct reloc *reloc;
+
+ reloc = find_reloc_by_dest(e->out, sym->sec, sym->offset);
+
+ if (!elf_create_reloc(e->out, obj_sec,
+ offsetof(struct klp_object_ext, callbacks) +
+ offsetof(struct klp_callbacks, pre_patch),
+ reloc->sym, reloc_addend(reloc), R_ABS64))
+ return -1;
+ }
+
+ if (snprintf_check(sym_name, SYM_NAME_LEN, KLP_POST_PATCH_PREFIX "%s", modname))
+ return -1;
+
+ sym = find_symbol_by_name(e->out, sym_name);
+ if (sym) {
+ struct reloc *reloc;
+
+ reloc = find_reloc_by_dest(e->out, sym->sec, sym->offset);
+
+ if (!elf_create_reloc(e->out, obj_sec,
+ offsetof(struct klp_object_ext, callbacks) +
+ offsetof(struct klp_callbacks, post_patch),
+ reloc->sym, reloc_addend(reloc), R_ABS64))
+ return -1;
+ }
+
+ if (snprintf_check(sym_name, SYM_NAME_LEN, KLP_PRE_UNPATCH_PREFIX "%s", modname))
+ return -1;
+
+ sym = find_symbol_by_name(e->out, sym_name);
+ if (sym) {
+ struct reloc *reloc;
+
+ reloc = find_reloc_by_dest(e->out, sym->sec, sym->offset);
+
+ if (!elf_create_reloc(e->out, obj_sec,
+ offsetof(struct klp_object_ext, callbacks) +
+ offsetof(struct klp_callbacks, pre_unpatch),
+ reloc->sym, reloc_addend(reloc), R_ABS64))
+ return -1;
+ }
+
+ if (snprintf_check(sym_name, SYM_NAME_LEN, KLP_POST_UNPATCH_PREFIX "%s", modname))
+ return -1;
+
+ sym = find_symbol_by_name(e->out, sym_name);
+ if (sym) {
+ struct reloc *reloc;
+
+ reloc = find_reloc_by_dest(e->out, sym->sec, sym->offset);
+
+ if (!elf_create_reloc(e->out, obj_sec,
+ offsetof(struct klp_object_ext, callbacks) +
+ offsetof(struct klp_callbacks, post_unpatch),
+ reloc->sym, reloc_addend(reloc), R_ABS64))
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Copy all .modinfo import_ns= tags to ensure all namespaced exported symbols
+ * can be accessed via normal relocs.
+ */
+static int copy_import_ns(struct elfs *e)
+{
+ struct section *patched_sec, *out_sec = NULL;
+ char *import_ns, *data_end;
+
+ patched_sec = find_section_by_name(e->patched, ".modinfo");
+ if (!patched_sec)
+ return 0;
+
+ import_ns = patched_sec->data->d_buf;
+ if (!import_ns)
+ return 0;
+
+ for (data_end = import_ns + sec_size(patched_sec);
+ import_ns < data_end;
+ import_ns += strlen(import_ns) + 1) {
+
+ import_ns = memmem(import_ns, data_end - import_ns, "import_ns=", 10);
+ if (!import_ns)
+ return 0;
+
+ if (!out_sec) {
+ out_sec = find_section_by_name(e->out, ".modinfo");
+ if (!out_sec) {
+ out_sec = elf_create_section(e->out, ".modinfo", 0,
+ patched_sec->sh.sh_entsize,
+ patched_sec->sh.sh_type,
+ patched_sec->sh.sh_addralign,
+ patched_sec->sh.sh_flags);
+ if (!out_sec)
+ return -1;
+ }
+ }
+
+ if (!elf_add_data(e->out, out_sec, import_ns, strlen(import_ns) + 1))
+ return -1;
+ }
+
+ return 0;
+}
+
+int cmd_klp_diff(int argc, const char **argv)
+{
+ struct elfs e = {0};
+
+ argc = parse_options(argc, argv, klp_diff_options, klp_diff_usage, 0);
+ if (argc != 3)
+ usage_with_options(klp_diff_usage, klp_diff_options);
+
+ objname = argv[0];
+
+ e.orig = elf_open_read(argv[0], O_RDONLY);
+ e.patched = elf_open_read(argv[1], O_RDONLY);
+ e.out = NULL;
+
+ if (!e.orig || !e.patched)
+ return -1;
+
+ if (read_exports())
+ return -1;
+
+ if (read_sym_checksums(e.orig))
+ return -1;
+
+ if (read_sym_checksums(e.patched))
+ return -1;
+
+ if (correlate_symbols(&e))
+ return -1;
+
+ if (mark_changed_functions(&e))
+ return 0;
+
+ e.out = elf_create_file(&e.orig->ehdr, argv[2]);
+ if (!e.out)
+ return -1;
+
+ if (clone_included_functions(&e))
+ return -1;
+
+ if (clone_special_sections(&e))
+ return -1;
+
+ if (create_klp_sections(&e))
+ return -1;
+
+ if (copy_import_ns(&e))
+ return -1;
+
+ if (elf_write(e.out))
+ return -1;
+
+ return elf_close(e.out);
+}
diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c
index 5c8b974ad0f9d..c8f611c1320d1 100644
--- a/tools/objtool/objtool.c
+++ b/tools/objtool/objtool.c
@@ -16,8 +16,6 @@
#include <objtool/objtool.h>
#include <objtool/warn.h>
-bool help;
-
static struct objtool_file file;
struct objtool_file *objtool_open_read(const char *filename)
@@ -71,6 +69,39 @@ int objtool_pv_add(struct objtool_file *f, int idx, struct symbol *func)
return 0;
}
+char *top_level_dir(const char *file)
+{
+ ssize_t len, self_len, file_len;
+ char self[PATH_MAX], *str;
+ int i;
+
+ len = readlink("/proc/self/exe", self, sizeof(self) - 1);
+ if (len <= 0)
+ return NULL;
+ self[len] = '\0';
+
+ for (i = 0; i < 3; i++) {
+ char *s = strrchr(self, '/');
+ if (!s)
+ return NULL;
+ *s = '\0';
+ }
+
+ self_len = strlen(self);
+ file_len = strlen(file);
+
+ str = malloc(self_len + file_len + 2);
+ if (!str)
+ return NULL;
+
+ memcpy(str, self, self_len);
+ str[self_len] = '/';
+ strcpy(str + self_len + 1, file);
+
+ return str;
+}
+
+
int main(int argc, const char **argv)
{
static const char *UNUSED = "OBJTOOL_NOT_IMPLEMENTED";
@@ -79,5 +110,11 @@ int main(int argc, const char **argv)
exec_cmd_init("objtool", UNUSED, UNUSED, UNUSED);
pager_init(UNUSED);
+ if (argc > 1 && !strcmp(argv[1], "klp")) {
+ argc--;
+ argv++;
+ return cmd_klp(argc, argv);
+ }
+
return objtool_run(argc, argv);
}
diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh
index 86d64e3ac6f73..e38167ca56a95 100755
--- a/tools/objtool/sync-check.sh
+++ b/tools/objtool/sync-check.sh
@@ -17,6 +17,7 @@ arch/x86/include/asm/emulate_prefix.h
arch/x86/lib/x86-opcode-map.txt
arch/x86/tools/gen-insn-attr-x86.awk
include/linux/interval_tree_generic.h
+include/linux/livepatch_external.h
include/linux/static_call_types.h
"
diff --git a/tools/objtool/weak.c b/tools/objtool/weak.c
index d83f607733b04..d6562f292259e 100644
--- a/tools/objtool/weak.c
+++ b/tools/objtool/weak.c
@@ -8,6 +8,8 @@
#include <stdbool.h>
#include <errno.h>
#include <objtool/objtool.h>
+#include <objtool/arch.h>
+#include <objtool/builtin.h>
#define UNSUPPORTED(name) \
({ \
@@ -24,3 +26,8 @@ int __weak orc_create(struct objtool_file *file)
{
UNSUPPORTED("ORC");
}
+
+int __weak cmd_klp(int argc, const char **argv)
+{
+ UNSUPPORTED("klp");
+}
--
2.50.0
^ permalink raw reply related
* [PATCH v4 50/63] objtool/klp: Add --debug-checksum=<funcs> to show per-instruction checksums
From: Josh Poimboeuf @ 2025-09-17 16:03 UTC (permalink / raw)
To: x86
Cc: linux-kernel, Petr Mladek, Miroslav Benes, Joe Lawrence,
live-patching, Song Liu, laokz, Jiri Kosina,
Marcos Paulo de Souza, Weinan Liu, Fazla Mehrab, Chen Zhongjin,
Puranjay Mohan, Dylan Hatch, Peter Zijlstra
In-Reply-To: <cover.1758067942.git.jpoimboe@kernel.org>
Add a --debug-checksum=<funcs> option to the check subcommand to print
the calculated checksum of each instruction in the given functions.
This is useful for determining where two versions of a function begin to
diverge.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
---
tools/objtool/builtin-check.c | 6 ++++
tools/objtool/check.c | 42 ++++++++++++++++++++++++
tools/objtool/include/objtool/builtin.h | 1 +
tools/objtool/include/objtool/checksum.h | 1 +
tools/objtool/include/objtool/elf.h | 1 +
tools/objtool/include/objtool/warn.h | 19 +++++++++++
6 files changed, 70 insertions(+)
diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c
index 101f05606a990..b20b0077449b2 100644
--- a/tools/objtool/builtin-check.c
+++ b/tools/objtool/builtin-check.c
@@ -94,6 +94,7 @@ static const struct option check_options[] = {
OPT_GROUP("Options:"),
OPT_BOOLEAN(0, "backtrace", &opts.backtrace, "unwind on error"),
OPT_BOOLEAN(0, "backup", &opts.backup, "create backup (.orig) file on warning/error"),
+ OPT_STRING(0, "debug-checksum", &opts.debug_checksum, "funcs", "enable checksum debug output"),
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
OPT_BOOLEAN(0, "link", &opts.link, "object is a linked object"),
OPT_BOOLEAN(0, "module", &opts.module, "object is part of a kernel module"),
@@ -168,6 +169,11 @@ static bool opts_valid(void)
}
#endif
+ if (opts.debug_checksum && !opts.checksum) {
+ ERROR("--debug-checksum requires --checksum");
+ return false;
+ }
+
if (opts.checksum ||
opts.hack_jump_label ||
opts.hack_noinstr ||
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index fec53407428e2..5eb0c92aa9fe5 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3590,6 +3590,44 @@ static bool skip_alt_group(struct instruction *insn)
return alt_insn->type == INSN_CLAC || alt_insn->type == INSN_STAC;
}
+static int checksum_debug_init(struct objtool_file *file)
+{
+ char *dup, *s;
+
+ if (!opts.debug_checksum)
+ return 0;
+
+ dup = strdup(opts.debug_checksum);
+ if (!dup) {
+ ERROR_GLIBC("strdup");
+ return -1;
+ }
+
+ s = dup;
+ while (*s) {
+ struct symbol *func;
+ char *comma;
+
+ comma = strchr(s, ',');
+ if (comma)
+ *comma = '\0';
+
+ func = find_symbol_by_name(file->elf, s);
+ if (!func || !is_func_sym(func))
+ WARN("--debug-checksum: can't find '%s'", s);
+ else
+ func->debug_checksum = 1;
+
+ if (!comma)
+ break;
+
+ s = comma + 1;
+ }
+
+ free(dup);
+ return 0;
+}
+
static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
struct instruction *insn)
{
@@ -4828,6 +4866,10 @@ int check(struct objtool_file *file)
cfi_hash_add(&init_cfi);
cfi_hash_add(&func_cfi);
+ ret = checksum_debug_init(file);
+ if (ret)
+ goto out;
+
ret = decode_sections(file);
if (ret)
goto out;
diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/include/objtool/builtin.h
index 338bdab6b9ad8..cee9fc0318777 100644
--- a/tools/objtool/include/objtool/builtin.h
+++ b/tools/objtool/include/objtool/builtin.h
@@ -32,6 +32,7 @@ struct opts {
/* options: */
bool backtrace;
bool backup;
+ const char *debug_checksum;
bool dryrun;
bool link;
bool mnop;
diff --git a/tools/objtool/include/objtool/checksum.h b/tools/objtool/include/objtool/checksum.h
index 927ca74b5c39e..7fe21608722ac 100644
--- a/tools/objtool/include/objtool/checksum.h
+++ b/tools/objtool/include/objtool/checksum.h
@@ -19,6 +19,7 @@ static inline void checksum_update(struct symbol *func,
const void *data, size_t size)
{
XXH3_64bits_update(func->csum.state, data, size);
+ dbg_checksum(func, insn, XXH3_64bits_digest(func->csum.state));
}
static inline void checksum_finish(struct symbol *func)
diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/objtool/elf.h
index bc7d8a6167f8f..a1f1762f89c49 100644
--- a/tools/objtool/include/objtool/elf.h
+++ b/tools/objtool/include/objtool/elf.h
@@ -82,6 +82,7 @@ struct symbol {
u8 nocfi : 1;
u8 cold : 1;
u8 prefix : 1;
+ u8 debug_checksum : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
diff --git a/tools/objtool/include/objtool/warn.h b/tools/objtool/include/objtool/warn.h
index cb8fe846d9ddd..29173a1368d79 100644
--- a/tools/objtool/include/objtool/warn.h
+++ b/tools/objtool/include/objtool/warn.h
@@ -102,4 +102,23 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define ERROR_FUNC(sec, offset, format, ...) __WARN_FUNC(ERROR_STR, sec, offset, format, ##__VA_ARGS__)
#define ERROR_INSN(insn, format, ...) WARN_FUNC(insn->sec, insn->offset, format, ##__VA_ARGS__)
+
+#define __dbg(format, ...) \
+ fprintf(stderr, \
+ "DEBUG: %s%s" format "\n", \
+ objname ?: "", \
+ objname ? ": " : "", \
+ ##__VA_ARGS__)
+
+#define dbg_checksum(func, insn, checksum) \
+({ \
+ if (unlikely(insn->sym && insn->sym->pfunc && \
+ insn->sym->pfunc->debug_checksum)) { \
+ char *insn_off = offstr(insn->sec, insn->offset); \
+ __dbg("checksum: %s %s %016lx", \
+ func->name, insn_off, checksum); \
+ free(insn_off); \
+ } \
+})
+
#endif /* _WARN_H */
--
2.50.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox