[PATCH v4 00/14] LoongArch: Add livepatch build (KLP) support

[PATCH v4 00/14] LoongArch: Add livepatch build (KLP) support

[George Guo]

This series adds LoongArch support for the klp-build livepatch tooling,
enabling automated livepatch module generation using objtool on
LoongArch.

It is based on Josh Poimboeuf's klp-build series [1] (see base-commit
below, unchanged from v1-v3) and extends it with the LoongArch-specific
objtool and build pieces, plus the toolchain and relocation fixes
required to make livepatch modules load, generate, and run correctly
under both GCC and Clang.

Collaboration
=============

Patch 1 ("objtool/LoongArch: Add arch_adjusted_addend() for KLP support")
and patch 7 ("LoongArch: Add livepatch build (KLP) support") overlap with
WangYuli's earlier LoongArch KLP work [2]. Both now carry
Co-developed-by/Signed-off-by: WangYuli.

WangYuli, please check that the attribution is recorded correctly. If you
consider any other patch in this series to overlap your work, point it out
and I will add the tags in the final version.

Overview
========

 - Patches 1-8: the LoongArch objtool hooks, special-section marking
   (including the inline ALTERNATIVE() replacement), the combined
   -fPIC / -mno-direct-extern-access build fix, EFI linking, the syscall
   patching macro, and wiring up LoongArch klp-build.
 - Patches 9-14: objtool/klp-diff fixes found by running the full
   klp-build flow under GCC and Clang.

Testing
=======

The klp-build integration tests were run on LoongArch under both GCC and
Clang: cmdline-string, proc-version, data-new, gcc-static-local-var-6,
syscall, new-globals, new-function, and shadow-newpid. All of them now
generate a livepatch module, load it, and run.

new-function and shadow-newpid did not build under Clang before this
version; the new objtool/klp-diff patches fix the Clang-specific
failures the suite surfaced. Joe Lawrence's test case was also
reproduced end to end on a Clang-built kernel in a VM (patch load,
tracepoint toggle, clean unload).

Changes since v3 [3]
====================

 - Merged the two build-flag patches (-mno-direct-extern-access and
   -fPIC) into one. The combined patch is one complete thought: for
   LoongArch KLP builds, replace the arch's KBUILD_CFLAGS_KERNEL (dropping
   -fPIE and -mdirect-extern-access) and add -fPIC via KCFLAGS. (Joe
   Lawrence)
 - Moved the LoongArch config checks (CONFIG_RELOCATABLE,
   CONFIG_AS_HAS_EXPLICIT_RELOCS) into validate_config(). (Joe Lawrence)
 - Patch 4 (EFI): fixed the changelog wording and added the
   "/* from the EFI stub */" comment on the wildcard, matching arm64 and
   riscv. (Joe Lawrence)
 - Patch 6 (syscall macro): pad the line continuations with tabs. (Joe
   Lawrence)
 - New patch 12: enable inline ALTERNATIVE() cloning. Joe hit
   "can't find new instruction" livepatching a function that contains an
   ALTERNATIVE(). Mark the .subsection 1 replacement with
   ANNOTATE_DATA_SPECIAL (patch 2) and define ARCH_HAS_INLINE_ALTS for
   LoongArch objtool (patch 12), mirroring arm64. (Joe Lawrence)
 - New patch 13: keep the tablejump annotation table entries. klp-diff
   dropped the table-side word of each .discard.tablejump_annotate pair,
   which crashed objtool on the patch module.
 - New patch 14: rewrite PC-relative data references to GOT. File-local
   static data uses a direct pcalau12i/addi.d pair that cannot reach
   vmlinux from the far livepatch module; convert it to a GOT-indirect
   load when the klp relocation is created. This affects GCC and Clang.

On patch 11 (fold paired ADD/SUB relocations): Clang's integrated
assembler emits the ADD64/SUB64 pair for a "sym - ." whose symbol is
undefined in the current object, where GAS with -mthin-add-sub emits a
single R_LARCH_64_PCREL. Reported to LLVM as issue #210052 [4]. The fold
is kept regardless, since shipping Clang emits the pair.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/log/?h=klp-build-arm64
[2] https://lore.kernel.org/loongarch/20251203091039.125259-1-wangyuli@aosc.io/
[3] https://lore.kernel.org/all/20260707072031.231066-1-dongtai.guo@linux.dev/
[4] https://github.com/llvm/llvm-project/issues/210052

George Guo (14):
  objtool/LoongArch: Add arch_adjusted_addend() for KLP support
  LoongArch: Mark special sections for KLP support
  livepatch/klp-build: use -fPIC and drop direct-extern-access on
    LoongArch
  LoongArch: Fix EFI linking with -fdata-sections
  objtool/klp: Add LoongArch jump opcode bytes support
  klp-build: Add LoongArch syscall patching macro
  LoongArch: Add livepatch build (KLP) support
  LoongArch: Select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY
  objtool/klp: Convert local label references
  objtool/klp: Fix ANNOTATE_DATA_SPECIAL parsing for local label
    references
  objtool/klp: Fold LoongArch paired ADD/SUB relocations into PCREL
  objtool/LoongArch: Enable inline alternative cloning for KLP
  objtool/klp: Keep LoongArch tablejump annotation table entries
  objtool/klp: Rewrite PC-relative data references to GOT on LoongArch

 arch/loongarch/Kconfig                        |   2 +
 arch/loongarch/include/asm/alternative-asm.h  |  10 +-
 arch/loongarch/include/asm/alternative.h      |  11 +-
 arch/loongarch/include/asm/asm-extable.h      |  10 +-
 arch/loongarch/include/asm/bug.h              |   1 +
 arch/loongarch/include/asm/jump_label.h       |   2 +
 arch/loongarch/kernel/asm-offsets.c           |  18 ++
 arch/loongarch/kernel/vmlinux.lds.S           |   2 +-
 include/linux/livepatch_helpers.h             |  22 ++
 scripts/livepatch/klp-build                   |  43 +++-
 tools/arch/loongarch/include/asm/inst.h       |  11 +
 tools/objtool/Makefile                        |   3 +-
 tools/objtool/arch/loongarch/decode.c         | 212 ++++++++++++++++++
 .../objtool/arch/loongarch/include/arch/elf.h |  35 +++
 tools/objtool/klp-diff.c                      | 105 ++++++++-
 15 files changed, 473 insertions(+), 14 deletions(-)


base-commit: 85afaba140a4b9f20fe8c8a64b24fc85f022d981
-- 
2.53.0

[PATCH v4 01/14] objtool/LoongArch: Add arch_adjusted_addend() for KLP support

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

Implement arch_adjusted_addend() to return the relocation addend value.

Co-developed-by: WangYuli <wangyl5933@chinaunicom.cn>
Signed-off-by: WangYuli <wangyl5933@chinaunicom.cn>
Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/arch/loongarch/decode.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/objtool/arch/loongarch/decode.c b/tools/objtool/arch/loongarch/decode.c
index 6cd288150f49..674e4efd138f 100644
--- a/tools/objtool/arch/loongarch/decode.c
+++ b/tools/objtool/arch/loongarch/decode.c
@@ -34,6 +34,11 @@ s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc)
 	return reloc_addend(reloc);
 }
 
+u64 arch_adjusted_addend(struct reloc *reloc)
+{
+	return reloc_addend(reloc);
+}
+
 bool arch_pc_relative_reloc(struct reloc *reloc)
 {
 	return false;
-- 
2.53.0

[PATCH v4 02/14] LoongArch: Mark special sections for KLP support

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

objtool needs to split the kernel's special sections into per-entry
symbols when generating livepatch modules. Mark them so it can:

 - .altinstructions and __ex_table are read-only, so set an explicit
   entry size via SHF_MERGE (ALT_INSTR_SIZE / EXTABLE_SIZE).

 - __bug_table and __jump_table are writable. SHF_MERGE cannot be
   combined with SHF_WRITE (Clang rejects it), so annotate each entry
   with ANNOTATE_DATA_SPECIAL instead, mirroring x86/arm64.

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 arch/loongarch/include/asm/alternative-asm.h | 10 ++++++++--
 arch/loongarch/include/asm/alternative.h     | 11 +++++++++--
 arch/loongarch/include/asm/asm-extable.h     | 10 ++++++++--
 arch/loongarch/include/asm/bug.h             |  1 +
 arch/loongarch/include/asm/jump_label.h      |  2 ++
 arch/loongarch/kernel/asm-offsets.c          | 18 ++++++++++++++++++
 6 files changed, 46 insertions(+), 6 deletions(-)

diff --git a/arch/loongarch/include/asm/alternative-asm.h b/arch/loongarch/include/asm/alternative-asm.h
index 7dc29bd9b2f0..92122a258a8e 100644
--- a/arch/loongarch/include/asm/alternative-asm.h
+++ b/arch/loongarch/include/asm/alternative-asm.h
@@ -5,6 +5,8 @@
 #ifdef __ASSEMBLER__
 
 #include <asm/asm.h>
+#include <generated/asm-offsets.h>
+#include <linux/annotate.h>
 
 /*
  * Issue one struct alt_instr descriptor entry (need to put it into
@@ -33,14 +35,16 @@
 	.fill - (((144f-143f)-(141b-140b)) > 0) * ((144f-143f)-(141b-140b)) / 4, 4, 0x03400000
 142 :
 
-	.pushsection .altinstructions, "a"
+	.pushsection .altinstructions, "aM", @progbits, ALT_INSTR_SIZE
 	altinstruction_entry 140b, 143f, \feature, 142b-140b, 144f-143f
 	.popsection
 
 	.subsection 1
+	ANNOTATE_DATA_SPECIAL
 143 :
 	\newinstr
 144 :
+	ANNOTATE_DATA_SPECIAL_END
 	.previous
 .endm
 
@@ -63,17 +67,19 @@
 		(alt_max_short(new_len1, new_len2) - (old_len)) / 4, 4, 0x03400000
 142 :
 
-	.pushsection .altinstructions, "a"
+	.pushsection .altinstructions, "aM", @progbits, ALT_INSTR_SIZE
 	altinstruction_entry 140b, 143f, \feature1, 142b-140b, 144f-143f, 142b-141b
 	altinstruction_entry 140b, 144f, \feature2, 142b-140b, 145f-144f, 142b-141b
 	.popsection
 
 	.subsection 1
+	ANNOTATE_DATA_SPECIAL
 143 :
 	\newinstr1
 144 :
 	\newinstr2
 145 :
+	ANNOTATE_DATA_SPECIAL_END
 	.previous
 .endm
 
diff --git a/arch/loongarch/include/asm/alternative.h b/arch/loongarch/include/asm/alternative.h
index b5bae21fb3c8..1d3ac2f667d1 100644
--- a/arch/loongarch/include/asm/alternative.h
+++ b/arch/loongarch/include/asm/alternative.h
@@ -7,6 +7,7 @@
 #include <linux/types.h>
 #include <linux/stddef.h>
 #include <linux/stringify.h>
+#include <linux/objtool.h>
 #include <asm/asm.h>
 
 struct alt_instr {
@@ -70,22 +71,28 @@ extern void apply_alternatives(struct alt_instr *start, struct alt_instr *end);
 /* alternative assembly primitive: */
 #define ALTERNATIVE(oldinstr, newinstr, feature)			\
 	OLDINSTR(oldinstr, 1)						\
-	".pushsection .altinstructions,\"a\"\n"				\
+	".pushsection .altinstructions, \"aM\", @progbits, "            \
+		 __stringify(ALT_INSTR_SIZE) "\n"                       \
 	ALTINSTR_ENTRY(feature, 1)					\
 	".popsection\n"							\
 	".subsection 1\n" \
+	ANNOTATE_DATA_SPECIAL "\n" \
 	ALTINSTR_REPLACEMENT(newinstr, feature, 1)			\
+	ANNOTATE_DATA_SPECIAL_END "\n" \
 	".previous\n"
 
 #define ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2)\
 	OLDINSTR_2(oldinstr, 1, 2)					\
-	".pushsection .altinstructions,\"a\"\n"				\
+	".pushsection .altinstructions, \"aM\", @progbits, "            \
+			__stringify(ALT_INSTR_SIZE) "\n"                \
 	ALTINSTR_ENTRY(feature1, 1)					\
 	ALTINSTR_ENTRY(feature2, 2)					\
 	".popsection\n"							\
 	".subsection 1\n" \
+	ANNOTATE_DATA_SPECIAL "\n" \
 	ALTINSTR_REPLACEMENT(newinstr1, feature1, 1)			\
 	ALTINSTR_REPLACEMENT(newinstr2, feature2, 2)			\
+	ANNOTATE_DATA_SPECIAL_END "\n" \
 	".previous\n"
 
 /*
diff --git a/arch/loongarch/include/asm/asm-extable.h b/arch/loongarch/include/asm/asm-extable.h
index d60bdf2e6377..2d7eef7c0b39 100644
--- a/arch/loongarch/include/asm/asm-extable.h
+++ b/arch/loongarch/include/asm/asm-extable.h
@@ -7,10 +7,15 @@
 #define EX_TYPE_UACCESS_ERR_ZERO	2
 #define EX_TYPE_BPF			3
 
+#ifndef COMPILE_OFFSETS
+#include <asm/asm-offsets.h>
+#endif
+
 #ifdef __ASSEMBLER__
 
 #define __ASM_EXTABLE_RAW(insn, fixup, type, data)	\
-	.pushsection	__ex_table, "a";		\
+	.pushsection    __ex_table, "aM", @progbits,    \
+			EXTABLE_SIZE;                   \
 	.balign		4;				\
 	.long		((insn) - .);			\
 	.long		((fixup) - .);			\
@@ -29,7 +34,8 @@
 #include <asm/gpr-num.h>
 
 #define __ASM_EXTABLE_RAW(insn, fixup, type, data)	\
-	".pushsection	__ex_table, \"a\"\n"		\
+	".pushsection   __ex_table, \"aM\", @progbits, "\
+	 __stringify(EXTABLE_SIZE) "\n"                 \
 	".balign	4\n"				\
 	".long		((" insn ") - .)\n"		\
 	".long		((" fixup ") - .)\n"		\
diff --git a/arch/loongarch/include/asm/bug.h b/arch/loongarch/include/asm/bug.h
index d090a5bec5eb..14e3acbaad8d 100644
--- a/arch/loongarch/include/asm/bug.h
+++ b/arch/loongarch/include/asm/bug.h
@@ -25,6 +25,7 @@
 #define __BUG_ENTRY(cond_str, flags)				\
 		.pushsection __bug_table, "aw";			\
 		.align 2;					\
+		__ANNOTATE_DATA_SPECIAL;				\
 	10000:	.long 10001f - .;				\
 		_BUGVERBOSE_LOCATION(WARN_CONDITION_STR(cond_str) __FILE__, __LINE__) \
 		.short flags;					\
diff --git a/arch/loongarch/include/asm/jump_label.h b/arch/loongarch/include/asm/jump_label.h
index 7ef4ae3abf08..d20f0fa24b6b 100644
--- a/arch/loongarch/include/asm/jump_label.h
+++ b/arch/loongarch/include/asm/jump_label.h
@@ -12,6 +12,7 @@
 #include <linux/types.h>
 #include <linux/stringify.h>
 #include <asm/asm.h>
+#include <linux/objtool.h>
 
 #define HAVE_JUMP_LABEL_BATCH
 
@@ -27,6 +28,7 @@
 #define JUMP_TABLE_ENTRY(key, label)			\
 	 ".pushsection	__jump_table, \"aw\"	\n\t"	\
 	 ".align	" __stringify(PTRLOG) "	\n\t"	\
+	 ANNOTATE_DATA_SPECIAL "\n\t"			\
 	 ".long		1b - ., " label " - .	\n\t"	\
 	 JUMP_LABEL_TYPE  key " - .		\n\t"	\
 	 ".popsection				\n\t"
diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c
index 2cc953f113ac..a611cbe557ed 100644
--- a/arch/loongarch/kernel/asm-offsets.c
+++ b/arch/loongarch/kernel/asm-offsets.c
@@ -18,6 +18,8 @@
 #include <asm/ftrace.h>
 #include <asm/sigframe.h>
 #include <vdso/datapage.h>
+#include <asm/alternative.h>
+#include <asm/extable.h>
 
 static void __used output_ptreg_defines(void)
 {
@@ -323,3 +325,19 @@ static void __used output_vdso_defines(void)
 	DEFINE(__VDSO_PAGES, VDSO_NR_PAGES);
 	BLANK();
 }
+
+static void __used output_extable_defines(void)
+{
+	COMMENT("LoongArch exception table entry offsets.");
+
+	DEFINE(EXTABLE_SIZE, sizeof(struct exception_table_entry));
+	BLANK();
+}
+
+static void __used output_alt_instr_defines(void)
+{
+	COMMENT("LoongArch alternative instructions offsets.");
+
+	DEFINE(ALT_INSTR_SIZE, sizeof(struct alt_instr));
+	BLANK();
+}
-- 
2.53.0

[PATCH v4 03/14] livepatch/klp-build: use -fPIC and drop direct-extern-access on LoongArch

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

On LoongArch, the klp relocation machinery redirects a livepatch's
reference to a core-kernel symbol through a GOT entry. klp-build extracts
the patched function into a separate module while the referenced symbol
stays in the core kernel, so the reference must be GOT-indirect for that
redirect to work. Two default compiler behaviours defeat this and make a
loaded livepatch module fault.

The first is a same-unit global (the syscall test patches sys_newuname()
in kernel/sys.c, which reads 'uts_sem', a rw_semaphore defined in the
same file). With CONFIG_RELOCATABLE=y the kernel is built -fPIE. For
a global defined in the same unit, -fPIE emits a direct PC-relative
reference (R_LARCH_PCALA_*) and skips the GOT, while -fPIC routes it
through the GOT (R_LARCH_GOT_PC_*). Once klp-build moves the patched
function into the livepatch module, the direct reference has no GOT
slot to redirect and faults.

The second is an extern global. -mdirect-extern-access replaces GOT-based
external symbol access with direct addressing. A livepatch reference to
such a symbol then has no GOT slot to fix up, and the wrong address faults
the kernel. This optimization was added by commit 38b10b269d04
("LoongArch: Tweak CFLAGS for Clang compatibility") as a nice-to-have that
reduces GOT accesses.

For LoongArch KLP builds, keep both forms GOT-indirect:

 - Add -fPIC via KCFLAGS. -fPIE is not enough; it optimizes away the very
   GOT indirection KLP relies on.
 - Disable direct-extern-access: -mno-direct-extern-access for GCC,
   -fno-direct-access-external-data for Clang.

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 scripts/livepatch/klp-build | 43 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/scripts/livepatch/klp-build b/scripts/livepatch/klp-build
index e83973567c87..27463def08f8 100755
--- a/scripts/livepatch/klp-build
+++ b/scripts/livepatch/klp-build
@@ -278,6 +278,13 @@ validate_config() {
 	[[ -x "$OBJTOOL" ]] && "$OBJTOOL" klp 2>&1 | command grep -q "not implemented" && \
 		die "objtool not built with KLP support; install xxhash-devel/libxxhash-dev (version >= 0.8) and recompile"
 
+	if [[ -v CONFIG_LOONGARCH ]]; then
+		[[ -v CONFIG_AS_HAS_EXPLICIT_RELOCS ]] ||	\
+			die "LoongArch klp-build requires CONFIG_AS_HAS_EXPLICIT_RELOCS=y"
+		[[ -v CONFIG_RELOCATABLE ]] ||			\
+			die "LoongArch klp-build requires CONFIG_RELOCATABLE=y"
+	fi
+
 	return 0
 }
 
@@ -556,6 +563,31 @@ build_kernel() {
 	local log="$TMP_DIR/build.log"
 	local cmd=()
 
+	local ARCH_KBUILD_CFLAGS_KERNEL=""
+	local ARCH_KCFLAGS=""
+
+	# For KLP, LoongArch symbol references must stay GOT-indirect so the klp
+	# relocation machinery can redirect a cross-object reference through a
+	# GOT entry.  Two default behaviours defeat that (CONFIG_RELOCATABLE and
+	# CONFIG_AS_HAS_EXPLICIT_RELOCS are checked in validate_config()):
+	#
+	#  - -fPIE (added under CONFIG_RELOCATABLE) emits a direct PC-relative
+	#    reference for a same-unit global and skips the GOT; -fPIC routes it
+	#    through the GOT.  Add -fPIC via KCFLAGS.
+	#  - -mdirect-extern-access replaces GOT access for an extern global with
+	#    direct addressing.  Disable it.
+	if [[ -v CONFIG_LOONGARCH ]]; then
+		ARCH_KCFLAGS="-fPIC"
+
+		if [[ "${CONFIG_CC_IS_CLANG:-}" == "y" ]]; then
+			ARCH_KBUILD_CFLAGS_KERNEL="-fno-direct-access-external-data"
+		else
+			ARCH_KBUILD_CFLAGS_KERNEL="-mno-direct-extern-access"
+		fi
+
+		status "LoongArch detected: adding $ARCH_KBUILD_CFLAGS_KERNEL to KBUILD_CFLAGS_KERNEL"
+	fi
+
 	cmd=("make")
 
 	# When a patch to a kernel module references a newly created unexported
@@ -581,7 +613,16 @@ build_kernel() {
 		cmd+=("-s")
 	fi
 	cmd+=("-j$JOBS")
-	cmd+=("KCFLAGS=-ffunction-sections -fdata-sections")
+	cmd+=("KCFLAGS=-ffunction-sections -fdata-sections${ARCH_KCFLAGS:+ $ARCH_KCFLAGS}")
+	# -fPIC is added for KLP via KCFLAGS above; the arch adds -fPIE via
+	# KBUILD_CFLAGS_KERNEL, which kbuild places after KCFLAGS on the
+	# built-in compile line.  -fPIC/-fPIE is last-one-wins, so -fPIE would
+	# win.  Setting KBUILD_CFLAGS_KERNEL on the command line replaces the
+	# arch value (not append), which drops -fPIE and lets -fPIC win.  Only
+	# do this when an arch needs it (LoongArch).
+	if [[ -n "$ARCH_KBUILD_CFLAGS_KERNEL" ]]; then
+		cmd+=("KBUILD_CFLAGS_KERNEL=$ARCH_KBUILD_CFLAGS_KERNEL")
+	fi
 	cmd+=("vmlinux")
 	cmd+=("modules")
 
-- 
2.53.0

[PATCH v4 04/14] LoongArch: Fix EFI linking with -fdata-sections

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

When building with -fdata-sections, the .init.bss section gets split
up into a bunch of .init.bss.<var> sections.  Make sure they get linked
into .init.bss.

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 arch/loongarch/kernel/vmlinux.lds.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S
index 840d944c2f73..570f936df34e 100644
--- a/arch/loongarch/kernel/vmlinux.lds.S
+++ b/arch/loongarch/kernel/vmlinux.lds.S
@@ -96,7 +96,7 @@ SECTIONS
 #endif
 
 	.init.bss : {
-		*(.init.bss)
+		*(.init.bss .init.bss.*)	/* from the EFI stub */
 	}
 	. = ALIGN(PECOFF_SEGMENT_ALIGN);
 	__initdata_end = .;
-- 
2.53.0

[PATCH v4 05/14] objtool/klp: Add LoongArch jump opcode bytes support

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

Implement arch_jump_opcode_bytes() for LoongArch so that klp checksums
for jump/call instructions with non-relocated destination offsets are
position-independent, mirroring the x86 and arm64 implementations.

Mask out the embedded branch offset while preserving the opcode and the
register operands, which differ per instruction format:

  - B/BL (reg0i26):                  no regs -> 0xfc000000
  - BEQZ/BNEZ/BCEQZ/BCNEZ (reg1i21): keep rj/cj -> 0xfc0003e0
  - JIRL/BEQ/BNE/.../BGEU (reg2i16): keep rj/rd -> 0xfc0003ff

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/arch/loongarch/decode.c | 41 +++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/tools/objtool/arch/loongarch/decode.c b/tools/objtool/arch/loongarch/decode.c
index 674e4efd138f..12facd0cc8d1 100644
--- a/tools/objtool/arch/loongarch/decode.c
+++ b/tools/objtool/arch/loongarch/decode.c
@@ -432,6 +432,47 @@ unsigned long arch_jump_table_sym_offset(struct reloc *reloc, struct reloc *tabl
 	}
 }
 
+size_t arch_jump_opcode_bytes(struct objtool_file *file, struct instruction *insn,
+			      unsigned char *buf)
+{
+	union loongarch_instruction *code;
+	u32 insn_word;
+
+	insn_word = le32toh(*(u32 *)(insn->sec->data->d_buf + insn->offset));
+	code = (union loongarch_instruction *)&insn_word;
+
+	switch (code->reg0i26_format.opcode) {
+	case b_op:
+	case bl_op:
+		/* reg0i26: 26-bit offset, no register operands */
+		insn_word &= 0xfc000000;
+		break;
+	case beqz_op:
+	case bnez_op:
+	case bceqz_op:		/* == bcnez_op */
+		/* reg1i21: keep opcode + rj/cj at bits[9:5] */
+		insn_word &= 0xfc0003e0;
+		break;
+	case jirl_op:
+	case beq_op:
+	case bne_op:
+	case blt_op:
+	case bge_op:
+	case bltu_op:
+	case bgeu_op:
+		/* reg2i16: keep opcode + rj/rd at bits[9:0] */
+		insn_word &= 0xfc0003ff;
+		break;
+	default:
+		break;
+	}
+
+	insn_word = htole32(insn_word);
+	memcpy(buf, &insn_word, sizeof(insn_word));
+
+	return LOONGARCH_INSN_SIZE;
+}
+
 #ifdef DISAS
 
 int arch_disas_info_init(struct disassemble_info *dinfo)
-- 
2.53.0

[PATCH v4 06/14] klp-build: Add LoongArch syscall patching macro

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

Add LoongArch support for KLP_SYSCALL_DEFINEx(), mirroring the generic
__SYSCALL_DEFINEx() pattern from include/linux/syscalls.h.

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 include/linux/livepatch_helpers.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/include/linux/livepatch_helpers.h b/include/linux/livepatch_helpers.h
index 4b647b83865f..d79e692e382d 100644
--- a/include/linux/livepatch_helpers.h
+++ b/include/linux/livepatch_helpers.h
@@ -91,6 +91,28 @@
 	}								\
 	static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__))
 
+#elif defined(CONFIG_LOONGARCH)
+
+#define __KLP_SYSCALL_DEFINEx(x, name, ...)				\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x,__SC_DECL,__VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\
+	asmlinkage long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__));	\
+	asmlinkage long __attribute__((optimize("-fno-optimize-sibling-calls")))\
+			__se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__))	\
+	{								\
+		long ret = __klp_do_sys##name(__MAP(x,__SC_CAST,__VA_ARGS__));\
+		__MAP(x,__SC_TEST,__VA_ARGS__);				\
+		__PROTECT(x, ret,__MAP(x,__SC_ARGS,__VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__))
+
 #endif
 
 #endif /* _LINUX_LIVEPATCH_HELPERS_H */
-- 
2.53.0

[PATCH v4 07/14] LoongArch: Add livepatch build (KLP) support

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

This allows automated livepatch module generation using objtool.

Co-developed-by: WangYuli <wangyl5933@chinaunicom.cn>
Signed-off-by: WangYuli <wangyl5933@chinaunicom.cn>
Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 arch/loongarch/Kconfig | 1 +
 tools/objtool/Makefile | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 3b042dbb2c41..1dbf51ba9d6a 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -159,6 +159,7 @@ config LOONGARCH
 	select HAVE_IOREMAP_PROT
 	select HAVE_IRQ_EXIT_ON_IRQ_STACK
 	select HAVE_IRQ_TIME_ACCOUNTING
+	select HAVE_KLP_BUILD
 	select HAVE_KPROBES
 	select HAVE_KPROBES_ON_FTRACE
 	select HAVE_KRETPROBES
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 94aabeee9736..83d645675ed9 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -8,7 +8,8 @@ ifeq ($(SRCARCH),x86)
 endif
 
 ifeq ($(SRCARCH),loongarch)
-	BUILD_ORC	   := y
+	BUILD_ORC    := y
+	ARCH_HAS_KLP := y
 endif
 
 ifeq ($(SRCARCH),arm64)
-- 
2.53.0

[PATCH v4 08/14] LoongArch: Select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

arch/loongarch/Makefile uses -fpatchable-function-entry=2 when
CONFIG_DYNAMIC_FTRACE is set, but nothing selects
FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY, so kbuild falls back to
FTRACE_MCOUNT_USE_RECORDMCOUNT and runs recordmcount on every object.
The pass is useless here: there are no _mcount calls to find, and the
real ftrace sites come from the __patchable_function_entries section
anyway.

It is not just wasted time.  recordmcount needs a non-weak symbol in
each text section, to use as the base of the __mcount_loc relocations
it writes (see find_secsym_ndx()).  GAS emits a section symbol for
every section, so GCC objects always have one.  Clang's integrated
assembler drops unreferenced section symbols, so a clang object must
use the function's own symbol -- but find_secsym_ndx() skips weak
symbols.  So with -ffunction-sections (which klp-build uses), a __weak
function like sched_clock is alone in its section with no usable
symbol, and recordmcount fails:

  Cannot find symbol for section 5: .text.sched_clock.
  kernel/sched/build_utility.o: failed

Select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY like arm64 and riscv
do, which disables the recordmcount pass altogether.

Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Suggested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 arch/loongarch/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 1dbf51ba9d6a..a1549003c871 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -88,6 +88,7 @@ config LOONGARCH
 	select CPU_PM
 	select EDAC_SUPPORT
 	select EFI
+	select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if DYNAMIC_FTRACE
 	select GENERIC_ATOMIC64 if 32BIT
 	select GENERIC_CLOCKEVENTS
 	select GENERIC_CMOS_UPDATE
-- 
2.53.0

[PATCH v4 09/14] objtool/klp: Convert local label references

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

Some toolchains reference an object through a local assembler label (.L*)
rather than a section symbol plus offset.  The assembler keeps the label
because a "section + constant offset" reference would go stale under
linker relaxation, while a symbol reference can be recomputed afterwards.
Such labels are assembler-local, absent from kallsyms, and are never
cloned into the livepatch object.

Two toolchains hit this:

  - GCC/GAS on LoongArch references special section entries (__ex_table,
    __bug_table, __jump_table, .altinstructions) through local text
    labels instead of a section symbol.

  - Clang emits a switch jump table through a local label (.LJTI*) in a
    SHF_MERGE|SHF_STRINGS .rodata section, where the table shares the
    section with the livepatch's klp_func name strings.

convert_reloc_secsym_to_sym() only handled the section symbol form and
returned early for a label reference, so the label was never converted
and the reference resolved incorrectly, with no error at build or load
time:

  - GCC: should_keep_special_sym() cannot correlate the entry with an
    included function and silently drops it, so the livepatch module is
    missing the patched function's __ex_table / __bug_table /
    __jump_table entries.

  - Clang: the jump table is not cloned, and its base label resolves into
    the name strings that occupy the same offset; the switch's indirect
    jump then lands on a wild address and the patched function oopses at
    runtime.

Redirect a local label reference off the label, mirroring the section
symbol case:

  - In a text section, to the containing function symbol.  A label with
    no containing function symbol (e.g. hand-written asm in a plain .text
    section) can't be correlated; skip it rather than failing, since such
    entries belong to unchanged code and are dropped anyway.

  - In a non-text section (e.g. the .rodata jump table), to the section
    symbol plus the full offset.

Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Suggested-by: Joe Lawrence <joe.lawrence@redhat.com>
Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/klp-diff.c | 49 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)

diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index b9624bd9439b..3e573b73fe6b 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -1423,8 +1423,55 @@ static int convert_reloc_secsym_to_sym(struct elf *elf, struct reloc *reloc)
 	if (!strcmp(reloc->sec->name, ".rela__patchable_function_entries"))
 		return convert_pfe_reloc(elf, reloc);
 
-	if (!is_sec_sym(sym))
+	if (!is_sec_sym(sym)) {
+		/*
+		 * Most toolchains reference special-section entries via the
+		 * section symbol plus an offset.  GCC/GAS on LoongArch instead
+		 * references a local text label (.L*): LoongArch linker
+		 * relaxation is the reason GAS keeps the label rather than
+		 * reducing it to a section symbol reference.  Such a label is
+		 * never cloned into the livepatch object, so the entry would be
+		 * silently dropped.  Redirect the relocation to the containing
+		 * function, mirroring the section-symbol case below.
+		 */
+		if (is_local_label(sym)) {
+			unsigned long offset = sym->offset + reloc_addend(reloc);
+
+			if (is_text_sec(sec)) {
+				sym = find_symbol_containing_inclusive(sec, offset);
+				if (!sym) {
+					/*
+					 * A local label with no containing function
+					 * symbol (e.g. hand-written asm in a plain .text
+					 * section).  It can't be correlated to a function,
+					 * so skip it rather than failing the build; such
+					 * entries belong to unchanged code and are dropped
+					 * anyway.
+					 */
+					return 1;
+				}
+
+				reloc->sym = sym;
+				set_reloc_sym(elf, reloc, sym->idx);
+				set_reloc_addend(elf, reloc, offset - sym->offset);
+			} else {
+				/*
+				 * A local label in a non-text section, e.g. Clang's
+				 * .LJTI* switch jump table in .rodata.  It isn't
+				 * cloned into the livepatch either, so redirect the
+				 * reloc to the section symbol plus the full offset,
+				 * mirroring the section-symbol case below.
+				 */
+				if (!sec->sym && !elf_create_section_symbol(elf, sec))
+					return -1;
+				reloc->sym = sec->sym;
+				set_reloc_sym(elf, reloc, sec->sym->idx);
+				set_reloc_addend(elf, reloc, offset);
+			}
+		}
+
 		return 0;
+	}
 
 	sym = find_symbol_containing_inclusive(sec, arch_adjusted_addend(reloc));
 	if (!sym) {
-- 
2.53.0

[PATCH v4 10/14] objtool/klp: Fix ANNOTATE_DATA_SPECIAL parsing for local label references

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

create_fake_symbols() takes each annotated entry's offset from the
annotate relocation's addend alone.  That works only when the
relocation points at the special section's section symbol.  GCC/GAS
on LoongArch points it at the entry's local label instead, with a
zero addend: every entry then gets offset 0, the "distance to the
next annotation" size becomes 0, and the fallback sets the size to
the rest of the section.

So every fake symbol starts at offset 0 and covers the whole special
section.  If any entry in the section belongs to a patched function,
should_keep_special_sym() keeps such a symbol, and the output module
gets the entire __bug_table (thousands of entries and klp
relocations) instead of just the patched function's entries.

Compute the offset as sym->offset + addend, which handles both the
section symbol form (sym->offset == 0) and the local label form
(addend == 0).

Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/klp-diff.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 3e573b73fe6b..a952e680e8a6 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -1877,7 +1877,12 @@ static int create_fake_symbols(struct elf *elf)
 		if (annotype(elf, sec, reloc) != ANNOTYPE_DATA_SPECIAL)
 			continue;
 
-		offset = reloc_addend(reloc);
+		/*
+		 * The annotated location may be referenced via the section
+		 * symbol plus addend, or via a local label (GCC/GAS on
+		 * LoongArch): sym->offset + addend covers both forms.
+		 */
+		offset = reloc->sym->offset + reloc_addend(reloc);
 
 		size = 0;
 		next_reloc = reloc;
@@ -1889,7 +1894,7 @@ static int create_fake_symbols(struct elf *elf)
 			    next_reloc->sym->sec != reloc->sym->sec)
 				continue;
 
-			size = reloc_addend(next_reloc) - offset;
+			size = (next_reloc->sym->offset + reloc_addend(next_reloc)) - offset;
 			break;
 		}
 
-- 
2.53.0

[PATCH v4 11/14] objtool/klp: Fold LoongArch paired ADD/SUB relocations into PCREL

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

On LoongArch, the "key - ." field of a __jump_table entry can come out
as two relocations at the same offset, R_LARCH_ADD64 plus
R_LARCH_SUB64.  clang's integrated assembler emits this pair when the
key symbol is not defined in the same translation unit (for example
__tracepoint_netif_rx); GAS, and clang for locally-defined keys,
produce a single R_LARCH_64_PCREL instead, which is why GCC builds do
not hit this.  objtool's elf_create_reloc() allows only one relocation
per offset, so cloning such an entry fails with:

  vmlinux.o: error: objtool: __jump_table_23683+0x8: duplicate reloc

When the SUB half points at the relocation's own position (the
"sym - ." pattern), the pair means the same thing as one PC-relative
relocation.  Add arch_normalize_paired_reloc() which rewrites the ADD
half to R_LARCH_32_PCREL/R_LARCH_64_PCREL and skips the SUB half.  The
LoongArch module loader supports both forms.

Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/arch/loongarch/decode.c         | 62 +++++++++++++++++++
 .../objtool/arch/loongarch/include/arch/elf.h | 18 ++++++
 tools/objtool/klp-diff.c                      | 13 ++++
 3 files changed, 93 insertions(+)

diff --git a/tools/objtool/arch/loongarch/decode.c b/tools/objtool/arch/loongarch/decode.c
index 12facd0cc8d1..273398ea4486 100644
--- a/tools/objtool/arch/loongarch/decode.c
+++ b/tools/objtool/arch/loongarch/decode.c
@@ -39,6 +39,68 @@ u64 arch_adjusted_addend(struct reloc *reloc)
 	return reloc_addend(reloc);
 }
 
+/*
+ * A cross-section difference like the "key - ." field of a __jump_table
+ * entry may come out as a paired R_LARCH_ADD plus R_LARCH_SUB relocation
+ * at the same offset: clang's integrated assembler does this when the
+ * symbol is not defined in the same translation unit (GAS, and clang for
+ * locally-defined symbols, emit a single PCREL instead).  objtool allows
+ * only one relocation per offset, so the pair breaks cloning.
+ *
+ * When the SUB half points at the reloc's own position ("sym - ."), the
+ * pair means the same as a single PC-relative relocation, which the
+ * module loader also supports: rewrite the ADD half to R_LARCH_*_PCREL
+ * and tell the caller to skip the SUB half.
+ *
+ * Return 1 to skip the reloc, 0 to proceed, -1 on error.
+ */
+int arch_normalize_paired_reloc(struct elf *elf, struct reloc *reloc)
+{
+	struct section *rsec = reloc->sec;
+	unsigned int sub_type, pcrel_type;
+	struct reloc *sub, *add;
+
+	switch (reloc_type(reloc)) {
+	case R_LARCH_ADD32:
+		sub_type = R_LARCH_SUB32;
+		pcrel_type = R_LARCH_32_PCREL;
+		break;
+	case R_LARCH_ADD64:
+		sub_type = R_LARCH_SUB64;
+		pcrel_type = R_LARCH_64_PCREL;
+		break;
+	case R_LARCH_SUB32:
+	case R_LARCH_SUB64:
+		/*
+		 * Skip only if the paired ADD (the preceding reloc) has
+		 * been rewritten to PCREL; otherwise leave the pair
+		 * intact so a failed conversion stays loud.
+		 */
+		add = reloc_idx(reloc) ? reloc - 1 : NULL;
+		if (add && reloc_offset(add) == reloc_offset(reloc) &&
+		    (reloc_type(add) == R_LARCH_32_PCREL ||
+		     reloc_type(add) == R_LARCH_64_PCREL))
+			return 1;
+		return 0;
+	default:
+		return 0;
+	}
+
+	/* The paired SUB reloc immediately follows the ADD */
+	sub = rsec_next_reloc(rsec, reloc);
+	if (!sub || reloc_offset(sub) != reloc_offset(reloc) ||
+	    reloc_type(sub) != sub_type)
+		return 0;
+
+	/* Only a "sym - ." difference is PC-relative */
+	if (sub->sym->sec != rsec->base ||
+	    sub->sym->offset + reloc_addend(sub) != reloc_offset(sub))
+		return 0;
+
+	set_reloc_type(elf, reloc, pcrel_type);
+	return 0;
+}
+
 bool arch_pc_relative_reloc(struct reloc *reloc)
 {
 	return false;
diff --git a/tools/objtool/arch/loongarch/include/arch/elf.h b/tools/objtool/arch/loongarch/include/arch/elf.h
index ec79062c9554..0103f27fccfc 100644
--- a/tools/objtool/arch/loongarch/include/arch/elf.h
+++ b/tools/objtool/arch/loongarch/include/arch/elf.h
@@ -15,6 +15,18 @@
 #ifndef R_LARCH_64
 #define R_LARCH_64		2
 #endif
+#ifndef R_LARCH_ADD32
+#define R_LARCH_ADD32		50
+#endif
+#ifndef R_LARCH_ADD64
+#define R_LARCH_ADD64		51
+#endif
+#ifndef R_LARCH_SUB32
+#define R_LARCH_SUB32		55
+#endif
+#ifndef R_LARCH_SUB64
+#define R_LARCH_SUB64		56
+#endif
 #ifndef R_LARCH_32_PCREL
 #define R_LARCH_32_PCREL	99
 #endif
@@ -34,4 +46,10 @@
 #define R_TEXT32		R_LARCH_32_PCREL
 #define R_TEXT64		R_LARCH_32_PCREL
 
+#define ARCH_HAS_PAIRED_RELOCS	1
+
+struct elf;
+struct reloc;
+int arch_normalize_paired_reloc(struct elf *elf, struct reloc *reloc);
+
 #endif /* _OBJTOOL_ARCH_ELF_H */
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index a952e680e8a6..92cf0fc3ff2f 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -1518,6 +1518,13 @@ static bool is_uncorrelated_section(struct section *sec)
 	       strstarts(sec->name, ".data..Lanon.");		/* Clang */
 }
 
+#ifndef ARCH_HAS_PAIRED_RELOCS
+static inline int arch_normalize_paired_reloc(struct elf *elf, struct reloc *reloc)
+{
+	return 0;
+}
+#endif
+
 /*
  * Convert a relocation symbol reference to the needed format: either a section
  * symbol or the underlying symbol itself.  Return -1 error, 0 success, 1 skip.
@@ -1525,10 +1532,16 @@ static bool is_uncorrelated_section(struct section *sec)
 static int convert_reloc_sym(struct elf *elf, struct reloc *reloc)
 {
 	struct section *sec = reloc->sym->sec;
+	int ret;
 
 	if (reloc_type(reloc) == R_NONE)
 		return 1;
 
+	/* Fold paired ADD/SUB relocs (LoongArch) into a single PCREL */
+	ret = arch_normalize_paired_reloc(elf, reloc);
+	if (ret)
+		return ret;
+
 	if (is_uncorrelated_section(sec))
 		return convert_reloc_sym_to_secsym(elf, reloc);
 
-- 
2.53.0

[PATCH v4 12/14] objtool/LoongArch: Enable inline alternative cloning for KLP

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

An ALTERNATIVE() emits its replacement instructions into a .subsection 1
placed after the patched function.  When objtool clones the function into
a livepatch module, clone_inline_alternatives() clones that replacement
block too, but only for architectures that define ARCH_HAS_INLINE_ALTS.

LoongArch uses the same .subsection layout as arm64, and its alternative
macros now mark the replacement with ANNOTATE_DATA_SPECIAL (see "LoongArch:
Mark special sections for KLP support").  Define ARCH_HAS_INLINE_ALTS so
the replacement is cloned; otherwise a livepatch for a function containing
an ALTERNATIVE() fails to build with "can't find new instruction".

Reproduced by building a livepatch for a function containing an
ALTERNATIVE() (the base-vmlinux test).
Link: https://lore.kernel.org/all/ak7QBSm3mmDlB-4u@redhat.com

Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/arch/loongarch/include/arch/elf.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/objtool/arch/loongarch/include/arch/elf.h b/tools/objtool/arch/loongarch/include/arch/elf.h
index 0103f27fccfc..e2843679f24e 100644
--- a/tools/objtool/arch/loongarch/include/arch/elf.h
+++ b/tools/objtool/arch/loongarch/include/arch/elf.h
@@ -46,6 +46,7 @@
 #define R_TEXT32		R_LARCH_32_PCREL
 #define R_TEXT64		R_LARCH_32_PCREL
 
+#define ARCH_HAS_INLINE_ALTS	1
 #define ARCH_HAS_PAIRED_RELOCS	1
 
 struct elf;
-- 
2.53.0

[PATCH v4 13/14] objtool/klp: Keep LoongArch tablejump annotation table entries

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

With -mannotate-tablejump, LoongArch compilers emit a
.discard.tablejump_annotate section.  Each entry is a pair of 8-byte
words: the address of a jump instruction and the address of its jump
table.  objtool reads these pairs to find switch jump tables when
decoding.

klp-diff creates one fake symbol per 8-byte word and clones a word only
if should_keep_special_sym() accepts it.  The default rule keeps a word
only if it references a function that was cloned into the output
module.  The instruction-side word references the function and is
kept.  The table-side word references the jump table via its .rodata
section symbol, which is not a function symbol, so it is dropped.

The cloned annotate section then holds only instruction-side words,
compacted together.  The pairing is destroyed.  Parsing the malformed
section crashes objtool on the patch module and no .ko is produced:

  Building patch module: livepatch-shadow-newpid.ko
  livepatch-shadow-newpid.o: error: SIGSEGV: objtool crash!

Keep all .discard.tablejump_annotate words whose referenced symbol has
been cloned.

Reproduced with the shadow-newpid test, which patches
proc_pid_status().  That function contains two switch jump tables.

Before, klp-diff clones only the instruction-side words:

  DEBUG: vmlinux.o: _discard_tablejump_annotate_57441 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x0: proc_pid_status+0xc70 [FUNC GLOBAL]
  DEBUG: vmlinux.o: _discard_tablejump_annotate_57443 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x8: proc_pid_status+0xd2c [FUNC GLOBAL]

After, the full pairs are kept, including the .rodata table words:

  DEBUG: vmlinux.o: _discard_tablejump_annotate_57441 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x0: proc_pid_status+0xc70 [FUNC GLOBAL]
  DEBUG: vmlinux.o: _discard_tablejump_annotate_57442 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x8: .rodata.proc_pid_status+0x0 [SECTION]
  DEBUG: vmlinux.o: _discard_tablejump_annotate_57443 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x10: proc_pid_status+0xd2c [FUNC GLOBAL]
  DEBUG: vmlinux.o: _discard_tablejump_annotate_57444 [+DATA]
  DEBUG: vmlinux.o:         .discard.tablejump_annotate+0x18: .rodata.proc_pid_status+0x80 [SECTION]

and the patch module builds.

Co-developed-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: Kexin Liu <liukexin@kylinos.cn>
Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/objtool/klp-diff.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index 92cf0fc3ff2f..f151ffc71184 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -1956,6 +1956,7 @@ static int create_fake_symbols(struct elf *elf)
 static bool should_keep_special_sym(struct elf *elf, struct symbol *sym)
 {
 	bool annotate_insn = !strcmp(sym->sec->name, ".discard.annotate_insn");
+	bool tablejump_annotate = !strcmp(sym->sec->name, ".discard.tablejump_annotate");
 	struct reloc *reloc;
 
 	if (is_sec_sym(sym) || !sym->sec->rsec)
@@ -1968,6 +1969,16 @@ static bool should_keep_special_sym(struct elf *elf, struct symbol *sym)
 		if (!reloc->sym->clone || is_undef_sym(reloc->sym->clone))
 			continue;
 
+		/*
+		 * .discard.tablejump_annotate (LoongArch -mannotate-tablejump)
+		 * holds pairs of words: a jump instruction and its jump table.
+		 * The table word references the table via its .rodata section
+		 * symbol, which the is_func_sym() rule below would drop,
+		 * breaking the pairing.  Keep both words of each entry.
+		 */
+		if (tablejump_annotate)
+			return true;
+
 		/*
 		 * Keep special section references to cloned functions.
 		 * In some cases annotate_insn can also reference cloned alt
-- 
2.53.0

[PATCH v4 14/14] objtool/klp: Rewrite PC-relative data references to GOT on LoongArch

[George Guo]

From: George Guo <guodongtai@kylinos.cn>

A livepatch module is mapped far more than the +-2GB reachable by a
pcalau12i/addi.d pair, so a klp relocation that resolves to a symbol in
vmlinux cannot be reached PC-relatively: the relocation overflows and
the patched function computes a wild pointer.  Far calls are already
handled by the module loader, which redirects them through a PLT stub,
but there is no such indirection for data, whose address is
materialized inline.

Compilers emit the direct PC-relative form for any symbol they consider
local.  In particular file-local 'static' data is never interposable, so
-fPIC does not route it through the GOT.  This is correct for vmlinux and
for ordinary modules; it only breaks once klp-diff extracts the function
into the far livepatch module while its data stays behind.  It is not
Clang-specific: GCC emits the same PC-relative form.

Add an arch hook, arch_klp_convert_reloc_to_got(), invoked when a klp
relocation is created, and implement it for LoongArch: rewrite the
reference to its GOT-indirect equivalent, which loads the full 64-bit
address from a GOT slot the module loader fills in:

	pcalau12i rd, %pc_hi20(sym)      ->  pcalau12i rd, %got_pc_hi20(sym)
	addi.d    rd, rj, %pc_lo12(sym)  ->  ld.d      rd, rj, %got_pc_lo12(sym)

The loader already sizes the module GOT from every SHT_RELA section that
targets executable code, including .klp.rela.*, so the slot is reserved
at load time and filled when the patch is enabled.

Only an adjacent, register-consistent pcalau12i/addi.d pair that
materializes an address can be converted.  A PCALA_LO12 on any other
instruction is a load or store that reads the symbol directly; going
through the GOT there needs an extra instruction to dereference the
slot, and the function cannot grow without shifting every offset,
relocation and ORC entry.  A pcalau12i shared by several addi.d with
different addends would likewise need one GOT slot per addend, which a
single GOT_PC_HI20 cannot express.  Reject both rather than emit a
livepatch that is quietly broken.

The LoongArch decoder in tools/ lacks the reg1i20 instruction format
used to identify pcalau12i; add it to the tools/ copy of asm/inst.h.

Reproduced by a livepatch for proc_pid_status() (the shadow-newpid
test), which reads the file-local task_state_array[].

Signed-off-by: George Guo <guodongtai@kylinos.cn>
---
 tools/arch/loongarch/include/asm/inst.h       |  11 ++
 tools/objtool/arch/loongarch/decode.c         | 104 ++++++++++++++++++
 .../objtool/arch/loongarch/include/arch/elf.h |  16 +++
 tools/objtool/klp-diff.c                      |  23 +++-
 4 files changed, 152 insertions(+), 2 deletions(-)

diff --git a/tools/arch/loongarch/include/asm/inst.h b/tools/arch/loongarch/include/asm/inst.h
index d68fad63c8b7..0d4caa6522f1 100644
--- a/tools/arch/loongarch/include/asm/inst.h
+++ b/tools/arch/loongarch/include/asm/inst.h
@@ -25,6 +25,10 @@ enum reg1i21_op {
 	bcnez_op	= 0x12, /* bits[9:8] = 0x01 */
 };
 
+enum reg1i20_op {
+	pcalau12i_op	= 0x0d,
+};
+
 enum reg2_op {
 	ertn_op		= 0x1920e,
 };
@@ -66,6 +70,12 @@ struct reg0i26_format {
 	unsigned int opcode : 6;
 };
 
+struct reg1i20_format {
+	unsigned int rd : 5;
+	unsigned int immediate : 20;
+	unsigned int opcode : 7;
+};
+
 struct reg1i21_format {
 	unsigned int immediate_h  : 5;
 	unsigned int rj : 5;
@@ -111,6 +121,7 @@ union loongarch_instruction {
 	unsigned int word;
 	struct reg0i15_format	reg0i15_format;
 	struct reg0i26_format	reg0i26_format;
+	struct reg1i20_format	reg1i20_format;
 	struct reg1i21_format	reg1i21_format;
 	struct reg2_format	reg2_format;
 	struct reg2i12_format	reg2i12_format;
diff --git a/tools/objtool/arch/loongarch/decode.c b/tools/objtool/arch/loongarch/decode.c
index 273398ea4486..99765b13058c 100644
--- a/tools/objtool/arch/loongarch/decode.c
+++ b/tools/objtool/arch/loongarch/decode.c
@@ -101,6 +101,110 @@ int arch_normalize_paired_reloc(struct elf *elf, struct reloc *reloc)
 	return 0;
 }
 
+static bool klp_read_insn(struct section *sec, unsigned long offset,
+			  union loongarch_instruction *insn)
+{
+	if (offset + sizeof(*insn) > sec_size(sec))
+		return false;
+
+	memcpy(insn, sec->data->d_buf + offset, sizeof(*insn));
+	return true;
+}
+
+/*
+ * A livepatch module is mapped far more than the +-2GB reachable by
+ * pcalau12i/addi.d, so a klp reloc which resolves to a symbol in vmlinux cannot
+ * be reached PC-relatively: the relocation overflows and the patched function
+ * computes a wild pointer.  Far calls are already handled by the module loader,
+ * which redirects them through a PLT stub, but there is no such indirection for
+ * data: the address is materialized inline.
+ *
+ * Compilers emit the direct PC-relative form for any symbol they consider local
+ * (in particular file-local 'static' data, which is never interposable, so
+ * -fPIC does not route it through the GOT).  This is correct for vmlinux and
+ * for ordinary modules; it only breaks once klp-diff extracts the function into
+ * a far away livepatch module while its data stays behind.
+ *
+ * Convert the reference to its GOT-indirect equivalent, which loads the full
+ * 64-bit address from a GOT slot the module loader fills in and so works at any
+ * distance:
+ *
+ *	pcalau12i rd, %pc_hi20(sym)	->  pcalau12i rd, %got_pc_hi20(sym)
+ *	addi.d    rd, rj, %pc_lo12(sym)	->  ld.d      rd, rj, %got_pc_lo12(sym)
+ *
+ * Only an adjacent pcalau12i/addi.d pair which materializes an address can be
+ * converted, and only that exact shape is accepted:
+ *
+ *  - A PCALA_LO12 on anything but addi.d is a load or store reading the symbol
+ *    directly.  Going through the GOT would need an extra instruction to
+ *    dereference the slot, and the function cannot grow without shifting every
+ *    offset, relocation and ORC entry.
+ *
+ *  - The pair must be adjacent and register-consistent.  If one pcalau12i is
+ *    shared by several addi.d with different addends, each needs its own GOT
+ *    slot, but the single GOT_PC_HI20 only spans one page: the result would be
+ *    correct only if those slots happened to share a page.  Requiring
+ *    adjacency rejects that (the second addi.d is not preceded by a pcalau12i),
+ *    as well as a pair the compiler scheduled apart.
+ *
+ * Bail out on anything else instead of emitting a livepatch which is quietly
+ * broken.
+ */
+int arch_klp_convert_reloc_to_got(struct elf *elf, struct section *sec,
+				  unsigned long offset, unsigned int *type)
+{
+	union loongarch_instruction hi, lo;
+	unsigned long hi_offset, lo_offset;
+
+	switch (*type) {
+	case R_LARCH_PCALA_HI20:
+		hi_offset = offset;
+		lo_offset = offset + LOONGARCH_INSN_SIZE;
+		break;
+
+	case R_LARCH_PCALA_LO12:
+		if (offset < LOONGARCH_INSN_SIZE) {
+			ERROR("%s+0x%lx: PCALA_LO12 with no preceding instruction",
+			      sec->name, offset);
+			return -1;
+		}
+		hi_offset = offset - LOONGARCH_INSN_SIZE;
+		lo_offset = offset;
+		break;
+
+	default:
+		return 0;
+	}
+
+	if (!klp_read_insn(sec, hi_offset, &hi) ||
+	    !klp_read_insn(sec, lo_offset, &lo)) {
+		ERROR("%s+0x%lx: PCALA pair runs past end of section",
+		      sec->name, offset);
+		return -1;
+	}
+
+	if (hi.reg1i20_format.opcode != pcalau12i_op ||
+	    lo.reg2i12_format.opcode != addid_op ||
+	    lo.reg2i12_format.rj != hi.reg1i20_format.rd) {
+		ERROR("%s+0x%lx: cannot make klp reference GOT-indirect: expected an adjacent pcalau12i/addi.d pair, found 0x%08x/0x%08x",
+		      sec->name, offset, hi.word, lo.word);
+		return -1;
+	}
+
+	if (*type == R_LARCH_PCALA_HI20) {
+		*type = R_LARCH_GOT_PC_HI20;
+		return 0;
+	}
+
+	/* addi.d rd, rj, %pc_lo12(sym) -> ld.d rd, rj, %got_pc_lo12(sym) */
+	lo.reg2i12_format.opcode = ldd_op;
+	if (elf_write_insn(elf, sec, lo_offset, sizeof(lo), (const char *)&lo))
+		return -1;
+
+	*type = R_LARCH_GOT_PC_LO12;
+	return 0;
+}
+
 bool arch_pc_relative_reloc(struct reloc *reloc)
 {
 	return false;
diff --git a/tools/objtool/arch/loongarch/include/arch/elf.h b/tools/objtool/arch/loongarch/include/arch/elf.h
index e2843679f24e..7a97cf69dc86 100644
--- a/tools/objtool/arch/loongarch/include/arch/elf.h
+++ b/tools/objtool/arch/loongarch/include/arch/elf.h
@@ -27,6 +27,18 @@
 #ifndef R_LARCH_SUB64
 #define R_LARCH_SUB64		56
 #endif
+#ifndef R_LARCH_PCALA_HI20
+#define R_LARCH_PCALA_HI20	71
+#endif
+#ifndef R_LARCH_PCALA_LO12
+#define R_LARCH_PCALA_LO12	72
+#endif
+#ifndef R_LARCH_GOT_PC_HI20
+#define R_LARCH_GOT_PC_HI20	75
+#endif
+#ifndef R_LARCH_GOT_PC_LO12
+#define R_LARCH_GOT_PC_LO12	76
+#endif
 #ifndef R_LARCH_32_PCREL
 #define R_LARCH_32_PCREL	99
 #endif
@@ -48,9 +60,13 @@
 
 #define ARCH_HAS_INLINE_ALTS	1
 #define ARCH_HAS_PAIRED_RELOCS	1
+#define ARCH_HAS_KLP_GOT_RELOCS	1
 
 struct elf;
 struct reloc;
+struct section;
 int arch_normalize_paired_reloc(struct elf *elf, struct reloc *reloc);
+int arch_klp_convert_reloc_to_got(struct elf *elf, struct section *sec,
+				  unsigned long offset, unsigned int *type);
 
 #endif /* _OBJTOOL_ARCH_ELF_H */
diff --git a/tools/objtool/klp-diff.c b/tools/objtool/klp-diff.c
index f151ffc71184..1538f8f92f1f 100644
--- a/tools/objtool/klp-diff.c
+++ b/tools/objtool/klp-diff.c
@@ -1525,6 +1525,14 @@ static inline int arch_normalize_paired_reloc(struct elf *elf, struct reloc *rel
 }
 #endif
 
+#ifndef ARCH_HAS_KLP_GOT_RELOCS
+static inline int arch_klp_convert_reloc_to_got(struct elf *elf, struct section *sec,
+						unsigned long offset, unsigned int *type)
+{
+	return 0;
+}
+#endif
+
 /*
  * Convert a relocation symbol reference to the needed format: either a section
  * symbol or the underlying symbol itself.  Return -1 error, 0 success, 1 skip.
@@ -1565,12 +1573,23 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
 	char sym_name[SYM_NAME_LEN];
 	struct klp_reloc klp_reloc;
 	unsigned long sympos;
+	unsigned int type;
 
 	if (!patched_sym->twin) {
 		ERROR("unexpected klp reloc for new symbol %s", patched_sym->name);
 		return -1;
 	}
 
+	/*
+	 * This reference resolves to a symbol which stays in the patched
+	 * object, arbitrarily far from the livepatch module.  Give the arch a
+	 * chance to convert a PC-relative reference into an indirect one which
+	 * can reach it (LoongArch: PCALA -> GOT).
+	 */
+	type = reloc_type(patched_reloc);
+	if (arch_klp_convert_reloc_to_got(e->out, sec, offset, &type))
+		return -1;
+
 	/*
 	 * Keep the original reloc intact for now to avoid breaking objtool run
 	 * which relies on proper relocations for many of its features.  This
@@ -1591,7 +1610,7 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
 		sym->clone = patched_sym;
 	}
 
-	if (!elf_create_reloc(e->out, sec, offset, sym, addend, reloc_type(patched_reloc)))
+	if (!elf_create_reloc(e->out, sec, offset, sym, addend, type))
 		return -1;
 
 	/*
@@ -1648,7 +1667,7 @@ static int clone_reloc_klp(struct elfs *e, struct reloc *patched_reloc,
 	klp_reloc_off = sec_size(klp_relocs);
 	memset(&klp_reloc, 0, sizeof(klp_reloc));
 
-	klp_reloc.type = reloc_type(patched_reloc);
+	klp_reloc.type = type;
 	if (!elf_add_data(e->out, klp_relocs, &klp_reloc, sizeof(klp_reloc), true))
 		return -1;
 
-- 
2.53.0