* [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048]
@ 2025-08-21 7:26 Kees Cook
2025-08-21 7:26 ` [RFC PATCH 1/7] sanitizer: Expand sanitizer flag from 32-bit to 64-bit Kees Cook
` (6 more replies)
0 siblings, 7 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Hi!
Repeating the start of the 3rd (core kcfi) patch's commit log:
This series implements the Linux Kernel Control Flow Integrity ABI,
which provides a function prototype based forward edge control flow
integrity protection by instrumenting every indirect call to check for
a hash value before the target function address. If the hash at the call
site and the hash at the target do not match, execution will trap.
Just to set expectations, this is an RFC because this is my first time
working on most of the affected areas in GCC, and it is likely I have
missed really obvious stuff, or gone about doing things in very wrong
ways. I tried to find the best way to do stuff, but I was left with many
questions. :) All that said, this works for x86_64 and aarch64 Linux
kernels. (I have implemented riscv64 as well, but I lack a viable test
environment -- I am working on this still.)
I tried to get comments and code style into reasonable shape so as
to not distract horribly from the actual implementation details, but
I suspect I'm still missing some correct style in places. The commit
logs are not in proper ChangeLog format either. I am planning to get
that sorted once this series has gotten some review and I've actually
got code in the right places.
I gave up on trying to keep the testsuite files wrapped at 80
characters. It seemed much less readable, but I will defer to whatever
folks want to see there. :)
Thanks!
-Kees
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107048
https://github.com/KSPP/linux/issues/369
Kees Cook (7):
sanitizer: Expand sanitizer flag from 32-bit to 64-bit
mangle: Introduce C typeinfo mangling API
kcfi: Add core Kernel Control Flow Integrity infrastructure
x86: Add x86_64 Kernel Control Flow Integrity implementation
aarch64: Add AArch64 Kernel Control Flow Integrity implementation
riscv: Add RISC-V Kernel Control Flow Integrity implementation
kcfi: Add regression test suite
gcc/Makefile.in | 2 +
gcc/asan.h | 4 +-
gcc/c-family/c-common.h | 2 +-
gcc/config/aarch64/aarch64-protos.h | 4 +
gcc/config/i386/i386-protos.h | 4 +
gcc/config/riscv/riscv-protos.h | 1 +
gcc/flag-types.h | 66 +-
gcc/kcfi.h | 85 ++
gcc/mangle.h | 29 +
gcc/opts.h | 8 +-
gcc/selftest.h | 1 +
gcc/tree-pass.h | 1 +
gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c | 36 +
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 83 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 83 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 75 ++
.../gcc.dg/kcfi/kcfi-cold-partition.c | 133 +++
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 116 +++
.../gcc.dg/kcfi/kcfi-insn-sequence.c | 42 +
.../gcc.dg/kcfi/kcfi-ipa-robustness.c | 54 ++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 96 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 39 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 41 +
.../gcc.dg/kcfi/kcfi-patchable-basic.c | 54 ++
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 36 +
.../gcc.dg/kcfi/kcfi-patchable-large.c | 47 +
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 52 ++
.../gcc.dg/kcfi/kcfi-patchable-none.c | 18 +
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 48 +
.../gcc.dg/kcfi/kcfi-pic-addressing.c | 98 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 111 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 56 ++
.../gcc.dg/kcfi/kcfi-type-mangling.c | 864 ++++++++++++++++++
gcc/c-family/c-attribs.cc | 19 +-
gcc/c/c-parser.cc | 4 +-
gcc/cfgcleanup.cc | 20 +
gcc/cfgexpand.cc | 61 ++
gcc/combine.cc | 1 +
gcc/common.opt | 2 +-
gcc/config/aarch64/aarch64.cc | 112 +++
gcc/config/aarch64/aarch64.md | 137 +++
gcc/config/i386/i386-options.cc | 3 +
gcc/config/i386/i386.cc | 128 +++
gcc/config/i386/i386.md | 144 +++
gcc/config/riscv/riscv.cc | 157 ++++
gcc/config/riscv/riscv.md | 49 +
gcc/cp/typeck.cc | 2 +-
gcc/d/d-attribs.cc | 8 +-
gcc/doc/invoke.texi | 76 ++
gcc/dwarf2asm.cc | 2 +-
gcc/emit-rtl.cc | 1 +
gcc/kcfi.cc | 783 ++++++++++++++++
gcc/mangle.cc | 548 +++++++++++
gcc/opts.cc | 17 +-
gcc/passes.cc | 1 +
gcc/passes.def | 3 +
gcc/recog.cc | 1 +
gcc/reg-notes.def | 6 +
gcc/targhooks.cc | 50 +-
gcc/testsuite/gcc.dg/kcfi/kcfi.exp | 36 +
gcc/toplev.cc | 8 +
gcc/varasm.cc | 18 +
62 files changed, 4721 insertions(+), 65 deletions(-)
create mode 100644 gcc/kcfi.h
create mode 100644 gcc/mangle.h
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-insn-sequence.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-basic.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-none.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-pic-addressing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-type-mangling.c
create mode 100644 gcc/kcfi.cc
create mode 100644 gcc/mangle.cc
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi.exp
--
2.34.1
^ permalink raw reply [flat|nested] 42+ messages in thread
* [RFC PATCH 1/7] sanitizer: Expand sanitizer flag from 32-bit to 64-bit
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
2025-08-21 7:26 ` [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API Kees Cook
` (5 subsequent siblings)
6 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Extend the sanitizer flags infrastructure from unsigned int (32-bit)
to unsigned long long (64-bit) to support the future addition of KCFI
(Kernel Control Flow Integrity) at bit position 32, which exceeds the
previous 32-bit limit.
Update all sanitizer-related data structures, function signatures,
and type handling across the compiler:
* common.opt: Change flag_sanitize from unsigned int to unsigned long long
* flag-types.h: Convert all SANITIZE_* enum values from 1UL << N to 1ULL << N
* opts.h: Update parse_sanitizer_options() and parse_no_sanitize_attribute()
function signatures to use unsigned long long parameters
* opts.cc: Update sanitizer parsing functions and sanitizer_opts_s
structure to handle 64-bit flag values, including special handling for
"all" flag (~0ULL)
* asan.h: Update sanitize_flags_p() to accept and return 64-bit values
* c-family/c-common.h: Update add_no_sanitize_value() signature
* c-family/c-attribs.cc: Update no_sanitize attribute handling functions
* c-family/c-attribs.cc: Replace unsigned_type_node with
long_long_unsigned_type_node for no_sanitize attribute storage
* d/d-attribs.cc: Replace d_uint_type with long_long_unsigned_type_node
for consistent 64-bit attribute representation across languages
* c/c-parser.cc: Update local flag_sanitize_save variables in declaration
parsing where sanitization is temporarily disabled for initializers
* cp/typeck.cc: Update flag_sanitize_save in pointer-to-member function
handling
* dwarf2asm.cc: Update save_flag_sanitize for ASan-disabled indirect
constants
Is using "unsigned long long" correct here, or is switching to
"uint64_t" preferred?
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/asan.h | 4 +--
gcc/c-family/c-common.h | 2 +-
gcc/flag-types.h | 64 +++++++++++++++++++--------------------
gcc/opts.h | 8 ++---
gcc/c-family/c-attribs.cc | 19 +++++++-----
gcc/c/c-parser.cc | 4 +--
gcc/common.opt | 2 +-
gcc/cp/typeck.cc | 2 +-
gcc/d/d-attribs.cc | 8 ++---
gcc/dwarf2asm.cc | 2 +-
gcc/opts.cc | 16 +++++-----
11 files changed, 67 insertions(+), 64 deletions(-)
diff --git a/gcc/asan.h b/gcc/asan.h
index 273d6745c58d..7434d43ac493 100644
--- a/gcc/asan.h
+++ b/gcc/asan.h
@@ -242,9 +242,9 @@ asan_protect_stack_decl (tree decl)
remove all flags mentioned in "no_sanitize" of DECL_ATTRIBUTES. */
inline bool
-sanitize_flags_p (unsigned int flag, const_tree fn = current_function_decl)
+sanitize_flags_p (unsigned long long flag, const_tree fn = current_function_decl)
{
- unsigned int result_flags = flag_sanitize & flag;
+ unsigned long long result_flags = flag_sanitize & flag;
if (result_flags == 0)
return false;
diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
index 7c7e21d2d0eb..e7940cc079f6 100644
--- a/gcc/c-family/c-common.h
+++ b/gcc/c-family/c-common.h
@@ -1713,7 +1713,7 @@ extern enum flt_eval_method
excess_precision_mode_join (enum flt_eval_method, enum flt_eval_method);
extern int c_flt_eval_method (bool ts18661_p);
-extern void add_no_sanitize_value (tree node, unsigned int flags);
+extern void add_no_sanitize_value (tree node, unsigned long long flags);
extern void maybe_add_include_fixit (rich_location *, const char *, bool);
extern void maybe_suggest_missing_token_insertion (rich_location *richloc,
diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index 9a3cc4a2e165..33c88a15ecbb 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -301,42 +301,42 @@ enum zero_init_padding_bits_kind {
/* Different instrumentation modes. */
enum sanitize_code {
/* AddressSanitizer. */
- SANITIZE_ADDRESS = 1UL << 0,
- SANITIZE_USER_ADDRESS = 1UL << 1,
- SANITIZE_KERNEL_ADDRESS = 1UL << 2,
+ SANITIZE_ADDRESS = 1ULL << 0,
+ SANITIZE_USER_ADDRESS = 1ULL << 1,
+ SANITIZE_KERNEL_ADDRESS = 1ULL << 2,
/* ThreadSanitizer. */
- SANITIZE_THREAD = 1UL << 3,
+ SANITIZE_THREAD = 1ULL << 3,
/* LeakSanitizer. */
- SANITIZE_LEAK = 1UL << 4,
+ SANITIZE_LEAK = 1ULL << 4,
/* UndefinedBehaviorSanitizer. */
- SANITIZE_SHIFT_BASE = 1UL << 5,
- SANITIZE_SHIFT_EXPONENT = 1UL << 6,
- SANITIZE_DIVIDE = 1UL << 7,
- SANITIZE_UNREACHABLE = 1UL << 8,
- SANITIZE_VLA = 1UL << 9,
- SANITIZE_NULL = 1UL << 10,
- SANITIZE_RETURN = 1UL << 11,
- SANITIZE_SI_OVERFLOW = 1UL << 12,
- SANITIZE_BOOL = 1UL << 13,
- SANITIZE_ENUM = 1UL << 14,
- SANITIZE_FLOAT_DIVIDE = 1UL << 15,
- SANITIZE_FLOAT_CAST = 1UL << 16,
- SANITIZE_BOUNDS = 1UL << 17,
- SANITIZE_ALIGNMENT = 1UL << 18,
- SANITIZE_NONNULL_ATTRIBUTE = 1UL << 19,
- SANITIZE_RETURNS_NONNULL_ATTRIBUTE = 1UL << 20,
- SANITIZE_OBJECT_SIZE = 1UL << 21,
- SANITIZE_VPTR = 1UL << 22,
- SANITIZE_BOUNDS_STRICT = 1UL << 23,
- SANITIZE_POINTER_OVERFLOW = 1UL << 24,
- SANITIZE_BUILTIN = 1UL << 25,
- SANITIZE_POINTER_COMPARE = 1UL << 26,
- SANITIZE_POINTER_SUBTRACT = 1UL << 27,
- SANITIZE_HWADDRESS = 1UL << 28,
- SANITIZE_USER_HWADDRESS = 1UL << 29,
- SANITIZE_KERNEL_HWADDRESS = 1UL << 30,
+ SANITIZE_SHIFT_BASE = 1ULL << 5,
+ SANITIZE_SHIFT_EXPONENT = 1ULL << 6,
+ SANITIZE_DIVIDE = 1ULL << 7,
+ SANITIZE_UNREACHABLE = 1ULL << 8,
+ SANITIZE_VLA = 1ULL << 9,
+ SANITIZE_NULL = 1ULL << 10,
+ SANITIZE_RETURN = 1ULL << 11,
+ SANITIZE_SI_OVERFLOW = 1ULL << 12,
+ SANITIZE_BOOL = 1ULL << 13,
+ SANITIZE_ENUM = 1ULL << 14,
+ SANITIZE_FLOAT_DIVIDE = 1ULL << 15,
+ SANITIZE_FLOAT_CAST = 1ULL << 16,
+ SANITIZE_BOUNDS = 1ULL << 17,
+ SANITIZE_ALIGNMENT = 1ULL << 18,
+ SANITIZE_NONNULL_ATTRIBUTE = 1ULL << 19,
+ SANITIZE_RETURNS_NONNULL_ATTRIBUTE = 1ULL << 20,
+ SANITIZE_OBJECT_SIZE = 1ULL << 21,
+ SANITIZE_VPTR = 1ULL << 22,
+ SANITIZE_BOUNDS_STRICT = 1ULL << 23,
+ SANITIZE_POINTER_OVERFLOW = 1ULL << 24,
+ SANITIZE_BUILTIN = 1ULL << 25,
+ SANITIZE_POINTER_COMPARE = 1ULL << 26,
+ SANITIZE_POINTER_SUBTRACT = 1ULL << 27,
+ SANITIZE_HWADDRESS = 1ULL << 28,
+ SANITIZE_USER_HWADDRESS = 1ULL << 29,
+ SANITIZE_KERNEL_HWADDRESS = 1ULL << 30,
/* Shadow Call Stack. */
- SANITIZE_SHADOW_CALL_STACK = 1UL << 31,
+ SANITIZE_SHADOW_CALL_STACK = 1ULL << 31,
SANITIZE_SHIFT = SANITIZE_SHIFT_BASE | SANITIZE_SHIFT_EXPONENT,
SANITIZE_UNDEFINED = SANITIZE_SHIFT | SANITIZE_DIVIDE | SANITIZE_UNREACHABLE
| SANITIZE_VLA | SANITIZE_NULL | SANITIZE_RETURN
diff --git a/gcc/opts.h b/gcc/opts.h
index ea92c4922a3b..65b8a1079f34 100644
--- a/gcc/opts.h
+++ b/gcc/opts.h
@@ -432,10 +432,10 @@ extern char *write_langs (unsigned int mask);
extern void print_ignored_options (void);
extern void handle_common_deferred_options (void);
extern void handle_deferred_dump_options (void);
-unsigned int parse_sanitizer_options (const char *, location_t, int,
- unsigned int, int, bool);
+unsigned long long parse_sanitizer_options (const char *, location_t, int,
+ unsigned long long, int, bool);
-unsigned int parse_no_sanitize_attribute (char *value);
+unsigned long long parse_no_sanitize_attribute (char *value);
extern bool common_handle_option (struct gcc_options *opts,
struct gcc_options *opts_set,
const struct cl_decoded_option *decoded,
@@ -477,7 +477,7 @@ extern bool opt_enum_arg_to_value (size_t opt_index, const char *arg,
extern const struct sanitizer_opts_s
{
const char *const name;
- unsigned int flag;
+ unsigned long long flag;
size_t len;
bool can_recover;
bool can_trap;
diff --git a/gcc/c-family/c-attribs.cc b/gcc/c-family/c-attribs.cc
index 1f4a0df12051..eed384818433 100644
--- a/gcc/c-family/c-attribs.cc
+++ b/gcc/c-family/c-attribs.cc
@@ -1409,24 +1409,27 @@ handle_cold_attribute (tree *node, tree name, tree ARG_UNUSED (args),
/* Add FLAGS for a function NODE to no_sanitize_flags in DECL_ATTRIBUTES. */
void
-add_no_sanitize_value (tree node, unsigned int flags)
+add_no_sanitize_value (tree node, unsigned long long flags)
{
+
tree attr = lookup_attribute ("no_sanitize", DECL_ATTRIBUTES (node));
if (attr)
{
- unsigned int old_value = tree_to_uhwi (TREE_VALUE (attr));
+ unsigned long long old_value = tree_to_uhwi (TREE_VALUE (attr));
flags |= old_value;
if (flags == old_value)
return;
- TREE_VALUE (attr) = build_int_cst (unsigned_type_node, flags);
+ TREE_VALUE (attr) = build_int_cst (long_long_unsigned_type_node, flags);
}
else
- DECL_ATTRIBUTES (node)
- = tree_cons (get_identifier ("no_sanitize"),
- build_int_cst (unsigned_type_node, flags),
- DECL_ATTRIBUTES (node));
+ {
+ DECL_ATTRIBUTES (node)
+ = tree_cons (get_identifier ("no_sanitize"),
+ build_int_cst (long_long_unsigned_type_node, flags),
+ DECL_ATTRIBUTES (node));
+ }
}
/* Handle a "no_sanitize" attribute; arguments as in
@@ -1436,7 +1439,7 @@ static tree
handle_no_sanitize_attribute (tree *node, tree name, tree args, int,
bool *no_add_attrs)
{
- unsigned int flags = 0;
+ unsigned long long flags = 0;
*no_add_attrs = true;
if (TREE_CODE (*node) != FUNCTION_DECL)
{
diff --git a/gcc/c/c-parser.cc b/gcc/c/c-parser.cc
index 4a13fc0d3842..7a0e1d6de6f3 100644
--- a/gcc/c/c-parser.cc
+++ b/gcc/c/c-parser.cc
@@ -2822,7 +2822,7 @@ c_parser_declaration_or_fndef (c_parser *parser, bool fndef_ok,
specs->constexpr_p, &richloc);
/* A parameter is initialized, which is invalid. Don't
attempt to instrument the initializer. */
- int flag_sanitize_save = flag_sanitize;
+ unsigned long long flag_sanitize_save = flag_sanitize;
if (nested && !empty_ok)
flag_sanitize = 0;
init = c_parser_expr_no_commas (parser, NULL);
@@ -2911,7 +2911,7 @@ c_parser_declaration_or_fndef (c_parser *parser, bool fndef_ok,
specs->constexpr_p, &richloc);
/* A parameter is initialized, which is invalid. Don't
attempt to instrument the initializer. */
- int flag_sanitize_save = flag_sanitize;
+ unsigned long long flag_sanitize_save = flag_sanitize;
if (TREE_CODE (d) == PARM_DECL)
flag_sanitize = 0;
init = c_parser_initializer (parser, d);
diff --git a/gcc/common.opt b/gcc/common.opt
index 70659fabebd5..f82f2b3f1b11 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -205,7 +205,7 @@ bool flag_opts_finished
; What the sanitizer should instrument
Variable
-unsigned int flag_sanitize
+unsigned long long flag_sanitize
; What sanitizers should recover from errors
Variable
diff --git a/gcc/cp/typeck.cc b/gcc/cp/typeck.cc
index f592894e01ab..c26b2fb8939b 100644
--- a/gcc/cp/typeck.cc
+++ b/gcc/cp/typeck.cc
@@ -4287,7 +4287,7 @@ get_member_function_from_ptrfunc (tree *instance_ptrptr, tree function,
idx = build1 (NOP_EXPR, vtable_index_type, e3);
switch (TARGET_PTRMEMFUNC_VBIT_LOCATION)
{
- int flag_sanitize_save;
+ unsigned long long flag_sanitize_save;
case ptrmemfunc_vbit_in_pfn:
e1 = cp_build_binary_op (input_location,
BIT_AND_EXPR, idx, integer_one_node,
diff --git a/gcc/d/d-attribs.cc b/gcc/d/d-attribs.cc
index 77315dc5d58d..a1712f996bdc 100644
--- a/gcc/d/d-attribs.cc
+++ b/gcc/d/d-attribs.cc
@@ -1406,7 +1406,7 @@ d_handle_no_sanitize_attribute (tree *node, tree name, tree args, int,
return NULL_TREE;
}
- unsigned int flags = 0;
+ unsigned long long flags = 0;
for (; args; args = TREE_CHAIN (args))
{
tree id = TREE_VALUE (args);
@@ -1424,16 +1424,16 @@ d_handle_no_sanitize_attribute (tree *node, tree name, tree args, int,
merge existing flags if no_sanitize was previously handled. */
if (tree attr = lookup_attribute ("no_sanitize", DECL_ATTRIBUTES (*node)))
{
- unsigned int old_value = tree_to_uhwi (TREE_VALUE (attr));
+ unsigned long long old_value = tree_to_uhwi (TREE_VALUE (attr));
flags |= old_value;
if (flags != old_value)
- TREE_VALUE (attr) = build_int_cst (d_uint_type, flags);
+ TREE_VALUE (attr) = build_int_cst (long_long_unsigned_type_node, flags);
}
else
{
DECL_ATTRIBUTES (*node) = tree_cons (get_identifier ("no_sanitize"),
- build_int_cst (d_uint_type, flags),
+ build_int_cst (long_long_unsigned_type_node, flags),
DECL_ATTRIBUTES (*node));
}
diff --git a/gcc/dwarf2asm.cc b/gcc/dwarf2asm.cc
index ec5c684da479..6f193a68bf9d 100644
--- a/gcc/dwarf2asm.cc
+++ b/gcc/dwarf2asm.cc
@@ -1041,7 +1041,7 @@ dw2_output_indirect_constant_1 (const char *sym, tree id)
sym_ref = gen_rtx_SYMBOL_REF (Pmode, sym);
/* Disable ASan for decl because redzones cause ABI breakage between GCC and
libstdc++ for `.LDFCM*' variables. See PR 78651 for details. */
- unsigned int save_flag_sanitize = flag_sanitize;
+ unsigned long long save_flag_sanitize = flag_sanitize;
flag_sanitize &= ~(SANITIZE_ADDRESS | SANITIZE_USER_ADDRESS
| SANITIZE_KERNEL_ADDRESS);
/* And also temporarily disable -fsection-anchors. These indirect constants
diff --git a/gcc/opts.cc b/gcc/opts.cc
index c21e66ba9171..5fd86aa89adb 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -2168,9 +2168,9 @@ const struct sanitizer_opts_s sanitizer_opts[] =
SANITIZER_OPT (pointer-overflow, SANITIZE_POINTER_OVERFLOW, true, true),
SANITIZER_OPT (builtin, SANITIZE_BUILTIN, true, true),
SANITIZER_OPT (shadow-call-stack, SANITIZE_SHADOW_CALL_STACK, false, false),
- SANITIZER_OPT (all, ~0U, true, true),
+ SANITIZER_OPT (all, ~0ULL, true, true),
#undef SANITIZER_OPT
- { NULL, 0U, 0UL, false, false }
+ { NULL, 0ULL, 0UL, false, false }
};
/* -fzero-call-used-regs= suboptions. */
@@ -2241,7 +2241,7 @@ get_closest_sanitizer_option (const string_fragment &arg,
{
/* -fsanitize=all is not valid, so don't offer it. */
if (code == OPT_fsanitize_
- && opts[i].flag == ~0U
+ && opts[i].flag == ~0ULL
&& value)
continue;
@@ -2268,9 +2268,9 @@ get_closest_sanitizer_option (const string_fragment &arg,
adjust previous FLAGS and return new ones. If COMPLAIN is false,
don't issue diagnostics. */
-unsigned int
+unsigned long long
parse_sanitizer_options (const char *p, location_t loc, int scode,
- unsigned int flags, int value, bool complain)
+ unsigned long long flags, int value, bool complain)
{
enum opt_code code = (enum opt_code) scode;
@@ -2296,7 +2296,7 @@ parse_sanitizer_options (const char *p, location_t loc, int scode,
&& memcmp (p, sanitizer_opts[i].name, len) == 0)
{
/* Handle both -fsanitize and -fno-sanitize cases. */
- if (value && sanitizer_opts[i].flag == ~0U)
+ if (value && sanitizer_opts[i].flag == ~0ULL)
{
if (code == OPT_fsanitize_)
{
@@ -2377,10 +2377,10 @@ parse_sanitizer_options (const char *p, location_t loc, int scode,
/* Parse string values of no_sanitize attribute passed in VALUE.
Values are separated with comma. */
-unsigned int
+unsigned long long
parse_no_sanitize_attribute (char *value)
{
- unsigned int flags = 0;
+ unsigned long long flags = 0;
unsigned int i;
char *q = strtok (value, ",");
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
2025-08-21 7:26 ` [RFC PATCH 1/7] sanitizer: Expand sanitizer flag from 32-bit to 64-bit Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
[not found] ` <CALvbMcAPV1eB6nocPAS=qR8SCiQyU43v911R8S7Ah_=G7yK-+g@mail.gmail.com>
2025-08-21 7:26 ` [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
` (4 subsequent siblings)
6 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
To support the KCFI type-id which needs to convert unique function
prototypes into unique 32-bit values, add a subset of the Itanium C++
mangling ABI for C typeinfo of function prototypes. This gets us to the
first step: a string representation of the function prototype.
Trying to extract only the C portions of the gcc/cp/mangle.cc code
seemed infeasible after a few attempts. So this is the minimal subset
of the mangling ABI needed to generate unique KCFI type ids.
I could not find a way to build a sensible selftest infrastructure for
this code. I wanted to do something like this:
#ifdef CHECKING_P
const char code[] = "
typedef struct { int x, y } xy_t;
extern int func(xy_t *p);
";
ASSERT_MANGLE (code, "_ZTSPFiP4xy_tE");
...
#endif
But I could not find any way to build a localized parser that could
parse the "code" string from which I could extract the "func" fndecl.
It would have been so much nicer to build the selftest directly into
mangle.cc here, but I couldn't figure it out. Instead, later patches
create a "kcfi" dump file, and the large kcfi testsuite validates
expected mangle strings as part of the type-id validation.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/Makefile.in | 1 +
gcc/mangle.h | 29 +++
gcc/selftest.h | 1 +
gcc/mangle.cc | 548 ++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 579 insertions(+)
create mode 100644 gcc/mangle.h
create mode 100644 gcc/mangle.cc
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index d7d5cbe72770..86f62611c1d4 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1619,6 +1619,7 @@ OBJS = \
lto-section-out.o \
lto-opts.o \
lto-compress.o \
+ mangle.o \
mcf.o \
mode-switching.o \
modulo-sched.o \
diff --git a/gcc/mangle.h b/gcc/mangle.h
new file mode 100644
index 000000000000..94521e1e7e5c
--- /dev/null
+++ b/gcc/mangle.h
@@ -0,0 +1,29 @@
+/* Itanium C++ ABI type mangling for GCC.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_MANGLE_H
+#define GCC_MANGLE_H
+
+#include "tree.h"
+
+/* Function type mangling following Itanium C++ ABI conventions.
+ Returns a static buffer containing the mangled type string. */
+extern const char *mangle_function_type (tree fntype_or_fndecl);
+
+#endif /* GCC_MANGLE_H */
diff --git a/gcc/mangle.cc b/gcc/mangle.cc
new file mode 100644
index 000000000000..830985251c81
--- /dev/null
+++ b/gcc/mangle.cc
@@ -0,0 +1,548 @@
+/* Itanium C++ ABI type mangling for GCC.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tree.h"
+#include "diagnostic-core.h"
+#include "stringpool.h"
+#include "stor-layout.h"
+#include "mangle.h"
+#include "selftest.h"
+
+/* Forward declaration for recursive type mangling. */
+static void mangle_type_to_buffer (tree type, char **p, char *end);
+
+/* Mangle a builtin type following Itanium C++ ABI for C types. */
+static void
+mangle_builtin_type_to_buffer (tree type, char **p, char *end)
+{
+ gcc_assert (type != NULL_TREE);
+ gcc_assert (p != NULL && *p != NULL && end != NULL);
+ gcc_assert (*p < end);
+
+ if (*p >= end)
+ return;
+
+ switch (TREE_CODE (type))
+ {
+ case VOID_TYPE:
+ **p = 'v';
+ (*p)++;
+ break;
+
+ case BOOLEAN_TYPE:
+ **p = 'b';
+ (*p)++;
+ break;
+
+ case INTEGER_TYPE:
+ /* Handle standard integer types using Itanium ABI codes. */
+ if (type == char_type_node)
+ {
+ **p = 'c';
+ (*p)++;
+ }
+ else if (type == signed_char_type_node)
+ {
+ **p = 'a';
+ (*p)++;
+ }
+ else if (type == unsigned_char_type_node)
+ {
+ **p = 'h';
+ (*p)++;
+ }
+ else if (type == short_integer_type_node)
+ {
+ **p = 's';
+ (*p)++;
+ }
+ else if (type == short_unsigned_type_node)
+ {
+ **p = 't';
+ (*p)++;
+ }
+ else if (type == integer_type_node)
+ {
+ **p = 'i';
+ (*p)++;
+ }
+ else if (type == unsigned_type_node)
+ {
+ **p = 'j';
+ (*p)++;
+ }
+ else if (type == long_integer_type_node)
+ {
+ **p = 'l';
+ (*p)++;
+ }
+ else if (type == long_unsigned_type_node)
+ {
+ **p = 'm';
+ (*p)++;
+ }
+ else if (type == long_long_integer_type_node)
+ {
+ **p = 'x';
+ (*p)++;
+ }
+ else if (type == long_long_unsigned_type_node)
+ {
+ **p = 'y';
+ (*p)++;
+ }
+ else
+ {
+ /* Fallback for other integer types - use precision-based encoding. */
+ *p += snprintf (*p, end - *p, "i%d", TYPE_PRECISION (type));
+ }
+ break;
+
+ case REAL_TYPE:
+ if (type == float_type_node)
+ {
+ **p = 'f';
+ (*p)++;
+ }
+ else if (type == double_type_node)
+ {
+ **p = 'd';
+ (*p)++;
+ }
+ else if (type == long_double_type_node)
+ {
+ **p = 'e';
+ (*p)++;
+ }
+ else
+ {
+ /* Fallback for other real types. */
+ *p += snprintf (*p, end - *p, "f%d", TYPE_PRECISION (type));
+ }
+ break;
+
+ default:
+ /* Unknown builtin type - this should never happen in a well-formed C program. */
+ error ("mangle: Unknown builtin type with %<TREE_CODE%> %d", TREE_CODE (type));
+ error ("mangle: %<TYPE_MODE%> = %d, %<TYPE_PRECISION%> = %d", TYPE_MODE (type), TYPE_PRECISION (type));
+ error ("mangle: Please report this as a bug with the above diagnostic information");
+ gcc_unreachable ();
+ }
+}
+
+/* Canonicalize typedef types to their underlying named struct/union types. */
+static tree
+canonicalize_typedef_type (tree type)
+{
+ /* Handle typedef types - canonicalize to named structs when possible. */
+ if (TYPE_NAME (type) && TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ tree type_decl = TYPE_NAME (type);
+
+ /* Check if this is a typedef (not the original struct declaration) */
+ if (DECL_ORIGINAL_TYPE (type_decl))
+ {
+ tree original_type = DECL_ORIGINAL_TYPE (type_decl);
+
+ /* If the original type is a named struct/union/enum, use that instead. */
+ if ((TREE_CODE (original_type) == RECORD_TYPE
+ || TREE_CODE (original_type) == UNION_TYPE
+ || TREE_CODE (original_type) == ENUMERAL_TYPE)
+ && TYPE_NAME (original_type)
+ && ((TREE_CODE (TYPE_NAME (original_type)) == TYPE_DECL
+ && DECL_NAME (TYPE_NAME (original_type)))
+ || TREE_CODE (TYPE_NAME (original_type)) == IDENTIFIER_NODE))
+ {
+ /* Recursively canonicalize in case the original type is also a typedef. */
+ return canonicalize_typedef_type (original_type);
+ }
+
+ /* For basic type typedefs (e.g., u8 -> unsigned char), canonicalize to original type. */
+ if (TREE_CODE (original_type) == INTEGER_TYPE
+ || TREE_CODE (original_type) == REAL_TYPE
+ || TREE_CODE (original_type) == POINTER_TYPE
+ || TREE_CODE (original_type) == ARRAY_TYPE
+ || TREE_CODE (original_type) == FUNCTION_TYPE
+ || TREE_CODE (original_type) == METHOD_TYPE
+ || TREE_CODE (original_type) == BOOLEAN_TYPE
+ || TREE_CODE (original_type) == COMPLEX_TYPE
+ || TREE_CODE (original_type) == VECTOR_TYPE)
+ {
+ /* Recursively canonicalize in case the original type is also a typedef. */
+ return canonicalize_typedef_type (original_type);
+ }
+ }
+ }
+
+ return type;
+}
+
+/* Recursively mangle a type following Itanium C++ ABI conventions. */
+static void
+mangle_type_to_buffer (tree type, char **p, char *end)
+{
+ gcc_assert (type != NULL_TREE);
+ gcc_assert (p != NULL && *p != NULL && end != NULL);
+ gcc_assert (*p < end);
+
+ if (*p >= end)
+ return;
+
+ /* Canonicalize typedef types to their underlying named struct types. */
+ type = canonicalize_typedef_type (type);
+
+ switch (TREE_CODE (type))
+ {
+ case POINTER_TYPE:
+ {
+ /* Pointer type: 'P' + qualifiers + pointed-to type. */
+ **p = 'P';
+ (*p)++;
+
+ /* Add qualifiers to the pointed-to type following Itanium C++ ABI ordering. */
+ tree pointed_to_type = TREE_TYPE (type);
+ if (TYPE_QUALS (pointed_to_type) != TYPE_UNQUALIFIED)
+ {
+ /* Emit qualifiers in Itanium ABI order: restrict, volatile, const. */
+ if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_RESTRICT)
+ {
+ **p = 'r';
+ (*p)++;
+ }
+ if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_VOLATILE)
+ {
+ **p = 'V';
+ (*p)++;
+ }
+ if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_CONST)
+ {
+ **p = 'K';
+ (*p)++;
+ }
+ /* Note: _Atomic is not typically used in kernel code. */
+ }
+
+ /* For KCFI's hybrid type system: preserve typedef names for compound types,
+ but use canonical forms for primitive types. */
+ tree target_type;
+ if (TREE_CODE (pointed_to_type) == RECORD_TYPE
+ || TREE_CODE (pointed_to_type) == UNION_TYPE
+ || TREE_CODE (pointed_to_type) == ENUMERAL_TYPE)
+ {
+ /* Compound type: preserve typedef information by using original type. */
+ target_type = pointed_to_type;
+ }
+ else
+ {
+ /* Primitive type: use canonical form to ensure structural typing. */
+ target_type = TYPE_MAIN_VARIANT (pointed_to_type);
+ }
+ mangle_type_to_buffer (target_type, p, end);
+ break;
+ }
+
+ case ARRAY_TYPE:
+ /* Array type: 'A' + size + '_' + element type (simplified). */
+ **p = 'A';
+ (*p)++;
+ if (TYPE_DOMAIN (type) && TYPE_MAX_VALUE (TYPE_DOMAIN (type)))
+ {
+ HOST_WIDE_INT size = tree_to_shwi (TYPE_MAX_VALUE (TYPE_DOMAIN (type))) + 1;
+ *p += snprintf (*p, end - *p, "%ld_", (long) size);
+ }
+ else
+ {
+ **p = '_';
+ (*p)++;
+ }
+ mangle_type_to_buffer (TREE_TYPE (type), p, end);
+ break;
+
+ case FUNCTION_TYPE:
+ {
+ /* Function type: 'F' + return type + parameter types + 'E' */
+ **p = 'F';
+ (*p)++;
+ mangle_type_to_buffer (TREE_TYPE (type), p, end);
+
+ /* Add parameter types. */
+ tree param_types = TYPE_ARG_TYPES (type);
+
+ if (param_types == NULL_TREE)
+ {
+ /* func() - variadic function, no parameter list.
+ Don't mangle any parameters. */
+ }
+ else
+ {
+ bool found_real_params = false;
+ for (tree param = param_types; param && *p < end; param = TREE_CHAIN (param))
+ {
+ tree param_type = TREE_VALUE (param);
+ if (param_type == void_type_node)
+ {
+ /* Check if this is the first parameter (explicit void) or a sentinel */
+ if (!found_real_params)
+ {
+ /* func(void) - explicit empty parameter list.
+ Mangle void to distinguish from variadic func(). */
+ mangle_type_to_buffer (void_type_node, p, end);
+ }
+ /* If we found real params before this void, it's a sentinel - stop */
+ break;
+ }
+
+ found_real_params = true;
+
+ /* For value parameters, ignore const/volatile qualifiers as they
+ don't affect the calling convention. const int and int are
+ passed identically by value. */
+ tree canonical_param_type = param_type;
+ if (TREE_CODE (param_type) != POINTER_TYPE
+ && TREE_CODE (param_type) != REFERENCE_TYPE
+ && TREE_CODE (param_type) != ARRAY_TYPE)
+ {
+ /* Strip qualifiers for non-pointer/reference value parameters. */
+ canonical_param_type = TYPE_MAIN_VARIANT (param_type);
+ }
+
+ mangle_type_to_buffer (canonical_param_type, p, end);
+ }
+ }
+
+ **p = 'E';
+ (*p)++;
+ break;
+ }
+
+ case RECORD_TYPE:
+ case UNION_TYPE:
+ case ENUMERAL_TYPE:
+ {
+ /* Struct/union/enum: use simplified representation for C types. */
+ const char *name = NULL;
+
+ if (TYPE_NAME (type))
+ {
+ if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ /* TYPE_DECL case: both named structs and typedef structs. */
+ tree decl_name = DECL_NAME (TYPE_NAME (type));
+ if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
+ {
+ name = IDENTIFIER_POINTER (decl_name);
+ }
+ }
+ else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+ {
+ /* Direct identifier case. */
+ name = IDENTIFIER_POINTER (TYPE_NAME (type));
+ }
+ }
+
+ /* If no name found through normal extraction, handle anonymous types following Itanium C++ ABI. */
+ if (!name && !TYPE_NAME (type))
+ {
+ static char anon_name[128];
+
+ if (TREE_CODE (type) == UNION_TYPE)
+ {
+ /* For anonymous unions, try to find first named field (Itanium ABI approach). */
+ tree field = TYPE_FIELDS (type);
+ while (field && !DECL_NAME (field))
+ field = DECL_CHAIN (field);
+
+ if (field && DECL_NAME (field))
+ {
+ const char *field_name = IDENTIFIER_POINTER (DECL_NAME (field));
+ snprintf (anon_name, sizeof(anon_name), "anon_union_by_%s", field_name);
+ }
+ else
+ {
+ /* No named fields - use Itanium-style Ut encoding. */
+ snprintf (anon_name, sizeof(anon_name), "Ut_unnamed_union");
+ }
+ }
+ else
+ {
+ /* For anonymous structs/enums, use Itanium-style Ut encoding with layout info for discrimination. */
+ const char *type_prefix = "";
+ if (TREE_CODE (type) == RECORD_TYPE)
+ type_prefix = "struct";
+ else if (TREE_CODE (type) == ENUMERAL_TYPE)
+ type_prefix = "enum";
+
+ /* Include size and field layout for better discrimination. */
+ HOST_WIDE_INT size = 0;
+ if (TYPE_SIZE (type) && tree_fits_shwi_p (TYPE_SIZE (type)))
+ size = tree_to_shwi (TYPE_SIZE (type));
+
+ /* Generate a hash based on field layout to distinguish same-sized anonymous types. */
+ unsigned layout_hash = 0;
+ if (TREE_CODE (type) == RECORD_TYPE)
+ {
+ for (tree field = TYPE_FIELDS (type); field; field = DECL_CHAIN (field))
+ {
+ if (TREE_CODE (field) == FIELD_DECL)
+ {
+ /* Hash field offset and type. */
+ if (DECL_FIELD_OFFSET (field))
+ {
+ HOST_WIDE_INT offset = tree_to_shwi (DECL_FIELD_OFFSET (field));
+ layout_hash = layout_hash * 31 + (unsigned)offset;
+ }
+
+ /* Hash field type. */
+ tree field_type = TREE_TYPE (field);
+ if (field_type && TYPE_MODE (field_type) != VOIDmode)
+ layout_hash = layout_hash * 37 + (unsigned)TYPE_MODE (field_type);
+ }
+ }
+ }
+
+ if (layout_hash != 0)
+ snprintf (anon_name, sizeof(anon_name), "Ut_%s_%ld_%x", type_prefix, (long)size, layout_hash);
+ else
+ snprintf (anon_name, sizeof(anon_name), "Ut_%s_%ld", type_prefix, (long)size);
+ }
+
+ name = anon_name;
+ }
+
+ if (name)
+ {
+ *p += snprintf (*p, end - *p, "%zu%s", strlen (name), name);
+ }
+ else
+ {
+ /* Always show diagnostic information for missing struct names. */
+ error ("mangle: No struct/union/enum name found for type code %d (%qs)",
+ TREE_CODE (type), get_tree_code_name (TREE_CODE (type)));
+ if (TYPE_NAME (type))
+ {
+ error ("mangle: %<TYPE_NAME%> exists but extraction failed");
+ error ("mangle: %<TYPE_NAME%> tree code = %d", TREE_CODE (TYPE_NAME (type)));
+ if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ tree decl_name = DECL_NAME (TYPE_NAME (type));
+ error ("mangle: %<TYPE_DECL%> %<DECL_NAME%> = %p", (void*)decl_name);
+ if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
+ error ("mangle: %<IDENTIFIER_NODE%> name = '%s'", IDENTIFIER_POINTER (decl_name));
+ }
+ else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+ {
+ error ("mangle: %<IDENTIFIER_NODE%> name = '%s'", IDENTIFIER_POINTER (TYPE_NAME (type)));
+ }
+ else
+ {
+ error ("mangle: Unknown %<TYPE_NAME%> tree code = %d", TREE_CODE (TYPE_NAME (type)));
+ }
+ }
+ else
+ {
+ error ("mangle: %<TYPE_NAME%> is NULL - anonymous struct/union/enum detected");
+ }
+
+ /* This indicates a missing case in our struct name extraction. */
+ error ("mangle: Please report this as a bug with the above diagnostic information");
+ gcc_unreachable ();
+ }
+ break;
+ }
+
+ default:
+ /* Handle builtin types. */
+ mangle_builtin_type_to_buffer (type, p, end);
+ break;
+ }
+}
+
+/* Compute canonical type name using Itanium C++ ABI mangling.
+ Accepts either FUNCTION_DECL (preferred for typedef preservation) or FUNCTION_TYPE. */
+const char *
+mangle_function_type (tree fntype_or_fndecl)
+{
+ gcc_assert (fntype_or_fndecl);
+
+ tree fntype = NULL;
+
+ /* Determine input type and extract function type. */
+ if (TREE_CODE (fntype_or_fndecl) == FUNCTION_TYPE)
+ {
+ /* Already FUNCTION_TYPE. */
+ fntype = fntype_or_fndecl;
+ }
+ else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
+ {
+ tree fndecl = fntype_or_fndecl;
+ tree base_fntype = TREE_TYPE (fndecl);
+
+ /* For FUNCTION_DECL, build a synthetic function type using DECL_ARGUMENTS
+ if available to preserve typedef information. */
+ tree parm = DECL_ARGUMENTS (fndecl);
+ if (parm)
+ {
+ /* Build parameter type list from DECL_ARGUMENTS. */
+ tree param_list = NULL_TREE;
+ tree *param_tail = ¶m_list;
+
+ for (; parm; parm = DECL_CHAIN (parm))
+ {
+ tree parm_type = TREE_TYPE (parm);
+ *param_tail = tree_cons (NULL_TREE, parm_type, NULL_TREE);
+ param_tail = &TREE_CHAIN (*param_tail);
+ }
+
+ /* Add void_type_node sentinel if the function takes no parameters. */
+ if (!param_list)
+ param_list = tree_cons (NULL_TREE, void_type_node, NULL_TREE);
+
+ /* Build synthetic function type with preserved parameter types. */
+ fntype = build_function_type (TREE_TYPE (base_fntype), param_list);
+ }
+ else
+ {
+ /* No DECL_ARGUMENTS - use the standard function type. */
+ fntype = base_fntype;
+ }
+ }
+ else
+ {
+ /* Must only be called with FUNCTION_DECL or FUNCTION_TYPE. */
+ gcc_unreachable ();
+ }
+
+ static char name_buf[512];
+ char *p = name_buf;
+ char *end = name_buf + sizeof (name_buf) - 1;
+
+ /* Typeinfo for a function prototype. */
+ p += sprintf(name_buf, "_ZTSP");
+
+ /* Use mangle_type_to_buffer for all cases. */
+ mangle_type_to_buffer (fntype, &p, end);
+
+ /* Ensure we didn't overflow the buffer. */
+ gcc_assert (p <= end);
+ *p = '\0';
+ return name_buf;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
2025-08-21 7:26 ` [RFC PATCH 1/7] sanitizer: Expand sanitizer flag from 32-bit to 64-bit Kees Cook
2025-08-21 7:26 ` [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
[not found] ` <CALvbMcA+8iHo+zCCvs4UdAg9PVQVtgOno-rtMS4i5YajrjkyGw@mail.gmail.com>
2025-08-28 14:57 ` Qing Zhao
2025-08-21 7:26 ` [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
` (3 subsequent siblings)
6 siblings, 2 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
This series implements the Linux Kernel Control Flow Integrity ABI,
which provides a function prototype based forward edge control flow
integrity protection by instrumenting every indirect call to check for
a hash value before the target function address. If the hash at the call
site and the hash at the target do not match, execution will trap.
Just to set expectations, this is an RFC because this is my first time
working on most of the affected areas in GCC, and it is likely I have
missed really obvious stuff, or gone about doing things in very wrong
ways. I tried to find the best way to do stuff, but I was left with many
questions. :) All that said, this works for x86_64 and aarch64 Linux
kernels. (I have implemented riscv64 as well, but I lack a viable test
environment -- I am working on this still.)
KCFI has a number of specific constraints. Some are tied to the
backend architecture, which I'll cover in more detail in later patches.
The constraints are:
- The KCFI scheme generates a unique 32-bit hash for each unique function
prototype, allowing for indirect call sites to verify that they are
calling into a matching _type_ of function pointer. This changes the
semantics of some optimization logic because now indirect calls to
different types cannot be merged. For example:
if (p->func_type_1)
return p->func_type_1();
if (p->func_type_2)
return p->func_type_2();
In final asm, the optimizer may collapse the second indirect call
into a jump to the first indirect call once it has loaded the function
pointer. KCFI must block cross-type merging otherwise there will be a
single KCFI check happening for only 1 type but being used by 2 target
types. The distinguishing characteristic for call merging becomes the
type, not the address/register usage.
- The check-call instruction sequence must be treated a single unit: it
cannot be rearranged or split or optimized. The pattern is that
indirect calls, "call *$target", get converted into:
mov $target_expression, %target (only present if the expression was
not already %target)
load -$offset(%target), %tmp
cmp $hash, %tmp
je .Lcheck_passed
.Ltrap$N:
trap
.Lcheck_passed$N:
call *%target
This pattern of call immediately after trap provides for the
"permissive" checking mode automatically: the trap gets handled,
a warning emitted, and then execution continues after the trap to
the call.
(x86_64 uses "mov -$hash, %tmp; addl -$offset(%target), %tmp; je"
to zero out the register before making the call. Also Linux needs
exactly these insns because it is both disassembling them during
trap handling and potentially live patching them at boot time to
be converted into a different series of instrutions, a scheme
know as FineIBT, making the insn sequence ABI.)
- KCFI check-call instrumentation must survive tail call optimization.
If an indirect call is turned into an indirect jump, KCFI checking
must still happen (but will still use the jmp).
- Functions that may be called indirectly have a preamble added,
__cfi_$original_func_name, that contains the $hash value:
__cfi_target_func:
.word $hash
target_func:
[regular function entry...]
(x86_64 uses a movl instruction to hold the hash and prefixed aligned
NOPs to maintain cache line alignment in the face of patchable function
entry...)
- The preamble needs to interact with patchable function entry so that
the hash appears further away from the actual start of the function
(leaving the prefix NOPs of the patchable function entry unchanged).
This means only _globally defined_ patchable function entry is supported
with KCFI (indrect call sites must know in advance what the offset is,
which may not be possible extern functions). For example, a "4,4"
patchable function entry would end up like:
__cfi_target_func:
.data $hash
nop nop nop nop
target_func:
[regular function entry...]
(Linux x86_64 uses an 11 byte prefix nop area resulting in 16 bytes
total including the movl. This region may be live patched at boot time
for FineIBT so the behavior here is also ABI.)
- External functions that are address-taken have a weak __kcfi_typeid_$funcname
symbol added with the hash value available so that the hash can be referenced
from assembly linkages, etc, where the hash values cannot be calculated (i.e
where C type information is missing):
.weak __kcfi_typeid_$func
.set __kcfi_typeid_$func, $hash
- On architectures that do not have a good way to encode additional
details in their trap (x86_64 and riscv64), the trap location
is identified as a KCFI trap via a relative address offset entry
emitted into the .kcfi_traps section for each indirect call site's
trap instruction. The previous check-call example's insn sequence has
a section push/pop inserted between the trap and call:
...
.Ltrap$N:
trap
.pushsection .kcfi_traps,"ao",@progbits,.text
.Lentry$N:
.long .Ltrap$N - .Lentry$N
.popsection
.Lcheck_passed$N:
call %target
(aarch64 encodes the register numbers that hold the expected hash
and the target address in the trap ESR and thereby does not need a
.kcfi_traps section at all.)
- The no_sanitize("kcfi") function attribute means that the marked function
must not produce KCFI checking for indirect calls, and that this
attribute must survive inlining. This is used rarely by Linux, but
is required to make BPF JIT trampolines work on older Linux kernel
versions. (The preamble code is very recently finally being generated
at JIT time on the last remaining Linux KCFI arch where this was
missing: aarch64.)
As a result of these constraints, there are some behavioral aspects
that need to be preserved across the middle-end and back-end, as I
understand them.
For indirect call sites:
- Keeping indirect calls from being merged (see above). I did this by
adding a wrapping type so that equality was tested based on type-id.
This is done in create_kcfi_wrapper_type(), via kcfi_instrument(),
via an early GIMPLE pass (pass_kcfi and pass_kcfi0). The wrapper type
is checked in gcc/cfgcleanup.cc, old_insns_match_p().
- Keeping typeid information available through to the RTL expansion
phase was done via a typeid note (REG_CALL_KCFI_TYPE) attached also
in create_kcfi_wrapper_type() in the same GIMPLE pass.
- REG_CALL_KCFI_TYPE notes needed to survive optimization passes so RTL
expansion could find them again. These are retained in gcc/emit-rtl.cc,
try_split(), and gcc/combine.cc, distribute_notes().
- The expansion to RTL is handled in gcc/cfgexpand.cc, expand_call_stmt().
This lets the call expansion logic run, but then marks the resulting
call RTL with REG_CALL_KCFI_TYPE so the last RTL instrumentation pass
can find it. To me, this logic feels the most weird. It seems like
there should be a better place to do this, but I do see that similar
behavioral needs are also here, like nocf_check, so perhaps it's not
far off.
- The same expand_call_stmt() logic also adds the clobbers that will be
present in the final KCFI check-call insn sequences, which need to be
added now so that register allocation is aware of them while working
through optimization passes. Without this, the clobbered registers
may get used in the RTL before we have replaced the call instructions
that actually have the clobbers associated with them. This is the
"have cake and eat it too" case where we have to do the check-call RTL
replacement very late, but need to make sure the clobbers are known
very early. Again, this seems like there should be a better way...
- To emit the exact KCFI check-call instruction sequences, a very late
RTL pass is used (pass_kcfi_final_instrumentation). In order to
handle sibling calls (tail calls), this pass chooses between either
a check-call insn sequence or a check-jump insn sequence (provided by
the per-arch back-end).
- To make sure instrumentation was skipped for inline functions, the
RTL pass walks the basic blocks to identify their function origins,
looking for the no_sanitize("kcfi") attribute, and skipping
instrumentation if found.
For indirect call targets:
- kcfi_emit_preamble_if_needed() uses function_needs_kcfi_preamble(),
and counter helpers, to emit the preablem, with patchable function
entry NOPs. This gets used both in default_print_patchable_function_entry()
and the per-arch path. I could not find a simpler way to deal with
patchable function entry besides splitting it up like this. I feel
like there should be a better way.
- gcc/varasm.cc, assemble_external_real() calls emit_kcfi_typeid_symbol()
to add the __kcfi_typeid symbols (see get_function_kcfi_type_id()
below).
To support the per-arch back-ends, there are some common helpers:
- A callback framework is added via struct kcfi_target_hooks for
backends to fill out.
- kcfi_emit_trap_with_section() handles the push/pop section and
generating the relative offset section entries.
- get_function_kcfi_type_id() generates the 32-bit hash value, using
compute_kcfi_type_id() and kcfi_hash_string() to hook to the mangling
API. The hash is FNV-1a right now: it doesn't need secrecy. It could be
replaced with any hash, though the hash will need to be coordinated
with Rust, which implements the KCFI ABI as well.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/Makefile.in | 1 +
gcc/flag-types.h | 2 +
gcc/kcfi.h | 85 +++++
gcc/tree-pass.h | 1 +
gcc/cfgcleanup.cc | 20 ++
gcc/cfgexpand.cc | 61 ++++
gcc/combine.cc | 1 +
gcc/doc/invoke.texi | 29 ++
gcc/emit-rtl.cc | 1 +
gcc/kcfi.cc | 783 ++++++++++++++++++++++++++++++++++++++++++++
gcc/opts.cc | 1 +
gcc/passes.cc | 1 +
gcc/passes.def | 3 +
gcc/recog.cc | 1 +
gcc/reg-notes.def | 6 +
gcc/targhooks.cc | 50 ++-
gcc/toplev.cc | 8 +
gcc/varasm.cc | 18 +
18 files changed, 1071 insertions(+), 1 deletion(-)
create mode 100644 gcc/kcfi.h
create mode 100644 gcc/kcfi.cc
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 86f62611c1d4..01cccd88fde0 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1593,6 +1593,7 @@ OBJS = \
ira-emit.o \
ira-lives.o \
jump.o \
+ kcfi.o \
langhooks.o \
late-combine.o \
lcm.o \
diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index 33c88a15ecbb..7ed6dab1bd4b 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -337,6 +337,8 @@ enum sanitize_code {
SANITIZE_KERNEL_HWADDRESS = 1ULL << 30,
/* Shadow Call Stack. */
SANITIZE_SHADOW_CALL_STACK = 1ULL << 31,
+ /* KCFI (Kernel Control Flow Integrity) */
+ SANITIZE_KCFI = 1ULL << 32,
SANITIZE_SHIFT = SANITIZE_SHIFT_BASE | SANITIZE_SHIFT_EXPONENT,
SANITIZE_UNDEFINED = SANITIZE_SHIFT | SANITIZE_DIVIDE | SANITIZE_UNREACHABLE
| SANITIZE_VLA | SANITIZE_NULL | SANITIZE_RETURN
diff --git a/gcc/kcfi.h b/gcc/kcfi.h
new file mode 100644
index 000000000000..23d679b73ba4
--- /dev/null
+++ b/gcc/kcfi.h
@@ -0,0 +1,85 @@
+/* Kernel Control Flow Integrity (KCFI) support for GCC.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_KCFI_H
+#define GCC_KCFI_H
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "rtl.h"
+
+/* Get KCFI type ID for a function declaration. */
+extern uint32_t get_function_kcfi_type_id (tree fndecl);
+
+/* KCFI target hooks for architecture-specific functionality. */
+
+struct kcfi_target_hooks {
+ /* Apply architecture-specific masking to type ID. */
+ uint32_t (*mask_type_id) (uint32_t type_id);
+
+ /* Generate bundled KCFI checked call (atomic check + call to prevent optimizer separation) */
+ rtx (*gen_kcfi_checked_call) (rtx call_insn, rtx target_reg, uint32_t expected_type, HOST_WIDE_INT prefix_nops);
+
+ /* Add architecture-specific register clobbers for KCFI calls. */
+ void (*add_kcfi_clobbers) (rtx_insn *call_insn);
+
+ /* Calculate architecture-specific prefix NOPs count (optional, returns prefix_nops unchanged if NULL) */
+ int (*calculate_prefix_nops) (HOST_WIDE_INT prefix_nops);
+
+ /* Emit architecture-specific type ID instruction (required for common preamble helper) */
+ void (*emit_type_id_instruction) (FILE *file, uint32_t type_id);
+};
+
+/* Global KCFI target hooks. */
+extern struct kcfi_target_hooks kcfi_target;
+
+/* Common helper for RTL patterns to emit .kcfi_traps section entry.
+ Call AFTER emitting trap label and instruction with the RTX label operand. */
+extern void kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx);
+
+/* RTL note management for KCFI. */
+
+/* Add KCFI type ID note to call instruction. */
+extern void add_kcfi_type_note (rtx_insn *call_insn, uint32_t type_id);
+
+/* Emit KCFI type ID symbol for address-taken functions. */
+extern void emit_kcfi_typeid_symbol (FILE *asm_file, tree decl, const char *name);
+
+/* KCFI preamble emission coordination. */
+
+/* Mark that KCFI preamble has been emitted to prevent duplication. */
+extern void mark_kcfi_preamble_emitted (void);
+
+/* Central manager for all KCFI preamble generation decisions. */
+extern void kcfi_emit_preamble_if_needed (FILE *file, tree decl,
+ bool is_patchable_context,
+ HOST_WIDE_INT prefix_nops,
+ const char *actual_fname);
+
+/* Pass creation functions. */
+class gimple_opt_pass;
+class rtl_opt_pass;
+namespace gcc { class context; }
+
+extern gimple_opt_pass *make_pass_kcfi (gcc::context *ctxt);
+extern gimple_opt_pass *make_pass_kcfi_O0 (gcc::context *ctxt);
+extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context *ctxt);
+
+#endif /* GCC_KCFI_H */
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 1c68a69350df..2affea230213 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -561,6 +561,7 @@ extern gimple_opt_pass *make_pass_fixup_cfg (gcc::context *ctxt);
extern gimple_opt_pass *make_pass_backprop (gcc::context *ctxt);
extern rtl_opt_pass *make_pass_expand (gcc::context *ctxt);
+extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context *ctxt);
extern rtl_opt_pass *make_pass_instantiate_virtual_regs (gcc::context *ctxt);
extern rtl_opt_pass *make_pass_rtl_fwprop (gcc::context *ctxt);
extern rtl_opt_pass *make_pass_rtl_fwprop_addr (gcc::context *ctxt);
diff --git a/gcc/cfgcleanup.cc b/gcc/cfgcleanup.cc
index d28d23231911..9707b7e22222 100644
--- a/gcc/cfgcleanup.cc
+++ b/gcc/cfgcleanup.cc
@@ -1238,6 +1238,26 @@ old_insns_match_p (int mode ATTRIBUTE_UNUSED, rtx_insn *i1, rtx_insn *i2)
if (RTX_FRAME_RELATED_P (i1) && !insns_have_identical_cfa_notes (i1, i2))
return dir_none;
+ /* KCFI (Kernel Control Flow Integrity): Do not cross-jump between different
+ KCFI check patterns. Each bundled KCFI call has a unique type ID that must
+ be preserved to prevent type confusion attacks. */
+ if (CALL_P (i1) && CALL_P (i2))
+ {
+ rtx kcfi_note1 = find_reg_note (i1, REG_CALL_KCFI_TYPE, NULL_RTX);
+ rtx kcfi_note2 = find_reg_note (i2, REG_CALL_KCFI_TYPE, NULL_RTX);
+
+ if (kcfi_note1 || kcfi_note2)
+ {
+ /* If only one has KCFI note, they're different. */
+ if (!kcfi_note1 || !kcfi_note2)
+ return dir_none;
+
+ /* If both have KCFI notes, compare the type IDs. */
+ if (!rtx_equal_p (XEXP (kcfi_note1, 0), XEXP (kcfi_note2, 0)))
+ return dir_none;
+ }
+ }
+
#ifdef STACK_REGS
/* If cross_jump_death_matters is not 0, the insn's mode
indicates whether or not the insn contains any stack-like
diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc
index 8950294abb60..432ccc7f864a 100644
--- a/gcc/cfgexpand.cc
+++ b/gcc/cfgexpand.cc
@@ -76,6 +76,7 @@ along with GCC; see the file COPYING3. If not see
#include "opts.h"
#include "gimple-range.h"
#include "rtl-iter.h"
+#include "kcfi.h"
/* Some systems use __main in a way incompatible with its use in gcc, in these
cases use the macros NAME__MAIN to give a quoted symbol and SYMBOL__MAIN to
@@ -3203,6 +3204,66 @@ expand_call_stmt (gcall *stmt)
else
expand_expr (exp, const0_rtx, VOIDmode, EXPAND_NORMAL);
+ /* Add KCFI annotations if this is an indirect call with KCFI wrapper type. */
+ if (sanitize_flags_p (SANITIZE_KCFI) && !gimple_call_fndecl (stmt))
+ {
+ tree fn_type = gimple_call_fntype (stmt);
+ gcc_assert (fn_type);
+ tree attr = lookup_attribute ("kcfi_type_id", TYPE_ATTRIBUTES (fn_type));
+ if (attr && TREE_VALUE (attr))
+ {
+ /* Check if this call site originated from a no_sanitize("kcfi") function
+ during inlining. If so, skip KCFI instrumentation in RTL phase too. */
+ bool should_skip_kcfi = false;
+ location_t call_location = gimple_location (stmt);
+ gcc_assert (call_location != UNKNOWN_LOCATION);
+
+ tree block = gimple_block (stmt);
+ while (block && TREE_CODE (block) == BLOCK)
+ {
+ tree fn_decl = BLOCK_ABSTRACT_ORIGIN (block);
+ if (fn_decl && TREE_CODE (fn_decl) == FUNCTION_DECL)
+ {
+ /* Found an inlined function - check if it has no_sanitize("kcfi"). */
+ if (!sanitize_flags_p (SANITIZE_KCFI, fn_decl))
+ {
+ should_skip_kcfi = true;
+ break;
+ }
+ break;
+ }
+ /* Move up the block chain to find parent inlined functions. */
+ block = BLOCK_SUPERCONTEXT (block);
+ }
+
+ if (!should_skip_kcfi)
+ {
+ uint32_t kcfi_type_id = (uint32_t) tree_to_uhwi (TREE_VALUE (attr));
+
+ /* Find the call that has been created. */
+ rtx_insn *call_insn = get_last_insn ();
+ while (call_insn && call_insn != before_call && !CALL_P (call_insn))
+ call_insn = PREV_INSN (call_insn);
+
+ if (call_insn && call_insn != before_call && CALL_P (call_insn))
+ {
+ /* Add KCFI type ID note for anti-merging protection. */
+ add_kcfi_type_note (call_insn, kcfi_type_id);
+
+ /* Add architecture-specific clobbers so register allocator knows
+ they'll be used. */
+ if (kcfi_target.add_kcfi_clobbers)
+ kcfi_target.add_kcfi_clobbers (call_insn);
+ }
+ else
+ {
+ error ("KCFI: Could not find call instruction for wrapper type");
+ gcc_unreachable ();
+ }
+ }
+ }
+ }
+
/* If the gimple call is an indirect call and has 'nocf_check'
attribute find a generated CALL insn to mark it as no
control-flow verification is needed. */
diff --git a/gcc/combine.cc b/gcc/combine.cc
index 4dbc1f6a4a4e..efaeb426d254 100644
--- a/gcc/combine.cc
+++ b/gcc/combine.cc
@@ -14525,6 +14525,7 @@ distribute_notes (rtx notes, rtx_insn *from_insn, rtx_insn *i3, rtx_insn *i2,
case REG_CALL_DECL:
case REG_UNTYPED_CALL:
case REG_CALL_NOCF_CHECK:
+ case REG_CALL_KCFI_TYPE:
/* These notes must remain with the call. It should not be
possible for both I2 and I3 to be a call. */
if (CALL_P (i3))
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index c1e708beacf3..c66f47336826 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -18294,6 +18294,35 @@ possible by specifying the command-line options
@option{--param hwasan-instrument-allocas=1} respectively. Using a random frame
tag is not implemented for kernel instrumentation.
+@opindex fsanitize=kcfi
+@item -fsanitize=kcfi
+Enable Kernel Control Flow Integrity (KCFI), a lightweight control
+flow integrity mechanism designed for operating system kernels.
+KCFI instruments indirect function calls to verify that the target
+function has the expected type signature at runtime. Each function
+receives a unique type identifier computed from a hash of its function
+prototype (including parameter types and return type). Before each
+indirect call, the implementation inserts a check to verify that the
+target function's type identifier matches the expected identifier
+for the call site, terminating the program if a mismatch is detected.
+This provides forward-edge control flow protection against attacks that
+attempt to redirect indirect calls to unintended targets.
+
+The implementation adds minimal runtime overhead and does not require
+runtime library support, making it suitable for kernel environments.
+The type identifier is placed before the function entry point,
+allowing runtime verification without additional metadata structures,
+and without changing the entry points of the target functions. Only
+functions that have referenced by their address receive the KCFI preamble
+instrumentation.
+
+KCFI is intended primarily for kernel code and may not be suitable
+for user-space applications that rely on techniques incompatible
+with strict type checking of indirect calls.
+
+Use @option{-fdump-tree-kcfi} to examine the computed type identifiers
+and their corresponding mangled type strings during compilation.
+
@opindex fsanitize=pointer-compare
@item -fsanitize=pointer-compare
Instrument comparison operation (<, <=, >, >=) with pointer operands.
diff --git a/gcc/emit-rtl.cc b/gcc/emit-rtl.cc
index f4fc92bb37a1..aa56112bc250 100644
--- a/gcc/emit-rtl.cc
+++ b/gcc/emit-rtl.cc
@@ -4056,6 +4056,7 @@ try_split (rtx pat, rtx_insn *trial, int last)
case REG_SETJMP:
case REG_TM:
case REG_CALL_NOCF_CHECK:
+ case REG_CALL_KCFI_TYPE:
case REG_CALL_ARG_LOCATION:
for (insn = insn_last; insn != NULL_RTX; insn = PREV_INSN (insn))
{
diff --git a/gcc/kcfi.cc b/gcc/kcfi.cc
new file mode 100644
index 000000000000..522bb97bf503
--- /dev/null
+++ b/gcc/kcfi.cc
@@ -0,0 +1,783 @@
+/* Kernel Control Flow Integrity (KCFI) support for GCC.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "target.h"
+#include "function.h"
+#include "tree.h"
+#include "tree-pass.h"
+#include "dumpfile.h"
+#include "basic-block.h"
+#include "gimple.h"
+#include "gimple-iterator.h"
+#include "cgraph.h"
+#include "kcfi.h"
+#include "stringpool.h"
+#include "attribs.h"
+#include "rtl.h"
+#include "cfg.h"
+#include "asan.h"
+#include "diagnostic-core.h"
+#include "memmodel.h"
+#include "emit-rtl.h"
+#include "output.h"
+#include "varasm.h"
+#include "opts.h"
+#include "mangle.h"
+
+/* Global KCFI target hooks structure - zero-initialized for safe defaults. */
+struct kcfi_target_hooks kcfi_target = { };
+
+/* Common KCFI utilities. */
+
+/* Common helper for RTL patterns to emit .kcfi_traps section entry. */
+void
+kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx)
+{
+ /* Convert trap label to string using standard GCC helper. */
+ char trap_name[64];
+ ASM_GENERATE_INTERNAL_LABEL (trap_name, "L", CODE_LABEL_NUMBER (trap_label_rtx));
+
+ /* Generate entry label name from trap label number. */
+ char entry_name[64];
+ ASM_GENERATE_INTERNAL_LABEL (entry_name, "Lentry", CODE_LABEL_NUMBER (trap_label_rtx));
+
+ /* Emit .kcfi_traps section entry using the converted labels. */
+ fprintf (file, "\t.pushsection\t.kcfi_traps,\"ao\",@progbits,.text\n");
+ assemble_name (file, entry_name);
+ fprintf (file, ":\n");
+ fprintf (file, "\t.long\t");
+ assemble_name (file, trap_name);
+ fprintf (file, " - ");
+ assemble_name (file, entry_name);
+ fprintf (file, "\n");
+ fprintf (file, "\t.popsection\n");
+}
+
+/* Hash function for KCFI type ID computation.
+ This implements a simple hash similar to FNV-1a. */
+static uint32_t
+kcfi_hash_string (const char *str)
+{
+ uint32_t hash = 2166136261U; /* FNV-1a 32-bit offset basis. */
+ for (const char *p = str; *p; p++)
+ {
+ hash ^= (unsigned char) *p;
+ hash *= 16777619U; /* FNV-1a 32-bit prime. */
+ }
+ return hash;
+}
+
+/* Compute KCFI type ID for a function declaration or function type (internal) */
+static uint32_t
+compute_kcfi_type_id (tree fntype_or_fndecl)
+{
+ if (!fntype_or_fndecl)
+ return 0;
+
+ const char *canonical_name = mangle_function_type (fntype_or_fndecl);
+ uint32_t base_type_id = kcfi_hash_string (canonical_name);
+
+ /* Apply target-specific masking if supported. */
+ if (kcfi_target.mask_type_id)
+ base_type_id = kcfi_target.mask_type_id (base_type_id);
+
+ /* Output to dump file if enabled */
+ if (dump_file && (dump_flags & TDF_DETAILS))
+ {
+ const char *name = NULL;
+ if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
+ {
+ if (DECL_NAME (fntype_or_fndecl))
+ name = IDENTIFIER_POINTER (DECL_NAME (fntype_or_fndecl));
+ }
+
+ if (name)
+ fprintf (dump_file, "KCFI type ID for function '%s': mangled='%s' typeid=0x%08x\n",
+ name, canonical_name, base_type_id);
+ else
+ fprintf (dump_file, "KCFI type ID: mangled='%s' typeid=0x%08x\n",
+ canonical_name, base_type_id);
+ }
+
+ return base_type_id;
+}
+
+/* Check if a function needs KCFI preamble generation.
+ ALL functions get preambles when -fsanitize=kcfi is enabled, regardless
+ of no_sanitize("kcfi") attribute. */
+static bool
+function_needs_kcfi_preamble (tree fndecl)
+{
+ /* Only instrument if KCFI is globally enabled. */
+ if (!(flag_sanitize & SANITIZE_KCFI))
+ return false;
+
+ struct cgraph_node *node = cgraph_node::get (fndecl);
+
+ /* Ignore cold partition functions: not reached via indirect call. */
+ if (node && node->split_part)
+ return false;
+
+ /* Ignore cold partition sections: cold partitions are never indirect call
+ targets. Only skip preambles for cold partitions (has_bb_partition = true)
+ not for entire cold-attributed functions (has_bb_partition = false). */
+ if (in_cold_section_p && crtl && crtl->has_bb_partition)
+ return false;
+
+ /* Check if function is truly address-taken using cgraph node analysis. */
+ bool addr_taken = (node && node->address_taken);
+
+ /* Only instrument functions that can be targets of indirect calls:
+ - Public functions (can be called externally)
+ - External declarations (from other modules)
+ - Functions with true address-taken status from cgraph analysis. */
+ return TREE_PUBLIC (fndecl) || DECL_EXTERNAL (fndecl) || addr_taken;
+}
+
+/* Function attribute to store KCFI type ID. */
+static tree kcfi_type_id_attr = NULL_TREE;
+
+/* Get KCFI type ID for a function declaration. */
+uint32_t
+get_function_kcfi_type_id (tree fndecl)
+{
+ if (!kcfi_type_id_attr)
+ kcfi_type_id_attr = get_identifier ("kcfi_type_id");
+
+ tree attr = lookup_attribute_by_prefix ("kcfi_type_id", DECL_ATTRIBUTES (fndecl));
+ if (attr && TREE_VALUE (attr) && TREE_VALUE (TREE_VALUE (attr)))
+ {
+ tree value = TREE_VALUE (TREE_VALUE (attr));
+ if (TREE_CODE (value) == INTEGER_CST)
+ return (uint32_t) TREE_INT_CST_LOW (value);
+ }
+
+ /* Compute and cache type ID using original parameter declarations. */
+ uint32_t type_id = compute_kcfi_type_id (fndecl);
+
+ tree type_id_tree = build_int_cst (unsigned_type_node, type_id);
+ tree attr_value = build_tree_list (NULL_TREE, type_id_tree);
+ attr = build_tree_list (kcfi_type_id_attr, attr_value);
+
+ DECL_ATTRIBUTES (fndecl) = chainon (DECL_ATTRIBUTES (fndecl), attr);
+
+ return type_id;
+}
+
+/* Get the number of patchable prefix NOPs for the current function. */
+static HOST_WIDE_INT
+get_current_function_patchable_prefix_nops (void)
+{
+ HOST_WIDE_INT prefix_nops = 0;
+
+ /* Check for function-specific patchable_function_entry attribute. */
+ tree patchable_attr = lookup_attribute ("patchable_function_entry",
+ DECL_ATTRIBUTES (current_function_decl));
+ if (patchable_attr)
+ {
+ tree pp_val = TREE_VALUE (patchable_attr);
+ /* total_nops = tree_to_uhwi (TREE_VALUE (pp_val)); */
+ if (TREE_CHAIN (pp_val))
+ prefix_nops = tree_to_uhwi (TREE_VALUE (TREE_CHAIN (pp_val)));
+ }
+ else
+ {
+ /* Use global configuration if no function-specific attribute. */
+ HOST_WIDE_INT total_nops, patch_area_entry;
+ parse_and_check_patch_area (flag_patchable_function_entry, false,
+ &total_nops, &patch_area_entry);
+ prefix_nops = patch_area_entry;
+ }
+
+ return prefix_nops;
+}
+
+/* Check if this is an indirect call that needs KCFI instrumentation. */
+static bool
+is_kcfi_indirect_call (tree fn)
+{
+ if (!fn)
+ return false;
+
+ /* Only functions WITHOUT no_sanitize("kcfi") should generate KCFI checks at
+ indirect call sites. */
+ if (!sanitize_flags_p (SANITIZE_KCFI, current_function_decl))
+ return false;
+
+ /* Direct function calls via ADDR_EXPR don't need KCFI checks. */
+ if (TREE_CODE (fn) == ADDR_EXPR)
+ return false;
+
+ /* Function pointers, variables, and other indirect calls need KCFI. */
+ return true;
+}
+
+/* Extract target from call instruction RTL pattern.
+ Handles the RTL pattern matching needed to find the rtx containing
+ the function pointer for indirect calls. */
+static rtx
+kcfi_find_call_target (rtx_insn *call_insn)
+{
+ if (!call_insn || (!CALL_P (call_insn) && !JUMP_P (call_insn)))
+ return NULL_RTX;
+
+ rtx call_pattern = PATTERN (call_insn);
+ rtx target_reg = NULL_RTX;
+
+ if (CALL_P (call_insn) && GET_CODE (call_pattern) == PARALLEL)
+ {
+ /* Handle PARALLEL patterns (need to extract CALL) */
+ rtx call_part = XVECEXP (call_pattern, 0, 0);
+ call_pattern = call_part;
+ }
+
+ if (CALL_P (call_insn) && GET_CODE (call_pattern) == CALL)
+ {
+ /* Handle CALL patterns. */
+ rtx mem = XEXP (call_pattern, 0);
+ if (GET_CODE (mem) == MEM)
+ {
+ target_reg = XEXP (mem, 0);
+ }
+ }
+ else if (CALL_P (call_insn) && GET_CODE (call_pattern) == SET)
+ {
+ /* Handle SET patterns (function calls with return values) */
+ rtx src = XEXP (call_pattern, 1);
+ if (GET_CODE (src) == CALL)
+ {
+ rtx mem = XEXP (src, 0);
+ if (GET_CODE (mem) == MEM)
+ {
+ target_reg = XEXP (mem, 0);
+ }
+ }
+ }
+ else if (JUMP_P (call_insn) && GET_CODE (call_pattern) == SET)
+ {
+ /* Handle jump patterns. */
+ rtx src = SET_SRC (call_pattern);
+ if (GET_CODE (src) == MEM) /* Direct indirect jump. */
+ {
+ target_reg = XEXP (src, 0);
+ }
+ }
+
+ return target_reg;
+}
+
+/* Add KCFI type ID note to call instruction. */
+void
+add_kcfi_type_note (rtx_insn *call_insn, uint32_t type_id)
+{
+ if (!call_insn || !CALL_P (call_insn))
+ return;
+
+ /* Create integer constant for type ID. */
+ rtx type_id_rtx = gen_int_mode (type_id, SImode);
+
+ /* Add note to call instruction. */
+ add_reg_note (call_insn, REG_CALL_KCFI_TYPE, type_id_rtx);
+}
+
+/* Global flag to coordinate KCFI preamble emission. */
+static bool kcfi_preamble_emitted = false;
+
+/* Mark that KCFI preamble has been emitted to prevent duplication. */
+void
+mark_kcfi_preamble_emitted (void)
+{
+ kcfi_preamble_emitted = true;
+}
+
+/* Emit KCFI type ID symbol for an address-taken function.
+ Centralized emission point to avoid duplication between
+ assemble_external_real() and assemble_start_function(). */
+void
+emit_kcfi_typeid_symbol (FILE *asm_file, tree decl, const char *name)
+{
+ uint32_t type_id = get_function_kcfi_type_id (decl);
+ fprintf (asm_file, "\t.weak\t__kcfi_typeid_%s\n", name);
+ fprintf (asm_file, "\t.set\t__kcfi_typeid_%s, 0x%08x\n", name, type_id);
+}
+
+/* Parse patchable function entry configuration from global flags. */
+static bool
+parse_patchable_function_entry_config (void)
+{
+ HOST_WIDE_INT total_nops, prefix_nops;
+
+ if (!flag_patchable_function_entry)
+ return false;
+
+ parse_and_check_patch_area (flag_patchable_function_entry, false, &total_nops, &prefix_nops);
+ return (total_nops > 0);
+}
+
+/* Common KCFI preamble helper that handles shared logic across architectures. */
+static void
+kcfi_emit_cfi_preamble (FILE *file, tree decl, HOST_WIDE_INT prefix_nops, const char *fname)
+{
+ /* Get type ID. */
+ uint32_t type_id = get_function_kcfi_type_id (decl);
+
+ /* Create symbol name for reuse. */
+ char cfi_symbol_name[256];
+ snprintf (cfi_symbol_name, sizeof(cfi_symbol_name), "__cfi_%s", fname);
+
+ /* Emit __cfi_ symbol with proper visibility. */
+ if (TREE_PUBLIC (decl))
+ {
+ if (DECL_WEAK (decl))
+ ASM_WEAKEN_LABEL (file, cfi_symbol_name);
+ else
+ targetm.asm_out.globalize_label (file, cfi_symbol_name);
+ }
+
+ /* Emit .type directive. */
+ ASM_OUTPUT_TYPE_DIRECTIVE (file, cfi_symbol_name, "function");
+ fprintf (file, "%s:\n", cfi_symbol_name);
+
+ /* Calculate architecture-specific prefix NOPs if hook provided. */
+ int final_nops = prefix_nops;
+ if (kcfi_target.calculate_prefix_nops)
+ final_nops = kcfi_target.calculate_prefix_nops (prefix_nops);
+
+ /* Emit architecture-specific prefix NOPs. */
+ for (int i = 0; i < final_nops; i++)
+ {
+ fprintf (file, "\tnop\n");
+ }
+
+ /* Emit architecture-specific type ID instruction. */
+ if (kcfi_target.emit_type_id_instruction)
+ kcfi_target.emit_type_id_instruction (file, type_id);
+
+ /* Mark end of __cfi_ symbol and emit size directive. */
+ char cfi_end_label[256];
+ snprintf (cfi_end_label, sizeof(cfi_end_label), ".Lcfi_func_end_%s", fname);
+ ASM_OUTPUT_LABEL (file, cfi_end_label);
+
+ ASM_OUTPUT_MEASURED_SIZE (file, cfi_symbol_name);
+}
+
+void
+kcfi_emit_preamble_if_needed (FILE *file, tree decl,
+ bool is_patchable_context,
+ HOST_WIDE_INT prefix_nops,
+ const char *actual_fname)
+{
+ /* Check if KCFI is enabled and function needs preamble. */
+ if (!function_needs_kcfi_preamble (decl))
+ return;
+
+ /* Use actual function name if provided, otherwise fall back to DECL_ASSEMBLER_NAME. */
+ const char *fname = actual_fname ? actual_fname : IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (decl));
+
+ /* Determine if we should emit here based on context. */
+ bool has_patchable_entries = parse_patchable_function_entry_config ();
+
+ if (is_patchable_context && has_patchable_entries)
+ {
+ /* Use the provided prefix_nops parameter. */
+ }
+ else if (!is_patchable_context && !has_patchable_entries)
+ {
+ /* Emit for standard non-patchable functions. */
+ prefix_nops = 0;
+ }
+ else
+ {
+ /* Skip to avoid duplication:
+ - Patchable context but no global patchable config
+ - Non-patchable context but has global patchable config. */
+ return;
+ }
+
+ kcfi_emit_cfi_preamble (file, decl, prefix_nops, fname);
+}
+
+/* KCFI Final Instrumentation Pass - Post-Optimization Implementation
+
+ This pass runs after all optimizations to insert KCFI checks immediately
+ before indirect calls/jumps, solving the fundamental problem where
+ optimization would insert code between KCFI checks and their protected
+ indirect transfers. */
+
+namespace {
+
+const pass_data pass_data_kcfi_final_instrumentation =
+{
+ RTL_PASS, /* type */
+ "kcfi_final_instrumentation", /* name */
+ OPTGROUP_NONE, /* optinfo_flags */
+ TV_NONE, /* tv_id */
+ 0, /* properties_required */
+ 0, /* properties_provided */
+ 0, /* properties_destroyed */
+ 0, /* todo_flags_start */
+ 0, /* todo_flags_finish */
+};
+
+class pass_kcfi_final_instrumentation : public rtl_opt_pass
+{
+public:
+ pass_kcfi_final_instrumentation (gcc::context *ctxt)
+ : rtl_opt_pass (pass_data_kcfi_final_instrumentation, ctxt)
+ {}
+
+ /* opt_pass methods: */
+ bool gate (function *) final override
+ {
+ /* Note: Target KCFI support is validated in toplev.cc during option processing.
+ If the target doesn't provide gen_kcfi_checked_call, compilation stops with
+ a "sorry" message before any passes run. */
+ return sanitize_flags_p (SANITIZE_KCFI);
+ }
+
+ unsigned int execute (function *) final override;
+};
+
+/* Check if an RTL instruction is an indirect call/jump marked for KCFI. */
+static bool
+is_marked_indirect_transfer (rtx_insn *insn, uint32_t *type_id)
+{
+ if (!insn || (!CALL_P (insn) && !JUMP_P (insn)))
+ return false;
+
+ /* Check for KCFI type ID note. */
+ rtx note = find_reg_note (insn, REG_CALL_KCFI_TYPE, NULL_RTX);
+ if (!note)
+ return false;
+
+ rtx type_rtx = XEXP (note, 0);
+ if (!CONST_INT_P (type_rtx))
+ return false;
+
+ *type_id = (uint32_t) INTVAL (type_rtx);
+
+ /* Verify it's actually an indirect transfer. */
+ rtx pattern = PATTERN (insn);
+
+ if (CALL_P (insn))
+ {
+ /* Handle call patterns. */
+ if (GET_CODE (pattern) == CALL)
+ {
+ rtx target = XEXP (pattern, 0);
+ return GET_CODE (target) == MEM;
+ }
+ else if (GET_CODE (pattern) == SET)
+ {
+ rtx src = SET_SRC (pattern);
+ if (GET_CODE (src) == CALL)
+ {
+ rtx target = XEXP (src, 0);
+ return GET_CODE (target) == MEM;
+ }
+ }
+ else if (GET_CODE (pattern) == PARALLEL)
+ {
+ for (int i = 0; i < XVECLEN (pattern, 0); i++)
+ {
+ rtx elem = XVECEXP (pattern, 0, i);
+ if (GET_CODE (elem) == CALL
+ || (GET_CODE (elem) == SET && GET_CODE (SET_SRC (elem)) == CALL))
+ {
+ rtx call_rtx = (GET_CODE (elem) == SET) ? SET_SRC (elem) : elem;
+ rtx target = XEXP (call_rtx, 0);
+ return GET_CODE (target) == MEM;
+ }
+ }
+ }
+ }
+ else if (JUMP_P (insn))
+ {
+ /* Handle jump patterns. */
+ if (GET_CODE (pattern) == SET)
+ {
+ rtx src = SET_SRC (pattern);
+ if (GET_CODE (src) == MEM) /* Direct indirect jump. */
+ return true;
+ /* Could also handle other indirect jump patterns here. */
+ }
+ }
+
+ return false;
+}
+
+/* Replace marked indirect transfer with KCFI-protected version. */
+unsigned int
+pass_kcfi_final_instrumentation::execute (function *fun)
+{
+ basic_block bb;
+
+ /* Scan all basic blocks for marked indirect transfers. */
+ FOR_EACH_BB_FN (bb, fun)
+ {
+ rtx_insn *insn, *next_insn;
+
+ /* Use safe iteration since we'll be modifying the instruction stream. */
+ for (insn = BB_HEAD (bb); insn && insn != NEXT_INSN (BB_END (bb)); insn = next_insn)
+ {
+ next_insn = NEXT_INSN (insn);
+
+ uint32_t type_id;
+ if (is_marked_indirect_transfer (insn, &type_id))
+ {
+ /* Extract target address from the call instruction. */
+ rtx target_addr = kcfi_find_call_target (insn);
+ if (!target_addr)
+ {
+ error ("KCFI: cannot find target for indirect call");
+ gcc_unreachable ();
+ }
+
+ HOST_WIDE_INT prefix_nops = get_current_function_patchable_prefix_nops ();
+
+ /* Generate bundled KCFI checked call. */
+ start_sequence ();
+
+ /* Pass the target as-is to the RTL pattern, which will handle
+ moving non-register targets to a register internally if needed. */
+ rtx bundled_call = kcfi_target.gen_kcfi_checked_call (insn, target_addr, type_id, prefix_nops);
+ if (!bundled_call)
+ {
+ error ("KCFI: instruction sequence creation failed");
+ gcc_unreachable ();
+ }
+
+ /* Emit as call_insn, not generic insn. */
+ emit_call_insn (bundled_call);
+ rtx replacement_seq = get_insns ();
+ if (!replacement_seq)
+ {
+ error ("KCFI: instruction sequence insertion failed");
+ gcc_unreachable ();
+ }
+
+ end_sequence ();
+
+ /* Check if original was a sibling call and preserve that flag. */
+ bool was_sibcall = CALL_P (insn) && SIBLING_CALL_P (insn);
+
+ /* Replace the original call entirely with bundled version. */
+ rtx_insn *new_insn = emit_insn_before (replacement_seq, insn);
+
+ /* If original was a sibling call, mark the new instruction as such. */
+ if (was_sibcall && new_insn && CALL_P (new_insn))
+ SIBLING_CALL_P (new_insn) = 1;
+
+ set_insn_deleted (insn); /* Mark original call as deleted. */
+ }
+ }
+ }
+
+ return 0;
+}
+
+} // anon namespace
+
+rtl_opt_pass *
+make_pass_kcfi_final_instrumentation (gcc::context *ctxt)
+{
+ return new pass_kcfi_final_instrumentation (ctxt);
+}
+
+/* KCFI GIMPLE pass implementation. */
+
+static bool
+gate_kcfi (void)
+{
+ return sanitize_flags_p (SANITIZE_KCFI);
+}
+
+/* Create a KCFI wrapper function type that embeds the type ID. */
+static tree
+create_kcfi_wrapper_type (tree original_fn_type, uint32_t type_id)
+{
+ /* Create a unique type name incorporating the type ID. */
+ char wrapper_name[64];
+ snprintf (wrapper_name, sizeof (wrapper_name), "__kcfi_wrapper_%x", type_id);
+
+ /* Build a new function type that's structurally identical but nominally different. */
+ tree wrapper_type = build_function_type (TREE_TYPE (original_fn_type),
+ TYPE_ARG_TYPES (original_fn_type));
+
+ /* Set the type name to make it distinct. */
+ TYPE_NAME (wrapper_type) = get_identifier (wrapper_name);
+
+ /* Attach kcfi_type_id attribute to the original function type for cfgexpand.cc */
+ tree attr_name = get_identifier ("kcfi_type_id");
+ tree attr_value = build_int_cst (unsigned_type_node, type_id);
+ tree attr = build_tree_list (attr_name, attr_value);
+ TYPE_ATTRIBUTES (original_fn_type) = chainon (TYPE_ATTRIBUTES (original_fn_type), attr);
+
+ return wrapper_type;
+}
+
+/* Replace indirect calls with KCFI wrapper types for anti-merging and clobber tracking. */
+static unsigned int
+kcfi_instrument (void)
+{
+ basic_block bb;
+
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ gimple_stmt_iterator gsi;
+ for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
+ {
+ gimple *stmt = gsi_stmt (gsi);
+
+ if (is_gimple_call (stmt))
+ {
+ gcall *call_stmt = as_a <gcall *> (stmt);
+
+ // Skip internal calls - we only instrument indirect calls
+ if (gimple_call_internal_p (call_stmt))
+ {
+ continue;
+ }
+
+ tree fndecl = gimple_call_fndecl (call_stmt);
+
+ // Only process indirect calls (no fndecl)
+ if (!fndecl)
+ {
+ tree fn = gimple_call_fn (call_stmt);
+ if (fn && is_kcfi_indirect_call (fn))
+ {
+ // Get the function type to compute KCFI type ID
+ tree fn_type = gimple_call_fntype (call_stmt);
+ gcc_assert (fn_type);
+ if (TREE_CODE (fn_type) == FUNCTION_TYPE)
+ {
+ uint32_t type_id = compute_kcfi_type_id (fn_type);
+
+ // Create KCFI wrapper type for this call
+ tree wrapper_type = create_kcfi_wrapper_type (fn_type, type_id);
+
+ // Create a temporary variable for the wrapped function pointer
+ tree wrapper_ptr_type = build_pointer_type (wrapper_type);
+ tree wrapper_tmp = create_tmp_var (wrapper_ptr_type, "kcfi_wrapper");
+
+ // Create assignment: wrapper_tmp = (wrapper_ptr_type) fn
+ tree cast_expr = build1 (NOP_EXPR, wrapper_ptr_type, fn);
+ gimple *cast_stmt = gimple_build_assign (wrapper_tmp, cast_expr);
+ gsi_insert_before (&gsi, cast_stmt, GSI_SAME_STMT);
+
+ // Update the call to use the wrapped function pointer
+ gimple_call_set_fn (call_stmt, wrapper_tmp);
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return 0;
+}
+
+namespace {
+
+const pass_data pass_data_kcfi =
+{
+ GIMPLE_PASS, /* type */
+ "kcfi", /* name */
+ OPTGROUP_NONE, /* optinfo_flags */
+ TV_NONE, /* tv_id */
+ ( PROP_ssa | PROP_cfg | PROP_gimple_leh ), /* properties_required */
+ 0, /* properties_provided */
+ 0, /* properties_destroyed */
+ 0, /* todo_flags_start */
+ TODO_update_ssa, /* todo_flags_finish */
+};
+
+class pass_kcfi : public gimple_opt_pass
+{
+public:
+ pass_kcfi (gcc::context *ctxt)
+ : gimple_opt_pass (pass_data_kcfi, ctxt)
+ {}
+
+ /* opt_pass methods: */
+ opt_pass * clone () final override { return new pass_kcfi (m_ctxt); }
+ bool gate (function *) final override
+ {
+ return gate_kcfi ();
+ }
+ unsigned int execute (function *) final override
+ {
+ return kcfi_instrument ();
+ }
+
+}; // class pass_kcfi
+
+} // anon namespace
+
+gimple_opt_pass *
+make_pass_kcfi (gcc::context *ctxt)
+{
+ return new pass_kcfi (ctxt);
+}
+
+namespace {
+
+const pass_data pass_data_kcfi_O0 =
+{
+ GIMPLE_PASS, /* type */
+ "kcfi0", /* name */
+ OPTGROUP_NONE, /* optinfo_flags */
+ TV_NONE, /* tv_id */
+ ( PROP_ssa | PROP_cfg | PROP_gimple_leh ), /* properties_required */
+ 0, /* properties_provided */
+ 0, /* properties_destroyed */
+ 0, /* todo_flags_start */
+ TODO_update_ssa, /* todo_flags_finish */
+};
+
+class pass_kcfi_O0 : public gimple_opt_pass
+{
+public:
+ pass_kcfi_O0 (gcc::context *ctxt)
+ : gimple_opt_pass (pass_data_kcfi_O0, ctxt)
+ {}
+
+ /* opt_pass methods: */
+ bool gate (function *) final override
+ {
+ return !optimize && gate_kcfi ();
+ }
+ unsigned int execute (function *) final override
+ {
+ return kcfi_instrument ();
+ }
+
+}; // class pass_kcfi_O0
+
+} // anon namespace
+
+gimple_opt_pass *
+make_pass_kcfi_O0 (gcc::context *ctxt)
+{
+ return new pass_kcfi_O0 (ctxt);
+}
diff --git a/gcc/opts.cc b/gcc/opts.cc
index 5fd86aa89adb..ab74414b9c5b 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -2168,6 +2168,7 @@ const struct sanitizer_opts_s sanitizer_opts[] =
SANITIZER_OPT (pointer-overflow, SANITIZE_POINTER_OVERFLOW, true, true),
SANITIZER_OPT (builtin, SANITIZE_BUILTIN, true, true),
SANITIZER_OPT (shadow-call-stack, SANITIZE_SHADOW_CALL_STACK, false, false),
+ SANITIZER_OPT (kcfi, SANITIZE_KCFI, false, true),
SANITIZER_OPT (all, ~0ULL, true, true),
#undef SANITIZER_OPT
{ NULL, 0ULL, 0UL, false, false }
diff --git a/gcc/passes.cc b/gcc/passes.cc
index a33c8d924a52..4c6ceac740ff 100644
--- a/gcc/passes.cc
+++ b/gcc/passes.cc
@@ -63,6 +63,7 @@ along with GCC; see the file COPYING3. If not see
#include "diagnostic-core.h" /* for fnotice */
#include "stringpool.h"
#include "attribs.h"
+#include "kcfi.h"
/* Reserved TODOs */
#define TODO_verify_il (1u << 31)
diff --git a/gcc/passes.def b/gcc/passes.def
index d528a0477d9a..d4c84ca7064d 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -275,6 +275,7 @@ along with GCC; see the file COPYING3. If not see
NEXT_PASS (pass_sink_code, false /* unsplit edges */);
NEXT_PASS (pass_sancov);
NEXT_PASS (pass_asan);
+ NEXT_PASS (pass_kcfi);
NEXT_PASS (pass_tsan);
NEXT_PASS (pass_dse, true /* use DR analysis */);
NEXT_PASS (pass_dce, false /* update_address_taken_p */, false /* remove_unused_locals */);
@@ -443,6 +444,7 @@ along with GCC; see the file COPYING3. If not see
NEXT_PASS (pass_sancov_O0);
NEXT_PASS (pass_lower_switch_O0);
NEXT_PASS (pass_asan_O0);
+ NEXT_PASS (pass_kcfi_O0);
NEXT_PASS (pass_tsan_O0);
NEXT_PASS (pass_musttail);
NEXT_PASS (pass_sanopt);
@@ -563,6 +565,7 @@ along with GCC; see the file COPYING3. If not see
NEXT_PASS (pass_delay_slots);
NEXT_PASS (pass_split_for_shorten_branches);
NEXT_PASS (pass_convert_to_eh_region_ranges);
+ NEXT_PASS (pass_kcfi_final_instrumentation);
NEXT_PASS (pass_shorten_branches);
NEXT_PASS (pass_set_nothrow_function_flags);
NEXT_PASS (pass_dwarf2_frame);
diff --git a/gcc/recog.cc b/gcc/recog.cc
index 67d7fa630692..127524cfb4e2 100644
--- a/gcc/recog.cc
+++ b/gcc/recog.cc
@@ -4026,6 +4026,7 @@ peep2_attempt (basic_block bb, rtx_insn *insn, int match_len, rtx_insn *attempt)
case REG_SETJMP:
case REG_TM:
case REG_CALL_NOCF_CHECK:
+ case REG_CALL_KCFI_TYPE:
add_reg_note (new_insn, REG_NOTE_KIND (note),
XEXP (note, 0));
break;
diff --git a/gcc/reg-notes.def b/gcc/reg-notes.def
index 68e137ceccaa..27924ee8afda 100644
--- a/gcc/reg-notes.def
+++ b/gcc/reg-notes.def
@@ -251,5 +251,11 @@ REG_NOTE (UNTYPED_CALL)
compiler when the option -fcf-protection=branch is specified. */
REG_NOTE (CALL_NOCF_CHECK)
+/* Contains the expected KCFI type ID for an indirect call instruction.
+ This note carries the 32-bit type identifier that should match the
+ type ID stored in the function preamble. Used by KCFI (Kernel Control
+ Flow Integrity) to validate indirect call targets at runtime. */
+REG_NOTE (CALL_KCFI_TYPE)
+
/* The values passed to callee, for debuginfo purposes. */
REG_NOTE (CALL_ARG_LOCATION)
diff --git a/gcc/targhooks.cc b/gcc/targhooks.cc
index e723bbbc4df6..7edd3d8619af 100644
--- a/gcc/targhooks.cc
+++ b/gcc/targhooks.cc
@@ -77,6 +77,7 @@ along with GCC; see the file COPYING3. If not see
#include "calls.h"
#include "expr.h"
#include "output.h"
+#include "kcfi.h"
#include "common/common-target.h"
#include "reload.h"
#include "intl.h"
@@ -2203,7 +2204,11 @@ default_compare_by_pieces_branch_ratio (machine_mode)
the location of the NOPs will be recorded in a special object section
called "__patchable_function_entries". This routine may be called
twice per function to put NOPs before and after the function
- entry. */
+ entry.
+
+ KCFI Integration: When KCFI is enabled and this is prefix emission (record_p = true),
+ this function will emit the KCFI preamble (symbols, NOPs, movl) before the prefix NOPs
+ to ensure correct memory layout per KCFI specification. */
void
default_print_patchable_function_entry (FILE *file,
@@ -2219,6 +2224,42 @@ default_print_patchable_function_entry (FILE *file,
code_num = recog_memoized (my_nop);
nop_templ = get_insn_template (code_num, my_nop);
+ /* KCFI Integration for prefix emission. */
+ bool kcfi_handled = false;
+ if (record_p && current_function_decl)
+ {
+ /* Parse patchable function entry configuration to get prefix NOPs. */
+ HOST_WIDE_INT total_nops = patch_area_size;
+ HOST_WIDE_INT prefix_nops = 0;
+
+ /* Check for function-specific patchable_function_entry attribute. */
+ tree patchable_attr = lookup_attribute ("patchable_function_entry",
+ DECL_ATTRIBUTES (current_function_decl));
+ if (patchable_attr)
+ {
+ tree pp_val = TREE_VALUE (patchable_attr);
+ total_nops = tree_to_uhwi (TREE_VALUE (pp_val));
+ if (TREE_CHAIN (pp_val))
+ prefix_nops = tree_to_uhwi (TREE_VALUE (TREE_CHAIN (pp_val)));
+ }
+ else
+ {
+ /* Use global configuration if no function-specific attribute. */
+ HOST_WIDE_INT patch_area_entry;
+ parse_and_check_patch_area (flag_patchable_function_entry, false,
+ &total_nops, &patch_area_entry);
+ prefix_nops = patch_area_entry;
+ }
+
+ /* Use centralized KCFI manager for patchable context. */
+ if (prefix_nops > 0 && patch_area_size == (unsigned HOST_WIDE_INT)prefix_nops)
+ {
+ kcfi_emit_preamble_if_needed (file, current_function_decl, true, prefix_nops, NULL);
+ kcfi_handled = true;
+ }
+ }
+
+ /* Standard patchable function entry handling. */
if (record_p && targetm_common.have_named_sections)
{
char buf[256];
@@ -2249,9 +2290,16 @@ default_print_patchable_function_entry (FILE *file,
ASM_OUTPUT_LABEL (file, buf);
}
+ /* Emit the patchable NOPs. */
unsigned i;
for (i = 0; i < patch_area_size; ++i)
output_asm_insn (nop_templ, NULL);
+
+ /* Mark that KCFI preamble was handled to prevent duplication in ix86_asm_output_function_label. */
+ if (kcfi_handled)
+ {
+ mark_kcfi_preamble_emitted ();
+ }
}
bool
diff --git a/gcc/toplev.cc b/gcc/toplev.cc
index 70dbb3e717f6..04331947b9da 100644
--- a/gcc/toplev.cc
+++ b/gcc/toplev.cc
@@ -67,6 +67,7 @@ along with GCC; see the file COPYING3. If not see
#include "attribs.h"
#include "asan.h"
#include "tsan.h"
+#include "kcfi.h"
#include "plugin.h"
#include "context.h"
#include "pass_manager.h"
@@ -524,6 +525,7 @@ compile_file (void)
insn_locations_init ();
targetm.asm_out.code_end ();
+
/* Do dbx symbols. */
timevar_push (TV_SYMOUT);
@@ -1733,6 +1735,12 @@ process_options ()
"requires %<-fno-exceptions%>");
}
+ if (flag_sanitize & SANITIZE_KCFI)
+ {
+ if (!kcfi_target.gen_kcfi_checked_call)
+ sorry ("%<-fsanitize=kcfi%> not supported by this target");
+ }
+
HOST_WIDE_INT patch_area_size, patch_area_start;
parse_and_check_patch_area (flag_patchable_function_entry, false,
&patch_area_size, &patch_area_start);
diff --git a/gcc/varasm.cc b/gcc/varasm.cc
index 000ad9e26f6f..9c4e3c223159 100644
--- a/gcc/varasm.cc
+++ b/gcc/varasm.cc
@@ -57,6 +57,7 @@ along with GCC; see the file COPYING3. If not see
#include "attribs.h"
#include "asan.h"
#include "rtl-iter.h"
+#include "kcfi.h"
#include "file-prefix-map.h" /* remap_debug_filename() */
#include "alloc-pool.h"
#include "toplev.h"
@@ -2210,6 +2211,10 @@ assemble_start_function (tree decl, const char *fnname)
ASM_OUTPUT_FUNCTION_LABEL (asm_out_file, fnname, current_function_decl);
#endif /* ASM_DECLARE_FUNCTION_NAME */
+ /* KCFI type ID symbols are only emitted for external function declarations,
+ handled by assemble_external_real(). Function definitions do not emit
+ __kcfi_typeid symbols. */
+
/* And the area after the label. Record it if we haven't done so yet. */
if (patch_area_size > patch_area_entry)
targetm.asm_out.print_patchable_function_entry (asm_out_file,
@@ -2752,6 +2757,19 @@ assemble_external_real (tree decl)
/* Some systems do require some output. */
SYMBOL_REF_USED (XEXP (rtl, 0)) = 1;
ASM_OUTPUT_EXTERNAL (asm_out_file, decl, XSTR (XEXP (rtl, 0), 0));
+
+ /* Emit KCFI type ID symbol for external function declarations that are address-taken. */
+ struct cgraph_node *node = (TREE_CODE (decl) == FUNCTION_DECL) ? cgraph_node::get (decl) : NULL;
+ if (flag_sanitize & SANITIZE_KCFI
+ && TREE_CODE (decl) == FUNCTION_DECL
+ && !DECL_INITIAL (decl) /* Only for external declarations (no function body) */
+ && node && node->address_taken) /* Use direct cgraph analysis for address-taken check. */
+ {
+ const char *name = XSTR (XEXP (rtl, 0), 0);
+ /* Strip any encoding prefixes like '*' from symbol name. */
+ name = targetm.strip_name_encoding (name);
+ emit_kcfi_typeid_symbol (asm_out_file, decl, name);
+ }
}
}
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (2 preceding siblings ...)
2025-08-21 7:26 ` [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
2025-08-21 9:29 ` Peter Zijlstra
2025-08-21 7:26 ` [RFC PATCH 5/7] aarch64: Add AArch64 " Kees Cook
` (2 subsequent siblings)
6 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Implement x86_64-specific KCFI backend:
- Function preamble generation with type IDs positioned at -(4+prefix_nops)
offset from function entry point.
- 16-byte alignment of KCFI preambles using calculated prefix NOPs:
aligned(prefix_nops + 5, 16) to maintain cache lines.
- Type-id hash avoids generating ENDBR instruction in type IDs
(0xfa1e0ff3/0xfb1e0ff3 are incremented by 1 to prevent execution).
- On-demand scratch register allocation strategy (r11 as needed).
The clobbers are available both early and late.
- Atomic bundled KCFI check + call/branch sequences using UNSPECV_KCFI
to prevent optimizer separation and maintain security properties.
- Uses the .kcfi_traps section for debugger/runtime metadata.
Assembly Code Pattern layout required by Linux kernel:
movl $inverse_type_id, %r10d ; Load expected type (0 - hash)
addl offset(%target), %r10d ; Add stored type ID from preamble
je .Lpass ; Branch if types match (sum == 0)
.Ltrap: ud2 ; Undefined instruction trap on mismatch
.Lpass: call/jmp *%target ; Execute validated indirect transfer
The initialization of the kcfi callbacks in ix86_option_override()
seems like a hack. I couldn't find a better place to do this.
Build and run tested on x86_64 Linux kernel with various CPU errata
handling alternatives and FineIBT.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/i386/i386-protos.h | 4 +
gcc/config/i386/i386-options.cc | 3 +
gcc/config/i386/i386.cc | 128 ++++++++++++++++++++++++++++
gcc/config/i386/i386.md | 144 ++++++++++++++++++++++++++++++++
gcc/doc/invoke.texi | 20 +++++
5 files changed, 299 insertions(+)
diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h
index 69bc0ee570dd..a5209077506c 100644
--- a/gcc/config/i386/i386-protos.h
+++ b/gcc/config/i386/i386-protos.h
@@ -36,6 +36,10 @@ extern void ix86_maybe_emit_epilogue_vzeroupper (void);
extern void ix86_expand_epilogue (int);
extern void ix86_expand_split_stack_prologue (void);
+/* KCFI support. */
+extern void ix86_kcfi_init (void);
+extern void kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx);
+
extern void ix86_output_addr_vec_elt (FILE *, int);
extern void ix86_output_addr_diff_elt (FILE *, int, int);
diff --git a/gcc/config/i386/i386-options.cc b/gcc/config/i386/i386-options.cc
index 09a35ef62980..f7726c3fdd8f 100644
--- a/gcc/config/i386/i386-options.cc
+++ b/gcc/config/i386/i386-options.cc
@@ -3180,6 +3180,9 @@ void
ix86_option_override (void)
{
ix86_option_override_internal (true, &global_options, &global_options_set);
+
+ /* Initialize KCFI target hooks for x86-64. */
+ ix86_kcfi_init ();
}
/* Remember the last target of ix86_set_current_function. */
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index 65e04d3760d5..1cecd6be2f57 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -98,6 +98,7 @@ along with GCC; see the file COPYING3. If not see
#include "i386-builtins.h"
#include "i386-expand.h"
#include "i386-features.h"
+#include "kcfi.h"
#include "function-abi.h"
#include "rtl-error.h"
#include "gimple-pretty-print.h"
@@ -1700,6 +1701,19 @@ ix86_function_naked (const_tree fn)
return false;
}
+/* Apply x86-64 specific masking to KCFI type ID. */
+static uint32_t
+ix86_kcfi_mask_type_id (uint32_t type_id)
+{
+ /* Avoid embedding ENDBR instructions in KCFI type IDs.
+ ENDBR64: 0xfa1e0ff3, ENDBR32: 0xfb1e0ff3
+ If the type ID matches either instruction encoding, increment by 1. */
+ if (type_id == 0xfa1e0ff3U || type_id == 0xfb1e0ff3U)
+ return type_id + 1;
+
+ return type_id;
+}
+
/* Write the extra assembler code needed to declare a function properly. */
void
@@ -1711,6 +1725,9 @@ ix86_asm_output_function_label (FILE *out_file, const char *fname,
if (cfun)
cfun->machine->function_label_emitted = true;
+ /* Handle KCFI preamble for non-patchable functions. */
+ kcfi_emit_preamble_if_needed (out_file, decl, false, 0, fname);
+
if (is_ms_hook)
{
int i, filler_count = (TARGET_64BIT ? 32 : 16);
@@ -28456,6 +28473,117 @@ ix86_set_handled_components (sbitmap components)
}
}
+/* Generate KCFI checked call - replaces indirect call with bundled KCFI check + call. */
+static rtx
+ix86_kcfi_gen_checked_call (rtx call_insn, rtx target_reg, uint32_t type_id, HOST_WIDE_INT prefix_nops)
+{
+ rtx inverse_type_id_rtx, offset_rtx, pass_label, trap_label, call_args;
+ bool is_sibcall = false;
+
+ /* Check if this is a sibling call (tail call) */
+ if (CALL_P (call_insn))
+ is_sibcall = SIBLING_CALL_P (call_insn);
+
+ /* Convert type ID to inverse for the check (0 - hash) */
+ uint32_t inverse_type_id = (uint32_t)(0 - type_id);
+ inverse_type_id_rtx = gen_int_mode (inverse_type_id, SImode);
+
+ /* Calculate variable offset: -(4 + prefix_nops) */
+ HOST_WIDE_INT offset = -(4 + prefix_nops);
+ offset_rtx = gen_int_mode (offset, DImode);
+
+ /* Generate unique labels for this check. */
+ pass_label = gen_label_rtx ();
+ trap_label = gen_label_rtx ();
+
+ /* Extract call arguments from original call insn. */
+ rtx pattern = PATTERN (call_insn);
+ if (GET_CODE (pattern) == CALL)
+ call_args = XEXP (pattern, 1);
+ else if (GET_CODE (pattern) == SET && GET_CODE (SET_SRC (pattern)) == CALL)
+ call_args = XEXP (SET_SRC (pattern), 1);
+ else if (GET_CODE (pattern) == PARALLEL)
+ {
+ /* Handle PARALLEL patterns (includes peephole2 optimizations and other legitimate cases) */
+ is_sibcall = true; /* PARALLEL indicates a sibling call. */
+ rtx first_elem = XVECEXP (pattern, 0, 0);
+ if (GET_CODE (first_elem) == CALL)
+ {
+ call_args = XEXP (first_elem, 1);
+ }
+ else if (GET_CODE (first_elem) == SET && GET_CODE (SET_SRC (first_elem)) == CALL)
+ {
+ call_args = XEXP (SET_SRC (first_elem), 1);
+ }
+ else
+ {
+ error ("KCFI: Unexpected PARALLEL pattern structure");
+ gcc_unreachable ();
+ }
+ }
+ else
+ {
+ /* This should never happen - all indirect calls should match one of the above patterns. */
+ error ("KCFI: Unexpected call pattern structure");
+ gcc_unreachable ();
+ }
+
+ rtx bundled_call;
+ if (is_sibcall)
+ {
+ /* Use sibling call pattern for tail calls. */
+ bundled_call = gen_kcfi_checked_sibcall (target_reg, call_args, inverse_type_id_rtx, offset_rtx, pass_label, trap_label);
+ }
+ else
+ {
+ /* Use regular call pattern. */
+ bundled_call = gen_kcfi_checked_call (target_reg, call_args, inverse_type_id_rtx, offset_rtx, pass_label, trap_label);
+ }
+
+ return bundled_call;
+}
+
+/* Calculate x86_64-specific KCFI prefix NOPs for 16-byte alignment. */
+static int
+ix86_kcfi_calculate_prefix_nops (HOST_WIDE_INT prefix_nops)
+{
+ /* Calculate KCFI NOPs needed: aligned(prefix_nops + 5, 16). */
+ return (16 - ((prefix_nops + 5) % 16)) % 16;
+}
+
+/* Emit x86_64-specific type ID instruction. */
+static void
+ix86_kcfi_emit_type_id_instruction (FILE *file, uint32_t type_id)
+{
+ /* Emit movl instruction with type ID. */
+ fprintf (file, "\tmovl\t$0x%08x, %%eax\n", type_id);
+}
+
+/* Add x86-64 specific register clobbers for KCFI calls. */
+static void
+ix86_kcfi_add_clobbers (rtx_insn *call_insn)
+{
+ /* Add r10/r11 clobbers so register allocator knows they'll be used. */
+ rtx usage = CALL_INSN_FUNCTION_USAGE (call_insn);
+ clobber_reg (&usage, gen_rtx_REG (DImode, R10_REG));
+ clobber_reg (&usage, gen_rtx_REG (DImode, R11_REG));
+ CALL_INSN_FUNCTION_USAGE (call_insn) = usage;
+}
+
+/* Initialize x86-64 KCFI target hooks. */
+void
+ix86_kcfi_init (void)
+{
+ if (TARGET_64BIT && (flag_sanitize & SANITIZE_KCFI))
+ {
+ kcfi_target.mask_type_id = ix86_kcfi_mask_type_id;
+ kcfi_target.gen_kcfi_checked_call = ix86_kcfi_gen_checked_call;
+ kcfi_target.add_kcfi_clobbers = ix86_kcfi_add_clobbers;
+ kcfi_target.calculate_prefix_nops = ix86_kcfi_calculate_prefix_nops;
+ kcfi_target.emit_type_id_instruction = ix86_kcfi_emit_type_id_instruction;
+ }
+}
+
#undef TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS
#define TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS ix86_get_separate_components
#undef TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB
diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
index a50475bdaf4c..acefc2246537 100644
--- a/gcc/config/i386/i386.md
+++ b/gcc/config/i386/i386.md
@@ -248,6 +248,7 @@
UNSPECV_RDGSBASE
UNSPECV_WRFSBASE
UNSPECV_WRGSBASE
+ UNSPECV_KCFI
UNSPECV_FXSAVE
UNSPECV_FXRSTOR
UNSPECV_FXSAVE64
@@ -30582,6 +30583,149 @@
(set_attr "type" "other")
(set_attr "mode" "<MODE>")])
+;; KCFI checked call - atomic KCFI check + indirect call bundle
+;; This prevents optimizer from separating KCFI checks from their protected calls
+(define_insn "kcfi_checked_call"
+ [(call (mem:QI (match_operand:DI 0 "nonimmediate_operand" "rm"))
+ (match_operand 1))
+ (unspec_volatile [(match_operand:SI 2 "const_int_operand" "n")
+ (match_operand:DI 3 "const_int_operand" "n")
+ (label_ref (match_operand 4))
+ (label_ref (match_operand 5))] UNSPECV_KCFI)
+ (clobber (reg:SI R10_REG))
+ (clobber (reg:SI R11_REG))
+ (clobber (reg:CC FLAGS_REG))]
+ "TARGET_64BIT && !SIBLING_CALL_P (insn)"
+{
+ rtx target_reg;
+ bool need_r11 = false;
+
+ /* If target is not in a register, move it to r11. */
+ if (!REG_P (operands[0]))
+ {
+ target_reg = gen_rtx_REG (DImode, R11_REG);
+ /* Emit the move to r11. */
+ rtx mov_to_r11[2] = { target_reg, operands[0] };
+ output_asm_insn ("movq\t%1, %0", mov_to_r11);
+ need_r11 = true;
+ }
+ else
+ {
+ target_reg = operands[0];
+ }
+
+ /* Choose scratch register: r10 by default, r11 if r10 is the target. */
+ bool target_is_r10 = (REG_P (target_reg) && REGNO (target_reg) == R10_REG);
+ int scratch_reg = target_is_r10 ? R11_REG : R10_REG;
+ const char *scratch_name = target_is_r10 ? "r11d" : "r10d";
+
+ /* Output complete KCFI check + call sequence atomically. */
+ char mov_insn[64];
+ sprintf (mov_insn, "movl\t$%%c2, %%%%%s", scratch_name);
+ output_asm_insn (mov_insn, operands);
+
+ /* Create memory operand for the addl instruction. */
+ rtx mem_op = gen_rtx_MEM (SImode, gen_rtx_PLUS (DImode, target_reg, operands[3]));
+ rtx temp_operands[2] = { mem_op, gen_rtx_REG (SImode, scratch_reg) };
+ output_asm_insn ("addl\t%0, %1", temp_operands);
+
+ output_asm_insn ("je\t%l4", operands);
+
+ /* Output trap label and instruction. */
+ output_asm_insn ("%l5:", operands);
+ output_asm_insn ("ud2", operands);
+
+ /* Use existing function with trap and entry label RTX. */
+ kcfi_emit_trap_with_section (asm_out_file, operands[5]);
+
+ /* Output pass label. */
+ output_asm_insn ("%l4:", operands);
+
+ /* Finally emit the protected call using the register we chose. */
+ if (need_r11)
+ {
+ rtx r11_operand = gen_rtx_REG (DImode, R11_REG);
+ output_asm_insn ("call\t*%0", &r11_operand);
+ return "";
+ }
+ else
+ return "call\t*%0";
+}
+ [(set_attr "type" "call")
+ (set_attr "mode" "DI")])
+
+;; KCFI checked sibling call - atomic KCFI check + indirect sibling call bundle
+;; This handles tail call optimization cases
+(define_insn "kcfi_checked_sibcall"
+ [(call (mem:QI (match_operand:DI 0 "nonimmediate_operand" "rm"))
+ (match_operand 1))
+ (unspec_volatile [(match_operand:SI 2 "const_int_operand" "n")
+ (match_operand:DI 3 "const_int_operand" "n")
+ (label_ref (match_operand 4))
+ (label_ref (match_operand 5))] UNSPECV_KCFI)
+ (clobber (reg:SI R10_REG))
+ (clobber (reg:SI R11_REG))
+ (clobber (reg:CC FLAGS_REG))]
+ "TARGET_64BIT && SIBLING_CALL_P (insn)"
+{
+ rtx target_reg;
+ bool need_r11 = false;
+
+ /* If target is not in a register, move it to r11. */
+ if (!REG_P (operands[0]))
+ {
+ target_reg = gen_rtx_REG (DImode, R11_REG);
+ /* Emit the move to r11. */
+ rtx mov_to_r11[2] = { target_reg, operands[0] };
+ output_asm_insn ("movq\t%1, %0", mov_to_r11);
+ need_r11 = true;
+ }
+ else
+ {
+ target_reg = operands[0];
+ }
+
+ /* Choose scratch register: r10 by default, r11 if r10 is the target. */
+ bool target_is_r10 = (REG_P (target_reg) && REGNO (target_reg) == R10_REG);
+ int scratch_reg = target_is_r10 ? R11_REG : R10_REG;
+ const char *scratch_name = target_is_r10 ? "r11d" : "r10d";
+
+ /* Output complete KCFI check + sibling call sequence atomically. */
+ char mov_insn[64];
+ sprintf (mov_insn, "movl\t$%%c2, %%%%%s", scratch_name);
+ output_asm_insn (mov_insn, operands);
+
+ /* Create memory operand for the addl instruction. */
+ rtx mem_op = gen_rtx_MEM (SImode, gen_rtx_PLUS (DImode, target_reg, operands[3]));
+ rtx temp_operands[2] = { mem_op, gen_rtx_REG (SImode, scratch_reg) };
+ output_asm_insn ("addl\t%0, %1", temp_operands);
+
+ output_asm_insn ("je\t%l4", operands);
+
+ /* Output trap label and instruction. */
+ output_asm_insn ("%l5:", operands);
+ output_asm_insn ("ud2", operands);
+
+ /* Use existing function with trap and entry label RTX. */
+ kcfi_emit_trap_with_section (asm_out_file, operands[5]);
+
+ /* Output pass label. */
+ output_asm_insn ("%l4:", operands);
+
+ /* Finally emit the protected sibling call (jmp) using the register we chose. */
+ if (need_r11)
+ {
+ rtx r11_operand = gen_rtx_REG (DImode, R11_REG);
+ output_asm_insn ("jmp\t*%0", &r11_operand);
+ return "";
+ }
+ else
+ return "jmp\t*%0";
+}
+ [(set_attr "type" "call")
+ (set_attr "mode" "DI")])
+
+
(include "mmx.md")
(include "sse.md")
(include "sync.md")
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index c66f47336826..f531a9f6ce33 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -18316,6 +18316,26 @@ and without changing the entry points of the target functions. Only
functions that have referenced by their address receive the KCFI preamble
instrumentation.
+Platform-specific implementation details:
+
+On x86_64, KCFI type identifiers are emitted as a @code{movl $ID, %eax}
+instruction before the function entry. The implementation ensures that
+type IDs never collide with ENDBR instruction encodings. When used with
+@option{-fpatchable-function-entry}, the type identifier is placed before
+any patchable NOPs, with appropriate alignment to maintain a 16-byte
+boundary for the function entry. The runtime check loads the type ID
+from the target function into @code{%r10d} and uses an @code{addl}
+instruction to add the negative expected type ID, effectively zeroing
+the register if the types match. A conditional jump follows to either
+continue execution or trap on mismatch. The check sequence uses
+@code{%r10d} and @code{%r11d} as scratch registers. Trap locations are
+recorded in a special @code{.kcfi_traps} section that maps trap sites
+to their corresponding function entry points, enabling debuggers and
+crash handlers to identify KCFI violations. The exact instruction
+sequences for both the KCFI preamble and the check-call bundle are
+considered ABI, as the Linux kernel may optionally rewrite these areas
+at boot time to mitigate detected CPU errata.
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 5/7] aarch64: Add AArch64 Kernel Control Flow Integrity implementation
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (3 preceding siblings ...)
2025-08-21 7:26 ` [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
2025-08-21 7:26 ` [RFC PATCH 6/7] riscv: Add RISC-V " Kees Cook
2025-08-21 7:26 ` [RFC PATCH 7/7] kcfi: Add regression test suite Kees Cook
6 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Implement AArch64-specific KCFI backend.
- Function preamble generation using .word directives for type ID storage
at offset from function entry point (no prefix NOPs needed due to
4-byte instruction alignment).
- Trap debugging through ESR (Exception Syndrome Register) encoding
in BRK instruction immediate values for precise failure analysis.
- Scratch register allocation using w16/w17 (x16/x17) following
AArch64 procedure call standard for intra-procedure-call registers.
- Support for both regular calls (BLR) and sibling calls (BR) with
appropriate register usage and jump instructions.
- Atomic bundled KCFI check + call/branch sequences using UNSPECV_KCFI_CHECK
to prevent optimizer separation and maintain security properties.
Assembly Code Pattern for AArch64:
ldur w16, [target, #-4] ; Load actual type ID from preamble
mov w17, #type_id_low ; Load expected type (lower 16 bits)
movk w17, #type_id_high, lsl #16 ; Load upper 16 bits if needed
cmp w16, w17 ; Compare type IDs directly
b.eq .Lpass ; Branch if types match
.Ltrap: brk #esr_value ; Enhanced trap with register info
.Lpass: blr/br target ; Execute validated indirect transfer
ESR (Exception Syndrome Register) Integration:
- BRK instruction immediate encoding format:
0x8000 | ((TypeIndex & 31) << 5) | (AddrIndex & 31)
- TypeIndex indicates which W register contains expected type (W17 = 17)
- AddrIndex indicates which X register contains target address (0-30)
- Example: brk #33313 (0x8221) = expected type in W17, target address in X1
Like x86, the callback initialization in aarch64_override_options()
seem hacky. Is there a better place for this?
Build and run tested with Linux kernel ARCH=arm64.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/aarch64/aarch64-protos.h | 4 +
gcc/config/aarch64/aarch64.cc | 112 +++++++++++++++++++++++
gcc/config/aarch64/aarch64.md | 137 ++++++++++++++++++++++++++++
gcc/doc/invoke.texi | 14 +++
4 files changed, 267 insertions(+)
diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
index 38c307cdc3a6..ff235305fbc1 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -1280,4 +1280,8 @@ extern bool aarch64_gcs_enabled ();
extern unsigned aarch64_data_alignment (const_tree exp, unsigned align);
extern unsigned aarch64_stack_alignment (const_tree exp, unsigned align);
+/* KCFI support. */
+extern void aarch64_kcfi_init (void);
+extern void kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx);
+
#endif /* GCC_AARCH64_PROTOS_H */
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index f4a2062b042a..fe5fbecb59b6 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -83,6 +83,7 @@
#include "rtlanal.h"
#include "tree-dfa.h"
#include "asan.h"
+#include "kcfi.h"
#include "aarch64-elf-metadata.h"
#include "aarch64-feature-deps.h"
#include "config/arm/aarch-common.h"
@@ -19437,6 +19438,9 @@ aarch64_override_options (void)
aarch64_override_options_internal (&global_options);
+ /* Initialize KCFI target hooks for AArch64. */
+ aarch64_kcfi_init ();
+
/* Save these options as the default ones in case we push and pop them later
while processing functions with potential target attributes. */
target_option_default_node = target_option_current_node
@@ -25473,6 +25477,9 @@ aarch64_declare_function_name (FILE *stream, const char* name,
aarch64_asm_output_variant_pcs (stream, fndecl, name);
+ /* Emit KCFI preamble for non-patchable functions. */
+ kcfi_emit_preamble_if_needed (stream, fndecl, false, 0, name);
+
/* Don't forget the type directive for ELF. */
ASM_OUTPUT_TYPE_DIRECTIVE (stream, name, "function");
ASM_OUTPUT_FUNCTION_LABEL (stream, name, fndecl);
@@ -32706,6 +32713,111 @@ aarch64_libgcc_floating_mode_supported_p
#undef TARGET_DOCUMENTATION_NAME
#define TARGET_DOCUMENTATION_NAME "AArch64"
+
+/* AArch64 doesn't need prefix NOPs (instructions are already 4-byte aligned) */
+static int
+aarch64_kcfi_calculate_prefix_nops (HOST_WIDE_INT prefix_nops ATTRIBUTE_UNUSED)
+{
+ /* AArch64 instructions are 4-byte aligned, no prefix NOPs needed for KCFI preamble. */
+ return 0;
+}
+
+/* Emit AArch64-specific type ID instruction. */
+static void
+aarch64_kcfi_emit_type_id_instruction (FILE *file, uint32_t type_id)
+{
+ /* Emit type ID as a 32-bit word. */
+ fprintf (file, "\t.word 0x%08x\n", type_id);
+}
+
+
+
+/* Generate AArch64 KCFI checked call bundle. */
+static rtx
+aarch64_kcfi_gen_checked_call (rtx call_insn, rtx target_reg, uint32_t expected_type,
+ HOST_WIDE_INT prefix_nops)
+{
+ /* For AArch64, we create an RTL bundle that combines the KCFI check
+ with the call instruction in an atomic sequence. */
+
+ if (!REG_P (target_reg))
+ {
+ /* If not a register, load it into x16. */
+ rtx temp = gen_rtx_REG (Pmode, 16);
+ emit_move_insn (temp, target_reg);
+ target_reg = temp;
+ }
+
+ /* Generate the bundled KCFI check + call pattern. */
+ rtx pattern;
+ if (CALL_P (call_insn))
+ {
+ rtx call_pattern = PATTERN (call_insn);
+
+ /* Create labels used by both call and sibcall patterns. */
+ rtx pass_label = gen_label_rtx ();
+ rtx trap_label = gen_label_rtx ();
+
+ /* Check if it's a sibling call. */
+ if (find_reg_note (call_insn, REG_NORETURN, NULL_RTX)
+ || (GET_CODE (call_pattern) == PARALLEL
+ && GET_CODE (XVECEXP (call_pattern, 0, XVECLEN (call_pattern, 0) - 1)) == RETURN))
+ {
+ /* Generate sibling call bundle. */
+ pattern = gen_aarch64_kcfi_checked_sibcall (target_reg,
+ gen_int_mode (expected_type, SImode),
+ gen_int_mode (prefix_nops, SImode),
+ pass_label,
+ trap_label);
+ }
+ else
+ {
+ /* Generate regular call bundle. */
+ pattern = gen_aarch64_kcfi_checked_call (target_reg,
+ gen_int_mode (expected_type, SImode),
+ gen_int_mode (prefix_nops, SImode),
+ pass_label,
+ trap_label);
+ }
+ }
+ else
+ {
+ error ("KCFI: Expected call instruction");
+ return NULL_RTX;
+ }
+
+ return pattern;
+}
+
+/* Add AArch64-specific register clobbers for KCFI calls. */
+static void
+aarch64_kcfi_add_clobbers (rtx_insn *call_insn)
+{
+ /* AArch64 KCFI uses w16 and w17 (x16 and x17) as scratch registers. */
+ rtx usage = CALL_INSN_FUNCTION_USAGE (call_insn);
+
+ /* Add w16 (x16) clobber. */
+ clobber_reg (&usage, gen_rtx_REG (SImode, 16));
+
+ /* Add w17 (x17) clobber. */
+ clobber_reg (&usage, gen_rtx_REG (SImode, 17));
+
+ CALL_INSN_FUNCTION_USAGE (call_insn) = usage;
+}
+
+/* Initialize AArch64 KCFI target hooks. */
+void
+aarch64_kcfi_init (void)
+{
+ if (flag_sanitize & SANITIZE_KCFI)
+ {
+ kcfi_target.gen_kcfi_checked_call = aarch64_kcfi_gen_checked_call;
+ kcfi_target.add_kcfi_clobbers = aarch64_kcfi_add_clobbers;
+ kcfi_target.calculate_prefix_nops = aarch64_kcfi_calculate_prefix_nops;
+ kcfi_target.emit_type_id_instruction = aarch64_kcfi_emit_type_id_instruction;
+ }
+}
+
struct gcc_target targetm = TARGET_INITIALIZER;
#include "gt-aarch64.h"
diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md
index a4ae6859da01..28f9aa651519 100644
--- a/gcc/config/aarch64/aarch64.md
+++ b/gcc/config/aarch64/aarch64.md
@@ -416,6 +416,7 @@
UNSPECV_TCANCEL ; Represent transaction cancel.
UNSPEC_RNDR ; Represent RNDR
UNSPEC_RNDRRS ; Represent RNDRRS
+ UNSPECV_KCFI_CHECK ; Represent KCFI check bundled with call
]
)
@@ -1342,6 +1343,142 @@
"brk #1000"
[(set_attr "type" "trap")])
+;; KCFI bundled check and call patterns
+;; These combine the KCFI check with the call in an atomic sequence
+
+(define_insn "aarch64_kcfi_checked_call"
+ [(parallel [(call (mem:DI (match_operand:DI 0 "register_operand" "r"))
+ (const_int 0))
+ (unspec:DI [(const_int 0)] UNSPEC_CALLEE_ABI)
+ (unspec_volatile:DI [(match_operand:SI 1 "const_int_operand" "n") ; type_id
+ (match_operand:SI 2 "const_int_operand" "n") ; prefix_nops
+ (label_ref (match_operand 3)) ; pass label
+ (label_ref (match_operand 4))] ; trap label
+ UNSPECV_KCFI_CHECK)
+ (clobber (reg:DI LR_REGNUM))
+ (clobber (reg:SI 16)) ; w16 - scratch for loaded type
+ (clobber (reg:SI 17))])] ; w17 - scratch for expected type
+ "flag_sanitize & SANITIZE_KCFI"
+ "*
+ {
+ uint32_t type_id = INTVAL (operands[1]);
+ HOST_WIDE_INT prefix_nops = INTVAL (operands[2]);
+ HOST_WIDE_INT offset = -(4 + prefix_nops);
+
+ /* AArch64 KCFI check sequence:
+ 1. Load actual type from function preamble
+ 2. Load expected type
+ 3. Compare and branch if equal
+ 4. Trap if mismatch
+ 5. Call target. */
+
+ static char ldur_buffer[64];
+ sprintf (ldur_buffer, \"ldur\\tw16, [%%0, #%ld]\", offset);
+ output_asm_insn (ldur_buffer, operands);
+
+ /* Load expected type - may need multiple instructions for large constants. */
+ if ((type_id & 0xffff0000) == 0)
+ {
+ static char mov_buffer[64];
+ sprintf (mov_buffer, \"mov\\tw17, #%u\", type_id);
+ output_asm_insn (mov_buffer, operands);
+ }
+ else
+ {
+ static char mov_buffer[64], movk_buffer[64];
+ sprintf (mov_buffer, \"mov\\tw17, #%u\", type_id & 0xffff);
+ output_asm_insn (mov_buffer, operands);
+ sprintf (movk_buffer, \"movk\\tw17, #%u, lsl #16\", (type_id >> 16) & 0xffff);
+ output_asm_insn (movk_buffer, operands);
+ }
+
+ output_asm_insn (\"cmp\\tw16, w17\", operands);
+ output_asm_insn (\"b.eq\\t%l3\", operands);
+
+ /* Generate unique trap ID and emit trap. */
+ output_asm_insn (\"%l4:\", operands);
+
+ /* Calculate and emit BRK with ESR encoding. */
+ unsigned type_index = 17; /* w17 contains expected type. */
+ unsigned addr_index = REGNO (operands[0]) - R0_REGNUM;
+ unsigned esr_value = 0x8000 | ((type_index & 31) << 5) | (addr_index & 31);
+
+ static char brk_buffer[32];
+ sprintf (brk_buffer, \"brk\\t#%u\", esr_value);
+ output_asm_insn (brk_buffer, operands);
+
+ output_asm_insn (\"%l3:\", operands);
+ output_asm_insn (\"\\tblr\\t%0\", operands);
+
+ return \"\";
+ }"
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
+(define_insn "aarch64_kcfi_checked_sibcall"
+ [(parallel [(call (mem:DI (match_operand:DI 0 "register_operand" "r"))
+ (const_int 0))
+ (unspec:DI [(const_int 0)] UNSPEC_CALLEE_ABI)
+ (unspec_volatile:DI [(match_operand:SI 1 "const_int_operand" "n") ; type_id
+ (match_operand:SI 2 "const_int_operand" "n") ; prefix_nops
+ (label_ref (match_operand 3)) ; pass label
+ (label_ref (match_operand 4))] ; trap label
+ UNSPECV_KCFI_CHECK)
+ (return)
+ (clobber (reg:SI 16)) ; w16 - scratch for loaded type
+ (clobber (reg:SI 17))])] ; w17 - scratch for expected type
+ "flag_sanitize & SANITIZE_KCFI"
+ "*
+ {
+ uint32_t type_id = INTVAL (operands[1]);
+ HOST_WIDE_INT prefix_nops = INTVAL (operands[2]);
+ HOST_WIDE_INT offset = -(4 + prefix_nops);
+
+ /* AArch64 KCFI check sequence for sibling calls. */
+
+ static char ldur_buffer[64];
+ sprintf (ldur_buffer, \"ldur\\tw16, [%%0, #%ld]\", offset);
+ output_asm_insn (ldur_buffer, operands);
+
+ /* Load expected type. */
+ if ((type_id & 0xffff0000) == 0)
+ {
+ static char mov_buffer[64];
+ sprintf (mov_buffer, \"mov\\tw17, #%u\", type_id);
+ output_asm_insn (mov_buffer, operands);
+ }
+ else
+ {
+ static char mov_buffer[64], movk_buffer[64];
+ sprintf (mov_buffer, \"mov\\tw17, #%u\", type_id & 0xffff);
+ output_asm_insn (mov_buffer, operands);
+ sprintf (movk_buffer, \"movk\\tw17, #%u, lsl #16\", (type_id >> 16) & 0xffff);
+ output_asm_insn (movk_buffer, operands);
+ }
+
+ output_asm_insn (\"cmp\\tw16, w17\", operands);
+ output_asm_insn (\"b.eq\\t%l3\", operands);
+
+ /* Generate unique trap ID and emit trap. */
+ output_asm_insn (\"%l4:\", operands);
+
+ /* Calculate and emit BRK with ESR encoding. */
+ unsigned type_index = 17; /* w17 contains expected type. */
+ unsigned addr_index = REGNO (operands[0]) - R0_REGNUM;
+ unsigned esr_value = 0x8000 | ((type_index & 31) << 5) | (addr_index & 31);
+
+ static char brk_buffer[32];
+ sprintf (brk_buffer, \"brk\\t#%u\", esr_value);
+ output_asm_insn (brk_buffer, operands);
+
+ output_asm_insn (\"%l3:\", operands);
+ output_asm_insn (\"\\tbr\\t%0\", operands);
+
+ return \"\";
+ }"
+ [(set_attr "type" "branch")
+ (set_attr "length" "24")])
+
(define_expand "prologue"
[(clobber (const_int 0))]
""
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index f531a9f6ce33..161c7024f842 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -18336,6 +18336,20 @@ sequences for both the KCFI preamble and the check-call bundle are
considered ABI, as the Linux kernel may optionally rewrite these areas
at boot time to mitigate detected CPU errata.
+On AArch64, KCFI type identifiers are emitted as a @code{.word ID}
+directive (a 32-bit constant) before the function entry. AArch64's
+natural 4-byte instruction alignment eliminates the need for additional
+padding NOPs. When used with @option{-fpatchable-function-entry}, the
+type identifier is placed before any patchable NOPs. The runtime check
+uses @code{x16} and @code{x17} as scratch registers. Type mismatches
+trigger a @code{brk} instruction with an immediate value that encodes
+both the expected type register index and the target address register
+index in the format @code{0x8000 | (type_reg << 5) | addr_reg}. This
+encoding is captured in the ESR (Exception Syndrome Register) when the
+trap is taken, allowing the kernel to identify both the KCFI violation
+and the involved registers for detailed diagnostics (eliminating the need
+for a separate @code{.kcfi_traps} section as used on x86_64).
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 6/7] riscv: Add RISC-V Kernel Control Flow Integrity implementation
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (4 preceding siblings ...)
2025-08-21 7:26 ` [RFC PATCH 5/7] aarch64: Add AArch64 " Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
2025-08-21 7:26 ` [RFC PATCH 7/7] kcfi: Add regression test suite Kees Cook
6 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Implement RISC-V-specific KCFI backend.
- Function preamble generation using .word directives for type ID storage
at offset from function entry point (no prefix NOPs needed due to
natural 4-byte instruction alignment).
- Scratch register allocation using t1/t2 (x6/x7) following RISC-V
procedure call standard for temporary registers.
- Support for both regular calls (JALR) and sibling calls (JR) with
appropriate register usage and jump instructions.
- Integration with .kcfi_traps section for debugger/runtime metadata
(like x86_64).
- Atomic bundled KCFI check + call/jump sequences using UNSPECV_KCFI_CHECK
to prevent optimizer separation and maintain security properties.
Assembly Code Pattern for RISC-V:
lw t1, -4(target_reg) ; Load actual type ID from preamble
lui t2, %hi(expected_type) ; Load expected type (upper 20 bits)
addiw t2, t2, %lo(expected_type) ; Add lower 12 bits (sign-extended)
beq t1, t2, .Lpass ; Branch if types match
.Ltrap: ebreak ; Environment break trap on mismatch
.Lpass: jalr/jr target_reg ; Execute validated indirect transfer
Build tested with Linux kernel ARCH=riscv (I am still building a proper
risc-v emulation setup). Run tested via userspace binaries.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/riscv/riscv-protos.h | 1 +
gcc/config/riscv/riscv.cc | 157 ++++++++++++++++++++++++++++++++
gcc/config/riscv/riscv.md | 49 ++++++++++
gcc/doc/invoke.texi | 13 +++
4 files changed, 220 insertions(+)
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 539321ff95b8..1d343c529934 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -126,6 +126,7 @@ extern bool riscv_split_64bit_move_p (rtx, rtx);
extern void riscv_split_doubleword_move (rtx, rtx);
extern const char *riscv_output_move (rtx, rtx);
extern const char *riscv_output_return ();
+extern const char *riscv_output_kcfi_checked_call (uint32_t, HOST_WIDE_INT, bool);
extern void riscv_declare_function_name (FILE *, const char *, tree);
extern void riscv_declare_function_size (FILE *, const char *, tree);
extern void riscv_asm_output_alias (FILE *, const tree, const tree);
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index 0a9fcef37029..5daa5427568d 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -81,6 +81,7 @@ along with GCC; see the file COPYING3. If not see
#include "cgraph.h"
#include "langhooks.h"
#include "gimplify.h"
+#include "kcfi.h"
/* This file should be included last. */
#include "target-def.h"
@@ -11156,6 +11157,9 @@ riscv_declare_function_name (FILE *stream, const char *name, tree fndecl)
fprintf (stream, "\t# tune = %s\n", local_tune_str);
}
}
+
+ /* Emit KCFI preamble for non-patchable functions. */
+ kcfi_emit_preamble_if_needed (stream, fndecl, false, 0, name);
}
void
@@ -11418,6 +11422,147 @@ riscv_convert_vector_chunks (struct gcc_options *opts)
return 1;
}
+/* KCFI (Kernel Control Flow Integrity) support. */
+
+/* Generate KCFI checked call RTL pattern following AArch64 approach. */
+static rtx
+riscv_kcfi_gen_checked_call (rtx call_insn, rtx target_reg, uint32_t expected_type,
+ HOST_WIDE_INT prefix_nops)
+{
+ /* For RISC-V, we create an RTL bundle that combines the KCFI check
+ with the call instruction in an atomic sequence. */
+
+ if (!REG_P (target_reg))
+ {
+ /* If not a register, load it into t1. */
+ rtx temp = gen_rtx_REG (Pmode, T1_REGNUM);
+ emit_move_insn (temp, target_reg);
+ target_reg = temp;
+ }
+
+ /* Generate the bundled KCFI check + call pattern. */
+ rtx pattern;
+ if (CALL_P (call_insn))
+ {
+ rtx call_pattern = PATTERN (call_insn);
+
+ /* Create labels used by both call and sibcall patterns. */
+ rtx pass_label = gen_label_rtx ();
+ rtx trap_label = gen_label_rtx ();
+
+ /* Check if it's a sibling call. */
+ if (find_reg_note (call_insn, REG_NORETURN, NULL_RTX)
+ || (GET_CODE (call_pattern) == PARALLEL
+ && GET_CODE (XVECEXP (call_pattern, 0, XVECLEN (call_pattern, 0) - 1)) == RETURN))
+ {
+ /* Generate sibling call bundle. */
+ pattern = gen_riscv_kcfi_checked_sibcall (target_reg,
+ gen_int_mode (expected_type, SImode),
+ gen_int_mode (prefix_nops, SImode),
+ pass_label,
+ trap_label);
+ }
+ else
+ {
+ /* Generate regular call bundle. */
+ pattern = gen_riscv_kcfi_checked_call (target_reg,
+ gen_int_mode (expected_type, SImode),
+ gen_int_mode (prefix_nops, SImode),
+ pass_label,
+ trap_label);
+ }
+ }
+ else
+ {
+ error ("KCFI: Expected call instruction");
+ gcc_unreachable ();
+ }
+
+ return pattern;
+}
+
+/* Add RISC-V specific register clobbers for KCFI instrumentation. */
+static void
+riscv_kcfi_add_clobbers (rtx_insn *call_insn)
+{
+ /* Add t1/t2 clobbers so register allocator knows they'll be used. */
+ rtx usage = CALL_INSN_FUNCTION_USAGE (call_insn);
+ clobber_reg (&usage, gen_rtx_REG (DImode, T1_REGNUM));
+ clobber_reg (&usage, gen_rtx_REG (DImode, T2_REGNUM));
+ CALL_INSN_FUNCTION_USAGE (call_insn) = usage;
+}
+
+/* Calculate prefix NOPs (RISC-V doesn't need additional NOPs). */
+static int
+riscv_kcfi_calculate_prefix_nops (HOST_WIDE_INT prefix_nops ATTRIBUTE_UNUSED)
+{
+ /* RISC-V instructions are 4-byte aligned, no additional NOPs needed. */
+ return 0;
+}
+
+/* Emit RISC-V type ID instruction. */
+static void
+riscv_kcfi_emit_type_id_instruction (FILE *file, uint32_t type_id)
+{
+ /* Emit .word directive with type ID. */
+ fprintf (file, "\t.word\t0x%08x\n", type_id);
+}
+
+/* Output KCFI checked call instruction sequence. */
+const char *
+riscv_output_kcfi_checked_call (uint32_t expected_type, HOST_WIDE_INT prefix_nops, bool sibling_call)
+{
+ static char buf[512];
+
+ /* Calculate offset for type ID load, accounting for prefix NOPs. */
+ HOST_WIDE_INT offset = -(4 + prefix_nops);
+
+ /* Generate unique labels. */
+ static int label_counter = 0;
+ int pass_label_num = ++label_counter;
+ int trap_label_num = ++label_counter;
+
+ /* Generate the KCFI check sequence:
+ lw t1, -4(target_reg) # Load actual type from function[-4]
+ lui t2, %hi(expected_type_id) # Load upper 20 bits of expected type
+ addiw t2, t2, %lo(expected_type_id) # Add lower 12 bits (sign-extended)
+ beq t1, t2, .Lpass # Branch if types match
+ .Ltrap:
+ ebreak # Environment break (trap on mismatch)
+ .Lpass:
+ jalr target_reg # Execute indirect function call
+ */
+
+ /* Manually split expected_type as required by agentic/kcfi-riscv.md:
+ - Upper 20 bits for lui instruction
+ - Lower 12 bits for addiw instruction (sign-extended) */
+ uint32_t hi20 = (expected_type >> 12) & 0xFFFFF; /* Upper 20 bits */
+ int32_t lo12 = ((int32_t)(expected_type << 20)) >> 20; /* Lower 12 bits, sign-extended */
+
+ snprintf (buf, sizeof (buf),
+ "lw\tt1, %ld(%%0)\n"
+ "\tlui\tt2, %u\n"
+ "\taddiw\tt2, t2, %d\n"
+ "\tbeq\tt1, t2, .Lkcfi_pass_%d\n"
+ ".Lkcfi_trap_%d:\n"
+ "\tebreak\n"
+ "\t.pushsection\t.kcfi_traps,\"ao\",@progbits,.text\n"
+ ".Lkcfi_trap_entry_%d:\n"
+ "\t.word\t.Lkcfi_trap_%d - .Lkcfi_trap_entry_%d\n"
+ "\t.popsection\n"
+ ".Lkcfi_pass_%d:\n"
+ "\t%s\t%%0",
+ offset, hi20, lo12,
+ pass_label_num,
+ trap_label_num,
+ trap_label_num,
+ trap_label_num, trap_label_num,
+ pass_label_num,
+ sibling_call ? "jr" : "jalr");
+
+ return buf;
+}
+
/* 'Unpack' up the internal tuning structs and update the options
in OPTS. The caller must have set up selected_tune and selected_arch
as all the other target-specific codegen decisions are
@@ -11525,6 +11670,7 @@ riscv_override_options_internal (struct gcc_options *opts)
opts->x_flag_cf_protection
= (cf_protection_level) (opts->x_flag_cf_protection | CF_SET);
}
+
}
/* Implement TARGET_OPTION_OVERRIDE. */
@@ -11715,6 +11861,16 @@ riscv_option_override (void)
riscv_override_options_internal (&global_options);
+ /* Initialize KCFI hooks if KCFI is enabled. */
+ if (flag_sanitize & SANITIZE_KCFI)
+ {
+ kcfi_target.gen_kcfi_checked_call = riscv_kcfi_gen_checked_call;
+ kcfi_target.add_kcfi_clobbers = riscv_kcfi_add_clobbers;
+ kcfi_target.calculate_prefix_nops = riscv_kcfi_calculate_prefix_nops;
+ kcfi_target.emit_type_id_instruction = riscv_kcfi_emit_type_id_instruction;
+ /* Note: mask_type_id is NULL - no masking needed for RISC-V. */
+ }
+
/* Save these options as the default ones in case we push and pop them later
while processing functions with potential target attributes. */
target_option_default_node = target_option_current_node
@@ -15795,6 +15951,7 @@ synthesize_and (rtx operands[3])
#define TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST \
riscv_builtin_vectorization_cost
+
#undef TARGET_VECTORIZE_CREATE_COSTS
#define TARGET_VECTORIZE_CREATE_COSTS riscv_vectorize_create_costs
diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md
index 578dd43441e2..6e9545e9d003 100644
--- a/gcc/config/riscv/riscv.md
+++ b/gcc/config/riscv/riscv.md
@@ -152,6 +152,9 @@
;; XTheadInt unspec
UNSPECV_XTHEADINT_PUSH
UNSPECV_XTHEADINT_POP
+
+ ;; KCFI unspec
+ UNSPECV_KCFI_CHECK
])
(define_constants
@@ -4078,6 +4081,52 @@
DONE;
})
+;; KCFI checked call patterns
+
+(define_insn "riscv_kcfi_checked_call"
+ [(parallel [(call (mem:DI (match_operand:DI 0 "register_operand" "r"))
+ (const_int 0))
+ (unspec:DI [(const_int 0)] UNSPEC_CALLEE_CC)
+ (unspec_volatile:DI [(match_operand:SI 1 "const_int_operand" "n") ; type_id
+ (match_operand:SI 2 "const_int_operand" "n") ; prefix_nops
+ (label_ref (match_operand 3)) ; pass label
+ (label_ref (match_operand 4))] ; trap label
+ UNSPECV_KCFI_CHECK)
+ (clobber (reg:DI RETURN_ADDR_REGNUM))
+ (clobber (reg:DI T1_REGNUM)) ; t1 - scratch for loaded type
+ (clobber (reg:DI T2_REGNUM))])] ; t2 - scratch for expected type
+ "flag_sanitize & SANITIZE_KCFI"
+ "*
+ {
+ uint32_t type_id = INTVAL (operands[1]);
+ HOST_WIDE_INT prefix_nops = INTVAL (operands[2]);
+ return riscv_output_kcfi_checked_call (type_id, prefix_nops, false);
+ }"
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
+(define_insn "riscv_kcfi_checked_sibcall"
+ [(parallel [(call (mem:DI (match_operand:DI 0 "register_operand" "r"))
+ (const_int 0))
+ (unspec:DI [(const_int 0)] UNSPEC_CALLEE_CC)
+ (unspec_volatile:DI [(match_operand:SI 1 "const_int_operand" "n") ; type_id
+ (match_operand:SI 2 "const_int_operand" "n") ; prefix_nops
+ (label_ref (match_operand 3)) ; pass label
+ (label_ref (match_operand 4))] ; trap label
+ UNSPECV_KCFI_CHECK)
+ (return)
+ (clobber (reg:DI T1_REGNUM)) ; t1 - scratch for loaded type
+ (clobber (reg:DI T2_REGNUM))])] ; t2 - scratch for expected type
+ "flag_sanitize & SANITIZE_KCFI"
+ "*
+ {
+ uint32_t type_id = INTVAL (operands[1]);
+ HOST_WIDE_INT prefix_nops = INTVAL (operands[2]);
+ return riscv_output_kcfi_checked_call (type_id, prefix_nops, true);
+ }"
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
(define_insn "nop"
[(const_int 0)]
""
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 161c7024f842..f82d0464590d 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -18350,6 +18350,19 @@ trap is taken, allowing the kernel to identify both the KCFI violation
and the involved registers for detailed diagnostics (eliminating the need
for a separate @code{.kcfi_traps} section as used on x86_64).
+On RISC-V, KCFI type identifiers are emitted as a @code{.word ID}
+directive (a 32-bit constant) before the function entry, similar to AArch64.
+RISC-V's natural 4-byte instruction alignment eliminates the need for
+additional padding NOPs. When used with @option{-fpatchable-function-entry},
+the type identifier is placed before any patchable NOPs. The runtime check
+loads the actual type using @code{lw t1, OFFSET(target_reg)}, where the
+offset accounts for any prefix NOPs, constructs the expected type using
+@code{lui} and @code{addiw} instructions into @code{t2}, and compares them
+with @code{beq}. Type mismatches trigger an @code{ebreak} instruction.
+Like x86_64, RISC-V uses a @code{.kcfi_traps} section to map trap locations
+to their corresponding function entry points for debugging (RISC-V lacks
+ESR-style trap encoding unlike AArch64).
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [RFC PATCH 7/7] kcfi: Add regression test suite
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (5 preceding siblings ...)
2025-08-21 7:26 ` [RFC PATCH 6/7] riscv: Add RISC-V " Kees Cook
@ 2025-08-21 7:26 ` Kees Cook
6 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 7:26 UTC (permalink / raw)
To: Qing Zhao
Cc: Kees Cook, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
Adds a test suite for KCFI (Kernel Control Flow Integrity) ABI.
Core Functionality Tests:
- kcfi-basics.c: Basic KCFI preamble generation, IPA analysis, and
__kcfi_typeid_ symbol emission for external address-taken functions.
- kcfi-type-mangling.c: Type ID hashing validation with Itanium C++ ABI
mangling for function signatures covering basic types, pointers, const
qualifiers, multiple parameters, return types, arrays, function pointers,
structs/unions/enums, typedef canonicalization, and recursive type
definitions.
- kcfi-no-sanitize.c: no_sanitize("kcfi") attribute support for
instrumentation disabling.
- kcfi-no-sanitize-inline.c: Regression test for preserving no_sanitize("kcfi")
attributes during function inlining. Tests that functions marked with both
no_sanitize("kcfi") and always_inline correctly suppress KCFI checks when
inlined.
- kcfi-trap-section.c: Trap section (.kcfi_traps) generation and
proper trap instruction placement.
Optimization and Code Generation Tests:
- kcfi-adjacency.c: Regression test ensuring KCFI checks remain
immediately adjacent to indirect calls with no intervening instructions.
- kcfi-call-sharing.c: Prevents optimizer from incorrectly sharing
KCFI checks between different function pointer types.
- kcfi-tail-calls.c: KCFI protection preservation when indirect calls
are converted to tail calls under optimization.
- kcfi-ipa-robustness.c: IPA pass robustness with compiler-generated
constructs like static_assert.
Addressing and Environment Tests:
- kcfi-complex-addressing.c: Structure member and array element function
pointers with complex addressing modes.
- kcfi-pic-addressing.c: Position-independent code with PIC addressing
modes and symbol references.
- kcfi-cold-partition.c: Cold function attributes and block partitioning
with -freorder-blocks-and-partition.
Architecture-Specific Verification:
- kcfi-aarch64-esr.c: AArch64 ESR encoding in BRK instructions with
proper exception register values.
- kcfi-offset-validation.c: Call site offset validation across
architectures.
- kcfi-insn-sequence.c: Exact instruction sequence verification for
KCFI runtime checks.
Patchable Function Entry Integration:
- kcfi-patchable-none.c: Standard case without patchable entries.
- kcfi-patchable-basic.c: Basic patchable entries (5,2 configuration).
- kcfi-patchable-prefix-only.c: Equal prefix/entry split (3,3
configuration).
- kcfi-patchable-entry-only.c: Entry-only NOPs (4,0 configuration).
- kcfi-patchable-medium.c: Medium prefix NOPs (8,4 configuration).
- kcfi-patchable-large.c: Large prefix NOPs (11,11 configuration).
Tests can be run via:
make check-gcc RUNTESTFLAGS='kcfi.exp'
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c | 36 +
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 83 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 83 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 75 ++
.../gcc.dg/kcfi/kcfi-cold-partition.c | 133 +++
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 116 +++
.../gcc.dg/kcfi/kcfi-insn-sequence.c | 42 +
.../gcc.dg/kcfi/kcfi-ipa-robustness.c | 54 ++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 96 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 39 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 41 +
.../gcc.dg/kcfi/kcfi-patchable-basic.c | 54 ++
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 36 +
.../gcc.dg/kcfi/kcfi-patchable-large.c | 47 +
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 52 ++
.../gcc.dg/kcfi/kcfi-patchable-none.c | 18 +
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 48 +
.../gcc.dg/kcfi/kcfi-pic-addressing.c | 98 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 111 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 56 ++
.../gcc.dg/kcfi/kcfi-type-mangling.c | 864 ++++++++++++++++++
gcc/testsuite/gcc.dg/kcfi/kcfi.exp | 36 +
22 files changed, 2218 insertions(+)
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-insn-sequence.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-basic.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-none.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-pic-addressing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-type-mangling.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi.exp
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c
new file mode 100644
index 000000000000..6bbbba0431b0
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-esr.c
@@ -0,0 +1,36 @@
+/* Test AArch64 KCFI ESR encoding in BRK instructions */
+/* { dg-do compile { target aarch64*-*-* } } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void target_function(int x, char y) {
+ /* Different signature to get different type ID */
+}
+
+int main() {
+ void (*func_ptr)(int, char) = target_function;
+
+ /* This should generate BRK with ESR encoding */
+ func_ptr(42, 'a');
+
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+
+/* AArch64 specific: Should have BRK instruction with proper ESR encoding
+ ESR format: 0x8000 | ((type_reg & 31) << 5) | (addr_reg & 31)
+
+ Test the ESR encoding by checking for the expected value.
+ Since we know this test uses x2, we expect ESR = 0x8000 | (17<<5) | 2 = 33314
+
+ A truly dynamic test would need to extract the register from blr and compute
+ the corresponding ESR, but DejaGnu's regex limitations make this complex.
+ This test validates the specific case and documents the encoding.
+ */
+/* { dg-final { scan-assembler "blr\\s+x2" { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler "brk\\s+#33314" { target aarch64*-*-* } } } */
+
+/* Should have KCFI check with type comparison */
+/* { dg-final { scan-assembler {ldur\t*w16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {cmp\t*w16, w17} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
new file mode 100644
index 000000000000..35678abe0e12
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -0,0 +1,83 @@
+/* Test KCFI check/transfer adjacency - regression test for instruction insertion */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+/* This test ensures that KCFI security checks remain immediately adjacent
+ to their corresponding indirect calls/jumps, with no executable instructions
+ between the type ID check and the control flow transfer. */
+
+/* External function pointers to prevent optimization */
+extern void (*complex_func_ptr)(int, int, int, int);
+extern int (*return_func_ptr)(int, int);
+
+/* Function with complex argument preparation that could tempt
+ the optimizer to insert instructions between KCFI check and call */
+__attribute__((noinline)) void test_complex_args(int a, int b, int c, int d) {
+ /* Complex argument expressions that might cause instruction scheduling */
+ complex_func_ptr(a * 2, b + c, d - a, (a << 1) | b);
+}
+
+/* Function with return value handling */
+__attribute__((noinline)) int test_return_value(int x, int y) {
+ /* Return value handling that shouldn't interfere with adjacency */
+ int result = return_func_ptr(x + 1, y * 2);
+ return result + 1;
+}
+
+/* Test struct field access that caused issues in try-catch.c */
+struct call_info {
+ void (*handler)(void);
+ int status;
+ int data;
+};
+
+extern struct call_info *global_call_info;
+
+__attribute__((noinline)) void test_struct_field_call(void) {
+ /* This pattern caused adjacency issues before the fix */
+ global_call_info->handler();
+}
+
+/* Test conditional indirect call */
+__attribute__((noinline)) void test_conditional_call(int flag) {
+ if (flag) {
+ global_call_info->handler();
+ }
+}
+
+/* Should have KCFI instrumentation for all indirect calls */
+
+/* x86_64: Complete KCFI check sequence should be present */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r1[01]d\n\taddl\t[^,]+, %r1[01]d\n\tje\t\.L[0-9]+\n\t\.L[0-9]+:\n\tud2} { target x86_64-*-* } } } */
+
+/* AArch64: Complete KCFI check sequence should be present */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-[0-9]+\]\n\tmov\tw17, #[0-9]+\n\tmovk\tw17, #[0-9]+, lsl #16\n\tcmp\tw16, w17\n\tb\.eq\t\.L[0-9]+\n\t\.L[0-9]+:\n\tbrk\t#[0-9]+} { target aarch64*-*-* } } } */
+
+/* RISC-V: Complete KCFI check sequence should be present */
+/* { dg-final { scan-assembler {lw\tt1, -4\([a-z0-9]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, -?[0-9]+\n\tbeq\tt1, t2, \.L[a-zA-Z0-9_]+\n\.L[a-zA-Z0-9_]+:\n\tebreak} { target riscv*-*-* } } } */
+
+/* Trap instructions are now validated as part of the complete compound patterns above */
+
+/* Should have trap section with entries */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have trap section (uses brk immediate instead) */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
+
+/* Critical adjacency requirement: After each KCFI type comparison,
+ there should be only a conditional jump and label before the indirect transfer.
+ No executable instructions should be inserted between the security check
+ and the control flow transfer.
+
+ Pattern on x86_64 should be:
+ movl $(-TYPEID), %r10d # Load inverse type ID
+ addl OFFSET(%reg), %r10d # Add actual type ID
+ je .LabelN # Jump if zero (match)
+ .LN: # Trap label (not executable)
+ ud2 # Trap instruction
+ .LabelN: # Call label (not executable)
+ call/jmp *TARGET # Indirect transfer
+
+ This test serves as a regression test to ensure this adjacency
+ property is maintained across compiler changes. */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
new file mode 100644
index 000000000000..db1fcfd8f5e2
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -0,0 +1,83 @@
+/* Test basic KCFI functionality - preamble generation */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+/* Extern function declarations - should NOT get KCFI preambles */
+extern void external_func(void);
+extern int external_func_int(int x);
+
+void regular_function(int x) {
+ /* This should get KCFI preamble */
+}
+
+void static_target_function(int x) {
+ /* Target function that can be called indirectly */
+}
+
+static void static_caller(void) {
+ /* Static function that makes an indirect call */
+ /* Should NOT get KCFI preamble (not address-taken) */
+ /* But must generate KCFI check for the indirect call */
+ void (*local_ptr)(int) = static_target_function;
+ local_ptr(42); /* This should generate KCFI check */
+}
+
+void (*func_ptr)(int) = regular_function;
+void (*ext_ptr)(void) = external_func; /* Make external_func address-taken */
+
+int main() {
+ func_ptr(42);
+ ext_ptr(); /* Indirect call to external_func */
+ external_func_int(10); /* Direct call to external_func_int */
+ static_caller(); /* Direct call to static function */
+ return 0;
+}
+
+/* Verify KCFI preamble exists for regular_function */
+/* { dg-final { scan-assembler {__cfi_regular_function:} } } */
+
+/* Target function should have preamble (address-taken) */
+/* { dg-final { scan-assembler {__cfi_static_target_function:} } } */
+
+/* Static caller should NOT have preamble (it's only called directly, not address-taken) */
+/* { dg-final { scan-assembler-not {__cfi_static_caller:} } } */
+
+/* x86_64: Verify type ID in preamble (after NOPs, before function label) */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t+nop\n.*\n\t+movl\t+\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+
+/* x86_64: Static function should generate complete KCFI check sequence */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d\n\tje\t\.L[0-9]+\n\t\.L[0-9]+:\n\tud2} { target x86_64-*-* } } } */
+
+/* AArch64: Verify type ID word in preamble */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word 0x[0-9a-f]+} { target aarch64*-*-* } } } */
+
+/* AArch64: Static function should generate complete KCFI check sequence */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]\n\tmov\tw17, #[0-9]+\n\tmovk\tw17, #[0-9]+, lsl #16\n\tcmp\tw16, w17\n\tb\.eq\t\.L[0-9]+\n\t\.L[0-9]+:\n\tbrk\t#[0-9]+} { target aarch64*-*-* } } } */
+
+/* RISC-V: Verify type ID word in preamble */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target riscv*-*-* } } } */
+
+/* RISC-V: Static function should generate KCFI check for indirect call */
+/* { dg-final { scan-assembler {lw\tt1, -4\([a-z0-9]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, -?[0-9]+\n\tbeq\tt1, t2, \.L[a-zA-Z0-9_]+\n\.L[a-zA-Z0-9_]+:\n\tebreak} { target riscv*-*-* } } } */
+
+/* Extern functions should NOT get KCFI preambles */
+/* { dg-final { scan-assembler-not {__cfi_external_func:} } } */
+/* { dg-final { scan-assembler-not {__cfi_external_func_int:} } } */
+
+/* Local functions should NOT get __kcfi_typeid_ symbols */
+/* Only external declarations that are address-taken should get __kcfi_typeid_ */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_regular_function} } } */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_main} } } */
+
+/* External address-taken functions should get __kcfi_typeid_ symbols */
+/* { dg-final { scan-assembler {__kcfi_typeid_external_func} } } */
+
+/* External functions that are only called directly should NOT get __kcfi_typeid_ symbols */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_external_func_int} } } */
+
+/* Should have trap section for KCFI checks */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have trap section (uses brk immediate instead) */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
new file mode 100644
index 000000000000..e8c480e81de2
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -0,0 +1,75 @@
+/* Test KCFI check sharing bug - optimizer incorrectly shares KCFI checks between different function types */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+/* Reproduce the pattern from Linux kernel internal_create_group where:
+ - Two different function pointer types (is_visible vs is_bin_visible)
+ - Both get loaded into the same register (%rcx)
+ - Optimizer creates shared KCFI check with wrong type ID
+ - This causes CFI failures in production kernel */
+
+struct kobject { int dummy; };
+struct attribute { int dummy; };
+struct bin_attribute { int dummy; };
+
+struct attribute_group {
+ const char *name;
+ int (*is_visible)(struct kobject *, struct attribute *, int); // Type ID A
+ int (*is_bin_visible)(struct kobject *, const struct bin_attribute *, int); // Type ID B
+ struct attribute **attrs;
+ const struct bin_attribute **bin_attrs;
+};
+
+/* Function that mimics __first_visible from kernel - gets inlined into caller */
+static int __first_visible(const struct attribute_group *grp, struct kobject *kobj)
+{
+ /* Path 1: Call is_visible function pointer */
+ if (grp->attrs && grp->attrs[0] && grp->is_visible)
+ return grp->is_visible(kobj, grp->attrs[0], 0);
+
+ /* Path 2: Call is_bin_visible function pointer */
+ if (grp->bin_attrs && grp->bin_attrs[0] && grp->is_bin_visible)
+ return grp->is_bin_visible(kobj, grp->bin_attrs[0], 0);
+
+ return 0;
+}
+
+/* Main function that triggers the optimization bug */
+int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *grp)
+{
+ /* This should inline __first_visible and create the problematic pattern where:
+ 1. Both function pointers get loaded into same register
+ 2. Optimizer shares KCFI check between them
+ 3. Uses wrong type ID for one of the calls */
+ return __first_visible(grp, kobj);
+}
+
+/* Each indirect call should have its own KCFI check with correct type ID
+
+ Should see:
+ 1. KCFI check for is_visible call with is_visible type ID
+ 2. KCFI check for is_bin_visible call with is_bin_visible type ID */
+
+/* Verify we have TWO different KCFI check sequences */
+/* Each check should have different type ID constants */
+/* x86: { dg-final { scan-assembler-times {movl\s+\$-?[0-9]+,\s+%r10d} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {mov\s+w17, #[0-9]+} 2 { target aarch64*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {lui\tt2, [0-9]+} 2 { target riscv*-*-* } } } */
+
+/* Verify the checks use DIFFERENT type IDs (not shared) */
+/* We should NOT see the same type ID used twice - that would indicate sharing bug */
+/* x86: { dg-final { scan-assembler-not {movl\s+\$(-?[0-9]+),\s+%r10d.*movl\s+\$\1,\s+%r10d} { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-not {mov\s+w17, #([0-9]+).*mov\s+w17, #\1} { target aarch64*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-not {lui\s+t2, ([0-9]+)\s.*lui\s+t2, \1\s} { target riscv*-*-* } } } */
+
+/* Verify each call follows its own check (not shared) */
+/* Should have 2 separate trap instructions */
+/* x86: { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {brk\s+#[0-9]+} 2 { target aarch64*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {ebreak} 2 { target riscv*-*-* } } } */
+
+/* Verify 2 separate call sites */
+/* x86: { dg-final { scan-assembler-times {jmp\s+\*%[a-z0-9]+} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: Allow both blr (regular call) and br (tail call) */
+/* AArch64: { dg-final { scan-assembler-times {br x[0-9]+} 2 { target aarch64*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {jalr\t[a-z0-9]+} 2 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
new file mode 100644
index 000000000000..f4fa9f6646a4
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
@@ -0,0 +1,133 @@
+/* Test KCFI cold function and cold partition behavior */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2 -freorder-blocks-and-partition" } */
+
+void regular_function(void) {
+ /* Regular function should get preamble */
+}
+
+/* Cold-attributed function should STILL get preamble (it's a regular function, just marked cold) */
+__attribute__((cold))
+void cold_attributed_function(void) {
+ /* This function has cold attribute but should still get KCFI preamble */
+}
+
+/* Hot-attributed function should get preamble */
+__attribute__((hot))
+void hot_attributed_function(void) {
+ /* This function is explicitly hot and should get KCFI preamble */
+}
+
+/* Global to prevent optimization from eliminating cold paths */
+extern void abort(void);
+
+/* Additional function to test that normal functions still get preambles */
+__attribute__((noinline))
+int another_regular_function(int x) {
+ return x + 42;
+}
+
+/* Function designed to generate cold partitions under optimization */
+__attribute__((noinline))
+void function_with_cold_partition(int condition) {
+ /* Hot path - very likely to execute */
+ if (__builtin_expect(condition == 42, 1)) {
+ /* Simple hot path that optimizer will keep inline */
+ return;
+ }
+
+ /* Cold paths that actually do something to prevent elimination */
+ if (__builtin_expect(condition < 0, 0)) {
+ /* Error path 1 - call abort to prevent elimination */
+ abort();
+ }
+
+ if (__builtin_expect(condition > 1000000, 0)) {
+ /* Error path 2 - call abort to prevent elimination */
+ abort();
+ }
+
+ if (__builtin_expect(condition == 999999, 0)) {
+ /* Error path 3 - more substantial cold code */
+ volatile int sum = 0;
+ for (volatile int i = 0; i < 100; i++) {
+ sum += i * condition;
+ }
+ if (sum > 0)
+ abort();
+ }
+
+ /* More cold paths - switch with many unlikely cases */
+ switch (condition) {
+ case 1000001: case 1000002: case 1000003: case 1000004: case 1000005:
+ case 1000006: case 1000007: case 1000008: case 1000009: case 1000010:
+ /* Each case does some work before abort */
+ volatile int work = condition * 2;
+ if (work > 0) abort();
+ break;
+ default:
+ if (condition != 42) {
+ /* Fallback cold path - substantial work */
+ volatile int result = 0;
+ for (volatile int j = 0; j < condition % 50; j++) {
+ result += j;
+ }
+ if (result >= 0) abort();
+ }
+ }
+}
+
+/* Test function pointers to ensure address-taken detection works */
+void test_function_pointers(void) {
+ void (*regular_ptr)(void) = regular_function;
+ void (*cold_ptr)(void) = cold_attributed_function;
+ void (*hot_ptr)(void) = hot_attributed_function;
+
+ regular_ptr();
+ cold_ptr();
+ hot_ptr();
+}
+
+int main() {
+ regular_function();
+ cold_attributed_function();
+ hot_attributed_function();
+ function_with_cold_partition(42); /* Normal case - stay in hot path */
+ another_regular_function(5);
+ test_function_pointers();
+ return 0;
+}
+
+/* Regular function should have preamble */
+/* { dg-final { scan-assembler "__cfi_regular_function:" } } */
+
+/* Cold-attributed function should STILL have preamble (it's a legitimate function) */
+/* { dg-final { scan-assembler "__cfi_cold_attributed_function:" } } */
+
+/* Hot-attributed function should have preamble */
+/* { dg-final { scan-assembler "__cfi_hot_attributed_function:" } } */
+
+/* Function that generates cold partitions should have preamble for main entry */
+/* { dg-final { scan-assembler "__cfi_function_with_cold_partition:" } } */
+
+/* Address-taken functions should have preambles */
+/* { dg-final { scan-assembler "__cfi_test_function_pointers:" } } */
+
+/* The function should generate a .cold partition */
+/* { dg-final { scan-assembler "function_with_cold_partition\\.cold:" } } */
+
+/* The .cold partition should NOT get a __cfi_ preamble since it's never reached via indirect calls */
+/* { dg-final { scan-assembler-not "__cfi_function_with_cold_partition\\.cold:" } } */
+
+/* Additional regular function should get preamble */
+/* { dg-final { scan-assembler "__cfi_another_regular_function:" } } */
+
+/* Test coverage summary:
+ 1. Cold-attributed function (__attribute__((cold))): SHOULD get preamble
+ 2. Cold partition (-freorder-blocks-and-partition): should NOT get preamble
+ 3. IPA split .part function (split_part=true): Logic in place, would skip if triggered
+
+ Note: IPA function splitting (creating .part functions with split_part=true) requires
+ specific optimization conditions that are difficult to trigger reliably in tests.
+ The KCFI logic correctly handles this case using the split_part flag check.
+*/
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
new file mode 100644
index 000000000000..ac7e2db2d654
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -0,0 +1,116 @@
+/* Test KCFI with complex addressing modes (structure members, array elements) */
+/* This is a regression test for the change_address_1 RTL error that occurred */
+/* when target_addr was PLUS(reg, offset) instead of a simple register */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+struct function_table {
+ int (*callback1)(int);
+ int (*callback2)(int, int);
+ void (*callback3)(void);
+ int data;
+};
+
+static int handler1(int x) {
+ return x * 2;
+}
+
+static int handler2(int x, int y) {
+ return x + y;
+}
+
+static void handler3(void) {
+ /* Empty handler */
+}
+
+/* Test indirect calls through structure members - this creates PLUS(reg, offset) addressing */
+int test_struct_members(struct function_table *table) {
+ int result = 0;
+
+ /* These indirect calls will generate complex addressing modes:
+ * call *(%rdi) - callback1 at offset 0
+ * call *8(%rdi) - callback2 at offset 8
+ * call *16(%rdi) - callback3 at offset 16
+ * Our KCFI instrumentation must handle PLUS(reg, struct_offset) + kcfi_offset */
+
+ result += table->callback1(10); /* MEM[PLUS(reg, 0)] -> PLUS(reg, -4) for KCFI */
+ result += table->callback2(5, 7); /* MEM[PLUS(reg, 8)] -> PLUS(reg, 4) for KCFI */
+ table->callback3(); /* MEM[PLUS(reg, 16)] -> PLUS(reg, 12) for KCFI */
+
+ return result;
+}
+
+/* Test indirect calls through array elements - another source of complex addressing */
+typedef int (*func_array_t)(int);
+
+int test_array_elements(func_array_t functions[], int index) {
+ /* This creates addressing like MEM[PLUS(PLUS(reg, index*8), 0)]
+ * which should be simplified to MEM[PLUS(reg, index*8)] */
+ return functions[index](42); /* Complex array indexing */
+}
+
+/* Test with global structure */
+static struct function_table global_table = {
+ .callback1 = handler1,
+ .callback2 = handler2,
+ .callback3 = handler3,
+ .data = 100
+};
+
+int test_global_struct(void) {
+ /* Access through global structure - may generate different addressing patterns */
+ return global_table.callback1(20) + global_table.callback2(3, 4);
+}
+
+/* Test nested structure access */
+struct nested_table {
+ struct function_table inner;
+ int extra_data;
+};
+
+int test_nested_struct(struct nested_table *nested) {
+ /* Even more complex addressing: nested structure member access */
+ return nested->inner.callback1(15); /* MEM[PLUS(reg, inner_offset + callback1_offset)] */
+}
+
+int main() {
+ struct function_table local_table = {
+ .callback1 = handler1,
+ .callback2 = handler2,
+ .callback3 = handler3,
+ .data = 50
+ };
+
+ func_array_t func_array[] = { handler1, handler1, handler1 };
+
+ int result = 0;
+ result += test_struct_members(&local_table);
+ result += test_array_elements(func_array, 1);
+ result += test_global_struct();
+
+ struct nested_table nested = { .inner = local_table, .extra_data = 200 };
+ result += test_nested_struct(&nested);
+
+ return result;
+}
+
+/* Verify that all address-taken functions get KCFI preambles */
+/* { dg-final { scan-assembler {__cfi_handler1:} } } */
+/* { dg-final { scan-assembler {__cfi_handler2:} } } */
+/* { dg-final { scan-assembler {__cfi_handler3:} } } */
+
+/* x86_64: Verify KCFI checks are generated for indirect calls through complex addressing */
+/* The key test: these should compile without change_address_1 errors (kernel-style encoding) */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {ud2} { target x86_64-*-* } } } */
+
+/* AArch64: Verify KCFI checks for complex addressing */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {brk} { target aarch64*-*-* } } } */
+
+/* RISC-V: Verify KCFI check sequence for complex addressing */
+/* { dg-final { scan-assembler {lw\tt1, -4\([a-z0-9]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, -?[0-9]+\n\tbeq\tt1, t2, \.L[a-zA-Z0-9_]+\n\.L[a-zA-Z0-9_]+:\n\tebreak} { target riscv*-*-* } } } */
+
+/* Should have trap section (x86 and RISC-V - AArch64 uses brk immediate) */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-insn-sequence.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-insn-sequence.c
new file mode 100644
index 000000000000..eac730c54956
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-insn-sequence.c
@@ -0,0 +1,42 @@
+/* Test for exact expected KCFI instrumentation instruction sequence */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void untaken_function(int x) {
+ /* Target function should get preamble */
+}
+
+void target_function(int x) {
+ /* Target function should get preamble */
+}
+
+int main() {
+ void (*func_ptr)(int) = target_function;
+
+ /* This indirect call should get KCFI check */
+ func_ptr(42);
+
+ untaken_function(15);
+
+ return 0;
+}
+
+/* Should have KCFI preamble for target */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+
+/* Should not have KCFI preamble for local non-address-take function */
+/* { dg-final { scan-assembler-not "__cfi_untaken-function:" } } */
+
+/* x86_64: Complete KCFI check sequence (kernel-style encoding) - updated for unified trap emission */
+/* Note: Single regex to enforce exact sequence - based on actual hexdump analysis */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%rax\), %r10d\n\tje\t\.L[0-9]+\n\t\.L[0-9]+:\n\tud2} { target x86_64-*-* } } } */
+
+/* AArch64: Complete KCFI check sequence */
+/* Load type ID, load expected type, compare, conditional branch, trap, branch target, call */
+/* Note: Single regex to enforce exact sequence - based on actual hexdump analysis */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]\n\tmov\tw17, #[0-9]+\n\tmovk\tw17, #[0-9]+, lsl #16\n\tcmp\tw16, w17\n\tb\.eq\t\.L[0-9]+\n\t\.L([^:]+):\n\tbrk\t#[0-9]+\n\t\.L[0-9]+:\n\t+blr\tx[0-9]+} { target aarch64*-*-* } } } */
+
+/* RISC-V: Complete KCFI check sequence */
+/* Load type ID, load expected type, compare, conditional branch, trap, branch target, call */
+/* Note: Single regex to enforce exact sequence - based on actual hexdump analysis */
+/* { dg-final { scan-assembler {lw\tt1, -[0-9]+\([^)]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, [0-9-]+\n\tbeq\tt1, t2, \.Lkcfi_pass_[0-9]+\n\.Lkcfi_trap_[0-9]+:\n\tebreak\n\t\.pushsection\t\.kcfi_traps[^\n]*\n\.Lkcfi_trap_entry_[0-9]+:\n\t\.word\t\.Lkcfi_trap_[0-9]+ - \.Lkcfi_trap_entry_[0-9]+\n\t\.popsection\n\.Lkcfi_pass_[0-9]+:\n\tjalr\t[^,\s]+} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
new file mode 100644
index 000000000000..2a5f1d311b6b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
@@ -0,0 +1,54 @@
+/* Test KCFI IPA pass robustness with compiler-generated constructs */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+#include <stddef.h>
+
+/* Test various compiler-generated constructs that could confuse IPA pass */
+
+/* 1. static_assert - this was causing the original crash */
+typedef struct {
+ int field1;
+ char field2;
+} test_struct_t;
+
+static_assert(offsetof(test_struct_t, field1) == 0, "layout check 1");
+static_assert(offsetof(test_struct_t, field2) == 4, "layout check 2");
+static_assert(sizeof(test_struct_t) >= 5, "size check");
+
+/* 2. Regular functions that should get KCFI analysis */
+void regular_function(void) {
+ /* Should get KCFI preamble */
+}
+
+static void static_function(void) {
+ /* With -O2: correctly identified as not address-taken, no preamble */
+}
+
+void address_taken_function(void) {
+ /* Should get KCFI preamble (address taken below) */
+}
+
+/* 3. Function pointer to create address-taken scenario */
+void (*func_ptr)(void) = address_taken_function;
+
+/* 4. More static_asserts mixed with function definitions */
+static_assert(sizeof(void*) == 8, "pointer size check");
+
+int main(void) {
+ regular_function(); /* Direct call */
+ static_function(); /* Direct call to static */
+ func_ptr(); /* Indirect call */
+
+ static_assert(sizeof(int) == 4, "int size check");
+
+ return 0;
+}
+
+/* Verify KCFI preambles are generated appropriately */
+/* { dg-final { scan-assembler "__cfi_regular_function:" } } */
+/* { dg-final { scan-assembler "__cfi_address_taken_function:" } } */
+/* { dg-final { scan-assembler "__cfi_main:" } } */
+
+/* With -O2: static_function correctly identified as not address-taken */
+/* { dg-final { scan-assembler-not "__cfi_static_function:" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
new file mode 100644
index 000000000000..2f0efc16486f
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -0,0 +1,96 @@
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+/* Test that no_sanitize("kcfi") attribute is preserved during inlining */
+
+extern void external_side_effect(int value);
+
+/* Regular function (should get KCFI checks) */
+__attribute__((noinline))
+void normal_function(void (*callback)(int))
+{
+ /* This indirect call must generate KCFI checks */
+ callback(300);
+ external_side_effect(300);
+}
+
+/* Regular function marked with no_sanitize("kcfi") (positive control) */
+__attribute__((noinline, no_sanitize("kcfi")))
+void sensitive_non_inline_function(void (*callback)(int))
+{
+ /* This indirect call should NOT generate KCFI checks */
+ callback(100);
+ external_side_effect(100);
+}
+
+/* Function marked with both no_sanitize("kcfi") and always_inline */
+__attribute__((always_inline, no_sanitize("kcfi")))
+static inline void sensitive_inline_function(void (*callback)(int))
+{
+ /* This indirect call should NOT generate KCFI checks when inlined */
+ callback(42);
+ external_side_effect(42);
+}
+
+/* Explicit wrapper for testing sensitive_inline_function behavior */
+__attribute__((noinline))
+void wrap_sensitive_inline(void (*callback)(int))
+{
+ sensitive_inline_function(callback);
+}
+
+/* Function marked with only always_inline (should get KCFI checks) */
+__attribute__((always_inline))
+static inline void normal_inline_function(void (*callback)(int))
+{
+ /* This indirect call must generate KCFI checks when inlined */
+ callback(200);
+ external_side_effect(200);
+}
+
+/* Explicit wrapper for testing normal_inline_function behavior */
+__attribute__((noinline))
+void wrap_normal_inline(void (*callback)(int))
+{
+ normal_inline_function(callback);
+}
+
+void test_callback(int value)
+{
+ external_side_effect(value);
+}
+
+static void (*volatile function_pointer)(int) = test_callback;
+
+int main(void)
+{
+ void (*fn_ptr)(int) = function_pointer;
+
+ normal_function(fn_ptr);
+ wrap_normal_inline(fn_ptr);
+ sensitive_non_inline_function(fn_ptr);
+ wrap_sensitive_inline(fn_ptr);
+
+ return 0;
+}
+
+/* Verify correct number of KCFI checks: exactly 2 */
+/* { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {brk\s+#33313} 2 { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-times {ebreak} 2 { target riscv*-*-* } } } */
+
+/* Positive controls: these should have KCFI checks */
+/* { dg-final { scan-assembler {normal_function:.*ud2.*\.size\s+normal_function} { target i?86-*-* x86_64-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*ud2.*\.size\s+wrap_normal_inline} { target i?86-*-* x86_64-*-* } } } */
+/* { dg-final { scan-assembler {normal_function:.*brk\s+#33313.*\.size\s+normal_function} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*brk\s+#33313.*\.size\s+wrap_normal_inline} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {normal_function:.*ebreak.*\.size\s+normal_function} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*ebreak.*\.size\s+wrap_normal_inline} { target riscv*-*-* } } } */
+
+/* Negative controls: these should NOT have KCFI checks */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ud2.*\.size\s+sensitive_non_inline_function} { target i?86-*-* x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ud2.*\.size\s+wrap_sensitive_inline} { target i?86-*-* x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*brk\s+#33313.*\.size\s+sensitive_non_inline_function} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*brk\s+#33313.*\.size\s+wrap_sensitive_inline} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ebreak.*\.size\s+sensitive_non_inline_function} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ebreak.*\.size\s+wrap_sensitive_inline} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
new file mode 100644
index 000000000000..1a48fd1407f3
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -0,0 +1,39 @@
+/* Test KCFI with no_sanitize attribute */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void target_function(void) {
+ /* This should get KCFI preamble */
+}
+
+void caller_with_checks(void) {
+ /* This function should generate KCFI checks */
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+}
+
+__attribute__((no_sanitize("kcfi")))
+void caller_no_checks(void) {
+ /* This function should NOT generate KCFI checks due to no_sanitize */
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+}
+
+int main() {
+ caller_with_checks(); /* This should generate checks inside */
+ caller_no_checks(); /* This should NOT generate checks inside */
+ return 0;
+}
+
+/* All functions should get preambles regardless of no_sanitize */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+/* { dg-final { scan-assembler "__cfi_caller_with_checks:" } } */
+/* { dg-final { scan-assembler "__cfi_caller_no_checks:" } } */
+/* { dg-final { scan-assembler "__cfi_main:" } } */
+
+/* caller_with_checks() should generate KCFI check (1 check) */
+/* caller_no_checks() should NOT generate KCFI check due to no_sanitize attribute */
+/* Total: exactly 1 KCFI check in the entire program */
+/* { dg-final { scan-assembler-times {addl\t-4\(%r[ad]x\), %r1[01]d} 1 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {ldur\tw16, \[x[0-9]+, #-4\]} 1 { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler-times {lw\tt1, -[0-9]+\(} 1 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
new file mode 100644
index 000000000000..735892c65997
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -0,0 +1,41 @@
+/* Test KCFI call-site offset validation across architectures */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void target_func_a(void) { }
+void target_func_b(int x) { }
+void target_func_c(int x, int y) { }
+
+int main() {
+ void (*ptr_a)(void) = target_func_a;
+ void (*ptr_b)(int) = target_func_b;
+ void (*ptr_c)(int, int) = target_func_c;
+
+ /* Multiple indirect calls - each should use -4 offset in standard case */
+ ptr_a();
+ ptr_b(1);
+ ptr_c(1, 2);
+
+ return 0;
+}
+
+/* Should have KCFI preambles for all functions */
+/* { dg-final { scan-assembler "__cfi_target_func_a:" } } */
+/* { dg-final { scan-assembler "__cfi_target_func_b:" } } */
+/* { dg-final { scan-assembler "__cfi_target_func_c:" } } */
+
+/* x86_64: All call sites should use -4 offset for KCFI type ID loads (kernel-style encoding) */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+
+/* AArch64: All call sites should use -4 offset */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
+
+/* RISC-V: All call sites should use -4 offset */
+/* { dg-final { scan-assembler {lw\tt1, -4\(} { target riscv*-*-* } } } */
+
+/* Should have trap section with multiple entries */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have trap section (uses brk immediate instead) */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-basic.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-basic.c
new file mode 100644
index 000000000000..f21de4021124
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-basic.c
@@ -0,0 +1,54 @@
+/* Test KCFI with patchable function entries - basic case */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fpatchable-function-entry=5,2" } */
+
+void test_function(int x) {
+ /* Function should get both KCFI preamble and patchable entries */
+}
+
+int main() {
+ test_function(42);
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_test_function:" } } */
+
+/* Should have patchable function entry section */
+/* { dg-final { scan-assembler "__patchable_function_entries" } } */
+
+/* x86_64: Should have exactly 2 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*\.type} { target x86_64-*-* } } } */
+
+/* x86_64: Should have exactly 3 entry NOPs between .cfi_startproc and pushq */
+/* { dg-final { scan-assembler {\.cfi_startproc\n\t*nop\n\t*nop\n\t*nop\n\t*pushq} { target x86_64-*-* } } } */
+
+/* x86_64: KCFI should have exactly 9 NOPs between __cfi_ and movl */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*movl} { target x86_64-*-* } } } */
+
+/* x86_64: Validate KCFI type ID is present */
+/* { dg-final { scan-assembler {movl\t\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+
+/* AArch64: Should have exactly 2 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*\.type} { target aarch64*-*-* } } } */
+
+/* AArch64: Should have exactly 3 entry NOPs between .cfi_startproc and sub sp */
+/* { dg-final { scan-assembler {\.cfi_startproc\n\t*nop\n\t*nop\n\t*nop\n\t*sub\t*sp} { target aarch64*-*-* } } } */
+
+/* AArch64: KCFI should have only .word immediate (no NOPs) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t\.word 0x[0-9a-f]+} { target aarch64*-*-* } } } */
+
+/* AArch64: Validate clean KCFI boundary - .word then immediate end/size */
+/* { dg-final { scan-assembler {\.word 0x[0-9a-f]+\n\.Lcfi_func_end_test_function:\n\t\.size\t__cfi_test_function, \.-__cfi_test_function} { target aarch64*-*-* } } } */
+
+/* RISC-V: Should have exactly 2 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*\.type} { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 3 entry NOPs before .cfi_startproc followed by addi sp */
+/* { dg-final { scan-assembler {nop\n\t*nop\n\t*nop\n\.LFB[0-9]+:\n\t*\.cfi_startproc\n\t*addi\t*sp} { target riscv*-*-* } } } */
+
+/* RISC-V: KCFI should have only .word immediate (no NOPs) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t\.word\t0x[0-9a-f]+} { target riscv*-*-* } } } */
+
+/* RISC-V: Validate clean KCFI boundary - .word then immediate end/size */
+/* { dg-final { scan-assembler {\.word\t0x[0-9a-f]+\n\.Lcfi_func_end_test_function:\n\t\.size\t__cfi_test_function, \.-__cfi_test_function} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
new file mode 100644
index 000000000000..68a42dd32995
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -0,0 +1,36 @@
+/* Test KCFI with patchable function entries - entry NOPs only */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fpatchable-function-entry=4,0" } */
+
+void test_function(void) {
+ /* All NOPs are entry NOPs, no prefix NOPs for KCFI */
+}
+
+int main() {
+ test_function();
+ return 0;
+}
+
+/* Should NOT have KCFI preamble (no prefix NOPs) */
+/* { dg-final { scan-assembler-not "__cfi_test_function:" } } */
+
+/* x86_64: All 4 NOPs are entry NOPs - should have exactly 4 entry NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*pushq} { target x86_64-*-* } } } */
+
+/* x86_64: No prefix NOPs - function type should come immediately before function */
+/* { dg-final { scan-assembler {\.type\t*test_function, @function\n*test_function:} { target x86_64-*-* } } } */
+
+/* AArch64: All 4 NOPs are entry NOPs - should have exactly 4 entry NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*stp} { target aarch64*-*-* } } } */
+
+/* AArch64: No prefix NOPs - function type should come immediately before function */
+/* { dg-final { scan-assembler {\.type\t*test_function, %function\n*test_function:} { target aarch64*-*-* } } } */
+
+/* RISC-V: All 4 NOPs are entry NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\.LFB} { target riscv*-*-* } } } */
+
+/* RISC-V: No prefix NOPs - function type should come immediately before function */
+/* { dg-final { scan-assembler {\.type\t*test_function, @function\n*test_function:} { target riscv*-*-* } } } */
+
+/* Should have patchable function entry section */
+/* { dg-final { scan-assembler "__patchable_function_entries" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
new file mode 100644
index 000000000000..015b021b9a0a
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -0,0 +1,47 @@
+/* Test KCFI with large patchable function entries */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fpatchable-function-entry=11,11" } */
+
+void test_function(void) {
+ /* 11 prefix NOPs, 0 entry NOPs - maximum prefix case */
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr(); /* Call site should use -(11+4) = -15 offset */
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_test_function:" } } */
+
+/* Should have patchable function entry section */
+/* { dg-final { scan-assembler "__patchable_function_entries" } } */
+
+/* x86_64: Should have exactly 11 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target x86_64-*-* } } } */
+
+/* x86_64: Should have 0 entry NOPs - function starts immediately with pushq (no __kcfi_typeid for definitions) */
+/* { dg-final { scan-assembler {test_function:\n\.LFB[0-9]+:\n\t*\.cfi_startproc\n\t*pushq\t*%rbp} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {\t*\.weak\t*__kcfi_typeid_test_function\n} { target x86_64-*-* } } } */
+
+/* x86_64: KCFI should have 0 NOPs - goes directly to movl (offsetToAlignment(11+5,16)=0) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t*movl} { target x86_64-*-* } } } */
+
+/* x86_64: Validate KCFI type ID is present */
+/* { dg-final { scan-assembler {movl\t\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+
+/* x86_64: Call site should use -15 offset (kernel-style encoding) */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-15\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+
+/* AArch64: Should have exactly 11 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target aarch64*-*-* } } } */
+
+/* AArch64: Call site should use -15 offset */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-15\]} { target aarch64*-*-* } } } */
+
+/* RISC-V: Should have 11 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target riscv*-*-* } } } */
+
+/* RISC-V: Call site should use -15 offset (same as x86/AArch64) */
+/* { dg-final { scan-assembler {lw\tt1, -15\(} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
new file mode 100644
index 000000000000..e6922cf81355
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -0,0 +1,52 @@
+/* Test KCFI with medium patchable function entries */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fpatchable-function-entry=8,4" } */
+
+void test_function(void) {
+ /* 4 prefix NOPs, 4 entry NOPs - should affect KCFI offset */
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr(); /* Call site should use -(4+4) = -8 offset */
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_test_function:" } } */
+
+/* Should have patchable function entry section */
+/* { dg-final { scan-assembler "__patchable_function_entries" } } */
+
+/* x86_64: Should have exactly 4 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target x86_64-*-* } } } */
+
+/* x86_64: Should have exactly 4 entry NOPs between .cfi_startproc and pushq */
+/* { dg-final { scan-assembler {\.cfi_startproc\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*pushq} { target x86_64-*-* } } } */
+
+/* x86_64: KCFI should have exactly 7 NOPs between __cfi_ and movl (offsetToAlignment(4+5,16)=7) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*movl} { target x86_64-*-* } } } */
+
+/* x86_64: Validate KCFI type ID is present */
+/* { dg-final { scan-assembler {movl\t\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+
+/* x86_64: Call site should use -8 offset (kernel-style encoding) */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-8\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+
+/* AArch64: Should have exactly 4 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target aarch64*-*-* } } } */
+
+/* AArch64: Should have exactly 4 entry NOPs after .cfi_startproc */
+/* { dg-final { scan-assembler {\.cfi_startproc\n\t*nop\n\t*nop\n\t*nop\n\t*nop} { target aarch64*-*-* } } } */
+
+/* AArch64: Call site should use -8 offset */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-8\]} { target aarch64*-*-* } } } */
+
+/* RISC-V: Should have exactly 4 prefix NOPs between .LPFE and .type */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*\.type} { target riscv*-*-* } } } */
+
+/* RISC-V: Should have 4 entry NOPs */
+/* { dg-final { scan-assembler {test_function:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\.LFB} { target riscv*-*-* } } } */
+
+/* RISC-V: Call site should use -8 offset (same as x86/AArch64) */
+/* { dg-final { scan-assembler {lw\tt1, -8\(} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-none.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-none.c
new file mode 100644
index 000000000000..01a611ca754d
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-none.c
@@ -0,0 +1,18 @@
+/* Test KCFI without patchable function entries - standard case */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void test_function(void) {
+ /* Standard KCFI without patchable entries */
+}
+
+int main() {
+ test_function();
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_test_function:" } } */
+
+/* Should NOT have patchable function entry section */
+/* { dg-final { scan-assembler-not "__patchable_function_entries" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
new file mode 100644
index 000000000000..5a500f3a8ea9
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -0,0 +1,48 @@
+/* Test KCFI with patchable function entries - prefix NOPs only */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fpatchable-function-entry=3,3" } */
+
+void test_function(void) {
+ /* All NOPs are prefix NOPs, should integrate with KCFI */
+}
+
+int main() {
+ test_function();
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_test_function:" } } */
+
+/* x86_64: All 3 NOPs are prefix NOPs - should have exactly 3 prefix NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*\.type\t*test_function} { target x86_64-*-* } } } */
+
+/* x86_64: No entry NOPs - function should start immediately with prologue (no __kcfi_typeid for definitions) */
+/* { dg-final { scan-assembler {test_function:\n\.LFB[0-9]+:\n\t*\.cfi_startproc\n\t*pushq\t*%rbp} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {\t*\.weak\t*__kcfi_typeid_test_function\n} { target x86_64-*-* } } } */
+
+/* x86_64: KCFI padding should have exactly 8 NOPs (offsetToAlignment(3+5,16)=8) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*nop\n\t*movl} { target x86_64-*-* } } } */
+
+/* AArch64: All 3 NOPs are prefix NOPs - should have exactly 3 prefix NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*\.type\t*test_function} { target aarch64*-*-* } } } */
+
+/* AArch64: No entry NOPs - function should start immediately with prologue (no __kcfi_typeid for definitions) */
+/* { dg-final { scan-assembler {test_function:\n\.LFB[0-9]+:\n\t*\.cfi_startproc\n\t*nop\n\t*ret} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {\t*\.weak\t*__kcfi_typeid_test_function\n} { target aarch64*-*-* } } } */
+
+/* AArch64: KCFI type ID should be immediate word (no alignment NOPs needed) */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t\.word 0x[0-9a-f]+} { target aarch64*-*-* } } } */
+
+/* RISC-V: All 3 NOPs are prefix NOPs - should have exactly 3 prefix NOPs */
+/* { dg-final { scan-assembler {\.LPFE[0-9]+:\n\t*nop\n\t*nop\n\t*nop\n\t*\.type\t*test_function} { target riscv*-*-* } } } */
+
+/* RISC-V: No entry NOPs - function should start immediately with .cfi_startproc */
+/* { dg-final { scan-assembler {test_function:\n\.LFB[0-9]+:\n\t*\.cfi_startproc} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-not {\t*\.weak\t*__kcfi_typeid_test_function\n} { target riscv*-*-* } } } */
+
+/* RISC-V: KCFI type ID should be immediate word */
+/* { dg-final { scan-assembler {__cfi_test_function:\n\t\.word\t0x[0-9a-f]+} { target riscv*-*-* } } } */
+
+/* Should have patchable function entry section */
+/* { dg-final { scan-assembler "__patchable_function_entries" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-pic-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-pic-addressing.c
new file mode 100644
index 000000000000..d6c580e1e502
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-pic-addressing.c
@@ -0,0 +1,98 @@
+/* Test KCFI with position-independent code addressing modes */
+/* This is a regression test for complex addressing like PLUS(PLUS(...), symbol_ref) */
+/* which can occur with PIC and caused change_address_1 RTL errors */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2 -fpic" } */
+
+/* Global function pointer table that creates PIC addressing */
+struct callbacks {
+ int (*handler1)(int);
+ void (*handler2)(void);
+ int (*handler3)(int, int);
+};
+
+static int simple_handler(int x) {
+ return x * 2;
+}
+
+static void void_handler(void) {
+ /* Empty handler */
+}
+
+static int complex_handler(int a, int b) {
+ return a + b;
+}
+
+/* Global structure that will require PIC addressing */
+struct callbacks global_callbacks = {
+ .handler1 = simple_handler,
+ .handler2 = void_handler,
+ .handler3 = complex_handler
+};
+
+/* Function that uses PIC addressing to access global callbacks */
+int test_pic_addressing(int value) {
+ /* These indirect calls through global structure create complex addressing
+ * like PLUS(PLUS(GOT_base, symbol_offset), struct_offset) which previously
+ * caused RTL errors in KCFI instrumentation */
+
+ int result = 0;
+ result += global_callbacks.handler1(value);
+
+ global_callbacks.handler2();
+
+ result += global_callbacks.handler3(value, result);
+
+ return result;
+}
+
+/* Test with function pointer arrays */
+static int (*func_array[])(int) = {
+ simple_handler,
+ simple_handler,
+ simple_handler
+};
+
+int test_pic_array(int index, int value) {
+ /* Array access with PIC can also create complex addressing */
+ return func_array[index % 3](value);
+}
+
+/* Test with dynamic PIC addressing */
+struct callbacks *get_callbacks(void) {
+ return &global_callbacks;
+}
+
+int test_dynamic_pic(int value) {
+ /* Dynamic access through function call creates very complex addressing */
+ struct callbacks *cb = get_callbacks();
+ return cb->handler1(value) + cb->handler3(value, value);
+}
+
+int main() {
+ int result = 0;
+ result += test_pic_addressing(10);
+ result += test_pic_array(1, 20);
+ result += test_dynamic_pic(5);
+ return result;
+}
+
+/* Verify that all address-taken functions get KCFI preambles */
+/* { dg-final { scan-assembler {__cfi_simple_handler:} } } */
+/* { dg-final { scan-assembler {__cfi_void_handler:} } } */
+/* { dg-final { scan-assembler {__cfi_complex_handler:} } } */
+
+/* x86_64: Verify KCFI checks are generated (should compile without RTL errors, kernel-style encoding) */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {ud2} { target x86_64-*-* } } } */
+
+/* AArch64: Verify KCFI checks */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {brk} { target aarch64*-*-* } } } */
+
+/* RISC-V: Verify complete KCFI check sequence */
+/* { dg-final { scan-assembler {lw\tt1, -4\([a-z0-9]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, -?[0-9]+\n\tbeq\tt1, t2, \.L[a-zA-Z0-9_]+\n\.L[a-zA-Z0-9_]+:\n\tebreak} { target riscv*-*-* } } } */
+
+/* Should have trap section */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.kcfi_traps} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
new file mode 100644
index 000000000000..7b8877de4254
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -0,0 +1,111 @@
+/* Test KCFI protection when indirect calls get converted to tail calls */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+typedef int (*func_ptr_t)(int);
+typedef void (*void_func_ptr_t)(void);
+
+struct function_table {
+ func_ptr_t process;
+ void_func_ptr_t cleanup;
+};
+
+/* Target functions */
+int process_data(int x) { return x * 2; }
+void cleanup_data(void) {}
+
+/* Initialize function table */
+volatile struct function_table vtable = {
+ .process = &process_data,
+ .cleanup = &cleanup_data
+};
+
+/* Test 1: Indirect call through struct member that should become tail call */
+int test_struct_indirect_call(int x) {
+ /* This is an indirect call that should be converted to tail call:
+ * Without -fno-optimize-sibling-calls: should become "jmp *vtable+0(%rip)" with KCFI
+ * With -fno-optimize-sibling-calls: should be "call *vtable+0(%rip)" with KCFI */
+ return vtable.process(x);
+}
+
+/* Test 2: Indirect call through function pointer parameter */
+int test_param_indirect_call(func_ptr_t handler, int x) {
+ /* This is an indirect call that should be converted to tail call:
+ * Without -fno-optimize-sibling-calls: should become "jmp *%rdi" with KCFI
+ * With -fno-optimize-sibling-calls: should be "call *%rdi" with KCFI */
+ return handler(x);
+}
+
+/* Test 3: Void indirect call through struct member */
+void test_void_indirect_call(void) {
+ /* This is an indirect call that should be converted to tail call:
+ * Without -fno-optimize-sibling-calls: should become "jmp *vtable+8(%rip)" with KCFI
+ * With -fno-optimize-sibling-calls: should be "call *vtable+8(%rip)" with KCFI */
+ vtable.cleanup();
+}
+
+/* Test 4: Non-tail call for comparison (should always be call with KCFI) */
+int test_non_tail_indirect_call(func_ptr_t handler, int x) {
+ /* This should never become a tail call - always "call *%rdi" with KCFI */
+ int result = handler(x);
+ return result + 1; /* Prevents tail call optimization */
+}
+
+/* Should have KCFI preambles for all functions (same for both architectures) */
+/* { dg-final { scan-assembler-times "__cfi_process_data:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_cleanup_data:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_struct_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_param_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_void_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_non_tail_indirect_call:" 1 } } */
+
+/* Should have exactly 4 KCFI checks for indirect calls (load type ID + compare) */
+/* { dg-final { scan-assembler-times {movl\t\$-?[0-9]+, %r10d} 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {addl\t-4\(%r[a-z0-9]+\), %r10d} 4 { target x86_64-*-* } } } */
+
+/* Should have exactly 4 trap sections and 4 trap instructions */
+/* { dg-final { scan-assembler-times "\\.kcfi_traps" 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times "ud2" 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times "\\.kcfi_traps" 4 { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-times "ebreak" 4 { target riscv*-*-* } } } */
+
+/* Should NOT have unprotected direct jumps to vtable */
+/* { dg-final { scan-assembler-not {jmp\t\*vtable\(%rip\)} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {jmp\t\*vtable\+8\(%rip\)} { target x86_64-*-* } } } */
+
+/* Should have exactly 3 protected tail calls (jmp through register after KCFI check) */
+/* { dg-final { scan-assembler-times {jmp\t\*%[a-z0-9]+} 3 { target x86_64-*-* } } } */
+
+/* Should have exactly 1 regular call (non-tail call case) */
+/* { dg-final { scan-assembler-times {call\t\*%[a-z0-9]+} 1 { target x86_64-*-* } } } */
+
+/* RISC-V: Should have exactly 4 complete KCFI check sequences */
+/* { dg-final { scan-assembler-times {lw\tt1, -4\([a-z0-9]+\)\n\tlui\tt2, [0-9]+\n\taddiw\tt2, t2, -?[0-9]+\n\tbeq\tt1, t2, \.L[a-zA-Z0-9_]+\n\.L[a-zA-Z0-9_]+:\n\tebreak} 4 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 4 KCFI checks for indirect calls (load type ID + compare) */
+/* { dg-final { scan-assembler-times {lw\tt1, -4\([a-z0-9]+\)} 4 { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-times {lui\tt2, [0-9]+} 4 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 3 protected tail calls (jr after KCFI check - no return address save) */
+/* { dg-final { scan-assembler-times {jalr\tx0, [a-z0-9]+, 0} 3 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 1 regular call (non-tail call case - saves return address) */
+/* { dg-final { scan-assembler-times {jalr\tx1, [a-z0-9]+, 0} 1 { target riscv*-*-* } } } */
+
+/* AArch64-specific patterns */
+/* Should have exactly 4 KCFI checks for indirect calls (load type ID from -4 offset + compare) */
+/* { dg-final { scan-assembler-times {ldur\tw16, \[x[0-9]+, #-4\]} 4 { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler-times {cmp\tw16, w17} 4 { target aarch64-*-* } } } */
+
+/* Should have exactly 4 trap instructions */
+/* { dg-final { scan-assembler-times {brk\t#[0-9]+} 4 { target aarch64-*-* } } } */
+
+/* Should have exactly 3 protected tail calls (br through register after KCFI check) */
+/* { dg-final { scan-assembler-times {br\tx[0-9]+} 3 { target aarch64-*-* } } } */
+
+/* Should have exactly 1 regular call (non-tail call case) */
+/* { dg-final { scan-assembler-times {blr\tx[0-9]+} 1 { target aarch64-*-* } } } */
+
+/* Type ID loading should use mov + movk pattern for 32-bit constants */
+/* { dg-final { scan-assembler {mov\tw17, #[0-9]+} { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler {movk\tw17, #[0-9]+, lsl #16} { target aarch64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
new file mode 100644
index 000000000000..8b0c4819e11b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -0,0 +1,56 @@
+/* Test KCFI trap section generation */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi" } */
+
+void target_function(void) {}
+
+int main() {
+ void (*func_ptr)(void) = target_function;
+
+ /* Multiple indirect calls to generate trap entries */
+ func_ptr();
+ func_ptr();
+
+ return 0;
+}
+
+/* Should have KCFI preamble */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+
+/* Should have exactly 2 trap labels in code */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ud2} 2 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*brk} 2 { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ebreak} 2 { target riscv*-*-* } } } */
+
+/* Should have .kcfi_traps section with exactly 2 entries (x86 and RISC-V - AArch64 uses brk immediate) */
+/* { dg-final { scan-assembler {\.pushsection\t*\.kcfi_traps,"ao",@progbits,\.text} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\t*\.long} 2 { target x86_64-*-* } } } */
+
+/* AArch64 should NOT have .kcfi_traps section (uses brk immediate instead) */
+/* { dg-final { scan-assembler-not {\.pushsection\t*\.kcfi_traps} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {\.L[^:]+:\n\t*\.long} { target aarch64*-*-* } } } */
+
+/* Each section entry must reference its corresponding trap label directly (x86 and RISC-V) */
+/* Entry references trap label directly (no offset needed) */
+/* { dg-final { scan-assembler {\.L([^:]+):\n\t*\.long\t*\.L([^\s]+)\s+-\s+\.L\1} { target x86_64-*-* } } } */
+
+/* RISC-V: Should have .kcfi_traps section with exactly 2 entries */
+/* { dg-final { scan-assembler {\.pushsection\t*\.kcfi_traps,"ao",@progbits,\.text} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-times {\.Lkcfi_trap_entry_[0-9]+:\n\t*\.word} 2 { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have RISC-V-style .kcfi_traps section entries */
+/* { dg-final { scan-assembler-not {\.Lkcfi_trap_entry_[0-9]+:\n\t*\.word} { target aarch64*-*-* } } } */
+
+/* RISC-V section entries must reference their corresponding trap labels */
+/* Entry references trap label directly (no offset needed) */
+/* { dg-final { scan-assembler {\.Lkcfi_trap_entry_([0-9]+):\n\t*\.word\t*\.Lkcfi_trap_\1\s+-\s+\.Lkcfi_trap_entry_\1} { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have RISC-V-style trap label references */
+/* { dg-final { scan-assembler-not {\.Lkcfi_trap_entry_([0-9]+):\n\t*\.word\t*\.Lkcfi_trap_\1} { target aarch64*-*-* } } } */
+
+/* Section must be properly closed (x86 and RISC-V) */
+/* { dg-final { scan-assembler {\.popsection} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {\.popsection} { target riscv*-*-* } } } */
+
+/* AArch64 should NOT have .popsection (no .kcfi_traps section to close) */
+/* { dg-final { scan-assembler-not {\.popsection} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-type-mangling.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-type-mangling.c
new file mode 100644
index 000000000000..36cdfd3ed890
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-type-mangling.c
@@ -0,0 +1,864 @@
+/* Test KCFI type ID hashing - verify different signatures generate different __kcfi_typeid_ symbols */
+/* { dg-do compile } */
+/* { dg-options "-fsanitize=kcfi -fdump-tree-kcfi0-details" } */
+
+/* Test __kcfi_typeid_ symbol generation for address-taken functions.
+ Verify precise type discrimination using Itanium C++ ABI mangling. */
+
+/* External function declarations - these will get __kcfi_typeid_ symbols when address-taken */
+extern void func_void(void); /* FvvE -> 0x68d6d5b6 */
+extern void func_char(char x); /* FvcE -> 0x14f227f7 */
+extern void func_int(int x); /* FviE -> 0x28e33ce9 */
+extern void func_long(long x); /* FvlE -> 0x64f06750 */
+
+/* Basic types - verify exact type IDs match with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_void\n\t\.set\t__kcfi_typeid_func_void, 0x68d6d5b6} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_char\n\t\.set\t__kcfi_typeid_func_char, 0x14f227f7} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_int\n\t\.set\t__kcfi_typeid_func_int, 0x28e33ce9} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_long\n\t\.set\t__kcfi_typeid_func_long, 0x64f06750} } } */
+
+/* Verify basic types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvvE' typeid=0x68d6d5b6} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvcE' typeid=0x14f227f7} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFviE' typeid=0x28e33ce9} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvlE' typeid=0x64f06750} kcfi0 } } */
+
+/* Count verification - basic types (void type used by multiple functions) */
+/* { dg-final { scan-assembler-times {0x68d6d5b6} 2 } } +1 from local_func_void preamble below */
+/* { dg-final { scan-assembler-times {0x14f227f7} 1 } } */
+/* { dg-final { scan-assembler-times {0x28e33ce9} 1 } } */
+/* { dg-final { scan-assembler-times {0x64f06750} 1 } } */
+
+/* Pointer parameter types - must all differ */
+extern void func_int_ptr(int *x); /* FvPiE -> 0x6d478137 */
+extern void func_char_ptr(char *x); /* FvPcE -> 0x81389629 */
+extern void func_void_ptr(void *x); /* FvPvE -> 0x5d1c8700 */
+
+/* Pointer types - verify they all differ with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_int_ptr\n\t\.set\t__kcfi_typeid_func_int_ptr, 0x6d478137} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_char_ptr\n\t\.set\t__kcfi_typeid_func_char_ptr, 0x81389629} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_void_ptr\n\t\.set\t__kcfi_typeid_func_void_ptr, 0x5d1c8700} } } */
+
+/* Verify pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPiE' typeid=0x6d478137} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPcE' typeid=0x81389629} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPvE' typeid=0x5d1c8700} kcfi0 } } */
+
+/* Count verification - pointer types (will appear twice due to array decay earlier) */
+/* { dg-final { scan-assembler-times {0x6d478137} 2 } } */
+/* { dg-final { scan-assembler-times {0x81389629} 2 } } */
+/* { dg-final { scan-assembler-times {0x5d1c8700} 1 } } */
+
+/* Const qualifier discrimination - const vs non-const must have different type IDs */
+extern void func_const_int_ptr(const int *x); /* FvPKiE -> const int* (must differ from int*) */
+extern void func_const_char_ptr(const char *x); /* FvPKcE -> const char* (must differ from char*) */
+extern void func_const_void_ptr(const void *x); /* FvPKvE -> const void* (must differ from void*) */
+
+/* Const qualifier types - verify const vs non-const have different type IDs */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_int_ptr\n\t\.set\t__kcfi_typeid_func_const_int_ptr, 0xcc6626a0} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_char_ptr\n\t\.set\t__kcfi_typeid_func_const_char_ptr, 0xb0750516} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_void_ptr\n\t\.set\t__kcfi_typeid_func_const_void_ptr, 0xdc8f8dd7} } } */
+
+/* Verify const qualifier types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPKiE' typeid=0xcc6626a0} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPKcE' typeid=0xb0750516} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPKvE' typeid=0xdc8f8dd7} kcfi0 } } */
+
+/* Count verification - const qualifier types should appear exactly once */
+/* { dg-final { scan-assembler-times {0xcc6626a0} 1 } } */
+/* { dg-final { scan-assembler-times {0xb0750516} 1 } } */
+/* { dg-final { scan-assembler-times {0xdc8f8dd7} 1 } } */
+
+/* Nested pointer types */
+extern void func_int_ptr_ptr(int **x); /* FvPPiE -> 0x3d62bef1 */
+extern void func_char_ptr_ptr(char **x); /* FvPPcE -> 0x494939ef */
+
+/* Nested pointers with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_int_ptr_ptr\n\t\.set\t__kcfi_typeid_func_int_ptr_ptr, 0x3d62bef1} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_char_ptr_ptr\n\t\.set\t__kcfi_typeid_func_char_ptr_ptr, 0x494939ef} } } */
+
+/* Verify nested pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPPiE' typeid=0x3d62bef1} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPPcE' typeid=0x494939ef} kcfi0 } } */
+
+/* Count verification - nested pointer types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x3d62bef1} 1 } } */
+/* { dg-final { scan-assembler-times {0x494939ef} 1 } } */
+
+/* Multiple parameter types - order matters */
+extern void func_int_char(int x, char y); /* FvicE -> 0x0e0cb81e */
+extern void func_char_int(char x, int y); /* FvciE -> 0xab661a02 */
+extern void func_two_int(int x, int y); /* FviiE -> 0x0a2649b8 */
+
+/* Multiple parameter tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_int_char\n\t\.set\t__kcfi_typeid_func_int_char, 0x0e0cb81e} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_char_int\n\t\.set\t__kcfi_typeid_func_char_int, 0xab661a02} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_two_int\n\t\.set\t__kcfi_typeid_func_two_int, 0x0a2649b8} } } */
+
+/* Verify multiple parameter types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvicE' typeid=0x0e0cb81e} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvciE' typeid=0xab661a02} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFviiE' typeid=0x0a2649b8} kcfi0 } } */
+
+/* Count verification - multiple parameter types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x0e0cb81e} 1 } } */
+/* { dg-final { scan-assembler-times {0xab661a02} 1 } } */
+/* { dg-final { scan-assembler-times {0x0a2649b8} 1 } } */
+
+/* Return types */
+extern int func_return_int(void); /* FivE -> 0x426f60ef */
+extern char func_return_char(void); /* FcvE -> 0x3688e5f1 */
+extern void* func_return_ptr(void); /* FPvvE -> 0x924cfbe6 */
+
+/* Return type tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_return_int\n\t\.set\t__kcfi_typeid_func_return_int, 0x4193590b} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_return_char\n\t\.set\t__kcfi_typeid_func_return_char, 0xe340f049} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_return_ptr\n\t\.set\t__kcfi_typeid_func_return_ptr, 0xa8267e10} } } */
+
+/* Verify return types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFivE' typeid=0x4193590b} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFcvE' typeid=0xe340f049} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFPvvE' typeid=0xa8267e10} kcfi0 } } */
+
+/* Count verification - return types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x4193590b} 1 } } */
+/* { dg-final { scan-assembler-times {0xe340f049} 1 } } */
+/* { dg-final { scan-assembler-times {0xa8267e10} 1 } } */
+
+/* Array parameters - decay to pointers */
+extern void func_int_array(int arr[]); /* FvPiE -> 0x6d478137 (same as int*) */
+extern void func_char_array(char arr[]); /* FvPcE -> 0x81389629 (same as char*) */
+
+/* Array decay validation - arrays should have SAME type ID as corresponding pointers */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_int_array\n\t\.set\t__kcfi_typeid_func_int_array, 0x6d478137} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_char_array\n\t\.set\t__kcfi_typeid_func_char_array, 0x81389629} } } */
+/* Counted below. */
+
+/* Function pointer parameters */
+extern void func_fptr_void(void (*fp)(void)); /* FvPFvvEE -> 0xbe908da1 */
+extern void func_fptr_int(void (*fp)(int)); /* FvPFviEE -> 0x8f3bc4fe */
+extern void func_fptr_ret_int(int (*fp)(void)); /* FvPFivEE -> 0xdcab0e2c */
+
+/* Function pointer parameter tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_fptr_void\n\t\.set\t__kcfi_typeid_func_fptr_void, 0x5f78c457} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_fptr_int\n\t\.set\t__kcfi_typeid_func_fptr_int, 0x8f3bc4fe} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_fptr_ret_int\n\t\.set\t__kcfi_typeid_func_fptr_ret_int, 0x5993cc14} } } */
+
+/* Verify function pointer parameter types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPFvvEE' typeid=0x5f78c457} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPFviEE' typeid=0x8f3bc4fe} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPFivEE' typeid=0x5993cc14} kcfi0 } } */
+
+/* Count verification - function pointer parameter types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x5f78c457} 1 } } */
+/* { dg-final { scan-assembler-times {0x8f3bc4fe} 1 } } */
+/* { dg-final { scan-assembler-times {0x5993cc14} 1 } } */
+
+/* Struct/union/enum parameter types: each struct name must produce different type IDs */
+struct test_struct_a { int x; };
+struct test_struct_b { int y; };
+struct test_struct_c { int z; };
+union test_union_a { int i; float f; };
+union test_union_b { int j; float g; };
+enum test_enum_a { ENUM_A_VAL1, ENUM_A_VAL2 };
+enum test_enum_b { ENUM_B_VAL1, ENUM_B_VAL2 };
+
+/* Functions taking struct pointers - must have different type IDs */
+extern void func_struct_a_ptr(struct test_struct_a *x); /* Fv14test_struct_aPiE -> unique */
+extern void func_struct_b_ptr(struct test_struct_b *x); /* Fv14test_struct_bPiE -> unique */
+extern void func_struct_c_ptr(struct test_struct_c *x); /* Fv14test_struct_cPiE -> unique */
+
+/* Struct pointer tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_a_ptr\n\t\.set\t__kcfi_typeid_func_struct_a_ptr, 0x778e8c02} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_b_ptr\n\t\.set\t__kcfi_typeid_func_struct_b_ptr, 0x1791c679} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_c_ptr\n\t\.set\t__kcfi_typeid_func_struct_c_ptr, 0x43944a54} } } */
+
+/* Verify struct pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP13test_struct_aE' typeid=0x778e8c02} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP13test_struct_bE' typeid=0x1791c679} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP13test_struct_cE' typeid=0x43944a54} kcfi0 } } */
+
+/* Count verification - struct pointer types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x778e8c02} 1 } } */
+/* { dg-final { scan-assembler-times {0x1791c679} 1 } } */
+/* { dg-final { scan-assembler-times {0x43944a54} 1 } } */
+
+/* Functions taking const struct pointers - must differ from non-const versions */
+extern void func_const_struct_a_ptr(const struct test_struct_a *x); /* FvPK14test_struct_aE -> unique, different from non-const */
+extern void func_const_struct_b_ptr(const struct test_struct_b *x); /* FvPK14test_struct_bE -> unique, different from non-const */
+extern void func_const_struct_c_ptr(const struct test_struct_c *x); /* FvPK14test_struct_cE -> unique, different from non-const */
+
+/* Const struct pointer tests with precise patterns - must differ from non-const */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_struct_a_ptr\n\t\.set\t__kcfi_typeid_func_const_struct_a_ptr, 0x763752c9} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_struct_b_ptr\n\t\.set\t__kcfi_typeid_func_const_struct_b_ptr, 0x5634e1d2} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_const_struct_c_ptr\n\t\.set\t__kcfi_typeid_func_const_struct_c_ptr, 0x32326a8f} } } */
+
+/* Verify const struct pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPK13test_struct_aE' typeid=0x763752c9} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPK13test_struct_bE' typeid=0x5634e1d2} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvPK13test_struct_cE' typeid=0x32326a8f} kcfi0 } } */
+
+/* Count verification - const struct pointer types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x763752c9} 1 } } */
+/* { dg-final { scan-assembler-times {0x5634e1d2} 1 } } */
+/* { dg-final { scan-assembler-times {0x32326a8f} 1 } } */
+
+extern void func_union_a_ptr(union test_union_a *x); /* Fv13test_union_aPiE -> unique */
+extern void func_union_b_ptr(union test_union_b *x); /* Fv13test_union_bPiE -> unique */
+extern void func_enum_a_ptr(enum test_enum_a *x); /* Fv11test_enum_aPiE -> unique */
+extern void func_enum_b_ptr(enum test_enum_b *x); /* Fv11test_enum_bPiE -> unique */
+
+/* Union member access discrimination test - prevents regression of union member bug */
+struct tasklet_like_struct {
+ int state;
+ union {
+ /* First union member - should NOT be used for callback calls */
+ void (*func)(unsigned long data);
+ /* Second union member - should be used for callback calls */
+ void (*callback)(struct tasklet_like_struct *t);
+ };
+ unsigned long data;
+};
+
+/* Function with callback signature - this should match when accessed via union->callback */
+extern void tasklet_callback_function(struct tasklet_like_struct *t); /* FvP19tasklet_like_structE -> unique */
+
+/* Function with func signature - this should NOT match callback calls */
+extern void tasklet_func_function(unsigned long data); /* FvmE -> different from callback */
+
+/* Union member access discrimination tests - MUST use correct union member type */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_tasklet_callback_function\n\t\.set\t__kcfi_typeid_tasklet_callback_function, 0x5634f4ec} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_tasklet_func_function\n\t\.set\t__kcfi_typeid_tasklet_func_function, 0x48edfca5} } } */
+
+/* Verify union member discrimination tests */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP19tasklet_like_structE' typeid=0x5634f4ec} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvmE' typeid=0x48edfca5} kcfi0 } } */
+
+/* Union pointer tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_union_a_ptr\n\t\.set\t__kcfi_typeid_func_union_a_ptr, 0xb03d9275} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_union_b_ptr\n\t\.set\t__kcfi_typeid_func_union_b_ptr, 0x003a3ece} } } */
+
+/* Verify union pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP12test_union_aE' typeid=0xb03d9275} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP12test_union_bE' typeid=0x003a3ece} kcfi0 } } */
+
+/* Count verification - union pointer types should appear exactly once */
+/* { dg-final { scan-assembler-times {0xb03d9275} 1 } } */
+/* { dg-final { scan-assembler-times {0x003a3ece} 1 } } */
+
+/* Enum pointer tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_enum_a_ptr\n\t\.set\t__kcfi_typeid_func_enum_a_ptr, 0x9a32df78} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_enum_b_ptr\n\t\.set\t__kcfi_typeid_func_enum_b_ptr, 0xaa2c3ce3} } } */
+
+/* Verify enum pointer types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP11test_enum_aE' typeid=0x9a32df78} kcfi0 } } */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP11test_enum_bE' typeid=0xaa2c3ce3} kcfi0 } } */
+
+/* Count verification - enum pointer types should appear exactly once */
+/* { dg-final { scan-assembler-times {0x9a32df78} 1 } } */
+/* { dg-final { scan-assembler-times {0xaa2c3ce3} 1 } } */
+
+/* Count verification - union member discrimination types should appear exactly once */
+/* The key test is that callback and func functions have DIFFERENT type IDs, proving union member discrimination works */
+/* { dg-final { scan-assembler-times {0x5634f4ec} 1 } } */
+/* { dg-final { scan-assembler-times {0x48edfca5} 1 } } */
+
+/* Indirect call through t->callback union must use correct callback type ID (0x5634f4ec).
+ The decimal value -1446311148 is the two's complement of 0x5634f4ec used in KCFI checks. */
+/* { dg-final { scan-assembler-times {\tmovl\t\$-1446311148, %r10d} 1 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {\tmov\tw17, #62700\n\tmovk\tw17, #22068, lsl #16} 1 { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler-times {\tlui\tt2, 353103\n\taddiw\tt2, t2, 1260} 1 { target riscv*-*-* } } } */
+
+/* Functions returning struct pointers - must have different type IDs */
+extern struct test_struct_a* func_ret_struct_a_ptr(void); /* F14test_struct_aPvE -> unique */
+extern struct test_struct_b* func_ret_struct_b_ptr(void); /* F14test_struct_bPvE -> unique */
+extern struct test_struct_c* func_ret_struct_c_ptr(void); /* F14test_struct_cPvE -> unique */
+
+/* Struct return pointer tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_a_ptr\n\t\.set\t__kcfi_typeid_func_ret_struct_a_ptr, 0x39c14222} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_b_ptr\n\t\.set\t__kcfi_typeid_func_ret_struct_b_ptr, 0xe26c7223} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_c_ptr\n\t\.set\t__kcfi_typeid_func_ret_struct_c_ptr, 0x4efac19c} } } */
+
+/* Verify struct return pointer types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFP13test_struct_avE' typeid=0x39c14222} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFP13test_struct_bvE' typeid=0xe26c7223} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFP13test_struct_cvE' typeid=0x4efac19c} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0x39c14222} 1 } } */
+/* { dg-final { scan-assembler-times {0xe26c7223} 1 } } */
+/* { dg-final { scan-assembler-times {0x4efac19c} 1 } } */
+
+/* Functions taking structs by value - must have different type IDs */
+extern void func_struct_a_val(struct test_struct_a x); /* Fv14test_struct_aE -> unique */
+extern void func_struct_b_val(struct test_struct_b x); /* Fv14test_struct_bE -> unique */
+extern void func_struct_c_val(struct test_struct_c x); /* Fv14test_struct_cE -> unique */
+
+/* Struct by-value parameter tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_a_val\n\t\.set\t__kcfi_typeid_func_struct_a_val, 0x3c685468} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_b_val\n\t\.set\t__kcfi_typeid_func_struct_b_val, 0xcc60e853} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_c_val\n\t\.set\t__kcfi_typeid_func_struct_c_val, 0xf0635f96} } } */
+
+/* Verify struct by-value parameter types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFv13test_struct_aE' typeid=0x3c685468} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFv13test_struct_bE' typeid=0xcc60e853} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFv13test_struct_cE' typeid=0xf0635f96} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0x3c685468} 1 } } */
+/* { dg-final { scan-assembler-times {0xcc60e853} 1 } } */
+/* { dg-final { scan-assembler-times {0xf0635f96} 1 } } */
+
+/* Functions returning structs by value - must have different type IDs */
+extern struct test_struct_a func_ret_struct_a_val(void); /* F14test_struct_avE -> unique */
+extern struct test_struct_b func_ret_struct_b_val(void); /* F14test_struct_bvE -> unique */
+extern struct test_struct_c func_ret_struct_c_val(void); /* F14test_struct_cvE -> unique */
+
+/* Struct return by-value tests with precise patterns */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_a_val\n\t\.set\t__kcfi_typeid_func_ret_struct_a_val, 0x872d60f8} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_b_val\n\t\.set\t__kcfi_typeid_func_ret_struct_b_val, 0x12ecd535} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_struct_c_val\n\t\.set\t__kcfi_typeid_func_ret_struct_c_val, 0x6f7b0b7e} } } */
+
+/* Verify struct return by-value types - using correct P prefix for function pointer */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPF13test_struct_avE' typeid=0x872d60f8} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPF13test_struct_bvE' typeid=0x12ecd535} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPF13test_struct_cvE' typeid=0x6f7b0b7e} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0x872d60f8} 1 } } */
+/* { dg-final { scan-assembler-times {0x12ecd535} 1 } } */
+/* { dg-final { scan-assembler-times {0x6f7b0b7e} 1 } } */
+
+/* Mixed struct parameters - order and type must matter */
+extern void func_struct_a_b(struct test_struct_a *a, struct test_struct_b *b); /* unique */
+extern void func_struct_b_a(struct test_struct_b *b, struct test_struct_a *a); /* different! */
+
+/* Mixed struct parameter tests - MUST be different (parameter order matters) */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_a_b\n\t\.set\t__kcfi_typeid_func_struct_a_b, 0x3858f101} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_b_a\n\t\.set\t__kcfi_typeid_func_struct_b_a, 0x9c8f2985} } } */
+
+/* Verify mixed struct parameter types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP13test_struct_aP13test_struct_bE' typeid=0x3858f101} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP13test_struct_bP13test_struct_aE' typeid=0x9c8f2985} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0x3858f101} 1 } } */
+/* { dg-final { scan-assembler-times {0x9c8f2985} 1 } } */
+
+/* Typedef structs - must be different from named structs */
+typedef struct { int value; } typedef_struct_x;
+typedef struct { int value; } typedef_struct_y; /* Same layout but different typedef name */
+extern void func_typedef_x_ptr(typedef_struct_x *x); /* Must be unique */
+extern void func_typedef_y_ptr(typedef_struct_y *x); /* Must be different from typedef_struct_x */
+
+/* Typedef struct tests - MUST be different from each other */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_typedef_x_ptr\n\t\.set\t__kcfi_typeid_func_typedef_x_ptr, 0xd04404df} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_typedef_y_ptr\n\t\.set\t__kcfi_typeid_func_typedef_y_ptr, 0xf4467c22} } } */
+
+/* Verify typedef struct pointer types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP16typedef_struct_xE' typeid=0xd04404df} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP16typedef_struct_yE' typeid=0xf4467c22} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0xd04404df} 1 } } */
+/* { dg-final { scan-assembler-times {0xf4467c22} 1 } } */
+
+/* Typedef vs open-coded function types - MUST have identical type IDs */
+typedef void (*func_ptr_typedef)(int x, char y);
+extern void func_with_typedef_param(func_ptr_typedef fp); /* Should match open-coded */
+extern void func_with_opencoded_param(void (*fp)(int x, char y)); /* Should match typedef */
+
+/* Function parameter types - typedef and open-coded should generate SAME type ID */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_with_typedef_param\n\t\.set\t__kcfi_typeid_func_with_typedef_param, 0xc3f653f3} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_with_opencoded_param\n\t\.set\t__kcfi_typeid_func_with_opencoded_param, 0xc3f653f3} } } */
+
+/* Verify function pointer parameter types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvPFvicEE' typeid=0xc3f653f3} kcfi0 } } */
+
+/* Verify exact count - each typedef/opencoded pair should generate exactly 2 symbols with identical values */
+/* { dg-final { scan-assembler-times {0xc3f653f3} 2 } } */
+
+/* Typedef vs open-coded function types - MUST have identical type IDs */
+typedef int (*ret_func_ptr_typedef)(void);
+extern ret_func_ptr_typedef func_ret_typedef_param(void); /* Should match open-coded */
+extern int (*func_ret_opencoded_param(void))(void); /* Should match typedef */
+
+/* Return function pointer types - typedef and open-coded should generate SAME type ID */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_typedef_param\n\t\.set\t__kcfi_typeid_func_ret_typedef_param, 0xff38a80c} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_ret_opencoded_param\n\t\.set\t__kcfi_typeid_func_ret_opencoded_param, 0xff38a80c} } } */
+
+/* Verify return function pointer types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFPFivEvE' typeid=0xff38a80c} kcfi0 } } */
+
+/* Verify exact count - each typedef/opencoded pair should generate exactly 2 symbols with identical values */
+/* { dg-final { scan-assembler-times {0xff38a80c} 2 } } */
+
+/* Anonymous struct via typedef - should get typedef name as struct name */
+typedef struct { int anon_member_1; } anon_typedef_1;
+typedef struct { int anon_member_2; } anon_typedef_2;
+extern void func_anon_typedef_1(anon_typedef_1 *param); /* Should use typedef name */
+extern void func_anon_typedef_2(anon_typedef_2 *param); /* Should be different from anon_typedef_1 */
+
+/* Anonymous typedef struct tests - MUST be different from each other */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_anon_typedef_1\n\t\.set\t__kcfi_typeid_func_anon_typedef_1, 0xeb2cfcf5} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_anon_typedef_2\n\t\.set\t__kcfi_typeid_func_anon_typedef_2, 0x3b29a94e} } } */
+
+/* Verify anonymous typedef struct types */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP14anon_typedef_1E' typeid=0xeb2cfcf5} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvP14anon_typedef_2E' typeid=0x3b29a94e} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0xeb2cfcf5} 1 } } */
+/* { dg-final { scan-assembler-times {0x3b29a94e} 1 } } */
+
+/* Local function definitions - these will NOT get __kcfi_typeid_ symbols (only external declarations do) */
+void local_func_void(void) { } /* FvvE -> 0x126cd6c8 */
+void local_func_short(short x) { } /* FvsE -> 0x34cb4ae7 */
+void local_func_uint(unsigned int x) { } /* FvjE -> 0x08e0cbf2 */
+void local_func_float(float x) { } /* FvfE -> 0x48ff45c6 */
+
+/* Local function validation - verify local function definitions do NOT get __kcfi_typeid_ symbols */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_void\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_short\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_uint\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_float\n} } } */
+
+/* Local pointer parameter types */
+void local_func_double_ptr(double *x) { } /* FvPdE -> 0x713f38be */
+void local_func_float_ptr(float *x) { } /* FvPfE -> 0xbd442d90 */
+
+/* Local pointer parameter types - should NOT emit symbols */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_double_ptr\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_float_ptr\n} } } */
+
+/* Local nested pointers */
+void local_func_void_ptr_ptr(void **x) { } /* FvPPvE -> 0x1d2eb12e */
+
+/* Local nested pointers - should NOT emit symbols */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_void_ptr_ptr\n} } } */
+
+/* Local mixed parameters */
+void local_func_ptr_val(int *x, int y) { } /* FvPiiE -> 0x79c26342 */
+void local_func_val_ptr(int x, int *y) { } /* FviPiE -> 0x467eba8c */
+
+/* Local mixed parameter validation - should NOT emit symbols */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_ptr_val\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_val_ptr\n} } } */
+
+/* Local return types */
+float local_func_return_float(void) { return 0.0f; } /* FfvE -> 0xf29546d8 */
+double local_func_return_double(void) { return 0.0; } /* FdvE -> 0x268f8886 */
+
+/* Local return type discrimination - should NOT emit symbols */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_return_float\n} } } */
+/* { dg-final { scan-assembler-not {\t\.weak\t__kcfi_typeid_local_func_return_double\n} } } */
+
+/* Verify local function mangle strings appear in KCFI dump (even though no symbols are emitted) */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvsE' typeid=0x34cb4ae7} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvfE' typeid=0x48ff45c6} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvPdE' typeid=0x713f38be} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvPfE' typeid=0xbd442d90} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvPPvE' typeid=0x1d2eb12e} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFvPiiE' typeid=0x79c26342} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFviPiE' typeid=0x467eba8c} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFfvE' typeid=0x3b5a51e6} kcfi0 } } */
+/* { dg-final { scan-tree-dump {KCFI type ID: mangled='_ZTSPFdvE' typeid=0x1043bac0} kcfi0 } } */
+
+struct not_void {
+ int nothing;
+};
+
+/* Function that takes addresses to make functions visible to KCFI */
+void test_address_taken(struct not_void *arg)
+{
+ /* External functions - taking addresses generates __kcfi_typeid_ symbols */
+ void (*p1)(void) = func_void;
+ void (*p2)(char) = func_char;
+ void (*p3)(int) = func_int;
+ void (*p4)(long) = func_long;
+
+ void (*p5)(int*) = func_int_ptr;
+ void (*p6)(char*) = func_char_ptr;
+ void (*p7)(void*) = func_void_ptr;
+
+ void (*p_const_int_ptr)(const int*) = func_const_int_ptr;
+ void (*p_const_char_ptr)(const char*) = func_const_char_ptr;
+ void (*p_const_void_ptr)(const void*) = func_const_void_ptr;
+
+ void (*p8)(int**) = func_int_ptr_ptr;
+ void (*p9)(char**) = func_char_ptr_ptr;
+
+ void (*p10)(int, char) = func_int_char;
+ void (*p11)(char, int) = func_char_int;
+ void (*p12)(int, int) = func_two_int;
+
+ int (*p13)(void) = func_return_int;
+ char (*p14)(void) = func_return_char;
+ void* (*p15)(void) = func_return_ptr;
+
+ /* Array parameters - should decay to pointers */
+ void (*p16)(int*) = func_int_array;
+ void (*p17)(char*) = func_char_array;
+
+ /* Function pointer parameters */
+ void (*p18)(void(*)(void)) = func_fptr_void;
+ void (*p19)(void(*)(int)) = func_fptr_int;
+ void (*p20)(int(*)(void)) = func_fptr_ret_int;
+
+ /* Struct/union/enum function pointers - taking addresses generates __kcfi_typeid_ symbols */
+ void (*p_struct_a_ptr)(struct test_struct_a*) = func_struct_a_ptr;
+ void (*p_struct_b_ptr)(struct test_struct_b*) = func_struct_b_ptr;
+ void (*p_struct_c_ptr)(struct test_struct_c*) = func_struct_c_ptr;
+
+ /* Const struct function pointers - taking addresses generates __kcfi_typeid_ symbols */
+ void (*p_const_struct_a_ptr)(const struct test_struct_a*) = func_const_struct_a_ptr;
+ void (*p_const_struct_b_ptr)(const struct test_struct_b*) = func_const_struct_b_ptr;
+ void (*p_const_struct_c_ptr)(const struct test_struct_c*) = func_const_struct_c_ptr;
+ void (*p_union_a_ptr)(union test_union_a*) = func_union_a_ptr;
+ void (*p_union_b_ptr)(union test_union_b*) = func_union_b_ptr;
+ void (*p_enum_a_ptr)(enum test_enum_a*) = func_enum_a_ptr;
+ void (*p_enum_b_ptr)(enum test_enum_b*) = func_enum_b_ptr;
+
+ struct test_struct_a* (*p_ret_struct_a_ptr)(void) = func_ret_struct_a_ptr;
+ struct test_struct_b* (*p_ret_struct_b_ptr)(void) = func_ret_struct_b_ptr;
+ struct test_struct_c* (*p_ret_struct_c_ptr)(void) = func_ret_struct_c_ptr;
+
+ void (*p_struct_a_val)(struct test_struct_a) = func_struct_a_val;
+ void (*p_struct_b_val)(struct test_struct_b) = func_struct_b_val;
+ void (*p_struct_c_val)(struct test_struct_c) = func_struct_c_val;
+
+ struct test_struct_a (*p_ret_struct_a_val)(void) = func_ret_struct_a_val;
+ struct test_struct_b (*p_ret_struct_b_val)(void) = func_ret_struct_b_val;
+ struct test_struct_c (*p_ret_struct_c_val)(void) = func_ret_struct_c_val;
+
+ void (*p_struct_a_b)(struct test_struct_a*, struct test_struct_b*) = func_struct_a_b;
+ void (*p_struct_b_a)(struct test_struct_b*, struct test_struct_a*) = func_struct_b_a;
+
+ void (*p_typedef_x_ptr)(typedef_struct_x*) = func_typedef_x_ptr;
+ void (*p_typedef_y_ptr)(typedef_struct_y*) = func_typedef_y_ptr;
+
+ /* Typedef vs open-coded function type assignments - should generate identical type IDs */
+ void (*p_with_typedef_param)(func_ptr_typedef) = func_with_typedef_param;
+ void (*p_with_opencoded_param)(void (*)(int, char)) = func_with_opencoded_param;
+ ret_func_ptr_typedef (*p_ret_typedef_param)(void) = func_ret_typedef_param;
+ int (*(*p_ret_opencoded_param)(void))(void) = func_ret_opencoded_param;
+
+ /* Anonymous struct typedef assignments - should generate unique type IDs */
+ void (*p_anon_typedef_1)(anon_typedef_1 *) = func_anon_typedef_1;
+ void (*p_anon_typedef_2)(anon_typedef_2 *) = func_anon_typedef_2;
+
+ /* Union member access discrimination test - take addresses to generate __kcfi_typeid_ symbols */
+ void (*p_tasklet_callback)(struct tasklet_like_struct *) = tasklet_callback_function;
+ void (*p_tasklet_func)(unsigned long) = tasklet_func_function;
+
+ /* Local functions - taking addresses does NOT generate __kcfi_typeid_ symbols (only external declarations do) */
+ void (*p21)(void) = local_func_void;
+ void (*p22)(short) = local_func_short;
+ void (*p23)(unsigned int) = local_func_uint;
+ void (*p24)(float) = local_func_float;
+
+ void (*p25)(double*) = local_func_double_ptr;
+ void (*p26)(float*) = local_func_float_ptr;
+
+ void (*p27)(void**) = local_func_void_ptr_ptr;
+
+ void (*p28)(int*, int) = local_func_ptr_val;
+ void (*p29)(int, int*) = local_func_val_ptr;
+
+ float (*p30)(void) = local_func_return_float;
+ double (*p31)(void) = local_func_return_double;
+
+ /* Use pointers to prevent optimization - external functions */
+ if (p1) p1();
+ if (p2) p2('x');
+ if (p3) p3(42);
+ if (p4) p4(42L);
+ if (p5) p5((int*)0);
+ if (p6) p6((char*)0);
+ if (p7) p7((void*)0);
+
+ /* Use const qualifier pointers to prevent optimization */
+ if (p_const_int_ptr) p_const_int_ptr((const int*)0);
+ if (p_const_char_ptr) p_const_char_ptr((const char*)0);
+ if (p_const_void_ptr) p_const_void_ptr((const void*)0);
+ if (p8) p8((int**)0);
+ if (p9) p9((char**)0);
+ if (p10) p10(1, 'x');
+ if (p11) p11('x', 1);
+ if (p12) p12(1, 2);
+ if (p13) p13();
+ if (p14) p14();
+ if (p15) p15();
+ if (p16) p16((int*)0);
+ if (p17) p17((char*)0);
+ if (p18) p18((void(*)(void))0);
+ if (p19) p19((void(*)(int))0);
+ if (p20) p20((int(*)(void))0);
+
+ /* Use pointers to prevent optimization - local functions */
+ if (p21) p21();
+ if (p22) p22(1);
+ if (p23) p23(1U);
+ if (p24) p24(1.0f);
+ if (p25) p25((double*)0);
+ if (p26) p26((float*)0);
+ if (p27) p27((void**)0);
+ if (p28) p28((int*)0, 1);
+ if (p29) p29(1, (int*)0);
+ if (p30) p30();
+ if (p31) p31();
+
+ /* Use struct/union/enum function pointers to generate KCFI type IDs */
+ if (p_struct_a_ptr) p_struct_a_ptr((struct test_struct_a*)0);
+ if (p_struct_b_ptr) p_struct_b_ptr((struct test_struct_b*)0);
+ if (p_struct_c_ptr) p_struct_c_ptr((struct test_struct_c*)0);
+ if (p_const_struct_a_ptr) p_const_struct_a_ptr((const struct test_struct_a*)0);
+ if (p_const_struct_b_ptr) p_const_struct_b_ptr((const struct test_struct_b*)0);
+ if (p_const_struct_c_ptr) p_const_struct_c_ptr((const struct test_struct_c*)0);
+ if (p_union_a_ptr) p_union_a_ptr((union test_union_a*)0);
+ if (p_union_b_ptr) p_union_b_ptr((union test_union_b*)0);
+ if (p_enum_a_ptr) p_enum_a_ptr((enum test_enum_a*)0);
+ if (p_enum_b_ptr) p_enum_b_ptr((enum test_enum_b*)0);
+
+ /* Use struct return type function pointers to generate type IDs */
+ if (p_ret_struct_a_ptr) p_ret_struct_a_ptr();
+ if (p_ret_struct_b_ptr) p_ret_struct_b_ptr();
+ if (p_ret_struct_c_ptr) p_ret_struct_c_ptr();
+
+ /* Use struct by-value parameter function pointers to generate type IDs */
+ struct test_struct_a dummy_a = {};
+ struct test_struct_b dummy_b = {};
+ struct test_struct_c dummy_c = {};
+ if (p_struct_a_val) p_struct_a_val(dummy_a);
+ if (p_struct_b_val) p_struct_b_val(dummy_b);
+ if (p_struct_c_val) p_struct_c_val(dummy_c);
+
+ /* Use struct return by-value function pointers to generate type IDs */
+ if (p_ret_struct_a_val) p_ret_struct_a_val();
+ if (p_ret_struct_b_val) p_ret_struct_b_val();
+ if (p_ret_struct_c_val) p_ret_struct_c_val();
+
+ /* Use multi-parameter struct function pointers to generate type IDs */
+ if (p_struct_a_b) p_struct_a_b((struct test_struct_a*)0, (struct test_struct_b*)0);
+ if (p_struct_b_a) p_struct_b_a((struct test_struct_b*)0, (struct test_struct_a*)0);
+
+ /* Use typedef struct function pointers to generate type IDs */
+ if (p_typedef_x_ptr) p_typedef_x_ptr((typedef_struct_x*)0);
+ if (p_typedef_y_ptr) p_typedef_y_ptr((typedef_struct_y*)0);
+
+ /* Use typedef vs open-coded function pointers to generate type IDs */
+ if (p_with_typedef_param) p_with_typedef_param((func_ptr_typedef)0);
+ if (p_with_opencoded_param) p_with_opencoded_param((void (*)(int, char))0);
+ if (p_ret_typedef_param) p_ret_typedef_param();
+ if (p_ret_opencoded_param) p_ret_opencoded_param();
+
+ /* Use anonymous typedef function pointers to generate type IDs */
+ if (p_anon_typedef_1) p_anon_typedef_1((anon_typedef_1*)0);
+ if (p_anon_typedef_2) p_anon_typedef_2((anon_typedef_2*)0);
+
+ /* Use tasklet func function pointer to generate the missing FvmE type ID */
+ if (p_tasklet_func) p_tasklet_func(0);
+
+ struct tasklet_like_struct test_tasklet = { };
+ test_tasklet.callback = tasklet_callback_function; /* Set callback function */
+
+ /* This indirect call through union->callback MUST generate type ID 0x5634f4ec (callback signature) */
+ /* NOT type ID 0x48edfca5 (func signature from first union member) */
+ /* Force indirect call through union member access */
+ struct tasklet_like_struct *volatile tasklet_ptr = &test_tasklet;
+ if (tasklet_ptr->callback) {
+ /* This call should match tasklet_callback_function type ID */
+ tasklet_ptr->callback(tasklet_ptr);
+ }
+}
+
+/* Named struct and its typedef should have IDENTICAL type IDs after canonicalization */
+struct named_for_typedef_test { int member; };
+typedef struct named_for_typedef_test named_for_typedef_test_t;
+
+extern void func_named_struct_param(struct named_for_typedef_test *param);
+extern void func_typedef_struct_param(named_for_typedef_test_t *param);
+
+/* Named struct typedef canonicalization - MUST have identical type IDs */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_named_struct_param\n\t\.set\t__kcfi_typeid_func_named_struct_param, 0x010f62ae} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_typedef_struct_param\n\t\.set\t__kcfi_typeid_func_typedef_struct_param, 0x010f62ae} } } */
+
+/* Verify named struct typedef canonicalization types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP22named_for_typedef_testE' typeid=0x010f62ae} kcfi0 } } */
+
+/* Verify exact count - both should generate exactly 2 symbols with identical values */
+/* { dg-final { scan-assembler-times {0x010f62ae} 2 } } */
+
+void test_named_struct_typedef_canonicalization(struct not_void *arg) {
+ /* These should be compatible after canonicalization */
+ void (*fp_struct)(struct named_for_typedef_test *) = func_named_struct_param;
+ void (*fp_typedef)(struct named_for_typedef_test *) = func_typedef_struct_param; /* Should work with canonicalization */
+
+ /* Take addresses to generate type IDs */
+ if (fp_struct) fp_struct((struct named_for_typedef_test *)0);
+ if (fp_typedef) fp_typedef((struct named_for_typedef_test *)0);
+}
+
+/* Basic type typedef canonicalization - typedef should canonicalize to underlying basic type */
+
+/* Basic type typedefs commonly used in kernel code */
+typedef unsigned char u8;
+typedef unsigned short u16;
+typedef unsigned int u32;
+
+/* Functions with basic type typedef vs original type parameters */
+extern void func_u8_param(u8 param);
+extern void func_unsigned_char_param(unsigned char param);
+extern void func_u16_param(u16 param);
+extern void func_unsigned_short_param(unsigned short param);
+extern void func_u32_param(u32 param);
+extern void func_unsigned_int_param(unsigned int param);
+
+void test_basic_typedef_canonicalization(struct not_void *arg) {
+ /* These should be compatible after canonicalization */
+ void (*fp_u8)(unsigned char) = func_u8_param; /* Should work with canonicalization */
+ void (*fp_uchar)(unsigned char) = func_unsigned_char_param; /* Should work normally */
+ void (*fp_u16)(unsigned short) = func_u16_param; /* Should work with canonicalization */
+ void (*fp_ushort)(unsigned short) = func_unsigned_short_param; /* Should work normally */
+ void (*fp_u32)(unsigned int) = func_u32_param; /* Should work with canonicalization */
+ void (*fp_uint)(unsigned int) = func_unsigned_int_param; /* Should work normally */
+
+ /* Take addresses to generate type IDs */
+ if (fp_u8) fp_u8(0);
+ if (fp_uchar) fp_uchar(0);
+ if (fp_u16) fp_u16(0);
+ if (fp_ushort) fp_ushort(0);
+ if (fp_u32) fp_u32(0);
+ if (fp_uint) fp_uint(0);
+}
+
+/* Basic type typedef canonicalization - MUST have identical type IDs after canonicalization */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u8_param\n\t\.set\t__kcfi_typeid_func_u8_param, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_unsigned_char_param\n\t\.set\t__kcfi_typeid_func_unsigned_char_param, 0x54e5c0c4} } } */
+
+/* Verify basic type canonicalization (u8/unsigned char) */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvhE' typeid=0x54e5c0c4} kcfi0 } } */
+
+/* Count test is below, which includes other tests that use this hash. */
+
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u16_param\n\t\.set\t__kcfi_typeid_func_u16_param, 0x34dc9408} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_unsigned_short_param\n\t\.set\t__kcfi_typeid_func_unsigned_short_param, 0x34dc9408} } } */
+
+/* Verify basic type canonicalization (u16/unsigned short) */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvtE' typeid=0x34dc9408} kcfi0 } } */
+
+/* { dg-final { scan-assembler-times {0x34dc9408} 2 } } */
+
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u32_param\n\t\.set\t__kcfi_typeid_func_u32_param, 0x08e0cbf2} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_unsigned_int_param\n\t\.set\t__kcfi_typeid_func_unsigned_int_param, 0x08e0cbf2} } } */
+
+/* Verify basic type canonicalization (u32/unsigned int) */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvjE' typeid=0x08e0cbf2} kcfi0 } } */
+
+/* Count test is below, which includes other tests that use this hash. */
+
+/* Verify exact count - each typedef/basic type pair should generate exactly 2 symbols with identical values */
+/* Note: Counts updated below to include recursive typedef tests */
+
+/* Recursive typedef canonicalization - test multi-level typedef chains */
+
+/* Kernel-style recursive typedef chains that need full canonicalization */
+typedef unsigned char __u8_recursive;
+typedef __u8_recursive u8_recursive;
+
+typedef unsigned int __u32_recursive;
+typedef __u32_recursive u32_recursive;
+
+/* Three-level typedef chains */
+typedef unsigned char base_u8_recursive_t;
+typedef base_u8_recursive_t mid_u8_recursive_t;
+typedef mid_u8_recursive_t top_u8_recursive_t;
+
+/* Struct recursive typedef chains */
+struct recursive_struct_test { int value; };
+typedef struct recursive_struct_test base_recursive_struct_t;
+typedef base_recursive_struct_t top_recursive_struct_t;
+
+/* Functions with recursive typedefs - MUST have same type IDs as canonical forms */
+extern void func_u8_recursive_chain(u8_recursive param); /* u8_recursive -> __u8_recursive -> unsigned char */
+extern void func_u8_recursive_mid(__u8_recursive param); /* __u8_recursive -> unsigned char */
+extern void func_u8_recursive_base(unsigned char param); /* unsigned char (baseline) */
+
+extern void func_u32_recursive_chain(u32_recursive param); /* u32_recursive -> __u32_recursive -> unsigned int */
+extern void func_u32_recursive_mid(__u32_recursive param); /* __u32_recursive -> unsigned int */
+extern void func_u32_recursive_base(unsigned int param); /* unsigned int (baseline) */
+
+extern void func_three_level_recursive(top_u8_recursive_t param); /* top -> mid -> base -> unsigned char */
+extern void func_three_level_mid(mid_u8_recursive_t param); /* mid -> base -> unsigned char */
+extern void func_three_level_base(base_u8_recursive_t param); /* base -> unsigned char */
+extern void func_three_level_final(unsigned char param); /* unsigned char (baseline) */
+
+extern void func_struct_recursive_chain(top_recursive_struct_t *param); /* Should resolve to struct name */
+extern void func_struct_recursive_mid(base_recursive_struct_t *param); /* Should resolve to struct name */
+extern void func_struct_recursive_original(struct recursive_struct_test *param); /* struct name (baseline) */
+
+void test_recursive_canonicalization(struct not_void *arg) {
+ /* Recursive typedef function pointers - should be compatible after full canonicalization */
+ void (*fp_u8_chain)(unsigned char) = func_u8_recursive_chain; /* Should work after 2-level canonicalization */
+ void (*fp_u8_mid)(unsigned char) = func_u8_recursive_mid; /* Should work after 1-level canonicalization */
+ void (*fp_u8_base)(unsigned char) = func_u8_recursive_base; /* Should work normally */
+
+ void (*fp_u32_chain)(unsigned int) = func_u32_recursive_chain; /* Should work after 2-level canonicalization */
+ void (*fp_u32_mid)(unsigned int) = func_u32_recursive_mid; /* Should work after 1-level canonicalization */
+ void (*fp_u32_base)(unsigned int) = func_u32_recursive_base; /* Should work normally */
+
+ void (*fp_three_chain)(unsigned char) = func_three_level_recursive; /* Should work after 3-level canonicalization */
+ void (*fp_three_mid)(unsigned char) = func_three_level_mid; /* Should work after 2-level canonicalization */
+ void (*fp_three_base)(unsigned char) = func_three_level_base; /* Should work after 1-level canonicalization */
+ void (*fp_three_final)(unsigned char) = func_three_level_final; /* Should work normally */
+
+ void (*fp_struct_chain)(struct recursive_struct_test *) = func_struct_recursive_chain; /* Should work after canonicalization */
+ void (*fp_struct_mid)(struct recursive_struct_test *) = func_struct_recursive_mid; /* Should work after canonicalization */
+ void (*fp_struct_orig)(struct recursive_struct_test *) = func_struct_recursive_original; /* Should work normally */
+
+ /* Use function pointers to prevent optimization */
+ if (fp_u8_chain) fp_u8_chain(0);
+ if (fp_u8_mid) fp_u8_mid(0);
+ if (fp_u8_base) fp_u8_base(0);
+ if (fp_u32_chain) fp_u32_chain(0);
+ if (fp_u32_mid) fp_u32_mid(0);
+ if (fp_u32_base) fp_u32_base(0);
+ if (fp_three_chain) fp_three_chain(0);
+ if (fp_three_mid) fp_three_mid(0);
+ if (fp_three_base) fp_three_base(0);
+ if (fp_three_final) fp_three_final(0);
+ if (fp_struct_chain) fp_struct_chain((struct recursive_struct_test *)0);
+ if (fp_struct_mid) fp_struct_mid((struct recursive_struct_test *)0);
+ if (fp_struct_orig) fp_struct_orig((struct recursive_struct_test *)0);
+}
+
+/* Recursive typedef canonicalization validation - MUST have identical type IDs after full canonicalization */
+
+/* u8 recursive chain - all should resolve to unsigned char */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u8_recursive_chain\n\t\.set\t__kcfi_typeid_func_u8_recursive_chain, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u8_recursive_mid\n\t\.set\t__kcfi_typeid_func_u8_recursive_mid, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u8_recursive_base\n\t\.set\t__kcfi_typeid_func_u8_recursive_base, 0x54e5c0c4} } } */
+
+/* u32 recursive chain - all should resolve to unsigned int */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u32_recursive_chain\n\t\.set\t__kcfi_typeid_func_u32_recursive_chain, 0x08e0cbf2} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u32_recursive_mid\n\t\.set\t__kcfi_typeid_func_u32_recursive_mid, 0x08e0cbf2} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_u32_recursive_base\n\t\.set\t__kcfi_typeid_func_u32_recursive_base, 0x08e0cbf2} } } */
+
+/* Three-level u8 recursive chain - all should resolve to unsigned char */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_three_level_recursive\n\t\.set\t__kcfi_typeid_func_three_level_recursive, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_three_level_mid\n\t\.set\t__kcfi_typeid_func_three_level_mid, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_three_level_base\n\t\.set\t__kcfi_typeid_func_three_level_base, 0x54e5c0c4} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_three_level_final\n\t\.set\t__kcfi_typeid_func_three_level_final, 0x54e5c0c4} } } */
+
+/* Struct recursive chain - all should resolve to same struct name */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_recursive_chain\n\t\.set\t__kcfi_typeid_func_struct_recursive_chain, 0x2e72122c} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_recursive_mid\n\t\.set\t__kcfi_typeid_func_struct_recursive_mid, 0x2e72122c} } } */
+/* { dg-final { scan-assembler {\t\.weak\t__kcfi_typeid_func_struct_recursive_original\n\t\.set\t__kcfi_typeid_func_struct_recursive_original, 0x2e72122c} } } */
+
+/* Update counts to include recursive typedef tests */
+/* Note: u8/unsigned char recursive tests add 7 more occurrences (actual count: 9) */
+/* { dg-final { scan-assembler-times {0x54e5c0c4} 9 } } */
+
+/* Note: u32/unsigned int recursive tests add 3 more occurrences (actual count: 6) */
+/* { dg-final { scan-assembler-times {0x08e0cbf2} 6 } } */
+
+/* Verify struct recursive typedef canonicalization types */
+/* { dg-final { scan-tree-dump {mangled='_ZTSPFvP21recursive_struct_testE' typeid=0x2e72122c} kcfi0 } } */
+
+/* Struct recursive: 3 identical type IDs */
+/* { dg-final { scan-assembler-times {0x2e72122c} 3 } } */
+
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi.exp b/gcc/testsuite/gcc.dg/kcfi/kcfi.exp
new file mode 100644
index 000000000000..2aebcbe1c01b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi.exp
@@ -0,0 +1,36 @@
+# Copyright (C) 2025 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>.
+
+# GCC testsuite for KCFI (Kernel Control Flow Integrity) tests.
+
+# Load support procs.
+load_lib gcc-dg.exp
+
+# If a testcase doesn't have special options, use these.
+global DEFAULT_CFLAGS
+if ![info exists DEFAULT_CFLAGS] then {
+ set DEFAULT_CFLAGS ""
+}
+
+# Initialize `dg'.
+dg-init
+
+# Main loop.
+dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] \
+ "" $DEFAULT_CFLAGS
+
+# All done.
+dg-finish
--
2.34.1
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
[not found] ` <CALvbMcAPV1eB6nocPAS=qR8SCiQyU43v911R8S7Ah_=G7yK-+g@mail.gmail.com>
@ 2025-08-21 8:29 ` Andrew Pinski
2025-08-21 16:16 ` Kees Cook
1 sibling, 0 replies; 42+ messages in thread
From: Andrew Pinski @ 2025-08-21 8:29 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 12:59 AM Andrew Pinski
<andrew.pinski@oss.qualcomm.com> wrote:
>
> On Thu, Aug 21, 2025 at 12:41 AM Kees Cook <kees@kernel.org> wrote:
>>
>> To support the KCFI type-id which needs to convert unique function
>> prototypes into unique 32-bit values, add a subset of the Itanium C++
>> mangling ABI for C typeinfo of function prototypes. This gets us to the
>> first step: a string representation of the function prototype.
>
>
> Can you explain why this is needed? Also it seems like the code is very sensitive to buffer overflows. Especially the way it is currently written.
> The C++ front-end version uses obstack_grow to overcome the buffer overflow issue.
So looking at where this is only used; You should calculate the hash
directly instead.
So basically places where you do *(p++) = 'x'; replace it with doing
the hash calculation.
Add a few functions like:
hash add_hash (hash, char);
hash add_hash (hash, char*);
hash add_hash_int (hash, int);
Etc.
Thanks,
Andrew
>
>>
>>
>> Trying to extract only the C portions of the gcc/cp/mangle.cc code
>> seemed infeasible after a few attempts. So this is the minimal subset
>> of the mangling ABI needed to generate unique KCFI type ids.
>>
>> I could not find a way to build a sensible selftest infrastructure for
>> this code. I wanted to do something like this:
>>
>> #ifdef CHECKING_P
>> const char code[] = "
>> typedef struct { int x, y } xy_t;
>> extern int func(xy_t *p);
>> ";
>>
>> ASSERT_MANGLE (code, "_ZTSPFiP4xy_tE");
>> ...
>> #endif
>>
>> But I could not find any way to build a localized parser that could
>> parse the "code" string from which I could extract the "func" fndecl.
>> It would have been so much nicer to build the selftest directly into
>> mangle.cc here, but I couldn't figure it out. Instead, later patches
>> create a "kcfi" dump file, and the large kcfi testsuite validates
>> expected mangle strings as part of the type-id validation.
>>
>> Signed-off-by: Kees Cook <kees@kernel.org>
>> ---
>> gcc/Makefile.in | 1 +
>> gcc/mangle.h | 29 +++
>> gcc/selftest.h | 1 +
>> gcc/mangle.cc | 548 ++++++++++++++++++++++++++++++++++++++++++++++++
>> 4 files changed, 579 insertions(+)
>> create mode 100644 gcc/mangle.h
>> create mode 100644 gcc/mangle.cc
>>
>> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
>> index d7d5cbe72770..86f62611c1d4 100644
>> --- a/gcc/Makefile.in
>> +++ b/gcc/Makefile.in
>> @@ -1619,6 +1619,7 @@ OBJS = \
>> lto-section-out.o \
>> lto-opts.o \
>> lto-compress.o \
>> + mangle.o \
>> mcf.o \
>> mode-switching.o \
>> modulo-sched.o \
>> diff --git a/gcc/mangle.h b/gcc/mangle.h
>> new file mode 100644
>> index 000000000000..94521e1e7e5c
>> --- /dev/null
>> +++ b/gcc/mangle.h
>> @@ -0,0 +1,29 @@
>> +/* Itanium C++ ABI type mangling for GCC.
>> + Copyright (C) 2025 Free Software Foundation, Inc.
>> +
>> +This file is part of GCC.
>> +
>> +GCC is free software; you can redistribute it and/or modify it under
>> +the terms of the GNU General Public License as published by the Free
>> +Software Foundation; either version 3, or (at your option) any later
>> +version.
>> +
>> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +for more details.
>> +
>> +You should have received a copy of the GNU General Public License
>> +along with GCC; see the file COPYING3. If not see
>> +<http://www.gnu.org/licenses/>. */
>> +
>> +#ifndef GCC_MANGLE_H
>> +#define GCC_MANGLE_H
>> +
>> +#include "tree.h"
>> +
>> +/* Function type mangling following Itanium C++ ABI conventions.
>> + Returns a static buffer containing the mangled type string. */
>> +extern const char *mangle_function_type (tree fntype_or_fndecl);
>> +
>> +#endif /* GCC_MANGLE_H */
>> diff --git a/gcc/mangle.cc b/gcc/mangle.cc
>> new file mode 100644
>> index 000000000000..830985251c81
>> --- /dev/null
>> +++ b/gcc/mangle.cc
>> @@ -0,0 +1,548 @@
>> +/* Itanium C++ ABI type mangling for GCC.
>> + Copyright (C) 2025 Free Software Foundation, Inc.
>> +
>> +This file is part of GCC.
>> +
>> +GCC is free software; you can redistribute it and/or modify it under
>> +the terms of the GNU General Public License as published by the Free
>> +Software Foundation; either version 3, or (at your option) any later
>> +version.
>> +
>> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +for more details.
>> +
>> +You should have received a copy of the GNU General Public License
>> +along with GCC; see the file COPYING3. If not see
>> +<http://www.gnu.org/licenses/>. */
>> +
>> +#include "config.h"
>> +#include "system.h"
>> +#include "coretypes.h"
>> +#include "tree.h"
>> +#include "diagnostic-core.h"
>> +#include "stringpool.h"
>> +#include "stor-layout.h"
>> +#include "mangle.h"
>> +#include "selftest.h"
>> +
>> +/* Forward declaration for recursive type mangling. */
>> +static void mangle_type_to_buffer (tree type, char **p, char *end);
>> +
>> +/* Mangle a builtin type following Itanium C++ ABI for C types. */
>> +static void
>> +mangle_builtin_type_to_buffer (tree type, char **p, char *end)
>> +{
>> + gcc_assert (type != NULL_TREE);
>> + gcc_assert (p != NULL && *p != NULL && end != NULL);
>> + gcc_assert (*p < end);
>> +
>> + if (*p >= end)
>> + return;
>> +
>> + switch (TREE_CODE (type))
>> + {
>> + case VOID_TYPE:
>> + **p = 'v';
>> + (*p)++;
>> + break;
>> +
>> + case BOOLEAN_TYPE:
>> + **p = 'b';
>> + (*p)++;
>
>
> I am not 100% sure this is always correct. because there could be a boolean type with a precision non 1.
>
>>
>> + break;
>> +
>> + case INTEGER_TYPE:
>> + /* Handle standard integer types using Itanium ABI codes. */
>> + if (type == char_type_node)
>> + {
>> + **p = 'c';
>> + (*p)++;
>> + }
>> + else if (type == signed_char_type_node)
>> + {
>> + **p = 'a';
>> + (*p)++;
>> + }
>> + else if (type == unsigned_char_type_node)
>> + {
>> + **p = 'h';
>> + (*p)++;
>> + }
>> + else if (type == short_integer_type_node)
>> + {
>> + **p = 's';
>> + (*p)++;
>> + }
>> + else if (type == short_unsigned_type_node)
>> + {
>> + **p = 't';
>> + (*p)++;
>> + }
>> + else if (type == integer_type_node)
>> + {
>> + **p = 'i';
>> + (*p)++;
>> + }
>> + else if (type == unsigned_type_node)
>> + {
>> + **p = 'j';
>> + (*p)++;
>> + }
>> + else if (type == long_integer_type_node)
>> + {
>> + **p = 'l';
>> + (*p)++;
>> + }
>> + else if (type == long_unsigned_type_node)
>> + {
>> + **p = 'm';
>> + (*p)++;
>> + }
>> + else if (type == long_long_integer_type_node)
>> + {
>> + **p = 'x';
>> + (*p)++;
>> + }
>> + else if (type == long_long_unsigned_type_node)
>> + {
>> + **p = 'y';
>> + (*p)++;
>> + }
>> + else
>> + {
>> + /* Fallback for other integer types - use precision-based encoding. */
>> + *p += snprintf (*p, end - *p, "i%d", TYPE_PRECISION (type));
>> + }
>> + break;
>> +
>> + case REAL_TYPE:
>> + if (type == float_type_node)
>> + {
>> + **p = 'f';
>> + (*p)++;
>> + }
>> + else if (type == double_type_node)
>> + {
>> + **p = 'd';
>> + (*p)++;
>> + }
>> + else if (type == long_double_type_node)
>> + {
>> + **p = 'e';
>> + (*p)++;
>> + }
>> + else
>> + {
>> + /* Fallback for other real types. */
>> + *p += snprintf (*p, end - *p, "f%d", TYPE_PRECISION (type));
>> + }
>> + break;
>
>
> You definitely miss NULLPTR_TYPE.
>
>>
>> +
>> + default:
>> + /* Unknown builtin type - this should never happen in a well-formed C program. */
>> + error ("mangle: Unknown builtin type with %<TREE_CODE%> %d", TREE_CODE (type));
>> + error ("mangle: %<TYPE_MODE%> = %d, %<TYPE_PRECISION%> = %d", TYPE_MODE (type), TYPE_PRECISION (type));
>> + error ("mangle: Please report this as a bug with the above diagnostic information");
>> + gcc_unreachable ();
>
>
> This is wrong way of doing this.
> It should be a sorry or a fatal_error instead. Maybe with an inform beforehand.
> You might also want to use get_tree_code_name, GET_MODE_NAME instead of printing out a number because the number tells nothing.
> Maybe even use print_tree or print_generic_expr to print out the tree instead of manually printing it.
>
>>
>> + }
>> +}
>> +
>> +/* Canonicalize typedef types to their underlying named struct/union types. */
>> +static tree
>> +canonicalize_typedef_type (tree type)
>> +{
>> + /* Handle typedef types - canonicalize to named structs when possible. */
>> + if (TYPE_NAME (type) && TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
>> + {
>> + tree type_decl = TYPE_NAME (type);
>> +
>> + /* Check if this is a typedef (not the original struct declaration) */
>> + if (DECL_ORIGINAL_TYPE (type_decl))
>> + {
>> + tree original_type = DECL_ORIGINAL_TYPE (type_decl);
>> +
>> + /* If the original type is a named struct/union/enum, use that instead. */
>> + if ((TREE_CODE (original_type) == RECORD_TYPE
>> + || TREE_CODE (original_type) == UNION_TYPE
>> + || TREE_CODE (original_type) == ENUMERAL_TYPE)
>> + && TYPE_NAME (original_type)
>> + && ((TREE_CODE (TYPE_NAME (original_type)) == TYPE_DECL
>> + && DECL_NAME (TYPE_NAME (original_type)))
>> + || TREE_CODE (TYPE_NAME (original_type)) == IDENTIFIER_NODE))
>> + {
>> + /* Recursively canonicalize in case the original type is also a typedef. */
>> + return canonicalize_typedef_type (original_type);
>> + }
>> +
>> + /* For basic type typedefs (e.g., u8 -> unsigned char), canonicalize to original type. */
>> + if (TREE_CODE (original_type) == INTEGER_TYPE
>> + || TREE_CODE (original_type) == REAL_TYPE
>> + || TREE_CODE (original_type) == POINTER_TYPE
>> + || TREE_CODE (original_type) == ARRAY_TYPE
>> + || TREE_CODE (original_type) == FUNCTION_TYPE
>> + || TREE_CODE (original_type) == METHOD_TYPE
>> + || TREE_CODE (original_type) == BOOLEAN_TYPE
>> + || TREE_CODE (original_type) == COMPLEX_TYPE
>> + || TREE_CODE (original_type) == VECTOR_TYPE)
>> + {
>> + /* Recursively canonicalize in case the original type is also a typedef. */
>> + return canonicalize_typedef_type (original_type);
>> + }
>> + }
>> + }
>> +
>> + return type;
>> +}
>> +
>> +/* Recursively mangle a type following Itanium C++ ABI conventions. */
>> +static void
>> +mangle_type_to_buffer (tree type, char **p, char *end)
>> +{
>> + gcc_assert (type != NULL_TREE);
>> + gcc_assert (p != NULL && *p != NULL && end != NULL);
>> + gcc_assert (*p < end);
>> +
>> + if (*p >= end)
>> + return;
>> +
>> + /* Canonicalize typedef types to their underlying named struct types. */
>> + type = canonicalize_typedef_type (type);
>> +
>> + switch (TREE_CODE (type))
>> + {
>> + case POINTER_TYPE:
>> + {
>> + /* Pointer type: 'P' + qualifiers + pointed-to type. */
>> + **p = 'P';
>> + (*p)++;
>> +
>> + /* Add qualifiers to the pointed-to type following Itanium C++ ABI ordering. */
>> + tree pointed_to_type = TREE_TYPE (type);
>> + if (TYPE_QUALS (pointed_to_type) != TYPE_UNQUALIFIED)
>> + {
>> + /* Emit qualifiers in Itanium ABI order: restrict, volatile, const. */
>> + if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_RESTRICT)
>> + {
>> + **p = 'r';
>> + (*p)++;
>> + }
>> + if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_VOLATILE)
>> + {
>> + **p = 'V';
>> + (*p)++;
>> + }
>> + if (TYPE_QUALS (pointed_to_type) & TYPE_QUAL_CONST)
>> + {
>> + **p = 'K';
>> + (*p)++;
>> + }
>> + /* Note: _Atomic is not typically used in kernel code. */
>> + }
>> +
>> + /* For KCFI's hybrid type system: preserve typedef names for compound types,
>> + but use canonical forms for primitive types. */
>> + tree target_type;
>> + if (TREE_CODE (pointed_to_type) == RECORD_TYPE
>> + || TREE_CODE (pointed_to_type) == UNION_TYPE
>> + || TREE_CODE (pointed_to_type) == ENUMERAL_TYPE)
>> + {
>> + /* Compound type: preserve typedef information by using original type. */
>> + target_type = pointed_to_type;
>> + }
>> + else
>> + {
>> + /* Primitive type: use canonical form to ensure structural typing. */
>> + target_type = TYPE_MAIN_VARIANT (pointed_to_type);
>> + }
>> + mangle_type_to_buffer (target_type, p, end);
>> + break;
>> + }
>> +
>> + case ARRAY_TYPE:
>> + /* Array type: 'A' + size + '_' + element type (simplified). */
>> + **p = 'A';
>> + (*p)++;
>> + if (TYPE_DOMAIN (type) && TYPE_MAX_VALUE (TYPE_DOMAIN (type)))
>> + {
>> + HOST_WIDE_INT size = tree_to_shwi (TYPE_MAX_VALUE (TYPE_DOMAIN (type))) + 1;
>
>
> No check to make sure array type is not a VLA. So this will ICE.
>
>>
>> + *p += snprintf (*p, end - *p, "%ld_", (long) size);
>> + }
>> + else
>> + {
>> + **p = '_';
>> + (*p)++;
>> + }
>> + mangle_type_to_buffer (TREE_TYPE (type), p, end);
>> + break;
>> +
>> + case FUNCTION_TYPE:
>> + {
>> + /* Function type: 'F' + return type + parameter types + 'E' */
>> + **p = 'F';
>> + (*p)++;
>> + mangle_type_to_buffer (TREE_TYPE (type), p, end);
>> +
>> + /* Add parameter types. */
>> + tree param_types = TYPE_ARG_TYPES (type);
>> +
>> + if (param_types == NULL_TREE)
>> + {
>> + /* func() - variadic function, no parameter list.
>> + Don't mangle any parameters. */
>> + }
>> + else
>> + {
>> + bool found_real_params = false;
>> + for (tree param = param_types; param && *p < end; param = TREE_CHAIN (param))
>> + {
>> + tree param_type = TREE_VALUE (param);
>> + if (param_type == void_type_node)
>> + {
>> + /* Check if this is the first parameter (explicit void) or a sentinel */
>> + if (!found_real_params)
>> + {
>> + /* func(void) - explicit empty parameter list.
>> + Mangle void to distinguish from variadic func(). */
>> + mangle_type_to_buffer (void_type_node, p, end);
>> + }
>> + /* If we found real params before this void, it's a sentinel - stop */
>> + break;
>> + }
>> +
>> + found_real_params = true;
>> +
>> + /* For value parameters, ignore const/volatile qualifiers as they
>> + don't affect the calling convention. const int and int are
>> + passed identically by value. */
>> + tree canonical_param_type = param_type;
>> + if (TREE_CODE (param_type) != POINTER_TYPE
>> + && TREE_CODE (param_type) != REFERENCE_TYPE
>> + && TREE_CODE (param_type) != ARRAY_TYPE)
>> + {
>> + /* Strip qualifiers for non-pointer/reference value parameters. */
>> + canonical_param_type = TYPE_MAIN_VARIANT (param_type);
>> + }
>> +
>> + mangle_type_to_buffer (canonical_param_type, p, end);
>> + }
>> + }
>> +
>> + **p = 'E';
>> + (*p)++;
>> + break;
>> + }
>> +
>> + case RECORD_TYPE:
>> + case UNION_TYPE:
>> + case ENUMERAL_TYPE:
>> + {
>> + /* Struct/union/enum: use simplified representation for C types. */
>> + const char *name = NULL;
>> +
>> + if (TYPE_NAME (type))
>> + {
>> + if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
>> + {
>> + /* TYPE_DECL case: both named structs and typedef structs. */
>> + tree decl_name = DECL_NAME (TYPE_NAME (type));
>> + if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
>> + {
>> + name = IDENTIFIER_POINTER (decl_name);
>> + }
>> + }
>> + else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
>> + {
>> + /* Direct identifier case. */
>> + name = IDENTIFIER_POINTER (TYPE_NAME (type));
>> + }
>> + }
>> +
>> + /* If no name found through normal extraction, handle anonymous types following Itanium C++ ABI. */
>> + if (!name && !TYPE_NAME (type))
>> + {
>> + static char anon_name[128];
>> +
>> + if (TREE_CODE (type) == UNION_TYPE)
>> + {
>> + /* For anonymous unions, try to find first named field (Itanium ABI approach). */
>> + tree field = TYPE_FIELDS (type);
>> + while (field && !DECL_NAME (field))
>> + field = DECL_CHAIN (field);
>> +
>> + if (field && DECL_NAME (field))
>> + {
>> + const char *field_name = IDENTIFIER_POINTER (DECL_NAME (field));
>> + snprintf (anon_name, sizeof(anon_name), "anon_union_by_%s", field_name);
>> + }
>> + else
>> + {
>> + /* No named fields - use Itanium-style Ut encoding. */
>> + snprintf (anon_name, sizeof(anon_name), "Ut_unnamed_union");
>> + }
>> + }
>> + else
>> + {
>> + /* For anonymous structs/enums, use Itanium-style Ut encoding with layout info for discrimination. */
>> + const char *type_prefix = "";
>> + if (TREE_CODE (type) == RECORD_TYPE)
>> + type_prefix = "struct";
>> + else if (TREE_CODE (type) == ENUMERAL_TYPE)
>> + type_prefix = "enum";
>> +
>> + /* Include size and field layout for better discrimination. */
>> + HOST_WIDE_INT size = 0;
>> + if (TYPE_SIZE (type) && tree_fits_shwi_p (TYPE_SIZE (type)))
>> + size = tree_to_shwi (TYPE_SIZE (type));
>> +
>> + /* Generate a hash based on field layout to distinguish same-sized anonymous types. */
>> + unsigned layout_hash = 0;
>> + if (TREE_CODE (type) == RECORD_TYPE)
>> + {
>> + for (tree field = TYPE_FIELDS (type); field; field = DECL_CHAIN (field))
>> + {
>> + if (TREE_CODE (field) == FIELD_DECL)
>> + {
>> + /* Hash field offset and type. */
>> + if (DECL_FIELD_OFFSET (field))
>> + {
>> + HOST_WIDE_INT offset = tree_to_shwi (DECL_FIELD_OFFSET (field));
>> + layout_hash = layout_hash * 31 + (unsigned)offset;
>> + }
>> +
>> + /* Hash field type. */
>> + tree field_type = TREE_TYPE (field);
>> + if (field_type && TYPE_MODE (field_type) != VOIDmode)
>> + layout_hash = layout_hash * 37 + (unsigned)TYPE_MODE (field_type);
>> + }
>> + }
>> + }
>> +
>> + if (layout_hash != 0)
>> + snprintf (anon_name, sizeof(anon_name), "Ut_%s_%ld_%x", type_prefix, (long)size, layout_hash);
>> + else
>> + snprintf (anon_name, sizeof(anon_name), "Ut_%s_%ld", type_prefix, (long)size);
>> + }
>> +
>> + name = anon_name;
>> + }
>> +
>> + if (name)
>> + {
>> + *p += snprintf (*p, end - *p, "%zu%s", strlen (name), name);
>> + }
>> + else
>> + {
>> + /* Always show diagnostic information for missing struct names. */
>> + error ("mangle: No struct/union/enum name found for type code %d (%qs)",
>> + TREE_CODE (type), get_tree_code_name (TREE_CODE (type)));
>> + if (TYPE_NAME (type))
>> + {
>> + error ("mangle: %<TYPE_NAME%> exists but extraction failed");
>> + error ("mangle: %<TYPE_NAME%> tree code = %d", TREE_CODE (TYPE_NAME (type)));
>> + if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
>> + {
>> + tree decl_name = DECL_NAME (TYPE_NAME (type));
>> + error ("mangle: %<TYPE_DECL%> %<DECL_NAME%> = %p", (void*)decl_name);
>> + if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
>> + error ("mangle: %<IDENTIFIER_NODE%> name = '%s'", IDENTIFIER_POINTER (decl_name));
>> + }
>> + else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
>> + {
>> + error ("mangle: %<IDENTIFIER_NODE%> name = '%s'", IDENTIFIER_POINTER (TYPE_NAME (type)));
>> + }
>> + else
>> + {
>> + error ("mangle: Unknown %<TYPE_NAME%> tree code = %d", TREE_CODE (TYPE_NAME (type)));
>> + }
>> + }
>> + else
>> + {
>> + error ("mangle: %<TYPE_NAME%> is NULL - anonymous struct/union/enum detected");
>> + }
>> +
>> + /* This indicates a missing case in our struct name extraction. */
>> + error ("mangle: Please report this as a bug with the above diagnostic information");
>> + gcc_unreachable ();
>
>
> See above about sorry/fatal_error.
>
>>
>> + }
>> + break;
>> + }
>> +
>> + default:
>> + /* Handle builtin types. */
>> + mangle_builtin_type_to_buffer (type, p, end);
>> + break;
>> + }
>> +}
>> +
>> +/* Compute canonical type name using Itanium C++ ABI mangling.
>> + Accepts either FUNCTION_DECL (preferred for typedef preservation) or FUNCTION_TYPE. */
>> +const char *
>> +mangle_function_type (tree fntype_or_fndecl)
>> +{
>> + gcc_assert (fntype_or_fndecl);
>> +
>> + tree fntype = NULL;
>> +
>> + /* Determine input type and extract function type. */
>> + if (TREE_CODE (fntype_or_fndecl) == FUNCTION_TYPE)
>> + {
>> + /* Already FUNCTION_TYPE. */
>> + fntype = fntype_or_fndecl;
>> + }
>> + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
>> + {
>> + tree fndecl = fntype_or_fndecl;
>> + tree base_fntype = TREE_TYPE (fndecl);
>> +
>> + /* For FUNCTION_DECL, build a synthetic function type using DECL_ARGUMENTS
>> + if available to preserve typedef information. */
>
>
> Why do the building? Seems like you could just do that work here. Also doesn't FUNCTION_DECL's type have exactly what you need?
>
>>
>> + tree parm = DECL_ARGUMENTS (fndecl);
>> + if (parm)
>> + {
>> + /* Build parameter type list from DECL_ARGUMENTS. */
>> + tree param_list = NULL_TREE;
>> + tree *param_tail = ¶m_list;
>> +
>> + for (; parm; parm = DECL_CHAIN (parm))
>> + {
>> + tree parm_type = TREE_TYPE (parm);
>> + *param_tail = tree_cons (NULL_TREE, parm_type, NULL_TREE);
>> + param_tail = &TREE_CHAIN (*param_tail);
>> + }
>> +
>> + /* Add void_type_node sentinel if the function takes no parameters. */
>> + if (!param_list)
>> + param_list = tree_cons (NULL_TREE, void_type_node, NULL_TREE);
>> +
>> + /* Build synthetic function type with preserved parameter types. */
>> + fntype = build_function_type (TREE_TYPE (base_fntype), param_list);
>> + }
>> + else
>> + {
>> + /* No DECL_ARGUMENTS - use the standard function type. */
>> + fntype = base_fntype;
>> + }
>> + }
>> + else
>> + {
>> + /* Must only be called with FUNCTION_DECL or FUNCTION_TYPE. */
>> + gcc_unreachable ();
>> + }
>> +
>> + static char name_buf[512];
>
>
> Why use a fixed size buffer on the stack? Why not use obstack or something to that extent. OR even std::string with a pre-allocated estimate.
>
> Thanks,
> Andrew
>
>>
>> + char *p = name_buf;
>> + char *end = name_buf + sizeof (name_buf) - 1;
>> +
>> + /* Typeinfo for a function prototype. */
>> + p += sprintf(name_buf, "_ZTSP");
>> +
>> + /* Use mangle_type_to_buffer for all cases. */
>> + mangle_type_to_buffer (fntype, &p, end);
>> +
>> + /* Ensure we didn't overflow the buffer. */
>> + gcc_assert (p <= end);
>> + *p = '\0';
>> + return name_buf;
>> +}
>> --
>> 2.34.1
>>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
[not found] ` <CALvbMcA+8iHo+zCCvs4UdAg9PVQVtgOno-rtMS4i5YajrjkyGw@mail.gmail.com>
@ 2025-08-21 9:12 ` Peter Zijlstra
2025-08-21 11:01 ` Richard Biener
[not found] ` <CA+=Sn1koTTQaXDnAVWtVU6ACWwhD08NR5nDJO236Pmcoi2X9qA@mail.gmail.com>
2025-08-22 5:10 ` Kees Cook
1 sibling, 2 replies; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-21 9:12 UTC (permalink / raw)
To: Andrew Pinski
Cc: Kees Cook, Qing Zhao, gcc-patches, Joseph Myers, Richard Biener,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> > +/* Compute KCFI type ID for a function declaration or function type
> > (internal) */
> > +static uint32_t
> > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > +{
> > + if (!fntype_or_fndecl)
> > + return 0;
> > +
> > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> >
>
> Now I am curious why this needs to be a mangled function name? Since the
> function in C the symbol is just its name.
> Is there documentation that says the hash needs to be based on all of the
> function arguments types?
The whole point of kCFI is to limit the targets of indirect calls to
functions of the same signature. The actual function name is immaterial.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-21 7:26 ` [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
@ 2025-08-21 9:29 ` Peter Zijlstra
2025-08-21 18:46 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-21 9:29 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 12:26:37AM -0700, Kees Cook wrote:
> Implement x86_64-specific KCFI backend:
>
> - Function preamble generation with type IDs positioned at -(4+prefix_nops)
> offset from function entry point.
>
> - 16-byte alignment of KCFI preambles using calculated prefix NOPs:
> aligned(prefix_nops + 5, 16) to maintain cache lines.
>
> - Type-id hash avoids generating ENDBR instruction in type IDs
> (0xfa1e0ff3/0xfb1e0ff3 are incremented by 1 to prevent execution).
>
> - On-demand scratch register allocation strategy (r11 as needed).
> The clobbers are available both early and late.
>
> - Atomic bundled KCFI check + call/branch sequences using UNSPECV_KCFI
> to prevent optimizer separation and maintain security properties.
>
> - Uses the .kcfi_traps section for debugger/runtime metadata.
>
> Assembly Code Pattern layout required by Linux kernel:
> movl $inverse_type_id, %r10d ; Load expected type (0 - hash)
> addl offset(%target), %r10d ; Add stored type ID from preamble
> je .Lpass ; Branch if types match (sum == 0)
> .Ltrap: ud2 ; Undefined instruction trap on mismatch
> .Lpass: call/jmp *%target ; Execute validated indirect transfer
>
> The initialization of the kcfi callbacks in ix86_option_override()
> seems like a hack. I couldn't find a better place to do this.
>
> Build and run tested on x86_64 Linux kernel with various CPU errata
> handling alternatives and FineIBT.
I'm a little confused, does this force r11 to be the indirect call
register like clang does? The code seems to suggest it is possible it
uses another register.
The current kernel FineIBT code hard assumes r11 for now.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 9:12 ` Peter Zijlstra
@ 2025-08-21 11:01 ` Richard Biener
2025-08-21 14:25 ` Peter Zijlstra
2025-08-21 19:57 ` Kees Cook
[not found] ` <CA+=Sn1koTTQaXDnAVWtVU6ACWwhD08NR5nDJO236Pmcoi2X9qA@mail.gmail.com>
1 sibling, 2 replies; 42+ messages in thread
From: Richard Biener @ 2025-08-21 11:01 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Andrew Pinski, Kees Cook, Qing Zhao, gcc-patches, Joseph Myers,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, 21 Aug 2025, Peter Zijlstra wrote:
> On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
>
> > > +/* Compute KCFI type ID for a function declaration or function type
> > > (internal) */
> > > +static uint32_t
> > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > +{
> > > + if (!fntype_or_fndecl)
> > > + return 0;
> > > +
> > > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > >
> >
> > Now I am curious why this needs to be a mangled function name? Since the
> > function in C the symbol is just its name.
> > Is there documentation that says the hash needs to be based on all of the
> > function arguments types?
>
> The whole point of kCFI is to limit the targets of indirect calls to
> functions of the same signature. The actual function name is immaterial.
What's the attack vector and how does kCFI achieve mitigating it?
Richard.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 11:01 ` Richard Biener
@ 2025-08-21 14:25 ` Peter Zijlstra
2025-08-21 18:09 ` Qing Zhao
2025-08-21 19:57 ` Kees Cook
1 sibling, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-21 14:25 UTC (permalink / raw)
To: Richard Biener
Cc: Andrew Pinski, Kees Cook, Qing Zhao, gcc-patches, Joseph Myers,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 01:01:37PM +0200, Richard Biener wrote:
> On Thu, 21 Aug 2025, Peter Zijlstra wrote:
>
> > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> >
> > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > (internal) */
> > > > +static uint32_t
> > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > +{
> > > > + if (!fntype_or_fndecl)
> > > > + return 0;
> > > > +
> > > > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > >
> > >
> > > Now I am curious why this needs to be a mangled function name? Since the
> > > function in C the symbol is just its name.
> > > Is there documentation that says the hash needs to be based on all of the
> > > function arguments types?
> >
> > The whole point of kCFI is to limit the targets of indirect calls to
> > functions of the same signature. The actual function name is immaterial.
>
> What's the attack vector and how does kCFI achieve mitigating it?
Any of the attacks that can result in scribbling a function pointer.
Typically a buffer overflow I suppose.
The way kCFI works is by changing the indirect call ABI. Traditionally
the indirect call is simply:
load-pointer-into-reg
call *%reg
kCFI changes every function to have a preamble like (with IBT and
retpolines and all the modern crap on):
__cfi_\func:
movl $0x12345678, %eax
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
nop
\func:
endbr64
...
And every indirect call site to:
load-pointer-into-r11
movl $(-0x12345678), %r10d
addl $-15(%r11), %r10d
je 2f
1:ud2
.pushsection .kcfi_traps
.long 1b - .
.popsection
2:cs call __x86_indirect_thunk_r11
Where 0x12345678 is the appropriate signature hash.
So if the loaded pointer is scribbled somehow, the hash embedded at the
callsite no longer matches the hash in the function preamble and #UD.
The kernel is also going to rewrite this if the hardware supports IBT,
into a FineIBT sequence (with too many variants :-/), but I'll spare you
those details.
Those people interested can find it here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/alternative.c#n1298
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
[not found] ` <CALvbMcAPV1eB6nocPAS=qR8SCiQyU43v911R8S7Ah_=G7yK-+g@mail.gmail.com>
2025-08-21 8:29 ` Andrew Pinski
@ 2025-08-21 16:16 ` Kees Cook
2025-08-21 16:24 ` Andrew Pinski
2025-08-21 19:14 ` Qing Zhao
1 sibling, 2 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 16:16 UTC (permalink / raw)
To: Andrew Pinski
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 12:59:06AM -0700, Andrew Pinski wrote:
> On Thu, Aug 21, 2025 at 12:41 AM Kees Cook <kees@kernel.org> wrote:
>
> > To support the KCFI type-id which needs to convert unique function
> > prototypes into unique 32-bit values, add a subset of the Itanium C++
> > mangling ABI for C typeinfo of function prototypes. This gets us to the
> > first step: a string representation of the function prototype.
> >
>
> Can you explain why this is needed? Also it seems like the code is very
As you suggest later, this could just twiddle hash bits instead of
building up a string, but I left this as a C mangler in case it might
be useful in the future, and for debugging. Trying to figure out why
certain prototypes got mismatched was super frustrating, and looking at
a hash value was pretty meaningless. :)
> sensitive to buffer overflows. Especially the way it is currently written.
> The C++ front-end version uses obstack_grow to overcome the buffer overflow
> issue.
Oh nice, yes, obstack looks perfect for this. (Or as you suggest below,
std::string).
If keeping the mangling string output is unwanted, maybe I could use
obstack when the dumpfile is active and report the string+hash at the
end of mangling, but make the actual output be the hash value?
> > + case BOOLEAN_TYPE:
> > + **p = 'b';
> > + (*p)++;
> >
>
> I am not 100% sure this is always correct. because there could be a boolean
> type with a precision non 1.
AIUI, the non-1 case isn't possible on the C end of things and since
this is strictly for C type representation, it's good as-is. But maybe
I'm missing something?
> > + else
> > + {
> > + /* Fallback for other real types. */
> > + *p += snprintf (*p, end - *p, "f%d", TYPE_PRECISION (type));
> > + }
> > + break;
> >
>
> You definitely miss NULLPTR_TYPE.
Isn't nullptr C++-only? I don't think I can create a C type that ends up
getting internally mapped to NULLPTR_TYPE.
> > +
> > + default:
> > + /* Unknown builtin type - this should never happen in a well-formed
> > C program. */
> > + error ("mangle: Unknown builtin type with %<TREE_CODE%> %d",
> > TREE_CODE (type));
> > + error ("mangle: %<TYPE_MODE%> = %d, %<TYPE_PRECISION%> = %d",
> > TYPE_MODE (type), TYPE_PRECISION (type));
> > + error ("mangle: Please report this as a bug with the above
> > diagnostic information");
> > + gcc_unreachable ();
> >
>
> This is wrong way of doing this.
> It should be a sorry or a fatal_error instead. Maybe with an inform
> beforehand.
> You might also want to use get_tree_code_name, GET_MODE_NAME instead of
> printing out a number because the number tells nothing.
> Maybe even use print_tree or print_generic_expr to print out the tree
> instead of manually printing it.
Ah, perfect; thanks!
> > + case ARRAY_TYPE:
> > + /* Array type: 'A' + size + '_' + element type (simplified). */
> > + **p = 'A';
> > + (*p)++;
> > + if (TYPE_DOMAIN (type) && TYPE_MAX_VALUE (TYPE_DOMAIN (type)))
> > + {
> > + HOST_WIDE_INT size = tree_to_shwi (TYPE_MAX_VALUE (TYPE_DOMAIN
> > (type))) + 1;
> >
>
> No check to make sure array type is not a VLA. So this will ICE.
Oops, yes. Thanks!
> > + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
> > + {
> > + tree fndecl = fntype_or_fndecl;
> > + tree base_fntype = TREE_TYPE (fndecl);
> > +
> > + /* For FUNCTION_DECL, build a synthetic function type using
> > DECL_ARGUMENTS
> > + if available to preserve typedef information. */
> >
>
> Why do the building? Seems like you could just do that work here. Also
> doesn't FUNCTION_DECL's type have exactly what you need?
I need the function prototype in three places:
- address-taken extern functions
- function preambles
- indirect call sites
At indirect call sites (during the early GIMPLE pass), I had a
FUNCTION_TYPE available that still had the full typedef information,
and I could use it fine. For the other two, it's later on and the
TREE_TYPE(fndecl)'s FUNCTION_TYPE had lost the typedef information (which
I need to be able to examine in cases where the typedef name was needed
for the mangling vs looking at the underlying types).
> > + tree parm = DECL_ARGUMENTS (fndecl);
> > + if (parm)
> > + {
> > + /* Build parameter type list from DECL_ARGUMENTS. */
> > + tree param_list = NULL_TREE;
> > + tree *param_tail = ¶m_list;
> > +
> > + for (; parm; parm = DECL_CHAIN (parm))
> > + {
> > + tree parm_type = TREE_TYPE (parm);
> > + *param_tail = tree_cons (NULL_TREE, parm_type, NULL_TREE);
> > + param_tail = &TREE_CHAIN (*param_tail);
> > + }
> > +
> > + /* Add void_type_node sentinel if the function takes no
> > parameters. */
> > + if (!param_list)
> > + param_list = tree_cons (NULL_TREE, void_type_node, NULL_TREE);
> > +
> > + /* Build synthetic function type with preserved parameter
> > types. */
> > + fntype = build_function_type (TREE_TYPE (base_fntype),
> > param_list);
> > + }
> > + else
> > + {
> > + /* No DECL_ARGUMENTS - use the standard function type. */
> > + fntype = base_fntype;
> > + }
> > + }
> > + else
> > + {
> > + /* Must only be called with FUNCTION_DECL or FUNCTION_TYPE. */
> > + gcc_unreachable ();
> > + }
Thanks for looking through all this!
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-21 16:16 ` Kees Cook
@ 2025-08-21 16:24 ` Andrew Pinski
2025-08-21 19:14 ` Qing Zhao
1 sibling, 0 replies; 42+ messages in thread
From: Andrew Pinski @ 2025-08-21 16:24 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 9:16 AM Kees Cook <kees@kernel.org> wrote:
>
> On Thu, Aug 21, 2025 at 12:59:06AM -0700, Andrew Pinski wrote:
> > On Thu, Aug 21, 2025 at 12:41 AM Kees Cook <kees@kernel.org> wrote:
> >
> > > To support the KCFI type-id which needs to convert unique function
> > > prototypes into unique 32-bit values, add a subset of the Itanium C++
> > > mangling ABI for C typeinfo of function prototypes. This gets us to the
> > > first step: a string representation of the function prototype.
> > >
> >
> > Can you explain why this is needed? Also it seems like the code is very
>
> As you suggest later, this could just twiddle hash bits instead of
> building up a string, but I left this as a C mangler in case it might
> be useful in the future, and for debugging. Trying to figure out why
> certain prototypes got mismatched was super frustrating, and looking at
> a hash value was pretty meaningless. :)
>
> > sensitive to buffer overflows. Especially the way it is currently written.
> > The C++ front-end version uses obstack_grow to overcome the buffer overflow
> > issue.
>
> Oh nice, yes, obstack looks perfect for this. (Or as you suggest below,
> std::string).
>
> If keeping the mangling string output is unwanted, maybe I could use
> obstack when the dumpfile is active and report the string+hash at the
> end of mangling, but make the actual output be the hash value?
>
> > > + case BOOLEAN_TYPE:
> > > + **p = 'b';
> > > + (*p)++;
> > >
> >
> > I am not 100% sure this is always correct. because there could be a boolean
> > type with a precision non 1.
>
> AIUI, the non-1 case isn't possible on the C end of things and since
> this is strictly for C type representation, it's good as-is. But maybe
> I'm missing something?
>
> > > + else
> > > + {
> > > + /* Fallback for other real types. */
> > > + *p += snprintf (*p, end - *p, "f%d", TYPE_PRECISION (type));
> > > + }
> > > + break;
> > >
> >
> > You definitely miss NULLPTR_TYPE.
>
> Isn't nullptr C++-only? I don't think I can create a C type that ends up
> getting internally mapped to NULLPTR_TYPE.
NO, it is part of C23.
`typeof(nullptr)` in C23 program will have a NULLPTR_TYPE.
I just posted some fixes for the C front-end in the area of
NULLPTR_TYPE which is why I know it exists.
>
> > > +
> > > + default:
> > > + /* Unknown builtin type - this should never happen in a well-formed
> > > C program. */
> > > + error ("mangle: Unknown builtin type with %<TREE_CODE%> %d",
> > > TREE_CODE (type));
> > > + error ("mangle: %<TYPE_MODE%> = %d, %<TYPE_PRECISION%> = %d",
> > > TYPE_MODE (type), TYPE_PRECISION (type));
> > > + error ("mangle: Please report this as a bug with the above
> > > diagnostic information");
> > > + gcc_unreachable ();
> > >
> >
> > This is wrong way of doing this.
> > It should be a sorry or a fatal_error instead. Maybe with an inform
> > beforehand.
> > You might also want to use get_tree_code_name, GET_MODE_NAME instead of
> > printing out a number because the number tells nothing.
> > Maybe even use print_tree or print_generic_expr to print out the tree
> > instead of manually printing it.
>
> Ah, perfect; thanks!
>
> > > + case ARRAY_TYPE:
> > > + /* Array type: 'A' + size + '_' + element type (simplified). */
> > > + **p = 'A';
> > > + (*p)++;
> > > + if (TYPE_DOMAIN (type) && TYPE_MAX_VALUE (TYPE_DOMAIN (type)))
> > > + {
> > > + HOST_WIDE_INT size = tree_to_shwi (TYPE_MAX_VALUE (TYPE_DOMAIN
> > > (type))) + 1;
> > >
> >
> > No check to make sure array type is not a VLA. So this will ICE.
>
> Oops, yes. Thanks!
>
> > > + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
> > > + {
> > > + tree fndecl = fntype_or_fndecl;
> > > + tree base_fntype = TREE_TYPE (fndecl);
> > > +
> > > + /* For FUNCTION_DECL, build a synthetic function type using
> > > DECL_ARGUMENTS
> > > + if available to preserve typedef information. */
> > >
> >
> > Why do the building? Seems like you could just do that work here. Also
> > doesn't FUNCTION_DECL's type have exactly what you need?
>
> I need the function prototype in three places:
>
> - address-taken extern functions
> - function preambles
> - indirect call sites
>
> At indirect call sites (during the early GIMPLE pass), I had a
> FUNCTION_TYPE available that still had the full typedef information,
> and I could use it fine. For the other two, it's later on and the
> TREE_TYPE(fndecl)'s FUNCTION_TYPE had lost the typedef information (which
> I need to be able to examine in cases where the typedef name was needed
> for the mangling vs looking at the underlying types).
What typedef information do you need that is missing? struct names? or
something different?
So from the looks of it there is a big issue with attack vector where
you have 2 different structs with the same name. Seems like the hash
should have included the fields of the struct instead of just a
mangled name of the struct. Can we break the hash for GCC's KFCI from
clang to include this extra hashing info? and maybe even fix up clang
later on?
Thanks,
Andrew Pinski
>
> > > + tree parm = DECL_ARGUMENTS (fndecl);
> > > + if (parm)
> > > + {
> > > + /* Build parameter type list from DECL_ARGUMENTS. */
> > > + tree param_list = NULL_TREE;
> > > + tree *param_tail = ¶m_list;
> > > +
> > > + for (; parm; parm = DECL_CHAIN (parm))
> > > + {
> > > + tree parm_type = TREE_TYPE (parm);
> > > + *param_tail = tree_cons (NULL_TREE, parm_type, NULL_TREE);
> > > + param_tail = &TREE_CHAIN (*param_tail);
> > > + }
> > > +
> > > + /* Add void_type_node sentinel if the function takes no
> > > parameters. */
> > > + if (!param_list)
> > > + param_list = tree_cons (NULL_TREE, void_type_node, NULL_TREE);
> > > +
> > > + /* Build synthetic function type with preserved parameter
> > > types. */
> > > + fntype = build_function_type (TREE_TYPE (base_fntype),
> > > param_list);
> > > + }
> > > + else
> > > + {
> > > + /* No DECL_ARGUMENTS - use the standard function type. */
> > > + fntype = base_fntype;
> > > + }
> > > + }
> > > + else
> > > + {
> > > + /* Must only be called with FUNCTION_DECL or FUNCTION_TYPE. */
> > > + gcc_unreachable ();
> > > + }
>
> Thanks for looking through all this!
>
> -Kees
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 14:25 ` Peter Zijlstra
@ 2025-08-21 18:09 ` Qing Zhao
2025-08-22 5:15 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Qing Zhao @ 2025-08-21 18:09 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Richard Biener, Andrew Pinski, Kees Cook, gcc-patches@gcc.gnu.org,
Joseph Myers, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening@vger.kernel.org
> On Aug 21, 2025, at 10:25, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Thu, Aug 21, 2025 at 01:01:37PM +0200, Richard Biener wrote:
>> On Thu, 21 Aug 2025, Peter Zijlstra wrote:
>>
>>> On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
>>>
>>>>> +/* Compute KCFI type ID for a function declaration or function type
>>>>> (internal) */
>>>>> +static uint32_t
>>>>> +compute_kcfi_type_id (tree fntype_or_fndecl)
>>>>> +{
>>>>> + if (!fntype_or_fndecl)
>>>>> + return 0;
>>>>> +
>>>>> + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
>>>>> + uint32_t base_type_id = kcfi_hash_string (canonical_name);
>>>>>
>>>>
>>>> Now I am curious why this needs to be a mangled function name? Since the
>>>> function in C the symbol is just its name.
>>>> Is there documentation that says the hash needs to be based on all of the
>>>> function arguments types?
>>>
>>> The whole point of kCFI is to limit the targets of indirect calls to
>>> functions of the same signature. The actual function name is immaterial.
>>
>> What's the attack vector and how does kCFI achieve mitigating it?
>
> Any of the attacks that can result in scribbling a function pointer.
> Typically a buffer overflow I suppose.
>
>
> The way kCFI works is by changing the indirect call ABI. Traditionally
> the indirect call is simply:
>
> load-pointer-into-reg
> call *%reg
>
> kCFI changes every function to have a preamble like (with IBT and
> retpolines and all the modern crap on):
Does “every function” mean all the function in the compilation? Not only the function whose address is taken?
Qing
>
> __cfi_\func:
> movl $0x12345678, %eax
> nop
> nop
> nop
> nop
> nop
> nop
> nop
> nop
> nop
> nop
> nop
> \func:
> endbr64
> ...
>
> And every indirect call site to:
>
> load-pointer-into-r11
> movl $(-0x12345678), %r10d
> addl $-15(%r11), %r10d
> je 2f
> 1:ud2
> .pushsection .kcfi_traps
> .long 1b - .
> .popsection
> 2:cs call __x86_indirect_thunk_r11
>
> Where 0x12345678 is the appropriate signature hash.
>
>
> So if the loaded pointer is scribbled somehow, the hash embedded at the
> callsite no longer matches the hash in the function preamble and #UD.
>
> The kernel is also going to rewrite this if the hardware supports IBT,
> into a FineIBT sequence (with too many variants :-/), but I'll spare you
> those details.
>
> Those people interested can find it here:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kernel/alternative.c#n1298
>
>
>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-21 9:29 ` Peter Zijlstra
@ 2025-08-21 18:46 ` Kees Cook
2025-08-21 19:03 ` Kees Cook
2025-08-22 8:19 ` Peter Zijlstra
0 siblings, 2 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 18:46 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 11:29:35AM +0200, Peter Zijlstra wrote:
> On Thu, Aug 21, 2025 at 12:26:37AM -0700, Kees Cook wrote:
> > Implement x86_64-specific KCFI backend:
> >
> > - Function preamble generation with type IDs positioned at -(4+prefix_nops)
> > offset from function entry point.
> >
> > - 16-byte alignment of KCFI preambles using calculated prefix NOPs:
> > aligned(prefix_nops + 5, 16) to maintain cache lines.
> >
> > - Type-id hash avoids generating ENDBR instruction in type IDs
> > (0xfa1e0ff3/0xfb1e0ff3 are incremented by 1 to prevent execution).
> >
> > - On-demand scratch register allocation strategy (r11 as needed).
> > The clobbers are available both early and late.
> >
> > - Atomic bundled KCFI check + call/branch sequences using UNSPECV_KCFI
> > to prevent optimizer separation and maintain security properties.
> >
> > - Uses the .kcfi_traps section for debugger/runtime metadata.
> >
> > Assembly Code Pattern layout required by Linux kernel:
> > movl $inverse_type_id, %r10d ; Load expected type (0 - hash)
> > addl offset(%target), %r10d ; Add stored type ID from preamble
> > je .Lpass ; Branch if types match (sum == 0)
> > .Ltrap: ud2 ; Undefined instruction trap on mismatch
> > .Lpass: call/jmp *%target ; Execute validated indirect transfer
> >
> > The initialization of the kcfi callbacks in ix86_option_override()
> > seems like a hack. I couldn't find a better place to do this.
> >
> > Build and run tested on x86_64 Linux kernel with various CPU errata
> > handling alternatives and FineIBT.
>
> I'm a little confused, does this force r11 to be the indirect call
> register like clang does? The code seems to suggest it is possible it
> uses another register.
>
> The current kernel FineIBT code hard assumes r11 for now.
Oh, it looked like it wasn't always r11. Does clang force the call
register to be r11? I only do that here if the call expression isn't a
register (similar to -mindirect-branch-register). Looking at the retpoline
implementation, I see __x86_indirect_thunk_* being generated for all the
general registers. Hm, but in looking now I see all the hard-coded r11 use
in the fineibt alternatives. I wonder if my boot testing is somehow not
triggering the FineIBT alternatives patching? I will investigate more...
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-21 18:46 ` Kees Cook
@ 2025-08-21 19:03 ` Kees Cook
2025-08-22 8:19 ` Peter Zijlstra
1 sibling, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-21 19:03 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 11:46:17AM -0700, Kees Cook wrote:
> On Thu, Aug 21, 2025 at 11:29:35AM +0200, Peter Zijlstra wrote:
> > On Thu, Aug 21, 2025 at 12:26:37AM -0700, Kees Cook wrote:
> > > Build and run tested on x86_64 Linux kernel with various CPU errata
> > > handling alternatives and FineIBT.
Turns out my configs were broken -- I only tested non-retpoline.
> >
> > I'm a little confused, does this force r11 to be the indirect call
> > register like clang does? The code seems to suggest it is possible it
> > uses another register.
> >
> > The current kernel FineIBT code hard assumes r11 for now.
>
> Oh, it looked like it wasn't always r11. Does clang force the call
> register to be r11? I only do that here if the call expression isn't a
> register (similar to -mindirect-branch-register). Looking at the retpoline
> implementation, I see __x86_indirect_thunk_* being generated for all the
> general registers. Hm, but in looking now I see all the hard-coded r11 use
> in the fineibt alternatives. I wonder if my boot testing is somehow not
> triggering the FineIBT alternatives patching? I will investigate more...
I've found my Kconfig problem now. Confirmed that this RFC does _not_
work with retpoline (much less FineIBT). I will get that fixed for the
next version.
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-21 16:16 ` Kees Cook
2025-08-21 16:24 ` Andrew Pinski
@ 2025-08-21 19:14 ` Qing Zhao
2025-08-21 21:29 ` Kees Cook
1 sibling, 1 reply; 42+ messages in thread
From: Qing Zhao @ 2025-08-21 19:14 UTC (permalink / raw)
To: Kees Cook
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
> On Aug 21, 2025, at 12:16, Kees Cook <kees@kernel.org> wrote:
>
>
>>> + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
>>> + {
>>> + tree fndecl = fntype_or_fndecl;
>>> + tree base_fntype = TREE_TYPE (fndecl);
>>> +
>>> + /* For FUNCTION_DECL, build a synthetic function type using
>>> DECL_ARGUMENTS
>>> + if available to preserve typedef information. */
>>>
>>
>> Why do the building? Seems like you could just do that work here. Also
>> doesn't FUNCTION_DECL's type have exactly what you need?
>
> I need the function prototype in three places:
>
> - address-taken extern functions
> - function preambles
> - indirect call sites
>
A little confused with the above:
From my understanding,
1. At each indirect call sites, we should generate the checking code to
A. load the hashed precomputed typeid from the callee’s preamble
B. compare it with the precomputed typeid for this call site
So, we need the function prototype of the indirect call site to compute the typeid for this call site.
2. For every “address-taken” function, we should generate the function
preamble, in which the precomputed typeid for this function is stored.
So, we need the function prototype of this function to compute the typeid for this function.
The above 2 should cover all the KCFI ABIs.
What I was confused is, why “address-taken external function” and “function preambles” are separated items?
For the function preambles, shall we generate for all the functions? Or only for address-taken functions in
the compilation?
> At indirect call sites (during the early GIMPLE pass), I had a
> FUNCTION_TYPE available that still had the full typedef information,
> and I could use it fine.
> For the other two, it's later on and the
> TREE_TYPE(fndecl)'s FUNCTION_TYPE had lost the typedef information (which
> I need to be able to examine in cases where the typedef name was needed
> for the mangling vs looking at the underlying types).
Then why not also compute the typeid for the function preamble during early GIMPLE phase
the same as the indirect call sites when all the typedef information is available?
Qing
>
>>> + tree parm = DECL_ARGUMENTS (fndecl);
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 11:01 ` Richard Biener
2025-08-21 14:25 ` Peter Zijlstra
@ 2025-08-21 19:57 ` Kees Cook
2025-08-22 6:53 ` Richard Biener
1 sibling, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-21 19:57 UTC (permalink / raw)
To: Richard Biener
Cc: Peter Zijlstra, Andrew Pinski, Qing Zhao, gcc-patches,
Joseph Myers, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 01:01:37PM +0200, Richard Biener wrote:
> On Thu, 21 Aug 2025, Peter Zijlstra wrote:
>
> > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> >
> > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > (internal) */
> > > > +static uint32_t
> > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > +{
> > > > + if (!fntype_or_fndecl)
> > > > + return 0;
> > > > +
> > > > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > >
> > >
> > > Now I am curious why this needs to be a mangled function name? Since the
> > > function in C the symbol is just its name.
> > > Is there documentation that says the hash needs to be based on all of the
> > > function arguments types?
> >
> > The whole point of kCFI is to limit the targets of indirect calls to
> > functions of the same signature. The actual function name is immaterial.
>
> What's the attack vector and how does kCFI achieve mitigating it?
To add some more detail to Peter's answer, the attack vector is dealing
with stored function pointers (generally on the heap, but potentially
also the stack), that can be changed by attacker-controlled writes (via
buffer overflows or use-after-free writes). The idea being that instead
of being able to call anywhere through an indirect call site that uses a
manipulated pointer, now the attacker is limited to a subset of "matching"
destinations. (KCFI depends on a system already enforcing W^X memory in
the sense that if an attacker can construct an executable region, they
could write whatever hash they want into in addition to whatever code.)
The general CFI ideas for this are discussed here, but focuses more on a
CFG analysis to construct valid call destinations, which tends to require
LTO, etc:
https://users.soe.ucsc.edu/~abadi/Papers/cfi-tissec-revised.pdf
Later refinement for using jump tables (constructed via CFG analysis
during LTO) was proposed here:
https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-tice.pdf
Linux used the above implementation from 2018 to 2022:
https://android-developers.googleblog.com/2018/10/control-flow-integrity-in-android-kernel.html
but the corner cases for target addresses not being the actual functions
(i.e. pointing into the jump table) was a continual source of problems,
and generating the jump tables required full LTO, which had its own set
of problems.
Looking at function prototypes as the source of call validity was
presented here, though still relied on LTO:
https://www.blackhat.com/docs/asia-17/materials/asia-17-Moreira-Drop-The-Rop-Fine-Grained-Control-Flow-Integrity-For-The-Linux-Kernel-wp.pdf
The KCFI approach built on the function-prototype idea, but avoided
needing LTO, and could be further updated to deal with CPU errata
(retpolines, etc):
https://lpc.events/event/16/contributions/1315/
This has been working very well now for 3 years, but has been limited to
only Linux built with Clang (e.g. Linux kernel CFI, first LLVM-CFI and now
KCFI, has been deployed on all Android phones and all Chrome OS devices
since 2018.)
I have been trying to find someone to work on KCFI in GCC since 2018,
(see my annual arm-waving at the Linux Plumbers conference Toolchain
track) and other than Dan Li who took a stab at it for aarch64 in 2023,
no one has stepped up to do it, so I thought I'd finally try tackling
it. :)
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-21 19:14 ` Qing Zhao
@ 2025-08-21 21:29 ` Kees Cook
2025-08-22 15:11 ` Qing Zhao
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-21 21:29 UTC (permalink / raw)
To: Qing Zhao
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
On Thu, Aug 21, 2025 at 07:14:31PM +0000, Qing Zhao wrote:
>
>
> > On Aug 21, 2025, at 12:16, Kees Cook <kees@kernel.org> wrote:
> >
> >
> >>> + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
> >>> + {
> >>> + tree fndecl = fntype_or_fndecl;
> >>> + tree base_fntype = TREE_TYPE (fndecl);
> >>> +
> >>> + /* For FUNCTION_DECL, build a synthetic function type using
> >>> DECL_ARGUMENTS
> >>> + if available to preserve typedef information. */
> >>>
> >>
> >> Why do the building? Seems like you could just do that work here. Also
> >> doesn't FUNCTION_DECL's type have exactly what you need?
> >
> > I need the function prototype in three places:
> >
> > - address-taken extern functions
> > - function preambles
> > - indirect call sites
> >
>
> A little confused with the above:
>
> From my understanding,
>
> 1. At each indirect call sites, we should generate the checking code to
> A. load the hashed precomputed typeid from the callee’s preamble
> B. compare it with the precomputed typeid for this call site
>
> So, we need the function prototype of the indirect call site to compute the typeid for this call site.
Correct.
> 2. For every “address-taken” function, we should generate the function
> preamble, in which the precomputed typeid for this function is stored.
>
> So, we need the function prototype of this function to compute the typeid for this function.
>
> The above 2 should cover all the KCFI ABIs.
For non-static functions, we cannot know if other compilation units may
make indirect calls to a given function, so those functions must always
have their kcfi preamble added. For static functions, if they are
address-taken by the current compilation unit, then they must get a kcfi
preamble added.
> What I was confused is, why “address-taken external function” and “function preambles” are separated items?
> For the function preambles, shall we generate for all the functions? Or only for address-taken functions in
> the compilation?
The other case is emitting the __ckfi_typeid_FUNC weak symbols, which is
used for link-time resolution with non-C code (i.e. raw .S assembly)
which doesn't have access to the C type system to calculate the hashes
on its own, and needs to have a way to build its own kcfi preambles. This
is how Linux constructs its assembly function entry points:
#ifndef __CFI_TYPE
#define __CFI_TYPE(name) \
.4byte __kcfi_typeid_##name
#endif
#define SYM_TYPED_ENTRY(name, linkage, align...) \
linkage(name) ASM_NL \
align ASM_NL \
__CFI_TYPE(name) ASM_NL \
name:
That way all the asm functions can be be indirect call targets without
knowing the hash value (which will be filled in at link time).
> > At indirect call sites (during the early GIMPLE pass), I had a
> > FUNCTION_TYPE available that still had the full typedef information,
> > and I could use it fine.
>
> > For the other two, it's later on and the
> > TREE_TYPE(fndecl)'s FUNCTION_TYPE had lost the typedef information (which
> > I need to be able to examine in cases where the typedef name was needed
> > for the mangling vs looking at the underlying types).
>
> Then why not also compute the typeid for the function preamble during early GIMPLE phase
> the same as the indirect call sites when all the typedef information is available?
I assume I just didn't see how yet. :) I wasn't able to identify nor
store the typeid for function definitions that ultimately end up getting
.s file output. For example, down in ix86_asm_output_function_label(),
I have the decl (but it's way late):
ix86_asm_output_function_label (FILE *out_file, const char *fname,
tree decl)
I couldn't figure out how to find these during the GIMPLE pass. Oh,
perhaps I can do this with an IPA pass? That should let me walk all
functions including externs. I'll give it a try...
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
[not found] ` <CALvbMcA+8iHo+zCCvs4UdAg9PVQVtgOno-rtMS4i5YajrjkyGw@mail.gmail.com>
2025-08-21 9:12 ` Peter Zijlstra
@ 2025-08-22 5:10 ` Kees Cook
2025-08-22 5:27 ` Andrew Pinski
1 sibling, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-22 5:10 UTC (permalink / raw)
To: Andrew Pinski
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Peter Zijlstra, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> On Thu, Aug 21, 2025 at 12:47 AM Kees Cook <kees@kernel.org> wrote:
> > +struct kcfi_target_hooks {
> > + /* Apply architecture-specific masking to type ID. */
> > + uint32_t (*mask_type_id) (uint32_t type_id);
> > +
> > + /* Generate bundled KCFI checked call (atomic check + call to prevent
> > optimizer separation) */
> > + rtx (*gen_kcfi_checked_call) (rtx call_insn, rtx target_reg, uint32_t
> > expected_type, HOST_WIDE_INT prefix_nops);
> > +
> > + /* Add architecture-specific register clobbers for KCFI calls. */
> > + void (*add_kcfi_clobbers) (rtx_insn *call_insn);
> > +
> > + /* Calculate architecture-specific prefix NOPs count (optional, returns
> > prefix_nops unchanged if NULL) */
> > + int (*calculate_prefix_nops) (HOST_WIDE_INT prefix_nops);
> > +
> > + /* Emit architecture-specific type ID instruction (required for common
> > preamble helper) */
> > + void (*emit_type_id_instruction) (FILE *file, uint32_t type_id);
> > +};
> >
>
> This should really be part of target.def so you also have the internals
> documentation done for these hooks.
> And then you could just use targetm.kcfi.xyz below.
> And for the hooks you would have TARGET_KCFI_XYZ being defined like the
> others one.
Ah-ha! Thank you. The macros to build that eluded me. I've updated to
this now.
> > +/* Pass creation functions. */
> > +class gimple_opt_pass;
> > +class rtl_opt_pass;
> > +namespace gcc { class context; }
> >
>
> This above is definitely bad form.
Ah yeah, sorry; this was a leftover. (And below.)
> > +
> > +extern gimple_opt_pass *make_pass_kcfi (gcc::context *ctxt);
> > +extern gimple_opt_pass *make_pass_kcfi_O0 (gcc::context *ctxt);
> > +extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context
> > *ctxt);
> >
>
> these 3 lines should just be in tree-pass.h with the other make_pass_*.
Yup, totally. I've moved these now.
> > diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc
> > index 8950294abb60..432ccc7f864a 100644
> > --- a/gcc/cfgexpand.cc
> > +++ b/gcc/cfgexpand.cc
> > @@ -76,6 +76,7 @@ along with GCC; see the file COPYING3. If not see
> > #include "opts.h"
> > #include "gimple-range.h"
> > #include "rtl-iter.h"
> > +#include "kcfi.h"
> >
> > /* Some systems use __main in a way incompatible with its use in gcc, in
> > these
> > cases use the macros NAME__MAIN to give a quoted symbol and
> > SYMBOL__MAIN to
> > @@ -3203,6 +3204,66 @@ expand_call_stmt (gcall *stmt)
> > else
> > expand_expr (exp, const0_rtx, VOIDmode, EXPAND_NORMAL);
> >
> > + /* Add KCFI annotations if this is an indirect call with KCFI wrapper
> > type. */
> > + if (sanitize_flags_p (SANITIZE_KCFI) && !gimple_call_fndecl (stmt))
> >
>
> Internal function calls with also have a null fndecl can you double check
> that you don't have an internal function call here?
I think I need this explained in more detail. This logic mostly matches
the nocf_check below:
if (gimple_call_nocf_check_p (stmt)
&& !gimple_call_fndecl (stmt))
Or do you mean the fn_type below?
>
>
> > + {
> > + tree fn_type = gimple_call_fntype (stmt);
> > + gcc_assert (fn_type);
... here? I haven't encountered this assert, if that's a useful signal. :)
> > + tree attr = lookup_attribute ("kcfi_type_id", TYPE_ATTRIBUTES
> > (fn_type));
> > + if (attr && TREE_VALUE (attr))
> > + {
> > + /* Check if this call site originated from a no_sanitize("kcfi")
> > function
> > + during inlining. If so, skip KCFI instrumentation in RTL
> > phase too. */
> >
>
> Why not add a bit to gimple_call and add while inlining instead of looking
> it up afterwards?
> I am not 100% sure the BLOCK will always be correct.
Ah, I think I see how: add a new GF_CALL_* flag? I'll go find where
inlining happens...
> > + bool should_skip_kcfi = false;
> > + location_t call_location = gimple_location (stmt);
> > + gcc_assert (call_location != UNKNOWN_LOCATION);
> > +
> > + tree block = gimple_block (stmt);
> > + while (block && TREE_CODE (block) == BLOCK)
> > + {
> > + tree fn_decl = BLOCK_ABSTRACT_ORIGIN (block);
> > + if (fn_decl && TREE_CODE (fn_decl) == FUNCTION_DECL)
> > + {
> > + /* Found an inlined function - check if it has
> > no_sanitize("kcfi"). */
> > + if (!sanitize_flags_p (SANITIZE_KCFI, fn_decl))
> > + {
> > + should_skip_kcfi = true;
> > + break;
> > + }
> > + break;
> > + }
> > + /* Move up the block chain to find parent inlined
> > functions. */
> > + block = BLOCK_SUPERCONTEXT (block);
> > + }
> > +
> > + if (!should_skip_kcfi)
> > + {
> > + uint32_t kcfi_type_id = (uint32_t) tree_to_uhwi (TREE_VALUE
> > (attr));
> > +
> > + /* Find the call that has been created. */
> > + rtx_insn *call_insn = get_last_insn ();
> > + while (call_insn && call_insn != before_call && !CALL_P
> > (call_insn))
> > + call_insn = PREV_INSN (call_insn);
> >
>
> This seems very fragile why not do this inside expand_call in calls.cc
> instead of expand_call_stmt ?
In expand_call, I couldn't find a way to have access to both the GIMPLE
(for the typeid) and the just-created RTL to attach the clobbers and
typeid. Initially I tried to just insert my own RTL in expand_call, but
there are some very complicated things going on in the default expansion,
so I needed to just let those run before I could try to tag the actually
produced call insn.
After looking through expand_call_stmt, I found the pattern used by
nocf_check, which seems to be doing very nearly the same thing KCFI
needed, so I repeated it (with great success). :)
I am, of course, happy to try moving it somewhere else!
> > +
> > + if (call_insn && call_insn != before_call && CALL_P
> > (call_insn))
> > + {
> > + /* Add KCFI type ID note for anti-merging protection. */
> > + add_kcfi_type_note (call_insn, kcfi_type_id);
> > +
> > + /* Add architecture-specific clobbers so register
> > allocator knows
> > + they'll be used. */
> > + if (kcfi_target.add_kcfi_clobbers)
> > + kcfi_target.add_kcfi_clobbers (call_insn);
> > + }
> > + else
> > + {
> > + error ("KCFI: Could not find call instruction for
> > wrapper type");
> > + gcc_unreachable ();
> >
>
> internal_error instead of error/gcc_unreachable.
Okay, I will fix these. And I see fatal_error as well; is that just for
user fixable issues? (i.e. I probably just need to use internal_error
exclusively for these kind of sanity checking pieces.)
> > +kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx)
> > +{
> > + /* Convert trap label to string using standard GCC helper. */
> > + char trap_name[64];
> > + ASM_GENERATE_INTERNAL_LABEL (trap_name, "L", CODE_LABEL_NUMBER
> > (trap_label_rtx));
> > +
> > + /* Generate entry label name from trap label number. */
> > + char entry_name[64];
> > + ASM_GENERATE_INTERNAL_LABEL (entry_name, "Lentry", CODE_LABEL_NUMBER
> > (trap_label_rtx));
> > +
> > + /* Emit .kcfi_traps section entry using the converted labels. */
> > + fprintf (file, "\t.pushsection\t.kcfi_traps,\"ao\",@progbits,.text\n");
> >
>
> This is totally wrong. instead look at varasm and use a section.
I've spent some more time looking through here and I *think* I see
how it's expected to be done with get_section. The trouble seems to be
with SECTION_LINK_ORDER which appears to need explicit special-casing
with a strcmp? (There's an exception for __patchable_function_entries
already.) It feels very weird. :P
My new change to default_elf_asm_named_section replaces:
/* For now, only section "__patchable_function_entries"
adopts flag SECTION_LINK_ORDER, internal label LPFE*
was emitted in default_print_patchable_function_entry,
just place it here for linked_to section. */
gcc_assert (!strcmp (name, "__patchable_function_entries"));
fprintf (asm_out_file, ",");
char buf[256];
ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
current_function_funcdef_no);
assemble_name_raw (asm_out_file, buf);
with:
if (!strcmp (name, "__patchable_function_entries"))
{
/* For patchable function entries, internal label LPFE*
was emitted in default_print_patchable_function_entry,
just place it here for linked_to section. */
fprintf (asm_out_file, ",");
char buf[256];
ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
current_function_funcdef_no);
assemble_name_raw (asm_out_file, buf);
}
else if (!strcmp (name, ".kcfi_traps"))
{
/* KCFI traps section links to .text section. */
fprintf (asm_out_file, ",.text");
}
else
internal_error ("unexpected use of SECTION_LINK_ORDER by section %qs",
name);
These strcmps seem odd. Do __patchable_function_entries and .kcfi_traps
need dedicated global sections like done for others (e.g. exception_section)?
I really can't tell what the right approach should be here.
> + assemble_name (file, entry_name);
> > + fprintf (file, ":\n");
> > + fprintf (file, "\t.long\t");
> > + assemble_name (file, trap_name);
> > + fprintf (file, " - ");
> > + assemble_name (file, entry_name);
> > + fprintf (file, "\n");
> >
>
> There are better ways of generating this subtraction.
I couldn't find an obvious helper for this. I looked for stuff doing
subtraction, relative offsets, etc. The closest I could find was
dw2_asm_output_delta, but it's quite specialized. Were you thinking that
using get_rtx* was the better way on this (like dw2_asm_output_delta
does)? I now currently have:
section *saved_section = in_section;
section *kcfi_traps_section = get_section (".kcfi_traps",
SECTION_LINK_ORDER, NULL);
switch_to_section (kcfi_traps_section);
ASM_OUTPUT_LABEL (file, entry_name);
rtx trap_symbol = gen_rtx_SYMBOL_REF (Pmode, trap_name);
rtx entry_symbol = gen_rtx_SYMBOL_REF (Pmode, entry_name);
rtx addr_diff = gen_rtx_MINUS (Pmode, trap_symbol, entry_symbol);
/* Emit the address difference as a 4-byte value. */
assemble_integer (addr_diff, 4, BITS_PER_UNIT, 1);
switch_to_section (saved_section);
> > + fprintf (file, "\t.popsection\n");
> > +}
> > +
> > +/* Hash function for KCFI type ID computation.
> > + This implements a simple hash similar to FNV-1a. */
> > +static uint32_t
> > +kcfi_hash_string (const char *str)
> > +{
> > + uint32_t hash = 2166136261U; /* FNV-1a 32-bit offset basis. */
> > + for (const char *p = str; *p; p++)
> > + {
> > + hash ^= (unsigned char) *p;
> > + hash *= 16777619U; /* FNV-1a 32-bit prime. */
> > + }
> > + return hash;
> > +}
> > +
> > +/* Compute KCFI type ID for a function declaration or function type
> > (internal) */
> > +static uint32_t
> > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > +{
> > + if (!fntype_or_fndecl)
> > + return 0;
> > +
> > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> >
>
> Now I am curious why this needs to be a mangled function name? Since the
> function in C the symbol is just its name.
> Is there documentation that says the hash needs to be based on all of the
> function arguments types?
> Also since you are producing a mangled name just to produce a hash and
> throwing away the mangled name (except writing it to a dump file which is
> just human readable), instead if better just to produce the hash from the
> function type instead of doing it in 2 steps?
I think this has been answered in other replies, but: yes, hash is based
on entire function prototype (and not function name). The mangling is
there to be human readable (it was an existing ABI so seemed a stable
intermediate). But yes, we could just go directly to the hash and build
the string only for the dumpfile, if no one wants to reuse this mangler?
Thanks for the review!
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 18:09 ` Qing Zhao
@ 2025-08-22 5:15 ` Kees Cook
2025-08-22 10:03 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-22 5:15 UTC (permalink / raw)
To: Qing Zhao
Cc: Peter Zijlstra, Richard Biener, Andrew Pinski,
gcc-patches@gcc.gnu.org, Joseph Myers, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening@vger.kernel.org
On Thu, Aug 21, 2025 at 06:09:08PM +0000, Qing Zhao wrote:
> > On Aug 21, 2025, at 10:25, Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Thu, Aug 21, 2025 at 01:01:37PM +0200, Richard Biener wrote:
> >> On Thu, 21 Aug 2025, Peter Zijlstra wrote:
> >>
> >>> On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> >>>
> >>>>> +/* Compute KCFI type ID for a function declaration or function type
> >>>>> (internal) */
> >>>>> +static uint32_t
> >>>>> +compute_kcfi_type_id (tree fntype_or_fndecl)
> >>>>> +{
> >>>>> + if (!fntype_or_fndecl)
> >>>>> + return 0;
> >>>>> +
> >>>>> + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> >>>>> + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> >>>>>
> >>>>
> >>>> Now I am curious why this needs to be a mangled function name? Since the
> >>>> function in C the symbol is just its name.
> >>>> Is there documentation that says the hash needs to be based on all of the
> >>>> function arguments types?
> >>>
> >>> The whole point of kCFI is to limit the targets of indirect calls to
> >>> functions of the same signature. The actual function name is immaterial.
> >>
> >> What's the attack vector and how does kCFI achieve mitigating it?
> >
> > Any of the attacks that can result in scribbling a function pointer.
> > Typically a buffer overflow I suppose.
> >
> >
> > The way kCFI works is by changing the indirect call ABI. Traditionally
> > the indirect call is simply:
> >
> > load-pointer-into-reg
> > call *%reg
> >
> > kCFI changes every function to have a preamble like (with IBT and
> > retpolines and all the modern crap on):
>
> Does “every function” mean all the function in the compilation? Not only the function whose address is taken?
I tried to explain the specific logic on how the set of functions getting
preambles is chosen in this other reply:
https://lore.kernel.org/linux-hardening/202508211258.8DEE293@keescook/
If that didn't answer your question, let me know and I'll try again. :)
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-22 5:10 ` Kees Cook
@ 2025-08-22 5:27 ` Andrew Pinski
0 siblings, 0 replies; 42+ messages in thread
From: Andrew Pinski @ 2025-08-22 5:27 UTC (permalink / raw)
To: Kees Cook
Cc: Andrew Pinski, Qing Zhao, gcc-patches, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening
On Thu, Aug 21, 2025 at 10:11 PM Kees Cook <kees@kernel.org> wrote:
>
> On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> > On Thu, Aug 21, 2025 at 12:47 AM Kees Cook <kees@kernel.org> wrote:
> > > +struct kcfi_target_hooks {
> > > + /* Apply architecture-specific masking to type ID. */
> > > + uint32_t (*mask_type_id) (uint32_t type_id);
> > > +
> > > + /* Generate bundled KCFI checked call (atomic check + call to prevent
> > > optimizer separation) */
> > > + rtx (*gen_kcfi_checked_call) (rtx call_insn, rtx target_reg, uint32_t
> > > expected_type, HOST_WIDE_INT prefix_nops);
> > > +
> > > + /* Add architecture-specific register clobbers for KCFI calls. */
> > > + void (*add_kcfi_clobbers) (rtx_insn *call_insn);
> > > +
> > > + /* Calculate architecture-specific prefix NOPs count (optional, returns
> > > prefix_nops unchanged if NULL) */
> > > + int (*calculate_prefix_nops) (HOST_WIDE_INT prefix_nops);
> > > +
> > > + /* Emit architecture-specific type ID instruction (required for common
> > > preamble helper) */
> > > + void (*emit_type_id_instruction) (FILE *file, uint32_t type_id);
> > > +};
> > >
> >
> > This should really be part of target.def so you also have the internals
> > documentation done for these hooks.
> > And then you could just use targetm.kcfi.xyz below.
> > And for the hooks you would have TARGET_KCFI_XYZ being defined like the
> > others one.
>
> Ah-ha! Thank you. The macros to build that eluded me. I've updated to
> this now.
>
> > > +/* Pass creation functions. */
> > > +class gimple_opt_pass;
> > > +class rtl_opt_pass;
> > > +namespace gcc { class context; }
> > >
> >
> > This above is definitely bad form.
>
> Ah yeah, sorry; this was a leftover. (And below.)
>
> > > +
> > > +extern gimple_opt_pass *make_pass_kcfi (gcc::context *ctxt);
> > > +extern gimple_opt_pass *make_pass_kcfi_O0 (gcc::context *ctxt);
> > > +extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context
> > > *ctxt);
> > >
> >
> > these 3 lines should just be in tree-pass.h with the other make_pass_*.
>
> Yup, totally. I've moved these now.
>
> > > diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc
> > > index 8950294abb60..432ccc7f864a 100644
> > > --- a/gcc/cfgexpand.cc
> > > +++ b/gcc/cfgexpand.cc
> > > @@ -76,6 +76,7 @@ along with GCC; see the file COPYING3. If not see
> > > #include "opts.h"
> > > #include "gimple-range.h"
> > > #include "rtl-iter.h"
> > > +#include "kcfi.h"
> > >
> > > /* Some systems use __main in a way incompatible with its use in gcc, in
> > > these
> > > cases use the macros NAME__MAIN to give a quoted symbol and
> > > SYMBOL__MAIN to
> > > @@ -3203,6 +3204,66 @@ expand_call_stmt (gcall *stmt)
> > > else
> > > expand_expr (exp, const0_rtx, VOIDmode, EXPAND_NORMAL);
> > >
> > > + /* Add KCFI annotations if this is an indirect call with KCFI wrapper
> > > type. */
> > > + if (sanitize_flags_p (SANITIZE_KCFI) && !gimple_call_fndecl (stmt))
> > >
> >
> > Internal function calls with also have a null fndecl can you double check
> > that you don't have an internal function call here?
>
> I think I need this explained in more detail. This logic mostly matches
> the nocf_check below:
>
> if (gimple_call_nocf_check_p (stmt)
> && !gimple_call_fndecl (stmt))
>
> Or do you mean the fn_type below?
So nocf check is on the GIMPLE_CALL first unlike your code. The nocf
is set only from the original function type so it will NOT be on calls
to internal function. You just need to check gimple_call_internal_p
first.
But then I missed that expand_call_stmt does a check for call internal
very first. So just add a comment/assert that you will not have an
internal call here.
Thanks,
Andrew
> >
> >
> > > + {
> > > + tree fn_type = gimple_call_fntype (stmt);
> > > + gcc_assert (fn_type);
>
> ... here? I haven't encountered this assert, if that's a useful signal. :)
>
> > > + tree attr = lookup_attribute ("kcfi_type_id", TYPE_ATTRIBUTES
> > > (fn_type));
> > > + if (attr && TREE_VALUE (attr))
> > > + {
> > > + /* Check if this call site originated from a no_sanitize("kcfi")
> > > function
> > > + during inlining. If so, skip KCFI instrumentation in RTL
> > > phase too. */
> > >
> >
> > Why not add a bit to gimple_call and add while inlining instead of looking
> > it up afterwards?
> > I am not 100% sure the BLOCK will always be correct.
>
> Ah, I think I see how: add a new GF_CALL_* flag? I'll go find where
> inlining happens...
>
> > > + bool should_skip_kcfi = false;
> > > + location_t call_location = gimple_location (stmt);
> > > + gcc_assert (call_location != UNKNOWN_LOCATION);
> > > +
> > > + tree block = gimple_block (stmt);
> > > + while (block && TREE_CODE (block) == BLOCK)
> > > + {
> > > + tree fn_decl = BLOCK_ABSTRACT_ORIGIN (block);
> > > + if (fn_decl && TREE_CODE (fn_decl) == FUNCTION_DECL)
> > > + {
> > > + /* Found an inlined function - check if it has
> > > no_sanitize("kcfi"). */
> > > + if (!sanitize_flags_p (SANITIZE_KCFI, fn_decl))
> > > + {
> > > + should_skip_kcfi = true;
> > > + break;
> > > + }
> > > + break;
> > > + }
> > > + /* Move up the block chain to find parent inlined
> > > functions. */
> > > + block = BLOCK_SUPERCONTEXT (block);
> > > + }
> > > +
> > > + if (!should_skip_kcfi)
> > > + {
> > > + uint32_t kcfi_type_id = (uint32_t) tree_to_uhwi (TREE_VALUE
> > > (attr));
> > > +
> > > + /* Find the call that has been created. */
> > > + rtx_insn *call_insn = get_last_insn ();
> > > + while (call_insn && call_insn != before_call && !CALL_P
> > > (call_insn))
> > > + call_insn = PREV_INSN (call_insn);
> > >
> >
> > This seems very fragile why not do this inside expand_call in calls.cc
> > instead of expand_call_stmt ?
>
> In expand_call, I couldn't find a way to have access to both the GIMPLE
> (for the typeid) and the just-created RTL to attach the clobbers and
> typeid. Initially I tried to just insert my own RTL in expand_call, but
> there are some very complicated things going on in the default expansion,
> so I needed to just let those run before I could try to tag the actually
> produced call insn.
>
> After looking through expand_call_stmt, I found the pattern used by
> nocf_check, which seems to be doing very nearly the same thing KCFI
> needed, so I repeated it (with great success). :)
>
> I am, of course, happy to try moving it somewhere else!
>
> > > +
> > > + if (call_insn && call_insn != before_call && CALL_P
> > > (call_insn))
> > > + {
> > > + /* Add KCFI type ID note for anti-merging protection. */
> > > + add_kcfi_type_note (call_insn, kcfi_type_id);
> > > +
> > > + /* Add architecture-specific clobbers so register
> > > allocator knows
> > > + they'll be used. */
> > > + if (kcfi_target.add_kcfi_clobbers)
> > > + kcfi_target.add_kcfi_clobbers (call_insn);
> > > + }
> > > + else
> > > + {
> > > + error ("KCFI: Could not find call instruction for
> > > wrapper type");
> > > + gcc_unreachable ();
> > >
> >
> > internal_error instead of error/gcc_unreachable.
>
> Okay, I will fix these. And I see fatal_error as well; is that just for
> user fixable issues? (i.e. I probably just need to use internal_error
> exclusively for these kind of sanity checking pieces.)
>
> > > +kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx)
> > > +{
> > > + /* Convert trap label to string using standard GCC helper. */
> > > + char trap_name[64];
> > > + ASM_GENERATE_INTERNAL_LABEL (trap_name, "L", CODE_LABEL_NUMBER
> > > (trap_label_rtx));
> > > +
> > > + /* Generate entry label name from trap label number. */
> > > + char entry_name[64];
> > > + ASM_GENERATE_INTERNAL_LABEL (entry_name, "Lentry", CODE_LABEL_NUMBER
> > > (trap_label_rtx));
> > > +
> > > + /* Emit .kcfi_traps section entry using the converted labels. */
> > > + fprintf (file, "\t.pushsection\t.kcfi_traps,\"ao\",@progbits,.text\n");
> > >
> >
> > This is totally wrong. instead look at varasm and use a section.
>
> I've spent some more time looking through here and I *think* I see
> how it's expected to be done with get_section. The trouble seems to be
> with SECTION_LINK_ORDER which appears to need explicit special-casing
> with a strcmp? (There's an exception for __patchable_function_entries
> already.) It feels very weird. :P
>
> My new change to default_elf_asm_named_section replaces:
>
> /* For now, only section "__patchable_function_entries"
> adopts flag SECTION_LINK_ORDER, internal label LPFE*
> was emitted in default_print_patchable_function_entry,
> just place it here for linked_to section. */
> gcc_assert (!strcmp (name, "__patchable_function_entries"));
> fprintf (asm_out_file, ",");
> char buf[256];
> ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
> current_function_funcdef_no);
> assemble_name_raw (asm_out_file, buf);
>
> with:
>
> if (!strcmp (name, "__patchable_function_entries"))
> {
> /* For patchable function entries, internal label LPFE*
> was emitted in default_print_patchable_function_entry,
> just place it here for linked_to section. */
> fprintf (asm_out_file, ",");
> char buf[256];
> ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
> current_function_funcdef_no);
> assemble_name_raw (asm_out_file, buf);
> }
> else if (!strcmp (name, ".kcfi_traps"))
> {
> /* KCFI traps section links to .text section. */
> fprintf (asm_out_file, ",.text");
> }
> else
> internal_error ("unexpected use of SECTION_LINK_ORDER by section %qs",
> name);
>
> These strcmps seem odd. Do __patchable_function_entries and .kcfi_traps
> need dedicated global sections like done for others (e.g. exception_section)?
> I really can't tell what the right approach should be here.
>
> > + assemble_name (file, entry_name);
> > > + fprintf (file, ":\n");
> > > + fprintf (file, "\t.long\t");
> > > + assemble_name (file, trap_name);
> > > + fprintf (file, " - ");
> > > + assemble_name (file, entry_name);
> > > + fprintf (file, "\n");
> > >
> >
> > There are better ways of generating this subtraction.
>
> I couldn't find an obvious helper for this. I looked for stuff doing
> subtraction, relative offsets, etc. The closest I could find was
> dw2_asm_output_delta, but it's quite specialized. Were you thinking that
> using get_rtx* was the better way on this (like dw2_asm_output_delta
> does)? I now currently have:
>
> section *saved_section = in_section;
>
> section *kcfi_traps_section = get_section (".kcfi_traps",
> SECTION_LINK_ORDER, NULL);
> switch_to_section (kcfi_traps_section);
>
> ASM_OUTPUT_LABEL (file, entry_name);
>
> rtx trap_symbol = gen_rtx_SYMBOL_REF (Pmode, trap_name);
> rtx entry_symbol = gen_rtx_SYMBOL_REF (Pmode, entry_name);
> rtx addr_diff = gen_rtx_MINUS (Pmode, trap_symbol, entry_symbol);
>
> /* Emit the address difference as a 4-byte value. */
> assemble_integer (addr_diff, 4, BITS_PER_UNIT, 1);
>
> switch_to_section (saved_section);
>
> > > + fprintf (file, "\t.popsection\n");
> > > +}
> > > +
> > > +/* Hash function for KCFI type ID computation.
> > > + This implements a simple hash similar to FNV-1a. */
> > > +static uint32_t
> > > +kcfi_hash_string (const char *str)
> > > +{
> > > + uint32_t hash = 2166136261U; /* FNV-1a 32-bit offset basis. */
> > > + for (const char *p = str; *p; p++)
> > > + {
> > > + hash ^= (unsigned char) *p;
> > > + hash *= 16777619U; /* FNV-1a 32-bit prime. */
> > > + }
> > > + return hash;
> > > +}
> > > +
> > > +/* Compute KCFI type ID for a function declaration or function type
> > > (internal) */
> > > +static uint32_t
> > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > +{
> > > + if (!fntype_or_fndecl)
> > > + return 0;
> > > +
> > > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > >
> >
> > Now I am curious why this needs to be a mangled function name? Since the
> > function in C the symbol is just its name.
> > Is there documentation that says the hash needs to be based on all of the
> > function arguments types?
> > Also since you are producing a mangled name just to produce a hash and
> > throwing away the mangled name (except writing it to a dump file which is
> > just human readable), instead if better just to produce the hash from the
> > function type instead of doing it in 2 steps?
>
> I think this has been answered in other replies, but: yes, hash is based
> on entire function prototype (and not function name). The mangling is
> there to be human readable (it was an existing ABI so seemed a stable
> intermediate). But yes, we could just go directly to the hash and build
> the string only for the dumpfile, if no one wants to reuse this mangler?
>
> Thanks for the review!
>
> -Kees
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 19:57 ` Kees Cook
@ 2025-08-22 6:53 ` Richard Biener
2025-08-22 19:23 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Richard Biener @ 2025-08-22 6:53 UTC (permalink / raw)
To: Kees Cook
Cc: Peter Zijlstra, Andrew Pinski, Qing Zhao, gcc-patches,
Joseph Myers, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, 21 Aug 2025, Kees Cook wrote:
> On Thu, Aug 21, 2025 at 01:01:37PM +0200, Richard Biener wrote:
> > On Thu, 21 Aug 2025, Peter Zijlstra wrote:
> >
> > > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> > >
> > > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > > (internal) */
> > > > > +static uint32_t
> > > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > > +{
> > > > > + if (!fntype_or_fndecl)
> > > > > + return 0;
> > > > > +
> > > > > + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> > > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > > >
> > > >
> > > > Now I am curious why this needs to be a mangled function name? Since the
> > > > function in C the symbol is just its name.
> > > > Is there documentation that says the hash needs to be based on all of the
> > > > function arguments types?
> > >
> > > The whole point of kCFI is to limit the targets of indirect calls to
> > > functions of the same signature. The actual function name is immaterial.
> >
> > What's the attack vector and how does kCFI achieve mitigating it?
>
> To add some more detail to Peter's answer, the attack vector is dealing
> with stored function pointers (generally on the heap, but potentially
> also the stack), that can be changed by attacker-controlled writes (via
> buffer overflows or use-after-free writes). The idea being that instead
> of being able to call anywhere through an indirect call site that uses a
> manipulated pointer, now the attacker is limited to a subset of "matching"
> destinations. (KCFI depends on a system already enforcing W^X memory in
> the sense that if an attacker can construct an executable region, they
> could write whatever hash they want into in addition to whatever code.)
>
> The general CFI ideas for this are discussed here, but focuses more on a
> CFG analysis to construct valid call destinations, which tends to require
> LTO, etc:
> https://users.soe.ucsc.edu/~abadi/Papers/cfi-tissec-revised.pdf
>
> Later refinement for using jump tables (constructed via CFG analysis
> during LTO) was proposed here:
> https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-tice.pdf
> Linux used the above implementation from 2018 to 2022:
> https://android-developers.googleblog.com/2018/10/control-flow-integrity-in-android-kernel.html
> but the corner cases for target addresses not being the actual functions
> (i.e. pointing into the jump table) was a continual source of problems,
> and generating the jump tables required full LTO, which had its own set
> of problems.
>
> Looking at function prototypes as the source of call validity was
> presented here, though still relied on LTO:
> https://www.blackhat.com/docs/asia-17/materials/asia-17-Moreira-Drop-The-Rop-Fine-Grained-Control-Flow-Integrity-For-The-Linux-Kernel-wp.pdf
>
> The KCFI approach built on the function-prototype idea, but avoided
> needing LTO, and could be further updated to deal with CPU errata
> (retpolines, etc):
> https://lpc.events/event/16/contributions/1315/
> This has been working very well now for 3 years, but has been limited to
> only Linux built with Clang (e.g. Linux kernel CFI, first LLVM-CFI and now
> KCFI, has been deployed on all Android phones and all Chrome OS devices
> since 2018.)
>
> I have been trying to find someone to work on KCFI in GCC since 2018,
> (see my annual arm-waving at the Linux Plumbers conference Toolchain
> track) and other than Dan Li who took a stab at it for aarch64 in 2023,
> no one has stepped up to do it, so I thought I'd finally try tackling
> it. :)
Thanks for the write-up. So I wonder whether a more general solution
would be to detach "hash value" computation and assignment and leave
that to external tooling that could do better than non-LTO can and
the compiler working with attributed function types instead? Say
void __attribute__((cfi_hash("voidvoidbutonlykind1"))) foo (void);
that decouples the way to arrive at the hashes from the instrumentation.
You could then of course have compiler assisted auto-attribution
based on function types. So implementation-wise I'd try to separate
both? Oh, and somehow it reminds me of vtable verification?
That said, how's the collision rate on the kernel side?
What's the usual exploitation of overwriting a call function pointer?
That is, what's a good useful function to target? I'd guess a
hypothetical setuid (int)? Somehow it feels like a more academic
thing, with other attack vectors being much more accessible?
Richard.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
[not found] ` <CA+=Sn1koTTQaXDnAVWtVU6ACWwhD08NR5nDJO236Pmcoi2X9qA@mail.gmail.com>
@ 2025-08-22 7:51 ` Peter Zijlstra
2025-08-22 8:24 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-22 7:51 UTC (permalink / raw)
To: Andrew Pinski
Cc: Andrew Pinski, Kees Cook, Qing Zhao, GCC Patches, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 02:22:30AM -0700, Andrew Pinski wrote:
> On Thu, Aug 21, 2025, 2:13 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> >
> > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > (internal) */
> > > > +static uint32_t
> > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > +{
> > > > + if (!fntype_or_fndecl)
> > > > + return 0;
> > > > +
> > > > + const char *canonical_name = mangle_function_type
> > (fntype_or_fndecl);
> > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > >
> > >
> > > Now I am curious why this needs to be a mangled function name? Since the
> > > function in C the symbol is just its name.
> > > Is there documentation that says the hash needs to be based on all of the
> > > function arguments types?
> >
> > The whole point of kCFI is to limit the targets of indirect calls to
> > functions of the same signature. The actual function name is immaterial.
> >
>
>
> So then just hash the function argument types. It only needs to be
> consistent for the objects that are compiled together right?
Function argument and return; but yes that could be done. Ideally the
kCFI implementation would be compatible between compilers. Specifically
rust is based on llvm and therefore generates kCFI that is compatible
with clang. Being able to mix GCC and rust code (as the kernel does)
would be nice.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-21 18:46 ` Kees Cook
2025-08-21 19:03 ` Kees Cook
@ 2025-08-22 8:19 ` Peter Zijlstra
2025-08-22 8:36 ` Kees Cook
1 sibling, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-22 8:19 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Thu, Aug 21, 2025 at 11:46:17AM -0700, Kees Cook wrote:
> On Thu, Aug 21, 2025 at 11:29:35AM +0200, Peter Zijlstra wrote:
> > The current kernel FineIBT code hard assumes r11 for now.
>
> Oh, it looked like it wasn't always r11. Does clang force the call
> register to be r11?
Yes. I'm not sure why, but that's what it does, unconditionally r11.
(funny extra detail, when using retpolines, clang sometimes generates
conditional tail-calls, it merged Jcc and JMP __x86_indirect_thunk_r11
into Jcc __x86_indirect_thunk_r11. I've never seen GCC do this).
> I only do that here if the call expression isn't a
> register (similar to -mindirect-branch-register). Looking at the retpoline
> implementation, I see __x86_indirect_thunk_* being generated for all the
> general registers.
Yeah, generating the whole set was easiest. The BPF JIT and custom asm
also have retpolines in. Eg.
arch/x86/kernel/ftrace_64.S: CALL_NOSPEC r8
arch/x86/platform/efi/efi_stub_64.S: CALL_NOSPEC rdi
> Hm, but in looking now I see all the hard-coded r11 use
> in the fineibt alternatives. I wonder if my boot testing is somehow not
> triggering the FineIBT alternatives patching? I will investigate more...
Right, GCC typically prefers rax (with a wide margin) but pretty much
every register gets used on a big enough code-base.
Random defconfig of the day gets me:
15227 __x86_indirect_thunk_rax
417 __x86_indirect_thunk_rdx
205 __x86_indirect_thunk_rcx
110 __x86_indirect_thunk_r13
108 __x86_indirect_thunk_r12
98 __x86_indirect_thunk_r8
97 __x86_indirect_thunk_rbp
96 __x86_indirect_thunk_r10
85 __x86_indirect_thunk_r14
38 __x86_indirect_thunk_r15
36 __x86_indirect_thunk_r9
34 __x86_indirect_thunk_rbx
28 __x86_indirect_thunk_r11
16 __x86_indirect_thunk_rsi
1 __x86_indirect_thunk_rdi
IIRC you clobber r10 for the hash usage, so you'll not generate indirect
calls through that, but other than that the code seems to preserve
random register if the address is already loaded in it, otherwise loads
into r11.
Anyway, I might be able to deal with the indirect call not being r11,
but it'll take a bit of prodding. Also it will shatter my plans to move
the hash to eax to save a few bytes in instruction encoding. Let me go
poke around with that UDB patch see what's possible.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-22 7:51 ` Peter Zijlstra
@ 2025-08-22 8:24 ` Peter Zijlstra
2025-08-22 8:47 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-22 8:24 UTC (permalink / raw)
To: Andrew Pinski
Cc: Andrew Pinski, Kees Cook, Qing Zhao, GCC Patches, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Fri, Aug 22, 2025 at 09:51:15AM +0200, Peter Zijlstra wrote:
> On Thu, Aug 21, 2025 at 02:22:30AM -0700, Andrew Pinski wrote:
> > On Thu, Aug 21, 2025, 2:13 AM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> > >
> > > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > > (internal) */
> > > > > +static uint32_t
> > > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > > +{
> > > > > + if (!fntype_or_fndecl)
> > > > > + return 0;
> > > > > +
> > > > > + const char *canonical_name = mangle_function_type
> > > (fntype_or_fndecl);
> > > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > > >
> > > >
> > > > Now I am curious why this needs to be a mangled function name? Since the
> > > > function in C the symbol is just its name.
> > > > Is there documentation that says the hash needs to be based on all of the
> > > > function arguments types?
> > >
> > > The whole point of kCFI is to limit the targets of indirect calls to
> > > functions of the same signature. The actual function name is immaterial.
> > >
> >
> >
> > So then just hash the function argument types. It only needs to be
> > consistent for the objects that are compiled together right?
>
> Function argument and return; but yes that could be done. Ideally the
> kCFI implementation would be compatible between compilers. Specifically
> rust is based on llvm and therefore generates kCFI that is compatible
> with clang. Being able to mix GCC and rust code (as the kernel does)
> would be nice.
FWIW, Kees, for this to actually work, we need this
CFI_ICALL_NORMALIZE_INTEGERS thing supported. Rust gets really upset
about LP64's whole 'long' vs 'long long' trainwreck :/
That is the -fsanitize-cfi-icall-experimental-normalize-integers
argument for clang (omg so long).
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-22 8:19 ` Peter Zijlstra
@ 2025-08-22 8:36 ` Kees Cook
2025-08-22 8:55 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-22 8:36 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Fri, Aug 22, 2025 at 10:19:15AM +0200, Peter Zijlstra wrote:
> Anyway, I might be able to deal with the indirect call not being r11,
> but it'll take a bit of prodding. Also it will shatter my plans to move
> the hash to eax to save a few bytes in instruction encoding. Let me go
> poke around with that UDB patch see what's possible.
I think I have it mostly working to force r11 when doing kcfi and
retpoline now, though I'm seeing a few glitches still. I'll keep working
on it.
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-22 8:24 ` Peter Zijlstra
@ 2025-08-22 8:47 ` Kees Cook
0 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-22 8:47 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Andrew Pinski, Andrew Pinski, Qing Zhao, GCC Patches,
Joseph Myers, Richard Biener, Jan Hubicka, Richard Earnshaw,
Richard Sandiford, Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng,
Palmer Dabbelt, Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening
On Fri, Aug 22, 2025 at 10:24:20AM +0200, Peter Zijlstra wrote:
> On Fri, Aug 22, 2025 at 09:51:15AM +0200, Peter Zijlstra wrote:
> > On Thu, Aug 21, 2025 at 02:22:30AM -0700, Andrew Pinski wrote:
> > > On Thu, Aug 21, 2025, 2:13 AM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > > On Thu, Aug 21, 2025 at 01:16:56AM -0700, Andrew Pinski wrote:
> > > >
> > > > > > +/* Compute KCFI type ID for a function declaration or function type
> > > > > > (internal) */
> > > > > > +static uint32_t
> > > > > > +compute_kcfi_type_id (tree fntype_or_fndecl)
> > > > > > +{
> > > > > > + if (!fntype_or_fndecl)
> > > > > > + return 0;
> > > > > > +
> > > > > > + const char *canonical_name = mangle_function_type
> > > > (fntype_or_fndecl);
> > > > > > + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> > > > > >
> > > > >
> > > > > Now I am curious why this needs to be a mangled function name? Since the
> > > > > function in C the symbol is just its name.
> > > > > Is there documentation that says the hash needs to be based on all of the
> > > > > function arguments types?
> > > >
> > > > The whole point of kCFI is to limit the targets of indirect calls to
> > > > functions of the same signature. The actual function name is immaterial.
> > > >
> > >
> > >
> > > So then just hash the function argument types. It only needs to be
> > > consistent for the objects that are compiled together right?
> >
> > Function argument and return; but yes that could be done. Ideally the
> > kCFI implementation would be compatible between compilers. Specifically
> > rust is based on llvm and therefore generates kCFI that is compatible
> > with clang. Being able to mix GCC and rust code (as the kernel does)
> > would be nice.
>
> FWIW, Kees, for this to actually work, we need this
> CFI_ICALL_NORMALIZE_INTEGERS thing supported. Rust gets really upset
> about LP64's whole 'long' vs 'long long' trainwreck :/
>
> That is the -fsanitize-cfi-icall-experimental-normalize-integers
> argument for clang (omg so long).
Yup! I forgot to include my "TODO" list in the RFC. It is:
* -fsanitize-cfi-icall-experimental-normalize-integers (but this option
needs a better name if it's going to be supported in GCC too for Rust
compat)
* -fsanitize-kcfi-arity
https://clang.llvm.org/docs/ControlFlowIntegrity.html#fsanitize-kcfi-arity
* cfi_salt function attribute
https://clang.llvm.org/docs/AttributeReference.html#cfi-salt
https://github.com/llvm/llvm-project/commit/aa4805a09052c1b6298718eeb6d30c33dd0d695f
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2025-08-22 8:36 ` Kees Cook
@ 2025-08-22 8:55 ` Peter Zijlstra
0 siblings, 0 replies; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-22 8:55 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches, Joseph Myers, Richard Biener, Jan Hubicka,
Richard Earnshaw, Richard Sandiford, Marcus Shawcroft,
Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt, Andrew Waterman,
Jim Wilson, Dan Li, linux-hardening
On Fri, Aug 22, 2025 at 01:36:43AM -0700, Kees Cook wrote:
> On Fri, Aug 22, 2025 at 10:19:15AM +0200, Peter Zijlstra wrote:
> > Anyway, I might be able to deal with the indirect call not being r11,
> > but it'll take a bit of prodding. Also it will shatter my plans to move
> > the hash to eax to save a few bytes in instruction encoding. Let me go
> > poke around with that UDB patch see what's possible.
>
> I think I have it mostly working to force r11 when doing kcfi and
> retpoline now, though I'm seeing a few glitches still. I'll keep working
> on it.
Awesome, thanks!
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-22 5:15 ` Kees Cook
@ 2025-08-22 10:03 ` Peter Zijlstra
0 siblings, 0 replies; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-22 10:03 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, Richard Biener, Andrew Pinski, gcc-patches@gcc.gnu.org,
Joseph Myers, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening@vger.kernel.org
On Thu, Aug 21, 2025 at 10:15:31PM -0700, Kees Cook wrote:
> On Thu, Aug 21, 2025 at 06:09:08PM +0000, Qing Zhao wrote:
> > > On Aug 21, 2025, at 10:25, Peter Zijlstra <peterz@infradead.org> wrote:
> > > kCFI changes every function to have a preamble like (with IBT and
> > > retpolines and all the modern crap on):
> >
> > Does “every function” mean all the function in the compilation? Not only the function whose address is taken?
>
> I tried to explain the specific logic on how the set of functions getting
> preambles is chosen in this other reply:
> https://lore.kernel.org/linux-hardening/202508211258.8DEE293@keescook/
>
> If that didn't answer your question, let me know and I'll try again. :)
One detail that might be interesting is that ENDBR and the __cfi_\func
preamble should be the same condition. The way these features are
exposed doesn't make this obvious.
And in fact, marking an address taken function with __noendbr will
result in a function that has a __cfi_\func preamble, but no ENDBR
(clang-20).
And while there is both a __noendbr and __nocfi attribute, they are
*VERY* different from one another. Where __noendbr inhibits the emission
of ENDBR and basically marks the function as impossible to reach with an
indirect call, the __nocfi attribute inhibits the CALL modification.
Notably, there is no function attribute to inhibit the __cfi_\func
preamble (and I'm arguing __noendbr should have that effect).
OTOH a function without __cfi_\func preamble but with ENDBR is 'unsafe',
but usable with a __nocfi call (typically reserved for calling external
code, like firmware). Anyway, we don't currently have means of
expressing this to the compiler (also, I don't care much in this case --
I think we should taint the kernel on EFI calls :-).
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-21 21:29 ` Kees Cook
@ 2025-08-22 15:11 ` Qing Zhao
2025-08-22 19:02 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Qing Zhao @ 2025-08-22 15:11 UTC (permalink / raw)
To: Kees Cook
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
> On Aug 21, 2025, at 17:29, Kees Cook <kees@kernel.org> wrote:
>
> On Thu, Aug 21, 2025 at 07:14:31PM +0000, Qing Zhao wrote:
>>
>>
>>> On Aug 21, 2025, at 12:16, Kees Cook <kees@kernel.org> wrote:
>>>
>>>
>>>>> + else if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
>>>>> + {
>>>>> + tree fndecl = fntype_or_fndecl;
>>>>> + tree base_fntype = TREE_TYPE (fndecl);
>>>>> +
>>>>> + /* For FUNCTION_DECL, build a synthetic function type using
>>>>> DECL_ARGUMENTS
>>>>> + if available to preserve typedef information. */
>>>>>
>>>>
>>>> Why do the building? Seems like you could just do that work here. Also
>>>> doesn't FUNCTION_DECL's type have exactly what you need?
>>>
>>> I need the function prototype in three places:
>>>
>>> - address-taken extern functions
>>> - function preambles
>>> - indirect call sites
>>>
>>
>> A little confused with the above:
>>
>> From my understanding,
>>
>> 1. At each indirect call sites, we should generate the checking code to
>> A. load the hashed precomputed typeid from the callee’s preamble
>> B. compare it with the precomputed typeid for this call site
>>
>> So, we need the function prototype of the indirect call site to compute the typeid for this call site.
>
> Correct.
>
>> 2. For every “address-taken” function, we should generate the function
>> preamble, in which the precomputed typeid for this function is stored.
>>
>> So, we need the function prototype of this function to compute the typeid for this function.
>>
>> The above 2 should cover all the KCFI ABIs.
>
> For non-static functions, we cannot know if other compilation units may
> make indirect calls to a given function, so those functions must always
> have their kcfi preamble added. For static functions, if they are
> address-taken by the current compilation unit, then they must get a kcfi
> preamble added.
Oh, yeah, I see. without lto or whole-program-mode, we cannot determine
whether a extern function is address taken or not. Therefore, we have to
treat ALL extern functions conservatively as address taken.
So, from my understanding, the complete list that need to compute the typeid from the function prototype is:
- At indirect call sites
- all indirect call sites; (At the call site)
- At function preambles
- all address-taken static functions (At the function definition)
- all extern functions (At function declaration or function definition?? Please see my question below)
>
>> What I was confused is, why “address-taken external function” and “function preambles” are separated items?
>> For the function preambles, shall we generate for all the functions? Or only for address-taken functions in
>> the compilation?
>
> The other case is emitting the __ckfi_typeid_FUNC weak symbols, which is
> used for link-time resolution with non-C code (i.e. raw .S assembly)
> which doesn't have access to the C type system to calculate the hashes
> on its own, and needs to have a way to build its own kcfi preambles.
So, for such functions, there should be an extern function declaration in the C code.
But the definition of such function is not available in the C code we are compiling.
Therefore the weak __ckfi_typeid_FUNC symbol is emitted at the function declaration
point for such function when we compile the C code?
And the typeid (the hash value) for such routine is computed at the function declaration
point too.
Is the above understanding correct?
Then for the other extern function whose definition is in the C code of other modules that might
be compiled later, should the typeid is computed at the declaration or the definition?
> This
> is how Linux constructs its assembly function entry points:
>
> #ifndef __CFI_TYPE
> #define __CFI_TYPE(name) \
> .4byte __kcfi_typeid_##name
> #endif
>
> #define SYM_TYPED_ENTRY(name, linkage, align...) \
> linkage(name) ASM_NL \
> align ASM_NL \
> __CFI_TYPE(name) ASM_NL \
> name:
>
> That way all the asm functions can be be indirect call targets without
> knowing the hash value (which will be filled in at link time).
Okay. I see. This is the case for the extern function whose definition is in the assembly file. (Not available in
the C code)
>
>>> At indirect call sites (during the early GIMPLE pass), I had a
>>> FUNCTION_TYPE available that still had the full typedef information,
>>> and I could use it fine.
>>
>>> For the other two, it's later on and the
>>> TREE_TYPE(fndecl)'s FUNCTION_TYPE had lost the typedef information (which
>>> I need to be able to examine in cases where the typedef name was needed
>>> for the mangling vs looking at the underlying types).
>>
>> Then why not also compute the typeid for the function preamble during early GIMPLE phase
>> the same as the indirect call sites when all the typedef information is available?
>
> I assume I just didn't see how yet. :) I wasn't able to identify nor
> store the typeid for function definitions that ultimately end up getting
> .s file output.
So, the problem only exists for the external functions whose definition is NOT in the C code?
Qing
> For example, down in ix86_asm_output_function_label(),
> I have the decl (but it's way late):
>
> ix86_asm_output_function_label (FILE *out_file, const char *fname,
> tree decl)
>
> I couldn't figure out how to find these during the GIMPLE pass. Oh,
> perhaps I can do this with an IPA pass? That should let me walk all
> functions including externs. I'll give it a try...
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-22 15:11 ` Qing Zhao
@ 2025-08-22 19:02 ` Kees Cook
2025-08-22 20:29 ` Qing Zhao
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-22 19:02 UTC (permalink / raw)
To: Qing Zhao
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
On Fri, Aug 22, 2025 at 03:11:16PM +0000, Qing Zhao wrote:
> > On Aug 21, 2025, at 17:29, Kees Cook <kees@kernel.org> wrote:
> > For non-static functions, we cannot know if other compilation units may
> > make indirect calls to a given function, so those functions must always
> > have their kcfi preamble added. For static functions, if they are
> > address-taken by the current compilation unit, then they must get a kcfi
> > preamble added.
>
> Oh, yeah, I see. without lto or whole-program-mode, we cannot determine
> whether a extern function is address taken or not. Therefore, we have to
> treat ALL extern functions conservatively as address taken.
>
> So, from my understanding, the complete list that need to compute the typeid from the function prototype is:
>
> - At indirect call sites
> - all indirect call sites; (At the call site)
> - At function preambles
> - all address-taken static functions (At the function definition)
> - all extern functions (At function declaration or function definition?? Please see my question below)
For "extern functions", the logic is split as:
- "all extern function definitions get preamble"
- "all extern function declarations without a definition that are
address-taken get __kcfi_typeid_ symbol"
> > The other case is emitting the __ckfi_typeid_FUNC weak symbols, which is
> > used for link-time resolution with non-C code (i.e. raw .S assembly)
> > which doesn't have access to the C type system to calculate the hashes
> > on its own, and needs to have a way to build its own kcfi preambles.
>
> So, for such functions, there should be an extern function declaration in the C code.
> But the definition of such function is not available in the C code we are compiling.
> Therefore the weak __ckfi_typeid_FUNC symbol is emitted at the function declaration
> point for such function when we compile the C code?
>
> And the typeid (the hash value) for such routine is computed at the function declaration
> point too.
>
> Is the above understanding correct?
Correct, the kcfi_typeid symbol and value are emitted at function
declaration point, but only if such function is address-taken.
> Then for the other extern function whose definition is in the C code of other modules that might
> be compiled later, should the typeid is computed at the declaration or the definition?
It is computed and emitted just for externs that are address-taken.
> > This
> > is how Linux constructs its assembly function entry points:
> >
> > #ifndef __CFI_TYPE
> > #define __CFI_TYPE(name) \
> > .4byte __kcfi_typeid_##name
> > #endif
> >
> > #define SYM_TYPED_ENTRY(name, linkage, align...) \
> > linkage(name) ASM_NL \
> > align ASM_NL \
> > __CFI_TYPE(name) ASM_NL \
> > name:
> >
> > That way all the asm functions can be be indirect call targets without
> > knowing the hash value (which will be filled in at link time).
>
> Okay. I see. This is the case for the extern function whose definition is in the assembly file. (Not available in
> the C code)
Right, and sometimes we have to explicitly perform a no-op
address-taking to make sure a symbol gets generated:
/*
* Force the compiler to emit 'sym' as a symbol, so that we can reference
* it from inline assembler. Necessary in case 'sym' could be inlined
* otherwise, or eliminated entirely due to lack of references that are
* visible to the compiler.
*/
#define ___ADDRESSABLE(sym, __attrs) \
static void * __used __attrs \
__UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
#define __ADDRESSABLE(sym) \
___ADDRESSABLE(sym, __section(".discard.addressable"))
$ git grep KCFI_REFERENCE
include/linux/compiler.h:#define KCFI_REFERENCE(sym) __ADDRESSABLE(sym)
arch/x86/include/asm/page_64.h:KCFI_REFERENCE(copy_page);
arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memset);
arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memmove);
arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_prog_runX);
arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_callback_fn);
> > I assume I just didn't see how yet. :) I wasn't able to identify nor
> > store the typeid for function definitions that ultimately end up getting
> > .s file output.
> So, the problem only exists for the external functions whose definition is NOT in the C code?
Yup!
> > I couldn't figure out how to find these during the GIMPLE pass. Oh,
> > perhaps I can do this with an IPA pass? That should let me walk all
> > functions including externs. I'll give it a try...
Adding the IPA pass to find all functions worked perfectly. I was able
to remove all the weird DECL reconstruction and just use the original
FUNCTION_TYPE info for the typeids.
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-22 6:53 ` Richard Biener
@ 2025-08-22 19:23 ` Kees Cook
0 siblings, 0 replies; 42+ messages in thread
From: Kees Cook @ 2025-08-22 19:23 UTC (permalink / raw)
To: Richard Biener
Cc: Peter Zijlstra, Andrew Pinski, Qing Zhao, gcc-patches,
Joseph Myers, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li, linux-hardening
On Fri, Aug 22, 2025 at 08:53:27AM +0200, Richard Biener wrote:
> Thanks for the write-up. So I wonder whether a more general solution
> would be to detach "hash value" computation and assignment and leave
> that to external tooling that could do better than non-LTO can and
> the compiler working with attributed function types instead? Say
>
> void __attribute__((cfi_hash("voidvoidbutonlykind1"))) foo (void);
Yeah, having a way to choose different hashes was part of Joao's thesis.
Additionally, using a function attribute to salt a given function's hash
had been on the KCFI TODO list for a few years and just got implemented
in Clang last week:
https://clang.llvm.org/docs/AttributeReference.html#cfi-salt
I intend to get that implemented too, but I wanted to get the core KCFI
landed first.
> that decouples the way to arrive at the hashes from the instrumentation.
> You could then of course have compiler assisted auto-attribution
> based on function types. So implementation-wise I'd try to separate
> both? Oh, and somehow it reminds me of vtable verification?
>
> That said, how's the collision rate on the kernel side?
The salting (and Joao's work) are designed specifically for dealing with
collisions; you can see the frequency graph in here:
https://security.googleblog.com/2018/10/posted-by-sami-tolvanen-staff-software.html
In the Android Linux kernel, indirect calls:
- 55% have <= 5 targets
- 7% have > 100 targets
Breaking up that long 7% tail has been the goal with either LTO analysis
to split up the groups or manual annotation (i.e. hash salt).
> What's the usual exploitation of overwriting a call function pointer?
> That is, what's a good useful function to target? I'd guess a
> hypothetical setuid (int)? Somehow it feels like a more academic
> thing, with other attack vectors being much more accessible?
For forward edge, it's usually calling a series of useful targets. Calling
into functions to disable SMAP/SMEP, turn off SELinux, setuid, etc. Each
of these specific cases have been hardened over the years (no non-inline
MSR manipulation, SELinux permissive knob removed, credential creation
doesn't take a default arg, etc).
What we've seen after the addition of CFI (and other hardening) in Linux
was a greater push by attackers toward the less "low-hanging fruit" of
"data only" attacks (e.g. write to the page tables, etc). Jann Horn has
a particularly sobering write-up about this via memory allocators:
https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-22 19:02 ` Kees Cook
@ 2025-08-22 20:29 ` Qing Zhao
2025-08-22 22:29 ` Kees Cook
0 siblings, 1 reply; 42+ messages in thread
From: Qing Zhao @ 2025-08-22 20:29 UTC (permalink / raw)
To: Kees Cook
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
> On Aug 22, 2025, at 15:02, Kees Cook <kees@kernel.org> wrote:
>
> On Fri, Aug 22, 2025 at 03:11:16PM +0000, Qing Zhao wrote:
>>> On Aug 21, 2025, at 17:29, Kees Cook <kees@kernel.org> wrote:
>>> For non-static functions, we cannot know if other compilation units may
>>> make indirect calls to a given function, so those functions must always
>>> have their kcfi preamble added. For static functions, if they are
>>> address-taken by the current compilation unit, then they must get a kcfi
>>> preamble added.
>>
>> Oh, yeah, I see. without lto or whole-program-mode, we cannot determine
>> whether a extern function is address taken or not. Therefore, we have to
>> treat ALL extern functions conservatively as address taken.
>>
>> So, from my understanding, the complete list that need to compute the typeid from the function prototype is:
>>
>> - At indirect call sites
>> - all indirect call sites; (At the call site)
>> - At function preambles
>> - all address-taken static functions (At the function definition)
>> - all extern functions (At function declaration or function definition?? Please see my question below)
>
> For "extern functions", the logic is split as:
> - "all extern function definitions get preamble"
> - "all extern function declarations without a definition that are
> address-taken get __kcfi_typeid_ symbol"
Okay.
>
>>> The other case is emitting the __ckfi_typeid_FUNC weak symbols, which is
>>> used for link-time resolution with non-C code (i.e. raw .S assembly)
>>> which doesn't have access to the C type system to calculate the hashes
>>> on its own, and needs to have a way to build its own kcfi preambles.
>>
>> So, for such functions, there should be an extern function declaration in the C code.
>> But the definition of such function is not available in the C code we are compiling.
>> Therefore the weak __ckfi_typeid_FUNC symbol is emitted at the function declaration
>> point for such function when we compile the C code?
>>
>> And the typeid (the hash value) for such routine is computed at the function declaration
>> point too.
>>
>> Is the above understanding correct?
>
> Correct, the kcfi_typeid symbol and value are emitted at function
> declaration point, but only if such function is address-taken.
>
>> Then for the other extern function whose definition is in the C code of other modules that might
>> be compiled later, should the typeid is computed at the declaration or the definition?
>
> It is computed and emitted just for externs that are address-taken.
Okay.
>
>>> This
>>> is how Linux constructs its assembly function entry points:
>>>
>>> #ifndef __CFI_TYPE
>>> #define __CFI_TYPE(name) \
>>> .4byte __kcfi_typeid_##name
>>> #endif
>>>
>>> #define SYM_TYPED_ENTRY(name, linkage, align...) \
>>> linkage(name) ASM_NL \
>>> align ASM_NL \
>>> __CFI_TYPE(name) ASM_NL \
>>> name:
>>>
>>> That way all the asm functions can be be indirect call targets without
>>> knowing the hash value (which will be filled in at link time).
>>
>> Okay. I see. This is the case for the extern function whose definition is in the assembly file. (Not available in
>> the C code)
>
> Right, and sometimes we have to explicitly perform a no-op
> address-taking to make sure a symbol gets generated:
>
> /*
> * Force the compiler to emit 'sym' as a symbol, so that we can reference
> * it from inline assembler. Necessary in case 'sym' could be inlined
> * otherwise, or eliminated entirely due to lack of references that are
> * visible to the compiler.
> */
> #define ___ADDRESSABLE(sym, __attrs) \
> static void * __used __attrs \
> __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
>
> #define __ADDRESSABLE(sym) \
> ___ADDRESSABLE(sym, __section(".discard.addressable"))
>
> $ git grep KCFI_REFERENCE
> include/linux/compiler.h:#define KCFI_REFERENCE(sym) __ADDRESSABLE(sym)
> arch/x86/include/asm/page_64.h:KCFI_REFERENCE(copy_page);
> arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memset);
> arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memmove);
> arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_prog_runX);
> arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_callback_fn);
I am curious on why the compiler eliminates an external routine completely in the file if it's address-taken in that file.
Why an additional no-op address-taken is needed here.
>
>
>>> I assume I just didn't see how yet. :) I wasn't able to identify nor
>>> store the typeid for function definitions that ultimately end up getting
>>> .s file output.
>> So, the problem only exists for the external functions whose definition is NOT in the C code?
>
> Yup!
>
>>> I couldn't figure out how to find these during the GIMPLE pass. Oh,
>>> perhaps I can do this with an IPA pass? That should let me walk all
>>> functions including externs. I'll give it a try...
>
> Adding the IPA pass to find all functions worked perfectly. I was able
> to remove all the weird DECL reconstruction and just use the original
> FUNCTION_TYPE info for the typeids.
Nice. So this new IPA pass is invoked even when O0?
Qing
>
> -Kees
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-22 20:29 ` Qing Zhao
@ 2025-08-22 22:29 ` Kees Cook
2025-08-25 8:13 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-08-22 22:29 UTC (permalink / raw)
To: Qing Zhao
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
On Fri, Aug 22, 2025 at 08:29:16PM +0000, Qing Zhao wrote:
> > On Aug 22, 2025, at 15:02, Kees Cook <kees@kernel.org> wrote:
> > Right, and sometimes we have to explicitly perform a no-op
> > address-taking to make sure a symbol gets generated:
> >
> > /*
> > * Force the compiler to emit 'sym' as a symbol, so that we can reference
> > * it from inline assembler. Necessary in case 'sym' could be inlined
> > * otherwise, or eliminated entirely due to lack of references that are
> > * visible to the compiler.
> > */
> > #define ___ADDRESSABLE(sym, __attrs) \
> > static void * __used __attrs \
> > __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
> >
> > #define __ADDRESSABLE(sym) \
> > ___ADDRESSABLE(sym, __section(".discard.addressable"))
> >
> > $ git grep KCFI_REFERENCE
> > include/linux/compiler.h:#define KCFI_REFERENCE(sym) __ADDRESSABLE(sym)
> > arch/x86/include/asm/page_64.h:KCFI_REFERENCE(copy_page);
> > arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memset);
> > arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memmove);
> > arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_prog_runX);
> > arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_callback_fn);
>
> I am curious on why the compiler eliminates an external routine completely in the file if it's address-taken in that file.
> Why an additional no-op address-taken is needed here.
If I am remembering correctly this is needed for rare cases where
a function built without a C definition is being used in Linux's
self-patching "alternatives" code swaps in one function for another,
and is being used indirectly. These cases end up not being visible to
compiler (so no address-taken), but the indirect call site is still
being instrumented. And the above list is the _entire_ list of such
corner cases: all really low-level things.
Peter may remember this better than me...
> > Adding the IPA pass to find all functions worked perfectly. I was able
> > to remove all the weird DECL reconstruction and just use the original
> > FUNCTION_TYPE info for the typeids.
>
> Nice. So this new IPA pass is invoked even when O0?
It seems like it works at any optimization level, yes.
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-22 22:29 ` Kees Cook
@ 2025-08-25 8:13 ` Peter Zijlstra
2025-08-25 13:56 ` Qing Zhao
0 siblings, 1 reply; 42+ messages in thread
From: Peter Zijlstra @ 2025-08-25 8:13 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening@vger.kernel.org
On Fri, Aug 22, 2025 at 03:29:11PM -0700, Kees Cook wrote:
> On Fri, Aug 22, 2025 at 08:29:16PM +0000, Qing Zhao wrote:
> > > On Aug 22, 2025, at 15:02, Kees Cook <kees@kernel.org> wrote:
> > > Right, and sometimes we have to explicitly perform a no-op
> > > address-taking to make sure a symbol gets generated:
> > >
> > > /*
> > > * Force the compiler to emit 'sym' as a symbol, so that we can reference
> > > * it from inline assembler. Necessary in case 'sym' could be inlined
> > > * otherwise, or eliminated entirely due to lack of references that are
> > > * visible to the compiler.
> > > */
> > > #define ___ADDRESSABLE(sym, __attrs) \
> > > static void * __used __attrs \
> > > __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
> > >
> > > #define __ADDRESSABLE(sym) \
> > > ___ADDRESSABLE(sym, __section(".discard.addressable"))
> > >
> > > $ git grep KCFI_REFERENCE
> > > include/linux/compiler.h:#define KCFI_REFERENCE(sym) __ADDRESSABLE(sym)
> > > arch/x86/include/asm/page_64.h:KCFI_REFERENCE(copy_page);
> > > arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memset);
> > > arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memmove);
> > > arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_prog_runX);
> > > arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_callback_fn);
> >
> > I am curious on why the compiler eliminates an external routine completely in the file if it's address-taken in that file.
> > Why an additional no-op address-taken is needed here.
>
> If I am remembering correctly this is needed for rare cases where
> a function built without a C definition is being used in Linux's
> self-patching "alternatives" code swaps in one function for another,
> and is being used indirectly. These cases end up not being visible to
> compiler (so no address-taken), but the indirect call site is still
> being instrumented. And the above list is the _entire_ list of such
> corner cases: all really low-level things.
>
> Peter may remember this better than me...
The above are all functions from assembly and JITs, the C compiler
simply never sees the function definition, only the declaration. The
above is used to force emit the __typeid symbol, such that assembly can
reference it and it all links correctly.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API
2025-08-25 8:13 ` Peter Zijlstra
@ 2025-08-25 13:56 ` Qing Zhao
0 siblings, 0 replies; 42+ messages in thread
From: Qing Zhao @ 2025-08-25 13:56 UTC (permalink / raw)
To: Peter Zijlstra, Kees Cook
Cc: Andrew Pinski, gcc-patches@gcc.gnu.org, Joseph Myers,
Richard Biener, Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening@vger.kernel.org
> On Aug 25, 2025, at 04:13, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Fri, Aug 22, 2025 at 03:29:11PM -0700, Kees Cook wrote:
>> On Fri, Aug 22, 2025 at 08:29:16PM +0000, Qing Zhao wrote:
>>>> On Aug 22, 2025, at 15:02, Kees Cook <kees@kernel.org> wrote:
>>>> Right, and sometimes we have to explicitly perform a no-op
>>>> address-taking to make sure a symbol gets generated:
>>>>
>>>> /*
>>>> * Force the compiler to emit 'sym' as a symbol, so that we can reference
>>>> * it from inline assembler. Necessary in case 'sym' could be inlined
>>>> * otherwise, or eliminated entirely due to lack of references that are
>>>> * visible to the compiler.
>>>> */
>>>> #define ___ADDRESSABLE(sym, __attrs) \
>>>> static void * __used __attrs \
>>>> __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
>>>>
>>>> #define __ADDRESSABLE(sym) \
>>>> ___ADDRESSABLE(sym, __section(".discard.addressable"))
>>>>
>>>> $ git grep KCFI_REFERENCE
>>>> include/linux/compiler.h:#define KCFI_REFERENCE(sym) __ADDRESSABLE(sym)
>>>> arch/x86/include/asm/page_64.h:KCFI_REFERENCE(copy_page);
>>>> arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memset);
>>>> arch/x86/include/asm/string_64.h:KCFI_REFERENCE(__memmove);
>>>> arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_prog_runX);
>>>> arch/x86/kernel/alternative.c:KCFI_REFERENCE(__bpf_callback_fn);
>>>
>>> I am curious on why the compiler eliminates an external routine completely in the file if it's address-taken in that file.
>>> Why an additional no-op address-taken is needed here.
>>
>> If I am remembering correctly this is needed for rare cases where
>> a function built without a C definition is being used in Linux's
>> self-patching "alternatives" code swaps in one function for another,
>> and is being used indirectly. These cases end up not being visible to
>> compiler (so no address-taken), but the indirect call site is still
>> being instrumented. And the above list is the _entire_ list of such
>> corner cases: all really low-level things.
>>
>> Peter may remember this better than me...
>
> The above are all functions from assembly and JITs, the C compiler
> simply never sees the function definition, only the declaration. The
> above is used to force emit the __typeid symbol, such that assembly can
> reference it and it all links correctly.
Okay, I see. Thanks for the information.
Qing
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-21 7:26 ` [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
[not found] ` <CALvbMcA+8iHo+zCCvs4UdAg9PVQVtgOno-rtMS4i5YajrjkyGw@mail.gmail.com>
@ 2025-08-28 14:57 ` Qing Zhao
2025-09-04 4:24 ` Kees Cook
1 sibling, 1 reply; 42+ messages in thread
From: Qing Zhao @ 2025-08-28 14:57 UTC (permalink / raw)
To: Kees Cook
Cc: gcc-patches@gcc.gnu.org, Joseph Myers, Richard Biener,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
Hi, Kees,
I have several suggestions and questions first for this patch:
1. Is -fsanitize=kcfi a C only feature? If so, you might need to mention this in
the documentation and also reject its usage in other languages.
2. There is no overall description of the design of this kcfi implementation
in the source code, I think it might be very helpful to provide such description
In the new module kcfi.cc <http://kcfi.cc/>. such documentation will be helpful for current
review and future maintenance of this feature.
> On Aug 21, 2025, at 03:26, Kees Cook <kees@kernel.org> wrote:
>
> This series implements the Linux Kernel Control Flow Integrity ABI,
> which provides a function prototype based forward edge control flow
> integrity protection by instrumenting every indirect call to check for
> a hash value before the target function address. If the hash at the call
> site and the hash at the target do not match, execution will trap.
>
> Just to set expectations, this is an RFC because this is my first time
> working on most of the affected areas in GCC, and it is likely I have
> missed really obvious stuff, or gone about doing things in very wrong
> ways. I tried to find the best way to do stuff, but I was left with many
> questions. :) All that said, this works for x86_64 and aarch64 Linux
> kernels. (I have implemented riscv64 as well, but I lack a viable test
> environment -- I am working on this still.)
>
> KCFI has a number of specific constraints. Some are tied to the
> backend architecture, which I'll cover in more detail in later patches.
> The constraints are:
>
> - The KCFI scheme generates a unique 32-bit hash for each unique function
> prototype, allowing for indirect call sites to verify that they are
> calling into a matching _type_ of function pointer. This changes the
> semantics of some optimization logic because now indirect calls to
> different types cannot be merged. For example:
>
> if (p->func_type_1)
> return p->func_type_1();
> if (p->func_type_2)
> return p->func_type_2();
>
> In final asm, the optimizer may collapse the second indirect call
> into a jump to the first indirect call once it has loaded the function
> pointer. KCFI must block cross-type merging otherwise there will be a
> single KCFI check happening for only 1 type but being used by 2 target
> types. The distinguishing characteristic for call merging becomes the
> type, not the address/register usage.
>
> - The check-call instruction sequence must be treated a single unit: it
> cannot be rearranged or split or optimized. The pattern is that
> indirect calls, "call *$target", get converted into:
>
> mov $target_expression, %target (only present if the expression was
> not already %target)
> load -$offset(%target), %tmp
> cmp $hash, %tmp
> je .Lcheck_passed
> .Ltrap$N:
> trap
> .Lcheck_passed$N:
> call *%target
>
> This pattern of call immediately after trap provides for the
> "permissive" checking mode automatically: the trap gets handled,
> a warning emitted, and then execution continues after the trap to
> the call.
>
> (x86_64 uses "mov -$hash, %tmp; addl -$offset(%target), %tmp; je"
> to zero out the register before making the call. Also Linux needs
> exactly these insns because it is both disassembling them during
> trap handling and potentially live patching them at boot time to
> be converted into a different series of instrutions, a scheme
> know as FineIBT, making the insn sequence ABI.)
>
> - KCFI check-call instrumentation must survive tail call optimization.
> If an indirect call is turned into an indirect jump, KCFI checking
> must still happen (but will still use the jmp).
>
> - Functions that may be called indirectly have a preamble added,
> __cfi_$original_func_name, that contains the $hash value:
>
> __cfi_target_func:
> .word $hash
> target_func:
> [regular function entry...]
>
> (x86_64 uses a movl instruction to hold the hash and prefixed aligned
> NOPs to maintain cache line alignment in the face of patchable function
> entry...)
>
> - The preamble needs to interact with patchable function entry so that
> the hash appears further away from the actual start of the function
> (leaving the prefix NOPs of the patchable function entry unchanged).
> This means only _globally defined_ patchable function entry is supported
> with KCFI (indrect call sites must know in advance what the offset is,
> which may not be possible extern functions). For example, a "4,4"
> patchable function entry would end up like:
>
> __cfi_target_func:
> .data $hash
> nop nop nop nop
> target_func:
> [regular function entry...]
>
> (Linux x86_64 uses an 11 byte prefix nop area resulting in 16 bytes
> total including the movl. This region may be live patched at boot time
> for FineIBT so the behavior here is also ABI.)
>
> - External functions that are address-taken have a weak __kcfi_typeid_$funcname
> symbol added with the hash value available so that the hash can be referenced
> from assembly linkages, etc, where the hash values cannot be calculated (i.e
> where C type information is missing):
>
> .weak __kcfi_typeid_$func
> .set __kcfi_typeid_$func, $hash
>
> - On architectures that do not have a good way to encode additional
> details in their trap (x86_64 and riscv64), the trap location
> is identified as a KCFI trap via a relative address offset entry
> emitted into the .kcfi_traps section for each indirect call site's
> trap instruction. The previous check-call example's insn sequence has
> a section push/pop inserted between the trap and call:
>
> ...
> .Ltrap$N:
> trap
> .pushsection .kcfi_traps,"ao",@progbits,.text
> .Lentry$N:
> .long .Ltrap$N - .Lentry$N
> .popsection
> .Lcheck_passed$N:
> call %target
>
> (aarch64 encodes the register numbers that hold the expected hash
> and the target address in the trap ESR and thereby does not need a
> .kcfi_traps section at all.)
>
> - The no_sanitize("kcfi") function attribute means that the marked function
> must not produce KCFI checking for indirect calls, and that this
> attribute must survive inlining. This is used rarely by Linux, but
> is required to make BPF JIT trampolines work on older Linux kernel
> versions. (The preamble code is very recently finally being generated
> at JIT time on the last remaining Linux KCFI arch where this was
> missing: aarch64.)
>
> As a result of these constraints, there are some behavioral aspects
> that need to be preserved across the middle-end and back-end, as I
> understand them.
>
> For indirect call sites:
>
> - Keeping indirect calls from being merged (see above). I did this by
> adding a wrapping type so that equality was tested based on type-id.
> This is done in create_kcfi_wrapper_type(), via kcfi_instrument(),
> via an early GIMPLE pass (pass_kcfi and pass_kcfi0). The wrapper type
> is checked in gcc/cfgcleanup.cc, old_insns_match_p().
First, Looks like that the routine “old_insns_match_p” checks the type-ids that were embedded
in the REG_NOTE ( REG_CALL_KCFI_TYPE) of the RTL. -:)
My understanding is: the computed type-id is recorded into to a “kcfi_type_id” attribute, and
then this “kcfi_type_id” attribute is attached to the original function type.
At the same time, A wrapper type is created for the original function type, whose typename is
“__kcfi_wrapper_type_id”.
I am confused:
1. Why the additional wrapper type is needed? why the original function type + “kcfi_type_id” not enough?
2. Why attaching the type-id as a “kcfi_type_id” attribute to the original function type?
Is there other way to attach the type-id to the original function type? Or to the original indirect call site?
Is it possible that we might provide a user level “kcfi_type_id” attribute to function type in the future? then
That will be conflict with the current “kcfi_type_id” attribute?
>
> - Keeping typeid information available through to the RTL expansion
> phase was done via a typeid note (REG_CALL_KCFI_TYPE) attached also
> in create_kcfi_wrapper_type() in the same GIMPLE pass.
This is done in the routine “expand_call_stmt” through call to “add_kcfi_type_note”.
The “type_id” is attached as a “kcfi_type_id” attribute to the original function type in the routine
“create_kcfi_wrapper_type”.
Please see my above question 2.
> - REG_CALL_KCFI_TYPE notes needed to survive optimization passes so RTL
> expansion could find them again. These are retained in gcc/emit-rtl.cc,
> try_split(), and gcc/combine.cc, distribute_notes().
Okay.
>
> - The expansion to RTL is handled in gcc/cfgexpand.cc, expand_call_stmt().
> This lets the call expansion logic run, but then marks the resulting
> call RTL with REG_CALL_KCFI_TYPE so the last RTL instrumentation pass
> can find it. To me, this logic feels the most weird. It seems like
> there should be a better place to do this, but I do see that similar
> behavioral needs are also here, like nocf_check, so perhaps it's not
> far off.
I think this should be okay.
>
> - The same expand_call_stmt() logic also adds the clobbers that will be
> present in the final KCFI check-call insn sequences, which need to be
> added now so that register allocation is aware of them while working
> through optimization passes. Without this, the clobbered registers
> may get used in the RTL before we have replaced the call instructions
> that actually have the clobbers associated with them. This is the
> "have cake and eat it too" case where we have to do the check-call RTL
> replacement very late, but need to make sure the clobbers are known
> very early. Again, this seems like there should be a better way...
>
> - To emit the exact KCFI check-call instruction sequences, a very late
> RTL pass is used (pass_kcfi_final_instrumentation). In order to
> handle sibling calls (tail calls), this pass chooses between either
> a check-call insn sequence or a check-jump insn sequence (provided by
> the per-arch back-end).
>
> - To make sure instrumentation was skipped for inline functions, the
> RTL pass walks the basic blocks to identify their function origins,
> looking for the no_sanitize("kcfi") attribute, and skipping
> instrumentation if found.
I think this was done inefficiently. It’s better to do this during inlining transformation to
propagate the no_sanitize information from the caller to the callee body and attach
the no_sanitize information to the indirect callsite.
>
> For indirect call targets:
>
> - kcfi_emit_preamble_if_needed() uses function_needs_kcfi_preamble(),
> and counter helpers, to emit the preablem, with patchable function
> entry NOPs. This gets used both in default_print_patchable_function_entry()
> and the per-arch path.
I saw that “kcfi_emit_preamble_if_needed” is called by “default_print_patchable_function_entry”,
But I didn’t see where it is called when the function is not patchable?
> I could not find a simpler way to deal with
> patchable function entry besides splitting it up like this. I feel
> like there should be a better way.
>
> - gcc/varasm.cc, assemble_external_real() calls emit_kcfi_typeid_symbol()
> to add the __kcfi_typeid symbols (see get_function_kcfi_type_id()
> below).
>
> To support the per-arch back-ends, there are some common helpers:
>
> - A callback framework is added via struct kcfi_target_hooks for
> backends to fill out.
>
> - kcfi_emit_trap_with_section() handles the push/pop section and
> generating the relative offset section entries.
>
> - get_function_kcfi_type_id() generates the 32-bit hash value, using
> compute_kcfi_type_id() and kcfi_hash_string() to hook to the mangling
> API. The hash is FNV-1a right now: it doesn't need secrecy. It could be
> replaced with any hash, though the hash will need to be coordinated
> with Rust, which implements the KCFI ABI as well.
One question here, the final type_id will be computed by the following 2 steps:
1. Itanium C++ mangling based on return type + parameter types of the function.
2. FNV-1a hash on the mangled string
If the hacker knows these, it should be quite easy for them to come up with a
matched typeid, is it?
Thanks.
Qing
>
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
> gcc/Makefile.in | 1 +
> gcc/flag-types.h | 2 +
> gcc/kcfi.h | 85 +++++
> gcc/tree-pass.h | 1 +
> gcc/cfgcleanup.cc | 20 ++
> gcc/cfgexpand.cc | 61 ++++
> gcc/combine.cc | 1 +
> gcc/doc/invoke.texi | 29 ++
> gcc/emit-rtl.cc | 1 +
> gcc/kcfi.cc | 783 ++++++++++++++++++++++++++++++++++++++++++++
> gcc/opts.cc | 1 +
> gcc/passes.cc | 1 +
> gcc/passes.def | 3 +
> gcc/recog.cc | 1 +
> gcc/reg-notes.def | 6 +
> gcc/targhooks.cc | 50 ++-
> gcc/toplev.cc | 8 +
> gcc/varasm.cc | 18 +
> 18 files changed, 1071 insertions(+), 1 deletion(-)
> create mode 100644 gcc/kcfi.h
> create mode 100644 gcc/kcfi.cc
>
> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
> index 86f62611c1d4..01cccd88fde0 100644
> --- a/gcc/Makefile.in
> +++ b/gcc/Makefile.in
> @@ -1593,6 +1593,7 @@ OBJS = \
> ira-emit.o \
> ira-lives.o \
> jump.o \
> + kcfi.o \
> langhooks.o \
> late-combine.o \
> lcm.o \
> diff --git a/gcc/flag-types.h b/gcc/flag-types.h
> index 33c88a15ecbb..7ed6dab1bd4b 100644
> --- a/gcc/flag-types.h
> +++ b/gcc/flag-types.h
> @@ -337,6 +337,8 @@ enum sanitize_code {
> SANITIZE_KERNEL_HWADDRESS = 1ULL << 30,
> /* Shadow Call Stack. */
> SANITIZE_SHADOW_CALL_STACK = 1ULL << 31,
> + /* KCFI (Kernel Control Flow Integrity) */
> + SANITIZE_KCFI = 1ULL << 32,
> SANITIZE_SHIFT = SANITIZE_SHIFT_BASE | SANITIZE_SHIFT_EXPONENT,
> SANITIZE_UNDEFINED = SANITIZE_SHIFT | SANITIZE_DIVIDE | SANITIZE_UNREACHABLE
> | SANITIZE_VLA | SANITIZE_NULL | SANITIZE_RETURN
> diff --git a/gcc/kcfi.h b/gcc/kcfi.h
> new file mode 100644
> index 000000000000..23d679b73ba4
> --- /dev/null
> +++ b/gcc/kcfi.h
> @@ -0,0 +1,85 @@
> +/* Kernel Control Flow Integrity (KCFI) support for GCC.
> + Copyright (C) 2025 Free Software Foundation, Inc.
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify it under
> +the terms of the GNU General Public License as published by the Free
> +Software Foundation; either version 3, or (at your option) any later
> +version.
> +
> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3. If not see
> +<http://www.gnu.org/licenses/>. */
> +
> +#ifndef GCC_KCFI_H
> +#define GCC_KCFI_H
> +
> +#include "config.h"
> +#include "system.h"
> +#include "coretypes.h"
> +#include "rtl.h"
> +
> +/* Get KCFI type ID for a function declaration. */
> +extern uint32_t get_function_kcfi_type_id (tree fndecl);
> +
> +/* KCFI target hooks for architecture-specific functionality. */
> +
> +struct kcfi_target_hooks {
> + /* Apply architecture-specific masking to type ID. */
> + uint32_t (*mask_type_id) (uint32_t type_id);
> +
> + /* Generate bundled KCFI checked call (atomic check + call to prevent optimizer separation) */
> + rtx (*gen_kcfi_checked_call) (rtx call_insn, rtx target_reg, uint32_t expected_type, HOST_WIDE_INT prefix_nops);
> +
> + /* Add architecture-specific register clobbers for KCFI calls. */
> + void (*add_kcfi_clobbers) (rtx_insn *call_insn);
> +
> + /* Calculate architecture-specific prefix NOPs count (optional, returns prefix_nops unchanged if NULL) */
> + int (*calculate_prefix_nops) (HOST_WIDE_INT prefix_nops);
> +
> + /* Emit architecture-specific type ID instruction (required for common preamble helper) */
> + void (*emit_type_id_instruction) (FILE *file, uint32_t type_id);
> +};
> +
> +/* Global KCFI target hooks. */
> +extern struct kcfi_target_hooks kcfi_target;
> +
> +/* Common helper for RTL patterns to emit .kcfi_traps section entry.
> + Call AFTER emitting trap label and instruction with the RTX label operand. */
> +extern void kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx);
> +
> +/* RTL note management for KCFI. */
> +
> +/* Add KCFI type ID note to call instruction. */
> +extern void add_kcfi_type_note (rtx_insn *call_insn, uint32_t type_id);
> +
> +/* Emit KCFI type ID symbol for address-taken functions. */
> +extern void emit_kcfi_typeid_symbol (FILE *asm_file, tree decl, const char *name);
> +
> +/* KCFI preamble emission coordination. */
> +
> +/* Mark that KCFI preamble has been emitted to prevent duplication. */
> +extern void mark_kcfi_preamble_emitted (void);
> +
> +/* Central manager for all KCFI preamble generation decisions. */
> +extern void kcfi_emit_preamble_if_needed (FILE *file, tree decl,
> + bool is_patchable_context,
> + HOST_WIDE_INT prefix_nops,
> + const char *actual_fname);
> +
> +/* Pass creation functions. */
> +class gimple_opt_pass;
> +class rtl_opt_pass;
> +namespace gcc { class context; }
> +
> +extern gimple_opt_pass *make_pass_kcfi (gcc::context *ctxt);
> +extern gimple_opt_pass *make_pass_kcfi_O0 (gcc::context *ctxt);
> +extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context *ctxt);
> +
> +#endif /* GCC_KCFI_H */
> diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
> index 1c68a69350df..2affea230213 100644
> --- a/gcc/tree-pass.h
> +++ b/gcc/tree-pass.h
> @@ -561,6 +561,7 @@ extern gimple_opt_pass *make_pass_fixup_cfg (gcc::context *ctxt);
> extern gimple_opt_pass *make_pass_backprop (gcc::context *ctxt);
>
> extern rtl_opt_pass *make_pass_expand (gcc::context *ctxt);
> +extern rtl_opt_pass *make_pass_kcfi_final_instrumentation (gcc::context *ctxt);
> extern rtl_opt_pass *make_pass_instantiate_virtual_regs (gcc::context *ctxt);
> extern rtl_opt_pass *make_pass_rtl_fwprop (gcc::context *ctxt);
> extern rtl_opt_pass *make_pass_rtl_fwprop_addr (gcc::context *ctxt);
> diff --git a/gcc/cfgcleanup.cc b/gcc/cfgcleanup.cc
> index d28d23231911..9707b7e22222 100644
> --- a/gcc/cfgcleanup.cc
> +++ b/gcc/cfgcleanup.cc
> @@ -1238,6 +1238,26 @@ old_insns_match_p (int mode ATTRIBUTE_UNUSED, rtx_insn *i1, rtx_insn *i2)
> if (RTX_FRAME_RELATED_P (i1) && !insns_have_identical_cfa_notes (i1, i2))
> return dir_none;
>
> + /* KCFI (Kernel Control Flow Integrity): Do not cross-jump between different
> + KCFI check patterns. Each bundled KCFI call has a unique type ID that must
> + be preserved to prevent type confusion attacks. */
> + if (CALL_P (i1) && CALL_P (i2))
> + {
> + rtx kcfi_note1 = find_reg_note (i1, REG_CALL_KCFI_TYPE, NULL_RTX);
> + rtx kcfi_note2 = find_reg_note (i2, REG_CALL_KCFI_TYPE, NULL_RTX);
> +
> + if (kcfi_note1 || kcfi_note2)
> + {
> + /* If only one has KCFI note, they're different. */
> + if (!kcfi_note1 || !kcfi_note2)
> + return dir_none;
> +
> + /* If both have KCFI notes, compare the type IDs. */
> + if (!rtx_equal_p (XEXP (kcfi_note1, 0), XEXP (kcfi_note2, 0)))
> + return dir_none;
> + }
> + }
> +
> #ifdef STACK_REGS
> /* If cross_jump_death_matters is not 0, the insn's mode
> indicates whether or not the insn contains any stack-like
> diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc
> index 8950294abb60..432ccc7f864a 100644
> --- a/gcc/cfgexpand.cc
> +++ b/gcc/cfgexpand.cc
> @@ -76,6 +76,7 @@ along with GCC; see the file COPYING3. If not see
> #include "opts.h"
> #include "gimple-range.h"
> #include "rtl-iter.h"
> +#include "kcfi.h"
>
> /* Some systems use __main in a way incompatible with its use in gcc, in these
> cases use the macros NAME__MAIN to give a quoted symbol and SYMBOL__MAIN to
> @@ -3203,6 +3204,66 @@ expand_call_stmt (gcall *stmt)
> else
> expand_expr (exp, const0_rtx, VOIDmode, EXPAND_NORMAL);
>
> + /* Add KCFI annotations if this is an indirect call with KCFI wrapper type. */
> + if (sanitize_flags_p (SANITIZE_KCFI) && !gimple_call_fndecl (stmt))
> + {
> + tree fn_type = gimple_call_fntype (stmt);
> + gcc_assert (fn_type);
> + tree attr = lookup_attribute ("kcfi_type_id", TYPE_ATTRIBUTES (fn_type));
> + if (attr && TREE_VALUE (attr))
> + {
> + /* Check if this call site originated from a no_sanitize("kcfi") function
> + during inlining. If so, skip KCFI instrumentation in RTL phase too. */
> + bool should_skip_kcfi = false;
> + location_t call_location = gimple_location (stmt);
> + gcc_assert (call_location != UNKNOWN_LOCATION);
> +
> + tree block = gimple_block (stmt);
> + while (block && TREE_CODE (block) == BLOCK)
> + {
> + tree fn_decl = BLOCK_ABSTRACT_ORIGIN (block);
> + if (fn_decl && TREE_CODE (fn_decl) == FUNCTION_DECL)
> + {
> + /* Found an inlined function - check if it has no_sanitize("kcfi"). */
> + if (!sanitize_flags_p (SANITIZE_KCFI, fn_decl))
> + {
> + should_skip_kcfi = true;
> + break;
> + }
> + break;
> + }
> + /* Move up the block chain to find parent inlined functions. */
> + block = BLOCK_SUPERCONTEXT (block);
> + }
> +
> + if (!should_skip_kcfi)
> + {
> + uint32_t kcfi_type_id = (uint32_t) tree_to_uhwi (TREE_VALUE (attr));
> +
> + /* Find the call that has been created. */
> + rtx_insn *call_insn = get_last_insn ();
> + while (call_insn && call_insn != before_call && !CALL_P (call_insn))
> + call_insn = PREV_INSN (call_insn);
> +
> + if (call_insn && call_insn != before_call && CALL_P (call_insn))
> + {
> + /* Add KCFI type ID note for anti-merging protection. */
> + add_kcfi_type_note (call_insn, kcfi_type_id);
> +
> + /* Add architecture-specific clobbers so register allocator knows
> + they'll be used. */
> + if (kcfi_target.add_kcfi_clobbers)
> + kcfi_target.add_kcfi_clobbers (call_insn);
> + }
> + else
> + {
> + error ("KCFI: Could not find call instruction for wrapper type");
> + gcc_unreachable ();
> + }
> + }
> + }
> + }
> +
> /* If the gimple call is an indirect call and has 'nocf_check'
> attribute find a generated CALL insn to mark it as no
> control-flow verification is needed. */
> diff --git a/gcc/combine.cc b/gcc/combine.cc
> index 4dbc1f6a4a4e..efaeb426d254 100644
> --- a/gcc/combine.cc
> +++ b/gcc/combine.cc
> @@ -14525,6 +14525,7 @@ distribute_notes (rtx notes, rtx_insn *from_insn, rtx_insn *i3, rtx_insn *i2,
> case REG_CALL_DECL:
> case REG_UNTYPED_CALL:
> case REG_CALL_NOCF_CHECK:
> + case REG_CALL_KCFI_TYPE:
> /* These notes must remain with the call. It should not be
> possible for both I2 and I3 to be a call. */
> if (CALL_P (i3))
> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index c1e708beacf3..c66f47336826 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -18294,6 +18294,35 @@ possible by specifying the command-line options
> @option{--param hwasan-instrument-allocas=1} respectively. Using a random frame
> tag is not implemented for kernel instrumentation.
>
> +@opindex fsanitize=kcfi
> +@item -fsanitize=kcfi
> +Enable Kernel Control Flow Integrity (KCFI), a lightweight control
> +flow integrity mechanism designed for operating system kernels.
> +KCFI instruments indirect function calls to verify that the target
> +function has the expected type signature at runtime. Each function
> +receives a unique type identifier computed from a hash of its function
> +prototype (including parameter types and return type). Before each
> +indirect call, the implementation inserts a check to verify that the
> +target function's type identifier matches the expected identifier
> +for the call site, terminating the program if a mismatch is detected.
> +This provides forward-edge control flow protection against attacks that
> +attempt to redirect indirect calls to unintended targets.
> +
> +The implementation adds minimal runtime overhead and does not require
> +runtime library support, making it suitable for kernel environments.
> +The type identifier is placed before the function entry point,
> +allowing runtime verification without additional metadata structures,
> +and without changing the entry points of the target functions. Only
> +functions that have referenced by their address receive the KCFI preamble
> +instrumentation.
> +
> +KCFI is intended primarily for kernel code and may not be suitable
> +for user-space applications that rely on techniques incompatible
> +with strict type checking of indirect calls.
> +
> +Use @option{-fdump-tree-kcfi} to examine the computed type identifiers
> +and their corresponding mangled type strings during compilation.
> +
> @opindex fsanitize=pointer-compare
> @item -fsanitize=pointer-compare
> Instrument comparison operation (<, <=, >, >=) with pointer operands.
> diff --git a/gcc/emit-rtl.cc b/gcc/emit-rtl.cc
> index f4fc92bb37a1..aa56112bc250 100644
> --- a/gcc/emit-rtl.cc
> +++ b/gcc/emit-rtl.cc
> @@ -4056,6 +4056,7 @@ try_split (rtx pat, rtx_insn *trial, int last)
> case REG_SETJMP:
> case REG_TM:
> case REG_CALL_NOCF_CHECK:
> + case REG_CALL_KCFI_TYPE:
> case REG_CALL_ARG_LOCATION:
> for (insn = insn_last; insn != NULL_RTX; insn = PREV_INSN (insn))
> {
> diff --git a/gcc/kcfi.cc b/gcc/kcfi.cc
> new file mode 100644
> index 000000000000..522bb97bf503
> --- /dev/null
> +++ b/gcc/kcfi.cc
> @@ -0,0 +1,783 @@
> +/* Kernel Control Flow Integrity (KCFI) support for GCC.
> + Copyright (C) 2025 Free Software Foundation, Inc.
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify it under
> +the terms of the GNU General Public License as published by the Free
> +Software Foundation; either version 3, or (at your option) any later
> +version.
> +
> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3. If not see
> +<http://www.gnu.org/licenses/>. */
> +
> +#include "config.h"
> +#include "system.h"
> +#include "coretypes.h"
> +#include "target.h"
> +#include "function.h"
> +#include "tree.h"
> +#include "tree-pass.h"
> +#include "dumpfile.h"
> +#include "basic-block.h"
> +#include "gimple.h"
> +#include "gimple-iterator.h"
> +#include "cgraph.h"
> +#include "kcfi.h"
> +#include "stringpool.h"
> +#include "attribs.h"
> +#include "rtl.h"
> +#include "cfg.h"
> +#include "asan.h"
> +#include "diagnostic-core.h"
> +#include "memmodel.h"
> +#include "emit-rtl.h"
> +#include "output.h"
> +#include "varasm.h"
> +#include "opts.h"
> +#include "mangle.h"
> +
> +/* Global KCFI target hooks structure - zero-initialized for safe defaults. */
> +struct kcfi_target_hooks kcfi_target = { };
> +
> +/* Common KCFI utilities. */
> +
> +/* Common helper for RTL patterns to emit .kcfi_traps section entry. */
> +void
> +kcfi_emit_trap_with_section (FILE *file, rtx trap_label_rtx)
> +{
> + /* Convert trap label to string using standard GCC helper. */
> + char trap_name[64];
> + ASM_GENERATE_INTERNAL_LABEL (trap_name, "L", CODE_LABEL_NUMBER (trap_label_rtx));
> +
> + /* Generate entry label name from trap label number. */
> + char entry_name[64];
> + ASM_GENERATE_INTERNAL_LABEL (entry_name, "Lentry", CODE_LABEL_NUMBER (trap_label_rtx));
> +
> + /* Emit .kcfi_traps section entry using the converted labels. */
> + fprintf (file, "\t.pushsection\t.kcfi_traps,\"ao\",@progbits,.text\n");
> + assemble_name (file, entry_name);
> + fprintf (file, ":\n");
> + fprintf (file, "\t.long\t");
> + assemble_name (file, trap_name);
> + fprintf (file, " - ");
> + assemble_name (file, entry_name);
> + fprintf (file, "\n");
> + fprintf (file, "\t.popsection\n");
> +}
> +
> +/* Hash function for KCFI type ID computation.
> + This implements a simple hash similar to FNV-1a. */
> +static uint32_t
> +kcfi_hash_string (const char *str)
> +{
> + uint32_t hash = 2166136261U; /* FNV-1a 32-bit offset basis. */
> + for (const char *p = str; *p; p++)
> + {
> + hash ^= (unsigned char) *p;
> + hash *= 16777619U; /* FNV-1a 32-bit prime. */
> + }
> + return hash;
> +}
> +
> +/* Compute KCFI type ID for a function declaration or function type (internal) */
> +static uint32_t
> +compute_kcfi_type_id (tree fntype_or_fndecl)
> +{
> + if (!fntype_or_fndecl)
> + return 0;
> +
> + const char *canonical_name = mangle_function_type (fntype_or_fndecl);
> + uint32_t base_type_id = kcfi_hash_string (canonical_name);
> +
> + /* Apply target-specific masking if supported. */
> + if (kcfi_target.mask_type_id)
> + base_type_id = kcfi_target.mask_type_id (base_type_id);
> +
> + /* Output to dump file if enabled */
> + if (dump_file && (dump_flags & TDF_DETAILS))
> + {
> + const char *name = NULL;
> + if (TREE_CODE (fntype_or_fndecl) == FUNCTION_DECL)
> + {
> + if (DECL_NAME (fntype_or_fndecl))
> + name = IDENTIFIER_POINTER (DECL_NAME (fntype_or_fndecl));
> + }
> +
> + if (name)
> + fprintf (dump_file, "KCFI type ID for function '%s': mangled='%s' typeid=0x%08x\n",
> + name, canonical_name, base_type_id);
> + else
> + fprintf (dump_file, "KCFI type ID: mangled='%s' typeid=0x%08x\n",
> + canonical_name, base_type_id);
> + }
> +
> + return base_type_id;
> +}
> +
> +/* Check if a function needs KCFI preamble generation.
> + ALL functions get preambles when -fsanitize=kcfi is enabled, regardless
> + of no_sanitize("kcfi") attribute. */
> +static bool
> +function_needs_kcfi_preamble (tree fndecl)
> +{
> + /* Only instrument if KCFI is globally enabled. */
> + if (!(flag_sanitize & SANITIZE_KCFI))
> + return false;
> +
> + struct cgraph_node *node = cgraph_node::get (fndecl);
> +
> + /* Ignore cold partition functions: not reached via indirect call. */
> + if (node && node->split_part)
> + return false;
> +
> + /* Ignore cold partition sections: cold partitions are never indirect call
> + targets. Only skip preambles for cold partitions (has_bb_partition = true)
> + not for entire cold-attributed functions (has_bb_partition = false). */
> + if (in_cold_section_p && crtl && crtl->has_bb_partition)
> + return false;
> +
> + /* Check if function is truly address-taken using cgraph node analysis. */
> + bool addr_taken = (node && node->address_taken);
> +
> + /* Only instrument functions that can be targets of indirect calls:
> + - Public functions (can be called externally)
> + - External declarations (from other modules)
> + - Functions with true address-taken status from cgraph analysis. */
> + return TREE_PUBLIC (fndecl) || DECL_EXTERNAL (fndecl) || addr_taken;
> +}
> +
> +/* Function attribute to store KCFI type ID. */
> +static tree kcfi_type_id_attr = NULL_TREE;
> +
> +/* Get KCFI type ID for a function declaration. */
> +uint32_t
> +get_function_kcfi_type_id (tree fndecl)
> +{
> + if (!kcfi_type_id_attr)
> + kcfi_type_id_attr = get_identifier ("kcfi_type_id");
> +
> + tree attr = lookup_attribute_by_prefix ("kcfi_type_id", DECL_ATTRIBUTES (fndecl));
> + if (attr && TREE_VALUE (attr) && TREE_VALUE (TREE_VALUE (attr)))
> + {
> + tree value = TREE_VALUE (TREE_VALUE (attr));
> + if (TREE_CODE (value) == INTEGER_CST)
> + return (uint32_t) TREE_INT_CST_LOW (value);
> + }
> +
> + /* Compute and cache type ID using original parameter declarations. */
> + uint32_t type_id = compute_kcfi_type_id (fndecl);
> +
> + tree type_id_tree = build_int_cst (unsigned_type_node, type_id);
> + tree attr_value = build_tree_list (NULL_TREE, type_id_tree);
> + attr = build_tree_list (kcfi_type_id_attr, attr_value);
> +
> + DECL_ATTRIBUTES (fndecl) = chainon (DECL_ATTRIBUTES (fndecl), attr);
> +
> + return type_id;
> +}
> +
> +/* Get the number of patchable prefix NOPs for the current function. */
> +static HOST_WIDE_INT
> +get_current_function_patchable_prefix_nops (void)
> +{
> + HOST_WIDE_INT prefix_nops = 0;
> +
> + /* Check for function-specific patchable_function_entry attribute. */
> + tree patchable_attr = lookup_attribute ("patchable_function_entry",
> + DECL_ATTRIBUTES (current_function_decl));
> + if (patchable_attr)
> + {
> + tree pp_val = TREE_VALUE (patchable_attr);
> + /* total_nops = tree_to_uhwi (TREE_VALUE (pp_val)); */
> + if (TREE_CHAIN (pp_val))
> + prefix_nops = tree_to_uhwi (TREE_VALUE (TREE_CHAIN (pp_val)));
> + }
> + else
> + {
> + /* Use global configuration if no function-specific attribute. */
> + HOST_WIDE_INT total_nops, patch_area_entry;
> + parse_and_check_patch_area (flag_patchable_function_entry, false,
> + &total_nops, &patch_area_entry);
> + prefix_nops = patch_area_entry;
> + }
> +
> + return prefix_nops;
> +}
> +
> +/* Check if this is an indirect call that needs KCFI instrumentation. */
> +static bool
> +is_kcfi_indirect_call (tree fn)
> +{
> + if (!fn)
> + return false;
> +
> + /* Only functions WITHOUT no_sanitize("kcfi") should generate KCFI checks at
> + indirect call sites. */
> + if (!sanitize_flags_p (SANITIZE_KCFI, current_function_decl))
> + return false;
> +
> + /* Direct function calls via ADDR_EXPR don't need KCFI checks. */
> + if (TREE_CODE (fn) == ADDR_EXPR)
> + return false;
> +
> + /* Function pointers, variables, and other indirect calls need KCFI. */
> + return true;
> +}
> +
> +/* Extract target from call instruction RTL pattern.
> + Handles the RTL pattern matching needed to find the rtx containing
> + the function pointer for indirect calls. */
> +static rtx
> +kcfi_find_call_target (rtx_insn *call_insn)
> +{
> + if (!call_insn || (!CALL_P (call_insn) && !JUMP_P (call_insn)))
> + return NULL_RTX;
> +
> + rtx call_pattern = PATTERN (call_insn);
> + rtx target_reg = NULL_RTX;
> +
> + if (CALL_P (call_insn) && GET_CODE (call_pattern) == PARALLEL)
> + {
> + /* Handle PARALLEL patterns (need to extract CALL) */
> + rtx call_part = XVECEXP (call_pattern, 0, 0);
> + call_pattern = call_part;
> + }
> +
> + if (CALL_P (call_insn) && GET_CODE (call_pattern) == CALL)
> + {
> + /* Handle CALL patterns. */
> + rtx mem = XEXP (call_pattern, 0);
> + if (GET_CODE (mem) == MEM)
> + {
> + target_reg = XEXP (mem, 0);
> + }
> + }
> + else if (CALL_P (call_insn) && GET_CODE (call_pattern) == SET)
> + {
> + /* Handle SET patterns (function calls with return values) */
> + rtx src = XEXP (call_pattern, 1);
> + if (GET_CODE (src) == CALL)
> + {
> + rtx mem = XEXP (src, 0);
> + if (GET_CODE (mem) == MEM)
> + {
> + target_reg = XEXP (mem, 0);
> + }
> + }
> + }
> + else if (JUMP_P (call_insn) && GET_CODE (call_pattern) == SET)
> + {
> + /* Handle jump patterns. */
> + rtx src = SET_SRC (call_pattern);
> + if (GET_CODE (src) == MEM) /* Direct indirect jump. */
> + {
> + target_reg = XEXP (src, 0);
> + }
> + }
> +
> + return target_reg;
> +}
> +
> +/* Add KCFI type ID note to call instruction. */
> +void
> +add_kcfi_type_note (rtx_insn *call_insn, uint32_t type_id)
> +{
> + if (!call_insn || !CALL_P (call_insn))
> + return;
> +
> + /* Create integer constant for type ID. */
> + rtx type_id_rtx = gen_int_mode (type_id, SImode);
> +
> + /* Add note to call instruction. */
> + add_reg_note (call_insn, REG_CALL_KCFI_TYPE, type_id_rtx);
> +}
> +
> +/* Global flag to coordinate KCFI preamble emission. */
> +static bool kcfi_preamble_emitted = false;
> +
> +/* Mark that KCFI preamble has been emitted to prevent duplication. */
> +void
> +mark_kcfi_preamble_emitted (void)
> +{
> + kcfi_preamble_emitted = true;
> +}
> +
> +/* Emit KCFI type ID symbol for an address-taken function.
> + Centralized emission point to avoid duplication between
> + assemble_external_real() and assemble_start_function(). */
> +void
> +emit_kcfi_typeid_symbol (FILE *asm_file, tree decl, const char *name)
> +{
> + uint32_t type_id = get_function_kcfi_type_id (decl);
> + fprintf (asm_file, "\t.weak\t__kcfi_typeid_%s\n", name);
> + fprintf (asm_file, "\t.set\t__kcfi_typeid_%s, 0x%08x\n", name, type_id);
> +}
> +
> +/* Parse patchable function entry configuration from global flags. */
> +static bool
> +parse_patchable_function_entry_config (void)
> +{
> + HOST_WIDE_INT total_nops, prefix_nops;
> +
> + if (!flag_patchable_function_entry)
> + return false;
> +
> + parse_and_check_patch_area (flag_patchable_function_entry, false, &total_nops, &prefix_nops);
> + return (total_nops > 0);
> +}
> +
> +/* Common KCFI preamble helper that handles shared logic across architectures. */
> +static void
> +kcfi_emit_cfi_preamble (FILE *file, tree decl, HOST_WIDE_INT prefix_nops, const char *fname)
> +{
> + /* Get type ID. */
> + uint32_t type_id = get_function_kcfi_type_id (decl);
> +
> + /* Create symbol name for reuse. */
> + char cfi_symbol_name[256];
> + snprintf (cfi_symbol_name, sizeof(cfi_symbol_name), "__cfi_%s", fname);
> +
> + /* Emit __cfi_ symbol with proper visibility. */
> + if (TREE_PUBLIC (decl))
> + {
> + if (DECL_WEAK (decl))
> + ASM_WEAKEN_LABEL (file, cfi_symbol_name);
> + else
> + targetm.asm_out.globalize_label (file, cfi_symbol_name);
> + }
> +
> + /* Emit .type directive. */
> + ASM_OUTPUT_TYPE_DIRECTIVE (file, cfi_symbol_name, "function");
> + fprintf (file, "%s:\n", cfi_symbol_name);
> +
> + /* Calculate architecture-specific prefix NOPs if hook provided. */
> + int final_nops = prefix_nops;
> + if (kcfi_target.calculate_prefix_nops)
> + final_nops = kcfi_target.calculate_prefix_nops (prefix_nops);
> +
> + /* Emit architecture-specific prefix NOPs. */
> + for (int i = 0; i < final_nops; i++)
> + {
> + fprintf (file, "\tnop\n");
> + }
> +
> + /* Emit architecture-specific type ID instruction. */
> + if (kcfi_target.emit_type_id_instruction)
> + kcfi_target.emit_type_id_instruction (file, type_id);
> +
> + /* Mark end of __cfi_ symbol and emit size directive. */
> + char cfi_end_label[256];
> + snprintf (cfi_end_label, sizeof(cfi_end_label), ".Lcfi_func_end_%s", fname);
> + ASM_OUTPUT_LABEL (file, cfi_end_label);
> +
> + ASM_OUTPUT_MEASURED_SIZE (file, cfi_symbol_name);
> +}
> +
> +void
> +kcfi_emit_preamble_if_needed (FILE *file, tree decl,
> + bool is_patchable_context,
> + HOST_WIDE_INT prefix_nops,
> + const char *actual_fname)
> +{
> + /* Check if KCFI is enabled and function needs preamble. */
> + if (!function_needs_kcfi_preamble (decl))
> + return;
> +
> + /* Use actual function name if provided, otherwise fall back to DECL_ASSEMBLER_NAME. */
> + const char *fname = actual_fname ? actual_fname : IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (decl));
> +
> + /* Determine if we should emit here based on context. */
> + bool has_patchable_entries = parse_patchable_function_entry_config ();
> +
> + if (is_patchable_context && has_patchable_entries)
> + {
> + /* Use the provided prefix_nops parameter. */
> + }
> + else if (!is_patchable_context && !has_patchable_entries)
> + {
> + /* Emit for standard non-patchable functions. */
> + prefix_nops = 0;
> + }
> + else
> + {
> + /* Skip to avoid duplication:
> + - Patchable context but no global patchable config
> + - Non-patchable context but has global patchable config. */
> + return;
> + }
> +
> + kcfi_emit_cfi_preamble (file, decl, prefix_nops, fname);
> +}
> +
> +/* KCFI Final Instrumentation Pass - Post-Optimization Implementation
> +
> + This pass runs after all optimizations to insert KCFI checks immediately
> + before indirect calls/jumps, solving the fundamental problem where
> + optimization would insert code between KCFI checks and their protected
> + indirect transfers. */
> +
> +namespace {
> +
> +const pass_data pass_data_kcfi_final_instrumentation =
> +{
> + RTL_PASS, /* type */
> + "kcfi_final_instrumentation", /* name */
> + OPTGROUP_NONE, /* optinfo_flags */
> + TV_NONE, /* tv_id */
> + 0, /* properties_required */
> + 0, /* properties_provided */
> + 0, /* properties_destroyed */
> + 0, /* todo_flags_start */
> + 0, /* todo_flags_finish */
> +};
> +
> +class pass_kcfi_final_instrumentation : public rtl_opt_pass
> +{
> +public:
> + pass_kcfi_final_instrumentation (gcc::context *ctxt)
> + : rtl_opt_pass (pass_data_kcfi_final_instrumentation, ctxt)
> + {}
> +
> + /* opt_pass methods: */
> + bool gate (function *) final override
> + {
> + /* Note: Target KCFI support is validated in toplev.cc during option processing.
> + If the target doesn't provide gen_kcfi_checked_call, compilation stops with
> + a "sorry" message before any passes run. */
> + return sanitize_flags_p (SANITIZE_KCFI);
> + }
> +
> + unsigned int execute (function *) final override;
> +};
> +
> +/* Check if an RTL instruction is an indirect call/jump marked for KCFI. */
> +static bool
> +is_marked_indirect_transfer (rtx_insn *insn, uint32_t *type_id)
> +{
> + if (!insn || (!CALL_P (insn) && !JUMP_P (insn)))
> + return false;
> +
> + /* Check for KCFI type ID note. */
> + rtx note = find_reg_note (insn, REG_CALL_KCFI_TYPE, NULL_RTX);
> + if (!note)
> + return false;
> +
> + rtx type_rtx = XEXP (note, 0);
> + if (!CONST_INT_P (type_rtx))
> + return false;
> +
> + *type_id = (uint32_t) INTVAL (type_rtx);
> +
> + /* Verify it's actually an indirect transfer. */
> + rtx pattern = PATTERN (insn);
> +
> + if (CALL_P (insn))
> + {
> + /* Handle call patterns. */
> + if (GET_CODE (pattern) == CALL)
> + {
> + rtx target = XEXP (pattern, 0);
> + return GET_CODE (target) == MEM;
> + }
> + else if (GET_CODE (pattern) == SET)
> + {
> + rtx src = SET_SRC (pattern);
> + if (GET_CODE (src) == CALL)
> + {
> + rtx target = XEXP (src, 0);
> + return GET_CODE (target) == MEM;
> + }
> + }
> + else if (GET_CODE (pattern) == PARALLEL)
> + {
> + for (int i = 0; i < XVECLEN (pattern, 0); i++)
> + {
> + rtx elem = XVECEXP (pattern, 0, i);
> + if (GET_CODE (elem) == CALL
> + || (GET_CODE (elem) == SET && GET_CODE (SET_SRC (elem)) == CALL))
> + {
> + rtx call_rtx = (GET_CODE (elem) == SET) ? SET_SRC (elem) : elem;
> + rtx target = XEXP (call_rtx, 0);
> + return GET_CODE (target) == MEM;
> + }
> + }
> + }
> + }
> + else if (JUMP_P (insn))
> + {
> + /* Handle jump patterns. */
> + if (GET_CODE (pattern) == SET)
> + {
> + rtx src = SET_SRC (pattern);
> + if (GET_CODE (src) == MEM) /* Direct indirect jump. */
> + return true;
> + /* Could also handle other indirect jump patterns here. */
> + }
> + }
> +
> + return false;
> +}
> +
> +/* Replace marked indirect transfer with KCFI-protected version. */
> +unsigned int
> +pass_kcfi_final_instrumentation::execute (function *fun)
> +{
> + basic_block bb;
> +
> + /* Scan all basic blocks for marked indirect transfers. */
> + FOR_EACH_BB_FN (bb, fun)
> + {
> + rtx_insn *insn, *next_insn;
> +
> + /* Use safe iteration since we'll be modifying the instruction stream. */
> + for (insn = BB_HEAD (bb); insn && insn != NEXT_INSN (BB_END (bb)); insn = next_insn)
> + {
> + next_insn = NEXT_INSN (insn);
> +
> + uint32_t type_id;
> + if (is_marked_indirect_transfer (insn, &type_id))
> + {
> + /* Extract target address from the call instruction. */
> + rtx target_addr = kcfi_find_call_target (insn);
> + if (!target_addr)
> + {
> + error ("KCFI: cannot find target for indirect call");
> + gcc_unreachable ();
> + }
> +
> + HOST_WIDE_INT prefix_nops = get_current_function_patchable_prefix_nops ();
> +
> + /* Generate bundled KCFI checked call. */
> + start_sequence ();
> +
> + /* Pass the target as-is to the RTL pattern, which will handle
> + moving non-register targets to a register internally if needed. */
> + rtx bundled_call = kcfi_target.gen_kcfi_checked_call (insn, target_addr, type_id, prefix_nops);
> + if (!bundled_call)
> + {
> + error ("KCFI: instruction sequence creation failed");
> + gcc_unreachable ();
> + }
> +
> + /* Emit as call_insn, not generic insn. */
> + emit_call_insn (bundled_call);
> + rtx replacement_seq = get_insns ();
> + if (!replacement_seq)
> + {
> + error ("KCFI: instruction sequence insertion failed");
> + gcc_unreachable ();
> + }
> +
> + end_sequence ();
> +
> + /* Check if original was a sibling call and preserve that flag. */
> + bool was_sibcall = CALL_P (insn) && SIBLING_CALL_P (insn);
> +
> + /* Replace the original call entirely with bundled version. */
> + rtx_insn *new_insn = emit_insn_before (replacement_seq, insn);
> +
> + /* If original was a sibling call, mark the new instruction as such. */
> + if (was_sibcall && new_insn && CALL_P (new_insn))
> + SIBLING_CALL_P (new_insn) = 1;
> +
> + set_insn_deleted (insn); /* Mark original call as deleted. */
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +} // anon namespace
> +
> +rtl_opt_pass *
> +make_pass_kcfi_final_instrumentation (gcc::context *ctxt)
> +{
> + return new pass_kcfi_final_instrumentation (ctxt);
> +}
> +
> +/* KCFI GIMPLE pass implementation. */
> +
> +static bool
> +gate_kcfi (void)
> +{
> + return sanitize_flags_p (SANITIZE_KCFI);
> +}
> +
> +/* Create a KCFI wrapper function type that embeds the type ID. */
> +static tree
> +create_kcfi_wrapper_type (tree original_fn_type, uint32_t type_id)
> +{
> + /* Create a unique type name incorporating the type ID. */
> + char wrapper_name[64];
> + snprintf (wrapper_name, sizeof (wrapper_name), "__kcfi_wrapper_%x", type_id);
> +
> + /* Build a new function type that's structurally identical but nominally different. */
> + tree wrapper_type = build_function_type (TREE_TYPE (original_fn_type),
> + TYPE_ARG_TYPES (original_fn_type));
> +
> + /* Set the type name to make it distinct. */
> + TYPE_NAME (wrapper_type) = get_identifier (wrapper_name);
> +
> + /* Attach kcfi_type_id attribute to the original function type for cfgexpand.cc */
> + tree attr_name = get_identifier ("kcfi_type_id");
> + tree attr_value = build_int_cst (unsigned_type_node, type_id);
> + tree attr = build_tree_list (attr_name, attr_value);
> + TYPE_ATTRIBUTES (original_fn_type) = chainon (TYPE_ATTRIBUTES (original_fn_type), attr);
> +
> + return wrapper_type;
> +}
> +
> +/* Replace indirect calls with KCFI wrapper types for anti-merging and clobber tracking. */
> +static unsigned int
> +kcfi_instrument (void)
> +{
> + basic_block bb;
> +
> + FOR_EACH_BB_FN (bb, cfun)
> + {
> + gimple_stmt_iterator gsi;
> + for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
> + {
> + gimple *stmt = gsi_stmt (gsi);
> +
> + if (is_gimple_call (stmt))
> + {
> + gcall *call_stmt = as_a <gcall *> (stmt);
> +
> + // Skip internal calls - we only instrument indirect calls
> + if (gimple_call_internal_p (call_stmt))
> + {
> + continue;
> + }
> +
> + tree fndecl = gimple_call_fndecl (call_stmt);
> +
> + // Only process indirect calls (no fndecl)
> + if (!fndecl)
> + {
> + tree fn = gimple_call_fn (call_stmt);
> + if (fn && is_kcfi_indirect_call (fn))
> + {
> + // Get the function type to compute KCFI type ID
> + tree fn_type = gimple_call_fntype (call_stmt);
> + gcc_assert (fn_type);
> + if (TREE_CODE (fn_type) == FUNCTION_TYPE)
> + {
> + uint32_t type_id = compute_kcfi_type_id (fn_type);
> +
> + // Create KCFI wrapper type for this call
> + tree wrapper_type = create_kcfi_wrapper_type (fn_type, type_id);
> +
> + // Create a temporary variable for the wrapped function pointer
> + tree wrapper_ptr_type = build_pointer_type (wrapper_type);
> + tree wrapper_tmp = create_tmp_var (wrapper_ptr_type, "kcfi_wrapper");
> +
> + // Create assignment: wrapper_tmp = (wrapper_ptr_type) fn
> + tree cast_expr = build1 (NOP_EXPR, wrapper_ptr_type, fn);
> + gimple *cast_stmt = gimple_build_assign (wrapper_tmp, cast_expr);
> + gsi_insert_before (&gsi, cast_stmt, GSI_SAME_STMT);
> +
> + // Update the call to use the wrapped function pointer
> + gimple_call_set_fn (call_stmt, wrapper_tmp);
> + }
> + }
> + }
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +namespace {
> +
> +const pass_data pass_data_kcfi =
> +{
> + GIMPLE_PASS, /* type */
> + "kcfi", /* name */
> + OPTGROUP_NONE, /* optinfo_flags */
> + TV_NONE, /* tv_id */
> + ( PROP_ssa | PROP_cfg | PROP_gimple_leh ), /* properties_required */
> + 0, /* properties_provided */
> + 0, /* properties_destroyed */
> + 0, /* todo_flags_start */
> + TODO_update_ssa, /* todo_flags_finish */
> +};
> +
> +class pass_kcfi : public gimple_opt_pass
> +{
> +public:
> + pass_kcfi (gcc::context *ctxt)
> + : gimple_opt_pass (pass_data_kcfi, ctxt)
> + {}
> +
> + /* opt_pass methods: */
> + opt_pass * clone () final override { return new pass_kcfi (m_ctxt); }
> + bool gate (function *) final override
> + {
> + return gate_kcfi ();
> + }
> + unsigned int execute (function *) final override
> + {
> + return kcfi_instrument ();
> + }
> +
> +}; // class pass_kcfi
> +
> +} // anon namespace
> +
> +gimple_opt_pass *
> +make_pass_kcfi (gcc::context *ctxt)
> +{
> + return new pass_kcfi (ctxt);
> +}
> +
> +namespace {
> +
> +const pass_data pass_data_kcfi_O0 =
> +{
> + GIMPLE_PASS, /* type */
> + "kcfi0", /* name */
> + OPTGROUP_NONE, /* optinfo_flags */
> + TV_NONE, /* tv_id */
> + ( PROP_ssa | PROP_cfg | PROP_gimple_leh ), /* properties_required */
> + 0, /* properties_provided */
> + 0, /* properties_destroyed */
> + 0, /* todo_flags_start */
> + TODO_update_ssa, /* todo_flags_finish */
> +};
> +
> +class pass_kcfi_O0 : public gimple_opt_pass
> +{
> +public:
> + pass_kcfi_O0 (gcc::context *ctxt)
> + : gimple_opt_pass (pass_data_kcfi_O0, ctxt)
> + {}
> +
> + /* opt_pass methods: */
> + bool gate (function *) final override
> + {
> + return !optimize && gate_kcfi ();
> + }
> + unsigned int execute (function *) final override
> + {
> + return kcfi_instrument ();
> + }
> +
> +}; // class pass_kcfi_O0
> +
> +} // anon namespace
> +
> +gimple_opt_pass *
> +make_pass_kcfi_O0 (gcc::context *ctxt)
> +{
> + return new pass_kcfi_O0 (ctxt);
> +}
> diff --git a/gcc/opts.cc b/gcc/opts.cc
> index 5fd86aa89adb..ab74414b9c5b 100644
> --- a/gcc/opts.cc
> +++ b/gcc/opts.cc
> @@ -2168,6 +2168,7 @@ const struct sanitizer_opts_s sanitizer_opts[] =
> SANITIZER_OPT (pointer-overflow, SANITIZE_POINTER_OVERFLOW, true, true),
> SANITIZER_OPT (builtin, SANITIZE_BUILTIN, true, true),
> SANITIZER_OPT (shadow-call-stack, SANITIZE_SHADOW_CALL_STACK, false, false),
> + SANITIZER_OPT (kcfi, SANITIZE_KCFI, false, true),
> SANITIZER_OPT (all, ~0ULL, true, true),
> #undef SANITIZER_OPT
> { NULL, 0ULL, 0UL, false, false }
> diff --git a/gcc/passes.cc b/gcc/passes.cc
> index a33c8d924a52..4c6ceac740ff 100644
> --- a/gcc/passes.cc
> +++ b/gcc/passes.cc
> @@ -63,6 +63,7 @@ along with GCC; see the file COPYING3. If not see
> #include "diagnostic-core.h" /* for fnotice */
> #include "stringpool.h"
> #include "attribs.h"
> +#include "kcfi.h"
>
> /* Reserved TODOs */
> #define TODO_verify_il (1u << 31)
> diff --git a/gcc/passes.def b/gcc/passes.def
> index d528a0477d9a..d4c84ca7064d 100644
> --- a/gcc/passes.def
> +++ b/gcc/passes.def
> @@ -275,6 +275,7 @@ along with GCC; see the file COPYING3. If not see
> NEXT_PASS (pass_sink_code, false /* unsplit edges */);
> NEXT_PASS (pass_sancov);
> NEXT_PASS (pass_asan);
> + NEXT_PASS (pass_kcfi);
> NEXT_PASS (pass_tsan);
> NEXT_PASS (pass_dse, true /* use DR analysis */);
> NEXT_PASS (pass_dce, false /* update_address_taken_p */, false /* remove_unused_locals */);
> @@ -443,6 +444,7 @@ along with GCC; see the file COPYING3. If not see
> NEXT_PASS (pass_sancov_O0);
> NEXT_PASS (pass_lower_switch_O0);
> NEXT_PASS (pass_asan_O0);
> + NEXT_PASS (pass_kcfi_O0);
> NEXT_PASS (pass_tsan_O0);
> NEXT_PASS (pass_musttail);
> NEXT_PASS (pass_sanopt);
> @@ -563,6 +565,7 @@ along with GCC; see the file COPYING3. If not see
> NEXT_PASS (pass_delay_slots);
> NEXT_PASS (pass_split_for_shorten_branches);
> NEXT_PASS (pass_convert_to_eh_region_ranges);
> + NEXT_PASS (pass_kcfi_final_instrumentation);
> NEXT_PASS (pass_shorten_branches);
> NEXT_PASS (pass_set_nothrow_function_flags);
> NEXT_PASS (pass_dwarf2_frame);
> diff --git a/gcc/recog.cc b/gcc/recog.cc
> index 67d7fa630692..127524cfb4e2 100644
> --- a/gcc/recog.cc
> +++ b/gcc/recog.cc
> @@ -4026,6 +4026,7 @@ peep2_attempt (basic_block bb, rtx_insn *insn, int match_len, rtx_insn *attempt)
> case REG_SETJMP:
> case REG_TM:
> case REG_CALL_NOCF_CHECK:
> + case REG_CALL_KCFI_TYPE:
> add_reg_note (new_insn, REG_NOTE_KIND (note),
> XEXP (note, 0));
> break;
> diff --git a/gcc/reg-notes.def b/gcc/reg-notes.def
> index 68e137ceccaa..27924ee8afda 100644
> --- a/gcc/reg-notes.def
> +++ b/gcc/reg-notes.def
> @@ -251,5 +251,11 @@ REG_NOTE (UNTYPED_CALL)
> compiler when the option -fcf-protection=branch is specified. */
> REG_NOTE (CALL_NOCF_CHECK)
>
> +/* Contains the expected KCFI type ID for an indirect call instruction.
> + This note carries the 32-bit type identifier that should match the
> + type ID stored in the function preamble. Used by KCFI (Kernel Control
> + Flow Integrity) to validate indirect call targets at runtime. */
> +REG_NOTE (CALL_KCFI_TYPE)
> +
> /* The values passed to callee, for debuginfo purposes. */
> REG_NOTE (CALL_ARG_LOCATION)
> diff --git a/gcc/targhooks.cc b/gcc/targhooks.cc
> index e723bbbc4df6..7edd3d8619af 100644
> --- a/gcc/targhooks.cc
> +++ b/gcc/targhooks.cc
> @@ -77,6 +77,7 @@ along with GCC; see the file COPYING3. If not see
> #include "calls.h"
> #include "expr.h"
> #include "output.h"
> +#include "kcfi.h"
> #include "common/common-target.h"
> #include "reload.h"
> #include "intl.h"
> @@ -2203,7 +2204,11 @@ default_compare_by_pieces_branch_ratio (machine_mode)
> the location of the NOPs will be recorded in a special object section
> called "__patchable_function_entries". This routine may be called
> twice per function to put NOPs before and after the function
> - entry. */
> + entry.
> +
> + KCFI Integration: When KCFI is enabled and this is prefix emission (record_p = true),
> + this function will emit the KCFI preamble (symbols, NOPs, movl) before the prefix NOPs
> + to ensure correct memory layout per KCFI specification. */
>
> void
> default_print_patchable_function_entry (FILE *file,
> @@ -2219,6 +2224,42 @@ default_print_patchable_function_entry (FILE *file,
> code_num = recog_memoized (my_nop);
> nop_templ = get_insn_template (code_num, my_nop);
>
> + /* KCFI Integration for prefix emission. */
> + bool kcfi_handled = false;
> + if (record_p && current_function_decl)
> + {
> + /* Parse patchable function entry configuration to get prefix NOPs. */
> + HOST_WIDE_INT total_nops = patch_area_size;
> + HOST_WIDE_INT prefix_nops = 0;
> +
> + /* Check for function-specific patchable_function_entry attribute. */
> + tree patchable_attr = lookup_attribute ("patchable_function_entry",
> + DECL_ATTRIBUTES (current_function_decl));
> + if (patchable_attr)
> + {
> + tree pp_val = TREE_VALUE (patchable_attr);
> + total_nops = tree_to_uhwi (TREE_VALUE (pp_val));
> + if (TREE_CHAIN (pp_val))
> + prefix_nops = tree_to_uhwi (TREE_VALUE (TREE_CHAIN (pp_val)));
> + }
> + else
> + {
> + /* Use global configuration if no function-specific attribute. */
> + HOST_WIDE_INT patch_area_entry;
> + parse_and_check_patch_area (flag_patchable_function_entry, false,
> + &total_nops, &patch_area_entry);
> + prefix_nops = patch_area_entry;
> + }
> +
> + /* Use centralized KCFI manager for patchable context. */
> + if (prefix_nops > 0 && patch_area_size == (unsigned HOST_WIDE_INT)prefix_nops)
> + {
> + kcfi_emit_preamble_if_needed (file, current_function_decl, true, prefix_nops, NULL);
> + kcfi_handled = true;
> + }
> + }
> +
> + /* Standard patchable function entry handling. */
> if (record_p && targetm_common.have_named_sections)
> {
> char buf[256];
> @@ -2249,9 +2290,16 @@ default_print_patchable_function_entry (FILE *file,
> ASM_OUTPUT_LABEL (file, buf);
> }
>
> + /* Emit the patchable NOPs. */
> unsigned i;
> for (i = 0; i < patch_area_size; ++i)
> output_asm_insn (nop_templ, NULL);
> +
> + /* Mark that KCFI preamble was handled to prevent duplication in ix86_asm_output_function_label. */
> + if (kcfi_handled)
> + {
> + mark_kcfi_preamble_emitted ();
> + }
> }
>
> bool
> diff --git a/gcc/toplev.cc b/gcc/toplev.cc
> index 70dbb3e717f6..04331947b9da 100644
> --- a/gcc/toplev.cc
> +++ b/gcc/toplev.cc
> @@ -67,6 +67,7 @@ along with GCC; see the file COPYING3. If not see
> #include "attribs.h"
> #include "asan.h"
> #include "tsan.h"
> +#include "kcfi.h"
> #include "plugin.h"
> #include "context.h"
> #include "pass_manager.h"
> @@ -524,6 +525,7 @@ compile_file (void)
> insn_locations_init ();
> targetm.asm_out.code_end ();
>
> +
> /* Do dbx symbols. */
> timevar_push (TV_SYMOUT);
>
> @@ -1733,6 +1735,12 @@ process_options ()
> "requires %<-fno-exceptions%>");
> }
>
> + if (flag_sanitize & SANITIZE_KCFI)
> + {
> + if (!kcfi_target.gen_kcfi_checked_call)
> + sorry ("%<-fsanitize=kcfi%> not supported by this target");
> + }
> +
> HOST_WIDE_INT patch_area_size, patch_area_start;
> parse_and_check_patch_area (flag_patchable_function_entry, false,
> &patch_area_size, &patch_area_start);
> diff --git a/gcc/varasm.cc b/gcc/varasm.cc
> index 000ad9e26f6f..9c4e3c223159 100644
> --- a/gcc/varasm.cc
> +++ b/gcc/varasm.cc
> @@ -57,6 +57,7 @@ along with GCC; see the file COPYING3. If not see
> #include "attribs.h"
> #include "asan.h"
> #include "rtl-iter.h"
> +#include "kcfi.h"
> #include "file-prefix-map.h" /* remap_debug_filename() */
> #include "alloc-pool.h"
> #include "toplev.h"
> @@ -2210,6 +2211,10 @@ assemble_start_function (tree decl, const char *fnname)
> ASM_OUTPUT_FUNCTION_LABEL (asm_out_file, fnname, current_function_decl);
> #endif /* ASM_DECLARE_FUNCTION_NAME */
>
> + /* KCFI type ID symbols are only emitted for external function declarations,
> + handled by assemble_external_real(). Function definitions do not emit
> + __kcfi_typeid symbols. */
> +
> /* And the area after the label. Record it if we haven't done so yet. */
> if (patch_area_size > patch_area_entry)
> targetm.asm_out.print_patchable_function_entry (asm_out_file,
> @@ -2752,6 +2757,19 @@ assemble_external_real (tree decl)
> /* Some systems do require some output. */
> SYMBOL_REF_USED (XEXP (rtl, 0)) = 1;
> ASM_OUTPUT_EXTERNAL (asm_out_file, decl, XSTR (XEXP (rtl, 0), 0));
> +
> + /* Emit KCFI type ID symbol for external function declarations that are address-taken. */
> + struct cgraph_node *node = (TREE_CODE (decl) == FUNCTION_DECL) ? cgraph_node::get (decl) : NULL;
> + if (flag_sanitize & SANITIZE_KCFI
> + && TREE_CODE (decl) == FUNCTION_DECL
> + && !DECL_INITIAL (decl) /* Only for external declarations (no function body) */
> + && node && node->address_taken) /* Use direct cgraph analysis for address-taken check. */
> + {
> + const char *name = XSTR (XEXP (rtl, 0), 0);
> + /* Strip any encoding prefixes like '*' from symbol name. */
> + name = targetm.strip_name_encoding (name);
> + emit_kcfi_typeid_symbol (asm_out_file, decl, name);
> + }
> }
> }
> #endif
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-08-28 14:57 ` Qing Zhao
@ 2025-09-04 4:24 ` Kees Cook
2025-09-04 7:16 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: Kees Cook @ 2025-09-04 4:24 UTC (permalink / raw)
To: Qing Zhao
Cc: gcc-patches@gcc.gnu.org, Joseph Myers, Richard Biener,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Peter Zijlstra, Dan Li,
linux-hardening@vger.kernel.org
On Thu, Aug 28, 2025 at 02:57:05PM +0000, Qing Zhao wrote:
> Hi, Kees,
>
> I have several suggestions and questions first for this patch:
>
> 1. Is -fsanitize=kcfi a C only feature? If so, you might need to mention this in
> the documentation and also reject its usage in other languages.
Yes, it's C only. I will note it and add a check for that. Thanks for
the reminder!
> 2. There is no overall description of the design of this kcfi implementation
> in the source code, I think it might be very helpful to provide such description
> In the new module kcfi.cc <http://kcfi.cc/>. such documentation will be helpful for current
> review and future maintenance of this feature.
Ah yeah, good idea. I will adapt the commit log.
>
> > On Aug 21, 2025, at 03:26, Kees Cook <kees@kernel.org> wrote:
> >
> > This series implements the Linux Kernel Control Flow Integrity ABI,
> > which provides a function prototype based forward edge control flow
> > integrity protection by instrumenting every indirect call to check for
> > a hash value before the target function address. If the hash at the call
> > site and the hash at the target do not match, execution will trap.
> >
> > Just to set expectations, this is an RFC because this is my first time
> > working on most of the affected areas in GCC, and it is likely I have
> > missed really obvious stuff, or gone about doing things in very wrong
> > ways. I tried to find the best way to do stuff, but I was left with many
> > questions. :) All that said, this works for x86_64 and aarch64 Linux
> > kernels. (I have implemented riscv64 as well, but I lack a viable test
> > environment -- I am working on this still.)
> >
> > KCFI has a number of specific constraints. Some are tied to the
> > backend architecture, which I'll cover in more detail in later patches.
> > The constraints are:
> >
> > - The KCFI scheme generates a unique 32-bit hash for each unique function
> > prototype, allowing for indirect call sites to verify that they are
> > calling into a matching _type_ of function pointer. This changes the
> > semantics of some optimization logic because now indirect calls to
> > different types cannot be merged. For example:
> >
> > if (p->func_type_1)
> > return p->func_type_1();
> > if (p->func_type_2)
> > return p->func_type_2();
> >
> > In final asm, the optimizer may collapse the second indirect call
> > into a jump to the first indirect call once it has loaded the function
> > pointer. KCFI must block cross-type merging otherwise there will be a
> > single KCFI check happening for only 1 type but being used by 2 target
> > types. The distinguishing characteristic for call merging becomes the
> > type, not the address/register usage.
> >
> > - The check-call instruction sequence must be treated a single unit: it
> > cannot be rearranged or split or optimized. The pattern is that
> > indirect calls, "call *$target", get converted into:
> >
> > mov $target_expression, %target (only present if the expression was
> > not already %target)
> > load -$offset(%target), %tmp
> > cmp $hash, %tmp
> > je .Lcheck_passed
> > .Ltrap$N:
> > trap
> > .Lcheck_passed$N:
> > call *%target
> >
> > This pattern of call immediately after trap provides for the
> > "permissive" checking mode automatically: the trap gets handled,
> > a warning emitted, and then execution continues after the trap to
> > the call.
> >
> > (x86_64 uses "mov -$hash, %tmp; addl -$offset(%target), %tmp; je"
> > to zero out the register before making the call. Also Linux needs
> > exactly these insns because it is both disassembling them during
> > trap handling and potentially live patching them at boot time to
> > be converted into a different series of instrutions, a scheme
> > know as FineIBT, making the insn sequence ABI.)
> >
> > - KCFI check-call instrumentation must survive tail call optimization.
> > If an indirect call is turned into an indirect jump, KCFI checking
> > must still happen (but will still use the jmp).
> >
> > - Functions that may be called indirectly have a preamble added,
> > __cfi_$original_func_name, that contains the $hash value:
> >
> > __cfi_target_func:
> > .word $hash
> > target_func:
> > [regular function entry...]
> >
> > (x86_64 uses a movl instruction to hold the hash and prefixed aligned
> > NOPs to maintain cache line alignment in the face of patchable function
> > entry...)
> >
> > - The preamble needs to interact with patchable function entry so that
> > the hash appears further away from the actual start of the function
> > (leaving the prefix NOPs of the patchable function entry unchanged).
> > This means only _globally defined_ patchable function entry is supported
> > with KCFI (indrect call sites must know in advance what the offset is,
> > which may not be possible extern functions). For example, a "4,4"
> > patchable function entry would end up like:
> >
> > __cfi_target_func:
> > .data $hash
> > nop nop nop nop
> > target_func:
> > [regular function entry...]
> >
> > (Linux x86_64 uses an 11 byte prefix nop area resulting in 16 bytes
> > total including the movl. This region may be live patched at boot time
> > for FineIBT so the behavior here is also ABI.)
> >
> > - External functions that are address-taken have a weak __kcfi_typeid_$funcname
> > symbol added with the hash value available so that the hash can be referenced
> > from assembly linkages, etc, where the hash values cannot be calculated (i.e
> > where C type information is missing):
> >
> > .weak __kcfi_typeid_$func
> > .set __kcfi_typeid_$func, $hash
> >
> > - On architectures that do not have a good way to encode additional
> > details in their trap (x86_64 and riscv64), the trap location
> > is identified as a KCFI trap via a relative address offset entry
> > emitted into the .kcfi_traps section for each indirect call site's
> > trap instruction. The previous check-call example's insn sequence has
> > a section push/pop inserted between the trap and call:
> >
> > ...
> > .Ltrap$N:
> > trap
> > .pushsection .kcfi_traps,"ao",@progbits,.text
> > .Lentry$N:
> > .long .Ltrap$N - .Lentry$N
> > .popsection
> > .Lcheck_passed$N:
> > call %target
> >
> > (aarch64 encodes the register numbers that hold the expected hash
> > and the target address in the trap ESR and thereby does not need a
> > .kcfi_traps section at all.)
> >
> > - The no_sanitize("kcfi") function attribute means that the marked function
> > must not produce KCFI checking for indirect calls, and that this
> > attribute must survive inlining. This is used rarely by Linux, but
> > is required to make BPF JIT trampolines work on older Linux kernel
> > versions. (The preamble code is very recently finally being generated
> > at JIT time on the last remaining Linux KCFI arch where this was
> > missing: aarch64.)
> >
> > As a result of these constraints, there are some behavioral aspects
> > that need to be preserved across the middle-end and back-end, as I
> > understand them.
> >
> > For indirect call sites:
> >
> > - Keeping indirect calls from being merged (see above). I did this by
> > adding a wrapping type so that equality was tested based on type-id.
> > This is done in create_kcfi_wrapper_type(), via kcfi_instrument(),
> > via an early GIMPLE pass (pass_kcfi and pass_kcfi0). The wrapper type
> > is checked in gcc/cfgcleanup.cc, old_insns_match_p().
>
> First, Looks like that the routine “old_insns_match_p” checks the type-ids that were embedded
> in the REG_NOTE ( REG_CALL_KCFI_TYPE) of the RTL. -:)
Once I started using the backend RTL patterns more correctly, this note
and all the logic attached to it went away. I'll send my v2 shortly.
> My understanding is: the computed type-id is recorded into to a “kcfi_type_id” attribute, and
> then this “kcfi_type_id” attribute is attached to the original function type.
Correct (so that we can use it when generating the preambles and RTL
define_insn output).
> At the same time, A wrapper type is created for the original function type, whose typename is
> “__kcfi_wrapper_type_id”.
>
> I am confused:
>
> 1. Why the additional wrapper type is needed? why the original function type + “kcfi_type_id” not enough?
This was created to keep calls some being combined during optimization.
e.g.:
if (func1) {
func1();
return;
}
if (func2) {
func2();
return;
}
this was getting constructed as:
if (func1) {
ptr = func1;
do_call:
call ptr;
return;
}
if (func2) {
ptr = func2;
goto do_call;
}
The above is normally totally fine because only address "matters", but
in the CFI world, the typeid is a part of the call. If func1 and
func2 don't share the same typeid, suddenly the "goto do_call" call path
will fail because the call (that checks the func1 typeid) will trigger a
mismatch wne func2 tries to be called through the func1-typeid-call.
> 2. Why attaching the type-id as a “kcfi_type_id” attribute to the original function type?
> Is there other way to attach the type-id to the original function type? Or to the original indirect call site?
In the v2 series I moved this to the IPA phase so I could attach
everything to the original fndecl.
> Is it possible that we might provide a user level “kcfi_type_id” attribute to function type in the future? then
> That will be conflict with the current “kcfi_type_id” attribute?
The closest I can imagine is the kcfi hash salt, which will just allow
the user to bump the typeid hash by a deterministic amount (to separate
the larger "pools" of hashes -- e.g. "void func(void)" is very common,
so the salt would allow for a slight modification to separate sets of
otherwise identical types).
> >
> > - Keeping typeid information available through to the RTL expansion
> > phase was done via a typeid note (REG_CALL_KCFI_TYPE) attached also
> > in create_kcfi_wrapper_type() in the same GIMPLE pass.
>
> This is done in the routine “expand_call_stmt” through call to “add_kcfi_type_note”.
> The “type_id” is attached as a “kcfi_type_id” attribute to the original function type in the routine
> “create_kcfi_wrapper_type”.
>
> Please see my above question 2.
In v2 this has been pretty radically changed -- expansion just reads the
kcfi_type_id directly into an RTL operand for a new KCFI insn type (i.e.
separate from CALL or SET), which wraps CALL.
> > - To make sure instrumentation was skipped for inline functions, the
> > RTL pass walks the basic blocks to identify their function origins,
> > looking for the no_sanitize("kcfi") attribute, and skipping
> > instrumentation if found.
>
> I think this was done inefficiently. It’s better to do this during inlining transformation to
> propagate the no_sanitize information from the caller to the callee body and attach
> the no_sanitize information to the indirect callsite.
Yes, I took Andrew's suggestion and implemented a flag that is set on
the insn itself and is checked at the callsite.
>
> >
> > For indirect call targets:
> >
> > - kcfi_emit_preamble_if_needed() uses function_needs_kcfi_preamble(),
> > and counter helpers, to emit the preablem, with patchable function
> > entry NOPs. This gets used both in default_print_patchable_function_entry()
> > and the per-arch path.
>
> I saw that “kcfi_emit_preamble_if_needed” is called by “default_print_patchable_function_entry”,
> But I didn’t see where it is called when the function is not patchable?
These were in the per-arch function label emission, but for v2 I've
moved them out after refactoring how the patchable function entry
integration works.
> > I could not find a simpler way to deal with
> > patchable function entry besides splitting it up like this. I feel
> > like there should be a better way.
> >
> > - gcc/varasm.cc, assemble_external_real() calls emit_kcfi_typeid_symbol()
> > to add the __kcfi_typeid symbols (see get_function_kcfi_type_id()
> > below).
> >
> > To support the per-arch back-ends, there are some common helpers:
> >
> > - A callback framework is added via struct kcfi_target_hooks for
> > backends to fill out.
> >
> > - kcfi_emit_trap_with_section() handles the push/pop section and
> > generating the relative offset section entries.
> >
> > - get_function_kcfi_type_id() generates the 32-bit hash value, using
> > compute_kcfi_type_id() and kcfi_hash_string() to hook to the mangling
> > API. The hash is FNV-1a right now: it doesn't need secrecy. It could be
> > replaced with any hash, though the hash will need to be coordinated
> > with Rust, which implements the KCFI ABI as well.
>
> One question here, the final type_id will be computed by the following 2 steps:
>
> 1. Itanium C++ mangling based on return type + parameter types of the function.
> 2. FNV-1a hash on the mangled string
>
> If the hacker knows these, it should be quite easy for them to come up with a
> matched typeid, is it?
The hashes aren't considered secret -- they need to be known/match between
compilation units, and even across languages (Rust). The KCFI mitigation
is fundamentally an "exploit surface reduction" measure in the sense
that it limits an attacker's set of callable functions to only matching
typeids (instead of all functions or all executable memory). I discuss
this a big more here:
https://gcc.gnu.org/pipermail/gcc-patches/2025-August/693059.html
Thanks for the review! I'll have v2 up shortly -- I'm still going
through the commit logs and tests to make sure they're sensible. :)
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2025-09-04 4:24 ` Kees Cook
@ 2025-09-04 7:16 ` Peter Zijlstra
0 siblings, 0 replies; 42+ messages in thread
From: Peter Zijlstra @ 2025-09-04 7:16 UTC (permalink / raw)
To: Kees Cook
Cc: Qing Zhao, gcc-patches@gcc.gnu.org, Joseph Myers, Richard Biener,
Jan Hubicka, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Dan Li,
linux-hardening@vger.kernel.org
On Wed, Sep 03, 2025 at 09:24:22PM -0700, Kees Cook wrote:
> > If the hacker knows these, it should be quite easy for them to come up with a
> > matched typeid, is it?
>
> The hashes aren't considered secret -- they need to be known/match between
> compilation units, and even across languages (Rust). The KCFI mitigation
> is fundamentally an "exploit surface reduction" measure in the sense
> that it limits an attacker's set of callable functions to only matching
> typeids (instead of all functions or all executable memory). I discuss
> this a big more here:
> https://gcc.gnu.org/pipermail/gcc-patches/2025-August/693059.html
Also note that the kernel with re-hash all the values at boot time once
more -- further increasing the difficulty of an attack.
^ permalink raw reply [flat|nested] 42+ messages in thread
end of thread, other threads:[~2025-09-04 7:16 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-21 7:26 [RFC PATCH 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
2025-08-21 7:26 ` [RFC PATCH 1/7] sanitizer: Expand sanitizer flag from 32-bit to 64-bit Kees Cook
2025-08-21 7:26 ` [RFC PATCH 2/7] mangle: Introduce C typeinfo mangling API Kees Cook
[not found] ` <CALvbMcAPV1eB6nocPAS=qR8SCiQyU43v911R8S7Ah_=G7yK-+g@mail.gmail.com>
2025-08-21 8:29 ` Andrew Pinski
2025-08-21 16:16 ` Kees Cook
2025-08-21 16:24 ` Andrew Pinski
2025-08-21 19:14 ` Qing Zhao
2025-08-21 21:29 ` Kees Cook
2025-08-22 15:11 ` Qing Zhao
2025-08-22 19:02 ` Kees Cook
2025-08-22 20:29 ` Qing Zhao
2025-08-22 22:29 ` Kees Cook
2025-08-25 8:13 ` Peter Zijlstra
2025-08-25 13:56 ` Qing Zhao
2025-08-21 7:26 ` [RFC PATCH 3/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
[not found] ` <CALvbMcA+8iHo+zCCvs4UdAg9PVQVtgOno-rtMS4i5YajrjkyGw@mail.gmail.com>
2025-08-21 9:12 ` Peter Zijlstra
2025-08-21 11:01 ` Richard Biener
2025-08-21 14:25 ` Peter Zijlstra
2025-08-21 18:09 ` Qing Zhao
2025-08-22 5:15 ` Kees Cook
2025-08-22 10:03 ` Peter Zijlstra
2025-08-21 19:57 ` Kees Cook
2025-08-22 6:53 ` Richard Biener
2025-08-22 19:23 ` Kees Cook
[not found] ` <CA+=Sn1koTTQaXDnAVWtVU6ACWwhD08NR5nDJO236Pmcoi2X9qA@mail.gmail.com>
2025-08-22 7:51 ` Peter Zijlstra
2025-08-22 8:24 ` Peter Zijlstra
2025-08-22 8:47 ` Kees Cook
2025-08-22 5:10 ` Kees Cook
2025-08-22 5:27 ` Andrew Pinski
2025-08-28 14:57 ` Qing Zhao
2025-09-04 4:24 ` Kees Cook
2025-09-04 7:16 ` Peter Zijlstra
2025-08-21 7:26 ` [RFC PATCH 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
2025-08-21 9:29 ` Peter Zijlstra
2025-08-21 18:46 ` Kees Cook
2025-08-21 19:03 ` Kees Cook
2025-08-22 8:19 ` Peter Zijlstra
2025-08-22 8:36 ` Kees Cook
2025-08-22 8:55 ` Peter Zijlstra
2025-08-21 7:26 ` [RFC PATCH 5/7] aarch64: Add AArch64 " Kees Cook
2025-08-21 7:26 ` [RFC PATCH 6/7] riscv: Add RISC-V " Kees Cook
2025-08-21 7:26 ` [RFC PATCH 7/7] kcfi: Add regression test suite Kees Cook
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).