* [PATCH v17 1/7] kcfi: Introduce KCFI typeinfo mangling API
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-05 15:40 ` [PATCH v17 2/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
` (5 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
To support the KCFI typeid and future Linux type-based allocators,
which need to convert unique types into unique 32-bit values, add a
mangling system based on the Itanium C++ mangling ABI, adapted for
C types. Introduce __builtin_linux_abi_kcfi_hash for the hash, and
__builtin_linux_abi_kcfi_name for testing and debugging (to see the
human-readable mangling form). Add tests for typeinfo validation and
error handling.
This ABI needs to match what is used by LLVM Rust (which matches the Clang
ABI) so that KCFI can work on mixed GCC with LLVM-Rust kernel builds.
Instead of inventing a new ABI, all use the existing Itanium C++ mangling
which matches KCFI's needs.
An important aspect of the C++ typeinfo behavior that is retained here
is that typedefs are treated as pass-through except when the underlying
type lacks a tag (i.e. anonymous struct, union, or enum). This provides a
distinction between those typedefs and typedefs used to provide _aliases_
(u8, uint16_t). But regardless, this is distinct from strict ISO C type
equivalency, so it remains a Linux KCFI ABI.
Assisted-by: LLM [tests]
gcc/ChangeLog:
* Makefile.in: Add kcfi-typeinfo.o.
* doc/extend.texi: Document typeinfo builtins.
* kcfi-typeinfo.h: New file, typeinfo mangling API.
* kcfi-typeinfo.cc: New file, implement typeinfo mangling.
* selftest.h (kcfi_typeinfo_cc_tests): Declare.
* selftest-run-tests.cc (selftest::run_tests): Call
kcfi_typeinfo_cc_tests.
gcc/c-family/ChangeLog:
* c-common.h (enum rid): Add typeinfo builtins.
* c-common.cc: Add typeinfo builtins.
gcc/c/ChangeLog:
* c-parser.cc (c_parser_get_builtin_type_arg): New function,
parse type.
(c_parser_postfix_expression): Add typeinfo builtins.
gcc/testsuite/ChangeLog:
* gcc.dg/builtin-typeinfo.c: New test, typeinfo mangling.
* gcc.dg/builtin-typeinfo-bitint.c: New test, BitInt.
* gcc.dg/builtin-typeinfo-c23.c: New test, C23 types.
* gcc.dg/builtin-typeinfo-errors.c: New test, validate bad
arguments are rejected.
* gcc.dg/builtin-typeinfo-fixed.c: New test, fixed point.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/doc/extend.texi | 99 ++
.../gcc.dg/builtin-typeinfo-bitint.c | 82 ++
gcc/testsuite/gcc.dg/builtin-typeinfo-c23.c | 72 ++
.../gcc.dg/builtin-typeinfo-errors.c | 28 +
gcc/testsuite/gcc.dg/builtin-typeinfo-fixed.c | 58 +
gcc/testsuite/gcc.dg/builtin-typeinfo.c | 429 +++++++
gcc/Makefile.in | 1 +
gcc/c-family/c-common.h | 1 +
gcc/kcfi-typeinfo.h | 31 +
gcc/selftest.h | 1 +
gcc/c-family/c-common.cc | 2 +
gcc/c/c-parser.cc | 72 ++
gcc/kcfi-typeinfo.cc | 1071 +++++++++++++++++
gcc/selftest-run-tests.cc | 1 +
14 files changed, 1948 insertions(+)
create mode 100644 gcc/testsuite/gcc.dg/builtin-typeinfo-bitint.c
create mode 100644 gcc/testsuite/gcc.dg/builtin-typeinfo-c23.c
create mode 100644 gcc/testsuite/gcc.dg/builtin-typeinfo-errors.c
create mode 100644 gcc/testsuite/gcc.dg/builtin-typeinfo-fixed.c
create mode 100644 gcc/testsuite/gcc.dg/builtin-typeinfo.c
create mode 100644 gcc/kcfi-typeinfo.h
create mode 100644 gcc/kcfi-typeinfo.cc
diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index cc182b850140..6491de935eda 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -18185,6 +18185,105 @@ which will cause a @code{NULL} pointer to be used for the unsafe case.
@enddefbuiltin
+@defbuiltin{{unsigned int} __builtin_linux_abi_kcfi_hash (@var{type})}
+
+The built-in function @code{__builtin_linux_abi_kcfi_hash} returns a hash value
+for the given type @var{type} (which is a type, not an expression).
+The hash is computed using the FNV-1a algorithm on the type's mangled
+name representation, which follows a subset of the
+@uref{https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling,
+Itanium C++ ABI} conventions adapted for C types. (See
+@code{__builtin_linux_abi_kcfi_name} for the string representation.)
+
+This built-in is primarily intended for kernel control flow integrity (KCFI)
+implementations and other type-aware runtime systems that need to generate
+consistent type identifiers. The hash value is a 32-bit unsigned integer.
+
+Key characteristics of the hash:
+@itemize @bullet
+@item
+The hash is consistent for the same type across different translation units.
+@item
+Typedefs are recursively canonicalized down to integral type name or named
+struct, union, or enum tag name.
+@item
+Typedefs of anonymous structs, unions, and enums preserve the typedef name
+in the hash calculation (e.g., @code{typedef struct @{ int x; @} foo_t;}
+uses @code{foo_t} in the hash).
+@item
+Type qualifiers (@code{const}, @code{volatile}, @code{restrict}) affect
+the hash value.
+@item
+Function types include parameter types and variadic markers in the hash.
+@end itemize
+
+For example:
+@smallexample
+typedef struct @{ int x; @} mytype_t;
+unsigned int hash1 = __builtin_linux_abi_kcfi_hash (mytype_t);
+unsigned int hash2 = __builtin_linux_abi_kcfi_hash (struct @{ int x; @});
+/* hash1 != hash2 because the typedef name is preserved */
+
+void func(int x, char y);
+unsigned int hash3 = __builtin_linux_abi_kcfi_hash (typeof (func));
+/* Returns hash for function type "void(int, char)" */
+@end smallexample
+
+@emph{Note:} This construct is only available for C and Objective-C@.
+For C++, see @code{std::type_info::hash_code}.
+
+@enddefbuiltin
+
+@defbuiltin{{const char *} __builtin_linux_abi_kcfi_name (@var{type})}
+
+The built-in function @code{__builtin_linux_abi_kcfi_name} returns a string
+containing the mangled name representation of the given type @var{type}
+(which is a type, not an expression). The string follows a subset of the
+@uref{https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling,
+Itanium C++ ABI} mangling conventions adapted for C types. (See
+@code{__builtin_linux_abi_kcfi_hash} for the unsigned 32-bit hash
+representation.)
+
+The returned string is a compile-time constant suitable for use in
+string comparisons, debugging output, or other type introspection needs.
+The string begins with @code{_ZTS} followed by the encoded type information.
+
+Mangling examples:
+@itemize @bullet
+@item
+@code{int} becomes @code{"_ZTSi"}
+@item
+@code{char *} becomes @code{"_ZTSPc"}
+@item
+@code{const int} becomes @code{"_ZTSKi"}
+@item
+@code{int[10]} becomes @code{"_ZTSA10_i"}
+@item
+@code{void (*)(int)} becomes @code{"_ZTSPFviE"}
+@item
+@code{struct foo} becomes @code{"_ZTS3foo"}
+@item
+@code{typedef struct @{ int x; @} bar_t;} becomes @code{"_ZTS5bar_t"}
+@end itemize
+
+The mangling preserves typedef names for anonymous compound types, which
+is particularly useful for distinguishing between different typedefs of
+structurally identical anonymous types:
+
+@smallexample
+typedef struct @{ int x; @} type_a;
+typedef struct @{ int x; @} type_b;
+/* name_a is "_ZTS6type_a", name_b is "_ZTS6type_b": different despite
+ identical structure. */
+const char *name_a = __builtin_linux_abi_kcfi_name (type_a);
+const char *name_b = __builtin_linux_abi_kcfi_name (type_b);
+@end smallexample
+
+@emph{Note:} This construct is only available for C and Objective-C@.
+For C++, see @code{std::type_info::name}.
+
+@enddefbuiltin
+
@defbuiltin{int __builtin_types_compatible_p (@var{type1}, @var{type2})}
You can use the built-in function @code{__builtin_types_compatible_p} to
diff --git a/gcc/testsuite/gcc.dg/builtin-typeinfo-bitint.c b/gcc/testsuite/gcc.dg/builtin-typeinfo-bitint.c
new file mode 100644
index 000000000000..f0a8156ec130
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-typeinfo-bitint.c
@@ -0,0 +1,82 @@
+/* Test KCFI type mangling of _BitInt types using
+ __builtin_linux_abi_kcfi_name. Bit-precise integers require C23. */
+/* { dg-do run } */
+/* { dg-require-effective-target bitint } */
+/* { dg-options "-std=c23" } */
+
+#include <stdio.h>
+#include <string.h>
+
+int pass, fail;
+
+#define TEST_STRING(expr, expected_string) \
+ do { \
+ const char *actual_string = __builtin_linux_abi_kcfi_name(typeof(expr)); \
+ printf("Testing %s: ", #expr); \
+ if (strcmp(actual_string, "_ZTS" expected_string) == 0) { \
+ printf("PASS (%s)\n", actual_string); \
+ pass ++; \
+ } else { \
+ printf("FAIL\n"); \
+ printf(" Expected: %s\n", "_ZTS" expected_string); \
+ printf(" Actual: %s\n", actual_string); \
+ fail ++; \
+ } \
+ } while (0)
+
+int main(void)
+{
+ printf("Testing KCFI Typeinfo Mangling of _BitInt\n");
+ printf("======================================================\n");
+
+ /* Bit-precise integer types follow the Itanium C++ ABI: DB<N>_ for
+ signed _BitInt(N), DU<N>_ for unsigned _BitInt(N). */
+ TEST_STRING(_BitInt(2), "DB2_");
+ TEST_STRING(_BitInt(7), "DB7_");
+ TEST_STRING(_BitInt(17), "DB17_");
+ TEST_STRING(_BitInt(32), "DB32_");
+ TEST_STRING(_BitInt(64), "DB64_");
+ TEST_STRING(signed _BitInt(17), "DB17_");
+
+ /* Unsigned bit-precise integers use DU. The minimum unsigned width
+ is 1 (the minimum signed width is 2). */
+ TEST_STRING(unsigned _BitInt(1), "DU1_");
+ TEST_STRING(unsigned _BitInt(7), "DU7_");
+ TEST_STRING(unsigned _BitInt(32), "DU32_");
+ TEST_STRING(unsigned _BitInt(64), "DU64_");
+
+ /* Qualifiers combine with the bit-precise encoding (Itanium order:
+ restrict, volatile, const). */
+ TEST_STRING(const _BitInt(32), "KDB32_");
+ TEST_STRING(volatile _BitInt(32), "VDB32_");
+
+ /* Pointers to _BitInt. */
+ TEST_STRING(_BitInt(32) *, "PDB32_");
+ TEST_STRING(unsigned _BitInt(7) *, "PDU7_");
+
+ /* _BitInt in function signatures (KCFI's actual use case). */
+ extern void func_bitint(_BitInt(32) x);
+ extern void func_ubitint(unsigned _BitInt(64) x);
+ extern _BitInt(64) func_ret_bitint(void);
+ extern void func_bitint_mixed(int x, _BitInt(32) y, unsigned _BitInt(7) z);
+ TEST_STRING(func_bitint, "FvDB32_E");
+ TEST_STRING(func_ubitint, "FvDU64_E");
+ TEST_STRING(func_ret_bitint, "FDB64_vE");
+ TEST_STRING(func_bitint_mixed, "FviDB32_DU7_E");
+
+ /* Distinct widths and signedness must produce distinct manglings. */
+ if (strcmp(__builtin_linux_abi_kcfi_name(typeof(_BitInt(32))),
+ __builtin_linux_abi_kcfi_name(typeof(_BitInt(64)))) == 0)
+ { printf("FAIL: _BitInt(32) and _BitInt(64) collide\n"); fail++; }
+ else
+ { printf("PASS: distinct widths are distinct\n"); pass++; }
+ if (strcmp(__builtin_linux_abi_kcfi_name(typeof(_BitInt(32))),
+ __builtin_linux_abi_kcfi_name(typeof(unsigned _BitInt(32)))) == 0)
+ { printf("FAIL: signed/unsigned _BitInt(32) collide\n"); fail++; }
+ else
+ { printf("PASS: signed and unsigned are distinct\n"); pass++; }
+
+ printf("\n================================================================\n");
+ printf("Passed: %d Failed: %d (%d total tests)\n", pass, fail, pass + fail);
+ return fail;
+}
diff --git a/gcc/testsuite/gcc.dg/builtin-typeinfo-c23.c b/gcc/testsuite/gcc.dg/builtin-typeinfo-c23.c
new file mode 100644
index 000000000000..bb7761643748
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-typeinfo-c23.c
@@ -0,0 +1,72 @@
+/* Test KCFI type mangling of C23 nullptr_t and the extended floating-point
+ formats using __builtin_linux_abi_kcfi_name. Optional-type blocks are
+ guarded by feature macros so the test runs on any target. */
+/* { dg-do run } */
+/* { dg-options "-std=c23" } */
+
+#include <stdio.h>
+#include <string.h>
+
+int pass, fail;
+
+#define TEST_STRING(expr, expected_string) \
+ do { \
+ const char *actual_string = __builtin_linux_abi_kcfi_name(typeof(expr)); \
+ printf("Testing %s: ", #expr); \
+ if (strcmp(actual_string, "_ZTS" expected_string) == 0) { \
+ printf("PASS (%s)\n", actual_string); \
+ pass ++; \
+ } else { \
+ printf("FAIL\n"); \
+ printf(" Expected: %s\n", "_ZTS" expected_string); \
+ printf(" Actual: %s\n", actual_string); \
+ fail ++; \
+ } \
+ } while (0)
+
+int main(void)
+{
+ printf("Testing KCFI Typeinfo Mangling of nullptr_t and extended floats\n");
+ printf("======================================================\n");
+
+ /* The type of C23 nullptr (std::nullptr_t): Dn. */
+ TEST_STRING(typeof(nullptr), "Dn");
+ TEST_STRING(typeof(nullptr) *, "PDn");
+ extern void func_nullptr(typeof(nullptr) x);
+ TEST_STRING(func_nullptr, "FvDnE");
+
+ /* Extended floating-point formats follow the Itanium C++ ABI. Target FP
+ types (e.g. x86 __float128) are handled by the target mangle hook.
+ Each block is guarded by its feature macro so unsupported targets
+ skip it rather than failing to compile. */
+#ifdef __FLT16_MANT_DIG__
+ TEST_STRING(_Float16, "DF16_");
+#endif
+#ifdef __FLT32_MANT_DIG__
+ TEST_STRING(_Float32, "DF32_");
+#endif
+#ifdef __FLT64_MANT_DIG__
+ TEST_STRING(_Float64, "DF64_");
+#endif
+#ifdef __FLT128_MANT_DIG__
+ TEST_STRING(_Float128, "DF128_");
+#endif
+#if defined(__SIZEOF_FLOAT128__) && defined(__FLT128_MANT_DIG__)
+ /* In C, __float128 is the same type as _Float128 where both exist, so it
+ must share the same mangling. */
+ if (strcmp(__builtin_linux_abi_kcfi_name(typeof(__float128)),
+ __builtin_linux_abi_kcfi_name(typeof(_Float128))) == 0)
+ { printf("PASS: __float128 matches _Float128\n"); pass++; }
+ else
+ { printf("FAIL: __float128 mangling differs from _Float128\n"); fail++; }
+#endif
+#ifdef __DEC32_MANT_DIG__
+ TEST_STRING(_Decimal32, "Df");
+ TEST_STRING(_Decimal64, "Dd");
+ TEST_STRING(_Decimal128, "De");
+#endif
+
+ printf("\n================================================================\n");
+ printf("Passed: %d Failed: %d (%d total tests)\n", pass, fail, pass + fail);
+ return fail;
+}
diff --git a/gcc/testsuite/gcc.dg/builtin-typeinfo-errors.c b/gcc/testsuite/gcc.dg/builtin-typeinfo-errors.c
new file mode 100644
index 000000000000..5d68425c229c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-typeinfo-errors.c
@@ -0,0 +1,28 @@
+/* Test error handling for __builtin_linux_abi_kcfi_name and __builtin_linux_abi_kcfi_hash. */
+/* { dg-do compile } */
+
+int main() {
+ /* Test missing arguments */
+ const char *result1 = __builtin_linux_abi_kcfi_name(); /* { dg-error "expected specifier-qualifier-list before '\\)'" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_name'" "" { target *-*-* } .-1 } */
+ unsigned int result2 = __builtin_linux_abi_kcfi_hash(); /* { dg-error "expected specifier-qualifier-list before '\\)'" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_hash'" "" { target *-*-* } .-1 } */
+
+ /* Test wrong argument types (expressions instead of type names) */
+ const char *result3 = __builtin_linux_abi_kcfi_name(42); /* { dg-error "expected specifier-qualifier-list before numeric constant" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_name'" "" { target *-*-* } .-1 } */
+ unsigned int result4 = __builtin_linux_abi_kcfi_hash(42); /* { dg-error "expected specifier-qualifier-list before numeric constant" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_hash'" "" { target *-*-* } .-1 } */
+
+ int x = 5;
+ const char *result5 = __builtin_linux_abi_kcfi_name(x); /* { dg-error "expected specifier-qualifier-list before" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_name'" "" { target *-*-* } .-1 } */
+ unsigned int result6 = __builtin_linux_abi_kcfi_hash(x); /* { dg-error "expected specifier-qualifier-list before" } */
+ /* { dg-error "expected type name in '__builtin_linux_abi_kcfi_hash'" "" { target *-*-* } .-1 } */
+
+ /* Test too many arguments */
+ const char *result7 = __builtin_linux_abi_kcfi_name(int, int); /* { dg-error "expected '\\)' before ','" } */
+ unsigned int result8 = __builtin_linux_abi_kcfi_hash(int, int); /* { dg-error "expected '\\)' before ','" } */
+
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/builtin-typeinfo-fixed.c b/gcc/testsuite/gcc.dg/builtin-typeinfo-fixed.c
new file mode 100644
index 000000000000..02ad44d76767
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-typeinfo-fixed.c
@@ -0,0 +1,58 @@
+/* Test KCFI type mangling of fixed-point types (ISO/IEC TR 18037) using
+ __builtin_linux_abi_kcfi_name. Fixed-point types have no Itanium C++ ABI
+ mangling, so KCFI emits a deterministic vendor-extended name. Rather than
+ hard-code the target-dependent encoding, verify the two properties KCFI
+ actually relies on: every fixed-point type mangles (no ICE) to a vendor
+ name, and distinct fixed-point types get distinct manglings. */
+/* { dg-do run } */
+/* { dg-require-effective-target fixed_point } */
+
+#include <stdio.h>
+#include <string.h>
+
+int pass, fail;
+
+/* Each mangling is the typeinfo name "_ZTS" followed by a non-empty
+ vendor-extended builtin type ("u"...). */
+#define CHECK_VENDOR(name, s) \
+ do { \
+ printf("%s = %s: ", name, s); \
+ if (strncmp(s, "_ZTSu", 5) == 0 && s[5] != '\0') { printf("PASS\n"); pass++; } \
+ else { printf("FAIL (not a vendor name)\n"); fail++; } \
+ } while (0)
+
+int main(void)
+{
+ const char *fr = __builtin_linux_abi_kcfi_name(typeof((_Fract) 0));
+ const char *ufr = __builtin_linux_abi_kcfi_name(typeof((unsigned _Fract) 0));
+ const char *sfr = __builtin_linux_abi_kcfi_name(typeof((_Sat _Fract) 0));
+ const char *lfr = __builtin_linux_abi_kcfi_name(typeof((long _Fract) 0));
+ const char *ac = __builtin_linux_abi_kcfi_name(typeof((_Accum) 0));
+ const char *sac = __builtin_linux_abi_kcfi_name(typeof((_Sat _Accum) 0));
+
+ CHECK_VENDOR("_Fract", fr);
+ CHECK_VENDOR("unsigned _Fract", ufr);
+ CHECK_VENDOR("_Sat _Fract", sfr);
+ CHECK_VENDOR("long _Fract", lfr);
+ CHECK_VENDOR("_Accum", ac);
+ CHECK_VENDOR("_Sat _Accum", sac);
+
+ /* Distinctness: signedness, saturation, size and the fract/accum
+ distinction must each change the mangling. */
+ const char *all[] = { fr, ufr, sfr, lfr, ac, sac };
+ int n = sizeof(all) / sizeof(all[0]);
+ for (int i = 0; i < n; i++)
+ for (int j = i + 1; j < n; j++)
+ {
+ if (strcmp(all[i], all[j]) == 0)
+ {
+ printf("FAIL: manglings %d and %d collide (%s)\n", i, j, all[i]);
+ fail++;
+ }
+ else
+ pass++;
+ }
+
+ printf("\nPassed: %d Failed: %d (%d total tests)\n", pass, fail, pass + fail);
+ return fail;
+}
diff --git a/gcc/testsuite/gcc.dg/builtin-typeinfo.c b/gcc/testsuite/gcc.dg/builtin-typeinfo.c
new file mode 100644
index 000000000000..d39bfb0438ad
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-typeinfo.c
@@ -0,0 +1,429 @@
+/* Test KCFI type mangling using __builtin_linux_abi_kcfi_name. */
+/* { dg-do run } */
+/* { dg-options "-std=gnu99" } */
+
+#include <stdio.h>
+#include <string.h>
+#include <stdarg.h>
+
+int pass, fail;
+
+#define TEST_STRING_FULL(expr, expected_full) \
+ do { \
+ const char *actual_string = __builtin_linux_abi_kcfi_name(typeof(expr)); \
+ const char *expected_str = (expected_full); \
+ printf("Testing %s: ", #expr); \
+ if (strcmp(actual_string, expected_str) == 0) { \
+ printf("PASS (%s)\n", actual_string); \
+ pass ++; \
+ } else { \
+ printf("FAIL\n"); \
+ printf(" Expected: %s\n", expected_str); \
+ printf(" Actual: %s\n", actual_string); \
+ fail ++; \
+ } \
+ } while (0)
+
+#define TEST_STRING(expr, expected_string) \
+ TEST_STRING_FULL(expr, "_ZTS" expected_string)
+
+int main(void)
+{
+ printf("Testing KCFI Typeinfo Mangling\n");
+ printf("======================================================\n");
+
+ /* Test basic types */
+ TEST_STRING(void, "v");
+ TEST_STRING(char, "c");
+ TEST_STRING(int, "i");
+ TEST_STRING(short, "s");
+ TEST_STRING(long, "l");
+ TEST_STRING(float, "f");
+ TEST_STRING(double, "d");
+
+ /* Test qualified types */
+ TEST_STRING(const int, "Ki");
+ TEST_STRING(volatile int, "Vi");
+
+ /* Test pointer types */
+ TEST_STRING(char*, "Pc");
+ TEST_STRING(int*, "Pi");
+ TEST_STRING(void*, "Pv");
+ TEST_STRING(const char*, "PKc");
+
+ /* Test array types */
+ TEST_STRING(int[10], "A10_i");
+ TEST_STRING(char[20], "A20_c");
+ TEST_STRING(short[], "A_s");
+
+ /* Test basic function types */
+ extern void func_void(void);
+ extern void func_char(char x);
+ extern void func_short(short x);
+ extern void func_int(int x);
+ extern void func_long(long x);
+ TEST_STRING(func_void, "FvvE");
+ TEST_STRING(func_char, "FvcE");
+ TEST_STRING(func_short, "FvsE");
+ TEST_STRING(func_int, "FviE");
+ TEST_STRING(func_long, "FvlE");
+
+ /* Test functions with unsigned types */
+ extern void func_unsigned_char(unsigned char x);
+ extern void func_unsigned_short(unsigned short x);
+ extern void func_unsigned_int(unsigned int x);
+ TEST_STRING(func_unsigned_char, "FvhE");
+ TEST_STRING(func_unsigned_short, "FvtE");
+ TEST_STRING(func_unsigned_int, "FvjE");
+
+ /* Test functions with signed types */
+ extern void func_signed_char(signed char x);
+ extern void func_signed_short(signed short x);
+ extern void func_signed_int(signed int x);
+ TEST_STRING(func_signed_char, "FvaE");
+ TEST_STRING(func_signed_short, "FvsE");
+ TEST_STRING(func_signed_int, "FviE");
+
+ /* Test functions with pointer types */
+ extern void func_void_ptr(void *x);
+ extern void func_char_ptr(char *x);
+ extern void func_short_ptr(short *x);
+ extern void func_int_ptr(int *x);
+ extern void func_int_array(int arr[]); /* Decays to "int *". */
+ extern void func_long_ptr(long *x);
+ TEST_STRING(func_void_ptr, "FvPvE");
+ TEST_STRING(func_char_ptr, "FvPcE");
+ TEST_STRING(func_short_ptr, "FvPsE");
+ TEST_STRING(func_int_ptr, "FvPiE");
+ TEST_STRING(func_int_array, "FvPiE");
+ TEST_STRING(func_long_ptr, "FvPlE");
+
+ /* Test functions with const qualifiers */
+ extern void func_const_void_ptr(const void *x);
+ extern void func_const_char_ptr(const char *x);
+ extern void func_const_short_ptr(const short *x);
+ extern void func_const_int_ptr(const int *x);
+ extern void func_const_long_ptr(const long *x);
+ TEST_STRING(func_const_void_ptr, "FvPKvE");
+ TEST_STRING(func_const_char_ptr, "FvPKcE");
+ TEST_STRING(func_const_short_ptr, "FvPKsE");
+ TEST_STRING(func_const_int_ptr, "FvPKiE");
+ TEST_STRING(func_const_long_ptr, "FvPKlE");
+
+ /* Test nested pointers */
+ extern void func_int_ptr_ptr(int **x);
+ extern void func_char_ptr_ptr(char **x);
+ TEST_STRING(func_int_ptr_ptr, "FvPPiE");
+ TEST_STRING(func_char_ptr_ptr, "FvPPcE");
+
+ /* Test top-level qualifiers on pointer parameters: as for other
+ parameters, they are not part of the function type, so they are
+ not mangled. The pointee's qualifiers are. */
+ struct qual_s;
+ extern void func_int_ptr_const(int *const x);
+ extern void func_int_ptr_restrict(int *restrict x);
+ extern void func_int_ptr_volatile(int *volatile x);
+ extern void func_const_int_ptr_const(const int *const x);
+ extern void func_struct_ptr_const(struct qual_s *const x);
+ extern void func_int_array_const(int x[const]);
+ TEST_STRING(func_int_ptr_const, "FvPiE");
+ TEST_STRING(func_int_ptr_restrict, "FvPiE");
+ TEST_STRING(func_int_ptr_volatile, "FvPiE");
+ TEST_STRING(func_const_int_ptr_const, "FvPKiE");
+ TEST_STRING(func_struct_ptr_const, "FvP6qual_sE");
+ TEST_STRING(func_int_array_const, "FvPiE");
+
+ /* Test _Bool (C23 bool). Expected strings match Clang. */
+ TEST_STRING(_Bool, "b");
+ extern void func_bool(_Bool x);
+ extern _Bool func_ret_bool(void);
+ extern void func_bool_ptr(_Bool *x);
+ extern void func_const_bool_ptr(const _Bool *x);
+ TEST_STRING(func_bool, "FvbE");
+ TEST_STRING(func_ret_bool, "FbvE");
+ TEST_STRING(func_bool_ptr, "FvPbE");
+ TEST_STRING(func_const_bool_ptr, "FvPKbE");
+
+ /* Test restrict below the top level, and the Itanium qualifier order
+ (restrict, volatile, const). Expected strings match Clang. */
+ extern void func_restrict_ptr_ptr(int *restrict *x);
+ extern void func_rvk_ptr_ptr(int *const volatile restrict *x);
+ extern void func_cv_int_ptr(const volatile int *x);
+ extern void func_cb_restrict(void (*cb)(int *restrict));
+ extern void func_cb_ret_restrict(int *restrict *(*cb)(void));
+ TEST_STRING(func_restrict_ptr_ptr, "FvPrPiE");
+ TEST_STRING(func_rvk_ptr_ptr, "FvPrVKPiE");
+ TEST_STRING(func_cv_int_ptr, "FvPVKiE");
+ TEST_STRING(func_cb_restrict, "FvPFvPiEE");
+ TEST_STRING(func_cb_ret_restrict, "FvPFPrPivEE");
+
+ /* Test multiple parameters */
+ extern void func_int_char(int x, char y);
+ extern void func_char_int(char x, int y);
+ extern void func_two_int(int x, int y);
+ TEST_STRING(func_int_char, "FvicE");
+ TEST_STRING(func_char_int, "FvciE");
+ TEST_STRING(func_two_int, "FviiE");
+
+ /* Test return types */
+ extern int func_return_int(void);
+ extern char func_return_char(void);
+ extern void* func_return_ptr(void);
+ TEST_STRING(func_return_int, "FivE");
+ TEST_STRING(func_return_char, "FcvE");
+ TEST_STRING(func_return_ptr, "FPvvE");
+
+ /* Test function pointer parameters */
+ extern void func_fptr_void(void (*fp)(void));
+ extern void func_fptr_int(void (*fp)(int));
+ extern void func_fptr_ret_int(int (*fp)(void));
+ TEST_STRING(func_fptr_void, "FvPFvvEE");
+ TEST_STRING(func_fptr_int, "FvPFviEE");
+ TEST_STRING(func_fptr_ret_int, "FvPFivEE");
+
+ /* Test variadic functions */
+ struct audit_context { int dummy; };
+ extern void func_variadic_simple(const char *fmt, ...);
+ extern void func_variadic_mixed(int x, const char *fmt, ...);
+ extern void func_variadic_multi(int x, char y, const char *fmt, ...);
+ extern void audit_log_pattern(struct audit_context *ctx, unsigned int gfp_mask,
+ int type, const char *fmt, ...);
+ TEST_STRING(func_variadic_simple, "FvPKczE");
+ TEST_STRING(func_variadic_mixed, "FviPKczE");
+ TEST_STRING(func_variadic_multi, "FvicPKczE");
+ TEST_STRING(audit_log_pattern, "FvP13audit_contextjiPKczE");
+
+ /* Test mixed const/non-const */
+ extern void func_const_mixed(int x, const char *fmt);
+ TEST_STRING(func_const_mixed, "FviPKcE");
+
+ /* Test named struct types */
+ struct test_struct_a { int x; };
+ struct test_struct_b { char y; };
+ struct test_struct_c { void *ptr; };
+ TEST_STRING(struct test_struct_a, "13test_struct_a");
+ extern void func_struct_a_ptr(struct test_struct_a *x);
+ extern void func_struct_b_ptr(struct test_struct_b *x);
+ extern void func_struct_c_ptr(struct test_struct_c *x);
+ TEST_STRING(func_struct_a_ptr, "FvP13test_struct_aE");
+ TEST_STRING(func_struct_b_ptr, "FvP13test_struct_bE");
+ TEST_STRING(func_struct_c_ptr, "FvP13test_struct_cE");
+
+ /* Test const named struct types */
+ extern void func_const_struct_a_ptr(const struct test_struct_a *x);
+ extern void func_const_struct_b_ptr(const struct test_struct_b *x);
+ extern void func_const_struct_c_ptr(const struct test_struct_c *x);
+ TEST_STRING(func_const_struct_a_ptr, "FvPK13test_struct_aE");
+ TEST_STRING(func_const_struct_b_ptr, "FvPK13test_struct_bE");
+ TEST_STRING(func_const_struct_c_ptr, "FvPK13test_struct_cE");
+
+ /* Test named union types */
+ union test_union_a { int x; float y; };
+ union test_union_b { char a; void *b; };
+ TEST_STRING(union test_union_a, "12test_union_a");
+ extern void func_union_a_ptr(union test_union_a *x);
+ extern void func_union_b_ptr(union test_union_b *x);
+ TEST_STRING(func_union_a_ptr, "FvP12test_union_aE");
+ TEST_STRING(func_union_b_ptr, "FvP12test_union_bE");
+
+ /* Test enum types: distinct from int */
+ enum test_enum_a { ENUM_A_VAL };
+ enum test_enum_b { ENUM_B_VAL };
+ TEST_STRING(enum test_enum_a, "11test_enum_a");
+ extern void func_enum_a_ptr(enum test_enum_a *x);
+ extern void func_enum_b_ptr(enum test_enum_b *x);
+ TEST_STRING(func_enum_a_ptr, "FvP11test_enum_aE");
+ TEST_STRING(func_enum_b_ptr, "FvP11test_enum_bE");
+
+ /* Test union member discrimination */
+ struct tasklet {
+ int state;
+ union {
+ void (*func)(unsigned long data);
+ void (*callback)(struct tasklet *t);
+ };
+ unsigned long data;
+ } tasklet_instance;
+ TEST_STRING(tasklet_instance, "7tasklet");
+ struct tasklet *p = &tasklet_instance;
+ extern void tasklet_callback_function(struct tasklet *t);
+ extern void tasklet_func_function(unsigned long data);
+ TEST_STRING(tasklet_func_function, "FvmE");
+ TEST_STRING(*p->func, "FvmE");
+ TEST_STRING(tasklet_callback_function, "FvP7taskletE");
+ TEST_STRING(*p->callback, "FvP7taskletE");
+
+ /* Test struct return pointers */
+ extern struct test_struct_a* func_ret_struct_a_ptr(void);
+ extern struct test_struct_b* func_ret_struct_b_ptr(void);
+ extern struct test_struct_c* func_ret_struct_c_ptr(void);
+ TEST_STRING(func_ret_struct_a_ptr, "FP13test_struct_avE");
+ TEST_STRING(func_ret_struct_b_ptr, "FP13test_struct_bvE");
+ TEST_STRING(func_ret_struct_c_ptr, "FP13test_struct_cvE");
+
+ /* Test struct by-value parameters */
+ extern void func_struct_a_val(struct test_struct_a x);
+ extern void func_struct_b_val(struct test_struct_b x);
+ extern void func_struct_c_val(struct test_struct_c x);
+ TEST_STRING(func_struct_a_val, "Fv13test_struct_aE");
+ TEST_STRING(func_struct_b_val, "Fv13test_struct_bE");
+ TEST_STRING(func_struct_c_val, "Fv13test_struct_cE");
+
+ /* Test struct return by-value */
+ extern struct test_struct_a func_ret_struct_a_val(void);
+ extern struct test_struct_b func_ret_struct_b_val(void);
+ extern struct test_struct_c func_ret_struct_c_val(void);
+ TEST_STRING(func_ret_struct_a_val, "F13test_struct_avE");
+ TEST_STRING(func_ret_struct_b_val, "F13test_struct_bvE");
+ TEST_STRING(func_ret_struct_c_val, "F13test_struct_cvE");
+
+ /* Test mixed struct parameters */
+ extern void func_struct_a_b(struct test_struct_a *a, struct test_struct_b *b);
+ extern void func_struct_b_a(struct test_struct_b *b, struct test_struct_a *a);
+ TEST_STRING(func_struct_a_b, "FvP13test_struct_aP13test_struct_bE");
+ TEST_STRING(func_struct_b_a, "FvP13test_struct_bP13test_struct_aE");
+
+ /* Test anonymous struct typedefs */
+ typedef struct { int x; } typedef_struct_x;
+ typedef struct { int y; } typedef_struct_y;
+ TEST_STRING(typedef_struct_x, "16typedef_struct_x");
+ extern void func_typedef_x_ptr(typedef_struct_x *x);
+ extern void func_typedef_y_ptr(typedef_struct_y *y);
+ TEST_STRING(func_typedef_x_ptr, "FvP16typedef_struct_xE");
+ TEST_STRING(func_typedef_y_ptr, "FvP16typedef_struct_yE");
+ extern void func_typedef_x(typedef_struct_x x);
+ TEST_STRING(func_typedef_x, "Fv16typedef_struct_xE");
+ extern void func_typedef_x_const(const typedef_struct_x x);
+ extern void func_typedef_x_ptr_const(typedef_struct_x *const x);
+ TEST_STRING(func_typedef_x_const, "Fv16typedef_struct_xE");
+ TEST_STRING(func_typedef_x_ptr_const, "FvP16typedef_struct_xE");
+
+ /* Test anonymous union typedefs */
+ typedef union { int x; short a; } typedef_union_x;
+ typedef union { int y; short b; } typedef_union_y;
+ TEST_STRING(typedef_union_x, "15typedef_union_x");
+ extern void func_typedef_union_x_ptr(typedef_union_x *x);
+ extern void func_typedef_union_y_ptr(typedef_union_y *y);
+ TEST_STRING(func_typedef_union_x_ptr, "FvP15typedef_union_xE");
+ TEST_STRING(func_typedef_union_y_ptr, "FvP15typedef_union_yE");
+ extern void func_typedef_union_x(typedef_union_x x);
+ TEST_STRING(func_typedef_union_x, "Fv15typedef_union_xE");
+
+ /* Test anonymous enum typedefs */
+ typedef enum { STEP_1, STEP_2 } typedef_enum_x;
+ typedef enum { STEP_A, STEP_B } typedef_enum_y;
+ TEST_STRING(typedef_enum_x, "14typedef_enum_x");
+ extern void func_typedef_enum_x_ptr(typedef_enum_x *x);
+ extern void func_typedef_enum_y_ptr(typedef_enum_y *y);
+ TEST_STRING(func_typedef_enum_x_ptr, "FvP14typedef_enum_xE");
+ TEST_STRING(func_typedef_enum_y_ptr, "FvP14typedef_enum_yE");
+ extern void func_typedef_enum_x(typedef_enum_x x);
+ TEST_STRING(func_typedef_enum_x, "Fv14typedef_enum_xE");
+
+ /* Test basic typedef vs open-coded function types: should be the same. */
+ typedef void (*func_type_typedef)(int, char);
+ TEST_STRING(func_type_typedef, "PFvicE");
+ extern void func_with_typedef_param(func_type_typedef fp);
+ extern void func_with_opencoded_param(void (*fp)(int, char));
+ TEST_STRING(func_with_typedef_param, "FvPFvicEE");
+ TEST_STRING(func_with_opencoded_param, "FvPFvicEE");
+
+ /* Test return function pointer types */
+ typedef int (*ret_func_type_typedef)(void);
+ TEST_STRING(ret_func_type_typedef, "PFivE");
+ extern ret_func_type_typedef func_ret_typedef_param(void);
+ extern int (*func_ret_opencoded_param(void))(void);
+ TEST_STRING(func_ret_typedef_param, "FPFivEvE");
+ TEST_STRING(func_ret_opencoded_param, "FPFivEvE");
+
+ /* Test additional type combos */
+ extern void func_float(float x);
+ extern void func_double_ptr(double *x);
+ extern void func_float_ptr(float *x);
+ extern void func_void_ptr_ptr(void **x);
+ extern void func_ptr_val(int *x, int y);
+ extern void func_val_ptr(int x, int *y);
+ extern float func_return_float(void);
+ extern double func_return_double(void);
+ TEST_STRING(func_float, "FvfE");
+ TEST_STRING(func_double_ptr, "FvPdE");
+ TEST_STRING(func_float_ptr, "FvPfE");
+ TEST_STRING(func_void_ptr_ptr, "FvPPvE");
+ TEST_STRING(func_ptr_val, "FvPiiE");
+ TEST_STRING(func_val_ptr, "FviPiE");
+ TEST_STRING(func_return_float, "FfvE");
+ TEST_STRING(func_return_double, "FdvE");
+
+ /* Test complex types (C99 _Complex): 'C' + element type. */
+ TEST_STRING(_Complex float, "Cf");
+ TEST_STRING(_Complex double, "Cd");
+ /* long double's own mangling is target-specific (e.g. "g" for 128-bit
+ IEEE long double), so build the expectation from it. */
+ char complex_ld[64];
+ snprintf(complex_ld, sizeof(complex_ld), "_ZTSC%s",
+ __builtin_linux_abi_kcfi_name(long double) + strlen("_ZTS"));
+ TEST_STRING_FULL(_Complex long double, complex_ld);
+ TEST_STRING(_Complex int, "Ci"); /* GNU extension. */
+ TEST_STRING(_Complex float *, "PCf");
+ extern void func_complex(_Complex double x);
+ extern _Complex float func_ret_complex(void);
+ TEST_STRING(func_complex, "FvCdE");
+ TEST_STRING(func_ret_complex, "FCfvE");
+
+ /* Test VLA types: should be all the same. */
+ extern void func_vla_1d(int n, int arr[n]);
+ extern void func_vla_empty(int n, int arr[]);
+ extern void func_vla_ptr(int n, int *arr);
+ TEST_STRING(func_vla_1d, "FviPiE");
+ TEST_STRING(func_vla_empty, "FviPiE");
+ TEST_STRING(func_vla_ptr, "FviPiE");
+
+ /* Test 2D VLA with fixed dimension: should be all the same. */
+ extern void func_vla_2d_first(int n, int arr[n][10]);
+ extern void func_vla_2d_empty(int n, int arr[][10]);
+ extern void func_vla_2d_ptr(int n, int (*arr)[10]);
+ TEST_STRING(func_vla_2d_first, "FviPA10_iE");
+ TEST_STRING(func_vla_2d_empty, "FviPA10_iE");
+ TEST_STRING(func_vla_2d_ptr, "FviPA10_iE");
+
+ /* Test 2D VLA with both dimensions variable: should be all the same. */
+ extern void func_vla_2d_both(int rows, int cols, int arr[rows][cols]);
+ extern void func_vla_2d_second(int rows, int cols, int arr[][cols]);
+ extern void func_vla_2d_star(int rows, int cols, int arr[*][cols]);
+ TEST_STRING(func_vla_2d_both, "FviiPA_iE");
+ TEST_STRING(func_vla_2d_second, "FviiPA_iE");
+ TEST_STRING(func_vla_2d_star, "FviiPA_iE");
+
+ /* Test a struct with a VLA member (a GCC extension). Mangling keys on
+ the struct tag, not its layout, so the VLA member never leaks into the
+ encoding: a named struct mangles as its tag name regardless. */
+ int vla_len = 8;
+ struct vla_member { int len; int data[vla_len]; };
+ struct vla_member vla_member_instance;
+ TEST_STRING(vla_member_instance, "10vla_member");
+ struct vla_member *vla_member_p = &vla_member_instance;
+ TEST_STRING(*vla_member_p, "10vla_member");
+ extern void func_vla_member_ptr(struct vla_member *x);
+ extern void func_vla_member_val(struct vla_member x);
+ TEST_STRING(func_vla_member_ptr, "FvP10vla_memberE");
+ TEST_STRING(func_vla_member_val, "Fv10vla_memberE");
+
+ /* Test recursive typedef canonicalization */
+ struct recursive_struct_test { int field; };
+ typedef struct recursive_struct_test recursive_struct_typedef_1;
+ typedef recursive_struct_typedef_1 recursive_struct_typedef_2;
+ extern void func_recursive_struct_test(struct recursive_struct_test *x);
+ TEST_STRING(func_recursive_struct_test, "FvP21recursive_struct_testE");
+
+ /* Test anonymous struct, union, enum types */
+ struct { int a; short b; } anon_struct;
+ union { int x; float y; } anon_union;
+ enum { ANON_VAL1, ANON_VAL2 } anon_enum;
+ TEST_STRING(anon_struct, "3$_0"); // <length>$_<counter>
+ TEST_STRING(anon_union, "3$_1"); // <length>$_<counter>
+ TEST_STRING(anon_enum, "3$_2"); // <length>$_<counter>
+
+ printf("\n================================================================\n");
+ printf("Passed: %d Failed: %d (%d total tests)\n", pass, fail, pass + fail);
+ return fail;
+}
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 83cb4884eb13..2560addbcf64 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1624,6 +1624,7 @@ OBJS = \
ira-emit.o \
ira-lives.o \
jump.o \
+ kcfi-typeinfo.o \
langhooks.o \
late-combine.o \
lcm.o \
diff --git a/gcc/c-family/c-common.h b/gcc/c-family/c-common.h
index 197db508c163..7dbc1b16057f 100644
--- a/gcc/c-family/c-common.h
+++ b/gcc/c-family/c-common.h
@@ -113,6 +113,7 @@ enum rid
RID_BUILTIN_HAS_ATTRIBUTE, RID_BUILTIN_ASSOC_BARRIER, RID_BUILTIN_STDC,
RID_BUILTIN_COUNTED_BY_REF, RID_BUILTIN_BSWAPG,
RID_BUILTIN_BITREVERSEG,
+ RID_BUILTIN_LINUX_ABI_KCFI_NAME, RID_BUILTIN_LINUX_ABI_KCFI_HASH,
RID_DFLOAT32, RID_DFLOAT64, RID_DFLOAT128, RID_DFLOAT64X,
/* TS 18661-3 keywords, in the same sequence as the TI_* values. */
diff --git a/gcc/kcfi-typeinfo.h b/gcc/kcfi-typeinfo.h
new file mode 100644
index 000000000000..092a682276af
--- /dev/null
+++ b/gcc/kcfi-typeinfo.h
@@ -0,0 +1,31 @@
+/* KCFI-compatible type mangling, based on Itanium C++ ABI.
+ Copyright (C) 2026 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_KCFI_TYPEINFO_H
+#define GCC_KCFI_TYPEINFO_H
+
+#include "tree.h"
+
+/* Get the typeinfo mangled name string for any C type. */
+extern std::string kcfi_typeinfo_get_name (tree type);
+
+/* Get the typeinfo hash for any C type. */
+extern uint32_t kcfi_typeinfo_get_hash (tree type);
+
+#endif /* GCC_KCFI_TYPEINFO_H */
diff --git a/gcc/selftest.h b/gcc/selftest.h
index acfc4a46e49b..a17a6fe1b789 100644
--- a/gcc/selftest.h
+++ b/gcc/selftest.h
@@ -240,6 +240,7 @@ extern void ipa_modref_tree_cc_tests ();
extern void json_cc_tests ();
extern void json_parser_cc_tests ();
extern void json_pointer_parsing_cc_tests ();
+extern void kcfi_typeinfo_cc_tests ();
extern void opt_suggestions_cc_tests ();
extern void optinfo_emit_json_cc_tests ();
extern void opts_cc_tests ();
diff --git a/gcc/c-family/c-common.cc b/gcc/c-family/c-common.cc
index 6380bec4515d..4836b60811d2 100644
--- a/gcc/c-family/c-common.cc
+++ b/gcc/c-family/c-common.cc
@@ -466,6 +466,8 @@ const struct c_common_resword c_common_reswords[] =
{ "__builtin_stdc_trailing_zeros", RID_BUILTIN_STDC, D_CONLY },
{ "__builtin_tgmath", RID_BUILTIN_TGMATH, D_CONLY },
{ "__builtin_offsetof", RID_OFFSETOF, 0 },
+ { "__builtin_linux_abi_kcfi_hash", RID_BUILTIN_LINUX_ABI_KCFI_HASH, D_CONLY },
+ { "__builtin_linux_abi_kcfi_name", RID_BUILTIN_LINUX_ABI_KCFI_NAME, D_CONLY },
{ "__builtin_types_compatible_p", RID_TYPES_COMPATIBLE_P, D_CONLY },
{ "__builtin_c23_va_start", RID_C23_VA_START, D_C23 | D_CXX26 },
{ "__builtin_va_arg", RID_VA_ARG, 0 },
diff --git a/gcc/c/c-parser.cc b/gcc/c/c-parser.cc
index 927aff6d5b8e..20a5f0e0a237 100644
--- a/gcc/c/c-parser.cc
+++ b/gcc/c/c-parser.cc
@@ -77,6 +77,7 @@ along with GCC; see the file COPYING3. If not see
#include "asan.h"
#include "c-family/c-ubsan.h"
#include "gcc-urlifier.h"
+#include "kcfi-typeinfo.h"
\f
/* We need to walk over decls with incomplete struct/union/enum types
after parsing the whole translation unit.
@@ -11245,6 +11246,38 @@ c_parser_has_attribute_expression (c_parser *parser)
return result;
}
+/* Parse the single type name argument of a builtin that takes a type name.
+ Returns true on success and stores the parsed type in *OUT_TYPE.
+ If successful, *OUT_CLOSE_PAREN_LOC is written with the location of
+ the closing parenthesis. */
+
+static bool
+c_parser_get_builtin_type_arg (c_parser *parser, const char *bname,
+ tree *out_type, location_t *out_close_paren_loc)
+{
+ matching_parens parens;
+ if (!parens.require_open (parser))
+ return false;
+
+ struct c_type_name *type_name = c_parser_type_name (parser);
+ if (type_name == NULL)
+ {
+ error_at (c_parser_peek_token (parser)->location,
+ "expected type name in %qs", bname);
+ return false;
+ }
+
+ *out_close_paren_loc = c_parser_peek_token (parser)->location;
+ parens.skip_until_found_close (parser);
+
+ tree type = groktypename (type_name, NULL, NULL);
+ if (type == error_mark_node)
+ return false;
+
+ *out_type = type;
+ return true;
+}
+
/* Helper function to read arguments of builtins which are interfaces
for the middle-end nodes like COMPLEX_EXPR, VEC_PERM_EXPR and
others. The name of the builtin is passed using BNAME parameter.
@@ -12283,6 +12316,45 @@ c_parser_postfix_expression (c_parser *parser)
set_c_expr_source_range (&expr, loc, close_paren_loc);
}
break;
+ case RID_BUILTIN_LINUX_ABI_KCFI_NAME:
+ {
+ c_parser_consume_token (parser);
+ location_t close_paren_loc;
+ tree type;
+ if (!c_parser_get_builtin_type_arg (parser,
+ "__builtin_linux_abi_kcfi_name",
+ &type, &close_paren_loc))
+ {
+ expr.set_error ();
+ break;
+ }
+
+ /* Call the typeinfo name function. */
+ std::string type_name = kcfi_typeinfo_get_name (type);
+ expr.value = build_string_literal (type_name.length () + 1,
+ type_name.c_str ());
+ set_c_expr_source_range (&expr, loc, close_paren_loc);
+ }
+ break;
+ case RID_BUILTIN_LINUX_ABI_KCFI_HASH:
+ {
+ c_parser_consume_token (parser);
+ location_t close_paren_loc;
+ tree type;
+ if (!c_parser_get_builtin_type_arg (parser,
+ "__builtin_linux_abi_kcfi_hash",
+ &type, &close_paren_loc))
+ {
+ expr.set_error ();
+ break;
+ }
+
+ /* Call the typeinfo hash function. */
+ uint32_t type_hash = kcfi_typeinfo_get_hash (type);
+ expr.value = build_int_cst (unsigned_type_node, type_hash);
+ set_c_expr_source_range (&expr, loc, close_paren_loc);
+ }
+ break;
case RID_BUILTIN_TGMATH:
{
vec<c_expr_t, va_gc> *cexpr_list;
diff --git a/gcc/kcfi-typeinfo.cc b/gcc/kcfi-typeinfo.cc
new file mode 100644
index 000000000000..735cbdbd3205
--- /dev/null
+++ b/gcc/kcfi-typeinfo.cc
@@ -0,0 +1,1071 @@
+/* KCFI-compatible type mangling, based on Itanium C++ ABI.
+ Copyright (C) 2026 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+/* Produces typeinfo mangling similar to Itanium C++ Mangling ABI, but
+ limited to types exposed within GCC for C language handling. The
+ hashes are used by KCFI (and future type-aware allocator support).
+ The strings are used for testing and debugging. */
+
+#define INCLUDE_STRING
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "tree.h"
+#include "target.h"
+#include "diagnostic-core.h"
+#include "stringpool.h"
+#include "stor-layout.h"
+#include "print-tree.h"
+#include "kcfi-typeinfo.h"
+
+/* Forward declaration for recursive type mangling. */
+
+static void mangle_type (tree type, std::string *out_str, uint32_t *hash_state);
+
+/* Helper to update FNV-1a hash with a single character. HASH_STATE is
+ the hash accumulator to update. C is the character to hash. */
+
+static inline void
+fnv1a_hash_char (uint32_t *hash_state, unsigned char c)
+{
+ *hash_state ^= c;
+ *hash_state *= 16777619U; /* FNV-1a 32-bit prime. */
+}
+
+/* Helper to append character to optional string and update hash using
+ FNV-1a. C is the character to append. OUT_STR is the optional string
+ to append to (NULL if not needed). HASH_STATE is the optional hash
+ accumulator to update (NULL if not needed). */
+
+static void
+append_char (char c, std::string *out_str, uint32_t *hash_state)
+{
+ if (out_str)
+ *out_str += c;
+ if (!hash_state)
+ return;
+ fnv1a_hash_char (hash_state, (unsigned char) c);
+}
+
+/* Helper to append string to optional string and update hash using
+ FNV-1a. STR is the string to append. OUT_STR is the optional string
+ to append to (NULL if not needed). HASH_STATE is the optional hash
+ accumulator to update (NULL if not needed). */
+
+static void
+append_string (const char *str, std::string *out_str, uint32_t *hash_state)
+{
+ if (out_str)
+ *out_str += str;
+ if (!hash_state)
+ return;
+ for (const char *p = str; *p; p++)
+ fnv1a_hash_char (hash_state, (unsigned char) *p);
+}
+
+/* Mangle a builtin type following Itanium C++ ABI for C types. TYPE is
+ the builtin type to mangle. OUT_STR is the optional string to append
+ mangling to (NULL if not needed). HASH_STATE is the optional hash
+ accumulator to update (NULL if not needed). */
+
+static void
+mangle_builtin_type (tree type, std::string *out_str, uint32_t *hash_state)
+{
+ gcc_assert (type != NULL_TREE);
+
+ /* Handle any target-specific fundamental types first, matching the
+ Itanium C++ ABI mangler (e.g. x86 __float128 -> "g", _Float16 -> "DF16_",
+ __bf16 -> "DF16b", and PowerPC MMA opaque types). */
+ if (const char *target_mangling = targetm.mangle_type (type))
+ {
+ append_string (target_mangling, out_str, hash_state);
+ return;
+ }
+
+ switch (TREE_CODE (type))
+ {
+ case VOID_TYPE:
+ append_char ('v', out_str, hash_state);
+ return;
+
+ case BOOLEAN_TYPE:
+ append_char ('b', out_str, hash_state);
+ return;
+
+ case INTEGER_TYPE:
+ if (type == char_type_node)
+ append_char ('c', out_str, hash_state);
+ else if (type == signed_char_type_node)
+ append_char ('a', out_str, hash_state);
+ else if (type == unsigned_char_type_node)
+ append_char ('h', out_str, hash_state);
+ else if (type == short_integer_type_node)
+ append_char ('s', out_str, hash_state);
+ else if (type == short_unsigned_type_node)
+ append_char ('t', out_str, hash_state);
+ else if (type == integer_type_node)
+ append_char ('i', out_str, hash_state);
+ else if (type == unsigned_type_node)
+ append_char ('j', out_str, hash_state);
+ else if (type == long_integer_type_node)
+ append_char ('l', out_str, hash_state);
+ else if (type == long_unsigned_type_node)
+ append_char ('m', out_str, hash_state);
+ else if (type == long_long_integer_type_node)
+ append_char ('x', out_str, hash_state);
+ else if (type == long_long_unsigned_type_node)
+ append_char ('y', out_str, hash_state);
+ else
+ {
+ /* Fallback for other integer types - use precision-based
+ encoding. */
+ append_char ('i', out_str, hash_state);
+ append_string (std::to_string (TYPE_PRECISION (type)).c_str (),
+ out_str, hash_state);
+ }
+ return;
+
+ case REAL_TYPE:
+ /* Encodings mirror the Itanium C++ ABI (see cp/mangle.cc). Target FP
+ types such as x86 __float128 ("g") are handled by the target hook
+ above. */
+ if (type == float_type_node)
+ append_char ('f', out_str, hash_state);
+ else if (type == double_type_node)
+ append_char ('d', out_str, hash_state);
+ else if (type == long_double_type_node)
+ append_char ('e', out_str, hash_state);
+ else if (type == dfloat32_type_node)
+ append_string ("Df", out_str, hash_state);
+ else if (type == dfloat64_type_node)
+ append_string ("Dd", out_str, hash_state);
+ else if (type == dfloat128_type_node)
+ append_string ("De", out_str, hash_state);
+ else if (type == float16_type_node)
+ append_string ("DF16_", out_str, hash_state);
+ else if (type == float32_type_node)
+ append_string ("DF32_", out_str, hash_state);
+ else if (type == float64_type_node)
+ append_string ("DF64_", out_str, hash_state);
+ else if (type == float128_type_node)
+ append_string ("DF128_", out_str, hash_state);
+ else if (type == float32x_type_node)
+ append_string ("DF32x", out_str, hash_state);
+ else if (type == float64x_type_node)
+ append_string ("DF64x", out_str, hash_state);
+ else if (type == float128x_type_node)
+ append_string ("DF128x", out_str, hash_state);
+ else if (type == bfloat16_type_node)
+ append_string ("DF16b", out_str, hash_state);
+ else
+ {
+ /* Unknown real type: emit a deterministic vendor-extended name
+ keyed on the machine mode, so distinct modes never collide. */
+ std::string name
+ = std::string ("rf") + GET_MODE_NAME (TYPE_MODE (type));
+ append_char ('u', out_str, hash_state);
+ append_string (std::to_string (name.length ()).c_str (),
+ out_str, hash_state);
+ append_string (name.c_str (), out_str, hash_state);
+ }
+ return;
+
+ case VECTOR_TYPE:
+ {
+ /* Handle vector types:
+ Dv<num-elements>_<element-type-encoding>
+ Example: uint8x16_t -> Dv16_h (vector of 16 unsigned char) */
+ tree vector_size = TYPE_SIZE_UNIT (type);
+ tree element_type = TREE_TYPE (type);
+ tree element_size = TYPE_SIZE_UNIT (element_type);
+
+ if (vector_size && element_size
+ && TREE_CODE (vector_size) == INTEGER_CST
+ && TREE_CODE (element_size) == INTEGER_CST)
+ {
+ append_char ('D', out_str, hash_state);
+ append_char ('v', out_str, hash_state);
+
+ unsigned HOST_WIDE_INT vec_bytes = tree_to_uhwi (vector_size);
+ unsigned HOST_WIDE_INT elem_bytes = tree_to_uhwi (element_size);
+ unsigned HOST_WIDE_INT num_elements = vec_bytes / elem_bytes;
+
+ /* Append number of elements. */
+ append_string (std::to_string (num_elements).c_str (),
+ out_str, hash_state);
+ append_char ('_', out_str, hash_state);
+
+ /* Recursively mangle the element type. */
+ mangle_type (element_type, out_str, hash_state);
+ return;
+ }
+
+ /* Scalable vectors (like RISC-V RVV types): use type name if
+ available, otherwise use a generic scalable vector encoding. */
+ if (TYPE_NAME (type))
+ {
+ const char *name = NULL;
+ if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ tree decl_name = DECL_NAME (TYPE_NAME (type));
+ if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
+ name = IDENTIFIER_POINTER (decl_name);
+ }
+ else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+ {
+ name = IDENTIFIER_POINTER (TYPE_NAME (type));
+ }
+
+ if (name)
+ {
+ /* Use vendor-extended type name encoding: u<length><name>. */
+ append_char ('u', out_str, hash_state);
+ append_string (std::to_string (strlen (name)).c_str (),
+ out_str, hash_state);
+ append_string (name, out_str, hash_state);
+ return;
+ }
+ }
+
+ /* Fallback for unnamed scalable vectors: encode as Dv0_<element>.
+ Using 0 elements indicates scalable/unknown size. */
+ append_string ("Dv0_", out_str, hash_state);
+ mangle_type (element_type, out_str, hash_state);
+ return;
+ }
+
+ case BITINT_TYPE:
+ /* Bit-precise integer types follow the Itanium C++ ABI:
+ DB<N>_ for signed _BitInt(N), DU<N>_ for unsigned _BitInt(N). */
+ append_char ('D', out_str, hash_state);
+ append_char (TYPE_UNSIGNED (type) ? 'U' : 'B', out_str, hash_state);
+ append_string (std::to_string (TYPE_PRECISION (type)).c_str (),
+ out_str, hash_state);
+ append_char ('_', out_str, hash_state);
+ return;
+
+ case NULLPTR_TYPE:
+ /* The type of C23 nullptr (std::nullptr_t): Dn. */
+ append_string ("Dn", out_str, hash_state);
+ return;
+
+ case FIXED_POINT_TYPE:
+ {
+ /* Fixed-point types (_Fract/_Accum, ISO/IEC TR 18037) have no
+ Itanium C++ ABI mangling. Emit a deterministic vendor-extended
+ name: the machine mode encodes size, signedness and the
+ fract/accum distinction; the saturating flag is separate. */
+ std::string name
+ = std::string ("fx") + GET_MODE_NAME (TYPE_MODE (type));
+ if (TYPE_SATURATING (type))
+ name += "s";
+ append_char ('u', out_str, hash_state);
+ append_string (std::to_string (name.length ()).c_str (),
+ out_str, hash_state);
+ append_string (name.c_str (), out_str, hash_state);
+ return;
+ }
+
+ case OPAQUE_TYPE:
+ {
+ /* Target opaque types (e.g. PowerPC MMA __vector_pair) are normally
+ mangled by the target hook above; if the target provides none, fall
+ back to a deterministic vendor-extended name keyed on the type name
+ or, lacking that, the machine mode. */
+ const char *name = NULL;
+ if (TYPE_NAME (type))
+ {
+ if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL
+ && DECL_NAME (TYPE_NAME (type)))
+ name = IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type)));
+ else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+ name = IDENTIFIER_POINTER (TYPE_NAME (type));
+ }
+ std::string vname = std::string ("op")
+ + (name ? name : GET_MODE_NAME (TYPE_MODE (type)));
+ append_char ('u', out_str, hash_state);
+ append_string (std::to_string (vname.length ()).c_str (),
+ out_str, hash_state);
+ append_string (vname.c_str (), out_str, hash_state);
+ return;
+ }
+
+ default:
+ break;
+ }
+
+ /* Unknown builtin type: this should never happen in a well-formed C. */
+ debug_tree (type);
+ internal_error ("mangle: Unknown builtin type - please report this as a bug");
+}
+
+/* Canonicalize typedef types to their underlying named struct/union types.
+ TYPE is the type to canonicalize. Returns the canonicalized type. */
+
+static tree
+canonicalize_typedef_type (tree type)
+{
+ /* Handle typedef types: canonicalize to named structs when possible. */
+ if (TYPE_NAME (type) && TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ tree type_decl = TYPE_NAME (type);
+
+ /* Check if this is a typedef (not the original struct declaration) */
+ if (DECL_ORIGINAL_TYPE (type_decl))
+ {
+ tree original_type = DECL_ORIGINAL_TYPE (type_decl);
+
+ /* Handle struct/union/enum types. */
+ if (TREE_CODE (original_type) == RECORD_TYPE
+ || TREE_CODE (original_type) == UNION_TYPE
+ || TREE_CODE (original_type) == ENUMERAL_TYPE)
+ {
+ /* Preserve typedef of anonymous struct/union/enum types. */
+ if (!TYPE_NAME (original_type))
+ return type;
+
+ /* Named compound type: canonicalize to it. */
+ return canonicalize_typedef_type (original_type);
+ }
+
+ /* For basic type typedefs (e.g., u8 -> unsigned char),
+ canonicalize to original type. */
+ if (TREE_CODE (original_type) == INTEGER_TYPE
+ || TREE_CODE (original_type) == REAL_TYPE
+ || TREE_CODE (original_type) == POINTER_TYPE
+ || TREE_CODE (original_type) == ARRAY_TYPE
+ || TREE_CODE (original_type) == FUNCTION_TYPE
+ || TREE_CODE (original_type) == METHOD_TYPE
+ || TREE_CODE (original_type) == BOOLEAN_TYPE
+ || TREE_CODE (original_type) == COMPLEX_TYPE
+ || TREE_CODE (original_type) == VECTOR_TYPE)
+ {
+ /* Recursively canonicalize in case the original type is
+ also a typedef. */
+ return canonicalize_typedef_type (original_type);
+ }
+ }
+ }
+
+ return type;
+}
+
+/* Return parameter type TYPE without its top-level const, volatile and
+ restrict qualifiers, which are not part of the function type ("int *const"
+ and "int *" parameters give compatible function types). Typedef names and
+ pointee qualifiers are kept. */
+
+static tree
+strip_param_qualifiers (tree type)
+{
+ int quals = TYPE_QUALS (type) & ~(TYPE_QUAL_CONST | TYPE_QUAL_VOLATILE
+ | TYPE_QUAL_RESTRICT);
+ return build_qualified_type (type, quals);
+}
+
+/* Recursively mangle a C type following Itanium C++ ABI. TYPE is the
+ type to mangle. OUT_STR is the optional string to append mangling to
+ (NULL if not needed). HASH_STATE is the optional hash accumulator to
+ update (NULL if not needed). */
+
+static void
+mangle_type (tree type, std::string *out_str, uint32_t *hash_state)
+{
+ gcc_assert (type != NULL_TREE);
+
+ /* Canonicalize typedef types to their underlying named struct types. */
+ type = canonicalize_typedef_type (type);
+
+ /* Save original qualified type for cases where we need typedef
+ information. */
+ tree qualified_type = type;
+
+ /* Centralized qualifier handling: emit qualifiers for this type,
+ then continue with unqualified version. */
+ if (TYPE_QUALS (type) != TYPE_UNQUALIFIED)
+ {
+ /* Emit qualifiers in Itanium ABI order: restrict, volatile, const. */
+ if (TYPE_QUALS (type) & TYPE_QUAL_RESTRICT)
+ append_char ('r', out_str, hash_state);
+ if (TYPE_QUALS (type) & TYPE_QUAL_VOLATILE)
+ append_char ('V', out_str, hash_state);
+ if (TYPE_QUALS (type) & TYPE_QUAL_CONST)
+ append_char ('K', out_str, hash_state);
+
+ /* Get unqualified version for further processing. */
+ type = TYPE_MAIN_VARIANT (type);
+ }
+
+ switch (TREE_CODE (type))
+ {
+ case POINTER_TYPE:
+ {
+ /* Pointer type: 'P' + pointed-to type. */
+ append_char ('P', out_str, hash_state);
+
+ /* Recursively mangle the pointed-to type. */
+ tree pointed_to_type = TREE_TYPE (type);
+ mangle_type (pointed_to_type, out_str, hash_state);
+ break;
+ }
+
+ case COMPLEX_TYPE:
+ /* Complex type (C99 _Complex): 'C' + element type. */
+ append_char ('C', out_str, hash_state);
+ mangle_type (TREE_TYPE (type), out_str, hash_state);
+ break;
+
+ case ARRAY_TYPE:
+ /* Array type: 'A' + size + '_' + element type (simplified). */
+ append_char ('A', out_str, hash_state);
+ if (TYPE_DOMAIN (type) && TYPE_MAX_VALUE (TYPE_DOMAIN (type)))
+ {
+ tree max_val = TYPE_MAX_VALUE (TYPE_DOMAIN (type));
+ /* Check if array size is compile-time constant to handle VLAs. */
+ if (TREE_CODE (max_val) == INTEGER_CST && tree_fits_shwi_p (max_val))
+ {
+ HOST_WIDE_INT size = tree_to_shwi (max_val) + 1;
+ append_string (std::to_string ((long) size).c_str (),
+ out_str, hash_state);
+ }
+ /* For VLAs or non-constant dimensions, emit empty size (A_). */
+ append_char ('_', out_str, hash_state);
+ }
+ else
+ {
+ /* No domain or no max value: emit A_. */
+ append_char ('_', out_str, hash_state);
+ }
+ mangle_type (TREE_TYPE (type), out_str, hash_state);
+ break;
+
+ case REFERENCE_TYPE:
+ /* Reference type: 'R' + referenced type.
+ Note: We must handle references to builtin types including compiler
+ builtins like __builtin_va_list used in functions like va_start. */
+ append_char ('R', out_str, hash_state);
+ mangle_type (TREE_TYPE (type), out_str, hash_state);
+ break;
+
+ case FUNCTION_TYPE:
+ {
+ /* Function type: 'F' + return type + parameter types + 'E' */
+ append_char ('F', out_str, hash_state);
+ mangle_type (TREE_TYPE (type), out_str, hash_state);
+
+ /* Add parameter types. */
+ tree param_types = TYPE_ARG_TYPES (type);
+
+ if (param_types == NULL_TREE)
+ {
+ /* func () - no parameter list (could be variadic). */
+ }
+ else
+ {
+ bool found_real_params = false;
+ for (tree param = param_types; param; param = TREE_CHAIN (param))
+ {
+ tree param_type = TREE_VALUE (param);
+ if (param_type == void_type_node)
+ {
+ /* Check if this is the first parameter (explicit void) or a
+ sentinel. */
+ if (!found_real_params)
+ {
+ /* func (void) - explicit empty parameter list.
+ Mangle void to distinguish from variadic func (). */
+ mangle_type (void_type_node, out_str, hash_state);
+ }
+ /* If we found real params before this void, it's a sentinel
+ so stop here. */
+ break;
+ }
+
+ found_real_params = true;
+
+ /* Top-level qualifiers on a parameter ("const int",
+ "int *const") are not part of the function type, so they
+ are not mangled. A pointer's pointee qualifiers ("const
+ int *") are, and are kept. */
+ tree canonical_param_type = strip_param_qualifiers (param_type);
+
+ if (TREE_CODE (param_type) != POINTER_TYPE
+ && TREE_CODE (param_type) != REFERENCE_TYPE
+ && TREE_CODE (param_type) != ARRAY_TYPE)
+ {
+ /* For non-pointer/reference value parameters, mangle the
+ main variant, which also drops typedef names. */
+ canonical_param_type = TYPE_MAIN_VARIANT (param_type);
+
+ /* Exception: preserve typedef information for anonymous
+ compound types. */
+ if (TYPE_NAME (param_type)
+ && TREE_CODE (TYPE_NAME (param_type)) == TYPE_DECL
+ && DECL_ORIGINAL_TYPE (TYPE_NAME (param_type)))
+ {
+ tree original_type
+ = DECL_ORIGINAL_TYPE (TYPE_NAME (param_type));
+ if ((TREE_CODE (original_type) == RECORD_TYPE
+ || TREE_CODE (original_type) == UNION_TYPE
+ || TREE_CODE (original_type) == ENUMERAL_TYPE)
+ && !TYPE_NAME (original_type))
+ {
+ /* Preserve typedef of an anonymous
+ struct/union/enum. */
+ canonical_param_type
+ = strip_param_qualifiers (param_type);
+ }
+ }
+ }
+
+ mangle_type (canonical_param_type, out_str, hash_state);
+ }
+ }
+
+ /* Check if this is a variadic function and add 'z' marker. */
+ if (stdarg_p (type))
+ {
+ append_char ('z', out_str, hash_state);
+ }
+
+ append_char ('E', out_str, hash_state);
+ break;
+ }
+
+ case RECORD_TYPE:
+ case UNION_TYPE:
+ case ENUMERAL_TYPE:
+ {
+ /* Struct/union/enum: use simplified representation for C types. */
+ const char *name = NULL;
+
+ /* For compound types, use the original qualified type to preserve
+ typedef info. */
+ if (TYPE_QUALS (qualified_type) != TYPE_UNQUALIFIED)
+ {
+ type = qualified_type;
+ }
+
+ if (TYPE_NAME (type))
+ {
+ if (TREE_CODE (TYPE_NAME (type)) == TYPE_DECL)
+ {
+ /* TYPE_DECL case: both named structs and typedef structs. */
+ tree decl_name = DECL_NAME (TYPE_NAME (type));
+ if (decl_name && TREE_CODE (decl_name) == IDENTIFIER_NODE)
+ {
+ name = IDENTIFIER_POINTER (decl_name);
+ }
+ }
+ else if (TREE_CODE (TYPE_NAME (type)) == IDENTIFIER_NODE)
+ {
+ /* Direct identifier case. */
+ name = IDENTIFIER_POINTER (TYPE_NAME (type));
+ }
+ }
+
+ if (name)
+ {
+ append_string (std::to_string (strlen (name)).c_str (),
+ out_str, hash_state);
+ append_string (name, out_str, hash_state);
+ break;
+ }
+
+ /* If no name found, use anonymous type format: <length>$_<counter>. */
+ static unsigned anon_counter = 0;
+ std::string anon_name = "$_" + std::to_string (anon_counter++);
+
+ append_string (std::to_string (anon_name.length ()).c_str (),
+ out_str, hash_state);
+ append_string (anon_name.c_str (), out_str, hash_state);
+ break;
+ }
+
+ default:
+ /* Handle builtin types. */
+ mangle_builtin_type (type, out_str, hash_state);
+ break;
+ }
+}
+
+/* Get the typeinfo mangled name string for any C type. TYPE is the type
+ to mangle. Returns the mangled type string following Itanium C++ ABI
+ conventions. */
+
+std::string
+kcfi_typeinfo_get_name (tree type)
+{
+ gcc_assert (type != NULL_TREE);
+ std::string result = "";
+
+ /* Add Itanium typeinfo-name mangling prefix. */
+ append_string ("_ZTS", &result, nullptr);
+ mangle_type (type, &result, nullptr);
+ return result;
+}
+
+/* Get the typeinfo hash for any C type. TYPE is the type to hash.
+ Returns the FNV-1a hash of the mangled type string. */
+
+uint32_t
+kcfi_typeinfo_get_hash (tree type)
+{
+ gcc_assert (type != NULL_TREE);
+ uint32_t hash_state = 2166136261U; /* FNV-1a 32-bit offset basis. */
+
+ /* Add Itanium typeinfo-name mangling prefix. */
+ append_string ("_ZTS", nullptr, &hash_state);
+ mangle_type (type, nullptr, &hash_state);
+ return hash_state;
+}
+
+#if CHECKING_P
+
+#include "selftest.h"
+
+namespace selftest {
+
+/* Bare Itanium mangling of TYPE, i.e. without the "_ZTS" typeinfo-name prefix
+ that the public entry points prepend. Used to unit-test the recursive
+ mangler independently of that prefix to make the type/string pairs easier
+ to read here. */
+
+static std::string
+kcfi_test_mangle (tree type)
+{
+ std::string result = "";
+ mangle_type (type, &result, nullptr);
+ return result;
+}
+
+/* Test mangling of basic builtin types. */
+
+static void
+test_typeinfo_basic_types ()
+{
+ ASSERT_STREQ ("v", kcfi_test_mangle (void_type_node).c_str ());
+ ASSERT_STREQ ("c", kcfi_test_mangle (char_type_node).c_str ());
+ ASSERT_STREQ ("a", kcfi_test_mangle (signed_char_type_node).c_str ());
+ ASSERT_STREQ ("h", kcfi_test_mangle (unsigned_char_type_node).c_str ());
+ ASSERT_STREQ ("s", kcfi_test_mangle (short_integer_type_node).c_str ());
+ ASSERT_STREQ ("t", kcfi_test_mangle (short_unsigned_type_node).c_str ());
+ ASSERT_STREQ ("i", kcfi_test_mangle (integer_type_node).c_str ());
+ ASSERT_STREQ ("j", kcfi_test_mangle (unsigned_type_node).c_str ());
+ ASSERT_STREQ ("l", kcfi_test_mangle (long_integer_type_node).c_str ());
+ ASSERT_STREQ ("m", kcfi_test_mangle (long_unsigned_type_node).c_str ());
+ ASSERT_STREQ ("x", kcfi_test_mangle (long_long_integer_type_node).c_str ());
+ ASSERT_STREQ ("y", kcfi_test_mangle (long_long_unsigned_type_node).c_str ());
+ ASSERT_STREQ ("f", kcfi_test_mangle (float_type_node).c_str ());
+ ASSERT_STREQ ("d", kcfi_test_mangle (double_type_node).c_str ());
+ ASSERT_STREQ ("b", kcfi_test_mangle (boolean_type_node).c_str ());
+
+ /* Several targets mangle long double specially depending on its format
+ (e.g. "g" for 128-bit IEEE long double), so expect the target's
+ mangling when it provides one, and the Itanium default otherwise. */
+ const char *ld_mangling = targetm.mangle_type (long_double_type_node);
+ ASSERT_STREQ (ld_mangling ? ld_mangling : "e",
+ kcfi_test_mangle (long_double_type_node).c_str ());
+}
+
+/* Test mangling of pointer types. */
+
+static void
+test_typeinfo_pointer_types ()
+{
+ tree char_ptr = build_pointer_type (char_type_node);
+ ASSERT_STREQ ("Pc", kcfi_test_mangle (char_ptr).c_str ());
+
+ tree int_ptr = build_pointer_type (integer_type_node);
+ ASSERT_STREQ ("Pi", kcfi_test_mangle (int_ptr).c_str ());
+
+ tree void_ptr = build_pointer_type (void_type_node);
+ ASSERT_STREQ ("Pv", kcfi_test_mangle (void_ptr).c_str ());
+
+ /* Pointer to pointer. */
+ tree int_ptr_ptr = build_pointer_type (int_ptr);
+ ASSERT_STREQ ("PPi", kcfi_test_mangle (int_ptr_ptr).c_str ());
+
+ tree void_ptr_ptr = build_pointer_type (void_ptr);
+ ASSERT_STREQ ("PPv", kcfi_test_mangle (void_ptr_ptr).c_str ());
+}
+
+/* Test mangling of qualified types. */
+
+static void
+test_typeinfo_qualified_types ()
+{
+ tree const_int = build_qualified_type (integer_type_node, TYPE_QUAL_CONST);
+ ASSERT_STREQ ("Ki", kcfi_test_mangle (const_int).c_str ());
+
+ tree volatile_int = build_qualified_type (integer_type_node,
+ TYPE_QUAL_VOLATILE);
+ ASSERT_STREQ ("Vi", kcfi_test_mangle (volatile_int).c_str ());
+
+ /* const char *. */
+ tree const_char = build_qualified_type (char_type_node, TYPE_QUAL_CONST);
+ tree const_char_ptr = build_pointer_type (const_char);
+ ASSERT_STREQ ("PKc", kcfi_test_mangle (const_char_ptr).c_str ());
+
+ /* const void *. */
+ tree const_void = build_qualified_type (void_type_node, TYPE_QUAL_CONST);
+ tree const_void_ptr = build_pointer_type (const_void);
+ ASSERT_STREQ ("PKv", kcfi_test_mangle (const_void_ptr).c_str ());
+
+ /* const int *. */
+ tree const_int_ptr = build_pointer_type (const_int);
+ ASSERT_STREQ ("PKi", kcfi_test_mangle (const_int_ptr).c_str ());
+}
+
+/* Test mangling of function types. */
+
+static void
+test_typeinfo_function_types ()
+{
+ /* void func(void) -> "FvvE". */
+ tree void_void = build_function_type_list (void_type_node,
+ void_type_node, NULL_TREE);
+ ASSERT_STREQ ("FvvE", kcfi_test_mangle (void_void).c_str ());
+
+ /* void func(int) -> "FviE". */
+ tree void_int = build_function_type_list (void_type_node,
+ integer_type_node, NULL_TREE);
+ ASSERT_STREQ ("FviE", kcfi_test_mangle (void_int).c_str ());
+
+ /* void func(char) -> "FvcE". */
+ tree void_char = build_function_type_list (void_type_node,
+ char_type_node, NULL_TREE);
+ ASSERT_STREQ ("FvcE", kcfi_test_mangle (void_char).c_str ());
+
+ /* void func(short) -> "FvsE". */
+ tree void_short = build_function_type_list (void_type_node,
+ short_integer_type_node,
+ NULL_TREE);
+ ASSERT_STREQ ("FvsE", kcfi_test_mangle (void_short).c_str ());
+
+ /* void func(long) -> "FvlE". */
+ tree void_long = build_function_type_list (void_type_node,
+ long_integer_type_node, NULL_TREE);
+ ASSERT_STREQ ("FvlE", kcfi_test_mangle (void_long).c_str ());
+
+ /* int func(void) -> "FivE". */
+ tree int_void = build_function_type_list (integer_type_node,
+ void_type_node, NULL_TREE);
+ ASSERT_STREQ ("FivE", kcfi_test_mangle (int_void).c_str ());
+
+ /* char func(void) -> "FcvE". */
+ tree char_void = build_function_type_list (char_type_node,
+ void_type_node, NULL_TREE);
+ ASSERT_STREQ ("FcvE", kcfi_test_mangle (char_void).c_str ());
+
+ /* void* func(void) -> "FPvvE". */
+ tree void_ptr = build_pointer_type (void_type_node);
+ tree voidptr_void = build_function_type_list (void_ptr,
+ void_type_node, NULL_TREE);
+ ASSERT_STREQ ("FPvvE", kcfi_test_mangle (voidptr_void).c_str ());
+}
+
+/* Test mangling of function types with pointer parameters. */
+
+static void
+test_typeinfo_function_pointer_params ()
+{
+ tree void_ptr = build_pointer_type (void_type_node);
+ tree char_ptr = build_pointer_type (char_type_node);
+ tree int_ptr = build_pointer_type (integer_type_node);
+ tree long_ptr = build_pointer_type (long_integer_type_node);
+
+ /* void func(void*) -> "FvPvE". */
+ tree void_voidptr = build_function_type_list (void_type_node,
+ void_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPvE", kcfi_test_mangle (void_voidptr).c_str ());
+
+ /* void func(char*) -> "FvPcE". */
+ tree void_charptr = build_function_type_list (void_type_node,
+ char_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPcE", kcfi_test_mangle (void_charptr).c_str ());
+
+ /* void func(int*) -> "FvPiE". */
+ tree void_intptr = build_function_type_list (void_type_node,
+ int_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPiE", kcfi_test_mangle (void_intptr).c_str ());
+
+ /* void func(long*) -> "FvPlE". */
+ tree void_longptr = build_function_type_list (void_type_node,
+ long_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPlE", kcfi_test_mangle (void_longptr).c_str ());
+
+ /* void func(const char*) -> "FvPKcE". */
+ tree const_char = build_qualified_type (char_type_node, TYPE_QUAL_CONST);
+ tree const_char_ptr = build_pointer_type (const_char);
+ tree void_constcharptr = build_function_type_list (void_type_node,
+ const_char_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPKcE", kcfi_test_mangle (void_constcharptr).c_str ());
+
+ /* void func(const void*) -> "FvPKvE". */
+ tree const_void = build_qualified_type (void_type_node, TYPE_QUAL_CONST);
+ tree const_void_ptr = build_pointer_type (const_void);
+ tree void_constvoidptr = build_function_type_list (void_type_node,
+ const_void_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPKvE", kcfi_test_mangle (void_constvoidptr).c_str ());
+
+ /* void func(int**) -> "FvPPiE". */
+ tree int_ptr_ptr = build_pointer_type (int_ptr);
+ tree void_intptrptr = build_function_type_list (void_type_node,
+ int_ptr_ptr, NULL_TREE);
+ ASSERT_STREQ ("FvPPiE", kcfi_test_mangle (void_intptrptr).c_str ());
+
+ /* Top-level qualifiers on a pointer parameter are not part of the
+ function type, so they are not mangled; the pointee's are. */
+ /* void func(int *const) -> "FvPiE". */
+ tree int_ptr_const = build_qualified_type (int_ptr, TYPE_QUAL_CONST);
+ tree void_intptrconst = build_function_type_list (void_type_node,
+ int_ptr_const, NULL_TREE);
+ ASSERT_STREQ ("FvPiE", kcfi_test_mangle (void_intptrconst).c_str ());
+
+ /* void func(int *restrict volatile) -> "FvPiE". */
+ tree int_ptr_rv = build_qualified_type (int_ptr, (TYPE_QUAL_RESTRICT
+ | TYPE_QUAL_VOLATILE));
+ tree void_intptrrv = build_function_type_list (void_type_node,
+ int_ptr_rv, NULL_TREE);
+ ASSERT_STREQ ("FvPiE", kcfi_test_mangle (void_intptrrv).c_str ());
+
+ /* void func(const char *const) -> "FvPKcE". */
+ tree const_char_ptr_const = build_qualified_type (const_char_ptr,
+ TYPE_QUAL_CONST);
+ tree void_ccpc = build_function_type_list (void_type_node,
+ const_char_ptr_const, NULL_TREE);
+ ASSERT_STREQ ("FvPKcE", kcfi_test_mangle (void_ccpc).c_str ());
+
+ /* Below the top level, restrict is mangled, in the Itanium order
+ restrict, volatile, const. */
+ /* void func(int *restrict *) -> "FvPrPiE". */
+ tree int_ptr_r = build_qualified_type (int_ptr, TYPE_QUAL_RESTRICT);
+ tree void_intptrrptr
+ = build_function_type_list (void_type_node, build_pointer_type (int_ptr_r),
+ NULL_TREE);
+ ASSERT_STREQ ("FvPrPiE", kcfi_test_mangle (void_intptrrptr).c_str ());
+
+ /* void func(int *const volatile restrict *) -> "FvPrVKPiE". */
+ tree int_ptr_cvr = build_qualified_type (int_ptr, (TYPE_QUAL_CONST
+ | TYPE_QUAL_VOLATILE
+ | TYPE_QUAL_RESTRICT));
+ tree void_intptrcvrptr
+ = build_function_type_list (void_type_node,
+ build_pointer_type (int_ptr_cvr), NULL_TREE);
+ ASSERT_STREQ ("FvPrVKPiE", kcfi_test_mangle (void_intptrcvrptr).c_str ());
+}
+
+/* Test mangling of function types with multiple parameters. */
+
+static void
+test_typeinfo_function_multi_params ()
+{
+ /* void func(int, char) -> "FvicE". */
+ tree void_int_char = build_function_type_list (void_type_node,
+ integer_type_node,
+ char_type_node, NULL_TREE);
+ ASSERT_STREQ ("FvicE", kcfi_test_mangle (void_int_char).c_str ());
+
+ /* void func(char, int) -> "FvciE". */
+ tree void_char_int = build_function_type_list (void_type_node,
+ char_type_node,
+ integer_type_node, NULL_TREE);
+ ASSERT_STREQ ("FvciE", kcfi_test_mangle (void_char_int).c_str ());
+
+ /* void func(int, int) -> "FviiE". */
+ tree void_int_int = build_function_type_list (void_type_node,
+ integer_type_node,
+ integer_type_node, NULL_TREE);
+ ASSERT_STREQ ("FviiE", kcfi_test_mangle (void_int_int).c_str ());
+
+ /* void func(int*, int) -> "FvPiiE". */
+ tree int_ptr = build_pointer_type (integer_type_node);
+ tree void_intptr_int = build_function_type_list (void_type_node,
+ int_ptr,
+ integer_type_node,
+ NULL_TREE);
+ ASSERT_STREQ ("FvPiiE", kcfi_test_mangle (void_intptr_int).c_str ());
+
+ /* void func(int, int*) -> "FviPiE". */
+ tree void_int_intptr = build_function_type_list (void_type_node,
+ integer_type_node,
+ int_ptr, NULL_TREE);
+ ASSERT_STREQ ("FviPiE", kcfi_test_mangle (void_int_intptr).c_str ());
+}
+
+/* Test mangling of function pointer types. */
+
+static void
+test_typeinfo_function_pointer_types ()
+{
+ /* void (*)(void) -> "PFvvE". */
+ tree void_void = build_function_type_list (void_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr_void_void = build_pointer_type (void_void);
+ ASSERT_STREQ ("PFvvE", kcfi_test_mangle (fptr_void_void).c_str ());
+
+ /* void (*)(int) -> "PFviE". */
+ tree void_int = build_function_type_list (void_type_node,
+ integer_type_node, NULL_TREE);
+ tree fptr_void_int = build_pointer_type (void_int);
+ ASSERT_STREQ ("PFviE", kcfi_test_mangle (fptr_void_int).c_str ());
+
+ /* int (*)(void) -> "PFivE". */
+ tree int_void = build_function_type_list (integer_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr_int_void = build_pointer_type (int_void);
+ ASSERT_STREQ ("PFivE", kcfi_test_mangle (fptr_int_void).c_str ());
+}
+
+/* Test mangling of functions with function pointer parameters. */
+
+static void
+test_typeinfo_function_fptr_params ()
+{
+ /* void func(void (*)(void)) -> "FvPFvvEE". */
+ tree void_void = build_function_type_list (void_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr_void_void = build_pointer_type (void_void);
+ tree func_fptr_void = build_function_type_list (void_type_node,
+ fptr_void_void, NULL_TREE);
+ ASSERT_STREQ ("FvPFvvEE", kcfi_test_mangle (func_fptr_void).c_str ());
+
+ /* void func(void (*)(int)) -> "FvPFviEE". */
+ tree void_int = build_function_type_list (void_type_node,
+ integer_type_node, NULL_TREE);
+ tree fptr_void_int = build_pointer_type (void_int);
+ tree func_fptr_int = build_function_type_list (void_type_node,
+ fptr_void_int, NULL_TREE);
+ ASSERT_STREQ ("FvPFviEE", kcfi_test_mangle (func_fptr_int).c_str ());
+
+ /* void func(int (*)(void)) -> "FvPFivEE". */
+ tree int_void = build_function_type_list (integer_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr_int_void = build_pointer_type (int_void);
+ tree func_fptr_ret_int = build_function_type_list (void_type_node,
+ fptr_int_void, NULL_TREE);
+ ASSERT_STREQ ("FvPFivEE", kcfi_test_mangle (func_fptr_ret_int).c_str ());
+}
+
+/* Test mangling of functions returning function pointers. */
+
+static void
+test_typeinfo_function_returning_fptr ()
+{
+ /* int (*func(void))(void) -> "FPFivEvE". */
+ tree int_void = build_function_type_list (integer_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr_int_void = build_pointer_type (int_void);
+ tree func_ret_fptr = build_function_type_list (fptr_int_void,
+ void_type_node, NULL_TREE);
+ ASSERT_STREQ ("FPFivEvE", kcfi_test_mangle (func_ret_fptr).c_str ());
+}
+
+/* Test mangling of array types. */
+
+static void
+test_typeinfo_array_types ()
+{
+ /* int[10] -> "A10_i". */
+ tree domain_10 = build_index_type (build_int_cst (sizetype, 9));
+ tree int_array_10 = build_array_type (integer_type_node, domain_10);
+ ASSERT_STREQ ("A10_i", kcfi_test_mangle (int_array_10).c_str ());
+
+ /* char[20] -> "A20_c". */
+ tree domain_20 = build_index_type (build_int_cst (sizetype, 19));
+ tree char_array_20 = build_array_type (char_type_node, domain_20);
+ ASSERT_STREQ ("A20_c", kcfi_test_mangle (char_array_20).c_str ());
+
+ /* int[] (incomplete array) -> "A_i". */
+ tree int_array_incomplete = build_array_type (integer_type_node, NULL_TREE);
+ ASSERT_STREQ ("A_i", kcfi_test_mangle (int_array_incomplete).c_str ());
+}
+
+/* Test that hash values are correct (FNV-1a of the mangled name). */
+
+static void
+test_typeinfo_hash_values ()
+{
+ /* Verify hash for a basic type: "_ZTSi" -> 0x5fbda8e8. */
+ ASSERT_EQ (0x5fbda8e8U, kcfi_typeinfo_get_hash (integer_type_node));
+
+ /* Verify hash for a function type: "_ZTSFviE" -> 0x70e35def. */
+ tree void_int = build_function_type_list (void_type_node,
+ integer_type_node, NULL_TREE);
+ ASSERT_EQ (0x70e35defU, kcfi_typeinfo_get_hash (void_int));
+
+ /* Verify hash for a pointer type: "_ZTSPKc" -> 0x05f5ee31. */
+ tree const_char = build_qualified_type (char_type_node, TYPE_QUAL_CONST);
+ tree const_char_ptr = build_pointer_type (const_char);
+ ASSERT_EQ (0x05f5ee31U, kcfi_typeinfo_get_hash (const_char_ptr));
+
+ /* Verify hash for a complex function type: "_ZTSFvPFvvEE" -> 0xc88f6251. */
+ tree void_void = build_function_type_list (void_type_node,
+ void_type_node, NULL_TREE);
+ tree fptr = build_pointer_type (void_void);
+ tree func_fptr = build_function_type_list (void_type_node, fptr, NULL_TREE);
+ ASSERT_EQ (0xc88f6251U, kcfi_typeinfo_get_hash (func_fptr));
+}
+
+/* Test that the public entry points emit the Itanium typeinfo-name "_ZTS"
+ prefix, so the KCFI type-id matches expected FNV-1a of mangleTypeName
+ (i.e. "_ZTS" + the bare type mangling). */
+
+static void
+test_typeinfo_zts_prefix ()
+{
+ tree int_int_int = build_function_type_list (integer_type_node,
+ integer_type_node,
+ integer_type_node, NULL_TREE);
+ tree int_int = build_function_type_list (integer_type_node,
+ integer_type_node, NULL_TREE);
+
+ /* The name carries the "_ZTS" prefix ahead of the bare "F...E" mangling. */
+ ASSERT_STREQ ("_ZTSFiiiE", kcfi_typeinfo_get_name (int_int_int).c_str ());
+ ASSERT_STREQ ("_ZTSFiiE", kcfi_typeinfo_get_name (int_int).c_str ());
+
+ /* The hashes match for the same prototypes: "_ZTSFiiiE" -> 0x70785afd,
+ "_ZTSFiiE" -> 0xb8360626. */
+ ASSERT_EQ (0x70785afdU, kcfi_typeinfo_get_hash (int_int_int));
+ ASSERT_EQ (0xb8360626U, kcfi_typeinfo_get_hash (int_int));
+}
+
+/* Run all typeinfo selftests. */
+
+void
+kcfi_typeinfo_cc_tests ()
+{
+ test_typeinfo_basic_types ();
+ test_typeinfo_pointer_types ();
+ test_typeinfo_qualified_types ();
+ test_typeinfo_function_types ();
+ test_typeinfo_function_pointer_params ();
+ test_typeinfo_function_multi_params ();
+ test_typeinfo_function_pointer_types ();
+ test_typeinfo_function_fptr_params ();
+ test_typeinfo_function_returning_fptr ();
+ test_typeinfo_array_types ();
+ test_typeinfo_hash_values ();
+ test_typeinfo_zts_prefix ();
+}
+
+} /* namespace selftest. */
+
+#endif /* CHECKING_P */
diff --git a/gcc/selftest-run-tests.cc b/gcc/selftest-run-tests.cc
index 07eb984e918e..c2b85493327b 100644
--- a/gcc/selftest-run-tests.cc
+++ b/gcc/selftest-run-tests.cc
@@ -92,6 +92,7 @@ selftest::run_tests ()
input_cc_tests ();
vec_perm_indices_cc_tests ();
tree_cc_tests ();
+ kcfi_typeinfo_cc_tests ();
convert_cc_tests ();
gimple_cc_tests ();
rtl_tests_cc_tests ();
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v17 2/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
2026-10-05 15:40 ` [PATCH v17 1/7] kcfi: Introduce KCFI typeinfo mangling API Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-09 13:35 ` Jeff Law
2026-10-05 15:40 ` [PATCH v17 3/7] kcfi: Add regression test suite Kees Cook
` (4 subsequent siblings)
6 siblings, 1 reply; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Implements the Linux Kernel Control Flow Integrity ABI, which provides a
function prototype based forward edge control flow integrity protection
by instrumenting every indirect call to check for a hash value before
the target function address. If the hash at the call site and the hash
at the target do not match, execution will trap.
See the start of kcfi.cc for design details.
gcc/ChangeLog:
* kcfi.h: New file with KCFI public interface declarations.
* kcfi.cc: New file implementing Kernel Control Flow Integrity
infrastructure.
* Makefile.in (OBJS): Add kcfi.o.
* flag-types.h (enum sanitize_code): Add SANITIZE_KCFI.
* gimple.h (enum gf_mask): Add GF_CALL_INLINED_FROM_KCFI_NOSANTIZE.
(gimple_call_set_inlined_from_kcfi_nosantize): New function.
(gimple_call_inlined_from_kcfi_nosantize_p): New function.
* df-scan.cc (df_uses_record): Add KCFI case to handle KCFI RTL
patterns and process wrapped RTL.
* doc/extend.texi: Update nocf_check for kcfi.
* doc/invoke.texi (fsanitize=kcfi): Add documentation for KCFI
sanitizer option.
* doc/tm.texi.in: Add Kernel Control Flow Integrity section with
TARGET_KCFI_SUPPORTED, TARGET_KCFI_MASK_TYPE_ID,
TARGET_KCFI_EMIT_TYPE_ID hooks.
* doc/tm.texi: Regenerate.
* final.cc (call_from_call_insn): Add KCFI case to handle
KCFI-wrapped calls. Add kcfi_next_labelno().
* opts.cc (sanitizer_opts): Add kcfi entry.
(parse_sanitizer_options): Exclude SANITIZE_KCFI from
-fsanitize-recover=all.
* rtl.def (KCFI): Add new RTL code for KCFI instrumentation.
* rtlanal.cc (reg_referenced_p): Add KCFI case.
* target.def: Add KCFI target hooks.
* toplev.cc (process_options): Add KCFI option processing.
* tree-inline.cc (copy_bb): Handle KCFI no_sanitize attribute
propagation during inlining.
* tree-ssa-tail-merge.cc: Include kcfi.h.
(gimple_equal_p): Don't treat calls that get different KCFI
checks as equal.
* varasm.cc (assemble_start_function): Emit KCFI preambles.
(assemble_external_real): Emit KCFI typeid symbols.
(default_elf_asm_named_section): Handle .kcfi_traps using
SECTION_LINK_ORDER flag.
gcc/c-family/ChangeLog:
* c-attribs.cc: Include asan.h.
(handle_nocf_check_attribute): Enable nocf_check under kcfi.
(handle_patchable_function_entry_attribute): Add error for using
patchable_function_entry attribute with -fsanitize=kcfi.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/kcfi.h | 71 ++++
gcc/kcfi.cc | 650 +++++++++++++++++++++++++++++++++++++
gcc/doc/extend.texi | 42 +++
gcc/doc/invoke.texi | 39 +++
gcc/doc/tm.texi | 32 ++
gcc/Makefile.in | 2 +
gcc/flag-types.h | 2 +
gcc/gimple.h | 22 ++
gcc/c-family/c-attribs.cc | 17 +-
gcc/df-scan.cc | 7 +
gcc/doc/tm.texi.in | 12 +
gcc/final.cc | 17 +
gcc/opts.cc | 4 +-
gcc/rtl.def | 6 +
gcc/rtlanal.cc | 5 +
gcc/target.def | 39 +++
gcc/toplev.cc | 12 +
gcc/tree-inline.cc | 10 +
gcc/tree-ssa-tail-merge.cc | 6 +
gcc/varasm.cc | 44 ++-
20 files changed, 1027 insertions(+), 12 deletions(-)
create mode 100644 gcc/kcfi.h
create mode 100644 gcc/kcfi.cc
diff --git a/gcc/kcfi.h b/gcc/kcfi.h
new file mode 100644
index 000000000000..ab6abb4adfa8
--- /dev/null
+++ b/gcc/kcfi.h
@@ -0,0 +1,71 @@
+/* Kernel Control Flow Integrity (KCFI) support for GCC.
+ Copyright (C) 2026 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_KCFI_H
+#define GCC_KCFI_H
+
+#include "coretypes.h"
+#include "rtl.h"
+
+/* Common helper for RTL patterns to emit .kcfi_traps section entry.
+ Call after emitting trap label and instruction with the trap symbol
+ reference. */
+extern void kcfi_emit_traps_section (FILE *file, rtx trap_label_sym,
+ int labelno);
+
+/* Extract KCFI type ID from current GIMPLE statement. */
+extern rtx internal_kcfi_get_type_id_for_expanding_gimple_call (void);
+
+/* Convenience wrapper to check for SANITIZE_KCFI. */
+inline rtx
+kcfi_get_type_id_for_expanding_gimple_call (void)
+{
+ if (!(flag_sanitize & SANITIZE_KCFI))
+ return NULL_RTX;
+ return internal_kcfi_get_type_id_for_expanding_gimple_call ();
+}
+
+/* Compare the KCFI checks two call statements get. */
+extern bool internal_kcfi_same_call_check_p (const gcall *, const gcall *);
+
+/* Return true if calls CALL1 and CALL2 get the same KCFI check (or none). */
+inline bool
+kcfi_same_call_check_p (const gcall *call1, const gcall *call2)
+{
+ if (!(flag_sanitize & SANITIZE_KCFI))
+ return true;
+ return internal_kcfi_same_call_check_p (call1, call2);
+}
+
+/* Emit KCFI type ID symbol for external address-taken functions. */
+extern void kcfi_emit_typeid_symbol (FILE *asm_file, tree fndecl);
+
+/* Emit KCFI preamble for potential indirect call targets. */
+extern void kcfi_emit_preamble (FILE *asm_file, tree fndecl,
+ const char *actual_fname);
+
+/* Get next KCFI label number for trap/call/entry label numbering. This
+ output-phase counter lives with the other unique-label counters in
+ final.cc. */
+extern int kcfi_next_labelno (void);
+
+/* Get the KCFI typeid offset for calculating callsite typeid offset. */
+extern HOST_WIDE_INT kcfi_get_typeid_offset (void);
+
+#endif /* GCC_KCFI_H */
diff --git a/gcc/kcfi.cc b/gcc/kcfi.cc
new file mode 100644
index 000000000000..0b2d49f0d78e
--- /dev/null
+++ b/gcc/kcfi.cc
@@ -0,0 +1,650 @@
+/* Kernel Control Flow Integrity (KCFI) support for GCC.
+ Copyright (C) 2026 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+/* KCFI ABI Design:
+
+The Linux Kernel Control Flow Integrity ABI provides a function prototype
+based forward edge control flow integrity protection by instrumenting
+every indirect call to check for a hash value before the target function
+address. If the hash at the call site and the hash at the target do not
+match, execution will trap.
+
+The general CFI ideas are discussed here, but focuses more on a CFG
+analysis to construct valid call destinations, which tends to require LTO:
+https://users.soe.ucsc.edu/~abadi/Papers/cfi-tissec-revised.pdf
+
+Later refinement for using jump tables (constructed via CFG analysis
+during LTO) was proposed here:
+https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-tice.pdf
+
+Linux used the above implementation from 2018 to 2022:
+https://android-developers.googleblog.com/2018/10/control-flow-integrity-in-android-kernel.html
+but the corner cases for target addresses not being the actual functions
+(i.e. pointing into the jump table) was a continual source of problems,
+and generating the jump tables required full LTO, which had its own set
+of problems.
+
+Looking at function prototypes as the source of call validity was
+presented here, though still relied on LTO:
+https://www.blackhat.com/docs/asia-17/materials/asia-17-Moreira-Drop-The-Rop-Fine-Grained-Control-Flow-Integrity-For-The-Linux-Kernel-wp.pdf
+
+The KCFI approach built on the function-prototype idea, but avoided
+needing LTO, and could be further updated to deal with CPU errata
+(retpolines, etc):
+https://lpc.events/event/16/contributions/1315/
+
+KCFI has a number of specific constraints. Some are tied to the
+backend architecture, which are covered in arch-specific code.
+The constraints are:
+
+- The KCFI scheme generates a unique 32-bit hash ("typeid") for each
+ unique function prototype, allowing for indirect call sites to verify
+ that they are calling into a matching _type_ of function pointer.
+ This changes the semantics of some optimization logic because now
+ indirect calls to different types cannot be merged. For example:
+
+ if (p->func_type_1)
+ return p->func_type_1 ();
+ if (p->func_type_2)
+ return p->func_type_2 ();
+
+ In final asm, the optimizer may collapse the second indirect call
+ into a jump to the first indirect call once it has loaded the function
+ pointer. KCFI must block cross-type merging otherwise there will be a
+ single KCFI check happening for only 1 type but being used by 2 target
+ types. The distinguishing characteristic for call merging becomes the
+ type, not the address/register usage.
+
+- The check-call instruction sequence must be treated as a single unit: it
+ cannot be rearranged or split or optimized. The pattern is that
+ indirect calls, "call *%target", get converted into:
+
+ mov $target_expression, %target ; only present if the expression was
+ ; not already in %target register
+ load -$offset(%target), %tmp ; load typeid hash from target preamble
+ cmp $typeid, %tmp ; compare expected typeid with loaded
+ je .Lkcfi_call$N ; success: jump to the indirect call
+ .Lkcfi_trap$N: ; label of trap insn
+ trap ; trap on failure, but arranged so
+ ; "permissive mode" falls through
+ .Lkcfi_call$N: ; label of call insn
+ call *%target ; actual indirect call
+
+ This pattern of call immediately after trap provides for the
+ "permissive" checking mode automatically: the trap gets handled,
+ a warning emitted, and then execution continues after the trap to
+ the call.
+
+- KCFI check-call instrumentation must survive tail call optimization.
+ If an indirect call is turned into an indirect jump, KCFI checking
+ must still happen (but it will use a jmp rather than a call).
+
+- Functions that may be called indirectly have a preamble added,
+ __cfi_$original_func_name, which contains the $typeid value:
+
+ __cfi_target_func:
+ .word $typeid
+ target_func:
+ [regular function entry...]
+
+- The preamble needs to interact with patchable function entry so that
+ the typeid appears further away from the actual start of the function
+ (leaving the prefix NOPs of the patchable function entry unchanged).
+ This means only _globally defined_ patchable function entry is supported
+ with KCFI (indrect call sites must know in advance what the offset is,
+ which may not be possible with extern functions that use a function
+ attribute to change their patchable function entry characteristics).
+ For example, a "4,4" patchable function entry would end up like:
+
+ __cfi_target_func:
+ .data $typeid
+ nop nop nop nop
+ target_func:
+ [regular function entry...]
+
+ Architectures may need to add alignment nops prior to the typeid to keep
+ __cfi_target_func aligned for function call conventions.
+
+- An external function that is address-taken but does not have a definition has
+ a weak __kcfi_typeid_$func symbol added at the declaration site. This weak
+ symbol has the typeid value available so that the typeid can be referenced
+ from assembly linkages, etc, where the typeid values cannot be calculated
+ (i.e where C type information is missing):
+
+ .weak __kcfi_typeid_$func
+ .set __kcfi_typeid_$func, $typeid
+
+- On architectures that do not have a good way to encode additional
+ details in their trap insn (e.g. x86_64 and riscv64), the trap location
+ is identified as a KCFI trap via a relative address offset entry
+ emitted into the .kcfi_traps section for each indirect call site's
+ trap instruction. The previous check-call example's insn sequence would
+ then have section changes inserted between the trap and call:
+
+ ...
+ .Lkcfi_trap$N:
+ trap
+ .section .kcfi_traps,"ao",@progbits,.text
+ .Lkcfi_entry$N:
+ .long .Lkcfi_trap$N - .Lkcfi_entry$N
+ .section .text
+ .Lkcfi_call$N:
+ call *%target
+
+ The .kcfi_traps section is SHF_LINK_ORDER, tied to the section the calling
+ function's code lives in (plain .text, or .text.$func under
+ -ffunction-sections, or a "section" attribute's section), so that under
+ --gc-sections the trap entries are retained or discarded together with the
+ function instead of the generic .text.
+
+ It is up to such architectures to decode instructions prior to the
+ trap to locate the typeid that the callsite was expecting.
+
+ For architectures that can encode immediates in their trap function
+ (e.g. aarch64 and arm32), this isn't needed: they just use immediate
+ codes that indicate a KCFI trap.
+
+- The no_sanitize("kcfi") function attribute means that the marked
+ function must not produce KCFI checking for indirect calls, and this
+ attribute must survive inlining. This is used rarely by Linux, but
+ is required to make BPF JIT trampolines work on older Linux kernel
+ versions.
+
+- The "nocf_check" function attribute can be used to supress the
+ KCFI preamble for a function, making that function unavailable
+ for indirect calls.
+
+As a result of these constraints, there are some behavioral aspects
+that need to be preserved across the middle-end and back-end.
+
+For indirect call sites:
+
+- Make sure KCFI expansion is skipped for inline functions that
+ are marked with no_sanitize("kcfi") by marking these calls
+ during GIMPLE inlining with a new flag which is checked during
+ expansion.
+
+- A function type's typeid is computed on demand from the type and
+ memoized in a hash_map keyed by the function type (kcfi_get_type_id),
+ so repeated uses of the same type share one computation.
+
+- Keep typeid information available through to the RTL expansion
+ phase via a new KCFI insn RTL pattern that wraps the CALL
+ and the typeid as expressions. Since per-arch RTL call logic
+ can be very complex, we don't want to replace the "define_expand"
+ patterns, but rather hook the tail end of expansion. This is
+ where kcfi_get_type_id_for_expanding_gimple_call gets used, which
+ uses currently_expanding_gimple_stmt internally, as done in
+ other cases where GIMPLE needs to be examined during expansion.
+
+- Keep indirect calls from being merged (see earlier example)
+ naturally by having typeid be the second argument of the KCFI insn
+ RTL pattern, so jump2 pass's use of rtx_equal_p see differing
+ typeids in the RTL.
+
+- Update register liveness analysis to look inside the new KCFI
+ RTL to find the CALL RTL and examine the registers in use there
+ so the allocator can track register usage correctly.
+
+- KCFI insn emission interacts with patchable function entry to
+ load the typeid from the target preamble, offset by prefix NOPs.
+
+For indirect call targets:
+
+- kcfi_emit_preamble interacts with patchable function entry to add
+ any needed alignment padding prior to emitting the typeid.
+
+- assemble_external_real calls kcfi_emit_typeid_symbol to add the
+ __kcfi_typeid_$func symbols.
+
+*/
+
+#define INCLUDE_STRING
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "target.h"
+#include "function.h"
+#include "tree.h"
+#include "tree-pass.h"
+#include "dumpfile.h"
+#include "basic-block.h"
+#include "gimple.h"
+#include "gimple-iterator.h"
+#include "cgraph.h"
+#include "kcfi.h"
+#include "stringpool.h"
+#include "attribs.h"
+#include "rtl.h"
+#include "cfg.h"
+#include "cfgrtl.h"
+#include "asan.h"
+#include "diagnostic-core.h"
+#include "memmodel.h"
+#include "print-tree.h"
+#include "emit-rtl.h"
+#include "output.h"
+#include "builtins.h"
+#include "varasm.h"
+#include "opts.h"
+#include "target.h"
+#include "flags.h"
+#include "kcfi-typeinfo.h"
+#include "insn-config.h"
+#include "recog.h"
+
+/* Callsite typeid loading offset. */
+static HOST_WIDE_INT kcfi_typeid_offset = 0;
+/* Count of needed __cfi_... preamble alignment padding NOPs. */
+static HOST_WIDE_INT kcfi_alignment_padding_nops = 0;
+/* NOP insn template. */
+static const char *kcfi_nop = NULL;
+/* True once the globals above have been computed for this codegen process. */
+static bool kcfi_prepared = false;
+
+static void kcfi_prepare_alignment_nops (void);
+
+/* Get the KCFI typeid offset. Returns the offset in bytes from the
+ function entry point where the KCFI type ID is stored. */
+
+HOST_WIDE_INT
+kcfi_get_typeid_offset (void)
+{
+ kcfi_prepare_alignment_nops ();
+ return kcfi_typeid_offset;
+}
+
+/* Common helper for RTL patterns to emit .kcfi_traps section entry.
+ FILE is the output assembly file stream. TRAP_LABEL_SYM is the RTX
+ symbol reference for the trap instruction label. LABELNO is the KCFI
+ label number to use for the entry label. */
+
+void
+kcfi_emit_traps_section (FILE *file, rtx trap_label_sym, int labelno)
+{
+ /* Generate entry label name with custom prefix. */
+ char entry_name[32];
+ ASM_GENERATE_INTERNAL_LABEL (entry_name, "Lkcfi_entry", labelno);
+
+ /* Save current section to restore later. */
+ section *saved_section = in_section;
+
+ /* Use varasm infrastructure for section handling.
+ .section .kcfi_traps,"ao",@progbits,<current function section> */
+ section *kcfi_traps_section = get_section (".kcfi_traps",
+ SECTION_LINK_ORDER, NULL);
+ switch_to_section (kcfi_traps_section);
+
+ /* Emit entry label for relative offset:
+ .Lkcfi_entry$N: */
+ ASM_OUTPUT_LABEL (file, entry_name);
+
+ /* Generate address difference using RTL infrastructure. */
+ rtx entry_label_sym = gen_rtx_SYMBOL_REF (Pmode, entry_name);
+ rtx addr_diff = gen_rtx_MINUS (Pmode, trap_label_sym, entry_label_sym);
+
+ /* Emit the address difference as a 4-byte value:
+ .long .Lkcfi_trap$N - .Lkcfi_entry$N */
+ assemble_integer (addr_diff, 4, BITS_PER_UNIT, 1);
+
+ /* Restore the previous section:
+ .text */
+ switch_to_section (saved_section);
+}
+
+/* Compute KCFI type ID for a function type. FNTYPE is the function
+ type tree node to compute the type ID for. Returns the 32-bit KCFI
+ type identifier hash. */
+
+static uint32_t
+compute_kcfi_type_id (tree fntype)
+{
+ gcc_assert (fntype);
+ gcc_assert (TREE_CODE (fntype) == FUNCTION_TYPE);
+
+ uint32_t type_id = kcfi_typeinfo_get_hash (fntype);
+
+ /* Apply target-specific masking if supported. */
+ if (targetm.kcfi.mask_type_id)
+ type_id = targetm.kcfi.mask_type_id (type_id);
+
+ /* Output to dump file if enabled. */
+ if (dump_file && (dump_flags & TDF_DETAILS))
+ {
+ std::string mangled_name = kcfi_typeinfo_get_name (fntype);
+ fprintf (dump_file, "KCFI type ID: mangled='%s' typeid=0x%08x\n",
+ mangled_name.c_str (), type_id);
+ }
+
+ return type_id;
+}
+
+/* Memoization cache mapping a FUNCTION_TYPE to its computed KCFI type ID.
+ The type ID is a deterministic function of the type, so this is purely a
+ per-process compile-time cache: it is never streamed for LTO (the ids are
+ recomputed lazily in each process), and GTY keeps it consistent with garbage
+ collection and PCH. Using a side map rather than a type attribute avoids
+ mutating (shared) TYPE_ATTRIBUTES lists and the per-query attribute walk. */
+static GTY(()) hash_map<tree, uint32_t> *kcfi_type_id_cache;
+
+/* Get KCFI type ID for a function type. Compute and cache it on first use.
+ FN_TYPE is the function type tree node. Returns the cached or newly
+ computed 32-bit KCFI type identifier. */
+
+static uint32_t
+kcfi_get_type_id (tree fn_type)
+{
+ gcc_assert (fn_type && TREE_CODE (fn_type) == FUNCTION_TYPE);
+
+ if (!kcfi_type_id_cache)
+ kcfi_type_id_cache = hash_map<tree, uint32_t>::create_ggc (32);
+
+ bool cache_hit;
+ uint32_t &type_id = kcfi_type_id_cache->get_or_insert (fn_type, &cache_hit);
+ if (!cache_hit)
+ type_id = compute_kcfi_type_id (fn_type);
+
+ return type_id;
+}
+
+/* Prepare the global KCFI alignment NOPs calculation, setting global variables
+ including kcfi_typeid_offset and kcfi_alignment_padding_nops based on
+ patchable function entry settings and function alignment requirements.
+ Called lazily from the consumers and guarded by kcfi_prepared so it runs at
+ most once, in the same codegen process that emits the code (which under LTO
+ is the ltrans process that consumes these values). */
+
+static void
+kcfi_prepare_alignment_nops (void)
+{
+ if (kcfi_prepared)
+ return;
+ kcfi_prepared = true;
+
+ /* Calculate where callsites load the 32-bit typeid from, based
+ on the target function address. Under default circumstances,
+ the typeid is located 4 bytes before the function entry, in
+ the __cfi_... preamble:
+
+ __cfi_func:
+ [typeid] // -4 from func
+ func:
+ [function body]
+
+ */
+ kcfi_typeid_offset = sizeof (uint32_t);
+
+ /* When Patchable Function Entry (PFE) is enabled, there may be NOP
+ instructions between the typeid and the function entry point.
+ Since KCFI callsites have no way to see PFE function attributes,
+ it can only base the calculations on the global
+ -fpatchable-function-entry=TOTAL[,PREFIX] flag. */
+ HOST_WIDE_INT prefix_nops = 0;
+ if (flag_patchable_function_entry)
+ {
+ HOST_WIDE_INT total_nops;
+ parse_and_check_patch_area (flag_patchable_function_entry, false,
+ &total_nops, &prefix_nops);
+ }
+
+ /* However, PFE is measured in NOP instruction counts, not bytes. So
+ we need to find out how many bytes they are, and we may need to
+ emit NOPs for alignment padding later. Prepare the NOP template. */
+ rtx_insn *nop_insn = make_insn_raw (gen_nop ());
+ int code_num = recog_memoized (nop_insn);
+ /* Keep a private copy: get_insn_template can return a pointer into a target
+ output routine's static scratch buffer, and nop_insn is GC-allocated, so we
+ must not cache the raw pointer for the life of the compilation. */
+ kcfi_nop = xstrdup (get_insn_template (code_num, nop_insn));
+ int nop_insn_bytes = get_attr_length (nop_insn);
+
+ /* Adjust the offset by how many NOP bytes may be between the typeid
+ and the function entry point.
+
+ __cfi_func:
+ [typeid] // -(4 + (prefix nop count * nop size)) from func
+ [prefix nops] // added when -fpatchable-function-entry is set
+ func:
+ [entry nops] // added when -fpatchable-function-entry is set
+ [function body]
+
+ At this point, kcfi_typeid_offset is ready and callsites can now
+ correctly find the typeid.
+
+ */
+ int prefix_nop_bytes = prefix_nops * nop_insn_bytes;
+ kcfi_typeid_offset += prefix_nop_bytes;
+
+ /* In the case where the KCFI preamble (and potentially the prefix NOPs)
+ are being used for alternative CFI implementations via live-patching,
+ the __cfi_... label itself needs to be usable as a callable function
+ target, so alignment NOPs may need to be added between the preamble
+ label and the typeid during KCFI preamble emission:
+
+ __cfi_func: // may need to be function entry aligned
+ [alignment padding nops] // may be needed when -falign-functions set
+ [typeid]
+ [prefix nops]
+ func:
+ [entry nops]
+ [function body]
+
+ But we only calculate alignment padding NOPs when -falign-functions
+ has been explicitly set.
+ */
+ if (align_functions.levels[0].log <= 0)
+ return;
+ int function_entry_alignment = align_functions.levels[0].get_value ();
+
+ /* Some architectures may be using an instruction for the typeid (though
+ this requires that the typeid is a trailing immediate value), but the
+ instruction will have a size greater than 4, which must be part of the
+ resulting alignment padding calculation. */
+ int typeid_insn_bytes = targetm.kcfi.emit_type_id
+ ? targetm.kcfi.emit_type_id (NULL, 0, NULL)
+ : sizeof (uint32_t);
+
+ /* Calculate needed architecture-specific alignment padding bytes. */
+ int needed_alignment_bytes = (function_entry_alignment
+ - ((prefix_nop_bytes + typeid_insn_bytes)
+ % function_entry_alignment))
+ % function_entry_alignment;
+
+ /* Calculate number of NOP instructions needed for alignment padding. */
+ if (needed_alignment_bytes % nop_insn_bytes != 0)
+ sorry ("KCFI function entry alignment padding bytes (%d) are not "
+ "a multiple of architecture NOP instruction size (%d)",
+ needed_alignment_bytes, nop_insn_bytes);
+ kcfi_alignment_padding_nops = needed_alignment_bytes / nop_insn_bytes;
+}
+
+/* Return true if call statement CALL_STMT in the current function gets a
+ KCFI check, storing the type ID it checks against in *TYPE_ID. Return
+ false if it gets no check. */
+
+static bool
+kcfi_call_type_id (const gcall *call_stmt, uint32_t *type_id)
+{
+ /* Internally checks for no_sanitize("kcfi") with current_function_decl. */
+ if (!sanitize_flags_p (SANITIZE_KCFI))
+ return false;
+
+ /* Only indirect calls need KCFI instrumentation. */
+ if (gimple_call_fndecl (call_stmt))
+ return false;
+
+ /* Skip calls originating from inlined no_sanitize("kcfi") functions. */
+ if (gimple_call_inlined_from_kcfi_nosantize_p (call_stmt))
+ return false;
+
+ /* Get function type of call. */
+ tree fn_type = gimple_call_fntype (call_stmt);
+ gcc_assert (fn_type);
+
+ *type_id = kcfi_get_type_id (fn_type);
+ return true;
+}
+
+/* Extract KCFI type ID from indirect call GIMPLE statement. Uses the
+ currently expanding GIMPLE statement to determine if KCFI instrumentation
+ is needed. Returns RTX constant with type ID, or NULL_RTX if no KCFI
+ instrumentation is required. */
+
+rtx
+internal_kcfi_get_type_id_for_expanding_gimple_call (void)
+{
+ gcc_assert (currently_expanding_gimple_stmt);
+ gcall *call_stmt = dyn_cast<gcall *> (currently_expanding_gimple_stmt);
+ gcc_assert (call_stmt);
+
+ uint32_t type_id;
+ if (!kcfi_call_type_id (call_stmt, &type_id))
+ return NULL_RTX;
+
+ return GEN_INT (type_id);
+}
+
+/* Return true if call statements CALL1 and CALL2 get the same KCFI check:
+ either neither gets one, or both are checked against the same type ID.
+ Passes that merge calls must not merge calls for which this is false, or
+ a call would be checked against the other's type. */
+
+bool
+internal_kcfi_same_call_check_p (const gcall *call1, const gcall *call2)
+{
+ uint32_t type_id1 = 0, type_id2 = 0;
+ bool checked1 = kcfi_call_type_id (call1, &type_id1);
+ bool checked2 = kcfi_call_type_id (call2, &type_id2);
+ return checked1 == checked2 && type_id1 == type_id2;
+}
+
+/* Emit KCFI type ID symbol for an address-taken external function.
+ ASM_FILE is the output assembly file stream. FNDECL is the external
+ function declaration tree node. Emits weak symbol definitions for
+ external functions that are address-taken. */
+
+void
+kcfi_emit_typeid_symbol (FILE *asm_file, tree fndecl)
+{
+ /* Only emit for external function declarations. */
+ if (TREE_CODE (fndecl) != FUNCTION_DECL || DECL_INITIAL (fndecl))
+ return;
+
+ /* Only emit for functions that are address-taken. */
+ struct cgraph_node *node = cgraph_node::get (fndecl);
+ if (!node || !node->address_taken)
+ return;
+
+ /* Get symbol name from RTL and strip encoding prefixes. */
+ rtx rtl = DECL_RTL (fndecl);
+ const char *name = XSTR (XEXP (rtl, 0), 0);
+ name = targetm.strip_name_encoding (name);
+
+ /* .weak __kcfi_typeid_{name} */
+ std::string symbol_name = std::string ("__kcfi_typeid_") + name;
+ ASM_WEAKEN_LABEL (asm_file, symbol_name.c_str ());
+
+ /* .set __kcfi_typeid_{name}, 0x{type_id} */
+ char val[16];
+ snprintf (val, sizeof (val), "0x%08x",
+ kcfi_get_type_id (TREE_TYPE (fndecl)));
+ ASM_OUTPUT_DEF (asm_file, symbol_name.c_str (), val);
+}
+
+/* Emit KCFI preamble before the function label. ASM_FILE is the output
+ assembly file stream. FNDECL is the function declaration tree node.
+ ACTUAL_FNAME is the actual function name to use, or NULL to use the
+ function's assembler name. Functions get preambles when -fsanitize=kcfi
+ is enabled, regardless of no_sanitize("kcfi") attribute. */
+
+void
+kcfi_emit_preamble (FILE *asm_file, tree fndecl, const char *actual_fname)
+{
+ /* Skip functions with nocf_check attribute. */
+ if (lookup_attribute ("nocf_check", TYPE_ATTRIBUTES (TREE_TYPE (fndecl))))
+ return;
+
+ struct cgraph_node *node = cgraph_node::get (fndecl);
+
+ /* Ignore cold partition functions: not reached via indirect call. */
+ if (node && node->split_part)
+ return;
+
+ /* Ignore cold partition sections: cold partitions are never indirect call
+ targets. Only skip preambles for cold partitions (has_bb_partition = true)
+ not for entire cold-attributed functions (has_bb_partition = false). */
+ if (in_cold_section_p && crtl && crtl->has_bb_partition)
+ return;
+
+ /* Check if function is truly address-taken using cgraph node analysis. */
+ bool addr_taken = (node && node->address_taken);
+
+ /* Only instrument functions that can be targets of indirect calls:
+ - Public functions (can be called externally)
+ - External declarations (from other modules)
+ - Functions with true address-taken status from cgraph analysis. */
+ if (!(TREE_PUBLIC (fndecl) || DECL_EXTERNAL (fndecl) || addr_taken))
+ return;
+
+ /* Use actual function name if provided, otherwise fall back to
+ DECL_ASSEMBLER_NAME. */
+ const char *fname = actual_fname
+ ? actual_fname
+ : IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (fndecl));
+
+ /* Drop any assembler-name encoding (e.g. the leading '*' on custom
+ asm () names); otherwise the "__cfi_" label would be malformed. */
+ fname = targetm.strip_name_encoding (fname);
+
+ /* Create symbol name for reuse. */
+ std::string cfi_symbol_name = std::string ("__cfi_") + fname;
+
+ /* Emit __cfi_ symbol with proper visibility. */
+ if (TREE_PUBLIC (fndecl))
+ {
+ if (DECL_WEAK (fndecl))
+ ASM_WEAKEN_LABEL (asm_file, cfi_symbol_name.c_str ());
+ else
+ targetm.asm_out.globalize_label (asm_file, cfi_symbol_name.c_str ());
+ }
+
+ /* Emit .type directive. */
+ ASM_OUTPUT_TYPE_DIRECTIVE (asm_file, cfi_symbol_name.c_str (), "function");
+ ASM_OUTPUT_LABEL (asm_file, cfi_symbol_name.c_str ());
+
+ /* Emit any needed alignment padding NOPs using target's NOP template. */
+ kcfi_prepare_alignment_nops ();
+ for (int i = 0; i < kcfi_alignment_padding_nops; i++)
+ output_asm_insn (kcfi_nop, NULL);
+
+ /* Emit type ID bytes. */
+ uint32_t type_id = kcfi_get_type_id (TREE_TYPE (fndecl));
+ if (targetm.kcfi.emit_type_id)
+ targetm.kcfi.emit_type_id (asm_file, type_id, fndecl);
+ else
+ fprintf (asm_file, "\t.word\t0x%08x\n", type_id);
+
+ /* Mark end of __cfi_ symbol and emit size directive. */
+ std::string cfi_end_label = std::string (".Lcfi_func_end_") + fname;
+ ASM_OUTPUT_LABEL (asm_file, cfi_end_label.c_str ());
+
+ ASM_OUTPUT_MEASURED_SIZE (asm_file, cfi_symbol_name.c_str ());
+}
+
+#include "gt-kcfi.h"
diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index 6491de935eda..348e14a0ae7d 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -3746,6 +3746,44 @@ void __attribute__ ((no_sanitize ("alignment,object-size")))
g () @{ /* @r{Do something.} */; @}
@end smallexample
+When @code{no_sanitize ("kcfi")} is applied to a function, it disables
+the generation of Kernel Control Flow Integrity (KCFI) instrumentation
+for indirect function calls within that function. This means that
+indirect calls in the marked function will not be checked against the
+target function's type signature.
+
+However, the function itself will still receive a KCFI preamble (type
+identifier) when compiled with @option{-fsanitize=kcfi}, allowing it to
+be safely called indirectly from other functions that do perform KCFI
+checks. In other words, @code{no_sanitize ("kcfi")} affects outgoing
+calls from the function, not incoming calls to the function.
+
+@smallexample
+void __attribute__ ((no_sanitize ("kcfi")))
+trusted_function(void (*callback)(int))
+@{
+ /* This indirect call will NOT be instrumented with KCFI checks. */
+ callback (42);
+@}
+
+void regular_function(void (*callback)(int))
+@{
+ /* This indirect call WILL be instrumented with KCFI checks. */
+ callback (42);
+@}
+@end smallexample
+
+This attribute is primarily used in kernel code for special contexts such
+as BPF JIT trampolines or other low-level code where KCFI instrumentation
+might interfere with the intended operation. The attribute survives
+inlining to ensure that @code{no_sanitize("kcfi")} functions do not generate
+KCFI checks even when inlined into a function that otherwise performs KCFI
+checks.
+
+Note: To disable KCFI preamble generation for functions so that they may
+explicitly not be called indirectly, use the @code{nocf_check} function
+attribute instead.
+
@atindex @code{no_sanitize_address}
@cindex functions that should not be sanitized
@item no_sanitize_address
@@ -4348,6 +4386,10 @@ script to place the sections with the @code{.persistent} prefix in the
right location. Specifically, some type of non-volatile, writable
memory is required.
+This attribute cannot be used with @option{-fsanitize=kcfi} because KCFI
+callsites cannot know about function-specific patchable entry settings on
+a preamble in a different translation unit.
+
@atindex @code{pure}
@cindex functions that have no side effects
@item pure
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index f89b984fd25f..f8a325b5153a 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -17531,6 +17531,45 @@ the available options are shown at startup of the instrumented program.
The option cannot be combined with @option{-fsanitize=thread} or
@option{-fsanitize=address}, and is currently only available on AArch64.
+@opindex fsanitize=kcfi
+@item -fsanitize=kcfi
+Enable Kernel Control Flow Integrity (KCFI), a lightweight control
+flow integrity mechanism designed for operating system kernels.
+KCFI instruments indirect function calls to verify that the target
+function has the expected type signature at runtime. Each function
+receives a unique type identifier computed from a hash of its function
+prototype (including parameter types and return type). Before each
+indirect call, the implementation inserts a check to verify that the
+target function's type identifier matches the expected identifier
+for the call site, issuing a trap instruction if a mismatch is detected.
+This provides forward-edge control flow protection against attacks that
+attempt to redirect indirect calls to unintended targets.
+
+The implementation adds minimal runtime overhead and does not require
+runtime library support, making it suitable for kernel environments.
+The type identifier is placed before the function entry point,
+allowing runtime verification without additional metadata structures,
+and without changing the entry points of the target functions.
+
+KCFI is intended primarily for kernel code and may not be suitable
+for user-space applications that rely on techniques incompatible
+with strict type checking of indirect calls.
+
+Note that KCFI is incompatible with function-specific
+@code{patchable_function_entry} attributes because KCFI call sites
+cannot know about function-specific patchable entry settings in different
+translation units. Only the global @option{-fpatchable-function-entry}
+command-line option is supported with KCFI.
+
+For the same reason, the location of the type identifier relative to a
+function entry point is part of a whole-program ABI: a call site computes
+that offset from its own translation unit's settings, so every translation
+unit that may be reached through an instrumented indirect call must agree on
+it.
+
+Use @option{-fdump-ipa-kcfi-details} to examine the computed type identifier
+hashes and their corresponding mangled type strings during compilation.
+
@opindex fsanitize=kernel-hwaddress
@item -fsanitize=kernel-hwaddress
Enable Hardware-assisted AddressSanitizer for compilation of the Linux kernel.
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 97afd0175220..8bea8b01e401 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -3221,6 +3221,7 @@ This describes the stack layout and calling conventions.
* Tail Calls::
* Shrink-wrapping separate components::
* Stack Smashing Protection::
+* Kernel Control Flow Integrity::
* Miscellaneous Register Hooks::
@end menu
@@ -5498,6 +5499,37 @@ should be allocated from heap memory and consumers should release them.
The result will be pruned to cases with PREFIX if not NULL.
@end deftypefn
+@node Kernel Control Flow Integrity
+@subsection Kernel Control Flow Integrity
+@cindex kernel control flow integrity
+@cindex KCFI
+
+@deftypefn {Target Hook} bool TARGET_KCFI_SUPPORTED (void)
+Return true if the target supports Kernel Control Flow Integrity (KCFI).
+This hook indicates whether the target has implemented the necessary RTL
+patterns and infrastructure to support KCFI instrumentation. The default
+implementation returns false.
+@end deftypefn
+
+@deftypefn {Target Hook} uint32_t TARGET_KCFI_MASK_TYPE_ID (uint32_t @var{type_id})
+Apply architecture-specific masking to KCFI type ID. This hook allows
+targets to apply bit masks or other transformations to the computed KCFI
+type identifier to match the target's specific requirements. The default
+implementation returns the type ID unchanged.
+@end deftypefn
+
+@deftypefn {Target Hook} int TARGET_KCFI_EMIT_TYPE_ID (FILE *@var{file}, uint32_t @var{type_id}, tree @var{fndecl})
+Emit architecture-specific type ID instruction for KCFI preambles
+and return the size of the instruction in bytes.
+@var{file} is the assembly output stream and @var{type_id} is the KCFI
+type identifier to emit. If @var{file} is NULL, skip emission and only
+return the size. If not overridden, the default fallback emits a
+@code{.word} directive with the type ID and returns 4 bytes. Targets can
+override this to emit different instruction sequences and return their
+corresponding sizes. @var{fndecl} is the function declaration, which
+targets can use to compute architecture-specific arity or other properties.
+@end deftypefn
+
@node Miscellaneous Register Hooks
@subsection Miscellaneous register hooks
@cindex miscellaneous register hooks
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 2560addbcf64..63a47c30aac5 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1625,6 +1625,7 @@ OBJS = \
ira-lives.o \
jump.o \
kcfi-typeinfo.o \
+ kcfi.o \
langhooks.o \
late-combine.o \
lcm.o \
@@ -3186,6 +3187,7 @@ GTFILES = $(CPPLIB_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
$(srcdir)/tsan.cc \
$(srcdir)/sanopt.cc \
$(srcdir)/sancov.cc \
+ $(srcdir)/kcfi.cc \
$(srcdir)/ipa-devirt.cc \
$(srcdir)/ipa-strub.cc \
$(srcdir)/internal-fn.h \
diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index c4eb8a0520a7..d35507ece51f 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -342,6 +342,8 @@ enum sanitize_code {
SANITIZE_MEMTAG_STACK = 1ULL << 32,
/* Memory Tagging. */
SANITIZE_MEMTAG = SANITIZE_MEMTAG_STACK,
+ /* KCFI (Kernel Control Flow Integrity) */
+ SANITIZE_KCFI = 1ULL << 33,
SANITIZE_SHIFT = SANITIZE_SHIFT_BASE | SANITIZE_SHIFT_EXPONENT,
SANITIZE_UNDEFINED = SANITIZE_SHIFT | SANITIZE_DIVIDE | SANITIZE_UNREACHABLE
| SANITIZE_VLA | SANITIZE_NULL | SANITIZE_RETURN
diff --git a/gcc/gimple.h b/gcc/gimple.h
index 82045c44bbeb..a61de841fd11 100644
--- a/gcc/gimple.h
+++ b/gcc/gimple.h
@@ -145,6 +145,7 @@ enum gf_mask {
GF_CALL_ALLOCA_FOR_VAR = 1 << 5,
GF_CALL_INTERNAL = 1 << 6,
GF_CALL_CTRL_ALTERING = 1 << 7,
+ GF_CALL_INLINED_FROM_KCFI_NOSANTIZE = 1 << 8,
GF_CALL_MUST_TAIL_CALL = 1 << 9,
GF_CALL_BY_DESCRIPTOR = 1 << 10,
GF_CALL_NOCF_CHECK = 1 << 11,
@@ -3499,6 +3500,27 @@ gimple_call_from_thunk_p (gcall *s)
return (s->subcode & GF_CALL_FROM_THUNK) != 0;
}
+/* If INLINED_FROM_KCFI_NOSANTIZE_P is true, mark GIMPLE_CALL S as being
+ inlined from a function with no_sanitize("kcfi"). */
+
+inline void
+gimple_call_set_inlined_from_kcfi_nosantize (gcall *s,
+ bool inlined_from_kcfi_nosantize_p)
+{
+ if (inlined_from_kcfi_nosantize_p)
+ s->subcode |= GF_CALL_INLINED_FROM_KCFI_NOSANTIZE;
+ else
+ s->subcode &= ~GF_CALL_INLINED_FROM_KCFI_NOSANTIZE;
+}
+
+/* Return true if GIMPLE_CALL S was inlined from a function with
+ no_sanitize("kcfi"). */
+
+inline bool
+gimple_call_inlined_from_kcfi_nosantize_p (const gcall *s)
+{
+ return (s->subcode & GF_CALL_INLINED_FROM_KCFI_NOSANTIZE) != 0;
+}
/* If FROM_NEW_OR_DELETE_P is true, mark GIMPLE_CALL S as being a call
to operator new or delete created from a new or delete expression. */
diff --git a/gcc/c-family/c-attribs.cc b/gcc/c-family/c-attribs.cc
index 470054f044ac..eb4c8b1a002f 100644
--- a/gcc/c-family/c-attribs.cc
+++ b/gcc/c-family/c-attribs.cc
@@ -49,6 +49,7 @@ along with GCC; see the file COPYING3. If not see
#include "gimplify.h"
#include "tree-pretty-print.h"
#include "gcc-rich-location.h"
+#include "asan.h"
#include "gcc-urlifier.h"
#include "attr-callback.h"
@@ -1536,8 +1537,11 @@ handle_nocf_check_attribute (tree *node, tree name,
warning (OPT_Wattributes, "%qE attribute ignored", name);
*no_add_attrs = true;
}
- else if (!(flag_cf_protection & CF_BRANCH))
+ else if (!(flag_cf_protection & CF_BRANCH)
+ && !(flag_sanitize & SANITIZE_KCFI))
{
+ /* Allow it with -fsanitize=kcfi, but leave this warning alone
+ to avoid confusion over this weird corner case. */
warning (OPT_Wattributes, "%qE attribute ignored. Use "
"%<-fcf-protection%> option to enable it",
name);
@@ -6014,6 +6018,17 @@ static tree
handle_patchable_function_entry_attribute (tree *, tree name, tree args,
int, bool *no_add_attrs)
{
+ /* Function-specific patchable_function_entry attribute is incompatible
+ with KCFI because KCFI callsites cannot know about function-specific
+ patchable entry settings on a preamble in a different translation
+ unit. */
+ if (sanitize_flags_p (SANITIZE_KCFI))
+ {
+ error ("%qE attribute cannot be used with %<-fsanitize=kcfi%>", name);
+ *no_add_attrs = true;
+ return NULL_TREE;
+ }
+
for (; args; args = TREE_CHAIN (args))
{
tree val = TREE_VALUE (args);
diff --git a/gcc/df-scan.cc b/gcc/df-scan.cc
index c6a5053ddbb6..e7ac751ca78f 100644
--- a/gcc/df-scan.cc
+++ b/gcc/df-scan.cc
@@ -2851,6 +2851,13 @@ df_uses_record (class df_collection_rec *collection_rec,
/* If we're clobbering a REG then we have a def so ignore. */
return;
+ case KCFI:
+ /* KCFI wraps other RTL - process the wrapped RTL. */
+ df_uses_record (collection_rec, &XEXP (x, 0), ref_type, bb, insn_info,
+ flags);
+ /* The type ID operand (XEXP (x, 1)) doesn't contain register uses. */
+ return;
+
case MEM:
df_uses_record (collection_rec,
&XEXP (x, 0), DF_REF_REG_MEM_LOAD,
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index 2113b9c679a3..aca8e292a6c5 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -2462,6 +2462,7 @@ This describes the stack layout and calling conventions.
* Tail Calls::
* Shrink-wrapping separate components::
* Stack Smashing Protection::
+* Kernel Control Flow Integrity::
* Miscellaneous Register Hooks::
@end menu
@@ -3841,6 +3842,17 @@ generic code.
@hook TARGET_GET_VALID_OPTION_VALUES
+@node Kernel Control Flow Integrity
+@subsection Kernel Control Flow Integrity
+@cindex kernel control flow integrity
+@cindex KCFI
+
+@hook TARGET_KCFI_SUPPORTED
+
+@hook TARGET_KCFI_MASK_TYPE_ID
+
+@hook TARGET_KCFI_EMIT_TYPE_ID
+
@node Miscellaneous Register Hooks
@subsection Miscellaneous register hooks
@cindex miscellaneous register hooks
diff --git a/gcc/final.cc b/gcc/final.cc
index b6be9be99b7f..b0330df035ec 100644
--- a/gcc/final.cc
+++ b/gcc/final.cc
@@ -78,6 +78,7 @@ along with GCC; see the file COPYING3. If not see
#include "stringpool.h"
#include "attribs.h"
#include "asan.h"
+#include "kcfi.h"
#include "rtl-iter.h"
#include "print-rtl.h"
#include "function-abi.h"
@@ -155,6 +156,19 @@ static rtx last_ignored_compare = 0;
static int insn_counter = 0;
+/* KCFI label counter, incremented during KCFI insn output to give each
+ indirect call site's trap/call/entry labels a unique number. Kept here
+ with the other output-phase label counters; declared in kcfi.h for its
+ architecture back-end callers. */
+
+static int kcfi_labelno = 0;
+
+int
+kcfi_next_labelno (void)
+{
+ return kcfi_labelno++;
+}
+
/* Number of unmatched NOTE_INSN_BLOCK_BEG notes we have seen. */
static int block_depth;
@@ -2088,6 +2102,9 @@ call_from_call_insn (const rtx_call_insn *insn)
case SET:
x = XEXP (x, 1);
break;
+ case KCFI:
+ x = XEXP (x, 0);
+ break;
}
}
return x;
diff --git a/gcc/opts.cc b/gcc/opts.cc
index 21a773f8da75..ad1fd54d2792 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -2227,6 +2227,7 @@ const struct sanitizer_opts_s sanitizer_opts[] =
SANITIZER_OPT (builtin, SANITIZE_BUILTIN, true, true),
SANITIZER_OPT (shadow-call-stack, SANITIZE_SHADOW_CALL_STACK, false, false),
SANITIZER_OPT (memtag-stack, SANITIZE_MEMTAG_STACK, false, false),
+ SANITIZER_OPT (kcfi, SANITIZE_KCFI, false, true),
SANITIZER_OPT (all, ~sanitize_code_type (0), true, true),
#undef SANITIZER_OPT
{ NULL, sanitize_code_type (0), 0UL, false, false }
@@ -2366,7 +2367,8 @@ parse_sanitizer_options (const char *p, location_t loc, int scode,
flags |= ~(SANITIZE_THREAD | SANITIZE_LEAK
| SANITIZE_UNREACHABLE | SANITIZE_RETURN
| SANITIZE_SHADOW_CALL_STACK
- | SANITIZE_MEMTAG_STACK);
+ | SANITIZE_MEMTAG_STACK
+ | SANITIZE_KCFI);
else /* if (code == OPT_fsanitize_trap_) */
flags |= (SANITIZE_UNDEFINED
| SANITIZE_UNDEFINED_NONDEFAULT);
diff --git a/gcc/rtl.def b/gcc/rtl.def
index cec11d832078..e01c756b562f 100644
--- a/gcc/rtl.def
+++ b/gcc/rtl.def
@@ -318,6 +318,12 @@ DEF_RTL_EXPR(CLOBBER, "clobber", "e", RTX_EXTRA)
DEF_RTL_EXPR(CALL, "call", "ee", RTX_EXTRA)
+/* KCFI wrapper for call expressions.
+ Operand 0 is the call expression.
+ Operand 1 is the KCFI type ID (const_int). */
+
+DEF_RTL_EXPR(KCFI, "kcfi", "ee", RTX_EXTRA)
+
/* Return from a subroutine. */
DEF_RTL_EXPR(RETURN, "return", "", RTX_EXTRA)
diff --git a/gcc/rtlanal.cc b/gcc/rtlanal.cc
index df4a69d180ae..251f2413df59 100644
--- a/gcc/rtlanal.cc
+++ b/gcc/rtlanal.cc
@@ -1177,6 +1177,11 @@ reg_referenced_p (const_rtx x, const_rtx body)
case IF_THEN_ELSE:
return reg_overlap_mentioned_p (x, body);
+ case KCFI:
+ /* For KCFI wrapper, only check the wrapped call. The type ID
+ (XEXP (body, 1)) is always a const_int. */
+ return reg_overlap_mentioned_p (x, XEXP (body, 0));
+
case TRAP_IF:
return reg_overlap_mentioned_p (x, TRAP_CONDITION (body));
diff --git a/gcc/target.def b/gcc/target.def
index 11334f9d443d..30862fa649bd 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -7737,6 +7737,45 @@ DEFHOOKPOD
The default value is NULL.",
const char *, NULL)
+/* Kernel Control Flow Integrity (KCFI) hooks. */
+#undef HOOK_PREFIX
+#define HOOK_PREFIX "TARGET_KCFI_"
+HOOK_VECTOR (TARGET_KCFI, kcfi)
+
+DEFHOOK
+(supported,
+ "Return true if the target supports Kernel Control Flow Integrity (KCFI).\n\
+This hook indicates whether the target has implemented the necessary RTL\n\
+patterns and infrastructure to support KCFI instrumentation. The default\n\
+implementation returns false.",
+ bool, (void),
+ hook_bool_void_false)
+
+DEFHOOK
+(mask_type_id,
+ "Apply architecture-specific masking to KCFI type ID. This hook allows\n\
+targets to apply bit masks or other transformations to the computed KCFI\n\
+type identifier to match the target's specific requirements. The default\n\
+implementation returns the type ID unchanged.",
+ uint32_t, (uint32_t type_id),
+ NULL)
+
+DEFHOOK
+(emit_type_id,
+ "Emit architecture-specific type ID instruction for KCFI preambles\n\
+and return the size of the instruction in bytes.\n\
+@var{file} is the assembly output stream and @var{type_id} is the KCFI\n\
+type identifier to emit. If @var{file} is NULL, skip emission and only\n\
+return the size. If not overridden, the default fallback emits a\n\
+@code{.word} directive with the type ID and returns 4 bytes. Targets can\n\
+override this to emit different instruction sequences and return their\n\
+corresponding sizes. @var{fndecl} is the function declaration, which\n\
+targets can use to compute architecture-specific arity or other properties.",
+ int, (FILE *file, uint32_t type_id, tree fndecl),
+ NULL)
+
+HOOK_VECTOR_END (kcfi)
+
/* Close the 'struct gcc_target' definition. */
HOOK_VECTOR_END (C90_EMPTY_HACK)
diff --git a/gcc/toplev.cc b/gcc/toplev.cc
index 30f5a3d00639..55ae40e9589a 100644
--- a/gcc/toplev.cc
+++ b/gcc/toplev.cc
@@ -68,6 +68,7 @@ along with GCC; see the file COPYING3. If not see
#include "attribs.h"
#include "asan.h"
#include "tsan.h"
+#include "kcfi.h"
#include "plugin.h"
#include "context.h"
#include "pass_manager.h"
@@ -1745,6 +1746,17 @@ process_options ()
"requires %<-fno-exceptions%>");
}
+ if (flag_sanitize & SANITIZE_KCFI)
+ {
+ if (!targetm.kcfi.supported ())
+ sorry ("%<-fsanitize=kcfi%> not supported by this target");
+
+ /* Compiling with -flto results in frontend language of GNU GIMPLE.
+ Allow it since the original source was C. */
+ if (!lang_GNU_C () && !startswith (lang_hooks.name, "GNU GIMPLE"))
+ sorry ("%<-fsanitize=kcfi%> is only supported for C");
+ }
+
HOST_WIDE_INT patch_area_size, patch_area_start;
parse_and_check_patch_area (flag_patchable_function_entry, false,
&patch_area_size, &patch_area_start);
diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc
index 8162b1f5051a..b451bcfdef61 100644
--- a/gcc/tree-inline.cc
+++ b/gcc/tree-inline.cc
@@ -2110,6 +2110,16 @@ copy_bb (copy_body_data *id, basic_block bb,
/* Advance iterator now before stmt is moved to seq_gsi. */
gsi_next (&stmts_gsi);
+ /* If inlining from a function with no_sanitize("kcfi"), mark any
+ call statements in the inlined body with the flag so they skip
+ KCFI instrumentation. */
+ if (is_gimple_call (stmt)
+ && !sanitize_flags_p (SANITIZE_KCFI, id->src_fn))
+ {
+ gcall *call = as_a <gcall *> (stmt);
+ gimple_call_set_inlined_from_kcfi_nosantize (call, true);
+ }
+
if (gimple_nop_p (stmt))
continue;
diff --git a/gcc/tree-ssa-tail-merge.cc b/gcc/tree-ssa-tail-merge.cc
index 6d81a7118b32..453cca46befd 100644
--- a/gcc/tree-ssa-tail-merge.cc
+++ b/gcc/tree-ssa-tail-merge.cc
@@ -208,6 +208,7 @@ along with GCC; see the file COPYING3. If not see
#include "tree-cfgcleanup.h"
#include "tree-ssa.h"
#include "gimple-match.h"
+#include "kcfi.h"
const int ignore_edge_flags = EDGE_DFS_BACK | EDGE_EXECUTABLE;
@@ -1148,6 +1149,11 @@ gimple_equal_p (same_succ *same_succ, gimple *s1, gimple *s2)
if (!gimple_call_same_target_p (s1, s2))
return false;
+ /* Calls through the same pointer value (e.g. via union members) can
+ still get different KCFI checks. */
+ if (!kcfi_same_call_check_p (as_a <gcall *> (s1), as_a <gcall *> (s2)))
+ return false;
+
t1 = gimple_call_chain (s1);
t2 = gimple_call_chain (s2);
if (!gimple_operand_equal_value_p (t1, t2))
diff --git a/gcc/varasm.cc b/gcc/varasm.cc
index bcaf80749517..fee88e5f4258 100644
--- a/gcc/varasm.cc
+++ b/gcc/varasm.cc
@@ -56,6 +56,7 @@ along with GCC; see the file COPYING3. If not see
#include "attribs.h"
#include "asan.h"
#include "rtl-iter.h"
+#include "kcfi.h"
#include "file-prefix-map.h" /* remap_debug_filename() */
#include "alloc-pool.h"
#include "toplev.h"
@@ -2202,6 +2203,10 @@ assemble_start_function (tree decl, const char *fnname)
unsigned short patch_area_size = crtl->patch_area_size;
unsigned short patch_area_entry = crtl->patch_area_entry;
+ /* Emit KCFI preamble before any patchable areas. */
+ if (flag_sanitize & SANITIZE_KCFI)
+ kcfi_emit_preamble (asm_out_file, decl, fnname);
+
/* Emit the patching area before the entry label, if any. */
if (patch_area_entry > 0)
targetm.asm_out.print_patchable_function_entry (asm_out_file,
@@ -2770,6 +2775,9 @@ assemble_external_real (tree decl)
/* Some systems do require some output. */
SYMBOL_REF_USED (XEXP (rtl, 0)) = 1;
ASM_OUTPUT_EXTERNAL (asm_out_file, decl, XSTR (XEXP (rtl, 0), 0));
+
+ if (flag_sanitize & SANITIZE_KCFI)
+ kcfi_emit_typeid_symbol (asm_out_file, decl);
}
}
#endif
@@ -7290,16 +7298,32 @@ default_elf_asm_named_section (const char *name, unsigned int flags,
fprintf (asm_out_file, ",%d", flags & SECTION_ENTSIZE);
if (flags & SECTION_LINK_ORDER)
{
- /* For now, only section "__patchable_function_entries"
- adopts flag SECTION_LINK_ORDER, internal label LPFE*
- was emitted in default_print_patchable_function_entry,
- just place it here for linked_to section. */
- gcc_assert (!strcmp (name, "__patchable_function_entries"));
- fprintf (asm_out_file, ",");
- char buf[256];
- ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
- current_function_funcdef_no);
- assemble_name_raw (asm_out_file, buf);
+ if (!strcmp (name, "__patchable_function_entries"))
+ {
+ /* For patchable function entries, internal label LPFE*
+ was emitted in default_print_patchable_function_entry,
+ just place it here for linked_to section. */
+ fprintf (asm_out_file, ",");
+ char buf[256];
+ ASM_GENERATE_INTERNAL_LABEL (buf, "LPFE",
+ current_function_funcdef_no);
+ assemble_name_raw (asm_out_file, buf);
+ }
+ else if (!strcmp (name, ".kcfi_traps"))
+ {
+ /* Link the KCFI traps section to the section the current
+ function's code lives in, so it is kept with the function
+ even under -ffunction-sections. */
+ section *fnsec
+ = current_function_decl ? current_function_section () : NULL;
+ if (fnsec && SECTION_STYLE (fnsec) == SECTION_NAMED)
+ fprintf (asm_out_file, ",%s", fnsec->named.name);
+ else
+ fprintf (asm_out_file, ",.text");
+ }
+ else
+ internal_error ("unexpected use of %<SECTION_LINK_ORDER%> by "
+ "section %qs", name);
}
if (HAVE_COMDAT_GROUP && (flags & SECTION_LINKONCE))
{
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v17 2/7] kcfi: Add core Kernel Control Flow Integrity infrastructure
2026-10-05 15:40 ` [PATCH v17 2/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
@ 2026-10-09 13:35 ` Jeff Law
0 siblings, 0 replies; 10+ messages in thread
From: Jeff Law @ 2026-10-09 13:35 UTC (permalink / raw)
To: Kees Cook, Andrea Pinski
Cc: Richard Biener, Jeffrey Law, Joseph Myers, Jakub Jelinek,
Martin Uecker, Peter Zijlstra, Ard Biesheuvel, Jan Hubicka,
Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
On 10/5/26 9:40 AM, Kees Cook wrote:
> Implements the Linux Kernel Control Flow Integrity ABI, which provides a
> function prototype based forward edge control flow integrity protection
> by instrumenting every indirect call to check for a hash value before
> the target function address. If the hash at the call site and the hash
> at the target do not match, execution will trap.
>
> See the start of kcfi.cc for design details.
>
> gcc/ChangeLog:
>
> * kcfi.h: New file with KCFI public interface declarations.
> * kcfi.cc: New file implementing Kernel Control Flow Integrity
> infrastructure.
> * Makefile.in (OBJS): Add kcfi.o.
> * flag-types.h (enum sanitize_code): Add SANITIZE_KCFI.
> * gimple.h (enum gf_mask): Add GF_CALL_INLINED_FROM_KCFI_NOSANTIZE.
> (gimple_call_set_inlined_from_kcfi_nosantize): New function.
> (gimple_call_inlined_from_kcfi_nosantize_p): New function.
> * df-scan.cc (df_uses_record): Add KCFI case to handle KCFI RTL
> patterns and process wrapped RTL.
> * doc/extend.texi: Update nocf_check for kcfi.
> * doc/invoke.texi (fsanitize=kcfi): Add documentation for KCFI
> sanitizer option.
> * doc/tm.texi.in: Add Kernel Control Flow Integrity section with
> TARGET_KCFI_SUPPORTED, TARGET_KCFI_MASK_TYPE_ID,
> TARGET_KCFI_EMIT_TYPE_ID hooks.
> * doc/tm.texi: Regenerate.
> * final.cc (call_from_call_insn): Add KCFI case to handle
> KCFI-wrapped calls. Add kcfi_next_labelno().
> * opts.cc (sanitizer_opts): Add kcfi entry.
> (parse_sanitizer_options): Exclude SANITIZE_KCFI from
> -fsanitize-recover=all.
> * rtl.def (KCFI): Add new RTL code for KCFI instrumentation.
> * rtlanal.cc (reg_referenced_p): Add KCFI case.
> * target.def: Add KCFI target hooks.
> * toplev.cc (process_options): Add KCFI option processing.
> * tree-inline.cc (copy_bb): Handle KCFI no_sanitize attribute
> propagation during inlining.
> * tree-ssa-tail-merge.cc: Include kcfi.h.
> (gimple_equal_p): Don't treat calls that get different KCFI
> checks as equal.
> * varasm.cc (assemble_start_function): Emit KCFI preambles.
> (assemble_external_real): Emit KCFI typeid symbols.
> (default_elf_asm_named_section): Handle .kcfi_traps using
> SECTION_LINK_ORDER flag.
>
> gcc/c-family/ChangeLog:
>
> * c-attribs.cc: Include asan.h.
> (handle_nocf_check_attribute): Enable nocf_check under kcfi.
> (handle_patchable_function_entry_attribute): Add error for using
> patchable_function_entry attribute with -fsanitize=kcfi.
>
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
>
>
> +
> +#include "coretypes.h"
> +#include "rtl.h"
In general we try to avoid having .h files include other .h files.
Instead push these into the appropriate .cc files. This make it easier
to spot potential layering violations since the use point is explicit in
the .cc file rather than blindly carried around in a .h file and
potentially getting exposed somewhere we don't want.
> +
> +/* Common helper for RTL patterns to emit .kcfi_traps section entry.
> + Call after emitting trap label and instruction with the trap symbol
> + reference. */
> +extern void kcfi_emit_traps_section (FILE *file, rtx trap_label_sym,
> + int labelno);
> +
> +/* Extract KCFI type ID from current GIMPLE statement. */
> +extern rtx internal_kcfi_get_type_id_for_expanding_gimple_call (void);
Pick one style. You've got the parameter names in one prototype, but
not the other. The general preference within GCC is without the name.
> +
> +- The check-call instruction sequence must be treated as a single unit: it
> + cannot be rearranged or split or optimized. The pattern is that
> + indirect calls, "call *%target", get converted into:
> +
> + mov $target_expression, %target ; only present if the expression was
> + ; not already in %target register
> + load -$offset(%target), %tmp ; load typeid hash from target preamble
> + cmp $typeid, %tmp ; compare expected typeid with loaded
> + je .Lkcfi_call$N ; success: jump to the indirect call
> + .Lkcfi_trap$N: ; label of trap insn
> + trap ; trap on failure, but arranged so
> + ; "permissive mode" falls through
> + .Lkcfi_call$N: ; label of call insn
> + call *%target ; actual indirect call
So I sound like a broken record at this point. I see the assertion that
it has to be an atomic unit, but nothing about why.
> + /* Keep a private copy: get_insn_template can return a pointer into a target
> + output routine's static scratch buffer, and nop_insn is GC-allocated, so we
> + must not cache the raw pointer for the life of the compilation. */
> + kcfi_nop = xstrdup (get_insn_template (code_num, nop_insn));
> + int nop_insn_bytes = get_attr_length (nop_insn);
Note that some ports do not track insn lengths, so you'll consistently
get back 0 on those. I would probably recommend asserting the size is
nonzero so that if you try to use this on a target without lengths
you'll get a compile-time error rather than a runtime surprise.
Also note that length computations are not guaranteed to be correct;
they are guaranteed to conservatively correct for things like
jump/branch relaxing to work. So in that context conservatively correct
means the reported size is no smaller than the actual size.
So you need to make sure you're safe if the port reports back a longer
size than what gets encoded into the final object.
> + int prefix_nop_bytes = prefix_nops * nop_insn_bytes;
> + kcfi_typeid_offset += prefix_nop_bytes;
> +
> + /* In the case where the KCFI preamble (and potentially the prefix NOPs)
> + are being used for alternative CFI implementations via live-patching,
> + the __cfi_... label itself needs to be usable as a callable function
> + target, so alignment NOPs may need to be added between the preamble
> + label and the typeid during KCFI preamble emission:
> +
> + __cfi_func: // may need to be function entry aligned
> + [alignment padding nops] // may be needed when -falign-functions set
> + [typeid]
> + [prefix nops]
> + func:
> + [entry nops]
> + [function body]
> +
> + But we only calculate alignment padding NOPs when -falign-functions
> + has been explicitly set.
> + */
> + if (align_functions.levels[0].log <= 0)
> + return;
So what happens if the port does not align functions to a useful boundary?
> +
> + /* Ignore cold partition sections: cold partitions are never indirect call
> + targets. Only skip preambles for cold partitions (has_bb_partition = true)
> + not for entire cold-attributed functions (has_bb_partition = false). */
> + if (in_cold_section_p && crtl && crtl->has_bb_partition)
> + return;
Is that an design decision in the partitioning code or is it an
implementation detail. If it's the former, than I'm inclined to allow
relying on it. If it's the latter, then it seems we're just setting up
a ticking time bomb. It could well be either, I don't know off the top
of my head.
I'm assuming that things like devirtualizing a call into a direct call
are safe, including the speculative forms.
Given we can pretty easily describe the generic call setup in RTL, how
painful would it be to instead do the instrumentation in a generic way
rather than twiddling each backend?
In that kind of design we'd force the target to provide the two marker
insns which mark the beginning and end of the atomic region. Target
independent code would be used to load the typeid hash from the preamble
(using the offset the target provides), compare it to the expected type
id, branch around a trap-like insn, the trap-like insn itself, then
finally the call. I'm not demanding this rethink, but I do want it to
be at least considered as a design question.
Jeff
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v17 3/7] kcfi: Add regression test suite
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
2026-10-05 15:40 ` [PATCH v17 1/7] kcfi: Introduce KCFI typeinfo mangling API Kees Cook
2026-10-05 15:40 ` [PATCH v17 2/7] kcfi: Add core Kernel Control Flow Integrity infrastructure Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-05 15:40 ` [PATCH v17 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
` (3 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Add test suite for KCFI (Kernel Control Flow Integrity) ABI, covering
core functionality, optimization and code generation, addressing,
architecture-specific KCFI sequence emission, and integration with
patchable function entry.
The arch-specific patterns themselves are added with the subsequent
architecture patches.
Tests can be run via:
make check-c RUNTESTFLAGS='kcfi.exp'
Assisted-by: LLM [tests]
gcc/testsuite/ChangeLog:
* lib/target-supports.exp: Add check_effective_target_kcfi.
* gcc.dg/kcfi/kcfi.exp: Add kcfi tests.
* gcc.dg/kcfi/kcfi-adjacency.c: New test.
* gcc.dg/kcfi/kcfi-asm-name.c: New test.
* gcc.dg/kcfi/kcfi-basics.c: New test.
* gcc.dg/kcfi/kcfi-call-sharing.c: New test.
* gcc.dg/kcfi/kcfi-cold-partition.c: New test.
* gcc.dg/kcfi/kcfi-complex-addressing.c: New test.
* gcc.dg/kcfi/kcfi-direct-call-shapes.c: New test.
* gcc.dg/kcfi/kcfi-ipa-robustness.c: New test.
* gcc.dg/kcfi/kcfi-lto-offset.c: New test.
* gcc.dg/kcfi/kcfi-move-preservation.c: New test.
* gcc.dg/kcfi/kcfi-no-sanitize-inline.c: New test.
* gcc.dg/kcfi/kcfi-no-sanitize.c: New test.
* gcc.dg/kcfi/kcfi-offset-validation.c: New test.
* gcc.dg/kcfi/kcfi-patchable-entry-only.c: New test.
* gcc.dg/kcfi/kcfi-patchable-incompatible.c: New test.
* gcc.dg/kcfi/kcfi-patchable-large.c: New test.
* gcc.dg/kcfi/kcfi-patchable-medium.c: New test.
* gcc.dg/kcfi/kcfi-patchable-prefix-only.c: New test.
* gcc.dg/kcfi/kcfi-qualified-params.c: New test.
* gcc.dg/kcfi/kcfi-runtime.c: New test.
* gcc.dg/kcfi/kcfi-tail-calls.c: New test.
* gcc.dg/kcfi/kcfi-tail-merge-run.c: New test.
* gcc.dg/kcfi/kcfi-tail-merge.c: New test.
* gcc.dg/kcfi/kcfi-trap-section.c: New test.
* gcc.dg/kcfi-compat/kcfi-compat.exp: Add kcfi compat tests.
* gcc.dg/kcfi-compat/kcfi-indirect-1.h: New test header.
* gcc.dg/kcfi-compat/kcfi-indirect-1_main.c: New test main.
* gcc.dg/kcfi-compat/kcfi-indirect-1_x.c: New test caller half.
* gcc.dg/kcfi-compat/kcfi-indirect-1_y.c: New test target half.
* gcc.dg/pch/kcfi-1.c: New test.
* gcc.dg/pch/kcfi-1.hs: New test header.
Signed-off-by: Kees Cook <kees@kernel.org>
---
.../gcc.dg/kcfi-compat/kcfi-compat.exp | 87 ++++++
gcc/testsuite/gcc.dg/kcfi/kcfi.exp | 51 ++++
gcc/testsuite/lib/target-supports.exp | 14 +
.../gcc.dg/kcfi-compat/kcfi-indirect-1.h | 26 ++
.../gcc.dg/kcfi-compat/kcfi-indirect-1_main.c | 22 ++
.../gcc.dg/kcfi-compat/kcfi-indirect-1_x.c | 43 +++
.../gcc.dg/kcfi-compat/kcfi-indirect-1_y.c | 40 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 49 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-asm-name.c | 16 +
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 74 +++++
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 61 ++++
.../gcc.dg/kcfi/kcfi-cold-partition.c | 126 ++++++++
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 131 +++++++++
.../gcc.dg/kcfi/kcfi-direct-call-shapes.c | 28 ++
.../gcc.dg/kcfi/kcfi-ipa-robustness.c | 54 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-lto-offset.c | 27 ++
.../gcc.dg/kcfi/kcfi-move-preservation.c | 41 +++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 74 +++++
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 31 ++
.../gcc.dg/kcfi/kcfi-offset-validation.c | 24 ++
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 14 +
.../gcc.dg/kcfi/kcfi-patchable-incompatible.c | 7 +
.../gcc.dg/kcfi/kcfi-patchable-large.c | 14 +
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 14 +
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 14 +
.../gcc.dg/kcfi/kcfi-qualified-params.c | 58 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-runtime.c | 276 ++++++++++++++++++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 60 ++++
.../gcc.dg/kcfi/kcfi-tail-merge-run.c | 54 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge.c | 36 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 17 ++
gcc/testsuite/gcc.dg/pch/kcfi-1.c | 50 ++++
gcc/testsuite/gcc.dg/pch/kcfi-1.hs | 20 ++
33 files changed, 1653 insertions(+)
create mode 100644 gcc/testsuite/gcc.dg/kcfi-compat/kcfi-compat.exp
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi.exp
create mode 100644 gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1.h
create mode 100644 gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_main.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_x.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_y.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-asm-name.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-lto-offset.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-incompatible.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-qualified-params.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-runtime.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge-run.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
create mode 100644 gcc/testsuite/gcc.dg/pch/kcfi-1.c
create mode 100644 gcc/testsuite/gcc.dg/pch/kcfi-1.hs
diff --git a/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-compat.exp b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-compat.exp
new file mode 100644
index 000000000000..973a92a8ad0c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-compat.exp
@@ -0,0 +1,87 @@
+# KCFI cross-TU / cross-compiler indirect-call ABI compatibility tests.
+# Copyright (C) 2026 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>.
+
+# Split each test into separately-compiled halves and verify that KCFI
+# indirect-call checks pass across the object boundary (i.e. that two
+# independent compilations agree on the type-id ABI). The _x TU is the
+# indirect caller and the _y TU holds the targets; the framework builds
+# each with the compiler under test and, when an alternate compiler is
+# configured, in every combination, so both compilers are exercised as
+# caller and as target.
+#
+# By default both halves are built with the compiler under test. To check
+# cross-compiler agreement with clang, pass ALT_CC_UNDER_TEST as a make
+# variable (DejaGnu writes it into site.exp; remove a stale site.exp first):
+#
+# rm gcc/site.exp
+# make check-c RUNTESTFLAGS="kcfi-compat.exp" \
+# ALT_CC_UNDER_TEST="/path/to/clang -Xclang -fsanitize-kcfi-hash=FNV-1a"
+
+if $tracelevel then {
+ strace $tracelevel
+}
+
+global GCC_UNDER_TEST
+global ALT_CC_UNDER_TEST
+global compat_save_gcc_under_test
+global compat_use_alt
+global compat_same_alt
+
+# c-compat.exp must be loaded before compat.exp (which guards on it).
+load_lib standard.exp
+load_lib gcc.exp
+load_lib c-compat.exp
+load_lib compat.exp
+
+gcc_init
+
+# Only run where -fsanitize=kcfi is supported (and can be linked and run).
+if { ![check_no_compiler_messages fsanitize_kcfi executable {
+ int main (void) { return 0; }
+} "-fsanitize=kcfi"] } {
+ return
+}
+
+# Save the compiler under test; each test switches it a couple of times.
+set compat_save_gcc_under_test $GCC_UNDER_TEST
+
+# Identifier for this suite, to avoid object-file name clashes.
+set sid "kcfi_compat"
+
+# Find out whether there is an alternate compiler to test.
+set compat_use_alt 0
+set compat_same_alt 0
+if [info exists ALT_CC_UNDER_TEST] then {
+ set compat_use_alt 1
+ if [string match "same" $ALT_CC_UNDER_TEST] then {
+ set compat_same_alt 1
+ }
+}
+
+# Initialize the compat framework globals (compat_save_TEST_ALWAYS_FLAGS,
+# compat_alt_*, DFP support, ...).
+compat_setup_dfp
+
+foreach src [lsort [find $srcdir/$subdir *_main.c]] {
+ if ![runtest_file_p $runtests $src] then {
+ continue
+ }
+ compat-execute $src $sid $compat_use_alt
+}
+
+# Restore the original compiler under test.
+compat-use-tst-compiler
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi.exp b/gcc/testsuite/gcc.dg/kcfi/kcfi.exp
new file mode 100644
index 000000000000..8c7f9421071c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi.exp
@@ -0,0 +1,51 @@
+# Copyright (C) 2026 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>.
+
+# GCC testsuite for KCFI (Kernel Control Flow Integrity) tests.
+
+# Load support procs.
+load_lib gcc-dg.exp
+
+# Skip tests if KCFI is not supported on this target.
+if { ![check_effective_target_kcfi] } {
+ return
+}
+
+# Save DEFAULT_CFLAGS to restore later.
+global DEFAULT_CFLAGS
+if [info exists DEFAULT_CFLAGS] then {
+ set save_default_cflags $DEFAULT_CFLAGS
+}
+
+# Enable KCFI for all the kcfi/ tests.
+set DEFAULT_CFLAGS "-fsanitize=kcfi"
+
+# Initialize `dg'.
+dg-init
+
+# Main loop.
+dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.c]] \
+ "" $DEFAULT_CFLAGS
+
+# All done.
+dg-finish
+
+# Restore original DEFAULT_CFLAGS.
+if [info exists save_default_cflags] {
+ set DEFAULT_CFLAGS $save_default_cflags
+} else {
+ unset DEFAULT_CFLAGS
+}
diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
index ae04b6a8daa1..6b6db81da2bb 100644
--- a/gcc/testsuite/lib/target-supports.exp
+++ b/gcc/testsuite/lib/target-supports.exp
@@ -14436,6 +14436,20 @@ proc check_effective_target_no_fsanitize_address {} {
return 0;
}
+# Return 1 if this target supports KCFI (Kernel Control Flow Integrity)
+
+proc check_effective_target_kcfi {} {
+ # We don't need a full executable to test this flag
+ return [check_no_compiler_messages kcfi assembly {
+ void func(void) {}
+ int main(void) {
+ void (*ptr)(void) = func;
+ ptr();
+ return 0;
+ }
+ } "-fsanitize=kcfi"]
+}
+
# Return 1 if the target supports GNU Retain, 0 otherwise.
# This means both SHF_GNU_RETAIN and SHT_INIT_ARRAY to simplify uses.
diff --git a/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1.h b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1.h
new file mode 100644
index 000000000000..1c399c9bc4e7
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1.h
@@ -0,0 +1,26 @@
+/* Shared declarations for the KCFI cross-TU indirect-call compat test.
+
+ Including this in both the caller (_x) and the targets (_y) keeps their view
+ of each target's prototype (and therefore the derived KCFI type-id) in
+ sync from a single source. */
+
+#ifndef KCFI_INDIRECT_1_H
+#define KCFI_INDIRECT_1_H
+
+typedef int (*fp1) (int);
+typedef int (*fp2) (int, int);
+typedef int (*fp3) (int *);
+
+/* Targets defined in the _y TU, called indirectly from the _x TU. */
+extern int tgt_inc (int);
+extern int tgt_add (int, int);
+extern int tgt_neg (int);
+extern int tgt_mul (int, int);
+/* Called through fp3: the top-level const on the parameter is not part of
+ the function type, so it must not change the type-id. */
+extern int tgt_deref (int *const);
+
+/* Entry point of the caller TU (_x), invoked from _main. */
+extern void run_caller (void);
+
+#endif /* KCFI_INDIRECT_1_H */
diff --git a/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_main.c b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_main.c
new file mode 100644
index 000000000000..799579f977d0
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_main.c
@@ -0,0 +1,22 @@
+/* KCFI cross-TU / cross-compiler indirect-call ABI compatibility test.
+
+ Bootstrap TU. main() lives here because the compat framework always builds
+ the _main TU with the compiler under test; it does nothing interesting but
+ hand control to the caller in the _x TU.
+
+ The cross-TU / cross-compiler pairing under test is _x (the indirect caller)
+ against _y (the targets): the framework builds each of those with the
+ compiler under test and, when ALT_CC_UNDER_TEST is set, an alternate
+ compiler (in every combination) so both compilers get exercised as the
+ caller and as the target. See kcfi-compat.exp. */
+
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+#include "kcfi-indirect-1.h"
+
+int
+main (void)
+{
+ run_caller ();
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_x.c b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_x.c
new file mode 100644
index 000000000000..fcea1c15b35e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_x.c
@@ -0,0 +1,43 @@
+/* KCFI cross-TU / cross-compiler compat test: the "caller" TU.
+
+ This TU holds the KCFI-checked indirect calls; the target functions are
+ defined in the _y TU and declared in the shared header. For each call to
+ succeed, the call-site type-id computed here (from the pointer's type) must
+ equal the __cfi_ preamble type-id the _y TU computed (from the function's
+ type). I.e. the two separately-compiled objects (potentially built by
+ different compilers) must agree on the KCFI type-id ABI. A mismatch traps
+ (SIGILL); success is running to completion. */
+
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+#include "kcfi-indirect-1.h"
+
+void abort (void);
+
+/* volatile keeps each call indirect, so it is actually KCFI-checked. */
+volatile fp1 p_inc, p_neg;
+volatile fp2 p_add, p_mul;
+volatile fp3 p_deref;
+
+void
+run_caller (void)
+{
+ int v = 42;
+
+ p_inc = tgt_inc;
+ p_add = tgt_add;
+ p_neg = tgt_neg;
+ p_mul = tgt_mul;
+ p_deref = tgt_deref;
+
+ if (p_inc (41) != 42)
+ abort ();
+ if (p_add (20, 22) != 42)
+ abort ();
+ if (p_neg (-42) != 42)
+ abort ();
+ if (p_mul (6, 7) != 42)
+ abort ();
+ if (p_deref (&v) != 42)
+ abort ();
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_y.c b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_y.c
new file mode 100644
index 000000000000..b67635c25fee
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi-compat/kcfi-indirect-1_y.c
@@ -0,0 +1,40 @@
+/* KCFI cross-TU / cross-compiler compat test: the "target" TU.
+
+ Defines the address-taken functions the caller (_x) invokes indirectly.
+ Being address-taken, each gets a __cfi_ preamble whose type-id is derived
+ from the function's own type; a call succeeds only if that matches the
+ caller's call-site type-id. */
+
+/* { dg-options "-fsanitize=kcfi -O2" } */
+
+#include "kcfi-indirect-1.h"
+
+int
+tgt_inc (int a)
+{
+ return a + 1;
+}
+
+int
+tgt_add (int a, int b)
+{
+ return a + b;
+}
+
+int
+tgt_neg (int a)
+{
+ return -a;
+}
+
+int
+tgt_mul (int a, int b)
+{
+ return a * b;
+}
+
+int
+tgt_deref (int *const p)
+{
+ return *p;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
new file mode 100644
index 000000000000..7c1cff986c01
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -0,0 +1,49 @@
+/* Test KCFI check/transfer adjacency - regression test for instruction
+ insertion. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+/* This test ensures that KCFI security checks remain immediately adjacent
+ to their corresponding indirect calls/jumps, with no executable instructions
+ between the type ID check and the control flow transfer. */
+
+/* External function pointers to prevent optimization. */
+extern void (*complex_func_ptr)(int, int, int, int);
+extern int (*return_func_ptr)(int, int);
+
+/* Function with complex argument preparation that could tempt
+ the optimizer to insert instructions between KCFI check and call. */
+__attribute__((noinline)) void test_complex_args(int a, int b, int c, int d) {
+ /* Complex argument expressions that might cause instruction scheduling. */
+ complex_func_ptr(a * 2, b + c, d - a, (a << 1) | b);
+}
+
+/* Function with return value handling. */
+__attribute__((noinline)) int test_return_value(int x, int y) {
+ /* Return value handling that shouldn't interfere with adjacency. */
+ int result = return_func_ptr(x + 1, y * 2);
+ return result + 1;
+}
+
+/* Test struct field access that caused issues in try-catch.c. */
+struct call_info {
+ void (*handler)(void);
+ int status;
+ int data;
+};
+
+extern struct call_info *global_call_info;
+
+__attribute__((noinline)) void test_struct_field_call(void) {
+ /* This pattern caused adjacency issues before the fix. */
+ global_call_info->handler();
+}
+
+/* Test conditional indirect call. */
+__attribute__((noinline)) void test_conditional_call(int flag) {
+ if (flag) {
+ global_call_info->handler();
+ }
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-asm-name.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-asm-name.c
new file mode 100644
index 000000000000..fbf34c30a591
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-asm-name.c
@@ -0,0 +1,16 @@
+/* Test that the KCFI preamble strips the assembler-name encoding when
+ forming the __cfi_ label. Functions with a custom asm() name have a
+ leading '*' in their DECL_ASSEMBLER_NAME; without stripping it the
+ preamble emits "__cfi_*name:", which the assembler rejects. Scanning the
+ emitted label is enough to catch this, so use dg-do compile (portable to
+ targets without an assembler). */
+/* { dg-do compile } */
+
+int realname (int x) __asm__ ("my_custom_name");
+int realname (int x) { return x + 1; }
+
+int call (int (*p) (int), int x) { return p (x); }
+
+/* The preamble label must be clean (no encoding '*'). */
+/* { dg-final { scan-assembler "__cfi_my_custom_name:" } } */
+/* { dg-final { scan-assembler-not {__cfi_\*} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
new file mode 100644
index 000000000000..ca833fed2971
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -0,0 +1,74 @@
+/* Test basic KCFI functionality - preamble generation. */
+/* { dg-do compile } */
+
+/* Extern function declarations - should NOT get KCFI preambles. */
+extern void external_func(void);
+extern int external_func_int(int x);
+
+void regular_function(int x) {
+ /* This should get KCFI preamble. */
+}
+
+void static_target_function(int x) {
+ /* Target function that can be called indirectly. */
+}
+
+__attribute__((nocf_check))
+void nocf_check_function(int x) {
+ /* This function has nocf_check attribute - should NOT get KCFI preamble. */
+}
+
+static void static_caller(void) {
+ /* Static function that makes an indirect call
+ Should NOT get KCFI preamble (not address-taken)
+ But must generate KCFI check for the indirect call. */
+ void (*local_ptr)(int) = static_target_function;
+ local_ptr(42); /* This should generate KCFI check. */
+}
+
+/* Make external_func address-taken. */
+void (*func_ptr)(int) = regular_function;
+void (*ext_ptr)(void) = external_func;
+void (__attribute__((nocf_check)) *nocf_ptr)(int) = nocf_check_function;
+
+int main() {
+ func_ptr(42);
+ ext_ptr(); /* Indirect call to external_func. */
+ external_func_int(10); /* Direct call to external_func_int. */
+ static_caller(); /* Direct call to static function. */
+ return 0;
+}
+
+/* Verify KCFI preamble exists for regular_function. */
+/* { dg-final { scan-assembler {__cfi_regular_function:} } } */
+
+/* Verify KCFI preamble symbol comes before main function symbol. */
+/* { dg-final { scan-assembler {__cfi_regular_function:.*regular_function:} } } */
+
+/* Target function should have preamble (address-taken). */
+/* { dg-final { scan-assembler {__cfi_static_target_function:} } } */
+
+/* Static caller should NOT have preamble (it's only called directly,
+ not address-taken). */
+/* { dg-final { scan-assembler-not {__cfi_static_caller:} } } */
+
+/* Function with nocf_check attribute should NOT have preamble. */
+/* { dg-final { scan-assembler-not {__cfi_nocf_check_function:} } } */
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
+
+/* Extern functions should NOT get KCFI preambles. */
+/* { dg-final { scan-assembler-not {__cfi_external_func:} } } */
+/* { dg-final { scan-assembler-not {__cfi_external_func_int:} } } */
+
+/* Local functions should NOT get __kcfi_typeid_ symbols. */
+/* Only external declarations that are address-taken should get __kcfi_typeid_ */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_regular_function} } } */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_main} } } */
+
+/* External address-taken functions should get __kcfi_typeid_ symbols. */
+/* { dg-final { scan-assembler {__kcfi_typeid_external_func} } } */
+
+/* External functions that are only called directly should NOT get
+ __kcfi_typeid_ symbols. */
+/* { dg-final { scan-assembler-not {__kcfi_typeid_external_func_int} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
new file mode 100644
index 000000000000..f72344f77124
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -0,0 +1,61 @@
+/* Test KCFI check sharing bug - optimizer incorrectly shares KCFI checks
+ between different function types. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+/* Reproduce the pattern from Linux kernel internal_create_group where:
+ - Two different function pointer types (is_visible vs is_bin_visible).
+ - Both get loaded into the same register (%rcx).
+ - Optimizer creates shared KCFI check with wrong type ID.
+ - This causes CFI failures in production kernel. */
+
+struct kobject { int dummy; };
+struct attribute { int dummy; };
+struct bin_attribute { int dummy; };
+
+struct attribute_group {
+ const char *name;
+ /* Type ID A. */
+ int (*is_visible)(struct kobject *, struct attribute *, int);
+ /* Type ID B. */
+ int (*is_bin_visible)(struct kobject *, const struct bin_attribute *, int);
+ /* Type ID B again. */
+ int (*is_bin_visible_again)(struct kobject *, const struct bin_attribute *, int);
+ struct attribute **attrs;
+ const struct bin_attribute **bin_attrs;
+};
+
+/* Function that mimics __first_visible from kernel - gets inlined into
+ caller. */
+static int __first_visible(const struct attribute_group *grp, struct kobject *kobj)
+{
+ /* Path 1: Call is_visible function pointer. */
+ if (grp->attrs && grp->attrs[0] && grp->is_visible)
+ return grp->is_visible(kobj, grp->attrs[0], 0);
+
+ /* Path 2: Call is_bin_visible function pointer. */
+ if (grp->bin_attrs && grp->bin_attrs[0] && grp->is_bin_visible)
+ return grp->is_bin_visible(kobj, grp->bin_attrs[0], 0);
+
+ /* Path 3: Call is_bin_visible_again function pointer. */
+ if (grp->bin_attrs && grp->bin_attrs[0] && grp->is_bin_visible_again)
+ return grp->is_bin_visible_again(kobj, grp->bin_attrs[0], 0);
+
+ return 0;
+}
+
+/* Main function that triggers the optimization bug. */
+int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *grp)
+{
+ /* This should inline __first_visible and create the problematic pattern where:
+ 1. Both function pointers get loaded into same register.
+ 2. Optimizer shares KCFI check between them.
+ 3. Uses wrong type ID for one of the calls. */
+ return __first_visible(grp, kobj);
+}
+
+/* Each indirect call should have its own KCFI check with correct type ID.
+
+ Should see:
+ 1. KCFI check for is_visible call with is_visible type ID A.
+ 2. KCFI check for is_bin_visible and is_bin_visible_again call with type ID B. */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
new file mode 100644
index 000000000000..87ffdba1f0ae
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-cold-partition.c
@@ -0,0 +1,126 @@
+/* Test KCFI cold function and cold partition behavior. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+/* { dg-additional-options "-freorder-blocks-and-partition" { target freorder } } */
+
+void regular_function(void) {
+ /* Regular function should get preamble. */
+}
+
+/* Cold-attributed function should STILL get preamble (it's a regular
+ function, just marked cold). */
+__attribute__((cold))
+void cold_attributed_function(void) {
+ /* This function has cold attribute but should still get KCFI preamble. */
+}
+
+/* Hot-attributed function should get preamble. */
+__attribute__((hot))
+void hot_attributed_function(void) {
+ /* This function is explicitly hot and should get KCFI preamble. */
+}
+
+/* Global to prevent optimization from eliminating cold paths. */
+extern void abort(void);
+
+/* Additional function to test that normal functions still get preambles. */
+__attribute__((noinline))
+int another_regular_function(int x) {
+ return x + 42;
+}
+
+/* Function designed to generate cold partitions under optimization. */
+__attribute__((noinline))
+void function_with_cold_partition(int condition) {
+ /* Hot path - very likely to execute. */
+ if (__builtin_expect(condition == 42, 1)) {
+ /* Simple hot path that optimizer will keep inline. */
+ return;
+ }
+
+ /* Cold paths that actually do something to prevent elimination. */
+ if (__builtin_expect(condition < 0, 0)) {
+ /* Error path 1 - call abort to prevent elimination. */
+ abort();
+ }
+
+ if (__builtin_expect(condition > 1000000, 0)) {
+ /* Error path 2 - call abort to prevent elimination. */
+ abort();
+ }
+
+ if (__builtin_expect(condition == 999999, 0)) {
+ /* Error path 3 - more substantial cold code. */
+ volatile int sum = 0;
+ for (volatile int i = 0; i < 100; i++) {
+ sum += i * condition;
+ }
+ if (sum > 0)
+ abort();
+ }
+
+ /* More cold paths - switch with many unlikely cases. */
+ switch (condition) {
+ case 1000001: case 1000002: case 1000003: case 1000004: case 1000005:
+ case 1000006: case 1000007: case 1000008: case 1000009: case 1000010:
+ /* Each case does some work before abort. */
+ volatile int work = condition * 2;
+ if (work > 0) abort();
+ break;
+ default:
+ if (condition != 42) {
+ /* Fallback cold path - substantial work. */
+ volatile int result = 0;
+ for (volatile int j = 0; j < condition % 50; j++) {
+ result += j;
+ }
+ if (result >= 0) abort();
+ }
+ }
+}
+
+/* Test function pointers to ensure address-taken detection works. */
+void test_function_pointers(void) {
+ void (*regular_ptr)(void) = regular_function;
+ void (*cold_ptr)(void) = cold_attributed_function;
+ void (*hot_ptr)(void) = hot_attributed_function;
+
+ regular_ptr();
+ cold_ptr();
+ hot_ptr();
+}
+
+int main() {
+ regular_function();
+ cold_attributed_function();
+ hot_attributed_function();
+ function_with_cold_partition(42); /* Normal case - stay in hot path. */
+ another_regular_function(5);
+ test_function_pointers();
+ return 0;
+}
+
+/* Regular function should have preamble. */
+/* { dg-final { scan-assembler "__cfi_regular_function:" } } */
+
+/* Cold-attributed function should STILL have preamble (it's a legitimate function) */
+/* { dg-final { scan-assembler "__cfi_cold_attributed_function:" } } */
+
+/* Hot-attributed function should have preamble. */
+/* { dg-final { scan-assembler "__cfi_hot_attributed_function:" } } */
+
+/* Function that generates cold partitions should have preamble for main entry. */
+/* { dg-final { scan-assembler "__cfi_function_with_cold_partition:" } } */
+
+/* Address-taken functions should have preambles. */
+/* { dg-final { scan-assembler "__cfi_test_function_pointers:" } } */
+
+/* The function should generate a .cold partition (only on targets that support freorder) */
+/* { dg-final { scan-assembler "function_with_cold_partition\\.cold:" { target freorder } } } */
+
+/* The .cold partition should NOT get a __cfi_ preamble since it's never
+ reached via indirect calls. */
+/* { dg-final { scan-assembler-not "__cfi_function_with_cold_partition\\.cold:" { target freorder } } } */
+
+/* Additional regular function should get preamble. */
+/* { dg-final { scan-assembler "__cfi_another_regular_function:" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
new file mode 100644
index 000000000000..c48b8d7ad552
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -0,0 +1,131 @@
+/* Test KCFI with complex addressing modes (structure members, array
+ elements). This is a regression test for the change_address_1 RTL
+ error that occurred when target_addr was PLUS(reg, offset) instead
+ of a simple register. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+struct function_table {
+ int (*callback1)(int);
+ int (*callback2)(int, int);
+ void (*callback3)(void);
+ int (*callback4)(void *, void *, void *, void *, void *, void *);
+ int data;
+};
+
+static int handler1(int x) {
+ return x * 2;
+}
+
+static int handler2(int x, int y) {
+ return x + y;
+}
+
+static void handler3(void) {
+ /* Empty handler. */
+}
+
+/* Test indirect calls through structure members - this creates
+ PLUS(reg, offset) addressing. */
+__attribute__((noinline))
+int test_struct_members(struct function_table *table) {
+ int result = 0;
+ int loop;
+
+ /* These indirect calls will generate complex addressing modes:
+ * call *(%rdi) - callback1 at offset 0
+ * call *8(%rdi) - callback2 at offset 8
+ * call *16(%rdi) - callback3 at offset 16
+ * KCFI must handle PLUS(reg, struct_offset) + kcfi_offset. */
+
+ for (loop = 0; loop < 16; loop++) {
+ result += table->callback1(10);
+ result += table->callback2(5, 7);
+ table->callback3();
+ result += table->callback4(handler1, handler2, handler3, &result, test_struct_members, table);
+ }
+
+ return result;
+}
+
+/* Test indirect calls through array elements - another source of
+ complex addressing. */
+typedef int (*func_array_t)(int);
+
+int test_array_elements(func_array_t functions[], int index) {
+ /* This creates addressing like MEM[PLUS(PLUS(reg, index*8), 0)]
+ which should be simplified to MEM[PLUS(reg, index*8)]. */
+ return functions[index](42);
+}
+
+/* Test with global structure. */
+static struct function_table global_table = {
+ .callback1 = handler1,
+ .callback2 = handler2,
+ .callback3 = handler3,
+ .data = 100
+};
+
+int test_global_struct(void) {
+ /* Access through global structure - may generate different
+ addressing patterns. */
+ return global_table.callback1(20) + global_table.callback2(3, 4);
+}
+
+/* Test nested structure access. */
+struct nested_table {
+ struct function_table inner;
+ int extra_data;
+};
+
+int test_nested_struct(struct nested_table *nested) {
+ /* Even more complex addressing: nested structure member access. */
+ return nested->inner.callback1(15);
+}
+
+int test_many_args(void *one, void *two, void *three, void *four, void *five, void *six)
+{
+ return (unsigned long)one + (unsigned long)two + (unsigned long)three
+ + (unsigned long)four + (unsigned long)five + (unsigned long)six;
+}
+
+int target_func(int a, int b, int c, int d) { return a + b + c + d; }
+
+/* Function to force r3 spill/reload by using 4+ arguments in indirect call */
+__attribute__((noinline))
+int force_r3_spill(int (*fp)(int, int, int, int)) {
+ int (*ip_reg)(int, int, int, int) = fp;
+ volatile int val = 0;
+ /* This should force r3 as scratch since ip is the target */
+ return ip_reg(val, val, val, val) + 5;
+}
+
+int main() {
+ struct function_table local_table = {
+ .callback1 = handler1,
+ .callback2 = handler2,
+ .callback3 = handler3,
+ .callback4 = test_many_args,
+ .data = 50
+ };
+
+ func_array_t func_array[] = { handler1, handler1, handler1 };
+
+ int result = 0;
+ result += test_struct_members(&local_table);
+ result += test_array_elements(func_array, 1);
+ result += test_global_struct();
+
+ struct nested_table nested = { .inner = local_table, .extra_data = 200 };
+ result += test_nested_struct(&nested);
+
+ /* Force r3 spill/reload pattern in ARM32 KCFI */
+ volatile int (*four_arg_ptr)(int, int, int, int) = target_func;
+ result += force_r3_spill(four_arg_ptr);
+
+ result += local_table.callback4(handler1, handler2, handler3, &result, main, &local_table);
+
+ return result;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
new file mode 100644
index 000000000000..4fce0771a55d
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
@@ -0,0 +1,28 @@
+/* Verify that direct calls of every syntactic shape (plain, weak,
+ aliased, static) produce zero KCFI checks, and that the only KCFI
+ check generated comes from the lone indirect call. This guards
+ against regressions where the direct-call detector would miss
+ non-SYMBOL_REF address forms (CONST, LABEL_REF) and wrongly wrap
+ a direct call. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+extern void plain_func(int);
+extern void weak_func(int) __attribute__((weak));
+void plain_func(int x) { __asm__("" : : "r"(x) : "memory"); }
+void weak_func(int x) { __asm__("" : : "r"(x) : "memory"); }
+extern void alias_func(int) __attribute__((alias("plain_func")));
+__attribute__((noinline)) static void static_func(int x) {
+ __asm__("" : : "r"(x) : "memory");
+}
+
+void caller(int sel, void (*ip)(int), int x) {
+ if (sel == 1) plain_func(x); /* direct, plain extern */
+ if (sel == 2) weak_func(x); /* direct, weak */
+ if (sel == 3) alias_func(x); /* direct, alias */
+ if (sel == 4) static_func(x); /* direct, static */
+ if (sel == 5) ip(x); /* indirect — the only KCFI-checked call */
+}
+
+/* Per-arch dg-final patterns are added by each arch's KCFI implementation
+ commit; the invariant is "exactly one typeid load = one indirect call". */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
new file mode 100644
index 000000000000..a43bcd4f3e3f
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-ipa-robustness.c
@@ -0,0 +1,54 @@
+/* Test KCFI IPA pass robustness with compiler-generated constructs. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+#include <stddef.h>
+
+/* Test various compiler-generated constructs that could confuse IPA pass. */
+
+/* static_assert - this was causing the original crash. */
+typedef struct {
+ int field1;
+ char field2;
+} test_struct_t;
+
+static_assert(offsetof(test_struct_t, field1) == 0, "layout check 1");
+static_assert(offsetof(test_struct_t, field2) == 4, "layout check 2");
+static_assert(sizeof(test_struct_t) >= 5, "size check");
+
+/* Regular functions that should get KCFI analysis. */
+void regular_function(void) {
+ /* Should get KCFI preamble. */
+}
+
+static void static_function(void) {
+ /* With -O2: correctly identified as not address-taken, no preamble. */
+}
+
+void address_taken_function(void) {
+ /* Should get KCFI preamble (address taken below) */
+}
+
+/* Function pointer to create address-taken scenario. */
+void (*func_ptr)(void) = address_taken_function;
+
+/* More static_asserts mixed with function definitions. */
+static_assert(sizeof(void*) >= 4, "pointer size check");
+
+int main(void) {
+ regular_function(); /* Direct call. */
+ static_function(); /* Direct call to static. */
+ func_ptr(); /* Indirect call. */
+
+ static_assert(sizeof(int) == 4, "int size check");
+
+ return 0;
+}
+
+/* Verify KCFI preambles are generated appropriately. */
+/* { dg-final { scan-assembler "__cfi_regular_function:" } } */
+/* { dg-final { scan-assembler "__cfi_address_taken_function:" } } */
+/* { dg-final { scan-assembler "__cfi_main:" } } */
+
+/* With -O2: static_function correctly identified as not address-taken. */
+/* { dg-final { scan-assembler-not "__cfi_static_function:" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-lto-offset.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-lto-offset.c
new file mode 100644
index 000000000000..029da383aca8
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-lto-offset.c
@@ -0,0 +1,27 @@
+/* Verify KCFI is correct under LTO. The callsite typeid-load offset (and the
+ preamble alignment padding) must be computed in whichever process emits the
+ code. Regression test for a bug where these were set in an IPA pass that
+ runs at WPA, so the globals were still 0 in the separate ltrans codegen
+ process: every indirect call then loaded the typeid from the wrong offset
+ and trapped at run time. */
+
+/* { dg-do run { target native } } */
+/* { dg-require-effective-target lto } */
+/* { dg-additional-options "-O2 -flto" } */
+
+typedef int (*fp_t) (int);
+
+/* volatile keeps the call indirect (and KCFI-checked) through LTO. */
+volatile fp_t sink;
+
+int target (int x) { return x + 1; }
+
+int do_call (int x) { return sink (x); }
+
+int
+main (void)
+{
+ sink = target;
+ /* With a wrong typeid offset this traps (SIGILL) instead of returning 42. */
+ return do_call (41) == 42 ? 0 : 1;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
new file mode 100644
index 000000000000..7d58fef3f920
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
@@ -0,0 +1,41 @@
+/* Test that KCFI preserves function pointer moves at -O2 optimization.
+ This test ensures that the combine pass doesn't incorrectly optimize away
+ the move instruction needed to transfer function pointers from argument
+ registers to the target registers used by KCFI patterns. */
+
+/* { dg-do compile } */
+/* { dg-additional-options "-O2 -std=gnu11" } */
+
+static int called_count = 0;
+
+/* Function taking one argument, returning void. */
+static __attribute__((noinline)) void increment_void(int *counter)
+{
+ (*counter)++;
+}
+
+/* Function taking one argument, returning int. */
+static __attribute__((noinline)) int increment_int(int *counter)
+{
+ (*counter)++;
+ return *counter;
+}
+
+/* Don't allow the compiler to inline the calls. */
+static __attribute__((noinline)) void indirect_call(void (*func)(int *))
+{
+ func(&called_count);
+}
+
+int main(void)
+{
+ /* This should work - matching prototype. */
+ indirect_call(increment_void);
+
+ /* This should trap - mismatched prototype. */
+ indirect_call((void *)increment_int);
+
+ return 0;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
new file mode 100644
index 000000000000..4a90390d1934
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -0,0 +1,74 @@
+/* Test that no_sanitize("kcfi") attribute is preserved during inlining. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+extern void external_side_effect(int value);
+
+/* Regular function (should get KCFI checks) */
+__attribute__((noinline))
+void normal_function(void (*callback)(int))
+{
+ /* This indirect call must generate KCFI checks. */
+ callback(300);
+ external_side_effect(300);
+}
+
+/* Regular function marked with no_sanitize("kcfi") (positive control) */
+__attribute__((noinline, no_sanitize("kcfi")))
+void sensitive_non_inline_function(void (*callback)(int))
+{
+ /* This indirect call should NOT generate KCFI checks. */
+ callback(100);
+ external_side_effect(100);
+}
+
+/* Function marked with both no_sanitize("kcfi") and always_inline. */
+__attribute__((always_inline, no_sanitize("kcfi")))
+static inline void sensitive_inline_function(void (*callback)(int))
+{
+ /* This indirect call should NOT generate KCFI checks when inlined. */
+ callback(42);
+ external_side_effect(42);
+}
+
+/* Explicit wrapper for testing sensitive_inline_function behavior. */
+__attribute__((noinline))
+void wrap_sensitive_inline(void (*callback)(int))
+{
+ sensitive_inline_function(callback);
+}
+
+/* Function marked with only always_inline (should get KCFI checks) */
+__attribute__((always_inline))
+static inline void normal_inline_function(void (*callback)(int))
+{
+ /* This indirect call must generate KCFI checks when inlined. */
+ callback(200);
+ external_side_effect(200);
+}
+
+/* Explicit wrapper for testing normal_inline_function behavior. */
+__attribute__((noinline))
+void wrap_normal_inline(void (*callback)(int))
+{
+ normal_inline_function(callback);
+}
+
+void test_callback(int value)
+{
+ external_side_effect(value);
+}
+
+static void (*volatile function_pointer)(int) = test_callback;
+
+int main(void)
+{
+ void (*fn_ptr)(int) = function_pointer;
+
+ normal_function(fn_ptr);
+ wrap_normal_inline(fn_ptr);
+ sensitive_non_inline_function(fn_ptr);
+ wrap_sensitive_inline(fn_ptr);
+
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
new file mode 100644
index 000000000000..124d26488635
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -0,0 +1,31 @@
+/* Test KCFI with no_sanitize attribute. */
+/* { dg-do compile } */
+
+void target_function(void) {
+ /* This should get KCFI preamble. */
+}
+
+void caller_with_checks(void) {
+ /* This function should generate KCFI checks. */
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+}
+
+__attribute__((no_sanitize("kcfi")))
+void caller_no_checks(void) {
+ /* This function should NOT generate KCFI checks due to no_sanitize. */
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+}
+
+int main() {
+ caller_with_checks(); /* This should generate checks inside. */
+ caller_no_checks(); /* This should NOT generate checks inside. */
+ return 0;
+}
+
+/* All functions should get preambles regardless of no_sanitize. */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+/* { dg-final { scan-assembler "__cfi_caller_with_checks:" } } */
+/* { dg-final { scan-assembler "__cfi_caller_no_checks:" } } */
+/* { dg-final { scan-assembler "__cfi_main:" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
new file mode 100644
index 000000000000..213a1a2892a5
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -0,0 +1,24 @@
+/* Test KCFI call-site offset validation across architectures. */
+/* { dg-do compile } */
+
+void target_func_a(void) { }
+void target_func_b(int x) { }
+void target_func_c(int x, int y) { }
+
+int main() {
+ void (*ptr_a)(void) = target_func_a;
+ void (*ptr_b)(int) = target_func_b;
+ void (*ptr_c)(int, int) = target_func_c;
+
+ /* Multiple indirect calls. */
+ ptr_a();
+ ptr_b(1);
+ ptr_c(1, 2);
+
+ return 0;
+}
+
+/* Should have KCFI preambles for all functions. */
+/* { dg-final { scan-assembler "__cfi_target_func_a:" } } */
+/* { dg-final { scan-assembler "__cfi_target_func_b:" } } */
+/* { dg-final { scan-assembler "__cfi_target_func_c:" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
new file mode 100644
index 000000000000..a6a2f4816fef
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -0,0 +1,14 @@
+/* Test KCFI with patchable function entries - entry NOPs only. */
+/* { dg-do compile } */
+/* { dg-additional-options "-fpatchable-function-entry=4,0" } */
+
+void test_function(void) {
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr();
+ return 0;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-incompatible.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-incompatible.c
new file mode 100644
index 000000000000..c6cf9ab720a3
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-incompatible.c
@@ -0,0 +1,7 @@
+/* The patchable_function_entry attribute is incompatible with KCFI. */
+/* { dg-do compile } */
+
+__attribute__((patchable_function_entry(4, 2)))
+int test_function(void) { /* { dg-error "'patchable_function_entry' attribute cannot be used with '-fsanitize=kcfi'" } */
+ return 42;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
new file mode 100644
index 000000000000..8c4ec30cecc5
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -0,0 +1,14 @@
+/* Test KCFI with large patchable function entries. */
+/* { dg-do compile } */
+/* { dg-additional-options "-fpatchable-function-entry=11,11" } */
+
+void test_function(void) {
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr();
+ return 0;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
new file mode 100644
index 000000000000..78a834ef2a97
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -0,0 +1,14 @@
+/* Test KCFI with medium patchable function entries. */
+/* { dg-do compile } */
+/* { dg-additional-options "-fpatchable-function-entry=8,4" } */
+
+void test_function(void) {
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr();
+ return 0;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
new file mode 100644
index 000000000000..1a4d8269ed56
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -0,0 +1,14 @@
+/* Test KCFI with patchable function entries - prefix NOPs only. */
+/* { dg-do compile } */
+/* { dg-additional-options "-fpatchable-function-entry=3,3" } */
+
+void test_function(void) {
+}
+
+int main() {
+ void (*func_ptr)(void) = test_function;
+ func_ptr();
+ return 0;
+}
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-qualified-params.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-qualified-params.c
new file mode 100644
index 000000000000..6285489d6eb6
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-qualified-params.c
@@ -0,0 +1,58 @@
+/* Test that top-level qualifiers on parameters are not part of the KCFI
+ type-id, as they are not part of the function type: calling through a
+ compatible pointer type whose parameters lack them must not trap. */
+/* { dg-do run { target native } } */
+/* { dg-additional-options "-O2" } */
+
+struct s { int v; };
+typedef struct { int v; } anon_t;
+
+int f_const (int *const p) { return *p; }
+int f_restrict (int *restrict p) { return *p; }
+int f_volatile (int *volatile p) { return *p; }
+int f_array (int p[const]) { return p[0]; }
+int f_const_pointee (const int *const p) { return *p; }
+int f_struct (struct s *const p) { return p->v; }
+int f_anon (const anon_t a) { return a.v; }
+int f_anon_ptr (anon_t *const p) { return p->v; }
+
+/* Volatile keeps each call indirect, so it is KCFI-checked. */
+int (*volatile ip) (int *);
+int (*volatile cip) (const int *);
+int (*volatile sp) (struct s *);
+int (*volatile ap) (anon_t);
+int (*volatile app) (anon_t *);
+
+int
+main (void)
+{
+ int v = 42;
+ struct s s = { 42 };
+ anon_t a = { 42 };
+
+ ip = f_const;
+ if (ip (&v) != 42)
+ __builtin_abort ();
+ ip = f_restrict;
+ if (ip (&v) != 42)
+ __builtin_abort ();
+ ip = f_volatile;
+ if (ip (&v) != 42)
+ __builtin_abort ();
+ ip = f_array;
+ if (ip (&v) != 42)
+ __builtin_abort ();
+ cip = f_const_pointee;
+ if (cip (&v) != 42)
+ __builtin_abort ();
+ sp = f_struct;
+ if (sp (&s) != 42)
+ __builtin_abort ();
+ ap = f_anon;
+ if (ap (a) != 42)
+ __builtin_abort ();
+ app = f_anon_ptr;
+ if (app (&a) != 42)
+ __builtin_abort ();
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-runtime.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-runtime.c
new file mode 100644
index 000000000000..2b5c13fac69c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-runtime.c
@@ -0,0 +1,276 @@
+/* Test KCFI runtime behavior: working calls and type mismatch trapping. */
+/* { dg-do run { target native } } */
+/* { dg-additional-options "-O2" } */
+
+#include <stdio.h>
+#include <signal.h>
+#include <setjmp.h>
+#include <stdlib.h>
+#include <string.h>
+
+/* Test functions with different signatures */
+static int func_int_void(void)
+{
+ return 42;
+}
+
+__attribute__((nocf_check))
+static int func_int_void_nocf_check(void)
+{
+ return 42;
+}
+
+static int func_int_int(int x)
+{
+ return x * 4;
+}
+
+/* Complex functions with many arguments to create register pressure */
+static long complex_calc_8args(int a, long b, int c, long d,
+ int e, long f, int g, long h)
+{
+ /* Do actual work with all arguments to keep them live */
+ /* Each calculation uses specific operations to ensure values are preserved */
+ long result = (a * 2) + (b / 3) - (c * 4) + (d / 5) +
+ (e * 6) - (f / 7) + (g * 8) - (h / 9);
+
+ return result;
+}
+
+static int complex_intermediate_6args(int x1, int x2, int x3,
+ int x4, int x5, int x6)
+{
+ /* Keep variables live by using them in multiple calculations */
+ int temp1 = x1 + x2;
+ int temp2 = x3 * x4;
+ int temp3 = x5 - x6;
+
+ /* Call another function with many args through pointer */
+ typedef long (*calc_ptr)(int, long, int, long, int, long, int, long);
+ volatile calc_ptr ptr = complex_calc_8args;
+
+ /* Pass derived values to force register preservation */
+ long result = ptr(temp1, temp2, temp3, x1 * x2,
+ x3 + x4, x5 * x6, temp1 + temp2, temp3 - x1);
+
+ /* Use original args again to ensure they stayed live */
+ return (int)(result + x1 - x2 + x3 - x4 + x5 - x6);
+}
+
+static int complex_outer_4args(int a, int b, int c, int d)
+{
+ /* Create local variables that must be preserved */
+ int local1 = a * a;
+ int local2 = b * b;
+ int local3 = c * c;
+ int local4 = d * d;
+ volatile int force_spill1 = local1 + local2;
+ volatile int force_spill2 = local3 + local4;
+
+ /* Call through function pointer with many args */
+ typedef int (*inter_ptr)(int, int, int, int, int, int);
+ volatile inter_ptr ptr = complex_intermediate_6args;
+
+ /* Pass combinations that require preserving originals */
+ int result = ptr(local1, local2, local3, local4,
+ a + b, c + d);
+
+ /* Force use of spilled values */
+ result += force_spill1 + force_spill2;
+
+ /* Use original arguments again */
+ return result + a - b + c - d;
+}
+
+/* Entry point for complex call chain */
+static int complex_entry_point(void)
+{
+ /* Start with many live values */
+ int v1 = 10, v2 = 20, v3 = 30, v4 = 40;
+ int v5 = 50, v6 = 60, v7 = 70, v8 = 80;
+
+ /* Keep them live with volatile stores */
+ volatile int keep_live1 = v1 + v5;
+ volatile int keep_live2 = v2 + v6;
+ volatile int keep_live3 = v3 + v7;
+ volatile int keep_live4 = v4 + v8;
+
+ /* Call through function pointer */
+ typedef int (*outer_ptr)(int, int, int, int);
+ volatile outer_ptr ptr = complex_outer_4args;
+
+ /* Use derived values to maintain register pressure */
+ int result = ptr(v1 * v2, v3 * v4, v5 * v6, v7 * v8);
+
+ /* Force original values to stay live */
+ result += keep_live1 + keep_live2 + keep_live3 + keep_live4;
+ result += v1 + v2 + v3 + v4 + v5 + v6 + v7 + v8;
+
+ /* Return the calculated result */
+ return result;
+}
+
+/* Global state for signal handling */
+static volatile int trap_occurred = 0;
+static jmp_buf trap_env;
+
+/* Signal handler for KCFI traps */
+static void trap_handler(int sig)
+{
+ trap_occurred = 1;
+ longjmp(trap_env, 1);
+}
+
+/* Compatible indirect call should work - simple version */
+static int test_compatible_call(void)
+{
+ typedef int (*int_void_ptr)(void);
+ volatile int_void_ptr ptr = func_int_void;
+
+ fprintf(stderr, "Calling %s(0x%08x) through %s(0x%08x) ...\n",
+ __builtin_linux_abi_kcfi_name(typeof(func_int_void)),
+ __builtin_linux_abi_kcfi_hash(typeof(func_int_void)),
+ __builtin_linux_abi_kcfi_name(typeof(*ptr)),
+ __builtin_linux_abi_kcfi_hash(typeof(*ptr)));
+
+ trap_occurred = 0;
+ /* This should work - same signature */
+ int result = ptr();
+
+ return (trap_occurred == 0 && result == 42) ? 1 : 0;
+}
+
+/* Compatible indirect call with complex register pressure */
+static int test_compatible_call_complex(void)
+{
+ typedef int (*int_void_ptr)(void);
+ volatile int_void_ptr ptr = complex_entry_point;
+
+ fprintf(stderr, "Calling complex chain %s(0x%08x) through %s(0x%08x) ...\n",
+ __builtin_linux_abi_kcfi_name(typeof(complex_entry_point)),
+ __builtin_linux_abi_kcfi_hash(typeof(complex_entry_point)),
+ __builtin_linux_abi_kcfi_name(typeof(*ptr)),
+ __builtin_linux_abi_kcfi_hash(typeof(*ptr)));
+
+ trap_occurred = 0;
+
+ /* This should work - complex call chain with register pressure */
+ int result = ptr();
+
+ /* Verify the complete call chain computed correctly with all register
+ values preserved through the high-pressure call sequence */
+ if (result != 657383831) {
+ fprintf(stderr, "ERROR: Incorrect final result %d (expected 657383831)\n", result);
+ return 0;
+ }
+
+ /* Check that no trap occurred and result is correct */
+ return (trap_occurred == 0) ? 1 : 0;
+}
+
+/* Compatible indirect call to nocf_check should not work */
+static int test_nocf_check_trap(void)
+{
+ trap_occurred = 0;
+
+ if (setjmp(trap_env) == 0) {
+ typedef int (__attribute__((nocf_check)) *int_void_ptr_nocf)(void);
+ volatile int_void_ptr_nocf ptr = func_int_void_nocf_check;
+
+ fprintf(stderr, "Calling %s(0x%08x) through %s(0x%08x) ...\n",
+ __builtin_linux_abi_kcfi_name(typeof(func_int_void_nocf_check)),
+ __builtin_linux_abi_kcfi_hash(typeof(func_int_void_nocf_check)),
+ __builtin_linux_abi_kcfi_name(typeof(*ptr)),
+ __builtin_linux_abi_kcfi_hash(typeof(*ptr)));
+
+ int result = ptr();
+
+ fprintf(stderr, "Yikes! Survived nocf_check call\n");
+
+ /* If we get here, the trap didn't occur */
+ return 0;
+ } else {
+ /* We caught the trap - this is expected */
+ return trap_occurred;
+ }
+}
+
+/* Type mismatch should trap */
+static int test_type_mismatch_trap(void)
+{
+ trap_occurred = 0;
+
+ if (setjmp(trap_env) == 0) {
+ /* Cast func_int_void to incompatible void(*)(void) type */
+ typedef void (*void_void_ptr)(void);
+ volatile void_void_ptr ptr = (void_void_ptr)func_int_void;
+
+ fprintf(stderr, "Calling %s(0x%08x) through %s(0x%08x) ...\n",
+ __builtin_linux_abi_kcfi_name(typeof(func_int_void)),
+ __builtin_linux_abi_kcfi_hash(typeof(func_int_void)),
+ __builtin_linux_abi_kcfi_name(typeof(*ptr)),
+ __builtin_linux_abi_kcfi_hash(typeof(*ptr)));
+
+ /* This should trap because type IDs don't match:
+ - func_int_void has type ID for int(void)
+ - but we're calling through void(void) pointer type */
+ ptr();
+
+ fprintf(stderr, "Yikes! Survived mismatched call\n");
+
+ /* If we get here, the trap didn't occur */
+ return 0;
+ } else {
+ /* We caught the trap - this is expected */
+ return trap_occurred;
+ }
+}
+
+int main(void)
+{
+ struct sigaction sa = {
+ .sa_handler = trap_handler,
+ .sa_flags = SA_NODEFER,
+ };
+ int failed = 4;
+
+ /* Install trap handler. */
+ if (sigaction(SIGILL, &sa, NULL)) {
+ perror("sigaction");
+ return 1;
+ }
+
+ /* Simple compatible call should work */
+ if (test_compatible_call()) {
+ printf("OK: simple matched indirect call succeeded\n");
+ failed--;
+ } else {
+ printf("FAIL: simple matched call\n");
+ }
+
+ /* Complex compatible call chain should work */
+ if (test_compatible_call_complex()) {
+ printf("OK: complex matched indirect call chain succeeded\n");
+ failed--;
+ } else {
+ printf("FAIL: complex matched call chain\n");
+ }
+
+ /* Using nocf_check should trap */
+ if (test_nocf_check_trap()) {
+ printf("OK: indirect call to nocf_check correctly trapped\n");
+ failed--;
+ } else {
+ printf("FAIL: nocf_check trap\n");
+ }
+
+ /* Type mismatch should trap */
+ if (test_type_mismatch_trap()) {
+ printf("OK: mismatched indirect call correctly trapped\n");
+ failed--;
+ } else {
+ printf("FAIL: type mismatch trap\n");
+ }
+
+ return failed;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
new file mode 100644
index 000000000000..9ddf178aa2b1
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -0,0 +1,60 @@
+/* Test KCFI protection when indirect calls get converted to tail calls. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+typedef int (*func_ptr_t)(int);
+typedef void (*void_func_ptr_t)(void);
+
+struct function_table {
+ func_ptr_t process;
+ void_func_ptr_t cleanup;
+};
+
+/* Target functions. */
+int process_data(int x) { return x * 2; }
+void cleanup_data(void) {}
+
+/* Initialize function table. */
+volatile struct function_table vtable = {
+ .process = &process_data,
+ .cleanup = &cleanup_data
+};
+
+/* Indirect call through struct member that should become tail call. */
+int test_struct_indirect_call(int x) {
+ /* This is an indirect call that should be converted to tail call:
+ Without -fno-optimize-sibling-calls should become "jmp *vtable+0(%rip)"
+ With -fno-optimize-sibling-calls should become "call *vtable+0(%rip)" */
+ return vtable.process(x);
+}
+
+/* Indirect call through function pointer parameter. */
+int test_param_indirect_call(func_ptr_t handler, int x) {
+ /* This is an indirect call that should be converted to tail call:
+ Without -fno-optimize-sibling-calls should become "jmp *%rdi"
+ With -fno-optimize-sibling-calls should be "call *%rdi" */
+ return handler(x);
+}
+
+/* Void indirect call through struct member. */
+void test_void_indirect_call(void) {
+ /* This is an indirect call that should be converted to tail call:
+ * Without -fno-optimize-sibling-calls: should become "jmp *vtable+8(%rip)"
+ * With -fno-optimize-sibling-calls: should be "call *vtable+8(%rip)" */
+ vtable.cleanup();
+}
+
+/* Non-tail call for comparison (should always be call). */
+int test_non_tail_indirect_call(func_ptr_t handler, int x) {
+ /* This should never become a tail call - always "call *%rdi" */
+ int result = handler(x);
+ return result + 1; /* Prevents tail call optimization. */
+}
+
+/* Should have KCFI preambles for all functions. */
+/* { dg-final { scan-assembler-times "__cfi_process_data:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_cleanup_data:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_struct_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_param_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_void_indirect_call:" 1 } } */
+/* { dg-final { scan-assembler-times "__cfi_test_non_tail_indirect_call:" 1 } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge-run.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge-run.c
new file mode 100644
index 000000000000..d1b975c38216
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge-run.c
@@ -0,0 +1,54 @@
+/* Test that tail merging does not merge indirect calls that get different
+ KCFI checks: each call through a union member must be checked against its
+ own type, so correctly typed targets must not trap. See
+ kcfi-tail-merge.c. */
+/* { dg-do run { target native } } */
+/* { dg-additional-options "-O2" } */
+
+union conv
+{
+ int (*int_conv) (int *);
+ int (*uint_conv) (unsigned int *);
+ int (*ulong_conv) (unsigned long *);
+};
+
+enum which { INT, UINT, ULONG };
+
+__attribute__ ((noipa)) int
+dispatch (enum which type, union conv conv, char *k)
+{
+ switch (type)
+ {
+ case INT:
+ return conv.int_conv ((int *) k);
+ case UINT:
+ return conv.uint_conv ((unsigned int *) k);
+ case ULONG:
+ return conv.ulong_conv ((unsigned long *) k);
+ }
+ return -1;
+}
+
+int int_fn (int *k) { return *k + 1; }
+int uint_fn (unsigned int *k) { return *k + 2; }
+int ulong_fn (unsigned long *k) { return *k + 3; }
+
+int
+main (void)
+{
+ int i = 40;
+ unsigned int u = 40;
+ unsigned long ul = 40;
+ union conv c;
+
+ c.int_conv = int_fn;
+ if (dispatch (INT, c, (char *) &i) != 41)
+ __builtin_abort ();
+ c.uint_conv = uint_fn;
+ if (dispatch (UINT, c, (char *) &u) != 42)
+ __builtin_abort ();
+ c.ulong_conv = ulong_fn;
+ if (dispatch (ULONG, c, (char *) &ul) != 43)
+ __builtin_abort ();
+ return 0;
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge.c
new file mode 100644
index 000000000000..5bc9307c34ff
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-merge.c
@@ -0,0 +1,36 @@
+/* Test that tail merging does not merge indirect calls that get different
+ KCFI checks. Calls through the members of a union share one pointer
+ value but have different function types, so each must keep its own
+ check. Reduced from proc_vec_conv() in the Linux kernel. */
+/* { dg-do compile } */
+/* { dg-additional-options "-O2" } */
+
+union conv
+{
+ int (*int_conv) (int *);
+ int (*uint_conv) (unsigned int *);
+ int (*ulong_conv) (unsigned long *);
+};
+
+enum which { INT, UINT, ULONG };
+
+int
+dispatch (enum which type, union conv conv, char *k)
+{
+ switch (type)
+ {
+ case INT:
+ return conv.int_conv ((int *) k);
+ case UINT:
+ return conv.uint_conv ((unsigned int *) k);
+ case ULONG:
+ return conv.ulong_conv ((unsigned long *) k);
+ }
+ return -1;
+}
+
+/* Three call-site checks; merging the calls would leave one. */
+/* x86: { dg-final { scan-assembler-times {movl\s+\$-?[0-9]+,\s+%r10d} 3 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {mov\s+w17, #[0-9]+} 3 { target aarch64*-*-* } } } */
+/* ARM 32-bit: { dg-final { scan-assembler-times {ldr\s+ip, \[(?:r[0-9]+|lr|ip), #-4\]} 3 { target arm*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {lui\tt2, [0-9]+} 3 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
new file mode 100644
index 000000000000..6d34ad6e1a0c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -0,0 +1,17 @@
+/* Test KCFI trap section generation. */
+/* { dg-do compile } */
+
+void target_function(void) {}
+
+int main() {
+ void (*func_ptr)(void) = target_function;
+
+ /* Multiple indirect calls to generate multiple trap entries. */
+ func_ptr();
+ func_ptr();
+
+ return 0;
+}
+
+/* Should have KCFI preamble. */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
diff --git a/gcc/testsuite/gcc.dg/pch/kcfi-1.c b/gcc/testsuite/gcc.dg/pch/kcfi-1.c
new file mode 100644
index 000000000000..8035396c7d85
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/pch/kcfi-1.c
@@ -0,0 +1,50 @@
+/* Test that KCFI is precompiled-header safe: compiling this TU with the header
+ pulled from a PCH must produce assembly identical to compiling it with the
+ header parsed normally. KCFI type-ids are computed at codegen from the
+ FUNCTION_TYPE trees, so a PCH-restored type must hash the same as a
+ freshly-parsed one; the __cfi_ preamble type-ids and the indirect-call check
+ immediates below would differ otherwise. It also guards the output-phase
+ label counter (kcfi_labelno): were it wrongly GTY-saved into the PCH, the
+ .Lkcfi_call labels would diverge between the two compilations. */
+
+/* { dg-options "-fsanitize=kcfi -I." } */
+/* { dg-require-effective-target kcfi } */
+
+#include "kcfi-1.h"
+
+/* Address-taken targets: each gets a __cfi_ preamble carrying its type-id. */
+int
+add (int a, int b)
+{
+ return a + b;
+}
+
+long
+deref (long *p)
+{
+ return *p;
+}
+
+void
+take (struct s *p)
+{
+ (void) p;
+}
+
+/* volatile keeps the calls indirect, so they are KCFI-checked. */
+volatile fp_ii p_add;
+volatile fp_lp p_deref;
+volatile fp_sp p_take;
+
+int
+caller (struct s *sp)
+{
+ long v = 41;
+
+ p_add = add;
+ p_deref = deref;
+ p_take = take;
+
+ p_take (sp);
+ return p_add (1, 2) + (int) p_deref (&v);
+}
diff --git a/gcc/testsuite/gcc.dg/pch/kcfi-1.hs b/gcc/testsuite/gcc.dg/pch/kcfi-1.hs
new file mode 100644
index 000000000000..5a32e2bc1dc9
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/pch/kcfi-1.hs
@@ -0,0 +1,20 @@
+/* Header precompiled for kcfi-1.c. It supplies the function types and
+ address-taken declarations whose KCFI type-ids kcfi-1.c relies on; the point
+ of the test is that those type-ids come out identical whether the
+ FUNCTION_TYPE trees are parsed fresh or restored from the PCH.
+
+ The header is precompiled with -fsanitize=kcfi so the PCH matches the way
+ kcfi-1.c is compiled; dg-require-effective-target keeps this UNSUPPORTED
+ (not failing) on targets without KCFI. */
+
+/* { dg-options "-fsanitize=kcfi" } */
+/* { dg-require-effective-target kcfi } */
+
+typedef int (*fp_ii) (int, int);
+typedef long (*fp_lp) (long *);
+struct s;
+typedef void (*fp_sp) (struct s *);
+
+extern int add (int, int);
+extern long deref (long *);
+extern void take (struct s *);
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v17 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (2 preceding siblings ...)
2026-10-05 15:40 ` [PATCH v17 3/7] kcfi: Add regression test suite Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-05 15:40 ` [PATCH v17 5/7] aarch64: Add AArch64 " Kees Cook
` (2 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Implement x86_64-specific KCFI backend:
- Implies -mindirect-branch-register since KCFI needs call target in
a register for typeid hash loading.
- Function preamble generation with type IDs positioned at -(4+prefix_nops)
offset from function entry point.
- Function-aligned KCFI preambles using calculated alignment padding NOPs:
aligned(prefix_nops + 5, $func_align) to maintain ability to call the
__cfi_ preamble directly in the case of Linux's FineIBT alternative
CFI sequences (live patched into place).
- Type-id hash avoids generating ENDBR instruction in type IDs
(0xfa1e0ff3/0xfb1e0ff3 are incremented by 1 to prevent execution).
- On-demand scratch register allocation strategy (r11 as needed), with
the clobbers added when KCFI is used.
- Incompatible with -ffixed-r10 or -ffixed-r11.
- Uses the .kcfi_traps section for debugger/runtime metadata.
- Introduces -fsanitize-kcfi-arity to enable function arg count to be
represented in the kcfi hash preamble for FineIBT.
Assembly Code Pattern layout required by Linux kernel:
movl $inverse_type_id, %r10d ; Load expected type (0 - hash)
addl offset(%target), %r10d ; Add stored type ID from preamble
je .Lkcfi_call ; Branch if types match (sum == 0)
.Lkcfi_trap: ud2 ; Undefined instruction trap on mismatch
.Lkcfi_call: call/jmp *%target ; Execute validated indirect transfer
Build and run tested on x86_64 Linux kernel with various CPU errata
handling alternatives, with and without FineIBT patching.
Assisted-by: LLM [tests]
gcc/ChangeLog:
* common.opt (fsanitize-kcfi-arity): New option.
* config/i386/i386.h: KCFI enables TARGET_INDIRECT_BRANCH_REGISTER.
* config/i386/i386-protos.h: Update ix86_output_call_insn.
* config/i386/i386-expand.cc (ix86_expand_call): Expand indirect
calls into KCFI RTL.
* config/i386/i386.cc (ix86_kcfi_mask_type_id): New function.
(ix86_output_call_insn): Emit KCFI assembly.
* config/i386/i386.md: Add KCFI RTL substitution patterns.
* doc/invoke.texi: Document x86 nuances.
* opts.cc (common_handle_option): Reset flag_sanitize_kcfi_arity.
gcc/testsuite/ChangeLog:
* gcc.dg/kcfi/kcfi-adjacency.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-basics.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-call-sharing.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-complex-addressing.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-direct-call-shapes.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-move-preservation.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-no-sanitize-inline.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-no-sanitize.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-offset-validation.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-patchable-entry-only.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-patchable-large.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-patchable-medium.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-patchable-prefix-only.c: Add x86 patterns.
* gcc.dg/kcfi/kcfi-tail-calls.c: Add x86 tail-call patterns.
* gcc.dg/kcfi/kcfi-trap-section.c: Add x86 trap patterns.
* gcc.dg/kcfi/kcfi-trap-section-per-func.c: New test.
* gcc.dg/kcfi/kcfi-x86-32bit.c: New test.
* gcc.dg/kcfi/kcfi-x86-arity.c: New test.
* gcc.dg/kcfi/kcfi-x86-endbr-collision.c: New test.
* gcc.dg/kcfi/kcfi-x86-fixed-r10.c: New test.
* gcc.dg/kcfi/kcfi-x86-fixed-r11.c: New test.
* gcc.dg/kcfi/kcfi-x86-retpoline-r11.c: New test.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/i386/i386-protos.h | 2 +-
gcc/config/i386/i386.h | 3 +-
gcc/config/i386/i386.md | 63 ++++--
gcc/config/i386/i386-expand.cc | 28 ++-
gcc/config/i386/i386.cc | 210 +++++++++++++++++-
gcc/doc/invoke.texi | 46 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 17 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 21 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 15 ++
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 18 ++
.../gcc.dg/kcfi/kcfi-direct-call-shapes.c | 1 +
.../gcc.dg/kcfi/kcfi-move-preservation.c | 19 ++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 11 +
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 5 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 5 +
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 24 ++
.../gcc.dg/kcfi/kcfi-patchable-large.c | 13 ++
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 20 ++
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 21 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 20 ++
.../gcc.dg/kcfi/kcfi-trap-section-per-func.c | 23 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 6 +
gcc/testsuite/gcc.dg/kcfi/kcfi-x86-32bit.c | 7 +
gcc/testsuite/gcc.dg/kcfi/kcfi-x86-arity.c | 93 ++++++++
.../gcc.dg/kcfi/kcfi-x86-endbr-collision.c | 55 +++++
.../gcc.dg/kcfi/kcfi-x86-fixed-r10.c | 7 +
.../gcc.dg/kcfi/kcfi-x86-fixed-r11.c | 7 +
.../gcc.dg/kcfi/kcfi-x86-retpoline-r11.c | 40 ++++
gcc/common.opt | 8 +
gcc/opts.cc | 1 +
30 files changed, 782 insertions(+), 27 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-32bit.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-arity.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-endbr-collision.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r10.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r11.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-x86-retpoline-r11.c
diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h
index 4e56bf70c8d5..dbf40eaa3c67 100644
--- a/gcc/config/i386/i386-protos.h
+++ b/gcc/config/i386/i386-protos.h
@@ -376,7 +376,7 @@ extern enum attr_cpu ix86_schedule;
#endif
extern bool ix86_nopic_noplt_attribute_p (rtx call_op);
-extern const char * ix86_output_call_insn (rtx_insn *insn, rtx call_op);
+extern const char * ix86_output_call_insn (rtx_insn *insn, rtx *operands);
extern const char * ix86_output_indirect_jmp (rtx call_op);
extern const char * ix86_output_function_return (bool long_p);
extern const char * ix86_output_indirect_function_return (rtx ret_op);
diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h
index 47725c560906..678aff983bcb 100644
--- a/gcc/config/i386/i386.h
+++ b/gcc/config/i386/i386.h
@@ -3108,7 +3108,8 @@ extern void debug_dispatch_window (int);
#define TARGET_INDIRECT_BRANCH_REGISTER \
(ix86_indirect_branch_register \
- || cfun->machine->indirect_branch_type != indirect_branch_keep)
+ || cfun->machine->indirect_branch_type != indirect_branch_keep \
+ || (flag_sanitize & SANITIZE_KCFI))
#define IX86_HLE_ACQUIRE (1 << 16)
#define IX86_HLE_RELEASE (1 << 17)
diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
index 0d4749e6e091..4b249d234c76 100644
--- a/gcc/config/i386/i386.md
+++ b/gcc/config/i386/i386.md
@@ -21165,11 +21165,19 @@
DONE;
})
-(define_insn "*call"
+(define_subst_attr "kcfi" "kcfi_subst" "" "_kcfi")
+
+(define_subst "kcfi_subst"
+ [(match_operand 0)]
+ ""
+ [(kcfi (match_dup 0)
+ (match_operand 2 "const_int_operand"))])
+
+(define_insn "*call<kcfi>"
[(call (mem:QI (match_operand:W 0 "call_insn_operand" "<c>BwBz"))
(match_operand 1))]
"!SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
;; This covers both call and sibcall since only GOT slot is allowed.
@@ -21180,7 +21188,8 @@
"TARGET_X32 && !TARGET_INDIRECT_BRANCH_REGISTER"
{
rtx fnaddr = gen_const_mem (DImode, XEXP (operands[0], 0));
- return ix86_output_call_insn (insn, fnaddr);
+ rtx fn_operands[] = { fnaddr };
+ return ix86_output_call_insn (insn, fn_operands);
}
[(set_attr "type" "call")])
@@ -21198,15 +21207,17 @@
{
rtx fnaddr = gen_rtx_PLUS (SImode, operands[0], operands[1]);
fnaddr = gen_const_mem (SImode, fnaddr);
- return ix86_output_call_insn (insn, fnaddr);
+ rtx fn_operands[] = { fnaddr };
+ return ix86_output_call_insn (insn, fn_operands);
}
[(set_attr "type" "call")])
-(define_insn "*sibcall"
+;; KCFI sibling call
+(define_insn "*sibcall<kcfi>"
[(call (mem:QI (match_operand:W 0 "sibcall_insn_operand" "UBsBz"))
(match_operand 1))]
"SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
(define_insn "*sibcall_memory"
@@ -21214,7 +21225,7 @@
(match_operand 1))
(unspec [(const_int 0)] UNSPEC_PEEPSIB)]
"!TARGET_X32 && !TARGET_INDIRECT_BRANCH_REGISTER"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
(define_peephole2
@@ -21265,7 +21276,7 @@
(plus:SI (reg:SI SP_REG)
(match_operand:SI 2 "immediate_operand" "i")))]
"!TARGET_64BIT && !SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
(define_insn "*sibcall_pop"
@@ -21275,7 +21286,7 @@
(plus:SI (reg:SI SP_REG)
(match_operand:SI 2 "immediate_operand" "i")))]
"!TARGET_64BIT && SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
(define_insn "*sibcall_pop_memory"
@@ -21286,7 +21297,7 @@
(match_operand:SI 2 "immediate_operand" "i")))
(unspec [(const_int 0)] UNSPEC_PEEPSIB)]
"!TARGET_64BIT && !TARGET_INDIRECT_BRANCH_REGISTER"
- "* return ix86_output_call_insn (insn, operands[0]);"
+ "* return ix86_output_call_insn (insn, operands);"
[(set_attr "type" "call")])
(define_peephole2
@@ -21363,12 +21374,22 @@
DONE;
})
-(define_insn "*call_value"
+(define_subst_attr "kcfiv" "kcfiv_subst" "" "_kcfi")
+
+(define_subst "kcfiv_subst"
+ [(set (match_operand 0)
+ (match_operand 1))]
+ ""
+ [(set (match_dup 0)
+ (kcfi (match_dup 1)
+ (match_operand 3 "const_int_operand")))])
+
+(define_insn "*call_value<kcfiv>"
[(set (match_operand 0)
(call (mem:QI (match_operand:W 1 "call_insn_operand" "<c>BwBz"))
(match_operand 2)))]
"!SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
;; This covers both call and sibcall since only GOT slot is allowed.
@@ -21381,7 +21402,8 @@
"TARGET_X32 && !TARGET_INDIRECT_BRANCH_REGISTER"
{
rtx fnaddr = gen_const_mem (DImode, XEXP (operands[1], 0));
- return ix86_output_call_insn (insn, fnaddr);
+ rtx fn_operands[] = { fnaddr };
+ return ix86_output_call_insn (insn, fn_operands);
}
[(set_attr "type" "callv")])
@@ -21400,16 +21422,17 @@
{
rtx fnaddr = gen_rtx_PLUS (SImode, operands[1], operands[2]);
fnaddr = gen_const_mem (SImode, fnaddr);
- return ix86_output_call_insn (insn, fnaddr);
+ rtx fn_operands[] = { fnaddr };
+ return ix86_output_call_insn (insn, fn_operands);
}
[(set_attr "type" "callv")])
-(define_insn "*sibcall_value"
+(define_insn "*sibcall_value<kcfiv>"
[(set (match_operand 0)
(call (mem:QI (match_operand:W 1 "sibcall_insn_operand" "UBsBz"))
(match_operand 2)))]
"SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
(define_insn "*sibcall_value_memory"
@@ -21418,7 +21441,7 @@
(match_operand 2)))
(unspec [(const_int 0)] UNSPEC_PEEPSIB)]
"!TARGET_X32 && !TARGET_INDIRECT_BRANCH_REGISTER"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
(define_peephole2
@@ -21475,7 +21498,7 @@
(plus:SI (reg:SI SP_REG)
(match_operand:SI 3 "immediate_operand" "i")))]
"!TARGET_64BIT && !SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
(define_insn "*sibcall_value_pop"
@@ -21486,7 +21509,7 @@
(plus:SI (reg:SI SP_REG)
(match_operand:SI 3 "immediate_operand" "i")))]
"!TARGET_64BIT && SIBLING_CALL_P (insn)"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
(define_insn "*sibcall_value_pop_memory"
@@ -21498,7 +21521,7 @@
(match_operand:SI 3 "immediate_operand" "i")))
(unspec [(const_int 0)] UNSPEC_PEEPSIB)]
"!TARGET_64BIT && !TARGET_INDIRECT_BRANCH_REGISTER"
- "* return ix86_output_call_insn (insn, operands[1]);"
+ "* return ix86_output_call_insn (insn, &operands[1]);"
[(set_attr "type" "callv")])
(define_peephole2
diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index 9e112d017345..a93f81f29518 100644
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -94,6 +94,7 @@ along with GCC; see the file COPYING3. If not see
#include "i386-builtins.h"
#include "i386-expand.h"
#include "asan.h"
+#include "kcfi.h"
#include "function-abi.h"
/* Split one or more double-mode RTL references into pairs of half-mode
@@ -11348,9 +11349,10 @@ ix86_expand_call (rtx retval, rtx fnaddr, rtx callarg1,
rtx vec[3];
rtx use = NULL, call;
unsigned int vec_len = 0;
+ bool is_direct_call = SYMBOL_REF_P (XEXP (fnaddr, 0));
tree fndecl;
- if (SYMBOL_REF_P (XEXP (fnaddr, 0)))
+ if (is_direct_call)
{
fndecl = SYMBOL_REF_DECL (XEXP (fnaddr, 0));
if (fndecl)
@@ -11374,7 +11376,7 @@ ix86_expand_call (rtx retval, rtx fnaddr, rtx callarg1,
if (TARGET_MACHO && !TARGET_64BIT)
{
#if TARGET_MACHO
- if (flag_pic && SYMBOL_REF_P (XEXP (fnaddr, 0)))
+ if (flag_pic && is_direct_call)
fnaddr = machopic_indirect_call_target (fnaddr);
#endif
}
@@ -11458,7 +11460,7 @@ ix86_expand_call (rtx retval, rtx fnaddr, rtx callarg1,
if (ix86_cmodel == CM_LARGE_PIC
&& !TARGET_PECOFF
&& MEM_P (fnaddr)
- && SYMBOL_REF_P (XEXP (fnaddr, 0))
+ && is_direct_call
&& !local_symbolic_operand (XEXP (fnaddr, 0), VOIDmode))
fnaddr = gen_rtx_MEM (QImode, construct_plt_address (XEXP (fnaddr, 0)));
/* Since x32 GOT slot is 64 bit with zero upper 32 bits, indirect
@@ -11490,6 +11492,26 @@ ix86_expand_call (rtx retval, rtx fnaddr, rtx callarg1,
call = gen_rtx_CALL (VOIDmode, fnaddr, callarg1);
+ /* Only indirect calls need KCFI instrumentation. After address
+ legitimization an indirect target is a REG; every symbolic form
+ (SYMBOL_REF, LABEL_REF, CONST) is a direct call. Use a strict
+ REG_P check here rather than the broader is_direct_call (which is
+ SYMBOL_REF_P and is shared with PIC/PLT logic) so that direct calls
+ that are not SYMBOL_REF are not wrongly wrapped. */
+ rtx kcfi_type_rtx = REG_P (XEXP (fnaddr, 0))
+ ? kcfi_get_type_id_for_expanding_gimple_call ()
+ : NULL_RTX;
+ if (kcfi_type_rtx)
+ {
+ /* Wrap call with KCFI. */
+ call = gen_rtx_KCFI (VOIDmode, call, kcfi_type_rtx);
+
+ /* Add KCFI clobbers for the insn sequence. */
+ clobber_reg (&use, gen_rtx_REG (DImode, R10_REG));
+ clobber_reg (&use, gen_rtx_REG (DImode, R11_REG));
+ clobber_reg (&use, gen_rtx_REG (CCmode, FLAGS_REG));
+ }
+
if (retval)
call = gen_rtx_SET (retval, call);
vec[vec_len++] = call;
diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index e9015d2427e7..2de84b7a3253 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -99,6 +99,7 @@ along with GCC; see the file COPYING3. If not see
#include "i386-builtins.h"
#include "i386-expand.h"
#include "i386-features.h"
+#include "kcfi.h"
#include "function-abi.h"
#include "rtl-error.h"
#include "gimple-pretty-print.h"
@@ -351,6 +352,26 @@ unsigned int const svr4_debugger_register_map[FIRST_PSEUDO_REGISTER] =
INVALID_REGNUM,
};
+/* KCFI arity (how many parameters a function has) is encoded in the
+ preamble's immediate register. The encoding is as follows:
+
+ Arity represented by
+ 0 by %eax (gcc regno = 0)
+ 1 by %ecx (gcc regno = 2)
+ 2 by %edx (gcc regno = 1)
+ 3 by %ebx (gcc regno = 3)
+ 4 by %esp (gcc regno = 7)
+ 5 by %ebp (gcc regno = 6)
+ 6 by %esi (gcc regno = 4)
+ 7 by %edi (gcc regno = 5)
+*/
+#define KCFI_ARITY_MAX_INDICATOR 7
+unsigned int const kcfi_arity_register_map[KCFI_ARITY_MAX_INDICATOR + 1] =
+{
+ /* eax to edi, as above. */
+ 0, 2, 1, 3, 7, 6, 4, 5
+};
+
/* Define parameter passing and return registers. */
static int const x86_64_int_parameter_registers[6] =
@@ -17760,11 +17781,89 @@ ix86_output_indirect_function_return (rtx ret_op)
return "";
}
-/* Output the assembly for a call instruction. */
+/* Output the assembly for a call instruction. INSN is the RTL instruction
+ being processed. OPERANDS is the array of RTL operands where operands[0]
+ is the call target, operands[1] is the call operand (unused on x86), and
+ operands[2] is the KCFI type ID constant (for KCFI-instrumented calls). */
const char *
-ix86_output_call_insn (rtx_insn *insn, rtx call_op)
+ix86_output_call_insn (rtx_insn *insn, rtx *operands)
{
+ rtx call_op = operands[0];
+ rtx pat = PATTERN (insn);
+
+ /* If KCFI, emit the check sequence first. */
+ if (GET_CODE (pat) == KCFI
+ || (GET_CODE (pat) == SET && GET_CODE (XEXP (pat, 1)) == KCFI))
+ {
+ /* Target is guaranteed to be in a register due to
+ TARGET_INDIRECT_BRANCH_REGISTER. */
+ gcc_assert (REG_P (call_op));
+
+ /* In thunk-extern mode, the register must be R11 for FineIBT
+ compatibility. */
+ if (cfun->machine->indirect_branch_type == indirect_branch_thunk_extern)
+ {
+ if (REGNO (call_op) != R11_REG)
+ {
+ /* Emit move from current target to R11. */
+ call_op = gen_rtx_REG (DImode, R11_REG);
+ rtx r11_operands[2] = { operands[0], call_op };
+ output_asm_insn ("movq\t%0, %1", r11_operands);
+ }
+ }
+
+ /* Get unique label number for this KCFI check. */
+ int labelno = kcfi_next_labelno ();
+
+ /* Generate custom label names. */
+ char trap_name[32];
+ char call_name[32];
+ ASM_GENERATE_INTERNAL_LABEL (trap_name, "Lkcfi_trap", labelno);
+ ASM_GENERATE_INTERNAL_LABEL (call_name, "Lkcfi_call", labelno);
+
+ /* Choose scratch register: r10 by default, r11 if r10 is the target. */
+ bool target_is_r10 = (REGNO (call_op) == R10_REG);
+ rtx scratch_reg = gen_rtx_REG (SImode, target_is_r10 ? R11_REG : R10_REG);
+
+ /* Get KCFI type ID from operand 2. */
+ uint32_t type_id = UINTVAL (operands[2]);
+
+ /* Convert to inverse for the check (0 - hash). */
+ uint32_t inverse_type_id = (uint32_t)(0 - type_id);
+
+ /* Calculate offset to typeid from target address. */
+ HOST_WIDE_INT offset = -kcfi_get_typeid_offset ();
+
+ /* Output complete KCFI check sequence. */
+ rtx inverse_type_id_rtx = gen_int_mode (inverse_type_id, SImode);
+ rtx mov_operands[2] = { inverse_type_id_rtx, scratch_reg };
+ output_asm_insn ("movl\t$%c0, %1", mov_operands);
+
+ /* Create memory operand for the addl instruction. */
+ rtx offset_rtx = gen_int_mode (offset, DImode);
+ rtx mem_op = gen_rtx_MEM (SImode,
+ gen_rtx_PLUS (DImode, call_op, offset_rtx));
+ rtx add_operands[2] = { mem_op, scratch_reg };
+ output_asm_insn ("addl\t%0, %1", add_operands);
+
+ /* Output conditional jump to call label. */
+ fputs ("\tje\t", asm_out_file);
+ assemble_name (asm_out_file, call_name);
+ fputc ('\n', asm_out_file);
+
+ /* Output trap label and instruction. */
+ ASM_OUTPUT_LABEL (asm_out_file, trap_name);
+ output_asm_insn ("ud2", operands);
+
+ /* Use common helper for trap section entry. */
+ rtx trap_label_sym = gen_rtx_SYMBOL_REF (Pmode, trap_name);
+ kcfi_emit_traps_section (asm_out_file, trap_label_sym, labelno);
+
+ /* Output pass/call label. */
+ ASM_OUTPUT_LABEL (asm_out_file, call_name);
+ }
+
bool direct_p = constant_call_address_operand (call_op, VOIDmode);
bool output_indirect_p
= (!TARGET_SEH
@@ -29632,6 +29731,113 @@ ix86_set_handled_components (sbitmap components)
}
}
+/* Apply x86-64 specific masking to KCFI type ID. TYPE_ID is the 32-bit
+ KCFI type identifier to potentially mask. Returns the type ID with
+ x86-64 specific adjustments to avoid embedding ENDBR64/32 instruction
+ sequences in the type identifier values. */
+
+static uint32_t
+ix86_kcfi_mask_type_id (uint32_t type_id)
+{
+ /* Note this deliberately does NOT use ix86_endbr_immediate_operand,
+ since that is gated by -fcf-protection, but the KCFI ABI requires
+ this always be masked. */
+ if (type_id == 0xfa1e0ff3U || type_id == 0xfb1e0ff3U)
+ return type_id + 1;
+
+ return type_id;
+}
+
+/* Return x86-64 specific function arity (number of integer register
+ arguments) of the given FNDECL. */
+
+static uint8_t
+ix86_kcfi_compute_type_arity (tree fndecl)
+{
+ tree args;
+ uint8_t arity = 0;
+
+ /* Only count arity if requested. */
+ if (!flag_sanitize_kcfi_arity)
+ return 0;
+
+ /* If fndecl is NULL, we can't determine arity - return 0. */
+ if (!fndecl)
+ return 0;
+
+ /* Count the number of registers used, disregard SSE registers. */
+ for (args = DECL_ARGUMENTS (fndecl); args; args = TREE_CHAIN (args))
+ {
+ int int_nregs, sse_nregs;
+ bool args_on_stack = false;
+ machine_mode mode = TYPE_MODE (TREE_TYPE (args));
+
+ args_on_stack = examine_argument (mode, TREE_TYPE (args), 0, &int_nregs,
+ &sse_nregs);
+ /* If we place arguments on the stack, return highest arity indicator. */
+ if (args_on_stack)
+ return KCFI_ARITY_MAX_INDICATOR;
+ arity += int_nregs;
+ }
+
+ /* Return KCFI_ARITY_MAX_INDICATOR if we have counted more than
+ KCFI_ARITY_MAX_INDICATOR arguments. */
+ return arity > KCFI_ARITY_MAX_INDICATOR ? KCFI_ARITY_MAX_INDICATOR : arity;
+}
+
+/* Emit x86_64-specific type ID instruction and return instruction size.
+ FILE is the output assembly file stream, or NULL for size calculation only.
+ TYPE_ID is the 32-bit KCFI type identifier to emit. FNDECL is the function
+ declaration, used to compute arity if needed. Returns the number
+ of bytes the instruction occupies (5 bytes for x86_64 movl instruction). */
+
+static int
+ix86_kcfi_emit_type_id (FILE *file, uint32_t type_id, tree fndecl)
+{
+ /* Compute function arity. */
+ uint8_t arity = ix86_kcfi_compute_type_arity (fndecl);
+
+ /* Choose register for movl instruction. */
+ gcc_assert (arity <= KCFI_ARITY_MAX_INDICATOR);
+ uint32_t regno = kcfi_arity_register_map[arity];
+
+ if (file)
+ fprintf (file, "\tmovl\t$0x%08x, %%e%s\n", type_id, reg_names[regno]);
+
+ /* x86_64 uses 5-byte movl instruction for type ID. */
+ return 5;
+}
+
+/* Return true if the target supports KCFI.
+ KCFI requires 64-bit mode (not x32) and the R10/R11 registers. */
+
+static bool
+ix86_kcfi_supported_p (void)
+{
+ if (!TARGET_64BIT || TARGET_X32)
+ {
+ error ("%<-fsanitize=kcfi%> is not supported for 32-bit x86 or x32 mode");
+ return false;
+ }
+ if (fixed_regs[R10_REG] || fixed_regs[R11_REG])
+ {
+ error ("%<-fsanitize=kcfi%> is not compatible with "
+ "%<-ffixed-r10%> or %<-ffixed-r11%> as KCFI requires these "
+ "registers for type checking");
+ return false;
+ }
+ return true;
+}
+
+#undef TARGET_KCFI_SUPPORTED
+#define TARGET_KCFI_SUPPORTED ix86_kcfi_supported_p
+
+#undef TARGET_KCFI_MASK_TYPE_ID
+#define TARGET_KCFI_MASK_TYPE_ID ix86_kcfi_mask_type_id
+
+#undef TARGET_KCFI_EMIT_TYPE_ID
+#define TARGET_KCFI_EMIT_TYPE_ID ix86_kcfi_emit_type_id
+
#undef TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS
#define TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS ix86_get_separate_components
#undef TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index f8a325b5153a..da258175b497 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -17551,6 +17551,52 @@ The type identifier is placed before the function entry point,
allowing runtime verification without additional metadata structures,
and without changing the entry points of the target functions.
+Platform-specific implementation details:
+
+On x86_64, KCFI type identifiers are emitted as a @code{movl $ID, %eax}
+instruction before the function entry. The implementation ensures that
+type IDs never collide with ENDBR instruction encodings. When used
+with @option{-fpatchable-function-entry}, the type identifier is
+placed before any patchable NOPs, with appropriate alignment to maintain
+the alignment specified by @code{-falign-functions}. KCFI automatically
+implies @option{-mindirect-branch-register}, forcing all indirect calls
+and jumps to use registers instead of memory operands. The runtime
+check loads the type ID from the target function into @code{%r10d} and
+uses an @code{addl} instruction to add the negative expected type ID,
+effectively zeroing the register if the types match. A conditional
+jump follows to either continue execution or trap on mismatch. The
+check sequence uses @code{%r10d} and @code{%r11d} as scratch registers.
+Trap locations are recorded in a special @code{.kcfi_traps} section
+that maps trap sites to their corresponding function entry points,
+enabling debuggers and crash handlers to identify KCFI violations.
+The exact instruction sequences for both the KCFI preamble and the
+check-call bundle are considered ABI, as the Linux kernel may
+optionally rewrite these areas at boot time to mitigate detected CPU
+errata.
+
+The @code{-fsanitize-kcfi-arity} option encodes the function arity
+(i.e., the number of arguments) into the @code{movl $ID, $REG} instruction,
+where the @code{$REG} encodes the function arity. This allows users, such
+as FineIBT to generate code in the kernel that is aware of how many register
+arguments a function takes. The encoding is as follows:
+
+@multitable @columnfractions 0.10 0.50 0.40
+@headitem Arity @tab Description @tab Register
+@item 0 @tab 0 parameters @tab @code{EAX}
+@item 1 @tab 1 parameter in RDI @tab @code{ECX}
+@item 2 @tab 2 parameters in RDI and RSI @tab @code{EDX}
+@item 3 @tab 3 parameters in RDI, RSI, and RDX @tab @code{EBX}
+@item 4 @tab 4 parameters in RDI, RSI, RDX, and RCX @tab @code{ESP}
+@item 5 @tab 5 parameters in RDI, RSI, RDX, RCX, and R8 @tab @code{EBP}
+@item 6 @tab 6 parameters in RDI, RSI, RDX, RCX, R8, and R9 @tab @code{ESI}
+@item 7 @tab At least one parameter may be passed on the stack @tab @code{EDI}
+@end multitable
+
+For example, if a function `foo` takes 3 register arguments, the KCFI
+header MOVri instruction would become something like this:
+
+@code{movl $199571451, %ebx # hash of foo's type = 0xBE537FB}
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
index 7c1cff986c01..7c59921e630c 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -46,4 +46,21 @@ __attribute__((noinline)) void test_conditional_call(int flag) {
}
}
+/*
+** test_complex_args: { target x86_64-*-* }
+** ...
+** movl \$-?[0-9]+, %r10d
+** addl -4\((%r[a-z0-9]+)\), %r10d
+** je .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ud2
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .long .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jmp \*\1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
index ca833fed2971..fe0a21d26df9 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -1,5 +1,6 @@
/* Test basic KCFI functionality - preamble generation. */
/* { dg-do compile } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
/* Extern function declarations - should NOT get KCFI preambles. */
extern void external_func(void);
@@ -55,6 +56,26 @@ int main() {
/* Function with nocf_check attribute should NOT have preamble. */
/* { dg-final { scan-assembler-not {__cfi_nocf_check_function:} } } */
+/* x86_64: Verify type ID in preamble (after NOPs, before function label) */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t+nop\n.*\n\t+movl\t+\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+
+/*
+** static_caller: { target x86_64-*-* }
+** ...
+** movl \$-?[0-9]+, %r10d
+** addl -4\((%r[a-z0-9]+)\), %r10d
+** je .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ud2
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .long .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** call \*\1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
/* Extern functions should NOT get KCFI preambles. */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
index f72344f77124..05165f0e2851 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -59,3 +59,18 @@ int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *
Should see:
1. KCFI check for is_visible call with is_visible type ID A.
2. KCFI check for is_bin_visible and is_bin_visible_again call with type ID B. */
+
+/* Verify we have TWO different KCFI check sequences. */
+/* Each check should have different type ID constants. */
+/* x86: { dg-final { scan-assembler-times {movl\s+\$-?[0-9]+,\s+%r10d} 2 { target i?86-*-* x86_64-*-* } } } */
+
+/* Verify the checks use DIFFERENT type IDs (not shared).
+ We should NOT see the same type ID used twice - that would indicate
+ unmerged sharing. */
+/* x86: { dg-final { scan-assembler-not {movl\s+\$(-?[0-9]+),\s+%r10d.*movl\s+\$\1,\s+%r10d} { target i?86-*-* x86_64-*-* } } } */
+
+/* Verify expected number of traps. */
+/* x86: { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
+
+/* Verify 2 separate call sites. */
+/* x86: { dg-final { scan-assembler-times {jmp\s+\*%[a-z0-9]+} 2 { target i?86-*-* x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
index c48b8d7ad552..ed415033c5c9 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -128,4 +128,22 @@ int main() {
return result;
}
+/* Standard KCFI handling. */
+/*
+** test_struct_members: { target x86_64-*-* }
+** ...
+** movl \$-?[0-9]+, %r10d
+** addl -4\((%r[a-z0-9]+)\), %r10d
+** je .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ud2
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .long .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** call \*\1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
index 4fce0771a55d..cda9bf6c82b8 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
@@ -26,3 +26,4 @@ void caller(int sel, void (*ip)(int), int x) {
/* Per-arch dg-final patterns are added by each arch's KCFI implementation
commit; the invariant is "exactly one typeid load = one indirect call". */
+/* { dg-final { scan-assembler-times {addl\t-4\(%r[a-z0-9]+\), %r10d} 1 { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
index 7d58fef3f920..5553ff47174b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
@@ -38,4 +38,23 @@ int main(void)
return 0;
}
+/*
+** indirect_call: { target x86_64-*-* }
+** ...
+** movq %rdi, (%rax)
+** movl \$called_count, %edi
+** movl \$-?[0-9]+, %r10d
+** addl -4\(\1\), %r10d
+** je .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ud2
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .long .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jmp \*\1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
index 4a90390d1934..9ed7e21fe8eb 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -72,3 +72,14 @@ int main(void)
return 0;
}
+
+/* Verify correct number of KCFI checks: exactly 2 */
+/* { dg-final { scan-assembler-times {ud2} 2 { target x86_64-*-* } } } */
+
+/* Positive controls: these should have KCFI checks. */
+/* { dg-final { scan-assembler {normal_function:.*ud2.*\.size\s+normal_function} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*ud2.*\.size\s+wrap_normal_inline} { target x86_64-*-* } } } */
+
+/* Negative controls: these should NOT have KCFI checks. */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ud2.*\.size\s+sensitive_non_inline_function} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ud2.*\.size\s+wrap_sensitive_inline} { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
index 124d26488635..95a8e8419e00 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -29,3 +29,8 @@ int main() {
/* { dg-final { scan-assembler "__cfi_caller_with_checks:" } } */
/* { dg-final { scan-assembler "__cfi_caller_no_checks:" } } */
/* { dg-final { scan-assembler "__cfi_main:" } } */
+
+/* caller_with_checks() should generate KCFI check.
+ caller_no_checks() should NOT generate KCFI check (no_sanitize).
+ So a total of exactly 1 KCFI check in the entire program. */
+/* { dg-final { scan-assembler-times {addl\t-4\(%r[ad]x\), %r1[01]d} 1 { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
index 213a1a2892a5..97d964feebd3 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -1,5 +1,6 @@
/* Test KCFI call-site offset validation across architectures. */
/* { dg-do compile } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
void target_func_a(void) { }
void target_func_b(int x) { }
@@ -22,3 +23,7 @@ int main() {
/* { dg-final { scan-assembler "__cfi_target_func_a:" } } */
/* { dg-final { scan-assembler "__cfi_target_func_b:" } } */
/* { dg-final { scan-assembler "__cfi_target_func_c:" } } */
+
+/* x86_64: All call sites should use -4 offset for KCFI type ID loads, even
+ with -falign-functions=16 (we're not using patchable entries here). */
+/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
index a6a2f4816fef..379356385a16 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -1,6 +1,7 @@
/* Test KCFI with patchable function entries - entry NOPs only. */
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=4,0" } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
void test_function(void) {
}
@@ -11,4 +12,27 @@ int main() {
return 0;
}
+/*
+** __cfi_test_function: { target x86_64-*-* }
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** movl \$0x[0-9a-f]+, %eax
+*/
+
+/*
+** main: { target x86_64-*-* }
+** ...
+** addl -4\(%r[a-z0-9]+\), %r10d
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
index 8c4ec30cecc5..06df3495bb23 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -1,6 +1,7 @@
/* Test KCFI with large patchable function entries. */
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=11,11" } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
void test_function(void) {
}
@@ -11,4 +12,16 @@ int main() {
return 0;
}
+/*
+** __cfi_test_function: { target x86_64-*-* }
+** movl \$0x[0-9a-f]+, %eax
+*/
+
+/*
+** main: { target x86_64-*-* }
+** ...
+** addl -15\(%r[a-z0-9]+\), %r10d
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
index 78a834ef2a97..ef87b135934b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -1,6 +1,7 @@
/* Test KCFI with medium patchable function entries. */
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=8,4" } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
void test_function(void) {
}
@@ -11,4 +12,23 @@ int main() {
return 0;
}
+/*
+** __cfi_test_function: { target x86_64-*-* }
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** movl \$0x[0-9a-f]+, %eax
+*/
+
+/*
+** main: { target x86_64-*-* }
+** ...
+** addl -8\(%r[a-z0-9]+\), %r10d
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
index 1a4d8269ed56..872814aa4171 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -1,6 +1,7 @@
/* Test KCFI with patchable function entries - prefix NOPs only. */
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=3,3" } */
+/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
void test_function(void) {
}
@@ -11,4 +12,24 @@ int main() {
return 0;
}
+/*
+** __cfi_test_function: { target x86_64-*-* }
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** nop
+** movl \$0x[0-9a-f]+, %eax
+*/
+
+/*
+** main: { target x86_64-*-* }
+** ...
+** addl -7\(%r[a-z0-9]+\), %r10d
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
index 9ddf178aa2b1..04a9eb1fd206 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -58,3 +58,23 @@ int test_non_tail_indirect_call(func_ptr_t handler, int x) {
/* { dg-final { scan-assembler-times "__cfi_test_param_indirect_call:" 1 } } */
/* { dg-final { scan-assembler-times "__cfi_test_void_indirect_call:" 1 } } */
/* { dg-final { scan-assembler-times "__cfi_test_non_tail_indirect_call:" 1 } } */
+
+/* Should have exactly 4 KCFI checks for indirect calls as
+ (load type ID + compare). */
+/* { dg-final { scan-assembler-times {movl\t\$-?[0-9]+, %r10d} 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {addl\t-4\(%r[a-z0-9]+\), %r10d} 4 { target x86_64-*-* } } } */
+
+/* Should have exactly 4 trap sections and 4 trap instructions. */
+/* { dg-final { scan-assembler-times "\\.kcfi_traps" 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times "ud2" 4 { target x86_64-*-* } } } */
+
+/* Should NOT have unprotected direct jumps to vtable. */
+/* { dg-final { scan-assembler-not {jmp\t\*vtable\(%rip\)} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {jmp\t\*vtable\+8\(%rip\)} { target x86_64-*-* } } } */
+
+/* Should have exactly 3 protected tail calls (jmp through register after
+ KCFI check). */
+/* { dg-final { scan-assembler-times {jmp\t\*%[a-z0-9]+} 3 { target x86_64-*-* } } } */
+
+/* Should have exactly 1 regular call (non-tail call case). */
+/* { dg-final { scan-assembler-times {call\t\*%[a-z0-9]+} 1 { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
new file mode 100644
index 000000000000..448805a37808
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
@@ -0,0 +1,23 @@
+/* The .kcfi_traps section is SHF_LINK_ORDER and must be tied to the
+ containing function's own text section, not the generic .text. When it
+ links to the generic .text, building with -ffunction-sections and
+ -Wl,--gc-sections silently drops the entire .kcfi_traps section (losing all
+ KCFI trap metadata, even for retained functions), because the empty generic
+ .text is garbage-collected and the link-ordered section goes with it.
+
+ Only x86_64 and RISC-V use a .kcfi_traps section; AArch64 and ARM encode the
+ trap information directly in the trap instruction's immediate. */
+
+/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-additional-options "-ffunction-sections" } */
+
+typedef int (*fn_t) (int);
+volatile fn_t ptr;
+
+int caller (int x) { return ptr (x); }
+
+/* Under -ffunction-sections the link-ordered target must be the function's
+ own text section (.text.caller), not the generic .text (which would be
+ garbage-collected, dropping the trap metadata). */
+/* { dg-final { scan-assembler {\.kcfi_traps,"ao",@progbits,\.text\.caller\n} } } */
+/* { dg-final { scan-assembler-not {\.kcfi_traps,"ao",@progbits,\.text\n} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
index 6d34ad6e1a0c..55c0829ccd7b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -15,3 +15,9 @@ int main() {
/* Should have KCFI preamble. */
/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+
+/* Should have exactly 2 trap labels in code. */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ud2} 2 { target x86_64-*-* } } } */
+
+/* x86_64 should exactly 2 .kcfi_traps sections. */
+/* { dg-final { scan-assembler-times {\.section\t\.kcfi_traps,"ao",@progbits,\.text} 2 { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-32bit.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-32bit.c
new file mode 100644
index 000000000000..3bbceb2925c5
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-32bit.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is rejected for 32-bit x86. */
+/* { dg-do compile { target i?86-*-* x86_64-*-* } } */
+/* { dg-additional-options "-m32" } */
+/* { dg-error ".-fsanitize=kcfi. is not supported for 32-bit x86 or x32 mode" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-arity.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-arity.c
new file mode 100644
index 000000000000..446ce1bf514e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-arity.c
@@ -0,0 +1,93 @@
+/* Test KCFI -fsanitize-kcfi-arity - preamble generation */
+/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-additional-options "-O2 -fsanitize-kcfi-arity" } */
+
+void regular_function(int x) {
+ /* This should get KCFI preamble */
+}
+
+/* These functions should get a KCFI arity corresponing to the number of
+ parameter.
+
+ On x86-64, the mov instruction in the preamble encodes the arity in the
+ used immediate register, as follows:
+
+ Arity Description Encoding in reg field
+ 0 0 parameters EAX
+ 1 1 parameter in RDI ECX
+ 2 2 parameters in RDI EDX
+ and RSI
+ 3 3 parameters in RDI, EBX
+ RSI, and RDX
+ 4 4 parameters in RDI, ESP
+ RSI, RDX, and RCX
+ 5 5 parameters in RDI, EBP
+ RSI, RDX, RCX, and R8
+ 6 6 parameters in RDI, ESI
+ RSI, RDX, RCX, R8, and R9
+ 7 At least one parameter EDI
+ may be passed on the stack
+*/
+void ind_fn_00(void) {}
+void ind_fn_01(int) {}
+void ind_fn_02(int, int) {}
+void ind_fn_03(int, int, int) {}
+void ind_fn_04(int, int, int, int) {}
+void ind_fn_05(int, int, int, int, int) {}
+void ind_fn_06(int, int, int, int, int, int) {}
+/* Arguments on stack arity from here on should be 7. */
+void ind_fn_07(int, int, int, int, int, int, int) {}
+void ind_fn_08(int, int, int, int, int, int, int, int) {}
+
+void ind_fn_float_01(int, float) {}
+void ind_fn_float_03(int, int, float, float, float) {}
+
+void (*func_ptr_00)(void) = ind_fn_00;
+void (*func_ptr_01)(int) = ind_fn_01;
+void (*func_ptr_02)(int, int) = ind_fn_02;
+void (*func_ptr_03)(int, int, int) = ind_fn_03;
+void (*func_ptr_04)(int, int, int, int) = ind_fn_04;
+void (*func_ptr_05)(int, int, int, int, int) = ind_fn_05;
+void (*func_ptr_06)(int, int, int, int, int, int) = ind_fn_06;
+void (*func_ptr_07)(int, int, int, int, int, int, int) = ind_fn_07;
+void (*func_ptr_08)(int, int, int, int, int, int, int, int) = ind_fn_08;
+
+
+void (*func_ptr_float_01)(int, float) = ind_fn_float_01;
+void (*func_ptr_float_03)(int, int, float, float, float) = ind_fn_float_03;
+
+int main() {
+ /* Function arity tests. */
+ func_ptr_00();
+ func_ptr_01(1);
+ func_ptr_02(1, 1);
+ func_ptr_03(1, 1, 1);
+ func_ptr_04(1, 1, 1, 1);
+ func_ptr_05(1, 1, 1, 1, 1);
+ func_ptr_06(1, 1, 1, 1, 1, 1);
+
+ /* Both of these put arguments on the stack so both get arity 7. */
+ func_ptr_07(1, 1, 1, 1, 1, 1, 1);
+ func_ptr_08(1, 1, 1, 1, 1, 1, 1, 1);
+
+ /* Float parameters should not be counted for arity. */
+ func_ptr_float_01(1, 1.0);
+ func_ptr_float_03(1, 1, 0.5, 0.5, 0.5);
+
+ return 0;
+}
+
+/* x86_64: Verify arity immediate register encoding in preamble. */
+/* { dg-final { scan-assembler {__cfi_ind_fn_00:.*\n\tmovl\t+\$0x[0-9a-f]+, %eax\n.*\nind_fn_00:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_01:.*\n\tmovl\t+\$0x[0-9a-f]+, %ecx\n.*\nind_fn_01:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_02:.*\n\tmovl\t+\$0x[0-9a-f]+, %edx\n.*\nind_fn_02:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_03:.*\n\tmovl\t+\$0x[0-9a-f]+, %ebx\n.*\nind_fn_03:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_04:.*\n\tmovl\t+\$0x[0-9a-f]+, %esp\n.*\nind_fn_04:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_05:.*\n\tmovl\t+\$0x[0-9a-f]+, %ebp\n.*\nind_fn_05:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_06:.*\n\tmovl\t+\$0x[0-9a-f]+, %esi\n.*\nind_fn_06:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_07:.*\n\tmovl\t+\$0x[0-9a-f]+, %edi\n.*\nind_fn_07:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_08:.*\n\tmovl\t+\$0x[0-9a-f]+, %edi\n.*\nind_fn_08:} { target x86_64-*-* } } } */
+
+/* x86_64: Verify arity is not affected by SSE registers. */
+/* { dg-final { scan-assembler {__cfi_ind_fn_float_01:.*\n\tmovl\t+\$0x[0-9a-f]+, %ecx\n.*\nind_fn_float_01:} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {__cfi_ind_fn_float_03:.*\n\tmovl\t+\$0x[0-9a-f]+, %edx\n.*\nind_fn_float_03:} { target x86_64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-endbr-collision.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-endbr-collision.c
new file mode 100644
index 000000000000..ead04a66488c
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-endbr-collision.c
@@ -0,0 +1,55 @@
+/* Verify the x86 KCFI type-id ENDBR avoidance appears in output. A type
+ whose raw KCFI type-id happens to equal an ENDBR instruction encoding
+ must have that id bumped by one, so the value stored in the __cfi_
+ preamble can never be mistaken for a CET landing pad. This matches
+ Clang's KCFI ABI (both ENDBR encodings, unconditionally), and exactly
+ those two 32-bit values are adjusted. E.g. the +/-1 neighbours of each
+ are left alone. */
+
+/* { dg-do compile { target { x86_64-*-* && lp64 } } } */
+/* { dg-additional-options "-O2" } */
+
+/* For the builtin, "void(*)(struct TAG)" should hash to RAW; only the
+ codegen should bump the type_id values on ENDBR64/32 collisions. */
+#define KCFI_CASE(TAG, FN, RAW) \
+ struct TAG; \
+ typedef void FN##_ft (struct TAG *); \
+ typedef void (*FN##_fp) (struct TAG *); \
+ _Static_assert (__builtin_linux_abi_kcfi_hash (FN##_ft) == RAW, \
+ #TAG " no longer hashes as expected; regenerate it"); \
+ void FN (struct TAG *q) { (void) q; } \
+ static volatile FN##_fp FN##_p;
+
+KCFI_CASE (jbjNNf, e64_lo, 0xfa1e0ff2U) /* ENDBR64 - 1 */
+KCFI_CASE (gVAv_j, e64, 0xfa1e0ff3U) /* ENDBR64 */
+KCFI_CASE (thNpCa, e64_hi, 0xfa1e0ff4U) /* ENDBR64 + 1 */
+KCFI_CASE (M6uRHd, e32_lo, 0xfb1e0ff2U) /* ENDBR32 - 1 */
+KCFI_CASE (Q0UXLa, e32, 0xfb1e0ff3U) /* ENDBR32 */
+KCFI_CASE (uev20d, e32_hi, 0xfb1e0ff4U) /* ENDBR32 + 1 */
+
+/* volatile keeps the calls indirect, so each target is address-taken. */
+#define KCFI_USE(FN) FN##_p = FN; FN##_p (0)
+
+void
+run (void)
+{
+ KCFI_USE (e64_lo);
+ KCFI_USE (e64);
+ KCFI_USE (e64_hi);
+ KCFI_USE (e32_lo);
+ KCFI_USE (e32);
+ KCFI_USE (e32_hi);
+}
+
+/* Neither ENDBR encoding is ever stored as a type-id (both were bumped away). */
+/* { dg-final { scan-assembler-not {\tmovl\t\$0xfa1e0ff3, %eax} } } */
+/* { dg-final { scan-assembler-not {\tmovl\t\$0xfb1e0ff3, %eax} } } */
+
+/* Each ENDBR value lands on its +1 (0xfa1e0ff4 / 0xfb1e0ff4), joining the
+ upper neighbour that already had that value; each appears exactly twice. */
+/* { dg-final { scan-assembler-times {\tmovl\t\$0xfa1e0ff4, %eax} 2 } } */
+/* { dg-final { scan-assembler-times {\tmovl\t\$0xfb1e0ff4, %eax} 2 } } */
+
+/* The lower neighbours are stored unchanged; each appear exactly once. */
+/* { dg-final { scan-assembler-times {\tmovl\t\$0xfa1e0ff2, %eax} 1 } } */
+/* { dg-final { scan-assembler-times {\tmovl\t\$0xfb1e0ff2, %eax} 1 } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r10.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r10.c
new file mode 100644
index 000000000000..d0879d161599
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r10.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is incompatible with -ffixed-r10 on x86_64. */
+/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-additional-options "-ffixed-r10" } */
+/* { dg-error ".-fsanitize=kcfi. is not compatible with .-ffixed-r10. or .-ffixed-r11. as KCFI requires these registers for type checking" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r11.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r11.c
new file mode 100644
index 000000000000..465f2cad6285
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-fixed-r11.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is incompatible with -ffixed-r11 on x86_64. */
+/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-additional-options "-ffixed-r11" } */
+/* { dg-error ".-fsanitize=kcfi. is not compatible with .-ffixed-r10. or .-ffixed-r11. as KCFI requires these registers for type checking" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-retpoline-r11.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-retpoline-r11.c
new file mode 100644
index 000000000000..056068b4f197
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-x86-retpoline-r11.c
@@ -0,0 +1,40 @@
+/* Test KCFI with retpoline thunk-extern flag forces r11 usage. */
+/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-additional-options "-O2 -mindirect-branch=thunk-extern" } */
+
+extern int external_target(void);
+
+/* Test regular call (not tail call) */
+__attribute__((noinline))
+int call_test(int (*func_ptr)(void)) {
+ /* This indirect call should use r11 when both KCFI and
+ -mindirect-branch=thunk-extern are enabled. */
+ int result = func_ptr(); /* Function parameter prevents direct optimization. */
+ return result + 1; /* Prevent tail call optimization. */
+}
+
+/* Reference external_target to generate the required symbol. */
+int (*external_func_ptr)(void) = external_target;
+
+/* Test function for sibcalls (tail calls) */
+__attribute__((noinline))
+void sibcall_test(int (**func_ptr)(void)) {
+ /* This sibcall should use r11 when both KCFI and
+ -mindirect-branch=thunk-extern are enabled. */
+ (*func_ptr)(); /* Tail call - should be optimized to sibcall. */
+}
+
+/* Should have weak symbol for external function. */
+/* { dg-final { scan-assembler "__kcfi_typeid_external_target" } } */
+
+/* When both KCFI and -mindirect-branch=thunk-extern are enabled,
+ indirect calls should always use r11 register and convert to extern thunks. */
+/* { dg-final { scan-assembler-times {call\s+} 1 } } */
+/* { dg-final { scan-assembler-times {call\s+__x86_indirect_thunk_r11} 1 } } */
+
+/* Sibcalls should also use r11 register and convert to extern thunks. */
+/* { dg-final { scan-assembler-times {jmp\s+} 1 } } */
+/* { dg-final { scan-assembler-times {jmp\s+__x86_indirect_thunk_r11} 1 } } */
+
+/* Should have exactly 2 KCFI traps (one per function) */
+/* { dg-final { scan-assembler-times {ud2} 2 } } */
diff --git a/gcc/common.opt b/gcc/common.opt
index 69a55b8d40d6..1dde799a932c 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -1165,6 +1165,14 @@ Enum(sanitize_coverage) String(trace-pc) Value(SANITIZE_COV_TRACE_PC)
EnumValue
Enum(sanitize_coverage) String(trace-cmp) Value(SANITIZE_COV_TRACE_CMP)
+fsanitize-kcfi-arity
+Common Driver Var(flag_sanitize_kcfi_arity)
+For supported targets, this feature extends kCFI by telling the compiler
+to record information about each indirect-callable function’s
+arity (i.e., the number of arguments passed in registers) into the
+binary. Some kernel CFI techniques, such as FineIBT, may be able to use
+this information to provide enhanced security.
+
fasan-shadow-offset=
Common Joined RejectNegative Var(common_deferred_options) Defer
-fasan-shadow-offset=<number> Use custom shadow memory offset.
diff --git a/gcc/opts.cc b/gcc/opts.cc
index ad1fd54d2792..6382550110ac 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -2853,6 +2853,7 @@ common_handle_option (struct gcc_options *opts,
SET_OPTION_IF_UNSET (opts, opts_set, param_asan_stack, 0);
SET_OPTION_IF_UNSET (opts, opts_set, param_asan_protect_allocas, 0);
SET_OPTION_IF_UNSET (opts, opts_set, param_asan_use_after_return, 0);
+ SET_OPTION_IF_UNSET (opts, opts_set, flag_sanitize_kcfi_arity, 0);
}
if (opts->x_flag_sanitize & SANITIZE_KERNEL_HWADDRESS)
{
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v17 5/7] aarch64: Add AArch64 Kernel Control Flow Integrity implementation
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (3 preceding siblings ...)
2026-10-05 15:40 ` [PATCH v17 4/7] x86: Add x86_64 Kernel Control Flow Integrity implementation Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-05 15:40 ` [PATCH v17 6/7] arm: Add ARM 32-bit " Kees Cook
2026-10-05 15:40 ` [PATCH v17 7/7] riscv: Add RISC-V " Kees Cook
6 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Implement AArch64-specific KCFI backend.
- Trap debugging through ESR (Exception Syndrome Register) encoding
in BRK instruction immediate values.
- Scratch register allocation using w16/w17 (x16/x17) following
AArch64 procedure call standard for intra-procedure-call registers,
which already makes x16/x17 available through existing clobbers.
Note that BTI uses x16/x17 AT the call site, and KCFI uses w16/w17
BEFORE the call (for the type hash comparison). These don't conflict
because:
- KCFI clobbers w16/w17 with hash values.
- Then the actual call happens via blr %target_reg (whatever
register the target is in).
- If SLS hardening is enabled, aarch64_indirect_call_asm will
create a thunk that moves target into x16 and does br x16.
- By the time the SLS thunk uses x16, KCFI is already done with it.
- Complementary with BTI (which uses a separate pass system to inject
landing instructions where needed).
- Does not interfere with SME, which uses attributes not function
prototypes for distinguishing functions.
Assembly Code Pattern for AArch64:
ldur w16, [target, #-4] ; Load actual type ID from preamble
mov w17, #type_id_low ; Load expected type (lower 16 bits)
movk w17, #type_id_high, lsl #16 ; Load upper 16 bits if needed
cmp w16, w17 ; Compare type IDs directly
b.eq .Lpass ; Branch if types match
.Ltrap: brk #esr_value ; Enhanced trap with register info
.Lpass: blr/br target ; Execute validated indirect transfer
ESR (Exception Syndrome Register) Integration:
- BRK instruction immediate encoding format:
0x8000 | ((TypeIndex & 31) << 5) | (AddrIndex & 31)
- TypeIndex indicates which W register contains expected type (W17 = 17)
- AddrIndex indicates which X register contains target address (0-30)
- Example: brk #33313 (0x8221) = expected type in W17, target address in X1
Build and run tested with Linux kernel ARCH=arm64.
Assisted-by: LLM [tests]
gcc/ChangeLog:
* config/aarch64/aarch64-protos.h (aarch64_output_kcfi_insn): Declare.
* config/aarch64/aarch64.cc (aarch64_expand_call): Wrap CALLs in
KCFI, with clobbers.
(aarch64_indirect_branch_asm): New function, extract common
logic for branch asm, like existing call asm helper.
(aarch64_output_kcfi_insn): Emit KCFI assembly.
* config/aarch64/aarch64.md: Add KCFI RTL patterns and replace
open-coded branch emission with aarch64_indirect_branch_asm.
* doc/invoke.texi: Document aarch64 nuances.
gcc/testsuite/ChangeLog:
* gcc.dg/kcfi/kcfi-aarch64-branch-range.c: New test.
* gcc.dg/kcfi/kcfi-aarch64-ilp32.c: New test.
* gcc.dg/kcfi/kcfi-aarch64-insn-length-sls.c: New test.
* gcc.dg/kcfi/kcfi-aarch64-insn-length.c: New test.
* gcc.dg/kcfi/kcfi-adjacency.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-basics.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-call-sharing.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-complex-addressing.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-direct-call-shapes.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-move-preservation.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-no-sanitize-inline.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-no-sanitize.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-offset-validation.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-patchable-entry-only.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-patchable-large.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-patchable-medium.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-patchable-prefix-only.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-tail-calls.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-trap-section.c: Add aarch64 patterns.
* gcc.dg/kcfi/kcfi-trap-encoding.c: New test.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/aarch64/aarch64-protos.h | 3 +
gcc/config/aarch64/aarch64.md | 66 +++++++++
gcc/config/aarch64/aarch64.cc | 127 ++++++++++++++++++
gcc/doc/invoke.texi | 14 ++
.../gcc.dg/kcfi/kcfi-aarch64-branch-range.c | 29 ++++
.../gcc.dg/kcfi/kcfi-aarch64-ilp32.c | 7 +
.../kcfi/kcfi-aarch64-insn-length-sls.c | 32 +++++
.../gcc.dg/kcfi/kcfi-aarch64-insn-length.c | 38 ++++++
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 20 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 21 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 4 +
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 16 +++
.../gcc.dg/kcfi/kcfi-direct-call-shapes.c | 3 +
.../gcc.dg/kcfi/kcfi-move-preservation.c | 20 +++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 5 +
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 1 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 3 +
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 12 ++
.../gcc.dg/kcfi/kcfi-patchable-large.c | 12 ++
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 12 ++
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 12 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 21 +++
.../gcc.dg/kcfi/kcfi-trap-encoding.c | 43 ++++++
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 4 +
24 files changed, 525 insertions(+)
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-branch-range.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-ilp32.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length-sls.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
index e8ae3d427949..af00040b7abe 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -1316,4 +1316,7 @@ extern unsigned aarch64_stack_alignment (const_tree exp, unsigned align);
extern rtx aarch64_gen_compare_zero_and_branch (rtx_code code, rtx x,
rtx_code_label *label);
+/* KCFI support. */
+extern const char *aarch64_output_kcfi_insn (rtx_insn *insn, rtx *operands);
+
#endif /* GCC_AARCH64_PROTOS_H */
diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md
index 7c8915ec3890..4d7aaf8f8c6c 100644
--- a/gcc/config/aarch64/aarch64.md
+++ b/gcc/config/aarch64/aarch64.md
@@ -1604,6 +1604,20 @@
}"
)
+;; KCFI indirect call
+(define_insn "*call_insn"
+ [(kcfi (call (mem:DI (match_operand:DI 0 "aarch64_call_insn_operand" "Ucr"))
+ (match_operand 1 "" ""))
+ (match_operand 3 "const_int_operand"))
+ (unspec:DI [(match_operand:DI 2 "const_int_operand")] UNSPEC_CALLEE_ABI)
+ (clobber (reg:DI LR_REGNUM))]
+ "!SIBLING_CALL_P (insn)"
+{
+ return aarch64_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "28")])
+
(define_insn "*call_insn"
[(call (mem:DI (match_operand:DI 0 "aarch64_call_insn_operand"))
(match_operand 1 "" ""))
@@ -1631,6 +1645,22 @@
}"
)
+;; KCFI call with return value
+(define_insn "*call_value_insn"
+ [(set (match_operand 0 "" "")
+ (kcfi (call (mem:DI (match_operand:DI 1 "aarch64_call_insn_operand"
+ "Ucr"))
+ (match_operand 2 "" ""))
+ (match_operand 4 "const_int_operand")))
+ (unspec:DI [(match_operand:DI 3 "const_int_operand")] UNSPEC_CALLEE_ABI)
+ (clobber (reg:DI LR_REGNUM))]
+ "!SIBLING_CALL_P (insn)"
+{
+ return aarch64_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "28")])
+
(define_insn "*call_value_insn"
[(set (match_operand 0 "" "")
(call (mem:DI (match_operand:DI 1 "aarch64_call_insn_operand"))
@@ -1671,6 +1701,23 @@
}
)
+;; KCFI sibling call
+(define_insn "*sibcall_insn"
+ [(kcfi (call (mem:DI (match_operand:DI 0 "aarch64_call_insn_operand" "Ucs"))
+ (match_operand 1 ""))
+ (match_operand 3 "const_int_operand"))
+ (unspec:DI [(match_operand:DI 2 "const_int_operand")] UNSPEC_CALLEE_ABI)
+ (return)]
+ "SIBLING_CALL_P (insn)"
+{
+ return aarch64_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "branch")
+ (set (attr "length")
+ (cond [(match_test "!aarch64_harden_sls_retbr_p ()") (const_int 28)
+ (match_test "TARGET_SB") (const_int 32)]
+ (const_int 36)))])
+
(define_insn "*sibcall_insn"
[(call (mem:DI (match_operand:DI 0 "aarch64_call_insn_operand" "Ucs, Usf"))
(match_operand 1 ""))
@@ -1686,6 +1733,25 @@
(set_attr "sls_length" "retbr,none")]
)
+;; KCFI sibling call with return value
+(define_insn "*sibcall_value_insn"
+ [(set (match_operand 0 "")
+ (kcfi (call (mem:DI (match_operand:DI 1 "aarch64_call_insn_operand"
+ "Ucs"))
+ (match_operand 2 ""))
+ (match_operand 4 "const_int_operand")))
+ (unspec:DI [(match_operand:DI 3 "const_int_operand")] UNSPEC_CALLEE_ABI)
+ (return)]
+ "SIBLING_CALL_P (insn)"
+{
+ return aarch64_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "branch")
+ (set (attr "length")
+ (cond [(match_test "!aarch64_harden_sls_retbr_p ()") (const_int 28)
+ (match_test "TARGET_SB") (const_int 32)]
+ (const_int 36)))])
+
(define_insn "*sibcall_value_insn"
[(set (match_operand 0 "")
(call (mem:DI
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index 107271b04045..1a8104e863f3 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -100,6 +100,7 @@
#include "ipa-cp.h"
#include "ipa-prop.h"
#include "ipa-fnsummary.h"
+#include "kcfi.h"
#include "hash-map.h"
#include "ifcvt.h"
#include "aarch64-sched-dispatch.h"
@@ -12538,6 +12539,16 @@ aarch64_expand_call (rtx result, rtx mem, rtx cookie, bool sibcall)
call = gen_rtx_CALL (VOIDmode, mem, const0_rtx);
+ /* Only indirect calls need KCFI instrumentation. */
+ bool is_direct_call = !REG_P (XEXP (mem, 0));
+ rtx kcfi_type_rtx = is_direct_call ? NULL_RTX
+ : kcfi_get_type_id_for_expanding_gimple_call ();
+ if (kcfi_type_rtx)
+ {
+ /* Wrap call in KCFI. */
+ call = gen_rtx_KCFI (VOIDmode, call, kcfi_type_rtx);
+ }
+
if (result != NULL_RTX)
call = gen_rtx_SET (result, call);
@@ -34967,6 +34978,122 @@ aarch64_libgcc_floating_mode_supported_p
#undef TARGET_DOCUMENTATION_NAME
#define TARGET_DOCUMENTATION_NAME "AArch64"
+/* Output the assembly for a KCFI checked call instruction. INSN is the
+ RTL instruction being processed. OPERANDS is the array of RTL operands
+ where operands[0] is the call target register, operands[3] is the KCFI
+ type ID constant. Returns the appropriate call instruction string. */
+
+const char *
+aarch64_output_kcfi_insn (rtx_insn *insn, rtx *operands)
+{
+ /* Target register is operands[0]. */
+ rtx target_reg = operands[0];
+ gcc_assert (REG_P (target_reg));
+
+ /* Get KCFI type ID from operand[3]. */
+ uint32_t type_id = UINTVAL (operands[3]);
+
+ /* Calculate typeid offset from call target. */
+ HOST_WIDE_INT offset = -kcfi_get_typeid_offset ();
+
+ /* Get unique label number for this KCFI check. */
+ int labelno = kcfi_next_labelno ();
+
+ /* Generate custom label names. */
+ char trap_name[32];
+ char call_name[32];
+ ASM_GENERATE_INTERNAL_LABEL (trap_name, "Lkcfi_trap", labelno);
+ ASM_GENERATE_INTERNAL_LABEL (call_name, "Lkcfi_call", labelno);
+
+ /* Pick scratch registers that do not alias the call target. The
+ scratch defaults are w16 (actual type) and w17 (expected type), but
+ for an indirect sibling call the target is itself constrained to
+ x16/x17 (TAILCALL_ADDR_REGS). Loading into a register that aliases
+ the target would clobber it before the branch. When the target is
+ x16 or x17, keep the expected type in the other of the two and load
+ the actual type into x9: a caller-saved temporary that is dead at
+ the call point. */
+ unsigned tregno = REGNO (target_reg);
+ unsigned actual_regno, expected_regno;
+ if (tregno == R16_REGNUM || tregno == R17_REGNUM)
+ {
+ expected_regno = (tregno == R16_REGNUM) ? R17_REGNUM : R16_REGNUM;
+ actual_regno = R9_REGNUM;
+ }
+ else
+ {
+ actual_regno = R16_REGNUM;
+ expected_regno = R17_REGNUM;
+ }
+
+ /* The scratch operands are the 32-bit type values, printed with the
+ %w modifier (w-names); target_reg is Pmode and prints as the 64-bit
+ base (x-name) in the load. */
+ rtx actual_reg = gen_rtx_REG (SImode, actual_regno);
+ rtx expected_reg = gen_rtx_REG (SImode, expected_regno);
+ rtx ops[3];
+
+ /* Load actual type from memory at offset using ldur. */
+ ops[0] = actual_reg;
+ ops[1] = target_reg;
+ ops[2] = GEN_INT (offset);
+ output_asm_insn ("ldur\t%w0, [%1, #%2]", ops);
+
+ /* Load expected type using mov/movk sequence. */
+ ops[0] = expected_reg;
+ ops[1] = GEN_INT (type_id & 0xFFFF);
+ output_asm_insn ("mov\t%w0, #%1", ops);
+ ops[1] = GEN_INT ((type_id >> 16) & 0xFFFF);
+ output_asm_insn ("movk\t%w0, #%1, lsl #16", ops);
+
+ /* Compare types. */
+ ops[0] = actual_reg;
+ ops[1] = expected_reg;
+ output_asm_insn ("cmp\t%w0, %w1", ops);
+
+ /* Output conditional branch to call label. */
+ fputs ("\tb.eq\t", asm_out_file);
+ assemble_name (asm_out_file, call_name);
+ fputc ('\n', asm_out_file);
+
+ /* Output trap label and BRK instruction. */
+ ASM_OUTPUT_LABEL (asm_out_file, trap_name);
+
+ /* Calculate and emit BRK with ESR encoding. The type index must name
+ the register that actually holds the expected type. */
+ unsigned type_index = expected_regno;
+ unsigned addr_index = REGNO (target_reg) - R0_REGNUM;
+ unsigned esr_value = 0x8000 | ((type_index & 31) << 5) | (addr_index & 31);
+
+ fprintf (asm_out_file, "\tbrk\t#%#x\n", esr_value);
+
+ /* Output call label. */
+ ASM_OUTPUT_LABEL (asm_out_file, call_name);
+
+ /* Return appropriate call instruction based on SIBLING_CALL_P. */
+ if (SIBLING_CALL_P (insn))
+ return aarch64_indirect_branch_asm (target_reg);
+ else
+ return aarch64_indirect_call_asm (target_reg);
+}
+
+/* Return true if the target supports KCFI.
+ KCFI is not supported for ILP32 due to pointer size requirements. */
+
+static bool
+aarch64_kcfi_supported_p (void)
+{
+ if (TARGET_ILP32)
+ {
+ error ("%<-fsanitize=kcfi%> is not supported for %<-mabi=ilp32%>");
+ return false;
+ }
+ return true;
+}
+
+#undef TARGET_KCFI_SUPPORTED
+#define TARGET_KCFI_SUPPORTED aarch64_kcfi_supported_p
+
struct gcc_target targetm = TARGET_INITIALIZER;
#include "gt-aarch64.h"
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index da258175b497..ee8b48a68517 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -17597,6 +17597,20 @@ header MOVri instruction would become something like this:
@code{movl $199571451, %ebx # hash of foo's type = 0xBE537FB}
+On AArch64, KCFI type identifiers are emitted as a @code{.word ID}
+directive (a 32-bit constant) before the function entry. AArch64's
+natural 4-byte instruction alignment eliminates the need for additional
+alignment NOPs. When used with @option{-fpatchable-function-entry}, the
+type identifier is placed before any prefix NOPs. The runtime check
+uses @code{x16} and @code{x17} as scratch registers. Type mismatches
+trigger a @code{brk} instruction with an immediate value that encodes
+both the expected type register index and the target address register
+index in the format @code{0x8000 | (type_reg << 5) | addr_reg}. This
+encoding is captured in the ESR (Exception Syndrome Register) when the
+trap is taken, allowing the kernel to identify both the KCFI violation
+and the involved registers for detailed diagnostics (eliminating the need
+for a separate @code{.kcfi_traps} section as used on x86_64).
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-branch-range.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-branch-range.c
new file mode 100644
index 000000000000..52a5866c59a8
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-branch-range.c
@@ -0,0 +1,29 @@
+/* Test that the AArch64 KCFI call patterns account for the length of the
+ whole check sequence. If each KCFI call is counted as one 4-byte
+ instruction, the tbz skipping the block below is assumed to be in range
+ (+-32KiB), and the assembler rejects it. See kcfi-aarch64-insn-length*.c
+ for the sibling call patterns. */
+/* { dg-do assemble { target aarch64*-*-* } } */
+/* { dg-additional-options "-O2" } */
+
+typedef void (*fn_t) (void);
+extern fn_t tbl[2048];
+
+#define C1(i) tbl[(i)] ();
+#define C4(i) C1(i) C1((i)+1) C1((i)+2) C1((i)+3)
+#define C16(i) C4(i) C4((i)+4) C4((i)+8) C4((i)+12)
+#define C64(i) C16(i) C16((i)+16) C16((i)+32) C16((i)+48)
+#define C256(i) C64(i) C64((i)+64) C64((i)+128) C64((i)+192)
+#define C1024(i) C256(i) C256((i)+256) C256((i)+512) C256((i)+768)
+
+/* About 1280 KCFI calls: roughly 41KiB of code, but only 10KiB if each
+ call is counted as 4 bytes. */
+void
+calls (unsigned long x)
+{
+ if (x & 1)
+ {
+ C1024 (0)
+ C256 (1024)
+ }
+}
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-ilp32.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-ilp32.c
new file mode 100644
index 000000000000..aff560020c7e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-ilp32.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is rejected for AArch64 ILP32. */
+/* { dg-do compile { target aarch64*-*-* } } */
+/* { dg-additional-options "-mabi=ilp32 -Wno-deprecated" } */
+/* { dg-error ".-fsanitize=kcfi. is not supported for .-mabi=ilp32." "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length-sls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length-sls.c
new file mode 100644
index 000000000000..4164efcb5d0e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length-sls.c
@@ -0,0 +1,32 @@
+/* Test the AArch64 KCFI sibling call length with straight-line speculation
+ hardening, which adds a barrier after the br: sb (4 bytes) when SB is
+ available, otherwise dsb sy; isb (8 bytes). Calls are unaffected. */
+/* { dg-do compile { target aarch64*-*-* } } */
+/* { dg-additional-options "-O2 -dp -mharden-sls=retbr" } */
+
+typedef void (*vfn_t) (void);
+
+__attribute__ ((target ("arch=armv8-a")))
+void
+sibcall_no_sb (vfn_t f)
+{
+ f ();
+}
+
+__attribute__ ((target ("arch=armv8-a+sb")))
+void
+sibcall_sb (vfn_t f)
+{
+ f ();
+}
+
+int
+call (vfn_t f)
+{
+ f ();
+ return 0;
+}
+
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=36\] \*sibcall_insn\n} 1 } } */
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=32\] \*sibcall_insn\n} 1 } } */
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=28\] \*call_insn\n} 1 } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length.c
new file mode 100644
index 000000000000..3a0ee389ef8a
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-aarch64-insn-length.c
@@ -0,0 +1,38 @@
+/* Test that the AArch64 KCFI call patterns report the length of the whole
+ check sequence (ldur, mov, movk, cmp, b.eq, brk, blr/br: 28 bytes), so
+ that branch shortening sees their true size. */
+/* { dg-do compile { target aarch64*-*-* } } */
+/* { dg-additional-options "-O2 -dp" } */
+
+typedef void (*vfn_t) (void);
+typedef int (*ifn_t) (void);
+
+void
+call (vfn_t f)
+{
+ f ();
+ f ();
+}
+
+int
+call_value (ifn_t f)
+{
+ return f () + 1;
+}
+
+void
+sibcall (vfn_t f)
+{
+ f ();
+}
+
+int
+sibcall_value (ifn_t f)
+{
+ return f ();
+}
+
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=28\] \*call_insn\n} 1 } } */
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=28\] \*call_value_insn\n} 1 } } */
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=28\] \*sibcall_insn\n} 2 } } */
+/* { dg-final { scan-assembler-times {\[c=[0-9]+ l=28\] \*sibcall_value_insn\n} 1 } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
index 7c59921e630c..47a33bf6393b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -63,4 +63,24 @@ __attribute__((noinline)) void test_conditional_call(int flag) {
** ...
*/
+/*
+** test_complex_args: { target aarch64*-*-* }
+** ...
+** ldur w[0-9]+, \[(x[0-9]+), #-4\]
+** mov w[0-9]+, #[0-9]+
+** movk w[0-9]+, #[0-9]+, lsl #16
+** cmp w[0-9]+, w[0-9]+
+** b.eq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** brk #0x[0-9a-f]+
+** .Lkcfi_call\2:
+** br \1
+** ...
+*/
+
+/* The typeid load must not clobber the call target: its destination
+ register must differ from the base (target) register. "ldur wN, [xN,..."
+ (same register number) means the target was overwritten by the load. */
+/* { dg-final { scan-assembler-not {ldur\tw([0-9]+), \[x\1,} { target aarch64*-*-* } } } */
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
index fe0a21d26df9..7f1c45f1fe94 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -59,6 +59,9 @@ int main() {
/* x86_64: Verify type ID in preamble (after NOPs, before function label) */
/* { dg-final { scan-assembler {__cfi_regular_function:\n\t+nop\n.*\n\t+movl\t+\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
+/* AArch64: Verify type ID word in preamble. */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target aarch64*-*-* } } } */
+
/*
** static_caller: { target x86_64-*-* }
** ...
@@ -76,6 +79,21 @@ int main() {
** ...
*/
+/*
+** static_caller: { target aarch64*-*-* }
+** ...
+** ldur w16, \[(x[0-9]+), #-4\]
+** mov w17, #[0-9]+
+** movk w17, #[0-9]+, lsl #16
+** cmp w16, w17
+** b.eq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** brk #0x[0-9a-f]+
+** .Lkcfi_call\2:
+** blr \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
/* Extern functions should NOT get KCFI preambles. */
@@ -93,3 +111,6 @@ int main() {
/* External functions that are only called directly should NOT get
__kcfi_typeid_ symbols. */
/* { dg-final { scan-assembler-not {__kcfi_typeid_external_func_int} } } */
+
+/* AArch64 should NOT have trap section (use immediate instructions instead). */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
index 05165f0e2851..c402a1678a73 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -63,14 +63,18 @@ int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *
/* Verify we have TWO different KCFI check sequences. */
/* Each check should have different type ID constants. */
/* x86: { dg-final { scan-assembler-times {movl\s+\$-?[0-9]+,\s+%r10d} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {mov\s+w17, #[0-9]+} 2 { target aarch64*-*-* } } } */
/* Verify the checks use DIFFERENT type IDs (not shared).
We should NOT see the same type ID used twice - that would indicate
unmerged sharing. */
/* x86: { dg-final { scan-assembler-not {movl\s+\$(-?[0-9]+),\s+%r10d.*movl\s+\$\1,\s+%r10d} { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-not {mov\s+w17, #([0-9]+).*mov\s+w17, #\1} { target aarch64*-*-* } } } */
/* Verify expected number of traps. */
/* x86: { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
/* Verify 2 separate call sites. */
/* x86: { dg-final { scan-assembler-times {jmp\s+\*%[a-z0-9]+} 2 { target i?86-*-* x86_64-*-* } } } */
+/* AArch64: { dg-final { scan-assembler-times {br\tx[0-9]+} 2 { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
index ed415033c5c9..0b23caa49ebb 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -146,4 +146,20 @@ int main() {
** ...
*/
+/* Standard KCFI handling. */
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[(x[0-9]+), #-4\]
+** mov w17, #[0-9]+
+** movk w17, #[0-9]+, lsl #16
+** cmp w16, w17
+** b.eq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** brk #0x[0-9a-f]+
+** .Lkcfi_call\2:
+** blr \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
index cda9bf6c82b8..6b4698447b7b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
@@ -27,3 +27,6 @@ void caller(int sel, void (*ip)(int), int x) {
/* Per-arch dg-final patterns are added by each arch's KCFI implementation
commit; the invariant is "exactly one typeid load = one indirect call". */
/* { dg-final { scan-assembler-times {addl\t-4\(%r[a-z0-9]+\), %r10d} 1 { target x86_64-*-* } } } */
+/* The typeid scratch register avoids the call target, so it is not always
+ w16 (a sibcall target lands in x16/x17, forcing the load into w9). */
+/* { dg-final { scan-assembler-times {ldur\tw[0-9]+, \[x[0-9]+, #-4\]} 1 { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
index 5553ff47174b..ad6130c9d323 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
@@ -57,4 +57,24 @@ int main(void)
** ...
*/
+/*
+** indirect_call: { target aarch64*-*-* }
+** ...
+** mov (x[0-9]+), x0
+** ...
+** ldur w[0-9]+, \[\1, #-4\]
+** mov w[0-9]+, #[0-9]+
+** movk w[0-9]+, #[0-9]+, lsl #16
+** cmp w[0-9]+, w[0-9]+
+** b.eq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** brk #0x[0-9a-f]+
+** .Lkcfi_call\2:
+** br \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
+
+/* AArch64 should NOT have trap section (use immediate instructions instead). */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
index 9ed7e21fe8eb..98591ed9b145 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -75,11 +75,16 @@ int main(void)
/* Verify correct number of KCFI checks: exactly 2 */
/* { dg-final { scan-assembler-times {ud2} 2 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
/* Positive controls: these should have KCFI checks. */
/* { dg-final { scan-assembler {normal_function:.*ud2.*\.size\s+normal_function} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler {wrap_normal_inline:.*ud2.*\.size\s+wrap_normal_inline} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler {normal_function:.*brk\s+#0x[0-9a-f]+.*\.size\s+normal_function} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_normal_inline} { target aarch64*-*-* } } } */
/* Negative controls: these should NOT have KCFI checks. */
/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ud2.*\.size\s+sensitive_non_inline_function} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ud2.*\.size\s+wrap_sensitive_inline} { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*brk\s+#0x[0-9a-f]+.*\.size\s+sensitive_non_inline_function} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_sensitive_inline} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
index 95a8e8419e00..af6d86803576 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -34,3 +34,4 @@ int main() {
caller_no_checks() should NOT generate KCFI check (no_sanitize).
So a total of exactly 1 KCFI check in the entire program. */
/* { dg-final { scan-assembler-times {addl\t-4\(%r[ad]x\), %r1[01]d} 1 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {ldur\tw16, \[x[0-9]+, #-4\]} 1 { target aarch64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
index 97d964feebd3..0ced5c43ae92 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -27,3 +27,6 @@ int main() {
/* x86_64: All call sites should use -4 offset for KCFI type ID loads, even
with -falign-functions=16 (we're not using patchable entries here). */
/* { dg-final { scan-assembler {movl\t\$-?[0-9]+, %r10d\n\taddl\t-4\(%r[a-z0-9]+\), %r10d} { target x86_64-*-* } } } */
+
+/* AArch64: All call sites should use -4 offset. */
+/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
index 379356385a16..7a251cbdee3b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -28,6 +28,11 @@ int main() {
** movl \$0x[0-9a-f]+, %eax
*/
+/*
+** __cfi_test_function: { target aarch64*-*-* }
+** .word 0x[0-9a-f]+
+*/
+
/*
** main: { target x86_64-*-* }
** ...
@@ -35,4 +40,11 @@ int main() {
** ...
*/
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[x[0-9]+, #-4\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
index 06df3495bb23..3ed5d16c8e91 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -17,6 +17,11 @@ int main() {
** movl \$0x[0-9a-f]+, %eax
*/
+/*
+** __cfi_test_function: { target aarch64*-*-* }
+** .word 0x[0-9a-f]+
+*/
+
/*
** main: { target x86_64-*-* }
** ...
@@ -24,4 +29,11 @@ int main() {
** ...
*/
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[x[0-9]+, #-48\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
index ef87b135934b..e354914209e9 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -24,6 +24,11 @@ int main() {
** movl \$0x[0-9a-f]+, %eax
*/
+/*
+** __cfi_test_function: { target aarch64*-*-* }
+** .word 0x[0-9a-f]+
+*/
+
/*
** main: { target x86_64-*-* }
** ...
@@ -31,4 +36,11 @@ int main() {
** ...
*/
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[x[0-9]+, #-20\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
index 872814aa4171..7a1dc4fa0e07 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -25,6 +25,11 @@ int main() {
** movl \$0x[0-9a-f]+, %eax
*/
+/*
+** __cfi_test_function: { target aarch64*-*-* }
+** .word 0x[0-9a-f]+
+*/
+
/*
** main: { target x86_64-*-* }
** ...
@@ -32,4 +37,11 @@ int main() {
** ...
*/
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[x[0-9]+, #-16\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
index 04a9eb1fd206..7ac40719504b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -78,3 +78,24 @@ int test_non_tail_indirect_call(func_ptr_t handler, int x) {
/* Should have exactly 1 regular call (non-tail call case). */
/* { dg-final { scan-assembler-times {call\t\*%[a-z0-9]+} 1 { target x86_64-*-* } } } */
+
+/* Should have exactly 4 KCFI checks for indirect calls (load type ID from
+ -4 offset + compare). The actual-type scratch register avoids the call
+ target, so it is w9 for the sibcall targets (x16) and w16 otherwise; the
+ expected type is in the other of w16/w17. */
+/* { dg-final { scan-assembler-times {ldur\tw[0-9]+, \[x[0-9]+, #-4\]} 4 { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler-times {cmp\tw[0-9]+, w1[67]} 4 { target aarch64-*-* } } } */
+
+/* Should have exactly 4 trap instructions. */
+/* { dg-final { scan-assembler-times {brk\t#0x[0-9a-f]+} 4 { target aarch64-*-* } } } */
+
+/* Should have exactly 3 protected tail calls (br through register after
+ KCFI check). */
+/* { dg-final { scan-assembler-times {br\tx[0-9]+} 3 { target aarch64-*-* } } } */
+
+/* Should have exactly 1 regular call (non-tail call case). */
+/* { dg-final { scan-assembler-times {blr\tx[0-9]+} 1 { target aarch64-*-* } } } */
+
+/* Type ID loading should use mov + movk pattern for 32-bit constants. */
+/* { dg-final { scan-assembler {mov\tw17, #[0-9]+} { target aarch64-*-* } } } */
+/* { dg-final { scan-assembler {movk\tw17, #[0-9]+, lsl #16} { target aarch64-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
new file mode 100644
index 000000000000..252e63c85760
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
@@ -0,0 +1,43 @@
+/* Test AArch64 and ARM32 KCFI trap encoding in BRK/UDF instructions. */
+/* { dg-do compile { target aarch64*-*-* } } */
+
+void target_function(int x, char y) {
+}
+
+int main() {
+ /* Volatile so the indirect call (and its KCFI check) survive -O2. */
+ void (*volatile func_ptr)(int, char) = target_function;
+
+ /* This should generate trap with immediate encoding. */
+ func_ptr(42, 'a');
+
+ return 0;
+}
+
+/* Should have KCFI preamble. */
+/* { dg-final { scan-assembler "__cfi_target_function:" } } */
+
+/* AArch64 specific: Should have BRK instruction with proper ESR encoding
+ ESR format: 0x8000 | ((type_reg & 31) << 5) | (addr_reg & 31)
+
+ Test the ESR encoding by checking for the expected value.
+ Since we know this test uses x2, we expect
+ ESR = 0x8000 | (17<<5) | 2 = 0x8222 (33314)
+ */
+
+/*
+** main: { target aarch64*-*-* }
+** ...
+** ldur w16, \[x[0-9]+, #-4\]
+** mov w17, #[0-9]+
+** movk w17, #[0-9]+, lsl #16
+** cmp w16, w17
+** b\.eq .Lkcfi_call[0-9]+
+** .Lkcfi_trap[0-9]+:
+** brk #0x8222
+** .Lkcfi_call[0-9]+:
+** blr x2
+** ...
+*/
+
+/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
index 55c0829ccd7b..e92873e51321 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -18,6 +18,10 @@ int main() {
/* Should have exactly 2 trap labels in code. */
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ud2} 2 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*brk} 2 { target aarch64*-*-* } } } */
/* x86_64 should exactly 2 .kcfi_traps sections. */
/* { dg-final { scan-assembler-times {\.section\t\.kcfi_traps,"ao",@progbits,\.text} 2 { target x86_64-*-* } } } */
+
+/* AArch64 should NOT have .kcfi_traps section. */
+/* { dg-final { scan-assembler-not {\.section\t+\.kcfi_traps} { target aarch64*-*-* } } } */
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v17 6/7] arm: Add ARM 32-bit Kernel Control Flow Integrity implementation
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (4 preceding siblings ...)
2026-10-05 15:40 ` [PATCH v17 5/7] aarch64: Add AArch64 " Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-05 15:40 ` [PATCH v17 7/7] riscv: Add RISC-V " Kees Cook
6 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Implement ARM 32-bit KCFI backend:
- Use eor instructions for 32-bit immediate loading.
- Trap debugging through UDF instruction immediate encoding following
AArch64 BRK pattern for encoding registers with useful contents.
- Scratch register allocation uses ip by default since it is most
commonly available as a caller-saved register. When, due to register
pressure, ip is the call target register, use r3. Since r3 is already
caller-saved, the allocator will have already arranged to reload r3
after the call if it is needed again. However, if r3 is being used
as the 4th argument to the call, we must internally spill/reload
it. Also uses r3 with -ffixed-ip or -ffixed-r12.
Assembly Code Pattern for ARM 32-bit:
ldr ip, [target, #-4] ; Load actual type ID from preamble
eor ip, ip, #byte1 ; OR type id byte 1
eor ip, ip, #byte2 << 8 ; OR type id byte 2
eor ip, ip, #byte3 << 16 ; OR type id byte 3
eors ip, ip, #byte4 << 24 ; OR type id byte 4 and set if reg is 0
beq .Lkcfi_call ; Branch if typeids matched
.Lkcfi_trap: udf #udf_value ; Undefined instruction trap with encoding
.Lkcfi_call: blx/bx target ; Execute validated indirect transfer
UDF Immediate Encoding (following AArch64 ESR pattern):
- UDF instruction immediate encoding format:
0x8000 | ((ExpectedTypeReg & 31) << 5) | (TargetAddrReg & 31)
- ExpectedTypeReg indicates which register contains expected type (R12 = 12)
- TargetAddrReg indicates which register contains target address (0-15)
- Example: udf #33154 (0x817A) = expected type in R12, target address in R2
Build and run tested with Linux kernel ARCH=arm.
Assisted-by: LLM [tests]
gcc/ChangeLog:
* config/arm/arm-protos.h: Declare KCFI helpers.
* config/arm/arm.cc (arm_maybe_wrap_call_with_kcfi): New function.
(arm_maybe_wrap_call_value_with_kcfi): New function.
(arm_output_kcfi_insn): Emit KCFI assembly.
* config/arm/arm.md: Add KCFI RTL patterns and hook expansion.
* doc/invoke.texi: Document arm32 nuances.
gcc/testsuite/ChangeLog:
* gcc.dg/kcfi/kcfi-adjacency.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-basics.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-call-sharing.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-complex-addressing.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-direct-call-shapes.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-move-preservation.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-no-sanitize-inline.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-no-sanitize.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-offset-validation.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-patchable-entry-only.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-patchable-large.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-patchable-medium.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-patchable-prefix-only.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-tail-calls.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-trap-encoding.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-trap-section.c: Add arm patterns.
* gcc.dg/kcfi/kcfi-arm-fixed-ip.c: New test.
* gcc.dg/kcfi/kcfi-arm-fixed-r12.c: New test.
* gcc.dg/kcfi/kcfi-arm-sibcall-r3.c: New test.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/arm/arm-protos.h | 4 +
gcc/config/arm/arm.md | 62 +++++
gcc/config/arm/arm.cc | 217 +++++++++++++++++-
gcc/doc/invoke.texi | 27 ++-
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 38 +++
gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-ip.c | 15 ++
.../gcc.dg/kcfi/kcfi-arm-fixed-r12.c | 15 ++
.../gcc.dg/kcfi/kcfi-arm-sibcall-r3.c | 50 ++++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 42 +++-
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 17 +-
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 22 ++
.../gcc.dg/kcfi/kcfi-direct-call-shapes.c | 3 +
.../gcc.dg/kcfi/kcfi-move-preservation.c | 42 +++-
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 5 +
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 3 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 5 +
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 9 +-
.../gcc.dg/kcfi/kcfi-patchable-large.c | 12 +-
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 12 +-
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 12 +-
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 20 ++
.../gcc.dg/kcfi/kcfi-trap-encoding.c | 48 +++-
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 5 +-
23 files changed, 659 insertions(+), 26 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-ip.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-r12.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-arm-sibcall-r3.c
diff --git a/gcc/config/arm/arm-protos.h b/gcc/config/arm/arm-protos.h
index b607cdb3fcc0..295647a0efeb 100644
--- a/gcc/config/arm/arm-protos.h
+++ b/gcc/config/arm/arm-protos.h
@@ -608,6 +608,10 @@ void arm_initialize_isa (sbitmap, const enum isa_feature *);
const char * arm_gen_far_branch (rtx *, int, const char * , const char *);
+rtx arm_maybe_wrap_call_with_kcfi (rtx, rtx);
+rtx arm_maybe_wrap_call_value_with_kcfi (rtx, rtx);
+const char *arm_output_kcfi_insn (rtx_insn *, rtx *);
+
bool arm_mve_immediate_check(rtx, machine_mode, bool);
opt_machine_mode arm_mve_data_mode (scalar_mode, poly_uint64);
diff --git a/gcc/config/arm/arm.md b/gcc/config/arm/arm.md
index febff17df0bf..a8cdb76f3ade 100644
--- a/gcc/config/arm/arm.md
+++ b/gcc/config/arm/arm.md
@@ -8635,6 +8635,7 @@
else
{
pat = gen_call_internal (operands[0], operands[1], operands[2]);
+ pat = arm_maybe_wrap_call_with_kcfi (pat, XEXP (operands[0], 0));
arm_emit_call_insn (pat, XEXP (operands[0], 0), false);
}
@@ -8693,6 +8694,20 @@
}
)
+;; KCFI indirect call - KCFI wraps just the call pattern
+(define_insn "*kcfi_call_reg"
+ [(kcfi (call (mem:SI (match_operand:SI 0 "s_register_operand" "r"))
+ (match_operand 1 "" ""))
+ (match_operand 2 "const_int_operand"))
+ (use (match_operand 3 "" ""))
+ (clobber (reg:SI LR_REGNUM))]
+ "TARGET_32BIT && !SIBLING_CALL_P (insn) && arm_ccfsm_state == 0"
+{
+ return arm_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "40")])
+
(define_insn "*call_reg_armv5"
[(call (mem:SI (match_operand:SI 0 "s_register_operand" "r"))
(match_operand 1 "" ""))
@@ -8759,6 +8774,7 @@
{
pat = gen_call_value_internal (operands[0], operands[1],
operands[2], operands[3]);
+ pat = arm_maybe_wrap_call_value_with_kcfi (pat, XEXP (operands[1], 0));
arm_emit_call_insn (pat, XEXP (operands[1], 0), false);
}
@@ -8805,6 +8821,21 @@
}
}")
+;; KCFI indirect call_value - KCFI wraps just the call pattern
+(define_insn "*kcfi_call_value_reg"
+ [(set (match_operand 0 "" "")
+ (kcfi (call (mem:SI (match_operand:SI 1 "s_register_operand" "r"))
+ (match_operand 2 "" ""))
+ (match_operand 3 "const_int_operand")))
+ (use (match_operand 4 "" ""))
+ (clobber (reg:SI LR_REGNUM))]
+ "TARGET_32BIT && !SIBLING_CALL_P (insn) && arm_ccfsm_state == 0"
+{
+ return arm_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "40")])
+
(define_insn "*call_value_reg_armv5"
[(set (match_operand 0 "" "")
(call (mem:SI (match_operand:SI 1 "s_register_operand" "r"))
@@ -8907,6 +8938,7 @@
operands[2] = const0_rtx;
pat = gen_sibcall_internal (operands[0], operands[1], operands[2]);
+ pat = arm_maybe_wrap_call_with_kcfi (pat, XEXP (operands[0], 0));
arm_emit_call_insn (pat, operands[0], true);
DONE;
}"
@@ -8941,11 +8973,26 @@
pat = gen_sibcall_value_internal (operands[0], operands[1],
operands[2], operands[3]);
+ pat = arm_maybe_wrap_call_value_with_kcfi (pat, XEXP (operands[1], 0));
arm_emit_call_insn (pat, operands[1], true);
DONE;
}"
)
+;; KCFI sibling call - KCFI wraps just the call pattern
+(define_insn "*kcfi_sibcall_insn"
+ [(kcfi (call (mem:SI (match_operand:SI 0 "s_register_operand" "Cs"))
+ (match_operand 1 "" ""))
+ (match_operand 2 "const_int_operand"))
+ (return)
+ (use (match_operand 3 "" ""))]
+ "TARGET_32BIT && SIBLING_CALL_P (insn) && arm_ccfsm_state == 0"
+{
+ return arm_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "40")])
+
(define_insn "*sibcall_insn"
[(call (mem:SI (match_operand:SI 0 "call_insn_operand" "Cs, US"))
(match_operand 1 "" ""))
@@ -8966,6 +9013,21 @@
[(set_attr "type" "call")]
)
+;; KCFI sibling call with return value - KCFI wraps just the call pattern
+(define_insn "*kcfi_sibcall_value_insn"
+ [(set (match_operand 0 "" "")
+ (kcfi (call (mem:SI (match_operand:SI 1 "s_register_operand" "Cs"))
+ (match_operand 2 "" ""))
+ (match_operand 3 "const_int_operand")))
+ (return)
+ (use (match_operand 4 "" ""))]
+ "TARGET_32BIT && SIBLING_CALL_P (insn) && arm_ccfsm_state == 0"
+{
+ return arm_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "40")])
+
(define_insn "*sibcall_value_insn"
[(set (match_operand 0 "" "")
(call (mem:SI (match_operand:SI 1 "call_insn_operand" "Cs,US"))
diff --git a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc
index 33b64995442f..6203459f3bc9 100644
--- a/gcc/config/arm/arm.cc
+++ b/gcc/config/arm/arm.cc
@@ -77,6 +77,8 @@
#include "aarch-common-protos.h"
#include "machmode.h"
#include "arm-builtins.h"
+#include "kcfi.h"
+#include "flags.h"
/* This file should be included last. */
#include "target-def.h"
@@ -23171,17 +23173,19 @@ thumb_force_lr_save (void)
we do have an indirect tailcall happening in this
particular case. */
static bool
-is_indirect_tailcall_p (rtx call)
+is_indirect_tailcall_p (rtx_insn *call)
{
- rtx pat = PATTERN (call);
-
- /* Indirect tail call. */
- pat = XVECEXP (pat, 0, 0);
- if (GET_CODE (pat) == SET)
- pat = SET_SRC (pat);
+ /* Use the generic accessor to reach the CALL expression so that we see
+ through any wrapper, in particular the KCFI call wrapper
+ (kcfi (call ...) typeid). An open-coded structural walk would peel one
+ level too few and misclassify a KCFI-instrumented indirect tail call as
+ direct, leaving r3 free to be used as stack-alignment padding and thus
+ clobbering the tail-call target. */
+ rtx pat = get_call_rtx_from (call);
- pat = XEXP (XEXP (pat, 0), 0);
- return REG_P (pat);
+ /* PAT is (call (mem ADDR) ...); the tail call is indirect when the target
+ ADDR is held in a register. */
+ return REG_P (XEXP (XEXP (pat, 0), 0));
}
/* Return true if r3 is used by any of the tail call insns in the
@@ -36107,6 +36111,201 @@ arm_mode_base_reg_class (machine_mode mode)
return MODE_BASE_REG_REG_CLASS (mode);
}
+/* Apply KCFI wrapping to call pattern if needed. PAT is the RTL call
+ pattern to potentially wrap with KCFI instrumentation. ADDR is the
+ call target address RTL expression. Returns the possibly modified
+ call pattern with KCFI wrapper applied for indirect calls. */
+
+rtx
+arm_maybe_wrap_call_with_kcfi (rtx pat, rtx addr)
+{
+ /* Only indirect calls need KCFI instrumentation. */
+ if (REG_P (addr))
+ {
+ rtx kcfi_type_rtx = kcfi_get_type_id_for_expanding_gimple_call ();
+ if (kcfi_type_rtx)
+ {
+ /* Extract the CALL from the PARALLEL and wrap it with KCFI. */
+ rtx call_rtx = XVECEXP (pat, 0, 0);
+ rtx kcfi_call = gen_rtx_KCFI (VOIDmode, call_rtx, kcfi_type_rtx);
+
+ /* Replace the CALL in the PARALLEL with the KCFI-wrapped call. */
+ XVECEXP (pat, 0, 0) = kcfi_call;
+ }
+ }
+ return pat;
+}
+
+/* Apply KCFI wrapping to call_value pattern if needed. PAT is the RTL
+ call_value pattern to potentially wrap with KCFI instrumentation. ADDR
+ is the call target address RTL expression. Returns the possibly modified
+ call pattern with KCFI wrapper applied for indirect calls. */
+
+rtx
+arm_maybe_wrap_call_value_with_kcfi (rtx pat, rtx addr)
+{
+ /* Only indirect calls need KCFI instrumentation. */
+ if (REG_P (addr))
+ {
+ rtx kcfi_type_rtx = kcfi_get_type_id_for_expanding_gimple_call ();
+ if (kcfi_type_rtx)
+ {
+ /* Extract the SET from the PARALLEL and wrap its CALL with KCFI. */
+ rtx set_rtx = XVECEXP (pat, 0, 0);
+ rtx call_rtx = SET_SRC (set_rtx);
+ rtx kcfi_call = gen_rtx_KCFI (VOIDmode, call_rtx, kcfi_type_rtx);
+
+ /* Replace the CALL in the SET with the KCFI-wrapped call. */
+ SET_SRC (set_rtx) = kcfi_call;
+ }
+ }
+ return pat;
+}
+
+/* Output the assembly for a KCFI checked call instruction. INSN is the
+ RTL instruction being processed. OPERANDS is the array of RTL operands
+ where operands[0] is the call target register, operands[2] is the KCFI
+ type ID constant. Returns an empty string as all output is handled by
+ direct assembly generation. */
+
+const char *
+arm_output_kcfi_insn (rtx_insn *insn, rtx *operands)
+{
+ /* KCFI type id. */
+ uint32_t type_id = UINTVAL (operands[2]);
+
+ /* Calculate typeid offset from call target. */
+ HOST_WIDE_INT offset = -kcfi_get_typeid_offset ();
+
+ /* Get unique label number for this KCFI check. */
+ int labelno = kcfi_next_labelno ();
+
+ /* Generate custom label names. */
+ char trap_name[32];
+ char call_name[32];
+ ASM_GENERATE_INTERNAL_LABEL (trap_name, "Lkcfi_trap", labelno);
+ ASM_GENERATE_INTERNAL_LABEL (call_name, "Lkcfi_call", labelno);
+
+ rtx temp_operands[6];
+
+ /* Normally we can use r12 as our scratch register. */
+ unsigned scratch_reg_num = IP_REGNUM;
+ /* If register pressure has made r12 our target register, we need to pick
+ a different register. We don't want to spill our target register
+ because on reload at the end of the KCFI check, we'd be producing
+ the very kind of call gadget we were trying to protect against:
+ "pop %target; call %target". In this case, use r3 as our scratch
+ register. But since r3 may be used for function arguments, we need
+ to check if it is being used for that and only spill/reload if that
+ happens. Any spill/reload of r3 due to making a call will already
+ have been managed by the register allocator, so we only have to care
+ about not clobbering the argument value it may be carrying into the
+ call here. Also use r3 when r12 is a fixed register. */
+ if (REGNO (operands[0]) == scratch_reg_num
+ || fixed_regs[scratch_reg_num])
+ scratch_reg_num = LAST_ARG_REGNUM;
+ rtx scratch_reg = gen_rtx_REG (SImode, scratch_reg_num);
+
+ /* We only need to spill r3 if it's actually used by the call. */
+ bool need_spill = (scratch_reg_num == LAST_ARG_REGNUM)
+ && reg_overlap_mentioned_p (scratch_reg, insn);
+
+ /* Calculate trap immediate. */
+ unsigned addr_reg_num = REGNO (operands[0]);
+ /* The scratch register is always clobbered by eor seq: use 0x1F. */
+ unsigned udf_immediate = 0x8000 | (0x1F << 5) | (addr_reg_num & 31);
+
+ /* Spill if needed. */
+ if (need_spill)
+ output_asm_insn ("push\t{%0}", &scratch_reg);
+
+ /* In Thumb-2 mode, function pointer values carry the Thumb bit (bit 0)
+ set, which would skew the typeid load by one byte. Mask it off into
+ the scratch register and load through that. In ARM mode the Thumb
+ bit is always 0, so the BIC is unnecessary. */
+ rtx addr_reg = operands[0];
+ if (TARGET_THUMB)
+ {
+ temp_operands[0] = scratch_reg;
+ temp_operands[1] = operands[0];
+ output_asm_insn ("bic\t%0, %1, #1", temp_operands);
+ addr_reg = scratch_reg;
+ }
+
+ /* Load actual type from memory into scratch register. */
+ rtx mem_op = gen_rtx_MEM (SImode,
+ gen_rtx_PLUS (SImode, addr_reg,
+ GEN_INT (offset)));
+ temp_operands[0] = scratch_reg;
+ temp_operands[1] = mem_op;
+ output_asm_insn ("ldr\t%0, %1", temp_operands);
+
+ /* Set up operands for EOR instructions - source and dest are the same. */
+ temp_operands[0] = scratch_reg;
+ temp_operands[1] = scratch_reg;
+
+ /* XOR with type_id byte 0. */
+ temp_operands[2] = GEN_INT (type_id & 0xFF);
+ output_asm_insn ("eor\t%0, %1, %2", temp_operands);
+
+ /* XOR with type_id byte 1 << 8. */
+ temp_operands[2] = GEN_INT (((type_id >> 8) & 0xFF) << 8);
+ output_asm_insn ("eor\t%0, %1, %2", temp_operands);
+
+ /* XOR with type_id byte 2 << 16. */
+ temp_operands[2] = GEN_INT (((type_id >> 16) & 0xFF) << 16);
+ output_asm_insn ("eor\t%0, %1, %2", temp_operands);
+
+ /* EORS with type_id byte 3 << 24 (sets flags). */
+ temp_operands[2] = GEN_INT (((type_id >> 24) & 0xFF) << 24);
+ output_asm_insn ("eors\t%0, %1, %2", temp_operands);
+
+ /* Reload if needed. */
+ if (need_spill)
+ output_asm_insn ("pop\t{%0}", &scratch_reg);
+
+ /* Output conditional branch to call label. */
+ fputs ("\tbeq\t", asm_out_file);
+ assemble_name (asm_out_file, call_name);
+ fputc ('\n', asm_out_file);
+
+ /* Output trap label and UDF instruction. */
+ ASM_OUTPUT_LABEL (asm_out_file, trap_name);
+ temp_operands[0] = GEN_INT (udf_immediate);
+ output_asm_insn ("udf\t%0", temp_operands);
+
+ /* Output pass/call label. */
+ ASM_OUTPUT_LABEL (asm_out_file, call_name);
+
+ /* Call or tail call instruction. */
+ if (SIBLING_CALL_P (insn))
+ output_asm_insn ("bx\t%0", operands);
+ else
+ output_asm_insn ("blx\t%0", operands);
+
+ return "";
+}
+
+/* KCFI is supported in ARM and Thumb-2 modes. Thumb-1 lacks the wide
+ immediate encodings for EOR, the UDF imm16 form, and BIC with immediate
+ that the check sequence relies on. */
+
+static bool
+arm_kcfi_supported_p (void)
+{
+ if (TARGET_THUMB1)
+ {
+ error ("%<-fsanitize=kcfi%> is not supported in Thumb-1 mode; "
+ "build with %<-marm%> or for an architecture that supports "
+ "Thumb-2");
+ return false;
+ }
+ return true;
+}
+
+#undef TARGET_KCFI_SUPPORTED
+#define TARGET_KCFI_SUPPORTED arm_kcfi_supported_p
+
#undef TARGET_DOCUMENTATION_NAME
#define TARGET_DOCUMENTATION_NAME "ARM"
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index ee8b48a68517..f0b1314d1731 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -17611,6 +17611,26 @@ trap is taken, allowing the kernel to identify both the KCFI violation
and the involved registers for detailed diagnostics (eliminating the need
for a separate @code{.kcfi_traps} section as used on x86_64).
+On ARM 32-bit, KCFI type identifiers are emitted as a @code{.word ID}
+directive (a 32-bit constant) before the function entry. ARM's
+natural 4-byte instruction alignment eliminates the need for additional
+alignment NOPs. When used with @option{-fpatchable-function-entry}, the
+type identifier is placed before any prefix NOPs. The runtime check uses
+@code{ip} (@code{r12}) as a scratch register, falling back to @code{r3} when
+@code{ip} is the call target or is a fixed register (spilling and restoring
+@code{r3} only when it carries a live argument). In Thumb-2 mode a
+@code{bic} first clears the function-pointer Thumb bit before the type
+identifier is loaded. The loaded type identifier is compared against the
+expected one using a sequence of four @code{eor} instructions (one per byte
+of the 32-bit value), the last of which sets the condition flags. Type
+mismatches trigger a @code{udf} instruction with an immediate value that
+encodes both the expected type register index and the target address
+register index in the format @code{0x8000 | (type_reg << 5) | addr_reg}. This
+encoding is captured in the UDF immediate field when the trap is taken,
+allowing the kernel to identify both the KCFI violation and the involved
+registers for detailed diagnostics (eliminating the need for a separate
+@code{.kcfi_traps} section as used on x86_64).
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
@@ -17625,7 +17645,12 @@ For the same reason, the location of the type identifier relative to a
function entry point is part of a whole-program ABI: a call site computes
that offset from its own translation unit's settings, so every translation
unit that may be reached through an instrumented indirect call must agree on
-it.
+it. On ARM this additionally requires a uniform instruction set state: the
+prefix-NOP size differs between ARM (4 bytes) and Thumb-2 (2 bytes), and
+Thumb function pointers carry the Thumb bit in bit 0, both of which feed the
+type-identifier location. KCFI therefore does not support mixing ARM and
+Thumb-2 instrumented code in a single program; all such translation units
+must be built in the same mode.
Use @option{-fdump-ipa-kcfi-details} to examine the computed type identifier
hashes and their corresponding mangled type strings during compilation.
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
index 47a33bf6393b..63fade749ddd 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -83,4 +83,42 @@ __attribute__((noinline)) void test_conditional_call(int flag) {
(same register number) means the target was overwritten by the load. */
/* { dg-final { scan-assembler-not {ldur\tw([0-9]+), \[x\1,} { target aarch64*-*-* } } } */
+/*
+** test_complex_args: { target arm_nothumb }
+** ...
+** push \{r3\}
+** ldr r3, \[ip, #-4\]
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eors r3, r3, #[0-9]+
+** pop \{r3\}
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\1:
+** bx ip
+** ...
+*/
+
+/* Thumb-2 inserts a BIC to strip the Thumb bit before the typeid load. */
+/*
+** test_complex_args: { target arm_thumb2 }
+** ...
+** push \{r3\}
+** bic r3, ip, #1
+** ldr r3, \[r3, #-4\]
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eors r3, r3, #[0-9]+
+** pop \{r3\}
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\1:
+** bx ip
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-ip.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-ip.c
new file mode 100644
index 000000000000..bb76078aa800
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-ip.c
@@ -0,0 +1,15 @@
+/* Test that KCFI works with -ffixed-ip on ARM by using r3 fallback. */
+/* { dg-do compile { target arm*-*-* } } */
+/* { dg-additional-options "-ffixed-ip" } */
+
+void target_function(void) {}
+
+int main() {
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+ return 0;
+}
+
+/* Should use r3 instead of ip for scratch register when ip is fixed. */
+/* { dg-final { scan-assembler "ldr\tr3, \\\[r\[0-9\]+, #-4\\\]" } } */
+/* { dg-final { scan-assembler "eor\tr3, r3, #\[0-9\]+" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-r12.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-r12.c
new file mode 100644
index 000000000000..dd3f1f001f5b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-fixed-r12.c
@@ -0,0 +1,15 @@
+/* Test that KCFI works with -ffixed-r12 on ARM by using r3 fallback. */
+/* { dg-do compile { target arm*-*-* } } */
+/* { dg-additional-options "-ffixed-r12" } */
+
+void target_function(void) {}
+
+int main() {
+ void (*func_ptr)(void) = target_function;
+ func_ptr();
+ return 0;
+}
+
+/* Should use r3 instead of ip for scratch register when ip is fixed. */
+/* { dg-final { scan-assembler "ldr\tr3, \\\[r\[0-9\]+, #-4\\\]" } } */
+/* { dg-final { scan-assembler "eor\tr3, r3, #\[0-9\]+" } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-sibcall-r3.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-sibcall-r3.c
new file mode 100644
index 000000000000..dd1833d7b37b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-arm-sibcall-r3.c
@@ -0,0 +1,50 @@
+/* Regression test for an arm32 KCFI indirect tail-call miscompile.
+
+ When a function makes a KCFI-instrumented indirect sibling call whose
+ target is held in r3, and the prologue needs a register for 8-byte stack
+ alignment padding while every callee-saved register is already in use,
+ arm_get_frame_offsets used to pick r3 as the padding register. r3 was
+ then pushed in the prologue and restored to its stale entry value by the
+ epilogue, clobbering the tail-call target so "bx r3" branched into
+ garbage (observed as a boot crash in cpuhp_invoke_callback).
+
+ The root cause was is_indirect_tailcall_p failing to look through the
+ KCFI call wrapper, so any_sibcall_could_use_r3 wrongly reported that no
+ sibcall could use r3 and the padding logic was free to steal it.
+
+ This function is shaped to occupy all callee-saved registers across a
+ call while keeping the indirect tail-call target in r3 and requiring
+ alignment padding, which is exactly the situation that exposed the bug.
+ r3 must NOT appear in the prologue push or epilogue pop. */
+
+/* { dg-do compile { target arm*-*-* } } */
+/* { dg-additional-options "-O2 -marm -mgeneral-regs-only" } */
+
+typedef int (*cb1_t) (unsigned int cpu);
+struct step { cb1_t single; cb1_t teardown; int multi; int x[16]; };
+extern struct step *get_step (int state);
+extern void barrier (void);
+extern void note (int);
+
+int invoke (unsigned int cpu, int state, int bringup)
+{
+ struct step *step = get_step (state);
+ int a = step->x[0], b = step->x[1], c = step->x[2];
+ int d = step->x[3], e = step->x[4], f = step->x[5];
+ cb1_t cb = bringup ? step->single : step->teardown;
+ barrier ();
+ if (a) note (a);
+ if (b) note (b);
+ if (c) note (c);
+ if (d) note (d);
+ if (e) note (e);
+ if (f) note (f);
+ return cb (cpu);
+}
+
+/* The indirect KCFI tail call goes through r3. */
+/* { dg-final { scan-assembler {bx\tr3} } } */
+/* r3 holds the tail-call target, so it must never be saved/restored around
+ the body; doing so would clobber the target across the epilogue. */
+/* { dg-final { scan-assembler-not {push\t.r3,} } } */
+/* { dg-final { scan-assembler-not {pop\t.r3,} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
index 7f1c45f1fe94..a917e1c70089 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -59,8 +59,8 @@ int main() {
/* x86_64: Verify type ID in preamble (after NOPs, before function label) */
/* { dg-final { scan-assembler {__cfi_regular_function:\n\t+nop\n.*\n\t+movl\t+\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
-/* AArch64: Verify type ID word in preamble. */
-/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target aarch64*-*-* } } } */
+/* AArch64, ARM32: Verify type ID word in preamble. */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target aarch64*-*-* arm*-*-* } } } */
/*
** static_caller: { target x86_64-*-* }
@@ -94,6 +94,40 @@ int main() {
** ...
*/
+/*
+** static_caller: { target arm_nothumb }
+** ...
+** ldr ip, \[(r[0-9]+), #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\2:
+** blx \1
+** ...
+*/
+
+/* Thumb-2 inserts a BIC to strip the Thumb bit before the typeid load. */
+/*
+** static_caller: { target arm_thumb2 }
+** ...
+** bic ip, (r[0-9]+), #1
+** ldr ip, \[ip, #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\2:
+** blx \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
/* Extern functions should NOT get KCFI preambles. */
@@ -112,5 +146,5 @@ int main() {
__kcfi_typeid_ symbols. */
/* { dg-final { scan-assembler-not {__kcfi_typeid_external_func_int} } } */
-/* AArch64 should NOT have trap section (use immediate instructions instead). */
-/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
+/* AArch64, ARM32 should NOT have trap section (use immediate instructions instead). */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
index c402a1678a73..ce5e17134929 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -60,21 +60,32 @@ int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *
1. KCFI check for is_visible call with is_visible type ID A.
2. KCFI check for is_bin_visible and is_bin_visible_again call with type ID B. */
-/* Verify we have TWO different KCFI check sequences. */
+/* Verify we have TWO different KCFI check sequences (except on arm, which
+ due to heavy register pressure ends up generating an unmergeable series
+ of instructions: the call target registers differ). */
/* Each check should have different type ID constants. */
/* x86: { dg-final { scan-assembler-times {movl\s+\$-?[0-9]+,\s+%r10d} 2 { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-times {mov\s+w17, #[0-9]+} 2 { target aarch64*-*-* } } } */
+/* ARM 32-bit: In Thumb-2 the load base is ip (post-Thumb-bit-strip), in ARM
+ mode it is the call target register or lr; accept any. */
+/* { dg-final { scan-assembler-times {ldr\s+ip, \[(?:r[0-9]+|lr|ip), #-4\]} 3 { target arm*-*-* } } } */
-/* Verify the checks use DIFFERENT type IDs (not shared).
+/* Verify the checks use DIFFERENT type IDs (not shared, except arm: see above).
We should NOT see the same type ID used twice - that would indicate
unmerged sharing. */
/* x86: { dg-final { scan-assembler-not {movl\s+\$(-?[0-9]+),\s+%r10d.*movl\s+\$\1,\s+%r10d} { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-not {mov\s+w17, #([0-9]+).*mov\s+w17, #\1} { target aarch64*-*-* } } } */
+/* ARM 32-bit: the first check (to .Lkcfi_call0) must not have its type ID
+ reused by a later check. Each check uses a single scratch register,
+ ip or r3. */
+/* { dg-final { scan-assembler-not {eor\s+(ip|r3), \1, #([0-9]+)\n\teor\s+\1, \1, #([0-9]+)\n\teor\s+\1, \1, #([0-9]+)\n\teors\s+\1, \1, #([0-9]+)\n(?:\tpop\t\{r3\}\n)?\tbeq\t\.Lkcfi_call0\n.*eor\s+(?:ip|r3), (?:ip|r3), #\2\n\teor\s+(?:ip|r3), (?:ip|r3), #\3\n\teor\s+(?:ip|r3), (?:ip|r3), #\4\n\teors\s+(?:ip|r3), (?:ip|r3), #\5} { target arm*-*-* } } } */
/* Verify expected number of traps. */
/* x86: { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
+/* ARM 32-bit: { dg-final { scan-assembler-times {udf\s+#[0-9]+} 3 { target arm*-*-* } } } */
-/* Verify 2 separate call sites. */
+/* Verify 2 separate call sites (except arm). */
/* x86: { dg-final { scan-assembler-times {jmp\s+\*%[a-z0-9]+} 2 { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-times {br\tx[0-9]+} 2 { target aarch64*-*-* } } } */
+/* ARM 32-bit: { dg-final { scan-assembler-times {bx\s+(?:r[0-9]+|lr)} 3 { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
index 0b23caa49ebb..5eec800e108d 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -4,6 +4,9 @@
of a simple register. */
/* { dg-do compile } */
/* { dg-additional-options "-O2" } */
+/* The force_r3_spill check verifies ARM-mode register-pressure codegen,
+ which differs in Thumb mode. Force ARM mode where supported. */
+/* { dg-additional-options "-marm" { target arm*-*-* } } */
struct function_table {
int (*callback1)(int);
@@ -162,4 +165,23 @@ int main() {
** ...
*/
+/* Looking for r3 fall-back due to register pressure. */
+/*
+** force_r3_spill: { target arm*-*-* }
+** ...
+** push \{r3\}
+** ldr r3, \[(ip), #-4\]
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eor r3, r3, #[0-9]+
+** eors r3, r3, #[0-9]+
+** pop \{r3\}
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\2:
+** blx \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
index 6b4698447b7b..2daf5aa94bcf 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
@@ -30,3 +30,6 @@ void caller(int sel, void (*ip)(int), int x) {
/* The typeid scratch register avoids the call target, so it is not always
w16 (a sibcall target lands in x16/x17, forcing the load into w9). */
/* { dg-final { scan-assembler-times {ldur\tw[0-9]+, \[x[0-9]+, #-4\]} 1 { target aarch64*-*-* } } } */
+/* The arm load base is the call target register in ARM mode and ip in
+ Thumb-2 (after the Thumb-bit strip); accept either. */
+/* { dg-final { scan-assembler-times {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} 1 { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
index ad6130c9d323..fac0e3aff0e4 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
@@ -74,7 +74,45 @@ int main(void)
** ...
*/
+/*
+** indirect_call: { target arm_nothumb }
+** ...
+** mov (r[0-9]+), r0
+** ...
+** ldr ip, \[\1, #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\2:
+** bx \1
+** ...
+*/
+
+/* Thumb-2 inserts a BIC to strip the Thumb bit before the typeid load. */
+/*
+** indirect_call: { target arm_thumb2 }
+** ...
+** mov (r[0-9]+), r0
+** ...
+** bic ip, \1, #1
+** ldr ip, \[ip, #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call([0-9]+)
+** .Lkcfi_trap[0-9]+:
+** udf #[0-9]+
+** .Lkcfi_call\2:
+** bx \1
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
-/* AArch64 should NOT have trap section (use immediate instructions instead). */
-/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* } } } */
+/* AArch64, ARM32 should NOT have trap section (use immediate instructions instead). */
+/* { dg-final { scan-assembler-not {\.kcfi_traps} { target aarch64*-*-* arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
index 98591ed9b145..279207a85cdc 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -76,15 +76,20 @@ int main(void)
/* Verify correct number of KCFI checks: exactly 2 */
/* { dg-final { scan-assembler-times {ud2} 2 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-times {udf\s+#[0-9]+} 2 { target arm*-*-* } } } */
/* Positive controls: these should have KCFI checks. */
/* { dg-final { scan-assembler {normal_function:.*ud2.*\.size\s+normal_function} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler {wrap_normal_inline:.*ud2.*\.size\s+wrap_normal_inline} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler {normal_function:.*brk\s+#0x[0-9a-f]+.*\.size\s+normal_function} { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler {wrap_normal_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_normal_inline} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler {normal_function:.*udf\t#[0-9]+.*\.size\s+normal_function} { target arm*-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*udf\t#[0-9]+.*\.size\s+wrap_normal_inline} { target arm*-*-* } } } */
/* Negative controls: these should NOT have KCFI checks. */
/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ud2.*\.size\s+sensitive_non_inline_function} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ud2.*\.size\s+wrap_sensitive_inline} { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*brk\s+#0x[0-9a-f]+.*\.size\s+sensitive_non_inline_function} { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_sensitive_inline} { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:[^\n]*udf\t#[0-9]+[^\n]*\.size\tsensitive_non_inline_function} { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:[^\n]*udf\t#[0-9]+[^\n]*\.size\twrap_sensitive_inline} { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
index af6d86803576..9355d633301b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -35,3 +35,6 @@ int main() {
So a total of exactly 1 KCFI check in the entire program. */
/* { dg-final { scan-assembler-times {addl\t-4\(%r[ad]x\), %r1[01]d} 1 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times {ldur\tw16, \[x[0-9]+, #-4\]} 1 { target aarch64-*-* } } } */
+/* In Thumb-2 the load base is ip (post-Thumb-bit-strip), in ARM mode it is
+ the call target register; accept either. */
+/* { dg-final { scan-assembler-times {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} 1 { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
index 0ced5c43ae92..cb1eb7c7ecc6 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -30,3 +30,8 @@ int main() {
/* AArch64: All call sites should use -4 offset. */
/* { dg-final { scan-assembler {ldur\tw16, \[x[0-9]+, #-4\]} { target aarch64*-*-* } } } */
+
+/* ARM 32-bit: All call sites should use -4 offset with EOR sequence.
+ In Thumb-2 the load base is ip (after the Thumb-bit strip); in ARM mode it
+ is the call target register directly. Accept either. */
+/* { dg-final { scan-assembler {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
index 7a251cbdee3b..612140a0f509 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -29,7 +29,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -47,4 +47,11 @@ int main() {
** ...
*/
+/*
+** main: { target arm*-*-* }
+** ...
+** ldr (r[0-9]+|ip), \[(r[0-9]+|ip), #-4\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
index 3ed5d16c8e91..7f7ada17cc14 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -2,6 +2,9 @@
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=11,11" } */
/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
+/* The typeid offset depends on patchable NOP size (4 bytes in ARM mode,
+ 2 bytes in Thumb mode). Force ARM mode for deterministic offsets. */
+/* { dg-additional-options "-marm" { target arm*-*-* } } */
void test_function(void) {
}
@@ -18,7 +21,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -36,4 +39,11 @@ int main() {
** ...
*/
+/*
+** main: { target arm*-*-* }
+** ...
+** ldr (r[0-9]+|ip), \[(r[0-9]+|ip), #-48\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
index e354914209e9..0a1f4e20851f 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -2,6 +2,9 @@
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=8,4" } */
/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
+/* The typeid offset depends on patchable NOP size (4 bytes in ARM mode,
+ 2 bytes in Thumb mode). Force ARM mode for deterministic offsets. */
+/* { dg-additional-options "-marm" { target arm*-*-* } } */
void test_function(void) {
}
@@ -25,7 +28,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -43,4 +46,11 @@ int main() {
** ...
*/
+/*
+** main: { target arm*-*-* }
+** ...
+** ldr ip, \[r[0-9]+, #-20\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
index 7a1dc4fa0e07..548a2f5d8b84 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -2,6 +2,9 @@
/* { dg-do compile } */
/* { dg-additional-options "-fpatchable-function-entry=3,3" } */
/* { dg-additional-options "-falign-functions=16" { target x86_64-*-* } } */
+/* The typeid offset depends on patchable NOP size (4 bytes in ARM mode,
+ 2 bytes in Thumb mode). Force ARM mode for deterministic offsets. */
+/* { dg-additional-options "-marm" { target arm*-*-* } } */
void test_function(void) {
}
@@ -26,7 +29,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -44,4 +47,11 @@ int main() {
** ...
*/
+/*
+** main: { target arm*-*-* }
+** ...
+** ldr (r[0-9]+|ip), \[(r[0-9]+|ip), #-16\]
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
index 7ac40719504b..544ae7807297 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -99,3 +99,23 @@ int test_non_tail_indirect_call(func_ptr_t handler, int x) {
/* Type ID loading should use mov + movk pattern for 32-bit constants. */
/* { dg-final { scan-assembler {mov\tw17, #[0-9]+} { target aarch64-*-* } } } */
/* { dg-final { scan-assembler {movk\tw17, #[0-9]+, lsl #16} { target aarch64-*-* } } } */
+
+/* Should have exactly 4 KCFI checks for indirect calls (load type ID from
+ -4 offset + EOR sequence). */
+/* In Thumb-2 the load base is ip (post-Thumb-bit-strip), in ARM mode it is
+ the call target register; accept either. */
+/* { dg-final { scan-assembler-times {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} 4 { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-times {eors\tip, ip, #[0-9]+} 4 { target arm*-*-* } } } */
+
+/* Should have exactly 4 trap instructions. */
+/* { dg-final { scan-assembler-times {udf\t#[0-9]+} 4 { target arm*-*-* } } } */
+
+/* Should have exactly 3 protected tail calls (bx through register after
+ KCFI check). */
+/* { dg-final { scan-assembler-times {bx\tr[0-9]+} 3 { target arm*-*-* } } } */
+
+/* Should have exactly 1 regular call (non-tail call case). */
+/* { dg-final { scan-assembler-times {blx\tr[0-9]+} 1 { target arm*-*-* } } } */
+
+/* Type ID loading should use 4 EOR instructions for 32-bit constants. */
+/* { dg-final { scan-assembler-times {eors?\tip, ip, #[0-9]+} 16 { target arm*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
index 252e63c85760..7f3e20ee23bc 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-encoding.c
@@ -1,5 +1,5 @@
/* Test AArch64 and ARM32 KCFI trap encoding in BRK/UDF instructions. */
-/* { dg-do compile { target aarch64*-*-* } } */
+/* { dg-do compile { target aarch64*-*-* arm*-*-* } } */
void target_function(int x, char y) {
}
@@ -40,4 +40,50 @@ int main() {
** ...
*/
+/* ARM32 specific: Should have UDF instruction with proper encoding
+ UDF format: 0x8000 | ((type_reg & 31) << 5) | (addr_reg & 31)
+
+ Since ARM32 spills and restores r3 before the trap, the type_reg
+ field uses 0x1F (31) to indicate "register was spilled" rather than
+ pointing to a live register. The addr_reg field contains the actual
+ target register number.
+
+ For this test case using r3, we expect:
+ UDF = 0x8000 | (31 << 5) | 3 = 33763
+ */
+
+/*
+** main: { target arm_nothumb }
+** ...
+** ldr ip, \[r[0-9]+, #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call[0-9]+
+** .Lkcfi_trap[0-9]+:
+** udf #33763
+** .Lkcfi_call[0-9]+:
+** blx r3
+** ...
+*/
+
+/* Thumb-2 inserts a BIC to strip the Thumb bit before the typeid load. */
+/*
+** main: { target arm_thumb2 }
+** ...
+** bic ip, r[0-9]+, #1
+** ldr ip, \[ip, #-4\]
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eor ip, ip, #[0-9]+
+** eors ip, ip, #[0-9]+
+** beq .Lkcfi_call[0-9]+
+** .Lkcfi_trap[0-9]+:
+** udf #33763
+** .Lkcfi_call[0-9]+:
+** blx r3
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
index e92873e51321..e02a320f2f92 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -19,9 +19,10 @@ int main() {
/* Should have exactly 2 trap labels in code. */
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ud2} 2 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*brk} 2 { target aarch64*-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*udf} 2 { target arm*-*-* } } } */
/* x86_64 should exactly 2 .kcfi_traps sections. */
/* { dg-final { scan-assembler-times {\.section\t\.kcfi_traps,"ao",@progbits,\.text} 2 { target x86_64-*-* } } } */
-/* AArch64 should NOT have .kcfi_traps section. */
-/* { dg-final { scan-assembler-not {\.section\t+\.kcfi_traps} { target aarch64*-*-* } } } */
+/* AArch64 and ARM 32-bit should NOT have .kcfi_traps section. */
+/* { dg-final { scan-assembler-not {\.section\t+\.kcfi_traps} { target aarch64*-*-* arm*-*-* } } } */
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v17 7/7] riscv: Add RISC-V Kernel Control Flow Integrity implementation
2026-10-05 15:40 [PATCH v17 0/7] Introduce Kernel Control Flow Integrity ABI [PR107048] Kees Cook
` (5 preceding siblings ...)
2026-10-05 15:40 ` [PATCH v17 6/7] arm: Add ARM 32-bit " Kees Cook
@ 2026-10-05 15:40 ` Kees Cook
2026-10-09 4:39 ` Jeff Law
6 siblings, 1 reply; 10+ messages in thread
From: Kees Cook @ 2026-10-05 15:40 UTC (permalink / raw)
To: Andrea Pinski
Cc: Kees Cook, Richard Biener, Jeffrey Law, Joseph Myers,
Jakub Jelinek, Martin Uecker, Peter Zijlstra, Ard Biesheuvel,
Jan Hubicka, Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
Implement RISC-V-specific KCFI backend. This is rv64-only: the emitted
typeid construction uses addiw, which exists only on RV64/RV128. An rv32
backend would need an alternate sequence (e.g. addi); since the only
current user of KCFI on riscv is the rv64 build of the Linux kernel, the
support hook rejects rv32.
- Scratch register allocation using t1/t2 (x6/x7) following RISC-V
procedure call standard for temporary registers (already
caller-saved), and t3 (x28) when either t1 or t2 is already the call
target register.
- Incompatible with -ffixed-t1, -ffixed-t2, or -ffixed-t3.
- Integration with .kcfi_traps section for debugger/runtime metadata
(like x86_64).
Assembly Code Pattern for RISC-V:
lw t1, -4(target_reg) ; Load actual type ID from preamble
lui t2, %hi(expected_type) ; Load expected type (upper 20 bits)
addiw t2, t2, %lo(expected_type) ; Add lower 12 bits (sign-extended)
beq t1, t2, .Lkcfi_call ; Branch if types match
.Lkcfi_trap: ebreak ; Environment break trap on mismatch
.Lkcfi_call: jalr/jr target_reg ; Execute validated indirect transfer
Build and run tested with Linux kernel ARCH=riscv.
Assisted-by: LLM [tests]
gcc/ChangeLog:
* config/riscv/riscv-protos.h: Declare KCFI helpers.
* config/riscv/riscv.cc (riscv_maybe_wrap_call_with_kcfi): New
function, to wrap calls.
(riscv_maybe_wrap_call_value_with_kcfi): New function, to
wrap calls with return values.
(riscv_output_kcfi_insn): New function to emit KCFI assembly.
* config/riscv/riscv.md: Add KCFI RTL patterns and hook expansion.
* doc/invoke.texi: Document riscv nuances.
gcc/testsuite/ChangeLog:
* gcc.dg/kcfi/kcfi-adjacency.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-basics.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-call-sharing.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-complex-addressing.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-direct-call-shapes.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-move-preservation.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-no-sanitize-inline.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-no-sanitize.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-offset-validation.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-patchable-entry-only.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-patchable-large.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-patchable-medium.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-patchable-prefix-only.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-tail-calls.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-trap-section.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-trap-section-per-func.c: Add riscv patterns.
* gcc.dg/kcfi/kcfi-riscv-32bit.c: New test.
* gcc.dg/kcfi/kcfi-riscv-fixed-t1.c: New test.
* gcc.dg/kcfi/kcfi-riscv-fixed-t2.c: New test.
* gcc.dg/kcfi/kcfi-riscv-fixed-t3.c: New test.
Signed-off-by: Kees Cook <kees@kernel.org>
---
gcc/config/riscv/riscv-protos.h | 3 +
gcc/config/riscv/riscv.md | 76 ++++++-
gcc/config/riscv/riscv.cc | 199 ++++++++++++++++++
gcc/doc/invoke.texi | 17 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c | 18 ++
gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c | 22 +-
gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c | 4 +
.../gcc.dg/kcfi/kcfi-complex-addressing.c | 19 ++
.../gcc.dg/kcfi/kcfi-direct-call-shapes.c | 1 +
.../gcc.dg/kcfi/kcfi-move-preservation.c | 20 ++
.../gcc.dg/kcfi/kcfi-no-sanitize-inline.c | 5 +
gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c | 1 +
.../gcc.dg/kcfi/kcfi-offset-validation.c | 3 +
.../gcc.dg/kcfi/kcfi-patchable-entry-only.c | 9 +-
.../gcc.dg/kcfi/kcfi-patchable-large.c | 10 +-
.../gcc.dg/kcfi/kcfi-patchable-medium.c | 9 +-
.../gcc.dg/kcfi/kcfi-patchable-prefix-only.c | 9 +-
gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-32bit.c | 7 +
.../gcc.dg/kcfi/kcfi-riscv-fixed-t1.c | 7 +
.../gcc.dg/kcfi/kcfi-riscv-fixed-t2.c | 7 +
.../gcc.dg/kcfi/kcfi-riscv-fixed-t3.c | 7 +
gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c | 23 ++
.../gcc.dg/kcfi/kcfi-trap-section-per-func.c | 2 +-
gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c | 5 +-
24 files changed, 466 insertions(+), 17 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-32bit.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t1.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t2.c
create mode 100644 gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t3.c
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index c0fb28397809..44f497882ff7 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -127,6 +127,9 @@ extern void riscv_split_doubleword_move (rtx, rtx);
extern void riscv_split_quadword_move (rtx, rtx);
extern const char *riscv_output_move (rtx, rtx);
extern const char *riscv_output_return ();
+extern rtx riscv_maybe_wrap_call_with_kcfi (rtx, rtx);
+extern rtx riscv_maybe_wrap_call_value_with_kcfi (rtx, rtx);
+extern const char *riscv_output_kcfi_insn (rtx_insn *, rtx *);
extern void riscv_declare_function_name (FILE *, const char *, tree);
extern void riscv_declare_function_size (FILE *, const char *, tree);
extern void riscv_asm_output_alias (FILE *, const tree, const tree);
diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md
index cfac4c960bf1..e02b3fc33dfc 100644
--- a/gcc/config/riscv/riscv.md
+++ b/gcc/config/riscv/riscv.md
@@ -4237,10 +4237,24 @@
""
{
rtx target = riscv_legitimize_call_address (XEXP (operands[0], 0), true);
- emit_call_insn (gen_sibcall_internal (target, operands[1]));
+ rtx pat = gen_sibcall_internal (target, operands[1]);
+ pat = riscv_maybe_wrap_call_with_kcfi (pat, target);
+ emit_call_insn (pat);
DONE;
})
+;; KCFI sibling call
+(define_insn "*kcfi_sibcall_insn"
+ [(kcfi (call (mem:SI (match_operand:P 0 "call_insn_operand" "l"))
+ (match_operand 1 ""))
+ (match_operand 2 "const_int_operand"))]
+ "SIBLING_CALL_P (insn)"
+{
+ return riscv_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
(define_insn "sibcall_internal"
[(call (mem:SI (match_operand 0 "call_insn_operand" "j,S,U"))
(match_operand 1 "" ""))]
@@ -4259,11 +4273,25 @@
""
{
rtx target = riscv_legitimize_call_address (XEXP (operands[1], 0), true);
- emit_call_insn (gen_sibcall_value_internal (operands[0], target,
- operands[2]));
+ rtx pat = gen_sibcall_value_internal (operands[0], target, operands[2]);
+ pat = riscv_maybe_wrap_call_value_with_kcfi (pat, target);
+ emit_call_insn (pat);
DONE;
})
+;; KCFI sibling call with return value
+(define_insn "*kcfi_sibcall_value_insn"
+ [(set (match_operand 0 "")
+ (kcfi (call (mem:SI (match_operand:P 1 "call_insn_operand" "l"))
+ (match_operand 2 ""))
+ (match_operand 3 "const_int_operand")))]
+ "SIBLING_CALL_P (insn)"
+{
+ return riscv_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
(define_insn "sibcall_value_internal"
[(set (match_operand 0 "" "")
(call (mem:SI (match_operand 1 "call_insn_operand" "j,S,U"))
@@ -4281,15 +4309,31 @@
(use (match_operand 2 ""))])]
""
{
+ rtx pat;
rtx addr = XEXP (operands[0], 0);
rtx target = riscv_legitimize_call_address (addr, false);
if (riscv_call_needs_lpad_p (addr))
- emit_call_insn (gen_call_internal_cfi (target, operands[1]));
+ pat = gen_call_internal_cfi (target, operands[1]);
else
- emit_call_insn (gen_call_internal (target, operands[1]));
+ pat = gen_call_internal (target, operands[1]);
+ pat = riscv_maybe_wrap_call_with_kcfi (pat, target);
+ emit_call_insn (pat);
DONE;
})
+;; KCFI indirect call
+(define_insn "*kcfi_call_internal"
+ [(kcfi (call (mem:SI (match_operand:P 0 "call_insn_operand" "l"))
+ (match_operand 1 "" ""))
+ (match_operand 2 "const_int_operand"))
+ (clobber (reg:SI RETURN_ADDR_REGNUM))]
+ "!SIBLING_CALL_P (insn)"
+{
+ return riscv_output_kcfi_insn (insn, operands);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
(define_insn "call_internal"
[(call (mem:SI (match_operand 0 "call_insn_operand" "l,S,U"))
(match_operand 1 "" ""))
@@ -4339,16 +4383,32 @@
(use (match_operand 3 ""))])]
""
{
+ rtx pat;
rtx addr = XEXP (operands[1], 0);
rtx target = riscv_legitimize_call_address (addr, false);
if (riscv_call_needs_lpad_p (addr))
- emit_call_insn (gen_call_value_internal_cfi (operands[0], target,
- operands[2]));
+ pat = gen_call_value_internal_cfi (operands[0], target, operands[2]);
else
- emit_call_insn (gen_call_value_internal (operands[0], target, operands[2]));
+ pat = gen_call_value_internal (operands[0], target, operands[2]);
+ pat = riscv_maybe_wrap_call_value_with_kcfi (pat, target);
+ emit_call_insn (pat);
DONE;
})
+;; KCFI call with return value
+(define_insn "*kcfi_call_value_insn"
+ [(set (match_operand 0 "" "")
+ (kcfi (call (mem:SI (match_operand:P 1 "call_insn_operand" "l"))
+ (match_operand 2 "" ""))
+ (match_operand 3 "const_int_operand")))
+ (clobber (reg:SI RETURN_ADDR_REGNUM))]
+ "!SIBLING_CALL_P (insn)"
+{
+ return riscv_output_kcfi_insn (insn, &operands[1]);
+}
+ [(set_attr "type" "call")
+ (set_attr "length" "24")])
+
(define_insn "call_value_internal"
[(set (match_operand 0 "" "")
(call (mem:SI (match_operand 1 "call_insn_operand" "l,S,U"))
diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc
index ae3fd8e3d72b..13907a90d7f1 100644
--- a/gcc/config/riscv/riscv.cc
+++ b/gcc/config/riscv/riscv.cc
@@ -82,6 +82,7 @@ along with GCC; see the file COPYING3. If not see
#include "cgraph.h"
#include "langhooks.h"
#include "gimplify.h"
+#include "kcfi.h"
/* This file should be included last. */
#include "target-def.h"
@@ -12344,6 +12345,180 @@ riscv_convert_vector_chunks (struct gcc_options *opts)
return 1;
}
+/* Apply KCFI wrapping to call pattern if needed. PAT is the RTL call
+ pattern to potentially wrap with KCFI instrumentation. ADDR is the
+ call target address RTL expression. Returns the possibly modified
+ call pattern with KCFI wrapper applied for indirect calls. */
+
+rtx
+riscv_maybe_wrap_call_with_kcfi (rtx pat, rtx addr)
+{
+ /* Only indirect calls need KCFI instrumentation. */
+ if (REG_P (addr))
+ {
+ rtx kcfi_type_rtx = kcfi_get_type_id_for_expanding_gimple_call ();
+ if (kcfi_type_rtx)
+ {
+ /* The call pattern is a PARALLEL when it has clobbers (regular
+ calls) and a bare CALL otherwise (sibling calls). Wrap the
+ CALL in either case. */
+ if (GET_CODE (pat) == PARALLEL)
+ {
+ rtx call_rtx = XVECEXP (pat, 0, 0);
+ rtx kcfi_call = gen_rtx_KCFI (VOIDmode, call_rtx, kcfi_type_rtx);
+ XVECEXP (pat, 0, 0) = kcfi_call;
+ }
+ else
+ {
+ gcc_assert (GET_CODE (pat) == CALL);
+ pat = gen_rtx_KCFI (VOIDmode, pat, kcfi_type_rtx);
+ }
+ }
+ }
+ return pat;
+}
+
+/* Apply KCFI wrapping to call_value pattern if needed. PAT is the RTL
+ call_value pattern to potentially wrap with KCFI instrumentation. ADDR
+ is the call target address RTL expression. Returns the possibly modified
+ call pattern with KCFI wrapper applied for indirect calls. */
+
+rtx
+riscv_maybe_wrap_call_value_with_kcfi (rtx pat, rtx addr)
+{
+ /* Only indirect calls need KCFI instrumentation. */
+ if (REG_P (addr))
+ {
+ rtx kcfi_type_rtx = kcfi_get_type_id_for_expanding_gimple_call ();
+ if (kcfi_type_rtx)
+ {
+ /* The call_value pattern is a PARALLEL when it has clobbers
+ (regular calls) and a bare SET otherwise (sibling calls).
+ Wrap the inner CALL in either case. */
+ rtx set_rtx = (GET_CODE (pat) == PARALLEL)
+ ? XVECEXP (pat, 0, 0) : pat;
+ gcc_assert (GET_CODE (set_rtx) == SET);
+ rtx call_rtx = SET_SRC (set_rtx);
+ rtx kcfi_call = gen_rtx_KCFI (VOIDmode, call_rtx, kcfi_type_rtx);
+ SET_SRC (set_rtx) = kcfi_call;
+ }
+ }
+ return pat;
+}
+
+/* Output the assembly for a KCFI checked call instruction. INSN is the
+ RTL instruction being processed. OPERANDS is the array of RTL operands
+ where operands[0] is the call target register, operands[2] is the KCFI
+ type ID constant. Returns an empty string as all output is handled by
+ direct assembly generation. */
+
+const char *
+riscv_output_kcfi_insn (rtx_insn *insn, rtx *operands)
+{
+ /* Target register. */
+ rtx target_reg = operands[0];
+ gcc_assert (REG_P (target_reg));
+
+ /* Get KCFI type ID. */
+ uint32_t expected_type = UINTVAL (operands[2]);
+
+ /* Calculate typeid offset from call target. */
+ HOST_WIDE_INT offset = -kcfi_get_typeid_offset ();
+
+ /* Choose scratch registers that don't conflict with target. */
+ unsigned temp1_regnum = T1_REGNUM;
+ unsigned temp2_regnum = T2_REGNUM;
+
+ if (REGNO (target_reg) == T1_REGNUM)
+ temp1_regnum = T3_REGNUM;
+ else if (REGNO (target_reg) == T2_REGNUM)
+ temp2_regnum = T3_REGNUM;
+
+ /* Get unique label number for this KCFI check. */
+ int labelno = kcfi_next_labelno ();
+
+ /* Generate custom label names. */
+ char trap_name[32];
+ char call_name[32];
+ ASM_GENERATE_INTERNAL_LABEL (trap_name, "Lkcfi_trap", labelno);
+ ASM_GENERATE_INTERNAL_LABEL (call_name, "Lkcfi_call", labelno);
+
+ /* Split expected_type for RISC-V immediate encoding.
+ If bit 11 is set, increment upper 20 bits to compensate for sign
+ extension. */
+ int32_t lo12 = ((int32_t)(expected_type << 20)) >> 20;
+ uint32_t hi20 = ((expected_type >> 12)
+ + ((expected_type & 0x800) ? 1 : 0)) & 0xFFFFF;
+
+ rtx temp_operands[3];
+
+ /* The check sequence (typeid load through indirect jump) must be emitted
+ as a single atomic unit: a mismatch must be caught before the jalr
+ executes, with no opportunity for the linker or assembler to interleave
+ anything. Disable linker relaxation (which can rewrite jalr/call forms
+ and shift offsets) and the C extension (which can shrink instructions
+ and perturb sizes the length attribute reports) for the duration. */
+ output_asm_insn (".option push", operands);
+ output_asm_insn (".option norelax", operands);
+ output_asm_insn (".option norvc", operands);
+
+ /* Load actual type from memory at offset. */
+ temp_operands[0] = gen_rtx_REG (SImode, temp1_regnum);
+ temp_operands[1] = gen_rtx_MEM (SImode,
+ gen_rtx_PLUS (DImode, target_reg,
+ GEN_INT (offset)));
+ output_asm_insn ("lw\t%0, %1", temp_operands);
+
+ /* Load expected type using lui + addiw for proper sign extension. */
+ temp_operands[0] = gen_rtx_REG (SImode, temp2_regnum);
+ temp_operands[1] = GEN_INT (hi20);
+ output_asm_insn ("lui\t%0, %1", temp_operands);
+
+ temp_operands[0] = gen_rtx_REG (SImode, temp2_regnum);
+ temp_operands[1] = gen_rtx_REG (SImode, temp2_regnum);
+ temp_operands[2] = GEN_INT (lo12);
+ output_asm_insn ("addiw\t%0, %1, %2", temp_operands);
+
+ /* Output conditional branch to call label. */
+ fprintf (asm_out_file, "\tbeq\t%s, %s, ",
+ reg_names[temp1_regnum], reg_names[temp2_regnum]);
+ assemble_name (asm_out_file, call_name);
+ fputc ('\n', asm_out_file);
+
+ /* Output trap label and ebreak instruction. */
+ ASM_OUTPUT_LABEL (asm_out_file, trap_name);
+ output_asm_insn ("ebreak", operands);
+
+ /* Use common helper for trap section entry. */
+ rtx trap_label_sym = gen_rtx_SYMBOL_REF (Pmode, trap_name);
+ kcfi_emit_traps_section (asm_out_file, trap_label_sym, labelno);
+
+ /* Output pass/call label. */
+ ASM_OUTPUT_LABEL (asm_out_file, call_name);
+
+ /* Execute the indirect call. */
+ if (SIBLING_CALL_P (insn))
+ {
+ /* Tail call uses x0 (zero register) to avoid saving return address. */
+ temp_operands[0] = gen_rtx_REG (DImode, 0);
+ temp_operands[1] = target_reg;
+ temp_operands[2] = const0_rtx;
+ output_asm_insn ("jalr\t%0, %1, %2", temp_operands);
+ }
+ else
+ {
+ /* Regular call uses x1 (return address register). */
+ temp_operands[0] = gen_rtx_REG (DImode, RETURN_ADDR_REGNUM);
+ temp_operands[1] = target_reg;
+ temp_operands[2] = const0_rtx;
+ output_asm_insn ("jalr\t%0, %1, %2", temp_operands);
+ }
+
+ output_asm_insn (".option pop", operands);
+
+ return "";
+}
+
/* 'Unpack' up the internal tuning structs and update the options
in OPTS. The caller must have set up selected_tune and selected_arch
as all the other target-specific codegen decisions are
@@ -17360,6 +17535,30 @@ riscv_memtag_tag_bitsize ()
#undef TARGET_MEMTAG_TAG_BITSIZE
#define TARGET_MEMTAG_TAG_BITSIZE riscv_memtag_tag_bitsize
+/* Return true if the target supports KCFI.
+ KCFI requires 64-bit mode and the T1, T2, and T3 registers. */
+
+static bool
+riscv_kcfi_supported_p (void)
+{
+ if (!TARGET_64BIT)
+ {
+ error ("%<-fsanitize=kcfi%> is not supported for 32-bit RISC-V");
+ return false;
+ }
+ if (fixed_regs[T1_REGNUM] || fixed_regs[T2_REGNUM] || fixed_regs[T3_REGNUM])
+ {
+ error ("%<-fsanitize=kcfi%> is not compatible with %<-ffixed-t1%>, "
+ "%<-ffixed-t2%>, or %<-ffixed-t3%> as KCFI requires these "
+ "registers for type checking");
+ return false;
+ }
+ return true;
+}
+
+#undef TARGET_KCFI_SUPPORTED
+#define TARGET_KCFI_SUPPORTED riscv_kcfi_supported_p
+
#undef TARGET_DOCUMENTATION_NAME
#define TARGET_DOCUMENTATION_NAME "RISC-V"
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index f0b1314d1731..b1ebae37ed89 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -17631,6 +17631,23 @@ allowing the kernel to identify both the KCFI violation and the involved
registers for detailed diagnostics (eliminating the need for a separate
@code{.kcfi_traps} section as used on x86_64).
+On 64-bit RISC-V, KCFI type identifiers are emitted as a @code{.word ID}
+directive (a 32-bit constant) before the function entry, similar to AArch64.
+RISC-V's natural instruction alignment eliminates the need for
+additional alignment NOPs. When used with @option{-fpatchable-function-entry},
+the type identifier is placed before any prefix NOPs. The runtime check
+loads the actual type using @code{lw t1, OFFSET(target_reg)}, where the
+offset accounts for any prefix NOPs, constructs the expected type using
+@code{lui} and @code{addiw} instructions into @code{t2}, and compares them
+with @code{beq}. @code{t3} is used as an alternative when @code{t1} or
+@code{t2} is the target call register. Because of the use of these
+registers, they cannot be fixed registers, so KCFI cannot be used with any
+of @code{-ffixed-t1}, @code{-ffixed-t2}, nor @code{-ffixed-t3}. Type
+mismatches trigger an @code{ebreak} instruction. Like x86_64, RISC-V
+uses a @code{.kcfi_traps} section to map trap locations to their
+corresponding function entry points for debugging (RISC-V lacks
+ESR-style trap encoding like used on AArch64).
+
KCFI is intended primarily for kernel code and may not be suitable
for user-space applications that rely on techniques incompatible
with strict type checking of indirect calls.
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
index 63fade749ddd..fad09c4f08d1 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-adjacency.c
@@ -121,4 +121,22 @@ __attribute__((noinline)) void test_conditional_call(int flag) {
** ...
*/
+/*
+** test_complex_args: { target riscv*-*-* }
+** ...
+** lw t1, -4\((a[0-9]+)\)
+** lui t2, [0-9]+
+** addiw t2, t2, -?[0-9]+
+** beq t1, t2, .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ebreak
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .4byte .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jalr zero, \1, 0
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
index a917e1c70089..9256d2a1a3c3 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-basics.c
@@ -59,8 +59,8 @@ int main() {
/* x86_64: Verify type ID in preamble (after NOPs, before function label) */
/* { dg-final { scan-assembler {__cfi_regular_function:\n\t+nop\n.*\n\t+movl\t+\$0x[0-9a-f]+, %eax} { target x86_64-*-* } } } */
-/* AArch64, ARM32: Verify type ID word in preamble. */
-/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target aarch64*-*-* arm*-*-* } } } */
+/* AArch64, ARM32, RISC-V: Verify type ID word in preamble. */
+/* { dg-final { scan-assembler {__cfi_regular_function:\n\t\.word\t0x[0-9a-f]+} { target aarch64*-*-* arm*-*-* riscv*-*-* } } } */
/*
** static_caller: { target x86_64-*-* }
@@ -128,6 +128,24 @@ int main() {
** ...
*/
+/*
+** static_caller: { target riscv*-*-* }
+** ...
+** lw t1, -4\((a[0-9]+)\)
+** lui t2, [0-9]+
+** addiw t2, t2, -?[0-9]+
+** beq t1, t2, .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ebreak
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .4byte .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jalr ra, \1, 0
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
/* Extern functions should NOT get KCFI preambles. */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
index ce5e17134929..f11556dd43de 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-call-sharing.c
@@ -69,6 +69,7 @@ int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *
/* ARM 32-bit: In Thumb-2 the load base is ip (post-Thumb-bit-strip), in ARM
mode it is the call target register or lr; accept any. */
/* { dg-final { scan-assembler-times {ldr\s+ip, \[(?:r[0-9]+|lr|ip), #-4\]} 3 { target arm*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {lui\tt2, [0-9]+} 2 { target riscv*-*-* } } } */
/* Verify the checks use DIFFERENT type IDs (not shared, except arm: see above).
We should NOT see the same type ID used twice - that would indicate
@@ -79,13 +80,16 @@ int test_kcfi_check_sharing(struct kobject *kobj, const struct attribute_group *
reused by a later check. Each check uses a single scratch register,
ip or r3. */
/* { dg-final { scan-assembler-not {eor\s+(ip|r3), \1, #([0-9]+)\n\teor\s+\1, \1, #([0-9]+)\n\teor\s+\1, \1, #([0-9]+)\n\teors\s+\1, \1, #([0-9]+)\n(?:\tpop\t\{r3\}\n)?\tbeq\t\.Lkcfi_call0\n.*eor\s+(?:ip|r3), (?:ip|r3), #\2\n\teor\s+(?:ip|r3), (?:ip|r3), #\3\n\teor\s+(?:ip|r3), (?:ip|r3), #\4\n\teors\s+(?:ip|r3), (?:ip|r3), #\5} { target arm*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-not {lui\s+t2, ([0-9]+)\s.*lui\s+t2, \1\s} { target riscv*-*-* } } } */
/* Verify expected number of traps. */
/* x86: { dg-final { scan-assembler-times {ud2} 2 { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
/* ARM 32-bit: { dg-final { scan-assembler-times {udf\s+#[0-9]+} 3 { target arm*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {ebreak} 2 { target riscv*-*-* } } } */
/* Verify 2 separate call sites (except arm). */
/* x86: { dg-final { scan-assembler-times {jmp\s+\*%[a-z0-9]+} 2 { target i?86-*-* x86_64-*-* } } } */
/* AArch64: { dg-final { scan-assembler-times {br\tx[0-9]+} 2 { target aarch64*-*-* } } } */
/* ARM 32-bit: { dg-final { scan-assembler-times {bx\s+(?:r[0-9]+|lr)} 3 { target arm*-*-* } } } */
+/* RISC-V: { dg-final { scan-assembler-times {jalr\t[a-z0-9]+} 2 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
index 5eec800e108d..b11014c29676 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-complex-addressing.c
@@ -184,4 +184,23 @@ int main() {
** ...
*/
+/* Standard KCFI handling. */
+/*
+** test_struct_members: { target riscv*-*-* }
+** ...
+** lw t1, -4\((a[0-9]+)\)
+** lui t2, [0-9]+
+** addiw t2, t2, -?[0-9]+
+** beq t1, t2, .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ebreak
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .4byte .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jalr ra, \1, 0
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
index 2daf5aa94bcf..1c72f0fc5757 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-direct-call-shapes.c
@@ -33,3 +33,4 @@ void caller(int sel, void (*ip)(int), int x) {
/* The arm load base is the call target register in ARM mode and ip in
Thumb-2 (after the Thumb-bit strip); accept either. */
/* { dg-final { scan-assembler-times {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} 1 { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-times {lw\tt1, -4\(a[0-9]+\)} 1 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
index fac0e3aff0e4..b78434cbe5e6 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-move-preservation.c
@@ -112,6 +112,26 @@ int main(void)
** ...
*/
+/*
+** indirect_call: { target riscv*-*-* }
+** ...
+** mv (a[0-9]+),a0
+** addi a0,a4,%lo\(called_count\)
+** lw t1, -4\(\1\)
+** lui t2, [0-9]+
+** addiw t2, t2, -?[0-9]+
+** beq t1, t2, .Lkcfi_call([0-9]+)
+** .Lkcfi_trap([0-9]+):
+** ebreak
+** .section .kcfi_traps,"ao",@progbits,.text
+** .Lkcfi_entry([0-9]+):
+** .4byte .Lkcfi_trap\3-.Lkcfi_entry\4
+** .text
+** .Lkcfi_call\2:
+** jalr zero, \1, 0
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.L.*|\.section|\.text} } } */
/* AArch64, ARM32 should NOT have trap section (use immediate instructions instead). */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
index 279207a85cdc..8bc353ed2d32 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize-inline.c
@@ -77,6 +77,7 @@ int main(void)
/* { dg-final { scan-assembler-times {ud2} 2 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times {brk\s+#0x[0-9a-f]+} 2 { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler-times {udf\s+#[0-9]+} 2 { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-times {ebreak} 2 { target riscv*-*-* } } } */
/* Positive controls: these should have KCFI checks. */
/* { dg-final { scan-assembler {normal_function:.*ud2.*\.size\s+normal_function} { target x86_64-*-* } } } */
@@ -85,6 +86,8 @@ int main(void)
/* { dg-final { scan-assembler {wrap_normal_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_normal_inline} { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler {normal_function:.*udf\t#[0-9]+.*\.size\s+normal_function} { target arm*-*-* } } } */
/* { dg-final { scan-assembler {wrap_normal_inline:.*udf\t#[0-9]+.*\.size\s+wrap_normal_inline} { target arm*-*-* } } } */
+/* { dg-final { scan-assembler {normal_function:.*ebreak.*\.size\s+normal_function} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler {wrap_normal_inline:.*ebreak.*\.size\s+wrap_normal_inline} { target riscv*-*-* } } } */
/* Negative controls: these should NOT have KCFI checks. */
/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ud2.*\.size\s+sensitive_non_inline_function} { target x86_64-*-* } } } */
@@ -93,3 +96,5 @@ int main(void)
/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*brk\s+#0x[0-9a-f]+.*\.size\s+wrap_sensitive_inline} { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler-not {sensitive_non_inline_function:[^\n]*udf\t#[0-9]+[^\n]*\.size\tsensitive_non_inline_function} { target arm*-*-* } } } */
/* { dg-final { scan-assembler-not {wrap_sensitive_inline:[^\n]*udf\t#[0-9]+[^\n]*\.size\twrap_sensitive_inline} { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-not {sensitive_non_inline_function:.*ebreak.*\.size\s+sensitive_non_inline_function} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-not {wrap_sensitive_inline:.*ebreak.*\.size\s+wrap_sensitive_inline} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
index 9355d633301b..86855eca9d86 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-no-sanitize.c
@@ -38,3 +38,4 @@ int main() {
/* In Thumb-2 the load base is ip (post-Thumb-bit-strip), in ARM mode it is
the call target register; accept either. */
/* { dg-final { scan-assembler-times {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} 1 { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-times {lw\tt1, -[0-9]+\(} 1 { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
index cb1eb7c7ecc6..8d48c5f2ef23 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-offset-validation.c
@@ -35,3 +35,6 @@ int main() {
In Thumb-2 the load base is ip (after the Thumb-bit strip); in ARM mode it
is the call target register directly. Accept either. */
/* { dg-final { scan-assembler {ldr\tip, \[(?:r[0-9]+|ip), #-4\]} { target arm*-*-* } } } */
+
+/* RISC-V: All call sites should use -4 offset. */
+/* { dg-final { scan-assembler {lw\tt1, -4\(} { target riscv*-*-* } } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
index 612140a0f509..db2ec8a5b64f 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-entry-only.c
@@ -29,7 +29,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* riscv*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -54,4 +54,11 @@ int main() {
** ...
*/
+/*
+** main: { target riscv*-*-* }
+** ...
+** lw t1, -4\(a[0-9]+\)
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
index 7f7ada17cc14..49a684b7ee15 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-large.c
@@ -21,7 +21,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* riscv*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -46,4 +46,12 @@ int main() {
** ...
*/
+
+/*
+** main: { target riscv*-*-* }
+** ...
+** lw t1, -48\(a[0-9]+\)
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
index 0a1f4e20851f..8d53407789cf 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-medium.c
@@ -28,7 +28,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* riscv*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -53,4 +53,11 @@ int main() {
** ...
*/
+/*
+** main: { target riscv*-*-* }
+** ...
+** lw t1, -20\(a[0-9]+\)
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
index 548a2f5d8b84..d85f7f907ce5 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-patchable-prefix-only.c
@@ -29,7 +29,7 @@ int main() {
*/
/*
-** __cfi_test_function: { target aarch64*-*-* arm*-*-* }
+** __cfi_test_function: { target aarch64*-*-* arm*-*-* riscv*-*-* }
** .word 0x[0-9a-f]+
*/
@@ -54,4 +54,11 @@ int main() {
** ...
*/
+/*
+** main: { target riscv*-*-* }
+** ...
+** lw t1, -16\(a[0-9]+\)
+** ...
+*/
+
/* { dg-final { check-function-bodies "**" "" "" { target *-*-* } {\.word} } } */
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-32bit.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-32bit.c
new file mode 100644
index 000000000000..8b2dc926c704
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-32bit.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is rejected for 32-bit RISC-V. */
+/* { dg-do compile { target riscv*-*-* } } */
+/* { dg-additional-options "-march=rv32gc -mabi=ilp32d" } */
+/* { dg-error ".-fsanitize=kcfi. is not supported for 32-bit RISC-V" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t1.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t1.c
new file mode 100644
index 000000000000..a5ef21092d63
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t1.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is incompatible with -ffixed-t1 on RISC-V. */
+/* { dg-do compile { target riscv*-*-* } } */
+/* { dg-additional-options "-ffixed-t1" } */
+/* { dg-error ".-fsanitize=kcfi. is not compatible with .-ffixed-t1., .-ffixed-t2., or .-ffixed-t3. as KCFI requires these registers for type checking" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t2.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t2.c
new file mode 100644
index 000000000000..3ead648e7154
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t2.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is incompatible with -ffixed-t2 on RISC-V. */
+/* { dg-do compile { target riscv*-*-* } } */
+/* { dg-additional-options "-ffixed-t2" } */
+/* { dg-error ".-fsanitize=kcfi. is not compatible with .-ffixed-t1., .-ffixed-t2., or .-ffixed-t3. as KCFI requires these registers for type checking" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t3.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t3.c
new file mode 100644
index 000000000000..98b579867a48
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-riscv-fixed-t3.c
@@ -0,0 +1,7 @@
+/* Test that KCFI is incompatible with -ffixed-t3 on RISC-V. */
+/* { dg-do compile { target riscv*-*-* } } */
+/* { dg-additional-options "-ffixed-t3" } */
+/* { dg-error ".-fsanitize=kcfi. is not compatible with .-ffixed-t1., .-ffixed-t2., or .-ffixed-t3. as KCFI requires these registers for type checking" "" { target *-*-* } 0 } */
+/* { dg-message "sorry, unimplemented: .-fsanitize=kcfi. not supported" "" { target *-*-* } 0 } */
+
+void foo (void) { }
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
index 544ae7807297..f8a76d3ac4f6 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-tail-calls.c
@@ -67,6 +67,8 @@ int test_non_tail_indirect_call(func_ptr_t handler, int x) {
/* Should have exactly 4 trap sections and 4 trap instructions. */
/* { dg-final { scan-assembler-times "\\.kcfi_traps" 4 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times "ud2" 4 { target x86_64-*-* } } } */
+/* { dg-final { scan-assembler-times "\\.kcfi_traps" 4 { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-times "ebreak" 4 { target riscv*-*-* } } } */
/* Should NOT have unprotected direct jumps to vtable. */
/* { dg-final { scan-assembler-not {jmp\t\*vtable\(%rip\)} { target x86_64-*-* } } } */
@@ -79,6 +81,27 @@ int test_non_tail_indirect_call(func_ptr_t handler, int x) {
/* Should have exactly 1 regular call (non-tail call case). */
/* { dg-final { scan-assembler-times {call\t\*%[a-z0-9]+} 1 { target x86_64-*-* } } } */
+/* RISC-V: Should have exactly 4 KCFI checks for indirect calls
+ (comparison instruction). */
+/* { dg-final { scan-assembler-times {beq\tt1, t2, \.Lkcfi_call[0-9]+} 4 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 4 KCFI checks for indirect calls as
+ (load type ID + compare). */
+/* { dg-final { scan-assembler-times {lw\tt1, -4\([a-z0-9]+\)} 4 { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler-times {lui\tt2, [0-9]+} 4 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 3 protected tail calls (jr after
+ KCFI check - no return address save). */
+/* { dg-final { scan-assembler-times {jalr\t(x0|zero), [a-z0-9]+, 0} 3 { target riscv*-*-* } } } */
+
+/* RISC-V: Should have exactly 1 regular call (non-tail call case - saves
+ return address). */
+/* { dg-final { scan-assembler-times {jalr\t(x1|ra), [a-z0-9]+, 0} 1 { target riscv*-*-* } } } */
+
+/* Type ID loading should use lui + addiw pattern for 32-bit constants. */
+/* { dg-final { scan-assembler {lui\tt2, [0-9]+} { target riscv*-*-* } } } */
+/* { dg-final { scan-assembler {addiw\tt2, t2, -?[0-9]+} { target riscv*-*-* } } } */
+
/* Should have exactly 4 KCFI checks for indirect calls (load type ID from
-4 offset + compare). The actual-type scratch register avoids the call
target, so it is w9 for the sibcall targets (x16) and w16 otherwise; the
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
index 448805a37808..7faeca95cac7 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section-per-func.c
@@ -8,7 +8,7 @@
Only x86_64 and RISC-V use a .kcfi_traps section; AArch64 and ARM encode the
trap information directly in the trap instruction's immediate. */
-/* { dg-do compile { target x86_64-*-* } } */
+/* { dg-do compile { target x86_64-*-* riscv*-*-* } } */
/* { dg-additional-options "-ffunction-sections" } */
typedef int (*fn_t) (int);
diff --git a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
index e02a320f2f92..5d1159cfb39b 100644
--- a/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
+++ b/gcc/testsuite/gcc.dg/kcfi/kcfi-trap-section.c
@@ -20,9 +20,10 @@ int main() {
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ud2} 2 { target x86_64-*-* } } } */
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*brk} 2 { target aarch64*-*-* } } } */
/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*udf} 2 { target arm*-*-* } } } */
+/* { dg-final { scan-assembler-times {\.L[^:]+:\n\s*ebreak} 2 { target riscv*-*-* } } } */
-/* x86_64 should exactly 2 .kcfi_traps sections. */
-/* { dg-final { scan-assembler-times {\.section\t\.kcfi_traps,"ao",@progbits,\.text} 2 { target x86_64-*-* } } } */
+/* x86_64 and RISC-V should exactly 2 .kcfi_traps sections. */
+/* { dg-final { scan-assembler-times {\.section\t\.kcfi_traps,"ao",@progbits,\.text} 2 { target x86_64-*-* riscv*-*-* } } } */
/* AArch64 and ARM 32-bit should NOT have .kcfi_traps section. */
/* { dg-final { scan-assembler-not {\.section\t+\.kcfi_traps} { target aarch64*-*-* arm*-*-* } } } */
--
2.55.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v17 7/7] riscv: Add RISC-V Kernel Control Flow Integrity implementation
2026-10-05 15:40 ` [PATCH v17 7/7] riscv: Add RISC-V " Kees Cook
@ 2026-10-09 4:39 ` Jeff Law
0 siblings, 0 replies; 10+ messages in thread
From: Jeff Law @ 2026-10-09 4:39 UTC (permalink / raw)
To: Kees Cook, Andrea Pinski
Cc: Richard Biener, Jeffrey Law, Joseph Myers, Jakub Jelinek,
Martin Uecker, Peter Zijlstra, Ard Biesheuvel, Jan Hubicka,
Uros Bizjak, Richard Earnshaw, Richard Sandiford,
Marcus Shawcroft, Kyrylo Tkachov, Kito Cheng, Palmer Dabbelt,
Andrew Waterman, Jim Wilson, Juergen Christ, Dan Li,
Sami Tolvanen, Ramon de C Valle, Joao Moreira, Nathan Chancellor,
Bill Wendling, Osterlund Sebastian, Constable Scott D,
gcc-patches, linux-hardening
On 10/5/26 9:40 AM, Kees Cook wrote:
> Implement RISC-V-specific KCFI backend. This is rv64-only: the emitted
> typeid construction uses addiw, which exists only on RV64/RV128. An rv32
> backend would need an alternate sequence (e.g. addi); since the only
> current user of KCFI on riscv is the rv64 build of the Linux kernel, the
> support hook rejects rv32.
Note that addiw explicitly sign extends the result from bit 32 out to
bit 63. That seems to match what you want with the implementation
(since you do a lw do load the ID from the preamble and that's
sign-extending from bit 32 to bit 63.
>
> - Scratch register allocation using t1/t2 (x6/x7) following RISC-V
> procedure call standard for temporary registers (already
> caller-saved), and t3 (x28) when either t1 or t2 is already the call
> target register.
>
> - Incompatible with -ffixed-t1, -ffixed-t2, or -ffixed-t3.
>
> - Integration with .kcfi_traps section for debugger/runtime metadata
> (like x86_64).
>
> Assembly Code Pattern for RISC-V:
> lw t1, -4(target_reg) ; Load actual type ID from preamble
> lui t2, %hi(expected_type) ; Load expected type (upper 20 bits)
> addiw t2, t2, %lo(expected_type) ; Add lower 12 bits (sign-extended)
> beq t1, t2, .Lkcfi_call ; Branch if types match
> .Lkcfi_trap: ebreak ; Environment break trap on mismatch
> .Lkcfi_call: jalr/jr target_reg ; Execute validated indirect transfer
So what I can't recall ever seeing is an explanation of why this
sequence needs to be emitted as a single assembly block. That's
something we generally try to avoid. Now if that design decision/need
is covered in the talk from the Cauldron, that's fine. I was
intercepted and couldn't get there in time, but I'll certainly make a
point to watch the whole thing.
And note that if it really needs to be an atomic sequence, there's still
ways to achieve that that avoid the blobs of assembly code. See below.
>
> Build and run tested with Linux kernel ARCH=riscv.
>
> Assisted-by: LLM [tests]
>
> gcc/ChangeLog:
>
> * config/riscv/riscv-protos.h: Declare KCFI helpers.
> * config/riscv/riscv.cc (riscv_maybe_wrap_call_with_kcfi): New
> function, to wrap calls.
> (riscv_maybe_wrap_call_value_with_kcfi): New function, to
> wrap calls with return values.
> (riscv_output_kcfi_insn): New function to emit KCFI assembly.
> * config/riscv/riscv.md: Add KCFI RTL patterns and hook expansion.
> * doc/invoke.texi: Document riscv nuances.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.dg/kcfi/kcfi-adjacency.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-basics.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-call-sharing.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-complex-addressing.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-direct-call-shapes.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-move-preservation.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-no-sanitize-inline.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-no-sanitize.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-offset-validation.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-patchable-entry-only.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-patchable-large.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-patchable-medium.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-patchable-prefix-only.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-tail-calls.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-trap-section.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-trap-section-per-func.c: Add riscv patterns.
> * gcc.dg/kcfi/kcfi-riscv-32bit.c: New test.
> * gcc.dg/kcfi/kcfi-riscv-fixed-t1.c: New test.
> * gcc.dg/kcfi/kcfi-riscv-fixed-t2.c: New test.
> * gcc.dg/kcfi/kcfi-riscv-fixed-t3.c: New test.
>
> Signed-off-by: Kees Cook <kees@kernel.org>
> ---
>
>
>
>
>
> +
> +/* Output the assembly for a KCFI checked call instruction. INSN is the
> + RTL instruction being processed. OPERANDS is the array of RTL operands
> + where operands[0] is the call target register, operands[2] is the KCFI
> + type ID constant. Returns an empty string as all output is handled by
> + direct assembly generation. */
> +
> +const char *
> +riscv_output_kcfi_insn (rtx_insn *insn, rtx *operands)
> +{
> + /* Target register. */
> + rtx target_reg = operands[0];
> + gcc_assert (REG_P (target_reg));
> +
> + /* Get KCFI type ID. */
> + uint32_t expected_type = UINTVAL (operands[2]);
Generally you don't want to be using types like uint32_t, uint64_t,
etc. Those are properties of the host, not the target. Given that
you're using "lw" and a sign extending "addiw", you probably want this
to be a HOST_WIDE_INT (vs an unsigned HOST_WIDE_INT). It probably
doesn't matter in practice here, but if you're pulling data out via
[U]INTVAL, then destination data type should generally be a
HOST_WIDE_INT or unsigned variant of the same.
> +
> + /* Calculate typeid offset from call target. */
> + HOST_WIDE_INT offset = -kcfi_get_typeid_offset ();
> +
> + /* Choose scratch registers that don't conflict with target. */
> + unsigned temp1_regnum = T1_REGNUM;
> + unsigned temp2_regnum = T2_REGNUM;
Why use fixed registers? It would seem to work better if you just
generated pseudos and let the register allocator do the right thing and
assign them to whatever temporary is best. That would also remove the
restrictions around -ffixed-reg.
> +
> + rtx temp_operands[3];
> +
> + /* The check sequence (typeid load through indirect jump) must be emitted
> + as a single atomic unit: a mismatch must be caught before the jalr
> + executes, with no opportunity for the linker or assembler to interleave
> + anything. Disable linker relaxation (which can rewrite jalr/call forms
> + and shift offsets) and the C extension (which can shrink instructions
> + and perturb sizes the length attribute reports) for the duration. */
> + output_asm_insn (".option push", operands);
> + output_asm_insn (".option norelax", operands);
> + output_asm_insn (".option norvc", operands);
Why does this need to be an atomic sequence with restrictions around
relaxing and compression avoidance? From other comments I'm guessing
you're rewriting that ebreak. At the least you need to explain why its
an atomic sequence.
> +
> + /* Load actual type from memory at offset. */
> + temp_operands[0] = gen_rtx_REG (SImode, temp1_regnum);
> + temp_operands[1] = gen_rtx_MEM (SImode,
> + gen_rtx_PLUS (DImode, target_reg,
> + GEN_INT (offset)));
> + output_asm_insn ("lw\t%0, %1", temp_operands);
So if we really don't need to generate an atomic sequence, then you've
got the skeleton for generating RTL here. You've got the register
destination and memory source. So instead of output_asm_insn, you'd do
something like:
emit_move_insn (temp_operands[0], temp_operands[1]);
> +
> + /* Load expected type using lui + addiw for proper sign extension. */
> + temp_operands[0] = gen_rtx_REG (SImode, temp2_regnum);
> + temp_operands[1] = GEN_INT (hi20);
> + output_asm_insn ("lui\t%0, %1", temp_operands);
> +
> + temp_operands[0] = gen_rtx_REG (SImode, temp2_regnum);
> + temp_operands[1] = gen_rtx_REG (SImode, temp2_regnum);
> + temp_operands[2] = GEN_INT (lo12);
> + output_asm_insn ("addiw\t%0, %1, %2", temp_operands);
All that turns into dest = force_reg (SImode, GEN_INT (expected_type));
> +
> + /* Output conditional branch to call label. */
> + fprintf (asm_out_file, "\tbeq\t%s, %s, ",
> + reg_names[temp1_regnum], reg_names[temp2_regnum]);
> + assemble_name (asm_out_file, call_name);
> + fputc ('\n', asm_out_file);
And you'd emit a conditional branch here via emit_jump_insn after
generating appropriate RTL.
> +
> + /* Output trap label and ebreak instruction. */
> + ASM_OUTPUT_LABEL (asm_out_file, trap_name);
> + output_asm_insn ("ebreak", operands);
This is a "trap" insn. So
emit_insn (gen_trap ()); or something along those lines.
> +
> + /* Use common helper for trap section entry. */
> + rtx trap_label_sym = gen_rtx_SYMBOL_REF (Pmode, trap_name);
> + kcfi_emit_traps_section (asm_out_file, trap_label_sym, labelno);
So do you do something fun like rewrite the trap? Is that why you're
generating the atomic sequence? ANd if so, we can still get you an
atomic sequence without emitting blobs of assembly code via emit_barrier
() at the start and end of the atomic sequence.
If you're rewriting the trap, then I'd probably do somethign like
1. Define a new insn that has the same basic properties as a barrier,
but emits the .option thingies at the start of the sequence.
2. Define another new insn also with barrier semantics that emits the
option pop.
3. Convert riscv_output_kcfi_insn to emit RTL instead and hook into the
define_expands. It'll need to generate the insns you created in steps
#1 and #2.
Note that steps #1 and #2 would in turn allow others to simplify other
parts of the RISC-V port as well. Not the main motivation, but worth
noting.
> +
> /* 'Unpack' up the internal tuning structs and update the options
> in OPTS. The caller must have set up selected_tune and selected_arch
> as all the other target-specific codegen decisions are
> @@ -17360,6 +17535,30 @@ riscv_memtag_tag_bitsize ()
> #undef TARGET_MEMTAG_TAG_BITSIZE
> #define TARGET_MEMTAG_TAG_BITSIZE riscv_memtag_tag_bitsize
>
> +/* Return true if the target supports KCFI.
> + KCFI requires 64-bit mode and the T1, T2, and T3 registers. */
> +
> +static bool
> +riscv_kcfi_supported_p (void)
We probably need to avoid for C++ due to thunks and Fortran due to
multiple entry points. And you may not have a good way to test for
those, particularly in an LTO build. We need to think about that
problem. There are ways to identify multiple entry points, but those
assume you've got a control flow graph for the current function. So
they probably won't work here and we can't really check for what
language is in use (think about LTO).
>
> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index f0b1314d1731..b1ebae37ed89 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -17631,6 +17631,23 @@ allowing the kernel to identify both the KCFI violation and the involved
> registers for detailed diagnostics (eliminating the need for a separate
> @code{.kcfi_traps} section as used on x86_64).
>
> +On 64-bit RISC-V, KCFI type identifiers are emitted as a @code{.word ID}
> +directive (a 32-bit constant) before the function entry, similar to AArch64.
> +RISC-V's natural instruction alignment eliminates the need for
> +additional alignment NOPs.
Really? Natural instruction alignment for the designs I expect to see
almost everywhere is 16 bits (via the C extension which is mandatory for
rva23 and used in nearly design I'm aware of). So you've got a bit of a
problem here as unaligned access isn't necessary guaranteed to work.
You could do something like force the function alignment to 4 bytes when
KFCI is on. A bit hackish, but probably sensible in practice.
> When used with @option{-fpatchable-function-entry},
> +the type identifier is placed before any prefix NOPs. The runtime check
> +loads the actual type using @code{lw t1, OFFSET(target_reg)}, where the
Is the offset always at -4? Just helps me understand the underlying
assumptions you need to make.
Jeff
^ permalink raw reply [flat|nested] 10+ messages in thread