[PATCH 0/3] Resolve ARM kCFI build failure in idpf xsk.c

[PATCH 0/3] Resolve ARM kCFI build failure in idpf xsk.c

[Nathan Chancellor]

Hi all,

This series resolves a build failure that is seen in
drivers/net/ethernet/intel/idpf/xsk.c after commit 9705d6552f58 ("idpf:
implement Rx path for AF_XDP") in 6.18-rc1 with ARCH=arm and
CONFIG_CFI=y. See patch 3 for a simplified reproducer on top of
defconfig.

I think this could go via hardening or net.

---
Nathan Chancellor (3):
      compiler_types: Introduce __nocfi_generic
      ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS
      libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

 arch/Kconfig                   | 7 +++++++
 arch/arm/Kconfig               | 2 ++
 include/linux/compiler_types.h | 6 ++++++
 include/net/libeth/xdp.h       | 2 +-
 4 files changed, 16 insertions(+), 1 deletion(-)
---
base-commit: 211ddde0823f1442e4ad052a2f30f050145ccada
change-id: 20251025-idpf-fix-arm-kcfi-build-error-65ae59616374

Best regards,
--  
Nathan Chancellor <nathan@kernel.org>

[PATCH 1/3] compiler_types: Introduce __nocfi_generic

[Nathan Chancellor]

There are two different ways that LLVM can expand kCFI operand bundles
in LLVM IR: generically in the middle end or using an architecture
specific sequence when lowering LLVM IR to machine code in the backend.
The generic pass allows any architecture to take advantage of kCFI but
the expansion of these bundles in the middle end can mess with
optimizations that may turn indirect calls into direct calls when the
call target is known at compile time, such as after inlining.

Add __nocfi_generic, dependent on an architecture selecting
CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS, to disable kCFI bundle
generation in functions where only the generic kCFI pass may cause
problems.

Link: https://github.com/ClangBuiltLinux/linux/issues/2124
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 arch/Kconfig                   | 7 +++++++
 include/linux/compiler_types.h | 6 ++++++
 2 files changed, 13 insertions(+)

diff --git a/arch/Kconfig b/arch/Kconfig
index 74ff01133532..61130b88964b 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -917,6 +917,13 @@ config ARCH_USES_CFI_TRAPS
 	  An architecture should select this option if it requires the
 	  .kcfi_traps section for KCFI trap handling.
 
+config ARCH_USES_CFI_GENERIC_LLVM_PASS
+	bool
+	help
+	  An architecture should select this option if it uses the generic
+	  KCFIPass in LLVM to expand kCFI bundles instead of architecture-specific
+	  lowering.
+
 config CFI
 	bool "Use Kernel Control Flow Integrity (kCFI)"
 	default CFI_CLANG
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index 59288a2c1ad2..1414be493738 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -461,6 +461,12 @@ struct ftrace_likely_data {
 # define __nocfi
 #endif
 
+#if defined(CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS)
+# define __nocfi_generic	__nocfi
+#else
+# define __nocfi_generic
+#endif
+
 /*
  * Any place that could be marked with the "alloc_size" attribute is also
  * a place to be marked with the "malloc" attribute, except those that may

-- 
2.51.1

[PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Nathan Chancellor]

Prior to clang 22.0.0 [1], ARM did not have an architecture specific
kCFI bundle lowering in the backend, which may cause issues. Select
CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS to enable use of __nocfi_generic.

Link: https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de [1]
Link: https://github.com/ClangBuiltLinux/linux/issues/2124
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 arch/arm/Kconfig | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 2e3f93b690f4..4fb985b76e97 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -44,6 +44,8 @@ config ARM
 	select ARCH_USE_BUILTIN_BSWAP
 	select ARCH_USE_CMPXCHG_LOCKREF
 	select ARCH_USE_MEMTEST
+	# https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de
+	select ARCH_USES_CFI_GENERIC_LLVM_PASS if CLANG_VERSION < 220000
 	select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU
 	select ARCH_WANT_GENERAL_HUGETLB
 	select ARCH_WANT_IPC_PARSE_VERSION

-- 
2.51.1

[PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Nathan Chancellor]

When building drivers/net/ethernet/intel/idpf/xsk.c for ARCH=arm with
CONFIG_CFI=y using a version of LLVM prior to 22.0.0, there is a
BUILD_BUG_ON failure:

  $ cat arch/arm/configs/repro.config
  CONFIG_BPF_SYSCALL=y
  CONFIG_CFI=y
  CONFIG_IDPF=y
  CONFIG_XDP_SOCKETS=y

  $ make -skj"$(nproc)" ARCH=arm LLVM=1 clean defconfig repro.config drivers/net/ethernet/intel/idpf/xsk.o
  In file included from drivers/net/ethernet/intel/idpf/xsk.c:4:
  include/net/libeth/xsk.h:205:2: error: call to '__compiletime_assert_728' declared with 'error' attribute: BUILD_BUG_ON failed: !__builtin_constant_p(tmo == libeth_xsktmo)
    205 |         BUILD_BUG_ON(!__builtin_constant_p(tmo == libeth_xsktmo));
        |         ^
  ...

libeth_xdp_tx_xmit_bulk() indirectly calls libeth_xsk_xmit_fill_buf()
but these functions are marked as __always_inline so that the compiler
can turn these indirect calls into direct ones and see that the tmo
parameter to __libeth_xsk_xmit_fill_buf_md() is ultimately libeth_xsktmo
from idpf_xsk_xmit().

Unfortunately, the generic kCFI pass in LLVM expands the kCFI bundles
from the indirect calls in libeth_xdp_tx_xmit_bulk() in such a way that
later optimizations cannot turn these calls into direct ones, making the
BUILD_BUG_ON fail because it cannot be proved at compile time that tmo
is libeth_xsktmo.

Disable the generic kCFI pass for libeth_xdp_tx_xmit_bulk() to ensure
these indirect calls can always be turned into direct calls to avoid
this error.

Closes: https://github.com/ClangBuiltLinux/linux/issues/2124
Fixes: 9705d6552f58 ("idpf: implement Rx path for AF_XDP")
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 include/net/libeth/xdp.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/net/libeth/xdp.h b/include/net/libeth/xdp.h
index bc3507edd589..898723ab62e8 100644
--- a/include/net/libeth/xdp.h
+++ b/include/net/libeth/xdp.h
@@ -513,7 +513,7 @@ struct libeth_xdp_tx_desc {
  * can't fail, but can send less frames if there's no enough free descriptors
  * available. The actual free space is returned by @prep from the driver.
  */
-static __always_inline u32
+static __always_inline __nocfi_generic u32
 libeth_xdp_tx_xmit_bulk(const struct libeth_xdp_tx_frame *bulk, void *xdpsq,
 			u32 n, bool unroll, u64 priv,
 			u32 (*prep)(void *xdpsq, struct libeth_xdpsq *sq),

-- 
2.51.1

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Przemek Kitszel]

On 10/25/25 22:53, Nathan Chancellor wrote:
> When building drivers/net/ethernet/intel/idpf/xsk.c for ARCH=arm with
> CONFIG_CFI=y using a version of LLVM prior to 22.0.0, there is a
> BUILD_BUG_ON failure:
> 
>    $ cat arch/arm/configs/repro.config
>    CONFIG_BPF_SYSCALL=y
>    CONFIG_CFI=y
>    CONFIG_IDPF=y
>    CONFIG_XDP_SOCKETS=y
> 
>    $ make -skj"$(nproc)" ARCH=arm LLVM=1 clean defconfig repro.config drivers/net/ethernet/intel/idpf/xsk.o
>    In file included from drivers/net/ethernet/intel/idpf/xsk.c:4:
>    include/net/libeth/xsk.h:205:2: error: call to '__compiletime_assert_728' declared with 'error' attribute: BUILD_BUG_ON failed: !__builtin_constant_p(tmo == libeth_xsktmo)
>      205 |         BUILD_BUG_ON(!__builtin_constant_p(tmo == libeth_xsktmo));
>          |         ^
>    ...
> 
> libeth_xdp_tx_xmit_bulk() indirectly calls libeth_xsk_xmit_fill_buf()
> but these functions are marked as __always_inline so that the compiler
> can turn these indirect calls into direct ones and see that the tmo
> parameter to __libeth_xsk_xmit_fill_buf_md() is ultimately libeth_xsktmo
> from idpf_xsk_xmit().
> 
> Unfortunately, the generic kCFI pass in LLVM expands the kCFI bundles
> from the indirect calls in libeth_xdp_tx_xmit_bulk() in such a way that
> later optimizations cannot turn these calls into direct ones, making the
> BUILD_BUG_ON fail because it cannot be proved at compile time that tmo
> is libeth_xsktmo.
> 

sorry, but from regular driver developer perspective, just after reading
your commit messages, I'm unable to tell what the fix is about, and from
that follows a bigger issue: how to write code in the future to avoid
such issues (it would be infeasible to always wait for a LLVM specialist
to come up with a fix ;))

was the tricky case to call __always_inline func from another that was
marked the same? Would it be also the case if one of the functions would
not be marked with __always_inline attribute, but still end up inlined?

what would be the cost of the alternative naive solution, to always add
__nocfi_generic to functions marked __always_inline?
(technically you would redefine __always_inline to have also
__nocfi_generic for the config combinations that require that)

sorry for my ignorance of not reading any of the attached URLs

> Disable the generic kCFI pass for libeth_xdp_tx_xmit_bulk() to ensure
> these indirect calls can always be turned into direct calls to avoid
> this error.
> 
> Closes: https://github.com/ClangBuiltLinux/linux/issues/2124
> Fixes: 9705d6552f58 ("idpf: implement Rx path for AF_XDP")
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
>   include/net/libeth/xdp.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/net/libeth/xdp.h b/include/net/libeth/xdp.h
> index bc3507edd589..898723ab62e8 100644
> --- a/include/net/libeth/xdp.h
> +++ b/include/net/libeth/xdp.h
> @@ -513,7 +513,7 @@ struct libeth_xdp_tx_desc {
>    * can't fail, but can send less frames if there's no enough free descriptors
>    * available. The actual free space is returned by @prep from the driver.
>    */
> -static __always_inline u32
> +static __always_inline __nocfi_generic u32
>   libeth_xdp_tx_xmit_bulk(const struct libeth_xdp_tx_frame *bulk, void *xdpsq,
>   			u32 n, bool unroll, u64 priv,
>   			u32 (*prep)(void *xdpsq, struct libeth_xdpsq *sq),
> 

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Alexander Lobakin]

From: Nathan Chancellor <nathan@kernel.org>
Date: Sat, 25 Oct 2025 21:53:20 +0100

> When building drivers/net/ethernet/intel/idpf/xsk.c for ARCH=arm with
> CONFIG_CFI=y using a version of LLVM prior to 22.0.0, there is a
> BUILD_BUG_ON failure:
> 
>   $ cat arch/arm/configs/repro.config
>   CONFIG_BPF_SYSCALL=y
>   CONFIG_CFI=y
>   CONFIG_IDPF=y
>   CONFIG_XDP_SOCKETS=y
> 
>   $ make -skj"$(nproc)" ARCH=arm LLVM=1 clean defconfig repro.config drivers/net/ethernet/intel/idpf/xsk.o
>   In file included from drivers/net/ethernet/intel/idpf/xsk.c:4:
>   include/net/libeth/xsk.h:205:2: error: call to '__compiletime_assert_728' declared with 'error' attribute: BUILD_BUG_ON failed: !__builtin_constant_p(tmo == libeth_xsktmo)
>     205 |         BUILD_BUG_ON(!__builtin_constant_p(tmo == libeth_xsktmo));
>         |         ^

I've been wondering why I keep getting these reports for some time
already as my CI bot's never been able to trigger this. So seems like
it's CFI...

>   ...
> 
> libeth_xdp_tx_xmit_bulk() indirectly calls libeth_xsk_xmit_fill_buf()
> but these functions are marked as __always_inline so that the compiler
> can turn these indirect calls into direct ones and see that the tmo
> parameter to __libeth_xsk_xmit_fill_buf_md() is ultimately libeth_xsktmo
> from idpf_xsk_xmit().
> 
> Unfortunately, the generic kCFI pass in LLVM expands the kCFI bundles
> from the indirect calls in libeth_xdp_tx_xmit_bulk() in such a way that
> later optimizations cannot turn these calls into direct ones, making the
> BUILD_BUG_ON fail because it cannot be proved at compile time that tmo
> is libeth_xsktmo.
> 
> Disable the generic kCFI pass for libeth_xdp_tx_xmit_bulk() to ensure
> these indirect calls can always be turned into direct calls to avoid
> this error.

Hmmm,

For this patch:

Acked-by: Alexander Lobakin <aleksander.lobakin@intel.com>

However,

The XSk metadata infra in the kernel relies on that when we call
xsk_tx_metadata_request(), we pass a static const struct with our
callbacks and then the compiler makes all these calls direct.
This is not limited to libeth (although I realize that it triggered
this build failure due to the way how I pass these callbacks), every
driver which implements XSk Tx metadata and calls
xsk_tx_metadata_request() relies on that these calls will be direct,
otherwise there'll be such performance penalty that is unacceptable
for XSk speeds.
Maybe xsk_tx_metadata_request() should be __nocfi as well? Or all
the callers of it?

> 
> Closes: https://github.com/ClangBuiltLinux/linux/issues/2124
> Fixes: 9705d6552f58 ("idpf: implement Rx path for AF_XDP")
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
>  include/net/libeth/xdp.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/net/libeth/xdp.h b/include/net/libeth/xdp.h
> index bc3507edd589..898723ab62e8 100644
> --- a/include/net/libeth/xdp.h
> +++ b/include/net/libeth/xdp.h
> @@ -513,7 +513,7 @@ struct libeth_xdp_tx_desc {
>   * can't fail, but can send less frames if there's no enough free descriptors
>   * available. The actual free space is returned by @prep from the driver.
>   */
> -static __always_inline u32
> +static __always_inline __nocfi_generic u32
>  libeth_xdp_tx_xmit_bulk(const struct libeth_xdp_tx_frame *bulk, void *xdpsq,
>  			u32 n, bool unroll, u64 priv,
>  			u32 (*prep)(void *xdpsq, struct libeth_xdpsq *sq),

Thanks,
Olek

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Sami Tolvanen]

Hi Nathan,

On Sat, Oct 25, 2025 at 1:53 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Prior to clang 22.0.0 [1], ARM did not have an architecture specific
> kCFI bundle lowering in the backend, which may cause issues. Select
> CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS to enable use of __nocfi_generic.
>
> Link: https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de [1]
> Link: https://github.com/ClangBuiltLinux/linux/issues/2124
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
>  arch/arm/Kconfig | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 2e3f93b690f4..4fb985b76e97 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -44,6 +44,8 @@ config ARM
>         select ARCH_USE_BUILTIN_BSWAP
>         select ARCH_USE_CMPXCHG_LOCKREF
>         select ARCH_USE_MEMTEST
> +       # https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de
> +       select ARCH_USES_CFI_GENERIC_LLVM_PASS if CLANG_VERSION < 220000

Instead of working around issues with the generic pass, would it make
more sense to just disable arm32 CFI with older Clang versions
entirely? Linus, any thoughts?

Sami

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Nathan Chancellor]

Hi Przemek,

On Mon, Oct 27, 2025 at 12:09:47PM +0100, Przemek Kitszel wrote:
> sorry, but from regular driver developer perspective, just after reading
> your commit messages, I'm unable to tell what the fix is about, and from
> that follows a bigger issue: how to write code in the future to avoid
> such issues (it would be infeasible to always wait for a LLVM specialist
> to come up with a fix ;))

Sorry about that! To be fair, I am not sure most driver developers would
write something as subtle as this code to get themselves into a
situation where I or another LLVM person would need to come along to fix
it.

> was the tricky case to call __always_inline func from another that was
> marked the same? Would it be also the case if one of the functions would
> not be marked with __always_inline attribute, but still end up inlined?

No, I think the tricky case here is that the code depends on the
compiler being able to turn all these indirect calls, as the (*xmit)()
and (*prep)() parameters in libeth_xdp_tx_xmit_bulk(), into direct calls
through inlining. I find that depending on optimizations for correctness
(i.e., passing the assertion that the result of a equality test can be
proven true or false at compile time) is risky because it can result in
flexible but performant code but it is also sensitive to interactions
with other compiler internals and optimizations, resulting in changes
such as these.

> what would be the cost of the alternative naive solution, to always add
> __nocfi_generic to functions marked __always_inline?
> (technically you would redefine __always_inline to have also
> __nocfi_generic for the config combinations that require that)

No, you would not want to do this because if any indirect call remained
in the inlining chain, there would be no caller side CFI code generation
and that indirect call would automatically fail (if I understand
correctly), which is why commit 894af4a1cde6 ("objtool: Validate kCFI
calls") now validates this rule at compile time for x86_64 (which is not
affected by this issue since it does not use the generic LLVM kCFI pass
like ARM prior to 22.0.0 does).

> sorry for my ignorance of not reading any of the attached URLs

No worries, I am not sure how much more they would help with
understanding the problem, which is subtle. If I can clarify anything
further, please me know but I am not sure it will matter too much since
we will likely just forbid using the generic kCFI pass to avoid this
issue, rather than applying __nocfi_generic.

Cheers,
Nathan

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Nathan Chancellor]

On Mon, Oct 27, 2025 at 03:59:51PM +0100, Alexander Lobakin wrote:
> Hmmm,
> 
> For this patch:
> 
> Acked-by: Alexander Lobakin <aleksander.lobakin@intel.com>

Thanks a lot for taking a look, even if it seems like we might not
actually go the route of working around this.

> However,
> 
> The XSk metadata infra in the kernel relies on that when we call
> xsk_tx_metadata_request(), we pass a static const struct with our
> callbacks and then the compiler makes all these calls direct.
> This is not limited to libeth (although I realize that it triggered
> this build failure due to the way how I pass these callbacks), every
> driver which implements XSk Tx metadata and calls
> xsk_tx_metadata_request() relies on that these calls will be direct,
> otherwise there'll be such performance penalty that is unacceptable
> for XSk speeds.

Hmmmm, I am not really sure how you could guarantee that these calls are
turned direct from indirect aside from placing compile time assertions
around like this... when you say "there'll be such performance penalty
that is unacceptable for XSk speeds", does that mean that everything
will function correctly but slower than expected or does the lack of
proper speed result in functionality degredation?

> Maybe xsk_tx_metadata_request() should be __nocfi as well? Or all
> the callers of it?

I would only expect __nocfi_generic to be useful for avoiding a problem
such as this. __nocfi would be too big of a hammer because it would
cause definite problems if these calls were emitted as indirect ones, as
they would not have the CFI setup on the caller side, resulting in
problems that are now flagged by commit 894af4a1cde6 ("objtool: Validate
kCFI calls") in mainline. It sounds like it could be useful on
xsk_tx_metadata_request() if we decide to further pursue this series but
given we could just bump the version of LLVM necessary for CONFIG_CFI on
ARM, we may just go that route.

Cheers,
Nathan

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Nathan Chancellor]

Hi Sami,

On Mon, Oct 27, 2025 at 08:53:49AM -0700, Sami Tolvanen wrote:
> Instead of working around issues with the generic pass, would it make
> more sense to just disable arm32 CFI with older Clang versions
> entirely? Linus, any thoughts?

That would certainly get to the heart of the problem. I have no real
strong opinion about keeping these older versions working, especially
since we have no idea how many people are actively using CONFIG_CFI on
ARM. I will say that this particular issue is rather exceptional (i.e.,
I don't know how often this would really come up in the future) because
this code is relying on the fact that these indirect calls will be made
direct by the compiler and checking for it, which does not seem like it
would be really common in the kernel otherwise. We would likely have to
forbid future use of the generic pass as well.

Cheers,
Nathan

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Linus Walleij]

On Mon, Oct 27, 2025 at 4:54 PM Sami Tolvanen <samitolvanen@google.com> wrote:
> On Sat, Oct 25, 2025 at 1:53 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > Prior to clang 22.0.0 [1], ARM did not have an architecture specific
> > kCFI bundle lowering in the backend, which may cause issues. Select
> > CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS to enable use of __nocfi_generic.
> >
> > Link: https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de [1]
> > Link: https://github.com/ClangBuiltLinux/linux/issues/2124
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>

I didn't know Kees stepped in and fixed this, christmas comes
early this year! I had it on my TODO to do this or get someone to
do this, but now it turns out I don't have to.

> > +       # https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de
> > +       select ARCH_USES_CFI_GENERIC_LLVM_PASS if CLANG_VERSION < 220000
>
> Instead of working around issues with the generic pass, would it make
> more sense to just disable arm32 CFI with older Clang versions
> entirely? Linus, any thoughts?

We have people using this with the default compilers that come with
Debiand and Fedora. I would say as soon as the latest release of
the major distributions supports this, we can drop support for older
compilers.

Yours,
Linus Walleij

RE: [Intel-wired-lan] [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Loktionov, Aleksandr]

> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf
> Of Nathan Chancellor
> Sent: Saturday, October 25, 2025 10:53 PM
> To: Kees Cook <kees@kernel.org>; Lobakin, Aleksander
> <aleksander.lobakin@intel.com>; David S. Miller <davem@davemloft.net>;
> Eric Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>;
> Paolo Abeni <pabeni@redhat.com>
> Cc: Simon Horman <horms@kernel.org>; Nathan Chancellor
> <nathan@kernel.org>; Nick Desaulniers
> <nick.desaulniers+lkml@gmail.com>; Bill Wendling <morbo@google.com>;
> Justin Stitt <justinstitt@google.com>; Sami Tolvanen
> <samitolvanen@google.com>; Russell King <linux@armlinux.org.uk>;
> Nguyen, Anthony L <anthony.l.nguyen@intel.com>; Michal Kubiak
> <michal.kubiak@intel.com>; linux-kernel@vger.kernel.org;
> llvm@lists.linux.dev; linux-arm-kernel@lists.infradead.org;
> netdev@vger.kernel.org; intel-wired-lan@lists.osuosl.org
> Subject: [Intel-wired-lan] [PATCH 3/3] libeth: xdp: Disable generic
> kCFI pass for libeth_xdp_tx_xmit_bulk()
> 
> When building drivers/net/ethernet/intel/idpf/xsk.c for ARCH=arm with
> CONFIG_CFI=y using a version of LLVM prior to 22.0.0, there is a
> BUILD_BUG_ON failure:
> 
>   $ cat arch/arm/configs/repro.config
>   CONFIG_BPF_SYSCALL=y
>   CONFIG_CFI=y
>   CONFIG_IDPF=y
>   CONFIG_XDP_SOCKETS=y
> 
>   $ make -skj"$(nproc)" ARCH=arm LLVM=1 clean defconfig repro.config
> drivers/net/ethernet/intel/idpf/xsk.o
>   In file included from drivers/net/ethernet/intel/idpf/xsk.c:4:
>   include/net/libeth/xsk.h:205:2: error: call to
> '__compiletime_assert_728' declared with 'error' attribute:
> BUILD_BUG_ON failed: !__builtin_constant_p(tmo == libeth_xsktmo)
>     205 |         BUILD_BUG_ON(!__builtin_constant_p(tmo ==
> libeth_xsktmo));
>         |         ^
>   ...
> 
> libeth_xdp_tx_xmit_bulk() indirectly calls libeth_xsk_xmit_fill_buf()
> but these functions are marked as __always_inline so that the compiler
> can turn these indirect calls into direct ones and see that the tmo
> parameter to __libeth_xsk_xmit_fill_buf_md() is ultimately
> libeth_xsktmo from idpf_xsk_xmit().
> 
> Unfortunately, the generic kCFI pass in LLVM expands the kCFI bundles
> from the indirect calls in libeth_xdp_tx_xmit_bulk() in such a way
> that later optimizations cannot turn these calls into direct ones,
> making the BUILD_BUG_ON fail because it cannot be proved at compile
> time that tmo is libeth_xsktmo.
> 
> Disable the generic kCFI pass for libeth_xdp_tx_xmit_bulk() to ensure
> these indirect calls can always be turned into direct calls to avoid
> this error.
> 
> Closes: https://github.com/ClangBuiltLinux/linux/issues/2124
> Fixes: 9705d6552f58 ("idpf: implement Rx path for AF_XDP")
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
>  include/net/libeth/xdp.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/net/libeth/xdp.h b/include/net/libeth/xdp.h index
> bc3507edd589..898723ab62e8 100644
> --- a/include/net/libeth/xdp.h
> +++ b/include/net/libeth/xdp.h
> @@ -513,7 +513,7 @@ struct libeth_xdp_tx_desc {
>   * can't fail, but can send less frames if there's no enough free
> descriptors
>   * available. The actual free space is returned by @prep from the
> driver.
>   */
> -static __always_inline u32
> +static __always_inline __nocfi_generic u32
>  libeth_xdp_tx_xmit_bulk(const struct libeth_xdp_tx_frame *bulk, void
> *xdpsq,
>  			u32 n, bool unroll, u64 priv,
>  			u32 (*prep)(void *xdpsq, struct libeth_xdpsq
> *sq),
> 
> --
> 2.51.1

Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Alexander Lobakin]

From: Nathan Chancellor <nathan@kernel.org>
Date: Mon, 27 Oct 2025 13:54:09 -0700

> On Mon, Oct 27, 2025 at 03:59:51PM +0100, Alexander Lobakin wrote:
>> Hmmm,
>>
>> For this patch:
>>
>> Acked-by: Alexander Lobakin <aleksander.lobakin@intel.com>
> 
> Thanks a lot for taking a look, even if it seems like we might not
> actually go the route of working around this.
> 
>> However,
>>
>> The XSk metadata infra in the kernel relies on that when we call
>> xsk_tx_metadata_request(), we pass a static const struct with our
>> callbacks and then the compiler makes all these calls direct.
>> This is not limited to libeth (although I realize that it triggered
>> this build failure due to the way how I pass these callbacks), every
>> driver which implements XSk Tx metadata and calls
>> xsk_tx_metadata_request() relies on that these calls will be direct,
>> otherwise there'll be such performance penalty that is unacceptable
>> for XSk speeds.
> 
> Hmmmm, I am not really sure how you could guarantee that these calls are
> turned direct from indirect aside from placing compile time assertions
> around like this... when you say "there'll be such performance penalty

You mean in case of CFI or in general? Because currently on both GCC and
Clang with both OPTIMIZE_FOR_{SIZE,SPEED} they get inlined in every driver.

> that is unacceptable for XSk speeds", does that mean that everything
> will function correctly but slower than expected or does the lack of
> proper speed result in functionality degredation?

Nothing would break, just work way slower than expected.
xsk_tx_metadata_request() is called for each Tx packet (when Tx metadata
is enabled). Average XSK Tx perf is ~35-40 Mpps (millions of packets per
second), often [much] higher. Having an indirect call there would divide
it by n.

> 
>> Maybe xsk_tx_metadata_request() should be __nocfi as well? Or all
>> the callers of it?
> 
> I would only expect __nocfi_generic to be useful for avoiding a problem
> such as this. __nocfi would be too big of a hammer because it would

Yep, sorry, I actually meant __nocfi_generic...

> cause definite problems if these calls were emitted as indirect ones, as
> they would not have the CFI setup on the caller side, resulting in
> problems that are now flagged by commit 894af4a1cde6 ("objtool: Validate
> kCFI calls") in mainline. It sounds like it could be useful on
> xsk_tx_metadata_request() if we decide to further pursue this series but
> given we could just bump the version of LLVM necessary for CONFIG_CFI on
> ARM, we may just go that route.
> 
> Cheers,
> Nathan

Thanks,
Olek

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Nathan Chancellor]

On Mon, Oct 27, 2025 at 11:56:21PM +0100, Linus Walleij wrote:
> On Mon, Oct 27, 2025 at 4:54 PM Sami Tolvanen <samitolvanen@google.com> wrote:
> > Instead of working around issues with the generic pass, would it make
> > more sense to just disable arm32 CFI with older Clang versions
> > entirely? Linus, any thoughts?
> 
> We have people using this with the default compilers that come with
> Debiand and Fedora. I would say as soon as the latest release of
> the major distributions supports this, we can drop support for older
> compilers.

Okay, I think that is reasonable enough. This is not a very large
workaround and I do not expect these type of workarounds to be necessary
frequently so I think it is worth keeping this working if people are
actually using it. That means we could mandate the backend version of
kCFI for ARM with Debian Forky in 2027.

Cheers,
Nathan

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Sami Tolvanen]

On Tue, Oct 28, 2025 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Mon, Oct 27, 2025 at 11:56:21PM +0100, Linus Walleij wrote:
> > On Mon, Oct 27, 2025 at 4:54 PM Sami Tolvanen <samitolvanen@google.com> wrote:
> > > Instead of working around issues with the generic pass, would it make
> > > more sense to just disable arm32 CFI with older Clang versions
> > > entirely? Linus, any thoughts?
> >
> > We have people using this with the default compilers that come with
> > Debiand and Fedora. I would say as soon as the latest release of
> > the major distributions supports this, we can drop support for older
> > compilers.
>
> Okay, I think that is reasonable enough. This is not a very large
> workaround and I do not expect these type of workarounds to be necessary
> frequently so I think it is worth keeping this working if people are
> actually using it. That means we could mandate the backend version of
> kCFI for ARM with Debian Forky in 2027.

Yeah, it's a bit unfortunate, but I agree that we shouldn't break
existing users until newer Clang is actually available in distros.

Sami

Re: [PATCH 3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

[Nathan Chancellor]

On Tue, Oct 28, 2025 at 05:29:30PM +0100, Alexander Lobakin wrote:
> From: Nathan Chancellor <nathan@kernel.org>
> Date: Mon, 27 Oct 2025 13:54:09 -0700
> 
> > On Mon, Oct 27, 2025 at 03:59:51PM +0100, Alexander Lobakin wrote:
> >> Hmmm,
> >>
> >> For this patch:
> >>
> >> Acked-by: Alexander Lobakin <aleksander.lobakin@intel.com>
> > 
> > Thanks a lot for taking a look, even if it seems like we might not
> > actually go the route of working around this.
> > 
> >> However,
> >>
> >> The XSk metadata infra in the kernel relies on that when we call
> >> xsk_tx_metadata_request(), we pass a static const struct with our
> >> callbacks and then the compiler makes all these calls direct.
> >> This is not limited to libeth (although I realize that it triggered
> >> this build failure due to the way how I pass these callbacks), every
> >> driver which implements XSk Tx metadata and calls
> >> xsk_tx_metadata_request() relies on that these calls will be direct,
> >> otherwise there'll be such performance penalty that is unacceptable
> >> for XSk speeds.
> > 
> > Hmmmm, I am not really sure how you could guarantee that these calls are
> > turned direct from indirect aside from placing compile time assertions
> > around like this... when you say "there'll be such performance penalty
> 
> You mean in case of CFI or in general? Because currently on both GCC and
> Clang with both OPTIMIZE_FOR_{SIZE,SPEED} they get inlined in every driver.

I mean in general but obviously that sort of optimization is high value
for the compiler to perform so I would only expect it not to occur in
extreme cases like sanitizers being enabled; I would expect no issues
when using a backend CFI implementation

> > that is unacceptable for XSk speeds", does that mean that everything
> > will function correctly but slower than expected or does the lack of
> > proper speed result in functionality degredation?
> 
> Nothing would break, just work way slower than expected.
> xsk_tx_metadata_request() is called for each Tx packet (when Tx metadata
> is enabled). Average XSK Tx perf is ~35-40 Mpps (millions of packets per
> second), often [much] higher. Having an indirect call there would divide
> it by n.

Ah okay.

> >> Maybe xsk_tx_metadata_request() should be __nocfi as well? Or all
> >> the callers of it?
> > 
> > I would only expect __nocfi_generic to be useful for avoiding a problem
> > such as this. __nocfi would be too big of a hammer because it would
> 
> Yep, sorry, I actually meant __nocfi_generic...

I figured, just wanted to make sure! This series needs to go to mainline
sooner rather than later, so maybe xsk_tx_metadata_request() could pick
up __nocfi_generic as a future change to net-next since there is no
obvious breakage? 32-bit ARM is the only architecture affected by this
change since all other architectures that support kCFI have a backend
specific lowering and I am guessing very few people would actually
notice this problem in practice.

Thanks again for chiming in and taking a look!

Cheers,
Nathan

Re: [PATCH 2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS

[Kees Cook]

On Mon, Oct 27, 2025 at 11:56:21PM +0100, Linus Walleij wrote:
> On Mon, Oct 27, 2025 at 4:54 PM Sami Tolvanen <samitolvanen@google.com> wrote:
> > On Sat, Oct 25, 2025 at 1:53 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > Prior to clang 22.0.0 [1], ARM did not have an architecture specific
> > > kCFI bundle lowering in the backend, which may cause issues. Select
> > > CONFIG_ARCH_USES_CFI_GENERIC_LLVM_PASS to enable use of __nocfi_generic.
> > >
> > > Link: https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de [1]
> > > Link: https://github.com/ClangBuiltLinux/linux/issues/2124
> > > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> 
> I didn't know Kees stepped in and fixed this, christmas comes
> early this year! I had it on my TODO to do this or get someone to
> do this, but now it turns out I don't have to.
> 
> > > +       # https://github.com/llvm/llvm-project/commit/d130f402642fba3d065aacb506cb061c899558de
> > > +       select ARCH_USES_CFI_GENERIC_LLVM_PASS if CLANG_VERSION < 220000
> >
> > Instead of working around issues with the generic pass, would it make
> > more sense to just disable arm32 CFI with older Clang versions
> > entirely? Linus, any thoughts?
> 
> We have people using this with the default compilers that come with
> Debiand and Fedora. I would say as soon as the latest release of
> the major distributions supports this, we can drop support for older
> compilers.

Okay, it seems like the consensus is to take this series so we don't
break existing users, even if they are rare.

Unless someone screams, I'll take this via the hardening tree...

-- 
Kees Cook

Re: [PATCH 0/3] Resolve ARM kCFI build failure in idpf xsk.c

[Kees Cook]

On Sat, 25 Oct 2025 21:53:17 +0100, Nathan Chancellor wrote:
> This series resolves a build failure that is seen in
> drivers/net/ethernet/intel/idpf/xsk.c after commit 9705d6552f58 ("idpf:
> implement Rx path for AF_XDP") in 6.18-rc1 with ARCH=arm and
> CONFIG_CFI=y. See patch 3 for a simplified reproducer on top of
> defconfig.
> 
> I think this could go via hardening or net.
> 
> [...]

Applied to for-linus/hardening, thanks!

[1/3] compiler_types: Introduce __nocfi_generic
      https://git.kernel.org/kees/c/39c89ee6e9c4
[2/3] ARM: Select ARCH_USES_CFI_GENERIC_LLVM_PASS
      https://git.kernel.org/kees/c/1ed9e6b1004f
[3/3] libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()
      https://git.kernel.org/kees/c/c57f5fee54df

Take care,

-- 
Kees Cook