[PATCH bpf 1/2] bpf, x86: Fix FineIBT #CP in BPF dispatcher's indirect-jump fallback

[PATCH bpf 1/2] bpf, x86: Fix FineIBT #CP in BPF dispatcher's indirect-jump fallback

[David Windsor]

commit 4f9087f16651 ("x86/cfi,bpf: Fix BPF JIT call") updated
emit_cfi() to emit FineIBT preambles for JIT-compiled BPF programs, but
did not update emit_bpf_dispatcher(). When prog->bpf_func is not in the
dispatcher table (e.g. an XDP_REDIRECT target program in a CPUMAP or
DEVMAP, or BPF_PROG_TEST_RUN of an unattached prog), the dispatcher
still jumps directly to prog->bpf_func. Without a FineIBT caller
sequence, under FineIBT this raises #CP:

  Missing ENDBR: __cfi_bpf_prog_..._xdp_dispatcher_unattached+0x10/0x10
  ------------[ cut here ]------------
  kernel BUG at arch/x86/kernel/cet.c:133!
  Oops: invalid opcode: 0000 [#1] SMP NOPTI
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
  RIP: 0010:do_kernel_cp_fault+0x126/0x130
  Call Trace:
   <TASK>
   exc_control_protection+0x46/0x80
   asm_exc_control_protection+0x2b/0x30
  RIP: 0010:__cfi_bpf_prog_..._xdp_dispatcher_unattached+0x10/0x10
   bpf_prog_test_run+0xda/0x1b0
   __sys_bpf+0x70b/0x990
   __x64_sys_bpf+0x2d/0x50
   x64_sys_call+0x1079/0x2d40
   do_syscall_64+0x12a/0x3c0
   entry_SYSCALL_64_after_hwframe+0x76/0x7e
   </TASK>

To fix this, we precede the indirect jump in emit_bpf_dispatcher() with
the FineIBT caller sequence: load cfi_bpf_hash into %eax and subtract
cfi_get_offset() from the target so it enters the preamble's real ENDBR.
Only needed for CFI_FINEIBT.

Fixes: 4f9087f16651 ("x86/cfi,bpf: Fix BPF JIT call")
Cc: stable@vger.kernel.org
Signed-off-by: David Windsor <dwindsor@gmail.com>
---
 arch/x86/net/bpf_jit_comp.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index de7515ea1bea..1c8249d8ca3b 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3740,6 +3740,15 @@ static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs, u8 *image,
 		if (err)
 			return err;
 
+		/* If running under FineIBT, enter the preamble so the following
+		 * indirect jump lands on a real ENDBR instead of the poison.
+		 */
+		if (cfi_mode == CFI_FINEIBT) {
+			EMIT1_off32(0xb8, cfi_bpf_hash);	/* mov $cfi_bpf_hash, %eax */
+			EMIT1(add_1mod(0x48, BPF_REG_3));	/* sub rdx, cfi_get_offset() */
+			EMIT2_off32(0x81, add_1reg(0xE8, BPF_REG_3), cfi_get_offset());
+		}
+
 		emit_indirect_jump(&prog, BPF_REG_3 /* R3 -> rdx */, image + (prog - buf));
 
 		*pprog = prog;
-- 
2.53.0

[PATCH bpf 2/2] selftests/bpf: Add XDP dispatcher FineIBT regression test

[David Windsor]

Add a regression test for the x86 BPF dispatcher's indirect-jump fallback
under FineIBT.

This test can crash the kernel on an unfixed FineIBT build and so should
be run in a VM.

Signed-off-by: David Windsor <dwindsor@gmail.com>
---
 .../bpf/prog_tests/xdp_dispatcher_fineibt.c   | 47 +++++++++++++++++++
 .../bpf/progs/xdp_dispatcher_fineibt.c        | 18 +++++++
 2 files changed, 65 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
 create mode 100644 tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c

diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c b/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
new file mode 100644
index 000000000000..2d3fc4034eb9
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 David Windsor */
+
+/*
+ * Regression test for the x86 BPF dispatcher's indirect-jump fallback
+ * under FineIBT.
+ *
+ * WARNING
+ * -------
+ *  This test can crash the kernel, thus should be run in a VM.
+ */
+#include <uapi/linux/if_link.h>
+#include <test_progs.h>
+#include <network_helpers.h>
+#include "xdp_dispatcher_fineibt.skel.h"
+
+#define IFINDEX_LO		1
+
+void test_xdp_dispatcher_fineibt(void)
+{
+	struct xdp_dispatcher_fineibt *skel;
+	int err, attached_fd, unattached_fd;
+
+	LIBBPF_OPTS(bpf_test_run_opts, topts,
+		.data_in = &pkt_v4,
+		.data_size_in = sizeof(pkt_v4),
+	);
+
+	skel = xdp_dispatcher_fineibt__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "skel_open_and_load"))
+		return;
+
+	attached_fd = bpf_program__fd(skel->progs.xdp_dispatcher_attached);
+	unattached_fd = bpf_program__fd(skel->progs.xdp_dispatcher_unattached);
+
+	err = bpf_xdp_attach(IFINDEX_LO, attached_fd, XDP_FLAGS_SKB_MODE, NULL);
+	if (!ASSERT_OK(err, "attach_prog"))
+		goto out;
+
+	err = bpf_prog_test_run_opts(unattached_fd, &topts);
+	ASSERT_OK(err, "unattached_test_run");
+	ASSERT_EQ(topts.retval, XDP_PASS, "unattached_retval");
+
+	bpf_xdp_detach(IFINDEX_LO, XDP_FLAGS_SKB_MODE, NULL);
+out:
+	xdp_dispatcher_fineibt__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c b/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c
new file mode 100644
index 000000000000..b49b84f01c8e
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c
@@ -0,0 +1,18 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 David Windsor */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+SEC("xdp")
+int xdp_dispatcher_attached(struct xdp_md *ctx)
+{
+	return XDP_PASS;
+}
+
+SEC("xdp")
+int xdp_dispatcher_unattached(struct xdp_md *ctx)
+{
+	return XDP_PASS;
+}
+
+char _license[] SEC("license") = "GPL";
-- 
2.53.0

Re: [PATCH bpf 2/2] selftests/bpf: Add XDP dispatcher FineIBT regression test

[Leon Hwang]

On 8/7/26 10:57, David Windsor wrote:
> Add a regression test for the x86 BPF dispatcher's indirect-jump fallback
> under FineIBT.
> 
> This test can crash the kernel on an unfixed FineIBT build and so should
> be run in a VM.
> 
> Signed-off-by: David Windsor <dwindsor@gmail.com>
> ---
>  .../bpf/prog_tests/xdp_dispatcher_fineibt.c   | 47 +++++++++++++++++++
>  .../bpf/progs/xdp_dispatcher_fineibt.c        | 18 +++++++
>  2 files changed, 65 insertions(+)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
>  create mode 100644 tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c
> 
> diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c b/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
> new file mode 100644
> index 000000000000..2d3fc4034eb9
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/xdp_dispatcher_fineibt.c
> @@ -0,0 +1,47 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (c) 2026 David Windsor */
> +
> +/*
> + * Regression test for the x86 BPF dispatcher's indirect-jump fallback
> + * under FineIBT.
> + *
> + * WARNING
> + * -------
> + *  This test can crash the kernel, thus should be run in a VM.
> + */
> +#include <uapi/linux/if_link.h>
> +#include <test_progs.h>
> +#include <network_helpers.h>
> +#include "xdp_dispatcher_fineibt.skel.h"
> +
> +#define IFINDEX_LO		1
> +
> +void test_xdp_dispatcher_fineibt(void)
> +{
> +	struct xdp_dispatcher_fineibt *skel;
> +	int err, attached_fd, unattached_fd;
> +
> +	LIBBPF_OPTS(bpf_test_run_opts, topts,
> +		.data_in = &pkt_v4,
> +		.data_size_in = sizeof(pkt_v4),
> +	);
> +
> +	skel = xdp_dispatcher_fineibt__open_and_load();
> +	if (!ASSERT_OK_PTR(skel, "skel_open_and_load"))
> +		return;
> +
> +	attached_fd = bpf_program__fd(skel->progs.xdp_dispatcher_attached);
> +	unattached_fd = bpf_program__fd(skel->progs.xdp_dispatcher_unattached);
> +
> +	err = bpf_xdp_attach(IFINDEX_LO, attached_fd, XDP_FLAGS_SKB_MODE, NULL);
> +	if (!ASSERT_OK(err, "attach_prog"))
> +		goto out;
> +
> +	err = bpf_prog_test_run_opts(unattached_fd, &topts);
> +	ASSERT_OK(err, "unattached_test_run");
> +	ASSERT_EQ(topts.retval, XDP_PASS, "unattached_retval");
> +
> +	bpf_xdp_detach(IFINDEX_LO, XDP_FLAGS_SKB_MODE, NULL);
> +out:
> +	xdp_dispatcher_fineibt__destroy(skel);
> +}
> diff --git a/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c b/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c
> new file mode 100644
> index 000000000000..b49b84f01c8e
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/xdp_dispatcher_fineibt.c
> @@ -0,0 +1,18 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (c) 2026 David Windsor */
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +
> +SEC("xdp")
> +int xdp_dispatcher_attached(struct xdp_md *ctx)
> +{
> +	return XDP_PASS;
> +}
> +
> +SEC("xdp")
> +int xdp_dispatcher_unattached(struct xdp_md *ctx)
> +{
> +	return XDP_PASS;
> +}
> +
> +char _license[] SEC("license") = "GPL";


Instead of creating xdp_dispatcher_fineibt.c, better to reuse
xdp_dummy.c. Then, in xdp_dispatcher_fineibt.c, you can create two
xdp_dummy skels, one for attaching, one for running.

And, add the test to xdp_attach.c instead of new file
prog_tests/xdp_dispatcher_fineibt.c?

Thanks,
Leon

Re: [PATCH bpf 1/2] bpf, x86: Fix FineIBT #CP in BPF dispatcher's indirect-jump fallback

[Leon Hwang]

On 8/7/26 10:57, David Windsor wrote:
> commit 4f9087f16651 ("x86/cfi,bpf: Fix BPF JIT call") updated
> emit_cfi() to emit FineIBT preambles for JIT-compiled BPF programs, but
> did not update emit_bpf_dispatcher(). When prog->bpf_func is not in the
> dispatcher table (e.g. an XDP_REDIRECT target program in a CPUMAP or
> DEVMAP, or BPF_PROG_TEST_RUN of an unattached prog), the dispatcher
> still jumps directly to prog->bpf_func. Without a FineIBT caller
> sequence, under FineIBT this raises #CP:
> 
>   Missing ENDBR: __cfi_bpf_prog_..._xdp_dispatcher_unattached+0x10/0x10
>   ------------[ cut here ]------------
>   kernel BUG at arch/x86/kernel/cet.c:133!
>   Oops: invalid opcode: 0000 [#1] SMP NOPTI
>   Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
>   RIP: 0010:do_kernel_cp_fault+0x126/0x130
>   Call Trace:
>    <TASK>
>    exc_control_protection+0x46/0x80
>    asm_exc_control_protection+0x2b/0x30
>   RIP: 0010:__cfi_bpf_prog_..._xdp_dispatcher_unattached+0x10/0x10
>    bpf_prog_test_run+0xda/0x1b0
>    __sys_bpf+0x70b/0x990
>    __x64_sys_bpf+0x2d/0x50
>    x64_sys_call+0x1079/0x2d40
>    do_syscall_64+0x12a/0x3c0
>    entry_SYSCALL_64_after_hwframe+0x76/0x7e
>    </TASK>
> 
> To fix this, we precede the indirect jump in emit_bpf_dispatcher() with
> the FineIBT caller sequence: load cfi_bpf_hash into %eax and subtract
> cfi_get_offset() from the target so it enters the preamble's real ENDBR.
> Only needed for CFI_FINEIBT.
> 
> Fixes: 4f9087f16651 ("x86/cfi,bpf: Fix BPF JIT call")
> Cc: stable@vger.kernel.org
> Signed-off-by: David Windsor <dwindsor@gmail.com>
> ---
>  arch/x86/net/bpf_jit_comp.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index de7515ea1bea..1c8249d8ca3b 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -3740,6 +3740,15 @@ static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs, u8 *image,
>  		if (err)
>  			return err;
>  
> +		/* If running under FineIBT, enter the preamble so the following
> +		 * indirect jump lands on a real ENDBR instead of the poison.
> +		 */


Comment style: /* should stay at its own line.

> +		if (cfi_mode == CFI_FINEIBT) {
> +			EMIT1_off32(0xb8, cfi_bpf_hash);	/* mov $cfi_bpf_hash, %eax */
> +			EMIT1(add_1mod(0x48, BPF_REG_3));	/* sub rdx, cfi_get_offset() */
> +			EMIT2_off32(0x81, add_1reg(0xE8, BPF_REG_3), cfi_get_offset());
> +		}


Better to emit_cfi()?

Thanks,
Leon

> +
>  		emit_indirect_jump(&prog, BPF_REG_3 /* R3 -> rdx */, image + (prog - buf));
>  
>  		*pprog = prog;

Re: [PATCH bpf 1/2] bpf, x86: Fix FineIBT #CP in BPF dispatcher's indirect-jump fallback

[David Windsor]

On Tue, Jul 7, 2026 at 11:30 PM Leon Hwang <leon.hwang@linux.dev> wrote:
>
> Comment style: /* should stay at its own line.
>
> > +             if (cfi_mode == CFI_FINEIBT) {
> > +                     EMIT1_off32(0xb8, cfi_bpf_hash);        /* mov $cfi_bpf_hash, %eax */
> > +                     EMIT1(add_1mod(0x48, BPF_REG_3));       /* sub rdx, cfi_get_offset() */
> > +                     EMIT2_off32(0x81, add_1reg(0xE8, BPF_REG_3), cfi_get_offset());
> > +             }
>
>
> Better to emit_cfi()?
>

Yes, thanks. Will do this for v2.