[PATCH bpf-next 0/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[PATCH bpf-next 0/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[Leon Hwang]

The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed a NULL pointer panic, but
does not support my case[0] that I want to update attached freplace prog to
PROG_ARRAY map.

This patchset fixes it to support updating attached freplace prog to
PROG_ARRAY map.

[0] https://lore.kernel.org/bpf/20240602122421.50892-1-hffilwlqm@gmail.com/

RFC PATCH -> v1:
 * Respin the PATCH with updating its message.

Leon Hwang (2):
  bpf: Fix updating attached freplace prog to PROG_ARRAY map
  selftests/bpf: Add testcase for updating attached freplace prog to
    PROG_ARRAY map

 include/linux/bpf_verifier.h                  |  4 +-
 .../selftests/bpf/prog_tests/tailcalls.c      | 76 ++++++++++++++++++-
 .../selftests/bpf/progs/tailcall_freplace.c   | 33 ++++++++
 .../testing/selftests/bpf/progs/tc_bpf2bpf.c  | 23 ++++++
 4 files changed, 133 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
 create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c

-- 
2.44.0

[PATCH bpf-next 1/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[Leon Hwang]

The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed the following panic,
which was caused by updating attached freplace prog to PROG_ARRAY map.

But, it does not support updating attached freplace prog to PROG_ARRAY
map.

[309049.036402] BUG: kernel NULL pointer dereference, address: 0000000000000004
[309049.036419] #PF: supervisor read access in kernel mode
[309049.036426] #PF: error_code(0x0000) - not-present page
[309049.036432] PGD 0 P4D 0
[309049.036437] Oops: 0000 [#1] PREEMPT SMP NOPTI
[309049.036444] CPU: 2 PID: 788148 Comm: test_progs Not tainted 6.8.0-31-generic #31-Ubuntu
[309049.036465] Hardware name: VMware, Inc. VMware20,1/440BX Desktop Reference Platform, BIOS VMW201.00V.21805430.B64.2305221830 05/22/2023
[309049.036477] RIP: 0010:bpf_prog_map_compatible+0x2a/0x140
[309049.036488] Code: 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 44 8b 6e 04 48 89 f3 41 83 fd 1c 75 0c 48 8b 46 38 48 8b 40 70 <44> 8b 68 04 f6 43 03 01 75 1c 48 8b 43 38 44 0f b6 a0 89 00 00 00
[309049.036505] RSP: 0018:ffffb2e080fd7ce0 EFLAGS: 00010246
[309049.036513] RAX: 0000000000000000 RBX: ffffb2e0807c1000 RCX: 0000000000000000
[309049.036521] RDX: 0000000000000000 RSI: ffffb2e0807c1000 RDI: ffff990290259e00
[309049.036528] RBP: ffffb2e080fd7d08 R08: 0000000000000000 R09: 0000000000000000
[309049.036536] R10: 0000000000000000 R11: 0000000000000000 R12: ffff990290259e00
[309049.036543] R13: 000000000000001c R14: ffff990290259e00 R15: ffff99028e29c400
[309049.036551] FS:  00007b82cbc28140(0000) GS:ffff9903b3f00000(0000) knlGS:0000000000000000
[309049.036559] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[309049.036566] CR2: 0000000000000004 CR3: 0000000101286002 CR4: 00000000003706f0
[309049.036573] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[309049.036581] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[309049.036588] Call Trace:
[309049.036592]  <TASK>
[309049.036597]  ? show_regs+0x6d/0x80
[309049.036604]  ? __die+0x24/0x80
[309049.036619]  ? page_fault_oops+0x99/0x1b0
[309049.036628]  ? do_user_addr_fault+0x2ee/0x6b0
[309049.036634]  ? exc_page_fault+0x83/0x1b0
[309049.036641]  ? asm_exc_page_fault+0x27/0x30
[309049.036649]  ? bpf_prog_map_compatible+0x2a/0x140
[309049.036656]  prog_fd_array_get_ptr+0x2c/0x70
[309049.036664]  bpf_fd_array_map_update_elem+0x37/0x130
[309049.036671]  bpf_map_update_value+0x1d3/0x260
[309049.036677]  map_update_elem+0x1fa/0x360
[309049.036683]  __sys_bpf+0x54c/0xa10
[309049.036689]  __x64_sys_bpf+0x1a/0x30
[309049.036694]  x64_sys_call+0x1936/0x25c0
[309049.036700]  do_syscall_64+0x7f/0x180
[309049.036706]  ? do_syscall_64+0x8c/0x180
[309049.036712]  ? do_syscall_64+0x8c/0x180
[309049.036717]  ? irqentry_exit+0x43/0x50
[309049.036723]  ? common_interrupt+0x54/0xb0
[309049.036729]  entry_SYSCALL_64_after_hwframe+0x73/0x7b

Since commit 1c123c567fb138eb ("bpf: Resolve fext program type when
checking map compatibility"), freplace prog can be used as tail-callee
of its target prog.
And the commit 3aac1ead5eb6b76f ("bpf: Move prog->aux->linked_prog and
trampoline into bpf_link on attach") sets prog->aux->dst_prog as NULL
when attach freplace prog to its target.

Then, as for following example:

tailcall_freplace.c:

// SPDX-License-Identifier: GPL-2.0

\#include <linux/bpf.h>
\#include <bpf/bpf_helpers.h>
\#include "bpf_legacy.h"

struct {
	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
	__uint(max_entries, 1);
	__uint(key_size, sizeof(__u32));
	__uint(value_size, sizeof(__u32));
} jmp_table SEC(".maps");

int count = 0;

__noinline int
subprog(struct __sk_buff *skb)
{
	volatile int ret = 1;

	count++;

	bpf_tail_call_static(skb, &jmp_table, 0);

	return ret;
}

SEC("freplace")
int entry(struct __sk_buff *skb)
{
	return subprog(skb);
}

char __license[] SEC("license") = "GPL";

tc_bpf2bpf.c:

// SPDX-License-Identifier: GPL-2.0

\#include <linux/bpf.h>
\#include <bpf/bpf_helpers.h>
\#include "bpf_legacy.h"

__noinline int
subprog(struct __sk_buff *skb)
{
	volatile int ret = 1;

	return ret;
}

SEC("tc")
int entry(struct __sk_buff *skb)
{
	return subprog(skb);
}

char __license[] SEC("license") = "GPL";

And freplace entry prog's target is the tc subprog.

After loading, the freplace jmp_table's owner type is
BPF_PROG_TYPE_SCHED_CLS.

Next, after attaching freplace prog to tc subprog, its prog->aux->
dst_prog is NULL.

Next, when update freplace prog to jmp_table, bpf_prog_map_compatible()
returns false because resolve_prog_type() returns BPF_PROG_TYPE_EXT instead
of BPF_PROG_TYPE_SCHED_CLS.

With this patch, resolve_prog_type() returns BPF_PROG_TYPE_SCHED_CLS to
support updating attached freplace prog to PROG_ARRY map for this
example.

Fixes: f7866c358733 ("bpf: Fix null pointer dereference in resolve_prog_type() for BPF_PROG_TYPE_EXT")
Cc: Toke Høiland-Jørgensen <toke@redhat.com>
Cc: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
---
 include/linux/bpf_verifier.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 5cea15c81b8a8..387e034e73d0e 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -874,8 +874,8 @@ static inline u32 type_flag(u32 type)
 /* only use after check_attach_btf_id() */
 static inline enum bpf_prog_type resolve_prog_type(const struct bpf_prog *prog)
 {
-	return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->dst_prog) ?
-		prog->aux->dst_prog->type : prog->type;
+	return prog->type == BPF_PROG_TYPE_EXT ?
+		prog->aux->saved_dst_prog_type : prog->type;
 }
 
 static inline bool bpf_prog_check_recur(const struct bpf_prog *prog)
-- 
2.44.0

[PATCH bpf-next 2/2] selftests/bpf: Add testcase for updating attached freplace prog to PROG_ARRAY map

[Leon Hwang]

Add a selftest to confirm the issue, which gets -EINVAL when update
attached freplace prog to PROG_ARRAY map, has been fixed.

cd tools/testing/selftests/bpf; ./test_progs -t tailcalls
327/25  tailcalls/tailcall_freplace:OK
327     tailcalls:OK
Summary: 1/25 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
---
 .../selftests/bpf/prog_tests/tailcalls.c      | 76 ++++++++++++++++++-
 .../selftests/bpf/progs/tailcall_freplace.c   | 33 ++++++++
 .../testing/selftests/bpf/progs/tc_bpf2bpf.c  | 23 ++++++
 3 files changed, 131 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
 create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c

diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
index e01fabb8cc415..f1145601c0005 100644
--- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c
+++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
@@ -5,7 +5,8 @@
 #include "tailcall_poke.skel.h"
 #include "tailcall_bpf2bpf_hierarchy2.skel.h"
 #include "tailcall_bpf2bpf_hierarchy3.skel.h"
-
+#include "tailcall_freplace.skel.h"
+#include "tc_bpf2bpf.skel.h"
 
 /* test_tailcall_1 checks basic functionality by patching multiple locations
  * in a single program for a single tail call slot with nop->jmp, jmp->nop
@@ -1495,6 +1496,77 @@ static void test_tailcall_bpf2bpf_hierarchy_3(void)
 	RUN_TESTS(tailcall_bpf2bpf_hierarchy3);
 }
 
+/* test_tailcall_freplace checks that the attached freplace prog is OK to
+ * update to PROG_ARRAY map.
+ */
+static void test_tailcall_freplace(void)
+{
+	struct tailcall_freplace *fr_skel = NULL;
+	struct tc_bpf2bpf *tc_skel = NULL;
+	struct bpf_link *fr_link = NULL;
+	int prog_fd, map_fd;
+	char buff[128] = {};
+	int err, key;
+
+	LIBBPF_OPTS(bpf_test_run_opts, topts,
+		    .data_in = buff,
+		    .data_size_in = sizeof(buff),
+		    .repeat = 1,
+	);
+
+	fr_skel = tailcall_freplace__open();
+	if (!ASSERT_OK_PTR(fr_skel, "open fr_skel"))
+		goto out;
+
+	tc_skel = tc_bpf2bpf__open_and_load();
+	if (!ASSERT_OK_PTR(tc_skel, "open tc_skel"))
+		goto out;
+
+	prog_fd = bpf_program__fd(tc_skel->progs.entry);
+	if (!ASSERT_GE(prog_fd, 0, "tc_skel entry prog_id"))
+		goto out;
+
+	err = bpf_program__set_attach_target(fr_skel->progs.entry,
+					     prog_fd, "subprog");
+	if (!ASSERT_OK(err, "set_attach_target"))
+		goto out;
+
+	err = tailcall_freplace__load(fr_skel);
+	if (!ASSERT_OK(err, "load fr_skel"))
+		goto out;
+
+	fr_link = bpf_program__attach_freplace(fr_skel->progs.entry,
+					       prog_fd, "subprog");
+	if (!ASSERT_OK_PTR(fr_link, "attach_freplace"))
+		goto out;
+
+	prog_fd = bpf_program__fd(fr_skel->progs.entry);
+	if (!ASSERT_GE(prog_fd, 0, "fr_skel entry prog_fd"))
+		goto out;
+
+	map_fd = bpf_map__fd(fr_skel->maps.jmp_table);
+	if (!ASSERT_GE(map_fd, 0, "fr_skel jmp_table map_fd"))
+		goto out;
+
+	key = 0;
+	err = bpf_map_update_elem(map_fd, &key, &prog_fd, BPF_ANY);
+	if (!ASSERT_OK(err, "update jmp_table"))
+		goto out;
+
+	prog_fd = bpf_program__fd(tc_skel->progs.entry);
+	if (!ASSERT_GE(prog_fd, 0, "prog_fd"))
+		goto out;
+
+	err = bpf_prog_test_run_opts(prog_fd, &topts);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(topts.retval, 34, "test_run retval");
+
+out:
+	bpf_link__destroy(fr_link);
+	tc_bpf2bpf__destroy(tc_skel);
+	tailcall_freplace__destroy(fr_skel);
+}
+
 void test_tailcalls(void)
 {
 	if (test__start_subtest("tailcall_1"))
@@ -1543,4 +1615,6 @@ void test_tailcalls(void)
 		test_tailcall_bpf2bpf_hierarchy_fentry_entry();
 	test_tailcall_bpf2bpf_hierarchy_2();
 	test_tailcall_bpf2bpf_hierarchy_3();
+	if (test__start_subtest("tailcall_freplace"))
+		test_tailcall_freplace();
 }
diff --git a/tools/testing/selftests/bpf/progs/tailcall_freplace.c b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
new file mode 100644
index 0000000000000..80b5fa386ed9c
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_legacy.h"
+
+struct {
+	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+	__uint(max_entries, 1);
+	__uint(key_size, sizeof(__u32));
+	__uint(value_size, sizeof(__u32));
+} jmp_table SEC(".maps");
+
+int count = 0;
+
+__noinline
+int subprog(struct __sk_buff *skb)
+{
+	count++;
+
+	bpf_tail_call_static(skb, &jmp_table, 0);
+
+	return count;
+}
+
+SEC("freplace")
+int entry(struct __sk_buff *skb)
+{
+	return subprog(skb);
+}
+
+char __license[] SEC("license") = "GPL";
+
diff --git a/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
new file mode 100644
index 0000000000000..4810961554585
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_legacy.h"
+
+__noinline
+int subprog(struct __sk_buff *skb)
+{
+	volatile int ret = 1;
+
+	asm volatile (""::"r+"(ret));
+	return ret;
+}
+
+SEC("tc")
+int entry(struct __sk_buff *skb)
+{
+	return subprog(skb);
+}
+
+char __license[] SEC("license") = "GPL";
+
-- 
2.44.0

Re: [PATCH bpf-next 1/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[Yonghong Song]

On 7/24/24 5:32 PM, Leon Hwang wrote:
> The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
> resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed the following panic,
> which was caused by updating attached freplace prog to PROG_ARRAY map.

I am confused here. You mentioned that commit f7866c3587337731
fixed the panic below. But looking at commit message:
   https://lore.kernel.org/bpf/20240711145819.254178-2-wutengda@huaweicloud.com
it does not seem the case.

>
> But, it does not support updating attached freplace prog to PROG_ARRAY
> map.

This seems true since this patch itself intends fixing this issue.

>
> [309049.036402] BUG: kernel NULL pointer dereference, address: 0000000000000004
> [309049.036419] #PF: supervisor read access in kernel mode
> [309049.036426] #PF: error_code(0x0000) - not-present page
> [309049.036432] PGD 0 P4D 0
> [309049.036437] Oops: 0000 [#1] PREEMPT SMP NOPTI
> [309049.036444] CPU: 2 PID: 788148 Comm: test_progs Not tainted 6.8.0-31-generic #31-Ubuntu
> [309049.036465] Hardware name: VMware, Inc. VMware20,1/440BX Desktop Reference Platform, BIOS VMW201.00V.21805430.B64.2305221830 05/22/2023
> [309049.036477] RIP: 0010:bpf_prog_map_compatible+0x2a/0x140
> [309049.036488] Code: 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 44 8b 6e 04 48 89 f3 41 83 fd 1c 75 0c 48 8b 46 38 48 8b 40 70 <44> 8b 68 04 f6 43 03 01 75 1c 48 8b 43 38 44 0f b6 a0 89 00 00 00
> [309049.036505] RSP: 0018:ffffb2e080fd7ce0 EFLAGS: 00010246
> [309049.036513] RAX: 0000000000000000 RBX: ffffb2e0807c1000 RCX: 0000000000000000
> [309049.036521] RDX: 0000000000000000 RSI: ffffb2e0807c1000 RDI: ffff990290259e00
> [309049.036528] RBP: ffffb2e080fd7d08 R08: 0000000000000000 R09: 0000000000000000
> [309049.036536] R10: 0000000000000000 R11: 0000000000000000 R12: ffff990290259e00
> [309049.036543] R13: 000000000000001c R14: ffff990290259e00 R15: ffff99028e29c400
> [309049.036551] FS:  00007b82cbc28140(0000) GS:ffff9903b3f00000(0000) knlGS:0000000000000000
> [309049.036559] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [309049.036566] CR2: 0000000000000004 CR3: 0000000101286002 CR4: 00000000003706f0
> [309049.036573] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [309049.036581] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [309049.036588] Call Trace:
> [309049.036592]  <TASK>
> [309049.036597]  ? show_regs+0x6d/0x80
> [309049.036604]  ? __die+0x24/0x80
> [309049.036619]  ? page_fault_oops+0x99/0x1b0
> [309049.036628]  ? do_user_addr_fault+0x2ee/0x6b0
> [309049.036634]  ? exc_page_fault+0x83/0x1b0
> [309049.036641]  ? asm_exc_page_fault+0x27/0x30
> [309049.036649]  ? bpf_prog_map_compatible+0x2a/0x140
> [309049.036656]  prog_fd_array_get_ptr+0x2c/0x70
> [309049.036664]  bpf_fd_array_map_update_elem+0x37/0x130
> [309049.036671]  bpf_map_update_value+0x1d3/0x260
> [309049.036677]  map_update_elem+0x1fa/0x360
> [309049.036683]  __sys_bpf+0x54c/0xa10
> [309049.036689]  __x64_sys_bpf+0x1a/0x30
> [309049.036694]  x64_sys_call+0x1936/0x25c0
> [309049.036700]  do_syscall_64+0x7f/0x180
> [309049.036706]  ? do_syscall_64+0x8c/0x180
> [309049.036712]  ? do_syscall_64+0x8c/0x180
> [309049.036717]  ? irqentry_exit+0x43/0x50
> [309049.036723]  ? common_interrupt+0x54/0xb0
> [309049.036729]  entry_SYSCALL_64_after_hwframe+0x73/0x7b

I actually tried your selftest (patch 2/2) without patch 1/1, I got the
following error:

All error logs:
tester_init:PASS:tester_log_buf 0 nsec
process_subtest:PASS:obj_open_mem 0 nsec
process_subtest:PASS:specs_alloc 0 nsec
test_tailcall_freplace:PASS:open fr_skel 0 nsec
test_tailcall_freplace:PASS:open tc_skel 0 nsec
test_tailcall_freplace:PASS:tc_skel entry prog_id 0 nsec
test_tailcall_freplace:PASS:set_attach_target 0 nsec
test_tailcall_freplace:PASS:load fr_skel 0 nsec
test_tailcall_freplace:PASS:attach_freplace 0 nsec
test_tailcall_freplace:PASS:fr_skel entry prog_fd 0 nsec
test_tailcall_freplace:PASS:fr_skel jmp_table map_fd 0 nsec
test_tailcall_freplace:FAIL:update jmp_table unexpected error: -22 (errno 22)
#328/25  tailcalls/tailcall_freplace:FAIL
#328     tailcalls:FAIL

I didn't see kernel panic.

>
> Since commit 1c123c567fb138eb ("bpf: Resolve fext program type when
> checking map compatibility"), freplace prog can be used as tail-callee
> of its target prog.

the tailcall target can be a freplace prog.

> And the commit 3aac1ead5eb6b76f ("bpf: Move prog->aux->linked_prog and
> trampoline into bpf_link on attach") sets prog->aux->dst_prog as NULL
> when attach freplace prog to its target.

when attach -> after attaching

>
> Then, as for following example:
>
> tailcall_freplace.c:
>
> // SPDX-License-Identifier: GPL-2.0
>
> \#include <linux/bpf.h>
> \#include <bpf/bpf_helpers.h>
> \#include "bpf_legacy.h"
>
> struct {
> 	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> 	__uint(max_entries, 1);
> 	__uint(key_size, sizeof(__u32));
> 	__uint(value_size, sizeof(__u32));
> } jmp_table SEC(".maps");
>
> int count = 0;
>
> __noinline int
> subprog(struct __sk_buff *skb)
> {
> 	volatile int ret = 1;
>
> 	count++;
>
> 	bpf_tail_call_static(skb, &jmp_table, 0);
>
> 	return ret;
> }

This subprog is not needed and could be misleading,
just inline subprog into entry prog, it should be okay.

>
> SEC("freplace")
> int entry(struct __sk_buff *skb)
> {
> 	return subprog(skb);
> }
>
> char __license[] SEC("license") = "GPL";
>
> tc_bpf2bpf.c:
>
> // SPDX-License-Identifier: GPL-2.0
>
> \#include <linux/bpf.h>
> \#include <bpf/bpf_helpers.h>
> \#include "bpf_legacy.h"
>
> __noinline int
> subprog(struct __sk_buff *skb)
> {
> 	volatile int ret = 1;
>
> 	return ret;
> }
>
> SEC("tc")
> int entry(struct __sk_buff *skb)
> {
> 	return subprog(skb);
> }
>
> char __license[] SEC("license") = "GPL";
>
> And freplace entry prog's target is the tc subprog.
>
> After loading, the freplace jmp_table's owner type is
> BPF_PROG_TYPE_SCHED_CLS.
>
> Next, after attaching freplace prog to tc subprog, its prog->aux->
> dst_prog is NULL.
>
> Next, when update freplace prog to jmp_table, bpf_prog_map_compatible()
> returns false because resolve_prog_type() returns BPF_PROG_TYPE_EXT instead
> of BPF_PROG_TYPE_SCHED_CLS.
>
> With this patch, resolve_prog_type() returns BPF_PROG_TYPE_SCHED_CLS to
> support updating attached freplace prog to PROG_ARRY map for this
> example.
>
> Fixes: f7866c358733 ("bpf: Fix null pointer dereference in resolve_prog_type() for BPF_PROG_TYPE_EXT")
> Cc: Toke Høiland-Jørgensen <toke@redhat.com>
> Cc: Martin KaFai Lau <martin.lau@kernel.org>
> Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> ---
>   include/linux/bpf_verifier.h | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> index 5cea15c81b8a8..387e034e73d0e 100644
> --- a/include/linux/bpf_verifier.h
> +++ b/include/linux/bpf_verifier.h
> @@ -874,8 +874,8 @@ static inline u32 type_flag(u32 type)
>   /* only use after check_attach_btf_id() */
>   static inline enum bpf_prog_type resolve_prog_type(const struct bpf_prog *prog)
>   {
> -	return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->dst_prog) ?
> -		prog->aux->dst_prog->type : prog->type;
> +	return prog->type == BPF_PROG_TYPE_EXT ?
> +		prog->aux->saved_dst_prog_type : prog->type;

If prog->aux->dst_prog is NULL, is it possible that prog->aux->saved_dst_prog_type
(0, corresponding to BPF_PROG_TYPE_UNSPEC) could be returned? Do we need to do
	return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->saved_dst_prog_type) ?
		prog->aux->saved_dst_prog_type : prog->type;

Maybe I missed something here?

>   }
>   
>   static inline bool bpf_prog_check_recur(const struct bpf_prog *prog)

Re: [PATCH bpf-next 2/2] selftests/bpf: Add testcase for updating attached freplace prog to PROG_ARRAY map

[Yonghong Song]

On 7/24/24 5:32 PM, Leon Hwang wrote:
> Add a selftest to confirm the issue, which gets -EINVAL when update
> attached freplace prog to PROG_ARRAY map, has been fixed.
>
> cd tools/testing/selftests/bpf; ./test_progs -t tailcalls
> 327/25  tailcalls/tailcall_freplace:OK
> 327     tailcalls:OK
> Summary: 1/25 PASSED, 0 SKIPPED, 0 FAILED
>
> Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> ---
>   .../selftests/bpf/prog_tests/tailcalls.c      | 76 ++++++++++++++++++-
>   .../selftests/bpf/progs/tailcall_freplace.c   | 33 ++++++++
>   .../testing/selftests/bpf/progs/tc_bpf2bpf.c  | 23 ++++++
>   3 files changed, 131 insertions(+), 1 deletion(-)
>   create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
>   create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> index e01fabb8cc415..f1145601c0005 100644
> --- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> +++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> @@ -5,7 +5,8 @@
>   #include "tailcall_poke.skel.h"
>   #include "tailcall_bpf2bpf_hierarchy2.skel.h"
>   #include "tailcall_bpf2bpf_hierarchy3.skel.h"
> -
> +#include "tailcall_freplace.skel.h"
> +#include "tc_bpf2bpf.skel.h"
>   
>   /* test_tailcall_1 checks basic functionality by patching multiple locations
>    * in a single program for a single tail call slot with nop->jmp, jmp->nop
> @@ -1495,6 +1496,77 @@ static void test_tailcall_bpf2bpf_hierarchy_3(void)
>   	RUN_TESTS(tailcall_bpf2bpf_hierarchy3);
>   }
>   
> +/* test_tailcall_freplace checks that the attached freplace prog is OK to
> + * update to PROG_ARRAY map.

update the prog_array map.

> + */
> +static void test_tailcall_freplace(void)
> +{
> +	struct tailcall_freplace *fr_skel = NULL;
> +	struct tc_bpf2bpf *tc_skel = NULL;
> +	struct bpf_link *fr_link = NULL;
> +	int prog_fd, map_fd;
> +	char buff[128] = {};
> +	int err, key;
> +
> +	LIBBPF_OPTS(bpf_test_run_opts, topts,
> +		    .data_in = buff,
> +		    .data_size_in = sizeof(buff),
> +		    .repeat = 1,
> +	);
> +
> +	fr_skel = tailcall_freplace__open();
> +	if (!ASSERT_OK_PTR(fr_skel, "open fr_skel"))

if (!ASSERT_OK_PTR(fr_skel, "open fr_skel"))
==>
if (!ASSERT_OK_PTR(fr_skel, "tailcall_freplace__open"))
Similar for below other ASSERT_* macros.

> +		goto out;

Let us just do 'return' here.

> +
> +	tc_skel = tc_bpf2bpf__open_and_load();
> +	if (!ASSERT_OK_PTR(tc_skel, "open tc_skel"))
> +		goto out;
> +
> +	prog_fd = bpf_program__fd(tc_skel->progs.entry);
> +	if (!ASSERT_GE(prog_fd, 0, "tc_skel entry prog_id"))
> +		goto out;

ASSERT_GE is not necessary, prog_fd should already be valid.

> +
> +	err = bpf_program__set_attach_target(fr_skel->progs.entry,
> +					     prog_fd, "subprog");
> +	if (!ASSERT_OK(err, "set_attach_target"))
> +		goto out;
> +
> +	err = tailcall_freplace__load(fr_skel);
> +	if (!ASSERT_OK(err, "load fr_skel"))
> +		goto out;
> +
> +	fr_link = bpf_program__attach_freplace(fr_skel->progs.entry,
> +					       prog_fd, "subprog");
> +	if (!ASSERT_OK_PTR(fr_link, "attach_freplace"))
> +		goto out;
> +
> +	prog_fd = bpf_program__fd(fr_skel->progs.entry);
> +	if (!ASSERT_GE(prog_fd, 0, "fr_skel entry prog_fd"))
> +		goto out;

prog_fd is valid here. No need ASSERT_GE.

> +
> +	map_fd = bpf_map__fd(fr_skel->maps.jmp_table);
> +	if (!ASSERT_GE(map_fd, 0, "fr_skel jmp_table map_fd"))
> +		goto out;

map_fd is valid. No need ASSERT_GE.

> +
> +	key = 0;
> +	err = bpf_map_update_elem(map_fd, &key, &prog_fd, BPF_ANY);
> +	if (!ASSERT_OK(err, "update jmp_table"))
> +		goto out;
> +
> +	prog_fd = bpf_program__fd(tc_skel->progs.entry);
> +	if (!ASSERT_GE(prog_fd, 0, "prog_fd"))
> +		goto out;

prog_fd is valid here.

> +
> +	err = bpf_prog_test_run_opts(prog_fd, &topts);
> +	ASSERT_OK(err, "test_run");
> +	ASSERT_EQ(topts.retval, 34, "test_run retval");
> +
> +out:
> +	bpf_link__destroy(fr_link);
> +	tc_bpf2bpf__destroy(tc_skel);
> +	tailcall_freplace__destroy(fr_skel);
> +}
> +
>   void test_tailcalls(void)
>   {
>   	if (test__start_subtest("tailcall_1"))
> @@ -1543,4 +1615,6 @@ void test_tailcalls(void)
>   		test_tailcall_bpf2bpf_hierarchy_fentry_entry();
>   	test_tailcall_bpf2bpf_hierarchy_2();
>   	test_tailcall_bpf2bpf_hierarchy_3();
> +	if (test__start_subtest("tailcall_freplace"))
> +		test_tailcall_freplace();
>   }
> diff --git a/tools/testing/selftests/bpf/progs/tailcall_freplace.c b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> new file mode 100644
> index 0000000000000..80b5fa386ed9c
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> @@ -0,0 +1,33 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_legacy.h"

bpf_legacy.h is not needed.

> +
> +struct {
> +	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> +	__uint(max_entries, 1);
> +	__uint(key_size, sizeof(__u32));
> +	__uint(value_size, sizeof(__u32));
> +} jmp_table SEC(".maps");
> +
> +int count = 0;
> +
> +__noinline
> +int subprog(struct __sk_buff *skb)
> +{
> +	count++;
> +
> +	bpf_tail_call_static(skb, &jmp_table, 0);
> +
> +	return count;
> +}

subprog() can be inlined into entry(). This
can avoid confusing vs. tc_bpf2bpf.c.

Better if you can differentiate two 'entry()' function
names, e.g., entry_freplace(), entry_tc(), it can make
it easy for people to understand your change in tailcalls.c.

> +
> +SEC("freplace")
> +int entry(struct __sk_buff *skb)
> +{
> +	return subprog(skb);
> +}
> +
> +char __license[] SEC("license") = "GPL";
> +
> diff --git a/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> new file mode 100644
> index 0000000000000..4810961554585
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_legacy.h"
> +
> +__noinline
> +int subprog(struct __sk_buff *skb)
> +{
> +	volatile int ret = 1;
> +
> +	asm volatile (""::"r+"(ret));
> +	return ret;
> +}
> +
> +SEC("tc")
> +int entry(struct __sk_buff *skb)
> +{
> +	return subprog(skb);
> +}
> +
> +char __license[] SEC("license") = "GPL";
> +

Re: [PATCH bpf-next 1/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[leon.hwang]

26 July 2024 at 04:58, "Yonghong Song" <yonghong.song@linux.dev> wrote:



> 
> On 7/24/24 5:32 PM, Leon Hwang wrote:
> 
> > 
> > The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
> > 
> >  resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed the following panic,
> > 
> >  which was caused by updating attached freplace prog to PROG_ARRAY map.
> > 
> 
> I am confused here. You mentioned that commit f7866c3587337731
> 
> fixed the panic below. But looking at commit message:
> 
>  https://lore.kernel.org/bpf/20240711145819.254178-2-wutengda@huaweicloud.com
> 
> it does not seem the case.

The commit fixed this panic meanwhile.

This panic seems confusing. I'll remove it in patch v2.

> 
> > 
> > But, it does not support updating attached freplace prog to PROG_ARRAY
> > 
> >  map.
> > 
> 
> This seems true since this patch itself intends fixing this issue.

Yes, it is to fix this issue.

> 
> > 
> > [309049.036402] BUG: kernel NULL pointer dereference, address: 0000000000000004
> > 
> >  [309049.036419] #PF: supervisor read access in kernel mode
> > 
> >  [309049.036426] #PF: error_code(0x0000) - not-present page
> > 
> >  [309049.036432] PGD 0 P4D 0
> > 
> >  [309049.036437] Oops: 0000 [#1] PREEMPT SMP NOPTI
> > 
> >  [309049.036444] CPU: 2 PID: 788148 Comm: test_progs Not tainted 6.8.0-31-generic #31-Ubuntu
> > 
> >  [309049.036465] Hardware name: VMware, Inc. VMware20,1/440BX Desktop Reference Platform, BIOS VMW201.00V.21805430.B64.2305221830 05/22/2023
> > 
> >  [309049.036477] RIP: 0010:bpf_prog_map_compatible+0x2a/0x140
> > 
> >  [309049.036488] Code: 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 49 89 fe 41 55 41 54 53 44 8b 6e 04 48 89 f3 41 83 fd 1c 75 0c 48 8b 46 38 48 8b 40 70 <44> 8b 68 04 f6 43 03 01 75 1c 48 8b 43 38 44 0f b6 a0 89 00 00 00
> > 
> >  [309049.036505] RSP: 0018:ffffb2e080fd7ce0 EFLAGS: 00010246
> > 
> >  [309049.036513] RAX: 0000000000000000 RBX: ffffb2e0807c1000 RCX: 0000000000000000
> > 
> >  [309049.036521] RDX: 0000000000000000 RSI: ffffb2e0807c1000 RDI: ffff990290259e00
> > 
> >  [309049.036528] RBP: ffffb2e080fd7d08 R08: 0000000000000000 R09: 0000000000000000
> > 
> >  [309049.036536] R10: 0000000000000000 R11: 0000000000000000 R12: ffff990290259e00
> > 
> >  [309049.036543] R13: 000000000000001c R14: ffff990290259e00 R15: ffff99028e29c400
> > 
> >  [309049.036551] FS: 00007b82cbc28140(0000) GS:ffff9903b3f00000(0000) knlGS:0000000000000000
> > 
> >  [309049.036559] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > 
> >  [309049.036566] CR2: 0000000000000004 CR3: 0000000101286002 CR4: 00000000003706f0
> > 
> >  [309049.036573] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > 
> >  [309049.036581] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > 
> >  [309049.036588] Call Trace:
> > 
> >  [309049.036592] <TASK>
> > 
> >  [309049.036597] ? show_regs+0x6d/0x80
> > 
> >  [309049.036604] ? __die+0x24/0x80
> > 
> >  [309049.036619] ? page_fault_oops+0x99/0x1b0
> > 
> >  [309049.036628] ? do_user_addr_fault+0x2ee/0x6b0
> > 
> >  [309049.036634] ? exc_page_fault+0x83/0x1b0
> > 
> >  [309049.036641] ? asm_exc_page_fault+0x27/0x30
> > 
> >  [309049.036649] ? bpf_prog_map_compatible+0x2a/0x140
> > 
> >  [309049.036656] prog_fd_array_get_ptr+0x2c/0x70
> > 
> >  [309049.036664] bpf_fd_array_map_update_elem+0x37/0x130
> > 
> >  [309049.036671] bpf_map_update_value+0x1d3/0x260
> > 
> >  [309049.036677] map_update_elem+0x1fa/0x360
> > 
> >  [309049.036683] __sys_bpf+0x54c/0xa10
> > 
> >  [309049.036689] __x64_sys_bpf+0x1a/0x30
> > 
> >  [309049.036694] x64_sys_call+0x1936/0x25c0
> > 
> >  [309049.036700] do_syscall_64+0x7f/0x180
> > 
> >  [309049.036706] ? do_syscall_64+0x8c/0x180
> > 
> >  [309049.036712] ? do_syscall_64+0x8c/0x180
> > 
> >  [309049.036717] ? irqentry_exit+0x43/0x50
> > 
> >  [309049.036723] ? common_interrupt+0x54/0xb0
> > 
> >  [309049.036729] entry_SYSCALL_64_after_hwframe+0x73/0x7b
> > 
> 
> I actually tried your selftest (patch 2/2) without patch 1/1, I got the
> 
> following error:
> 
> All error logs:
> 
> tester_init:PASS:tester_log_buf 0 nsec
> 
> process_subtest:PASS:obj_open_mem 0 nsec
> 
> process_subtest:PASS:specs_alloc 0 nsec
> 
> test_tailcall_freplace:PASS:open fr_skel 0 nsec
> 
> test_tailcall_freplace:PASS:open tc_skel 0 nsec
> 
> test_tailcall_freplace:PASS:tc_skel entry prog_id 0 nsec
> 
> test_tailcall_freplace:PASS:set_attach_target 0 nsec
> 
> test_tailcall_freplace:PASS:load fr_skel 0 nsec
> 
> test_tailcall_freplace:PASS:attach_freplace 0 nsec
> 
> test_tailcall_freplace:PASS:fr_skel entry prog_fd 0 nsec
> 
> test_tailcall_freplace:PASS:fr_skel jmp_table map_fd 0 nsec
> 
> test_tailcall_freplace:FAIL:update jmp_table unexpected error: -22 (errno 22)
> 
> #328/25 tailcalls/tailcall_freplace:FAIL
> 
> #328 tailcalls:FAIL
> 
> I didn't see kernel panic.

Indeed.

> 
> > 
> > Since commit 1c123c567fb138eb ("bpf: Resolve fext program type when
> > 
> >  checking map compatibility"), freplace prog can be used as tail-callee
> > 
> >  of its target prog.
> > 
> 
> the tailcall target can be a freplace prog.

Ack.

> 
> > 
> > And the commit 3aac1ead5eb6b76f ("bpf: Move prog->aux->linked_prog and
> > 
> >  trampoline into bpf_link on attach") sets prog->aux->dst_prog as NULL
> > 
> >  when attach freplace prog to its target.
> > 
> 
> when attach -> after attaching

Ack.

> 
> > 
> > Then, as for following example:
> > 
> >  tailcall_freplace.c:
> > 
> >  // SPDX-License-Identifier: GPL-2.0
> > 
> >  \#include <linux/bpf.h>
> > 
> >  \#include <bpf/bpf_helpers.h>
> > 
> >  \#include "bpf_legacy.h"
> > 
> >  struct {
> > 
> >  __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> > 
> >  __uint(max_entries, 1);
> > 
> >  __uint(key_size, sizeof(__u32));
> > 
> >  __uint(value_size, sizeof(__u32));
> > 
> >  } jmp_table SEC(".maps");
> > 
> >  int count = 0;
> > 
> >  __noinline int
> > 
> >  subprog(struct __sk_buff *skb)
> > 
> >  {
> > 
> >  volatile int ret = 1;
> > 
> >  count++;
> > 
> >  bpf_tail_call_static(skb, &jmp_table, 0);
> > 
> >  return ret;
> > 
> >  }
> > 
> 
> This subprog is not needed and could be misleading,
> 
> just inline subprog into entry prog, it should be okay.

Ack.

> 
> > 
> > SEC("freplace")
> > 
> >  int entry(struct __sk_buff *skb)
> > 
> >  {
> > 
> >  return subprog(skb);
> > 
> >  }
> > 
> >  char __license[] SEC("license") = "GPL";
> > 
> >  tc_bpf2bpf.c:
> > 
> >  // SPDX-License-Identifier: GPL-2.0
> > 
> >  \#include <linux/bpf.h>
> > 
> >  \#include <bpf/bpf_helpers.h>
> > 
> >  \#include "bpf_legacy.h"
> > 
> >  __noinline int
> > 
> >  subprog(struct __sk_buff *skb)
> > 
> >  {
> > 
> >  volatile int ret = 1;
> > 
> >  return ret;
> > 
> >  }
> > 
> >  SEC("tc")
> > 
> >  int entry(struct __sk_buff *skb)
> > 
> >  {
> > 
> >  return subprog(skb);
> > 
> >  }
> > 
> >  char __license[] SEC("license") = "GPL";
> > 
> >  And freplace entry prog's target is the tc subprog.
> > 
> >  After loading, the freplace jmp_table's owner type is
> > 
> >  BPF_PROG_TYPE_SCHED_CLS.
> > 
> >  Next, after attaching freplace prog to tc subprog, its prog->aux->
> > 
> >  dst_prog is NULL.
> > 
> >  Next, when update freplace prog to jmp_table, bpf_prog_map_compatible()
> > 
> >  returns false because resolve_prog_type() returns BPF_PROG_TYPE_EXT instead
> > 
> >  of BPF_PROG_TYPE_SCHED_CLS.
> > 
> >  With this patch, resolve_prog_type() returns BPF_PROG_TYPE_SCHED_CLS to
> > 
> >  support updating attached freplace prog to PROG_ARRY map for this
> > 
> >  example.
> > 
> >  Fixes: f7866c358733 ("bpf: Fix null pointer dereference in resolve_prog_type() for BPF_PROG_TYPE_EXT")
> > 
> >  Cc: Toke Høiland-Jørgensen <toke@redhat.com>
> > 
> >  Cc: Martin KaFai Lau <martin.lau@kernel.org>
> > 
> >  Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> > 
> >  ---
> > 
> >  include/linux/bpf_verifier.h | 4 ++--
> > 
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> >  diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> > 
> >  index 5cea15c81b8a8..387e034e73d0e 100644
> > 
> >  --- a/include/linux/bpf_verifier.h
> > 
> >  +++ b/include/linux/bpf_verifier.h
> > 
> >  @@ -874,8 +874,8 @@ static inline u32 type_flag(u32 type)
> > 
> >  /* only use after check_attach_btf_id() */
> > 
> >  static inline enum bpf_prog_type resolve_prog_type(const struct bpf_prog *prog)
> > 
> >  {
> > 
> >  - return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->dst_prog) ?
> > 
> >  - prog->aux->dst_prog->type : prog->type;
> > 
> >  + return prog->type == BPF_PROG_TYPE_EXT ?
> > 
> >  + prog->aux->saved_dst_prog_type : prog->type;
> > 
> 
> If prog->aux->dst_prog is NULL, is it possible that prog->aux->saved_dst_prog_type
> 
> (0, corresponding to BPF_PROG_TYPE_UNSPEC) could be returned? Do we need to do
> 
>  return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->saved_dst_prog_type) ?
> 
>  prog->aux->saved_dst_prog_type : prog->type;
> 
> Maybe I missed something here?

It seems better to check prog->aux->saved_dst_prog_type. But I don't think so.

prog->aux->saved_dst_prog_type is set in check_attach_btf_id(). And there is no
resolve_prog_type() before check_attach_btf_id() in bpf_check().

Therefore, resolve_prog_type() must be called after check_attach_btf_id().

Thanks,
Leon

> 
> > 
> > }
> > 
> >  > static inline bool bpf_prog_check_recur(const struct bpf_prog *prog)
> >
>

Re: [PATCH bpf-next 2/2] selftests/bpf: Add testcase for updating attached freplace prog to PROG_ARRAY map

[leon.hwang]

Hi Yonghong,

Thank you for your review.

26 July 2024 at 05:11, "Yonghong Song" <yonghong.song@linux.dev> wrote:



> 
> On 7/24/24 5:32 PM, Leon Hwang wrote:
> 
> > 
> > Add a selftest to confirm the issue, which gets -EINVAL when update
> > 
> >  attached freplace prog to PROG_ARRAY map, has been fixed.
> > 
> >  cd tools/testing/selftests/bpf; ./test_progs -t tailcalls
> > 
> >  327/25 tailcalls/tailcall_freplace:OK
> > 
> >  327 tailcalls:OK
> > 
> >  Summary: 1/25 PASSED, 0 SKIPPED, 0 FAILED
> > 
> >  Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> > 
> >  ---
> > 
> >  .../selftests/bpf/prog_tests/tailcalls.c | 76 ++++++++++++++++++-
> > 
> >  .../selftests/bpf/progs/tailcall_freplace.c | 33 ++++++++
> > 
> >  .../testing/selftests/bpf/progs/tc_bpf2bpf.c | 23 ++++++
> > 
> >  3 files changed, 131 insertions(+), 1 deletion(-)
> > 
> >  create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> >  diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> > 
> >  index e01fabb8cc415..f1145601c0005 100644
> > 
> >  --- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> > 
> >  +++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c
> > 
> >  @@ -5,7 +5,8 @@
> > 
> >  #include "tailcall_poke.skel.h"
> > 
> >  #include "tailcall_bpf2bpf_hierarchy2.skel.h"
> > 
> >  #include "tailcall_bpf2bpf_hierarchy3.skel.h"
> > 
> >  -
> > 
> >  +#include "tailcall_freplace.skel.h"
> > 
> >  +#include "tc_bpf2bpf.skel.h"
> > 
> >  > /* test_tailcall_1 checks basic functionality by patching multiple locations
> > 
> >  * in a single program for a single tail call slot with nop->jmp, jmp->nop
> > 
> >  @@ -1495,6 +1496,77 @@ static void test_tailcall_bpf2bpf_hierarchy_3(void)
> > 
> >  RUN_TESTS(tailcall_bpf2bpf_hierarchy3);
> > 
> >  }
> > 
> >  > +/* test_tailcall_freplace checks that the attached freplace prog is OK to
> > 
> >  + * update to PROG_ARRAY map.
> > 
> 
> update the prog_array map.

Ack.

> 
> > 
> > + */
> > 
> >  +static void test_tailcall_freplace(void)
> > 
> >  +{
> > 
> >  + struct tailcall_freplace *fr_skel = NULL;
> > 
> >  + struct tc_bpf2bpf *tc_skel = NULL;
> > 
> >  + struct bpf_link *fr_link = NULL;
> > 
> >  + int prog_fd, map_fd;
> > 
> >  + char buff[128] = {};
> > 
> >  + int err, key;
> > 
> >  +
> > 
> >  + LIBBPF_OPTS(bpf_test_run_opts, topts,
> > 
> >  + .data_in = buff,
> > 
> >  + .data_size_in = sizeof(buff),
> > 
> >  + .repeat = 1,
> > 
> >  + );
> > 
> >  +
> > 
> >  + fr_skel = tailcall_freplace__open();
> > 
> >  + if (!ASSERT_OK_PTR(fr_skel, "open fr_skel"))
> > 
> 
> if (!ASSERT_OK_PTR(fr_skel, "open fr_skel"))
> 
> ==>
> 
> if (!ASSERT_OK_PTR(fr_skel, "tailcall_freplace__open"))
> 
> Similar for below other ASSERT_* macros.

Ack.

> 
> > 
> > + goto out;
> > 
> 
> Let us just do 'return' here.

Ack.

> 
> > 
> > +
> > 
> >  + tc_skel = tc_bpf2bpf__open_and_load();
> > 
> >  + if (!ASSERT_OK_PTR(tc_skel, "open tc_skel"))
> > 
> >  + goto out;
> > 
> >  +
> > 
> >  + prog_fd = bpf_program__fd(tc_skel->progs.entry);
> > 
> >  + if (!ASSERT_GE(prog_fd, 0, "tc_skel entry prog_id"))
> > 
> >  + goto out;
> > 
> 
> ASSERT_GE is not necessary, prog_fd should already be valid.

Ack.

I should read bpf_program__fd() source code before using it.

> 
> > 
> > +
> > 
> >  + err = bpf_program__set_attach_target(fr_skel->progs.entry,
> > 
> >  + prog_fd, "subprog");
> > 
> >  + if (!ASSERT_OK(err, "set_attach_target"))
> > 
> >  + goto out;
> > 
> >  +
> > 
> >  + err = tailcall_freplace__load(fr_skel);
> > 
> >  + if (!ASSERT_OK(err, "load fr_skel"))
> > 
> >  + goto out;
> > 
> >  +
> > 
> >  + fr_link = bpf_program__attach_freplace(fr_skel->progs.entry,
> > 
> >  + prog_fd, "subprog");
> > 
> >  + if (!ASSERT_OK_PTR(fr_link, "attach_freplace"))
> > 
> >  + goto out;
> > 
> >  +
> > 
> >  + prog_fd = bpf_program__fd(fr_skel->progs.entry);
> > 
> >  + if (!ASSERT_GE(prog_fd, 0, "fr_skel entry prog_fd"))
> > 
> >  + goto out;
> > 
> 
> prog_fd is valid here. No need ASSERT_GE.

Ack.

> 
> > 
> > +
> > 
> >  + map_fd = bpf_map__fd(fr_skel->maps.jmp_table);
> > 
> >  + if (!ASSERT_GE(map_fd, 0, "fr_skel jmp_table map_fd"))
> > 
> >  + goto out;
> > 
> 
> map_fd is valid. No need ASSERT_GE.

Ack.

> 
> > 
> > +
> > 
> >  + key = 0;
> > 
> >  + err = bpf_map_update_elem(map_fd, &key, &prog_fd, BPF_ANY);
> > 
> >  + if (!ASSERT_OK(err, "update jmp_table"))
> > 
> >  + goto out;
> > 
> >  +
> > 
> >  + prog_fd = bpf_program__fd(tc_skel->progs.entry);
> > 
> >  + if (!ASSERT_GE(prog_fd, 0, "prog_fd"))
> > 
> >  + goto out;
> > 
> 
> prog_fd is valid here.

Ack.

> 
> > 
> > +
> > 
> >  + err = bpf_prog_test_run_opts(prog_fd, &topts);
> > 
> >  + ASSERT_OK(err, "test_run");
> > 
> >  + ASSERT_EQ(topts.retval, 34, "test_run retval");
> > 
> >  +
> > 
> >  +out:
> > 
> >  + bpf_link__destroy(fr_link);
> > 
> >  + tc_bpf2bpf__destroy(tc_skel);
> > 
> >  + tailcall_freplace__destroy(fr_skel);
> > 
> >  +}
> > 
> >  +
> > 
> >  void test_tailcalls(void)
> > 
> >  {
> > 
> >  if (test__start_subtest("tailcall_1"))
> > 
> >  @@ -1543,4 +1615,6 @@ void test_tailcalls(void)
> > 
> >  test_tailcall_bpf2bpf_hierarchy_fentry_entry();
> > 
> >  test_tailcall_bpf2bpf_hierarchy_2();
> > 
> >  test_tailcall_bpf2bpf_hierarchy_3();
> > 
> >  + if (test__start_subtest("tailcall_freplace"))
> > 
> >  + test_tailcall_freplace();
> > 
> >  }
> > 
> >  diff --git a/tools/testing/selftests/bpf/progs/tailcall_freplace.c b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  new file mode 100644
> > 
> >  index 0000000000000..80b5fa386ed9c
> > 
> >  --- /dev/null
> > 
> >  +++ b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  @@ -0,0 +1,33 @@
> > 
> >  +// SPDX-License-Identifier: GPL-2.0
> > 
> >  +
> > 
> >  +#include <linux/bpf.h>
> > 
> >  +#include <bpf/bpf_helpers.h>
> > 
> >  +#include "bpf_legacy.h"
> > 
> 
> bpf_legacy.h is not needed.

Ack.

> 
> > 
> > +
> > 
> >  +struct {
> > 
> >  + __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> > 
> >  + __uint(max_entries, 1);
> > 
> >  + __uint(key_size, sizeof(__u32));
> > 
> >  + __uint(value_size, sizeof(__u32));
> > 
> >  +} jmp_table SEC(".maps");
> > 
> >  +
> > 
> >  +int count = 0;
> > 
> >  +
> > 
> >  +__noinline
> > 
> >  +int subprog(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + count++;
> > 
> >  +
> > 
> >  + bpf_tail_call_static(skb, &jmp_table, 0);
> > 
> >  +
> > 
> >  + return count;
> > 
> >  +}
> > 
> 
> subprog() can be inlined into entry(). This
> 
> can avoid confusing vs. tc_bpf2bpf.c.
> 
> Better if you can differentiate two 'entry()' function
> 
> names, e.g., entry_freplace(), entry_tc(), it can make
> 
> it easy for people to understand your change in tailcalls.c.

Indeed. Thanks for your suggestion.

> 
> > 
> > +
> > 
> >  +SEC("freplace")
> > 
> >  +int entry(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + return subprog(skb);
> > 
> >  +}
> > 
> >  +
> > 
> >  +char __license[] SEC("license") = "GPL";
> > 
> >  +
> > 
> >  diff --git a/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> >  new file mode 100644
> > 
> >  index 0000000000000..4810961554585
> > 
> >  --- /dev/null
> > 
> >  +++ b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> >  @@ -0,0 +1,23 @@
> > 
> >  +// SPDX-License-Identifier: GPL-2.0
> > 
> >  +
> > 
> >  +#include <linux/bpf.h>
> > 
> >  +#include <bpf/bpf_helpers.h>
> > 
> >  +#include "bpf_legacy.h"
> > 
> >  +
> > 
> >  +__noinline
> > 
> >  +int subprog(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + volatile int ret = 1;
> > 
> >  +
> > 
> >  + asm volatile (""::"r+"(ret));
> > 
> >  + return ret;
> > 
> >  +}
> > 
> >  +
> > 
> >  +SEC("tc")
> > 
> >  +int entry(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + return subprog(skb);
> > 
> >  +}
> > 
> >  +
> > 
> >  +char __license[] SEC("license") = "GPL";
> > 
> >  +
> >
>

Re: [PATCH bpf-next 1/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[Yonghong Song]

On 7/25/24 8:27 PM, leon.hwang@linux.dev wrote:
> 26 July 2024 at 04:58, "Yonghong Song" <yonghong.song@linux.dev> wrote:
>
>
>
>> On 7/24/24 5:32 PM, Leon Hwang wrote:
>>
>>> The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
>>>
>>>   resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed the following panic,
>>>
>>>   which was caused by updating attached freplace prog to PROG_ARRAY map.
>>>
>> I am confused here. You mentioned that commit f7866c3587337731
>>
>> fixed the panic below. But looking at commit message:
>>
>>   https://lore.kernel.org/bpf/20240711145819.254178-2-wutengda@huaweicloud.com
>>
>> it does not seem the case.
> The commit fixed this panic meanwhile.
>
> This panic seems confusing. I'll remove it in patch v2.
>
[...]

>>>   ---
>>>
>>>   include/linux/bpf_verifier.h | 4 ++--
>>>
>>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>>   diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
>>>
>>>   index 5cea15c81b8a8..387e034e73d0e 100644
>>>
>>>   --- a/include/linux/bpf_verifier.h
>>>
>>>   +++ b/include/linux/bpf_verifier.h
>>>
>>>   @@ -874,8 +874,8 @@ static inline u32 type_flag(u32 type)
>>>
>>>   /* only use after check_attach_btf_id() */
>>>
>>>   static inline enum bpf_prog_type resolve_prog_type(const struct bpf_prog *prog)
>>>
>>>   {
>>>
>>>   - return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->dst_prog) ?
>>>
>>>   - prog->aux->dst_prog->type : prog->type;
>>>
>>>   + return prog->type == BPF_PROG_TYPE_EXT ?
>>>
>>>   + prog->aux->saved_dst_prog_type : prog->type;
>>>
>> If prog->aux->dst_prog is NULL, is it possible that prog->aux->saved_dst_prog_type
>>
>> (0, corresponding to BPF_PROG_TYPE_UNSPEC) could be returned? Do we need to do
>>
>>   return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->saved_dst_prog_type) ?
>>
>>   prog->aux->saved_dst_prog_type : prog->type;
>>
>> Maybe I missed something here?
> It seems better to check prog->aux->saved_dst_prog_type. But I don't think so.
>
> prog->aux->saved_dst_prog_type is set in check_attach_btf_id(). And there is no
> resolve_prog_type() before check_attach_btf_id() in bpf_check().
>
> Therefore, resolve_prog_type() must be called after check_attach_btf_id().

In check_attach_btf_id(), I see
         if (tgt_prog) {
                 prog->aux->saved_dst_prog_type = tgt_prog->type;
                 prog->aux->saved_dst_attach_type = tgt_prog->expected_attach_type;
         }

So it is possible prog->aux->saved_dst_prog_type is 0 (default value).
I don't know that if tgt_prog is NULL, whether later resolve_prog_type()
will be called or not. Need more checking here.

Re: [PATCH bpf-next 2/2] selftests/bpf: Add testcase for updating attached freplace prog to PROG_ARRAY map

[Yonghong Song]

On 7/24/24 5:32 PM, Leon Hwang wrote:
> Add a selftest to confirm the issue, which gets -EINVAL when update
> attached freplace prog to PROG_ARRAY map, has been fixed.
>
> cd tools/testing/selftests/bpf; ./test_progs -t tailcalls
> 327/25  tailcalls/tailcall_freplace:OK
> 327     tailcalls:OK
> Summary: 1/25 PASSED, 0 SKIPPED, 0 FAILED
>
> Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> ---
>   .../selftests/bpf/prog_tests/tailcalls.c      | 76 ++++++++++++++++++-
>   .../selftests/bpf/progs/tailcall_freplace.c   | 33 ++++++++
>   .../testing/selftests/bpf/progs/tc_bpf2bpf.c  | 23 ++++++
>   3 files changed, 131 insertions(+), 1 deletion(-)
>   create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
>   create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
>
[...]
> diff --git a/tools/testing/selftests/bpf/progs/tailcall_freplace.c b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> new file mode 100644
> index 0000000000000..80b5fa386ed9c
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> @@ -0,0 +1,33 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_legacy.h"
> +
> +struct {
> +	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> +	__uint(max_entries, 1);
> +	__uint(key_size, sizeof(__u32));
> +	__uint(value_size, sizeof(__u32));
> +} jmp_table SEC(".maps");
> +
> +int count = 0;
> +
> +__noinline
> +int subprog(struct __sk_buff *skb)
> +{
> +	count++;
> +
> +	bpf_tail_call_static(skb, &jmp_table, 0);
> +
> +	return count;
> +}
> +
> +SEC("freplace")
> +int entry(struct __sk_buff *skb)
> +{
> +	return subprog(skb);
> +}
> +
> +char __license[] SEC("license") = "GPL";
> +

extra line in the above.

> diff --git a/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> new file mode 100644
> index 0000000000000..4810961554585
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_legacy.h"
> +
> +__noinline
> +int subprog(struct __sk_buff *skb)
> +{
> +	volatile int ret = 1;
> +
> +	asm volatile (""::"r+"(ret));
> +	return ret;
> +}
> +
> +SEC("tc")
> +int entry(struct __sk_buff *skb)
> +{
> +	return subprog(skb);
> +}
> +
> +char __license[] SEC("license") = "GPL";
> +

extra line in the above.

Re: [PATCH bpf-next 1/2] bpf: Fix updating attached freplace prog to PROG_ARRAY map

[Leon Hwang]

26 July 2024 at 14:15, "Yonghong Song" <yonghong.song@linux.dev> wrote:

> 
> On 7/25/24 8:27 PM, leon.hwang@linux.dev wrote:
> 
> > 
> > 26 July 2024 at 04:58, "Yonghong Song" <yonghong.song@linux.dev> wrote:
> > 
> > > 
> > > On 7/24/24 5:32 PM, Leon Hwang wrote:
> > > 
> > 
> >  The commit f7866c3587337731 ("bpf: Fix null pointer dereference in
> > 
> >  resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed the following panic,
> > 
> >  which was caused by updating attached freplace prog to PROG_ARRAY map.
> > 
> > > 
> > > I am confused here. You mentioned that commit f7866c3587337731
> > > 
> > >  fixed the panic below. But looking at commit message:
> > > 
> > >  https://lore.kernel.org/bpf/20240711145819.254178-2-wutengda@huaweicloud.com
> > > 
> > >  it does not seem the case.
> > > 
> > 
> >  The commit fixed this panic meanwhile.
> > 
> >  This panic seems confusing. I'll remove it in patch v2.
> > 
> 
> [...]
> 
> > 
> > ---
> > 
> >  include/linux/bpf_verifier.h | 4 ++--
> > 
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> >  diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
> > 
> >  index 5cea15c81b8a8..387e034e73d0e 100644
> > 
> >  --- a/include/linux/bpf_verifier.h
> > 
> >  +++ b/include/linux/bpf_verifier.h
> > 
> >  @@ -874,8 +874,8 @@ static inline u32 type_flag(u32 type)
> > 
> >  /* only use after check_attach_btf_id() */
> > 
> >  static inline enum bpf_prog_type resolve_prog_type(const struct bpf_prog *prog)
> > 
> >  {
> > 
> >  - return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->dst_prog) ?
> > 
> >  - prog->aux->dst_prog->type : prog->type;
> > 
> >  + return prog->type == BPF_PROG_TYPE_EXT ?
> > 
> >  + prog->aux->saved_dst_prog_type : prog->type;
> > 
> > > 
> > > If prog->aux->dst_prog is NULL, is it possible that prog->aux->saved_dst_prog_type
> > > 
> > >  (0, corresponding to BPF_PROG_TYPE_UNSPEC) could be returned? Do we need to do
> > > 
> > >  return (prog->type == BPF_PROG_TYPE_EXT && prog->aux->saved_dst_prog_type) ?
> > > 
> > >  prog->aux->saved_dst_prog_type : prog->type;
> > > 
> > >  Maybe I missed something here?
> > > 
> > 
> >  It seems better to check prog->aux->saved_dst_prog_type. But I don't think so.
> > 
> >  prog->aux->saved_dst_prog_type is set in check_attach_btf_id(). And there is no
> > 
> >  resolve_prog_type() before check_attach_btf_id() in bpf_check().
> > 
> >  Therefore, resolve_prog_type() must be called after check_attach_btf_id().
> > 
> 
> In check_attach_btf_id(), I see
> 
>  if (tgt_prog) {
> 
>  prog->aux->saved_dst_prog_type = tgt_prog->type;
> 
>  prog->aux->saved_dst_attach_type = tgt_prog->expected_attach_type;
> 
>  }
> 
> So it is possible prog->aux->saved_dst_prog_type is 0 (default value).
> 
> I don't know that if tgt_prog is NULL, whether later resolve_prog_type()
> 
> will be called or not. Need more checking here.
>

This is the case that commit f7866c3587337731 ("bpf: Fix null pointer dereference
in resolve_prog_type() for BPF_PROG_TYPE_EXT") fixed, which is loading freplace
prog without tgt_prog.

With this patch, while loading freplace prog without tgt_prog, resolve_prog_type()
returns 0 instead of BPF_PROG_TYPE_EXT.

It's better to return a meaningful prog type in resolve_prog_type() anyway.

I accept your suggestion.

Thanks,
Leon

Re: [PATCH bpf-next 2/2] selftests/bpf: Add testcase for updating attached freplace prog to PROG_ARRAY map

[Leon Hwang]

26 July 2024 at 14:16, "Yonghong Song" <yonghong.song@linux.dev> wrote:

> 
> On 7/24/24 5:32 PM, Leon Hwang wrote:
> 
> > 
> > Add a selftest to confirm the issue, which gets -EINVAL when update
> > 
> >  attached freplace prog to PROG_ARRAY map, has been fixed.
> > 
> >  cd tools/testing/selftests/bpf; ./test_progs -t tailcalls
> > 
> >  327/25 tailcalls/tailcall_freplace:OK
> > 
> >  327 tailcalls:OK
> > 
> >  Summary: 1/25 PASSED, 0 SKIPPED, 0 FAILED
> > 
> >  Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> > 
> >  ---
> > 
> >  .../selftests/bpf/prog_tests/tailcalls.c | 76 ++++++++++++++++++-
> > 
> >  .../selftests/bpf/progs/tailcall_freplace.c | 33 ++++++++
> > 
> >  .../testing/selftests/bpf/progs/tc_bpf2bpf.c | 23 ++++++
> > 
> >  3 files changed, 131 insertions(+), 1 deletion(-)
> > 
> >  create mode 100644 tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  create mode 100644 tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> 
> [...]
> 
> > 
> > diff --git a/tools/testing/selftests/bpf/progs/tailcall_freplace.c b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  new file mode 100644
> > 
> >  index 0000000000000..80b5fa386ed9c
> > 
> >  --- /dev/null
> > 
> >  +++ b/tools/testing/selftests/bpf/progs/tailcall_freplace.c
> > 
> >  @@ -0,0 +1,33 @@
> > 
> >  +// SPDX-License-Identifier: GPL-2.0
> > 
> >  +
> > 
> >  +#include <linux/bpf.h>
> > 
> >  +#include <bpf/bpf_helpers.h>
> > 
> >  +#include "bpf_legacy.h"
> > 
> >  +
> > 
> >  +struct {
> > 
> >  + __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
> > 
> >  + __uint(max_entries, 1);
> > 
> >  + __uint(key_size, sizeof(__u32));
> > 
> >  + __uint(value_size, sizeof(__u32));
> > 
> >  +} jmp_table SEC(".maps");
> > 
> >  +
> > 
> >  +int count = 0;
> > 
> >  +
> > 
> >  +__noinline
> > 
> >  +int subprog(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + count++;
> > 
> >  +
> > 
> >  + bpf_tail_call_static(skb, &jmp_table, 0);
> > 
> >  +
> > 
> >  + return count;
> > 
> >  +}
> > 
> >  +
> > 
> >  +SEC("freplace")
> > 
> >  +int entry(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + return subprog(skb);
> > 
> >  +}
> > 
> >  +
> > 
> >  +char __license[] SEC("license") = "GPL";
> > 
> >  +
> > 
> 
> extra line in the above.

Ack.

> 
> > 
> > diff --git a/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> >  new file mode 100644
> > 
> >  index 0000000000000..4810961554585
> > 
> >  --- /dev/null
> > 
> >  +++ b/tools/testing/selftests/bpf/progs/tc_bpf2bpf.c
> > 
> >  @@ -0,0 +1,23 @@
> > 
> >  +// SPDX-License-Identifier: GPL-2.0
> > 
> >  +
> > 
> >  +#include <linux/bpf.h>
> > 
> >  +#include <bpf/bpf_helpers.h>
> > 
> >  +#include "bpf_legacy.h"
> > 
> >  +
> > 
> >  +__noinline
> > 
> >  +int subprog(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + volatile int ret = 1;
> > 
> >  +
> > 
> >  + asm volatile (""::"r+"(ret));
> > 
> >  + return ret;
> > 
> >  +}
> > 
> >  +
> > 
> >  +SEC("tc")
> > 
> >  +int entry(struct __sk_buff *skb)
> > 
> >  +{
> > 
> >  + return subprog(skb);
> > 
> >  +}
> > 
> >  +
> > 
> >  +char __license[] SEC("license") = "GPL";
> > 
> >  +
> > 
> 
> extra line in the above.
>

Ack.

Thanks,
Leon