[PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[Song Liu]

Change in struct file [1] moves f_ref to the 3rd cache line. This makes
deferencing file pointer as a 8-byte variable invalid, because
btf_struct_walk() will walk into f_lock, which is 4-byte long.

Fix the selftests to deference the file pointer as a 4-byte variable.

[1] commit e249056c91a2 ("fs: place f_ref to 3rd cache line in struct
                          file to resolve false sharing")
Reported-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Song Liu <song@kernel.org>
---
 tools/testing/selftests/bpf/progs/test_module_attach.c    | 2 +-
 tools/testing/selftests/bpf/progs/test_subprogs_extable.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/test_module_attach.c b/tools/testing/selftests/bpf/progs/test_module_attach.c
index fb07f5773888..7f3c233943b3 100644
--- a/tools/testing/selftests/bpf/progs/test_module_attach.c
+++ b/tools/testing/selftests/bpf/progs/test_module_attach.c
@@ -117,7 +117,7 @@ int BPF_PROG(handle_fexit_ret, int arg, struct file *ret)
 
 	bpf_probe_read_kernel(&buf, 8, ret);
 	bpf_probe_read_kernel(&buf, 8, (char *)ret + 256);
-	*(volatile long long *)ret;
+	*(volatile int *)ret;
 	*(volatile int *)&ret->f_mode;
 	return 0;
 }
diff --git a/tools/testing/selftests/bpf/progs/test_subprogs_extable.c b/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
index e2a21fbd4e44..dcac69f5928a 100644
--- a/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
+++ b/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
@@ -21,7 +21,7 @@ static __u64 test_cb(struct bpf_map *map, __u32 *key, __u64 *val, void *data)
 SEC("fexit/bpf_testmod_return_ptr")
 int BPF_PROG(handle_fexit_ret_subprogs, int arg, struct file *ret)
 {
-	*(volatile long *)ret;
+	*(volatile int *)ret;
 	*(volatile int *)&ret->f_mode;
 	bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
 	triggered++;
@@ -31,7 +31,7 @@ int BPF_PROG(handle_fexit_ret_subprogs, int arg, struct file *ret)
 SEC("fexit/bpf_testmod_return_ptr")
 int BPF_PROG(handle_fexit_ret_subprogs2, int arg, struct file *ret)
 {
-	*(volatile long *)ret;
+	*(volatile int *)ret;
 	*(volatile int *)&ret->f_mode;
 	bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
 	triggered++;
@@ -41,7 +41,7 @@ int BPF_PROG(handle_fexit_ret_subprogs2, int arg, struct file *ret)
 SEC("fexit/bpf_testmod_return_ptr")
 int BPF_PROG(handle_fexit_ret_subprogs3, int arg, struct file *ret)
 {
-	*(volatile long *)ret;
+	*(volatile int *)ret;
 	*(volatile int *)&ret->f_mode;
 	bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
 	triggered++;
-- 
2.47.1

Re: [PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[Andrii Nakryiko]

On Thu, Mar 27, 2025 at 11:55 AM Song Liu <song@kernel.org> wrote:
>
> Change in struct file [1] moves f_ref to the 3rd cache line. This makes
> deferencing file pointer as a 8-byte variable invalid, because
> btf_struct_walk() will walk into f_lock, which is 4-byte long.
>
> Fix the selftests to deference the file pointer as a 4-byte variable.
>
> [1] commit e249056c91a2 ("fs: place f_ref to 3rd cache line in struct
>                           file to resolve false sharing")
> Reported-by: Jakub Kicinski <kuba@kernel.org>
> Signed-off-by: Song Liu <song@kernel.org>
> ---
>  tools/testing/selftests/bpf/progs/test_module_attach.c    | 2 +-
>  tools/testing/selftests/bpf/progs/test_subprogs_extable.c | 6 +++---
>  2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/progs/test_module_attach.c b/tools/testing/selftests/bpf/progs/test_module_attach.c
> index fb07f5773888..7f3c233943b3 100644
> --- a/tools/testing/selftests/bpf/progs/test_module_attach.c
> +++ b/tools/testing/selftests/bpf/progs/test_module_attach.c
> @@ -117,7 +117,7 @@ int BPF_PROG(handle_fexit_ret, int arg, struct file *ret)
>
>         bpf_probe_read_kernel(&buf, 8, ret);
>         bpf_probe_read_kernel(&buf, 8, (char *)ret + 256);
> -       *(volatile long long *)ret;
> +       *(volatile int *)ret;

we already have `*(volatile int *)&ret->f_mode;` below, do we really
need this int casting case?.. Maybe instead of guessing the size of
file's first field, let's just remove `*(volatile long long *)ret;`
altogether?

>         *(volatile int *)&ret->f_mode;
>         return 0;
>  }
> diff --git a/tools/testing/selftests/bpf/progs/test_subprogs_extable.c b/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
> index e2a21fbd4e44..dcac69f5928a 100644
> --- a/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
> +++ b/tools/testing/selftests/bpf/progs/test_subprogs_extable.c
> @@ -21,7 +21,7 @@ static __u64 test_cb(struct bpf_map *map, __u32 *key, __u64 *val, void *data)
>  SEC("fexit/bpf_testmod_return_ptr")
>  int BPF_PROG(handle_fexit_ret_subprogs, int arg, struct file *ret)
>  {
> -       *(volatile long *)ret;
> +       *(volatile int *)ret;
>         *(volatile int *)&ret->f_mode;
>         bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
>         triggered++;
> @@ -31,7 +31,7 @@ int BPF_PROG(handle_fexit_ret_subprogs, int arg, struct file *ret)
>  SEC("fexit/bpf_testmod_return_ptr")
>  int BPF_PROG(handle_fexit_ret_subprogs2, int arg, struct file *ret)
>  {
> -       *(volatile long *)ret;
> +       *(volatile int *)ret;
>         *(volatile int *)&ret->f_mode;
>         bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
>         triggered++;
> @@ -41,7 +41,7 @@ int BPF_PROG(handle_fexit_ret_subprogs2, int arg, struct file *ret)
>  SEC("fexit/bpf_testmod_return_ptr")
>  int BPF_PROG(handle_fexit_ret_subprogs3, int arg, struct file *ret)
>  {
> -       *(volatile long *)ret;
> +       *(volatile int *)ret;
>         *(volatile int *)&ret->f_mode;
>         bpf_for_each_map_elem(&test_array, test_cb, NULL, 0);
>         triggered++;
> --
> 2.47.1
>

Re: [PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[Song Liu]

> On Mar 28, 2025, at 10:30 AM, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> 
> On Thu, Mar 27, 2025 at 11:55 AM Song Liu <song@kernel.org> wrote:
>> 
>> Change in struct file [1] moves f_ref to the 3rd cache line. This makes
>> deferencing file pointer as a 8-byte variable invalid, because
>> btf_struct_walk() will walk into f_lock, which is 4-byte long.
>> 
>> Fix the selftests to deference the file pointer as a 4-byte variable.
>> 
>> [1] commit e249056c91a2 ("fs: place f_ref to 3rd cache line in struct
>>                          file to resolve false sharing")
>> Reported-by: Jakub Kicinski <kuba@kernel.org>
>> Signed-off-by: Song Liu <song@kernel.org>
>> ---
>> tools/testing/selftests/bpf/progs/test_module_attach.c    | 2 +-
>> tools/testing/selftests/bpf/progs/test_subprogs_extable.c | 6 +++---
>> 2 files changed, 4 insertions(+), 4 deletions(-)
>> 
>> diff --git a/tools/testing/selftests/bpf/progs/test_module_attach.c b/tools/testing/selftests/bpf/progs/test_module_attach.c
>> index fb07f5773888..7f3c233943b3 100644
>> --- a/tools/testing/selftests/bpf/progs/test_module_attach.c
>> +++ b/tools/testing/selftests/bpf/progs/test_module_attach.c
>> @@ -117,7 +117,7 @@ int BPF_PROG(handle_fexit_ret, int arg, struct file *ret)
>> 
>>        bpf_probe_read_kernel(&buf, 8, ret);
>>        bpf_probe_read_kernel(&buf, 8, (char *)ret + 256);
>> -       *(volatile long long *)ret;
>> +       *(volatile int *)ret;
> 
> we already have `*(volatile int *)&ret->f_mode;` below, do we really
> need this int casting case?.. Maybe instead of guessing the size of
> file's first field, let's just remove `*(volatile long long *)ret;`
> altogether?

I was assuming the original test covers two cases:
  1) deref ret itself;
  2) deref a member of ret (ret->f_mode);

Therefore, instead of doing something like

   *(volatile long long *)&ret->f_ref;  /* first member of file */

I got current version. 

If we don't need the first case, we sure can remove it. 

Thanks,
Song

> 
>>        *(volatile int *)&ret->f_mode;
>>        return 0;

Re: [PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[Alexei Starovoitov]

On Fri, Mar 28, 2025 at 10:57 AM Song Liu <songliubraving@meta.com> wrote:
>
>
>
> > On Mar 28, 2025, at 10:30 AM, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Mar 27, 2025 at 11:55 AM Song Liu <song@kernel.org> wrote:
> >>
> >> Change in struct file [1] moves f_ref to the 3rd cache line. This makes
> >> deferencing file pointer as a 8-byte variable invalid, because
> >> btf_struct_walk() will walk into f_lock, which is 4-byte long.
> >>
> >> Fix the selftests to deference the file pointer as a 4-byte variable.
> >>
> >> [1] commit e249056c91a2 ("fs: place f_ref to 3rd cache line in struct
> >>                          file to resolve false sharing")
> >> Reported-by: Jakub Kicinski <kuba@kernel.org>
> >> Signed-off-by: Song Liu <song@kernel.org>
> >> ---
> >> tools/testing/selftests/bpf/progs/test_module_attach.c    | 2 +-
> >> tools/testing/selftests/bpf/progs/test_subprogs_extable.c | 6 +++---
> >> 2 files changed, 4 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/tools/testing/selftests/bpf/progs/test_module_attach.c b/tools/testing/selftests/bpf/progs/test_module_attach.c
> >> index fb07f5773888..7f3c233943b3 100644
> >> --- a/tools/testing/selftests/bpf/progs/test_module_attach.c
> >> +++ b/tools/testing/selftests/bpf/progs/test_module_attach.c
> >> @@ -117,7 +117,7 @@ int BPF_PROG(handle_fexit_ret, int arg, struct file *ret)
> >>
> >>        bpf_probe_read_kernel(&buf, 8, ret);
> >>        bpf_probe_read_kernel(&buf, 8, (char *)ret + 256);
> >> -       *(volatile long long *)ret;
> >> +       *(volatile int *)ret;
> >
> > we already have `*(volatile int *)&ret->f_mode;` below, do we really
> > need this int casting case?.. Maybe instead of guessing the size of
> > file's first field, let's just remove `*(volatile long long *)ret;`
> > altogether?
>
> I was assuming the original test covers two cases:
>   1) deref ret itself;
>   2) deref a member of ret (ret->f_mode);
>
> Therefore, instead of doing something like
>
>    *(volatile long long *)&ret->f_ref;  /* first member of file */
>
> I got current version.
>
> If we don't need the first case, we sure can remove it.

The idea of the patch was to test the load from the address
returned from bpf_testmod_return_ptr() twice.
Once as that exact value and another with some offset,
since JIT processing logic is different whether insn->off is zero.
Doing &ret->f_lock /* first member of file */
sort-of works, but the comment will be stale eventually.
I think the current fix is the best:
-       *(volatile long long *)ret;
+       *(volatile int *)ret;

This way the load will have guaranteed insn->off == 0,
and when file layout changes we will notice the breakage right away.
Like happened this time.

So I'm thinking of applying this patch as-is when bpf-next is ready.

Re: [PATCH bpf-next] selftests/bpf: Fix tests after change in struct file

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to bpf/bpf.git (master)
by Alexei Starovoitov <ast@kernel.org>:

On Thu, 27 Mar 2025 11:55:28 -0700 you wrote:
> Change in struct file [1] moves f_ref to the 3rd cache line. This makes
> deferencing file pointer as a 8-byte variable invalid, because
> btf_struct_walk() will walk into f_lock, which is 4-byte long.
> 
> Fix the selftests to deference the file pointer as a 4-byte variable.
> 
> [1] commit e249056c91a2 ("fs: place f_ref to 3rd cache line in struct
>                           file to resolve false sharing")
> Reported-by: Jakub Kicinski <kuba@kernel.org>
> Signed-off-by: Song Liu <song@kernel.org>
> 
> [...]

Here is the summary with links:
  - [bpf-next] selftests/bpf: Fix tests after change in struct file
    https://git.kernel.org/bpf/bpf/c/bd06a13f44e1

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html