[PATCH -tip 1/1] x86/bpf: Fix BPF percpu accesses

[PATCH -tip 1/1] x86/bpf: Fix BPF percpu accesses

[Brian Gerst]

Due to commit 9d7de2aa8b41 ("Use relative percpu offsets"), percpu
addresses went from positive offsets from the GSBASE to negative kernel
virtual addresses.  The BPF verifier has an optimization for x86-64 that
loads the address of cpu_number into a register, but was only doing a
32-bit load which truncates negative addresses.  Change it to a 64-bit
load so that the address is properly sign-extended.

Fixes: 9d7de2aa8b41 ("Use relative percpu offsets")
Signed-off-by: Brian Gerst <brgerst@gmail.com>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
---
 kernel/bpf/verifier.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 60611df77957..f4859516b190 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -21707,7 +21707,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
 			 * way, it's fine to back out this inlining logic
 			 */
 #ifdef CONFIG_SMP
-			insn_buf[0] = BPF_MOV32_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);
+			insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);
 			insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0);
 			insn_buf[2] = BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0);
 			cnt = 3;
-- 
2.48.1

[tip: x86/asm] x86/bpf: Fix BPF percpu accesses

[tip-bot2 for Brian Gerst]

The following commit has been merged into the x86/asm branch of tip:

Commit-ID:     18cdd90aba794333f4c6dce39f5c3fe642af5575
Gitweb:        https://git.kernel.org/tip/18cdd90aba794333f4c6dce39f5c3fe642af5575
Author:        Brian Gerst <brgerst@gmail.com>
AuthorDate:    Thu, 27 Feb 2025 14:53:02 -05:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Thu, 27 Feb 2025 21:10:03 +01:00

x86/bpf: Fix BPF percpu accesses

Due to this recent commit in the x86 tree:

  9d7de2aa8b41 ("Use relative percpu offsets")

percpu addresses went from positive offsets from the GSBASE to negative
kernel virtual addresses.  The BPF verifier has an optimization for
x86-64 that loads the address of cpu_number into a register, but was only
doing a 32-bit load which truncates negative addresses.

Change it to a 64-bit load so that the address is properly sign-extended.

Fixes: 9d7de2aa8b41 ("Use relative percpu offsets")
Signed-off-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Uros Bizjak <ubizjak@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20250227195302.1667654-1-brgerst@gmail.com
---
 kernel/bpf/verifier.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9971c03..f74263b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -21692,7 +21692,7 @@ patch_map_ops_generic:
 			 * way, it's fine to back out this inlining logic
 			 */
 #ifdef CONFIG_SMP
-			insn_buf[0] = BPF_MOV32_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);
+			insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);
 			insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0);
 			insn_buf[2] = BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0);
 			cnt = 3;

Re: [PATCH -tip 1/1] x86/bpf: Fix BPF percpu accesses

[Andrii Nakryiko]

On Thu, Feb 27, 2025 at 11:53 AM Brian Gerst <brgerst@gmail.com> wrote:
>
> Due to commit 9d7de2aa8b41 ("Use relative percpu offsets"), percpu
> addresses went from positive offsets from the GSBASE to negative kernel
> virtual addresses.  The BPF verifier has an optimization for x86-64 that
> loads the address of cpu_number into a register, but was only doing a
> 32-bit load which truncates negative addresses.  Change it to a 64-bit
> load so that the address is properly sign-extended.
>
> Fixes: 9d7de2aa8b41 ("Use relative percpu offsets")
> Signed-off-by: Brian Gerst <brgerst@gmail.com>
> Cc: Andrii Nakryiko <andrii@kernel.org>
> Cc: Alexei Starovoitov <ast@kernel.org>
> ---
>  kernel/bpf/verifier.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 60611df77957..f4859516b190 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -21707,7 +21707,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
>                          * way, it's fine to back out this inlining logic
>                          */
>  #ifdef CONFIG_SMP
> -                       insn_buf[0] = BPF_MOV32_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);
> +                       insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)&pcpu_hot.cpu_number);

wouldn't you need to at least drop the (u32) cast? Otherwise what's
really the point?

>                         insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0);
>                         insn_buf[2] = BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0);
>                         cnt = 3;
> --
> 2.48.1
>