* Re: [PATCH v3 4/7] riscv/runtime-const: Introduce runtime_const_mask_32()
From: Guo Ren @ 2026-04-03 9:42 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Sebastian Andrzej Siewior, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Samuel Holland, David Laight, Darren Hart,
Davidlohr Bueso, André Almeida, linux-arch, linux-kernel,
linux-s390, linux-riscv, linux-arm-kernel, Alexandre Ghiti,
Charlie Jenkins, Charles Mirabile
In-Reply-To: <20260402112250.2138-5-kprateek.nayak@amd.com>
On Thu, Apr 2, 2026 at 7:39 PM K Prateek Nayak <kprateek.nayak@amd.com> wrote:
>
> Futex hash computation requires a mask operation with read-only after
> init data that will be converted to a runtime constant in the subsequent
> commit.
>
> Introduce runtime_const_mask_32 to further optimize the mask operation
> in the futex hash computation hot path. GCC generates a:
>
> lui a0, 0x12346 # upper; +0x800 then >>12 for correct rounding
> addi a0, a0, 0x678 # lower 12 bits
> and a1, a1, a0 # a1 = a1 & a0
>
> pattern to tackle arbitrary 32-bit masks and the same was also suggested
> by Claude which is implemented here. The (__mask & val) operation is
> intentionally placed outside of asm block to allow compilers to further
> optimize it if possible.
>
> __runtime_fixup_ptr() already patches a "lui + addi" sequence which has
> been reused to patch the same sequence for __runtime_fixup_mask().
>
> Assisted-by: Claude:claude-sonnet-4-5
> Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
> ---
> Changelog v2..v3:
>
> o Moved the "&" operation outside the inline asm block to allow for
> compilers to further optimize it if possible. (Based on David's
> comment on ARM64 bits).
> ---
> arch/riscv/include/asm/runtime-const.h | 22 ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
> diff --git a/arch/riscv/include/asm/runtime-const.h b/arch/riscv/include/asm/runtime-const.h
> index d766e2b9e6df..85efba8ecf12 100644
> --- a/arch/riscv/include/asm/runtime-const.h
> +++ b/arch/riscv/include/asm/runtime-const.h
> @@ -153,6 +153,22 @@
> __ret; \
> })
>
> +#define runtime_const_mask_32(val, sym) \
> +({ \
> + u32 __mask; \
> + asm_inline(".option push\n\t" \
> + ".option norvc\n\t" \
> + "1:\t" \
> + "lui %[__mask],0x89abd\n\t" \
> + "addi %[__mask],%[__mask],-0x211\n\t" \
Ref include/uapi/linux/reboot.h:
#define LINUX_REBOOT_CMD_CAD_ON 0x89ABCDEF
#define RUNTIME_MAGIC 0x89ABCDEF
"lui %[__mask], %%hi(RUNTIME_MAGIC)\n\t"
"addi %[__mask], %[__mask], %%lo(RUNTIME_MAGIC)\n\t"
> + ".option pop\n\t" \
> + ".pushsection runtime_mask_" #sym ",\"a\"\n\t" \
> + ".long 1b - .\n\t" \
> + ".popsection" \
> + : [__mask] "=r" (__mask)); \
> + (__mask & val); \
> +})
> +
> #define runtime_const_init(type, sym) do { \
> extern s32 __start_runtime_##type##_##sym[]; \
> extern s32 __stop_runtime_##type##_##sym[]; \
> @@ -256,6 +272,12 @@ static inline void __runtime_fixup_shift(void *where, unsigned long val)
> mutex_unlock(&text_mutex);
> }
>
> +static inline void __runtime_fixup_mask(void *where, unsigned long val)
> +{
> + __runtime_fixup_32(where, where + 4, val);
> + __runtime_fixup_caches(where, 2);
> +}
> +
> static inline void runtime_const_fixup(void (*fn)(void *, unsigned long),
> unsigned long val, s32 *start, s32 *end)
> {
> --
> 2.34.1
>
>
--
Best Regards
Guo Ren
^ permalink raw reply
* Re: [PATCH v5 3/3] arm64,ppc64le/kdump: pass dm-crypt keys to kdump kernel
From: Coiby Xu @ 2026-04-03 9:36 UTC (permalink / raw)
To: Sourabh Jain
Cc: kexec, linux-arm-kernel, linuxppc-dev, devicetree,
Arnaud Lefebvre, Baoquan he, Dave Young, Kairui Song, Pingfan Liu,
Andrew Morton, Krzysztof Kozlowski, Rob Herring, Thomas Staudt,
Will Deacon, Christophe Leroy (CS GROUP), Catalin Marinas,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Saravana Kannan, open list
In-Reply-To: <51761fcf-955f-45e2-97a5-2b49d8e79d04@linux.ibm.com>
On Thu, Apr 02, 2026 at 04:24:14PM +0530, Sourabh Jain wrote:
>
>
>On 25/02/26 11:33, Coiby Xu wrote:
>>CONFIG_CRASH_DM_CRYPT has been introduced to support LUKS-encrypted
>>device dump target by addressing two challenges [1],
>> - Kdump kernel may not be able to decrypt the LUKS partition. For some
>> machines, a system administrator may not have a chance to enter the
>> password to decrypt the device in kdump initramfs after the 1st kernel
>> crashes
>>
>> - LUKS2 by default use the memory-hard Argon2 key derivation function
>> which is quite memory-consuming compared to the limited memory reserved
>> for kdump.
>>
>>To also enable this feature for ARM64 and PowerPC, the missing piece is
>>to let the kdump kernel know where to find the dm-crypt keys which are
>>randomly stored in memory reserved for kdump. Introduce a new device
>>tree property dmcryptkeys [2] as similar to elfcorehdr to pass the
>>memory address of the stored info of dm-crypt keys to the kdump kernel.
>>Since this property is only needed by the kdump kernel, it won't be
>>exposed to user space.
>>
>>[1] https://lore.kernel.org/all/20250502011246.99238-1-coxu@redhat.com/
>>[2] https://github.com/devicetree-org/dt-schema/pull/181
>>
>>Cc: Arnaud Lefebvre <arnaud.lefebvre@clever-cloud.com>
>>Cc: Baoquan he <bhe@redhat.com>
>>Cc: Dave Young <dyoung@redhat.com>
>>Cc: Kairui Song <ryncsn@gmail.com>
>>Cc: Pingfan Liu <kernelfans@gmail.com>
>>Cc: Andrew Morton <akpm@linux-foundation.org>
>>Cc: Krzysztof Kozlowski <krzk@kernel.org>
>>Cc: Rob Herring <robh@kernel.org>
>>Cc: Thomas Staudt <tstaudt@de.ibm.com>
>>Cc: Sourabh Jain <sourabhjain@linux.ibm.com>
>>Cc: Will Deacon <will@kernel.org>
>>Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
>>Signed-off-by: Coiby Xu <coxu@redhat.com>
>>---
>> arch/arm64/kernel/machine_kexec_file.c | 4 ++++
>> arch/powerpc/kexec/elf_64.c | 4 ++++
>> drivers/of/fdt.c | 21 +++++++++++++++++++++
>> drivers/of/kexec.c | 19 +++++++++++++++++++
>> 4 files changed, 48 insertions(+)
>>
>>diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c
>>index fba260ad87a9..e31fabed378a 100644
>>--- a/arch/arm64/kernel/machine_kexec_file.c
>>+++ b/arch/arm64/kernel/machine_kexec_file.c
>>@@ -134,6 +134,10 @@ int load_other_segments(struct kimage *image,
>> kexec_dprintk("Loaded elf core header at 0x%lx bufsz=0x%lx memsz=0x%lx\n",
>> image->elf_load_addr, kbuf.bufsz, kbuf.memsz);
>>+
>>+ ret = crash_load_dm_crypt_keys(image);
>>+ if (ret)
>>+ goto out_err;
>> }
>> #endif
>>diff --git a/arch/powerpc/kexec/elf_64.c b/arch/powerpc/kexec/elf_64.c
>>index 5d6d616404cf..ea50a072debf 100644
>>--- a/arch/powerpc/kexec/elf_64.c
>>+++ b/arch/powerpc/kexec/elf_64.c
>>@@ -79,6 +79,10 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,
>> goto out;
>> }
>>+ ret = crash_load_dm_crypt_keys(image);
>>+ if (ret)
>>+ goto out;
>>+
>> /* Setup cmdline for kdump kernel case */
>> modified_cmdline = setup_kdump_cmdline(image, cmdline,
>> cmdline_len);
>>diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
>>index 331646d667b9..2967e4aff807 100644
>>--- a/drivers/of/fdt.c
>>+++ b/drivers/of/fdt.c
>>@@ -866,6 +866,26 @@ static void __init early_init_dt_check_for_elfcorehdr(unsigned long node)
>> elfcorehdr_addr, elfcorehdr_size);
>> }
>>+static void __init early_init_dt_check_for_dmcryptkeys(unsigned long node)
>>+{
>>+ const char *prop_name = "linux,dmcryptkeys";
>>+ const __be32 *prop;
>>+
>>+ if (!IS_ENABLED(CONFIG_CRASH_DM_CRYPT))
>>+ return;
>>+
>>+ pr_debug("Looking for dmcryptkeys property... ");
>>+
>>+ prop = of_get_flat_dt_prop(node, prop_name, NULL);
>>+ if (!prop)
>>+ return;
>>+
>>+ dm_crypt_keys_addr = dt_mem_next_cell(dt_root_addr_cells, &prop);
>>+
>>+ /* Property only accessible to crash dump kernel */
>>+ fdt_delprop(initial_boot_params, node, prop_name);
>>+}
>>+
>> static unsigned long chosen_node_offset = -FDT_ERR_NOTFOUND;
>> /*
>>@@ -1097,6 +1117,7 @@ int __init early_init_dt_scan_chosen(char *cmdline)
>> early_init_dt_check_for_initrd(node);
>> early_init_dt_check_for_elfcorehdr(node);
>>+ early_init_dt_check_for_dmcryptkeys(node);
>> rng_seed = of_get_flat_dt_prop(node, "rng-seed", &l);
>> if (rng_seed && l > 0) {
>>diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c
>>index c4cf3552c018..fbd253f0d3c5 100644
>>--- a/drivers/of/kexec.c
>>+++ b/drivers/of/kexec.c
>>@@ -423,6 +423,25 @@ void *of_kexec_alloc_and_setup_fdt(const struct kimage *image,
>> if (ret)
>> goto out;
>>+ if (image->dm_crypt_keys_addr != 0) {
>>+ ret = fdt_appendprop_addrrange(fdt, 0, chosen_node,
>>+ "linux,dmcryptkeys",
>>+ image->dm_crypt_keys_addr,
>>+ image->dm_crypt_keys_sz);
>>+
>>+ if (ret)
>>+ goto out;
>>+
>>+ /*
>>+ * Avoid dmcryptkeys from being stomped on in kdump kernel by
>>+ * setting up memory reserve map.
>>+ */
>>+ ret = fdt_add_mem_rsv(fdt, image->dm_crypt_keys_addr,
>>+ image->dm_crypt_keys_sz);
>>+ if (ret)
>>+ goto out;
>>+ }
>>+
>> #ifdef CONFIG_CRASH_DUMP
>> /* add linux,usable-memory-range */
>> ret = fdt_appendprop_addrrange(fdt, 0, chosen_node,
>
>The above changes look good to me.
>
>Feel free to add:
>Reviewed-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Thanks for reviewing the patch!
>
>But while reading crash_load_dm_crypt_keys() I noticed a possibility of a
>double free at the address pointed by `keys_header`:
>
>In crash_load_dm_crypt_keys()/crash_dump_dm_crypt.c
> snip...
>
> kbuf.buffer = keys_header;
>
> snip....
>
> r = kexec_add_buffer(&kbuf);
> if (r) {
> pr_err("Failed to call kexec_add_buffer, ret=%d\n", r);
> kvfree((void *)kbuf.buffer); <---
>First Free
> return r;
> }
>
>Since `keys_header` is not reset, the next call to build_keys_header()
>will cause a double free at `keys_header`.
>
>static int build_keys_header(void)
>{
>
> snip...
>
> if (keys_header != NULL)
> kvfree(keys_header);
>
> snip...
>}
>
>What do you think?
>
>- Sourabh Jain
Good catch! I'll send a patch to address this issue. Thanks!
--
Best regards,
Coiby
^ permalink raw reply
* Re: [PATCH 6.1.y 5/8] nvme-apple: remove an extra queue reference
From: Fedor Pchelkin @ 2026-04-03 9:35 UTC (permalink / raw)
To: Heyne, Maximilian
Cc: Christoph Hellwig, Sagi Grimberg, stable@vger.kernel.org,
Sven Peter, Chaitanya Kulkarni, Keith Busch, Jens Axboe,
Hector Martin, Alyssa Rosenzweig, James E.J. Bottomley,
Martin K. Petersen, Alim Akhtar, Avri Altman, Bart Van Assche,
Sasha Levin, Peter Wang, Greg Kroah-Hartman, Seunghui Lee,
Sanjeev Yadav, Wonkon Kim, Brian Kao, Hannes Reinecke, Ming Lei,
linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org
In-Reply-To: <20260402-anti-cuba-1ea02cea@mheyne-amazon>
On Thu, 02. Apr 12:31, Heyne, Maximilian wrote:
> On Wed, Apr 01, 2026 at 11:45:57PM +0300, Fedor Pchelkin wrote:
> > Hello,
> >
> > "Heyne, Maximilian" <mheyne@amazon.de> wrote:
> > > From: Christoph Hellwig <hch@lst.de>
> > >
> > > [ Upstream commit 941f7298c70c7668416e7845fa76eb72c07d966b ]
> > >
> > > Now that blk_mq_destroy_queue does not release the queue reference, there
> > > is no need for a second admin queue reference to be held by the
> > > apple_nvme structure.
> >
> > This patch is probably buggy in upstream. It removes extra reference
> > ->get, but doesn't remove the corresponding ->put which is located
> > inside apple_nvme_free_ctrl().
>
> Now I'm seeing this as well. Has the same problem as the pci driver in
> 6.1 where blk_put_queue is called from nvme_free_ctrl() and again from
> apple_nvme_free_ctrl(). Thank you for catching this. I don't have the
> hardware to test this.
>
> Are you going to send a fix upstream? It's looks to be broken on master,
> too.
I don't have the needed hardware either but will send the patch for review.
> >
> > That said, the other part of the backport series FWIW looks good to me,
> > and I've also verified it resolves the 6.1.y regression.
>
> You may leave a Tested-by if you want ;-)
I'll leave it for v2 then.
^ permalink raw reply
* [PATCH bpf-next v11 4/5] bpf, x86: Emit ENDBR for indirect jump targets
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260403090915.473493-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support CET/IBT, the indirect jump selftest triggers
a kernel panic because the indirect jump targets lack ENDBR
instructions.
To fix it, emit an ENDBR instruction to each indirect jump target. Since
the ENDBR instruction shifts the position of original jited instructions,
fix the instruction address calculation wherever the addresses are used.
For reference, below is a sample panic log.
Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cet.c:133!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
...
? 0xffffffffc00fb258
? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
bpf_prog_test_run_syscall+0x110/0x2f0
? fdget+0xba/0xe0
__sys_bpf+0xe4b/0x2590
? __kmalloc_node_track_caller_noprof+0x1c7/0x680
? bpf_prog_test_run_syscall+0x215/0x2f0
__x64_sys_bpf+0x21/0x30
do_syscall_64+0x85/0x620
? bpf_prog_test_run_syscall+0x1e2/0x2f0
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72d9a5faa230..ea9e707e8abf 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
#define EMIT_ENDBR() EMIT(gen_endbr(), 4)
#define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4)
#else
-#define EMIT_ENDBR()
-#define EMIT_ENDBR_POISON()
+#define EMIT_ENDBR() do { } while (0)
+#define EMIT_ENDBR_POISON() do { } while (0)
#endif
static bool is_imm8(int value)
@@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
-static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image,
- int oldproglen, struct jit_context *ctx, bool jmp_padding)
+static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
+ u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
bool tail_call_reachable = bpf_prog->aux->tail_call_reachable;
struct bpf_insn *insn = bpf_prog->insnsi;
@@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
void __percpu *priv_stack_ptr;
int i, excnt = 0;
int ilen, proglen = 0;
- u8 *prog = temp;
+ u8 *ip, *prog = temp;
u32 stack_depth;
int err;
@@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
dst_reg = X86_REG_R9;
}
+ if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1))
+ EMIT_ENDBR();
+
+ ip = image + addrs[i - 1] + (prog - temp);
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off))
/* call */
case BPF_JMP | BPF_CALL: {
- u8 *ip = image + addrs[i - 1];
-
func = (u8 *) __bpf_call_base + imm32;
if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off))
if (imm32)
emit_bpf_tail_call_direct(bpf_prog,
&bpf_prog->aux->poke_tab[imm32 - 1],
- &prog, image + addrs[i - 1],
+ &prog,
+ ip,
callee_regs_used,
stack_depth,
ctx);
@@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off))
&prog,
callee_regs_used,
stack_depth,
- image + addrs[i - 1],
+ ip,
ctx);
break;
@@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off))
break;
case BPF_JMP | BPF_JA | BPF_X:
- emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+ emit_indirect_jump(&prog, insn->dst_reg, ip);
break;
case BPF_JMP | BPF_JA:
case BPF_JMP32 | BPF_JA:
@@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off))
ctx->cleanup_addr = proglen;
if (bpf_prog_was_classic(bpf_prog) &&
!ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
- u8 *ip = image + addrs[i - 1];
-
if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
return -EINVAL;
}
@@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
for (pass = 0; pass < MAX_PASSES || image; pass++) {
if (!padding && pass >= PADDING_PASSES)
padding = true;
- proglen = do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, padding);
+ proglen = do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, padding);
if (proglen <= 0) {
out_image:
image = NULL;
--
2.43.0
^ permalink raw reply related
* [PATCH bpf-next v11 3/5] bpf: Add helper to detect indirect jump targets
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260403090915.473493-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Introduce helper bpf_insn_is_indirect_target to check whether a BPF
instruction is an indirect jump target.
Since the verifier knows which instructions are indirect jump targets,
add a new flag indirect_target to struct bpf_insn_aux_data to mark
them. The verifier sets this flag when verifying an indirect jump target
instruction, and the helper checks the flag to determine whether an
instruction is an indirect jump target.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
include/linux/bpf.h | 2 ++
include/linux/bpf_verifier.h | 9 +++++----
kernel/bpf/core.c | 9 +++++++++
kernel/bpf/verifier.c | 18 ++++++++++++++++++
4 files changed, 34 insertions(+), 4 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 05b34a6355b0..90760e250865 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1541,6 +1541,8 @@ bool bpf_has_frame_pointer(unsigned long ip);
int bpf_jit_charge_modmem(u32 size);
void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 512f7becc3dd..3f55ecb0ffb5 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -578,16 +578,17 @@ struct bpf_insn_aux_data {
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
- bool jmp_point;
- bool prune_point;
+ u32 jmp_point:1;
+ u32 prune_point:1;
/* ensure we check state equivalence and save state checkpoint and
* this instruction, regardless of any heuristics
*/
- bool force_checkpoint;
+ u32 force_checkpoint:1;
/* true if instruction is a call to a helper function that
* accepts callback function as a parameter.
*/
- bool calls_callback;
+ u32 calls_callback:1;
+ u32 indirect_target:1; /* if it is an indirect jump target */
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 0327008ba5e3..2539ae1b4395 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1578,6 +1578,15 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
env->blinded_insn_aux_len = clone->len;
return clone;
}
+
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx)
+{
+ if (!env)
+ return false;
+ insn_idx += prog->aux->subprog_start;
+ return env->insn_aux_data[insn_idx].indirect_target;
+}
#endif /* CONFIG_BPF_JIT */
/* Base function for offset calculation. Needs to go into .text section,
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 3378b373a461..4d8b6dd56d80 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4049,6 +4049,11 @@ static bool is_jmp_point(struct bpf_verifier_env *env, int insn_idx)
return env->insn_aux_data[insn_idx].jmp_point;
}
+static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].indirect_target = true;
+}
+
#define LR_FRAMENO_BITS 3
#define LR_SPI_BITS 6
#define LR_ENTRY_BITS (LR_SPI_BITS + LR_FRAMENO_BITS + 1)
@@ -21227,12 +21232,14 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
}
for (i = 0; i < n - 1; i++) {
+ mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
env->insn_idx, env->cur_state->speculative);
if (IS_ERR(other_branch))
return PTR_ERR(other_branch);
}
env->insn_idx = env->gotox_tmp_buf->items[n-1];
+ mark_indirect_target(env, env->insn_idx);
return 0;
}
@@ -22158,6 +22165,17 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
data[i].seen = old_seen;
data[i].zext_dst = insn_has_def32(insn + i);
}
+
+ /* The indirect_target flag of the original instruction was moved to the last of the
+ * new instructions by the above memmove and memset, but the indirect jump target is
+ * actually the first instruction, so move it back. This also matches with the behavior
+ * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
+ * instruction.
+ */
+ if (data[off + cnt - 1].indirect_target) {
+ data[off].indirect_target = 1;
+ data[off + cnt - 1].indirect_target = 0;
+ }
}
static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)
--
2.43.0
^ permalink raw reply related
* [PATCH bpf-next v11 2/5] bpf: Pass bpf_verifier_env to JIT
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260403090915.473493-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will
use env->insn_aux_data in the JIT stage to detect indirect jump targets.
Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c
code without verifier, introduce helper __bpf_prog_select_runtime()
to accept the env parameter.
Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and
switch to call __bpf_prog_select_runtime() in the verifier, with env
variable passed. The original bpf_prog_select_runtime() is preserved for
cbpf and lib/test_bpf.c, where env is NULL.
Now all constants blinding calls are moved into the verifier, except
the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted
by bpf_patch_insn_data() function for normal cases, so there is no need
to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 2 +-
arch/arm/net/bpf_jit_32.c | 2 +-
arch/arm64/net/bpf_jit_comp.c | 2 +-
arch/loongarch/net/bpf_jit.c | 2 +-
arch/mips/net/bpf_jit_comp.c | 2 +-
arch/parisc/net/bpf_jit_core.c | 2 +-
arch/powerpc/net/bpf_jit_comp.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 2 +-
arch/s390/net/bpf_jit_comp.c | 2 +-
arch/sparc/net/bpf_jit_comp_64.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
include/linux/filter.h | 4 +-
kernel/bpf/core.c | 63 ++++++++++++--------------------
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 20 ++++++----
16 files changed, 50 insertions(+), 65 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 973ceae48675..639a2736f029 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -1400,7 +1400,7 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog)
* (re)locations involved that their addresses are not known
* during the first run.
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
vm_dump(prog);
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index e6b1bb2de627..1628b6fc70a4 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2142,7 +2142,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header;
struct jit_ctx ctx;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index cd5a72fff500..7212ec89dfe3 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2006,7 +2006,7 @@ struct arm64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
struct bpf_binary_header *header;
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index fcc8c0c29fb0..5149ce4cef7e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1920,7 +1920,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index d2b6c955f18e..6ee4abe6a1f7 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -909,7 +909,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index 35dca372b5df..172770132440 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 711028bebea3..27fecb4cc063 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -129,7 +129,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
u32 proglen;
u32 alloclen;
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 527baa50dc68..768ac686b359 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 2dfc279b1be2..94128fe6be23 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2312,7 +2312,7 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
/*
* Compile eBPF program "fp"
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index e83e29137566..2fa0e9375127 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1477,7 +1477,7 @@ struct sparc64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 77d00a8dec87..72d9a5faa230 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3713,7 +3713,7 @@ struct x64_jit_data {
#define MAX_PASSES 20
#define PADDING_PASSES (MAX_PASSES - 5)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5f259577614a..852baf2e4db4 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2518,7 +2518,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
int proglen, oldproglen = 0;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index d396e55c9a1d..d7ae6fa1998b 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1107,6 +1107,8 @@ static inline int sk_filter_reason(struct sock *sk, struct sk_buff *skb,
return sk_filter_trim_cap(sk, skb, 1, reason);
}
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err);
struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err);
void bpf_prog_free(struct bpf_prog *fp);
@@ -1152,7 +1154,7 @@ u64 __bpf_call_base(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
((u64 (*)(u64, u64, u64, u64, u64, const struct bpf_insn *)) \
(void *)__bpf_call_base)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog);
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_compile(struct bpf_prog *prog);
bool bpf_jit_needs_zext(void);
bool bpf_jit_inlines_helper_call(s32 imm);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index b458a85b3e18..0327008ba5e3 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1489,23 +1489,6 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
bpf_prog_clone_free(fp_other);
}
-static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
-{
-#ifdef CONFIG_BPF_SYSCALL
- struct bpf_map *map;
- int i;
-
- if (len <= 1)
- return;
-
- for (i = 0; i < prog->aux->used_map_cnt; i++) {
- map = prog->aux->used_maps[i];
- if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
- bpf_insn_array_adjust(map, off, len);
- }
-#endif
-}
-
/* Now this function is used only to blind the main prog and must be invoked only when
* bpf_prog_need_blind() returns true.
*/
@@ -1580,12 +1563,6 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
if (env)
env->prog = clone;
- else
- /* Instructions arrays must be updated using absolute xlated offsets.
- * The arrays have already been adjusted by bpf_patch_insn_data() when
- * env is not NULL.
- */
- adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2559,14 +2536,14 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
-static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
#ifdef CONFIG_BPF_JIT
bool blinded = false;
struct bpf_prog *orig_prog = prog;
if (bpf_prog_need_blind(orig_prog)) {
- prog = bpf_jit_blind_constants(NULL, orig_prog);
+ prog = bpf_jit_blind_constants(env, orig_prog);
/* If blinding was requested and we failed during blinding, we must fall
* back to the interpreter.
*/
@@ -2575,7 +2552,7 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
blinded = true;
}
- prog = bpf_int_jit_compile(prog);
+ prog = bpf_int_jit_compile(env, prog);
if (blinded) {
if (!prog->jited) {
bpf_jit_prog_release_other(orig_prog, prog);
@@ -2588,18 +2565,8 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
return prog;
}
-/**
- * bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with BPF program
- * @err: pointer to error variable
- *
- * Try to JIT eBPF program, if JIT is not available, use interpreter.
- * The BPF program will be executed via bpf_prog_run() function.
- *
- * Return: the &fp argument along with &err set to 0 for success or
- * a negative errno code on failure
- */
-struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err)
{
/* In case of BPF to BPF calls, verifier did all the prep
* work with regards to JITing, etc.
@@ -2627,7 +2594,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_prog_jit_compile(fp);
+ fp = bpf_prog_jit_compile(env, fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
@@ -2653,6 +2620,22 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
return fp;
}
+
+/**
+ * bpf_prog_select_runtime - select exec runtime for BPF program
+ * @fp: bpf_prog populated with BPF program
+ * @err: pointer to error variable
+ *
+ * Try to JIT eBPF program, if JIT is not available, use interpreter.
+ * The BPF program will be executed via bpf_prog_run() function.
+ *
+ * Return: the &fp argument along with &err set to 0 for success or
+ * a negative errno code on failure
+ */
+struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+{
+ return __bpf_prog_select_runtime(NULL, fp, err);
+}
EXPORT_SYMBOL_GPL(bpf_prog_select_runtime);
static unsigned int __bpf_prog_ret1(const void *ctx,
@@ -3140,7 +3123,7 @@ const struct bpf_func_proto bpf_tail_call_proto = {
* It is encouraged to implement bpf_int_jit_compile() instead, so that
* eBPF and implicitly also cBPF can get JITed!
*/
-struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
return prog;
}
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index e1505c9cd09e..553dca175640 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3090,10 +3090,6 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
if (err < 0)
goto free_used_maps;
- prog = bpf_prog_select_runtime(prog, &err);
- if (err < 0)
- goto free_used_maps;
-
err = bpf_prog_mark_insn_arrays_ready(prog);
if (err < 0)
goto free_used_maps;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 0d6bfccf0dd7..3378b373a461 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -23142,7 +23142,7 @@ static int __jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
- func[i] = bpf_int_jit_compile(func[i]);
+ func[i] = bpf_int_jit_compile(env, func[i]);
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -23186,7 +23186,7 @@ static int __jit_subprogs(struct bpf_verifier_env *env)
}
for (i = 0; i < env->subprog_cnt; i++) {
old_bpf_func = func[i]->bpf_func;
- tmp = bpf_int_jit_compile(func[i]);
+ tmp = bpf_int_jit_compile(env, func[i]);
if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) {
verbose(env, "JIT doesn't support bpf-to-bpf calls\n");
err = -ENOTSUPP;
@@ -26504,6 +26504,16 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
convert_pseudo_ld_imm64(env);
}
+ /* extension progs temporarily inherit the attach_type of their targets
+ for verification purposes, so set it back to zero before returning
+ */
+ if (env->prog->type == BPF_PROG_TYPE_EXT)
+ env->prog->expected_attach_type = 0;
+
+ env->prog = __bpf_prog_select_runtime(env, env->prog, &ret);
+ if (ret)
+ goto err_release_maps;
+
adjust_btf_func(env);
err_release_maps:
@@ -26517,12 +26527,6 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
if (!env->prog->aux->used_btfs)
release_btfs(env);
- /* extension progs temporarily inherit the attach_type of their targets
- for verification purposes, so set it back to zero before returning
- */
- if (env->prog->type == BPF_PROG_TYPE_EXT)
- env->prog->expected_attach_type = 0;
-
*prog = env->prog;
module_put(env->attach_btf_mod);
--
2.43.0
^ permalink raw reply related
* [PATCH bpf-next v11 0/5] emit ENDBR/BTI instructions for indirect jump targets
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.
The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, allowing
the JIT to consult env->insn_aux_data to determine which instructions are
indirect jump targets.
During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not use env->insn_aux_data directly.
To avoid this mismatch, and given that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.
v11:
- Restore env->subprog_info after jit_subprogs() fails
- Clear prog->jit_requested and prog->blinding_requested on failure
- Use the actual env->insn_aux_data size in clear_insn_aux_data() on failure
v10: https://lore.kernel.org/bpf/20260324122052.342751-1-xukuohai@huaweicloud.com
- Fix the incorrect call_imm restore in jit_subprogs
- Define a dummy void version of bpf_jit_prog_release_other and
bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)
v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com/
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)
v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com/
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
to adjust_insn_aux_data()
v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch
v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com/
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier
v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com/
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
each prog
v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com/
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
jump targets, and emit ENDBR on x86
v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com/
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)
v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com/
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)
v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com/
Xu Kuohai (5):
bpf: Move constants blinding out of arch-specific JITs
bpf: Pass bpf_verifier_env to JIT
bpf: Add helper to detect indirect jump targets
bpf, x86: Emit ENDBR for indirect jump targets
bpf, arm64: Emit BTI for indirect jump target
arch/arc/net/bpf_jit_core.c | 41 +++-----
arch/arm/net/bpf_jit_32.c | 43 ++-------
arch/arm64/net/bpf_jit_comp.c | 87 ++++++-----------
arch/loongarch/net/bpf_jit.c | 61 ++++--------
arch/mips/net/bpf_jit_comp.c | 22 +----
arch/parisc/net/bpf_jit_core.c | 75 ++++++---------
arch/powerpc/net/bpf_jit_comp.c | 70 +++++---------
arch/riscv/net/bpf_jit_core.c | 63 +++++-------
arch/s390/net/bpf_jit_comp.c | 61 ++++--------
arch/sparc/net/bpf_jit_comp_64.c | 63 +++++-------
arch/x86/net/bpf_jit_comp.c | 73 +++++---------
arch/x86/net/bpf_jit_comp32.c | 35 +------
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 10 +-
include/linux/filter.h | 37 ++++++-
kernel/bpf/core.c | 127 ++++++++++++++++--------
kernel/bpf/syscall.c | 4 -
kernel/bpf/verifier.c | 159 ++++++++++++++++++++++++-------
18 files changed, 480 insertions(+), 553 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH bpf-next v11 1/5] bpf: Move constants blinding out of arch-specific JITs
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260403090915.473493-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
During the JIT stage, constants blinding rewrites instructions but only
rewrites the private instruction copy of the JITed subprog, leaving the
global env->prog->insni and env->insn_aux_data untouched. This causes a
mismatch between subprog instructions and the global state, making it
difficult to use the global data in the JIT.
To avoid this mismatch, and given that all arch-specific JITs already
support constants blinding, move it to the generic verifier code, and
switch to rewrite the global env->prog->insnsi with the global states
adjusted, as other rewrites in the verifier do.
This removes the constants blinding calls in each JIT, which are largely
duplicated code across architectures.
Since constants blinding is only required for JIT, and there are two
JIT entry functions, jit_subprogs() for BPF programs with multiple
subprogs and bpf_prog_select_runtime() for programs with no subprogs,
move the constants blinding invocation into these two functions.
In the verifier path, bpf_patch_insn_data() is used to keep global
verifier auxiliary data in sync with patched instructions. A key
question is whether this global auxiliary data should be restored
on the failure path.
Besides instructions, bpf_patch_insn_data() adjusts:
- prog->aux->poke_tab
- env->insn_array_maps
- env->subprog_info
- env->insn_aux_data
For prog->aux->poke_tab, it is only used by JIT or only meaningful after
JIT succeeds, so it does not need to be restored on the failure path.
For env->insn_array_maps, when JIT fails, programs using insn arrays
are rejected by bpf_insn_array_ready() due to missing JIT addresses.
Hence, env->insn_array_maps is only meaningful for JIT and does not need
to be restored.
For subprog_info, if jit_subprogs fails and CONFIG_BPF_JIT_ALWAYS_ON
is not enabled, kernel falls back to interpreter. In this case,
env->subprog_info is used to determine subprogram stack depth. So it
must be restored on failure.
For env->insn_aux_data, it is freed by clean_insn_aux_data() at the
end of bpf_check(). Before freeing, clean_insn_aux_data() loops over
env->insn_aux_data to release jump targets recorded in it. The loop
uses env->prog->len as the array length, but this length no longer
matches the actual size of the adjusted env->insn_aux_data array after
constants blinding.
There are two possible approaches to address it.
One is to clone the original env->insn_aux_data before blinding and
restore it after failure, similar to env->prog. But since insn_aux_data
is only used by clean_insn_aux_data() before being freed on the failure
path, and it is a big array, this approach introduces unnecessary memory
allocation and data copying overhead.
The other approach is to keep insn_aux_data as adjusted after failure,
since it will be freed shortly, and record its actual size for the loop
in clean_insn_aux_data(). This adds almost no extra overhead, and is
therefore adopted.
For classic BPF programs, constants blinding works as before since it
is still invoked from bpf_prog_select_runtime().
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Hari Bathini <hbathini@linux.ibm.com> # powerpc jit
Reviewed-by: Pu Lehui <pulehui@huawei.com> # riscv jit
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 39 ++++------
arch/arm/net/bpf_jit_32.c | 41 ++---------
arch/arm64/net/bpf_jit_comp.c | 72 +++++-------------
arch/loongarch/net/bpf_jit.c | 59 +++++----------
arch/mips/net/bpf_jit_comp.c | 20 +----
arch/parisc/net/bpf_jit_core.c | 73 ++++++++-----------
arch/powerpc/net/bpf_jit_comp.c | 68 +++++++----------
arch/riscv/net/bpf_jit_core.c | 61 ++++++----------
arch/s390/net/bpf_jit_comp.c | 59 +++++----------
arch/sparc/net/bpf_jit_comp_64.c | 61 ++++++----------
arch/x86/net/bpf_jit_comp.c | 43 ++---------
arch/x86/net/bpf_jit_comp32.c | 33 +--------
include/linux/bpf_verifier.h | 1 +
include/linux/filter.h | 33 ++++++++-
kernel/bpf/core.c | 75 ++++++++++++++++---
kernel/bpf/verifier.c | 121 +++++++++++++++++++++++++------
16 files changed, 383 insertions(+), 476 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 1421eeced0f5..973ceae48675 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -79,7 +79,6 @@ struct arc_jit_data {
* The JIT pertinent context that is used by different functions.
*
* prog: The current eBPF program being handled.
- * orig_prog: The original eBPF program before any possible change.
* jit: The JIT buffer and its length.
* bpf_header: The JITed program header. "jit.buf" points inside it.
* emit: If set, opcodes are written to memory; else, a dry-run.
@@ -94,12 +93,10 @@ struct arc_jit_data {
* need_extra_pass: A forecast if an "extra_pass" will occur.
* is_extra_pass: Indicates if the current pass is an extra pass.
* user_bpf_prog: True, if VM opcodes come from a real program.
- * blinded: True if "constant blinding" step returned a new "prog".
* success: Indicates if the whole JIT went OK.
*/
struct jit_context {
struct bpf_prog *prog;
- struct bpf_prog *orig_prog;
struct jit_buffer jit;
struct bpf_binary_header *bpf_header;
bool emit;
@@ -114,7 +111,6 @@ struct jit_context {
bool need_extra_pass;
bool is_extra_pass;
bool user_bpf_prog;
- bool blinded;
bool success;
};
@@ -161,13 +157,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog)
{
memset(ctx, 0, sizeof(*ctx));
- ctx->orig_prog = prog;
-
- /* If constant blinding was requested but failed, scram. */
- ctx->prog = bpf_jit_blind_constants(prog);
- if (IS_ERR(ctx->prog))
- return PTR_ERR(ctx->prog);
- ctx->blinded = (ctx->prog != ctx->orig_prog);
+ ctx->prog = prog;
/* If the verifier doesn't zero-extend, then we have to do it. */
ctx->do_zext = !ctx->prog->aux->verifier_zext;
@@ -214,14 +204,6 @@ static inline void maybe_free(struct jit_context *ctx, void **mem)
*/
static void jit_ctx_cleanup(struct jit_context *ctx)
{
- if (ctx->blinded) {
- /* if all went well, release the orig_prog. */
- if (ctx->success)
- bpf_jit_prog_release_other(ctx->prog, ctx->orig_prog);
- else
- bpf_jit_prog_release_other(ctx->orig_prog, ctx->prog);
- }
-
maybe_free(ctx, (void **)&ctx->bpf2insn);
maybe_free(ctx, (void **)&ctx->jit_data);
@@ -229,12 +211,19 @@ static void jit_ctx_cleanup(struct jit_context *ctx)
ctx->bpf2insn_valid = false;
/* Freeing "bpf_header" is enough. "jit.buf" is a sub-array of it. */
- if (!ctx->success && ctx->bpf_header) {
- bpf_jit_binary_free(ctx->bpf_header);
- ctx->bpf_header = NULL;
- ctx->jit.buf = NULL;
- ctx->jit.index = 0;
- ctx->jit.len = 0;
+ if (!ctx->success) {
+ if (ctx->bpf_header) {
+ bpf_jit_binary_free(ctx->bpf_header);
+ ctx->bpf_header = NULL;
+ ctx->jit.buf = NULL;
+ ctx->jit.index = 0;
+ ctx->jit.len = 0;
+ }
+ if (ctx->is_extra_pass) {
+ ctx->prog->bpf_func = NULL;
+ ctx->prog->jited = 0;
+ ctx->prog->jited_len = 0;
+ }
}
ctx->emit = false;
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..e6b1bb2de627 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2144,9 +2144,7 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
- bool tmp_blinded = false;
struct jit_ctx ctx;
unsigned int tmp_idx;
unsigned int image_size;
@@ -2156,20 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- /* If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
-
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
@@ -2179,10 +2164,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* we must fall back to the interpreter
*/
ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
- if (ctx.offsets == NULL) {
- prog = orig_prog;
- goto out;
- }
+ if (ctx.offsets == NULL)
+ return prog;
/* 1) fake pass to find in the length of the JITed code,
* to compute ctx->offsets and other context variables
@@ -2194,10 +2177,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* being successful in the second pass, so just fall back
* to the interpreter.
*/
- if (build_body(&ctx)) {
- prog = orig_prog;
+ if (build_body(&ctx))
goto out_off;
- }
tmp_idx = ctx.idx;
build_prologue(&ctx);
@@ -2213,10 +2194,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx += ctx.imm_count;
if (ctx.imm_count) {
ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
- if (ctx.imms == NULL) {
- prog = orig_prog;
+ if (ctx.imms == NULL)
goto out_off;
- }
}
#else
/* there's nothing about the epilogue on ARMv7 */
@@ -2238,10 +2217,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Not able to allocate memory for the structure then
* we must fall back to the interpretation
*/
- if (header == NULL) {
- prog = orig_prog;
+ if (header == NULL)
goto out_imms;
- }
/* 2.) Actual pass to generate final JIT code */
ctx.target = (u32 *) image_ptr;
@@ -2278,16 +2255,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
#endif
out_off:
kfree(ctx.offsets);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free:
image_ptr = NULL;
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_imms;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index adf84962d579..cd5a72fff500 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2009,14 +2009,12 @@ struct arm64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header = NULL;
struct arm64_jit_data *jit_data;
void __percpu *priv_stack_ptr = NULL;
bool was_classic = bpf_prog_was_classic(prog);
int priv_stack_alloc_sz;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
@@ -2025,26 +2023,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int exentry_idx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -2056,10 +2041,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -2079,10 +2062,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kvzalloc_objs(int, prog->len + 1);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_off;
- }
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
@@ -2095,15 +2076,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* BPF line info needs ctx->offset[i] to be the offset of
* instruction[i] in jited image, so build prologue first.
*/
- if (build_prologue(&ctx, was_classic)) {
- prog = orig_prog;
+ if (build_prologue(&ctx, was_classic))
goto out_off;
- }
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_off;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx, was_classic);
@@ -2121,10 +2098,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_off;
- }
/* Pass 2: Determine jited position and result for each instruction */
@@ -2152,10 +2127,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free_hdr;
- }
ctx.epilogue_offset = ctx.idx;
ctx.exentry_idx = exentry_idx;
@@ -2164,19 +2137,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Pass 3: Adjust jump offset and write final image */
if (build_body(&ctx, extra_pass) ||
- WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset)) {
- prog = orig_prog;
+ WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
- }
build_epilogue(&ctx, was_classic);
build_plt(&ctx);
/* Extra pass to validate JITed code. */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free_hdr;
- }
/* update the real prog size */
prog_size = sizeof(u32) * ctx.idx;
@@ -2193,16 +2162,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (extra_pass && ctx.idx > jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d > %d\n",
ctx.idx, jit_data->ctx.idx);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
goto out_free_hdr;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header has been freed */
ro_header = NULL;
- prog = orig_prog;
- goto out_off;
+ header = NULL;
+ goto out_free_hdr;
}
/*
* The instructions have now been copied to the ROX region from
@@ -2245,13 +2211,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size,
sizeof(header->size));
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 9cb796e16379..fcc8c0c29fb0 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1922,43 +1922,26 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- bool tmp_blinded = false, extra_pass = false;
+ bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header;
- struct bpf_prog *tmp, *orig_prog = prog;
/*
* If BPF JIT was not enabled then we must fall back to
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- if (IS_ERR(tmp))
- return orig_prog;
-
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1978,17 +1961,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.offset = kvcalloc(prog->len + 1, sizeof(u32), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_offset;
- }
/* 1. Initial fake pass to compute ctx->idx and set ctx->flags */
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_offset;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
@@ -2004,10 +1983,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
&header, &image_ptr, jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_offset;
- }
/* 2. Now, the actual pass to generate final JIT code */
/*
@@ -2027,17 +2004,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.num_exentries = 0;
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free;
- }
build_epilogue(&ctx);
/* 3. Extra pass to validate JITed code */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free;
- }
/* And we're done */
if (bpf_jit_enable > 1)
@@ -2050,9 +2023,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_free;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header have been freed */
ro_header = NULL;
- prog = orig_prog;
+ header = NULL;
goto out_free;
}
/*
@@ -2084,13 +2057,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ? tmp : orig_prog);
-
return prog;
out_free:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size, sizeof(header->size));
bpf_jit_binary_pack_free(ro_header, header);
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index e355dfca4400..d2b6c955f18e 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -911,10 +911,8 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
- bool tmp_blinded = false;
unsigned int tmp_idx;
unsigned int image_size;
u8 *image_ptr;
@@ -925,19 +923,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
- /*
- * If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.program = prog;
@@ -1025,14 +1011,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited_len = image_size;
out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
kfree(ctx.descriptors);
return prog;
out_err:
- prog = orig_prog;
if (header)
bpf_jit_binary_free(header);
goto out;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index a5eb6b51e27a..35dca372b5df 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -44,30 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, prologue_len, i;
struct hppa_jit_data *jit_data;
struct hppa_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
@@ -81,10 +70,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!ctx->offset)
+ goto out_err;
for (i = 0; i < prog->len; i++) {
prev_ninsns += 20;
ctx->offset[i] = prev_ninsns;
@@ -93,10 +80,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
for (i = 0; i < NR_JIT_ITERATIONS; i++) {
pass++;
ctx->ninsns = 0;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (build_body(ctx, extra_pass, ctx->offset))
+ goto out_err;
ctx->body_len = ctx->ninsns;
bpf_jit_build_prologue(ctx);
ctx->prologue_len = ctx->ninsns - ctx->body_len;
@@ -116,10 +101,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->image,
sizeof(long),
bpf_fill_ill_insns);
- if (!jit_data->header) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!jit_data->header)
+ goto out_err;
ctx->insns = (u32 *)jit_data->image;
/*
@@ -134,8 +117,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
if (jit_data->header)
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
if (extable_size)
@@ -148,8 +130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx);
if (build_body(ctx, extra_pass, NULL)) {
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
bpf_jit_build_epilogue(ctx);
@@ -160,20 +141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{ extern int machine_restart(char *); machine_restart(""); }
}
+ if (!prog->is_func || extra_pass) {
+ if (bpf_jit_binary_lock_ro(jit_data->header)) {
+ bpf_jit_binary_free(jit_data->header);
+ goto out_err;
+ }
+ bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
+ }
+
prog->bpf_func = (void *)ctx->insns;
prog->jited = 1;
prog->jited_len = prog_size;
- bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
-
if (!prog->is_func || extra_pass) {
- if (bpf_jit_binary_lock_ro(jit_data->header)) {
- bpf_jit_binary_free(jit_data->header);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
- goto out_offset;
- }
prologue_len = ctx->epilogue_offset - ctx->body_len;
for (i = 0; i < prog->len; i++)
ctx->offset[i] += prologue_len;
@@ -183,14 +163,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
+
if (HPPA_JIT_REBOOT)
{ extern int machine_restart(char *); machine_restart(""); }
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_offset;
}
u64 hppa_div64(u64 div, u64 divisor)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index a62a9a92b7b5..711028bebea3 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -142,9 +142,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
int flen;
struct bpf_binary_header *fhdr = NULL;
struct bpf_binary_header *hdr = NULL;
- struct bpf_prog *org_fp = fp;
- struct bpf_prog *tmp_fp;
- bool bpf_blinded = false;
bool extra_pass = false;
u8 *fimage = NULL;
u32 *fcode_base;
@@ -152,24 +149,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u32 fixup_len;
if (!fp->jit_requested)
- return org_fp;
-
- tmp_fp = bpf_jit_blind_constants(org_fp);
- if (IS_ERR(tmp_fp))
- return org_fp;
-
- if (tmp_fp != org_fp) {
- bpf_blinded = true;
- fp = tmp_fp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = org_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
@@ -194,10 +180,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
addrs = kcalloc(flen + 1, sizeof(*addrs), GFP_KERNEL);
- if (addrs == NULL) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (addrs == NULL)
+ goto out_err;
memset(&cgctx, 0, sizeof(struct codegen_context));
bpf_jit_init_reg_mapping(&cgctx);
@@ -211,11 +195,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
cgctx.exception_cb = fp->aux->exception_cb;
/* Scouting faux-generate pass 0 */
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
/* We hit something illegal or unsupported. */
- fp = org_fp;
- goto out_addrs;
- }
+ goto out_err;
/*
* If we have seen a tail call, we need a second pass.
@@ -226,10 +208,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
+ goto out_err;
}
bpf_jit_realloc_regs(&cgctx);
@@ -250,10 +230,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
bpf_jit_fill_ill_insns);
- if (!fhdr) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (!fhdr)
+ goto out_err;
if (extable_len)
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
@@ -272,8 +250,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
extra_pass)) {
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
bpf_jit_binary_pack_free(fhdr, hdr);
- fp = org_fp;
- goto out_addrs;
+ goto out_err;
}
bpf_jit_build_epilogue(code_base, &cgctx);
@@ -295,15 +272,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
((u64 *)image)[1] = local_paca->kernel_toc;
#endif
+ if (!fp->is_func || extra_pass) {
+ if (bpf_jit_binary_pack_finalize(fhdr, hdr))
+ goto out_err;
+ }
+
fp->bpf_func = (void *)fimage;
fp->jited = 1;
fp->jited_len = cgctx.idx * 4 + FUNCTION_DESCR_SIZE;
if (!fp->is_func || extra_pass) {
- if (bpf_jit_binary_pack_finalize(fhdr, hdr)) {
- fp = org_fp;
- goto out_addrs;
- }
bpf_prog_fill_jited_linfo(fp, addrs);
out_addrs:
kfree(addrs);
@@ -318,11 +296,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
jit_data->hdr = hdr;
}
-out:
- if (bpf_blinded)
- bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);
-
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto out_addrs;
}
/*
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index b3581e926436..527baa50dc68 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -44,29 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, i;
struct rv_jit_data *jit_data;
struct rv_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
if (!jit_data) {
- prog = orig_prog;
- goto out;
+ return prog;
}
prog->aux->jit_data = jit_data;
}
@@ -83,15 +73,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
+ if (!ctx->offset)
goto out_offset;
- }
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_offset;
- }
for (i = 0; i < prog->len; i++) {
prev_ninsns += 32;
@@ -105,10 +91,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
ctx->prologue_len = ctx->ninsns;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, ctx->offset))
goto out_offset;
- }
ctx->epilogue_offset = ctx->ninsns;
bpf_jit_build_epilogue(ctx);
@@ -126,10 +110,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
bpf_fill_ill_insns);
- if (!jit_data->ro_header) {
- prog = orig_prog;
+ if (!jit_data->ro_header)
goto out_offset;
- }
/*
* Use the image(RW) for writing the JITed instructions. But also save
@@ -150,7 +132,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (i == NR_JIT_ITERATIONS) {
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
- prog = orig_prog;
goto out_free_hdr;
}
@@ -163,26 +144,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->nexentries = 0;
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_free_hdr;
- }
bpf_jit_build_epilogue(ctx);
if (bpf_jit_enable > 1)
bpf_jit_dump(prog->len, prog_size, pass, ctx->insns);
- prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
- prog->jited = 1;
- prog->jited_len = prog_size - cfi_get_offset();
-
if (!prog->is_func || extra_pass) {
if (WARN_ON(bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header))) {
/* ro_header has been freed */
jit_data->ro_header = NULL;
- prog = orig_prog;
- goto out_offset;
+ jit_data->header = NULL;
+ goto out_free_hdr;
}
+ }
+
+ prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
+ prog->jited = 1;
+ prog->jited_len = prog_size - cfi_get_offset();
+
+ if (!prog->is_func || extra_pass) {
/*
* The instructions have now been copied to the ROX region from
* where they will execute.
@@ -198,14 +180,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (jit_data->header) {
bpf_arch_text_copy(&jit_data->ro_header->size, &jit_data->header->size,
sizeof(jit_data->header->size));
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index d08d159b6319..2dfc279b1be2 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2314,36 +2314,20 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
*/
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
- struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
- bool tmp_blinded = false;
bool extra_pass = false;
struct bpf_jit jit;
int pass;
if (!fp->jit_requested)
- return orig_fp;
-
- tmp = bpf_jit_blind_constants(fp);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_fp;
- if (tmp != fp) {
- tmp_blinded = true;
- fp = tmp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = orig_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
if (jit_data->ctx.addrs) {
@@ -2356,34 +2340,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
memset(&jit, 0, sizeof(jit));
jit.addrs = kvcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
- if (jit.addrs == NULL) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (jit.addrs == NULL)
+ goto out_err;
/*
* Three initial passes:
* - 1/2: Determine clobbered registers
* - 3: Calculate program size and addrs array
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp, extra_pass)) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (bpf_jit_prog(&jit, fp, extra_pass))
+ goto out_err;
}
/*
* Final pass: Allocate and generate program
*/
header = bpf_jit_alloc(&jit, fp);
- if (!header) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (!header)
+ goto out_err;
skip_init_ctx:
if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
if (bpf_jit_enable > 1) {
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
@@ -2392,8 +2369,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
if (!fp->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
} else {
jit_data->header = header;
@@ -2411,11 +2387,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
kfree(jit_data);
fp->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(fp, fp == orig_fp ?
- tmp : orig_fp);
+
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto free_addrs;
}
bool bpf_jit_supports_kfunc_call(void)
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index b23d1c645ae5..e83e29137566 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1479,37 +1479,22 @@ struct sparc64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
u32 prev_image_size, image_size;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
int pass, i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1527,10 +1512,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kmalloc_array(prog->len, sizeof(unsigned int), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (ctx.offset == NULL)
+ goto out_err;
/* Longest sequence emitted is for bswap32, 12 instructions. Pre-cook
* the offset array so that we converge faster.
@@ -1543,10 +1526,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx = 0;
build_prologue(&ctx);
- if (build_body(&ctx)) {
- prog = orig_prog;
- goto out_off;
- }
+ if (build_body(&ctx))
+ goto out_err;
build_epilogue(&ctx);
if (bpf_jit_enable > 1)
@@ -1569,10 +1550,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = sizeof(u32) * ctx.idx;
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
- if (header == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (header == NULL)
+ goto out_err;
ctx.image = (u32 *)image_ptr;
skip_init_ctx:
@@ -1582,8 +1561,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (build_body(&ctx)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
build_epilogue(&ctx);
@@ -1592,8 +1570,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf_jit: Failed to converge, prev_size=%u size=%d\n",
prev_image_size, ctx.idx * 4);
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
if (bpf_jit_enable > 1)
@@ -1604,8 +1581,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (!prog->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
} else {
jit_data->ctx = ctx;
@@ -1624,9 +1600,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_off;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e9b78040d703..77d00a8dec87 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3717,13 +3717,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
void __percpu *priv_stack_ptr = NULL;
struct x64_jit_data *jit_data;
int priv_stack_alloc_sz;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
bool extra_pass = false;
bool padding = false;
u8 *rw_image = NULL;
@@ -3733,27 +3731,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -3765,10 +3749,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 8) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 8, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -3786,10 +3768,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto skip_init_addrs;
}
addrs = kvmalloc_objs(*addrs, prog->len + 1);
- if (!addrs) {
- prog = orig_prog;
+ if (!addrs)
goto out_addrs;
- }
/*
* Before first pass, make a rough estimation of addrs[]
@@ -3820,8 +3800,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
sizeof(rw_header->size));
bpf_jit_binary_pack_free(header, rw_header);
}
- /* Fall back to interpreter mode */
- prog = orig_prog;
if (extra_pass) {
prog->bpf_func = NULL;
prog->jited = 0;
@@ -3852,10 +3830,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
prog->aux->extable = (void *) image + roundup(proglen, align);
}
oldproglen = proglen;
@@ -3908,8 +3884,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image + cfi_get_offset();
prog->jited = 1;
prog->jited_len = proglen - cfi_get_offset();
- } else {
- prog = orig_prog;
}
if (!image || !prog->is_func || extra_pass) {
@@ -3925,10 +3899,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
}
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index dda423025c3d..5f259577614a 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2521,35 +2521,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
u8 *image = NULL;
int *addrs;
int pass;
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
addrs = kmalloc_objs(*addrs, prog->len);
- if (!addrs) {
- prog = orig_prog;
- goto out;
- }
+ if (!addrs)
+ return prog;
/*
* Before first pass, make a rough estimation of addrs[]
@@ -2574,7 +2558,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image = NULL;
if (header)
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_addrs;
}
if (image) {
@@ -2588,10 +2571,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (proglen == oldproglen) {
header = bpf_jit_binary_alloc(proglen, &image,
1, jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
}
oldproglen = proglen;
cond_resched();
@@ -2604,16 +2585,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image;
prog->jited = 1;
prog->jited_len = proglen;
- } else {
- prog = orig_prog;
}
out_addrs:
kfree(addrs);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
}
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index b129e0aaee20..512f7becc3dd 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -827,6 +827,7 @@ struct bpf_verifier_env {
u32 free_list_size;
u32 explored_states_size;
u32 num_backedges;
+ u32 blinded_insn_aux_len;
bpfptr_t fd_array;
/* bit mask to keep track of whether a register has been accessed
diff --git a/include/linux/filter.h b/include/linux/filter.h
index e40d4071a345..d396e55c9a1d 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1183,6 +1183,18 @@ static inline bool bpf_dump_raw_ok(const struct cred *cred)
struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
const struct bpf_insn *patch, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len);
+#else
+static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+#endif /* CONFIG_BPF_SYSCALL */
+
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
static inline bool xdp_return_frame_no_direct(void)
@@ -1309,9 +1321,14 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
const char *bpf_jit_get_prog_name(struct bpf_prog *prog);
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *fp);
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other);
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return prog->blinding_requested && !prog->blinded;
+}
+
static inline void bpf_jit_dump(unsigned int flen, unsigned int proglen,
u32 pass, void *image)
{
@@ -1450,6 +1467,20 @@ static inline void bpf_prog_kallsyms_del(struct bpf_prog *fp)
{
}
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return false;
+}
+
+static inline
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
+{
+ return prog;
+}
+
+static inline void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
+{
+}
#endif /* CONFIG_BPF_JIT */
void bpf_prog_kallsyms_del_all(struct bpf_prog *fp);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 1af5fb3f21d9..b458a85b3e18 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1506,7 +1506,10 @@ static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
#endif
}
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
+/* Now this function is used only to blind the main prog and must be invoked only when
+ * bpf_prog_need_blind() returns true.
+ */
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
struct bpf_prog *clone, *tmp;
@@ -1514,13 +1517,17 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
struct bpf_insn *insn;
int i, rewritten;
- if (!prog->blinding_requested || prog->blinded)
- return prog;
+ if (env)
+ prog = env->prog;
clone = bpf_prog_clone_create(prog, GFP_USER);
if (!clone)
return ERR_PTR(-ENOMEM);
+ /* make sure bpf_patch_insn_data() patches the correct prog */
+ if (env)
+ env->prog = clone;
+
insn_cnt = clone->len;
insn = clone->insnsi;
@@ -1548,21 +1555,37 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!rewritten)
continue;
- tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
- if (IS_ERR(tmp)) {
+ if (env)
+ tmp = bpf_patch_insn_data(env, i, insn_buff, rewritten);
+ else
+ tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
+
+ if (IS_ERR_OR_NULL(tmp)) {
+ if (env) {
+ /* record the actual length of the adjusted env->insn_aux_data */
+ env->blinded_insn_aux_len = clone->len;
+ /* restore the original prog */
+ env->prog = prog;
+ }
/* Patching may have repointed aux->prog during
* realloc from the original one, so we need to
* fix it up here on error.
*/
bpf_jit_prog_release_other(prog, clone);
- return tmp;
+ return IS_ERR(tmp) ? tmp : ERR_PTR(-ENOMEM);
}
clone = tmp;
insn_delta = rewritten - 1;
- /* Instructions arrays must be updated using absolute xlated offsets */
- adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+ if (env)
+ env->prog = clone;
+ else
+ /* Instructions arrays must be updated using absolute xlated offsets.
+ * The arrays have already been adjusted by bpf_patch_insn_data() when
+ * env is not NULL.
+ */
+ adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -1571,6 +1594,11 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
}
clone->blinded = 1;
+ /* also record the actual length of env->insn_aux_data on success, so it can be freed
+ * correctly when the parent function fails.
+ */
+ if (env)
+ env->blinded_insn_aux_len = clone->len;
return clone;
}
#endif /* CONFIG_BPF_JIT */
@@ -2531,6 +2559,35 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+{
+#ifdef CONFIG_BPF_JIT
+ bool blinded = false;
+ struct bpf_prog *orig_prog = prog;
+
+ if (bpf_prog_need_blind(orig_prog)) {
+ prog = bpf_jit_blind_constants(NULL, orig_prog);
+ /* If blinding was requested and we failed during blinding, we must fall
+ * back to the interpreter.
+ */
+ if (IS_ERR(prog))
+ return orig_prog;
+ blinded = true;
+ }
+
+ prog = bpf_int_jit_compile(prog);
+ if (blinded) {
+ if (!prog->jited) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ prog = orig_prog;
+ } else {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ }
+ }
+#endif
+ return prog;
+}
+
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
* @fp: bpf_prog populated with BPF program
@@ -2570,7 +2627,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_int_jit_compile(fp);
+ fp = bpf_prog_jit_compile(fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a431b7d50e1b..0d6bfccf0dd7 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -22215,8 +22215,8 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-static struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -22983,7 +22983,30 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
return 0;
}
-static int jit_subprogs(struct bpf_verifier_env *env)
+static u32 *dup_subprog_starts(struct bpf_verifier_env *env)
+{
+ u32 *starts = NULL;
+
+#ifndef CONFIG_BPF_JIT_ALWAYS_ON
+ starts = kvmalloc_objs(u32, env->subprog_cnt, GFP_KERNEL_ACCOUNT);
+ if (!starts)
+ return NULL;
+
+ for (int i = 0; i < env->subprog_cnt; i++)
+ starts[i] = env->subprog_info[i].start;
+#endif
+ return starts;
+}
+
+static void restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)
+{
+#ifndef CONFIG_BPF_JIT_ALWAYS_ON
+ for (int i = 0; i < env->subprog_cnt; i++)
+ env->subprog_info[i].start = orig_starts[i];
+#endif
+}
+
+static int __jit_subprogs(struct bpf_verifier_env *env)
{
struct bpf_prog *prog = env->prog, **func, *tmp;
int i, j, subprog_start, subprog_end = 0, len, subprog;
@@ -22991,10 +23014,6 @@ static int jit_subprogs(struct bpf_verifier_env *env)
struct bpf_insn *insn;
void *old_bpf_func;
int err, num_exentries;
- int old_len, subprog_start_adjustment = 0;
-
- if (env->subprog_cnt <= 1)
- return 0;
for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
if (!bpf_pseudo_func(insn) && !bpf_pseudo_call(insn))
@@ -23063,10 +23082,11 @@ static int jit_subprogs(struct bpf_verifier_env *env)
goto out_free;
func[i]->is_func = 1;
func[i]->sleepable = prog->sleepable;
+ func[i]->blinded = prog->blinded;
func[i]->aux->func_idx = i;
/* Below members will be freed only at prog->aux */
func[i]->aux->btf = prog->aux->btf;
- func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
+ func[i]->aux->subprog_start = subprog_start;
func[i]->aux->func_info = prog->aux->func_info;
func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -23122,15 +23142,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
-
- /*
- * To properly pass the absolute subprog start to jit
- * all instruction adjustments should be accumulated
- */
- old_len = func[i]->len;
func[i] = bpf_int_jit_compile(func[i]);
- subprog_start_adjustment += func[i]->len - old_len;
-
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -23256,16 +23268,77 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
kfree(func);
out_undo_insn:
+ bpf_prog_jit_attempt_done(prog);
+ return err;
+}
+
+static int jit_subprogs(struct bpf_verifier_env *env)
+{
+ int err, i;
+ bool blinded = false;
+ struct bpf_insn *insn;
+ struct bpf_prog *prog, *orig_prog;
+ u32 *orig_subprog_starts;
+
+ if (env->subprog_cnt <= 1)
+ return 0;
+
+ prog = orig_prog = env->prog;
+ if (bpf_prog_need_blind(orig_prog)) {
+ if (!IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON)) {
+ orig_subprog_starts = dup_subprog_starts(env);
+ if (!orig_subprog_starts) {
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ }
+ prog = bpf_jit_blind_constants(env, NULL);
+ if (IS_ERR(prog)) {
+ err = -ENOMEM;
+ goto out_restore;
+ }
+ blinded = true;
+ }
+
+ err = __jit_subprogs(env);
+ if (blinded) {
+ if (err) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ /* roll back to the clean original prog */
+ env->prog = orig_prog;
+ goto out_restore;
+ } else {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ if (!IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON))
+ kvfree(orig_subprog_starts);
+ }
+ } else if (err) {
+ /* We will fall back to interpreter mode when err is not -EFAULT, before
+ * that, insn->off and insn->imm should be restored to their original values
+ * since they were modified by __jit_subprogs.
+ */
+ if (err != EFAULT) {
+ for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
+ if (!bpf_pseudo_call(insn))
+ continue;
+ insn->off = 0;
+ insn->imm = env->insn_aux_data[i].call_imm;
+ }
+ }
+ goto out_cleanup;
+ }
+
+ return 0;
+
+out_restore:
+ if (!IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON)) {
+ restore_subprog_starts(env, orig_subprog_starts);
+ kvfree(orig_subprog_starts);
+ }
+out_cleanup:
/* cleanup main prog to be interpreted */
prog->jit_requested = 0;
prog->blinding_requested = 0;
- for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
- if (!bpf_pseudo_call(insn))
- continue;
- insn->off = 0;
- insn->imm = env->insn_aux_data[i].call_imm;
- }
- bpf_prog_jit_attempt_done(prog);
return err;
}
@@ -26456,7 +26529,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
err_unlock:
if (!is_priv)
mutex_unlock(&bpf_verifier_lock);
- clear_insn_aux_data(env, 0, env->prog->len);
+ clear_insn_aux_data(env, 0, env->blinded_insn_aux_len ? : env->prog->len);
vfree(env->insn_aux_data);
err_free_env:
bpf_stack_liveness_free(env);
--
2.43.0
^ permalink raw reply related
* [PATCH bpf-next v11 5/5] bpf, arm64: Emit BTI for indirect jump target
From: Xu Kuohai @ 2026-04-03 9:09 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260403090915.473493-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support BTI, the indirect jump selftest triggers a kernel
panic because there is no BTI instructions at the indirect jump targets.
Fix it by emitting a BTI instruction for each indirect jump target.
For reference, below is a sample panic log.
Internal error: Oops - BTI: 0000000036000003 [#1] SMP
...
Call trace:
bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x54/0xf8 (P)
bpf_prog_run_pin_on_cpu+0x140/0x468
bpf_prog_test_run_syscall+0x280/0x3b8
bpf_prog_test_run+0x22c/0x2c0
Fixes: f4a66cf1cb14 ("bpf: arm64: Add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 7212ec89dfe3..aa5cd240cdda 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1198,8 +1198,8 @@ static int add_exception_handler(const struct bpf_insn *insn,
* >0 - successfully JITed a 16-byte eBPF instruction.
* <0 - failed to JIT.
*/
-static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
- bool extra_pass)
+static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn *insn,
+ struct jit_ctx *ctx, bool extra_pass)
{
const u8 code = insn->code;
u8 dst = bpf2a64[insn->dst_reg];
@@ -1224,6 +1224,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
int ret;
bool sign_extend;
+ if (bpf_insn_is_indirect_target(env, ctx->prog, i))
+ emit_bti(A64_BTI_J, ctx);
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
@@ -1899,7 +1902,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
-static int build_body(struct jit_ctx *ctx, bool extra_pass)
+static int build_body(struct bpf_verifier_env *env, struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
int i;
@@ -1918,7 +1921,7 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
int ret;
ctx->offset[i] = ctx->idx;
- ret = build_insn(insn, ctx, extra_pass);
+ ret = build_insn(env, insn, ctx, extra_pass);
if (ret > 0) {
i++;
ctx->offset[i] = ctx->idx;
@@ -2079,7 +2082,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
if (build_prologue(&ctx, was_classic))
goto out_off;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_off;
ctx.epilogue_offset = ctx.idx;
@@ -2127,7 +2130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_free_hdr;
ctx.epilogue_offset = ctx.idx;
@@ -2136,7 +2139,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
ctx.write = true;
/* Pass 3: Adjust jump offset and write final image */
- if (build_body(&ctx, extra_pass) ||
+ if (build_body(env, &ctx, extra_pass) ||
WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
--
2.43.0
^ permalink raw reply related
* Re: [PATCH 6/6] ARM: dts: renesas: rskrza1: Drop superfluous cells
From: Geert Uytterhoeven @ 2026-04-03 9:24 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-7-marek.vasut+renesas@mailbox.org>
On Sat, 28 Mar 2026 at 00:43, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Drop superfluous address-cells and size-cells to fix DTC warning:
> "
> arch/arm/boot/dts/renesas/r7s72100-rskrza1.dts:32.17-72.4: Warning (avoid_unnecessary_addr_size): /flash@18000000: unnecessary #address-cells/#size-cells without "ranges", "dma-ranges" or child "reg" or "ranges" property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
My bad...
Fixes: 98537eb77d3ef185 ("ARM: dts: renesas: rskrza1: Add FLASH nodes")
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 5/6] ARM: dts: renesas: genmai: Drop superfluous cells
From: Geert Uytterhoeven @ 2026-04-03 9:23 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-6-marek.vasut+renesas@mailbox.org>
On Sat, 28 Mar 2026 at 00:43, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Drop superfluous address-cells and size-cells to fix DTC warning:
> "
> arch/arm/boot/dts/renesas/r7s72100-genmai.dts:28.17-55.4: Warning (avoid_unnecessary_addr_size): /flash@18000000: unnecessary #address-cells/#size-cells without "ranges", "dma-ranges" or child "reg" or "ranges" property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
My bad...
Fixes: 30e0a8cf886cb459 ("ARM: dts: renesas: genmai: Add FLASH nodes")
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 4/6] ARM: dts: renesas: r7s72100: Add missing unit to bus node
From: Geert Uytterhoeven @ 2026-04-03 9:22 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-5-marek.vasut+renesas@mailbox.org>
On Sat, 28 Mar 2026 at 00:43, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing unit to bus node to fix the following DTC warning:
> "
> arch/arm/boot/dts/renesas/r7s72100.dtsi:40.11-46.4: Warning (unit_address_vs_reg): /bus: node has a reg or ranges property, but no unit name
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 3/6] ARM: dts: renesas: r8a7792: Add missing unit to bus node
From: Geert Uytterhoeven @ 2026-04-03 9:22 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-4-marek.vasut+renesas@mailbox.org>
On Sat, 28 Mar 2026 at 00:43, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing unit to bus node to fix the following DTC warning:
> "
> arch/arm/boot/dts/renesas/r8a7792.dtsi:89.12-94.4: Warning (unit_address_vs_reg): /bus: node has a reg or ranges property, but no unit name
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 2/6] ARM: dts: renesas: r8a7779: Add missing unit to bus node
From: Geert Uytterhoeven @ 2026-04-03 9:22 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-3-marek.vasut+renesas@mailbox.org>
On Sat, 28 Mar 2026 at 00:43, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing unit to bus node to fix the following DTC warning:
> "
> arch/arm/boot/dts/renesas/r8a7779.dtsi:707.12-712.4: Warning (unit_address_vs_reg): /bus: node has a reg or ranges property, but no unit name
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 1/6] ARM: dts: renesas: r8a7778: Add missing unit to bus node
From: Geert Uytterhoeven @ 2026-04-03 9:21 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260327234244.91707-2-marek.vasut+renesas@mailbox.org>
Hi Marek,
On Sat, 28 Mar 2026 at 00:42, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing unit to bus node to fix the following DTC warning:
> "
> arch/arm/boot/dts/renesas/r8a7778.dtsi:43.12-48.4: Warning (unit_address_vs_reg): /bus: node has a reg or ranges property, but no unit name
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
> --- a/arch/arm/boot/dts/renesas/r8a7778.dtsi
> +++ b/arch/arm/boot/dts/renesas/r8a7778.dtsi
> @@ -40,7 +40,7 @@ aliases {
> spi2 = &hspi2;
> };
>
> - lbsc: bus {
> + lbsc: bus@0 {
Note for the future: if we ever add proper LBSC support (including its
own compatible value and reg property), this should be changed to the
address in the reg property. This applies to the first four patches.
> compatible = "simple-bus";
> #address-cells = <1>;
> #size-cells = <1>;
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH] arm64: pi: validate bootargs before parsing them
From: Will Deacon @ 2026-04-03 9:09 UTC (permalink / raw)
To: Pengpeng Hou; +Cc: Catalin Marinas, linux-arm-kernel, linux-kernel, robh, ardb
In-Reply-To: <20260403143004.4-arm64-pi-bootargs-pengpeng@iscas.ac.cn>
On Fri, Apr 03, 2026 at 11:56:05AM +0800, Pengpeng Hou wrote:
> get_bootargs_cmdline() fetches the raw bootargs property from the FDT
> and immediately calls strlen() on it before later passing the same
> pointer into the early command-line parser. Flat DT properties are
> external boot input, and this path does not prove that bootargs is
> NUL-terminated within its declared bounds.
>
> Use fdt_stringlist_get() so malformed unterminated bootargs are
> rejected before the local parser treats them as C strings.
>
> Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
> ---
> arch/arm64/kernel/pi/idreg-override.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c
> index bc57b290e5e7..310ed279ef26 100644
> --- a/arch/arm64/kernel/pi/idreg-override.c
> +++ b/arch/arm64/kernel/pi/idreg-override.c
> @@ -373,11 +373,11 @@ static __init const u8 *get_bootargs_cmdline(const void *fdt, int node)
> if (node < 0)
> return NULL;
>
> - prop = fdt_getprop(fdt, node, bootargs, NULL);
> + prop = fdt_stringlist_get(fdt, node, bootargs, 0, NULL);
> if (!prop)
> return NULL;
>
> - return strlen(prop) ? prop : NULL;
> + return *prop ? prop : NULL;
> }
I'm not exactly sure why we need to go out of our way to handle a
malformed DT at this stage, tbh. If it's corrupted in other ways (e.g.
random ASCII replacement) then we're probably still going to crash and
burn. There's nothing particularly special about NUL terminators.
If we _do_ decide that this is worth fixing, what about the other
early callers of fdt_getprop (kaslr, EFI stub)?
Will
^ permalink raw reply
* Re: [PATCH 4/4] arm64: dts: renesas: rzg2l-smarc: Fix missing cells and reg into DU subnode
From: Geert Uytterhoeven @ 2026-04-03 9:03 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260326042411.215241-5-marek.vasut+renesas@mailbox.org>
On Thu, 26 Mar 2026 at 05:24, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing cells and reg DT property into DU subnode to fix
> the following warning:
>
> "
> arch/arm64/boot/dts/renesas/rz-smarc-du-adv7513.dtsi:29.10-33.5: Warning (unit_address_vs_reg): /fragment@1/__overlay__/ports/port@0: node has a unit name, but no reg or ranges property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 3/4] arm64: dts: renesas: rzg2l-smarc: Fix missing cells and reg into CSI2 subnode
From: Geert Uytterhoeven @ 2026-04-03 9:03 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260326042411.215241-4-marek.vasut+renesas@mailbox.org>
On Thu, 26 Mar 2026 at 05:24, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing cells and reg DT property into CSI2 subnode to fix
> the following warning:
>
> "
> arch/arm64/boot/dts/renesas/rz-smarc-cru-csi-ov5645.dtsi:49.10-55.5: Warning (unit_address_vs_reg): /fragment@2/__overlay__/ports/port@0: node has a unit name, but no reg or ranges property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 2/4] arm64: dts: renesas: Fix missing cells and reg in Salvator-X panel DTO
From: Geert Uytterhoeven @ 2026-04-03 9:02 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260326042411.215241-3-marek.vasut+renesas@mailbox.org>
On Thu, 26 Mar 2026 at 05:24, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing cells and reg DT property into Salvator-X panel DTO to fix
> the following warning:
>
> "
> arch/arm64/boot/dts/renesas/salvator-panel-aa104xd12.dtso:30.10-34.5: Warning (unit_address_vs_reg): /fragment@2/__overlay__/ports/port@1: node has a unit name, but no reg or ranges property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 1/4] arm64: dts: renesas: Fix missing cells and reg in Draak/Ebisu panel DTO
From: Geert Uytterhoeven @ 2026-04-03 9:01 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-kernel, linux-renesas-soc
In-Reply-To: <20260326042411.215241-2-marek.vasut+renesas@mailbox.org>
On Thu, 26 Mar 2026 at 05:24, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Add missing cells and reg DT property into Draak/Ebisu panel DTO to fix
> the following warning:
>
> "
> arch/arm64/boot/dts/renesas/draak-ebisu-panel-aa104xd12.dtso:30.10-34.5: Warning (unit_address_vs_reg): /fragment@2/__overlay__/ports/port@1: node has a unit name, but no reg or ranges property
> "
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.2.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH v3] coresight: etm3x: Fix cntr_val_show() to match cntr_val_store() behavior
From: Kuan-Wei Chiu @ 2026-04-03 8:57 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: James Clark, mike.leach, alexander.shishkin, gregkh,
mathieu.poirier, leo.yan, Al.Grant, jserv, marscheng, ericchancf,
milesjiang, nickpan, coresight, linux-arm-kernel, linux-kernel,
stable
In-Reply-To: <bb521240-ab53-4c5a-aa1d-6b140ed4262e@arm.com>
Hi Suzuki,
On Mon, Feb 02, 2026 at 09:33:59AM +0000, Suzuki K Poulose wrote:
> Hello
>
> On 02/02/2026 05:09, Kuan-Wei Chiu wrote:
> > On Tue, Dec 02, 2025 at 09:26:19AM +0000, James Clark wrote:
> > >
> > >
> > > On 02/12/2025 8:26 am, Kuan-Wei Chiu wrote:
> > > > The cntr_val_show() function was intended to print the values of all
> > > > counters using a loop. However, due to a buffer overwrite issue with
> > > > sprintf(), it effectively only displayed the value of the last counter.
> > > >
> > > > The companion function, cntr_val_store(), allows users to modify a
> > > > specific counter selected by 'cntr_idx'. To maintain consistency
> > > > between read and write operations and to align with the ETM4x driver
> > > > behavior, modify cntr_val_show() to report only the value of the
> > > > currently selected counter.
> > > >
> > > > This change removes the loop and the "counter %d:" prefix, printing
> > > > only the hexadecimal value. It also adopts sysfs_emit() for standard
> > > > sysfs output formatting.
> > > >
> > > > Fixes: a939fc5a71ad ("coresight-etm: add CoreSight ETM/PTM driver")
> > > > Cc: stable@vger.kernel.org
> > > > Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
> > > > ---
> > > > Build test only.
> > > >
> > > > Changes in v3:
> > > > - Switch format specifier to %#x to include the 0x prefix.
> > > > - Add Cc stable
> > > >
> > > > v2: https://lore.kernel.org/lkml/20251201095228.1905489-1-visitorckw@gmail.com/
> > > >
> > > > .../hwtracing/coresight/coresight-etm3x-sysfs.c | 15 ++++-----------
> > > > 1 file changed, 4 insertions(+), 11 deletions(-)
> > > >
> > > > diff --git a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
> > > > index 762109307b86..b3c67e96a82a 100644
> > > > --- a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
> > > > +++ b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
> > > > @@ -717,26 +717,19 @@ static DEVICE_ATTR_RW(cntr_rld_event);
> > > > static ssize_t cntr_val_show(struct device *dev,
> > > > struct device_attribute *attr, char *buf)
> > > > {
> > > > - int i, ret = 0;
> > > > u32 val;
> > > > struct etm_drvdata *drvdata = dev_get_drvdata(dev->parent);
> > > > struct etm_config *config = &drvdata->config;
> > > > if (!coresight_get_mode(drvdata->csdev)) {
> > > > spin_lock(&drvdata->spinlock);
> > > > - for (i = 0; i < drvdata->nr_cntr; i++)
> > > > - ret += sprintf(buf, "counter %d: %x\n",
> > > > - i, config->cntr_val[i]);
> > > > + val = config->cntr_val[config->cntr_idx];
> > > > spin_unlock(&drvdata->spinlock);
> > > > - return ret;
> > > > - }
> > > > -
> > > > - for (i = 0; i < drvdata->nr_cntr; i++) {
> > > > - val = etm_readl(drvdata, ETMCNTVRn(i));
> > > > - ret += sprintf(buf, "counter %d: %x\n", i, val);
> > > > + } else {
> > > > + val = etm_readl(drvdata, ETMCNTVRn(config->cntr_idx));
> > > > }
> > > > - return ret;
> > > > + return sysfs_emit(buf, "%#x\n", val);
> > > > }
> > > > static ssize_t cntr_val_store(struct device *dev,
> > >
> > > Reviewed-by: James Clark <james.clark@linaro.org>
> > >
> > Thanks for the review!
> > Is there anything else I need to do for this fix to land?
>
> Thanks for the patch, I will queue this for the next release (v7.1).
>
Just a gentle ping.
Since the v7.1 merge window is presumably opening in about a week, I
noticed this patch isn't in linux-next yet and wanted to send a quick
reminder. Thanks.
Regards,
Kuan-Wei
^ permalink raw reply
* Re: [PATCH v5 23/27] clk: mediatek: Add MT8196 disp-ao clock support
From: Jason-JH Lin (林睿祥) @ 2026-04-03 8:54 UTC (permalink / raw)
To: Laura Nao
Cc: Guangjie Song (宋光杰), robh@kernel.org,
kernel@collabora.com, Sirius Wang (王皓昱),
Nancy Lin (林欣螢), AngeloGioacchino Del Regno,
linux-mediatek@lists.infradead.org, conor+dt@kernel.org,
Paul-pl Chen (陳柏霖),
linux-kernel@vger.kernel.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
richardcochran@gmail.com, krzk+dt@kernel.org,
mturquette@baylibre.com, Nicolas Prado, p.zabel@pengutronix.de,
Singo Chang (張興國), wenst@chromium.org,
linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org,
linux-clk@vger.kernel.org, matthias.bgg@gmail.com,
sboyd@kernel.org
In-Reply-To: <20260402100538.27291-1-laura.nao@collabora.com>
[snip]
> > > +static const struct of_device_id of_match_clk_mt8196_vdisp_ao[]
> > > = {
> > > + { .compatible = "mediatek,mt8196-vdisp-ao", .data =
> > > &mm_v_mcd },
> >
> > Hi Laura,
> >
> > We are going to send mtk-mmsys driver for MT8196 recently, but we
> > found
> > the compatible name is used here.
> >
> > As your commit message, vdisp-ao is integrated with the mtk-mmsys
> > driver, which registers the vdisp-ao clock driver via
> > platform_device_register_data().
> >
> > Shouldn't this compatible name belong to mmsys driver for MT8196?
> >
>
> That's right, my fault for missing that! Thanks for the heads up.
>
> I'm aware Angelo is currently restructuring mediatek-drm (including
> mmsys and mutex), and that might affect the way vdisp-ao is loaded
> too.
> So I'm not sure whether it makes sense to send a patch to fix this
> right away.
OK, we'll try to contact Angelo from other places.
Thanks for your confirmation!
Regards,
Jason-JH.Lin
>
> Best,
>
> Laura
>
^ permalink raw reply
* Re: [EXT] [PATCH v1 resend 0/7] Timer driver module support
From: Daniel Lezcano @ 2026-04-03 8:50 UTC (permalink / raw)
To: Zhipeng Wang, Daniel Lezcano, tglx@kernel.org
Cc: shawnguo@kernel.org, jstultz@google.com,
linux-kernel@vger.kernel.org, Matthias Brugger,
AngeloGioacchino Del Regno,
moderated list:ARM/Mediatek SoC support,
moderated list:ARM/Mediatek SoC support
In-Reply-To: <AS8PR04MB842096EB48B0B607711702EEEB5EA@AS8PR04MB8420.eurprd04.prod.outlook.com>
Hi Zhipeng,
On 4/3/26 09:59, Zhipeng Wang wrote:
>
[ ... ]
>> 2.43.0
> Hi Daniel,
>
> Apologies for the delayed response. Thank you for working on the timer driver module support series. I've
> tested your patches on i.MX platforms and they work well with the TPM
> timer driver.
>
> Based on your framework, I've converted the i.MX TPM timer driver to
> use TIMER_PDEV_DECLARE. The driver has been tested on:
> - i.MX7ULP (ARM, no alternative timer): built-in with CONFIG_EARLY_TIMER=y
> - i.MX8ULP (ARM64, has ARM Generic Timer): built-in, module, and
> CONFIG_EARLY_TIMER=y configurations
>
> All test cases passed without issues.
thanks for testing
I'll apply patch 2 and 3 and send a v2 for the next release and wait for
more comments.
-- Daniel
[ ... ]
^ permalink raw reply
* [PATCH 0/3] firmware: arm_scmi: Miscellaneous improvements
From: Geert Uytterhoeven @ 2026-04-03 8:41 UTC (permalink / raw)
To: Sudeep Holla, Cristian Marussi, Marek Vasut
Cc: arm-scmi, linux-arm-kernel, linux-renesas-soc, Geert Uytterhoeven
Hi all,
This patch series contains miscellaneous improvements for SCMI quirk
handling and SCMI instance iteration.
Thanks for your comments!
Geert Uytterhoeven (3):
firmware: arm_scmi: quirk: Improve quirk range parsing
firmware: arm_scmi: quirk: Simplify quirk table iteration
firmware: arm_scmi: Convert to list_for_each_entry()
drivers/firmware/arm_scmi/driver.c | 4 +---
drivers/firmware/arm_scmi/quirks.c | 15 ++++++---------
2 files changed, 7 insertions(+), 12 deletions(-)
--
2.43.0
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* [PATCH 2/3] firmware: arm_scmi: quirk: Simplify quirk table iteration
From: Geert Uytterhoeven @ 2026-04-03 8:41 UTC (permalink / raw)
To: Sudeep Holla, Cristian Marussi, Marek Vasut
Cc: arm-scmi, linux-arm-kernel, linux-renesas-soc, Geert Uytterhoeven
In-Reply-To: <cover.1775205358.git.geert+renesas@glider.be>
The current table entry is assigned in both the init and loop
expressions of the for-statement. Merge this into a single assignment
in the conditional expression, to simplify the code.
While at it, make the loop counter unsigned and loop-local.
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
drivers/firmware/arm_scmi/quirks.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/firmware/arm_scmi/quirks.c b/drivers/firmware/arm_scmi/quirks.c
index b1d9cd9fa2427879..2b38ba3f59a13c9e 100644
--- a/drivers/firmware/arm_scmi/quirks.c
+++ b/drivers/firmware/arm_scmi/quirks.c
@@ -258,10 +258,8 @@ static int scmi_quirk_range_parse(struct scmi_quirk *quirk)
void scmi_quirks_initialize(void)
{
struct scmi_quirk *quirk;
- int i;
- for (i = 0, quirk = scmi_quirks_table[0]; quirk;
- i++, quirk = scmi_quirks_table[i]) {
+ for (unsigned int i = 0; (quirk = scmi_quirks_table[i]); i++) {
int ret;
ret = scmi_quirk_range_parse(quirk);
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox