* Re: [PATCH 02/14] KVM: arm64: Track host-unmapped MMIO regions in a static array
From: Vincent Donnefort @ 2026-03-24 10:46 UTC (permalink / raw)
To: Sebastian Ene
Cc: alexandru.elisei, kvmarm, linux-arm-kernel, linux-kernel,
android-kvm, catalin.marinas, dbrazdil, joey.gouly, kees,
mark.rutland, maz, oupton, perlarsen, qperret, rananta, smostafa,
suzuki.poulose, tabba, tglx, bgrzesik, will, yuzenghui
In-Reply-To: <20260310124933.830025-3-sebastianene@google.com>
On Tue, Mar 10, 2026 at 12:49:21PM +0000, Sebastian Ene wrote:
> Introduce a registry to track protected MMIO regions that are unmapped
> from the host stage-2 page tables. These regions are stored in a
> fixed-size array and their ownership is donated to the hypervisor during
> initialization to ensure host-exclusion and persistent tracking.
>
> Signed-off-by: Sebastian Ene <sebastianene@google.com>
> ---
> arch/arm64/include/asm/kvm_pkvm.h | 10 ++++++++++
> arch/arm64/kvm/hyp/nvhe/mem_protect.c | 3 +++
> arch/arm64/kvm/hyp/nvhe/setup.c | 25 +++++++++++++++++++++++++
> 3 files changed, 38 insertions(+)
>
> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
> index 757076ad4ec9..48ec7d519399 100644
> --- a/arch/arm64/include/asm/kvm_pkvm.h
> +++ b/arch/arm64/include/asm/kvm_pkvm.h
> @@ -17,6 +17,16 @@
>
> #define HYP_MEMBLOCK_REGIONS 128
>
> +#define PKVM_PROTECTED_REGS_NUM 8
> +
> +struct pkvm_protected_reg {
> + u64 start_pfn;
> + size_t num_pages;
nit: "u64 pfn, u64 nr_pages" to align with everywhere else.
> +};
> +
> +extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[];
> +extern unsigned int kvm_nvhe_sym(num_protected_reg);
> +
> int pkvm_init_host_vm(struct kvm *kvm);
> int pkvm_create_hyp_vm(struct kvm *kvm);
> bool pkvm_hyp_vm_is_created(struct kvm *kvm);
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 0808367c52e5..7c125836b533 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -23,6 +23,9 @@
>
> struct host_mmu host_mmu;
>
> +struct pkvm_protected_reg pkvm_protected_regs[PKVM_PROTECTED_REGS_NUM];
> +unsigned int num_protected_reg;
> +
> static struct hyp_pool host_s2_pool;
>
> static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm);
> diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c
> index 90bd014e952f..ad5b96085e1b 100644
> --- a/arch/arm64/kvm/hyp/nvhe/setup.c
> +++ b/arch/arm64/kvm/hyp/nvhe/setup.c
> @@ -284,6 +284,27 @@ static int fix_hyp_pgtable_refcnt(void)
> &walker);
> }
>
> +static int unmap_protected_regions(void)
> +{
> + struct pkvm_protected_reg *reg;
> + int i, ret, j = 0;
> +
> + for (i = 0; i < num_protected_reg; i++) {
> + reg = &pkvm_protected_regs[i];
> + for (j = 0; j < reg->num_pages; j++) {
> + ret = __pkvm_host_donate_hyp_mmio(reg->start_pfn + j);
If this is to make this static at boot, we don't even need __pkvm_host_donate_hyp_mmio()
We can just map the region early enough in the hypervisor pkvm_create_mappings()
in recreate_hyp_mappings() and then let fix_host_ownership() do the host
stage2 unmapping.
> + if (ret)
> + goto err_setup;
> + }
> + }
> +
> + return 0;
> +err_setup:
> + for (j = j - 1; j >= 0; j--)
> + __pkvm_hyp_donate_host_mmio(reg->start_pfn + j);
> + return ret;
> +}
> +
> void __noreturn __pkvm_init_finalise(void)
> {
> struct kvm_cpu_context *host_ctxt = host_data_ptr(host_ctxt);
> @@ -324,6 +345,10 @@ void __noreturn __pkvm_init_finalise(void)
> if (ret)
> goto out;
>
> + ret = unmap_protected_regions();
> + if (ret)
> + goto out;
> +
> ret = hyp_ffa_init(ffa_proxy_pages);
> if (ret)
> goto out;
> --
> 2.53.0.473.g4a7958ca14-goog
>
^ permalink raw reply
* Re: [PATCH 1/2] arm64/entry: Fix involuntary preemption exception masking
From: Mark Rutland @ 2026-03-24 10:51 UTC (permalink / raw)
To: Jinjie Ruan
Cc: vladimir.murzin, peterz, catalin.marinas, linux-kernel, tglx,
luto, will, linux-arm-kernel
In-Reply-To: <c4e823cd-bd35-0e4b-d3fd-52688bfdf4ab@huawei.com>
On Tue, Mar 24, 2026 at 11:14:28AM +0800, Jinjie Ruan wrote:
> On 2026/3/20 19:30, Mark Rutland wrote:
> > On arm64, involuntary kernel preemption has been subtly broken since the
> > move to the generic irq entry code. When preemption occurs, the new task
> > may run with SError and Debug exceptions masked unexpectedly, leading to
> > a loss of RAS events, breakpoints, watchpoints, and single-step
> > exceptions.
>
> We can also add a check in arch_irqentry_exit_need_resched to prevent
> schedule-out when the DA bit is set.
That *might* be good enough for a backport to stable, but that's not the
right fix going forwards.
Checking the DAIF.DA bits will prevent preemption from anything other
than real interrupts, and Thomas said we should preempt wherever
possible:
https://lore.kernel.org/linux-arm-kernel/87h5qak2uv.ffs@tglx/
The interrupt handling path is also inconsistent w.r.t. masking DAIF
upon return, and with that cleaned up it'll look the same as other
exceptions, and we can't rely on DAIF.DA.
I think there are alternative options here; I have a half-written reply
to Thomas's other message:
https://lore.kernel.org/linux-arm-kernel/87fr5six4d.ffs@tglx/
... and I'll try to finish that up and get it out shortly.
Mark.
^ permalink raw reply
* Re: [PATCH v4 04/21] mm: avoid deadlock when holding rmap on mmap_prepare error
From: Vlastimil Babka (SUSE) @ 2026-03-24 10:55 UTC (permalink / raw)
To: Lorenzo Stoakes (Oracle), Andrew Morton
Cc: Jonathan Corbet, Clemens Ladisch, Arnd Bergmann,
Greg Kroah-Hartman, K . Y . Srinivasan, Haiyang Zhang, Wei Liu,
Dexuan Cui, Long Li, Alexander Shishkin, Maxime Coquelin,
Alexandre Torgue, Miquel Raynal, Richard Weinberger,
Vignesh Raghavendra, Bodo Stroesser, Martin K . Petersen,
David Howells, Marc Dionne, Alexander Viro, Christian Brauner,
Jan Kara, David Hildenbrand, Liam R . Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jann Horn, Pedro Falcato,
linux-kernel, linux-doc, linux-hyperv, linux-stm32,
linux-arm-kernel, linux-mtd, linux-staging, linux-scsi,
target-devel, linux-afs, linux-fsdevel, linux-mm, Ryan Roberts
In-Reply-To: <d44248be9da68258b07c2c59d4e73485ee0ca943.1774045440.git.ljs@kernel.org>
On 3/20/26 23:39, Lorenzo Stoakes (Oracle) wrote:
> Commit ac0a3fc9c07d ("mm: add ability to take further action in
> vm_area_desc") added the ability for drivers to instruct mm to take actions
> after the .mmap_prepare callback is complete.
>
> To make life simpler and safer, this is done before the VMA/mmap write lock
> is dropped but when the VMA is completely established.
>
> So on error, we simply munmap() the VMA.
>
> As part of this implementation, unfortunately a horrible hack had to be
> implemented to support some questionable behaviour hugetlb relies upon -
> that is that the file rmap lock is held until the operation is complete.
>
> The implementation, for convenience, did this in mmap_action_finish() so
> both the VMA and mmap_prepare compatibility layer paths would have this
> correctly handled.
>
> However, it turns out there is a mistake here - the rmap lock cannot be
> held on munmap, as free_pgtables() -> unlink_file_vma_batch_add() ->
> unlink_file_vma_batch_process() takes the file rmap lock.
>
> We therefore currently have a deadlock issue that might arise.
>
> Resolve this by leaving it to callers to handle the unmap.
>
> The compatibility layer does not support this rmap behaviour, so we simply
> have it unmap on error after calling mmap_action_complete().
>
> In the VMA implementation, we only perform the unmap after the rmap lock is
> dropped.
>
> This resolves the issue by ensuring the rmap lock is always dropped when
> the unmap occurs.
>
> Fixes: ac0a3fc9c07d ("mm: add ability to take further action in vm_area_desc")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH 03/14] KVM: arm64: Support host MMIO trap handlers for unmapped devices
From: Vincent Donnefort @ 2026-03-24 10:59 UTC (permalink / raw)
To: Sebastian Ene
Cc: alexandru.elisei, kvmarm, linux-arm-kernel, linux-kernel,
android-kvm, catalin.marinas, dbrazdil, joey.gouly, kees,
mark.rutland, maz, oupton, perlarsen, qperret, rananta, smostafa,
suzuki.poulose, tabba, tglx, bgrzesik, will, yuzenghui
In-Reply-To: <20260310124933.830025-4-sebastianene@google.com>
On Tue, Mar 10, 2026 at 12:49:22PM +0000, Sebastian Ene wrote:
> Introduce a mechanism to register callbacks for MMIO accesses to regions
> unmapped from the host Stage-2 page tables.
>
> This infrastructure allows the hypervisor to intercept host accesses to
> protected or emulated devices. When a Stage-2 fault occurs on a
> registered device region, the hypervisor will invoke the associated
> callback to emulate the access.
>
> Signed-off-by: Sebastian Ene <sebastianene@google.com>
> ---
> arch/arm64/include/asm/kvm_arm.h | 3 ++
> arch/arm64/include/asm/kvm_pkvm.h | 6 ++++
> arch/arm64/kvm/hyp/nvhe/mem_protect.c | 41 +++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/nvhe/setup.c | 3 ++
> 4 files changed, 53 insertions(+)
>
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 3f9233b5a130..8fe1e80ab3f4 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -304,6 +304,9 @@
>
> /* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
> #define HPFAR_MASK (~UL(0xf))
> +
> +#define FAR_MASK GENMASK_ULL(11, 0)
> +
> /*
> * We have
> * PAR [PA_Shift - 1 : 12] = PA [PA_Shift - 1 : 12]
> diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
> index 48ec7d519399..5321ced2f50a 100644
> --- a/arch/arm64/include/asm/kvm_pkvm.h
> +++ b/arch/arm64/include/asm/kvm_pkvm.h
> @@ -19,9 +19,15 @@
>
> #define PKVM_PROTECTED_REGS_NUM 8
>
> +struct pkvm_protected_reg;
> +
> +typedef void (pkvm_emulate_handler)(struct pkvm_protected_reg *region, u64 offset, bool write,
> + u64 *reg, u8 reg_size);
> +
> struct pkvm_protected_reg {
> u64 start_pfn;
> size_t num_pages;
> + pkvm_emulate_handler *cb;
> };
>
> extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[];
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 7c125836b533..f405d2fbd88f 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -13,6 +13,7 @@
> #include <asm/stage2_pgtable.h>
>
> #include <hyp/fault.h>
> +#include <hyp/adjust_pc.h>
>
> #include <nvhe/gfp.h>
> #include <nvhe/memory.h>
> @@ -608,6 +609,41 @@ static int host_stage2_idmap(u64 addr)
> return ret;
> }
>
> +static bool handle_host_mmio_trap(struct kvm_cpu_context *host_ctxt, u64 esr, u64 addr)
> +{
> + u64 offset, reg_value = 0, start, end;
> + u8 reg_size, reg_index;
> + bool write;
> + int i;
> +
> + for (i = 0; i < num_protected_reg; i++) {
This is potentially slow for a fast path. As this is an array, we could sort it
and do a binary search, just like find_mem_range?
> + start = pkvm_protected_regs[i].start_pfn << PAGE_SHIFT;
> + end = start + (pkvm_protected_regs[i].num_pages << PAGE_SHIFT);
> +
> + if (start > addr || addr > end)
> + continue;
> +
> + reg_size = BIT((esr & ESR_ELx_SAS) >> ESR_ELx_SAS_SHIFT);
> + reg_index = (esr & ESR_ELx_SRT_MASK) >> ESR_ELx_SRT_SHIFT;
> + write = (esr & ESR_ELx_WNR) == ESR_ELx_WNR;
> + offset = addr - start;
> +
> + if (write)
> + reg_value = host_ctxt->regs.regs[reg_index];
> +
> + pkvm_protected_regs[i].cb(&pkvm_protected_regs[i], offset, write,
> + ®_value, reg_size);
> +
> + if (!write)
> + host_ctxt->regs.regs[reg_index] = reg_value;
> +
> + kvm_skip_host_instr();
> + return true;
> + }
> +
> + return false;
> +}
> +
> void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt)
> {
> struct kvm_vcpu_fault_info fault;
> @@ -630,6 +666,11 @@ void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt)
> */
> BUG_ON(!(fault.hpfar_el2 & HPFAR_EL2_NS));
> addr = FIELD_GET(HPFAR_EL2_FIPA, fault.hpfar_el2) << 12;
> + addr |= fault.far_el2 & FAR_MASK;
> +
> + if (ESR_ELx_EC(esr) == ESR_ELx_EC_DABT_LOW && !addr_is_memory(addr) &&
> + handle_host_mmio_trap(host_ctxt, esr, addr))
> + return;
>
> ret = host_stage2_idmap(addr);
> BUG_ON(ret && ret != -EAGAIN);
> diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c
> index ad5b96085e1b..f91dfebe9980 100644
> --- a/arch/arm64/kvm/hyp/nvhe/setup.c
> +++ b/arch/arm64/kvm/hyp/nvhe/setup.c
> @@ -296,6 +296,9 @@ static int unmap_protected_regions(void)
> if (ret)
> goto err_setup;
> }
> +
> + if (reg->cb)
> + reg->cb = kern_hyp_va(reg->cb);
> }
>
> return 0;
> --
> 2.53.0.473.g4a7958ca14-goog
>
^ permalink raw reply
* [PATCH v2 1/2] arm64: dts: imx8qm-mek: switch Type-C connector power-role to dual
From: Xu Yang @ 2026-03-24 11:04 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, Frank.Li, s.hauer, kernel, festevam
Cc: devicetree, imx, linux-arm-kernel, linux-kernel, jun.li
When attach to PC Type-A port, the USB device controller does not function
at all. Because it is configured as source-only and a Type-A port doesn't
support PD capability, a data role swap is impossible.
Actually, PTN5110THQ is configured for Source role only at POR, but after
POR it can operate as a DRP (Dual-Role Power). By switching the power-role
to dual, the port can operate as a sink and enter device mode when attach
to Type-A port.
Since the board design uses EN_SRC to control the 5V VBUS path and EN_SNK
to control the 12V VBUS output, to avoid outputting a higher VBUS when in
sink role, we set the operation current limit to 0mA so that SW will not
control EN_SNK at all.
Signed-off-by: Xu Yang <xu.yang_2@nxp.com>
---
Changes in v2:
- improve commit message
- add comments for 0mA operational current
---
arch/arm64/boot/dts/freescale/imx8qm-mek.dts | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/freescale/imx8qm-mek.dts b/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
index dadc136aec6e..011a89d85961 100644
--- a/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
+++ b/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
@@ -611,9 +611,17 @@ ptn5110: tcpc@51 {
usb_con1: connector {
compatible = "usb-c-connector";
label = "USB-C";
- power-role = "source";
+ power-role = "dual";
data-role = "dual";
+ try-power-role = "sink";
source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ /*
+ * Set operational current to 0mA as we don't want EN_SNK
+ * enable 12V VBUS switch when it work as a sink.
+ */
+ sink-pdos = <PDO_FIXED(5000, 0, PDO_FIXED_USB_COMM)>;
+ op-sink-microwatt = <0>;
+ self-powered;
ports {
#address-cells = <1>;
--
2.34.1
^ permalink raw reply related
* [PATCH v2 2/2] arm64: dts: imx8qxp-mek: switch Type-C connector power-role to dual
From: Xu Yang @ 2026-03-24 11:04 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, Frank.Li, s.hauer, kernel, festevam
Cc: devicetree, imx, linux-arm-kernel, linux-kernel, jun.li
In-Reply-To: <20260324110459.2838767-1-xu.yang_2@nxp.com>
When attach to PC Type-A port, the USB device controller does not function
at all. Because it is configured as source-only and a Type-A port doesn't
support PD capability, a data role swap is impossible.
Actually, PTN5110THQ is configured for Source role only at POR, but after
POR it can operate as a DRP (Dual-Role Power). By switching the power-role
to dual, the port can operate as a sink and enter device mode when attach
to Type-A port.
Since the board design uses EN_SRC to control the 5V VBUS path and EN_SNK
to control the 12V VBUS output, to avoid outputting a higher VBUS when in
sink role, we set the operation current limit to 0mA so that SW will not
control EN_SNK at all.
Signed-off-by: Xu Yang <xu.yang_2@nxp.com>
---
Changes in v2:
- improve commit message
- add comments for 0mA operational current
---
arch/arm64/boot/dts/freescale/imx8qxp-mek.dts | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
index 40a0bc9f4e84..623169f7ddb5 100644
--- a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
+++ b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
@@ -566,9 +566,17 @@ ptn5110: tcpc@50 {
usb_con1: connector {
compatible = "usb-c-connector";
label = "USB-C";
- power-role = "source";
+ power-role = "dual";
data-role = "dual";
+ try-power-role = "sink";
source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ /*
+ * Set operational current to 0mA as we don't want EN_SNK
+ * enable 12V VBUS switch when it work as a sink.
+ */
+ sink-pdos = <PDO_FIXED(5000, 0, PDO_FIXED_USB_COMM)>;
+ op-sink-microwatt = <0>;
+ self-powered;
ports {
#address-cells = <1>;
--
2.34.1
^ permalink raw reply related
* Re: [PATCH v7 1/5] clk: bcm: rpi: Manage clock rate in prepare/unprepare callbacks
From: Maíra Canal @ 2026-03-24 11:09 UTC (permalink / raw)
To: Stephen Boyd, Chema Casanova, Dave Stevenson, Florian Fainelli,
Iago Toral Quiroga, Maxime Ripard, Melissa Wen, Michael Turquette,
Nicolas Saenz Julienne, Philipp Zabel, Stefan Wahren
Cc: linux-clk, dri-devel, linux-rpi-kernel, linux-arm-kernel,
Broadcom internal kernel review list, kernel-dev
In-Reply-To: <177431178481.5403.10860855444629990495@lazor>
Hi Stephen,
On 23/03/26 21:23, Stephen Boyd wrote:
> Quoting Maíra Canal (2026-03-12 14:34:23)
>> On current firmware versions, RPI_FIRMWARE_SET_CLOCK_STATE doesn't
>> actually power off the clock. To achieve meaningful power savings, the
>> clock rate must be set to the minimum before disabling. This might be
>> fixed in future firmware releases.
>>
>> Rather than pushing rate management to clock consumers, handle it
>> directly in the clock framework's prepare/unprepare callbacks. In
>> unprepare, set the rate to the minimum before disabling the clock.
>> In prepare, for clocks marked with `maximize` (currently v3d),
>> restore the rate to the maximum after enabling.
>>
>> Signed-off-by: Maíra Canal <mcanal@igalia.com>
>> ---
>
> Acked-by: Stephen Boyd <sboyd@kernel.org>
When possible, could you apply this patch to clk-next? Thank you and
Maxime for the reviews!
Best regards,
- Maíra
^ permalink raw reply
* [PATCH AUTOSEL 6.19-6.6] arm64/scs: Fix handling of advance_loc4
From: Sasha Levin @ 2026-03-24 11:19 UTC (permalink / raw)
To: patches, stable
Cc: Pepper Gray, Will Deacon, Sasha Levin, catalin.marinas,
linux-arm-kernel, linux-kernel
In-Reply-To: <20260324111931.3257972-1-sashal@kernel.org>
From: Pepper Gray <hello@peppergray.xyz>
[ Upstream commit d499e9627d70b1269020d59b95ed3e18bee6b8cd ]
DW_CFA_advance_loc4 is defined but no handler is implemented. Its
CFA opcode defaults to EDYNSCS_INVALID_CFA_OPCODE triggering an
error which wrongfully prevents modules from loading.
Link: https://bugs.gentoo.org/971060
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
The context around the insertion point is identical in v6.6 and v6.12.
The fix adds a new `case DW_CFA_advance_loc4:` between `advance_loc2`
and `DW_CFA_def_cfa`. This should apply cleanly to both stable trees.
## PHASE 3-7 SYNTHESIS
Let me now compile all findings:
### Step 3.1: BLAME THE CHANGED LINES
The switch statement was introduced in commit `3b619e22c4601b` by Ard
Biesheuvel in v6.2-rc1. The `DW_CFA_advance_loc4` constant was defined
but never given a case handler — the bug has existed since the code was
first introduced.
### Step 3.2: FIXES TAG
No Fixes: tag present. The implicit fix target is `3b619e22c4601b`
("arm64: implement dynamic shadow call stack for Clang") from v6.2.
### Step 3.3-3.5: FILE HISTORY AND DEPENDENCIES
The file has been modified 7 times total. The fix is self-contained — it
adds a new case to an existing switch statement following the exact
pattern of `DW_CFA_advance_loc1` and `DW_CFA_advance_loc2`. No
dependencies on other patches.
### Step 5: CODE SEMANTIC ANALYSIS
- `scs_handle_fde_frame()` is called from `scs_patch()` which is called
from:
1. `map_kernel.c` — during early boot (vmlinux SCS patching)
2. `module.c` — during module loading
- The amdgpu driver generates `DW_CFA_advance_loc4` opcodes (likely due
to very large functions), triggering the bug on module load.
### Step 7: SUBSYSTEM AND CRITICALITY
- **Subsystem:** arm64/scs — Shadow Call Stack security feature
- **Criticality:** IMPORTANT — affects arm64 platforms with SCS enabled
(hardened kernels, Android)
### Step 8: IMPACT AND RISK ASSESSMENT
**Who is affected:** arm64 users with CONFIG_SHADOW_CALL_STACK=y and
CONFIG_DYNAMIC_SCS=y loading modules with large functions (e.g.,
amdgpu).
**Trigger:** Loading any kernel module whose compiled code generates
`DW_CFA_advance_loc4` DWARF opcodes (functions spanning >64KB of
instructions).
**Failure mode in stable (6.6.y, 6.12.y):** SCS patching silently fails
— the error return is not checked, so the module loads but without
proper Shadow Call Stack protection. This is a **security degradation**
— SCS is designed to protect against Return-Oriented Programming
attacks.
**Failure mode in mainline (v6.18+):** Module loading fails entirely
(due to `6d4a0fbd34a40`). The Gentoo bug report confirms amdgpu fails to
load on ARM64 hardened kernels.
**Fix quality:**
- 8 lines added, following the exact pattern of `advance_loc1` (1 byte)
and `advance_loc2` (2 bytes) but for 4 bytes
- Obviously correct — it reads 4 bytes and advances the location counter
- Signed off by Will Deacon (arm64 maintainer)
- Minimal, surgical, no side effects
- One minor style nit: `break` is outdented compared to the other cases,
but functionally correct
## PHASE 9: FINAL SYNTHESIS
### Evidence FOR backporting:
1. Fixes a real bug that prevents module loading on arm64 (confirmed by
Gentoo bug report with amdgpu)
2. In stable trees, the bug silently disables Shadow Call Stack security
protection for affected modules
3. The buggy code has been present since v6.2 (affects 6.6.y, 6.12.y
stable trees)
4. Fix is small (8 lines), obviously correct, follows the exact pattern
of adjacent code
5. Signed off by Will Deacon (arm64 maintainer)
6. Link to real user bug report (Gentoo #971060) — actual users hit this
7. Self-contained — no dependencies on other patches
8. Should apply cleanly to stable (same code context exists in 6.6 and
6.12)
### Evidence AGAINST backporting:
- None significant. The only minor concern is that in current stable
trees the error is silently ignored (module still loads), so the
immediate user-visible impact is lower (security degradation rather
than module load failure). But this is still a bug worth fixing.
### Stable Rules Checklist:
1. **Obviously correct and tested?** YES — follows the pattern of
loc1/loc2, tested by Gentoo users
2. **Fixes a real bug?** YES — prevents module loading (mainline) or
silently breaks SCS (stable)
3. **Important issue?** YES — security feature bypass on hardened arm64
kernels
4. **Small and contained?** YES — 8 lines in one file
5. **No new features or APIs?** YES — just adds missing case handler
6. **Can apply to stable trees?** YES — context is identical in 6.6 and
6.12
## Verification
- [Phase 1] Parsed subject: arm64/scs subsystem, "Fix" action verb,
missing advance_loc4 handler
- [Phase 1] Parsed tags: Link to bugs.gentoo.org/971060, Signed-off-by
Will Deacon (arm64 maintainer)
- [Phase 2] Diff analysis: +8 lines in single file, adds
DW_CFA_advance_loc4 case to existing switch
- [Phase 2] Pattern follows DW_CFA_advance_loc1 (1 byte) and
DW_CFA_advance_loc2 (2 bytes) exactly
- [Phase 3] git blame: switch statement introduced in 3b619e22c4601b
(v6.2-rc1), bug present since then
- [Phase 3] git show v6.1: file does not exist — bug only affects 6.2+
- [Phase 3] git show v6.6, v6.12: confirmed DW_CFA_advance_loc4 defined
but no case handler in both
- [Phase 3] git show 6d4a0fbd34a40: confirmed this commit (v6.18) made
module loading actually fail on SCS errors
- [Phase 3] v6.6/v6.12 module.c: SCS patch error return is NOT checked —
module loads with broken SCS
- [Phase 4] WebFetch bugs.gentoo.org/971060: confirmed amdgpu module
fails to load on ARM64 hardened kernel
- [Phase 4] WebFetch lore.kernel.org: found patch discussion, accepted
by Will Deacon, pulled in arm64 fixes
- [Phase 5] scs_handle_fde_frame called from scs_patch, which is called
from module.c and map_kernel.c
- [Phase 6] Confirmed context around insertion point is identical in
v6.6 and v6.12 — clean apply expected
- [Phase 7] Subsystem: arm64/scs, IMPORTANT criticality (security
feature for arm64)
- [Phase 8] Failure mode: security degradation (stable) or module load
failure (mainline), severity HIGH
**YES**
arch/arm64/kernel/pi/patch-scs.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/arch/arm64/kernel/pi/patch-scs.c b/arch/arm64/kernel/pi/patch-scs.c
index bbe7d30ed12b3..dac568e4a54f2 100644
--- a/arch/arm64/kernel/pi/patch-scs.c
+++ b/arch/arm64/kernel/pi/patch-scs.c
@@ -192,6 +192,14 @@ static int scs_handle_fde_frame(const struct eh_frame *frame,
size -= 2;
break;
+ case DW_CFA_advance_loc4:
+ loc += *opcode++ * code_alignment_factor;
+ loc += (*opcode++ << 8) * code_alignment_factor;
+ loc += (*opcode++ << 16) * code_alignment_factor;
+ loc += (*opcode++ << 24) * code_alignment_factor;
+ size -= 4;
+ break;
+
case DW_CFA_def_cfa:
case DW_CFA_offset_extended:
size = skip_xleb128(&opcode, size);
--
2.51.0
^ permalink raw reply related
* Re: [PATCH net-next 0/6] net: stmmac: cleanup stmmac_xmit()
From: patchwork-bot+netdevbpf @ 2026-03-24 11:20 UTC (permalink / raw)
To: Russell King
Cc: andrew, alexandre.torgue, andrew+netdev, davem, edumazet, kuba,
linux-arm-kernel, linux-stm32, netdev, pabeni
In-Reply-To: <ab15_JvLGFtUH_3x@shell.armlinux.org.uk>
Hello:
This series was applied to netdev/net-next.git (main)
by Paolo Abeni <pabeni@redhat.com>:
On Fri, 20 Mar 2026 16:46:52 +0000 you wrote:
> This series continues on from part 2 of the descriptor cleanups, making
> stmmac_xmit() more readable.
>
> drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 103 +++++++++++-----------
> 1 file changed, 52 insertions(+), 51 deletions(-)
Here is the summary with links:
- [net-next,1/6] net: stmmac: move stmmac_xmit() skb head handling
https://git.kernel.org/netdev/net-next/c/fcbf106189aa
- [net-next,2/6] net: stmmac: move first xmit descriptor SARC and TBS config
https://git.kernel.org/netdev/net-next/c/cafacdc48a74
- [net-next,3/6] net: stmmac: move stmmac_xmit() first entry index code
https://git.kernel.org/netdev/net-next/c/7b8683115034
- [net-next,4/6] net: stmmac: move stmmac_xmit() initial variable init
https://git.kernel.org/netdev/net-next/c/2e3bfeb1bc0c
- [net-next,5/6] net: stmmac: use first_desc for TBS
https://git.kernel.org/netdev/net-next/c/557ccd54ba14
- [net-next,6/6] net: stmmac: elminate tbs_desc in stmmac_xmit()
https://git.kernel.org/netdev/net-next/c/cd1f306e2815
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply
* Re: [PATCH v2 1/2] dt-bindings: watchdog: Convert TS-4800 to DT schema
From: Guenter Roeck @ 2026-03-24 11:36 UTC (permalink / raw)
To: Eduard Bostina, daniel.baluta, simona.toaca, d-gole, m-chawdhry,
Wim Van Sebroeck, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Frank Li, Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Mark Brown, linux-watchdog, devicetree, linux-kernel, imx,
linux-arm-kernel
In-Reply-To: <20260323175948.302441-2-egbostina@gmail.com>
On 3/23/26 10:59, Eduard Bostina wrote:
> Convert the Technologic Systems TS-4800 watchdog timer bindings
> to DT schema.
>
> Signed-off-by: Eduard Bostina <egbostina@gmail.com>
Reviewed-by: Guenter Roeck <linux@roeck-us.net>
^ permalink raw reply
* Re: [PATCH v4 2/2] arm64: dts: cix: Add scmi powerdomain nodes for sky1
From: Peter Chen @ 2026-03-24 11:45 UTC (permalink / raw)
To: Gary Yang
Cc: lpieralisi, kwilczynski, mani, robh, bhelgaas, krzk+dt, conor+dt,
linux-pci, devicetree, linux-kernel, linux-arm-kernel,
cix-kernel-upstream
In-Reply-To: <20260313114914.1564115-3-gary.yang@cixtech.com>
On 26-03-13 19:49:14, Gary Yang wrote:
> Add a second SCMI channel using SMC transport to communicate with TF-A
> for power domain management on the Sky1 SoC.
>
> Signed-off-by: Gary Yang <gary.yang@cixtech.com>
Applied, thanks.
Peter
> ---
> arch/arm64/boot/dts/cix/sky1-power.h | 33 ++++++++++++++++++++++++++++
> arch/arm64/boot/dts/cix/sky1.dtsi | 21 ++++++++++++++++++
> 2 files changed, 54 insertions(+)
> create mode 100644 arch/arm64/boot/dts/cix/sky1-power.h
>
> diff --git a/arch/arm64/boot/dts/cix/sky1-power.h b/arch/arm64/boot/dts/cix/sky1-power.h
> new file mode 100644
> index 000000000000..53f4a3af36b3
> --- /dev/null
> +++ b/arch/arm64/boot/dts/cix/sky1-power.h
> @@ -0,0 +1,33 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright 2026 Cix Technology Group Co., Ltd.
> + */
> +
> +#ifndef __SKY1_POWER_H__
> +#define __SKY1_POWER_H__
> +
> +/* The Rich OS need flow the macro */
> +#define SKY1_PD_AUDIO 0
> +#define SKY1_PD_PCIE_CTRL0 1
> +#define SKY1_PD_PCIE_DUMMY 2
> +#define SKY1_PD_PCIEHUB 3
> +#define SKY1_PD_MMHUB 4
> +#define SKY1_PD_MMHUB_SMMU 5
> +#define SKY1_PD_DPU0 6
> +#define SKY1_PD_DPU1 7
> +#define SKY1_PD_DPU2 8
> +#define SKY1_PD_DPU3 9
> +#define SKY1_PD_DPU4 10
> +#define SKY1_PD_VPU_TOP 11
> +#define SKY1_PD_VPU_CORE0 12
> +#define SKY1_PD_VPU_CORE1 13
> +#define SKY1_PD_VPU_CORE2 14
> +#define SKY1_PD_VPU_CORE3 15
> +#define SKY1_PD_NPU_CORE0 16
> +#define SKY1_PD_NPU_CORE1 17
> +#define SKY1_PD_NPU_CORE2 18
> +#define SKY1_PD_NPU_TOP 19
> +#define SKY1_PD_ISP0 20
> +#define SKY1_PD_GPU 21
> +
> +#endif
> diff --git a/arch/arm64/boot/dts/cix/sky1.dtsi b/arch/arm64/boot/dts/cix/sky1.dtsi
> index 64b76905cbff..495ea91a63f5 100644
> --- a/arch/arm64/boot/dts/cix/sky1.dtsi
> +++ b/arch/arm64/boot/dts/cix/sky1.dtsi
> @@ -6,6 +6,7 @@
>
> #include <dt-bindings/interrupt-controller/arm-gic.h>
> #include <dt-bindings/clock/cix,sky1.h>
> +#include "sky1-power.h"
>
> / {
> interrupt-parent = <&gic>;
> @@ -168,6 +169,19 @@ scmi_clk: protocol@14 {
> #clock-cells = <1>;
> };
> };
> +
> + ap_to_tfa_scmi: scmi-1 {
> + compatible = "arm,scmi-smc";
> + arm,smc-id = <0xc2000001>;
> + #address-cells = <1>;
> + #size-cells = <0>;
> + shmem = <&ap_tfa_scmi_mem>;
> +
> + smc_devpd: protocol@11 {
> + reg = <0x11>;
> + #power-domain-cells = <1>;
> + };
> + };
> };
>
> pmu-a520 {
> @@ -428,6 +442,7 @@ pcie_x8_rc: pcie@a010000 {
> #size-cells = <2>;
> bus-range = <0xc0 0xff>;
> device_type = "pci";
> + power-domains = <&smc_devpd SKY1_PD_PCIE_CTRL0>;
> #interrupt-cells = <1>;
> interrupt-map-mask = <0 0 0 0x7>;
> interrupt-map = <0 0 0 1 &gic 0 0 GIC_SPI 407 IRQ_TYPE_LEVEL_HIGH 0>,
> @@ -572,6 +587,12 @@ iomuxc_s5: pinctrl@16007000 {
> compatible = "cix,sky1-pinctrl-s5";
> reg = <0x0 0x16007000 0x0 0x1000>;
> };
> +
> + ap_tfa_scmi_mem: shmem@84380000 {
> + compatible = "arm,scmi-shmem";
> + reg = <0x0 0x84380000 0x0 0x80>;
> + reg-io-width = <4>;
> + };
> };
>
> timer {
> --
> 2.49.0
>
--
Best regards,
Peter
^ permalink raw reply
* Re: [PATCH v2 1/1] arm64: dts: cix: add FCH(S0)/S5 GPIO controllers for sky1
From: Peter Chen @ 2026-03-24 11:47 UTC (permalink / raw)
To: Zichar Zhang
Cc: robh, krzk+dt, conor+dt, fugang.duan, linusw, cix-kernel-upstream,
devicetree, linux-kernel, linux-arm-kernel
In-Reply-To: <20260312094923.3473444-2-zichar.zhang@cixtech.com>
On 26-03-12 17:49:23, Zichar Zhang wrote:
> Add Cadence GPIO controller nodes for Sky1 FCH(S0) and S5 domains in
> sky1.dtsi, and enable those controllers on sky1-orion-o6.
>
> Signed-off-by: Zichar Zhang <zichar.zhang@cixtech.com>
Applied, thanks.
Peter
> ---
> arch/arm64/boot/dts/cix/sky1-orion-o6.dts | 28 +++++
> arch/arm64/boot/dts/cix/sky1.dtsi | 119 ++++++++++++++++++++++
> 2 files changed, 147 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/cix/sky1-orion-o6.dts b/arch/arm64/boot/dts/cix/sky1-orion-o6.dts
> index 4dee8cd0b86d..e39c87774c12 100644
> --- a/arch/arm64/boot/dts/cix/sky1-orion-o6.dts
> +++ b/arch/arm64/boot/dts/cix/sky1-orion-o6.dts
> @@ -36,6 +36,22 @@ linux,cma {
>
> };
>
> +&fch_gpio0 {
> + status = "okay";
> +};
> +
> +&fch_gpio1 {
> + status = "okay";
> +};
> +
> +&fch_gpio2 {
> + status = "okay";
> +};
> +
> +&fch_gpio3 {
> + status = "okay";
> +};
> +
> &iomuxc {
> pinctrl-names = "default";
> pinctrl-0 = <&pinctrl_hog>;
> @@ -86,6 +102,18 @@ &pcie_x1_1_rc {
> status = "okay";
> };
>
> +&s5_gpio0 {
> + status = "okay";
> +};
> +
> +&s5_gpio1 {
> + status = "okay";
> +};
> +
> +&s5_gpio2 {
> + status = "okay";
> +};
> +
> &uart2 {
> status = "okay";
> };
> diff --git a/arch/arm64/boot/dts/cix/sky1.dtsi b/arch/arm64/boot/dts/cix/sky1.dtsi
> index 72f3b195a927..3772548e8c0b 100644
> --- a/arch/arm64/boot/dts/cix/sky1.dtsi
> +++ b/arch/arm64/boot/dts/cix/sky1.dtsi
> @@ -185,6 +185,13 @@ psci {
> method = "smc";
> };
>
> + s5_gpio_apb_clk: clock-100000000 {
> + compatible = "fixed-clock";
> + #clock-cells = <0>;
> + clock-frequency = <100000000>;
> + clock-output-names = "s5_gpio_apb_clk";
> + };
> +
> soc@0 {
> compatible = "simple-bus";
> ranges = <0 0 0 0 0x20 0>;
> @@ -348,6 +355,70 @@ i3c1: i3c@4100000 {
> status = "disabled";
> };
>
> + fch_gpio0: gpio-controller@4120000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x4120000 0x0 0x1000>;
> + clocks = <&scmi_clk CLK_TREE_FCH_GPIO_APB>;
> +
> + interrupts = <GIC_SPI 304 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <32>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> + fch_gpio1: gpio-controller@4130000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x4130000 0x0 0x1000>;
> + clocks = <&scmi_clk CLK_TREE_FCH_GPIO_APB>;
> +
> + interrupts = <GIC_SPI 305 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <32>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> + fch_gpio2: gpio-controller@4140000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x4140000 0x0 0x1000>;
> + clocks = <&scmi_clk CLK_TREE_FCH_GPIO_APB>;
> +
> + interrupts = <GIC_SPI 306 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <32>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> + fch_gpio3: gpio-controller@4150000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x4150000 0x0 0x1000>;
> + clocks = <&scmi_clk CLK_TREE_FCH_GPIO_APB>;
> +
> + interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <17>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> syscon: syscon@4160000 {
> compatible = "cix,sky1-system-control", "syscon";
> reg = <0x0 0x4160000 0x0 0x100>;
> @@ -587,6 +658,54 @@ s5_syscon: syscon@16000000 {
> #reset-cells = <1>;
> };
>
> + s5_gpio0: gpio-controller@16004000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x16004000 0x0 0x1000>;
> + clocks = <&s5_gpio_apb_clk>;
> +
> + interrupts = <GIC_SPI 372 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <32>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> + s5_gpio1: gpio-controller@16005000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x16005000 0x0 0x1000>;
> + clocks = <&s5_gpio_apb_clk>;
> +
> + interrupts = <GIC_SPI 373 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <10>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> + s5_gpio2: gpio-controller@16006000 {
> + compatible = "cdns,gpio-r1p02";
> + reg = <0x0 0x16006000 0x0 0x1000>;
> + clocks = <&s5_gpio_apb_clk>;
> +
> + interrupts = <GIC_SPI 374 IRQ_TYPE_LEVEL_HIGH 0>;
> +
> + gpio-controller;
> + #gpio-cells = <2>;
> + ngpios = <10>;
> +
> + interrupt-controller;
> + #interrupt-cells = <2>;
> + status = "disabled";
> + };
> +
> iomuxc_s5: pinctrl@16007000 {
> compatible = "cix,sky1-pinctrl-s5";
> reg = <0x0 0x16007000 0x0 0x1000>;
> --
> 2.34.1
>
--
Best regards,
Peter
^ permalink raw reply
* [PATCH v5 0/5] drm/msm: add RGB101010 pixel format
From: Alexander Koskovich @ 2026-03-24 11:47 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich, Dmitry Baryshkov,
Konrad Dybcio
This series adds support for the RGB101010 (30bpp) pixel format used by some
newer panels.
Tested on the BOE BF068MWM-TD0 panel (10 bit DSC) on the Nothing Phone (3a).
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
Changes in v5:
- Add comment & warning about RGB101010 support in compression_en && !wide_bus_en case
- Move RGB101010 check further down so it has access to msm_dsi_host_is_wide_bus_enabled()
- Use default case for meson_dw_mipi_dsi pixel format switchs
- Link to v4: https://lore.kernel.org/r/20260321-dsi-rgb101010-support-v4-0-eb28ecebbfe8@pm.me
Changes in v4:
- Dropped dpu_encoder_phys_vid patch
- Needs further discussion and rest of these changes can be independent of that
- Pick back up dropped drm_mipi_dsi.h patch (oops)
- Fix meson driver warning from adding RGB101010 to drm_mipi_dsi.h
- msm_dsi_host_version_ge -> msm_dsi_host_version_geq
- Fix msm_dsi_host_version_geq to handle major > major
- Link to v3: https://lore.kernel.org/r/20260319-dsi-rgb101010-support-v3-0-85b99df2d090@pm.me
Changes in v3:
- Fix typo for MSM8998 DSI version name (V2_2 -> V_2_0)
- Add msm_dsi_host_version_is_gt per Konrad and use for RGB101010 check
- Fix up comment & commit message for video mode DSC INTF timing width change per Neil/Konrad
- Link to v2: https://lore.kernel.org/r/20260318-dsi-rgb101010-support-v2-0-698b7612eaeb@pm.me
Changes in v2:
- Only allow RGB101010 if MSM_DSI_6G_VER >= V2.1.0
- Link to v1: https://lore.kernel.org/r/20260318-dsi-rgb101010-support-v1-0-6021eb79e796@pm.me
---
Alexander Koskovich (5):
drm/mipi-dsi: add RGB101010 pixel format
drm/meson: use default case for unsupported DSI pixel formats
drm/msm/dsi: rename MSM8998 DSI version from V2_2_0 to V2_0_0
drm/msm/dsi: add DSI version >= comparison helper
drm/msm/dsi: Add support for RGB101010 pixel format
drivers/gpu/drm/meson/meson_dw_mipi_dsi.c | 6 ++---
drivers/gpu/drm/msm/dsi/dsi_cfg.c | 4 ++--
drivers/gpu/drm/msm/dsi/dsi_cfg.h | 2 +-
drivers/gpu/drm/msm/dsi/dsi_host.c | 34 +++++++++++++++++++++++++--
drivers/gpu/drm/msm/registers/display/dsi.xml | 5 +++-
include/drm/drm_mipi_dsi.h | 4 ++++
6 files changed, 45 insertions(+), 10 deletions(-)
---
base-commit: f338e77383789c0cae23ca3d48adcc5e9e137e3c
change-id: 20260318-dsi-rgb101010-support-4956b1cd8657
Best regards,
--
Alexander Koskovich <akoskovich@pm.me>
^ permalink raw reply
* [PATCH v5 1/5] drm/mipi-dsi: add RGB101010 pixel format
From: Alexander Koskovich @ 2026-03-24 11:48 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich, Dmitry Baryshkov
In-Reply-To: <20260324-dsi-rgb101010-support-v5-0-ff6afc904115@pm.me>
Add MIPI_DSI_FMT_RGB101010 for 30 bit (10,10,10 RGB) pixel format,
corresponding to the packed 30 bit pixel stream defined in MIPI DSI
v1.3 Section 8.8.17.
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
include/drm/drm_mipi_dsi.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/drm/drm_mipi_dsi.h b/include/drm/drm_mipi_dsi.h
index 3aba7b380c8d..a822e9e876af 100644
--- a/include/drm/drm_mipi_dsi.h
+++ b/include/drm/drm_mipi_dsi.h
@@ -140,6 +140,7 @@ struct mipi_dsi_host *of_find_mipi_dsi_host_by_node(struct device_node *node);
#define MIPI_DSI_HS_PKT_END_ALIGNED BIT(12)
enum mipi_dsi_pixel_format {
+ MIPI_DSI_FMT_RGB101010,
MIPI_DSI_FMT_RGB888,
MIPI_DSI_FMT_RGB666,
MIPI_DSI_FMT_RGB666_PACKED,
@@ -235,6 +236,9 @@ extern const struct bus_type mipi_dsi_bus_type;
static inline int mipi_dsi_pixel_format_to_bpp(enum mipi_dsi_pixel_format fmt)
{
switch (fmt) {
+ case MIPI_DSI_FMT_RGB101010:
+ return 30;
+
case MIPI_DSI_FMT_RGB888:
case MIPI_DSI_FMT_RGB666:
return 24;
--
2.53.0
^ permalink raw reply related
* [PATCH v5 2/5] drm/meson: use default case for unsupported DSI pixel formats
From: Alexander Koskovich @ 2026-03-24 11:48 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich
In-Reply-To: <20260324-dsi-rgb101010-support-v5-0-ff6afc904115@pm.me>
Use default instead of listing unsupported formats explicitly, so the
switch statements don't need updating each time a new pixel format is
added.
Reviewed-by: Neil Armstrong <neil.armstrong@linaro.org>
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
drivers/gpu/drm/meson/meson_dw_mipi_dsi.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/meson/meson_dw_mipi_dsi.c b/drivers/gpu/drm/meson/meson_dw_mipi_dsi.c
index 66c73c512b0e..4412bd678108 100644
--- a/drivers/gpu/drm/meson/meson_dw_mipi_dsi.c
+++ b/drivers/gpu/drm/meson/meson_dw_mipi_dsi.c
@@ -119,8 +119,7 @@ static int dw_mipi_dsi_phy_init(void *priv_data)
dpi_data_format = DPI_COLOR_18BIT_CFG_2;
venc_data_width = VENC_IN_COLOR_18B;
break;
- case MIPI_DSI_FMT_RGB666_PACKED:
- case MIPI_DSI_FMT_RGB565:
+ default:
return -EINVAL;
}
@@ -232,8 +231,7 @@ static int meson_dw_mipi_dsi_host_attach(void *priv_data,
break;
case MIPI_DSI_FMT_RGB666:
break;
- case MIPI_DSI_FMT_RGB666_PACKED:
- case MIPI_DSI_FMT_RGB565:
+ default:
dev_err(mipi_dsi->dev, "invalid pixel format %d\n", device->format);
return -EINVAL;
}
--
2.53.0
^ permalink raw reply related
* [PATCH v5 3/5] drm/msm/dsi: rename MSM8998 DSI version from V2_2_0 to V2_0_0
From: Alexander Koskovich @ 2026-03-24 11:48 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich, Konrad Dybcio,
Dmitry Baryshkov
In-Reply-To: <20260324-dsi-rgb101010-support-v5-0-ff6afc904115@pm.me>
The MSM8998 DSI controller is v2.0.0 as stated in commit 7b8c9e203039
("drm/msm/dsi: Add support for MSM8998 DSI controller"). The value was
always correct just the name was wrong.
Rename and reorder to maintain version sorting.
Fixes: 7b8c9e203039 ("drm/msm/dsi: Add support for MSM8998 DSI controller")
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
drivers/gpu/drm/msm/dsi/dsi_cfg.c | 4 ++--
drivers/gpu/drm/msm/dsi/dsi_cfg.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/msm/dsi/dsi_cfg.c b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
index bd3c51c350e7..da3fe6824495 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_cfg.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
@@ -317,10 +317,10 @@ static const struct msm_dsi_cfg_handler dsi_cfg_handlers[] = {
&msm8996_dsi_cfg, &msm_dsi_6g_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V1_4_2,
&msm8976_dsi_cfg, &msm_dsi_6g_host_ops},
+ {MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_0_0,
+ &msm8998_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_1_0,
&sdm660_dsi_cfg, &msm_dsi_6g_v2_host_ops},
- {MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_2_0,
- &msm8998_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_2_1,
&sdm845_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_3_0,
diff --git a/drivers/gpu/drm/msm/dsi/dsi_cfg.h b/drivers/gpu/drm/msm/dsi/dsi_cfg.h
index 5dc812028bd5..ccf06679608e 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_cfg.h
+++ b/drivers/gpu/drm/msm/dsi/dsi_cfg.h
@@ -19,8 +19,8 @@
#define MSM_DSI_6G_VER_MINOR_V1_3_1 0x10030001
#define MSM_DSI_6G_VER_MINOR_V1_4_1 0x10040001
#define MSM_DSI_6G_VER_MINOR_V1_4_2 0x10040002
+#define MSM_DSI_6G_VER_MINOR_V2_0_0 0x20000000
#define MSM_DSI_6G_VER_MINOR_V2_1_0 0x20010000
-#define MSM_DSI_6G_VER_MINOR_V2_2_0 0x20000000
#define MSM_DSI_6G_VER_MINOR_V2_2_1 0x20020001
#define MSM_DSI_6G_VER_MINOR_V2_3_0 0x20030000
#define MSM_DSI_6G_VER_MINOR_V2_3_1 0x20030001
--
2.53.0
^ permalink raw reply related
* [PATCH v5 4/5] drm/msm/dsi: add DSI version >= comparison helper
From: Alexander Koskovich @ 2026-03-24 11:48 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich, Konrad Dybcio,
Dmitry Baryshkov
In-Reply-To: <20260324-dsi-rgb101010-support-v5-0-ff6afc904115@pm.me>
Add a helper for checking if the DSI hardware version is greater
than or equal to a given version, for use in a future change.
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
drivers/gpu/drm/msm/dsi/dsi_host.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index db6da99375a1..b55ffe2fdec3 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -782,13 +782,21 @@ static void dsi_ctrl_disable(struct msm_dsi_host *msm_host)
dsi_write(msm_host, REG_DSI_CTRL, 0);
}
+static bool msm_dsi_host_version_geq(struct msm_dsi_host *msm_host,
+ u32 major, u32 minor)
+{
+ return msm_host->cfg_hnd->major > major ||
+ (msm_host->cfg_hnd->major == major &&
+ msm_host->cfg_hnd->minor >= minor);
+}
+
bool msm_dsi_host_is_wide_bus_enabled(struct mipi_dsi_host *host)
{
struct msm_dsi_host *msm_host = to_msm_dsi_host(host);
return msm_host->dsc &&
- (msm_host->cfg_hnd->major == MSM_DSI_VER_MAJOR_6G &&
- msm_host->cfg_hnd->minor >= MSM_DSI_6G_VER_MINOR_V2_5_0);
+ msm_dsi_host_version_geq(msm_host, MSM_DSI_VER_MAJOR_6G,
+ MSM_DSI_6G_VER_MINOR_V2_5_0);
}
static void dsi_ctrl_enable(struct msm_dsi_host *msm_host,
--
2.53.0
^ permalink raw reply related
* [PATCH v5 5/5] drm/msm/dsi: Add support for RGB101010 pixel format
From: Alexander Koskovich @ 2026-03-24 11:48 UTC (permalink / raw)
To: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Jeffrey Hugo,
Neil Armstrong, Martin Blumenstingl
Cc: dri-devel, linux-kernel, linux-arm-msm, freedreno, linux-amlogic,
linux-arm-kernel, Alexander Koskovich, Konrad Dybcio,
Dmitry Baryshkov
In-Reply-To: <20260324-dsi-rgb101010-support-v5-0-ff6afc904115@pm.me>
Add video and command mode destination format mappings for RGB101010,
and extend the VID_CFG0 DST_FORMAT bitfield to 3 bits to accommodate
the new format value.
Make sure this is guarded behind MSM_DSI_6G_VER >= V2.1.0 as anything
older does not support this.
Required for 10 bit panels such as the BOE BF068MWM-TD0 found on the
Nothing Phone (3a).
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
---
drivers/gpu/drm/msm/dsi/dsi_host.c | 22 ++++++++++++++++++++++
drivers/gpu/drm/msm/registers/display/dsi.xml | 5 ++++-
2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index b55ffe2fdec3..992407bbea84 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -757,6 +757,7 @@ static inline enum dsi_vid_dst_format
dsi_get_vid_fmt(const enum mipi_dsi_pixel_format mipi_fmt)
{
switch (mipi_fmt) {
+ case MIPI_DSI_FMT_RGB101010: return VID_DST_FORMAT_RGB101010;
case MIPI_DSI_FMT_RGB888: return VID_DST_FORMAT_RGB888;
case MIPI_DSI_FMT_RGB666: return VID_DST_FORMAT_RGB666_LOOSE;
case MIPI_DSI_FMT_RGB666_PACKED: return VID_DST_FORMAT_RGB666;
@@ -769,6 +770,7 @@ static inline enum dsi_cmd_dst_format
dsi_get_cmd_fmt(const enum mipi_dsi_pixel_format mipi_fmt)
{
switch (mipi_fmt) {
+ case MIPI_DSI_FMT_RGB101010: return CMD_DST_FORMAT_RGB101010;
case MIPI_DSI_FMT_RGB888: return CMD_DST_FORMAT_RGB888;
case MIPI_DSI_FMT_RGB666_PACKED:
case MIPI_DSI_FMT_RGB666: return CMD_DST_FORMAT_RGB666;
@@ -1713,6 +1715,26 @@ static int dsi_host_attach(struct mipi_dsi_host *host,
if (dsi->dsc)
msm_host->dsc = dsi->dsc;
+ if (msm_host->format == MIPI_DSI_FMT_RGB101010) {
+ if (!msm_dsi_host_version_geq(msm_host, MSM_DSI_VER_MAJOR_6G,
+ MSM_DSI_6G_VER_MINOR_V2_1_0)) {
+ DRM_DEV_ERROR(&msm_host->pdev->dev,
+ "RGB101010 not supported on this DSI controller\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Downstream overrides RGB101010 back to RGB888 when DSC is enabled
+ * but widebus is not. Using RGB101010 in this case may require some
+ * extra changes.
+ */
+ if (msm_host->dsc &&
+ !msm_dsi_host_is_wide_bus_enabled(&msm_host->base)) {
+ dev_warn(&msm_host->pdev->dev,
+ "RGB101010 with DSC but without widebus, may need extra changes\n");
+ }
+ }
+
ret = dsi_dev_attach(msm_host->pdev);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/msm/registers/display/dsi.xml b/drivers/gpu/drm/msm/registers/display/dsi.xml
index c7a7b633d747..e40125f75175 100644
--- a/drivers/gpu/drm/msm/registers/display/dsi.xml
+++ b/drivers/gpu/drm/msm/registers/display/dsi.xml
@@ -15,6 +15,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd">
<value name="VID_DST_FORMAT_RGB666" value="1"/>
<value name="VID_DST_FORMAT_RGB666_LOOSE" value="2"/>
<value name="VID_DST_FORMAT_RGB888" value="3"/>
+ <value name="VID_DST_FORMAT_RGB101010" value="4"/>
</enum>
<enum name="dsi_rgb_swap">
<value name="SWAP_RGB" value="0"/>
@@ -39,6 +40,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd">
<value name="CMD_DST_FORMAT_RGB565" value="6"/>
<value name="CMD_DST_FORMAT_RGB666" value="7"/>
<value name="CMD_DST_FORMAT_RGB888" value="8"/>
+ <value name="CMD_DST_FORMAT_RGB101010" value="9"/>
</enum>
<enum name="dsi_lane_swap">
<value name="LANE_SWAP_0123" value="0"/>
@@ -142,7 +144,8 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd">
</reg32>
<reg32 offset="0x0000c" name="VID_CFG0">
<bitfield name="VIRT_CHANNEL" low="0" high="1" type="uint"/> <!-- always zero? -->
- <bitfield name="DST_FORMAT" low="4" high="5" type="dsi_vid_dst_format"/>
+ <!-- high was 5 before DSI 6G 2.1.0 -->
+ <bitfield name="DST_FORMAT" low="4" high="6" type="dsi_vid_dst_format"/>
<bitfield name="TRAFFIC_MODE" low="8" high="9" type="dsi_traffic_mode"/>
<bitfield name="BLLP_POWER_STOP" pos="12" type="boolean"/>
<bitfield name="EOF_BLLP_POWER_STOP" pos="15" type="boolean"/>
--
2.53.0
^ permalink raw reply related
* [PATCH bpf-next v10 5/5] bpf, arm64: Emit BTI for indirect jump target
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260324122052.342751-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support BTI, the indirect jump selftest triggers a kernel
panic because there is no BTI instructions at the indirect jump targets.
Fix it by emitting a BTI instruction for each indirect jump target.
For reference, below is a sample panic log.
Internal error: Oops - BTI: 0000000036000003 [#1] SMP
...
Call trace:
bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x54/0xf8 (P)
bpf_prog_run_pin_on_cpu+0x140/0x468
bpf_prog_test_run_syscall+0x280/0x3b8
bpf_prog_test_run+0x22c/0x2c0
Fixes: f4a66cf1cb14 ("bpf: arm64: Add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 7212ec89dfe3..aa5cd240cdda 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1198,8 +1198,8 @@ static int add_exception_handler(const struct bpf_insn *insn,
* >0 - successfully JITed a 16-byte eBPF instruction.
* <0 - failed to JIT.
*/
-static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
- bool extra_pass)
+static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn *insn,
+ struct jit_ctx *ctx, bool extra_pass)
{
const u8 code = insn->code;
u8 dst = bpf2a64[insn->dst_reg];
@@ -1224,6 +1224,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
int ret;
bool sign_extend;
+ if (bpf_insn_is_indirect_target(env, ctx->prog, i))
+ emit_bti(A64_BTI_J, ctx);
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
@@ -1899,7 +1902,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
-static int build_body(struct jit_ctx *ctx, bool extra_pass)
+static int build_body(struct bpf_verifier_env *env, struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
int i;
@@ -1918,7 +1921,7 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
int ret;
ctx->offset[i] = ctx->idx;
- ret = build_insn(insn, ctx, extra_pass);
+ ret = build_insn(env, insn, ctx, extra_pass);
if (ret > 0) {
i++;
ctx->offset[i] = ctx->idx;
@@ -2079,7 +2082,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
if (build_prologue(&ctx, was_classic))
goto out_off;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_off;
ctx.epilogue_offset = ctx.idx;
@@ -2127,7 +2130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_free_hdr;
ctx.epilogue_offset = ctx.idx;
@@ -2136,7 +2139,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
ctx.write = true;
/* Pass 3: Adjust jump offset and write final image */
- if (build_body(&ctx, extra_pass) ||
+ if (build_body(env, &ctx, extra_pass) ||
WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v10 3/5] bpf: Add helper to detect indirect jump targets
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260324122052.342751-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Introduce helper bpf_insn_is_indirect_target to check whether a BPF
instruction is an indirect jump target.
Since the verifier knows which instructions are indirect jump targets,
add a new flag indirect_target to struct bpf_insn_aux_data to mark
them. The verifier sets this flag when verifying an indirect jump target
instruction, and the helper checks the flag to determine whether an
instruction is an indirect jump target.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
include/linux/bpf.h | 2 ++
include/linux/bpf_verifier.h | 9 +++++----
kernel/bpf/core.c | 9 +++++++++
kernel/bpf/verifier.c | 18 ++++++++++++++++++
4 files changed, 34 insertions(+), 4 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 05b34a6355b0..90760e250865 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1541,6 +1541,8 @@ bool bpf_has_frame_pointer(unsigned long ip);
int bpf_jit_charge_modmem(u32 size);
void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 090aa26d1c98..6431b94746cf 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -578,16 +578,17 @@ struct bpf_insn_aux_data {
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
- bool jmp_point;
- bool prune_point;
+ u32 jmp_point:1;
+ u32 prune_point:1;
/* ensure we check state equivalence and save state checkpoint and
* this instruction, regardless of any heuristics
*/
- bool force_checkpoint;
+ u32 force_checkpoint:1;
/* true if instruction is a call to a helper function that
* accepts callback function as a parameter.
*/
- bool calls_callback;
+ u32 calls_callback:1;
+ u32 indirect_target:1; /* if it is an indirect jump target */
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fa3d6d8d76da..6e0c3f327144 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1572,6 +1572,15 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
clone->blinded = 1;
return clone;
}
+
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx)
+{
+ if (!env)
+ return false;
+ insn_idx += prog->aux->subprog_start;
+ return env->insn_aux_data[insn_idx].indirect_target;
+}
#endif /* CONFIG_BPF_JIT */
/* Base function for offset calculation. Needs to go into .text section,
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 89fb700d818f..3b81747a1b61 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4028,6 +4028,11 @@ static bool is_jmp_point(struct bpf_verifier_env *env, int insn_idx)
return env->insn_aux_data[insn_idx].jmp_point;
}
+static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].indirect_target = true;
+}
+
#define LR_FRAMENO_BITS 3
#define LR_SPI_BITS 6
#define LR_ENTRY_BITS (LR_SPI_BITS + LR_FRAMENO_BITS + 1)
@@ -21117,12 +21122,14 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
}
for (i = 0; i < n - 1; i++) {
+ mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
env->insn_idx, env->cur_state->speculative);
if (IS_ERR(other_branch))
return PTR_ERR(other_branch);
}
env->insn_idx = env->gotox_tmp_buf->items[n-1];
+ mark_indirect_target(env, env->insn_idx);
return 0;
}
@@ -22048,6 +22055,17 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
data[i].seen = old_seen;
data[i].zext_dst = insn_has_def32(insn + i);
}
+
+ /* The indirect_target flag of the original instruction was moved to the last of the
+ * new instructions by the above memmove and memset, but the indirect jump target is
+ * actually the first instruction, so move it back. This also matches with the behavior
+ * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
+ * instruction.
+ */
+ if (data[off + cnt - 1].indirect_target) {
+ data[off].indirect_target = 1;
+ data[off + cnt - 1].indirect_target = 0;
+ }
}
static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v10 4/5] bpf, x86: Emit ENDBR for indirect jump targets
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260324122052.342751-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support CET/IBT, the indirect jump selftest triggers
a kernel panic because the indirect jump targets lack ENDBR
instructions.
To fix it, emit an ENDBR instruction to each indirect jump target. Since
the ENDBR instruction shifts the position of original jited instructions,
fix the instruction address calculation wherever the addresses are used.
For reference, below is a sample panic log.
Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cet.c:133!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
...
? 0xffffffffc00fb258
? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
bpf_prog_test_run_syscall+0x110/0x2f0
? fdget+0xba/0xe0
__sys_bpf+0xe4b/0x2590
? __kmalloc_node_track_caller_noprof+0x1c7/0x680
? bpf_prog_test_run_syscall+0x215/0x2f0
__x64_sys_bpf+0x21/0x30
do_syscall_64+0x85/0x620
? bpf_prog_test_run_syscall+0x1e2/0x2f0
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72d9a5faa230..ea9e707e8abf 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
#define EMIT_ENDBR() EMIT(gen_endbr(), 4)
#define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4)
#else
-#define EMIT_ENDBR()
-#define EMIT_ENDBR_POISON()
+#define EMIT_ENDBR() do { } while (0)
+#define EMIT_ENDBR_POISON() do { } while (0)
#endif
static bool is_imm8(int value)
@@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
-static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image,
- int oldproglen, struct jit_context *ctx, bool jmp_padding)
+static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
+ u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
bool tail_call_reachable = bpf_prog->aux->tail_call_reachable;
struct bpf_insn *insn = bpf_prog->insnsi;
@@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
void __percpu *priv_stack_ptr;
int i, excnt = 0;
int ilen, proglen = 0;
- u8 *prog = temp;
+ u8 *ip, *prog = temp;
u32 stack_depth;
int err;
@@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
dst_reg = X86_REG_R9;
}
+ if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1))
+ EMIT_ENDBR();
+
+ ip = image + addrs[i - 1] + (prog - temp);
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off))
/* call */
case BPF_JMP | BPF_CALL: {
- u8 *ip = image + addrs[i - 1];
-
func = (u8 *) __bpf_call_base + imm32;
if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off))
if (imm32)
emit_bpf_tail_call_direct(bpf_prog,
&bpf_prog->aux->poke_tab[imm32 - 1],
- &prog, image + addrs[i - 1],
+ &prog,
+ ip,
callee_regs_used,
stack_depth,
ctx);
@@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off))
&prog,
callee_regs_used,
stack_depth,
- image + addrs[i - 1],
+ ip,
ctx);
break;
@@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off))
break;
case BPF_JMP | BPF_JA | BPF_X:
- emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+ emit_indirect_jump(&prog, insn->dst_reg, ip);
break;
case BPF_JMP | BPF_JA:
case BPF_JMP32 | BPF_JA:
@@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off))
ctx->cleanup_addr = proglen;
if (bpf_prog_was_classic(bpf_prog) &&
!ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
- u8 *ip = image + addrs[i - 1];
-
if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
return -EINVAL;
}
@@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
for (pass = 0; pass < MAX_PASSES || image; pass++) {
if (!padding && pass >= PADDING_PASSES)
padding = true;
- proglen = do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, padding);
+ proglen = do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, padding);
if (proglen <= 0) {
out_image:
image = NULL;
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v10 0/5] emit ENDBR/BTI instructions for indirect jump targets
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.
The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, so the JIT
knows which instructions are indirect jump targets.
During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not lookup env->insn_aux_data directly.
To avoid this mismatch, and considering that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.
v10:
- Fix the incorrect call_imm restore in jit_subprogs
- Define a dummy void version of bpf_jit_prog_release_other and
bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)
v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com/
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)
v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com/
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
to adjust_insn_aux_data()
v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch
v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com/
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier
v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com/
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
each prog
v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com/
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
jump targets, and emit ENDBR on x86
v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com/
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)
v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com/
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)
v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com/
Xu Kuohai (5):
bpf: Move constants blinding out of arch-specific JITs
bpf: Pass bpf_verifier_env to JIT
bpf: Add helper to detect indirect jump targets
bpf, x86: Emit ENDBR for indirect jump targets
bpf, arm64: Emit BTI for indirect jump target
arch/arc/net/bpf_jit_core.c | 41 ++++-------
arch/arm/net/bpf_jit_32.c | 43 +++--------
arch/arm64/net/bpf_jit_comp.c | 87 ++++++++---------------
arch/loongarch/net/bpf_jit.c | 61 +++++-----------
arch/mips/net/bpf_jit_comp.c | 22 +-----
arch/parisc/net/bpf_jit_core.c | 75 ++++++++------------
arch/powerpc/net/bpf_jit_comp.c | 70 +++++++-----------
arch/riscv/net/bpf_jit_core.c | 63 ++++++-----------
arch/s390/net/bpf_jit_comp.c | 61 ++++++----------
arch/sparc/net/bpf_jit_comp_64.c | 63 ++++++-----------
arch/x86/net/bpf_jit_comp.c | 73 ++++++-------------
arch/x86/net/bpf_jit_comp32.c | 35 ++-------
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 9 +--
include/linux/filter.h | 27 ++++++-
kernel/bpf/core.c | 118 +++++++++++++++++++++----------
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 79 +++++++++++++++------
18 files changed, 392 insertions(+), 541 deletions(-)
--
2.47.3
^ permalink raw reply
* [PATCH bpf-next v10 2/5] bpf: Pass bpf_verifier_env to JIT
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260324122052.342751-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will
use env->insn_aux_data in the JIT stage to detect indirect jump targets.
Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c
code without verifier, introduce helper __bpf_prog_select_runtime()
to accept the env parameter.
Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and
switch to call __bpf_prog_select_runtime() in the verifier, with env
variable passed. The original bpf_prog_select_runtime() is preserved for
cbpf and lib/test_bpf.c, where env is NULL.
Now all constants blinding calls are moved into the verifier, except
the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted
by bpf_patch_insn_data() function for normal cases, so there is no need
to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 2 +-
arch/arm/net/bpf_jit_32.c | 2 +-
arch/arm64/net/bpf_jit_comp.c | 2 +-
arch/loongarch/net/bpf_jit.c | 2 +-
arch/mips/net/bpf_jit_comp.c | 2 +-
arch/parisc/net/bpf_jit_core.c | 2 +-
arch/powerpc/net/bpf_jit_comp.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 2 +-
arch/s390/net/bpf_jit_comp.c | 2 +-
arch/sparc/net/bpf_jit_comp_64.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
include/linux/filter.h | 4 +-
kernel/bpf/core.c | 63 ++++++++++++--------------------
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 8 +++-
16 files changed, 44 insertions(+), 59 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 973ceae48675..639a2736f029 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -1400,7 +1400,7 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog)
* (re)locations involved that their addresses are not known
* during the first run.
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
vm_dump(prog);
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index e6b1bb2de627..1628b6fc70a4 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2142,7 +2142,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header;
struct jit_ctx ctx;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index cd5a72fff500..7212ec89dfe3 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2006,7 +2006,7 @@ struct arm64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
struct bpf_binary_header *header;
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index fcc8c0c29fb0..5149ce4cef7e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1920,7 +1920,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index d2b6c955f18e..6ee4abe6a1f7 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -909,7 +909,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index 35dca372b5df..172770132440 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 711028bebea3..27fecb4cc063 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -129,7 +129,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
u32 proglen;
u32 alloclen;
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 527baa50dc68..768ac686b359 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 2dfc279b1be2..94128fe6be23 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2312,7 +2312,7 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
/*
* Compile eBPF program "fp"
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index e83e29137566..2fa0e9375127 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1477,7 +1477,7 @@ struct sparc64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 77d00a8dec87..72d9a5faa230 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3713,7 +3713,7 @@ struct x64_jit_data {
#define MAX_PASSES 20
#define PADDING_PASSES (MAX_PASSES - 5)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5f259577614a..852baf2e4db4 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2518,7 +2518,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
int proglen, oldproglen = 0;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index d8c97f463e0f..fe19faae35bb 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1108,6 +1108,8 @@ static inline int sk_filter_reason(struct sock *sk, struct sk_buff *skb,
return sk_filter_trim_cap(sk, skb, 1, reason);
}
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err);
struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err);
void bpf_prog_free(struct bpf_prog *fp);
@@ -1153,7 +1155,7 @@ u64 __bpf_call_base(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
((u64 (*)(u64, u64, u64, u64, u64, const struct bpf_insn *)) \
(void *)__bpf_call_base)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog);
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_compile(struct bpf_prog *prog);
bool bpf_jit_needs_zext(void);
bool bpf_jit_inlines_helper_call(s32 imm);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 8f52f4b6c3af..fa3d6d8d76da 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1490,23 +1490,6 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
bpf_prog_clone_free(fp_other);
}
-static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
-{
-#ifdef CONFIG_BPF_SYSCALL
- struct bpf_map *map;
- int i;
-
- if (len <= 1)
- return;
-
- for (i = 0; i < prog->aux->used_map_cnt; i++) {
- map = prog->aux->used_maps[i];
- if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
- bpf_insn_array_adjust(map, off, len);
- }
-#endif
-}
-
struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
@@ -1579,12 +1562,6 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
if (env)
env->prog = clone;
- else
- /* Instructions arrays must be updated using absolute xlated offsets.
- * The arrays have already been adjusted by bpf_patch_insn_data() when
- * env is not NULL.
- */
- adjust_insn_arrays(clone, subprog_start + i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2553,13 +2530,13 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
-static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
#ifdef CONFIG_BPF_JIT
bool blinded = false;
struct bpf_prog *orig_prog = prog;
- prog = bpf_jit_blind_constants(NULL, orig_prog);
+ prog = bpf_jit_blind_constants(env, orig_prog);
/* If blinding was requested and we failed during blinding, we must fall
* back to the interpreter.
*/
@@ -2569,7 +2546,7 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
if (prog != orig_prog)
blinded = true;
- prog = bpf_int_jit_compile(prog);
+ prog = bpf_int_jit_compile(env, prog);
if (blinded) {
if (!prog->jited) {
bpf_jit_prog_release_other(orig_prog, prog);
@@ -2582,18 +2559,8 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
return prog;
}
-/**
- * bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with BPF program
- * @err: pointer to error variable
- *
- * Try to JIT eBPF program, if JIT is not available, use interpreter.
- * The BPF program will be executed via bpf_prog_run() function.
- *
- * Return: the &fp argument along with &err set to 0 for success or
- * a negative errno code on failure
- */
-struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err)
{
/* In case of BPF to BPF calls, verifier did all the prep
* work with regards to JITing, etc.
@@ -2621,7 +2588,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_prog_jit_compile(fp);
+ fp = bpf_prog_jit_compile(env, fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
@@ -2647,6 +2614,22 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
return fp;
}
+
+/**
+ * bpf_prog_select_runtime - select exec runtime for BPF program
+ * @fp: bpf_prog populated with BPF program
+ * @err: pointer to error variable
+ *
+ * Try to JIT eBPF program, if JIT is not available, use interpreter.
+ * The BPF program will be executed via bpf_prog_run() function.
+ *
+ * Return: the &fp argument along with &err set to 0 for success or
+ * a negative errno code on failure
+ */
+struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+{
+ return __bpf_prog_select_runtime(NULL, fp, err);
+}
EXPORT_SYMBOL_GPL(bpf_prog_select_runtime);
static unsigned int __bpf_prog_ret1(const void *ctx,
@@ -3134,7 +3117,7 @@ const struct bpf_func_proto bpf_tail_call_proto = {
* It is encouraged to implement bpf_int_jit_compile() instead, so that
* eBPF and implicitly also cBPF can get JITed!
*/
-struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
return prog;
}
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 274039e36465..fb07be46c936 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3090,10 +3090,6 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
if (err < 0)
goto free_used_maps;
- prog = bpf_prog_select_runtime(prog, &err);
- if (err < 0)
- goto free_used_maps;
-
err = bpf_prog_mark_insn_arrays_ready(prog);
if (err < 0)
goto free_used_maps;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 1b2b99f2d859..89fb700d818f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -23024,7 +23024,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
- func[i] = bpf_int_jit_compile(func[i]);
+ func[i] = bpf_int_jit_compile(env, func[i]);
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -23068,7 +23068,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
for (i = 0; i < env->subprog_cnt; i++) {
old_bpf_func = func[i]->bpf_func;
- tmp = bpf_int_jit_compile(func[i]);
+ tmp = bpf_int_jit_compile(env, func[i]);
if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) {
verbose(env, "JIT doesn't support bpf-to-bpf calls\n");
err = -ENOTSUPP;
@@ -26339,6 +26339,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
convert_pseudo_ld_imm64(env);
}
+ env->prog = __bpf_prog_select_runtime(env, env->prog, &ret);
+ if (ret)
+ goto err_release_maps;
+
adjust_btf_func(env);
err_release_maps:
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v10 1/5] bpf: Move constants blinding out of arch-specific JITs
From: Xu Kuohai @ 2026-03-24 12:20 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Yonghong Song, Puranjay Mohan,
Anton Protopopov, Alexis Lothoré, Shahab Vahedi,
Russell King, Tiezhu Yang, Hengqi Chen, Johan Almbladh,
Paul Burton, Hari Bathini, Christophe Leroy, Naveen N Rao,
Luke Nelson, Xi Wang, Björn Töpel, Pu Lehui,
Ilya Leoshkevich, Heiko Carstens, Vasily Gorbik, David S . Miller,
Wang YanQing
In-Reply-To: <20260324122052.342751-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
During the JIT stage, constants blinding rewrites instructions but only
rewrites the private instruction copy of the JITed subprog, leaving the
global instructions and insn_aux_data unchanged. This causes a mismatch
between subprog instructions and the global state, making it difficult
to look up the global insn_aux_data in the JIT.
To avoid this mismatch, and given that all arch-specific JITs already
support constants blinding, move it to the generic verifier code, and
switch to rewrite the global env->insnsi with the global states
adjusted, as other rewrites in the verifier do.
This removes the constants blinding calls in each JIT, which are largely
duplicated code across architectures.
Since constants blinding is only required for JIT, and there are two
entry functions for JIT, jit_subprogs() and bpf_prog_select_runtime(),
move the constants blinding invocation into the two functions.
If constants blinding fails, or if it succeeds but the subsequent JIT
compilation fails, kernel falls back to running the BPF program with
interpreter. To ensure a correct rollback, the program cloning before
instruction rewriting in the constants blinding is preserved. During
the blinding process, only the cloned instructions are patched, leaving
the original program untouched.
Since bpf_patch_insn_data() is chosen for the constants blinding in the
verifier path, and it adjusts the global auxiliary data in the verifier
state, a key question is whether this auxiliary data should be restored
when JIT fails?
Besides instructions, bpf_patch_insn_data() adjusts env->insn_aux_data,
env->subprog_info, prog->aux->poke_tab and env->insn_array_maps. env->
insn_aux_data and env->subprog_info are no longer used after JIT failure
and are freed at the end of bpf_check(). prog->aux->poke_tab is only
used by JIT. And when the JIT fails, programs using insn_array would be
rejected by bpf_insn_array_ready() function since no JITed addresses
available. This means env->insn_array_maps is only useful for JIT.
Therefore, all the auxiliary data adjusted does not need to be restored.
For classic BPF programs, constants blinding works as before since it
is still invoked from bpf_prog_select_runtime().
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Hari Bathini <hbathini@linux.ibm.com> # powerpc jit
Reviewed-by: Pu Lehui <pulehui@huawei.com> # riscv jit
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 39 ++++++-----------
arch/arm/net/bpf_jit_32.c | 41 +++---------------
arch/arm64/net/bpf_jit_comp.c | 72 +++++++++----------------------
arch/loongarch/net/bpf_jit.c | 59 ++++++++------------------
arch/mips/net/bpf_jit_comp.c | 20 +--------
arch/parisc/net/bpf_jit_core.c | 73 +++++++++++++-------------------
arch/powerpc/net/bpf_jit_comp.c | 68 +++++++++++------------------
arch/riscv/net/bpf_jit_core.c | 61 ++++++++++----------------
arch/s390/net/bpf_jit_comp.c | 59 +++++++++-----------------
arch/sparc/net/bpf_jit_comp_64.c | 61 +++++++++-----------------
arch/x86/net/bpf_jit_comp.c | 43 +++----------------
arch/x86/net/bpf_jit_comp32.c | 33 ++-------------
include/linux/filter.h | 23 +++++++++-
kernel/bpf/core.c | 66 +++++++++++++++++++++++++----
kernel/bpf/verifier.c | 53 ++++++++++++++---------
15 files changed, 301 insertions(+), 470 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 1421eeced0f5..973ceae48675 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -79,7 +79,6 @@ struct arc_jit_data {
* The JIT pertinent context that is used by different functions.
*
* prog: The current eBPF program being handled.
- * orig_prog: The original eBPF program before any possible change.
* jit: The JIT buffer and its length.
* bpf_header: The JITed program header. "jit.buf" points inside it.
* emit: If set, opcodes are written to memory; else, a dry-run.
@@ -94,12 +93,10 @@ struct arc_jit_data {
* need_extra_pass: A forecast if an "extra_pass" will occur.
* is_extra_pass: Indicates if the current pass is an extra pass.
* user_bpf_prog: True, if VM opcodes come from a real program.
- * blinded: True if "constant blinding" step returned a new "prog".
* success: Indicates if the whole JIT went OK.
*/
struct jit_context {
struct bpf_prog *prog;
- struct bpf_prog *orig_prog;
struct jit_buffer jit;
struct bpf_binary_header *bpf_header;
bool emit;
@@ -114,7 +111,6 @@ struct jit_context {
bool need_extra_pass;
bool is_extra_pass;
bool user_bpf_prog;
- bool blinded;
bool success;
};
@@ -161,13 +157,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog)
{
memset(ctx, 0, sizeof(*ctx));
- ctx->orig_prog = prog;
-
- /* If constant blinding was requested but failed, scram. */
- ctx->prog = bpf_jit_blind_constants(prog);
- if (IS_ERR(ctx->prog))
- return PTR_ERR(ctx->prog);
- ctx->blinded = (ctx->prog != ctx->orig_prog);
+ ctx->prog = prog;
/* If the verifier doesn't zero-extend, then we have to do it. */
ctx->do_zext = !ctx->prog->aux->verifier_zext;
@@ -214,14 +204,6 @@ static inline void maybe_free(struct jit_context *ctx, void **mem)
*/
static void jit_ctx_cleanup(struct jit_context *ctx)
{
- if (ctx->blinded) {
- /* if all went well, release the orig_prog. */
- if (ctx->success)
- bpf_jit_prog_release_other(ctx->prog, ctx->orig_prog);
- else
- bpf_jit_prog_release_other(ctx->orig_prog, ctx->prog);
- }
-
maybe_free(ctx, (void **)&ctx->bpf2insn);
maybe_free(ctx, (void **)&ctx->jit_data);
@@ -229,12 +211,19 @@ static void jit_ctx_cleanup(struct jit_context *ctx)
ctx->bpf2insn_valid = false;
/* Freeing "bpf_header" is enough. "jit.buf" is a sub-array of it. */
- if (!ctx->success && ctx->bpf_header) {
- bpf_jit_binary_free(ctx->bpf_header);
- ctx->bpf_header = NULL;
- ctx->jit.buf = NULL;
- ctx->jit.index = 0;
- ctx->jit.len = 0;
+ if (!ctx->success) {
+ if (ctx->bpf_header) {
+ bpf_jit_binary_free(ctx->bpf_header);
+ ctx->bpf_header = NULL;
+ ctx->jit.buf = NULL;
+ ctx->jit.index = 0;
+ ctx->jit.len = 0;
+ }
+ if (ctx->is_extra_pass) {
+ ctx->prog->bpf_func = NULL;
+ ctx->prog->jited = 0;
+ ctx->prog->jited_len = 0;
+ }
}
ctx->emit = false;
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..e6b1bb2de627 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2144,9 +2144,7 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
- bool tmp_blinded = false;
struct jit_ctx ctx;
unsigned int tmp_idx;
unsigned int image_size;
@@ -2156,20 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- /* If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
-
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
@@ -2179,10 +2164,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* we must fall back to the interpreter
*/
ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
- if (ctx.offsets == NULL) {
- prog = orig_prog;
- goto out;
- }
+ if (ctx.offsets == NULL)
+ return prog;
/* 1) fake pass to find in the length of the JITed code,
* to compute ctx->offsets and other context variables
@@ -2194,10 +2177,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* being successful in the second pass, so just fall back
* to the interpreter.
*/
- if (build_body(&ctx)) {
- prog = orig_prog;
+ if (build_body(&ctx))
goto out_off;
- }
tmp_idx = ctx.idx;
build_prologue(&ctx);
@@ -2213,10 +2194,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx += ctx.imm_count;
if (ctx.imm_count) {
ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
- if (ctx.imms == NULL) {
- prog = orig_prog;
+ if (ctx.imms == NULL)
goto out_off;
- }
}
#else
/* there's nothing about the epilogue on ARMv7 */
@@ -2238,10 +2217,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Not able to allocate memory for the structure then
* we must fall back to the interpretation
*/
- if (header == NULL) {
- prog = orig_prog;
+ if (header == NULL)
goto out_imms;
- }
/* 2.) Actual pass to generate final JIT code */
ctx.target = (u32 *) image_ptr;
@@ -2278,16 +2255,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
#endif
out_off:
kfree(ctx.offsets);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free:
image_ptr = NULL;
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_imms;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index adf84962d579..cd5a72fff500 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2009,14 +2009,12 @@ struct arm64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header = NULL;
struct arm64_jit_data *jit_data;
void __percpu *priv_stack_ptr = NULL;
bool was_classic = bpf_prog_was_classic(prog);
int priv_stack_alloc_sz;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
@@ -2025,26 +2023,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int exentry_idx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -2056,10 +2041,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -2079,10 +2062,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kvzalloc_objs(int, prog->len + 1);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_off;
- }
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
@@ -2095,15 +2076,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* BPF line info needs ctx->offset[i] to be the offset of
* instruction[i] in jited image, so build prologue first.
*/
- if (build_prologue(&ctx, was_classic)) {
- prog = orig_prog;
+ if (build_prologue(&ctx, was_classic))
goto out_off;
- }
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_off;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx, was_classic);
@@ -2121,10 +2098,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_off;
- }
/* Pass 2: Determine jited position and result for each instruction */
@@ -2152,10 +2127,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free_hdr;
- }
ctx.epilogue_offset = ctx.idx;
ctx.exentry_idx = exentry_idx;
@@ -2164,19 +2137,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Pass 3: Adjust jump offset and write final image */
if (build_body(&ctx, extra_pass) ||
- WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset)) {
- prog = orig_prog;
+ WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
- }
build_epilogue(&ctx, was_classic);
build_plt(&ctx);
/* Extra pass to validate JITed code. */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free_hdr;
- }
/* update the real prog size */
prog_size = sizeof(u32) * ctx.idx;
@@ -2193,16 +2162,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (extra_pass && ctx.idx > jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d > %d\n",
ctx.idx, jit_data->ctx.idx);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
goto out_free_hdr;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header has been freed */
ro_header = NULL;
- prog = orig_prog;
- goto out_off;
+ header = NULL;
+ goto out_free_hdr;
}
/*
* The instructions have now been copied to the ROX region from
@@ -2245,13 +2211,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size,
sizeof(header->size));
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 9cb796e16379..fcc8c0c29fb0 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1922,43 +1922,26 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- bool tmp_blinded = false, extra_pass = false;
+ bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header;
- struct bpf_prog *tmp, *orig_prog = prog;
/*
* If BPF JIT was not enabled then we must fall back to
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- if (IS_ERR(tmp))
- return orig_prog;
-
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1978,17 +1961,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.offset = kvcalloc(prog->len + 1, sizeof(u32), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_offset;
- }
/* 1. Initial fake pass to compute ctx->idx and set ctx->flags */
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_offset;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
@@ -2004,10 +1983,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
&header, &image_ptr, jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_offset;
- }
/* 2. Now, the actual pass to generate final JIT code */
/*
@@ -2027,17 +2004,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.num_exentries = 0;
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free;
- }
build_epilogue(&ctx);
/* 3. Extra pass to validate JITed code */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free;
- }
/* And we're done */
if (bpf_jit_enable > 1)
@@ -2050,9 +2023,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_free;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header have been freed */
ro_header = NULL;
- prog = orig_prog;
+ header = NULL;
goto out_free;
}
/*
@@ -2084,13 +2057,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ? tmp : orig_prog);
-
return prog;
out_free:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size, sizeof(header->size));
bpf_jit_binary_pack_free(ro_header, header);
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index e355dfca4400..d2b6c955f18e 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -911,10 +911,8 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
- bool tmp_blinded = false;
unsigned int tmp_idx;
unsigned int image_size;
u8 *image_ptr;
@@ -925,19 +923,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
- /*
- * If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.program = prog;
@@ -1025,14 +1011,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited_len = image_size;
out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
kfree(ctx.descriptors);
return prog;
out_err:
- prog = orig_prog;
if (header)
bpf_jit_binary_free(header);
goto out;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index a5eb6b51e27a..35dca372b5df 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -44,30 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, prologue_len, i;
struct hppa_jit_data *jit_data;
struct hppa_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
@@ -81,10 +70,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!ctx->offset)
+ goto out_err;
for (i = 0; i < prog->len; i++) {
prev_ninsns += 20;
ctx->offset[i] = prev_ninsns;
@@ -93,10 +80,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
for (i = 0; i < NR_JIT_ITERATIONS; i++) {
pass++;
ctx->ninsns = 0;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (build_body(ctx, extra_pass, ctx->offset))
+ goto out_err;
ctx->body_len = ctx->ninsns;
bpf_jit_build_prologue(ctx);
ctx->prologue_len = ctx->ninsns - ctx->body_len;
@@ -116,10 +101,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->image,
sizeof(long),
bpf_fill_ill_insns);
- if (!jit_data->header) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!jit_data->header)
+ goto out_err;
ctx->insns = (u32 *)jit_data->image;
/*
@@ -134,8 +117,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
if (jit_data->header)
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
if (extable_size)
@@ -148,8 +130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx);
if (build_body(ctx, extra_pass, NULL)) {
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
bpf_jit_build_epilogue(ctx);
@@ -160,20 +141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{ extern int machine_restart(char *); machine_restart(""); }
}
+ if (!prog->is_func || extra_pass) {
+ if (bpf_jit_binary_lock_ro(jit_data->header)) {
+ bpf_jit_binary_free(jit_data->header);
+ goto out_err;
+ }
+ bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
+ }
+
prog->bpf_func = (void *)ctx->insns;
prog->jited = 1;
prog->jited_len = prog_size;
- bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
-
if (!prog->is_func || extra_pass) {
- if (bpf_jit_binary_lock_ro(jit_data->header)) {
- bpf_jit_binary_free(jit_data->header);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
- goto out_offset;
- }
prologue_len = ctx->epilogue_offset - ctx->body_len;
for (i = 0; i < prog->len; i++)
ctx->offset[i] += prologue_len;
@@ -183,14 +163,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
+
if (HPPA_JIT_REBOOT)
{ extern int machine_restart(char *); machine_restart(""); }
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_offset;
}
u64 hppa_div64(u64 div, u64 divisor)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index a62a9a92b7b5..711028bebea3 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -142,9 +142,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
int flen;
struct bpf_binary_header *fhdr = NULL;
struct bpf_binary_header *hdr = NULL;
- struct bpf_prog *org_fp = fp;
- struct bpf_prog *tmp_fp;
- bool bpf_blinded = false;
bool extra_pass = false;
u8 *fimage = NULL;
u32 *fcode_base;
@@ -152,24 +149,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u32 fixup_len;
if (!fp->jit_requested)
- return org_fp;
-
- tmp_fp = bpf_jit_blind_constants(org_fp);
- if (IS_ERR(tmp_fp))
- return org_fp;
-
- if (tmp_fp != org_fp) {
- bpf_blinded = true;
- fp = tmp_fp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = org_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
@@ -194,10 +180,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
addrs = kcalloc(flen + 1, sizeof(*addrs), GFP_KERNEL);
- if (addrs == NULL) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (addrs == NULL)
+ goto out_err;
memset(&cgctx, 0, sizeof(struct codegen_context));
bpf_jit_init_reg_mapping(&cgctx);
@@ -211,11 +195,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
cgctx.exception_cb = fp->aux->exception_cb;
/* Scouting faux-generate pass 0 */
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
/* We hit something illegal or unsupported. */
- fp = org_fp;
- goto out_addrs;
- }
+ goto out_err;
/*
* If we have seen a tail call, we need a second pass.
@@ -226,10 +208,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
+ goto out_err;
}
bpf_jit_realloc_regs(&cgctx);
@@ -250,10 +230,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
bpf_jit_fill_ill_insns);
- if (!fhdr) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (!fhdr)
+ goto out_err;
if (extable_len)
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
@@ -272,8 +250,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
extra_pass)) {
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
bpf_jit_binary_pack_free(fhdr, hdr);
- fp = org_fp;
- goto out_addrs;
+ goto out_err;
}
bpf_jit_build_epilogue(code_base, &cgctx);
@@ -295,15 +272,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
((u64 *)image)[1] = local_paca->kernel_toc;
#endif
+ if (!fp->is_func || extra_pass) {
+ if (bpf_jit_binary_pack_finalize(fhdr, hdr))
+ goto out_err;
+ }
+
fp->bpf_func = (void *)fimage;
fp->jited = 1;
fp->jited_len = cgctx.idx * 4 + FUNCTION_DESCR_SIZE;
if (!fp->is_func || extra_pass) {
- if (bpf_jit_binary_pack_finalize(fhdr, hdr)) {
- fp = org_fp;
- goto out_addrs;
- }
bpf_prog_fill_jited_linfo(fp, addrs);
out_addrs:
kfree(addrs);
@@ -318,11 +296,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
jit_data->hdr = hdr;
}
-out:
- if (bpf_blinded)
- bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);
-
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto out_addrs;
}
/*
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index b3581e926436..527baa50dc68 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -44,29 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, i;
struct rv_jit_data *jit_data;
struct rv_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
if (!jit_data) {
- prog = orig_prog;
- goto out;
+ return prog;
}
prog->aux->jit_data = jit_data;
}
@@ -83,15 +73,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
+ if (!ctx->offset)
goto out_offset;
- }
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_offset;
- }
for (i = 0; i < prog->len; i++) {
prev_ninsns += 32;
@@ -105,10 +91,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
ctx->prologue_len = ctx->ninsns;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, ctx->offset))
goto out_offset;
- }
ctx->epilogue_offset = ctx->ninsns;
bpf_jit_build_epilogue(ctx);
@@ -126,10 +110,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
bpf_fill_ill_insns);
- if (!jit_data->ro_header) {
- prog = orig_prog;
+ if (!jit_data->ro_header)
goto out_offset;
- }
/*
* Use the image(RW) for writing the JITed instructions. But also save
@@ -150,7 +132,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (i == NR_JIT_ITERATIONS) {
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
- prog = orig_prog;
goto out_free_hdr;
}
@@ -163,26 +144,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->nexentries = 0;
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_free_hdr;
- }
bpf_jit_build_epilogue(ctx);
if (bpf_jit_enable > 1)
bpf_jit_dump(prog->len, prog_size, pass, ctx->insns);
- prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
- prog->jited = 1;
- prog->jited_len = prog_size - cfi_get_offset();
-
if (!prog->is_func || extra_pass) {
if (WARN_ON(bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header))) {
/* ro_header has been freed */
jit_data->ro_header = NULL;
- prog = orig_prog;
- goto out_offset;
+ jit_data->header = NULL;
+ goto out_free_hdr;
}
+ }
+
+ prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
+ prog->jited = 1;
+ prog->jited_len = prog_size - cfi_get_offset();
+
+ if (!prog->is_func || extra_pass) {
/*
* The instructions have now been copied to the ROX region from
* where they will execute.
@@ -198,14 +180,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (jit_data->header) {
bpf_arch_text_copy(&jit_data->ro_header->size, &jit_data->header->size,
sizeof(jit_data->header->size));
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index d08d159b6319..2dfc279b1be2 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2314,36 +2314,20 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
*/
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
- struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
- bool tmp_blinded = false;
bool extra_pass = false;
struct bpf_jit jit;
int pass;
if (!fp->jit_requested)
- return orig_fp;
-
- tmp = bpf_jit_blind_constants(fp);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_fp;
- if (tmp != fp) {
- tmp_blinded = true;
- fp = tmp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = orig_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
if (jit_data->ctx.addrs) {
@@ -2356,34 +2340,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
memset(&jit, 0, sizeof(jit));
jit.addrs = kvcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
- if (jit.addrs == NULL) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (jit.addrs == NULL)
+ goto out_err;
/*
* Three initial passes:
* - 1/2: Determine clobbered registers
* - 3: Calculate program size and addrs array
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp, extra_pass)) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (bpf_jit_prog(&jit, fp, extra_pass))
+ goto out_err;
}
/*
* Final pass: Allocate and generate program
*/
header = bpf_jit_alloc(&jit, fp);
- if (!header) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (!header)
+ goto out_err;
skip_init_ctx:
if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
if (bpf_jit_enable > 1) {
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
@@ -2392,8 +2369,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
if (!fp->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
} else {
jit_data->header = header;
@@ -2411,11 +2387,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
kfree(jit_data);
fp->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(fp, fp == orig_fp ?
- tmp : orig_fp);
+
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto free_addrs;
}
bool bpf_jit_supports_kfunc_call(void)
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index b23d1c645ae5..e83e29137566 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1479,37 +1479,22 @@ struct sparc64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
u32 prev_image_size, image_size;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
int pass, i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1527,10 +1512,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kmalloc_array(prog->len, sizeof(unsigned int), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (ctx.offset == NULL)
+ goto out_err;
/* Longest sequence emitted is for bswap32, 12 instructions. Pre-cook
* the offset array so that we converge faster.
@@ -1543,10 +1526,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx = 0;
build_prologue(&ctx);
- if (build_body(&ctx)) {
- prog = orig_prog;
- goto out_off;
- }
+ if (build_body(&ctx))
+ goto out_err;
build_epilogue(&ctx);
if (bpf_jit_enable > 1)
@@ -1569,10 +1550,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = sizeof(u32) * ctx.idx;
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
- if (header == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (header == NULL)
+ goto out_err;
ctx.image = (u32 *)image_ptr;
skip_init_ctx:
@@ -1582,8 +1561,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (build_body(&ctx)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
build_epilogue(&ctx);
@@ -1592,8 +1570,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf_jit: Failed to converge, prev_size=%u size=%d\n",
prev_image_size, ctx.idx * 4);
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
if (bpf_jit_enable > 1)
@@ -1604,8 +1581,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (!prog->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
} else {
jit_data->ctx = ctx;
@@ -1624,9 +1600,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_off;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e9b78040d703..77d00a8dec87 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3717,13 +3717,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
void __percpu *priv_stack_ptr = NULL;
struct x64_jit_data *jit_data;
int priv_stack_alloc_sz;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
bool extra_pass = false;
bool padding = false;
u8 *rw_image = NULL;
@@ -3733,27 +3731,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -3765,10 +3749,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 8) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 8, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -3786,10 +3768,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto skip_init_addrs;
}
addrs = kvmalloc_objs(*addrs, prog->len + 1);
- if (!addrs) {
- prog = orig_prog;
+ if (!addrs)
goto out_addrs;
- }
/*
* Before first pass, make a rough estimation of addrs[]
@@ -3820,8 +3800,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
sizeof(rw_header->size));
bpf_jit_binary_pack_free(header, rw_header);
}
- /* Fall back to interpreter mode */
- prog = orig_prog;
if (extra_pass) {
prog->bpf_func = NULL;
prog->jited = 0;
@@ -3852,10 +3830,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
prog->aux->extable = (void *) image + roundup(proglen, align);
}
oldproglen = proglen;
@@ -3908,8 +3884,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image + cfi_get_offset();
prog->jited = 1;
prog->jited_len = proglen - cfi_get_offset();
- } else {
- prog = orig_prog;
}
if (!image || !prog->is_func || extra_pass) {
@@ -3925,10 +3899,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
}
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index dda423025c3d..5f259577614a 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2521,35 +2521,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
u8 *image = NULL;
int *addrs;
int pass;
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
addrs = kmalloc_objs(*addrs, prog->len);
- if (!addrs) {
- prog = orig_prog;
- goto out;
- }
+ if (!addrs)
+ return prog;
/*
* Before first pass, make a rough estimation of addrs[]
@@ -2574,7 +2558,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image = NULL;
if (header)
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_addrs;
}
if (image) {
@@ -2588,10 +2571,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (proglen == oldproglen) {
header = bpf_jit_binary_alloc(proglen, &image,
1, jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
}
oldproglen = proglen;
cond_resched();
@@ -2604,16 +2585,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image;
prog->jited = 1;
prog->jited_len = proglen;
- } else {
- prog = orig_prog;
}
out_addrs:
kfree(addrs);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
}
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 44d7ae95ddbc..d8c97f463e0f 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1184,6 +1184,18 @@ static inline bool bpf_dump_raw_ok(const struct cred *cred)
struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
const struct bpf_insn *patch, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len);
+#else
+static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+#endif
+
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
static inline bool xdp_return_frame_no_direct(void)
@@ -1310,7 +1322,7 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
const char *bpf_jit_get_prog_name(struct bpf_prog *prog);
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *fp);
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other);
static inline void bpf_jit_dump(unsigned int flen, unsigned int proglen,
@@ -1451,6 +1463,15 @@ static inline void bpf_prog_kallsyms_del(struct bpf_prog *fp)
{
}
+static inline
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
+{
+ return prog;
+}
+
+static inline void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
+{
+}
#endif /* CONFIG_BPF_JIT */
void bpf_prog_kallsyms_del_all(struct bpf_prog *fp);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 67eb12b637a5..8f52f4b6c3af 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1507,13 +1507,16 @@ static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
#endif
}
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
struct bpf_prog *clone, *tmp;
int insn_delta, insn_cnt;
struct bpf_insn *insn;
- int i, rewritten;
+ int i, rewritten, subprog_start;
+
+ if (env)
+ prog = env->prog;
if (!prog->blinding_requested || prog->blinded)
return prog;
@@ -1522,8 +1525,13 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!clone)
return ERR_PTR(-ENOMEM);
+ /* make sure bpf_patch_insn_data() patches the correct prog */
+ if (env)
+ env->prog = clone;
+
insn_cnt = clone->len;
insn = clone->insnsi;
+ subprog_start = prog->aux->subprog_start;
for (i = 0; i < insn_cnt; i++, insn++) {
if (bpf_pseudo_func(insn)) {
@@ -1549,21 +1557,34 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!rewritten)
continue;
- tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
- if (IS_ERR(tmp)) {
+ if (env)
+ tmp = bpf_patch_insn_data(env, subprog_start + i, insn_buff, rewritten);
+ else
+ tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
+
+ if (IS_ERR_OR_NULL(tmp)) {
+ /* restore the original prog */
+ if (env)
+ env->prog = prog;
/* Patching may have repointed aux->prog during
* realloc from the original one, so we need to
* fix it up here on error.
*/
bpf_jit_prog_release_other(prog, clone);
- return tmp;
+ return IS_ERR(tmp) ? tmp : ERR_PTR(-ENOMEM);
}
clone = tmp;
insn_delta = rewritten - 1;
- /* Instructions arrays must be updated using absolute xlated offsets */
- adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+ if (env)
+ env->prog = clone;
+ else
+ /* Instructions arrays must be updated using absolute xlated offsets.
+ * The arrays have already been adjusted by bpf_patch_insn_data() when
+ * env is not NULL.
+ */
+ adjust_insn_arrays(clone, subprog_start + i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2532,6 +2553,35 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+{
+#ifdef CONFIG_BPF_JIT
+ bool blinded = false;
+ struct bpf_prog *orig_prog = prog;
+
+ prog = bpf_jit_blind_constants(NULL, orig_prog);
+ /* If blinding was requested and we failed during blinding, we must fall
+ * back to the interpreter.
+ */
+ if (IS_ERR(prog))
+ return orig_prog;
+
+ if (prog != orig_prog)
+ blinded = true;
+
+ prog = bpf_int_jit_compile(prog);
+ if (blinded) {
+ if (!prog->jited) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ prog = orig_prog;
+ } else {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ }
+ }
+#endif
+ return prog;
+}
+
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
* @fp: bpf_prog populated with BPF program
@@ -2571,7 +2621,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_int_jit_compile(fp);
+ fp = bpf_prog_jit_compile(fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index cd008b146ee5..1b2b99f2d859 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -22105,8 +22105,8 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-static struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -22875,17 +22875,23 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
static int jit_subprogs(struct bpf_verifier_env *env)
{
- struct bpf_prog *prog = env->prog, **func, *tmp;
+ struct bpf_prog *orig_prog = env->prog, *prog, **func, *tmp;
int i, j, subprog_start, subprog_end = 0, len, subprog;
struct bpf_map *map_ptr;
struct bpf_insn *insn;
void *old_bpf_func;
int err, num_exentries;
- int old_len, subprog_start_adjustment = 0;
+ bool blinded = false;
if (env->subprog_cnt <= 1)
return 0;
+ prog = bpf_jit_blind_constants(env, NULL);
+ if (IS_ERR(prog))
+ return -ENOMEM;
+ if (prog != orig_prog)
+ blinded = true;
+
for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
if (!bpf_pseudo_func(insn) && !bpf_pseudo_call(insn))
continue;
@@ -22896,8 +22902,13 @@ static int jit_subprogs(struct bpf_verifier_env *env)
*/
subprog = find_subprog(env, i + insn->imm + 1);
if (verifier_bug_if(subprog < 0, env, "No program to jit at insn %d",
- i + insn->imm + 1))
+ i + insn->imm + 1)) {
+ if (blinded) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ env->prog = orig_prog;
+ }
return -EFAULT;
+ }
/* temporarily remember subprog id inside insn instead of
* aux_data, since next loop will split up all insns into funcs
*/
@@ -22953,10 +22964,11 @@ static int jit_subprogs(struct bpf_verifier_env *env)
goto out_free;
func[i]->is_func = 1;
func[i]->sleepable = prog->sleepable;
+ func[i]->blinded = prog->blinded;
func[i]->aux->func_idx = i;
/* Below members will be freed only at prog->aux */
func[i]->aux->btf = prog->aux->btf;
- func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
+ func[i]->aux->subprog_start = subprog_start;
func[i]->aux->func_info = prog->aux->func_info;
func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -23012,15 +23024,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
-
- /*
- * To properly pass the absolute subprog start to jit
- * all instruction adjustments should be accumulated
- */
- old_len = func[i]->len;
func[i] = bpf_int_jit_compile(func[i]);
- subprog_start_adjustment += func[i]->len - old_len;
-
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -23124,6 +23128,10 @@ static int jit_subprogs(struct bpf_verifier_env *env)
prog->aux->bpf_exception_cb = (void *)func[env->exception_callback_subprog]->bpf_func;
prog->aux->exception_boundary = func[0]->aux->exception_boundary;
bpf_prog_jit_attempt_done(prog);
+
+ if (blinded)
+ bpf_jit_prog_release_other(prog, orig_prog);
+
return 0;
out_free:
/* We failed JIT'ing, so at this point we need to unregister poke
@@ -23146,14 +23154,21 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
kfree(func);
out_undo_insn:
+ if (blinded) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ env->prog = prog = orig_prog;
+ }
/* cleanup main prog to be interpreted */
prog->jit_requested = 0;
prog->blinding_requested = 0;
- for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
- if (!bpf_pseudo_call(insn))
- continue;
- insn->off = 0;
- insn->imm = env->insn_aux_data[i].call_imm;
+ /* we already rolled back to the clean orig_prog when blinded is true */
+ if (!blinded) {
+ for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
+ if (!bpf_pseudo_call(insn))
+ continue;
+ insn->off = 0;
+ insn->imm = env->insn_aux_data[i].call_imm;
+ }
}
bpf_prog_jit_attempt_done(prog);
return err;
--
2.47.3
^ permalink raw reply related
* Re: [PATCH v3 1/6] can: flexcan: use dedicated IRQ handlers for multi-IRQ platforms
From: Marc Kleine-Budde @ 2026-03-24 11:56 UTC (permalink / raw)
To: Ciprian Costea
Cc: Vincent Mailhol, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Frank Li, Sascha Hauer, Fabio Estevam, Pengutronix Kernel Team,
linux-can, devicetree, linux-kernel, imx, linux-arm-kernel,
NXP S32 Linux Team, Christophe Lizzi, Alberto Ruiz,
Enric Balletbo, Eric Chanudet
In-Reply-To: <20260323135827.2129371-2-ciprianmarian.costea@oss.nxp.com>
[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]
On 23.03.2026 14:58:22, Ciprian Costea wrote:
> From: Ciprian Marian Costea <ciprianmarian.costea@oss.nxp.com>
>
> On platforms with multiple IRQ lines (S32G2, MCF5441X), all lines are
> registered to the same flexcan_irq() handler. Since these are distinct IRQ
> numbers, they can be dispatched concurrently on different CPUs. Both
> instances then read the same iflag and ESR registers unconditionally,
> leading to duplicate frame processing.
>
> Fix this by splitting the monolithic handler into focused parts:
> - flexcan_do_mb(): processes mailbox events
> - flexcan_do_state(): processes device state change events
> - flexcan_do_berr(): processes bus error events
>
> Introduce dedicated IRQ handlers for multi-IRQ platforms:
> - flexcan_irq_mb(): mailbox-only, used for mb-0, mb-1 IRQ lines
> - flexcan_irq_boff(): state-change-only, used for boff/state IRQ line
> - flexcan_irq_berr(): bus-error-only, used for berr IRQ line
>
> The combined flexcan_irq() handler is preserved for single-IRQ
> platforms with no functional change.
Thanks for implementing this.
Can you take care of the S32G2 which has 2 mailbox IRQs, too? Please in
a separate patch.
My idea was to take the "irq" argument of the IRQ handler and the quirks
and figure out if you are the first or second mailbox IRQ handler.
Convert these
| struct flexcan_priv {
| [...]
| u64 rx_mask;
| u64 tx_mask;
| [...]
| }
into a struct and put an array of 2 of these structs into "struct
flexcan_priv". Use correct mask array depending on IRQ handler.
regards,
Marc
--
Pengutronix e.K. | Marc Kleine-Budde |
Embedded Linux | https://www.pengutronix.de |
Vertretung Nürnberg | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox