* Re: [PATCH 06/11] Drivers: hv: Make sint vector architecture neutral in MSHV_VTL
From: Naman Jain @ 2026-04-13 11:47 UTC (permalink / raw)
To: Michael Kelley, K . Y . Srinivasan, Haiyang Zhang, Wei Liu,
Dexuan Cui, Long Li, Catalin Marinas, Will Deacon,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
x86@kernel.org, H . Peter Anvin, Arnd Bergmann, Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti
Cc: Marc Zyngier, Timothy Hayes, Lorenzo Pieralisi, mrigendrachaubey,
ssengar@linux.microsoft.com, linux-hyperv@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
linux-riscv@lists.infradead.org
In-Reply-To: <SN6PR02MB4157521DEF9EA2471B6F3359D450A@SN6PR02MB4157.namprd02.prod.outlook.com>
On 4/1/2026 10:27 PM, Michael Kelley wrote:
> From: Naman Jain <namjain@linux.microsoft.com> Sent: Monday, March 16, 2026 5:13 AM
>>
>> Generalize Synthetic interrupt source vector (sint) to use
>> vmbus_interrupt variable instead, which automatically takes care of
>> architectures where HYPERVISOR_CALLBACK_VECTOR is not present (arm64).
>
> Sashiko AI raised an interesting question about the startup timing --
> whether the vmbus_platform_driver_probe() is guaranteed to have
> set vmbus_interrupt before the VTL functions below run and use it.
> What causes the mshv_vtl.ko module to be loaded, and hence run
> mshv_vtl_init()?
There is no race condition here. The init ordering guarantees that
vmbus_interrupt is always set before mshv_vtl_synic_enable_regs()
reads it.
The call chain for setting vmbus_interrupt:
subsys_initcall(hv_acpi_init) [level 4]
-> platform_driver_register(&vmbus_platform_driver) and so on.
The call chain for reading vmbus_interrupt:
module_init(mshv_vtl_init) [level 6]
-> hv_vtl_setup_synic()
-> cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, ...,
mshv_vtl_alloc_context, ...)
-> mshv_vtl_alloc_context()
-> mshv_vtl_synic_enable_regs()
-> sint.vector = vmbus_interrupt
do_initcalls() processes sections in order 0 through 7, so
hv_acpi_init() (level 4) is guaranteed to complete before
mshv_vtl_init() (level 6) runs.
Regarding memory leak on cpu offline/online or module load/unload- it is
beyond the scope of this series, I will fix it separately.
I may need some more time in addressing comments on rest of the patches.
Please bear with me.
Regards,
Naman
>
>>
>> Signed-off-by: Roman Kisel <romank@linux.microsoft.com>
>> Signed-off-by: Naman Jain <namjain@linux.microsoft.com>
>> ---
>> drivers/hv/mshv_vtl_main.c | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/hv/mshv_vtl_main.c b/drivers/hv/mshv_vtl_main.c
>> index b607b6e7e121..91517b45d526 100644
>> --- a/drivers/hv/mshv_vtl_main.c
>> +++ b/drivers/hv/mshv_vtl_main.c
>> @@ -234,7 +234,7 @@ static void mshv_vtl_synic_enable_regs(unsigned int cpu)
>> union hv_synic_sint sint;
>>
>> sint.as_uint64 = 0;
>> - sint.vector = HYPERVISOR_CALLBACK_VECTOR;
>> + sint.vector = vmbus_interrupt;
>> sint.masked = false;
>> sint.auto_eoi = hv_recommend_using_aeoi();
>>
>> @@ -753,7 +753,7 @@ static void mshv_vtl_synic_mask_vmbus_sint(void *info)
>> const u8 *mask = info;
>>
>> sint.as_uint64 = 0;
>> - sint.vector = HYPERVISOR_CALLBACK_VECTOR;
>> + sint.vector = vmbus_interrupt;
>> sint.masked = (*mask != 0);
>> sint.auto_eoi = hv_recommend_using_aeoi();
>>
>> --
>> 2.43.0
>>
>
> Assuming there's no timing problem vs. the VMBus driver,
>
> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
^ permalink raw reply
* Re: [PATCH v2 1/3] arm64: mm: Fix rodata=full block mapping support for realm guests
From: Kevin Brodsky @ 2026-04-13 11:47 UTC (permalink / raw)
To: Ryan Roberts, Catalin Marinas, Will Deacon,
David Hildenbrand (Arm), Dev Jain, Yang Shi, Suzuki K Poulose,
Jinjiang Tu
Cc: linux-arm-kernel, linux-kernel, stable
In-Reply-To: <20260330161705.3349825-2-ryan.roberts@arm.com>
On 30/03/2026 18:17, Ryan Roberts wrote:
> +bool page_alloc_available __ro_after_init;
> +
> +void __init mem_init(void)
> +{
> + page_alloc_available = true;
I knew I had seen a simpler solution to this: vmemmap_alloc_block() uses
slab_is_available() to tell whether the buddy allocator is available.
AFAICT this becomes true somewhere in kmem_cache_init(), which is called
just after mem_init(). Probably good enough for our purpose?
- Kevin
> + swiotlb_update_mem_attributes();
> +}
^ permalink raw reply
* Re: [PATCH] drm/bridge: stm_lvds: Do not fail atomic_check on disabled connector
From: Raphael Gallais-Pou @ 2026-04-13 11:48 UTC (permalink / raw)
To: dri-devel, Marek Vasut
Cc: Alexandre Torgue, David Airlie, Maarten Lankhorst,
Maxime Coquelin, Maxime Ripard, Philippe Cornu, Simona Vetter,
Thomas Zimmermann, Yannick Fertre, linux-arm-kernel, linux-kernel,
linux-stm32
In-Reply-To: <20260409024928.344010-1-marex@nabladev.com>
On Thu, 09 Apr 2026 04:48:41 +0200, Marek Vasut wrote:
> If the connector is disabled, the new connector state has .crtc field
> set to NULL and there is nothing more to validate after that point.
> The .crtc field being NULL is not an error. Test for .crtc being NULL,
> and if it is NULL, exit early with return 0.
>
> This fixes a failure in suspend/resume path, where the connector is
> already disabled, but .atomic_check is called, fails, returns -EINVAL
> and blocks the suspend entry.
>
> [...]
Applied, thanks!
[1/1] drm/bridge: stm_lvds: Do not fail atomic_check on disabled connector
commit: eecdd4bd6e47bf0c8ff1e049771fa5bab7074c7c
Best regards,
--
Raphael Gallais-Pou <raphael.gallais-pou@foss.st.com>
^ permalink raw reply
* Re: [PATCH v3 2/4] mm: use tiered folio allocation for VM_EXEC readahead
From: Usama Arif @ 2026-04-13 11:48 UTC (permalink / raw)
To: Jan Kara
Cc: Andrew Morton, david, willy, ryan.roberts, linux-mm, r, ajd,
apopple, baohua, baolin.wang, brauner, catalin.marinas, dev.jain,
kees, kevin.brodsky, lance.yang, Liam.Howlett, linux-arm-kernel,
linux-fsdevel, linux-kernel, Lorenzo Stoakes, mhocko, npache,
pasha.tatashin, rmclure, rppt, surenb, vbabka, Al Viro,
wilts.infradead.org, ziy, hannes, kas, shakeel.butt, leitao,
kernel-team
In-Reply-To: <aji7zs42th272khtxesk6dfcrgf7ddr5r5n62wgzeqooyexgxf@5ns3i47f5nlg>
On 13/04/2026 12:03, Jan Kara wrote:
> On Thu 02-04-26 11:08:23, Usama Arif wrote:
>> When executable pages are faulted via do_sync_mmap_readahead(), request
>> a folio order that enables the best hardware TLB coalescing available:
>>
>> - If the VMA is large enough to contain a full PMD, request
>> HPAGE_PMD_ORDER so the folio can be PMD-mapped. This benefits
>> architectures where PMD_SIZE is reasonable (e.g. 2M on x86-64
>> and arm64 with 4K pages). VM_EXEC VMAs are very unlikely to be
>> large enough for 512M pages on ARM to take into affect.
>
> I'm not sure relying on PMD_SIZE will be too much for a VMA is a great
> strategy. With 16k PAGE_SIZE the PMD would be 32MB large which would fit in
> the .text size but already looks a bit too much? Mapping with PMD sized
> folios brings some benefits but at the same time it costs because now parts
> of VMA that would be never paged in are pulled into memory and also LRU
> tracking now happens with this very large granularity making it fairly
> inefficient (big folios have much higher chances of getting accessed
> similarly often making LRU order mostly random). We are already getting
> reports of people with small machines (phones etc.) where the memory
> overhead of large folios (in the page cache) is simply too much. So I'd
> have a bigger peace of mind if we capped folio size at 2MB for now until we
> come with a more sophisticated heuristic of picking sensible folio order
> given the machine size. Now I'm not really an MM person so my feeling here
> may be just wrong but I wanted to voice this concern from what I can see...
>
> Honza
>
>
Thanks for the feedback! I agree, it makes sense. I did that in the previous
revision [1]. I will reinistante that in the next one.
[1] https://lore.kernel.org/all/20260320140315.979307-3-usama.arif@linux.dev/
>> - Otherwise, fall back to exec_folio_order(), which returns the
>> minimum order for hardware PTE coalescing for arm64:
>> - arm64 4K: order 4 (64K) for contpte (16 PTEs → 1 iTLB entry)
>> - arm64 16K: order 2 (64K) for HPA (4 pages → 1 TLB entry)
>> - arm64 64K: order 5 (2M) for contpte (32 PTEs → 1 iTLB entry)
>> - generic: order 0 (no coalescing)
>>
>> Update the arm64 exec_folio_order() to return ilog2(SZ_2M >>
>> PAGE_SHIFT) on 64K page configurations, where the previous SZ_64K
>> value collapsed to order 0 (a single page) and provided no coalescing
>> benefit.
>>
>> Use ~__GFP_RECLAIM so the allocation is opportunistic: if a large
>> folio is readily available, use it, otherwise fall back to smaller
>> folios without stalling on reclaim or compaction. The existing fallback
>> in page_cache_ra_order() handles this naturally.
>>
>> The readahead window is already clamped to the VMA boundaries, so
>> ra->size naturally caps the folio order via ilog2(ra->size) in
>> page_cache_ra_order().
>>
>> Signed-off-by: Usama Arif <usama.arif@linux.dev>
>> ---
>> arch/arm64/include/asm/pgtable.h | 16 +++++++++----
>> mm/filemap.c | 40 +++++++++++++++++++++++---------
>> mm/internal.h | 3 ++-
>> mm/readahead.c | 7 +++---
>> 4 files changed, 45 insertions(+), 21 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
>> index 52bafe79c10a..9ce9f73a6f35 100644
>> --- a/arch/arm64/include/asm/pgtable.h
>> +++ b/arch/arm64/include/asm/pgtable.h
>> @@ -1591,12 +1591,18 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
>> #define arch_wants_old_prefaulted_pte cpu_has_hw_af
>>
>> /*
>> - * Request exec memory is read into pagecache in at least 64K folios. This size
>> - * can be contpte-mapped when 4K base pages are in use (16 pages into 1 iTLB
>> - * entry), and HPA can coalesce it (4 pages into 1 TLB entry) when 16K base
>> - * pages are in use.
>> + * Request exec memory is read into pagecache in folios large enough for
>> + * hardware TLB coalescing. On 4K and 16K page configs this is 64K, which
>> + * enables contpte mapping (16 × 4K) and HPA coalescing (4 × 16K). On
>> + * 64K page configs, contpte requires 2M (32 × 64K).
>> */
>> -#define exec_folio_order() ilog2(SZ_64K >> PAGE_SHIFT)
>> +#define exec_folio_order exec_folio_order
>> +static inline unsigned int exec_folio_order(void)
>> +{
>> + if (PAGE_SIZE == SZ_64K)
>> + return ilog2(SZ_2M >> PAGE_SHIFT);
>> + return ilog2(SZ_64K >> PAGE_SHIFT);
>> +}
>>
>> static inline bool pud_sect_supported(void)
>> {
>> diff --git a/mm/filemap.c b/mm/filemap.c
>> index a4ea869b2ca1..7ffea986b3b4 100644
>> --- a/mm/filemap.c
>> +++ b/mm/filemap.c
>> @@ -3311,6 +3311,7 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
>> DEFINE_READAHEAD(ractl, file, ra, mapping, vmf->pgoff);
>> struct file *fpin = NULL;
>> vm_flags_t vm_flags = vmf->vma->vm_flags;
>> + gfp_t gfp = readahead_gfp_mask(mapping);
>> bool force_thp_readahead = false;
>> unsigned short mmap_miss;
>>
>> @@ -3363,28 +3364,45 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
>> ra->size *= 2;
>> ra->async_size = HPAGE_PMD_NR;
>> ra->order = HPAGE_PMD_ORDER;
>> - page_cache_ra_order(&ractl, ra);
>> + page_cache_ra_order(&ractl, ra, gfp);
>> return fpin;
>> }
>>
>> if (vm_flags & VM_EXEC) {
>> /*
>> - * Allow arch to request a preferred minimum folio order for
>> - * executable memory. This can often be beneficial to
>> - * performance if (e.g.) arm64 can contpte-map the folio.
>> - * Executable memory rarely benefits from readahead, due to its
>> - * random access nature, so set async_size to 0.
>> + * Request large folios for executable memory to enable
>> + * hardware PTE coalescing and PMD mappings:
>> *
>> - * Limit to the boundaries of the VMA to avoid reading in any
>> - * pad that might exist between sections, which would be a waste
>> - * of memory.
>> + * - If the VMA is large enough for a PMD, request
>> + * HPAGE_PMD_ORDER so the folio can be PMD-mapped.
>> + * - Otherwise, use exec_folio_order() which returns
>> + * the minimum order for hardware TLB coalescing
>> + * (e.g. arm64 contpte/HPA).
>> + *
>> + * Use ~__GFP_RECLAIM so large folio allocation is
>> + * opportunistic — if memory isn't readily available,
>> + * fall back to smaller folios rather than stalling on
>> + * reclaim or compaction.
>> + *
>> + * Executable memory rarely benefits from speculative
>> + * readahead due to its random access nature, so set
>> + * async_size to 0.
>> + *
>> + * Limit to the boundaries of the VMA to avoid reading
>> + * in any pad that might exist between sections, which
>> + * would be a waste of memory.
>> */
>> + gfp &= ~__GFP_RECLAIM;
>> struct vm_area_struct *vma = vmf->vma;
>> unsigned long start = vma->vm_pgoff;
>> unsigned long end = start + vma_pages(vma);
>> unsigned long ra_end;
>>
>> - ra->order = exec_folio_order();
>> + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
>> + vma_pages(vma) >= HPAGE_PMD_NR)
>> + ra->order = HPAGE_PMD_ORDER;
>> + else
>> + ra->order = exec_folio_order();
>> ra->start = round_down(vmf->pgoff, 1UL << ra->order);
>> ra->start = max(ra->start, start);
>> ra_end = round_up(ra->start + ra->ra_pages, 1UL << ra->order);
>> @@ -3403,7 +3421,7 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
>>
>> fpin = maybe_unlock_mmap_for_io(vmf, fpin);
>> ractl._index = ra->start;
>> - page_cache_ra_order(&ractl, ra);
>> + page_cache_ra_order(&ractl, ra, gfp);
>> return fpin;
>> }
>>
>> diff --git a/mm/internal.h b/mm/internal.h
>> index 475bd281a10d..e624cb619057 100644
>> --- a/mm/internal.h
>> +++ b/mm/internal.h
>> @@ -545,7 +545,8 @@ int zap_vma_for_reaping(struct vm_area_struct *vma);
>> int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio,
>> gfp_t gfp);
>>
>> -void page_cache_ra_order(struct readahead_control *, struct file_ra_state *);
>> +void page_cache_ra_order(struct readahead_control *, struct file_ra_state *,
>> + gfp_t gfp);
>> void force_page_cache_ra(struct readahead_control *, unsigned long nr);
>> static inline void force_page_cache_readahead(struct address_space *mapping,
>> struct file *file, pgoff_t index, unsigned long nr_to_read)
>> diff --git a/mm/readahead.c b/mm/readahead.c
>> index 7b05082c89ea..b3dc08cf180c 100644
>> --- a/mm/readahead.c
>> +++ b/mm/readahead.c
>> @@ -465,7 +465,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index,
>> }
>>
>> void page_cache_ra_order(struct readahead_control *ractl,
>> - struct file_ra_state *ra)
>> + struct file_ra_state *ra, gfp_t gfp)
>> {
>> struct address_space *mapping = ractl->mapping;
>> pgoff_t start = readahead_index(ractl);
>> @@ -475,7 +475,6 @@ void page_cache_ra_order(struct readahead_control *ractl,
>> pgoff_t mark = index + ra->size - ra->async_size;
>> unsigned int nofs;
>> int err = 0;
>> - gfp_t gfp = readahead_gfp_mask(mapping);
>> unsigned int new_order = ra->order;
>>
>> trace_page_cache_ra_order(mapping->host, start, ra);
>> @@ -626,7 +625,7 @@ void page_cache_sync_ra(struct readahead_control *ractl,
>> readit:
>> ra->order = 0;
>> ractl->_index = ra->start;
>> - page_cache_ra_order(ractl, ra);
>> + page_cache_ra_order(ractl, ra, readahead_gfp_mask(ractl->mapping));
>> }
>> EXPORT_SYMBOL_GPL(page_cache_sync_ra);
>>
>> @@ -697,7 +696,7 @@ void page_cache_async_ra(struct readahead_control *ractl,
>> ra->size -= end - aligned_end;
>> ra->async_size = ra->size;
>> ractl->_index = ra->start;
>> - page_cache_ra_order(ractl, ra);
>> + page_cache_ra_order(ractl, ra, readahead_gfp_mask(ractl->mapping));
>> }
>> EXPORT_SYMBOL_GPL(page_cache_async_ra);
>>
>> --
>> 2.52.0
>>
^ permalink raw reply
* Re: [PATCH RFC 00/12] Add support for DisplayPort link training information report
From: Kory Maincent @ 2026-04-13 12:10 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: Ville Syrjälä, Jani Nikula, Rodrigo Vivi,
Joonas Lahtinen, Tvrtko Ursulin, David Airlie, Simona Vetter,
Dave Airlie, Jesse Barnes, Eric Anholt, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Andrzej Hajda, Neil Armstrong,
Robert Foss, Laurent Pinchart, Jonas Karlman, Jernej Skrabec,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Chris Wilson, Thomas Petazzoni,
Mark Yacoub, Sean Paul, Louis Chauvet, intel-gfx, intel-xe,
dri-devel, linux-kernel, linux-mediatek, linux-arm-kernel,
Simona Vetter
In-Reply-To: <u4ononk4cpccx77gvlywtfen5rmyslvr72v7olkhdrjf65aqce@xo777vofhcan>
On Fri, 10 Apr 2026 00:36:09 +0300
Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> wrote:
> On Thu, Apr 09, 2026 at 11:36:21PM +0300, Ville Syrjälä wrote:
> > On Thu, Apr 09, 2026 at 07:08:16PM +0200, Kory Maincent wrote:
> > > DisplayPort link training negotiates the physical-layer parameters needed
> > > for a reliable connection: lane count, link rate, voltage swing,
> > > pre-emphasis, and optionally Display Stream Compression (DSC). Currently,
> > > each driver exposes this state in its own way, often through
> > > driver-specific debugfs entries, with no standard interface for userspace
> > > diagnostic and monitoring tools.
> > >
> > > This series introduces a generic, DRM-managed framework for exposing DP
> > > link training state as standard connector properties, modeled after the
> > > existing HDMI helper drmm_connector_hdmi_init().
> > >
> > > The new drmm_connector_dp_init() helper initializes a DP connector and
> > > registers the following connector properties to expose the negotiated link
> > > state to userspace:
> > >
> > > - num_lanes: negotiated lane count (1, 2 or 4)
> > > - link_rate: negotiated link rate
> > > - dsc_en: whether Display Stream Compression is active
> > > - voltage_swingN: per-lane voltage swing level (lanes 0-3)
> > > - pre_emphasisN: per-lane pre-emphasis level (lanes 0-3)
> >
> > I don't see why any real userspace would be interested in those (apart
> > from maybe DSC). If this is just for diagnostics and whatnot then I
> > think sysfs/debugfs could be a better fit.
>
> I'd agree here. Please consider implementing it as a debugfs interface,
> possibly reusing the Intel's format.
Sorry, I completely forgot to include a paragraph explaining the rationale
behind using DRM properties.
This DisplayPort link information report was requested by OSes to allow them to
assess the capabilities of each DisplayPort connector on the system, and to
guide users from the most to least capable ones. It will also enable the OS to
warn the user when a cable is too long or experiencing noise (indicated by high
voltage swing and pre-emphasis levels).
Since this is information that OSes will consume on a regular basis, exposing
it directly as DRM properties seems the most appropriate approach.
Regards,
--
Köry Maincent, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
^ permalink raw reply
* [PATCH v4 2/2] usb: mtu3: add support remote wakeup of mt8196
From: Chunfeng Yun @ 2026-04-13 12:17 UTC (permalink / raw)
To: Greg Kroah-Hartman, AngeloGioacchino Del Regno
Cc: Chunfeng Yun, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Matthias Brugger, linux-usb, linux-arm-kernel, linux-mediatek,
devicetree, linux-kernel
In-Reply-To: <20260413121727.4702-1-chunfeng.yun@mediatek.com>
There are three USB controllers on mt8196, each controller's wakeup
control is different, add some specific versions for them.
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
Signed-off-by: Chunfeng Yun <chunfeng.yun@mediatek.com>
---
v4: add reviewed-by
v3: add the ommitted third dual-role controller add acked by Conor
v2: new patch for dual-role controllers
---
drivers/usb/mtu3/mtu3_host.c | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/drivers/usb/mtu3/mtu3_host.c b/drivers/usb/mtu3/mtu3_host.c
index 7c657ea2dabd..8138b3f3096a 100644
--- a/drivers/usb/mtu3/mtu3_host.c
+++ b/drivers/usb/mtu3/mtu3_host.c
@@ -46,6 +46,14 @@
#define WC1_IS_P_95 BIT(12)
#define WC1_IS_EN_P0_95 BIT(6)
+/* mt8196 */
+#define PERI_WK_CTRL0_8196 0x08
+#define WC0_IS_EN_P0_96 BIT(0)
+#define WC0_IS_EN_P1_96 BIT(7)
+
+#define PERI_WK_CTRL1_8196 0x10
+#define WC1_IS_EN_P2_96 BIT(0)
+
/* mt2712 etc */
#define PERI_SSUSB_SPM_CTRL 0x0
#define SSC_IP_SLEEP_EN BIT(4)
@@ -59,6 +67,9 @@ enum ssusb_uwk_vers {
SSUSB_UWK_V1_3, /* mt8195 IP0 */
SSUSB_UWK_V1_5 = 105, /* mt8195 IP2 */
SSUSB_UWK_V1_6, /* mt8195 IP3 */
+ SSUSB_UWK_V1_7, /* mt8196 IP0 */
+ SSUSB_UWK_V1_8, /* mt8196 IP1 */
+ SSUSB_UWK_V1_9, /* mt8196 IP2 */
};
/*
@@ -100,6 +111,21 @@ static void ssusb_wakeup_ip_sleep_set(struct ssusb_mtk *ssusb, bool enable)
msk = WC0_IS_EN_P3_95 | WC0_IS_C_95(0x7) | WC0_IS_P_95;
val = enable ? (WC0_IS_EN_P3_95 | WC0_IS_C_95(0x1)) : 0;
break;
+ case SSUSB_UWK_V1_7:
+ reg = ssusb->uwk_reg_base + PERI_WK_CTRL0_8196;
+ msk = WC0_IS_EN_P0_96;
+ val = enable ? msk : 0;
+ break;
+ case SSUSB_UWK_V1_8:
+ reg = ssusb->uwk_reg_base + PERI_WK_CTRL0_8196;
+ msk = WC0_IS_EN_P1_96;
+ val = enable ? msk : 0;
+ break;
+ case SSUSB_UWK_V1_9:
+ reg = ssusb->uwk_reg_base + PERI_WK_CTRL1_8196;
+ msk = WC1_IS_EN_P2_96;
+ val = enable ? msk : 0;
+ break;
case SSUSB_UWK_V2:
reg = ssusb->uwk_reg_base + PERI_SSUSB_SPM_CTRL;
msk = SSC_IP_SLEEP_EN | SSC_SPM_INT_EN;
--
2.45.2
^ permalink raw reply related
* [PATCH v4 1/2] dt-bindings: usb: mtu3: add support mt8196
From: Chunfeng Yun @ 2026-04-13 12:17 UTC (permalink / raw)
To: Greg Kroah-Hartman, AngeloGioacchino Del Regno
Cc: Chunfeng Yun, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Matthias Brugger, linux-usb, linux-arm-kernel, linux-mediatek,
devicetree, linux-kernel, Conor Dooley
There are three USB controllers on mt8196, each controller's wakeup
control is different, add some specific versions for them, and add
compatilbe for mt8196.
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
Signed-off-by: Chunfeng Yun <chunfeng.yun@mediatek.com>
---
v4: add reviewed-by
v3: add the ommitted third dual-role controller suggested by Angelo
v2: add wakeup for dual-role controllers
---
Documentation/devicetree/bindings/usb/mediatek,mtu3.yaml | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/Documentation/devicetree/bindings/usb/mediatek,mtu3.yaml b/Documentation/devicetree/bindings/usb/mediatek,mtu3.yaml
index 21fc6bbe954f..d148e938d647 100644
--- a/Documentation/devicetree/bindings/usb/mediatek,mtu3.yaml
+++ b/Documentation/devicetree/bindings/usb/mediatek,mtu3.yaml
@@ -28,6 +28,7 @@ properties:
- mediatek,mt8188-mtu3
- mediatek,mt8192-mtu3
- mediatek,mt8195-mtu3
+ - mediatek,mt8196-mtu3
- mediatek,mt8365-mtu3
- const: mediatek,mtu3
@@ -200,7 +201,10 @@ properties:
103 - used by mt8195, IP0, specific 1.03;
105 - used by mt8195, IP2, specific 1.05;
106 - used by mt8195, IP3, specific 1.06;
- enum: [1, 2, 101, 102, 103, 105, 106]
+ 107 - used by mt8196, IP0, specific 1.07;
+ 108 - used by mt8196, IP1, specific 1.08;
+ 109 - used by mt8196, IP2, specific 1.09;
+ enum: [1, 2, 101, 102, 103, 105, 106, 107, 108, 109]
mediatek,u3p-dis-msk:
$ref: /schemas/types.yaml#/definitions/uint32
--
2.45.2
^ permalink raw reply related
* [PATCH V2 RESEND 2/2] phy: mediatek: xsphy: add support to set disconnect threshold
From: Chunfeng Yun @ 2026-04-13 12:28 UTC (permalink / raw)
To: Vinod Koul, AngeloGioacchino Del Regno
Cc: Chunfeng Yun, Neil Armstrong, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, linux-arm-kernel, linux-mediatek,
linux-phy, devicetree, linux-kernel
In-Reply-To: <20260413122836.4848-1-chunfeng.yun@mediatek.com>
Add a property to tune usb2 phy's disconnect threshold.
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
Signed-off-by: Chunfeng Yun <chunfeng.yun@mediatek.com>
---
v2: change property name
---
drivers/phy/mediatek/phy-mtk-xsphy.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/phy/mediatek/phy-mtk-xsphy.c b/drivers/phy/mediatek/phy-mtk-xsphy.c
index c0ddb9273cc3..46345e4f4189 100644
--- a/drivers/phy/mediatek/phy-mtk-xsphy.c
+++ b/drivers/phy/mediatek/phy-mtk-xsphy.c
@@ -61,6 +61,7 @@
#define XSP_USBPHYACR6 ((SSUSB_SIFSLV_U2PHY_COM) + 0x018)
#define P2A6_RG_BC11_SW_EN BIT(23)
#define P2A6_RG_OTG_VBUSCMP_EN BIT(20)
+#define PA6_RG_U2_DISCTH GENMASK(7, 4)
#define XSP_U2PHYDTM1 ((SSUSB_SIFSLV_U2PHY_COM) + 0x06C)
#define P2D_FORCE_IDDIG BIT(9)
@@ -107,6 +108,7 @@ struct xsphy_instance {
int eye_src;
int eye_vrt;
int eye_term;
+ int discth;
};
struct mtk_xsphy {
@@ -256,9 +258,12 @@ static void phy_parse_property(struct mtk_xsphy *xsphy,
&inst->eye_vrt);
device_property_read_u32(dev, "mediatek,eye-term",
&inst->eye_term);
- dev_dbg(dev, "intr:%d, src:%d, vrt:%d, term:%d\n",
+ device_property_read_u32(dev, "mediatek,discth",
+ &inst->discth);
+ dev_dbg(dev, "intr:%d, src:%d, vrt:%d, term:%d, discth:%d\n",
inst->efuse_intr, inst->eye_src,
- inst->eye_vrt, inst->eye_term);
+ inst->eye_vrt, inst->eye_term,
+ inst->discth);
break;
case PHY_TYPE_USB3:
device_property_read_u32(dev, "mediatek,efuse-intr",
@@ -301,6 +306,9 @@ static void u2_phy_props_set(struct mtk_xsphy *xsphy,
if (inst->eye_term)
mtk_phy_update_field(pbase + XSP_USBPHYACR1, P2A1_RG_TERM_SEL,
inst->eye_term);
+ if (inst->discth)
+ mtk_phy_update_field(pbase + XSP_USBPHYACR6, PA6_RG_U2_DISCTH,
+ inst->discth);
}
static void u3_phy_props_set(struct mtk_xsphy *xsphy,
--
2.45.2
^ permalink raw reply related
* [PATCH v2 RESEND 1/2] dt-bindings: phy: mediatek,xsphy: add property to set disconnect threshold
From: Chunfeng Yun @ 2026-04-13 12:28 UTC (permalink / raw)
To: Vinod Koul, AngeloGioacchino Del Regno
Cc: Chunfeng Yun, Neil Armstrong, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, linux-arm-kernel, linux-mediatek,
linux-phy, devicetree, linux-kernel
Add a property to tune usb2 phy's disconnect threshold.
And add a compatible for mt8196.
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
Signed-off-by: Chunfeng Yun <chunfeng.yun@mediatek.com>
---
v2: change property name
---
Documentation/devicetree/bindings/phy/mediatek,xsphy.yaml | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/devicetree/bindings/phy/mediatek,xsphy.yaml b/Documentation/devicetree/bindings/phy/mediatek,xsphy.yaml
index 0bed847bb4ad..9017a9c93eb9 100644
--- a/Documentation/devicetree/bindings/phy/mediatek,xsphy.yaml
+++ b/Documentation/devicetree/bindings/phy/mediatek,xsphy.yaml
@@ -50,6 +50,7 @@ properties:
- mediatek,mt3611-xsphy
- mediatek,mt3612-xsphy
- mediatek,mt7988-xsphy
+ - mediatek,mt8196-xsphy
- const: mediatek,xsphy
reg:
@@ -130,6 +131,13 @@ patternProperties:
minimum: 1
maximum: 7
+ mediatek,disconnect-threshold:
+ description:
+ The selection of disconnect threshold (U2 phy)
+ $ref: /schemas/types.yaml#/definitions/uint32
+ minimum: 1
+ maximum: 15
+
mediatek,efuse-intr:
description:
The selection of Internal Resistor (U2/U3 phy)
--
2.45.2
^ permalink raw reply related
* Re: [PATCH] clk: bcm: rpi: Mark VEC clock as CLK_IGNORE_UNUSED
From: Maíra Canal @ 2026-04-13 12:29 UTC (permalink / raw)
To: Michael Turquette, Stephen Boyd, Florian Fainelli,
Broadcom internal kernel review list, Mark Brown, Maxime Ripard,
Stefan Wahren, Dom Cobley, Dave Stevenson
Cc: linux-clk, linux-rpi-kernel, linux-arm-kernel, kernel-dev
In-Reply-To: <20260401111416.562279-2-mcanal@igalia.com>
Hi Stephen,
It would be great to land this patch in the next release together with
commit 672299736af6 ("clk: bcm: rpi: Manage clock rate in prepare/
unprepare callbacks"). When possible, could you take a look at it?
Best regards,
- Maíra
On 4/1/26 08:13, Maíra Canal wrote:
> On Raspberry Pi 3B, the VEC clock is used by the VideoCore firmware
> display driver, which remains active until the vc4 driver loads and
> sends NOTIFY_DISPLAY_DONE. If this clock is disabled during boot, a bus
> lockup happens and the firmware becomes unresponsive, causing a complete
> system lockup.
>
> Mark the VEC clock with CLK_IGNORE_UNUSED so it survives the unused
> clock disablement and remains available until the vc4 driver takes over
> display management.
>
> Fixes: 672299736af6 ("clk: bcm: rpi: Manage clock rate in prepare/unprepare callbacks")
> Reported-by: Mark Brown <broonie@kernel.org>
> Signed-off-by: Maíra Canal <mcanal@igalia.com>
> ---
> drivers/clk/bcm/clk-raspberrypi.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/clk/bcm/clk-raspberrypi.c b/drivers/clk/bcm/clk-raspberrypi.c
> index df2d246eb6ef..f1a99de6de4f 100644
> --- a/drivers/clk/bcm/clk-raspberrypi.c
> +++ b/drivers/clk/bcm/clk-raspberrypi.c
> @@ -160,6 +160,13 @@ raspberrypi_clk_variants[RPI_FIRMWARE_NUM_CLK_ID] = {
> [RPI_FIRMWARE_VEC_CLK_ID] = {
> .export = true,
> .minimize = true,
> +
> + /*
> + * If this clock is disabled during boot, it causes a bus
> + * lockup in RPi 3B. Therefore, make sure it's left enabled
> + * during boot.
> + */
> + .flags = CLK_IGNORE_UNUSED,
> },
> [RPI_FIRMWARE_DISP_CLK_ID] = {
> .export = true,
^ permalink raw reply
* [PATCH bpf-next 0/2] bpf, arm64/riscv: Remove redundant icache flush after pack allocator finalize
From: Puranjay Mohan @ 2026-04-13 12:32 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman,
Kumar Kartikeya Dwivedi, Song Liu, Yonghong Song, Jiri Olsa,
Xu Kuohai, Catalin Marinas, Will Deacon, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, linux-arm-kernel, linux-riscv,
linux-kernel
When the BPF prog pack allocator was added for arm64 and riscv, the
existing bpf_flush_icache() calls were retained after
bpf_jit_binary_pack_finalize(). However, the finalize path copies the
JITed code via architecture text patching routines (__text_poke on arm64,
patch_text_nosync on riscv) that already perform a full
flush_icache_range() internally. The subsequent bpf_flush_icache()
repeats the same cache maintenance on the same range.
Remove the redundant flush and the now-unused bpf_flush_icache()
definitions on both architectures.
Puranjay Mohan (2):
bpf, arm64: Remove redundant bpf_flush_icache() after pack allocator
finalize
bpf, riscv: Remove redundant bpf_flush_icache() after pack allocator
finalize
arch/arm64/net/bpf_jit_comp.c | 11 -----------
arch/riscv/net/bpf_jit.h | 5 -----
arch/riscv/net/bpf_jit_core.c | 7 -------
3 files changed, 23 deletions(-)
base-commit: 71b500afd2f7336f5b6c6026f2af546fc079be26
--
2.52.0
^ permalink raw reply
* [PATCH bpf-next 2/2] bpf, riscv: Remove redundant bpf_flush_icache() after pack allocator finalize
From: Puranjay Mohan @ 2026-04-13 12:32 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman,
Kumar Kartikeya Dwivedi, Song Liu, Yonghong Song, Jiri Olsa,
Xu Kuohai, Catalin Marinas, Will Deacon, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, linux-arm-kernel, linux-riscv,
linux-kernel
In-Reply-To: <20260413123256.3296452-1-puranjay@kernel.org>
bpf_flush_icache() calls flush_icache_range() to clean the data cache
and invalidate the instruction cache for the JITed code region. However,
since commit 48a8f78c50bd ("bpf, riscv: use prog pack allocator in the
BPF JIT"), this flush is redundant.
bpf_jit_binary_pack_finalize() copies the JITed instructions to the ROX
region via bpf_arch_text_copy() -> patch_text_nosync(), and
patch_text_nosync() already calls flush_icache_range() on the written
range. The subsequent bpf_flush_icache() repeats the same cache
maintenance on an overlapping range.
Remove the redundant bpf_flush_icache() call and its now-unused
definition.
Fixes: 48a8f78c50bd ("bpf, riscv: use prog pack allocator in the BPF JIT")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
arch/riscv/net/bpf_jit.h | 5 -----
arch/riscv/net/bpf_jit_core.c | 7 -------
2 files changed, 12 deletions(-)
diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h
index 632ced07bca4..549537cad86b 100644
--- a/arch/riscv/net/bpf_jit.h
+++ b/arch/riscv/net/bpf_jit.h
@@ -105,11 +105,6 @@ static inline void bpf_fill_ill_insns(void *area, unsigned int size)
memset(area, 0, size);
}
-static inline void bpf_flush_icache(void *start, void *end)
-{
- flush_icache_range((unsigned long)start, (unsigned long)end);
-}
-
/* Emit a 4-byte riscv instruction. */
static inline void emit(const u32 insn, struct rv_jit_context *ctx)
{
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index b3581e926436..f7fd4afc3ca3 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -183,13 +183,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog = orig_prog;
goto out_offset;
}
- /*
- * The instructions have now been copied to the ROX region from
- * where they will execute.
- * Write any modified data cache blocks out to memory and
- * invalidate the corresponding blocks in the instruction cache.
- */
- bpf_flush_icache(jit_data->ro_header, ctx->ro_insns + ctx->ninsns);
for (i = 0; i < prog->len; i++)
ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
bpf_prog_fill_jited_linfo(prog, ctx->offset);
--
2.52.0
^ permalink raw reply related
* [PATCH bpf-next 1/2] bpf, arm64: Remove redundant bpf_flush_icache() after pack allocator finalize
From: Puranjay Mohan @ 2026-04-13 12:32 UTC (permalink / raw)
To: bpf
Cc: Puranjay Mohan, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman,
Kumar Kartikeya Dwivedi, Song Liu, Yonghong Song, Jiri Olsa,
Xu Kuohai, Catalin Marinas, Will Deacon, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, linux-arm-kernel, linux-riscv,
linux-kernel
In-Reply-To: <20260413123256.3296452-1-puranjay@kernel.org>
bpf_flush_icache() calls flush_icache_range() to clean the data cache
and invalidate the instruction cache for the JITed code region. However,
since commit 1dad391daef1 ("bpf, arm64: use bpf_prog_pack for memory
management"), this flush is redundant.
bpf_jit_binary_pack_finalize() copies the JITed instructions to the ROX
region via bpf_arch_text_copy() -> aarch64_insn_copy() -> __text_poke(),
and __text_poke() already calls flush_icache_range() on the written
range. The subsequent bpf_flush_icache() repeats the same cache
maintenance on an overlapping range, including an unnecessary second
synchronous IPI to all CPUs via kick_all_cpus_sync().
Remove the redundant bpf_flush_icache() call and its now-unused
definition.
Fixes: 1dad391daef1 ("bpf, arm64: use bpf_prog_pack for memory management")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
arch/arm64/net/bpf_jit_comp.c | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index adf84962d579..e88b0917adec 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1961,11 +1961,6 @@ static int validate_ctx(struct jit_ctx *ctx)
return 0;
}
-static inline void bpf_flush_icache(void *start, void *end)
-{
- flush_icache_range((unsigned long)start, (unsigned long)end);
-}
-
static void priv_stack_init_guard(void __percpu *priv_stack_ptr, int alloc_size)
{
int cpu, underflow_idx = (alloc_size - PRIV_STACK_GUARD_SZ) >> 3;
@@ -2204,12 +2199,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog = orig_prog;
goto out_off;
}
- /*
- * The instructions have now been copied to the ROX region from
- * where they will execute. Now the data cache has to be cleaned to
- * the PoU and the I-cache has to be invalidated for the VAs.
- */
- bpf_flush_icache(ro_header, ctx.ro_image + ctx.idx);
} else {
jit_data->ctx = ctx;
jit_data->ro_image = ro_image_ptr;
--
2.52.0
^ permalink raw reply related
* Re: [PATCH RFC 10/12] drm/i915/display/dp: Adopt dp_connector helpers to expose link training state
From: Kory Maincent @ 2026-04-13 12:34 UTC (permalink / raw)
To: Jani Nikula
Cc: Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin, David Airlie,
Simona Vetter, Dave Airlie, Jesse Barnes, Eric Anholt,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Chun-Kuang Hu, Philipp Zabel,
Matthias Brugger, AngeloGioacchino Del Regno, Chris Wilson,
Thomas Petazzoni, Mark Yacoub, Sean Paul, Louis Chauvet,
intel-gfx, intel-xe, dri-devel, linux-kernel, linux-mediatek,
linux-arm-kernel, Simona Vetter
In-Reply-To: <e253ca4fa0b493032a7b35a0a20689b9d9e0c4e7@intel.com>
On Fri, 10 Apr 2026 19:26:53 +0300
Jani Nikula <jani.nikula@linux.intel.com> wrote:
> On Thu, 09 Apr 2026, Kory Maincent <kory.maincent@bootlin.com> wrote:
> > Switch the i915 DP connector initialization from drmm_connector_init()
> > to drmm_connector_dp_init(), providing the source link capabilities
> > (supported lane counts, link rates, DSC support, voltage swing and
> > pre-emphasis levels).
> >
> > Add intel_dp_report_link_train() to collect the negotiated link
> > parameters (rate, lane count, DSC enable, per-lane voltage swing and
> > pre-emphasis) and report them via
> > drm_connector_dp_set_link_train_properties() once link training completes
> > successfully.
> >
> > Reset the link training properties via
> > drm_connector_dp_reset_link_train_properties() when the connector is
> > reported as disconnected or when the display device is disabled, so
> > the exposed state always reflects the current link status.
> >
> > Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> > ---
> > drivers/gpu/drm/i915/display/intel_dp.c | 31
> > +++++++++++++++++++--- .../gpu/drm/i915/display/intel_dp_link_training.c |
> > 25 +++++++++++++++++ 2 files changed, 52 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/display/intel_dp.c
> > b/drivers/gpu/drm/i915/display/intel_dp.c index
> > 2af64de9c81de..641406bdc0cc9 100644 ---
> > a/drivers/gpu/drm/i915/display/intel_dp.c +++
> > b/drivers/gpu/drm/i915/display/intel_dp.c @@ -45,6 +45,7 @@
> > #include <drm/display/drm_hdmi_helper.h>
> > #include <drm/drm_atomic_helper.h>
> > #include <drm/drm_crtc.h>
> > +#include <drm/drm_dp_connector.h>
> > #include <drm/drm_edid.h>
> > #include <drm/drm_fixed.h>
> > #include <drm/drm_managed.h>
> > @@ -6337,8 +6338,10 @@ intel_dp_detect(struct drm_connector *_connector,
> > drm_WARN_ON(display->drm,
> > !drm_modeset_is_locked(&display->drm->mode_config.connection_mutex));
> >
> > - if (!intel_display_device_enabled(display))
> > + if (!intel_display_device_enabled(display)) {
> > + drm_connector_dp_reset_link_train_properties(_connector);
> > return connector_status_disconnected;
> > + }
> >
> > if (!intel_display_driver_check_access(display))
> > return connector->base.status;
> > @@ -6388,6 +6391,8 @@ intel_dp_detect(struct drm_connector *_connector,
> >
> > intel_dp_tunnel_disconnect(intel_dp);
> >
> > + drm_connector_dp_reset_link_train_properties(_connector);
> > +
> > goto out_unset_edid;
> > }
> >
> > @@ -7162,10 +7167,12 @@ intel_dp_init_connector(struct intel_digital_port
> > *dig_port, struct intel_connector *connector)
> > {
> > struct intel_display *display = to_intel_display(dig_port);
> > + struct drm_connector_dp_link_train_caps link_caps;
> > struct intel_dp *intel_dp = &dig_port->dp;
> > struct intel_encoder *encoder = &dig_port->base;
> > struct drm_device *dev = encoder->base.dev;
> > enum port port = encoder->port;
> > + u32 *rates;
> > int type;
> >
> > if (drm_WARN(dev, dig_port->max_lanes < 1,
> > @@ -7213,8 +7220,25 @@ intel_dp_init_connector(struct intel_digital_port
> > *dig_port, type == DRM_MODE_CONNECTOR_eDP ? "eDP" : "DP",
> > encoder->base.base.id, encoder->base.name);
> >
> > - drmm_connector_init(dev, &connector->base,
> > &intel_dp_connector_funcs,
> > - type, &intel_dp->aux.ddc);
> > + intel_dp_set_source_rates(intel_dp);
> > + link_caps.nlanes = DRM_DP_1LANE | DRM_DP_2LANE | DRM_DP_4LANE;
> > + link_caps.nrates = intel_dp->num_source_rates;
> > + rates = kzalloc_objs(*rates, intel_dp->num_source_rates);
> > + if (!rates)
> > + goto fail;
> > +
> > + for (int i = 0; i < intel_dp->num_source_rates; i++)
> > + rates[i] = intel_dp->source_rates[i];
> > +
> > + link_caps.rates = rates;
> > + link_caps.dsc = true;
>
> You have a source, you have a sink, and you have a link between the two.
>
> Source rates do not reflect the link rates common between source and
> sink.
>
> DSC depends on source and sink, and it's not statically "true" for
> either, and depends on a bunch of things.
At init, we are reporting the capabilities of the source. So we list every
link rates that the source can achieve and we report that the source is DSC
capable which it is IIUC the code. Or maybe I am missing something?
Regards,
--
Köry Maincent, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
^ permalink raw reply
* [PATCH v4 2/4] pwm: sun50i: Add H616 PWM support
From: bigunclemax @ 2026-04-13 12:39 UTC (permalink / raw)
To: richard.genoud
Cc: conor+dt, devicetree, jernej.skrabec, joao, jstultz, krzk+dt,
linux-arm-kernel, linux-kernel, linux-pwm, linux-sunxi, p.zabel,
paulk, robh, samuel, thomas.petazzoni, u.kleine-koenig, wens
In-Reply-To: <20260305091959.2530374-3-richard.genoud@bootlin.com>
Hi Richard,
> +
> +/* PWM Capture Fall Lock Register */
> +#define H616_PWM_CFLR(x) (0x74 + (x) * 0x20)
> +
> +#define H616_PWM_PAIR_IDX(chan) ((chan) >> 2)
> +
It looks like there's a typo or a mistake in the PAIR_IDX calculation.
It should be like ((chan) >> 1).
For example, for the 5th channel the result will be 1, but it should be 2.
Best regards
Maksim
^ permalink raw reply
* Re: [PATCH 0/3] arm64/virt: Add Arm CCA measurement register support
From: Jason Gunthorpe @ 2026-04-13 12:59 UTC (permalink / raw)
To: Sami Mujawar, Dan Williams
Cc: linux-arm-kernel, linux-kernel, catalin.marinas, will, thuth,
Suzuki.Poulose, steven.price, gshan, YeoReum.Yun
In-Reply-To: <20260413084957.327661-1-sami.mujawar@arm.com>
On Mon, Apr 13, 2026 at 09:49:54AM +0100, Sami Mujawar wrote:
> This series adds support for Arm Confidential Compute Architecture (CCA)
> measurement registers in the Linux kernel, enabling guest Realms to
> access, extend, and expose measurement values for attestation and runtime
> integrity tracking.
>
> The Realm Management Monitor (RMM) defines a set of measurement registers
> consisting of a Realm Initial Measurement (RIM) and a number of Realm
> Extensible Measurements (REMs). This series introduces the necessary
> infrastructure to interact with these registers via the RSI interface
> and exposes them to userspace through the TSM measurement framework.
>
> At a high level, the series includes:
> - Helper interfaces for reading and extending measurement
> registers via RSI
> - Definitions for Realm hash algorithms as defined by the
> RMM specification
> - Integration with the TSM measurement subsystem and sysfs
> exposure for userspace visibility and interaction
>
> After applying this series, measurement registers are exposed under:
> /sys/devices/virtual/misc/arm_cca_guest/measurements/
I'm surprised we get some random sysfs files? How does some more
generic userspace figure out to use this vs a TPM or some other
platform's version of it?
I also think exposing PCRs as was done for TPM in sysfs was something
of a mistake.. Allowing extension without logging is too low level and
is very hard to build an entire attestation system around.
I really think we are missing a subsystem here, TPM has sort of been
filling this role in a non-generic way, but we should have a
common uAPI for platform measurement & attestation:
- Discover available measurements
- Report signed measurements, with ingesting a nonce
- Report measurement logs
- Extend measurements and udpate logs
- Report certificates used in signing
- General reporting of various kinds of attestation evidence
And it would be nice for the PCI devices and others to plug into the
general framework as well instead of building a parallel TSM framework
for handling evidence.
Isn't this also sort of incomplete? Doesn't anything serious need
signed measurements? Isnt't there alot more data that comes out of RMM
than just a few measurement registers?
Jason
^ permalink raw reply
* Re: [PATCH RFC 09/12] drm: Introduce drmm_connector_dp_init() with link training state properties
From: Kory Maincent @ 2026-04-13 12:59 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: Jani Nikula, Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin,
David Airlie, Simona Vetter, Dave Airlie, Jesse Barnes,
Eric Anholt, Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Chun-Kuang Hu, Philipp Zabel,
Matthias Brugger, AngeloGioacchino Del Regno, Chris Wilson,
Thomas Petazzoni, Mark Yacoub, Sean Paul, Louis Chauvet,
intel-gfx, intel-xe, dri-devel, linux-kernel, linux-mediatek,
linux-arm-kernel, Simona Vetter
In-Reply-To: <gmaxonri7y3k43pxsnxfnd23mydgrwsw2322v6m4yjoksqm7aj@4ehgvhzg2ubb>
On Fri, 10 Apr 2026 00:53:08 +0300
Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> wrote:
> On Thu, Apr 09, 2026 at 07:08:25PM +0200, Kory Maincent wrote:
> > Add a managed DisplayPort connector initialization helper,
> > drmm_connector_dp_init(), modeled after the existing HDMI counterpart
> > drmm_connector_hdmi_init(). Cleanup is handled automatically via a
> > DRM-managed action.
> >
> > The helper creates the following immutable connector properties to expose
> > DP link training capabilities and state to userspace:
> >
> > - num_lanes: bitmask of supported lane counts (1, 2, 4)
> > - link_rate: Array of supported link rates.
> > - dsc_en: Display Stream Compression supported
> > - voltage_swingN: per-lane voltage swing level bitmask
> > - pre-emphasisN: per-lane pre-emphasis level bitmask
> >
> > Link rates are passed by the driver in deca-kbps, following the DRM
> > convention, but exposed to userspace in kbps for clarity.
> >
> > Two additional helpers are provided to update and reset those properties
> > at runtime:
> > - drm_connector_dp_set_link_train_properties()
> > - drm_connector_dp_reset_link_train_properties()
> >
...
> > +/**
> > + * struct drm_connector_dp_link_train - DRM DisplayPort link training
> > + * information report
> > + */
> > +struct drm_connector_dp_link_train {
>
> THese define the current DP state. As such, they definitely make sense
> to be a part of the drm_connector.
> > + /**
> > + * @nlanes: The number of lanes used
> > + */
> > + u8 nlanes;
> > +
> > + /**
> > + * @rates: Link rate value selected in deca-kbps
> > + */
> > + u32 rate;
> > +
> > + /**
> > + * @dsc: Display Stream Compression enabled
> > + */
> > + bool dsc_en;
> > +
> > + /**
> > + * @v_swings: Array listing the bitmask voltage swing level per
> > lanes
> > + */
> > + u8 v_swing[4];
> > +
> > + /**
> > + * @pre_emph: Array listing the bitmask pre-emphasis level per
> > lanes
> > + */
> > + u8 pre_emph[4];
>
> Please consider following struct phy_configure_opts_dp (or using it as
> is). Overall, please refer the talk and (more important) the lightning
> resumee at this XDC. I have some bits and pieces ready in spite of that
> proposal, but I didn't have time to finish it.
I didn't know this phy_configure_opts_dp struct. This indeed could make sense
to reuse and modify that structure for our needs.
About your XDC talk. What were the developments you were talking about? Could I
have a look at them?
Regards,
--
Köry Maincent, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
^ permalink raw reply
* Re: [PATCH RFC 10/12] drm/i915/display/dp: Adopt dp_connector helpers to expose link training state
From: Jani Nikula @ 2026-04-13 13:05 UTC (permalink / raw)
To: Kory Maincent
Cc: Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin, David Airlie,
Simona Vetter, Dave Airlie, Jesse Barnes, Eric Anholt,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Chun-Kuang Hu, Philipp Zabel,
Matthias Brugger, AngeloGioacchino Del Regno, Chris Wilson,
Thomas Petazzoni, Mark Yacoub, Sean Paul, Louis Chauvet,
intel-gfx, intel-xe, dri-devel, linux-kernel, linux-mediatek,
linux-arm-kernel, Simona Vetter
In-Reply-To: <20260413143402.76f5c3c9@kmaincent-XPS-13-7390>
On Mon, 13 Apr 2026, Kory Maincent <kory.maincent@bootlin.com> wrote:
> On Fri, 10 Apr 2026 19:26:53 +0300
> Jani Nikula <jani.nikula@linux.intel.com> wrote:
>
>> On Thu, 09 Apr 2026, Kory Maincent <kory.maincent@bootlin.com> wrote:
>> > Switch the i915 DP connector initialization from drmm_connector_init()
>> > to drmm_connector_dp_init(), providing the source link capabilities
>> > (supported lane counts, link rates, DSC support, voltage swing and
>> > pre-emphasis levels).
>> >
>> > Add intel_dp_report_link_train() to collect the negotiated link
>> > parameters (rate, lane count, DSC enable, per-lane voltage swing and
>> > pre-emphasis) and report them via
>> > drm_connector_dp_set_link_train_properties() once link training completes
>> > successfully.
>> >
>> > Reset the link training properties via
>> > drm_connector_dp_reset_link_train_properties() when the connector is
>> > reported as disconnected or when the display device is disabled, so
>> > the exposed state always reflects the current link status.
>> >
>> > Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
>> > ---
>> > drivers/gpu/drm/i915/display/intel_dp.c | 31
>> > +++++++++++++++++++--- .../gpu/drm/i915/display/intel_dp_link_training.c |
>> > 25 +++++++++++++++++ 2 files changed, 52 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/drivers/gpu/drm/i915/display/intel_dp.c
>> > b/drivers/gpu/drm/i915/display/intel_dp.c index
>> > 2af64de9c81de..641406bdc0cc9 100644 ---
>> > a/drivers/gpu/drm/i915/display/intel_dp.c +++
>> > b/drivers/gpu/drm/i915/display/intel_dp.c @@ -45,6 +45,7 @@
>> > #include <drm/display/drm_hdmi_helper.h>
>> > #include <drm/drm_atomic_helper.h>
>> > #include <drm/drm_crtc.h>
>> > +#include <drm/drm_dp_connector.h>
>> > #include <drm/drm_edid.h>
>> > #include <drm/drm_fixed.h>
>> > #include <drm/drm_managed.h>
>> > @@ -6337,8 +6338,10 @@ intel_dp_detect(struct drm_connector *_connector,
>> > drm_WARN_ON(display->drm,
>> > !drm_modeset_is_locked(&display->drm->mode_config.connection_mutex));
>> >
>> > - if (!intel_display_device_enabled(display))
>> > + if (!intel_display_device_enabled(display)) {
>> > + drm_connector_dp_reset_link_train_properties(_connector);
>> > return connector_status_disconnected;
>> > + }
>> >
>> > if (!intel_display_driver_check_access(display))
>> > return connector->base.status;
>> > @@ -6388,6 +6391,8 @@ intel_dp_detect(struct drm_connector *_connector,
>> >
>> > intel_dp_tunnel_disconnect(intel_dp);
>> >
>> > + drm_connector_dp_reset_link_train_properties(_connector);
>> > +
>> > goto out_unset_edid;
>> > }
>> >
>> > @@ -7162,10 +7167,12 @@ intel_dp_init_connector(struct intel_digital_port
>> > *dig_port, struct intel_connector *connector)
>> > {
>> > struct intel_display *display = to_intel_display(dig_port);
>> > + struct drm_connector_dp_link_train_caps link_caps;
>> > struct intel_dp *intel_dp = &dig_port->dp;
>> > struct intel_encoder *encoder = &dig_port->base;
>> > struct drm_device *dev = encoder->base.dev;
>> > enum port port = encoder->port;
>> > + u32 *rates;
>> > int type;
>> >
>> > if (drm_WARN(dev, dig_port->max_lanes < 1,
>> > @@ -7213,8 +7220,25 @@ intel_dp_init_connector(struct intel_digital_port
>> > *dig_port, type == DRM_MODE_CONNECTOR_eDP ? "eDP" : "DP",
>> > encoder->base.base.id, encoder->base.name);
>> >
>> > - drmm_connector_init(dev, &connector->base,
>> > &intel_dp_connector_funcs,
>> > - type, &intel_dp->aux.ddc);
>> > + intel_dp_set_source_rates(intel_dp);
>> > + link_caps.nlanes = DRM_DP_1LANE | DRM_DP_2LANE | DRM_DP_4LANE;
>> > + link_caps.nrates = intel_dp->num_source_rates;
>> > + rates = kzalloc_objs(*rates, intel_dp->num_source_rates);
>> > + if (!rates)
>> > + goto fail;
>> > +
>> > + for (int i = 0; i < intel_dp->num_source_rates; i++)
>> > + rates[i] = intel_dp->source_rates[i];
>> > +
>> > + link_caps.rates = rates;
>> > + link_caps.dsc = true;
>>
>> You have a source, you have a sink, and you have a link between the two.
>>
>> Source rates do not reflect the link rates common between source and
>> sink.
>>
>> DSC depends on source and sink, and it's not statically "true" for
>> either, and depends on a bunch of things.
>
> At init, we are reporting the capabilities of the source. So we list every
> link rates that the source can achieve and we report that the source is DSC
> capable which it is IIUC the code. Or maybe I am missing something?
IMO link caps is the intersection of the source and sink caps. If the
sink is unknown, i.e. its caps are the empty set, then the link caps
should also be the empty set.
If you need to know the source caps, then they need to be presented
separately.
Moreover, the source does not unconditionally support DSC. See
intel_dp_has_dsc().
BR,
Jani.
--
Jani Nikula, Intel
^ permalink raw reply
* [PATCH bpf-next v14 4/5] bpf, x86: Emit ENDBR for indirect jump targets
From: Xu Kuohai @ 2026-04-13 12:45 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <cover.1776062885.git.xukuohai@hotmail.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support CET/IBT, the indirect jump selftest triggers
a kernel panic because the indirect jump targets lack ENDBR
instructions.
To fix it, emit an ENDBR instruction to each indirect jump target. Since
the ENDBR instruction shifts the position of original jited instructions,
fix the instruction address calculation wherever the addresses are used.
For reference, below is a sample panic log.
Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cet.c:133!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
...
? 0xffffffffc00fb258
? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
bpf_prog_test_run_syscall+0x110/0x2f0
? fdget+0xba/0xe0
__sys_bpf+0xe4b/0x2590
? __kmalloc_node_track_caller_noprof+0x1c7/0x680
? bpf_prog_test_run_syscall+0x215/0x2f0
__x64_sys_bpf+0x21/0x30
do_syscall_64+0x85/0x620
? bpf_prog_test_run_syscall+0x1e2/0x2f0
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72d9a5faa230..ea9e707e8abf 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
#define EMIT_ENDBR() EMIT(gen_endbr(), 4)
#define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4)
#else
-#define EMIT_ENDBR()
-#define EMIT_ENDBR_POISON()
+#define EMIT_ENDBR() do { } while (0)
+#define EMIT_ENDBR_POISON() do { } while (0)
#endif
static bool is_imm8(int value)
@@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
-static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image,
- int oldproglen, struct jit_context *ctx, bool jmp_padding)
+static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
+ u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
bool tail_call_reachable = bpf_prog->aux->tail_call_reachable;
struct bpf_insn *insn = bpf_prog->insnsi;
@@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
void __percpu *priv_stack_ptr;
int i, excnt = 0;
int ilen, proglen = 0;
- u8 *prog = temp;
+ u8 *ip, *prog = temp;
u32 stack_depth;
int err;
@@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
dst_reg = X86_REG_R9;
}
+ if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1))
+ EMIT_ENDBR();
+
+ ip = image + addrs[i - 1] + (prog - temp);
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off))
/* call */
case BPF_JMP | BPF_CALL: {
- u8 *ip = image + addrs[i - 1];
-
func = (u8 *) __bpf_call_base + imm32;
if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off))
if (imm32)
emit_bpf_tail_call_direct(bpf_prog,
&bpf_prog->aux->poke_tab[imm32 - 1],
- &prog, image + addrs[i - 1],
+ &prog,
+ ip,
callee_regs_used,
stack_depth,
ctx);
@@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off))
&prog,
callee_regs_used,
stack_depth,
- image + addrs[i - 1],
+ ip,
ctx);
break;
@@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off))
break;
case BPF_JMP | BPF_JA | BPF_X:
- emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+ emit_indirect_jump(&prog, insn->dst_reg, ip);
break;
case BPF_JMP | BPF_JA:
case BPF_JMP32 | BPF_JA:
@@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off))
ctx->cleanup_addr = proglen;
if (bpf_prog_was_classic(bpf_prog) &&
!ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
- u8 *ip = image + addrs[i - 1];
-
if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
return -EINVAL;
}
@@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
for (pass = 0; pass < MAX_PASSES || image; pass++) {
if (!padding && pass >= PADDING_PASSES)
padding = true;
- proglen = do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, padding);
+ proglen = do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, padding);
if (proglen <= 0) {
out_image:
image = NULL;
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v14 2/5] bpf: Pass bpf_verifier_env to JIT
From: Xu Kuohai @ 2026-04-13 12:44 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <cover.1776062885.git.xukuohai@hotmail.com>
From: Xu Kuohai <xukuohai@huawei.com>
Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will
use env->insn_aux_data in the JIT stage to detect indirect jump targets.
Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c
code without verifier, introduce helper __bpf_prog_select_runtime()
to accept the env parameter.
Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and
switch to call __bpf_prog_select_runtime() in the verifier, with env
variable passed. The original bpf_prog_select_runtime() is preserved for
cbpf and lib/test_bpf.c, where env is NULL.
Now all constants blinding calls are moved into the verifier, except
the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted
by bpf_patch_insn_data() function for normal cases, so there is no need
to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 2 +-
arch/arm/net/bpf_jit_32.c | 2 +-
arch/arm64/net/bpf_jit_comp.c | 2 +-
arch/loongarch/net/bpf_jit.c | 2 +-
arch/mips/net/bpf_jit_comp.c | 2 +-
arch/parisc/net/bpf_jit_core.c | 2 +-
arch/powerpc/net/bpf_jit_comp.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 2 +-
arch/s390/net/bpf_jit_comp.c | 2 +-
arch/sparc/net/bpf_jit_comp_64.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
include/linux/filter.h | 17 ++++++-
kernel/bpf/core.c | 86 ++++++++++++++++----------------
kernel/bpf/fixups.c | 10 ++--
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 14 +++---
17 files changed, 84 insertions(+), 71 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 973ceae48675..639a2736f029 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -1400,7 +1400,7 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog)
* (re)locations involved that their addresses are not known
* during the first run.
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
vm_dump(prog);
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index e6b1bb2de627..1628b6fc70a4 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2142,7 +2142,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header;
struct jit_ctx ctx;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index cd5a72fff500..7212ec89dfe3 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2006,7 +2006,7 @@ struct arm64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
struct bpf_binary_header *header;
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index fcc8c0c29fb0..5149ce4cef7e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1920,7 +1920,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index d2b6c955f18e..6ee4abe6a1f7 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -909,7 +909,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index 35dca372b5df..172770132440 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 711028bebea3..27fecb4cc063 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -129,7 +129,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
u32 proglen;
u32 alloclen;
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 527baa50dc68..768ac686b359 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 2dfc279b1be2..94128fe6be23 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2312,7 +2312,7 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
/*
* Compile eBPF program "fp"
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index e83e29137566..2fa0e9375127 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1477,7 +1477,7 @@ struct sparc64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 77d00a8dec87..72d9a5faa230 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3713,7 +3713,7 @@ struct x64_jit_data {
#define MAX_PASSES 20
#define PADDING_PASSES (MAX_PASSES - 5)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5f259577614a..852baf2e4db4 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2518,7 +2518,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
int proglen, oldproglen = 0;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index d396e55c9a1d..83f37d38c5c1 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1107,6 +1107,8 @@ static inline int sk_filter_reason(struct sock *sk, struct sk_buff *skb,
return sk_filter_trim_cap(sk, skb, 1, reason);
}
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err);
struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err);
void bpf_prog_free(struct bpf_prog *fp);
@@ -1152,7 +1154,7 @@ u64 __bpf_call_base(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
((u64 (*)(u64, u64, u64, u64, u64, const struct bpf_insn *)) \
(void *)__bpf_call_base)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog);
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_compile(struct bpf_prog *prog);
bool bpf_jit_needs_zext(void);
bool bpf_jit_inlines_helper_call(s32 imm);
@@ -1187,12 +1189,25 @@ struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
#ifdef CONFIG_BPF_SYSCALL
struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len);
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env);
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux);
#else
static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len)
{
return ERR_PTR(-ENOTSUPP);
}
+
+static inline struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ return NULL;
+}
+
+static inline void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+}
#endif /* CONFIG_BPF_SYSCALL */
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc9fb3c07866..79361aa11757 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1491,23 +1491,6 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
bpf_prog_clone_free(fp_other);
}
-static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
-{
-#ifdef CONFIG_BPF_SYSCALL
- struct bpf_map *map;
- int i;
-
- if (len <= 1)
- return;
-
- for (i = 0; i < prog->aux->used_map_cnt; i++) {
- map = prog->aux->used_maps[i];
- if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
- bpf_insn_array_adjust(map, off, len);
- }
-#endif
-}
-
/*
* Now this function is used only to blind the main prog and must be invoked only when
* bpf_prog_need_blind() returns true.
@@ -1580,13 +1563,6 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
if (env)
env->prog = clone;
- else
- /*
- * Instructions arrays must be updated using absolute xlated offsets.
- * The arrays have already been adjusted by bpf_patch_insn_data() when
- * env is not NULL.
- */
- adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2555,47 +2531,55 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
-static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
#ifdef CONFIG_BPF_JIT
struct bpf_prog *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
if (!bpf_prog_need_blind(prog))
- return bpf_int_jit_compile(prog);
+ return bpf_int_jit_compile(env, prog);
+
+ if (env) {
+ /*
+ * If env is not NULL, we are called from the end of bpf_check(), at this
+ * point, only insn_aux_data is used after failure, so it should be restored
+ * on failure.
+ */
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux)
+ return prog;
+ }
orig_prog = prog;
- prog = bpf_jit_blind_constants(NULL, prog);
+ prog = bpf_jit_blind_constants(env, prog);
/*
* If blinding was requested and we failed during blinding, we must fall
* back to the interpreter.
*/
if (IS_ERR(prog))
- return orig_prog;
+ goto out_restore;
- prog = bpf_int_jit_compile(prog);
+ prog = bpf_int_jit_compile(env, prog);
if (prog->jited) {
bpf_jit_prog_release_other(prog, orig_prog);
+ if (env)
+ vfree(orig_insn_aux);
return prog;
}
bpf_jit_prog_release_other(orig_prog, prog);
+
+out_restore:
prog = orig_prog;
+ if (env)
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
#endif
return prog;
}
-/**
- * bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with BPF program
- * @err: pointer to error variable
- *
- * Try to JIT eBPF program, if JIT is not available, use interpreter.
- * The BPF program will be executed via bpf_prog_run() function.
- *
- * Return: the &fp argument along with &err set to 0 for success or
- * a negative errno code on failure
- */
-struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err)
{
/* In case of BPF to BPF calls, verifier did all the prep
* work with regards to JITing, etc.
@@ -2623,7 +2607,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_prog_jit_compile(fp);
+ fp = bpf_prog_jit_compile(env, fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
@@ -2649,6 +2633,22 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
return fp;
}
+
+/**
+ * bpf_prog_select_runtime - select exec runtime for BPF program
+ * @fp: bpf_prog populated with BPF program
+ * @err: pointer to error variable
+ *
+ * Try to JIT eBPF program, if JIT is not available, use interpreter.
+ * The BPF program will be executed via bpf_prog_run() function.
+ *
+ * Return: the &fp argument along with &err set to 0 for success or
+ * a negative errno code on failure
+ */
+struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+{
+ return __bpf_prog_select_runtime(NULL, fp, err);
+}
EXPORT_SYMBOL_GPL(bpf_prog_select_runtime);
static unsigned int __bpf_prog_ret1(const void *ctx,
@@ -3136,7 +3136,7 @@ const struct bpf_func_proto bpf_tail_call_proto = {
* It is encouraged to implement bpf_int_jit_compile() instead, so that
* eBPF and implicitly also cBPF can get JITed!
*/
-struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
return prog;
}
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index ec8afd6a9369..6ef213429a2a 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -991,7 +991,7 @@ static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_s
env->subprog_info[i].start = orig_starts[i];
}
-static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
{
size_t size;
void *new_aux;
@@ -1003,8 +1003,8 @@ static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *
return new_aux;
}
-static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
- struct bpf_insn_aux_data *orig_insn_aux)
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
{
/* the expanded elements are zero-filled, so no special handling is required */
vfree(env->insn_aux_data);
@@ -1147,7 +1147,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
- func[i] = bpf_int_jit_compile(func[i]);
+ func[i] = bpf_int_jit_compile(env, func[i]);
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1191,7 +1191,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
for (i = 0; i < env->subprog_cnt; i++) {
old_bpf_func = func[i]->bpf_func;
- tmp = bpf_int_jit_compile(func[i]);
+ tmp = bpf_int_jit_compile(env, func[i]);
if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) {
verbose(env, "JIT doesn't support bpf-to-bpf calls\n");
err = -ENOTSUPP;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index b73b25c63073..a3c0214ca934 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3083,10 +3083,6 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
if (err < 0)
goto free_used_maps;
- prog = bpf_prog_select_runtime(prog, &err);
- if (err < 0)
- goto free_used_maps;
-
err = bpf_prog_mark_insn_arrays_ready(prog);
if (err < 0)
goto free_used_maps;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e4980128151..e804e0da3500 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -20155,6 +20155,14 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
adjust_btf_func(env);
+ /* extension progs temporarily inherit the attach_type of their targets
+ for verification purposes, so set it back to zero before returning
+ */
+ if (env->prog->type == BPF_PROG_TYPE_EXT)
+ env->prog->expected_attach_type = 0;
+
+ env->prog = __bpf_prog_select_runtime(env, env->prog, &ret);
+
err_release_maps:
if (ret)
release_insn_arrays(env);
@@ -20166,12 +20174,6 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
if (!env->prog->aux->used_btfs)
release_btfs(env);
- /* extension progs temporarily inherit the attach_type of their targets
- for verification purposes, so set it back to zero before returning
- */
- if (env->prog->type == BPF_PROG_TYPE_EXT)
- env->prog->expected_attach_type = 0;
-
*prog = env->prog;
module_put(env->attach_btf_mod);
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v14 0/5] emit ENDBR/BTI instructions for indirect
From: Xu Kuohai @ 2026-04-13 12:44 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
From: Xu Kuohai <xukuohai@hotmail.com>
On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.
The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, allowing
the JIT to consult env->insn_aux_data to determine which instructions are
indirect jump targets.
During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not use env->insn_aux_data directly.
To avoid this mismatch, and given that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.
v14:
- Rebase
- Fix comment style
- Fix incorrect variable and function name used in commit message
v13: https://lore.kernel.org/bpf/20260411133847.1042658-1-xukuohai@huaweicloud.com
- Use vmalloc to allocate memory for insn_aux_data copies to match with vfree
- Do not free the copied memory of insn_aux_data when restoring from failure
- Code cleanup
v12: https://lore.kernel.org/bpf/20260403132811.753894-1-xukuohai@huaweicloud.com
- Restore env->insn_aux_data on JIT failure
- Fix incorrect error code sign (-EFAULT vs EFAULT)
- Fix incorrect prog used in the restore path
v11: https://lore.kernel.org/bpf/20260403090915.473493-1-xukuohai@huaweicloud.com
- Restore env->subprog_info after jit_subprogs() fails
- Clear prog->jit_requested and prog->blinding_requested on failure
- Use the actual env->insn_aux_data size in clear_insn_aux_data() on failure
v10: https://lore.kernel.org/bpf/20260324122052.342751-1-xukuohai@huaweicloud.com
- Fix the incorrect call_imm restore in jit_subprogs
- Define a dummy void version of bpf_jit_prog_release_other and
bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)
v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)
v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
to adjust_insn_aux_data()
v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch
v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier
v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
each prog
v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
jump targets, and emit ENDBR on x86
v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)
v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)
v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com
Xu Kuohai (5):
bpf: Move constants blinding out of arch-specific JITs
bpf: Pass bpf_verifier_env to JIT
bpf: Add helper to detect indirect jump targets
bpf, x86: Emit ENDBR for indirect jump targets
bpf, arm64: Emit BTI for indirect jump target
arch/arc/net/bpf_jit_core.c | 41 +++-----
arch/arm/net/bpf_jit_32.c | 43 ++-------
arch/arm64/net/bpf_jit_comp.c | 87 ++++++-----------
arch/loongarch/net/bpf_jit.c | 61 ++++--------
arch/mips/net/bpf_jit_comp.c | 22 +----
arch/parisc/net/bpf_jit_core.c | 75 ++++++---------
arch/powerpc/net/bpf_jit_comp.c | 70 +++++---------
arch/riscv/net/bpf_jit_core.c | 63 +++++-------
arch/s390/net/bpf_jit_comp.c | 61 ++++--------
arch/sparc/net/bpf_jit_comp_64.c | 63 +++++-------
arch/x86/net/bpf_jit_comp.c | 73 +++++---------
arch/x86/net/bpf_jit_comp32.c | 35 +------
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 9 +-
include/linux/filter.h | 50 +++++++++-
kernel/bpf/core.c | 138 ++++++++++++++++++--------
kernel/bpf/fixups.c | 160 ++++++++++++++++++++++++++-----
kernel/bpf/syscall.c | 4 -
kernel/bpf/verifier.c | 21 ++--
19 files changed, 526 insertions(+), 552 deletions(-)
--
2.47.3
^ permalink raw reply
* [PATCH bpf-next v14 3/5] bpf: Add helper to detect indirect jump targets
From: Xu Kuohai @ 2026-04-13 12:45 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <cover.1776062885.git.xukuohai@hotmail.com>
From: Xu Kuohai <xukuohai@huawei.com>
Introduce helper bpf_insn_is_indirect_target to check whether a BPF
instruction is an indirect jump target.
Since the verifier knows which instructions are indirect jump targets,
add a new flag indirect_target to struct bpf_insn_aux_data to mark
them. The verifier sets this flag when verifying an indirect jump target
instruction, and the helper checks the flag to determine whether an
instruction is an indirect jump target.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> #v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> #v12
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
include/linux/bpf.h | 2 ++
include/linux/bpf_verifier.h | 9 +++++----
kernel/bpf/core.c | 9 +++++++++
kernel/bpf/fixups.c | 12 ++++++++++++
kernel/bpf/verifier.c | 7 +++++++
5 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 0136a108d083..b4b703c90ca9 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1541,6 +1541,8 @@ bool bpf_has_frame_pointer(unsigned long ip);
int bpf_jit_charge_modmem(u32 size);
void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 53e8664cb566..b148f816f25b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -630,16 +630,17 @@ struct bpf_insn_aux_data {
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
- bool jmp_point;
- bool prune_point;
+ u32 jmp_point:1;
+ u32 prune_point:1;
/* ensure we check state equivalence and save state checkpoint and
* this instruction, regardless of any heuristics
*/
- bool force_checkpoint;
+ u32 force_checkpoint:1;
/* true if instruction is a call to a helper function that
* accepts callback function as a parameter.
*/
- bool calls_callback;
+ u32 calls_callback:1;
+ u32 indirect_target:1; /* if it is an indirect jump target */
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 79361aa11757..8b018ff48875 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1573,6 +1573,15 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
clone->blinded = 1;
return clone;
}
+
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx)
+{
+ if (!env)
+ return false;
+ insn_idx += prog->aux->subprog_start;
+ return env->insn_aux_data[insn_idx].indirect_target;
+}
#endif /* CONFIG_BPF_JIT */
/* Base function for offset calculation. Needs to go into .text section,
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 6ef213429a2a..67f14e24d7d7 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -183,6 +183,18 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
data[i].seen = old_seen;
data[i].zext_dst = insn_has_def32(insn + i);
}
+
+ /*
+ * The indirect_target flag of the original instruction was moved to the last of the
+ * new instructions by the above memmove and memset, but the indirect jump target is
+ * actually the first instruction, so move it back. This also matches with the behavior
+ * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
+ * instruction.
+ */
+ if (data[off + cnt - 1].indirect_target) {
+ data[off].indirect_target = 1;
+ data[off + cnt - 1].indirect_target = 0;
+ }
}
static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e804e0da3500..1e36b9e91277 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3497,6 +3497,11 @@ static int insn_stack_access_flags(int frameno, int spi)
return INSN_F_STACK_ACCESS | (spi << INSN_F_SPI_SHIFT) | frameno;
}
+static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].indirect_target = true;
+}
+
#define LR_FRAMENO_BITS 3
#define LR_SPI_BITS 6
#define LR_ENTRY_BITS (LR_SPI_BITS + LR_FRAMENO_BITS + 1)
@@ -17545,12 +17550,14 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
}
for (i = 0; i < n - 1; i++) {
+ mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
env->insn_idx, env->cur_state->speculative);
if (IS_ERR(other_branch))
return PTR_ERR(other_branch);
}
env->insn_idx = env->gotox_tmp_buf->items[n-1];
+ mark_indirect_target(env, env->insn_idx);
return INSN_IDX_UPDATED;
}
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v14 5/5] bpf, arm64: Emit BTI for indirect jump target
From: Xu Kuohai @ 2026-04-13 12:45 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <cover.1776062885.git.xukuohai@hotmail.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support BTI, the indirect jump selftest triggers a kernel
panic because there is no BTI instructions at the indirect jump targets.
Fix it by emitting a BTI instruction for each indirect jump target.
For reference, below is a sample panic log.
Internal error: Oops - BTI: 0000000036000003 [#1] SMP
...
Call trace:
bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x54/0xf8 (P)
bpf_prog_run_pin_on_cpu+0x140/0x468
bpf_prog_test_run_syscall+0x280/0x3b8
bpf_prog_test_run+0x22c/0x2c0
Fixes: f4a66cf1cb14 ("bpf: arm64: Add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 7212ec89dfe3..aa5cd240cdda 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1198,8 +1198,8 @@ static int add_exception_handler(const struct bpf_insn *insn,
* >0 - successfully JITed a 16-byte eBPF instruction.
* <0 - failed to JIT.
*/
-static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
- bool extra_pass)
+static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn *insn,
+ struct jit_ctx *ctx, bool extra_pass)
{
const u8 code = insn->code;
u8 dst = bpf2a64[insn->dst_reg];
@@ -1224,6 +1224,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
int ret;
bool sign_extend;
+ if (bpf_insn_is_indirect_target(env, ctx->prog, i))
+ emit_bti(A64_BTI_J, ctx);
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
@@ -1899,7 +1902,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
-static int build_body(struct jit_ctx *ctx, bool extra_pass)
+static int build_body(struct bpf_verifier_env *env, struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
int i;
@@ -1918,7 +1921,7 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
int ret;
ctx->offset[i] = ctx->idx;
- ret = build_insn(insn, ctx, extra_pass);
+ ret = build_insn(env, insn, ctx, extra_pass);
if (ret > 0) {
i++;
ctx->offset[i] = ctx->idx;
@@ -2079,7 +2082,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
if (build_prologue(&ctx, was_classic))
goto out_off;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_off;
ctx.epilogue_offset = ctx.idx;
@@ -2127,7 +2130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_free_hdr;
ctx.epilogue_offset = ctx.idx;
@@ -2136,7 +2139,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
ctx.write = true;
/* Pass 3: Adjust jump offset and write final image */
- if (build_body(&ctx, extra_pass) ||
+ if (build_body(env, &ctx, extra_pass) ||
WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
--
2.47.3
^ permalink raw reply related
* [PATCH bpf-next v14 1/5] bpf: Move constants blinding out of arch-specific JITs
From: Xu Kuohai @ 2026-04-13 12:44 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <cover.1776062885.git.xukuohai@hotmail.com>
From: Xu Kuohai <xukuohai@huawei.com>
During the JIT stage, constants blinding rewrites instructions but only
rewrites the private instruction copy of the JITed subprog, leaving the
global env->prog->insnsi and env->insn_aux_data untouched. This causes a
mismatch between subprog instructions and the global state, making it
difficult to use the global data in the JIT.
To avoid this mismatch, and given that all arch-specific JITs already
support constants blinding, move it to the generic verifier code, and
switch to rewrite the global env->prog->insnsi with the global states
adjusted, as other rewrites in the verifier do.
This removes the constants blinding calls in each JIT, which are largely
duplicated code across architectures.
Since constants blinding is only required for JIT, and there are two
JIT entry functions, jit_subprogs() for BPF programs with multiple
subprogs and bpf_prog_select_runtime() for programs with no subprogs,
move the constants blinding invocation into these two functions.
In the verifier path, bpf_patch_insn_data() is used to keep global
verifier auxiliary data in sync with patched instructions. A key
question is whether this global auxiliary data should be restored
on the failure path.
Besides instructions, bpf_patch_insn_data() adjusts:
- prog->aux->poke_tab
- env->insn_array_maps
- env->subprog_info
- env->insn_aux_data
For prog->aux->poke_tab, it is only used by JIT or only meaningful after
JIT succeeds, so it does not need to be restored on the failure path.
For env->insn_array_maps, when JIT fails, programs using insn arrays
are rejected by bpf_insn_array_ready() due to missing JIT addresses.
Hence, env->insn_array_maps is only meaningful for JIT and does not need
to be restored.
For subprog_info, if jit_subprogs fails and CONFIG_BPF_JIT_ALWAYS_ON
is not enabled, kernel falls back to interpreter. In this case,
env->subprog_info is used to determine subprogram stack depth. So it
must be restored on failure.
For env->insn_aux_data, it is freed by clear_insn_aux_data() at the
end of bpf_check(). Before freeing, clear_insn_aux_data() loops over
env->insn_aux_data to release jump targets recorded in it. The loop
uses env->prog->len as the array length, but this length no longer
matches the actual size of the adjusted env->insn_aux_data array after
constants blinding.
To address it, a simple approach is to keep insn_aux_data as adjusted
after failure, since it will be freed shortly, and record its actual size
for the loop in clear_insn_aux_data(). But since clear_insn_aux_data()
uses the same index to loop over both env->prog->insnsi and env->insn_aux_data,
this approach results in incorrect index for the insnsi array. So an
alternative approach is adopted: clone the original env->insn_aux_data
before blinding and restore it after failure, similar to env->prog.
For classic BPF programs, constants blinding works as before since it
is still invoked from bpf_prog_select_runtime().
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Hari Bathini <hbathini@linux.ibm.com> # powerpc jit
Reviewed-by: Pu Lehui <pulehui@huawei.com> # riscv jit
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 39 +++------
arch/arm/net/bpf_jit_32.c | 41 ++-------
arch/arm64/net/bpf_jit_comp.c | 72 +++++-----------
arch/loongarch/net/bpf_jit.c | 59 ++++---------
arch/mips/net/bpf_jit_comp.c | 20 +----
arch/parisc/net/bpf_jit_core.c | 73 +++++++---------
arch/powerpc/net/bpf_jit_comp.c | 68 ++++++---------
arch/riscv/net/bpf_jit_core.c | 61 +++++--------
arch/s390/net/bpf_jit_comp.c | 59 +++++--------
arch/sparc/net/bpf_jit_comp_64.c | 61 +++++--------
arch/x86/net/bpf_jit_comp.c | 43 ++-------
arch/x86/net/bpf_jit_comp32.c | 33 +------
include/linux/filter.h | 33 ++++++-
kernel/bpf/core.c | 69 +++++++++++++--
kernel/bpf/fixups.c | 144 ++++++++++++++++++++++++++-----
15 files changed, 400 insertions(+), 475 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 1421eeced0f5..973ceae48675 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -79,7 +79,6 @@ struct arc_jit_data {
* The JIT pertinent context that is used by different functions.
*
* prog: The current eBPF program being handled.
- * orig_prog: The original eBPF program before any possible change.
* jit: The JIT buffer and its length.
* bpf_header: The JITed program header. "jit.buf" points inside it.
* emit: If set, opcodes are written to memory; else, a dry-run.
@@ -94,12 +93,10 @@ struct arc_jit_data {
* need_extra_pass: A forecast if an "extra_pass" will occur.
* is_extra_pass: Indicates if the current pass is an extra pass.
* user_bpf_prog: True, if VM opcodes come from a real program.
- * blinded: True if "constant blinding" step returned a new "prog".
* success: Indicates if the whole JIT went OK.
*/
struct jit_context {
struct bpf_prog *prog;
- struct bpf_prog *orig_prog;
struct jit_buffer jit;
struct bpf_binary_header *bpf_header;
bool emit;
@@ -114,7 +111,6 @@ struct jit_context {
bool need_extra_pass;
bool is_extra_pass;
bool user_bpf_prog;
- bool blinded;
bool success;
};
@@ -161,13 +157,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog)
{
memset(ctx, 0, sizeof(*ctx));
- ctx->orig_prog = prog;
-
- /* If constant blinding was requested but failed, scram. */
- ctx->prog = bpf_jit_blind_constants(prog);
- if (IS_ERR(ctx->prog))
- return PTR_ERR(ctx->prog);
- ctx->blinded = (ctx->prog != ctx->orig_prog);
+ ctx->prog = prog;
/* If the verifier doesn't zero-extend, then we have to do it. */
ctx->do_zext = !ctx->prog->aux->verifier_zext;
@@ -214,14 +204,6 @@ static inline void maybe_free(struct jit_context *ctx, void **mem)
*/
static void jit_ctx_cleanup(struct jit_context *ctx)
{
- if (ctx->blinded) {
- /* if all went well, release the orig_prog. */
- if (ctx->success)
- bpf_jit_prog_release_other(ctx->prog, ctx->orig_prog);
- else
- bpf_jit_prog_release_other(ctx->orig_prog, ctx->prog);
- }
-
maybe_free(ctx, (void **)&ctx->bpf2insn);
maybe_free(ctx, (void **)&ctx->jit_data);
@@ -229,12 +211,19 @@ static void jit_ctx_cleanup(struct jit_context *ctx)
ctx->bpf2insn_valid = false;
/* Freeing "bpf_header" is enough. "jit.buf" is a sub-array of it. */
- if (!ctx->success && ctx->bpf_header) {
- bpf_jit_binary_free(ctx->bpf_header);
- ctx->bpf_header = NULL;
- ctx->jit.buf = NULL;
- ctx->jit.index = 0;
- ctx->jit.len = 0;
+ if (!ctx->success) {
+ if (ctx->bpf_header) {
+ bpf_jit_binary_free(ctx->bpf_header);
+ ctx->bpf_header = NULL;
+ ctx->jit.buf = NULL;
+ ctx->jit.index = 0;
+ ctx->jit.len = 0;
+ }
+ if (ctx->is_extra_pass) {
+ ctx->prog->bpf_func = NULL;
+ ctx->prog->jited = 0;
+ ctx->prog->jited_len = 0;
+ }
}
ctx->emit = false;
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..e6b1bb2de627 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2144,9 +2144,7 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
- bool tmp_blinded = false;
struct jit_ctx ctx;
unsigned int tmp_idx;
unsigned int image_size;
@@ -2156,20 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- /* If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
-
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
@@ -2179,10 +2164,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* we must fall back to the interpreter
*/
ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
- if (ctx.offsets == NULL) {
- prog = orig_prog;
- goto out;
- }
+ if (ctx.offsets == NULL)
+ return prog;
/* 1) fake pass to find in the length of the JITed code,
* to compute ctx->offsets and other context variables
@@ -2194,10 +2177,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* being successful in the second pass, so just fall back
* to the interpreter.
*/
- if (build_body(&ctx)) {
- prog = orig_prog;
+ if (build_body(&ctx))
goto out_off;
- }
tmp_idx = ctx.idx;
build_prologue(&ctx);
@@ -2213,10 +2194,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx += ctx.imm_count;
if (ctx.imm_count) {
ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
- if (ctx.imms == NULL) {
- prog = orig_prog;
+ if (ctx.imms == NULL)
goto out_off;
- }
}
#else
/* there's nothing about the epilogue on ARMv7 */
@@ -2238,10 +2217,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Not able to allocate memory for the structure then
* we must fall back to the interpretation
*/
- if (header == NULL) {
- prog = orig_prog;
+ if (header == NULL)
goto out_imms;
- }
/* 2.) Actual pass to generate final JIT code */
ctx.target = (u32 *) image_ptr;
@@ -2278,16 +2255,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
#endif
out_off:
kfree(ctx.offsets);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free:
image_ptr = NULL;
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_imms;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index adf84962d579..cd5a72fff500 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2009,14 +2009,12 @@ struct arm64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header = NULL;
struct arm64_jit_data *jit_data;
void __percpu *priv_stack_ptr = NULL;
bool was_classic = bpf_prog_was_classic(prog);
int priv_stack_alloc_sz;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
@@ -2025,26 +2023,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int exentry_idx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -2056,10 +2041,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -2079,10 +2062,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kvzalloc_objs(int, prog->len + 1);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_off;
- }
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
@@ -2095,15 +2076,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* BPF line info needs ctx->offset[i] to be the offset of
* instruction[i] in jited image, so build prologue first.
*/
- if (build_prologue(&ctx, was_classic)) {
- prog = orig_prog;
+ if (build_prologue(&ctx, was_classic))
goto out_off;
- }
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_off;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx, was_classic);
@@ -2121,10 +2098,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_off;
- }
/* Pass 2: Determine jited position and result for each instruction */
@@ -2152,10 +2127,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free_hdr;
- }
ctx.epilogue_offset = ctx.idx;
ctx.exentry_idx = exentry_idx;
@@ -2164,19 +2137,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Pass 3: Adjust jump offset and write final image */
if (build_body(&ctx, extra_pass) ||
- WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset)) {
- prog = orig_prog;
+ WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
- }
build_epilogue(&ctx, was_classic);
build_plt(&ctx);
/* Extra pass to validate JITed code. */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free_hdr;
- }
/* update the real prog size */
prog_size = sizeof(u32) * ctx.idx;
@@ -2193,16 +2162,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (extra_pass && ctx.idx > jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d > %d\n",
ctx.idx, jit_data->ctx.idx);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
goto out_free_hdr;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header has been freed */
ro_header = NULL;
- prog = orig_prog;
- goto out_off;
+ header = NULL;
+ goto out_free_hdr;
}
/*
* The instructions have now been copied to the ROX region from
@@ -2245,13 +2211,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size,
sizeof(header->size));
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 9cb796e16379..fcc8c0c29fb0 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1922,43 +1922,26 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- bool tmp_blinded = false, extra_pass = false;
+ bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header;
- struct bpf_prog *tmp, *orig_prog = prog;
/*
* If BPF JIT was not enabled then we must fall back to
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- if (IS_ERR(tmp))
- return orig_prog;
-
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1978,17 +1961,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.offset = kvcalloc(prog->len + 1, sizeof(u32), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_offset;
- }
/* 1. Initial fake pass to compute ctx->idx and set ctx->flags */
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_offset;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
@@ -2004,10 +1983,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
&header, &image_ptr, jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_offset;
- }
/* 2. Now, the actual pass to generate final JIT code */
/*
@@ -2027,17 +2004,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.num_exentries = 0;
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free;
- }
build_epilogue(&ctx);
/* 3. Extra pass to validate JITed code */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free;
- }
/* And we're done */
if (bpf_jit_enable > 1)
@@ -2050,9 +2023,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_free;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header have been freed */
ro_header = NULL;
- prog = orig_prog;
+ header = NULL;
goto out_free;
}
/*
@@ -2084,13 +2057,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ? tmp : orig_prog);
-
return prog;
out_free:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size, sizeof(header->size));
bpf_jit_binary_pack_free(ro_header, header);
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index e355dfca4400..d2b6c955f18e 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -911,10 +911,8 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
- bool tmp_blinded = false;
unsigned int tmp_idx;
unsigned int image_size;
u8 *image_ptr;
@@ -925,19 +923,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
- /*
- * If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.program = prog;
@@ -1025,14 +1011,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited_len = image_size;
out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
kfree(ctx.descriptors);
return prog;
out_err:
- prog = orig_prog;
if (header)
bpf_jit_binary_free(header);
goto out;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index a5eb6b51e27a..35dca372b5df 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -44,30 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, prologue_len, i;
struct hppa_jit_data *jit_data;
struct hppa_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
@@ -81,10 +70,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!ctx->offset)
+ goto out_err;
for (i = 0; i < prog->len; i++) {
prev_ninsns += 20;
ctx->offset[i] = prev_ninsns;
@@ -93,10 +80,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
for (i = 0; i < NR_JIT_ITERATIONS; i++) {
pass++;
ctx->ninsns = 0;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (build_body(ctx, extra_pass, ctx->offset))
+ goto out_err;
ctx->body_len = ctx->ninsns;
bpf_jit_build_prologue(ctx);
ctx->prologue_len = ctx->ninsns - ctx->body_len;
@@ -116,10 +101,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->image,
sizeof(long),
bpf_fill_ill_insns);
- if (!jit_data->header) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!jit_data->header)
+ goto out_err;
ctx->insns = (u32 *)jit_data->image;
/*
@@ -134,8 +117,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
if (jit_data->header)
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
if (extable_size)
@@ -148,8 +130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx);
if (build_body(ctx, extra_pass, NULL)) {
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
bpf_jit_build_epilogue(ctx);
@@ -160,20 +141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{ extern int machine_restart(char *); machine_restart(""); }
}
+ if (!prog->is_func || extra_pass) {
+ if (bpf_jit_binary_lock_ro(jit_data->header)) {
+ bpf_jit_binary_free(jit_data->header);
+ goto out_err;
+ }
+ bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
+ }
+
prog->bpf_func = (void *)ctx->insns;
prog->jited = 1;
prog->jited_len = prog_size;
- bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
-
if (!prog->is_func || extra_pass) {
- if (bpf_jit_binary_lock_ro(jit_data->header)) {
- bpf_jit_binary_free(jit_data->header);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
- goto out_offset;
- }
prologue_len = ctx->epilogue_offset - ctx->body_len;
for (i = 0; i < prog->len; i++)
ctx->offset[i] += prologue_len;
@@ -183,14 +163,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
+
if (HPPA_JIT_REBOOT)
{ extern int machine_restart(char *); machine_restart(""); }
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_offset;
}
u64 hppa_div64(u64 div, u64 divisor)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index a62a9a92b7b5..711028bebea3 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -142,9 +142,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
int flen;
struct bpf_binary_header *fhdr = NULL;
struct bpf_binary_header *hdr = NULL;
- struct bpf_prog *org_fp = fp;
- struct bpf_prog *tmp_fp;
- bool bpf_blinded = false;
bool extra_pass = false;
u8 *fimage = NULL;
u32 *fcode_base;
@@ -152,24 +149,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u32 fixup_len;
if (!fp->jit_requested)
- return org_fp;
-
- tmp_fp = bpf_jit_blind_constants(org_fp);
- if (IS_ERR(tmp_fp))
- return org_fp;
-
- if (tmp_fp != org_fp) {
- bpf_blinded = true;
- fp = tmp_fp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = org_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
@@ -194,10 +180,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
addrs = kcalloc(flen + 1, sizeof(*addrs), GFP_KERNEL);
- if (addrs == NULL) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (addrs == NULL)
+ goto out_err;
memset(&cgctx, 0, sizeof(struct codegen_context));
bpf_jit_init_reg_mapping(&cgctx);
@@ -211,11 +195,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
cgctx.exception_cb = fp->aux->exception_cb;
/* Scouting faux-generate pass 0 */
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
/* We hit something illegal or unsupported. */
- fp = org_fp;
- goto out_addrs;
- }
+ goto out_err;
/*
* If we have seen a tail call, we need a second pass.
@@ -226,10 +208,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
+ goto out_err;
}
bpf_jit_realloc_regs(&cgctx);
@@ -250,10 +230,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
bpf_jit_fill_ill_insns);
- if (!fhdr) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (!fhdr)
+ goto out_err;
if (extable_len)
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
@@ -272,8 +250,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
extra_pass)) {
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
bpf_jit_binary_pack_free(fhdr, hdr);
- fp = org_fp;
- goto out_addrs;
+ goto out_err;
}
bpf_jit_build_epilogue(code_base, &cgctx);
@@ -295,15 +272,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
((u64 *)image)[1] = local_paca->kernel_toc;
#endif
+ if (!fp->is_func || extra_pass) {
+ if (bpf_jit_binary_pack_finalize(fhdr, hdr))
+ goto out_err;
+ }
+
fp->bpf_func = (void *)fimage;
fp->jited = 1;
fp->jited_len = cgctx.idx * 4 + FUNCTION_DESCR_SIZE;
if (!fp->is_func || extra_pass) {
- if (bpf_jit_binary_pack_finalize(fhdr, hdr)) {
- fp = org_fp;
- goto out_addrs;
- }
bpf_prog_fill_jited_linfo(fp, addrs);
out_addrs:
kfree(addrs);
@@ -318,11 +296,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
jit_data->hdr = hdr;
}
-out:
- if (bpf_blinded)
- bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);
-
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto out_addrs;
}
/*
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index b3581e926436..527baa50dc68 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -44,29 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, i;
struct rv_jit_data *jit_data;
struct rv_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
if (!jit_data) {
- prog = orig_prog;
- goto out;
+ return prog;
}
prog->aux->jit_data = jit_data;
}
@@ -83,15 +73,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
+ if (!ctx->offset)
goto out_offset;
- }
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_offset;
- }
for (i = 0; i < prog->len; i++) {
prev_ninsns += 32;
@@ -105,10 +91,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
ctx->prologue_len = ctx->ninsns;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, ctx->offset))
goto out_offset;
- }
ctx->epilogue_offset = ctx->ninsns;
bpf_jit_build_epilogue(ctx);
@@ -126,10 +110,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
bpf_fill_ill_insns);
- if (!jit_data->ro_header) {
- prog = orig_prog;
+ if (!jit_data->ro_header)
goto out_offset;
- }
/*
* Use the image(RW) for writing the JITed instructions. But also save
@@ -150,7 +132,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (i == NR_JIT_ITERATIONS) {
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
- prog = orig_prog;
goto out_free_hdr;
}
@@ -163,26 +144,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->nexentries = 0;
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_free_hdr;
- }
bpf_jit_build_epilogue(ctx);
if (bpf_jit_enable > 1)
bpf_jit_dump(prog->len, prog_size, pass, ctx->insns);
- prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
- prog->jited = 1;
- prog->jited_len = prog_size - cfi_get_offset();
-
if (!prog->is_func || extra_pass) {
if (WARN_ON(bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header))) {
/* ro_header has been freed */
jit_data->ro_header = NULL;
- prog = orig_prog;
- goto out_offset;
+ jit_data->header = NULL;
+ goto out_free_hdr;
}
+ }
+
+ prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
+ prog->jited = 1;
+ prog->jited_len = prog_size - cfi_get_offset();
+
+ if (!prog->is_func || extra_pass) {
/*
* The instructions have now been copied to the ROX region from
* where they will execute.
@@ -198,14 +180,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (jit_data->header) {
bpf_arch_text_copy(&jit_data->ro_header->size, &jit_data->header->size,
sizeof(jit_data->header->size));
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index d08d159b6319..2dfc279b1be2 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2314,36 +2314,20 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
*/
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
- struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
- bool tmp_blinded = false;
bool extra_pass = false;
struct bpf_jit jit;
int pass;
if (!fp->jit_requested)
- return orig_fp;
-
- tmp = bpf_jit_blind_constants(fp);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_fp;
- if (tmp != fp) {
- tmp_blinded = true;
- fp = tmp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = orig_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
if (jit_data->ctx.addrs) {
@@ -2356,34 +2340,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
memset(&jit, 0, sizeof(jit));
jit.addrs = kvcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
- if (jit.addrs == NULL) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (jit.addrs == NULL)
+ goto out_err;
/*
* Three initial passes:
* - 1/2: Determine clobbered registers
* - 3: Calculate program size and addrs array
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp, extra_pass)) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (bpf_jit_prog(&jit, fp, extra_pass))
+ goto out_err;
}
/*
* Final pass: Allocate and generate program
*/
header = bpf_jit_alloc(&jit, fp);
- if (!header) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (!header)
+ goto out_err;
skip_init_ctx:
if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
if (bpf_jit_enable > 1) {
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
@@ -2392,8 +2369,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
if (!fp->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
} else {
jit_data->header = header;
@@ -2411,11 +2387,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
kfree(jit_data);
fp->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(fp, fp == orig_fp ?
- tmp : orig_fp);
+
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto free_addrs;
}
bool bpf_jit_supports_kfunc_call(void)
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index b23d1c645ae5..e83e29137566 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1479,37 +1479,22 @@ struct sparc64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
u32 prev_image_size, image_size;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
int pass, i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1527,10 +1512,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kmalloc_array(prog->len, sizeof(unsigned int), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (ctx.offset == NULL)
+ goto out_err;
/* Longest sequence emitted is for bswap32, 12 instructions. Pre-cook
* the offset array so that we converge faster.
@@ -1543,10 +1526,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx = 0;
build_prologue(&ctx);
- if (build_body(&ctx)) {
- prog = orig_prog;
- goto out_off;
- }
+ if (build_body(&ctx))
+ goto out_err;
build_epilogue(&ctx);
if (bpf_jit_enable > 1)
@@ -1569,10 +1550,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = sizeof(u32) * ctx.idx;
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
- if (header == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (header == NULL)
+ goto out_err;
ctx.image = (u32 *)image_ptr;
skip_init_ctx:
@@ -1582,8 +1561,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (build_body(&ctx)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
build_epilogue(&ctx);
@@ -1592,8 +1570,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf_jit: Failed to converge, prev_size=%u size=%d\n",
prev_image_size, ctx.idx * 4);
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
if (bpf_jit_enable > 1)
@@ -1604,8 +1581,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (!prog->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
} else {
jit_data->ctx = ctx;
@@ -1624,9 +1600,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_off;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e9b78040d703..77d00a8dec87 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3717,13 +3717,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
void __percpu *priv_stack_ptr = NULL;
struct x64_jit_data *jit_data;
int priv_stack_alloc_sz;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
bool extra_pass = false;
bool padding = false;
u8 *rw_image = NULL;
@@ -3733,27 +3731,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -3765,10 +3749,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 8) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 8, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -3786,10 +3768,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto skip_init_addrs;
}
addrs = kvmalloc_objs(*addrs, prog->len + 1);
- if (!addrs) {
- prog = orig_prog;
+ if (!addrs)
goto out_addrs;
- }
/*
* Before first pass, make a rough estimation of addrs[]
@@ -3820,8 +3800,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
sizeof(rw_header->size));
bpf_jit_binary_pack_free(header, rw_header);
}
- /* Fall back to interpreter mode */
- prog = orig_prog;
if (extra_pass) {
prog->bpf_func = NULL;
prog->jited = 0;
@@ -3852,10 +3830,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
prog->aux->extable = (void *) image + roundup(proglen, align);
}
oldproglen = proglen;
@@ -3908,8 +3884,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image + cfi_get_offset();
prog->jited = 1;
prog->jited_len = proglen - cfi_get_offset();
- } else {
- prog = orig_prog;
}
if (!image || !prog->is_func || extra_pass) {
@@ -3925,10 +3899,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
}
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index dda423025c3d..5f259577614a 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2521,35 +2521,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
u8 *image = NULL;
int *addrs;
int pass;
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
addrs = kmalloc_objs(*addrs, prog->len);
- if (!addrs) {
- prog = orig_prog;
- goto out;
- }
+ if (!addrs)
+ return prog;
/*
* Before first pass, make a rough estimation of addrs[]
@@ -2574,7 +2558,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image = NULL;
if (header)
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_addrs;
}
if (image) {
@@ -2588,10 +2571,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (proglen == oldproglen) {
header = bpf_jit_binary_alloc(proglen, &image,
1, jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
}
oldproglen = proglen;
cond_resched();
@@ -2604,16 +2585,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image;
prog->jited = 1;
prog->jited_len = proglen;
- } else {
- prog = orig_prog;
}
out_addrs:
kfree(addrs);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
}
diff --git a/include/linux/filter.h b/include/linux/filter.h
index e40d4071a345..d396e55c9a1d 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1183,6 +1183,18 @@ static inline bool bpf_dump_raw_ok(const struct cred *cred)
struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
const struct bpf_insn *patch, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len);
+#else
+static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+#endif /* CONFIG_BPF_SYSCALL */
+
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
static inline bool xdp_return_frame_no_direct(void)
@@ -1309,9 +1321,14 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
const char *bpf_jit_get_prog_name(struct bpf_prog *prog);
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *fp);
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other);
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return prog->blinding_requested && !prog->blinded;
+}
+
static inline void bpf_jit_dump(unsigned int flen, unsigned int proglen,
u32 pass, void *image)
{
@@ -1450,6 +1467,20 @@ static inline void bpf_prog_kallsyms_del(struct bpf_prog *fp)
{
}
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return false;
+}
+
+static inline
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
+{
+ return prog;
+}
+
+static inline void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
+{
+}
#endif /* CONFIG_BPF_JIT */
void bpf_prog_kallsyms_del_all(struct bpf_prog *fp);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 066b86e7233c..fc9fb3c07866 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1508,7 +1508,11 @@ static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
#endif
}
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
+/*
+ * Now this function is used only to blind the main prog and must be invoked only when
+ * bpf_prog_need_blind() returns true.
+ */
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
struct bpf_prog *clone, *tmp;
@@ -1516,13 +1520,17 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
struct bpf_insn *insn;
int i, rewritten;
- if (!prog->blinding_requested || prog->blinded)
- return prog;
+ if (WARN_ON_ONCE(env && env->prog != prog))
+ return ERR_PTR(-EINVAL);
clone = bpf_prog_clone_create(prog, GFP_USER);
if (!clone)
return ERR_PTR(-ENOMEM);
+ /* make sure bpf_patch_insn_data() patches the correct prog */
+ if (env)
+ env->prog = clone;
+
insn_cnt = clone->len;
insn = clone->insnsi;
@@ -1550,21 +1558,35 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!rewritten)
continue;
- tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
- if (IS_ERR(tmp)) {
+ if (env)
+ tmp = bpf_patch_insn_data(env, i, insn_buff, rewritten);
+ else
+ tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
+
+ if (IS_ERR_OR_NULL(tmp)) {
+ if (env)
+ /* restore the original prog */
+ env->prog = prog;
/* Patching may have repointed aux->prog during
* realloc from the original one, so we need to
* fix it up here on error.
*/
bpf_jit_prog_release_other(prog, clone);
- return tmp;
+ return IS_ERR(tmp) ? tmp : ERR_PTR(-ENOMEM);
}
clone = tmp;
insn_delta = rewritten - 1;
- /* Instructions arrays must be updated using absolute xlated offsets */
- adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+ if (env)
+ env->prog = clone;
+ else
+ /*
+ * Instructions arrays must be updated using absolute xlated offsets.
+ * The arrays have already been adjusted by bpf_patch_insn_data() when
+ * env is not NULL.
+ */
+ adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2533,6 +2555,35 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+{
+#ifdef CONFIG_BPF_JIT
+ struct bpf_prog *orig_prog;
+
+ if (!bpf_prog_need_blind(prog))
+ return bpf_int_jit_compile(prog);
+
+ orig_prog = prog;
+ prog = bpf_jit_blind_constants(NULL, prog);
+ /*
+ * If blinding was requested and we failed during blinding, we must fall
+ * back to the interpreter.
+ */
+ if (IS_ERR(prog))
+ return orig_prog;
+
+ prog = bpf_int_jit_compile(prog);
+ if (prog->jited) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ return prog;
+ }
+
+ bpf_jit_prog_release_other(orig_prog, prog);
+ prog = orig_prog;
+#endif
+ return prog;
+}
+
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
* @fp: bpf_prog populated with BPF program
@@ -2572,7 +2623,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_int_jit_compile(fp);
+ fp = bpf_prog_jit_compile(fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 67c9b28767e1..ec8afd6a9369 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -232,8 +232,8 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-static struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -973,7 +973,45 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
return 0;
}
-int bpf_jit_subprogs(struct bpf_verifier_env *env)
+static u32 *bpf_dup_subprog_starts(struct bpf_verifier_env *env)
+{
+ u32 *starts = NULL;
+
+ starts = kvmalloc_objs(u32, env->subprog_cnt, GFP_KERNEL_ACCOUNT);
+ if (starts) {
+ for (int i = 0; i < env->subprog_cnt; i++)
+ starts[i] = env->subprog_info[i].start;
+ }
+ return starts;
+}
+
+static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)
+{
+ for (int i = 0; i < env->subprog_cnt; i++)
+ env->subprog_info[i].start = orig_starts[i];
+}
+
+static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ size_t size;
+ void *new_aux;
+
+ size = array_size(sizeof(struct bpf_insn_aux_data), env->prog->len);
+ new_aux = __vmalloc(size, GFP_KERNEL_ACCOUNT);
+ if (new_aux)
+ memcpy(new_aux, env->insn_aux_data, size);
+ return new_aux;
+}
+
+static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+ /* the expanded elements are zero-filled, so no special handling is required */
+ vfree(env->insn_aux_data);
+ env->insn_aux_data = orig_insn_aux;
+}
+
+static int jit_subprogs(struct bpf_verifier_env *env)
{
struct bpf_prog *prog = env->prog, **func, *tmp;
int i, j, subprog_start, subprog_end = 0, len, subprog;
@@ -981,10 +1019,6 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
struct bpf_insn *insn;
void *old_bpf_func;
int err, num_exentries;
- int old_len, subprog_start_adjustment = 0;
-
- if (env->subprog_cnt <= 1)
- return 0;
for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
if (!bpf_pseudo_func(insn) && !bpf_pseudo_call(insn))
@@ -1053,10 +1087,11 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
goto out_free;
func[i]->is_func = 1;
func[i]->sleepable = prog->sleepable;
+ func[i]->blinded = prog->blinded;
func[i]->aux->func_idx = i;
/* Below members will be freed only at prog->aux */
func[i]->aux->btf = prog->aux->btf;
- func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
+ func[i]->aux->subprog_start = subprog_start;
func[i]->aux->func_info = prog->aux->func_info;
func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -1112,15 +1147,7 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
-
- /*
- * To properly pass the absolute subprog start to jit
- * all instruction adjustments should be accumulated
- */
- old_len = func[i]->len;
func[i] = bpf_int_jit_compile(func[i]);
- subprog_start_adjustment += func[i]->len - old_len;
-
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1246,16 +1273,87 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
}
kfree(func);
out_undo_insn:
+ bpf_prog_jit_attempt_done(prog);
+ return err;
+}
+
+int bpf_jit_subprogs(struct bpf_verifier_env *env)
+{
+ int err, i;
+ bool blinded = false;
+ struct bpf_insn *insn;
+ struct bpf_prog *prog, *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
+ u32 *orig_subprog_starts;
+
+ if (env->subprog_cnt <= 1)
+ return 0;
+
+ prog = orig_prog = env->prog;
+ if (bpf_prog_need_blind(prog)) {
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux) {
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ orig_subprog_starts = bpf_dup_subprog_starts(env);
+ if (!orig_subprog_starts) {
+ vfree(orig_insn_aux);
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ prog = bpf_jit_blind_constants(env, prog);
+ if (IS_ERR(prog)) {
+ err = -ENOMEM;
+ prog = orig_prog;
+ goto out_restore;
+ }
+ blinded = true;
+ }
+
+ err = jit_subprogs(env);
+ if (err)
+ goto out_jit_err;
+
+ if (blinded) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ kvfree(orig_subprog_starts);
+ vfree(orig_insn_aux);
+ }
+
+ return 0;
+
+out_jit_err:
+ if (blinded) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ /* roll back to the clean original prog */
+ prog = env->prog = orig_prog;
+ goto out_restore;
+ } else {
+ if (err != -EFAULT) {
+ /*
+ * We will fall back to interpreter mode when err is not -EFAULT, before
+ * that, insn->off and insn->imm should be restored to their original
+ * values since they were modified by __jit_subprogs.
+ */
+ for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
+ if (!bpf_pseudo_call(insn))
+ continue;
+ insn->off = 0;
+ insn->imm = env->insn_aux_data[i].call_imm;
+ }
+ }
+ goto out_cleanup;
+ }
+
+out_restore:
+ bpf_restore_subprog_starts(env, orig_subprog_starts);
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
+ kvfree(orig_subprog_starts);
+out_cleanup:
/* cleanup main prog to be interpreted */
prog->jit_requested = 0;
prog->blinding_requested = 0;
- for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
- if (!bpf_pseudo_call(insn))
- continue;
- insn->off = 0;
- insn->imm = env->insn_aux_data[i].call_imm;
- }
- bpf_prog_jit_attempt_done(prog);
return err;
}
--
2.47.3
^ permalink raw reply related
* Re: [PATCH v2] pmdomain: imx: Make IMX8M/IMX9 BLK_CTRL tristate
From: Marco Felsch @ 2026-04-13 13:11 UTC (permalink / raw)
To: Zhipeng Wang
Cc: ulfh, Frank.Li, s.hauer, imx, linux-pm, xuegang.liu, jindong.yue,
linux-kernel, kernel, festevam, linux-arm-kernel
In-Reply-To: <20260413053049.3041177-1-zhipeng.wang_1@nxp.com>
On 26-04-13, Zhipeng Wang wrote:
> Convert IMX8M_BLK_CTRL and IMX9_BLK_CTRL from bool to tristate
> to allow building as loadable modules.
Out of curiosity, why do you want to have a PM driver to be buildable as
module?
Regards,
Marco
> Add prompt strings to make these options visible and configurable
> in menuconfig, keeping them enabled by default on appropriate platforms.
>
> Also remove the IMX_GPCV2_PM_DOMAINS dependency from IMX9_BLK_CTRL.
> This dependency was incorrect from the beginning - i.MX93 uses a
> different power domain architecture compared to i.MX8M series:
>
> - i.MX8M uses GPCv2 (General Power Controller v2) for power domain
> management, hence IMX8M_BLK_CTRL correctly depends on it.
>
> - i.MX93 uses BLK_CTRL directly without GPCv2. The hardware doesn't
> have GPCv2 at all.
>
> Signed-off-by: Zhipeng Wang <zhipeng.wang_1@nxp.com>
> ---
> drivers/pmdomain/imx/Kconfig | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/pmdomain/imx/Kconfig b/drivers/pmdomain/imx/Kconfig
> index 00203615c65e..9168d183b0c5 100644
> --- a/drivers/pmdomain/imx/Kconfig
> +++ b/drivers/pmdomain/imx/Kconfig
> @@ -10,15 +10,18 @@ config IMX_GPCV2_PM_DOMAINS
> default y if SOC_IMX7D
>
> config IMX8M_BLK_CTRL
> - bool
> - default SOC_IMX8M && IMX_GPCV2_PM_DOMAINS
> + tristate "i.MX8M BLK CTRL driver"
> + depends on SOC_IMX8M
> + depends on IMX_GPCV2_PM_DOMAINS
> depends on PM_GENERIC_DOMAINS
> depends on COMMON_CLK
> + default y
>
> config IMX9_BLK_CTRL
> - bool
> - default SOC_IMX9 && IMX_GPCV2_PM_DOMAINS
> + tristate "i.MX93 BLK CTRL driver"
> + depends on SOC_IMX9
> depends on PM_GENERIC_DOMAINS
> + default y
>
> config IMX_SCU_PD
> bool "IMX SCU Power Domain driver"
> --
> 2.34.1
>
>
>
--
#gernperDu
#CallMeByMyFirstName
Pengutronix e.K. | |
Steuerwalder Str. 21 | https://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox