* Re: [EXT] Re: [PATCH v2] pmdomain: imx: Make IMX8M/IMX9 BLK_CTRL tristate
From: Daniel Baluta @ 2026-04-16 6:04 UTC (permalink / raw)
To: Zhipeng Wang, Marco Felsch
Cc: ulfh@kernel.org, Frank Li, s.hauer@pengutronix.de,
imx@lists.linux.dev, linux-pm@vger.kernel.org, Xuegang Liu,
Jindong Yue, linux-kernel@vger.kernel.org, kernel@pengutronix.de,
festevam@gmail.com, linux-arm-kernel@lists.infradead.org
In-Reply-To: <AMBPR04MB123344C1F802528A10CEA7FADEB252@AMBPR04MB12334.eurprd04.prod.outlook.com>
On 4/14/26 04:59, Zhipeng Wang wrote:
> > On 26-04-13, Zhipeng Wang wrote:
>>> Convert IMX8M_BLK_CTRL and IMX9_BLK_CTRL from bool to tristate to
>>> allow building as loadable modules.
>> Out of curiosity, why do you want to have a PM driver to be buildable as
>> module?
>>
>> Regards,
>> Marco
>>
> Hi Marco,
>
> Thank you for your question.
>
> The primary motivation is to support Google's GKI (Generic Kernel Image)
> requirement for Android devices.
>
> GKI separates the kernel into two parts:
> 1. A unified kernel image (GKI) that is common across all Android devices
> 2. Vendor-specific drivers that must be built as loadable modules
>
> Under the GKI architecture, SoC-specific drivers like IMX8M/IMX9 BLK_CTRL
> cannot be built into the core kernel image. Instead, they must be loadable
> modules that vendors can ship separately. This allows:
>
> - A single kernel binary to support multiple hardware platforms
> - Vendors to update their drivers independently without rebuilding the entire kernel
> - Better compliance with Android's kernel update and security policies
>
Can you please add the below line in the commit message?
> For i.MX8M/i.MX9 devices running Android with GKI kernels, the BLK_CTRL
> drivers need to be loaded as modules during boot. Without tristate support,
> these devices cannot properly initialize their power domains, making them
> non-functional under GKI.
^ permalink raw reply
* Re: [PATCH net-next 5/6] net: stmmac: move PHY handling out of __stmmac_open()/release()
From: Alexander Stein @ 2026-04-16 6:20 UTC (permalink / raw)
To: Russell King (Oracle)
Cc: Andrew Lunn, Heiner Kallweit, Alexandre Torgue, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, linux-arm-kernel,
linux-stm32, Maxime Coquelin, netdev, Paolo Abeni
In-Reply-To: <ad-LtOBrKREM1tCk@shell.armlinux.org.uk>
Am Mittwoch, 15. April 2026, 14:59:32 CEST schrieb Russell King (Oracle):
> On Wed, Apr 15, 2026 at 08:08:40AM +0200, Alexander Stein wrote:
> > Hi,
> >
> > Am Dienstag, 23. September 2025, 13:26:19 CEST schrieb Russell King (Oracle):
> > > Move the PHY attachment/detachment from the network driver out of
> > > __stmmac_open() and __stmmac_release() into stmmac_open() and
> > > stmmac_release() where these actions will only happen when the
> > > interface is administratively brought up or down. It does not make
> > > sense to detach and re-attach the PHY during a change of MTU.
> >
> > Sorry for coming up now. But I recently noticed this commit breaks changing
> > the MTU on i.MX8MP. Once I simply change the MTU I run into some DMA error:
> > $ ip link set dev end1 mtu 1400
> > imx-dwmac 30bf0000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-0
> > imx-dwmac 30bf0000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-1
> > imx-dwmac 30bf0000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-2
> > imx-dwmac 30bf0000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-3
> > imx-dwmac 30bf0000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-4
> > imx-dwmac 30bf0000.ethernet end1: Link is Down
> > imx-dwmac 30bf0000.ethernet end1: Failed to reset the dma
> > imx-dwmac 30bf0000.ethernet end1: stmmac_hw_setup: DMA engine initialization failed
>
> This basically means that a clock is missing. Please provide more
> information:
>
> - what kernel version are you using?
Currently I am using v6.18.22.
$ ethtool -i end1
driver: st_gmac
version: 6.18.22
firmware-version:
expansion-rom-version:
bus-info: 30bf0000.ethernet
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
> - has EEE been negotiated?
No. It is marked as not supported
$ ethtool --show-eee end1
EEE settings for end1:
EEE status: not supported
> - does the problem persist when EEE is disabled?
As EEE is not supported the problem occurs even with EEE disabled.
> - which PHY is attached to stmmac?
It is a TI DP83867.
imx-dwmac 30bf0000.ethernet eth1: PHY [stmmac-1:03] driver [TI DP83867] (irq=136)
> - which PHY interface mode is being used to connect the PHY to stmmac?
For this interface
> phy-mode = "rgmii-id";
is set.
In case it is helpful. My platform is arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts
Thanks for assisting. If there a further questions, don't hesitate to ask.
Thanks and best regards
Alexander
--
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/
^ permalink raw reply
* Re: [PATCH v3 3/3] dt-bindings: i3c: Add AST2600 I3C global registers
From: Krzysztof Kozlowski @ 2026-04-16 6:21 UTC (permalink / raw)
To: Dawid Glazik
Cc: Alexandre Belloni, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, linux-aspeed, linux-i3c, devicetree,
linux-arm-kernel, Frank Li, Maciej Lawniczak
In-Reply-To: <cb0d16bf-988d-403e-8a8e-c85bf2a208d0@linux.intel.com>
On 15/04/2026 20:21, Dawid Glazik wrote:
> On 4/9/2026 9:30 AM, Krzysztof Kozlowski wrote:
>> On 09/04/2026 09:28, Krzysztof Kozlowski wrote:
>>> On Wed, Apr 08, 2026 at 10:34:35PM +0200, Dawid Glazik wrote:
>>>> Introduce the device-tree bindings for I3C global registers found on
>>>> AST2600 SoCs.
>>>>
>>>> Signed-off-by: Dawid Glazik <dawid.glazik@linux.intel.com>
>>>> ---
>>>> I wasn't sure if I should add newline at the end of the
>>>> file or not so I took
>>>> https://github.com/torvalds/linux/tree/master/Documentation/devicetree/bindings/i3c
>>>> as an example.
>>>
>>> Answer is: you cannot have patch warnings.
>>>
>>> Documentation/devicetree/bindings/i3c does not have patch warning, does
>>> it?
>>
>> And if you tested this code with standard tools, you would see that...
>>
>> Best regards,
>> Krzysztof
>
> Thank you for the review and feedback. This is my first contribution to
> Linux kernel so I'm still learning the process and toolchain. I
> apologize for the rookie mistakes. I will address all the issues you've
> pointed out and resubmit the series.
So get the patch reviewed by Intel colleagues which would tell you what
tools you must run and what warnings are accepted or not (and patch
warning is never accepted).
Best regards,
Krzysztof
^ permalink raw reply
* Re: [PATCH v8 next 00/10] arm_mpam: Introduce Narrow-PARTID feature
From: Shaopeng Tan (Fujitsu) @ 2026-04-16 6:29 UTC (permalink / raw)
To: ben.horgan@arm.com, Dave.Martin@arm.com, james.morse@arm.com,
reinette.chatre@intel.com, fenghuay@nvidia.com, tglx@kernel.org,
will@kernel.org, hpa@zytor.com, bp@alien8.de, babu.moger@amd.com,
dave.hansen@linux.intel.com, mingo@redhat.com,
tony.luck@intel.com, gshan@redhat.com, catalin.marinas@arm.com
Cc: linux-arm-kernel@lists.infradead.org, x86@kernel.org,
linux-kernel@vger.kernel.org, wangkefeng.wang@huawei.com
In-Reply-To: <20260413085405.1166412-1-zengheng4@huawei.com>
Hello Zeng Heng,
Could you tell me which branch this patch series based on?
Best regards,
Shaopent TAN
^ permalink raw reply
* Re: [PATCH] arm_pmu: acpi: fix reference leak on failed device registration
From: Guangshuo Li @ 2026-04-16 6:34 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mark Rutland, Will Deacon, Anshuman Khandual, linux-arm-kernel,
linux-perf-users, linux-kernel, stable
In-Reply-To: <2026041603-guts-crested-ef76@gregkh>
Hi Mark, Greg,
Thanks for the feedback.
On Thu, 16 Apr 2026 at 12:41, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Wed, Apr 15, 2026 at 07:19:06PM +0100, Mark Rutland wrote:
> > Hi,
> >
> > Thanks for the patch, but from a quick skim, I don't think this is the right
> > fix.
> >
> > Greg, I think we might want to rework the core API here; question for
> > you at the end.
> >
> > On Thu, Apr 16, 2026 at 01:41:59AM +0800, Guangshuo Li wrote:
> > > When platform_device_register() fails in arm_acpi_register_pmu_device(),
> > > the embedded struct device in pdev has already been initialized by
> > > device_initialize(), but the failure path only unregisters the GSI and
> > > does not drop the device reference for the current platform device:
> > >
> > > arm_acpi_register_pmu_device()
> > > -> platform_device_register(pdev)
> > > -> device_initialize(&pdev->dev)
> > > -> setup_pdev_dma_masks(pdev)
> > > -> platform_device_add(pdev)
> > >
> > > This leads to a reference leak when platform_device_register() fails.
> >
> > AFAICT you're saying that the reference was taken *within*
> > platform_device_register(), and then platform_device_register() itself
> > has failed. I think it's surprising that platform_device_register()
> > doesn't clean that up itself in the case of an error.
> >
> > There are *tonnes* of calls to platform_device_register() throughout the
> > kernel that don't even bother to check the return value, and many that
> > just pass the return onto a caller that can't possibly know to call
> > platform_device_put().
> >
> > Code in the same file as platform_device_register() expects it to clean up
> > after itself, e.g.
> >
> > | int platform_add_devices(struct platform_device **devs, int num)
> > | {
> > | int i, ret = 0;
> > |
> > | for (i = 0; i < num; i++) {
> > | ret = platform_device_register(devs[i]);
> > | if (ret) {
> > | while (--i >= 0)
> > | platform_device_unregister(devs[i]);
> > | break;
> > | }
> > | }
> > |
> > | return ret;
> > | }
> >
> > That's been there since the initial git commit, and back then,
> > platform_device_register() didn't mention that callers needed to perform
> > any cleanup.
> >
> > I see a comment was added to platform_device_register() in commit:
> >
> > 67e532a42cf4 ("driver core: platform: document registration-failure requirement")
> >
> > ... and that copied the commend added for device_register() in commit:
> >
> > 5739411acbaa ("Driver core: Clarify device cleanup.")
> >
> > ... but the potential brokenness is so widespread, and the behaviour is
> > so surprising, that I'd argue the real but is that device_register()
> > doesn't clean up in case of error. I don't think it's worth changing
> > this single instance given the prevalance and churn fixing all of that
> > would involve.
> >
> > I think it would be far better to fix the core driver API such that when
> > those functions return an error, they've already cleaned up for
> > themselves.
> >
> > Greg, am I missing some functional reason why we can't rework
> > device_register() and friends to handle cleanup themselves? I appreciate
> > that'll involve churn for some callers, but AFAICT the majority of
> > callers don't have the required cleanup.
>
> Yes, we should fix the platform core code here, this should not be
> required to do everywhere as obviously we all got it wrong.
>
> Guangshuo, can you submit a patch to do that instead and ask for all of
> your other patches to not be applied as well?
>
> thanks,
>
> greg k-h
I agree that fixing this in the platform core makes more sense than
handling it in individual callers.
I'll look into the core code and send a patch for that instead. I'll
also ask for my other related patches not to be applied.
Thanks,
Guangshuo
^ permalink raw reply
* Re: [PATCH 0/7] TQMLX2160A-MBLS2160A DT fixes/updates
From: Alexander Stein @ 2026-04-16 6:39 UTC (permalink / raw)
To: Frank Li, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Shawn Guo
Cc: linux-arm-kernel, linux, devicetree, linux-kernel, Nora Schiffer
In-Reply-To: <20260331141915.2918927-1-alexander.stein@ew.tq-group.com>
Hi Frank,
Am Dienstag, 31. März 2026, 16:19:01 CEST schrieb Alexander Stein:
> Hi,
>
> this series adds small fixes and improvements for TQMLX2160A DTs.
> The DT overlays address specific hardware behaviour when serdes is configured
> differently.
Any feedback here?
Thanks and best regards
Alexander
>
> Best regards,
> Alexander
>
> Alexander Stein (1):
> arm64: dts: fsl-lx2160a-tqmlx2160a: Remove deprecated properties
>
> Nora Schiffer (6):
> arm64: dts: fsl-lx2160a-tqmlx2160a: fix LED polarity
> arm64: dts: fsl-lx2160a-tqmlx2160a-mblx2160a: use DPMAC 17 and 18 for
> SGMII in SERDES2 configs 7 and 11
> arm64: dts: fsl-lx2160a-tqmlx2160a: add aliases for all 18 DPMAC
> instances
> arm64: dts: fsl-lx2160a-tqmlx2160a-mbls2160a: add various GPIO hogs
> arm64: dts: fsl-lx2160a-tqmlx2160a-mbls2160a: enable pcs_mdio17 and
> pcs_mdio18 in appropriate overlays
> arm64: dts: fsl-lx2160a-tqmlx2160a-mbls2160a: specify Ethernet PHY
> reset GPIOs
>
> .../fsl-lx2160a-tqmlx2160a-mblx2160a.dts | 306 +++++++++++++++++-
> ...l-lx2160a-tqmlx2160a-mblx2160a_x_11_x.dtso | 20 ++
> ...sl-lx2160a-tqmlx2160a-mblx2160a_x_7_x.dtso | 20 ++
> .../dts/freescale/fsl-lx2160a-tqmlx2160a.dtsi | 23 +-
> 4 files changed, 357 insertions(+), 12 deletions(-)
>
>
--
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/
^ permalink raw reply
* Re: [PATCH] arm64: cpufeature: Fix GCIE field ordering in ftr_id_aa64pfr2
From: Marc Zyngier @ 2026-04-16 6:42 UTC (permalink / raw)
To: Mukesh Ojha; +Cc: Catalin Marinas, Will Deacon, linux-arm-kernel, linux-kernel
In-Reply-To: <20260415200031.1885440-1-mukesh.ojha@oss.qualcomm.com>
On Wed, 15 Apr 2026 21:00:31 +0100,
Mukesh Ojha <mukesh.ojha@oss.qualcomm.com> wrote:
>
> The ftr_id_aa64pfr2[] array must be sorted in descending order of
> shift value so that the overlap validation in init_cpu_features()
> works correctly. The GCIE field (bits 15:12, shift=12) was placed
> last in the array, after MTEFAR (bits 11:8, shift=8) and
> MTESTOREONLY (bits 7:4, shift=4), causing a spurious warning at
> boot:
>
> [ 0.000000] SYS_ID_AA64PFR2_EL1 has feature overlap at shift 12
> [ 0.000000] WARNING: arch/arm64/kernel/cpufeature.c:989 at init_cpu_features+0x144/0x3d0, CPU#0:
> swapper/0
> ..
>
> [ 0.000000] pc : init_cpu_features+0x144/0x3d0
> [ 0.000000] lr : init_cpu_features+0x144/0x3d0
> [ 0.000000] sp : ffffc08678f03dc0
>
> ...
> 0.000000] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffffc08678f14000
> [ 0.000000] Call trace:
> [ 0.000000] init_cpu_features+0x144/0x3d0 (P)
> [ 0.000000] cpuinfo_store_boot_cpu+0x4c/0x5c
> [ 0.000000] smp_prepare_boot_cpu+0x28/0x38
> [ 0.000000] start_kernel+0x1d4/0x848
> [ 0.000000] __primary_switched+0x88/0x90
>
> This is because the overlap check computes (shift + width) > prev_shift,
> i.e. (12 + 4) > 8, which triggers since GCIE occupies bits above MTEFAR
> but was listed after it.
>
> Fix the ordering to match the register layout: FPMR(35:32), GCIE(15:12),
> MTEFAR(11:8), MTESTOREONLY(7:4).
>
> Fixes: 899ff451fcee ("KVM: arm64: Advertise ID_AA64PFR2_EL1.GCIE")
> Signed-off-by: Mukesh Ojha <mukesh.ojha@oss.qualcomm.com>
This was present in next-20260403, identified the following day, a fix
proposed in [1], and the fixed commit appeared in next-20260407 (which
was the subsequent -next build) as 7e629348df81b.
May I humbly suggest that you check with the latest -next branch
before spending time on this sort of things? Two weeks is a pretty
long time...
Thanks,
M.
[1] https://lore.kernel.org/all/874ilqcu3c.wl-maz@kernel.org/
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply
* Re: [PATCH v5 07/12] coresight: etm4x: fix inconsistencies with sysfs configuration
From: Yeoreum Yun @ 2026-04-16 6:49 UTC (permalink / raw)
To: Jie Gan
Cc: coresight, linux-arm-kernel, linux-kernel, suzuki.poulose,
mike.leach, james.clark, alexander.shishkin, leo.yan
In-Reply-To: <b9528488-b0c7-410b-b91b-b05c21fd0c08@oss.qualcomm.com>
Hi Jie,
>
>
> On 4/16/2026 12:55 AM, Yeoreum Yun wrote:
> > The current ETM4x configuration via sysfs can lead to
> > several inconsistencies:
> >
> > - If the configuration is modified via sysfs while a perf session is
> > active, the running configuration may differ before a sched-out and
> > after a subsequent sched-in.
> >
> > - If a perf session and a sysfs session enable tracing concurrently,
> > the configuration from configfs may become corrupted.
> >
> > - There is a risk of corrupting drvdata->config if a perf session enables
> > tracing while cscfg_csdev_disable_active_config() is being handled in
> > etm4_disable_sysfs().
> >
> > To resolve these issues, separate the configuration into:
> >
> > - active_config: the configuration applied to the current session
> > - config: the configuration set via sysfs
> >
> > Additionally:
> >
> > - Apply the configuration from configfs after taking the appropriate mode.
> >
> > - Since active_config and related fields are accessed only by the local CPU
> > in etm4_enable/disable_sysfs_smp_call() (similar to perf enable/disable),
> > remove the lock/unlock from the sysfs enable/disable path and
> > startup/dying_cpu except when to access config fields.
> >
> > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > ---
>
> <...>
>
> > @@ -618,23 +624,45 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
> > static void etm4_enable_sysfs_smp_call(void *info)
> > {
> > struct etm4_enable_arg *arg = info;
> > + struct etmv4_drvdata *drvdata;
> > struct coresight_device *csdev;
> > if (WARN_ON(!arg))
> > return;
> > - csdev = arg->drvdata->csdev;
> > + drvdata = arg->drvdata;
> > + csdev = drvdata->csdev;
> > if (!coresight_take_mode(csdev, CS_MODE_SYSFS)) {
> > /* Someone is already using the tracer */
> > arg->rc = -EBUSY;
> > return;
> > }
> > - arg->rc = etm4_enable_hw(arg->drvdata);
> > + drvdata->active_config = arg->config;
> > - /* The tracer didn't start */
> > + if (arg->cfg_hash) {
> > + arg->rc = cscfg_csdev_enable_active_config(csdev,
> > + arg->cfg_hash,
> > + arg->preset);
> > + if (arg->rc)
> > + goto err;
> > + }
> > +
> > + drvdata->trcid = arg->trace_id;
> > +
> > + /* Tracer will never be paused in sysfs mode */
> > + drvdata->paused = false;
> > +
> > + arg->rc = etm4_enable_hw(drvdata);
> > if (arg->rc)
> > - coresight_set_mode(csdev, CS_MODE_DISABLED);
>
> needs disable the active config in error path:
> cscfg_csdev_disable_active_config(drvdata->csdev);
You're right. I missed it. Thanks!
[...]
--
Sincerely,
Yeoreum Yun
^ permalink raw reply
* Re: [PATCH v5 04/12] coresight: etm4x: exclude ss_status from drvdata->config
From: Yeoreum Yun @ 2026-04-16 6:54 UTC (permalink / raw)
To: Jie Gan
Cc: coresight, linux-arm-kernel, linux-kernel, suzuki.poulose,
mike.leach, james.clark, alexander.shishkin, leo.yan
In-Reply-To: <778a826b-918d-4f7c-95a9-1cdb013618d8@oss.qualcomm.com>
Hi Jie,
>
>
> On 4/16/2026 12:55 AM, Yeoreum Yun wrote:
> > The purpose of TRCSSCSRn register is to show status of
> > the corresponding Single-shot Comparator Control and input supports.
> > That means writable field's purpose for reset or restore from idle status
> > not for configuration.
> >
> > Therefore, exclude ss_status from drvdata->config, move it to etm4x_caps
> > and rename it to ss_smp.
> >
> > This includes remove TRCSSCRn from configurable item and
> > remove saving in etm4_disable_hw().
> >
> > Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
> > ---
> > .../hwtracing/coresight/coresight-etm4x-cfg.c | 1 -
> > .../coresight/coresight-etm4x-core.c | 19 ++++++-------------
> > .../coresight/coresight-etm4x-sysfs.c | 7 ++-----
> > drivers/hwtracing/coresight/coresight-etm4x.h | 7 ++++++-
> > 4 files changed, 14 insertions(+), 20 deletions(-)
> >
> > diff --git a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> > index c302072b293a..d14d7c8a23e5 100644
> > --- a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> > +++ b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
> > @@ -86,7 +86,6 @@ static int etm4_cfg_map_reg_offset(struct etmv4_drvdata *drvdata,
> > off_mask = (offset & GENMASK(11, 5));
> > do {
> > CHECKREGIDX(TRCSSCCRn(0), ss_ctrl, idx, off_mask);
> > - CHECKREGIDX(TRCSSCSRn(0), ss_status, idx, off_mask);
> > CHECKREGIDX(TRCSSPCICRn(0), ss_pe_cmp, idx, off_mask);
> > } while (0);
> > } else if ((offset >= TRCCIDCVRn(0)) && (offset <= TRCVMIDCVRn(7))) {
> > diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> > index b2b092a76eb5..f55338a4989d 100644
> > --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> > +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> > @@ -91,7 +91,7 @@ static bool etm4x_sspcicrn_present(struct etmv4_drvdata *drvdata, int n)
> > const struct etmv4_caps *caps = &drvdata->caps;
> > return (n < caps->nr_ss_cmp) && caps->nr_pe_cmp &&
> > - (drvdata->config.ss_status[n] & TRCSSCSRn_PC);
> > + (caps->ss_cmp[n] & TRCSSCSRn_PC);
> > }
> > u64 etm4x_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
> > @@ -573,11 +573,9 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
> > etm4x_relaxed_write32(csa, config->res_ctrl[i], TRCRSCTLRn(i));
> > for (i = 0; i < caps->nr_ss_cmp; i++) {
> > - /* always clear status bit on restart if using single-shot */
> > - if (config->ss_ctrl[i] || config->ss_pe_cmp[i])
> > - config->ss_status[i] &= ~TRCSSCSRn_STATUS;
> > etm4x_relaxed_write32(csa, config->ss_ctrl[i], TRCSSCCRn(i));
> > - etm4x_relaxed_write32(csa, config->ss_status[i], TRCSSCSRn(i));
> > + /* always clear status and pending bits on restart if using single-shot */
> > + etm4x_relaxed_write32(csa, 0x0, TRCSSCSRn(i));
> > if (etm4x_sspcicrn_present(drvdata, i))
> > etm4x_relaxed_write32(csa, config->ss_pe_cmp[i], TRCSSPCICRn(i));
> > }
> > @@ -1055,12 +1053,6 @@ static void etm4_disable_hw(struct etmv4_drvdata *drvdata)
> > etm4_disable_trace_unit(drvdata);
> > - /* read the status of the single shot comparators */
> > - for (i = 0; i < caps->nr_ss_cmp; i++) {
> > - config->ss_status[i] =
> > - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
> > - }
> > -
> > /* read back the current counter values */
> > for (i = 0; i < caps->nr_cntr; i++) {
> > config->cntr_val[i] =
> > @@ -1503,8 +1495,9 @@ static void etm4_init_arch_data(void *info)
> > */
> > caps->nr_ss_cmp = FIELD_GET(TRCIDR4_NUMSSCC_MASK, etmidr4);
> > for (i = 0; i < caps->nr_ss_cmp; i++) {
> > - drvdata->config.ss_status[i] =
> > - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
> > + caps->ss_cmp[i] = etm4x_relaxed_read32(csa, TRCSSCSRn(i));
> > + caps->ss_cmp[i] &= (TRCSSCSRn_PC | TRCSSCSRn_DV |
> > + TRCSSCSRn_DA | TRCSSCSRn_INST);
>
> Just re-go through this patch and had a question here:
>
> I’m not sure whether this new change should be documented in the ABI, given
> that the TRCSSCSRn_STATUS bit is masked. In my opinion, this change breaks
> the existing ABI description.
>
> Description from the ABI document:
>
> What: /sys/bus/coresight/devices/etm<N>/sshot_status
> Date: December 2019
> KernelVersion: 5.5
> Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
> Description: (Read) Print the current value of the selected single
> shot status register.
But, as I mentioned another thread:
- https://lore.kernel.org/all/ad5yV2FoNbGGLE6R@e129823.arm.com/
Till now, sysfs doesn't show the *current value* of the single shot
state since the config->ss_status is updated enabled/disabled sysfs
session. an I think once the session is disabled, other status bits
(currently STATUS and PENDING bits) don't have any meaning.
I think it's enough to change the doc's Description for this.
Any thought?
--
Sincerely,
Yeoreum Yun
^ permalink raw reply
* [PATCH bpf v15 4/5] bpf, x86: Emit ENDBR for indirect jump targets
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support CET/IBT, the indirect jump selftest triggers
a kernel panic because the indirect jump targets lack ENDBR
instructions.
To fix it, emit an ENDBR instruction to each indirect jump target. Since
the ENDBR instruction shifts the position of original jited instructions,
fix the instruction address calculation wherever the addresses are used.
For reference, below is a sample panic log.
Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cet.c:133!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
...
? 0xffffffffc00fb258
? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
bpf_prog_test_run_syscall+0x110/0x2f0
? fdget+0xba/0xe0
__sys_bpf+0xe4b/0x2590
? __kmalloc_node_track_caller_noprof+0x1c7/0x680
? bpf_prog_test_run_syscall+0x215/0x2f0
__x64_sys_bpf+0x21/0x30
do_syscall_64+0x85/0x620
? bpf_prog_test_run_syscall+0x1e2/0x2f0
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72d9a5faa230..ea9e707e8abf 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
#define EMIT_ENDBR() EMIT(gen_endbr(), 4)
#define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4)
#else
-#define EMIT_ENDBR()
-#define EMIT_ENDBR_POISON()
+#define EMIT_ENDBR() do { } while (0)
+#define EMIT_ENDBR_POISON() do { } while (0)
#endif
static bool is_imm8(int value)
@@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
-static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image,
- int oldproglen, struct jit_context *ctx, bool jmp_padding)
+static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
+ u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
bool tail_call_reachable = bpf_prog->aux->tail_call_reachable;
struct bpf_insn *insn = bpf_prog->insnsi;
@@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
void __percpu *priv_stack_ptr;
int i, excnt = 0;
int ilen, proglen = 0;
- u8 *prog = temp;
+ u8 *ip, *prog = temp;
u32 stack_depth;
int err;
@@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
dst_reg = X86_REG_R9;
}
+ if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1))
+ EMIT_ENDBR();
+
+ ip = image + addrs[i - 1] + (prog - temp);
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off))
/* call */
case BPF_JMP | BPF_CALL: {
- u8 *ip = image + addrs[i - 1];
-
func = (u8 *) __bpf_call_base + imm32;
if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off))
if (imm32)
emit_bpf_tail_call_direct(bpf_prog,
&bpf_prog->aux->poke_tab[imm32 - 1],
- &prog, image + addrs[i - 1],
+ &prog,
+ ip,
callee_regs_used,
stack_depth,
ctx);
@@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off))
&prog,
callee_regs_used,
stack_depth,
- image + addrs[i - 1],
+ ip,
ctx);
break;
@@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off))
break;
case BPF_JMP | BPF_JA | BPF_X:
- emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+ emit_indirect_jump(&prog, insn->dst_reg, ip);
break;
case BPF_JMP | BPF_JA:
case BPF_JMP32 | BPF_JA:
@@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off))
ctx->cleanup_addr = proglen;
if (bpf_prog_was_classic(bpf_prog) &&
!ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
- u8 *ip = image + addrs[i - 1];
-
if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
return -EINVAL;
}
@@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
for (pass = 0; pass < MAX_PASSES || image; pass++) {
if (!padding && pass >= PADDING_PASSES)
padding = true;
- proglen = do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, padding);
+ proglen = do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, padding);
if (proglen <= 0) {
out_image:
image = NULL;
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 3/5] bpf: Add helper to detect indirect jump targets
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Introduce helper bpf_insn_is_indirect_target to check whether a BPF
instruction is an indirect jump target.
Since the verifier knows which instructions are indirect jump targets,
add a new flag indirect_target to struct bpf_insn_aux_data to mark
them. The verifier sets this flag when verifying an indirect jump target
instruction, and the helper checks the flag to determine whether an
instruction is an indirect jump target.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> #v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> #v12
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
include/linux/bpf.h | 2 ++
include/linux/bpf_verifier.h | 9 +++++----
kernel/bpf/core.c | 9 +++++++++
kernel/bpf/fixups.c | 12 ++++++++++++
kernel/bpf/verifier.c | 7 +++++++
5 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 0136a108d083..b4b703c90ca9 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1541,6 +1541,8 @@ bool bpf_has_frame_pointer(unsigned long ip);
int bpf_jit_charge_modmem(u32 size);
void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 53e8664cb566..b148f816f25b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -630,16 +630,17 @@ struct bpf_insn_aux_data {
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
- bool jmp_point;
- bool prune_point;
+ u32 jmp_point:1;
+ u32 prune_point:1;
/* ensure we check state equivalence and save state checkpoint and
* this instruction, regardless of any heuristics
*/
- bool force_checkpoint;
+ u32 force_checkpoint:1;
/* true if instruction is a call to a helper function that
* accepts callback function as a parameter.
*/
- bool calls_callback;
+ u32 calls_callback:1;
+ u32 indirect_target:1; /* if it is an indirect jump target */
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 79361aa11757..8b018ff48875 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1573,6 +1573,15 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
clone->blinded = 1;
return clone;
}
+
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx)
+{
+ if (!env)
+ return false;
+ insn_idx += prog->aux->subprog_start;
+ return env->insn_aux_data[insn_idx].indirect_target;
+}
#endif /* CONFIG_BPF_JIT */
/* Base function for offset calculation. Needs to go into .text section,
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 6c86980cc9e8..fba9e8c00878 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -183,6 +183,18 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
data[i].seen = old_seen;
data[i].zext_dst = insn_has_def32(insn + i);
}
+
+ /*
+ * The indirect_target flag of the original instruction was moved to the last of the
+ * new instructions by the above memmove and memset, but the indirect jump target is
+ * actually the first instruction, so move it back. This also matches with the behavior
+ * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
+ * instruction.
+ */
+ if (data[off + cnt - 1].indirect_target) {
+ data[off].indirect_target = 1;
+ data[off + cnt - 1].indirect_target = 0;
+ }
}
static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e804e0da3500..1e36b9e91277 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3497,6 +3497,11 @@ static int insn_stack_access_flags(int frameno, int spi)
return INSN_F_STACK_ACCESS | (spi << INSN_F_SPI_SHIFT) | frameno;
}
+static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].indirect_target = true;
+}
+
#define LR_FRAMENO_BITS 3
#define LR_SPI_BITS 6
#define LR_ENTRY_BITS (LR_SPI_BITS + LR_FRAMENO_BITS + 1)
@@ -17545,12 +17550,14 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
}
for (i = 0; i < n - 1; i++) {
+ mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
env->insn_idx, env->cur_state->speculative);
if (IS_ERR(other_branch))
return PTR_ERR(other_branch);
}
env->insn_idx = env->gotox_tmp_buf->items[n-1];
+ mark_indirect_target(env, env->insn_idx);
return INSN_IDX_UPDATED;
}
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 0/5] emit ENDBR/BTI instructions for indirect
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.
The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, allowing
the JIT to consult env->insn_aux_data to determine which instructions are
indirect jump targets.
During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not use env->insn_aux_data directly.
To avoid this mismatch, and given that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.
v15:
- Rebase and target bpf tree
- Resotre subprog_start of the fake 'exit' subprog on failure
- Fix wrong function name used in comment
v14: https://lore.kernel.org/all/cover.1776062885.git.xukuohai@hotmail.com/
- Rebase
- Fix comment style
- Fix incorrect variable and function name used in commit message
v13: https://lore.kernel.org/bpf/20260411133847.1042658-1-xukuohai@huaweicloud.com
- Use vmalloc to allocate memory for insn_aux_data copies to match with vfree
- Do not free the copied memory of insn_aux_data when restoring from failure
- Code cleanup
v12: https://lore.kernel.org/bpf/20260403132811.753894-1-xukuohai@huaweicloud.com
- Restore env->insn_aux_data on JIT failure
- Fix incorrect error code sign (-EFAULT vs EFAULT)
- Fix incorrect prog used in the restore path
v11: https://lore.kernel.org/bpf/20260403090915.473493-1-xukuohai@huaweicloud.com
- Restore env->subprog_info after jit_subprogs() fails
- Clear prog->jit_requested and prog->blinding_requested on failure
- Use the actual env->insn_aux_data size in clear_insn_aux_data() on failure
v10: https://lore.kernel.org/bpf/20260324122052.342751-1-xukuohai@huaweicloud.com
- Fix the incorrect call_imm restore in jit_subprogs
- Define a dummy void version of bpf_jit_prog_release_other and
bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)
v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)
v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
to adjust_insn_aux_data()
v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch
v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier
v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
each prog
v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
jump targets, and emit ENDBR on x86
v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)
v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)
v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com
Xu Kuohai (5):
bpf: Move constants blinding out of arch-specific JITs
bpf: Pass bpf_verifier_env to JIT
bpf: Add helper to detect indirect jump targets
bpf, x86: Emit ENDBR for indirect jump targets
bpf, arm64: Emit BTI for indirect jump target
arch/arc/net/bpf_jit_core.c | 41 +++-----
arch/arm/net/bpf_jit_32.c | 43 ++------
arch/arm64/net/bpf_jit_comp.c | 87 ++++++-----------
arch/loongarch/net/bpf_jit.c | 61 ++++--------
arch/mips/net/bpf_jit_comp.c | 22 +----
arch/parisc/net/bpf_jit_core.c | 75 ++++++--------
arch/powerpc/net/bpf_jit_comp.c | 74 ++++++--------
arch/riscv/net/bpf_jit_core.c | 63 +++++-------
arch/s390/net/bpf_jit_comp.c | 61 ++++--------
arch/sparc/net/bpf_jit_comp_64.c | 63 +++++-------
arch/x86/net/bpf_jit_comp.c | 73 +++++---------
arch/x86/net/bpf_jit_comp32.c | 35 +------
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 9 +-
include/linux/filter.h | 50 +++++++++-
kernel/bpf/core.c | 138 ++++++++++++++++++--------
kernel/bpf/fixups.c | 162 ++++++++++++++++++++++++++-----
kernel/bpf/syscall.c | 4 -
kernel/bpf/verifier.c | 21 ++--
19 files changed, 529 insertions(+), 555 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH bpf v15 2/5] bpf: Pass bpf_verifier_env to JIT
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will
use env->insn_aux_data in the JIT stage to detect indirect jump targets.
Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c
code without verifier, introduce helper __bpf_prog_select_runtime()
to accept the env parameter.
Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and
switch to call __bpf_prog_select_runtime() in the verifier, with env
variable passed. The original bpf_prog_select_runtime() is preserved for
cbpf and lib/test_bpf.c, where env is NULL.
Now all constants blinding calls are moved into the verifier, except
the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted
by bpf_patch_insn_data() function for normal cases, so there is no need
to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Hengqi Chen <hengqi.chen@gmail.com> # v14
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 2 +-
arch/arm/net/bpf_jit_32.c | 2 +-
arch/arm64/net/bpf_jit_comp.c | 2 +-
arch/loongarch/net/bpf_jit.c | 2 +-
arch/mips/net/bpf_jit_comp.c | 2 +-
arch/parisc/net/bpf_jit_core.c | 2 +-
arch/powerpc/net/bpf_jit_comp.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 2 +-
arch/s390/net/bpf_jit_comp.c | 2 +-
arch/sparc/net/bpf_jit_comp_64.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
include/linux/filter.h | 17 ++++++-
kernel/bpf/core.c | 86 ++++++++++++++++----------------
kernel/bpf/fixups.c | 10 ++--
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 14 +++---
17 files changed, 84 insertions(+), 71 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 973ceae48675..639a2736f029 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -1400,7 +1400,7 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog)
* (re)locations involved that their addresses are not known
* during the first run.
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
vm_dump(prog);
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index e6b1bb2de627..1628b6fc70a4 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2142,7 +2142,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header;
struct jit_ctx ctx;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index d310d1c35192..bd8757952507 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2000,7 +2000,7 @@ struct arm64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
struct bpf_binary_header *header;
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index fcc8c0c29fb0..5149ce4cef7e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1920,7 +1920,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index d2b6c955f18e..6ee4abe6a1f7 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -909,7 +909,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index 35dca372b5df..172770132440 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 2bae4699e78f..53ab97ad6074 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -162,7 +162,7 @@ static void priv_stack_check_guard(void __percpu *priv_stack_ptr, int alloc_size
}
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
u32 proglen;
u32 alloclen;
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 36f0aea8096d..4365d07aaf54 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 2dfc279b1be2..94128fe6be23 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2312,7 +2312,7 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
/*
* Compile eBPF program "fp"
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index e83e29137566..2fa0e9375127 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1477,7 +1477,7 @@ struct sparc64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 77d00a8dec87..72d9a5faa230 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3713,7 +3713,7 @@ struct x64_jit_data {
#define MAX_PASSES 20
#define PADDING_PASSES (MAX_PASSES - 5)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5f259577614a..852baf2e4db4 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2518,7 +2518,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
int proglen, oldproglen = 0;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 9fa4d4090093..1ec6d5ba64cc 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1108,6 +1108,8 @@ sk_filter_reason(struct sock *sk, struct sk_buff *skb)
return sk_filter_trim_cap(sk, skb, 1);
}
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err);
struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err);
void bpf_prog_free(struct bpf_prog *fp);
@@ -1153,7 +1155,7 @@ u64 __bpf_call_base(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
((u64 (*)(u64, u64, u64, u64, u64, const struct bpf_insn *)) \
(void *)__bpf_call_base)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog);
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_compile(struct bpf_prog *prog);
bool bpf_jit_needs_zext(void);
bool bpf_jit_inlines_helper_call(s32 imm);
@@ -1188,12 +1190,25 @@ struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
#ifdef CONFIG_BPF_SYSCALL
struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len);
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env);
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux);
#else
static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len)
{
return ERR_PTR(-ENOTSUPP);
}
+
+static inline struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ return NULL;
+}
+
+static inline void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+}
#endif /* CONFIG_BPF_SYSCALL */
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc9fb3c07866..79361aa11757 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1491,23 +1491,6 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
bpf_prog_clone_free(fp_other);
}
-static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
-{
-#ifdef CONFIG_BPF_SYSCALL
- struct bpf_map *map;
- int i;
-
- if (len <= 1)
- return;
-
- for (i = 0; i < prog->aux->used_map_cnt; i++) {
- map = prog->aux->used_maps[i];
- if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
- bpf_insn_array_adjust(map, off, len);
- }
-#endif
-}
-
/*
* Now this function is used only to blind the main prog and must be invoked only when
* bpf_prog_need_blind() returns true.
@@ -1580,13 +1563,6 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
if (env)
env->prog = clone;
- else
- /*
- * Instructions arrays must be updated using absolute xlated offsets.
- * The arrays have already been adjusted by bpf_patch_insn_data() when
- * env is not NULL.
- */
- adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2555,47 +2531,55 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
-static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
#ifdef CONFIG_BPF_JIT
struct bpf_prog *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
if (!bpf_prog_need_blind(prog))
- return bpf_int_jit_compile(prog);
+ return bpf_int_jit_compile(env, prog);
+
+ if (env) {
+ /*
+ * If env is not NULL, we are called from the end of bpf_check(), at this
+ * point, only insn_aux_data is used after failure, so it should be restored
+ * on failure.
+ */
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux)
+ return prog;
+ }
orig_prog = prog;
- prog = bpf_jit_blind_constants(NULL, prog);
+ prog = bpf_jit_blind_constants(env, prog);
/*
* If blinding was requested and we failed during blinding, we must fall
* back to the interpreter.
*/
if (IS_ERR(prog))
- return orig_prog;
+ goto out_restore;
- prog = bpf_int_jit_compile(prog);
+ prog = bpf_int_jit_compile(env, prog);
if (prog->jited) {
bpf_jit_prog_release_other(prog, orig_prog);
+ if (env)
+ vfree(orig_insn_aux);
return prog;
}
bpf_jit_prog_release_other(orig_prog, prog);
+
+out_restore:
prog = orig_prog;
+ if (env)
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
#endif
return prog;
}
-/**
- * bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with BPF program
- * @err: pointer to error variable
- *
- * Try to JIT eBPF program, if JIT is not available, use interpreter.
- * The BPF program will be executed via bpf_prog_run() function.
- *
- * Return: the &fp argument along with &err set to 0 for success or
- * a negative errno code on failure
- */
-struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err)
{
/* In case of BPF to BPF calls, verifier did all the prep
* work with regards to JITing, etc.
@@ -2623,7 +2607,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_prog_jit_compile(fp);
+ fp = bpf_prog_jit_compile(env, fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
@@ -2649,6 +2633,22 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
return fp;
}
+
+/**
+ * bpf_prog_select_runtime - select exec runtime for BPF program
+ * @fp: bpf_prog populated with BPF program
+ * @err: pointer to error variable
+ *
+ * Try to JIT eBPF program, if JIT is not available, use interpreter.
+ * The BPF program will be executed via bpf_prog_run() function.
+ *
+ * Return: the &fp argument along with &err set to 0 for success or
+ * a negative errno code on failure
+ */
+struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+{
+ return __bpf_prog_select_runtime(NULL, fp, err);
+}
EXPORT_SYMBOL_GPL(bpf_prog_select_runtime);
static unsigned int __bpf_prog_ret1(const void *ctx,
@@ -3136,7 +3136,7 @@ const struct bpf_func_proto bpf_tail_call_proto = {
* It is encouraged to implement bpf_int_jit_compile() instead, so that
* eBPF and implicitly also cBPF can get JITed!
*/
-struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
return prog;
}
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 721b830b5ef2..6c86980cc9e8 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -993,7 +993,7 @@ static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_s
env->subprog_info[env->subprog_cnt].start = env->prog->len;
}
-static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
{
size_t size;
void *new_aux;
@@ -1005,8 +1005,8 @@ static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *
return new_aux;
}
-static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
- struct bpf_insn_aux_data *orig_insn_aux)
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
{
/* the expanded elements are zero-filled, so no special handling is required */
vfree(env->insn_aux_data);
@@ -1150,7 +1150,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->token = prog->aux->token;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
- func[i] = bpf_int_jit_compile(func[i]);
+ func[i] = bpf_int_jit_compile(env, func[i]);
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1194,7 +1194,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
for (i = 0; i < env->subprog_cnt; i++) {
old_bpf_func = func[i]->bpf_func;
- tmp = bpf_int_jit_compile(func[i]);
+ tmp = bpf_int_jit_compile(env, func[i]);
if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) {
verbose(env, "JIT doesn't support bpf-to-bpf calls\n");
err = -ENOTSUPP;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index b73b25c63073..a3c0214ca934 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3083,10 +3083,6 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
if (err < 0)
goto free_used_maps;
- prog = bpf_prog_select_runtime(prog, &err);
- if (err < 0)
- goto free_used_maps;
-
err = bpf_prog_mark_insn_arrays_ready(prog);
if (err < 0)
goto free_used_maps;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e4980128151..e804e0da3500 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -20155,6 +20155,14 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
adjust_btf_func(env);
+ /* extension progs temporarily inherit the attach_type of their targets
+ for verification purposes, so set it back to zero before returning
+ */
+ if (env->prog->type == BPF_PROG_TYPE_EXT)
+ env->prog->expected_attach_type = 0;
+
+ env->prog = __bpf_prog_select_runtime(env, env->prog, &ret);
+
err_release_maps:
if (ret)
release_insn_arrays(env);
@@ -20166,12 +20174,6 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
if (!env->prog->aux->used_btfs)
release_btfs(env);
- /* extension progs temporarily inherit the attach_type of their targets
- for verification purposes, so set it back to zero before returning
- */
- if (env->prog->type == BPF_PROG_TYPE_EXT)
- env->prog->expected_attach_type = 0;
-
*prog = env->prog;
module_put(env->attach_btf_mod);
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 5/5] bpf, arm64: Emit BTI for indirect jump target
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support BTI, the indirect jump selftest triggers a kernel
panic because there is no BTI instructions at the indirect jump targets.
Fix it by emitting a BTI instruction for each indirect jump target.
For reference, below is a sample panic log.
Internal error: Oops - BTI: 0000000036000003 [#1] SMP
...
Call trace:
bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x54/0xf8 (P)
bpf_prog_run_pin_on_cpu+0x140/0x468
bpf_prog_test_run_syscall+0x280/0x3b8
bpf_prog_test_run+0x22c/0x2c0
Fixes: f4a66cf1cb14 ("bpf: arm64: Add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index bd8757952507..0816c40fc7af 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1197,8 +1197,8 @@ static int add_exception_handler(const struct bpf_insn *insn,
* >0 - successfully JITed a 16-byte eBPF instruction.
* <0 - failed to JIT.
*/
-static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
- bool extra_pass)
+static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn *insn,
+ struct jit_ctx *ctx, bool extra_pass)
{
const u8 code = insn->code;
u8 dst = bpf2a64[insn->dst_reg];
@@ -1223,6 +1223,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
int ret;
bool sign_extend;
+ if (bpf_insn_is_indirect_target(env, ctx->prog, i))
+ emit_bti(A64_BTI_J, ctx);
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
@@ -1898,7 +1901,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
-static int build_body(struct jit_ctx *ctx, bool extra_pass)
+static int build_body(struct bpf_verifier_env *env, struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
int i;
@@ -1917,7 +1920,7 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
int ret;
ctx->offset[i] = ctx->idx;
- ret = build_insn(insn, ctx, extra_pass);
+ ret = build_insn(env, insn, ctx, extra_pass);
if (ret > 0) {
i++;
ctx->offset[i] = ctx->idx;
@@ -2073,7 +2076,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
if (build_prologue(&ctx, was_classic))
goto out_off;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_off;
ctx.epilogue_offset = ctx.idx;
@@ -2121,7 +2124,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_free_hdr;
ctx.epilogue_offset = ctx.idx;
@@ -2130,7 +2133,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
ctx.write = true;
/* Pass 3: Adjust jump offset and write final image */
- if (build_body(&ctx, extra_pass) ||
+ if (build_body(env, &ctx, extra_pass) ||
WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 1/5] bpf: Move constants blinding out of arch-specific JITs
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
During the JIT stage, constants blinding rewrites instructions but only
rewrites the private instruction copy of the JITed subprog, leaving the
global env->prog->insnsi and env->insn_aux_data untouched. This causes a
mismatch between subprog instructions and the global state, making it
difficult to use the global data in the JIT.
To avoid this mismatch, and given that all arch-specific JITs already
support constants blinding, move it to the generic verifier code, and
switch to rewrite the global env->prog->insnsi with the global states
adjusted, as other rewrites in the verifier do.
This removes the constants blinding calls in each JIT, which are largely
duplicated code across architectures.
Since constants blinding is only required for JIT, and there are two
JIT entry functions, jit_subprogs() for BPF programs with multiple
subprogs and bpf_prog_select_runtime() for programs with no subprogs,
move the constants blinding invocation into these two functions.
In the verifier path, bpf_patch_insn_data() is used to keep global
verifier auxiliary data in sync with patched instructions. A key
question is whether this global auxiliary data should be restored
on the failure path.
Besides instructions, bpf_patch_insn_data() adjusts:
- prog->aux->poke_tab
- env->insn_array_maps
- env->subprog_info
- env->insn_aux_data
For prog->aux->poke_tab, it is only used by JIT or only meaningful after
JIT succeeds, so it does not need to be restored on the failure path.
For env->insn_array_maps, when JIT fails, programs using insn arrays
are rejected by bpf_insn_array_ready() due to missing JIT addresses.
Hence, env->insn_array_maps is only meaningful for JIT and does not need
to be restored.
For subprog_info, if jit_subprogs fails and CONFIG_BPF_JIT_ALWAYS_ON
is not enabled, kernel falls back to interpreter. In this case,
env->subprog_info is used to determine subprogram stack depth. So it
must be restored on failure.
For env->insn_aux_data, it is freed by clear_insn_aux_data() at the
end of bpf_check(). Before freeing, clear_insn_aux_data() loops over
env->insn_aux_data to release jump targets recorded in it. The loop
uses env->prog->len as the array length, but this length no longer
matches the actual size of the adjusted env->insn_aux_data array after
constants blinding.
To address it, a simple approach is to keep insn_aux_data as adjusted
after failure, since it will be freed shortly, and record its actual size
for the loop in clear_insn_aux_data(). But since clear_insn_aux_data()
uses the same index to loop over both env->prog->insnsi and env->insn_aux_data,
this approach results in incorrect index for the insnsi array. So an
alternative approach is adopted: clone the original env->insn_aux_data
before blinding and restore it after failure, similar to env->prog.
For classic BPF programs, constants blinding works as before since it
is still invoked from bpf_prog_select_runtime().
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Hari Bathini <hbathini@linux.ibm.com> # powerpc jit
Reviewed-by: Pu Lehui <pulehui@huawei.com> # riscv jit
Acked-by: Hengqi Chen <hengqi.chen@gmail.com> # loongarch jit
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 39 +++------
arch/arm/net/bpf_jit_32.c | 41 ++-------
arch/arm64/net/bpf_jit_comp.c | 72 +++++----------
arch/loongarch/net/bpf_jit.c | 59 ++++---------
arch/mips/net/bpf_jit_comp.c | 20 +----
arch/parisc/net/bpf_jit_core.c | 73 ++++++----------
arch/powerpc/net/bpf_jit_comp.c | 72 ++++++---------
arch/riscv/net/bpf_jit_core.c | 61 +++++--------
arch/s390/net/bpf_jit_comp.c | 59 +++++--------
arch/sparc/net/bpf_jit_comp_64.c | 61 +++++--------
arch/x86/net/bpf_jit_comp.c | 43 ++-------
arch/x86/net/bpf_jit_comp32.c | 33 +------
include/linux/filter.h | 33 ++++++-
kernel/bpf/core.c | 69 +++++++++++++--
kernel/bpf/fixups.c | 146 ++++++++++++++++++++++++++-----
15 files changed, 403 insertions(+), 478 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 1421eeced0f5..973ceae48675 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -79,7 +79,6 @@ struct arc_jit_data {
* The JIT pertinent context that is used by different functions.
*
* prog: The current eBPF program being handled.
- * orig_prog: The original eBPF program before any possible change.
* jit: The JIT buffer and its length.
* bpf_header: The JITed program header. "jit.buf" points inside it.
* emit: If set, opcodes are written to memory; else, a dry-run.
@@ -94,12 +93,10 @@ struct arc_jit_data {
* need_extra_pass: A forecast if an "extra_pass" will occur.
* is_extra_pass: Indicates if the current pass is an extra pass.
* user_bpf_prog: True, if VM opcodes come from a real program.
- * blinded: True if "constant blinding" step returned a new "prog".
* success: Indicates if the whole JIT went OK.
*/
struct jit_context {
struct bpf_prog *prog;
- struct bpf_prog *orig_prog;
struct jit_buffer jit;
struct bpf_binary_header *bpf_header;
bool emit;
@@ -114,7 +111,6 @@ struct jit_context {
bool need_extra_pass;
bool is_extra_pass;
bool user_bpf_prog;
- bool blinded;
bool success;
};
@@ -161,13 +157,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog)
{
memset(ctx, 0, sizeof(*ctx));
- ctx->orig_prog = prog;
-
- /* If constant blinding was requested but failed, scram. */
- ctx->prog = bpf_jit_blind_constants(prog);
- if (IS_ERR(ctx->prog))
- return PTR_ERR(ctx->prog);
- ctx->blinded = (ctx->prog != ctx->orig_prog);
+ ctx->prog = prog;
/* If the verifier doesn't zero-extend, then we have to do it. */
ctx->do_zext = !ctx->prog->aux->verifier_zext;
@@ -214,14 +204,6 @@ static inline void maybe_free(struct jit_context *ctx, void **mem)
*/
static void jit_ctx_cleanup(struct jit_context *ctx)
{
- if (ctx->blinded) {
- /* if all went well, release the orig_prog. */
- if (ctx->success)
- bpf_jit_prog_release_other(ctx->prog, ctx->orig_prog);
- else
- bpf_jit_prog_release_other(ctx->orig_prog, ctx->prog);
- }
-
maybe_free(ctx, (void **)&ctx->bpf2insn);
maybe_free(ctx, (void **)&ctx->jit_data);
@@ -229,12 +211,19 @@ static void jit_ctx_cleanup(struct jit_context *ctx)
ctx->bpf2insn_valid = false;
/* Freeing "bpf_header" is enough. "jit.buf" is a sub-array of it. */
- if (!ctx->success && ctx->bpf_header) {
- bpf_jit_binary_free(ctx->bpf_header);
- ctx->bpf_header = NULL;
- ctx->jit.buf = NULL;
- ctx->jit.index = 0;
- ctx->jit.len = 0;
+ if (!ctx->success) {
+ if (ctx->bpf_header) {
+ bpf_jit_binary_free(ctx->bpf_header);
+ ctx->bpf_header = NULL;
+ ctx->jit.buf = NULL;
+ ctx->jit.index = 0;
+ ctx->jit.len = 0;
+ }
+ if (ctx->is_extra_pass) {
+ ctx->prog->bpf_func = NULL;
+ ctx->prog->jited = 0;
+ ctx->prog->jited_len = 0;
+ }
}
ctx->emit = false;
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..e6b1bb2de627 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2144,9 +2144,7 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
- bool tmp_blinded = false;
struct jit_ctx ctx;
unsigned int tmp_idx;
unsigned int image_size;
@@ -2156,20 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- /* If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
-
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
@@ -2179,10 +2164,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* we must fall back to the interpreter
*/
ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
- if (ctx.offsets == NULL) {
- prog = orig_prog;
- goto out;
- }
+ if (ctx.offsets == NULL)
+ return prog;
/* 1) fake pass to find in the length of the JITed code,
* to compute ctx->offsets and other context variables
@@ -2194,10 +2177,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* being successful in the second pass, so just fall back
* to the interpreter.
*/
- if (build_body(&ctx)) {
- prog = orig_prog;
+ if (build_body(&ctx))
goto out_off;
- }
tmp_idx = ctx.idx;
build_prologue(&ctx);
@@ -2213,10 +2194,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx += ctx.imm_count;
if (ctx.imm_count) {
ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
- if (ctx.imms == NULL) {
- prog = orig_prog;
+ if (ctx.imms == NULL)
goto out_off;
- }
}
#else
/* there's nothing about the epilogue on ARMv7 */
@@ -2238,10 +2217,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Not able to allocate memory for the structure then
* we must fall back to the interpretation
*/
- if (header == NULL) {
- prog = orig_prog;
+ if (header == NULL)
goto out_imms;
- }
/* 2.) Actual pass to generate final JIT code */
ctx.target = (u32 *) image_ptr;
@@ -2278,16 +2255,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
#endif
out_off:
kfree(ctx.offsets);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free:
image_ptr = NULL;
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_imms;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 524b67c0867e..d310d1c35192 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2003,14 +2003,12 @@ struct arm64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header = NULL;
struct arm64_jit_data *jit_data;
void __percpu *priv_stack_ptr = NULL;
bool was_classic = bpf_prog_was_classic(prog);
int priv_stack_alloc_sz;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
@@ -2019,26 +2017,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int exentry_idx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -2050,10 +2035,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -2073,10 +2056,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kvzalloc_objs(int, prog->len + 1);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_off;
- }
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
@@ -2089,15 +2070,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* BPF line info needs ctx->offset[i] to be the offset of
* instruction[i] in jited image, so build prologue first.
*/
- if (build_prologue(&ctx, was_classic)) {
- prog = orig_prog;
+ if (build_prologue(&ctx, was_classic))
goto out_off;
- }
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_off;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx, was_classic);
@@ -2115,10 +2092,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_off;
- }
/* Pass 2: Determine jited position and result for each instruction */
@@ -2146,10 +2121,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free_hdr;
- }
ctx.epilogue_offset = ctx.idx;
ctx.exentry_idx = exentry_idx;
@@ -2158,19 +2131,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Pass 3: Adjust jump offset and write final image */
if (build_body(&ctx, extra_pass) ||
- WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset)) {
- prog = orig_prog;
+ WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
- }
build_epilogue(&ctx, was_classic);
build_plt(&ctx);
/* Extra pass to validate JITed code. */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free_hdr;
- }
/* update the real prog size */
prog_size = sizeof(u32) * ctx.idx;
@@ -2187,16 +2156,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (extra_pass && ctx.idx > jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d > %d\n",
ctx.idx, jit_data->ctx.idx);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
goto out_free_hdr;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header has been freed */
ro_header = NULL;
- prog = orig_prog;
- goto out_off;
+ header = NULL;
+ goto out_free_hdr;
}
} else {
jit_data->ctx = ctx;
@@ -2233,13 +2199,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size,
sizeof(header->size));
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 9cb796e16379..fcc8c0c29fb0 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1922,43 +1922,26 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- bool tmp_blinded = false, extra_pass = false;
+ bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header;
- struct bpf_prog *tmp, *orig_prog = prog;
/*
* If BPF JIT was not enabled then we must fall back to
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- if (IS_ERR(tmp))
- return orig_prog;
-
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1978,17 +1961,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.offset = kvcalloc(prog->len + 1, sizeof(u32), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_offset;
- }
/* 1. Initial fake pass to compute ctx->idx and set ctx->flags */
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_offset;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
@@ -2004,10 +1983,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
&header, &image_ptr, jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_offset;
- }
/* 2. Now, the actual pass to generate final JIT code */
/*
@@ -2027,17 +2004,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.num_exentries = 0;
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free;
- }
build_epilogue(&ctx);
/* 3. Extra pass to validate JITed code */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free;
- }
/* And we're done */
if (bpf_jit_enable > 1)
@@ -2050,9 +2023,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_free;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header have been freed */
ro_header = NULL;
- prog = orig_prog;
+ header = NULL;
goto out_free;
}
/*
@@ -2084,13 +2057,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ? tmp : orig_prog);
-
return prog;
out_free:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size, sizeof(header->size));
bpf_jit_binary_pack_free(ro_header, header);
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index e355dfca4400..d2b6c955f18e 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -911,10 +911,8 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
- bool tmp_blinded = false;
unsigned int tmp_idx;
unsigned int image_size;
u8 *image_ptr;
@@ -925,19 +923,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
- /*
- * If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.program = prog;
@@ -1025,14 +1011,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited_len = image_size;
out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
kfree(ctx.descriptors);
return prog;
out_err:
- prog = orig_prog;
if (header)
bpf_jit_binary_free(header);
goto out;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index a5eb6b51e27a..35dca372b5df 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -44,30 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, prologue_len, i;
struct hppa_jit_data *jit_data;
struct hppa_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
@@ -81,10 +70,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!ctx->offset)
+ goto out_err;
for (i = 0; i < prog->len; i++) {
prev_ninsns += 20;
ctx->offset[i] = prev_ninsns;
@@ -93,10 +80,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
for (i = 0; i < NR_JIT_ITERATIONS; i++) {
pass++;
ctx->ninsns = 0;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (build_body(ctx, extra_pass, ctx->offset))
+ goto out_err;
ctx->body_len = ctx->ninsns;
bpf_jit_build_prologue(ctx);
ctx->prologue_len = ctx->ninsns - ctx->body_len;
@@ -116,10 +101,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->image,
sizeof(long),
bpf_fill_ill_insns);
- if (!jit_data->header) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!jit_data->header)
+ goto out_err;
ctx->insns = (u32 *)jit_data->image;
/*
@@ -134,8 +117,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
if (jit_data->header)
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
if (extable_size)
@@ -148,8 +130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx);
if (build_body(ctx, extra_pass, NULL)) {
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
bpf_jit_build_epilogue(ctx);
@@ -160,20 +141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{ extern int machine_restart(char *); machine_restart(""); }
}
+ if (!prog->is_func || extra_pass) {
+ if (bpf_jit_binary_lock_ro(jit_data->header)) {
+ bpf_jit_binary_free(jit_data->header);
+ goto out_err;
+ }
+ bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
+ }
+
prog->bpf_func = (void *)ctx->insns;
prog->jited = 1;
prog->jited_len = prog_size;
- bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
-
if (!prog->is_func || extra_pass) {
- if (bpf_jit_binary_lock_ro(jit_data->header)) {
- bpf_jit_binary_free(jit_data->header);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
- goto out_offset;
- }
prologue_len = ctx->epilogue_offset - ctx->body_len;
for (i = 0; i < prog->len; i++)
ctx->offset[i] += prologue_len;
@@ -183,14 +163,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
+
if (HPPA_JIT_REBOOT)
{ extern int machine_restart(char *); machine_restart(""); }
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_offset;
}
u64 hppa_div64(u64 div, u64 divisor)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 50103b3794fb..2bae4699e78f 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -177,9 +177,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
void __percpu *priv_stack_ptr = NULL;
struct bpf_binary_header *fhdr = NULL;
struct bpf_binary_header *hdr = NULL;
- struct bpf_prog *org_fp = fp;
- struct bpf_prog *tmp_fp = NULL;
- bool bpf_blinded = false;
bool extra_pass = false;
u8 *fimage = NULL;
u32 *fcode_base = NULL;
@@ -187,24 +184,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u32 fixup_len;
if (!fp->jit_requested)
- return org_fp;
-
- tmp_fp = bpf_jit_blind_constants(org_fp);
- if (IS_ERR(tmp_fp))
- return org_fp;
-
- if (tmp_fp != org_fp) {
- bpf_blinded = true;
- fp = tmp_fp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = org_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
@@ -219,10 +205,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
priv_stack_alloc_size = round_up(fp->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_size, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- fp = org_fp;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_size);
fp->aux->priv_stack_ptr = priv_stack_ptr;
@@ -249,10 +233,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
addrs = kcalloc(flen + 1, sizeof(*addrs), GFP_KERNEL);
- if (addrs == NULL) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (addrs == NULL)
+ goto out_err;
memset(&cgctx, 0, sizeof(struct codegen_context));
bpf_jit_init_reg_mapping(&cgctx);
@@ -279,11 +261,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
/* Scouting faux-generate pass 0 */
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
/* We hit something illegal or unsupported. */
- fp = org_fp;
- goto out_addrs;
- }
+ goto out_err;
/*
* If we have seen a tail call, we need a second pass.
@@ -294,10 +274,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
+ goto out_err;
}
bpf_jit_realloc_regs(&cgctx);
@@ -318,10 +296,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
bpf_jit_fill_ill_insns);
- if (!fhdr) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (!fhdr)
+ goto out_err;
if (extable_len)
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
@@ -340,8 +316,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
extra_pass)) {
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
bpf_jit_binary_pack_free(fhdr, hdr);
- fp = org_fp;
- goto out_addrs;
+ goto out_err;
}
bpf_jit_build_epilogue(code_base, &cgctx);
@@ -363,15 +338,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
((u64 *)image)[1] = local_paca->kernel_toc;
#endif
+ if (!fp->is_func || extra_pass) {
+ if (bpf_jit_binary_pack_finalize(fhdr, hdr))
+ goto out_err;
+ }
+
fp->bpf_func = (void *)fimage;
fp->jited = 1;
fp->jited_len = cgctx.idx * 4 + FUNCTION_DESCR_SIZE;
if (!fp->is_func || extra_pass) {
- if (bpf_jit_binary_pack_finalize(fhdr, hdr)) {
- fp = org_fp;
- goto out_addrs;
- }
bpf_prog_fill_jited_linfo(fp, addrs);
/*
* On ABI V1, executable code starts after the function
@@ -398,11 +374,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
jit_data->hdr = hdr;
}
-out:
- if (bpf_blinded)
- bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);
-
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto out_addrs;
}
/*
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index f7fd4afc3ca3..36f0aea8096d 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -44,29 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, i;
struct rv_jit_data *jit_data;
struct rv_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
if (!jit_data) {
- prog = orig_prog;
- goto out;
+ return prog;
}
prog->aux->jit_data = jit_data;
}
@@ -83,15 +73,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
+ if (!ctx->offset)
goto out_offset;
- }
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_offset;
- }
for (i = 0; i < prog->len; i++) {
prev_ninsns += 32;
@@ -105,10 +91,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
ctx->prologue_len = ctx->ninsns;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, ctx->offset))
goto out_offset;
- }
ctx->epilogue_offset = ctx->ninsns;
bpf_jit_build_epilogue(ctx);
@@ -126,10 +110,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
bpf_fill_ill_insns);
- if (!jit_data->ro_header) {
- prog = orig_prog;
+ if (!jit_data->ro_header)
goto out_offset;
- }
/*
* Use the image(RW) for writing the JITed instructions. But also save
@@ -150,7 +132,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (i == NR_JIT_ITERATIONS) {
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
- prog = orig_prog;
goto out_free_hdr;
}
@@ -163,26 +144,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->nexentries = 0;
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_free_hdr;
- }
bpf_jit_build_epilogue(ctx);
if (bpf_jit_enable > 1)
bpf_jit_dump(prog->len, prog_size, pass, ctx->insns);
- prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
- prog->jited = 1;
- prog->jited_len = prog_size - cfi_get_offset();
-
if (!prog->is_func || extra_pass) {
if (WARN_ON(bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header))) {
/* ro_header has been freed */
jit_data->ro_header = NULL;
- prog = orig_prog;
- goto out_offset;
+ jit_data->header = NULL;
+ goto out_free_hdr;
}
+ }
+
+ prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
+ prog->jited = 1;
+ prog->jited_len = prog_size - cfi_get_offset();
+
+ if (!prog->is_func || extra_pass) {
for (i = 0; i < prog->len; i++)
ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
bpf_prog_fill_jited_linfo(prog, ctx->offset);
@@ -191,14 +173,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (jit_data->header) {
bpf_arch_text_copy(&jit_data->ro_header->size, &jit_data->header->size,
sizeof(jit_data->header->size));
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index d08d159b6319..2dfc279b1be2 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2314,36 +2314,20 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
*/
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
- struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
- bool tmp_blinded = false;
bool extra_pass = false;
struct bpf_jit jit;
int pass;
if (!fp->jit_requested)
- return orig_fp;
-
- tmp = bpf_jit_blind_constants(fp);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_fp;
- if (tmp != fp) {
- tmp_blinded = true;
- fp = tmp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = orig_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
if (jit_data->ctx.addrs) {
@@ -2356,34 +2340,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
memset(&jit, 0, sizeof(jit));
jit.addrs = kvcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
- if (jit.addrs == NULL) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (jit.addrs == NULL)
+ goto out_err;
/*
* Three initial passes:
* - 1/2: Determine clobbered registers
* - 3: Calculate program size and addrs array
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp, extra_pass)) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (bpf_jit_prog(&jit, fp, extra_pass))
+ goto out_err;
}
/*
* Final pass: Allocate and generate program
*/
header = bpf_jit_alloc(&jit, fp);
- if (!header) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (!header)
+ goto out_err;
skip_init_ctx:
if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
if (bpf_jit_enable > 1) {
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
@@ -2392,8 +2369,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
if (!fp->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
} else {
jit_data->header = header;
@@ -2411,11 +2387,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
kfree(jit_data);
fp->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(fp, fp == orig_fp ?
- tmp : orig_fp);
+
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto free_addrs;
}
bool bpf_jit_supports_kfunc_call(void)
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index b23d1c645ae5..e83e29137566 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1479,37 +1479,22 @@ struct sparc64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
u32 prev_image_size, image_size;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
int pass, i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1527,10 +1512,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kmalloc_array(prog->len, sizeof(unsigned int), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (ctx.offset == NULL)
+ goto out_err;
/* Longest sequence emitted is for bswap32, 12 instructions. Pre-cook
* the offset array so that we converge faster.
@@ -1543,10 +1526,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx = 0;
build_prologue(&ctx);
- if (build_body(&ctx)) {
- prog = orig_prog;
- goto out_off;
- }
+ if (build_body(&ctx))
+ goto out_err;
build_epilogue(&ctx);
if (bpf_jit_enable > 1)
@@ -1569,10 +1550,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = sizeof(u32) * ctx.idx;
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
- if (header == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (header == NULL)
+ goto out_err;
ctx.image = (u32 *)image_ptr;
skip_init_ctx:
@@ -1582,8 +1561,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (build_body(&ctx)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
build_epilogue(&ctx);
@@ -1592,8 +1570,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf_jit: Failed to converge, prev_size=%u size=%d\n",
prev_image_size, ctx.idx * 4);
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
if (bpf_jit_enable > 1)
@@ -1604,8 +1581,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (!prog->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
} else {
jit_data->ctx = ctx;
@@ -1624,9 +1600,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_off;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e9b78040d703..77d00a8dec87 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3717,13 +3717,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
void __percpu *priv_stack_ptr = NULL;
struct x64_jit_data *jit_data;
int priv_stack_alloc_sz;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
bool extra_pass = false;
bool padding = false;
u8 *rw_image = NULL;
@@ -3733,27 +3731,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -3765,10 +3749,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 8) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 8, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -3786,10 +3768,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto skip_init_addrs;
}
addrs = kvmalloc_objs(*addrs, prog->len + 1);
- if (!addrs) {
- prog = orig_prog;
+ if (!addrs)
goto out_addrs;
- }
/*
* Before first pass, make a rough estimation of addrs[]
@@ -3820,8 +3800,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
sizeof(rw_header->size));
bpf_jit_binary_pack_free(header, rw_header);
}
- /* Fall back to interpreter mode */
- prog = orig_prog;
if (extra_pass) {
prog->bpf_func = NULL;
prog->jited = 0;
@@ -3852,10 +3830,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
prog->aux->extable = (void *) image + roundup(proglen, align);
}
oldproglen = proglen;
@@ -3908,8 +3884,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image + cfi_get_offset();
prog->jited = 1;
prog->jited_len = proglen - cfi_get_offset();
- } else {
- prog = orig_prog;
}
if (!image || !prog->is_func || extra_pass) {
@@ -3925,10 +3899,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
}
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index dda423025c3d..5f259577614a 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2521,35 +2521,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
u8 *image = NULL;
int *addrs;
int pass;
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
addrs = kmalloc_objs(*addrs, prog->len);
- if (!addrs) {
- prog = orig_prog;
- goto out;
- }
+ if (!addrs)
+ return prog;
/*
* Before first pass, make a rough estimation of addrs[]
@@ -2574,7 +2558,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image = NULL;
if (header)
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_addrs;
}
if (image) {
@@ -2588,10 +2571,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (proglen == oldproglen) {
header = bpf_jit_binary_alloc(proglen, &image,
1, jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
}
oldproglen = proglen;
cond_resched();
@@ -2604,16 +2585,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image;
prog->jited = 1;
prog->jited_len = proglen;
- } else {
- prog = orig_prog;
}
out_addrs:
kfree(addrs);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
}
diff --git a/include/linux/filter.h b/include/linux/filter.h
index f552170eacf4..9fa4d4090093 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1184,6 +1184,18 @@ static inline bool bpf_dump_raw_ok(const struct cred *cred)
struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
const struct bpf_insn *patch, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len);
+#else
+static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+#endif /* CONFIG_BPF_SYSCALL */
+
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
static inline bool xdp_return_frame_no_direct(void)
@@ -1310,9 +1322,14 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
const char *bpf_jit_get_prog_name(struct bpf_prog *prog);
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *fp);
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other);
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return prog->blinding_requested && !prog->blinded;
+}
+
static inline void bpf_jit_dump(unsigned int flen, unsigned int proglen,
u32 pass, void *image)
{
@@ -1451,6 +1468,20 @@ static inline void bpf_prog_kallsyms_del(struct bpf_prog *fp)
{
}
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return false;
+}
+
+static inline
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
+{
+ return prog;
+}
+
+static inline void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
+{
+}
#endif /* CONFIG_BPF_JIT */
void bpf_prog_kallsyms_del_all(struct bpf_prog *fp);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 066b86e7233c..fc9fb3c07866 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1508,7 +1508,11 @@ static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
#endif
}
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
+/*
+ * Now this function is used only to blind the main prog and must be invoked only when
+ * bpf_prog_need_blind() returns true.
+ */
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
struct bpf_prog *clone, *tmp;
@@ -1516,13 +1520,17 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
struct bpf_insn *insn;
int i, rewritten;
- if (!prog->blinding_requested || prog->blinded)
- return prog;
+ if (WARN_ON_ONCE(env && env->prog != prog))
+ return ERR_PTR(-EINVAL);
clone = bpf_prog_clone_create(prog, GFP_USER);
if (!clone)
return ERR_PTR(-ENOMEM);
+ /* make sure bpf_patch_insn_data() patches the correct prog */
+ if (env)
+ env->prog = clone;
+
insn_cnt = clone->len;
insn = clone->insnsi;
@@ -1550,21 +1558,35 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!rewritten)
continue;
- tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
- if (IS_ERR(tmp)) {
+ if (env)
+ tmp = bpf_patch_insn_data(env, i, insn_buff, rewritten);
+ else
+ tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
+
+ if (IS_ERR_OR_NULL(tmp)) {
+ if (env)
+ /* restore the original prog */
+ env->prog = prog;
/* Patching may have repointed aux->prog during
* realloc from the original one, so we need to
* fix it up here on error.
*/
bpf_jit_prog_release_other(prog, clone);
- return tmp;
+ return IS_ERR(tmp) ? tmp : ERR_PTR(-ENOMEM);
}
clone = tmp;
insn_delta = rewritten - 1;
- /* Instructions arrays must be updated using absolute xlated offsets */
- adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+ if (env)
+ env->prog = clone;
+ else
+ /*
+ * Instructions arrays must be updated using absolute xlated offsets.
+ * The arrays have already been adjusted by bpf_patch_insn_data() when
+ * env is not NULL.
+ */
+ adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2533,6 +2555,35 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+{
+#ifdef CONFIG_BPF_JIT
+ struct bpf_prog *orig_prog;
+
+ if (!bpf_prog_need_blind(prog))
+ return bpf_int_jit_compile(prog);
+
+ orig_prog = prog;
+ prog = bpf_jit_blind_constants(NULL, prog);
+ /*
+ * If blinding was requested and we failed during blinding, we must fall
+ * back to the interpreter.
+ */
+ if (IS_ERR(prog))
+ return orig_prog;
+
+ prog = bpf_int_jit_compile(prog);
+ if (prog->jited) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ return prog;
+ }
+
+ bpf_jit_prog_release_other(orig_prog, prog);
+ prog = orig_prog;
+#endif
+ return prog;
+}
+
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
* @fp: bpf_prog populated with BPF program
@@ -2572,7 +2623,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_int_jit_compile(fp);
+ fp = bpf_prog_jit_compile(fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index dd00a680e4ea..721b830b5ef2 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -232,8 +232,8 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-static struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -973,7 +973,47 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
return 0;
}
-int bpf_jit_subprogs(struct bpf_verifier_env *env)
+static u32 *bpf_dup_subprog_starts(struct bpf_verifier_env *env)
+{
+ u32 *starts = NULL;
+
+ starts = kvmalloc_objs(u32, env->subprog_cnt, GFP_KERNEL_ACCOUNT);
+ if (starts) {
+ for (int i = 0; i < env->subprog_cnt; i++)
+ starts[i] = env->subprog_info[i].start;
+ }
+ return starts;
+}
+
+static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)
+{
+ for (int i = 0; i < env->subprog_cnt; i++)
+ env->subprog_info[i].start = orig_starts[i];
+ /* restore the start of fake 'exit' subprog as well */
+ env->subprog_info[env->subprog_cnt].start = env->prog->len;
+}
+
+static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ size_t size;
+ void *new_aux;
+
+ size = array_size(sizeof(struct bpf_insn_aux_data), env->prog->len);
+ new_aux = __vmalloc(size, GFP_KERNEL_ACCOUNT);
+ if (new_aux)
+ memcpy(new_aux, env->insn_aux_data, size);
+ return new_aux;
+}
+
+static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+ /* the expanded elements are zero-filled, so no special handling is required */
+ vfree(env->insn_aux_data);
+ env->insn_aux_data = orig_insn_aux;
+}
+
+static int jit_subprogs(struct bpf_verifier_env *env)
{
struct bpf_prog *prog = env->prog, **func, *tmp;
int i, j, subprog_start, subprog_end = 0, len, subprog;
@@ -981,10 +1021,6 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
struct bpf_insn *insn;
void *old_bpf_func;
int err, num_exentries;
- int old_len, subprog_start_adjustment = 0;
-
- if (env->subprog_cnt <= 1)
- return 0;
for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
if (!bpf_pseudo_func(insn) && !bpf_pseudo_call(insn))
@@ -1053,10 +1089,11 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
goto out_free;
func[i]->is_func = 1;
func[i]->sleepable = prog->sleepable;
+ func[i]->blinded = prog->blinded;
func[i]->aux->func_idx = i;
/* Below members will be freed only at prog->aux */
func[i]->aux->btf = prog->aux->btf;
- func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
+ func[i]->aux->subprog_start = subprog_start;
func[i]->aux->func_info = prog->aux->func_info;
func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -1113,15 +1150,7 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->token = prog->aux->token;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
-
- /*
- * To properly pass the absolute subprog start to jit
- * all instruction adjustments should be accumulated
- */
- old_len = func[i]->len;
func[i] = bpf_int_jit_compile(func[i]);
- subprog_start_adjustment += func[i]->len - old_len;
-
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1247,16 +1276,87 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
}
kfree(func);
out_undo_insn:
+ bpf_prog_jit_attempt_done(prog);
+ return err;
+}
+
+int bpf_jit_subprogs(struct bpf_verifier_env *env)
+{
+ int err, i;
+ bool blinded = false;
+ struct bpf_insn *insn;
+ struct bpf_prog *prog, *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
+ u32 *orig_subprog_starts;
+
+ if (env->subprog_cnt <= 1)
+ return 0;
+
+ prog = orig_prog = env->prog;
+ if (bpf_prog_need_blind(prog)) {
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux) {
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ orig_subprog_starts = bpf_dup_subprog_starts(env);
+ if (!orig_subprog_starts) {
+ vfree(orig_insn_aux);
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ prog = bpf_jit_blind_constants(env, prog);
+ if (IS_ERR(prog)) {
+ err = -ENOMEM;
+ prog = orig_prog;
+ goto out_restore;
+ }
+ blinded = true;
+ }
+
+ err = jit_subprogs(env);
+ if (err)
+ goto out_jit_err;
+
+ if (blinded) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ kvfree(orig_subprog_starts);
+ vfree(orig_insn_aux);
+ }
+
+ return 0;
+
+out_jit_err:
+ if (blinded) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ /* roll back to the clean original prog */
+ prog = env->prog = orig_prog;
+ goto out_restore;
+ } else {
+ if (err != -EFAULT) {
+ /*
+ * We will fall back to interpreter mode when err is not -EFAULT, before
+ * that, insn->off and insn->imm should be restored to their original
+ * values since they were modified by jit_subprogs.
+ */
+ for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
+ if (!bpf_pseudo_call(insn))
+ continue;
+ insn->off = 0;
+ insn->imm = env->insn_aux_data[i].call_imm;
+ }
+ }
+ goto out_cleanup;
+ }
+
+out_restore:
+ bpf_restore_subprog_starts(env, orig_subprog_starts);
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
+ kvfree(orig_subprog_starts);
+out_cleanup:
/* cleanup main prog to be interpreted */
prog->jit_requested = 0;
prog->blinding_requested = 0;
- for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
- if (!bpf_pseudo_call(insn))
- continue;
- insn->off = 0;
- insn->imm = env->insn_aux_data[i].call_imm;
- }
- bpf_prog_jit_attempt_done(prog);
return err;
}
--
2.43.0
^ permalink raw reply related
* Re: [PATCH v5 04/12] coresight: etm4x: exclude ss_status from drvdata->config
From: Jie Gan @ 2026-04-16 7:20 UTC (permalink / raw)
To: Yeoreum Yun
Cc: coresight, linux-arm-kernel, linux-kernel, suzuki.poulose,
mike.leach, james.clark, alexander.shishkin, leo.yan
In-Reply-To: <aeCHnToKG43oKQNF@e129823.arm.com>
On 4/16/2026 2:54 PM, Yeoreum Yun wrote:
> Hi Jie,
>
Hi Yeoreum,
>>
>>
>> On 4/16/2026 12:55 AM, Yeoreum Yun wrote:
>>> The purpose of TRCSSCSRn register is to show status of
>>> the corresponding Single-shot Comparator Control and input supports.
>>> That means writable field's purpose for reset or restore from idle status
>>> not for configuration.
>>>
>>> Therefore, exclude ss_status from drvdata->config, move it to etm4x_caps
>>> and rename it to ss_smp.
>>>
>>> This includes remove TRCSSCRn from configurable item and
>>> remove saving in etm4_disable_hw().
>>>
>>> Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
>>> ---
>>> .../hwtracing/coresight/coresight-etm4x-cfg.c | 1 -
>>> .../coresight/coresight-etm4x-core.c | 19 ++++++-------------
>>> .../coresight/coresight-etm4x-sysfs.c | 7 ++-----
>>> drivers/hwtracing/coresight/coresight-etm4x.h | 7 ++++++-
>>> 4 files changed, 14 insertions(+), 20 deletions(-)
>>>
>>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> index c302072b293a..d14d7c8a23e5 100644
>>> --- a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> @@ -86,7 +86,6 @@ static int etm4_cfg_map_reg_offset(struct etmv4_drvdata *drvdata,
>>> off_mask = (offset & GENMASK(11, 5));
>>> do {
>>> CHECKREGIDX(TRCSSCCRn(0), ss_ctrl, idx, off_mask);
>>> - CHECKREGIDX(TRCSSCSRn(0), ss_status, idx, off_mask);
>>> CHECKREGIDX(TRCSSPCICRn(0), ss_pe_cmp, idx, off_mask);
>>> } while (0);
>>> } else if ((offset >= TRCCIDCVRn(0)) && (offset <= TRCVMIDCVRn(7))) {
>>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> index b2b092a76eb5..f55338a4989d 100644
>>> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> @@ -91,7 +91,7 @@ static bool etm4x_sspcicrn_present(struct etmv4_drvdata *drvdata, int n)
>>> const struct etmv4_caps *caps = &drvdata->caps;
>>> return (n < caps->nr_ss_cmp) && caps->nr_pe_cmp &&
>>> - (drvdata->config.ss_status[n] & TRCSSCSRn_PC);
>>> + (caps->ss_cmp[n] & TRCSSCSRn_PC);
>>> }
>>> u64 etm4x_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
>>> @@ -573,11 +573,9 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
>>> etm4x_relaxed_write32(csa, config->res_ctrl[i], TRCRSCTLRn(i));
>>> for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - /* always clear status bit on restart if using single-shot */
>>> - if (config->ss_ctrl[i] || config->ss_pe_cmp[i])
>>> - config->ss_status[i] &= ~TRCSSCSRn_STATUS;
>>> etm4x_relaxed_write32(csa, config->ss_ctrl[i], TRCSSCCRn(i));
>>> - etm4x_relaxed_write32(csa, config->ss_status[i], TRCSSCSRn(i));
>>> + /* always clear status and pending bits on restart if using single-shot */
>>> + etm4x_relaxed_write32(csa, 0x0, TRCSSCSRn(i));
>>> if (etm4x_sspcicrn_present(drvdata, i))
>>> etm4x_relaxed_write32(csa, config->ss_pe_cmp[i], TRCSSPCICRn(i));
>>> }
>>> @@ -1055,12 +1053,6 @@ static void etm4_disable_hw(struct etmv4_drvdata *drvdata)
>>> etm4_disable_trace_unit(drvdata);
>>> - /* read the status of the single shot comparators */
>>> - for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - config->ss_status[i] =
>>> - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> - }
>>> -
>>> /* read back the current counter values */
>>> for (i = 0; i < caps->nr_cntr; i++) {
>>> config->cntr_val[i] =
>>> @@ -1503,8 +1495,9 @@ static void etm4_init_arch_data(void *info)
>>> */
>>> caps->nr_ss_cmp = FIELD_GET(TRCIDR4_NUMSSCC_MASK, etmidr4);
>>> for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - drvdata->config.ss_status[i] =
>>> - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> + caps->ss_cmp[i] = etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> + caps->ss_cmp[i] &= (TRCSSCSRn_PC | TRCSSCSRn_DV |
>>> + TRCSSCSRn_DA | TRCSSCSRn_INST);
>>
>> Just re-go through this patch and had a question here:
>>
>> I’m not sure whether this new change should be documented in the ABI, given
>> that the TRCSSCSRn_STATUS bit is masked. In my opinion, this change breaks
>> the existing ABI description.
>>
>> Description from the ABI document:
>>
>> What: /sys/bus/coresight/devices/etm<N>/sshot_status
>> Date: December 2019
>> KernelVersion: 5.5
>> Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
>> Description: (Read) Print the current value of the selected single
>> shot status register.
>
> But, as I mentioned another thread:
> - https://lore.kernel.org/all/ad5yV2FoNbGGLE6R@e129823.arm.com/
>
> Till now, sysfs doesn't show the *current value* of the single shot
> state since the config->ss_status is updated enabled/disabled sysfs
> session. an I think once the session is disabled, other status bits
> (currently STATUS and PENDING bits) don't have any meaning.
>
> I think it's enough to change the doc's Description for this.
>
> Any thought?
Thanks for the info. I have missed this info, we are calling it
'comparator' instead of 'status' by focusing on the capability bits.
Make sense to me.
Thanks,
Jie
>
> --
> Sincerely,
> Yeoreum Yun
^ permalink raw reply
* Re: [PATCH] arm_pmu: acpi: fix reference leak on failed device registration
From: Johan Hovold @ 2026-04-16 7:23 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mark Rutland, Guangshuo Li, Will Deacon, Anshuman Khandual,
linux-arm-kernel, linux-perf-users, linux-kernel, stable
In-Reply-To: <2026041603-guts-crested-ef76@gregkh>
On Thu, Apr 16, 2026 at 06:40:55AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Apr 15, 2026 at 07:19:06PM +0100, Mark Rutland wrote:
> > AFAICT you're saying that the reference was taken *within*
> > platform_device_register(), and then platform_device_register() itself
> > has failed. I think it's surprising that platform_device_register()
> > doesn't clean that up itself in the case of an error.
> >
> > There are *tonnes* of calls to platform_device_register() throughout the
> > kernel that don't even bother to check the return value, and many that
> > just pass the return onto a caller that can't possibly know to call
> > platform_device_put().
> >
> > Code in the same file as platform_device_register() expects it to clean up
> > after itself, e.g.
> >
> > | int platform_add_devices(struct platform_device **devs, int num)
> > | {
> > | int i, ret = 0;
> > |
> > | for (i = 0; i < num; i++) {
> > | ret = platform_device_register(devs[i]);
> > | if (ret) {
> > | while (--i >= 0)
> > | platform_device_unregister(devs[i]);
> > | break;
> > | }
> > | }
> > |
> > | return ret;
> > | }
> >
> > That's been there since the initial git commit, and back then,
> > platform_device_register() didn't mention that callers needed to perform
> > any cleanup.
> >
> > I see a comment was added to platform_device_register() in commit:
> >
> > 67e532a42cf4 ("driver core: platform: document registration-failure requirement")
> >
> > ... and that copied the commend added for device_register() in commit:
> >
> > 5739411acbaa ("Driver core: Clarify device cleanup.")
> >
> > ... but the potential brokenness is so widespread, and the behaviour is
> > so surprising, that I'd argue the real but is that device_register()
> > doesn't clean up in case of error. I don't think it's worth changing
> > this single instance given the prevalance and churn fixing all of that
> > would involve.
> >
> > I think it would be far better to fix the core driver API such that when
> > those functions return an error, they've already cleaned up for
> > themselves.
> >
> > Greg, am I missing some functional reason why we can't rework
> > device_register() and friends to handle cleanup themselves? I appreciate
> > that'll involve churn for some callers, but AFAICT the majority of
> > callers don't have the required cleanup.
>
> Yes, we should fix the platform core code here, this should not be
> required to do everywhere as obviously we all got it wrong.
It's not just the platform code as this directly reflects the behaviour
of device_register() as Mark pointed out.
It is indeed an unfortunate quirk of the driver model, but one can argue
that having a registration function that frees its argument on errors
would be even worse. And even more so when many (or most) users get this
right.
So if we want to change this, I think we would need to deprecate
device_register() in favour of explicit device_initialize() and
device_add().
That said, most users of platform_device_register() appear to operate
on static platform devices which don't even have a release function and
would trigger a WARN() if we ever drop the reference (which is arguably
worse than leaking a tiny bit of memory).
So leaving things as-is is also an option.
Johan
^ permalink raw reply
* [PATCH net v3 0/3] net: airoha: Fix airoha_qdma_cleanup_tx_queue() processing
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
Add missing bits in airoha_qdma_cleanup_tx_queue routine.
Fix airoha_qdma_cleanup_tx_queue processing errors intorduced in commit
'3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order
DMA tx descriptors")'.
---
Changes in v3:
- Move ndesc initialization fix in a dedicated patch.
- Add patch 2/3 to move entries to queue head in case of DMA mapping
failure in airoha_dev_xmit().
- Cosmetics.
- Link to v2: https://lore.kernel.org/r/20260414-airoha_qdma_cleanup_tx_queue-fix-net-v2-1-875de57cc022@kernel.org
Changes in v2:
- Move q->ndesc initialization at end of airoha_qdma_init_tx routine in
order to avoid any possible NULL pointer dereference in
airoha_qdma_cleanup_tx_queue()
- Check if q->tx_list is empty in airoha_qdma_cleanup_tx_queue()
- Link to v1: https://lore.kernel.org/r/20260410-airoha_qdma_cleanup_tx_queue-fix-net-v1-1-b7171c8f1e78@kernel.org
---
Lorenzo Bianconi (3):
net: airoha: Move ndesc initialization at end of airoha_qdma_init_tx()
net: airoha: Move entries to queue head in case of DMA mapping failure in airoha_dev_xmit()
net: airoha: Add missing bits in airoha_qdma_cleanup_tx_queue()
drivers/net/ethernet/airoha/airoha_eth.c | 42 ++++++++++++++++++++++++++------
1 file changed, 35 insertions(+), 7 deletions(-)
---
base-commit: 3f20012a3964f487ae1e9ff942e2f35d4e9595bf
change-id: 20260410-airoha_qdma_cleanup_tx_queue-fix-net-93375f5ee80f
Best regards,
--
Lorenzo Bianconi <lorenzo@kernel.org>
^ permalink raw reply
* [PATCH net v3 1/3] net: airoha: Move ndesc initialization at end of airoha_qdma_init_tx()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
If queue entry list allocation fails in airoha_qdma_init_tx_queue routine,
airoha_qdma_cleanup_tx_queue() will trigger a NULL pointer dereference
accessing the queue entry array. The issue is due to the early ndesc
initialization in airoha_qdma_init_tx_queue(). Fix the issue moving ndesc
initialization at end of airoha_qdma_init_tx routine.
Fixes: 3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order DMA tx descriptors")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index e1ab15f1ee7d..690bfaf8d7d9 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -954,27 +954,27 @@ static int airoha_qdma_init_tx_queue(struct airoha_queue *q,
dma_addr_t dma_addr;
spin_lock_init(&q->lock);
- q->ndesc = size;
q->qdma = qdma;
q->free_thr = 1 + MAX_SKB_FRAGS;
INIT_LIST_HEAD(&q->tx_list);
- q->entry = devm_kzalloc(eth->dev, q->ndesc * sizeof(*q->entry),
+ q->entry = devm_kzalloc(eth->dev, size * sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
- q->desc = dmam_alloc_coherent(eth->dev, q->ndesc * sizeof(*q->desc),
+ q->desc = dmam_alloc_coherent(eth->dev, size * sizeof(*q->desc),
&dma_addr, GFP_KERNEL);
if (!q->desc)
return -ENOMEM;
- for (i = 0; i < q->ndesc; i++) {
+ for (i = 0; i < size; i++) {
u32 val = FIELD_PREP(QDMA_DESC_DONE_MASK, 1);
list_add_tail(&q->entry[i].list, &q->tx_list);
WRITE_ONCE(q->desc[i].ctrl, cpu_to_le32(val));
}
+ q->ndesc = size;
/* xmit ring drop default setting */
airoha_qdma_set(qdma, REG_TX_RING_BLOCKING(qid),
--
2.53.0
^ permalink raw reply related
* [PATCH net v3 2/3] net: airoha: Move entries to queue head in case of DMA mapping failure in airoha_dev_xmit()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
In order to respect the original descriptor order and avoid any
potential IOMMU fault or memory corruption, move pending queue entries
to the head of hw queue tx_list if the DMA mapping of current inflight
packet fails in airoha_dev_xmit routine.
Fixes: 3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order DMA tx descriptors")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index 690bfaf8d7d9..dfe8a0bd2abd 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -2053,7 +2053,7 @@ static netdev_tx_t airoha_dev_xmit(struct sk_buff *skb,
dma_unmap_single(dev->dev.parent, e->dma_addr, e->dma_len,
DMA_TO_DEVICE);
e->dma_addr = 0;
- list_move_tail(&e->list, &q->tx_list);
+ list_move(&e->list, &q->tx_list);
}
spin_unlock_bh(&q->lock);
--
2.53.0
^ permalink raw reply related
* [PATCH net v3 3/3] net: airoha: Add missing bits in airoha_qdma_cleanup_tx_queue()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
Similar to airoha_qdma_cleanup_rx_queue(), reset DMA TX descriptors in
airoha_qdma_cleanup_tx_queue routine. Moreover, reset TX_DMA_IDX to
TX_CPU_IDX to notify the NIC the QDMA TX ring is empty.
Fixes: 23020f0493270 ("net: airoha: Introduce ethernet support for EN7581 SoC")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 32 ++++++++++++++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index dfe8a0bd2abd..903682e88e89 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -1039,12 +1039,15 @@ static int airoha_qdma_init_tx(struct airoha_qdma *qdma)
static void airoha_qdma_cleanup_tx_queue(struct airoha_queue *q)
{
- struct airoha_eth *eth = q->qdma->eth;
- int i;
+ struct airoha_qdma *qdma = q->qdma;
+ struct airoha_eth *eth = qdma->eth;
+ int i, qid = q - &qdma->q_tx[0];
+ u16 index = 0;
spin_lock_bh(&q->lock);
for (i = 0; i < q->ndesc; i++) {
struct airoha_queue_entry *e = &q->entry[i];
+ struct airoha_qdma_desc *desc = &q->desc[i];
if (!e->dma_addr)
continue;
@@ -1055,8 +1058,33 @@ static void airoha_qdma_cleanup_tx_queue(struct airoha_queue *q)
e->dma_addr = 0;
e->skb = NULL;
list_add_tail(&e->list, &q->tx_list);
+
+ /* Reset DMA descriptor */
+ WRITE_ONCE(desc->ctrl, 0);
+ WRITE_ONCE(desc->addr, 0);
+ WRITE_ONCE(desc->data, 0);
+ WRITE_ONCE(desc->msg0, 0);
+ WRITE_ONCE(desc->msg1, 0);
+ WRITE_ONCE(desc->msg2, 0);
+
q->queued--;
}
+
+ if (!list_empty(&q->tx_list)) {
+ struct airoha_queue_entry *e;
+
+ e = list_first_entry(&q->tx_list, struct airoha_queue_entry,
+ list);
+ index = e - q->entry;
+ }
+ /* Set TX_DMA_IDX to TX_CPU_IDX to notify the hw the QDMA TX ring is
+ * empty.
+ */
+ airoha_qdma_rmw(qdma, REG_TX_CPU_IDX(qid), TX_RING_CPU_IDX_MASK,
+ FIELD_PREP(TX_RING_CPU_IDX_MASK, index));
+ airoha_qdma_rmw(qdma, REG_TX_DMA_IDX(qid), TX_RING_DMA_IDX_MASK,
+ FIELD_PREP(TX_RING_DMA_IDX_MASK, index));
+
spin_unlock_bh(&q->lock);
}
--
2.53.0
^ permalink raw reply related
* [PATCH v7 0/3] pinctrl: aspeed: Add AST2700 SoC0 support
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
AST2700 is composed of two interconnected SoC instances, each providing
its own pin control hardware. This series introduces bindings describing
the AST2700 pinctrl architecture and adds pinctrl driver support for the
SoC0 instance.
The bindings document the AST2700 dual-SoC design and follow common
pinctrl conventions, while the SoC0 driver implementation builds upon
the existing ASPEED pinctrl infrastructure.
---
Changes in v7:
- Fix schema validation error by ensuring if/then/else keywords are used
correctly.
- Remove unnecessary restrictions on AST2700 SoC0 pinctrl pin configuration
properties.
- Link to v6: https://lore.kernel.org/r/20260414-upstream_pinctrl-v6-0-709f2127da33@aspeedtech.com
Changes in v6:
- Restrict AST2700 SoC0 pinctrl pin configuration properties
(`drive-strength` and `bias-*`) to `pins`-based state nodes in the
binding schema.
- Move `memory-region` and `memory-region-names` in the AST2x00 SCU
binding to top-level descriptions, and keep the conditional schema
only to disallow them for non-AST2700 SCU0 compatibles.
- Add bias pull-up, pull-down, and disable support for AST2700 SoC0
GPIO18A/GPIO18B pins in the pinctrl driver.
- Fix the USB2 Port B XH/XHP mux selector definitions to use the
correct `PORTB_U2_XH_DESC` setting.
- Link to v5: https://lore.kernel.org/r/20260331-upstream_pinctrl-v5-0-8994f59ff367@aspeedtech.com
Changes in v5:
- Complete the AST2700 SCU0 binding and disallow child nodes that are
not relevant for the hardware (p2a-control and smp-memram).
- Add examples for both the AST2700 SCU0 binding and the pinctrl binding,
ensuring they are valid against the schema.
- Rework the pinctrl binding example to be self-contained and independent
of the SCU binding.
- Reorder the binding patches so the pinctrl binding is introduced before
the SCU binding update, allowing the SCU example to be added cleanly.
- Adjust the binding accordingly to restrict drive-strength to the
supported values.
- Update the drive-strength table to match hardware-defined values.
- Link to v4: https://lore.kernel.org/r/20260306-upstream_pinctrl-v4-0-ad4e8ab8b489@aspeedtech.com
Changes in v4:
- Rename series title to "pinctrl: aspeed: Add AST2700 SoC0 support"
to make it specific to SoC0.
- Remove unnecessary SCU example from bindings.
- Fix Makefile newline to avoid patch warning.
- Make pinctrl data structures const and align with existing Aspeed drivers.
- Sort the arrays and enums alphabetically.
- Minor cleanups for consistency, no functional changes.
- Link to v3: https://lore.kernel.org/r/20260120-upstream_pinctrl-v3-0-868fbf8413b5@aspeedtech.com
Changes in v3:
dt-bindings: pinctrl: aspeed: AST2700 pinctrl improvements
- Improved binding descriptions for SoC0 and SoC1 to better explain the
AST2700 dual-SoC architecture with independent pin control blocks
- Switched from additionalProperties to patternProperties using the
'-state$' suffix to restrict child node naming
- Removed per-binding examples based on review feedback
- Added additionalProperties: false at the top level for stricter schema
validation
- Dropped the aspeed,ast2700-soc1-pinctrl binding, as the SoC1 pinctrl
registers follow a regular layout and can be described using an
existing generic pinctrl binding
- Updated the function and group enum lists to match the definitions
used by the AST2700 pinctrl driver
dt-bindings: mfd: aspeed: Add AST2700 SCU example with pinctrl
- Added a complete AST2700 SCU0 example demonstrating pinctrl integration
- Example covers both pin function/group configuration and pin
drive-strength settings
- Updated child node naming to use the '-state' suffix, following common
pinctrl conventions
pinctrl: aspeed: AST2700 SoC0 driver improvements
- Refactored pin and signal declarations to use common ASPEED pinmux
macros (SIG_EXPR_LIST_DECL_SEMG, SIG_EXPR_LIST_DECL_SESG, PIN_DECL_*)
- Added SCU010 register definition for hardware strap control
- Reworked code structure to better align with existing ASPEED pinctrl
drivers
- Link to v2: https://lore.kernel.org/r/20250904103401.88287-1-billy_tsai@aspeedtech.com
Changes in v2:
- Update pinctrl aspeed binding files.
- Update the commit message for pinctrl binding patch.
- Link to v1: https://lore.kernel.org/r/20250829073030.2749482-1-billy_tsai@aspeedtech.com
---
Billy Tsai (3):
dt-bindings: pinctrl: Add aspeed,ast2700-soc0-pinctrl
dt-bindings: mfd: aspeed,ast2x00-scu: Describe AST2700 SCU0
pinctrl: aspeed: Add AST2700 SoC0 support
.../bindings/mfd/aspeed,ast2x00-scu.yaml | 113 ++++
.../pinctrl/aspeed,ast2700-soc0-pinctrl.yaml | 162 +++++
drivers/pinctrl/aspeed/Kconfig | 9 +
drivers/pinctrl/aspeed/Makefile | 1 +
drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c | 749 +++++++++++++++++++++
5 files changed, 1034 insertions(+)
---
base-commit: af4e9ef3d78420feb8fe58cd9a1ab80c501b3c08
change-id: 20251215-upstream_pinctrl-8f195df0a975
Best regards,
--
Billy Tsai <billy_tsai@aspeedtech.com>
^ permalink raw reply
* [PATCH v7 2/3] dt-bindings: mfd: aspeed,ast2x00-scu: Describe AST2700 SCU0
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
AST2700 consists of two interconnected SoC instances, each with its own
System Control Unit (SCU). The SCU0 provides pin control, interrupt
controllers, clocks, resets, and address-space mappings for the
Secondary and Tertiary Service Processors (SSP and TSP).
Describe the SSP/TSP address mappings using the standard
memory-region and memory-region-names properties.
Disallow legacy child nodes that are not present on AST2700, including
p2a-control and smp-memram. The latter is unnecessary as software can
access the scratch registers via the SCU syscon.
Also allow the AST2700 SoC0 pin controller to be described as a child
node of the SCU0, and add an example illustrating the SCU0 layout,
including reserved-memory, interrupt controllers, and pinctrl.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
.../bindings/mfd/aspeed,ast2x00-scu.yaml | 113 +++++++++++++++++++++
1 file changed, 113 insertions(+)
diff --git a/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml b/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
index a87f31fce019..215ff59b38ea 100644
--- a/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
+++ b/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
@@ -46,6 +46,17 @@ properties:
'#reset-cells':
const: 1
+ memory-region:
+ items:
+ - description: Region mapped through the first SSP address window.
+ - description: Region mapped through the second SSP address window.
+ - description: Region mapped through the TSP address window.
+ memory-region-names:
+ items:
+ - const: ssp-0
+ - const: ssp-1
+ - const: tsp
+
patternProperties:
'^p2a-control@[0-9a-f]+$':
description: >
@@ -87,6 +98,7 @@ patternProperties:
- aspeed,ast2400-pinctrl
- aspeed,ast2500-pinctrl
- aspeed,ast2600-pinctrl
+ - aspeed,ast2700-soc0-pinctrl
required:
- compatible
@@ -156,6 +168,30 @@ required:
- '#clock-cells'
- '#reset-cells'
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ anyOf:
+ - const: aspeed,ast2700-scu0
+ - const: aspeed,ast2700-scu1
+ then:
+ patternProperties:
+ '^p2a-control@[0-9a-f]+$': false
+ '^smp-memram@[0-9a-f]+$': false
+
+ - if:
+ not:
+ properties:
+ compatible:
+ contains:
+ const: aspeed,ast2700-scu0
+ then:
+ properties:
+ memory-region: false
+ memory-region-names: false
+
additionalProperties: false
examples:
@@ -180,4 +216,81 @@ examples:
reg = <0x7c 0x4>, <0x150 0x8>;
};
};
+
+ - |
+ / {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ ssp_region_0: memory@400000000 {
+ reg = <0x4 0x00000000 0x0 0x01000000>;
+ no-map;
+ };
+
+ ssp_region_1: memory@401000000 {
+ reg = <0x4 0x01000000 0x0 0x01000000>;
+ no-map;
+ };
+
+ tsp_region: memory@402000000 {
+ reg = <0x4 0x02000000 0x0 0x01000000>;
+ no-map;
+ };
+ };
+
+ bus {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ syscon@12c02000 {
+ compatible = "aspeed,ast2700-scu0", "syscon", "simple-mfd";
+ reg = <0 0x12c02000 0 0x1000>;
+ ranges = <0x0 0x0 0x12c02000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+
+ memory-region = <&ssp_region_0>, <&ssp_region_1>,
+ <&tsp_region>;
+ memory-region-names = "ssp-0", "ssp-1", "tsp";
+
+ silicon-id@0 {
+ compatible = "aspeed,ast2700-silicon-id", "aspeed,silicon-id";
+ reg = <0x0 0x4>;
+ };
+
+ interrupt-controller@1b0 {
+ compatible = "aspeed,ast2700-scu-ic0";
+ reg = <0x1b0 0x4>;
+ #interrupt-cells = <1>;
+ interrupts-extended = <&intc0 97>;
+ interrupt-controller;
+ };
+
+ interrupt-controller@1e0 {
+ compatible = "aspeed,ast2700-scu-ic1";
+ reg = <0x1e0 0x4>;
+ #interrupt-cells = <1>;
+ interrupts-extended = <&intc0 98>;
+ interrupt-controller;
+ };
+
+ pinctrl@400 {
+ compatible = "aspeed,ast2700-soc0-pinctrl";
+ reg = <0x400 0x318>;
+ emmc-state {
+ function = "EMMC";
+ groups = "EMMCG1";
+ };
+ };
+ };
+ };
+ };
+
...
--
2.34.1
^ permalink raw reply related
* [PATCH v7 1/3] dt-bindings: pinctrl: Add aspeed,ast2700-soc0-pinctrl
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
Add a device tree binding for the pin controller found in the
ASPEED AST2700 SoC0.
The controller manages various peripheral functions such as eMMC, USB,
VGA DDC, JTAG, and PCIe root complex signals.
Describe the AST2700 SoC0 pin controller using standard pin multiplexing
and configuration properties.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
.../pinctrl/aspeed,ast2700-soc0-pinctrl.yaml | 162 +++++++++++++++++++++
1 file changed, 162 insertions(+)
diff --git a/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml
new file mode 100644
index 000000000000..947f3cd09fcc
--- /dev/null
+++ b/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml
@@ -0,0 +1,162 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ASPEED AST2700 SoC0 Pin Controller
+
+maintainers:
+ - Billy Tsai <billy_tsai@aspeedtech.com>
+
+description:
+ The AST2700 features a dual-SoC architecture with two interconnected SoCs,
+ each having its own System Control Unit (SCU) for independent pin control.
+ This pin controller manages the pin multiplexing for SoC0.
+
+ The SoC0 pin controller manages pin functions including eMMC, VGA DDC,
+ dual USB3/USB2 ports (A and B), JTAG, and PCIe root complex interfaces.
+
+properties:
+ compatible:
+ const: aspeed,ast2700-soc0-pinctrl
+ reg:
+ maxItems: 1
+
+patternProperties:
+ '-state$':
+ type: object
+ allOf:
+ - $ref: pinmux-node.yaml#
+ - $ref: pincfg-node.yaml#
+
+ additionalProperties: false
+
+ properties:
+ function:
+ enum:
+ - EMMC
+ - JTAGDDR
+ - JTAGM0
+ - JTAGPCIEA
+ - JTAGPCIEB
+ - JTAGPSP
+ - JTAGSSP
+ - JTAGTSP
+ - JTAGUSB3A
+ - JTAGUSB3B
+ - PCIERC0PERST
+ - PCIERC1PERST
+ - TSPRSTN
+ - UFSCLKI
+ - USB2AD0
+ - USB2AD1
+ - USB2AH
+ - USB2AHP
+ - USB2AHPD0
+ - USB2AXH
+ - USB2AXH2B
+ - USB2AXHD1
+ - USB2AXHP
+ - USB2AXHP2B
+ - USB2AXHPD1
+ - USB2BD0
+ - USB2BD1
+ - USB2BH
+ - USB2BHP
+ - USB2BHPD0
+ - USB2BXH
+ - USB2BXH2A
+ - USB2BXHD1
+ - USB2BXHP
+ - USB2BXHP2A
+ - USB2BXHPD1
+ - USB3AXH
+ - USB3AXH2B
+ - USB3AXHD
+ - USB3AXHP
+ - USB3AXHP2B
+ - USB3AXHPD
+ - USB3BXH
+ - USB3BXH2A
+ - USB3BXHD
+ - USB3BXHP
+ - USB3BXHP2A
+ - USB3BXHPD
+ - VB
+ - VGADDC
+
+ groups:
+ enum:
+ - EMMCCDN
+ - EMMCG1
+ - EMMCG4
+ - EMMCG8
+ - EMMCWPN
+ - JTAG0
+ - PCIERC0PERST
+ - PCIERC1PERST
+ - TSPRSTN
+ - UFSCLKI
+ - USB2A
+ - USB2AAP
+ - USB2ABP
+ - USB2ADAP
+ - USB2AH
+ - USB2AHAP
+ - USB2B
+ - USB2BAP
+ - USB2BBP
+ - USB2BDBP
+ - USB2BH
+ - USB2BHBP
+ - USB3A
+ - USB3AAP
+ - USB3ABP
+ - USB3B
+ - USB3BAP
+ - USB3BBP
+ - VB0
+ - VB1
+ - VGADDC
+ pins:
+ enum:
+ - AB13
+ - AB14
+ - AC13
+ - AC14
+ - AD13
+ - AD14
+ - AE13
+ - AE14
+ - AE15
+ - AF13
+ - AF14
+ - AF15
+
+ drive-strength:
+ enum: [3, 6, 8, 11, 16, 18, 20, 23, 30, 32, 33, 35, 37, 38, 39, 41]
+
+ bias-disable: true
+ bias-pull-up: true
+ bias-pull-down: true
+
+required:
+ - compatible
+ - reg
+
+allOf:
+ - $ref: pinctrl.yaml#
+
+additionalProperties: false
+
+examples:
+ - |
+ pinctrl@400 {
+ compatible = "aspeed,ast2700-soc0-pinctrl";
+ reg = <0x400 0x318>;
+ emmc-state {
+ function = "EMMC";
+ groups = "EMMCG1";
+ };
+ };
--
2.34.1
^ permalink raw reply related
* [PATCH v7 3/3] pinctrl: aspeed: Add AST2700 SoC0 support
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
Add pinctrl support for the SoC0 instance of the ASPEED AST2700.
AST2700 consists of two interconnected SoC instances, each with its own
pinctrl register block.
The SoC0 pinctrl hardware closely follows the design found in previous
ASPEED BMC generations, allowing the driver to build upon the common
ASPEED pinctrl infrastructure.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
drivers/pinctrl/aspeed/Kconfig | 9 +
drivers/pinctrl/aspeed/Makefile | 1 +
drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c | 749 ++++++++++++++++++++++++
3 files changed, 759 insertions(+)
diff --git a/drivers/pinctrl/aspeed/Kconfig b/drivers/pinctrl/aspeed/Kconfig
index 1a4e5b9ed471..f9672cca891e 100644
--- a/drivers/pinctrl/aspeed/Kconfig
+++ b/drivers/pinctrl/aspeed/Kconfig
@@ -31,3 +31,12 @@ config PINCTRL_ASPEED_G6
help
Say Y here to enable pin controller support for Aspeed's 6th
generation SoCs. GPIO is provided by a separate GPIO driver.
+
+config PINCTRL_ASPEED_G7_SOC0
+ bool "Aspeed G7 SoC pin control"
+ depends on (ARCH_ASPEED || COMPILE_TEST) && OF
+ select PINCTRL_ASPEED
+ help
+ Say Y here to enable pin controller support for the SoC0 instance
+ of Aspeed's 7th generation SoCs. GPIO is provided by a separate
+ GPIO driver.
diff --git a/drivers/pinctrl/aspeed/Makefile b/drivers/pinctrl/aspeed/Makefile
index db2a7600ae2b..0de524ca2c72 100644
--- a/drivers/pinctrl/aspeed/Makefile
+++ b/drivers/pinctrl/aspeed/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_PINCTRL_ASPEED) += pinctrl-aspeed.o pinmux-aspeed.o
obj-$(CONFIG_PINCTRL_ASPEED_G4) += pinctrl-aspeed-g4.o
obj-$(CONFIG_PINCTRL_ASPEED_G5) += pinctrl-aspeed-g5.o
obj-$(CONFIG_PINCTRL_ASPEED_G6) += pinctrl-aspeed-g6.o
+obj-$(CONFIG_PINCTRL_ASPEED_G7_SOC0) += pinctrl-aspeed-g7-soc0.o
diff --git a/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c b/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c
new file mode 100644
index 000000000000..35a28b677318
--- /dev/null
+++ b/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c
@@ -0,0 +1,749 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bitops.h>
+#include <linux/bits.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include "pinctrl-aspeed.h"
+#include "pinmux-aspeed.h"
+#include "../pinctrl-utils.h"
+
+#define SCU200 0x200 /* System Reset Control #1 */
+
+#define SCU010 0x010 /* Hardware Strap Register */
+#define SCU400 0x400 /* Multi-function Pin Control #1 */
+#define SCU404 0x404 /* Multi-function Pin Control #2 */
+#define SCU408 0x408 /* Multi-function Pin Control #3 */
+#define SCU40C 0x40C /* Multi-function Pin Control #3 */
+#define SCU410 0x410 /* USB Multi-function Control Register */
+#define SCU414 0x414 /* VGA Function Control Register */
+
+#define SCU480 0x480 /* GPIO18A0 IO Control Register */
+#define SCU484 0x484 /* GPIO18A1 IO Control Register */
+#define SCU488 0x488 /* GPIO18A2 IO Control Register */
+#define SCU48C 0x48c /* GPIO18A3 IO Control Register */
+#define SCU490 0x490 /* GPIO18A4 IO Control Register */
+#define SCU494 0x494 /* GPIO18A5 IO Control Register */
+#define SCU498 0x498 /* GPIO18A6 IO Control Register */
+#define SCU49C 0x49c /* GPIO18A7 IO Control Register */
+#define SCU4A0 0x4A0 /* GPIO18B0 IO Control Register */
+#define SCU4A4 0x4A4 /* GPIO18B1 IO Control Register */
+#define SCU4A8 0x4A8 /* GPIO18B2 IO Control Register */
+#define SCU4AC 0x4AC /* GPIO18B3 IO Control Register */
+
+enum {
+ AC14,
+ AE15,
+ AD14,
+ AE14,
+ AF14,
+ AB13,
+ AB14,
+ AF15,
+ AF13,
+ AC13,
+ AD13,
+ AE13,
+ JTAG_PORT,
+ PCIERC0_PERST,
+ PCIERC1_PERST,
+ PORTA_MODE,
+ PORTA_U2,
+ PORTB_MODE,
+ PORTB_U2,
+ PORTA_U2_PHY,
+ PORTB_U2_PHY,
+ PORTA_U3,
+ PORTB_U3,
+ PORTA_U3_PHY,
+ PORTB_U3_PHY,
+};
+
+SIG_EXPR_LIST_DECL_SEMG(AC14, EMMCCLK, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 0));
+SIG_EXPR_LIST_DECL_SESG(AC14, VB1CS, VB1, SIG_DESC_SET(SCU404, 0));
+PIN_DECL_2(AC14, GPIO18A0, EMMCCLK, VB1CS);
+
+SIG_EXPR_LIST_DECL_SEMG(AE15, EMMCCMD, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 1));
+SIG_EXPR_LIST_DECL_SESG(AE15, VB1CK, VB1, SIG_DESC_SET(SCU404, 1));
+PIN_DECL_2(AE15, GPIO18A1, EMMCCMD, VB1CK);
+
+SIG_EXPR_LIST_DECL_SEMG(AD14, EMMCDAT0, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 2));
+SIG_EXPR_LIST_DECL_SESG(AD14, VB1MOSI, VB1, SIG_DESC_SET(SCU404, 2));
+PIN_DECL_2(AD14, GPIO18A2, EMMCDAT0, VB1MOSI);
+
+SIG_EXPR_LIST_DECL_SEMG(AE14, EMMCDAT1, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 3));
+SIG_EXPR_LIST_DECL_SESG(AE14, VB1MISO, VB1, SIG_DESC_SET(SCU404, 3));
+PIN_DECL_2(AE14, GPIO18A3, EMMCDAT1, VB1MISO);
+
+SIG_EXPR_LIST_DECL_SEMG(AF14, EMMCDAT2, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 4));
+PIN_DECL_1(AF14, GPIO18A4, EMMCDAT2);
+
+SIG_EXPR_LIST_DECL_SEMG(AB13, EMMCDAT3, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 5));
+PIN_DECL_1(AB13, GPIO18A5, EMMCDAT3);
+
+SIG_EXPR_LIST_DECL_SEMG(AB14, EMMCCDN, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 6));
+SIG_EXPR_LIST_DECL_SESG(AB14, VB0CS, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_2(AB14, GPIO18A6, EMMCCDN, VB0CS);
+
+SIG_EXPR_LIST_DECL_SEMG(AF15, EMMCWPN, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 7));
+SIG_EXPR_LIST_DECL_SESG(AF15, VB0CK, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_2(AF15, GPIO18A7, EMMCWPN, VB0CK);
+
+SIG_EXPR_LIST_DECL_SESG(AF13, TSPRSTN, TSPRSTN, SIG_DESC_SET(SCU010, 9));
+SIG_EXPR_LIST_DECL_SEMG(AF13, EMMCDAT4, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 8));
+SIG_EXPR_LIST_DECL_SESG(AF13, VB0MOSI, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_3(AF13, GPIO18B0, TSPRSTN, EMMCDAT4, VB0MOSI);
+
+SIG_EXPR_LIST_DECL_SESG(AC13, UFSCLKI, UFSCLKI, SIG_DESC_SET(SCU010, 19));
+SIG_EXPR_LIST_DECL_SEMG(AC13, EMMCDAT5, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 9));
+SIG_EXPR_LIST_DECL_SESG(AC13, VB0MISO, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_3(AC13, GPIO18B1, UFSCLKI, EMMCDAT5, VB0MISO);
+
+SIG_EXPR_LIST_DECL_SEMG(AD13, EMMCDAT6, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 10));
+SIG_EXPR_LIST_DECL_SESG(AD13, DDCCLK, VGADDC, SIG_DESC_SET(SCU404, 10));
+PIN_DECL_2(AD13, GPIO18B2, EMMCDAT6, DDCCLK);
+
+SIG_EXPR_LIST_DECL_SEMG(AE13, EMMCDAT7, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 11));
+SIG_EXPR_LIST_DECL_SESG(AE13, DDCDAT, VGADDC, SIG_DESC_SET(SCU404, 11));
+PIN_DECL_2(AE13, GPIO18B3, EMMCDAT7, DDCDAT);
+
+GROUP_DECL(EMMCG1, AC14, AE15, AD14);
+GROUP_DECL(EMMCG4, AC14, AE15, AD14, AE14, AF14, AB13);
+GROUP_DECL(EMMCG8, AC14, AE15, AD14, AE14, AF14, AB13, AF13, AC13, AD13, AE13);
+GROUP_DECL(EMMCWPN, AF15);
+GROUP_DECL(EMMCCDN, AB14);
+FUNC_DECL_(EMMC, "EMMCG1", "EMMCG4", "EMMCG8", "EMMCWPN", "EMMCCDN");
+
+GROUP_DECL(VB1, AC14, AE15, AD14, AE14);
+GROUP_DECL(VB0, AF15, AB14, AF13, AC13);
+FUNC_DECL_2(VB, VB1, VB0);
+
+FUNC_GROUP_DECL(TSPRSTN, AF13);
+
+FUNC_GROUP_DECL(UFSCLKI, AC13);
+
+FUNC_GROUP_DECL(VGADDC, AD13, AE13);
+
+/* JTAG Port Selection */
+#define JTAG_PORT_PSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x0, 0 }
+#define JTAG_PORT_SSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x41, 0 }
+#define JTAG_PORT_TSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x42, 0 }
+#define JTAG_PORT_DDR_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x43, 0 }
+#define JTAG_PORT_USB3A_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x44, 0 }
+#define JTAG_PORT_USB3B_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x45, 0 }
+#define JTAG_PORT_PCIEA_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x46, 0 }
+#define JTAG_PORT_PCIEB_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x47, 0 }
+#define JTAG_PORT_JTAGM0_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x8, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPSP, JTAG0, JTAGPSP, JTAG_PORT_PSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGSSP, JTAG0, JTAGSSP, JTAG_PORT_SSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGTSP, JTAG0, JTAGTSP, JTAG_PORT_TSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGDDR, JTAG0, JTAGDDR, JTAG_PORT_DDR_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGUSB3A, JTAG0, JTAGUSB3A, JTAG_PORT_USB3A_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGUSB3B, JTAG0, JTAGUSB3B, JTAG_PORT_USB3B_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPCIEA, JTAG0, JTAGPCIEA, JTAG_PORT_PCIEA_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPCIEB, JTAG0, JTAGPCIEB, JTAG_PORT_PCIEB_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGM0, JTAG0, JTAGM0, JTAG_PORT_JTAGM0_DESC);
+PIN_DECL_(JTAG_PORT, SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPSP), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGSSP),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGTSP), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGDDR),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGUSB3A), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGUSB3B),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPCIEA), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPCIEB),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGM0));
+
+GROUP_DECL(JTAG0, JTAG_PORT);
+
+FUNC_DECL_1(JTAGPSP, JTAG0);
+FUNC_DECL_1(JTAGSSP, JTAG0);
+FUNC_DECL_1(JTAGTSP, JTAG0);
+FUNC_DECL_1(JTAGDDR, JTAG0);
+FUNC_DECL_1(JTAGUSB3A, JTAG0);
+FUNC_DECL_1(JTAGUSB3B, JTAG0);
+FUNC_DECL_1(JTAGPCIEA, JTAG0);
+FUNC_DECL_1(JTAGPCIEB, JTAG0);
+FUNC_DECL_1(JTAGM0, JTAG0);
+
+/* PCIe Reset Control */
+SIG_EXPR_LIST_DECL_SESG(PCIERC0_PERST, PCIERC0PERST, PCIERC0PERST, SIG_DESC_SET(SCU200, 21));
+PIN_DECL_(PCIERC0_PERST, SIG_EXPR_LIST_PTR(PCIERC0_PERST, PCIERC0PERST));
+FUNC_GROUP_DECL(PCIERC0PERST, PCIERC0_PERST);
+
+SIG_EXPR_LIST_DECL_SESG(PCIERC1_PERST, PCIERC1PERST, PCIERC1PERST, SIG_DESC_SET(SCU200, 19));
+PIN_DECL_(PCIERC1_PERST, SIG_EXPR_LIST_PTR(PCIERC1_PERST, PCIERC1PERST));
+FUNC_GROUP_DECL(PCIERC1PERST, PCIERC1_PERST);
+
+#define PORTA_MODE_HPD0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 0, 0 }
+#define PORTA_MODE_D0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 1, 0 }
+#define PORTA_MODE_H_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 2, 0 }
+#define PORTA_MODE_HP_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AHPD0, USB2AH, USB2AHPD0, PORTA_MODE_HPD0_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AH, USB2AHAP, USB2AH, PORTA_MODE_H_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AHP, USB2AHAP, USB2AHP, PORTA_MODE_HP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AD0, USB2AHAP, USB2AD0, PORTA_MODE_D0_DESC);
+PIN_DECL_(PORTA_MODE, SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AHPD0),
+ SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AH), SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AHP),
+ SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AD0));
+
+#define PORTA_U2_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 0, 0 }
+#define PORTA_U2_D1_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 1, 0 }
+#define PORTA_U2_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 2, 0 }
+#define PORTA_U2_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHD1, USB2A, USB2AXHD1, PORTA_U2_XHD_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHPD1, USB2A, USB2AXHPD1, PORTA_U2_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXH, USB2AAP, USB2AXH, PORTA_U2_XH_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHP, USB2AAP, USB2AXHP, PORTA_U2_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXH2B, USB2ABP, USB2AXH2B, PORTA_U2_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHP2B, USB2ABP, USB2AXHP2B, PORTA_U2_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AD1, USB2ADAP, USB2AD1, PORTA_U2_D1_DESC);
+PIN_DECL_(PORTA_U2, SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHD1), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHPD1),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXH), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHP),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXH2B), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHP2B),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AD1));
+
+#define PORTB_MODE_HPD0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 0, 0 }
+#define PORTB_MODE_D0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 1, 0 }
+#define PORTB_MODE_H_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 2, 0 }
+#define PORTB_MODE_HP_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BHPD0, USB2BH, USB2BHPD0, PORTB_MODE_HPD0_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BH, USB2BHBP, USB2BH, PORTB_MODE_H_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BHP, USB2BHBP, USB2BHP, PORTB_MODE_HP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BD0, USB2BHBP, USB2BD0, PORTB_MODE_D0_DESC);
+PIN_DECL_(PORTB_MODE, SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BHPD0),
+ SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BH), SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BHP),
+ SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BD0));
+
+#define PORTB_U2_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 0, 0 }
+#define PORTB_U2_D1_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 1, 0 }
+#define PORTB_U2_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 2, 0 }
+#define PORTB_U2_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHD1, USB2B, USB2BXHD1, PORTB_U2_XHD_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHPD1, USB2B, USB2BXHPD1, PORTB_U2_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXH, USB2BBP, USB2BXH, PORTB_U2_XH_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHP, USB2BBP, USB2BXHP, PORTB_U2_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXH2A, USB2BAP, USB2BXH2A, PORTB_U2_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHP2A, USB2BAP, USB2BXHP2A, PORTB_U2_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BD1, USB2BDBP, USB2BD1, PORTB_U2_D1_DESC);
+PIN_DECL_(PORTB_U2, SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHD1), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHPD1),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXH), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHP),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXH2A), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHP2A),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BD1));
+/*
+ * USB2 virtual PHY pins.
+ *
+ * PORTA_U2_PHY and PORTB_U2_PHY are logical endpoints, not package pins.
+ * They alias existing USB2 expressions so pin groups can model direct and
+ * cross-coupled routing for host and mode paths.
+ *
+ * - USB2AAP/USB2ADAP/USB2AHAP: use PORTA_U2_PHY
+ * - USB2ABP : use PORTB_U2_PHY
+ * - USB2BBP/USB2BDBP/USB2BHBP: use PORTB_U2_PHY
+ * - USB2BAP : use PORTA_U2_PHY
+ *
+ * They do not have any registers to configure this behaviour; the goal is
+ * simply for the driver to prevent conflicting selections. For example,
+ * selecting group USB2ABP and USB2BBP at the same time should not be
+ * allowed.
+ */
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AXH, USB2AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AXHP, USB2AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2BXH2A, USB2BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2BXHP2A, USB2BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AD1, USB2ADAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AH, USB2AHAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AHP, USB2AHAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AD0, USB2AHAP);
+PIN_DECL_(PORTA_U2_PHY, SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AXH),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AXHP), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2BXH2A),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2BXHP2A), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AD1),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AH), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AHP),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AD0));
+
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2AXH2B, USB2ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2AXHP2B, USB2ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BXH, USB2BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BXHP, USB2BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BD1, USB2BDBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BH, USB2BHBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BHP, USB2BHBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BD0, USB2BHBP);
+PIN_DECL_(PORTB_U2_PHY, SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2AXH2B),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2AXHP2B), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BXH),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BXHP), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BD1),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BH), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BHP),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BD0));
+
+GROUP_DECL(USB2A, PORTA_U2);
+GROUP_DECL(USB2AAP, PORTA_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2ABP, PORTA_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2ADAP, PORTA_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2AH, PORTA_MODE);
+GROUP_DECL(USB2AHAP, PORTA_MODE, PORTA_U2_PHY);
+
+FUNC_DECL_1(USB2AXHD1, USB2A);
+FUNC_DECL_1(USB2AXHPD1, USB2A);
+FUNC_DECL_1(USB2AXH, USB2AAP);
+FUNC_DECL_1(USB2AXHP, USB2AAP);
+FUNC_DECL_1(USB2AXH2B, USB2ABP);
+FUNC_DECL_1(USB2AXHP2B, USB2ABP);
+FUNC_DECL_1(USB2AD1, USB2ADAP);
+FUNC_DECL_1(USB2AHPD0, USB2AH);
+FUNC_DECL_1(USB2AH, USB2AHAP);
+FUNC_DECL_1(USB2AHP, USB2AHAP);
+FUNC_DECL_1(USB2AD0, USB2AHAP);
+
+GROUP_DECL(USB2B, PORTB_U2);
+GROUP_DECL(USB2BBP, PORTB_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2BAP, PORTB_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2BDBP, PORTB_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2BH, PORTB_MODE);
+GROUP_DECL(USB2BHBP, PORTB_MODE, PORTB_U2_PHY);
+
+FUNC_DECL_1(USB2BXHD1, USB2B);
+FUNC_DECL_1(USB2BXHPD1, USB2B);
+FUNC_DECL_1(USB2BXH, USB2BBP);
+FUNC_DECL_1(USB2BXHP, USB2BBP);
+FUNC_DECL_1(USB2BXH2A, USB2BAP);
+FUNC_DECL_1(USB2BXHP2A, USB2BAP);
+FUNC_DECL_1(USB2BD1, USB2BDBP);
+FUNC_DECL_1(USB2BHPD0, USB2BH);
+FUNC_DECL_1(USB2BH, USB2BHBP);
+FUNC_DECL_1(USB2BHP, USB2BHBP);
+FUNC_DECL_1(USB2BD0, USB2BHBP);
+
+#define PORTA_U3_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 0, 0 }
+#define PORTA_U3_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 2, 0 }
+#define PORTA_U3_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHD, USB3A, USB3AXHD, PORTA_U3_XHD_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHPD, USB3A, USB3AXHPD, PORTA_U3_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXH, USB3AAP, USB3AXH, PORTA_U3_XH_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHP, USB3AAP, USB3AXHP, PORTA_U3_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXH2B, USB3ABP, USB3AXH2B, PORTA_U3_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHP2B, USB3ABP, USB3AXHP2B, PORTA_U3_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+PIN_DECL_(PORTA_U3, SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHD), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHPD),
+ SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXH), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHP),
+ SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXH2B), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHP2B));
+
+#define PORTB_U3_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 0, 0 }
+#define PORTB_U3_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 2, 0 }
+#define PORTB_U3_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHD, USB3B, USB3BXHD, PORTB_U3_XHD_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHPD, USB3B, USB3BXHPD, PORTB_U3_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXH, USB3BBP, USB3BXH, PORTB_U3_XH_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHP, USB3BBP, USB3BXHP, PORTB_U3_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXH2A, USB3BAP, USB3BXH2A, PORTB_U3_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHP2A, USB3BAP, USB3BXHP2A, PORTB_U3_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+PIN_DECL_(PORTB_U3, SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHD), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHPD),
+ SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXH), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHP),
+ SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXH2A), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHP2A));
+
+/*
+ * USB3 virtual PHY pins.
+ *
+ * PORTA_U3_PHY and PORTB_U3_PHY are logical endpoints, not package pins.
+ * They alias existing USB3 expressions so pin groups can model both direct and
+ * cross-coupled routing to PHY A/B.
+ *
+ * - USB3AAP: PORTA_U3 + PORTA_U3_PHY (A -> PHY A)
+ * - USB3ABP: PORTA_U3 + PORTB_U3_PHY (A -> PHY B)
+ * - USB3BBP: PORTB_U3 + PORTB_U3_PHY (B -> PHY B)
+ * - USB3BAP: PORTB_U3 + PORTA_U3_PHY (B -> PHY A)
+ *
+ * They do not have any registers to configure this behavior; the goal is
+ * simply for the driver to prevent conflicting selections. For example,
+ * selecting group USB3ABP and USB3BBP at the same time should not be
+ * allowed.
+ */
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3AXH, USB3AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3AXHP, USB3AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3BXH2A, USB3BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3BXHP2A, USB3BAP);
+PIN_DECL_(PORTA_U3_PHY, SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3AXH),
+ SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3AXHP), SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3BXH2A),
+ SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3BXHP2A));
+
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3AXH2B, USB3ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3AXHP2B, USB3ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3BXH, USB3BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3BXHP, USB3BBP);
+PIN_DECL_(PORTB_U3_PHY, SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3AXH2B),
+ SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3AXHP2B), SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3BXH),
+ SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3BXHP));
+
+/* USB3A xHCI to vHUB */
+GROUP_DECL(USB3A, PORTA_U3);
+/* USB3A xHCI to USB3A PHY */
+GROUP_DECL(USB3AAP, PORTA_U3, PORTA_U3_PHY);
+/* USB3A xHCI to USB3B PHY */
+GROUP_DECL(USB3ABP, PORTA_U3, PORTB_U3_PHY);
+
+FUNC_DECL_1(USB3AXHD, USB3A);
+FUNC_DECL_1(USB3AXHPD, USB3A);
+FUNC_DECL_1(USB3AXH, USB3AAP);
+FUNC_DECL_1(USB3AXHP, USB3AAP);
+FUNC_DECL_1(USB3AXH2B, USB3ABP);
+FUNC_DECL_1(USB3AXHP2B, USB3ABP);
+
+/* USB3B xHCI to vHUB */
+GROUP_DECL(USB3B, PORTB_U3);
+/* USB3B xHCI to USB3A PHY */
+GROUP_DECL(USB3BAP, PORTB_U3, PORTA_U3_PHY);
+/* USB3B xHCI to USB3B PHY */
+GROUP_DECL(USB3BBP, PORTB_U3, PORTB_U3_PHY);
+
+FUNC_DECL_1(USB3BXHD, USB3B);
+FUNC_DECL_1(USB3BXHPD, USB3B);
+FUNC_DECL_1(USB3BXH, USB3BBP);
+FUNC_DECL_1(USB3BXHP, USB3BBP);
+FUNC_DECL_1(USB3BXH2A, USB3BAP);
+FUNC_DECL_1(USB3BXHP2A, USB3BAP);
+
+static const struct pinctrl_pin_desc aspeed_g7_soc0_pins[] = {
+ ASPEED_PINCTRL_PIN(AC14),
+ ASPEED_PINCTRL_PIN(AE15),
+ ASPEED_PINCTRL_PIN(AD14),
+ ASPEED_PINCTRL_PIN(AE14),
+ ASPEED_PINCTRL_PIN(AF14),
+ ASPEED_PINCTRL_PIN(AB13),
+ ASPEED_PINCTRL_PIN(AB14),
+ ASPEED_PINCTRL_PIN(AF15),
+ ASPEED_PINCTRL_PIN(AF13),
+ ASPEED_PINCTRL_PIN(AC13),
+ ASPEED_PINCTRL_PIN(AD13),
+ ASPEED_PINCTRL_PIN(AE13),
+ ASPEED_PINCTRL_PIN(JTAG_PORT),
+ ASPEED_PINCTRL_PIN(PCIERC0_PERST),
+ ASPEED_PINCTRL_PIN(PCIERC1_PERST),
+ ASPEED_PINCTRL_PIN(PORTA_MODE),
+ ASPEED_PINCTRL_PIN(PORTA_U2),
+ ASPEED_PINCTRL_PIN(PORTA_U3),
+ ASPEED_PINCTRL_PIN(PORTA_U2_PHY),
+ ASPEED_PINCTRL_PIN(PORTA_U3_PHY),
+ ASPEED_PINCTRL_PIN(PORTB_MODE),
+ ASPEED_PINCTRL_PIN(PORTB_U2),
+ ASPEED_PINCTRL_PIN(PORTB_U3),
+ ASPEED_PINCTRL_PIN(PORTB_U2_PHY),
+ ASPEED_PINCTRL_PIN(PORTB_U3_PHY),
+};
+
+static const struct aspeed_pin_group aspeed_g7_soc0_groups[] = {
+ ASPEED_PINCTRL_GROUP(EMMCCDN),
+ ASPEED_PINCTRL_GROUP(EMMCG1),
+ ASPEED_PINCTRL_GROUP(EMMCG4),
+ ASPEED_PINCTRL_GROUP(EMMCG8),
+ ASPEED_PINCTRL_GROUP(EMMCWPN),
+ ASPEED_PINCTRL_GROUP(TSPRSTN),
+ ASPEED_PINCTRL_GROUP(UFSCLKI),
+ ASPEED_PINCTRL_GROUP(VB0),
+ ASPEED_PINCTRL_GROUP(VB1),
+ ASPEED_PINCTRL_GROUP(VGADDC),
+ /* JTAG groups */
+ ASPEED_PINCTRL_GROUP(JTAG0),
+ /* PCIE RC groups */
+ ASPEED_PINCTRL_GROUP(PCIERC0PERST),
+ ASPEED_PINCTRL_GROUP(PCIERC1PERST),
+ /* USB3A groups */
+ ASPEED_PINCTRL_GROUP(USB3A),
+ ASPEED_PINCTRL_GROUP(USB3AAP),
+ ASPEED_PINCTRL_GROUP(USB3ABP),
+ /* USB3B groups */
+ ASPEED_PINCTRL_GROUP(USB3B),
+ ASPEED_PINCTRL_GROUP(USB3BAP),
+ ASPEED_PINCTRL_GROUP(USB3BBP),
+ /* USB2A groups */
+ ASPEED_PINCTRL_GROUP(USB2A),
+ ASPEED_PINCTRL_GROUP(USB2AAP),
+ ASPEED_PINCTRL_GROUP(USB2ABP),
+ ASPEED_PINCTRL_GROUP(USB2ADAP),
+ ASPEED_PINCTRL_GROUP(USB2AH),
+ ASPEED_PINCTRL_GROUP(USB2AHAP),
+ /* USB2B groups */
+ ASPEED_PINCTRL_GROUP(USB2B),
+ ASPEED_PINCTRL_GROUP(USB2BAP),
+ ASPEED_PINCTRL_GROUP(USB2BBP),
+ ASPEED_PINCTRL_GROUP(USB2BDBP),
+ ASPEED_PINCTRL_GROUP(USB2BH),
+ ASPEED_PINCTRL_GROUP(USB2BHBP),
+};
+
+static const struct aspeed_pin_function aspeed_g7_soc0_functions[] = {
+ ASPEED_PINCTRL_FUNC(EMMC),
+ ASPEED_PINCTRL_FUNC(TSPRSTN),
+ ASPEED_PINCTRL_FUNC(UFSCLKI),
+ ASPEED_PINCTRL_FUNC(VB),
+ ASPEED_PINCTRL_FUNC(VGADDC),
+ /* JTAG functions */
+ ASPEED_PINCTRL_FUNC(JTAGDDR),
+ ASPEED_PINCTRL_FUNC(JTAGM0),
+ ASPEED_PINCTRL_FUNC(JTAGPCIEA),
+ ASPEED_PINCTRL_FUNC(JTAGPCIEB),
+ ASPEED_PINCTRL_FUNC(JTAGPSP),
+ ASPEED_PINCTRL_FUNC(JTAGSSP),
+ ASPEED_PINCTRL_FUNC(JTAGTSP),
+ ASPEED_PINCTRL_FUNC(JTAGUSB3A),
+ ASPEED_PINCTRL_FUNC(JTAGUSB3B),
+ /* PCIE RC functions */
+ ASPEED_PINCTRL_FUNC(PCIERC0PERST),
+ ASPEED_PINCTRL_FUNC(PCIERC1PERST),
+ /* USB3A functions */
+ ASPEED_PINCTRL_FUNC(USB3AXH),
+ ASPEED_PINCTRL_FUNC(USB3AXH2B),
+ ASPEED_PINCTRL_FUNC(USB3AXHD),
+ ASPEED_PINCTRL_FUNC(USB3AXHP),
+ ASPEED_PINCTRL_FUNC(USB3AXHP2B),
+ ASPEED_PINCTRL_FUNC(USB3AXHPD),
+ /* USB3B functions */
+ ASPEED_PINCTRL_FUNC(USB3BXH),
+ ASPEED_PINCTRL_FUNC(USB3BXH2A),
+ ASPEED_PINCTRL_FUNC(USB3BXHD),
+ ASPEED_PINCTRL_FUNC(USB3BXHP),
+ ASPEED_PINCTRL_FUNC(USB3BXHP2A),
+ ASPEED_PINCTRL_FUNC(USB3BXHPD),
+ /* USB2A functions */
+ ASPEED_PINCTRL_FUNC(USB2AD0),
+ ASPEED_PINCTRL_FUNC(USB2AD1),
+ ASPEED_PINCTRL_FUNC(USB2AH),
+ ASPEED_PINCTRL_FUNC(USB2AHP),
+ ASPEED_PINCTRL_FUNC(USB2AHPD0),
+ ASPEED_PINCTRL_FUNC(USB2AXH),
+ ASPEED_PINCTRL_FUNC(USB2AXH2B),
+ ASPEED_PINCTRL_FUNC(USB2AXHD1),
+ ASPEED_PINCTRL_FUNC(USB2AXHP),
+ ASPEED_PINCTRL_FUNC(USB2AXHP2B),
+ ASPEED_PINCTRL_FUNC(USB2AXHPD1),
+ /* USB2B functions */
+ ASPEED_PINCTRL_FUNC(USB2BD0),
+ ASPEED_PINCTRL_FUNC(USB2BD1),
+ ASPEED_PINCTRL_FUNC(USB2BH),
+ ASPEED_PINCTRL_FUNC(USB2BHP),
+ ASPEED_PINCTRL_FUNC(USB2BHPD0),
+ ASPEED_PINCTRL_FUNC(USB2BXH),
+ ASPEED_PINCTRL_FUNC(USB2BXH2A),
+ ASPEED_PINCTRL_FUNC(USB2BXHD1),
+ ASPEED_PINCTRL_FUNC(USB2BXHP),
+ ASPEED_PINCTRL_FUNC(USB2BXHP2A),
+ ASPEED_PINCTRL_FUNC(USB2BXHPD1),
+};
+
+static const struct pinmux_ops aspeed_g7_soc0_pinmux_ops = {
+ .get_functions_count = aspeed_pinmux_get_fn_count,
+ .get_function_name = aspeed_pinmux_get_fn_name,
+ .get_function_groups = aspeed_pinmux_get_fn_groups,
+ .set_mux = aspeed_pinmux_set_mux,
+ .gpio_request_enable = aspeed_gpio_request_enable,
+ .strict = true,
+};
+
+static const struct pinctrl_ops aspeed_g7_soc0_pinctrl_ops = {
+ .get_groups_count = aspeed_pinctrl_get_groups_count,
+ .get_group_name = aspeed_pinctrl_get_group_name,
+ .get_group_pins = aspeed_pinctrl_get_group_pins,
+ .pin_dbg_show = aspeed_pinctrl_pin_dbg_show,
+ .dt_node_to_map = pinconf_generic_dt_node_to_map_all,
+ .dt_free_map = pinctrl_utils_free_map,
+};
+
+static const struct pinconf_ops aspeed_g7_soc0_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = aspeed_pin_config_get,
+ .pin_config_set = aspeed_pin_config_set,
+ .pin_config_group_get = aspeed_pin_config_group_get,
+ .pin_config_group_set = aspeed_pin_config_group_set,
+};
+
+/* pinctrl_desc */
+static const struct pinctrl_desc aspeed_g7_soc0_pinctrl_desc = {
+ .name = "aspeed-g7-soc0-pinctrl",
+ .pins = aspeed_g7_soc0_pins,
+ .npins = ARRAY_SIZE(aspeed_g7_soc0_pins),
+ .pctlops = &aspeed_g7_soc0_pinctrl_ops,
+ .pmxops = &aspeed_g7_soc0_pinmux_ops,
+ .confops = &aspeed_g7_soc0_pinconf_ops,
+};
+
+static const struct aspeed_pin_config aspeed_g7_soc0_configs[] = {
+ /* GPIO18A */
+ { PIN_CONFIG_DRIVE_STRENGTH, { AC14, AC14 }, SCU480, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AC14, AC14 }, SCU480, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AC14, AC14 }, SCU480, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AC14, AC14 }, SCU480, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE15, AE15 }, SCU484, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE15, AE15 }, SCU484, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE15, AE15 }, SCU484, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE15, AE15 }, SCU484, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AD14, AD14 }, SCU488, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AD14, AD14 }, SCU488, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AD14, AD14 }, SCU488, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AD14, AD14 }, SCU488, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE14, AE14 }, SCU48C, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE14, AE14 }, SCU48C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE14, AE14 }, SCU48C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE14, AE14 }, SCU48C, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF14, AF14 }, SCU490, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF14, AF14 }, SCU490, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF14, AF14 }, SCU490, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF14, AF14 }, SCU490, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AB13, AB13 }, SCU494, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AB13, AB13 }, SCU494, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AB13, AB13 }, SCU494, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AB13, AB13 }, SCU494, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AB14, AB14 }, SCU498, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AB14, AB14 }, SCU498, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AB14, AB14 }, SCU498, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AB14, AB14 }, SCU498, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF15, AF15 }, SCU49C, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF15, AF15 }, SCU49C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF15, AF15 }, SCU49C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF15, AF15 }, SCU49C, BIT(5) },
+ /* GPIO18B */
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF13, AF13 }, SCU4A0, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF13, AF13 }, SCU4A0, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF13, AF13 }, SCU4A0, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF13, AF13 }, SCU4A0, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AC13, AC13 }, SCU4A4, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AC13, AC13 }, SCU4A4, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AC13, AC13 }, SCU4A4, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AC13, AC13 }, SCU4A4, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AD13, AD13 }, SCU4A8, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AD13, AD13 }, SCU4A8, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AD13, AD13 }, SCU4A8, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AD13, AD13 }, SCU4A8, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE13, AE13 }, SCU4AC, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE13, AE13 }, SCU4AC, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE13, AE13 }, SCU4AC, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE13, AE13 }, SCU4AC, BIT(5) },
+};
+
+static const struct aspeed_pin_config_map aspeed_g7_soc0_pin_config_map[] = {
+ { PIN_CONFIG_BIAS_PULL_DOWN, -1, 2, GENMASK(1, 0) },
+ { PIN_CONFIG_BIAS_PULL_UP, -1, 3, GENMASK(1, 0) },
+ { PIN_CONFIG_BIAS_DISABLE, -1, 0, BIT_MASK(0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 3, 0, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 6, 1, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 8, 2, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 11, 3, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 16, 4, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 18, 5, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 20, 6, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 23, 7, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 30, 8, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 32, 9, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 33, 10, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 35, 11, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 37, 12, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 38, 13, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 39, 14, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 41, 15, GENMASK(3, 0) },
+
+};
+
+static int aspeed_g7_soc0_sig_expr_set(struct aspeed_pinmux_data *ctx,
+ const struct aspeed_sig_expr *expr, bool enable)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < expr->ndescs; i++) {
+ const struct aspeed_sig_desc *desc = &expr->descs[i];
+ u32 pattern = enable ? desc->enable : desc->disable;
+ u32 val = (pattern << __ffs(desc->mask));
+
+ if (!ctx->maps[desc->ip])
+ return -ENODEV;
+
+ WARN_ON_ONCE(desc->ip != ASPEED_IP_SCU);
+
+ ret = regmap_update_bits(ctx->maps[desc->ip], desc->reg,
+ desc->mask, val);
+ if (ret)
+ return ret;
+ }
+
+ ret = aspeed_sig_expr_eval(ctx, expr, enable);
+ if (ret < 0)
+ return ret;
+
+ return ret ? 0 : -EPERM;
+}
+
+static const struct aspeed_pinmux_ops aspeed_g7_soc0_ops = {
+ .set = aspeed_g7_soc0_sig_expr_set,
+};
+
+static struct aspeed_pinctrl_data aspeed_g7_soc0_pinctrl_data = {
+ .pins = aspeed_g7_soc0_pins,
+ .npins = ARRAY_SIZE(aspeed_g7_soc0_pins),
+ .pinmux = {
+ .ops = &aspeed_g7_soc0_ops,
+ .groups = aspeed_g7_soc0_groups,
+ .ngroups = ARRAY_SIZE(aspeed_g7_soc0_groups),
+ .functions = aspeed_g7_soc0_functions,
+ .nfunctions = ARRAY_SIZE(aspeed_g7_soc0_functions),
+ },
+ .configs = aspeed_g7_soc0_configs,
+ .nconfigs = ARRAY_SIZE(aspeed_g7_soc0_configs),
+ .confmaps = aspeed_g7_soc0_pin_config_map,
+ .nconfmaps = ARRAY_SIZE(aspeed_g7_soc0_pin_config_map),
+};
+
+static int aspeed_g7_soc0_pinctrl_probe(struct platform_device *pdev)
+{
+ return aspeed_pinctrl_probe(pdev, &aspeed_g7_soc0_pinctrl_desc,
+ &aspeed_g7_soc0_pinctrl_data);
+}
+
+static const struct of_device_id aspeed_g7_soc0_pinctrl_match[] = {
+ { .compatible = "aspeed,ast2700-soc0-pinctrl" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, aspeed_g7_soc0_pinctrl_match);
+
+static struct platform_driver aspeed_g7_soc0_pinctrl_driver = {
+ .probe = aspeed_g7_soc0_pinctrl_probe,
+ .driver = {
+ .name = "aspeed-g7-soc0-pinctrl",
+ .of_match_table = aspeed_g7_soc0_pinctrl_match,
+ .suppress_bind_attrs = true,
+ },
+};
+
+static int __init aspeed_g7_soc0_pinctrl_init(void)
+{
+ return platform_driver_register(&aspeed_g7_soc0_pinctrl_driver);
+}
+arch_initcall(aspeed_g7_soc0_pinctrl_init);
--
2.34.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox