* Re: [PATCH net] net: airoha: Fix possible TX queue stall in airoha_qdma_tx_napi_poll()
From: Paolo Abeni @ 2026-04-16 8:44 UTC (permalink / raw)
To: Lorenzo Bianconi, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260413-airoha-txq-potential-stall-v1-1-7830363b1543@kernel.org>
On 4/13/26 10:29 AM, Lorenzo Bianconi wrote:
> Since multiple net_device TX queues can share the same hw QDMA TX queue,
> there is no guarantee we have inflight packets queued in hw belonging to a
> net_device TX queue stopped in the xmit path because hw QDMA TX queue
> can be full. In this corner case the net_device TX queue will never be
> re-activated. In order to avoid any potential net_device TX queue stall,
> we need to wake all the net_device TX queues feeding the same hw QDMA TX
> queue in airoha_qdma_tx_napi_poll routine.
>
> Fixes: 23020f0493270 ("net: airoha: Introduce ethernet support for EN7581 SoC")
> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
> ---
> drivers/net/ethernet/airoha/airoha_eth.c | 30 ++++++++++++++++++++++++++----
> 1 file changed, 26 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
> index 9e995094c32a..e7610f36b8e4 100644
> --- a/drivers/net/ethernet/airoha/airoha_eth.c
> +++ b/drivers/net/ethernet/airoha/airoha_eth.c
> @@ -855,6 +855,19 @@ static int airoha_qdma_init_rx(struct airoha_qdma *qdma)
> return 0;
> }
>
> +static void airoha_qdma_wake_tx_queues(struct airoha_qdma *qdma)
> +{
> + struct airoha_eth *eth = qdma->eth;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(eth->ports); i++) {
> + struct airoha_gdm_port *port = eth->ports[i];
> +
> + if (port && port->qdma == qdma)
> + netif_tx_wake_all_queues(port->dev);
> + }
> +}
> +
> static int airoha_qdma_tx_napi_poll(struct napi_struct *napi, int budget)
> {
> struct airoha_tx_irq_queue *irq_q;
> @@ -931,12 +944,21 @@ static int airoha_qdma_tx_napi_poll(struct napi_struct *napi, int budget)
>
> txq = netdev_get_tx_queue(skb->dev, queue);
> netdev_tx_completed_queue(txq, 1, skb->len);
> - if (netif_tx_queue_stopped(txq) &&
> - q->ndesc - q->queued >= q->free_thr)
> - netif_tx_wake_queue(txq);
> -
> dev_kfree_skb_any(skb);
> }
> +
> + if (q->ndesc - q->queued == q->free_thr) {
Sashiko says:
---
Can this exact equality check cause a permanent TX queue stall?
The previous logic checked if the free space was greater than or equal
to q->free_thr. If the xmit path stops the queue because the free space
drops to exactly q->free_thr, the hardware queue will have exactly
q->free_thr free slots.
When the NAPI poll routine subsequently reaps a completed descriptor,
q->queued is decremented, increasing the free space to q->free_thr + 1.
Since the free space is no longer exactly equal to the threshold, this
condition evaluates to false.
As NAPI continues to reap more descriptors, the free space strictly
increases, meaning the exact equality check will never evaluate to true
and the netdev TX queue will remain permanently stalled.
---
Please, try to triage sashiko comments proactively. Especially on NIC
drivers, validating the AI statements is extremely cumbersome for the
maintainers.
Thanks,
Paolo
^ permalink raw reply
* [PATCH] serial: mxs-auart: Compare the return value of gpiod_get_direction against GPIO_LINE_DIRECTION_IN
From: Nikola Z. Ivanov @ 2026-04-16 8:32 UTC (permalink / raw)
To: gregkh, jirislaby, Frank.Li, s.hauer, kernel, festevam
Cc: linux-kernel, linux-serial, imx, linux-arm-kernel,
Nikola Z. Ivanov
The GPIO_LINE_DIRECTION_* definitions have just recently been exposed to
gpio consumers.h by breaking them out in a separate defs.h file.
Use this to validate the gpio direction instead of the hard-coded literal.
Signed-off-by: Nikola Z. Ivanov <zlatistiv@gmail.com>
---
drivers/tty/serial/mxs-auart.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/tty/serial/mxs-auart.c b/drivers/tty/serial/mxs-auart.c
index cc65c9fb6446..6c6df4d5c21f 100644
--- a/drivers/tty/serial/mxs-auart.c
+++ b/drivers/tty/serial/mxs-auart.c
@@ -1519,7 +1519,7 @@ static int mxs_auart_init_gpios(struct mxs_auart_port *s, struct device *dev)
for (i = 0; i < UART_GPIO_MAX; i++) {
gpiod = mctrl_gpio_to_gpiod(s->gpios, i);
- if (gpiod && (gpiod_get_direction(gpiod) == 1))
+ if (gpiod && (gpiod_get_direction(gpiod) == GPIO_LINE_DIRECTION_IN))
s->gpio_irq[i] = gpiod_to_irq(gpiod);
else
s->gpio_irq[i] = -EINVAL;
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v2 1/8] dt-bindings: mfd: khadas: Add new compatible for Khadas VIM4 MCU
From: Ronald Claveau @ 2026-04-16 8:25 UTC (permalink / raw)
To: Rob Herring
Cc: Neil Armstrong, Lee Jones, Krzysztof Kozlowski, Conor Dooley,
Andi Shyti, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Beniamino Galvani, Rafael J. Wysocki, Daniel Lezcano, Zhang Rui,
Lukasz Luba, Liam Girdwood, Mark Brown, linux-amlogic, devicetree,
linux-kernel, linux-i2c, linux-arm-kernel, linux-pm
In-Reply-To: <20260415214815.GA602572-robh@kernel.org>
On 4/15/26 11:48 PM, Rob Herring wrote:
> On Fri, Apr 03, 2026 at 06:08:34PM +0200, Ronald Claveau wrote:
>> The Khadas VIM4 MCU register is slightly different
>> from previous boards' MCU.
>> This board also features a switchable power source for its fan.
>>
>> Signed-off-by: Ronald Claveau <linux-kernel-dev@aliel.fr>
>> ---
>> Documentation/devicetree/bindings/mfd/khadas,mcu.yaml | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/mfd/khadas,mcu.yaml b/Documentation/devicetree/bindings/mfd/khadas,mcu.yaml
>> index 084960fd5a1fd..67769ef5d58b1 100644
>> --- a/Documentation/devicetree/bindings/mfd/khadas,mcu.yaml
>> +++ b/Documentation/devicetree/bindings/mfd/khadas,mcu.yaml
>> @@ -18,6 +18,7 @@ properties:
>> compatible:
>> enum:
>> - khadas,mcu # MCU revision is discoverable
>
> The revision is no longer discoverable as was claimed?
>
The firmware revision is still discoverable, and via the same register,
but the VIM4 MCU has a different register layout (eg: no DEVICE_NO
register). The new compatible is needed to describe a different MCU
variant, not a different revision of the same MCU.
I will remove the comment as it is confusing with new boards.
>> + - khadas,vim4-mcu
>>
>> "#cooling-cells": # Only needed for boards having FAN control feature
>> const: 2
>> @@ -25,6 +26,10 @@ properties:
>> reg:
>> maxItems: 1
>>
>> + fan-supply:
>> + description: Phandle to the regulator that powers the fan.
>> + $ref: /schemas/types.yaml#/definitions/phandle
>> +
>> required:
>> - compatible
>> - reg
>>
>> --
>> 2.49.0
>>
--
Best regards,
Ronald
^ permalink raw reply
* Re: [PATCH v4 0/4] Introduce Allwinner H616 PWM controller
From: Richard GENOUD @ 2026-04-16 8:22 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Uwe Kleine-König, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Chen-Yu Tsai, Jernej Skrabec, Samuel Holland,
Philipp Zabel, Thomas Petazzoni, John Stultz, Joao Schim,
linux-pwm, devicetree, linux-arm-kernel, linux-sunxi,
linux-kernel
In-Reply-To: <adfe2z2YeBxm_6oR@shepard>
Le 09/04/2026 à 19:16, Paul Kocialkowski a écrit :
> Hi Richard,
>
> On Thu 05 Mar 26, 10:19, Richard Genoud wrote:
>> Allwinner H616 PWM controller is quite different from the A10 one.
>
> As I've mentionned before, this PWM controller is not specific to the H616
> but also appears in other chips, so the name of the driver and registers
> should not mention H616.
>
> After further investigation, I can see multiple versions of this new PWM IP
> being used in different chips, starting with the R40/V40 (sun8iw11) in 2016.
>
> The latest downstream BSP driver has a list of the different generations:
> https://github.com/radxa/allwinner-bsp/blob/cubie-aiot-v1.4.6/drivers/pwm/pwm-sunxi.c#L1901
>
> We have a first generation called v100/v101 for the following chips:
> H616, R328 and R40. A second generation is called v200 and brings slight
> register layout differences for A133, D1/T113-S3 and V851. Subsequent
> iterations (v201-5) are used in more recent chips like A527 and A733 and
> seem register-compatible with v200 (from a quick look).
>
> So what I suggest here is to rename the driver "sun8i-pwm" and eventually add
> a list of generations to the driver and different registers when needed, with
> an appropriate suffix in their name.
>
> But since you're currently only dealing with H616, this work can be done later
> when introducing support for more chips.
ok, I'm fine with that :)
>
>> It can drive 6 PWM channels, and like for the A10, each channel has a
>> bypass that permits to output a clock, bypassing the PWM logic, when
>> enabled.
>>
>> But, the channels are paired 2 by 2, sharing a first set of
>> MUX/prescaler/gate.
>> Then, for each channel, there's another prescaler (that will be bypassed
>> if the bypass is enabled for this channel).
>>
>> It looks like that:
>> _____ ______ ________
>> OSC24M --->| | | | | |
>> APB1 ----->| Mux |--->| Gate |--->| /div_m |-----> PWM_clock_src_xy
>> |_____| |______| |________|
>> ________
>> | |
>> +->| /div_k |---> PWM_clock_x
>> | |________|
>> | ______
>> | | |
>> +-->| Gate |----> PWM_bypass_clock_x
>> | |______|
>> PWM_clock_src_xy -----+ ________
>> | | |
>> +->| /div_k |---> PWM_clock_y
>> | |________|
>> | ______
>> | | |
>> +-->| Gate |----> PWM_bypass_clock_y
>> |______|
>>
>> Where xy can be 0/1, 2/3, 4/5
>>
>> PWM_clock_x/y serve for the PWM purpose.
>> PWM_bypass_clock_x/y serve for the clock-provider purpose.
>> The common clock framework has been used to manage those clocks.
>>
>> This PWM driver serves as a clock-provider for PWM_bypass_clocks.
>> This is needed for example by the embedded AC300 PHY which clock comes
>> from PMW5 pin (PB12).
>>
>> Usually, to get a clock from a PWM driver, we use the pwm-clock driver
>> so that the PWM driver doesn't need to be a clk-provider itself.
>> While this works in most cases, here it just doesn't.
>> That's because the pwm-clock request a period from the PWM driver,
>> without any clue that it actually wants a clock at a specific frequency,
>> and not a PWM signal with duty cycle capability.
>
> From what I understand the pwm-clock driver will either assume a fixed rate
> set in device-tree or deduce the rate from the pwm period. In any case it will
> check that the pwm period (which it cannot change) is the same as the requested
> clock period.
>
> So I agree that pwm-clock is unable to change the clock rate at runtime and will
> just use whatever frequency the pwm is running at (which is typically set
> in the device-tree consumer property).
>
>> So, the PWM driver doesn't know if it can use the bypass or not, it
>> doesn't even have the real accurate frequency information (23809524 Hz
>> instead of 24MHz) because PWM drivers only deal with periods.
>
> I agree that the driver needs to register as a proper clock provider in
> addition to pwm. But what happens if the same PWM clock is requested both from
> the clk side and the pwm side?
The first to request it is the winner :)
The other ones will receive a -EBUSY
In h616_pwm_request() and h616_pwm_of_clk_get(), the channel mode is
checked, and if it's free to use, it's set as either PWM or CLK mode so
that it can't be requested a second time.
>
>> With pwm-clock, we loose a precious information along the way (that we
>> actually want a clock and not a PWM signal).
>> That's ok with simple PWM drivers that don't have multiple input clocks,
>> but in this case, without this information, we can't know for sure which
>> clock to use.
>> And here, for instance, if we ask for a 24MHz clock, pwm-clock will
>> requests 42ns (assigned-clocks doesn't help for that matter). The logic
>> is to select the highest clock (100MHz) with no prescaler and a duty
>> cycle value of 2/4 => we have 25MHz instead of 24MHz.
>> And that's a perfectly fine choice for a PMW, because we still can
>> change the duty cycle in the range [0-4]/4.
>> But obviously for a clock, we don't care about the duty cycle, but more
>> about the clock accuracy.
>>
>> And actually, this PWM is really a PWM AND a real clock when the bypass
>> is set.
>
> Make sense to me.
>
>> This series is based onto v6.19-rc4
>>
>> NB: checkpatch is not happy with patch 2, but it's a false positive.
>> It doesn't detect that PWM_XY_SRC_MUX/GATE/DIV are structures, but as
>> it's more readable like that, I prefer keeping it that way.
>>
>> NB2: for geopolitical reasons, I didn't re-use the old series that Paul
>> was referring to.
>>
>> Changes since v3:
>> - gather Acked-by/Tested-by
>> - fix cast from pointer to integer of different size (kernel test robot
>> with arc platform)
>> - add devm_action for clk_hw_unregister_composite as suggested by Philipp
>> - remove now unused pwm_remove as suggested by Philipp
>>
>> Changes since v2:
>> - use U32_MAX instead of defining UINT32_MAX
>> - add a comment on U32_MAX usage in clk_round_rate()
>> - change clk_table_div_m (use macros)
>> - fix formatting (double space, superfluous comma, extra line feed)
>> - fix the parent clock order
>> - simplify code by using scoped_guard()
>> - add missing const in to_h616_pwm_chip() and rename to
>> h616_pwm_from_chip()
>> - add/remove missing/superflous error messages
>> - rename cnt->period_ticks, duty_cnt->duty_ticks
>> - fix PWM_PERIOD_MAX
>> - add .remove() callback
>> - fix DIV_ROUND_CLOSEST_ULL->DIV_ROUND_UP_ULL
>> - add H616_ prefix
>> - protect _reg in macros
>> - switch to waveforms instead of apply/get_state
>> - shrink struct h616_pwm_channel
>> - rebase on v6.19-rc4
>>
>> Changes since v1:
>> - rebase onto v6.19-rc1
>> - add missing headers
>> - remove MODULE_ALIAS (suggested by Krzysztof)
>> - use sun4i-pwm binding instead of creating a new one (suggested by Krzysztof)
>> - retrieve the parent clocks from the devicetree
>> - switch num_parents to unsigned int
>>
>> Richard Genoud (4):
>> dt-bindings: pwm: allwinner: add h616 pwm compatible
>> pwm: sun50i: Add H616 PWM support
>> arm64: dts: allwinner: h616: add PWM controller
>> MAINTAINERS: Add entry on Allwinner H616 PWM driver
>>
>> .../bindings/pwm/allwinner,sun4i-a10-pwm.yaml | 19 +-
>> MAINTAINERS | 5 +
>> .../arm64/boot/dts/allwinner/sun50i-h616.dtsi | 47 +
>> drivers/pwm/Kconfig | 12 +
>> drivers/pwm/Makefile | 1 +
>> drivers/pwm/pwm-sun50i-h616.c | 936 ++++++++++++++++++
>> 6 files changed, 1019 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/pwm/pwm-sun50i-h616.c
>>
>>
>> base-commit: 11439c4635edd669ae435eec308f4ab8a0804808
>
Regards,
Richard
^ permalink raw reply
* Re: [PATCH net,v2 1/1] net: stmmac: Update default_an_inband before passing value to phylink_config
From: KhaiWenTan @ 2026-04-16 8:22 UTC (permalink / raw)
To: Paolo Abeni, andrew+netdev, davem, edumazet, kuba,
mcoquelin.stm32, alexandre.torgue, rmk+kernel, maxime.chevallier,
ovidiu.panait.rb, vladimir.oltean
Cc: netdev, linux-stm32, linux-arm-kernel, linux-kernel,
yoong.siang.song, hong.aun.looi, khai.wen.tan
In-Reply-To: <72d1b0b7-c8df-463e-a2d9-bf5ff04ba33c@redhat.com>
On 4/16/2026 4:12 PM, Paolo Abeni wrote:
> On 4/13/26 4:03 AM, KhaiWenTan wrote:
>> get_interfaces() will update both the plat->phy_interfaces and
>> mdio_bus_data->default_an_inband based on reading a SERDES register. As
>> get_interfaces() will be called after default_an_inband had already been
>> read, dwmac-intel regressed as a result with incorrect default_an_inband
>> value in phylink_config.
>>
>> Therefore, we moved the priv->plat->get_interfaces() to be executed first
>> before assigning mdio_bus_data->default_an_inband to
>> config->default_an_inband to ensure default_an_inband is in correct value.
>>
>> Fixes: d3836052fe09 ("net: stmmac: intel: convert speed_mode_2500() to get_interfaces()")
>> Signed-off-by: KhaiWenTan <khai.wen.tan@linux.intel.com>
> Since Jakub sent the net-next PR and forwarded the trees, this patch
> does not apply anymore. Please rebase and repost. You can retain
> Russell's reviewed-by tag.
>
> Thanks,
>
> Paolo
Thank you Paolo, will be rebasing the patch and update a v3.
^ permalink raw reply
* Re: [PATCH net,v2 1/1] net: stmmac: Update default_an_inband before passing value to phylink_config
From: Paolo Abeni @ 2026-04-16 8:12 UTC (permalink / raw)
To: KhaiWenTan, andrew+netdev, davem, edumazet, kuba, mcoquelin.stm32,
alexandre.torgue, rmk+kernel, maxime.chevallier, ovidiu.panait.rb,
vladimir.oltean
Cc: netdev, linux-stm32, linux-arm-kernel, linux-kernel,
yoong.siang.song, hong.aun.looi, khai.wen.tan
In-Reply-To: <20260413020339.68426-1-khai.wen.tan@linux.intel.com>
On 4/13/26 4:03 AM, KhaiWenTan wrote:
> get_interfaces() will update both the plat->phy_interfaces and
> mdio_bus_data->default_an_inband based on reading a SERDES register. As
> get_interfaces() will be called after default_an_inband had already been
> read, dwmac-intel regressed as a result with incorrect default_an_inband
> value in phylink_config.
>
> Therefore, we moved the priv->plat->get_interfaces() to be executed first
> before assigning mdio_bus_data->default_an_inband to
> config->default_an_inband to ensure default_an_inband is in correct value.
>
> Fixes: d3836052fe09 ("net: stmmac: intel: convert speed_mode_2500() to get_interfaces()")
> Signed-off-by: KhaiWenTan <khai.wen.tan@linux.intel.com>
Since Jakub sent the net-next PR and forwarded the trees, this patch
does not apply anymore. Please rebase and repost. You can retain
Russell's reviewed-by tag.
Thanks,
Paolo
^ permalink raw reply
* Re: [PATCH 1/6] drm/connector: report IRQ_HPD events to drm_connector_oob_hotplug_event()
From: Tomi Valkeinen @ 2026-04-16 8:10 UTC (permalink / raw)
To: Dmitry Baryshkov
Cc: dri-devel, linux-kernel, linux-usb, intel-gfx, intel-xe,
linux-amlogic, linux-arm-kernel, linux-arm-msm, freedreno,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Heikki Krogerus, Greg Kroah-Hartman, Andrzej Hajda,
Neil Armstrong, Robert Foss, Laurent Pinchart, Jonas Karlman,
Jernej Skrabec, Adrien Grassein, Jani Nikula, Rodrigo Vivi,
Joonas Lahtinen, Tvrtko Ursulin, Kevin Hilman, Jerome Brunet,
Martin Blumenstingl, Rob Clark, Dmitry Baryshkov, Abhinav Kumar,
Jessica Zhang, Sean Paul, Marijn Suijten, Bjorn Andersson,
Konrad Dybcio, Pengyu Luo, Nikita Travkin, Yongxing Mou
In-Reply-To: <20260416-hpd-irq-events-v1-1-1ab1f1cfb2b2@oss.qualcomm.com>
Hi,
On 16/04/2026 02:22, Dmitry Baryshkov wrote:
> The DisplayPort standard defines a special kind of events called IRQ.
> These events are used to notify DP Source about the events on the Sink
> side. It is extremely important for DP MST handling, where the MST
> events are reported through this IRQ.
>
> In case of the USB-C DP AltMode there is no actual HPD pulse, but the
> events are ported through the bits in the AltMode VDOs.
>
> Extend the drm_connector_oob_hotplug_event() interface and report IRQ
> events to the DisplayPort Sink drivers.
>
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> ---
> drivers/gpu/drm/drm_connector.c | 4 +++-
> drivers/usb/typec/altmodes/displayport.c | 12 ++++++++----
> include/drm/drm_connector.h | 3 ++-
> 3 files changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
> index 47dc53c4a738..5fdacbd84bd7 100644
> --- a/drivers/gpu/drm/drm_connector.c
> +++ b/drivers/gpu/drm/drm_connector.c
> @@ -3510,6 +3510,7 @@ struct drm_connector *drm_connector_find_by_fwnode(struct fwnode_handle *fwnode)
> * drm_connector_oob_hotplug_event - Report out-of-band hotplug event to connector
> * @connector_fwnode: fwnode_handle to report the event on
> * @status: hot plug detect logical state
> + * @irq_hpd: HPD pulse detected
> *
> * On some hardware a hotplug event notification may come from outside the display
> * driver / device. An example of this is some USB Type-C setups where the hardware
> @@ -3520,7 +3521,8 @@ struct drm_connector *drm_connector_find_by_fwnode(struct fwnode_handle *fwnode)
> * a drm_connector reference through calling drm_connector_find_by_fwnode().
> */
> void drm_connector_oob_hotplug_event(struct fwnode_handle *connector_fwnode,
> - enum drm_connector_status status)
> + enum drm_connector_status status,
> + bool irq_hpd)
I find the "IRQ HPD" naming always confusing, even if I'm somewhat
familiar with DP, but if someone has mainly worked on HDMI, I'm sure
it's even worse.
Can we define this a bit more precisely? Is 'irq_hpd' only for
displayport? If so, perhaps 'dp_irq_hpd' or 'displayport_irq_hpd'. I
might even call it 'dp_hpd_pulse', but maybe that's not good as the spec
talks about HPD pulse for both short and long ones (although in the
kernel doc you just write "HPD pulse")... The kernel doc could be
expanded a bit to make it clear what this flag indicates.
Tomi
^ permalink raw reply
* Re: [PATCH v4 2/4] pwm: sun50i: Add H616 PWM support
From: Richard GENOUD @ 2026-04-16 7:57 UTC (permalink / raw)
To: bigunclemax
Cc: conor+dt, devicetree, jernej.skrabec, joao, jstultz, krzk+dt,
linux-arm-kernel, linux-kernel, linux-pwm, linux-sunxi, p.zabel,
paulk, robh, samuel, thomas.petazzoni, u.kleine-koenig, wens
In-Reply-To: <20260413123920.2459916-1-bigunclemax@gmail.com>
Le 13/04/2026 à 14:39, bigunclemax@gmail.com a écrit :
> Hi Richard,
>
>> +
>> +/* PWM Capture Fall Lock Register */
>> +#define H616_PWM_CFLR(x) (0x74 + (x) * 0x20)
>> +
>> +#define H616_PWM_PAIR_IDX(chan) ((chan) >> 2)
>> +
>
> It looks like there's a typo or a mistake in the PAIR_IDX calculation.
> It should be like ((chan) >> 1).
> For example, for the 5th channel the result will be 1, but it should be 2.
Good catch!
I mainly tested with PWM1 as it's the only one easily accessible with a
scope.
I'll change that in the next iteration.
Thanks!
>
> Best regards
> Maksim
>
^ permalink raw reply
* Re: [PATCH v4 2/4] pwm: sun50i: Add H616 PWM support
From: Richard GENOUD @ 2026-04-16 7:53 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Uwe Kleine-König, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Chen-Yu Tsai, Jernej Skrabec, Samuel Holland,
Philipp Zabel, Thomas Petazzoni, John Stultz, Joao Schim,
linux-pwm, devicetree, linux-arm-kernel, linux-sunxi,
linux-kernel
In-Reply-To: <adfiPo4Jq1IRMM0h@shepard>
Hi Paul,
Le 09/04/2026 à 19:30, Paul Kocialkowski a écrit :
> Hi Richard,
>
> On Thu 05 Mar 26, 10:19, Richard Genoud wrote:
>> +/* PWM IRQ Enable Register */
>> +#define H616_PWM_IER 0x0
>
> I think it would make more sense to keep the full register names from the
> manual after the suffix and stick to them. It makes things easier when
> comparing the code with documentation or the reference implementation.
>
> So something like SUN8I_PWM_PIER here.
Ok, that make sense.
>
>> +
>> +/* PWM IRQ Status Register */
>> +#define H616_PWM_ISR 0x4
>> +
>> +/* PWM Capture IRQ Enable Register */
>> +#define H616_PWM_CIER 0x10
>> +
>> +/* PWM Capture IRQ Status Register */
>> +#define H616_PWM_CISR 0x14
>> +
>> +/* PWMCC Pairs Clock Configuration Registers */
>> +#define H616_PWM_XY_CLK_CR(pair) (0x20 + ((pair) * 0x4))
>> +#define H616_PWM_XY_CLK_CR_SRC_SHIFT 7
>> +#define H616_PWM_XY_CLK_CR_SRC_MASK 1
>> +#define H616_PWM_XY_CLK_CR_GATE_BIT 4
>> +#define H616_PWM_XY_CLK_CR_BYPASS_BIT(chan) ((chan) % 2 + 5)
>> +#define H616_PWM_XY_CLK_CR_DIV_M_SHIFT 0
>> +
>> +/* PWMCC Pairs Dead Zone Control Registers */
>> +#define H616_PWM_XY_DZ(pair) (0x30 + ((pair) * 0x4))
>> +
>> +/* PWM Enable Register */
>> +#define H616_PWM_ENR 0x40
>> +#define H616_PWM_ENABLE(x) BIT(x)
>> +
>> +/* PWM Capture Enable Register */
>> +#define H616_PWM_CER 0x44
>> +
>> +/* PWM Control Register */
>> +#define H616_PWM_CTRL_REG(chan) (0x60 + (chan) * 0x20)
>
> You're sometimes calling the register offset _REG and sometimes not.
> Both options are fine but you need to keep it consistent across the whole
> definitions. I would be enclined to not use it after using the register names
> coming from the manual as suggested above.
I see what you mean, so H616_PWM_CTRL_REG() would become SUN8I_PWM_PCR()
>
> Also you're sometimes using "chan", sometimes "ch" for the argument to the
> register macros. This is inconsistent and you might as well just use "c"
> everywhere so it doesn't take too much space.
Indeed.
>
>> +#define H616_PWM_CTRL_PRESCAL_K_SHIFT 0
>> +#define H616_PWM_CTRL_PRESCAL_K_WIDTH 8
>> +#define H616_PWM_CTRL_ACTIVE_STATE BIT(8)
>> +
>> +/* PWM Period Register */
>> +#define H616_PWM_PERIOD_REG(ch) (0x64 + (ch) * 0x20)
>> +#define H616_PWM_PERIOD_MASK GENMASK(31, 16)
>> +#define H616_PWM_DUTY_MASK GENMASK(15, 0)
>> +#define H616_PWM_REG_PERIOD(reg) (FIELD_GET(H616_PWM_PERIOD_MASK, reg) + 1)
>> +#define H616_PWM_REG_DUTY(reg) FIELD_GET(H616_PWM_DUTY_MASK, reg)
>> +#define H616_PWM_PERIOD(prd) FIELD_PREP(H616_PWM_PERIOD_MASK, (prd) - 1)
>> +#define H616_PWM_DUTY(dty) FIELD_PREP(H616_PWM_DUTY_MASK, dty)
>> +#define H616_PWM_PERIOD_MAX (FIELD_MAX(H616_PWM_PERIOD_MASK) + 1)
>
> Using REG as a prefix feels a bit confusing here. I would rather see:
> #define SUN8I_PWM_PPR(c) (0x64 + (c) * 0x20)
> #define SUN8I_PWM_PPR_PERIOD(p) FIELD_PREP(...)
> #define SUN8I_PWM_PPR_PERIOD_VALUE(r) FIELD_GET(...)
> #define SUN8I_PWN_PPR_PERIOD_MAX FIELD_MAX(...)
> #define SUN8I_PWM_PPR_DUTY(d) FIELD_PREP(...)
> #define SUN8I_PWM_PPR_DUTY_VALUE(r) FIELD_GET(...)
That's right, that would be less confusing.
>
>> +
>> +/* PWM Count Register */
>> +#define H616_PWM_CNT_REG(x) (0x68 + (x) * 0x20)
>> +
>> +/* PWM Capture Control Register */
>> +#define H616_PWM_CCR(x) (0x6c + (x) * 0x20)
>> +
>> +/* PWM Capture Rise Lock Register */
>> +#define H616_PWM_CRLR(x) (0x70 + (x) * 0x20)
>> +
>> +/* PWM Capture Fall Lock Register */
>> +#define H616_PWM_CFLR(x) (0x74 + (x) * 0x20)
>> +
>> +#define H616_PWM_PAIR_IDX(chan) ((chan) >> 2)
>> +
>> +/*
>> + * Block diagram of the PWM clock controller:
>> + *
>> + * _____ ______ ________
>> + * OSC24M --->| | | | | |
>> + * APB1 ----->| Mux |--->| Gate |--->| /div_m |-----> H616_PWM_clock_src_xy
>> + * |_____| |______| |________|
>> + * ________
>> + * | |
>> + * +->| /div_k |---> H616_PWM_clock_x
>> + * | |________|
>> + * | ______
>> + * | | |
>> + * +-->| Gate |----> H616_PWM_bypass_clock_x
>> + * | |______|
>> + * H616_PWM_clock_src_xy ----+ ________
>> + * | | |
>> + * +->| /div_k |---> H616_PWM_clock_y
>> + * | |________|
>> + * | ______
>> + * | | |
>> + * +-->| Gate |----> H616_PWM_bypass_clock_y
>> + * |______|
>> + *
>> + * NB: when the bypass is set, all the PWM logic is bypassed.
>> + * So, the duty cycle and polarity can't be modified (we just have a clock).
>> + * The bypass in PWM mode is used to achieve a 1/2 relative duty cycle with the
>> + * fastest clock.
>> + *
>> + * H616_PWM_clock_x/y serve for the PWM purpose.
>> + * H616_PWM_bypass_clock_x/y serve for the clock-provider purpose.
>> + *
>> + */
>> +
>> +/*
>> + * Table used for /div_m (diviser before obtaining H616_PWM_clock_src_xy)
>> + * It's actually CLK_DIVIDER_POWER_OF_TWO, but limited to /256
>> + */
>> +#define CLK_TABLE_DIV_M_ENTRY(i) { \
>> + .val = (i), .div = 1 << (i) \
>> +}
>> +
>> +static const struct clk_div_table clk_table_div_m[] = {
>> + CLK_TABLE_DIV_M_ENTRY(0),
>> + CLK_TABLE_DIV_M_ENTRY(1),
>> + CLK_TABLE_DIV_M_ENTRY(2),
>> + CLK_TABLE_DIV_M_ENTRY(3),
>> + CLK_TABLE_DIV_M_ENTRY(4),
>> + CLK_TABLE_DIV_M_ENTRY(5),
>> + CLK_TABLE_DIV_M_ENTRY(6),
>> + CLK_TABLE_DIV_M_ENTRY(7),
>> + CLK_TABLE_DIV_M_ENTRY(8),
>> + { /* sentinel */ }
>> +};
>> +
>> +#define H616_PWM_XY_SRC_GATE(_pair, _reg) \
>> +struct clk_gate gate_xy_##_pair = { \
>> + .reg = (void *)(_reg), \
>> + .bit_idx = H616_PWM_XY_CLK_CR_GATE_BIT, \
>> + .hw.init = &(struct clk_init_data){ \
>> + .ops = &clk_gate_ops, \
>> + } \
>> +}
>> +
>> +#define H616_PWM_XY_SRC_MUX(_pair, _reg) \
>> +struct clk_mux mux_xy_##_pair = { \
>> + .reg = (void *)(_reg), \
>> + .shift = H616_PWM_XY_CLK_CR_SRC_SHIFT, \
>> + .mask = H616_PWM_XY_CLK_CR_SRC_MASK, \
>> + .flags = CLK_MUX_ROUND_CLOSEST, \
>> + .hw.init = &(struct clk_init_data){ \
>> + .ops = &clk_mux_ops, \
>> + } \
>> +}
>> +
>> +#define H616_PWM_XY_SRC_DIV(_pair, _reg) \
>> +struct clk_divider rate_xy_##_pair = { \
>> + .reg = (void *)(_reg), \
>> + .shift = H616_PWM_XY_CLK_CR_DIV_M_SHIFT, \
>> + .table = clk_table_div_m, \
>> + .hw.init = &(struct clk_init_data){ \
>> + .ops = &clk_divider_ops, \
>> + } \
>> +}
>> +
>> +#define H616_PWM_X_DIV(_idx, _reg) \
>> +struct clk_divider rate_x_##_idx = { \
>> + .reg = (void *)(_reg), \
>> + .shift = H616_PWM_CTRL_PRESCAL_K_SHIFT, \
>> + .width = H616_PWM_CTRL_PRESCAL_K_WIDTH, \
>> + .hw.init = &(struct clk_init_data){ \
>> + .ops = &clk_divider_ops, \
>> + } \
>> +}
>> +
>> +#define H616_PWM_X_BYPASS_GATE(_idx) \
>> +struct clk_gate gate_x_bypass_##_idx = { \
>> + .reg = (void *)H616_PWM_ENR, \
>> + .bit_idx = _idx, \
>> + .hw.init = &(struct clk_init_data){ \
>> + .ops = &clk_gate_ops, \
>> + } \
>> +}
>> +
>> +#define H616_PWM_XY_CLK_SRC(_pair, _reg) \
>> + static H616_PWM_XY_SRC_MUX(_pair, _reg); \
>> + static H616_PWM_XY_SRC_GATE(_pair, _reg); \
>> + static H616_PWM_XY_SRC_DIV(_pair, _reg)
>> +
>> +#define H616_PWM_X_CLK(_idx) \
>> + static H616_PWM_X_DIV(_idx, H616_PWM_CTRL_REG(_idx))
>> +
>> +#define H616_PWM_X_BYPASS_CLK(_idx) \
>> + H616_PWM_X_BYPASS_GATE(_idx)
>> +
>> +#define REF_CLK_XY_SRC(_pair) \
>> + { \
>> + .name = "pwm-clk-src" #_pair, \
>> + .mux_hw = &mux_xy_##_pair.hw, \
>> + .gate_hw = &gate_xy_##_pair.hw, \
>> + .rate_hw = &rate_xy_##_pair.hw, \
>> + }
>> +
>> +#define REF_CLK_X(_idx, _pair) \
>> + { \
>> + .name = "pwm-clk" #_idx, \
>> + .parent_names = (const char *[]){ "pwm-clk-src" #_pair }, \
>> + .num_parents = 1, \
>> + .rate_hw = &rate_x_##_idx.hw, \
>> + .flags = CLK_SET_RATE_PARENT, \
>> + }
>> +
>> +#define REF_CLK_BYPASS(_idx, _pair) \
>> + { \
>> + .name = "pwm-clk-bypass" #_idx, \
>> + .parent_names = (const char *[]){ "pwm-clk-src" #_pair }, \
>> + .num_parents = 1, \
>> + .gate_hw = &gate_x_bypass_##_idx.hw, \
>> + .flags = CLK_SET_RATE_PARENT, \
>> + }
>> +
>> +/*
>> + * H616_PWM_clock_src_xy generation:
>> + * _____ ______ ________
>> + * OSC24M --->| | | | | |
>> + * APB1 ----->| Mux |--->| Gate |--->| /div_m |-----> H616_PWM_clock_src_xy
>> + * |_____| |______| |________|
>> + */
>> +H616_PWM_XY_CLK_SRC(01, H616_PWM_XY_CLK_CR(0));
>> +H616_PWM_XY_CLK_SRC(23, H616_PWM_XY_CLK_CR(1));
>> +H616_PWM_XY_CLK_SRC(45, H616_PWM_XY_CLK_CR(2));
>> +
>> +/*
>> + * H616_PWM_clock_x_div generation:
>> + * ________
>> + * | | H616_PWM_clock_x/y
>> + * H616_PWM_clock_src_xy --->| /div_k |--------------->
>> + * |________|
>> + */
>> +H616_PWM_X_CLK(0);
>> +H616_PWM_X_CLK(1);
>> +H616_PWM_X_CLK(2);
>> +H616_PWM_X_CLK(3);
>> +H616_PWM_X_CLK(4);
>> +H616_PWM_X_CLK(5);
>> +
>> +/*
>> + * H616_PWM_bypass_clock_xy generation:
>> + * ______
>> + * | |
>> + * H616_PWM_clock_src_xy ---->| Gate |-------> H616_PWM_bypass_clock_x
>> + * |______|
>> + *
>> + * The gate is actually H616_PWM_ENR register.
>> + */
>> +H616_PWM_X_BYPASS_CLK(0);
>> +H616_PWM_X_BYPASS_CLK(1);
>> +H616_PWM_X_BYPASS_CLK(2);
>> +H616_PWM_X_BYPASS_CLK(3);
>> +H616_PWM_X_BYPASS_CLK(4);
>> +H616_PWM_X_BYPASS_CLK(5);
>> +
>> +struct clk_pwm_data {
>> + const char *name;
>> + const char **parent_names;
>> + unsigned int num_parents;
>> + struct clk_hw *mux_hw;
>> + struct clk_hw *rate_hw;
>> + struct clk_hw *gate_hw;
>> + unsigned long flags;
>> +};
>> +
>> +#define CLK_BYPASS(h616chip, ch) ((h616chip)->data->npwm + (ch))
>> +#define CLK_XY_SRC_IDX(h616chip, ch) ((h616chip)->data->npwm * 2 + ((ch) >> 1))
>> +static struct clk_pwm_data pwmcc_data[] = {
>> + REF_CLK_X(0, 01),
>> + REF_CLK_X(1, 01),
>> + REF_CLK_X(2, 23),
>> + REF_CLK_X(3, 23),
>> + REF_CLK_X(4, 45),
>> + REF_CLK_X(5, 45),
>> + REF_CLK_BYPASS(0, 01),
>> + REF_CLK_BYPASS(1, 01),
>> + REF_CLK_BYPASS(2, 23),
>> + REF_CLK_BYPASS(3, 23),
>> + REF_CLK_BYPASS(4, 45),
>> + REF_CLK_BYPASS(5, 45),
>> + REF_CLK_XY_SRC(01),
>> + REF_CLK_XY_SRC(23),
>> + REF_CLK_XY_SRC(45),
>> + { /* sentinel */ }
>> +};
>
> We'll probably need a way to tie these static definitions to a particular
> instance of the unit for a given chip. But I guess that can be done later
> when adding more chips to the driver.
>
> I'm not too versed in the clk and pwm APIs but the rest generally looks good
> to me.
Thanks!
>
> All the best,
>
> Paul
>
^ permalink raw reply
* [PATCH v7 3/3] pinctrl: aspeed: Add AST2700 SoC0 support
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
Add pinctrl support for the SoC0 instance of the ASPEED AST2700.
AST2700 consists of two interconnected SoC instances, each with its own
pinctrl register block.
The SoC0 pinctrl hardware closely follows the design found in previous
ASPEED BMC generations, allowing the driver to build upon the common
ASPEED pinctrl infrastructure.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
drivers/pinctrl/aspeed/Kconfig | 9 +
drivers/pinctrl/aspeed/Makefile | 1 +
drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c | 749 ++++++++++++++++++++++++
3 files changed, 759 insertions(+)
diff --git a/drivers/pinctrl/aspeed/Kconfig b/drivers/pinctrl/aspeed/Kconfig
index 1a4e5b9ed471..f9672cca891e 100644
--- a/drivers/pinctrl/aspeed/Kconfig
+++ b/drivers/pinctrl/aspeed/Kconfig
@@ -31,3 +31,12 @@ config PINCTRL_ASPEED_G6
help
Say Y here to enable pin controller support for Aspeed's 6th
generation SoCs. GPIO is provided by a separate GPIO driver.
+
+config PINCTRL_ASPEED_G7_SOC0
+ bool "Aspeed G7 SoC pin control"
+ depends on (ARCH_ASPEED || COMPILE_TEST) && OF
+ select PINCTRL_ASPEED
+ help
+ Say Y here to enable pin controller support for the SoC0 instance
+ of Aspeed's 7th generation SoCs. GPIO is provided by a separate
+ GPIO driver.
diff --git a/drivers/pinctrl/aspeed/Makefile b/drivers/pinctrl/aspeed/Makefile
index db2a7600ae2b..0de524ca2c72 100644
--- a/drivers/pinctrl/aspeed/Makefile
+++ b/drivers/pinctrl/aspeed/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_PINCTRL_ASPEED) += pinctrl-aspeed.o pinmux-aspeed.o
obj-$(CONFIG_PINCTRL_ASPEED_G4) += pinctrl-aspeed-g4.o
obj-$(CONFIG_PINCTRL_ASPEED_G5) += pinctrl-aspeed-g5.o
obj-$(CONFIG_PINCTRL_ASPEED_G6) += pinctrl-aspeed-g6.o
+obj-$(CONFIG_PINCTRL_ASPEED_G7_SOC0) += pinctrl-aspeed-g7-soc0.o
diff --git a/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c b/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c
new file mode 100644
index 000000000000..35a28b677318
--- /dev/null
+++ b/drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c
@@ -0,0 +1,749 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bitops.h>
+#include <linux/bits.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include "pinctrl-aspeed.h"
+#include "pinmux-aspeed.h"
+#include "../pinctrl-utils.h"
+
+#define SCU200 0x200 /* System Reset Control #1 */
+
+#define SCU010 0x010 /* Hardware Strap Register */
+#define SCU400 0x400 /* Multi-function Pin Control #1 */
+#define SCU404 0x404 /* Multi-function Pin Control #2 */
+#define SCU408 0x408 /* Multi-function Pin Control #3 */
+#define SCU40C 0x40C /* Multi-function Pin Control #3 */
+#define SCU410 0x410 /* USB Multi-function Control Register */
+#define SCU414 0x414 /* VGA Function Control Register */
+
+#define SCU480 0x480 /* GPIO18A0 IO Control Register */
+#define SCU484 0x484 /* GPIO18A1 IO Control Register */
+#define SCU488 0x488 /* GPIO18A2 IO Control Register */
+#define SCU48C 0x48c /* GPIO18A3 IO Control Register */
+#define SCU490 0x490 /* GPIO18A4 IO Control Register */
+#define SCU494 0x494 /* GPIO18A5 IO Control Register */
+#define SCU498 0x498 /* GPIO18A6 IO Control Register */
+#define SCU49C 0x49c /* GPIO18A7 IO Control Register */
+#define SCU4A0 0x4A0 /* GPIO18B0 IO Control Register */
+#define SCU4A4 0x4A4 /* GPIO18B1 IO Control Register */
+#define SCU4A8 0x4A8 /* GPIO18B2 IO Control Register */
+#define SCU4AC 0x4AC /* GPIO18B3 IO Control Register */
+
+enum {
+ AC14,
+ AE15,
+ AD14,
+ AE14,
+ AF14,
+ AB13,
+ AB14,
+ AF15,
+ AF13,
+ AC13,
+ AD13,
+ AE13,
+ JTAG_PORT,
+ PCIERC0_PERST,
+ PCIERC1_PERST,
+ PORTA_MODE,
+ PORTA_U2,
+ PORTB_MODE,
+ PORTB_U2,
+ PORTA_U2_PHY,
+ PORTB_U2_PHY,
+ PORTA_U3,
+ PORTB_U3,
+ PORTA_U3_PHY,
+ PORTB_U3_PHY,
+};
+
+SIG_EXPR_LIST_DECL_SEMG(AC14, EMMCCLK, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 0));
+SIG_EXPR_LIST_DECL_SESG(AC14, VB1CS, VB1, SIG_DESC_SET(SCU404, 0));
+PIN_DECL_2(AC14, GPIO18A0, EMMCCLK, VB1CS);
+
+SIG_EXPR_LIST_DECL_SEMG(AE15, EMMCCMD, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 1));
+SIG_EXPR_LIST_DECL_SESG(AE15, VB1CK, VB1, SIG_DESC_SET(SCU404, 1));
+PIN_DECL_2(AE15, GPIO18A1, EMMCCMD, VB1CK);
+
+SIG_EXPR_LIST_DECL_SEMG(AD14, EMMCDAT0, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 2));
+SIG_EXPR_LIST_DECL_SESG(AD14, VB1MOSI, VB1, SIG_DESC_SET(SCU404, 2));
+PIN_DECL_2(AD14, GPIO18A2, EMMCDAT0, VB1MOSI);
+
+SIG_EXPR_LIST_DECL_SEMG(AE14, EMMCDAT1, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 3));
+SIG_EXPR_LIST_DECL_SESG(AE14, VB1MISO, VB1, SIG_DESC_SET(SCU404, 3));
+PIN_DECL_2(AE14, GPIO18A3, EMMCDAT1, VB1MISO);
+
+SIG_EXPR_LIST_DECL_SEMG(AF14, EMMCDAT2, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 4));
+PIN_DECL_1(AF14, GPIO18A4, EMMCDAT2);
+
+SIG_EXPR_LIST_DECL_SEMG(AB13, EMMCDAT3, EMMCG4, EMMC, SIG_DESC_SET(SCU400, 5));
+PIN_DECL_1(AB13, GPIO18A5, EMMCDAT3);
+
+SIG_EXPR_LIST_DECL_SEMG(AB14, EMMCCDN, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 6));
+SIG_EXPR_LIST_DECL_SESG(AB14, VB0CS, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_2(AB14, GPIO18A6, EMMCCDN, VB0CS);
+
+SIG_EXPR_LIST_DECL_SEMG(AF15, EMMCWPN, EMMCG1, EMMC, SIG_DESC_SET(SCU400, 7));
+SIG_EXPR_LIST_DECL_SESG(AF15, VB0CK, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_2(AF15, GPIO18A7, EMMCWPN, VB0CK);
+
+SIG_EXPR_LIST_DECL_SESG(AF13, TSPRSTN, TSPRSTN, SIG_DESC_SET(SCU010, 9));
+SIG_EXPR_LIST_DECL_SEMG(AF13, EMMCDAT4, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 8));
+SIG_EXPR_LIST_DECL_SESG(AF13, VB0MOSI, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_3(AF13, GPIO18B0, TSPRSTN, EMMCDAT4, VB0MOSI);
+
+SIG_EXPR_LIST_DECL_SESG(AC13, UFSCLKI, UFSCLKI, SIG_DESC_SET(SCU010, 19));
+SIG_EXPR_LIST_DECL_SEMG(AC13, EMMCDAT5, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 9));
+SIG_EXPR_LIST_DECL_SESG(AC13, VB0MISO, VB0, SIG_DESC_SET(SCU010, 17));
+PIN_DECL_3(AC13, GPIO18B1, UFSCLKI, EMMCDAT5, VB0MISO);
+
+SIG_EXPR_LIST_DECL_SEMG(AD13, EMMCDAT6, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 10));
+SIG_EXPR_LIST_DECL_SESG(AD13, DDCCLK, VGADDC, SIG_DESC_SET(SCU404, 10));
+PIN_DECL_2(AD13, GPIO18B2, EMMCDAT6, DDCCLK);
+
+SIG_EXPR_LIST_DECL_SEMG(AE13, EMMCDAT7, EMMCG8, EMMC, SIG_DESC_SET(SCU400, 11));
+SIG_EXPR_LIST_DECL_SESG(AE13, DDCDAT, VGADDC, SIG_DESC_SET(SCU404, 11));
+PIN_DECL_2(AE13, GPIO18B3, EMMCDAT7, DDCDAT);
+
+GROUP_DECL(EMMCG1, AC14, AE15, AD14);
+GROUP_DECL(EMMCG4, AC14, AE15, AD14, AE14, AF14, AB13);
+GROUP_DECL(EMMCG8, AC14, AE15, AD14, AE14, AF14, AB13, AF13, AC13, AD13, AE13);
+GROUP_DECL(EMMCWPN, AF15);
+GROUP_DECL(EMMCCDN, AB14);
+FUNC_DECL_(EMMC, "EMMCG1", "EMMCG4", "EMMCG8", "EMMCWPN", "EMMCCDN");
+
+GROUP_DECL(VB1, AC14, AE15, AD14, AE14);
+GROUP_DECL(VB0, AF15, AB14, AF13, AC13);
+FUNC_DECL_2(VB, VB1, VB0);
+
+FUNC_GROUP_DECL(TSPRSTN, AF13);
+
+FUNC_GROUP_DECL(UFSCLKI, AC13);
+
+FUNC_GROUP_DECL(VGADDC, AD13, AE13);
+
+/* JTAG Port Selection */
+#define JTAG_PORT_PSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x0, 0 }
+#define JTAG_PORT_SSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x41, 0 }
+#define JTAG_PORT_TSP_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x42, 0 }
+#define JTAG_PORT_DDR_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x43, 0 }
+#define JTAG_PORT_USB3A_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x44, 0 }
+#define JTAG_PORT_USB3B_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x45, 0 }
+#define JTAG_PORT_PCIEA_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x46, 0 }
+#define JTAG_PORT_PCIEB_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x47, 0 }
+#define JTAG_PORT_JTAGM0_DESC { ASPEED_IP_SCU, SCU408, GENMASK(12, 5), 0x8, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPSP, JTAG0, JTAGPSP, JTAG_PORT_PSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGSSP, JTAG0, JTAGSSP, JTAG_PORT_SSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGTSP, JTAG0, JTAGTSP, JTAG_PORT_TSP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGDDR, JTAG0, JTAGDDR, JTAG_PORT_DDR_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGUSB3A, JTAG0, JTAGUSB3A, JTAG_PORT_USB3A_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGUSB3B, JTAG0, JTAGUSB3B, JTAG_PORT_USB3B_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPCIEA, JTAG0, JTAGPCIEA, JTAG_PORT_PCIEA_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGPCIEB, JTAG0, JTAGPCIEB, JTAG_PORT_PCIEB_DESC);
+SIG_EXPR_LIST_DECL_SEMG(JTAG_PORT, JTAGM0, JTAG0, JTAGM0, JTAG_PORT_JTAGM0_DESC);
+PIN_DECL_(JTAG_PORT, SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPSP), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGSSP),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGTSP), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGDDR),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGUSB3A), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGUSB3B),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPCIEA), SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGPCIEB),
+ SIG_EXPR_LIST_PTR(JTAG_PORT, JTAGM0));
+
+GROUP_DECL(JTAG0, JTAG_PORT);
+
+FUNC_DECL_1(JTAGPSP, JTAG0);
+FUNC_DECL_1(JTAGSSP, JTAG0);
+FUNC_DECL_1(JTAGTSP, JTAG0);
+FUNC_DECL_1(JTAGDDR, JTAG0);
+FUNC_DECL_1(JTAGUSB3A, JTAG0);
+FUNC_DECL_1(JTAGUSB3B, JTAG0);
+FUNC_DECL_1(JTAGPCIEA, JTAG0);
+FUNC_DECL_1(JTAGPCIEB, JTAG0);
+FUNC_DECL_1(JTAGM0, JTAG0);
+
+/* PCIe Reset Control */
+SIG_EXPR_LIST_DECL_SESG(PCIERC0_PERST, PCIERC0PERST, PCIERC0PERST, SIG_DESC_SET(SCU200, 21));
+PIN_DECL_(PCIERC0_PERST, SIG_EXPR_LIST_PTR(PCIERC0_PERST, PCIERC0PERST));
+FUNC_GROUP_DECL(PCIERC0PERST, PCIERC0_PERST);
+
+SIG_EXPR_LIST_DECL_SESG(PCIERC1_PERST, PCIERC1PERST, PCIERC1PERST, SIG_DESC_SET(SCU200, 19));
+PIN_DECL_(PCIERC1_PERST, SIG_EXPR_LIST_PTR(PCIERC1_PERST, PCIERC1PERST));
+FUNC_GROUP_DECL(PCIERC1PERST, PCIERC1_PERST);
+
+#define PORTA_MODE_HPD0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 0, 0 }
+#define PORTA_MODE_D0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 1, 0 }
+#define PORTA_MODE_H_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 2, 0 }
+#define PORTA_MODE_HP_DESC { ASPEED_IP_SCU, SCU410, GENMASK(25, 24), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AHPD0, USB2AH, USB2AHPD0, PORTA_MODE_HPD0_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AH, USB2AHAP, USB2AH, PORTA_MODE_H_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AHP, USB2AHAP, USB2AHP, PORTA_MODE_HP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTA_MODE, USB2AD0, USB2AHAP, USB2AD0, PORTA_MODE_D0_DESC);
+PIN_DECL_(PORTA_MODE, SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AHPD0),
+ SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AH), SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AHP),
+ SIG_EXPR_LIST_PTR(PORTA_MODE, USB2AD0));
+
+#define PORTA_U2_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 0, 0 }
+#define PORTA_U2_D1_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 1, 0 }
+#define PORTA_U2_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 2, 0 }
+#define PORTA_U2_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(3, 2), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHD1, USB2A, USB2AXHD1, PORTA_U2_XHD_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHPD1, USB2A, USB2AXHPD1, PORTA_U2_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXH, USB2AAP, USB2AXH, PORTA_U2_XH_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHP, USB2AAP, USB2AXHP, PORTA_U2_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXH2B, USB2ABP, USB2AXH2B, PORTA_U2_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AXHP2B, USB2ABP, USB2AXHP2B, PORTA_U2_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U2, USB2AD1, USB2ADAP, USB2AD1, PORTA_U2_D1_DESC);
+PIN_DECL_(PORTA_U2, SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHD1), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHPD1),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXH), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHP),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXH2B), SIG_EXPR_LIST_PTR(PORTA_U2, USB2AXHP2B),
+ SIG_EXPR_LIST_PTR(PORTA_U2, USB2AD1));
+
+#define PORTB_MODE_HPD0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 0, 0 }
+#define PORTB_MODE_D0_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 1, 0 }
+#define PORTB_MODE_H_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 2, 0 }
+#define PORTB_MODE_HP_DESC { ASPEED_IP_SCU, SCU410, GENMASK(29, 28), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BHPD0, USB2BH, USB2BHPD0, PORTB_MODE_HPD0_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BH, USB2BHBP, USB2BH, PORTB_MODE_H_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BHP, USB2BHBP, USB2BHP, PORTB_MODE_HP_DESC);
+SIG_EXPR_LIST_DECL_SEMG(PORTB_MODE, USB2BD0, USB2BHBP, USB2BD0, PORTB_MODE_D0_DESC);
+PIN_DECL_(PORTB_MODE, SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BHPD0),
+ SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BH), SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BHP),
+ SIG_EXPR_LIST_PTR(PORTB_MODE, USB2BD0));
+
+#define PORTB_U2_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 0, 0 }
+#define PORTB_U2_D1_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 1, 0 }
+#define PORTB_U2_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 2, 0 }
+#define PORTB_U2_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(7, 6), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHD1, USB2B, USB2BXHD1, PORTB_U2_XHD_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHPD1, USB2B, USB2BXHPD1, PORTB_U2_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXH, USB2BBP, USB2BXH, PORTB_U2_XH_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHP, USB2BBP, USB2BXHP, PORTB_U2_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXH2A, USB2BAP, USB2BXH2A, PORTB_U2_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BXHP2A, USB2BAP, USB2BXHP2A, PORTB_U2_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U2, USB2BD1, USB2BDBP, USB2BD1, PORTB_U2_D1_DESC);
+PIN_DECL_(PORTB_U2, SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHD1), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHPD1),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXH), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHP),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXH2A), SIG_EXPR_LIST_PTR(PORTB_U2, USB2BXHP2A),
+ SIG_EXPR_LIST_PTR(PORTB_U2, USB2BD1));
+/*
+ * USB2 virtual PHY pins.
+ *
+ * PORTA_U2_PHY and PORTB_U2_PHY are logical endpoints, not package pins.
+ * They alias existing USB2 expressions so pin groups can model direct and
+ * cross-coupled routing for host and mode paths.
+ *
+ * - USB2AAP/USB2ADAP/USB2AHAP: use PORTA_U2_PHY
+ * - USB2ABP : use PORTB_U2_PHY
+ * - USB2BBP/USB2BDBP/USB2BHBP: use PORTB_U2_PHY
+ * - USB2BAP : use PORTA_U2_PHY
+ *
+ * They do not have any registers to configure this behaviour; the goal is
+ * simply for the driver to prevent conflicting selections. For example,
+ * selecting group USB2ABP and USB2BBP at the same time should not be
+ * allowed.
+ */
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AXH, USB2AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AXHP, USB2AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2BXH2A, USB2BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2BXHP2A, USB2BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AD1, USB2ADAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AH, USB2AHAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AHP, USB2AHAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U2_PHY, USB2AD0, USB2AHAP);
+PIN_DECL_(PORTA_U2_PHY, SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AXH),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AXHP), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2BXH2A),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2BXHP2A), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AD1),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AH), SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AHP),
+ SIG_EXPR_LIST_PTR(PORTA_U2_PHY, USB2AD0));
+
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2AXH2B, USB2ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2AXHP2B, USB2ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BXH, USB2BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BXHP, USB2BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BD1, USB2BDBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BH, USB2BHBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BHP, USB2BHBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U2_PHY, USB2BD0, USB2BHBP);
+PIN_DECL_(PORTB_U2_PHY, SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2AXH2B),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2AXHP2B), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BXH),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BXHP), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BD1),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BH), SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BHP),
+ SIG_EXPR_LIST_PTR(PORTB_U2_PHY, USB2BD0));
+
+GROUP_DECL(USB2A, PORTA_U2);
+GROUP_DECL(USB2AAP, PORTA_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2ABP, PORTA_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2ADAP, PORTA_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2AH, PORTA_MODE);
+GROUP_DECL(USB2AHAP, PORTA_MODE, PORTA_U2_PHY);
+
+FUNC_DECL_1(USB2AXHD1, USB2A);
+FUNC_DECL_1(USB2AXHPD1, USB2A);
+FUNC_DECL_1(USB2AXH, USB2AAP);
+FUNC_DECL_1(USB2AXHP, USB2AAP);
+FUNC_DECL_1(USB2AXH2B, USB2ABP);
+FUNC_DECL_1(USB2AXHP2B, USB2ABP);
+FUNC_DECL_1(USB2AD1, USB2ADAP);
+FUNC_DECL_1(USB2AHPD0, USB2AH);
+FUNC_DECL_1(USB2AH, USB2AHAP);
+FUNC_DECL_1(USB2AHP, USB2AHAP);
+FUNC_DECL_1(USB2AD0, USB2AHAP);
+
+GROUP_DECL(USB2B, PORTB_U2);
+GROUP_DECL(USB2BBP, PORTB_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2BAP, PORTB_U2, PORTA_U2_PHY);
+GROUP_DECL(USB2BDBP, PORTB_U2, PORTB_U2_PHY);
+GROUP_DECL(USB2BH, PORTB_MODE);
+GROUP_DECL(USB2BHBP, PORTB_MODE, PORTB_U2_PHY);
+
+FUNC_DECL_1(USB2BXHD1, USB2B);
+FUNC_DECL_1(USB2BXHPD1, USB2B);
+FUNC_DECL_1(USB2BXH, USB2BBP);
+FUNC_DECL_1(USB2BXHP, USB2BBP);
+FUNC_DECL_1(USB2BXH2A, USB2BAP);
+FUNC_DECL_1(USB2BXHP2A, USB2BAP);
+FUNC_DECL_1(USB2BD1, USB2BDBP);
+FUNC_DECL_1(USB2BHPD0, USB2BH);
+FUNC_DECL_1(USB2BH, USB2BHBP);
+FUNC_DECL_1(USB2BHP, USB2BHBP);
+FUNC_DECL_1(USB2BD0, USB2BHBP);
+
+#define PORTA_U3_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 0, 0 }
+#define PORTA_U3_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 2, 0 }
+#define PORTA_U3_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(1, 0), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHD, USB3A, USB3AXHD, PORTA_U3_XHD_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHPD, USB3A, USB3AXHPD, PORTA_U3_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXH, USB3AAP, USB3AXH, PORTA_U3_XH_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHP, USB3AAP, USB3AXHP, PORTA_U3_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXH2B, USB3ABP, USB3AXH2B, PORTA_U3_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 9));
+SIG_EXPR_LIST_DECL_SEMG(PORTA_U3, USB3AXHP2B, USB3ABP, USB3AXHP2B, PORTA_U3_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 9));
+PIN_DECL_(PORTA_U3, SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHD), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHPD),
+ SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXH), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHP),
+ SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXH2B), SIG_EXPR_LIST_PTR(PORTA_U3, USB3AXHP2B));
+
+#define PORTB_U3_XHD_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 0, 0 }
+#define PORTB_U3_XH_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 2, 0 }
+#define PORTB_U3_XH2E_DESC { ASPEED_IP_SCU, SCU410, GENMASK(5, 4), 3, 0 }
+
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHD, USB3B, USB3BXHD, PORTB_U3_XHD_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHPD, USB3B, USB3BXHPD, PORTB_U3_XHD_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXH, USB3BBP, USB3BXH, PORTB_U3_XH_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHP, USB3BBP, USB3BXHP, PORTB_U3_XH_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXH2A, USB3BAP, USB3BXH2A, PORTB_U3_XH2E_DESC,
+ SIG_DESC_SET(SCU410, 10));
+SIG_EXPR_LIST_DECL_SEMG(PORTB_U3, USB3BXHP2A, USB3BAP, USB3BXHP2A, PORTB_U3_XH2E_DESC,
+ SIG_DESC_CLEAR(SCU410, 10));
+PIN_DECL_(PORTB_U3, SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHD), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHPD),
+ SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXH), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHP),
+ SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXH2A), SIG_EXPR_LIST_PTR(PORTB_U3, USB3BXHP2A));
+
+/*
+ * USB3 virtual PHY pins.
+ *
+ * PORTA_U3_PHY and PORTB_U3_PHY are logical endpoints, not package pins.
+ * They alias existing USB3 expressions so pin groups can model both direct and
+ * cross-coupled routing to PHY A/B.
+ *
+ * - USB3AAP: PORTA_U3 + PORTA_U3_PHY (A -> PHY A)
+ * - USB3ABP: PORTA_U3 + PORTB_U3_PHY (A -> PHY B)
+ * - USB3BBP: PORTB_U3 + PORTB_U3_PHY (B -> PHY B)
+ * - USB3BAP: PORTB_U3 + PORTA_U3_PHY (B -> PHY A)
+ *
+ * They do not have any registers to configure this behavior; the goal is
+ * simply for the driver to prevent conflicting selections. For example,
+ * selecting group USB3ABP and USB3BBP at the same time should not be
+ * allowed.
+ */
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3AXH, USB3AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3AXHP, USB3AAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3BXH2A, USB3BAP);
+SIG_EXPR_LIST_ALIAS(PORTA_U3_PHY, USB3BXHP2A, USB3BAP);
+PIN_DECL_(PORTA_U3_PHY, SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3AXH),
+ SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3AXHP), SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3BXH2A),
+ SIG_EXPR_LIST_PTR(PORTA_U3_PHY, USB3BXHP2A));
+
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3AXH2B, USB3ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3AXHP2B, USB3ABP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3BXH, USB3BBP);
+SIG_EXPR_LIST_ALIAS(PORTB_U3_PHY, USB3BXHP, USB3BBP);
+PIN_DECL_(PORTB_U3_PHY, SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3AXH2B),
+ SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3AXHP2B), SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3BXH),
+ SIG_EXPR_LIST_PTR(PORTB_U3_PHY, USB3BXHP));
+
+/* USB3A xHCI to vHUB */
+GROUP_DECL(USB3A, PORTA_U3);
+/* USB3A xHCI to USB3A PHY */
+GROUP_DECL(USB3AAP, PORTA_U3, PORTA_U3_PHY);
+/* USB3A xHCI to USB3B PHY */
+GROUP_DECL(USB3ABP, PORTA_U3, PORTB_U3_PHY);
+
+FUNC_DECL_1(USB3AXHD, USB3A);
+FUNC_DECL_1(USB3AXHPD, USB3A);
+FUNC_DECL_1(USB3AXH, USB3AAP);
+FUNC_DECL_1(USB3AXHP, USB3AAP);
+FUNC_DECL_1(USB3AXH2B, USB3ABP);
+FUNC_DECL_1(USB3AXHP2B, USB3ABP);
+
+/* USB3B xHCI to vHUB */
+GROUP_DECL(USB3B, PORTB_U3);
+/* USB3B xHCI to USB3A PHY */
+GROUP_DECL(USB3BAP, PORTB_U3, PORTA_U3_PHY);
+/* USB3B xHCI to USB3B PHY */
+GROUP_DECL(USB3BBP, PORTB_U3, PORTB_U3_PHY);
+
+FUNC_DECL_1(USB3BXHD, USB3B);
+FUNC_DECL_1(USB3BXHPD, USB3B);
+FUNC_DECL_1(USB3BXH, USB3BBP);
+FUNC_DECL_1(USB3BXHP, USB3BBP);
+FUNC_DECL_1(USB3BXH2A, USB3BAP);
+FUNC_DECL_1(USB3BXHP2A, USB3BAP);
+
+static const struct pinctrl_pin_desc aspeed_g7_soc0_pins[] = {
+ ASPEED_PINCTRL_PIN(AC14),
+ ASPEED_PINCTRL_PIN(AE15),
+ ASPEED_PINCTRL_PIN(AD14),
+ ASPEED_PINCTRL_PIN(AE14),
+ ASPEED_PINCTRL_PIN(AF14),
+ ASPEED_PINCTRL_PIN(AB13),
+ ASPEED_PINCTRL_PIN(AB14),
+ ASPEED_PINCTRL_PIN(AF15),
+ ASPEED_PINCTRL_PIN(AF13),
+ ASPEED_PINCTRL_PIN(AC13),
+ ASPEED_PINCTRL_PIN(AD13),
+ ASPEED_PINCTRL_PIN(AE13),
+ ASPEED_PINCTRL_PIN(JTAG_PORT),
+ ASPEED_PINCTRL_PIN(PCIERC0_PERST),
+ ASPEED_PINCTRL_PIN(PCIERC1_PERST),
+ ASPEED_PINCTRL_PIN(PORTA_MODE),
+ ASPEED_PINCTRL_PIN(PORTA_U2),
+ ASPEED_PINCTRL_PIN(PORTA_U3),
+ ASPEED_PINCTRL_PIN(PORTA_U2_PHY),
+ ASPEED_PINCTRL_PIN(PORTA_U3_PHY),
+ ASPEED_PINCTRL_PIN(PORTB_MODE),
+ ASPEED_PINCTRL_PIN(PORTB_U2),
+ ASPEED_PINCTRL_PIN(PORTB_U3),
+ ASPEED_PINCTRL_PIN(PORTB_U2_PHY),
+ ASPEED_PINCTRL_PIN(PORTB_U3_PHY),
+};
+
+static const struct aspeed_pin_group aspeed_g7_soc0_groups[] = {
+ ASPEED_PINCTRL_GROUP(EMMCCDN),
+ ASPEED_PINCTRL_GROUP(EMMCG1),
+ ASPEED_PINCTRL_GROUP(EMMCG4),
+ ASPEED_PINCTRL_GROUP(EMMCG8),
+ ASPEED_PINCTRL_GROUP(EMMCWPN),
+ ASPEED_PINCTRL_GROUP(TSPRSTN),
+ ASPEED_PINCTRL_GROUP(UFSCLKI),
+ ASPEED_PINCTRL_GROUP(VB0),
+ ASPEED_PINCTRL_GROUP(VB1),
+ ASPEED_PINCTRL_GROUP(VGADDC),
+ /* JTAG groups */
+ ASPEED_PINCTRL_GROUP(JTAG0),
+ /* PCIE RC groups */
+ ASPEED_PINCTRL_GROUP(PCIERC0PERST),
+ ASPEED_PINCTRL_GROUP(PCIERC1PERST),
+ /* USB3A groups */
+ ASPEED_PINCTRL_GROUP(USB3A),
+ ASPEED_PINCTRL_GROUP(USB3AAP),
+ ASPEED_PINCTRL_GROUP(USB3ABP),
+ /* USB3B groups */
+ ASPEED_PINCTRL_GROUP(USB3B),
+ ASPEED_PINCTRL_GROUP(USB3BAP),
+ ASPEED_PINCTRL_GROUP(USB3BBP),
+ /* USB2A groups */
+ ASPEED_PINCTRL_GROUP(USB2A),
+ ASPEED_PINCTRL_GROUP(USB2AAP),
+ ASPEED_PINCTRL_GROUP(USB2ABP),
+ ASPEED_PINCTRL_GROUP(USB2ADAP),
+ ASPEED_PINCTRL_GROUP(USB2AH),
+ ASPEED_PINCTRL_GROUP(USB2AHAP),
+ /* USB2B groups */
+ ASPEED_PINCTRL_GROUP(USB2B),
+ ASPEED_PINCTRL_GROUP(USB2BAP),
+ ASPEED_PINCTRL_GROUP(USB2BBP),
+ ASPEED_PINCTRL_GROUP(USB2BDBP),
+ ASPEED_PINCTRL_GROUP(USB2BH),
+ ASPEED_PINCTRL_GROUP(USB2BHBP),
+};
+
+static const struct aspeed_pin_function aspeed_g7_soc0_functions[] = {
+ ASPEED_PINCTRL_FUNC(EMMC),
+ ASPEED_PINCTRL_FUNC(TSPRSTN),
+ ASPEED_PINCTRL_FUNC(UFSCLKI),
+ ASPEED_PINCTRL_FUNC(VB),
+ ASPEED_PINCTRL_FUNC(VGADDC),
+ /* JTAG functions */
+ ASPEED_PINCTRL_FUNC(JTAGDDR),
+ ASPEED_PINCTRL_FUNC(JTAGM0),
+ ASPEED_PINCTRL_FUNC(JTAGPCIEA),
+ ASPEED_PINCTRL_FUNC(JTAGPCIEB),
+ ASPEED_PINCTRL_FUNC(JTAGPSP),
+ ASPEED_PINCTRL_FUNC(JTAGSSP),
+ ASPEED_PINCTRL_FUNC(JTAGTSP),
+ ASPEED_PINCTRL_FUNC(JTAGUSB3A),
+ ASPEED_PINCTRL_FUNC(JTAGUSB3B),
+ /* PCIE RC functions */
+ ASPEED_PINCTRL_FUNC(PCIERC0PERST),
+ ASPEED_PINCTRL_FUNC(PCIERC1PERST),
+ /* USB3A functions */
+ ASPEED_PINCTRL_FUNC(USB3AXH),
+ ASPEED_PINCTRL_FUNC(USB3AXH2B),
+ ASPEED_PINCTRL_FUNC(USB3AXHD),
+ ASPEED_PINCTRL_FUNC(USB3AXHP),
+ ASPEED_PINCTRL_FUNC(USB3AXHP2B),
+ ASPEED_PINCTRL_FUNC(USB3AXHPD),
+ /* USB3B functions */
+ ASPEED_PINCTRL_FUNC(USB3BXH),
+ ASPEED_PINCTRL_FUNC(USB3BXH2A),
+ ASPEED_PINCTRL_FUNC(USB3BXHD),
+ ASPEED_PINCTRL_FUNC(USB3BXHP),
+ ASPEED_PINCTRL_FUNC(USB3BXHP2A),
+ ASPEED_PINCTRL_FUNC(USB3BXHPD),
+ /* USB2A functions */
+ ASPEED_PINCTRL_FUNC(USB2AD0),
+ ASPEED_PINCTRL_FUNC(USB2AD1),
+ ASPEED_PINCTRL_FUNC(USB2AH),
+ ASPEED_PINCTRL_FUNC(USB2AHP),
+ ASPEED_PINCTRL_FUNC(USB2AHPD0),
+ ASPEED_PINCTRL_FUNC(USB2AXH),
+ ASPEED_PINCTRL_FUNC(USB2AXH2B),
+ ASPEED_PINCTRL_FUNC(USB2AXHD1),
+ ASPEED_PINCTRL_FUNC(USB2AXHP),
+ ASPEED_PINCTRL_FUNC(USB2AXHP2B),
+ ASPEED_PINCTRL_FUNC(USB2AXHPD1),
+ /* USB2B functions */
+ ASPEED_PINCTRL_FUNC(USB2BD0),
+ ASPEED_PINCTRL_FUNC(USB2BD1),
+ ASPEED_PINCTRL_FUNC(USB2BH),
+ ASPEED_PINCTRL_FUNC(USB2BHP),
+ ASPEED_PINCTRL_FUNC(USB2BHPD0),
+ ASPEED_PINCTRL_FUNC(USB2BXH),
+ ASPEED_PINCTRL_FUNC(USB2BXH2A),
+ ASPEED_PINCTRL_FUNC(USB2BXHD1),
+ ASPEED_PINCTRL_FUNC(USB2BXHP),
+ ASPEED_PINCTRL_FUNC(USB2BXHP2A),
+ ASPEED_PINCTRL_FUNC(USB2BXHPD1),
+};
+
+static const struct pinmux_ops aspeed_g7_soc0_pinmux_ops = {
+ .get_functions_count = aspeed_pinmux_get_fn_count,
+ .get_function_name = aspeed_pinmux_get_fn_name,
+ .get_function_groups = aspeed_pinmux_get_fn_groups,
+ .set_mux = aspeed_pinmux_set_mux,
+ .gpio_request_enable = aspeed_gpio_request_enable,
+ .strict = true,
+};
+
+static const struct pinctrl_ops aspeed_g7_soc0_pinctrl_ops = {
+ .get_groups_count = aspeed_pinctrl_get_groups_count,
+ .get_group_name = aspeed_pinctrl_get_group_name,
+ .get_group_pins = aspeed_pinctrl_get_group_pins,
+ .pin_dbg_show = aspeed_pinctrl_pin_dbg_show,
+ .dt_node_to_map = pinconf_generic_dt_node_to_map_all,
+ .dt_free_map = pinctrl_utils_free_map,
+};
+
+static const struct pinconf_ops aspeed_g7_soc0_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = aspeed_pin_config_get,
+ .pin_config_set = aspeed_pin_config_set,
+ .pin_config_group_get = aspeed_pin_config_group_get,
+ .pin_config_group_set = aspeed_pin_config_group_set,
+};
+
+/* pinctrl_desc */
+static const struct pinctrl_desc aspeed_g7_soc0_pinctrl_desc = {
+ .name = "aspeed-g7-soc0-pinctrl",
+ .pins = aspeed_g7_soc0_pins,
+ .npins = ARRAY_SIZE(aspeed_g7_soc0_pins),
+ .pctlops = &aspeed_g7_soc0_pinctrl_ops,
+ .pmxops = &aspeed_g7_soc0_pinmux_ops,
+ .confops = &aspeed_g7_soc0_pinconf_ops,
+};
+
+static const struct aspeed_pin_config aspeed_g7_soc0_configs[] = {
+ /* GPIO18A */
+ { PIN_CONFIG_DRIVE_STRENGTH, { AC14, AC14 }, SCU480, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AC14, AC14 }, SCU480, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AC14, AC14 }, SCU480, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AC14, AC14 }, SCU480, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE15, AE15 }, SCU484, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE15, AE15 }, SCU484, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE15, AE15 }, SCU484, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE15, AE15 }, SCU484, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AD14, AD14 }, SCU488, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AD14, AD14 }, SCU488, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AD14, AD14 }, SCU488, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AD14, AD14 }, SCU488, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE14, AE14 }, SCU48C, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE14, AE14 }, SCU48C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE14, AE14 }, SCU48C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE14, AE14 }, SCU48C, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF14, AF14 }, SCU490, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF14, AF14 }, SCU490, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF14, AF14 }, SCU490, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF14, AF14 }, SCU490, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AB13, AB13 }, SCU494, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AB13, AB13 }, SCU494, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AB13, AB13 }, SCU494, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AB13, AB13 }, SCU494, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AB14, AB14 }, SCU498, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AB14, AB14 }, SCU498, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AB14, AB14 }, SCU498, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AB14, AB14 }, SCU498, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF15, AF15 }, SCU49C, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF15, AF15 }, SCU49C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF15, AF15 }, SCU49C, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF15, AF15 }, SCU49C, BIT(5) },
+ /* GPIO18B */
+ { PIN_CONFIG_DRIVE_STRENGTH, { AF13, AF13 }, SCU4A0, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AF13, AF13 }, SCU4A0, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AF13, AF13 }, SCU4A0, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AF13, AF13 }, SCU4A0, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AC13, AC13 }, SCU4A4, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AC13, AC13 }, SCU4A4, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AC13, AC13 }, SCU4A4, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AC13, AC13 }, SCU4A4, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AD13, AD13 }, SCU4A8, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AD13, AD13 }, SCU4A8, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AD13, AD13 }, SCU4A8, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AD13, AD13 }, SCU4A8, BIT(5) },
+ { PIN_CONFIG_DRIVE_STRENGTH, { AE13, AE13 }, SCU4AC, GENMASK(3, 0) },
+ { PIN_CONFIG_BIAS_PULL_DOWN, { AE13, AE13 }, SCU4AC, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_PULL_UP, { AE13, AE13 }, SCU4AC, GENMASK(5, 4) },
+ { PIN_CONFIG_BIAS_DISABLE, { AE13, AE13 }, SCU4AC, BIT(5) },
+};
+
+static const struct aspeed_pin_config_map aspeed_g7_soc0_pin_config_map[] = {
+ { PIN_CONFIG_BIAS_PULL_DOWN, -1, 2, GENMASK(1, 0) },
+ { PIN_CONFIG_BIAS_PULL_UP, -1, 3, GENMASK(1, 0) },
+ { PIN_CONFIG_BIAS_DISABLE, -1, 0, BIT_MASK(0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 3, 0, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 6, 1, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 8, 2, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 11, 3, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 16, 4, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 18, 5, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 20, 6, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 23, 7, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 30, 8, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 32, 9, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 33, 10, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 35, 11, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 37, 12, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 38, 13, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 39, 14, GENMASK(3, 0) },
+ { PIN_CONFIG_DRIVE_STRENGTH, 41, 15, GENMASK(3, 0) },
+
+};
+
+static int aspeed_g7_soc0_sig_expr_set(struct aspeed_pinmux_data *ctx,
+ const struct aspeed_sig_expr *expr, bool enable)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < expr->ndescs; i++) {
+ const struct aspeed_sig_desc *desc = &expr->descs[i];
+ u32 pattern = enable ? desc->enable : desc->disable;
+ u32 val = (pattern << __ffs(desc->mask));
+
+ if (!ctx->maps[desc->ip])
+ return -ENODEV;
+
+ WARN_ON_ONCE(desc->ip != ASPEED_IP_SCU);
+
+ ret = regmap_update_bits(ctx->maps[desc->ip], desc->reg,
+ desc->mask, val);
+ if (ret)
+ return ret;
+ }
+
+ ret = aspeed_sig_expr_eval(ctx, expr, enable);
+ if (ret < 0)
+ return ret;
+
+ return ret ? 0 : -EPERM;
+}
+
+static const struct aspeed_pinmux_ops aspeed_g7_soc0_ops = {
+ .set = aspeed_g7_soc0_sig_expr_set,
+};
+
+static struct aspeed_pinctrl_data aspeed_g7_soc0_pinctrl_data = {
+ .pins = aspeed_g7_soc0_pins,
+ .npins = ARRAY_SIZE(aspeed_g7_soc0_pins),
+ .pinmux = {
+ .ops = &aspeed_g7_soc0_ops,
+ .groups = aspeed_g7_soc0_groups,
+ .ngroups = ARRAY_SIZE(aspeed_g7_soc0_groups),
+ .functions = aspeed_g7_soc0_functions,
+ .nfunctions = ARRAY_SIZE(aspeed_g7_soc0_functions),
+ },
+ .configs = aspeed_g7_soc0_configs,
+ .nconfigs = ARRAY_SIZE(aspeed_g7_soc0_configs),
+ .confmaps = aspeed_g7_soc0_pin_config_map,
+ .nconfmaps = ARRAY_SIZE(aspeed_g7_soc0_pin_config_map),
+};
+
+static int aspeed_g7_soc0_pinctrl_probe(struct platform_device *pdev)
+{
+ return aspeed_pinctrl_probe(pdev, &aspeed_g7_soc0_pinctrl_desc,
+ &aspeed_g7_soc0_pinctrl_data);
+}
+
+static const struct of_device_id aspeed_g7_soc0_pinctrl_match[] = {
+ { .compatible = "aspeed,ast2700-soc0-pinctrl" },
+ {}
+};
+MODULE_DEVICE_TABLE(of, aspeed_g7_soc0_pinctrl_match);
+
+static struct platform_driver aspeed_g7_soc0_pinctrl_driver = {
+ .probe = aspeed_g7_soc0_pinctrl_probe,
+ .driver = {
+ .name = "aspeed-g7-soc0-pinctrl",
+ .of_match_table = aspeed_g7_soc0_pinctrl_match,
+ .suppress_bind_attrs = true,
+ },
+};
+
+static int __init aspeed_g7_soc0_pinctrl_init(void)
+{
+ return platform_driver_register(&aspeed_g7_soc0_pinctrl_driver);
+}
+arch_initcall(aspeed_g7_soc0_pinctrl_init);
--
2.34.1
^ permalink raw reply related
* [PATCH v7 1/3] dt-bindings: pinctrl: Add aspeed,ast2700-soc0-pinctrl
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
Add a device tree binding for the pin controller found in the
ASPEED AST2700 SoC0.
The controller manages various peripheral functions such as eMMC, USB,
VGA DDC, JTAG, and PCIe root complex signals.
Describe the AST2700 SoC0 pin controller using standard pin multiplexing
and configuration properties.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
.../pinctrl/aspeed,ast2700-soc0-pinctrl.yaml | 162 +++++++++++++++++++++
1 file changed, 162 insertions(+)
diff --git a/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml b/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml
new file mode 100644
index 000000000000..947f3cd09fcc
--- /dev/null
+++ b/Documentation/devicetree/bindings/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml
@@ -0,0 +1,162 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/pinctrl/aspeed,ast2700-soc0-pinctrl.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ASPEED AST2700 SoC0 Pin Controller
+
+maintainers:
+ - Billy Tsai <billy_tsai@aspeedtech.com>
+
+description:
+ The AST2700 features a dual-SoC architecture with two interconnected SoCs,
+ each having its own System Control Unit (SCU) for independent pin control.
+ This pin controller manages the pin multiplexing for SoC0.
+
+ The SoC0 pin controller manages pin functions including eMMC, VGA DDC,
+ dual USB3/USB2 ports (A and B), JTAG, and PCIe root complex interfaces.
+
+properties:
+ compatible:
+ const: aspeed,ast2700-soc0-pinctrl
+ reg:
+ maxItems: 1
+
+patternProperties:
+ '-state$':
+ type: object
+ allOf:
+ - $ref: pinmux-node.yaml#
+ - $ref: pincfg-node.yaml#
+
+ additionalProperties: false
+
+ properties:
+ function:
+ enum:
+ - EMMC
+ - JTAGDDR
+ - JTAGM0
+ - JTAGPCIEA
+ - JTAGPCIEB
+ - JTAGPSP
+ - JTAGSSP
+ - JTAGTSP
+ - JTAGUSB3A
+ - JTAGUSB3B
+ - PCIERC0PERST
+ - PCIERC1PERST
+ - TSPRSTN
+ - UFSCLKI
+ - USB2AD0
+ - USB2AD1
+ - USB2AH
+ - USB2AHP
+ - USB2AHPD0
+ - USB2AXH
+ - USB2AXH2B
+ - USB2AXHD1
+ - USB2AXHP
+ - USB2AXHP2B
+ - USB2AXHPD1
+ - USB2BD0
+ - USB2BD1
+ - USB2BH
+ - USB2BHP
+ - USB2BHPD0
+ - USB2BXH
+ - USB2BXH2A
+ - USB2BXHD1
+ - USB2BXHP
+ - USB2BXHP2A
+ - USB2BXHPD1
+ - USB3AXH
+ - USB3AXH2B
+ - USB3AXHD
+ - USB3AXHP
+ - USB3AXHP2B
+ - USB3AXHPD
+ - USB3BXH
+ - USB3BXH2A
+ - USB3BXHD
+ - USB3BXHP
+ - USB3BXHP2A
+ - USB3BXHPD
+ - VB
+ - VGADDC
+
+ groups:
+ enum:
+ - EMMCCDN
+ - EMMCG1
+ - EMMCG4
+ - EMMCG8
+ - EMMCWPN
+ - JTAG0
+ - PCIERC0PERST
+ - PCIERC1PERST
+ - TSPRSTN
+ - UFSCLKI
+ - USB2A
+ - USB2AAP
+ - USB2ABP
+ - USB2ADAP
+ - USB2AH
+ - USB2AHAP
+ - USB2B
+ - USB2BAP
+ - USB2BBP
+ - USB2BDBP
+ - USB2BH
+ - USB2BHBP
+ - USB3A
+ - USB3AAP
+ - USB3ABP
+ - USB3B
+ - USB3BAP
+ - USB3BBP
+ - VB0
+ - VB1
+ - VGADDC
+ pins:
+ enum:
+ - AB13
+ - AB14
+ - AC13
+ - AC14
+ - AD13
+ - AD14
+ - AE13
+ - AE14
+ - AE15
+ - AF13
+ - AF14
+ - AF15
+
+ drive-strength:
+ enum: [3, 6, 8, 11, 16, 18, 20, 23, 30, 32, 33, 35, 37, 38, 39, 41]
+
+ bias-disable: true
+ bias-pull-up: true
+ bias-pull-down: true
+
+required:
+ - compatible
+ - reg
+
+allOf:
+ - $ref: pinctrl.yaml#
+
+additionalProperties: false
+
+examples:
+ - |
+ pinctrl@400 {
+ compatible = "aspeed,ast2700-soc0-pinctrl";
+ reg = <0x400 0x318>;
+ emmc-state {
+ function = "EMMC";
+ groups = "EMMCG1";
+ };
+ };
--
2.34.1
^ permalink raw reply related
* [PATCH v7 2/3] dt-bindings: mfd: aspeed,ast2x00-scu: Describe AST2700 SCU0
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
In-Reply-To: <20260416-upstream_pinctrl-v7-0-d72762253163@aspeedtech.com>
AST2700 consists of two interconnected SoC instances, each with its own
System Control Unit (SCU). The SCU0 provides pin control, interrupt
controllers, clocks, resets, and address-space mappings for the
Secondary and Tertiary Service Processors (SSP and TSP).
Describe the SSP/TSP address mappings using the standard
memory-region and memory-region-names properties.
Disallow legacy child nodes that are not present on AST2700, including
p2a-control and smp-memram. The latter is unnecessary as software can
access the scratch registers via the SCU syscon.
Also allow the AST2700 SoC0 pin controller to be described as a child
node of the SCU0, and add an example illustrating the SCU0 layout,
including reserved-memory, interrupt controllers, and pinctrl.
Signed-off-by: Billy Tsai <billy_tsai@aspeedtech.com>
---
.../bindings/mfd/aspeed,ast2x00-scu.yaml | 113 +++++++++++++++++++++
1 file changed, 113 insertions(+)
diff --git a/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml b/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
index a87f31fce019..215ff59b38ea 100644
--- a/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
+++ b/Documentation/devicetree/bindings/mfd/aspeed,ast2x00-scu.yaml
@@ -46,6 +46,17 @@ properties:
'#reset-cells':
const: 1
+ memory-region:
+ items:
+ - description: Region mapped through the first SSP address window.
+ - description: Region mapped through the second SSP address window.
+ - description: Region mapped through the TSP address window.
+ memory-region-names:
+ items:
+ - const: ssp-0
+ - const: ssp-1
+ - const: tsp
+
patternProperties:
'^p2a-control@[0-9a-f]+$':
description: >
@@ -87,6 +98,7 @@ patternProperties:
- aspeed,ast2400-pinctrl
- aspeed,ast2500-pinctrl
- aspeed,ast2600-pinctrl
+ - aspeed,ast2700-soc0-pinctrl
required:
- compatible
@@ -156,6 +168,30 @@ required:
- '#clock-cells'
- '#reset-cells'
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ anyOf:
+ - const: aspeed,ast2700-scu0
+ - const: aspeed,ast2700-scu1
+ then:
+ patternProperties:
+ '^p2a-control@[0-9a-f]+$': false
+ '^smp-memram@[0-9a-f]+$': false
+
+ - if:
+ not:
+ properties:
+ compatible:
+ contains:
+ const: aspeed,ast2700-scu0
+ then:
+ properties:
+ memory-region: false
+ memory-region-names: false
+
additionalProperties: false
examples:
@@ -180,4 +216,81 @@ examples:
reg = <0x7c 0x4>, <0x150 0x8>;
};
};
+
+ - |
+ / {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ reserved-memory {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ ssp_region_0: memory@400000000 {
+ reg = <0x4 0x00000000 0x0 0x01000000>;
+ no-map;
+ };
+
+ ssp_region_1: memory@401000000 {
+ reg = <0x4 0x01000000 0x0 0x01000000>;
+ no-map;
+ };
+
+ tsp_region: memory@402000000 {
+ reg = <0x4 0x02000000 0x0 0x01000000>;
+ no-map;
+ };
+ };
+
+ bus {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ syscon@12c02000 {
+ compatible = "aspeed,ast2700-scu0", "syscon", "simple-mfd";
+ reg = <0 0x12c02000 0 0x1000>;
+ ranges = <0x0 0x0 0x12c02000 0x1000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+
+ memory-region = <&ssp_region_0>, <&ssp_region_1>,
+ <&tsp_region>;
+ memory-region-names = "ssp-0", "ssp-1", "tsp";
+
+ silicon-id@0 {
+ compatible = "aspeed,ast2700-silicon-id", "aspeed,silicon-id";
+ reg = <0x0 0x4>;
+ };
+
+ interrupt-controller@1b0 {
+ compatible = "aspeed,ast2700-scu-ic0";
+ reg = <0x1b0 0x4>;
+ #interrupt-cells = <1>;
+ interrupts-extended = <&intc0 97>;
+ interrupt-controller;
+ };
+
+ interrupt-controller@1e0 {
+ compatible = "aspeed,ast2700-scu-ic1";
+ reg = <0x1e0 0x4>;
+ #interrupt-cells = <1>;
+ interrupts-extended = <&intc0 98>;
+ interrupt-controller;
+ };
+
+ pinctrl@400 {
+ compatible = "aspeed,ast2700-soc0-pinctrl";
+ reg = <0x400 0x318>;
+ emmc-state {
+ function = "EMMC";
+ groups = "EMMCG1";
+ };
+ };
+ };
+ };
+ };
+
...
--
2.34.1
^ permalink raw reply related
* [PATCH v7 0/3] pinctrl: aspeed: Add AST2700 SoC0 support
From: Billy Tsai @ 2026-04-16 7:29 UTC (permalink / raw)
To: Lee Jones, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Joel Stanley, Andrew Jeffery, Linus Walleij, Billy Tsai,
Bartosz Golaszewski, Ryan Chen
Cc: Andrew Jeffery, devicetree, linux-arm-kernel, linux-aspeed,
linux-kernel, openbmc, linux-gpio, linux-clk
AST2700 is composed of two interconnected SoC instances, each providing
its own pin control hardware. This series introduces bindings describing
the AST2700 pinctrl architecture and adds pinctrl driver support for the
SoC0 instance.
The bindings document the AST2700 dual-SoC design and follow common
pinctrl conventions, while the SoC0 driver implementation builds upon
the existing ASPEED pinctrl infrastructure.
---
Changes in v7:
- Fix schema validation error by ensuring if/then/else keywords are used
correctly.
- Remove unnecessary restrictions on AST2700 SoC0 pinctrl pin configuration
properties.
- Link to v6: https://lore.kernel.org/r/20260414-upstream_pinctrl-v6-0-709f2127da33@aspeedtech.com
Changes in v6:
- Restrict AST2700 SoC0 pinctrl pin configuration properties
(`drive-strength` and `bias-*`) to `pins`-based state nodes in the
binding schema.
- Move `memory-region` and `memory-region-names` in the AST2x00 SCU
binding to top-level descriptions, and keep the conditional schema
only to disallow them for non-AST2700 SCU0 compatibles.
- Add bias pull-up, pull-down, and disable support for AST2700 SoC0
GPIO18A/GPIO18B pins in the pinctrl driver.
- Fix the USB2 Port B XH/XHP mux selector definitions to use the
correct `PORTB_U2_XH_DESC` setting.
- Link to v5: https://lore.kernel.org/r/20260331-upstream_pinctrl-v5-0-8994f59ff367@aspeedtech.com
Changes in v5:
- Complete the AST2700 SCU0 binding and disallow child nodes that are
not relevant for the hardware (p2a-control and smp-memram).
- Add examples for both the AST2700 SCU0 binding and the pinctrl binding,
ensuring they are valid against the schema.
- Rework the pinctrl binding example to be self-contained and independent
of the SCU binding.
- Reorder the binding patches so the pinctrl binding is introduced before
the SCU binding update, allowing the SCU example to be added cleanly.
- Adjust the binding accordingly to restrict drive-strength to the
supported values.
- Update the drive-strength table to match hardware-defined values.
- Link to v4: https://lore.kernel.org/r/20260306-upstream_pinctrl-v4-0-ad4e8ab8b489@aspeedtech.com
Changes in v4:
- Rename series title to "pinctrl: aspeed: Add AST2700 SoC0 support"
to make it specific to SoC0.
- Remove unnecessary SCU example from bindings.
- Fix Makefile newline to avoid patch warning.
- Make pinctrl data structures const and align with existing Aspeed drivers.
- Sort the arrays and enums alphabetically.
- Minor cleanups for consistency, no functional changes.
- Link to v3: https://lore.kernel.org/r/20260120-upstream_pinctrl-v3-0-868fbf8413b5@aspeedtech.com
Changes in v3:
dt-bindings: pinctrl: aspeed: AST2700 pinctrl improvements
- Improved binding descriptions for SoC0 and SoC1 to better explain the
AST2700 dual-SoC architecture with independent pin control blocks
- Switched from additionalProperties to patternProperties using the
'-state$' suffix to restrict child node naming
- Removed per-binding examples based on review feedback
- Added additionalProperties: false at the top level for stricter schema
validation
- Dropped the aspeed,ast2700-soc1-pinctrl binding, as the SoC1 pinctrl
registers follow a regular layout and can be described using an
existing generic pinctrl binding
- Updated the function and group enum lists to match the definitions
used by the AST2700 pinctrl driver
dt-bindings: mfd: aspeed: Add AST2700 SCU example with pinctrl
- Added a complete AST2700 SCU0 example demonstrating pinctrl integration
- Example covers both pin function/group configuration and pin
drive-strength settings
- Updated child node naming to use the '-state' suffix, following common
pinctrl conventions
pinctrl: aspeed: AST2700 SoC0 driver improvements
- Refactored pin and signal declarations to use common ASPEED pinmux
macros (SIG_EXPR_LIST_DECL_SEMG, SIG_EXPR_LIST_DECL_SESG, PIN_DECL_*)
- Added SCU010 register definition for hardware strap control
- Reworked code structure to better align with existing ASPEED pinctrl
drivers
- Link to v2: https://lore.kernel.org/r/20250904103401.88287-1-billy_tsai@aspeedtech.com
Changes in v2:
- Update pinctrl aspeed binding files.
- Update the commit message for pinctrl binding patch.
- Link to v1: https://lore.kernel.org/r/20250829073030.2749482-1-billy_tsai@aspeedtech.com
---
Billy Tsai (3):
dt-bindings: pinctrl: Add aspeed,ast2700-soc0-pinctrl
dt-bindings: mfd: aspeed,ast2x00-scu: Describe AST2700 SCU0
pinctrl: aspeed: Add AST2700 SoC0 support
.../bindings/mfd/aspeed,ast2x00-scu.yaml | 113 ++++
.../pinctrl/aspeed,ast2700-soc0-pinctrl.yaml | 162 +++++
drivers/pinctrl/aspeed/Kconfig | 9 +
drivers/pinctrl/aspeed/Makefile | 1 +
drivers/pinctrl/aspeed/pinctrl-aspeed-g7-soc0.c | 749 +++++++++++++++++++++
5 files changed, 1034 insertions(+)
---
base-commit: af4e9ef3d78420feb8fe58cd9a1ab80c501b3c08
change-id: 20251215-upstream_pinctrl-8f195df0a975
Best regards,
--
Billy Tsai <billy_tsai@aspeedtech.com>
^ permalink raw reply
* [PATCH net v3 3/3] net: airoha: Add missing bits in airoha_qdma_cleanup_tx_queue()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
Similar to airoha_qdma_cleanup_rx_queue(), reset DMA TX descriptors in
airoha_qdma_cleanup_tx_queue routine. Moreover, reset TX_DMA_IDX to
TX_CPU_IDX to notify the NIC the QDMA TX ring is empty.
Fixes: 23020f0493270 ("net: airoha: Introduce ethernet support for EN7581 SoC")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 32 ++++++++++++++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index dfe8a0bd2abd..903682e88e89 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -1039,12 +1039,15 @@ static int airoha_qdma_init_tx(struct airoha_qdma *qdma)
static void airoha_qdma_cleanup_tx_queue(struct airoha_queue *q)
{
- struct airoha_eth *eth = q->qdma->eth;
- int i;
+ struct airoha_qdma *qdma = q->qdma;
+ struct airoha_eth *eth = qdma->eth;
+ int i, qid = q - &qdma->q_tx[0];
+ u16 index = 0;
spin_lock_bh(&q->lock);
for (i = 0; i < q->ndesc; i++) {
struct airoha_queue_entry *e = &q->entry[i];
+ struct airoha_qdma_desc *desc = &q->desc[i];
if (!e->dma_addr)
continue;
@@ -1055,8 +1058,33 @@ static void airoha_qdma_cleanup_tx_queue(struct airoha_queue *q)
e->dma_addr = 0;
e->skb = NULL;
list_add_tail(&e->list, &q->tx_list);
+
+ /* Reset DMA descriptor */
+ WRITE_ONCE(desc->ctrl, 0);
+ WRITE_ONCE(desc->addr, 0);
+ WRITE_ONCE(desc->data, 0);
+ WRITE_ONCE(desc->msg0, 0);
+ WRITE_ONCE(desc->msg1, 0);
+ WRITE_ONCE(desc->msg2, 0);
+
q->queued--;
}
+
+ if (!list_empty(&q->tx_list)) {
+ struct airoha_queue_entry *e;
+
+ e = list_first_entry(&q->tx_list, struct airoha_queue_entry,
+ list);
+ index = e - q->entry;
+ }
+ /* Set TX_DMA_IDX to TX_CPU_IDX to notify the hw the QDMA TX ring is
+ * empty.
+ */
+ airoha_qdma_rmw(qdma, REG_TX_CPU_IDX(qid), TX_RING_CPU_IDX_MASK,
+ FIELD_PREP(TX_RING_CPU_IDX_MASK, index));
+ airoha_qdma_rmw(qdma, REG_TX_DMA_IDX(qid), TX_RING_DMA_IDX_MASK,
+ FIELD_PREP(TX_RING_DMA_IDX_MASK, index));
+
spin_unlock_bh(&q->lock);
}
--
2.53.0
^ permalink raw reply related
* [PATCH net v3 2/3] net: airoha: Move entries to queue head in case of DMA mapping failure in airoha_dev_xmit()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
In order to respect the original descriptor order and avoid any
potential IOMMU fault or memory corruption, move pending queue entries
to the head of hw queue tx_list if the DMA mapping of current inflight
packet fails in airoha_dev_xmit routine.
Fixes: 3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order DMA tx descriptors")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index 690bfaf8d7d9..dfe8a0bd2abd 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -2053,7 +2053,7 @@ static netdev_tx_t airoha_dev_xmit(struct sk_buff *skb,
dma_unmap_single(dev->dev.parent, e->dma_addr, e->dma_len,
DMA_TO_DEVICE);
e->dma_addr = 0;
- list_move_tail(&e->list, &q->tx_list);
+ list_move(&e->list, &q->tx_list);
}
spin_unlock_bh(&q->lock);
--
2.53.0
^ permalink raw reply related
* [PATCH net v3 1/3] net: airoha: Move ndesc initialization at end of airoha_qdma_init_tx()
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
In-Reply-To: <20260416-airoha_qdma_cleanup_tx_queue-fix-net-v3-0-2b69f5788580@kernel.org>
If queue entry list allocation fails in airoha_qdma_init_tx_queue routine,
airoha_qdma_cleanup_tx_queue() will trigger a NULL pointer dereference
accessing the queue entry array. The issue is due to the early ndesc
initialization in airoha_qdma_init_tx_queue(). Fix the issue moving ndesc
initialization at end of airoha_qdma_init_tx routine.
Fixes: 3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order DMA tx descriptors")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/airoha/airoha_eth.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
index e1ab15f1ee7d..690bfaf8d7d9 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.c
+++ b/drivers/net/ethernet/airoha/airoha_eth.c
@@ -954,27 +954,27 @@ static int airoha_qdma_init_tx_queue(struct airoha_queue *q,
dma_addr_t dma_addr;
spin_lock_init(&q->lock);
- q->ndesc = size;
q->qdma = qdma;
q->free_thr = 1 + MAX_SKB_FRAGS;
INIT_LIST_HEAD(&q->tx_list);
- q->entry = devm_kzalloc(eth->dev, q->ndesc * sizeof(*q->entry),
+ q->entry = devm_kzalloc(eth->dev, size * sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
- q->desc = dmam_alloc_coherent(eth->dev, q->ndesc * sizeof(*q->desc),
+ q->desc = dmam_alloc_coherent(eth->dev, size * sizeof(*q->desc),
&dma_addr, GFP_KERNEL);
if (!q->desc)
return -ENOMEM;
- for (i = 0; i < q->ndesc; i++) {
+ for (i = 0; i < size; i++) {
u32 val = FIELD_PREP(QDMA_DESC_DONE_MASK, 1);
list_add_tail(&q->entry[i].list, &q->tx_list);
WRITE_ONCE(q->desc[i].ctrl, cpu_to_le32(val));
}
+ q->ndesc = size;
/* xmit ring drop default setting */
airoha_qdma_set(qdma, REG_TX_RING_BLOCKING(qid),
--
2.53.0
^ permalink raw reply related
* [PATCH net v3 0/3] net: airoha: Fix airoha_qdma_cleanup_tx_queue() processing
From: Lorenzo Bianconi @ 2026-04-16 7:27 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Lorenzo Bianconi, Simon Horman
Cc: linux-arm-kernel, linux-mediatek, netdev
Add missing bits in airoha_qdma_cleanup_tx_queue routine.
Fix airoha_qdma_cleanup_tx_queue processing errors intorduced in commit
'3f47e67dff1f7 ("net: airoha: Add the capability to consume out-of-order
DMA tx descriptors")'.
---
Changes in v3:
- Move ndesc initialization fix in a dedicated patch.
- Add patch 2/3 to move entries to queue head in case of DMA mapping
failure in airoha_dev_xmit().
- Cosmetics.
- Link to v2: https://lore.kernel.org/r/20260414-airoha_qdma_cleanup_tx_queue-fix-net-v2-1-875de57cc022@kernel.org
Changes in v2:
- Move q->ndesc initialization at end of airoha_qdma_init_tx routine in
order to avoid any possible NULL pointer dereference in
airoha_qdma_cleanup_tx_queue()
- Check if q->tx_list is empty in airoha_qdma_cleanup_tx_queue()
- Link to v1: https://lore.kernel.org/r/20260410-airoha_qdma_cleanup_tx_queue-fix-net-v1-1-b7171c8f1e78@kernel.org
---
Lorenzo Bianconi (3):
net: airoha: Move ndesc initialization at end of airoha_qdma_init_tx()
net: airoha: Move entries to queue head in case of DMA mapping failure in airoha_dev_xmit()
net: airoha: Add missing bits in airoha_qdma_cleanup_tx_queue()
drivers/net/ethernet/airoha/airoha_eth.c | 42 ++++++++++++++++++++++++++------
1 file changed, 35 insertions(+), 7 deletions(-)
---
base-commit: 3f20012a3964f487ae1e9ff942e2f35d4e9595bf
change-id: 20260410-airoha_qdma_cleanup_tx_queue-fix-net-93375f5ee80f
Best regards,
--
Lorenzo Bianconi <lorenzo@kernel.org>
^ permalink raw reply
* Re: [PATCH] arm_pmu: acpi: fix reference leak on failed device registration
From: Johan Hovold @ 2026-04-16 7:23 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Mark Rutland, Guangshuo Li, Will Deacon, Anshuman Khandual,
linux-arm-kernel, linux-perf-users, linux-kernel, stable
In-Reply-To: <2026041603-guts-crested-ef76@gregkh>
On Thu, Apr 16, 2026 at 06:40:55AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Apr 15, 2026 at 07:19:06PM +0100, Mark Rutland wrote:
> > AFAICT you're saying that the reference was taken *within*
> > platform_device_register(), and then platform_device_register() itself
> > has failed. I think it's surprising that platform_device_register()
> > doesn't clean that up itself in the case of an error.
> >
> > There are *tonnes* of calls to platform_device_register() throughout the
> > kernel that don't even bother to check the return value, and many that
> > just pass the return onto a caller that can't possibly know to call
> > platform_device_put().
> >
> > Code in the same file as platform_device_register() expects it to clean up
> > after itself, e.g.
> >
> > | int platform_add_devices(struct platform_device **devs, int num)
> > | {
> > | int i, ret = 0;
> > |
> > | for (i = 0; i < num; i++) {
> > | ret = platform_device_register(devs[i]);
> > | if (ret) {
> > | while (--i >= 0)
> > | platform_device_unregister(devs[i]);
> > | break;
> > | }
> > | }
> > |
> > | return ret;
> > | }
> >
> > That's been there since the initial git commit, and back then,
> > platform_device_register() didn't mention that callers needed to perform
> > any cleanup.
> >
> > I see a comment was added to platform_device_register() in commit:
> >
> > 67e532a42cf4 ("driver core: platform: document registration-failure requirement")
> >
> > ... and that copied the commend added for device_register() in commit:
> >
> > 5739411acbaa ("Driver core: Clarify device cleanup.")
> >
> > ... but the potential brokenness is so widespread, and the behaviour is
> > so surprising, that I'd argue the real but is that device_register()
> > doesn't clean up in case of error. I don't think it's worth changing
> > this single instance given the prevalance and churn fixing all of that
> > would involve.
> >
> > I think it would be far better to fix the core driver API such that when
> > those functions return an error, they've already cleaned up for
> > themselves.
> >
> > Greg, am I missing some functional reason why we can't rework
> > device_register() and friends to handle cleanup themselves? I appreciate
> > that'll involve churn for some callers, but AFAICT the majority of
> > callers don't have the required cleanup.
>
> Yes, we should fix the platform core code here, this should not be
> required to do everywhere as obviously we all got it wrong.
It's not just the platform code as this directly reflects the behaviour
of device_register() as Mark pointed out.
It is indeed an unfortunate quirk of the driver model, but one can argue
that having a registration function that frees its argument on errors
would be even worse. And even more so when many (or most) users get this
right.
So if we want to change this, I think we would need to deprecate
device_register() in favour of explicit device_initialize() and
device_add().
That said, most users of platform_device_register() appear to operate
on static platform devices which don't even have a release function and
would trigger a WARN() if we ever drop the reference (which is arguably
worse than leaking a tiny bit of memory).
So leaving things as-is is also an option.
Johan
^ permalink raw reply
* Re: [PATCH v5 04/12] coresight: etm4x: exclude ss_status from drvdata->config
From: Jie Gan @ 2026-04-16 7:20 UTC (permalink / raw)
To: Yeoreum Yun
Cc: coresight, linux-arm-kernel, linux-kernel, suzuki.poulose,
mike.leach, james.clark, alexander.shishkin, leo.yan
In-Reply-To: <aeCHnToKG43oKQNF@e129823.arm.com>
On 4/16/2026 2:54 PM, Yeoreum Yun wrote:
> Hi Jie,
>
Hi Yeoreum,
>>
>>
>> On 4/16/2026 12:55 AM, Yeoreum Yun wrote:
>>> The purpose of TRCSSCSRn register is to show status of
>>> the corresponding Single-shot Comparator Control and input supports.
>>> That means writable field's purpose for reset or restore from idle status
>>> not for configuration.
>>>
>>> Therefore, exclude ss_status from drvdata->config, move it to etm4x_caps
>>> and rename it to ss_smp.
>>>
>>> This includes remove TRCSSCRn from configurable item and
>>> remove saving in etm4_disable_hw().
>>>
>>> Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
>>> ---
>>> .../hwtracing/coresight/coresight-etm4x-cfg.c | 1 -
>>> .../coresight/coresight-etm4x-core.c | 19 ++++++-------------
>>> .../coresight/coresight-etm4x-sysfs.c | 7 ++-----
>>> drivers/hwtracing/coresight/coresight-etm4x.h | 7 ++++++-
>>> 4 files changed, 14 insertions(+), 20 deletions(-)
>>>
>>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> index c302072b293a..d14d7c8a23e5 100644
>>> --- a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
>>> @@ -86,7 +86,6 @@ static int etm4_cfg_map_reg_offset(struct etmv4_drvdata *drvdata,
>>> off_mask = (offset & GENMASK(11, 5));
>>> do {
>>> CHECKREGIDX(TRCSSCCRn(0), ss_ctrl, idx, off_mask);
>>> - CHECKREGIDX(TRCSSCSRn(0), ss_status, idx, off_mask);
>>> CHECKREGIDX(TRCSSPCICRn(0), ss_pe_cmp, idx, off_mask);
>>> } while (0);
>>> } else if ((offset >= TRCCIDCVRn(0)) && (offset <= TRCVMIDCVRn(7))) {
>>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> index b2b092a76eb5..f55338a4989d 100644
>>> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> @@ -91,7 +91,7 @@ static bool etm4x_sspcicrn_present(struct etmv4_drvdata *drvdata, int n)
>>> const struct etmv4_caps *caps = &drvdata->caps;
>>> return (n < caps->nr_ss_cmp) && caps->nr_pe_cmp &&
>>> - (drvdata->config.ss_status[n] & TRCSSCSRn_PC);
>>> + (caps->ss_cmp[n] & TRCSSCSRn_PC);
>>> }
>>> u64 etm4x_sysreg_read(u32 offset, bool _relaxed, bool _64bit)
>>> @@ -573,11 +573,9 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
>>> etm4x_relaxed_write32(csa, config->res_ctrl[i], TRCRSCTLRn(i));
>>> for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - /* always clear status bit on restart if using single-shot */
>>> - if (config->ss_ctrl[i] || config->ss_pe_cmp[i])
>>> - config->ss_status[i] &= ~TRCSSCSRn_STATUS;
>>> etm4x_relaxed_write32(csa, config->ss_ctrl[i], TRCSSCCRn(i));
>>> - etm4x_relaxed_write32(csa, config->ss_status[i], TRCSSCSRn(i));
>>> + /* always clear status and pending bits on restart if using single-shot */
>>> + etm4x_relaxed_write32(csa, 0x0, TRCSSCSRn(i));
>>> if (etm4x_sspcicrn_present(drvdata, i))
>>> etm4x_relaxed_write32(csa, config->ss_pe_cmp[i], TRCSSPCICRn(i));
>>> }
>>> @@ -1055,12 +1053,6 @@ static void etm4_disable_hw(struct etmv4_drvdata *drvdata)
>>> etm4_disable_trace_unit(drvdata);
>>> - /* read the status of the single shot comparators */
>>> - for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - config->ss_status[i] =
>>> - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> - }
>>> -
>>> /* read back the current counter values */
>>> for (i = 0; i < caps->nr_cntr; i++) {
>>> config->cntr_val[i] =
>>> @@ -1503,8 +1495,9 @@ static void etm4_init_arch_data(void *info)
>>> */
>>> caps->nr_ss_cmp = FIELD_GET(TRCIDR4_NUMSSCC_MASK, etmidr4);
>>> for (i = 0; i < caps->nr_ss_cmp; i++) {
>>> - drvdata->config.ss_status[i] =
>>> - etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> + caps->ss_cmp[i] = etm4x_relaxed_read32(csa, TRCSSCSRn(i));
>>> + caps->ss_cmp[i] &= (TRCSSCSRn_PC | TRCSSCSRn_DV |
>>> + TRCSSCSRn_DA | TRCSSCSRn_INST);
>>
>> Just re-go through this patch and had a question here:
>>
>> I’m not sure whether this new change should be documented in the ABI, given
>> that the TRCSSCSRn_STATUS bit is masked. In my opinion, this change breaks
>> the existing ABI description.
>>
>> Description from the ABI document:
>>
>> What: /sys/bus/coresight/devices/etm<N>/sshot_status
>> Date: December 2019
>> KernelVersion: 5.5
>> Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
>> Description: (Read) Print the current value of the selected single
>> shot status register.
>
> But, as I mentioned another thread:
> - https://lore.kernel.org/all/ad5yV2FoNbGGLE6R@e129823.arm.com/
>
> Till now, sysfs doesn't show the *current value* of the single shot
> state since the config->ss_status is updated enabled/disabled sysfs
> session. an I think once the session is disabled, other status bits
> (currently STATUS and PENDING bits) don't have any meaning.
>
> I think it's enough to change the doc's Description for this.
>
> Any thought?
Thanks for the info. I have missed this info, we are calling it
'comparator' instead of 'status' by focusing on the capability bits.
Make sense to me.
Thanks,
Jie
>
> --
> Sincerely,
> Yeoreum Yun
^ permalink raw reply
* [PATCH bpf v15 1/5] bpf: Move constants blinding out of arch-specific JITs
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
During the JIT stage, constants blinding rewrites instructions but only
rewrites the private instruction copy of the JITed subprog, leaving the
global env->prog->insnsi and env->insn_aux_data untouched. This causes a
mismatch between subprog instructions and the global state, making it
difficult to use the global data in the JIT.
To avoid this mismatch, and given that all arch-specific JITs already
support constants blinding, move it to the generic verifier code, and
switch to rewrite the global env->prog->insnsi with the global states
adjusted, as other rewrites in the verifier do.
This removes the constants blinding calls in each JIT, which are largely
duplicated code across architectures.
Since constants blinding is only required for JIT, and there are two
JIT entry functions, jit_subprogs() for BPF programs with multiple
subprogs and bpf_prog_select_runtime() for programs with no subprogs,
move the constants blinding invocation into these two functions.
In the verifier path, bpf_patch_insn_data() is used to keep global
verifier auxiliary data in sync with patched instructions. A key
question is whether this global auxiliary data should be restored
on the failure path.
Besides instructions, bpf_patch_insn_data() adjusts:
- prog->aux->poke_tab
- env->insn_array_maps
- env->subprog_info
- env->insn_aux_data
For prog->aux->poke_tab, it is only used by JIT or only meaningful after
JIT succeeds, so it does not need to be restored on the failure path.
For env->insn_array_maps, when JIT fails, programs using insn arrays
are rejected by bpf_insn_array_ready() due to missing JIT addresses.
Hence, env->insn_array_maps is only meaningful for JIT and does not need
to be restored.
For subprog_info, if jit_subprogs fails and CONFIG_BPF_JIT_ALWAYS_ON
is not enabled, kernel falls back to interpreter. In this case,
env->subprog_info is used to determine subprogram stack depth. So it
must be restored on failure.
For env->insn_aux_data, it is freed by clear_insn_aux_data() at the
end of bpf_check(). Before freeing, clear_insn_aux_data() loops over
env->insn_aux_data to release jump targets recorded in it. The loop
uses env->prog->len as the array length, but this length no longer
matches the actual size of the adjusted env->insn_aux_data array after
constants blinding.
To address it, a simple approach is to keep insn_aux_data as adjusted
after failure, since it will be freed shortly, and record its actual size
for the loop in clear_insn_aux_data(). But since clear_insn_aux_data()
uses the same index to loop over both env->prog->insnsi and env->insn_aux_data,
this approach results in incorrect index for the insnsi array. So an
alternative approach is adopted: clone the original env->insn_aux_data
before blinding and restore it after failure, similar to env->prog.
For classic BPF programs, constants blinding works as before since it
is still invoked from bpf_prog_select_runtime().
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Hari Bathini <hbathini@linux.ibm.com> # powerpc jit
Reviewed-by: Pu Lehui <pulehui@huawei.com> # riscv jit
Acked-by: Hengqi Chen <hengqi.chen@gmail.com> # loongarch jit
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 39 +++------
arch/arm/net/bpf_jit_32.c | 41 ++-------
arch/arm64/net/bpf_jit_comp.c | 72 +++++----------
arch/loongarch/net/bpf_jit.c | 59 ++++---------
arch/mips/net/bpf_jit_comp.c | 20 +----
arch/parisc/net/bpf_jit_core.c | 73 ++++++----------
arch/powerpc/net/bpf_jit_comp.c | 72 ++++++---------
arch/riscv/net/bpf_jit_core.c | 61 +++++--------
arch/s390/net/bpf_jit_comp.c | 59 +++++--------
arch/sparc/net/bpf_jit_comp_64.c | 61 +++++--------
arch/x86/net/bpf_jit_comp.c | 43 ++-------
arch/x86/net/bpf_jit_comp32.c | 33 +------
include/linux/filter.h | 33 ++++++-
kernel/bpf/core.c | 69 +++++++++++++--
kernel/bpf/fixups.c | 146 ++++++++++++++++++++++++++-----
15 files changed, 403 insertions(+), 478 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 1421eeced0f5..973ceae48675 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -79,7 +79,6 @@ struct arc_jit_data {
* The JIT pertinent context that is used by different functions.
*
* prog: The current eBPF program being handled.
- * orig_prog: The original eBPF program before any possible change.
* jit: The JIT buffer and its length.
* bpf_header: The JITed program header. "jit.buf" points inside it.
* emit: If set, opcodes are written to memory; else, a dry-run.
@@ -94,12 +93,10 @@ struct arc_jit_data {
* need_extra_pass: A forecast if an "extra_pass" will occur.
* is_extra_pass: Indicates if the current pass is an extra pass.
* user_bpf_prog: True, if VM opcodes come from a real program.
- * blinded: True if "constant blinding" step returned a new "prog".
* success: Indicates if the whole JIT went OK.
*/
struct jit_context {
struct bpf_prog *prog;
- struct bpf_prog *orig_prog;
struct jit_buffer jit;
struct bpf_binary_header *bpf_header;
bool emit;
@@ -114,7 +111,6 @@ struct jit_context {
bool need_extra_pass;
bool is_extra_pass;
bool user_bpf_prog;
- bool blinded;
bool success;
};
@@ -161,13 +157,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog)
{
memset(ctx, 0, sizeof(*ctx));
- ctx->orig_prog = prog;
-
- /* If constant blinding was requested but failed, scram. */
- ctx->prog = bpf_jit_blind_constants(prog);
- if (IS_ERR(ctx->prog))
- return PTR_ERR(ctx->prog);
- ctx->blinded = (ctx->prog != ctx->orig_prog);
+ ctx->prog = prog;
/* If the verifier doesn't zero-extend, then we have to do it. */
ctx->do_zext = !ctx->prog->aux->verifier_zext;
@@ -214,14 +204,6 @@ static inline void maybe_free(struct jit_context *ctx, void **mem)
*/
static void jit_ctx_cleanup(struct jit_context *ctx)
{
- if (ctx->blinded) {
- /* if all went well, release the orig_prog. */
- if (ctx->success)
- bpf_jit_prog_release_other(ctx->prog, ctx->orig_prog);
- else
- bpf_jit_prog_release_other(ctx->orig_prog, ctx->prog);
- }
-
maybe_free(ctx, (void **)&ctx->bpf2insn);
maybe_free(ctx, (void **)&ctx->jit_data);
@@ -229,12 +211,19 @@ static void jit_ctx_cleanup(struct jit_context *ctx)
ctx->bpf2insn_valid = false;
/* Freeing "bpf_header" is enough. "jit.buf" is a sub-array of it. */
- if (!ctx->success && ctx->bpf_header) {
- bpf_jit_binary_free(ctx->bpf_header);
- ctx->bpf_header = NULL;
- ctx->jit.buf = NULL;
- ctx->jit.index = 0;
- ctx->jit.len = 0;
+ if (!ctx->success) {
+ if (ctx->bpf_header) {
+ bpf_jit_binary_free(ctx->bpf_header);
+ ctx->bpf_header = NULL;
+ ctx->jit.buf = NULL;
+ ctx->jit.index = 0;
+ ctx->jit.len = 0;
+ }
+ if (ctx->is_extra_pass) {
+ ctx->prog->bpf_func = NULL;
+ ctx->prog->jited = 0;
+ ctx->prog->jited_len = 0;
+ }
}
ctx->emit = false;
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..e6b1bb2de627 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2144,9 +2144,7 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
- bool tmp_blinded = false;
struct jit_ctx ctx;
unsigned int tmp_idx;
unsigned int image_size;
@@ -2156,20 +2154,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- /* If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
-
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
@@ -2179,10 +2164,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* we must fall back to the interpreter
*/
ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
- if (ctx.offsets == NULL) {
- prog = orig_prog;
- goto out;
- }
+ if (ctx.offsets == NULL)
+ return prog;
/* 1) fake pass to find in the length of the JITed code,
* to compute ctx->offsets and other context variables
@@ -2194,10 +2177,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* being successful in the second pass, so just fall back
* to the interpreter.
*/
- if (build_body(&ctx)) {
- prog = orig_prog;
+ if (build_body(&ctx))
goto out_off;
- }
tmp_idx = ctx.idx;
build_prologue(&ctx);
@@ -2213,10 +2194,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx += ctx.imm_count;
if (ctx.imm_count) {
ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
- if (ctx.imms == NULL) {
- prog = orig_prog;
+ if (ctx.imms == NULL)
goto out_off;
- }
}
#else
/* there's nothing about the epilogue on ARMv7 */
@@ -2238,10 +2217,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Not able to allocate memory for the structure then
* we must fall back to the interpretation
*/
- if (header == NULL) {
- prog = orig_prog;
+ if (header == NULL)
goto out_imms;
- }
/* 2.) Actual pass to generate final JIT code */
ctx.target = (u32 *) image_ptr;
@@ -2278,16 +2255,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
#endif
out_off:
kfree(ctx.offsets);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free:
image_ptr = NULL;
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_imms;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 524b67c0867e..d310d1c35192 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2003,14 +2003,12 @@ struct arm64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header = NULL;
struct arm64_jit_data *jit_data;
void __percpu *priv_stack_ptr = NULL;
bool was_classic = bpf_prog_was_classic(prog);
int priv_stack_alloc_sz;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
@@ -2019,26 +2017,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int exentry_idx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -2050,10 +2035,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -2073,10 +2056,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kvzalloc_objs(int, prog->len + 1);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_off;
- }
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena);
@@ -2089,15 +2070,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* BPF line info needs ctx->offset[i] to be the offset of
* instruction[i] in jited image, so build prologue first.
*/
- if (build_prologue(&ctx, was_classic)) {
- prog = orig_prog;
+ if (build_prologue(&ctx, was_classic))
goto out_off;
- }
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_off;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx, was_classic);
@@ -2115,10 +2092,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_off;
- }
/* Pass 2: Determine jited position and result for each instruction */
@@ -2146,10 +2121,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free_hdr;
- }
ctx.epilogue_offset = ctx.idx;
ctx.exentry_idx = exentry_idx;
@@ -2158,19 +2131,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Pass 3: Adjust jump offset and write final image */
if (build_body(&ctx, extra_pass) ||
- WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset)) {
- prog = orig_prog;
+ WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
- }
build_epilogue(&ctx, was_classic);
build_plt(&ctx);
/* Extra pass to validate JITed code. */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free_hdr;
- }
/* update the real prog size */
prog_size = sizeof(u32) * ctx.idx;
@@ -2187,16 +2156,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (extra_pass && ctx.idx > jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d > %d\n",
ctx.idx, jit_data->ctx.idx);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
goto out_free_hdr;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header has been freed */
ro_header = NULL;
- prog = orig_prog;
- goto out_off;
+ header = NULL;
+ goto out_free_hdr;
}
} else {
jit_data->ctx = ctx;
@@ -2233,13 +2199,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size,
sizeof(header->size));
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 9cb796e16379..fcc8c0c29fb0 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1922,43 +1922,26 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- bool tmp_blinded = false, extra_pass = false;
+ bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
int image_size, prog_size, extable_size;
struct jit_ctx ctx;
struct jit_data *jit_data;
struct bpf_binary_header *header;
struct bpf_binary_header *ro_header;
- struct bpf_prog *tmp, *orig_prog = prog;
/*
* If BPF JIT was not enabled then we must fall back to
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- if (IS_ERR(tmp))
- return orig_prog;
-
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1978,17 +1961,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx.offset = kvcalloc(prog->len + 1, sizeof(u32), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
+ if (ctx.offset == NULL)
goto out_offset;
- }
/* 1. Initial fake pass to compute ctx->idx and set ctx->flags */
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_offset;
- }
ctx.epilogue_offset = ctx.idx;
build_epilogue(&ctx);
@@ -2004,10 +1983,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
&header, &image_ptr, jit_fill_hole);
- if (!ro_header) {
- prog = orig_prog;
+ if (!ro_header)
goto out_offset;
- }
/* 2. Now, the actual pass to generate final JIT code */
/*
@@ -2027,17 +2004,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.num_exentries = 0;
build_prologue(&ctx);
- if (build_body(&ctx, extra_pass)) {
- prog = orig_prog;
+ if (build_body(&ctx, extra_pass))
goto out_free;
- }
build_epilogue(&ctx);
/* 3. Extra pass to validate JITed code */
- if (validate_ctx(&ctx)) {
- prog = orig_prog;
+ if (validate_ctx(&ctx))
goto out_free;
- }
/* And we're done */
if (bpf_jit_enable > 1)
@@ -2050,9 +2023,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_free;
}
if (WARN_ON(bpf_jit_binary_pack_finalize(ro_header, header))) {
- /* ro_header has been freed */
+ /* ro_header and header have been freed */
ro_header = NULL;
- prog = orig_prog;
+ header = NULL;
goto out_free;
}
/*
@@ -2084,13 +2057,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ? tmp : orig_prog);
-
return prog;
out_free:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+
if (header) {
bpf_arch_text_copy(&ro_header->size, &header->size, sizeof(header->size));
bpf_jit_binary_pack_free(ro_header, header);
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index e355dfca4400..d2b6c955f18e 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -911,10 +911,8 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
- bool tmp_blinded = false;
unsigned int tmp_idx;
unsigned int image_size;
u8 *image_ptr;
@@ -925,19 +923,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
* the interpreter.
*/
if (!prog->jit_requested)
- return orig_prog;
- /*
- * If constant blinding was enabled and we failed during blinding
- * then we must fall back to the interpreter. Otherwise, we save
- * the new JITed code.
- */
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
memset(&ctx, 0, sizeof(ctx));
ctx.program = prog;
@@ -1025,14 +1011,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->jited_len = image_size;
out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
kfree(ctx.descriptors);
return prog;
out_err:
- prog = orig_prog;
if (header)
bpf_jit_binary_free(header);
goto out;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index a5eb6b51e27a..35dca372b5df 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -44,30 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, prologue_len, i;
struct hppa_jit_data *jit_data;
struct hppa_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
@@ -81,10 +70,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!ctx->offset)
+ goto out_err;
for (i = 0; i < prog->len; i++) {
prev_ninsns += 20;
ctx->offset[i] = prev_ninsns;
@@ -93,10 +80,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
for (i = 0; i < NR_JIT_ITERATIONS; i++) {
pass++;
ctx->ninsns = 0;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (build_body(ctx, extra_pass, ctx->offset))
+ goto out_err;
ctx->body_len = ctx->ninsns;
bpf_jit_build_prologue(ctx);
ctx->prologue_len = ctx->ninsns - ctx->body_len;
@@ -116,10 +101,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->image,
sizeof(long),
bpf_fill_ill_insns);
- if (!jit_data->header) {
- prog = orig_prog;
- goto out_offset;
- }
+ if (!jit_data->header)
+ goto out_err;
ctx->insns = (u32 *)jit_data->image;
/*
@@ -134,8 +117,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
if (jit_data->header)
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
if (extable_size)
@@ -148,8 +130,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx);
if (build_body(ctx, extra_pass, NULL)) {
bpf_jit_binary_free(jit_data->header);
- prog = orig_prog;
- goto out_offset;
+ goto out_err;
}
bpf_jit_build_epilogue(ctx);
@@ -160,20 +141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{ extern int machine_restart(char *); machine_restart(""); }
}
+ if (!prog->is_func || extra_pass) {
+ if (bpf_jit_binary_lock_ro(jit_data->header)) {
+ bpf_jit_binary_free(jit_data->header);
+ goto out_err;
+ }
+ bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
+ }
+
prog->bpf_func = (void *)ctx->insns;
prog->jited = 1;
prog->jited_len = prog_size;
- bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
-
if (!prog->is_func || extra_pass) {
- if (bpf_jit_binary_lock_ro(jit_data->header)) {
- bpf_jit_binary_free(jit_data->header);
- prog->bpf_func = NULL;
- prog->jited = 0;
- prog->jited_len = 0;
- goto out_offset;
- }
prologue_len = ctx->epilogue_offset - ctx->body_len;
for (i = 0; i < prog->len; i++)
ctx->offset[i] += prologue_len;
@@ -183,14 +163,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
+
if (HPPA_JIT_REBOOT)
{ extern int machine_restart(char *); machine_restart(""); }
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_offset;
}
u64 hppa_div64(u64 div, u64 divisor)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 50103b3794fb..2bae4699e78f 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -177,9 +177,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
void __percpu *priv_stack_ptr = NULL;
struct bpf_binary_header *fhdr = NULL;
struct bpf_binary_header *hdr = NULL;
- struct bpf_prog *org_fp = fp;
- struct bpf_prog *tmp_fp = NULL;
- bool bpf_blinded = false;
bool extra_pass = false;
u8 *fimage = NULL;
u32 *fcode_base = NULL;
@@ -187,24 +184,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u32 fixup_len;
if (!fp->jit_requested)
- return org_fp;
-
- tmp_fp = bpf_jit_blind_constants(org_fp);
- if (IS_ERR(tmp_fp))
- return org_fp;
-
- if (tmp_fp != org_fp) {
- bpf_blinded = true;
- fp = tmp_fp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = org_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
@@ -219,10 +205,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
priv_stack_alloc_size = round_up(fp->aux->stack_depth, 16) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_size, 16, GFP_KERNEL);
- if (!priv_stack_ptr) {
- fp = org_fp;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_size);
fp->aux->priv_stack_ptr = priv_stack_ptr;
@@ -249,10 +233,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
addrs = kcalloc(flen + 1, sizeof(*addrs), GFP_KERNEL);
- if (addrs == NULL) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (addrs == NULL)
+ goto out_err;
memset(&cgctx, 0, sizeof(struct codegen_context));
bpf_jit_init_reg_mapping(&cgctx);
@@ -279,11 +261,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
}
/* Scouting faux-generate pass 0 */
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
/* We hit something illegal or unsupported. */
- fp = org_fp;
- goto out_addrs;
- }
+ goto out_err;
/*
* If we have seen a tail call, we need a second pass.
@@ -294,10 +274,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
- if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
+ goto out_err;
}
bpf_jit_realloc_regs(&cgctx);
@@ -318,10 +296,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
bpf_jit_fill_ill_insns);
- if (!fhdr) {
- fp = org_fp;
- goto out_addrs;
- }
+ if (!fhdr)
+ goto out_err;
if (extable_len)
fp->aux->extable = (void *)fimage + FUNCTION_DESCR_SIZE + proglen + fixup_len;
@@ -340,8 +316,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
extra_pass)) {
bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
bpf_jit_binary_pack_free(fhdr, hdr);
- fp = org_fp;
- goto out_addrs;
+ goto out_err;
}
bpf_jit_build_epilogue(code_base, &cgctx);
@@ -363,15 +338,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
((u64 *)image)[1] = local_paca->kernel_toc;
#endif
+ if (!fp->is_func || extra_pass) {
+ if (bpf_jit_binary_pack_finalize(fhdr, hdr))
+ goto out_err;
+ }
+
fp->bpf_func = (void *)fimage;
fp->jited = 1;
fp->jited_len = cgctx.idx * 4 + FUNCTION_DESCR_SIZE;
if (!fp->is_func || extra_pass) {
- if (bpf_jit_binary_pack_finalize(fhdr, hdr)) {
- fp = org_fp;
- goto out_addrs;
- }
bpf_prog_fill_jited_linfo(fp, addrs);
/*
* On ABI V1, executable code starts after the function
@@ -398,11 +374,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
jit_data->hdr = hdr;
}
-out:
- if (bpf_blinded)
- bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);
-
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto out_addrs;
}
/*
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index f7fd4afc3ca3..36f0aea8096d 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -44,29 +44,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
- bool tmp_blinded = false, extra_pass = false;
- struct bpf_prog *tmp, *orig_prog = prog;
+ bool extra_pass = false;
int pass = 0, prev_ninsns = 0, i;
struct rv_jit_data *jit_data;
struct rv_jit_context *ctx;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
if (!jit_data) {
- prog = orig_prog;
- goto out;
+ return prog;
}
prog->aux->jit_data = jit_data;
}
@@ -83,15 +73,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena);
ctx->prog = prog;
ctx->offset = kzalloc_objs(int, prog->len);
- if (!ctx->offset) {
- prog = orig_prog;
+ if (!ctx->offset)
goto out_offset;
- }
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_offset;
- }
for (i = 0; i < prog->len; i++) {
prev_ninsns += 32;
@@ -105,10 +91,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
ctx->prologue_len = ctx->ninsns;
- if (build_body(ctx, extra_pass, ctx->offset)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, ctx->offset))
goto out_offset;
- }
ctx->epilogue_offset = ctx->ninsns;
bpf_jit_build_epilogue(ctx);
@@ -126,10 +110,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
bpf_fill_ill_insns);
- if (!jit_data->ro_header) {
- prog = orig_prog;
+ if (!jit_data->ro_header)
goto out_offset;
- }
/*
* Use the image(RW) for writing the JITed instructions. But also save
@@ -150,7 +132,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (i == NR_JIT_ITERATIONS) {
pr_err("bpf-jit: image did not converge in <%d passes!\n", i);
- prog = orig_prog;
goto out_free_hdr;
}
@@ -163,26 +144,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx->nexentries = 0;
bpf_jit_build_prologue(ctx, bpf_is_subprog(prog));
- if (build_body(ctx, extra_pass, NULL)) {
- prog = orig_prog;
+ if (build_body(ctx, extra_pass, NULL))
goto out_free_hdr;
- }
bpf_jit_build_epilogue(ctx);
if (bpf_jit_enable > 1)
bpf_jit_dump(prog->len, prog_size, pass, ctx->insns);
- prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
- prog->jited = 1;
- prog->jited_len = prog_size - cfi_get_offset();
-
if (!prog->is_func || extra_pass) {
if (WARN_ON(bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header))) {
/* ro_header has been freed */
jit_data->ro_header = NULL;
- prog = orig_prog;
- goto out_offset;
+ jit_data->header = NULL;
+ goto out_free_hdr;
}
+ }
+
+ prog->bpf_func = (void *)ctx->ro_insns + cfi_get_offset();
+ prog->jited = 1;
+ prog->jited_len = prog_size - cfi_get_offset();
+
+ if (!prog->is_func || extra_pass) {
for (i = 0; i < prog->len; i++)
ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
bpf_prog_fill_jited_linfo(prog, ctx->offset);
@@ -191,14 +173,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
out_free_hdr:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
if (jit_data->header) {
bpf_arch_text_copy(&jit_data->ro_header->size, &jit_data->header->size,
sizeof(jit_data->header->size));
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index d08d159b6319..2dfc279b1be2 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2314,36 +2314,20 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
*/
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
- struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
- bool tmp_blinded = false;
bool extra_pass = false;
struct bpf_jit jit;
int pass;
if (!fp->jit_requested)
- return orig_fp;
-
- tmp = bpf_jit_blind_constants(fp);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_fp;
- if (tmp != fp) {
- tmp_blinded = true;
- fp = tmp;
- }
+ return fp;
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- fp = orig_fp;
- goto out;
- }
+ if (!jit_data)
+ return fp;
fp->aux->jit_data = jit_data;
}
if (jit_data->ctx.addrs) {
@@ -2356,34 +2340,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
memset(&jit, 0, sizeof(jit));
jit.addrs = kvcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
- if (jit.addrs == NULL) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (jit.addrs == NULL)
+ goto out_err;
/*
* Three initial passes:
* - 1/2: Determine clobbered registers
* - 3: Calculate program size and addrs array
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp, extra_pass)) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (bpf_jit_prog(&jit, fp, extra_pass))
+ goto out_err;
}
/*
* Final pass: Allocate and generate program
*/
header = bpf_jit_alloc(&jit, fp);
- if (!header) {
- fp = orig_fp;
- goto free_addrs;
- }
+ if (!header)
+ goto out_err;
skip_init_ctx:
if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
if (bpf_jit_enable > 1) {
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
@@ -2392,8 +2369,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
if (!fp->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- fp = orig_fp;
- goto free_addrs;
+ goto out_err;
}
} else {
jit_data->header = header;
@@ -2411,11 +2387,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
kfree(jit_data);
fp->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(fp, fp == orig_fp ?
- tmp : orig_fp);
+
return fp;
+
+out_err:
+ if (extra_pass) {
+ fp->bpf_func = NULL;
+ fp->jited = 0;
+ fp->jited_len = 0;
+ }
+ goto free_addrs;
}
bool bpf_jit_supports_kfunc_call(void)
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index b23d1c645ae5..e83e29137566 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1479,37 +1479,22 @@ struct sparc64_jit_data {
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
- struct bpf_prog *tmp, *orig_prog = prog;
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
u32 prev_image_size, image_size;
- bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
u8 *image_ptr;
int pass, i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /* If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
@@ -1527,10 +1512,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.prog = prog;
ctx.offset = kmalloc_array(prog->len, sizeof(unsigned int), GFP_KERNEL);
- if (ctx.offset == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (ctx.offset == NULL)
+ goto out_err;
/* Longest sequence emitted is for bswap32, 12 instructions. Pre-cook
* the offset array so that we converge faster.
@@ -1543,10 +1526,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
ctx.idx = 0;
build_prologue(&ctx);
- if (build_body(&ctx)) {
- prog = orig_prog;
- goto out_off;
- }
+ if (build_body(&ctx))
+ goto out_err;
build_epilogue(&ctx);
if (bpf_jit_enable > 1)
@@ -1569,10 +1550,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = sizeof(u32) * ctx.idx;
header = bpf_jit_binary_alloc(image_size, &image_ptr,
sizeof(u32), jit_fill_hole);
- if (header == NULL) {
- prog = orig_prog;
- goto out_off;
- }
+ if (header == NULL)
+ goto out_err;
ctx.image = (u32 *)image_ptr;
skip_init_ctx:
@@ -1582,8 +1561,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (build_body(&ctx)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
build_epilogue(&ctx);
@@ -1592,8 +1570,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
pr_err("bpf_jit: Failed to converge, prev_size=%u size=%d\n",
prev_image_size, ctx.idx * 4);
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
if (bpf_jit_enable > 1)
@@ -1604,8 +1581,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (!prog->is_func || extra_pass) {
if (bpf_jit_binary_lock_ro(header)) {
bpf_jit_binary_free(header);
- prog = orig_prog;
- goto out_off;
+ goto out_err;
}
} else {
jit_data->ctx = ctx;
@@ -1624,9 +1600,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
+
+out_err:
+ if (extra_pass) {
+ prog->bpf_func = NULL;
+ prog->jited = 0;
+ prog->jited_len = 0;
+ }
+ goto out_off;
}
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index e9b78040d703..77d00a8dec87 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3717,13 +3717,11 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
void __percpu *priv_stack_ptr = NULL;
struct x64_jit_data *jit_data;
int priv_stack_alloc_sz;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
bool extra_pass = false;
bool padding = false;
u8 *rw_image = NULL;
@@ -3733,27 +3731,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc_obj(*jit_data);
- if (!jit_data) {
- prog = orig_prog;
- goto out;
- }
+ if (!jit_data)
+ return prog;
prog->aux->jit_data = jit_data;
}
priv_stack_ptr = prog->aux->priv_stack_ptr;
@@ -3765,10 +3749,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
priv_stack_alloc_sz = round_up(prog->aux->stack_depth, 8) +
2 * PRIV_STACK_GUARD_SZ;
priv_stack_ptr = __alloc_percpu_gfp(priv_stack_alloc_sz, 8, GFP_KERNEL);
- if (!priv_stack_ptr) {
- prog = orig_prog;
+ if (!priv_stack_ptr)
goto out_priv_stack;
- }
priv_stack_init_guard(priv_stack_ptr, priv_stack_alloc_sz);
prog->aux->priv_stack_ptr = priv_stack_ptr;
@@ -3786,10 +3768,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto skip_init_addrs;
}
addrs = kvmalloc_objs(*addrs, prog->len + 1);
- if (!addrs) {
- prog = orig_prog;
+ if (!addrs)
goto out_addrs;
- }
/*
* Before first pass, make a rough estimation of addrs[]
@@ -3820,8 +3800,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
sizeof(rw_header->size));
bpf_jit_binary_pack_free(header, rw_header);
}
- /* Fall back to interpreter mode */
- prog = orig_prog;
if (extra_pass) {
prog->bpf_func = NULL;
prog->jited = 0;
@@ -3852,10 +3830,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
prog->aux->extable = (void *) image + roundup(proglen, align);
}
oldproglen = proglen;
@@ -3908,8 +3884,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image + cfi_get_offset();
prog->jited = 1;
prog->jited_len = proglen - cfi_get_offset();
- } else {
- prog = orig_prog;
}
if (!image || !prog->is_func || extra_pass) {
@@ -3925,10 +3899,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
kfree(jit_data);
prog->aux->jit_data = NULL;
}
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
+
return prog;
}
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index dda423025c3d..5f259577614a 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2521,35 +2521,19 @@ bool bpf_jit_needs_zext(void)
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
- struct bpf_prog *tmp, *orig_prog = prog;
int proglen, oldproglen = 0;
struct jit_context ctx = {};
- bool tmp_blinded = false;
u8 *image = NULL;
int *addrs;
int pass;
int i;
if (!prog->jit_requested)
- return orig_prog;
-
- tmp = bpf_jit_blind_constants(prog);
- /*
- * If blinding was requested and we failed during blinding,
- * we must fall back to the interpreter.
- */
- if (IS_ERR(tmp))
- return orig_prog;
- if (tmp != prog) {
- tmp_blinded = true;
- prog = tmp;
- }
+ return prog;
addrs = kmalloc_objs(*addrs, prog->len);
- if (!addrs) {
- prog = orig_prog;
- goto out;
- }
+ if (!addrs)
+ return prog;
/*
* Before first pass, make a rough estimation of addrs[]
@@ -2574,7 +2558,6 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image = NULL;
if (header)
bpf_jit_binary_free(header);
- prog = orig_prog;
goto out_addrs;
}
if (image) {
@@ -2588,10 +2571,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (proglen == oldproglen) {
header = bpf_jit_binary_alloc(proglen, &image,
1, jit_fill_hole);
- if (!header) {
- prog = orig_prog;
+ if (!header)
goto out_addrs;
- }
}
oldproglen = proglen;
cond_resched();
@@ -2604,16 +2585,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog->bpf_func = (void *)image;
prog->jited = 1;
prog->jited_len = proglen;
- } else {
- prog = orig_prog;
}
out_addrs:
kfree(addrs);
-out:
- if (tmp_blinded)
- bpf_jit_prog_release_other(prog, prog == orig_prog ?
- tmp : orig_prog);
return prog;
}
diff --git a/include/linux/filter.h b/include/linux/filter.h
index f552170eacf4..9fa4d4090093 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1184,6 +1184,18 @@ static inline bool bpf_dump_raw_ok(const struct cred *cred)
struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
const struct bpf_insn *patch, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len);
+#else
+static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return ERR_PTR(-ENOTSUPP);
+}
+#endif /* CONFIG_BPF_SYSCALL */
+
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
static inline bool xdp_return_frame_no_direct(void)
@@ -1310,9 +1322,14 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
const char *bpf_jit_get_prog_name(struct bpf_prog *prog);
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *fp);
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other);
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return prog->blinding_requested && !prog->blinded;
+}
+
static inline void bpf_jit_dump(unsigned int flen, unsigned int proglen,
u32 pass, void *image)
{
@@ -1451,6 +1468,20 @@ static inline void bpf_prog_kallsyms_del(struct bpf_prog *fp)
{
}
+static inline bool bpf_prog_need_blind(const struct bpf_prog *prog)
+{
+ return false;
+}
+
+static inline
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
+{
+ return prog;
+}
+
+static inline void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
+{
+}
#endif /* CONFIG_BPF_JIT */
void bpf_prog_kallsyms_del_all(struct bpf_prog *fp);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 066b86e7233c..fc9fb3c07866 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1508,7 +1508,11 @@ static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
#endif
}
-struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
+/*
+ * Now this function is used only to blind the main prog and must be invoked only when
+ * bpf_prog_need_blind() returns true.
+ */
+struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_insn insn_buff[16], aux[2];
struct bpf_prog *clone, *tmp;
@@ -1516,13 +1520,17 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
struct bpf_insn *insn;
int i, rewritten;
- if (!prog->blinding_requested || prog->blinded)
- return prog;
+ if (WARN_ON_ONCE(env && env->prog != prog))
+ return ERR_PTR(-EINVAL);
clone = bpf_prog_clone_create(prog, GFP_USER);
if (!clone)
return ERR_PTR(-ENOMEM);
+ /* make sure bpf_patch_insn_data() patches the correct prog */
+ if (env)
+ env->prog = clone;
+
insn_cnt = clone->len;
insn = clone->insnsi;
@@ -1550,21 +1558,35 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
if (!rewritten)
continue;
- tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
- if (IS_ERR(tmp)) {
+ if (env)
+ tmp = bpf_patch_insn_data(env, i, insn_buff, rewritten);
+ else
+ tmp = bpf_patch_insn_single(clone, i, insn_buff, rewritten);
+
+ if (IS_ERR_OR_NULL(tmp)) {
+ if (env)
+ /* restore the original prog */
+ env->prog = prog;
/* Patching may have repointed aux->prog during
* realloc from the original one, so we need to
* fix it up here on error.
*/
bpf_jit_prog_release_other(prog, clone);
- return tmp;
+ return IS_ERR(tmp) ? tmp : ERR_PTR(-ENOMEM);
}
clone = tmp;
insn_delta = rewritten - 1;
- /* Instructions arrays must be updated using absolute xlated offsets */
- adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+ if (env)
+ env->prog = clone;
+ else
+ /*
+ * Instructions arrays must be updated using absolute xlated offsets.
+ * The arrays have already been adjusted by bpf_patch_insn_data() when
+ * env is not NULL.
+ */
+ adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2533,6 +2555,35 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+{
+#ifdef CONFIG_BPF_JIT
+ struct bpf_prog *orig_prog;
+
+ if (!bpf_prog_need_blind(prog))
+ return bpf_int_jit_compile(prog);
+
+ orig_prog = prog;
+ prog = bpf_jit_blind_constants(NULL, prog);
+ /*
+ * If blinding was requested and we failed during blinding, we must fall
+ * back to the interpreter.
+ */
+ if (IS_ERR(prog))
+ return orig_prog;
+
+ prog = bpf_int_jit_compile(prog);
+ if (prog->jited) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ return prog;
+ }
+
+ bpf_jit_prog_release_other(orig_prog, prog);
+ prog = orig_prog;
+#endif
+ return prog;
+}
+
/**
* bpf_prog_select_runtime - select exec runtime for BPF program
* @fp: bpf_prog populated with BPF program
@@ -2572,7 +2623,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_int_jit_compile(fp);
+ fp = bpf_prog_jit_compile(fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index dd00a680e4ea..721b830b5ef2 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -232,8 +232,8 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-static struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -973,7 +973,47 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
return 0;
}
-int bpf_jit_subprogs(struct bpf_verifier_env *env)
+static u32 *bpf_dup_subprog_starts(struct bpf_verifier_env *env)
+{
+ u32 *starts = NULL;
+
+ starts = kvmalloc_objs(u32, env->subprog_cnt, GFP_KERNEL_ACCOUNT);
+ if (starts) {
+ for (int i = 0; i < env->subprog_cnt; i++)
+ starts[i] = env->subprog_info[i].start;
+ }
+ return starts;
+}
+
+static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)
+{
+ for (int i = 0; i < env->subprog_cnt; i++)
+ env->subprog_info[i].start = orig_starts[i];
+ /* restore the start of fake 'exit' subprog as well */
+ env->subprog_info[env->subprog_cnt].start = env->prog->len;
+}
+
+static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ size_t size;
+ void *new_aux;
+
+ size = array_size(sizeof(struct bpf_insn_aux_data), env->prog->len);
+ new_aux = __vmalloc(size, GFP_KERNEL_ACCOUNT);
+ if (new_aux)
+ memcpy(new_aux, env->insn_aux_data, size);
+ return new_aux;
+}
+
+static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+ /* the expanded elements are zero-filled, so no special handling is required */
+ vfree(env->insn_aux_data);
+ env->insn_aux_data = orig_insn_aux;
+}
+
+static int jit_subprogs(struct bpf_verifier_env *env)
{
struct bpf_prog *prog = env->prog, **func, *tmp;
int i, j, subprog_start, subprog_end = 0, len, subprog;
@@ -981,10 +1021,6 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
struct bpf_insn *insn;
void *old_bpf_func;
int err, num_exentries;
- int old_len, subprog_start_adjustment = 0;
-
- if (env->subprog_cnt <= 1)
- return 0;
for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
if (!bpf_pseudo_func(insn) && !bpf_pseudo_call(insn))
@@ -1053,10 +1089,11 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
goto out_free;
func[i]->is_func = 1;
func[i]->sleepable = prog->sleepable;
+ func[i]->blinded = prog->blinded;
func[i]->aux->func_idx = i;
/* Below members will be freed only at prog->aux */
func[i]->aux->btf = prog->aux->btf;
- func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
+ func[i]->aux->subprog_start = subprog_start;
func[i]->aux->func_info = prog->aux->func_info;
func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -1113,15 +1150,7 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->token = prog->aux->token;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
-
- /*
- * To properly pass the absolute subprog start to jit
- * all instruction adjustments should be accumulated
- */
- old_len = func[i]->len;
func[i] = bpf_int_jit_compile(func[i]);
- subprog_start_adjustment += func[i]->len - old_len;
-
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1247,16 +1276,87 @@ int bpf_jit_subprogs(struct bpf_verifier_env *env)
}
kfree(func);
out_undo_insn:
+ bpf_prog_jit_attempt_done(prog);
+ return err;
+}
+
+int bpf_jit_subprogs(struct bpf_verifier_env *env)
+{
+ int err, i;
+ bool blinded = false;
+ struct bpf_insn *insn;
+ struct bpf_prog *prog, *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
+ u32 *orig_subprog_starts;
+
+ if (env->subprog_cnt <= 1)
+ return 0;
+
+ prog = orig_prog = env->prog;
+ if (bpf_prog_need_blind(prog)) {
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux) {
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ orig_subprog_starts = bpf_dup_subprog_starts(env);
+ if (!orig_subprog_starts) {
+ vfree(orig_insn_aux);
+ err = -ENOMEM;
+ goto out_cleanup;
+ }
+ prog = bpf_jit_blind_constants(env, prog);
+ if (IS_ERR(prog)) {
+ err = -ENOMEM;
+ prog = orig_prog;
+ goto out_restore;
+ }
+ blinded = true;
+ }
+
+ err = jit_subprogs(env);
+ if (err)
+ goto out_jit_err;
+
+ if (blinded) {
+ bpf_jit_prog_release_other(prog, orig_prog);
+ kvfree(orig_subprog_starts);
+ vfree(orig_insn_aux);
+ }
+
+ return 0;
+
+out_jit_err:
+ if (blinded) {
+ bpf_jit_prog_release_other(orig_prog, prog);
+ /* roll back to the clean original prog */
+ prog = env->prog = orig_prog;
+ goto out_restore;
+ } else {
+ if (err != -EFAULT) {
+ /*
+ * We will fall back to interpreter mode when err is not -EFAULT, before
+ * that, insn->off and insn->imm should be restored to their original
+ * values since they were modified by jit_subprogs.
+ */
+ for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
+ if (!bpf_pseudo_call(insn))
+ continue;
+ insn->off = 0;
+ insn->imm = env->insn_aux_data[i].call_imm;
+ }
+ }
+ goto out_cleanup;
+ }
+
+out_restore:
+ bpf_restore_subprog_starts(env, orig_subprog_starts);
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
+ kvfree(orig_subprog_starts);
+out_cleanup:
/* cleanup main prog to be interpreted */
prog->jit_requested = 0;
prog->blinding_requested = 0;
- for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
- if (!bpf_pseudo_call(insn))
- continue;
- insn->off = 0;
- insn->imm = env->insn_aux_data[i].call_imm;
- }
- bpf_prog_jit_attempt_done(prog);
return err;
}
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 5/5] bpf, arm64: Emit BTI for indirect jump target
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support BTI, the indirect jump selftest triggers a kernel
panic because there is no BTI instructions at the indirect jump targets.
Fix it by emitting a BTI instruction for each indirect jump target.
For reference, below is a sample panic log.
Internal error: Oops - BTI: 0000000036000003 [#1] SMP
...
Call trace:
bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x54/0xf8 (P)
bpf_prog_run_pin_on_cpu+0x140/0x468
bpf_prog_test_run_syscall+0x280/0x3b8
bpf_prog_test_run+0x22c/0x2c0
Fixes: f4a66cf1cb14 ("bpf: arm64: Add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index bd8757952507..0816c40fc7af 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1197,8 +1197,8 @@ static int add_exception_handler(const struct bpf_insn *insn,
* >0 - successfully JITed a 16-byte eBPF instruction.
* <0 - failed to JIT.
*/
-static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
- bool extra_pass)
+static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn *insn,
+ struct jit_ctx *ctx, bool extra_pass)
{
const u8 code = insn->code;
u8 dst = bpf2a64[insn->dst_reg];
@@ -1223,6 +1223,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
int ret;
bool sign_extend;
+ if (bpf_insn_is_indirect_target(env, ctx->prog, i))
+ emit_bti(A64_BTI_J, ctx);
+
switch (code) {
/* dst = src */
case BPF_ALU | BPF_MOV | BPF_X:
@@ -1898,7 +1901,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
-static int build_body(struct jit_ctx *ctx, bool extra_pass)
+static int build_body(struct bpf_verifier_env *env, struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
int i;
@@ -1917,7 +1920,7 @@ static int build_body(struct jit_ctx *ctx, bool extra_pass)
int ret;
ctx->offset[i] = ctx->idx;
- ret = build_insn(insn, ctx, extra_pass);
+ ret = build_insn(env, insn, ctx, extra_pass);
if (ret > 0) {
i++;
ctx->offset[i] = ctx->idx;
@@ -2073,7 +2076,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
if (build_prologue(&ctx, was_classic))
goto out_off;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_off;
ctx.epilogue_offset = ctx.idx;
@@ -2121,7 +2124,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* Dont write body instructions to memory for now */
ctx.write = false;
- if (build_body(&ctx, extra_pass))
+ if (build_body(env, &ctx, extra_pass))
goto out_free_hdr;
ctx.epilogue_offset = ctx.idx;
@@ -2130,7 +2133,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
ctx.write = true;
/* Pass 3: Adjust jump offset and write final image */
- if (build_body(&ctx, extra_pass) ||
+ if (build_body(env, &ctx, extra_pass) ||
WARN_ON_ONCE(ctx.idx != ctx.epilogue_offset))
goto out_free_hdr;
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 2/5] bpf: Pass bpf_verifier_env to JIT
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will
use env->insn_aux_data in the JIT stage to detect indirect jump targets.
Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c
code without verifier, introduce helper __bpf_prog_select_runtime()
to accept the env parameter.
Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and
switch to call __bpf_prog_select_runtime() in the verifier, with env
variable passed. The original bpf_prog_select_runtime() is preserved for
cbpf and lib/test_bpf.c, where env is NULL.
Now all constants blinding calls are moved into the verifier, except
the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted
by bpf_patch_insn_data() function for normal cases, so there is no need
to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Hengqi Chen <hengqi.chen@gmail.com> # v14
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arc/net/bpf_jit_core.c | 2 +-
arch/arm/net/bpf_jit_32.c | 2 +-
arch/arm64/net/bpf_jit_comp.c | 2 +-
arch/loongarch/net/bpf_jit.c | 2 +-
arch/mips/net/bpf_jit_comp.c | 2 +-
arch/parisc/net/bpf_jit_core.c | 2 +-
arch/powerpc/net/bpf_jit_comp.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 2 +-
arch/s390/net/bpf_jit_comp.c | 2 +-
arch/sparc/net/bpf_jit_comp_64.c | 2 +-
arch/x86/net/bpf_jit_comp.c | 2 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
include/linux/filter.h | 17 ++++++-
kernel/bpf/core.c | 86 ++++++++++++++++----------------
kernel/bpf/fixups.c | 10 ++--
kernel/bpf/syscall.c | 4 --
kernel/bpf/verifier.c | 14 +++---
17 files changed, 84 insertions(+), 71 deletions(-)
diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c
index 973ceae48675..639a2736f029 100644
--- a/arch/arc/net/bpf_jit_core.c
+++ b/arch/arc/net/bpf_jit_core.c
@@ -1400,7 +1400,7 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog)
* (re)locations involved that their addresses are not known
* during the first run.
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
vm_dump(prog);
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index e6b1bb2de627..1628b6fc70a4 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -2142,7 +2142,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header;
struct jit_ctx ctx;
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index d310d1c35192..bd8757952507 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2000,7 +2000,7 @@ struct arm64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
int image_size, prog_size, extable_size, extable_align, extable_offset;
struct bpf_binary_header *header;
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index fcc8c0c29fb0..5149ce4cef7e 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1920,7 +1920,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
bool extra_pass = false;
u8 *image_ptr, *ro_image_ptr;
diff --git a/arch/mips/net/bpf_jit_comp.c b/arch/mips/net/bpf_jit_comp.c
index d2b6c955f18e..6ee4abe6a1f7 100644
--- a/arch/mips/net/bpf_jit_comp.c
+++ b/arch/mips/net/bpf_jit_comp.c
@@ -909,7 +909,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
struct jit_context ctx;
diff --git a/arch/parisc/net/bpf_jit_core.c b/arch/parisc/net/bpf_jit_core.c
index 35dca372b5df..172770132440 100644
--- a/arch/parisc/net/bpf_jit_core.c
+++ b/arch/parisc/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 2bae4699e78f..53ab97ad6074 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -162,7 +162,7 @@ static void priv_stack_check_guard(void __percpu *priv_stack_ptr, int alloc_size
}
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
u32 proglen;
u32 alloclen;
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 36f0aea8096d..4365d07aaf54 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -41,7 +41,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
unsigned int prog_size = 0, extable_size = 0;
bool extra_pass = false;
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index 2dfc279b1be2..94128fe6be23 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -2312,7 +2312,7 @@ static struct bpf_binary_header *bpf_jit_alloc(struct bpf_jit *jit,
/*
* Compile eBPF program "fp"
*/
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)
{
struct bpf_binary_header *header;
struct s390_jit_data *jit_data;
diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c
index e83e29137566..2fa0e9375127 100644
--- a/arch/sparc/net/bpf_jit_comp_64.c
+++ b/arch/sparc/net/bpf_jit_comp_64.c
@@ -1477,7 +1477,7 @@ struct sparc64_jit_data {
struct jit_ctx ctx;
};
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct sparc64_jit_data *jit_data;
struct bpf_binary_header *header;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 77d00a8dec87..72d9a5faa230 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3713,7 +3713,7 @@ struct x64_jit_data {
#define MAX_PASSES 20
#define PADDING_PASSES (MAX_PASSES - 5)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *rw_header = NULL;
struct bpf_binary_header *header = NULL;
diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c
index 5f259577614a..852baf2e4db4 100644
--- a/arch/x86/net/bpf_jit_comp32.c
+++ b/arch/x86/net/bpf_jit_comp32.c
@@ -2518,7 +2518,7 @@ bool bpf_jit_needs_zext(void)
return true;
}
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
struct bpf_binary_header *header = NULL;
int proglen, oldproglen = 0;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 9fa4d4090093..1ec6d5ba64cc 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1108,6 +1108,8 @@ sk_filter_reason(struct sock *sk, struct sk_buff *skb)
return sk_filter_trim_cap(sk, skb, 1);
}
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err);
struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err);
void bpf_prog_free(struct bpf_prog *fp);
@@ -1153,7 +1155,7 @@ u64 __bpf_call_base(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
((u64 (*)(u64, u64, u64, u64, u64, const struct bpf_insn *)) \
(void *)__bpf_call_base)
-struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog);
+struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog);
void bpf_jit_compile(struct bpf_prog *prog);
bool bpf_jit_needs_zext(void);
bool bpf_jit_inlines_helper_call(s32 imm);
@@ -1188,12 +1190,25 @@ struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off,
#ifdef CONFIG_BPF_SYSCALL
struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len);
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env);
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux);
#else
static inline struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
const struct bpf_insn *patch, u32 len)
{
return ERR_PTR(-ENOTSUPP);
}
+
+static inline struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+{
+ return NULL;
+}
+
+static inline void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
+{
+}
#endif /* CONFIG_BPF_SYSCALL */
int bpf_remove_insns(struct bpf_prog *prog, u32 off, u32 cnt);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc9fb3c07866..79361aa11757 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1491,23 +1491,6 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
bpf_prog_clone_free(fp_other);
}
-static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
-{
-#ifdef CONFIG_BPF_SYSCALL
- struct bpf_map *map;
- int i;
-
- if (len <= 1)
- return;
-
- for (i = 0; i < prog->aux->used_map_cnt; i++) {
- map = prog->aux->used_maps[i];
- if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
- bpf_insn_array_adjust(map, off, len);
- }
-#endif
-}
-
/*
* Now this function is used only to blind the main prog and must be invoked only when
* bpf_prog_need_blind() returns true.
@@ -1580,13 +1563,6 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
if (env)
env->prog = clone;
- else
- /*
- * Instructions arrays must be updated using absolute xlated offsets.
- * The arrays have already been adjusted by bpf_patch_insn_data() when
- * env is not NULL.
- */
- adjust_insn_arrays(clone, i, rewritten);
/* Walk new program and skip insns we just inserted. */
insn = clone->insnsi + i + insn_delta;
@@ -2555,47 +2531,55 @@ static bool bpf_prog_select_interpreter(struct bpf_prog *fp)
return select_interpreter;
}
-static struct bpf_prog *bpf_prog_jit_compile(struct bpf_prog *prog)
+static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
#ifdef CONFIG_BPF_JIT
struct bpf_prog *orig_prog;
+ struct bpf_insn_aux_data *orig_insn_aux;
if (!bpf_prog_need_blind(prog))
- return bpf_int_jit_compile(prog);
+ return bpf_int_jit_compile(env, prog);
+
+ if (env) {
+ /*
+ * If env is not NULL, we are called from the end of bpf_check(), at this
+ * point, only insn_aux_data is used after failure, so it should be restored
+ * on failure.
+ */
+ orig_insn_aux = bpf_dup_insn_aux_data(env);
+ if (!orig_insn_aux)
+ return prog;
+ }
orig_prog = prog;
- prog = bpf_jit_blind_constants(NULL, prog);
+ prog = bpf_jit_blind_constants(env, prog);
/*
* If blinding was requested and we failed during blinding, we must fall
* back to the interpreter.
*/
if (IS_ERR(prog))
- return orig_prog;
+ goto out_restore;
- prog = bpf_int_jit_compile(prog);
+ prog = bpf_int_jit_compile(env, prog);
if (prog->jited) {
bpf_jit_prog_release_other(prog, orig_prog);
+ if (env)
+ vfree(orig_insn_aux);
return prog;
}
bpf_jit_prog_release_other(orig_prog, prog);
+
+out_restore:
prog = orig_prog;
+ if (env)
+ bpf_restore_insn_aux_data(env, orig_insn_aux);
#endif
return prog;
}
-/**
- * bpf_prog_select_runtime - select exec runtime for BPF program
- * @fp: bpf_prog populated with BPF program
- * @err: pointer to error variable
- *
- * Try to JIT eBPF program, if JIT is not available, use interpreter.
- * The BPF program will be executed via bpf_prog_run() function.
- *
- * Return: the &fp argument along with &err set to 0 for success or
- * a negative errno code on failure
- */
-struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
+ int *err)
{
/* In case of BPF to BPF calls, verifier did all the prep
* work with regards to JITing, etc.
@@ -2623,7 +2607,7 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
if (*err)
return fp;
- fp = bpf_prog_jit_compile(fp);
+ fp = bpf_prog_jit_compile(env, fp);
bpf_prog_jit_attempt_done(fp);
if (!fp->jited && jit_needed) {
*err = -ENOTSUPP;
@@ -2649,6 +2633,22 @@ struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
return fp;
}
+
+/**
+ * bpf_prog_select_runtime - select exec runtime for BPF program
+ * @fp: bpf_prog populated with BPF program
+ * @err: pointer to error variable
+ *
+ * Try to JIT eBPF program, if JIT is not available, use interpreter.
+ * The BPF program will be executed via bpf_prog_run() function.
+ *
+ * Return: the &fp argument along with &err set to 0 for success or
+ * a negative errno code on failure
+ */
+struct bpf_prog *bpf_prog_select_runtime(struct bpf_prog *fp, int *err)
+{
+ return __bpf_prog_select_runtime(NULL, fp, err);
+}
EXPORT_SYMBOL_GPL(bpf_prog_select_runtime);
static unsigned int __bpf_prog_ret1(const void *ctx,
@@ -3136,7 +3136,7 @@ const struct bpf_func_proto bpf_tail_call_proto = {
* It is encouraged to implement bpf_int_jit_compile() instead, so that
* eBPF and implicitly also cBPF can get JITed!
*/
-struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_prog *prog)
+struct bpf_prog * __weak bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)
{
return prog;
}
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 721b830b5ef2..6c86980cc9e8 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -993,7 +993,7 @@ static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_s
env->subprog_info[env->subprog_cnt].start = env->prog->len;
}
-static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
+struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *env)
{
size_t size;
void *new_aux;
@@ -1005,8 +1005,8 @@ static struct bpf_insn_aux_data *bpf_dup_insn_aux_data(struct bpf_verifier_env *
return new_aux;
}
-static void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
- struct bpf_insn_aux_data *orig_insn_aux)
+void bpf_restore_insn_aux_data(struct bpf_verifier_env *env,
+ struct bpf_insn_aux_data *orig_insn_aux)
{
/* the expanded elements are zero-filled, so no special handling is required */
vfree(env->insn_aux_data);
@@ -1150,7 +1150,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
func[i]->aux->token = prog->aux->token;
if (!i)
func[i]->aux->exception_boundary = env->seen_exception;
- func[i] = bpf_int_jit_compile(func[i]);
+ func[i] = bpf_int_jit_compile(env, func[i]);
if (!func[i]->jited) {
err = -ENOTSUPP;
goto out_free;
@@ -1194,7 +1194,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
}
for (i = 0; i < env->subprog_cnt; i++) {
old_bpf_func = func[i]->bpf_func;
- tmp = bpf_int_jit_compile(func[i]);
+ tmp = bpf_int_jit_compile(env, func[i]);
if (tmp != func[i] || func[i]->bpf_func != old_bpf_func) {
verbose(env, "JIT doesn't support bpf-to-bpf calls\n");
err = -ENOTSUPP;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index b73b25c63073..a3c0214ca934 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3083,10 +3083,6 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
if (err < 0)
goto free_used_maps;
- prog = bpf_prog_select_runtime(prog, &err);
- if (err < 0)
- goto free_used_maps;
-
err = bpf_prog_mark_insn_arrays_ready(prog);
if (err < 0)
goto free_used_maps;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9e4980128151..e804e0da3500 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -20155,6 +20155,14 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
adjust_btf_func(env);
+ /* extension progs temporarily inherit the attach_type of their targets
+ for verification purposes, so set it back to zero before returning
+ */
+ if (env->prog->type == BPF_PROG_TYPE_EXT)
+ env->prog->expected_attach_type = 0;
+
+ env->prog = __bpf_prog_select_runtime(env, env->prog, &ret);
+
err_release_maps:
if (ret)
release_insn_arrays(env);
@@ -20166,12 +20174,6 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
if (!env->prog->aux->used_btfs)
release_btfs(env);
- /* extension progs temporarily inherit the attach_type of their targets
- for verification purposes, so set it back to zero before returning
- */
- if (env->prog->type == BPF_PROG_TYPE_EXT)
- env->prog->expected_attach_type = 0;
-
*prog = env->prog;
module_put(env->attach_btf_mod);
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 0/5] emit ENDBR/BTI instructions for indirect
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.
The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, allowing
the JIT to consult env->insn_aux_data to determine which instructions are
indirect jump targets.
During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not use env->insn_aux_data directly.
To avoid this mismatch, and given that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.
v15:
- Rebase and target bpf tree
- Resotre subprog_start of the fake 'exit' subprog on failure
- Fix wrong function name used in comment
v14: https://lore.kernel.org/all/cover.1776062885.git.xukuohai@hotmail.com/
- Rebase
- Fix comment style
- Fix incorrect variable and function name used in commit message
v13: https://lore.kernel.org/bpf/20260411133847.1042658-1-xukuohai@huaweicloud.com
- Use vmalloc to allocate memory for insn_aux_data copies to match with vfree
- Do not free the copied memory of insn_aux_data when restoring from failure
- Code cleanup
v12: https://lore.kernel.org/bpf/20260403132811.753894-1-xukuohai@huaweicloud.com
- Restore env->insn_aux_data on JIT failure
- Fix incorrect error code sign (-EFAULT vs EFAULT)
- Fix incorrect prog used in the restore path
v11: https://lore.kernel.org/bpf/20260403090915.473493-1-xukuohai@huaweicloud.com
- Restore env->subprog_info after jit_subprogs() fails
- Clear prog->jit_requested and prog->blinding_requested on failure
- Use the actual env->insn_aux_data size in clear_insn_aux_data() on failure
v10: https://lore.kernel.org/bpf/20260324122052.342751-1-xukuohai@huaweicloud.com
- Fix the incorrect call_imm restore in jit_subprogs
- Define a dummy void version of bpf_jit_prog_release_other and
bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)
v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)
v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
to adjust_insn_aux_data()
v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch
v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier
v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
each prog
v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
jump targets, and emit ENDBR on x86
v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)
v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)
v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com
Xu Kuohai (5):
bpf: Move constants blinding out of arch-specific JITs
bpf: Pass bpf_verifier_env to JIT
bpf: Add helper to detect indirect jump targets
bpf, x86: Emit ENDBR for indirect jump targets
bpf, arm64: Emit BTI for indirect jump target
arch/arc/net/bpf_jit_core.c | 41 +++-----
arch/arm/net/bpf_jit_32.c | 43 ++------
arch/arm64/net/bpf_jit_comp.c | 87 ++++++-----------
arch/loongarch/net/bpf_jit.c | 61 ++++--------
arch/mips/net/bpf_jit_comp.c | 22 +----
arch/parisc/net/bpf_jit_core.c | 75 ++++++--------
arch/powerpc/net/bpf_jit_comp.c | 74 ++++++--------
arch/riscv/net/bpf_jit_core.c | 63 +++++-------
arch/s390/net/bpf_jit_comp.c | 61 ++++--------
arch/sparc/net/bpf_jit_comp_64.c | 63 +++++-------
arch/x86/net/bpf_jit_comp.c | 73 +++++---------
arch/x86/net/bpf_jit_comp32.c | 35 +------
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 9 +-
include/linux/filter.h | 50 +++++++++-
kernel/bpf/core.c | 138 ++++++++++++++++++--------
kernel/bpf/fixups.c | 162 ++++++++++++++++++++++++++-----
kernel/bpf/syscall.c | 4 -
kernel/bpf/verifier.c | 21 ++--
19 files changed, 529 insertions(+), 555 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH bpf v15 3/5] bpf: Add helper to detect indirect jump targets
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
Introduce helper bpf_insn_is_indirect_target to check whether a BPF
instruction is an indirect jump target.
Since the verifier knows which instructions are indirect jump targets,
add a new flag indirect_target to struct bpf_insn_aux_data to mark
them. The verifier sets this flag when verifying an indirect jump target
instruction, and the helper checks the flag to determine whether an
instruction is an indirect jump target.
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> #v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> #v12
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
include/linux/bpf.h | 2 ++
include/linux/bpf_verifier.h | 9 +++++----
kernel/bpf/core.c | 9 +++++++++
kernel/bpf/fixups.c | 12 ++++++++++++
kernel/bpf/verifier.c | 7 +++++++
5 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 0136a108d083..b4b703c90ca9 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1541,6 +1541,8 @@ bool bpf_has_frame_pointer(unsigned long ip);
int bpf_jit_charge_modmem(u32 size);
void bpf_jit_uncharge_modmem(u32 size);
bool bpf_prog_has_trampoline(const struct bpf_prog *prog);
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx);
#else
static inline int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr,
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 53e8664cb566..b148f816f25b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -630,16 +630,17 @@ struct bpf_insn_aux_data {
/* below fields are initialized once */
unsigned int orig_idx; /* original instruction index */
- bool jmp_point;
- bool prune_point;
+ u32 jmp_point:1;
+ u32 prune_point:1;
/* ensure we check state equivalence and save state checkpoint and
* this instruction, regardless of any heuristics
*/
- bool force_checkpoint;
+ u32 force_checkpoint:1;
/* true if instruction is a call to a helper function that
* accepts callback function as a parameter.
*/
- bool calls_callback;
+ u32 calls_callback:1;
+ u32 indirect_target:1; /* if it is an indirect jump target */
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 79361aa11757..8b018ff48875 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1573,6 +1573,15 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_verifier_env *env, struct bp
clone->blinded = 1;
return clone;
}
+
+bool bpf_insn_is_indirect_target(const struct bpf_verifier_env *env, const struct bpf_prog *prog,
+ int insn_idx)
+{
+ if (!env)
+ return false;
+ insn_idx += prog->aux->subprog_start;
+ return env->insn_aux_data[insn_idx].indirect_target;
+}
#endif /* CONFIG_BPF_JIT */
/* Base function for offset calculation. Needs to go into .text section,
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 6c86980cc9e8..fba9e8c00878 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -183,6 +183,18 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
data[i].seen = old_seen;
data[i].zext_dst = insn_has_def32(insn + i);
}
+
+ /*
+ * The indirect_target flag of the original instruction was moved to the last of the
+ * new instructions by the above memmove and memset, but the indirect jump target is
+ * actually the first instruction, so move it back. This also matches with the behavior
+ * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
+ * instruction.
+ */
+ if (data[off + cnt - 1].indirect_target) {
+ data[off].indirect_target = 1;
+ data[off + cnt - 1].indirect_target = 0;
+ }
}
static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e804e0da3500..1e36b9e91277 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3497,6 +3497,11 @@ static int insn_stack_access_flags(int frameno, int spi)
return INSN_F_STACK_ACCESS | (spi << INSN_F_SPI_SHIFT) | frameno;
}
+static void mark_indirect_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].indirect_target = true;
+}
+
#define LR_FRAMENO_BITS 3
#define LR_SPI_BITS 6
#define LR_ENTRY_BITS (LR_SPI_BITS + LR_FRAMENO_BITS + 1)
@@ -17545,12 +17550,14 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
}
for (i = 0; i < n - 1; i++) {
+ mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
env->insn_idx, env->cur_state->speculative);
if (IS_ERR(other_branch))
return PTR_ERR(other_branch);
}
env->insn_idx = env->gotox_tmp_buf->items[n-1];
+ mark_indirect_target(env, env->insn_idx);
return INSN_IDX_UPDATED;
}
--
2.43.0
^ permalink raw reply related
* [PATCH bpf v15 4/5] bpf, x86: Emit ENDBR for indirect jump targets
From: Xu Kuohai @ 2026-04-16 6:43 UTC (permalink / raw)
To: bpf, linux-kernel, linux-arm-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Yonghong Song, Puranjay Mohan, Anton Protopopov,
Alexis Lothoré, Shahab Vahedi, Russell King, Tiezhu Yang,
Hengqi Chen, Johan Almbladh, Paul Burton, Hari Bathini,
Christophe Leroy, Naveen N Rao, Luke Nelson, Xi Wang,
Björn Töpel, Pu Lehui, Ilya Leoshkevich, Heiko Carstens,
Vasily Gorbik, David S . Miller, Wang YanQing
In-Reply-To: <20260416064341.151802-1-xukuohai@huaweicloud.com>
From: Xu Kuohai <xukuohai@huawei.com>
On CPUs that support CET/IBT, the indirect jump selftest triggers
a kernel panic because the indirect jump targets lack ENDBR
instructions.
To fix it, emit an ENDBR instruction to each indirect jump target. Since
the ENDBR instruction shifts the position of original jited instructions,
fix the instruction address calculation wherever the addresses are used.
For reference, below is a sample panic log.
Missing ENDBR: bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cet.c:133!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
...
? 0xffffffffc00fb258
? bpf_prog_2e5f1c71c13ac3e0_big_jump_table+0x97/0xe1
bpf_prog_test_run_syscall+0x110/0x2f0
? fdget+0xba/0xe0
__sys_bpf+0xe4b/0x2590
? __kmalloc_node_track_caller_noprof+0x1c7/0x680
? bpf_prog_test_run_syscall+0x215/0x2f0
__x64_sys_bpf+0x21/0x30
do_syscall_64+0x85/0x620
? bpf_prog_test_run_syscall+0x1e2/0x2f0
Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> # v8
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com> # v12
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/x86/net/bpf_jit_comp.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72d9a5faa230..ea9e707e8abf 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -58,8 +58,8 @@ static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
#define EMIT_ENDBR() EMIT(gen_endbr(), 4)
#define EMIT_ENDBR_POISON() EMIT(gen_endbr_poison(), 4)
#else
-#define EMIT_ENDBR()
-#define EMIT_ENDBR_POISON()
+#define EMIT_ENDBR() do { } while (0)
+#define EMIT_ENDBR_POISON() do { } while (0)
#endif
static bool is_imm8(int value)
@@ -1649,8 +1649,8 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
return 0;
}
-static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image,
- int oldproglen, struct jit_context *ctx, bool jmp_padding)
+static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
+ u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
{
bool tail_call_reachable = bpf_prog->aux->tail_call_reachable;
struct bpf_insn *insn = bpf_prog->insnsi;
@@ -1663,7 +1663,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
void __percpu *priv_stack_ptr;
int i, excnt = 0;
int ilen, proglen = 0;
- u8 *prog = temp;
+ u8 *ip, *prog = temp;
u32 stack_depth;
int err;
@@ -1734,6 +1734,11 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
dst_reg = X86_REG_R9;
}
+ if (bpf_insn_is_indirect_target(env, bpf_prog, i - 1))
+ EMIT_ENDBR();
+
+ ip = image + addrs[i - 1] + (prog - temp);
+
switch (insn->code) {
/* ALU */
case BPF_ALU | BPF_ADD | BPF_X:
@@ -2440,8 +2445,6 @@ st: if (is_imm8(insn->off))
/* call */
case BPF_JMP | BPF_CALL: {
- u8 *ip = image + addrs[i - 1];
-
func = (u8 *) __bpf_call_base + imm32;
if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2465,7 +2468,8 @@ st: if (is_imm8(insn->off))
if (imm32)
emit_bpf_tail_call_direct(bpf_prog,
&bpf_prog->aux->poke_tab[imm32 - 1],
- &prog, image + addrs[i - 1],
+ &prog,
+ ip,
callee_regs_used,
stack_depth,
ctx);
@@ -2474,7 +2478,7 @@ st: if (is_imm8(insn->off))
&prog,
callee_regs_used,
stack_depth,
- image + addrs[i - 1],
+ ip,
ctx);
break;
@@ -2639,7 +2643,7 @@ st: if (is_imm8(insn->off))
break;
case BPF_JMP | BPF_JA | BPF_X:
- emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+ emit_indirect_jump(&prog, insn->dst_reg, ip);
break;
case BPF_JMP | BPF_JA:
case BPF_JMP32 | BPF_JA:
@@ -2729,8 +2733,6 @@ st: if (is_imm8(insn->off))
ctx->cleanup_addr = proglen;
if (bpf_prog_was_classic(bpf_prog) &&
!ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
- u8 *ip = image + addrs[i - 1];
-
if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
return -EINVAL;
}
@@ -3791,7 +3793,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
for (pass = 0; pass < MAX_PASSES || image; pass++) {
if (!padding && pass >= PADDING_PASSES)
padding = true;
- proglen = do_jit(prog, addrs, image, rw_image, oldproglen, &ctx, padding);
+ proglen = do_jit(env, prog, addrs, image, rw_image, oldproglen, &ctx, padding);
if (proglen <= 0) {
out_image:
image = NULL;
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox