* [PATCH v2 0/4] drm/verisilicon: add Nuvoton MA35D1 DCU Lite support
From: Joey Lu @ 2026-05-19 5:51 UTC (permalink / raw)
To: zhengxingda, maarten.lankhorst, mripard, tzimmermann, airlied,
simona, robh, krzk+dt, conor+dt
Cc: ychuang3, schung, yclu4, dri-devel, devicetree, linux-arm-kernel,
linux-kernel, Joey Lu
This series adds support for the Verisilicon DCU Lite display controller
as integrated in the Nuvoton MA35D1 SoC.
The Verisilicon DC driver and its DT binding were originally written by
Icenowy Zheng <zhengxingda@iscas.ac.cn> for the T-Head TH1520 SoC, which
carries a DC8200 IP block. The present series builds on that foundation
with gratitude to Icenowy for the original work.
The DCU Lite is a different variant in the DC IP family. While the two
IPs share a broadly similar register layout, a number of differences
prevent the existing driver from working on the MA35D1 without
modification:
- No CONFIG_EX commit path: the DC8200 staging registers
(FB_CONFIG_EX, FB_TOP_LEFT, FB_BOTTOM_RIGHT, FB_BLEND_CONFIG,
PANEL_CONFIG_EX) are absent. The DCU Lite uses enable (bit 0) and
reset (bit 4) bits in FB_CONFIG for direct framebuffer updates, and
requires a per-frame VALID bit toggle (FB_CONFIG bit 3) to latch
configuration changes.
- No PANEL_START register: panel output begins when
PANEL_CONFIG.RUNNING is set; the DC8200 multi-display sync start
register at 0x1CCC does not exist.
- Different IRQ registers: DISP_IRQ_STA at 0x147C / DISP_IRQ_EN at
0x1480, versus the DC8200's TOP_IRQ_ACK at 0x0010 / TOP_IRQ_EN at
0x0014.
- Simpler clock topology: two clocks ("core" bus gate and "pix0" pixel
divider); no axi or ahb clocks required.
- Single display output: no per-output indexing beyond index 0 is
needed.
- Hardware-discoverable identity: the DCU Lite exposes chip identity
registers whose model field reads 0x0 (revision 0x5560,
customer_id 0x305), allowing the existing vs_fill_chip_identity()
path to identify the variant purely through register reads. No
separate OF compatible string is introduced.
Patch 1 generalises the verisilicon,dc DT binding to accommodate
variants with flexible clock/reset counts and a single output, using
allOf/if-then-else to keep per-variant constraints in-schema.
Patches 2-4 introduce the driver changes in three logical steps:
register-level constants and the DCU Lite chip identity table entry;
the vs_dc_funcs hardware ops table with DC8200 ops extracted into
vs_dc8200.c; and finally the DCU Lite ops in vs_dcu_lite.c with the
necessary Kconfig and clock-optionality changes.
All patches have been tested on Nuvoton MA35D1 hardware.
Changes from v1:
- Corrected "DC8000" to "DC8200" throughout (the existing supported
IP is DC8200, not DC8000).
- Dropped the separate nuvoton,ma35d1-dcu.yaml; variant constraints
are now expressed inline in verisilicon,dc.yaml via allOf/if-then-else.
The MA35D1 uses the generic "verisilicon,dc" compatible string.
- Replaced the vs_dc_info platform-data flags approach with a
vs_dc_funcs hardware ops table, giving cleaner per-variant dispatch
without scattering if/else branches across multiple files.
- DCU Lite variant is identified through hardware registers rather than
the OF match table.
- Series split from 2 patches to 4 for clearer logical progression.
- Renamed plane ops in vs_dc_funcs: plane_enable/disable to
plane_enable_ex/disable_ex, plane_update_ext to plane_update_ex.
Joey Lu (4):
dt-bindings: display: verisilicon,dc: generalize for single-output
variants
drm/verisilicon: add model ID constants and DCU Lite chip identity
drm/verisilicon: introduce per-variant hardware ops table
drm/verisilicon: add Nuvoton MA35D1 DCU Lite display controller
support
.../bindings/display/verisilicon,dc.yaml | 135 ++++++++++++++----
drivers/gpu/drm/verisilicon/Kconfig | 2 +-
drivers/gpu/drm/verisilicon/Makefile | 2 +-
drivers/gpu/drm/verisilicon/vs_bridge.c | 20 +--
drivers/gpu/drm/verisilicon/vs_crtc.c | 38 ++++-
drivers/gpu/drm/verisilicon/vs_crtc_regs.h | 1 +
drivers/gpu/drm/verisilicon/vs_dc.c | 13 +-
drivers/gpu/drm/verisilicon/vs_dc.h | 33 +++++
drivers/gpu/drm/verisilicon/vs_dc8200.c | 107 ++++++++++++++
drivers/gpu/drm/verisilicon/vs_dcu_lite.c | 78 ++++++++++
drivers/gpu/drm/verisilicon/vs_hwdb.c | 16 ++-
drivers/gpu/drm/verisilicon/vs_hwdb.h | 3 +
.../gpu/drm/verisilicon/vs_primary_plane.c | 32 +----
.../drm/verisilicon/vs_primary_plane_regs.h | 3 +
14 files changed, 398 insertions(+), 85 deletions(-)
create mode 100644 drivers/gpu/drm/verisilicon/vs_dc8200.c
create mode 100644 drivers/gpu/drm/verisilicon/vs_dcu_lite.c
--
2.43.0
^ permalink raw reply
* [PATCH] arm64: dts: imx943-evk-sdwifi: add a new dtso to support SDIW612 WiFi
From: Sherry Sun (OSS) @ 2026-05-19 5:39 UTC (permalink / raw)
To: Frank.Li, s.hauer, kernel, festevam, robh, krzk+dt, conor+dt
Cc: imx, linux-arm-kernel, devicetree, linux-kernel
From: Sherry Sun <sherry.sun@nxp.com>
Add a new imx943-evk-sdwifi.dtso to support SDIW612 WiFi chip on
imx943-evk board, the default imx943-evk.dtb is used to support PCIE
AW693 WiFi.
Use separate dts for SDIW612 and PCIe AW693 WiFi to avoid the shared
regulator between SDIO and PCIe buses, the random probe order between
the two buses may break the PCIe initialization sequence which cause
AW693 has probability of failing to detect.
Signed-off-by: Sherry Sun <sherry.sun@nxp.com>
---
arch/arm64/boot/dts/freescale/Makefile | 3 +++
.../boot/dts/freescale/imx943-evk-sdwifi.dtso | 15 +++++++++++++++
arch/arm64/boot/dts/freescale/imx943-evk.dts | 2 +-
3 files changed, 19 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/boot/dts/freescale/imx943-evk-sdwifi.dtso
diff --git a/arch/arm64/boot/dts/freescale/Makefile b/arch/arm64/boot/dts/freescale/Makefile
index 0a4dabac5de4..657e0915ca69 100644
--- a/arch/arm64/boot/dts/freescale/Makefile
+++ b/arch/arm64/boot/dts/freescale/Makefile
@@ -516,6 +516,9 @@ imx943-evk-pcie0-ep-dtbs += imx943-evk.dtb imx-pcie0-ep.dtbo
imx943-evk-pcie1-ep-dtbs += imx943-evk.dtb imx-pcie1-ep.dtbo
dtb-$(CONFIG_ARCH_MXC) += imx943-evk-pcie0-ep.dtb imx943-evk-pcie1-ep.dtb
+imx943-evk-sdwifi-dtbs := imx943-evk.dtb imx943-evk-sdwifi.dtbo
+dtb-$(CONFIG_ARCH_MXC) += imx943-evk-sdwifi.dtb
+
dtb-$(CONFIG_ARCH_MXC) += imx95-15x15-ab2.dtb
dtb-$(CONFIG_ARCH_MXC) += imx95-15x15-evk.dtb
dtb-$(CONFIG_ARCH_MXC) += imx95-15x15-frdm.dtb
diff --git a/arch/arm64/boot/dts/freescale/imx943-evk-sdwifi.dtso b/arch/arm64/boot/dts/freescale/imx943-evk-sdwifi.dtso
new file mode 100644
index 000000000000..59cc1c27b9b9
--- /dev/null
+++ b/arch/arm64/boot/dts/freescale/imx943-evk-sdwifi.dtso
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+/*
+ * Copyright 2026 NXP
+ */
+
+/dts-v1/;
+/plugin/;
+
+&pcie0 {
+ status = "disabled";
+};
+
+&usdhc3 {
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/freescale/imx943-evk.dts b/arch/arm64/boot/dts/freescale/imx943-evk.dts
index fe4fc512d95d..1346a6a56883 100644
--- a/arch/arm64/boot/dts/freescale/imx943-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx943-evk.dts
@@ -1153,7 +1153,7 @@ &usdhc3 {
keep-power-in-suspend;
non-removable;
wakeup-source;
- status = "okay";
+ status = "disabled";
};
&wdog3 {
base-commit: 5f9e9f83aee0fa8f2124c6f192505de2cdf7c5dc
--
2.37.1
^ permalink raw reply related
* Re: [PATCH v14 05/44] arm64: RMI: Add wrappers for RMI calls
From: Aneesh Kumar K.V @ 2026-05-19 5:35 UTC (permalink / raw)
To: Steven Price, kvm, kvmarm
Cc: Steven Price, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Emi Kisanuki,
Vishal Annapurve, WeiLin.Chang, Lorenzo.Pieralisi2
In-Reply-To: <20260513131757.116630-6-steven.price@arm.com>
Steven Price <steven.price@arm.com> writes:
> The wrappers make the call sites easier to read and deal with the
> boiler plate of handling the error codes from the RMM.
>
> Signed-off-by: Steven Price <steven.price@arm.com>
> +#define rmi_smccc(...) do { \
> + arm_smccc_1_1_invoke(__VA_ARGS__); \
> +} while (RMI_RETURN_STATUS(res.a0) == RMI_BUSY || \
> + RMI_RETURN_STATUS(res.a0) == RMI_BLOCKED)
> +
I guess this is not used. Also, that would require the call site to have a struct arm_smccc_res res.
-aneesh
^ permalink raw reply
* Re: [PATCH v3 3/4] PCI: endpoint: Add API for DOE initialization and setup in EPC core
From: Aksh Garg @ 2026-05-19 5:30 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: linux-pci, linux-doc, kwilczynski, bhelgaas, corbet, kishon,
skhan, lukas, cassel, alistair, linux-arm-kernel, linux-kernel,
s-vadapalli, danishanwar, srk
In-Reply-To: <mn7rnqdunp4mq45a7ypf26rfpzjr2gik7w4p7hpj4x3r3fzfzz@dlrsn27u5mbf>
On 15/05/26 18:17, Manivannan Sadhasivam wrote:
> On Fri, May 15, 2026 at 10:21:52AM +0530, Aksh Garg wrote:
>>
>>
>> On 14/05/26 13:38, Manivannan Sadhasivam wrote:
>>> On Mon, Apr 27, 2026 at 10:47:24AM +0530, Aksh Garg wrote:
>>>> Add pci_epc_setup_doe() API in EPC core driver to initialize and setup
>>>> the DOE framework for an endpoint controller. The API discovers the DOE
>>>> capabilities (extended capability ID 0x2E), and registers each discovered
>>>> DOE mailbox for all the functions in the endpoint controller. This API
>>>> should be invoked by the controller driver during probe based on the
>>>> doe_capable feature.
>>>>
>>>> Add pci_epc_destroy_doe() API in EPC core driver for cleanup of DOE
>>>> resources, which should be invoked by the controller driver during
>>>> controller cleanup based on the doe_capable feature.
>>>>
>>>> Co-developed-by: Siddharth Vadapalli <s-vadapalli@ti.com>
>>>> Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
>>>> Signed-off-by: Aksh Garg <a-garg7@ti.com>
>>>> ---
>>>>
>>>> Changes from v2 to v3:
>>>> - Rebased on 7.1-rc1.
>>>>
>>>> Changes since v1:
>>>> - New patch added to v2 (not present in v1)
>>>>
>>>> v2: https://lore.kernel.org/all/20260401073022.215805-4-a-garg7@ti.com/
>>>>
>>>> This patch is introduced based on the feedback provided by Manivannan
>>>> Sadhasivam at [1].
>>>>
>>>
>>> Sweet! But I was expecting you to add atleast one EPC driver implementation to
>>> make use of these APIs.
>>>
>>> Also, why can't you call these APIs from the EPC core directly? Maybe during
>>> pci_epc_init_notify() once the register accesses become valid.
>>
>> Can we add the DOE initialization API to pci_epc_init_notify()? This
>> API seems to be called to notify the EPF drivers that the EPC device's
>> initialization has been completed, as the name and description suggests.
>
> That's correct. But there is no harm in calling something like
> pci_epc_init_capabilities() inside its definition. Only concern would be that
> pci_epc_init_notify() is mostly called from threaded IRQ handlers. So loading
> the handler would not be recommended. But since it is threaded anyway and we
> don't have a better place to call, it would be OK.
>
> We could've called this from pci_epc_{create/start}, but some controllers won't
> allow accessing CSRs without REFCLK. So only after pci_init_notify(), CSRs can
> be accessed.
>
>> As 'pci_epc_doe_setup' is a part of EPC initialization, I thought the
>> EPC drivers should call this API before calling the pci_epc_init_notify().
>>
>> However, I agree with your suggestion to call the DOE setup API directly
>> from the EPC core instead of sprinkling over the EPC drivers. I would
>> recommend renaming the pci_epc_init_notify() API (and hence the
>> pci_epc_deinit_notify() as well) to something like pci_epc_init_complete(),
>> and add the DOE setup API/logic just before the
>> logic of notifying the EPF devices.
>>
>
> No need to rename this API. Just use as is:
>
> pci_epc_init_notify()
> -> pci_epc_init_capabilities()
> -> pci_epc_init_doe()
> -> epf->event_ops->epc_init()
Thank you for the suggestion, I will incorporate these changes in v4 series.
Regards,
Aksh Garg
>
> - Mani
>
^ permalink raw reply
* Re: [PATCH] spi: aspeed: Replace VLA parameter with flat pointer in calibration helper
From: Cédric Le Goater @ 2026-05-19 5:24 UTC (permalink / raw)
To: Chin-Ting Kuo, broonie, joel, andrew, linux-aspeed, openbmc,
linux-spi, linux-arm-kernel, linux-kernel, BMC-SW
Cc: kernel test robot
In-Reply-To: <20260518095708.2502537-3-chin-ting_kuo@aspeedtech.com>
On 5/18/26 11:57, Chin-Ting Kuo wrote:
> aspeed_spi_ast2600_optimized_timing() declared its buffer argument as a
> variable-length array parameter (u8 buf[rows][cols]), which causes a
> sparse warning. Replace the VLA parameter with a plain u8 * and compute
> the 2-D index manually. The corresponding call site is also updated.
>
> Reported-by: kernel test robot <lkp@intel.com>
> Closes: https://lore.kernel.org/oe-kbuild-all/202605180441.uD3toFRJ-lkp@intel.com/
> Signed-off-by: Chin-Ting Kuo <chin-ting_kuo@aspeedtech.com>
> ---
> drivers/spi/spi-aspeed-smc.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/spi/spi-aspeed-smc.c b/drivers/spi/spi-aspeed-smc.c
> index 808659a1f460..c5275700de3d 100644
> --- a/drivers/spi/spi-aspeed-smc.c
> +++ b/drivers/spi/spi-aspeed-smc.c
> @@ -1467,8 +1467,7 @@ static int aspeed_spi_do_calibration(struct aspeed_spi_chip *chip)
> * must contains the highest number of consecutive "pass"
> * results and not span across multiple rows.
> */
> -static u32 aspeed_spi_ast2600_optimized_timing(u32 rows, u32 cols,
> - u8 buf[rows][cols])
> +static u32 aspeed_spi_ast2600_optimized_timing(u32 rows, u32 cols, u8 *buf)
> {
> int r = 0, c = 0;
> int max = 0;
> @@ -1478,7 +1477,7 @@ static u32 aspeed_spi_ast2600_optimized_timing(u32 rows, u32 cols,
> for (j = 0; j < cols;) {
> int k = j;
>
> - while (k < cols && buf[i][k])
> + while (k < cols && buf[i * cols + k])
> k++;
>
> if (k - j > max) {
> @@ -1541,7 +1540,7 @@ static int aspeed_spi_ast2600_calibrate(struct aspeed_spi_chip *chip, u32 hdiv,
> }
> }
>
> - calib_point = aspeed_spi_ast2600_optimized_timing(6, 17, calib_res);
> + calib_point = aspeed_spi_ast2600_optimized_timing(6, 17, &calib_res[0][0]);
> /* No good setting for this frequency */
> if (calib_point == 0)
> return -1;
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Thanks,
C.
^ permalink raw reply
* Re: [PATCH v3 2/4] PCI: endpoint: Add DOE mailbox support for endpoint functions
From: Aksh Garg @ 2026-05-19 5:23 UTC (permalink / raw)
To: Manivannan Sadhasivam
Cc: linux-pci, linux-doc, kwilczynski, bhelgaas, corbet, kishon,
skhan, lukas, cassel, alistair, linux-arm-kernel, linux-kernel,
s-vadapalli, danishanwar, srk
In-Reply-To: <ies3cbldthjv4vgraibgo642pfuvcr3lsixgxeisqa34ygkpbf@dd2qstv5fiig>
On 15/05/26 18:10, Manivannan Sadhasivam wrote:
> On Fri, May 15, 2026 at 11:05:29AM +0530, Aksh Garg wrote:
>>
>>
>> On 14/05/26 13:33, Manivannan Sadhasivam wrote:
>>> On Mon, Apr 27, 2026 at 10:47:23AM +0530, Aksh Garg wrote:
>>>> DOE (Data Object Exchange) is a standard PCIe extended capability
>>>> feature introduced in the Data Object Exchange (DOE) ECN for
>>>> PCIe r5.0. It provides a communication mechanism primarily used for
>>>> implementing PCIe security features such as device authentication, and
>>>> secure link establishment. Think of DOE as a sophisticated mailbox
>>>> system built into PCIe. The root complex can send structured requests
>>>> to the endpoint device through DOE mailboxes, and the endpoint device
>>>> responds with appropriate data.
>>>>
>>>> Add the DOE support for PCIe endpoint devices, enabling endpoint
>>>> functions to process the DOE requests from the host. The implementation
>>>> provides framework APIs for EPC core driver and controller drivers to
>>>> register mailboxes, and request processing with workqueues ensuring
>>>> sequential handling per mailbox, and parallel handling across mailboxes.
>>>> The Discovery protocol is handled internally by the DOE core.
>>>>
>>>> This implementation complements the existing DOE implementation for
>>>> root complex in drivers/pci/doe.c.
>>>>
>>>> Co-developed-by: Siddharth Vadapalli <s-vadapalli@ti.com>
>>>> Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
>>>> Signed-off-by: Aksh Garg <a-garg7@ti.com>
>>>> ---
>>>> +
>>>> +/*
>>>> + * Global registry of protocol handlers.
>>>> + * When a new DOE protocol, library is added, add an entry to this array.
>>>> + */
>>>> +static const struct pci_doe_protocol pci_doe_protocols[] = {
>>>> + {
>>>> + .vid = PCI_VENDOR_ID_PCI_SIG,
>>>> + .type = PCI_DOE_FEATURE_DISCOVERY,
>>>> + .handler = pci_ep_doe_handle_discovery,
>>>> + },
>>>> +};
>>>> +
>>>> +/*
>>>> + * Combines function number and capability offset into a unique lookup key
>>>> + * for storing/retrieving DOE mailboxes in an xarray.
>>>> + */
>>>> +#define PCI_DOE_MB_KEY(func, offset) \
>>>> + (((unsigned long)(func) << 16) | (offset))
>>>> +#define PCI_DOE_PROTOCOL_COUNT ARRAY_SIZE(pci_doe_protocols)
>>>> +
>>>> +/**
>>>> + * pci_ep_doe_init() - Initialize the DOE framework for a controller in EP mode
>>>> + * @epc: PCI endpoint controller
>>>> + *
>>>> + * Initialize the DOE framework data structures. This only initializes
>>>> + * the xarray that will hold the mailboxes.
>>>> + *
>>>> + * RETURNS: 0 on success, -errno on failure
>>>
>>> kernel-doc format to describe return value is 'Return:' or 'Returns:".
>>
>> Thanks for pointing this out. I will update this.
>>
>>>
>>>> + */
>>>> +int pci_ep_doe_init(struct pci_epc *epc)
>>>> +{
>>>> + if (!epc)
>>>> + return -EINVAL;
>>>> +
>>>> + xa_init(&epc->doe_mbs);
>>>> + return 0;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(pci_ep_doe_init);
>>>> +
>>
>> [...]
>>
>>>> +
>>>> +/**
>>>> + * pci_ep_doe_process_request() - Process DOE request on endpoint
>>>> + * @epc: PCI endpoint controller
>>>> + * @func_no: Physical function number
>>>> + * @cap_offset: DOE capability offset
>>>> + * @vendor: Vendor ID from request header
>>>> + * @type: Protocol type from request header
>>>> + * @request: Request payload in CPU-native format
>>>> + * @request_sz: Size of request payload (bytes)
>>>> + * @complete: Callback to invoke upon completion
>>>> + *
>>>> + * Asynchronously process a DOE request received on the endpoint. The request
>>>> + * payload should not include the DOE header (vendor/type/length). The protocol
>>>> + * handler will allocate the response buffer, which the caller (controller driver)
>>>> + * must free after use.
>>>> + *
>>>> + * This function returns immediately after queuing the request. The completion
>>>> + * callback will be invoked asynchronously from workqueue context once the
>>>> + * request is processed. The callback receives the function number and capability
>>>> + * offset to identify the mailbox, along with a status code (0 on success, -errno
>>>> + * on failure), and other required arguments.
>>>> + *
>>>> + * As per DOE specification, a mailbox processes one request at a time.
>>>> + * Therefore, this function will never be called concurrently for the same
>>>> + * mailbox by different callers.
>>>> + *
>>>> + * The caller is responsible for the conversion of the received DOE request
>>>> + * with le32_to_cpu() before calling this function.
>>>> + * Similarly, it is responsible for converting the response payload with
>>>> + * cpu_to_le32() before sending it back over the DOE mailbox.
>>>> + *
>>>> + * The caller is also responsible for ensuring that the request size
>>>> + * is within the limits defined by PCI_DOE_MAX_LENGTH.
>>>> + *
>>>> + * RETURNS: 0 if the request was successfully queued, -errno on failure
>>>> + */
>>>> +int pci_ep_doe_process_request(struct pci_epc *epc, u8 func_no, u16 cap_offset,
>>>> + u16 vendor, u8 type, const void *request, size_t request_sz,
>>>> + pci_ep_doe_complete_t complete)
>>>> +{
>>>> + struct pci_ep_doe_mb *doe_mb;
>>>> + struct pci_ep_doe_task *task;
>>>> + int rc;
>>>> +
>>>> + doe_mb = pci_ep_doe_get_mailbox(epc, func_no, cap_offset);
>>>> + if (!doe_mb) {
>>>> + kfree(request);
>>>> + return -ENODEV;
>>>> + }
>>>> +
>>>> + task = kzalloc_obj(*task, GFP_KERNEL);
>>>> + if (!task) {
>>>> + kfree(request);
>>>> + return -ENOMEM;
>>>> + }
>>>> +
>>>> + task->feat.vid = vendor;
>>>> + task->feat.type = type;
>>>> + task->request_pl = request;
>>>> + task->request_pl_sz = request_sz;
>>>> + task->response_pl = NULL;
>>>> + task->response_pl_sz = 0;
>>>> + task->complete = complete;
>>>> +
>>>> + rc = pci_ep_doe_submit_task(doe_mb, task);
>>>> + if (rc) {
>>>> + kfree(request);
>>>> + kfree(task);
>>>> + return rc;
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(pci_ep_doe_process_request);
>>>
>>> So who is supposed to call this API? EPC driver that receives the DOE interrupt?
>>
>> Yes, the EPC drivers that receive the DOE interrupts are expected to
>> call this API.
>>
>>> But I don't see the any callers of this and below exported APIs in this series.
>>> Either you should add the callers or limit this series just to adding the DOE
>>> skeleton implementation with a clear follow-up.
>>
>> I currently am working on the EPC driver implementation for a platform
>> which has not been up-streamed yet. I plan to use these APIs to support
>> the DOE feature for that driver. Currently, I am not aware of any
>> platform whose EPC driver supports DOE feature and its interrupts, hence
>> I see no real callers of these APIs to include in this patch series.
>>
>> Would it be appropriate to add a dummy [NOT-FOR-MERGING] demonstration
>> patch over an existing EPC driver, showing how these DOE APIs would be
>> integrated into an EPC driver?
>>
>
> Usually we don't add APIs without any callers. But if you have a realistic time
> frame and guarantee that you are going to add EPC driver support soon, then we
> can have these APIs merged first.
>
Hi Mani,
The expected timeline for adding the upstream support for the platform
is by the end of Q3. Once it gets merged, we would post the patches to
add its EPC as well as downstream driver support on top of it.
> For demonstration purpose, you can just show the EPC integration as a snippet in
> cover letter or point to the downstream driver for reference (if it is not a
> secret sauce).
Sure, I will add a dummy EPC integration code in the cover letter to
demonstrate the usage of those APIs.
Thanks.
>
> - Mani
>
^ permalink raw reply
* Re: [PATCH 1/2] spi: aspeed: Fix missing __iomem annotation in output transfer path
From: Cédric Le Goater @ 2026-05-19 5:15 UTC (permalink / raw)
To: Chin-Ting Kuo, broonie, joel, andrew, linux-aspeed, openbmc,
linux-spi, linux-arm-kernel, linux-kernel, BMC-SW
Cc: kernel test robot
In-Reply-To: <20260518095708.2502537-2-chin-ting_kuo@aspeedtech.com>
On 5/18/26 11:57, Chin-Ting Kuo wrote:
> The dst parameter of aspeed_spi_user_transfer_tx() is an MMIO address
> obtained from chip->ahb_base, but it was typed as void * instead of
> void __iomem *. This caused a sparse warning report. Fix the
> parameter type to void __iomem * and drop the now-unnecessary
> cast at the call site.
>
> Reported-by: kernel test robot <lkp@intel.com>
> Closes: https://lore.kernel.org/oe-kbuild-all/202605180441.uD3toFRJ-lkp@intel.com/
> Signed-off-by: Chin-Ting Kuo <chin-ting_kuo@aspeedtech.com>
> ---
> drivers/spi/spi-aspeed-smc.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/spi/spi-aspeed-smc.c b/drivers/spi/spi-aspeed-smc.c
> index c21323e07d3c..808659a1f460 100644
> --- a/drivers/spi/spi-aspeed-smc.c
> +++ b/drivers/spi/spi-aspeed-smc.c
> @@ -891,7 +891,7 @@ static int aspeed_spi_user_unprepare_msg(struct spi_controller *ctlr,
> static void aspeed_spi_user_transfer_tx(struct aspeed_spi *aspi,
> struct spi_device *spi,
> const u8 *tx_buf, u8 *rx_buf,
> - void *dst, u32 len)
> + void __iomem *dst, u32 len)
> {
> const struct aspeed_spi_data *data = aspi->data;
> bool full_duplex_transfer = data->full_duplex && tx_buf == rx_buf;
> @@ -936,7 +936,7 @@ static int aspeed_spi_user_transfer(struct spi_controller *ctlr,
> aspeed_spi_set_io_mode(chip, CTRL_IO_QUAD_DATA);
>
> aspeed_spi_user_transfer_tx(aspi, spi, tx_buf, rx_buf,
> - (void *)ahb_base, xfer->len);
> + ahb_base, xfer->len);
> }
>
> if (rx_buf && rx_buf != tx_buf) {
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Thanks,
C.
^ permalink raw reply
* [PATCH v4 08/24] iommu: Change group->devices to RCU-protected list
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
To allow lockless iterations of the group->devices list in an ISR context
that cannot hold the group->mutex, change the list to be RCU protected.
Mark the existing __dev_to_gdev() for group->mutex case only. A subsequent
change will add another __dev_to_gdev_rcu() for RCU case.
Hold grp_dev->dev across the RCU grace period using synchronize_rcu(), in
__iommu_group_free_device(). Without that, the driver core might free the
struct device while an RCU reader is still mid-iteration.
Note: a call_rcu() callback runs in softirq context, but put_device() may
sleep -- the device release path can invoke devres_release_all() and
->release callbacks that take mutexes. Use synchronize_rcu() to defer the
put_device() to the (sleepable) caller context instead.
Note that in bus_iommu_probe() there is a for_each_group_device marked as
FIXME, which can't take either mutex or RCU read lock. Plainly replace it
with list_for_each_entry for a status quo.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/iommu.c | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 2f8f3ea13f490..4116b28258bde 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -97,8 +97,10 @@ struct group_device {
/* Iterate over each struct group_device in a struct iommu_group */
#define for_each_group_device(group, pos) \
- list_for_each_entry(pos, &(group)->devices, list)
+ list_for_each_entry_rcu(pos, &(group)->devices, list, \
+ lockdep_is_held(&(group)->mutex))
+/* Caller must hold dev->iommu_group->mutex. */
static struct group_device *__dev_to_gdev(struct device *dev)
{
struct iommu_group *group = dev->iommu_group;
@@ -688,7 +690,7 @@ static int __iommu_probe_device(struct device *dev, struct list_head *group_list
* The gdev must be in the list before calling
* iommu_setup_default_domain()
*/
- list_add_tail(&gdev->list, &group->devices);
+ list_add_tail_rcu(&gdev->list, &group->devices);
WARN_ON(group->default_domain && !group->domain);
if (group->default_domain)
iommu_create_device_direct_mappings(group->default_domain, dev);
@@ -719,7 +721,7 @@ static int __iommu_probe_device(struct device *dev, struct list_head *group_list
return 0;
err_remove_gdev:
- list_del(&gdev->list);
+ list_del_rcu(&gdev->list);
__iommu_group_empty_assert_owner_cnt(group);
err_put_group:
iommu_deinit_device(dev);
@@ -762,6 +764,10 @@ static void __iommu_group_free_device(struct iommu_group *group,
trace_remove_device_from_group(group->id, dev);
kfree(grp_dev->name);
+
+ /* Wait for any in-flight reader to drop the reference to gdev->dev */
+ synchronize_rcu();
+ put_device(grp_dev->dev);
kfree(grp_dev);
}
@@ -779,7 +785,7 @@ static void __iommu_group_remove_device(struct device *dev)
/* Must drop the recovery_cnt when removing a blocked device */
if (device->blocked && !WARN_ON(group->recovery_cnt == 0))
group->recovery_cnt--;
- list_del(&device->list);
+ list_del_rcu(&device->list);
__iommu_group_empty_assert_owner_cnt(group);
if (dev_has_iommu(dev))
iommu_deinit_device(dev);
@@ -1298,6 +1304,8 @@ static struct group_device *iommu_group_alloc_device(struct iommu_group *group,
return ERR_PTR(-ENOMEM);
device->dev = dev;
+ /* Keep dev alive for any in-flight RCU reader of grp_dev->dev. */
+ get_device(dev);
ret = sysfs_create_link(&dev->kobj, &group->kobj, "iommu_group");
if (ret)
@@ -1337,6 +1345,7 @@ static struct group_device *iommu_group_alloc_device(struct iommu_group *group,
err_remove_link:
sysfs_remove_link(&dev->kobj, "iommu_group");
err_free_device:
+ put_device(dev);
kfree(device);
dev_err(dev, "Failed to add to iommu group %d: %d\n", group->id, ret);
return ERR_PTR(ret);
@@ -1362,7 +1371,7 @@ int iommu_group_add_device(struct iommu_group *group, struct device *dev)
rcu_assign_pointer(dev_iommu_group_rcu(dev), group);
mutex_lock(&group->mutex);
- list_add_tail(&gdev->list, &group->devices);
+ list_add_tail_rcu(&gdev->list, &group->devices);
mutex_unlock(&group->mutex);
return 0;
}
@@ -2011,9 +2020,11 @@ static int bus_iommu_probe(const struct bus_type *bus)
* FIXME: Mis-locked because the ops->probe_finalize() call-back
* of some IOMMU drivers calls arm_iommu_attach_device() which
* in-turn might call back into IOMMU core code, where it tries
- * to take group->mutex, resulting in a deadlock.
+ * to take group->mutex, resulting in a deadlock. Unfortunately,
+ * as iommu_group_do_probe_finalize() can sleep, rcu_read_lock()
+ * cannot be held to mitigate this.
*/
- for_each_group_device(group, gdev)
+ list_for_each_entry(gdev, &group->devices, list)
iommu_group_do_probe_finalize(gdev->dev);
}
--
2.43.0
^ permalink raw reply related
* Re: [REGRESSION] Bluetooth: MT7922 fails to initialize after "Bluetooth: btmtk: validate WMT event SKB length before struct access"
From: Thorsten Leemhuis @ 2026-05-19 5:04 UTC (permalink / raw)
To: Baley Eccles, linux-bluetooth
Cc: Marcel Holtmann, Luiz Augusto von Dentz, Matthias Brugger,
AngeloGioacchino Del Regno, linux-kernel, linux-arm-kernel,
linux-mediatek, regressions, stable
In-Reply-To: <CADCSNFD0Ut-jJohTQFczjBgaVf=mBrc2rq4hJQncVZpF4bCoxw@mail.gmail.com>
On 5/19/26 06:31, Baley Eccles wrote:
> Subject: [REGRESSION] Bluetooth: MT7922 fails to initialize after
> "Bluetooth: btmtk: validate WMT event SKB length before struct access"
>
> Hi all,
>
> I have experienced and looked into a regression on a MediaTek MT7922
> adapter. Bluetooth works on v6.18.29, fails on v6.18.30, and reverting
> the bisected commit fixes it.
Thx for the report, there are quite a few similar ones already; the
problem is known and the fix (see the link below) should be heading to
mainline this week and from there go to various stable series.
Ciao, Thorsten
#regzbot dup:
https://lore.kernel.org/linux-bluetooth/770d36b07311bf88210c187923f243fb9f126f04.1777058551.git.pav@iki.fi/
> Hardware:
> MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:7922]
> Subsystem: AzureWave ASUS PCE-AXE59BT [1a3b:5300]
>
> Good kernel:
> 6.18.29-p2-gentoo-dist
> Upstream base: v6.18.29
>
> Bad kernel:
> 6.18.30-p1-gentoo-dist
> Upstream base: v6.18.30
>
> Failure:
> bluetoothctl list prints nothing / no default controller is available.
>
> Bad dmesg:
> Bluetooth: hci0: HW/SW Version: 0x008a008a, Build Time: 20260224103448
> Bluetooth: hci0: Failed to send wmt func ctrl (-22)
> Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is
> advertised, but not supported.
>
> Good dmesg:
> Bluetooth: hci0: HW/SW Version: 0x008a008a, Build Time: 20260224103448
> Bluetooth: hci0: Device setup in 129909 usecs
> Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is
> advertised, but not supported.
> Bluetooth: hci0: AOSP extensions version v1.00
> Bluetooth: hci0: AOSP quality report is supported
> Bluetooth: MGMT ver 1.23
>
> Firmware:
> /lib/firmware/mediatek/BT_RAM_CODE_MT7922_1_1_hdr.bin
> /lib/firmware/mediatek/WIFI_MT7922_patch_mcu_1_1_hdr.bin
> /lib/firmware/mediatek/WIFI_RAM_CODE_MT7922_1.bin
>
> Bisect result:
> 624fb79dadc1b65757986a9d0fdde5c0cf3fe179 is the first bad commit
>
> Bluetooth: btmtk: validate WMT event SKB length before struct access
>
> This is a stable backport of upstream commit:
> 634a4408c0615c523cf7531790f4f14a422b9206
>
> Reverting 624fb79dadc1b65757986a9d0fdde5c0cf3fe179 on top of v6.18.30
> fixes the issue and Bluetooth works again.
>
> Please let me know if there is any additional logging or testing I can provide.
>
> #regzbot introduced: 624fb79dadc1b65757986a9d0fdde5c0cf3fe179
>
> Cheers,
> Baley
^ permalink raw reply
* [REGRESSION] Bluetooth: MT7922 fails to initialize after "Bluetooth: btmtk: validate WMT event SKB length before struct access"
From: Baley Eccles @ 2026-05-19 4:31 UTC (permalink / raw)
To: linux-bluetooth
Cc: Marcel Holtmann, Luiz Augusto von Dentz, Matthias Brugger,
AngeloGioacchino Del Regno, linux-kernel, linux-arm-kernel,
linux-mediatek, regressions, stable
Subject: [REGRESSION] Bluetooth: MT7922 fails to initialize after
"Bluetooth: btmtk: validate WMT event SKB length before struct access"
Hi all,
I have experienced and looked into a regression on a MediaTek MT7922
adapter. Bluetooth works on v6.18.29, fails on v6.18.30, and reverting
the bisected commit fixes it.
Hardware:
MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:7922]
Subsystem: AzureWave ASUS PCE-AXE59BT [1a3b:5300]
Good kernel:
6.18.29-p2-gentoo-dist
Upstream base: v6.18.29
Bad kernel:
6.18.30-p1-gentoo-dist
Upstream base: v6.18.30
Failure:
bluetoothctl list prints nothing / no default controller is available.
Bad dmesg:
Bluetooth: hci0: HW/SW Version: 0x008a008a, Build Time: 20260224103448
Bluetooth: hci0: Failed to send wmt func ctrl (-22)
Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is
advertised, but not supported.
Good dmesg:
Bluetooth: hci0: HW/SW Version: 0x008a008a, Build Time: 20260224103448
Bluetooth: hci0: Device setup in 129909 usecs
Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is
advertised, but not supported.
Bluetooth: hci0: AOSP extensions version v1.00
Bluetooth: hci0: AOSP quality report is supported
Bluetooth: MGMT ver 1.23
Firmware:
/lib/firmware/mediatek/BT_RAM_CODE_MT7922_1_1_hdr.bin
/lib/firmware/mediatek/WIFI_MT7922_patch_mcu_1_1_hdr.bin
/lib/firmware/mediatek/WIFI_RAM_CODE_MT7922_1.bin
Bisect result:
624fb79dadc1b65757986a9d0fdde5c0cf3fe179 is the first bad commit
Bluetooth: btmtk: validate WMT event SKB length before struct access
This is a stable backport of upstream commit:
634a4408c0615c523cf7531790f4f14a422b9206
Reverting 624fb79dadc1b65757986a9d0fdde5c0cf3fe179 on top of v6.18.30
fixes the issue and Bluetooth works again.
Please let me know if there is any additional logging or testing I can provide.
#regzbot introduced: 624fb79dadc1b65757986a9d0fdde5c0cf3fe179
Cheers,
Baley
^ permalink raw reply
* Re: [PATCH] media: mediatek: mdp: avoid double free on video register failure
From: Guangshuo Li @ 2026-05-19 4:14 UTC (permalink / raw)
To: kernel test robot
Cc: Minghsiu Tsai, Houlong Wei, Andrew-CT Chen, Mauro Carvalho Chehab,
Matthias Brugger, AngeloGioacchino Del Regno, Hans Verkuil,
linux-kernel, linux-arm-kernel, linux-mediatek, llvm,
oe-kbuild-all, linux-media
In-Reply-To: <202605190845.KlMSPp80-lkp@intel.com>
Thanks for the report.
On Tue, 19 May 2026 at 10:13, kernel test robot <lkp@intel.com> wrote:
>
> Hi Guangshuo,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on linuxtv-media-pending/master]
> [also build test ERROR on media-tree/master linus/master v7.1-rc4 next-20260518]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
>
> url: https://github.com/intel-lab-lkp/linux/commits/Guangshuo-Li/media-mediatek-mdp-avoid-double-free-on-video-register-failure/20260518-211648
> base: https://git.linuxtv.org/media-ci/media-pending.git master
> patch link: https://lore.kernel.org/r/20260518125500.1000083-1-lgs201920130244%40gmail.com
> patch subject: [PATCH] media: mediatek: mdp: avoid double free on video register failure
> config: hexagon-allmodconfig (https://download.01.org/0day-ci/archive/20260519/202605190845.KlMSPp80-lkp@intel.com/config)
> compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
> reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260519/202605190845.KlMSPp80-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202605190845.KlMSPp80-lkp@intel.com/
>
> All errors (new ones prefixed by >>):
>
> >> drivers/media/platform/mediatek/mdp/mtk_mdp_m2m.c:1217:33: error: expected ';' after expression
> 1217 | video_device_release(mdp->vdev)
> | ^
> | ;
> 1 error generated.
>
>
> vim +1217 drivers/media/platform/mediatek/mdp/mtk_mdp_m2m.c
>
> 1172
> 1173 int mtk_mdp_register_m2m_device(struct mtk_mdp_dev *mdp)
> 1174 {
> 1175 struct device *dev = &mdp->pdev->dev;
> 1176 int ret;
> 1177
> 1178 mdp->variant = &mtk_mdp_default_variant;
> 1179 mdp->vdev = video_device_alloc();
> 1180 if (!mdp->vdev) {
> 1181 dev_err(dev, "failed to allocate video device\n");
> 1182 ret = -ENOMEM;
> 1183 goto err_video_alloc;
> 1184 }
> 1185 mdp->vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> 1186 mdp->vdev->fops = &mtk_mdp_m2m_fops;
> 1187 mdp->vdev->ioctl_ops = &mtk_mdp_m2m_ioctl_ops;
> 1188 mdp->vdev->release = video_device_release_empty;
> 1189 mdp->vdev->lock = &mdp->lock;
> 1190 mdp->vdev->vfl_dir = VFL_DIR_M2M;
> 1191 mdp->vdev->v4l2_dev = &mdp->v4l2_dev;
> 1192 snprintf(mdp->vdev->name, sizeof(mdp->vdev->name), "%s:m2m",
> 1193 MTK_MDP_MODULE_NAME);
> 1194 video_set_drvdata(mdp->vdev, mdp);
> 1195
> 1196 mdp->m2m_dev = v4l2_m2m_init(&mtk_mdp_m2m_ops);
> 1197 if (IS_ERR(mdp->m2m_dev)) {
> 1198 dev_err(dev, "failed to initialize v4l2-m2m device\n");
> 1199 ret = PTR_ERR(mdp->m2m_dev);
> 1200 goto err_m2m_init;
> 1201 }
> 1202
> 1203 ret = video_register_device(mdp->vdev, VFL_TYPE_VIDEO, 2);
> 1204 if (ret) {
> 1205 dev_err(dev, "failed to register video device\n");
> 1206 goto err_vdev_register;
> 1207 }
> 1208 mdp->vdev->release = video_device_release;
> 1209
> 1210 v4l2_info(&mdp->v4l2_dev, "driver registered as /dev/video%d",
> 1211 mdp->vdev->num);
> 1212 return 0;
> 1213
> 1214 err_vdev_register:
> 1215 v4l2_m2m_release(mdp->m2m_dev);
> 1216 err_m2m_init:
> > 1217 video_device_release(mdp->vdev)
> 1218 mdp->vdev = NULL;
> 1219 err_video_alloc:
> 1220
> 1221 return ret;
> 1222 }
> 1223
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
This build failure was caused by my oversight. I missed the semicolon after
video_device_release(mdp->vdev).
I will send a v2 to fix this issue.
Sorry for the noise.
^ permalink raw reply
* [PATCH v4 24/24] iommu/arm-smmu-v3: Block ATS upon an ATC invalidation timeout
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Currently, when GERROR_CMDQ_ERR occurs, the arm_smmu_cmdq_skip_err() won't
do anything for the CMDQ_ERR_CERROR_ATC_INV_IDX.
When a device wasn't responsive to an ATC invalidation request, this often
results in constant CMDQ errors:
unexpected global error reported (0x00000001), this could be serious
CMDQ error (cons 0x0302bb84): ATC invalidate timeout
unexpected global error reported (0x00000001), this could be serious
CMDQ error (cons 0x0302bb88): ATC invalidate timeout
unexpected global error reported (0x00000001), this could be serious
CMDQ error (cons 0x0302bb8c): ATC invalidate timeout
...
An ATC invalidation timeout indicates that the device failed to respond to
a protocol-critical coherency request, which means that device's internal
ATS state is desynchronized from the SMMU.
Furthermore, ignoring the timeout leaves the system in an unsafe state, as
the device cache may retain stale ATC entries for memory pages that the OS
has already reclaimed and reassigned. This might lead to data corruption.
Isolate the device that is confirmed to be unresponsive by a surgical STE
update to unset its EATS bit so as to reject any further ATS transaction,
which could corrupt the memory.
Also, set the master->ats_broken flag that is revertible after the device
completes a reset. This flag avoids further ATS requests and invalidations
from happening.
Finally, report this broken device to the IOMMU core to isolate the device
in the core level too.
Since the three steps above are invoked in an invalidation path (which can
be an atomic context), hold the ats_broken_lock instead of any mutex.
For batched ATC_INV commands, SMMU hardware only reports a timeout at the
CMD_SYNC, which could follow the batch issued for multiple devices. So, it
isn't straightforward to identify which command in a batch resulted in the
timeout. Fortunately, the invs array has a sorted list of ATC entries. So,
the issued batch must be sorted as well. This makes it possible to retry
the ATC_INV command for each unique Stream ID in the batch to identify the
unresponsive master.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 +++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 118 +++++++++++++++++++-
2 files changed, 133 insertions(+), 3 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index e3eb4c4a62d3a..43d4a35500500 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -831,6 +831,24 @@ arm_smmu_invs_iter_next(struct arm_smmu_invs *invs, size_t next, size_t *idx)
for (cur = arm_smmu_invs_iter_next(invs, 0, &(idx)); cur; \
cur = arm_smmu_invs_iter_next(invs, idx + 1, &(idx)))
+static inline struct arm_smmu_master *
+arm_smmu_invs_find_ats_master(struct arm_smmu_invs *invs,
+ struct arm_smmu_device *smmu, u32 sid)
+{
+ struct arm_smmu_inv *cur;
+ size_t i;
+
+ if (!invs->has_ats)
+ return NULL;
+
+ arm_smmu_invs_for_each_entry(invs, i, cur) {
+ if (cur->smmu == smmu && arm_smmu_inv_is_ats(cur) &&
+ cur->id == sid)
+ return cur->master;
+ }
+ return NULL;
+}
+
static inline struct arm_smmu_invs *arm_smmu_invs_alloc(size_t num_invs)
{
struct arm_smmu_invs *new_invs;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index ee864046f0baa..0323fd3f33b7f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -107,8 +107,13 @@ static const char * const event_class_str[] = {
[3] = "Reserved",
};
+static struct arm_smmu_ste *
+arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid);
static int arm_smmu_alloc_cd_tables(struct arm_smmu_master *master);
static bool arm_smmu_ats_supported(struct arm_smmu_master *master);
+static void arm_smmu_cmdq_batch_retry(struct arm_smmu_device *smmu,
+ struct arm_smmu_invs *invs,
+ struct arm_smmu_cmdq_batch *cmds);
static void parse_driver_options(struct arm_smmu_device *smmu)
{
@@ -905,8 +910,13 @@ static int arm_smmu_cmdq_batch_issue(struct arm_smmu_device *smmu,
struct arm_smmu_cmdq_batch *cmds,
bool sync)
{
- return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
- cmds->num, sync);
+ int ret = arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
+ cmds->num, sync);
+
+ /* Identify the timed-out master via cmds->invs */
+ if (ret == -EIO && cmds->invs)
+ arm_smmu_cmdq_batch_retry(smmu, cmds->invs, cmds);
+ return ret;
}
static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
@@ -924,7 +934,11 @@ static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
}
if (cmds->num == CMDQ_BATCH_ENTRIES) {
- arm_smmu_cmdq_batch_issue(smmu, cmds, false);
+ /*
+ * Force sync for ATS-bearing batches so the timeout is caught
+ * here, not at a later unrelated batch's CMD_SYNC.
+ */
+ arm_smmu_cmdq_batch_issue(smmu, cmds, cmds->has_ats);
arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd, cmds->invs);
}
@@ -945,6 +959,104 @@ static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu,
return arm_smmu_cmdq_batch_issue(smmu, cmds, true);
}
+static void arm_smmu_master_disable_ats(struct arm_smmu_master *master)
+{
+ struct arm_smmu_cmd cmd = arm_smmu_make_cmd_op(CMDQ_OP_CFGI_STE);
+ struct arm_smmu_device *smmu = master->smmu;
+ struct arm_smmu_cmdq_batch cmds;
+ struct arm_smmu_inv *cur;
+ size_t i;
+
+ lockdep_assert_held(&master->ats_broken_lock);
+
+ /* Disable STE.EATS on every SID */
+ arm_smmu_cmdq_batch_init_cmd(smmu, &cmds, &cmd, NULL);
+ arm_smmu_invs_for_each_entry(master->ats_invs, i, cur) {
+ struct arm_smmu_ste *step =
+ arm_smmu_get_step_for_sid(smmu, cur->id);
+
+ /* EATS is safe to update. See arm_smmu_get_ste_update_safe() */
+ WRITE_ONCE(step->data[1],
+ step->data[1] & ~cpu_to_le64(STRTAB_STE_1_EATS));
+
+ arm_smmu_cmdq_batch_add_cmd(
+ smmu, &cmds, arm_smmu_make_cmd_cfgi_ste(cur->id, true));
+ }
+ if (arm_smmu_cmdq_batch_submit(smmu, &cmds))
+ dev_err_ratelimited(smmu->dev,
+ "failed to disable ATS for master\n");
+
+ /* Pair with lockless readers */
+ WRITE_ONCE(master->ats_broken, true);
+
+ /* Lastly, report to the core to schedule a full blocking procedure */
+ iommu_report_device_broken(master->dev);
+
+ /*
+ * When a concurrent pci_dev_reset_iommu_done() runs after this report
+ * (e.g. an AER recovery in flight), the broken_worker may transiently
+ * block a recovering device. pci_dev_reset_iommu_done() will lift it
+ * immediately. Net end-state is correct.
+ */
+}
+
+static void arm_smmu_cmdq_batch_retry(struct arm_smmu_device *smmu,
+ struct arm_smmu_invs *invs,
+ struct arm_smmu_cmdq_batch *cmds)
+{
+ struct arm_smmu_cmd atc = {};
+ int i;
+
+ /* Only a timed out ATC_INV command needs a retry */
+ if (!invs->has_ats)
+ return;
+
+ for (i = 0; i < cmds->num; i++) {
+ struct arm_smmu_cmdq *cmdq = cmds->cmdq;
+ struct arm_smmu_master *master = NULL;
+ unsigned long flags;
+ u32 sid;
+ int ret;
+
+ /* Only need to retry ATC invalidations */
+ if (FIELD_GET(CMDQ_0_OP, cmds->cmds[i].data[0]) !=
+ CMDQ_OP_ATC_INV)
+ continue;
+
+ /* Only need to retry with one ATC_INV per Stream ID (device) */
+ sid = FIELD_GET(CMDQ_ATC_0_SID, cmds->cmds[i].data[0]);
+ if (atc.data[0] &&
+ sid == FIELD_GET(CMDQ_ATC_0_SID, atc.data[0]))
+ continue;
+
+ master = arm_smmu_invs_find_ats_master(invs, smmu, sid);
+ if (WARN_ON(!master))
+ continue;
+
+ atc = cmds->cmds[i];
+ /*
+ * Hold ats_broken_lock across the per-master re-issue and the
+ * possible disable_ats, so a concurrent reset_device_done()
+ * cannot clear ats_broken between the timeout observation and
+ * the quarantine action.
+ */
+ spin_lock_irqsave(&master->ats_broken_lock, flags);
+ /*
+ * A previous retry on a sibling SID may have already disabled
+ * ATS across all the STEs owned by this master's SIDs. Skip it.
+ */
+ if (master->ats_broken) {
+ spin_unlock_irqrestore(&master->ats_broken_lock, flags);
+ continue;
+ }
+
+ ret = arm_smmu_cmdq_issue_cmdlist(smmu, cmdq, &atc, 1, true);
+ if (ret == -EIO)
+ arm_smmu_master_disable_ats(master);
+ spin_unlock_irqrestore(&master->ats_broken_lock, flags);
+ }
+}
+
static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused,
struct iommu_page_response *resp)
{
--
2.43.0
^ permalink raw reply related
* [PATCH v4 15/24] iommu/arm-smmu-v3: Co-clear pending CMDQ_ERR when CMD_SYNC times out
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Once arm_smmu_cmdq_poll_until_sync() returns, arm_smmu_cmdq_issue_cmdlist()
tests its CMD_SYNC slot in atc_sync_timeouts to decide whether there was an
ATC_INV timeout.
On the other hand, when that poll timed out, the GERROR ISR might have been
delayed past the poll deadline, so the atc_sync_timeouts test could miss an
ATC_INV timeout, classifying it as a generic CMD_SYNC timeout and bypassing
the per-device quarantine.
Add two cmdq_err_handler impl functions:
- arm_smmu_cmdq_err_handler() reads SMMU GERROR/GERRORN.
- tegra241_vcmdq_handle_cmdq_err() reads VCMDQ GERROR/GERRORN.
Co-clear any pending CMDQ_ERR in the issuer, when the polling on a CMD_SYNC
times out. Each cmdq impl serializes the synchronous drain against its own
IRQ handler with cmdq->cmdq_err_lock.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 36 ++++++++++++++++++-
.../iommu/arm/arm-smmu-v3/tegra241-cmdqv.c | 23 +++++++++++-
2 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index fc0757359b783..7f81fd2e92480 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -813,6 +813,15 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu,
sync_prod = llq.prod;
ret = arm_smmu_cmdq_poll_until_sync(smmu, cmdq, &llq);
+ /*
+ * When the poll above timed out, the GERROR ISR might have been
+ * delayed past the poll deadline, so the atc_sync_timeouts test
+ * below could miss our ATC_INV timeout. Thus, drain any pending
+ * CMDQ_ERR synchronously first via the per-cmdq callback.
+ */
+ if (ret && cmdq->cmdq_err_handler)
+ cmdq->cmdq_err_handler(smmu, cmdq);
+
/*
* Test atc_sync_timeouts first and see if there is ATC timeout
* resulted from this cmdlist. Return -EIO to separate from the
@@ -2251,6 +2260,31 @@ static irqreturn_t arm_smmu_priq_thread(int irq, void *dev)
static int arm_smmu_device_disable(struct arm_smmu_device *smmu);
+/*
+ * Drain a pending CMDQ_ERR on the primary cmdq. Installed as the primary
+ * cmdq's cmdq_err_handler so arm_smmu_cmdq_issue_cmdlist() can drain after
+ * a CMD_SYNC poll timeout; serialized against arm_smmu_gerror_handler() by
+ * cmdq->cmdq_err_lock.
+ */
+static void arm_smmu_cmdq_err_handler(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq *cmdq)
+{
+ u32 gerror, gerrorn;
+
+ guard(raw_spinlock_irqsave)(&cmdq->cmdq_err_lock);
+
+ gerror = readl_relaxed(smmu->base + ARM_SMMU_GERROR);
+ gerrorn = readl_relaxed(smmu->base + ARM_SMMU_GERRORN);
+
+ if (!((gerror ^ gerrorn) & GERROR_CMDQ_ERR))
+ return;
+
+ __arm_smmu_cmdq_skip_err(smmu, cmdq);
+
+ /* Toggle only the CMDQ_ERR bit; other bits are left for the ISR. */
+ writel(gerrorn ^ GERROR_CMDQ_ERR, smmu->base + ARM_SMMU_GERRORN);
+}
+
static irqreturn_t arm_smmu_gerror_handler(int irq, void *dev)
{
u32 gerror, gerrorn, active;
@@ -4399,7 +4433,7 @@ static int arm_smmu_init_queues(struct arm_smmu_device *smmu)
if (ret)
return ret;
- ret = arm_smmu_cmdq_init(smmu, &smmu->cmdq, NULL);
+ ret = arm_smmu_cmdq_init(smmu, &smmu->cmdq, arm_smmu_cmdq_err_handler);
if (ret)
return ret;
diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
index fb2f8f68fa344..e04107f0490c9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
+++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
@@ -337,6 +337,27 @@ static void tegra241_vintf0_handle_error(struct tegra241_vintf *vintf)
}
}
+static void tegra241_vcmdq_handle_cmdq_err(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq *cmdq)
+{
+ struct tegra241_vcmdq *vcmdq =
+ container_of(cmdq, struct tegra241_vcmdq, cmdq);
+ u32 gerror, gerrorn;
+
+ guard(raw_spinlock_irqsave)(&cmdq->cmdq_err_lock);
+
+ gerror = readl_relaxed(REG_VCMDQ_PAGE0(vcmdq, GERROR));
+ gerrorn = readl_relaxed(REG_VCMDQ_PAGE0(vcmdq, GERRORN));
+
+ if (!((gerror ^ gerrorn) & GERROR_CMDQ_ERR))
+ return;
+
+ __arm_smmu_cmdq_skip_err(smmu, cmdq);
+
+ /* Toggle only the CMDQ_ERR bit on this VCMDQ's GERRORN */
+ writel(gerrorn ^ GERROR_CMDQ_ERR, REG_VCMDQ_PAGE0(vcmdq, GERRORN));
+}
+
static irqreturn_t tegra241_cmdqv_isr(int irq, void *devid)
{
struct tegra241_cmdqv *cmdqv = (struct tegra241_cmdqv *)devid;
@@ -652,7 +673,7 @@ static int tegra241_vcmdq_alloc_smmu_cmdq(struct tegra241_vcmdq *vcmdq)
q->q_base = q->base_dma & VCMDQ_ADDR;
q->q_base |= FIELD_PREP(VCMDQ_LOG2SIZE, q->llq.max_n_shift);
- return arm_smmu_cmdq_init(smmu, cmdq, NULL);
+ return arm_smmu_cmdq_init(smmu, cmdq, tegra241_vcmdq_handle_cmdq_err);
}
/* VINTF Logical VCMDQ Resource Helpers */
--
2.43.0
^ permalink raw reply related
* [PATCH v4 16/24] iommu/arm-smmu-v3: Co-clear pending CMDQ_ERR when queue_has_space() fails
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
It's unusual when the command queue fails the queue_has_space() test. There
must be something stalling the HW so the queue does not advance.
Currently, a possible scenario: arm_smmu_cmdq_issue_cmdlist() may be called
in an IRQ context where IRQ is already disabled. E.g., ata_sg_clean() in
drivers/ata/libata-core.c
When GERROR is affined to the CPU currently running with IRQs disabled, the
GERROR ISR will not run and a CERROR_ILL will not be cleared, which stalls
the HW; arm_smmu_cmdq_poll_until_not_full() then either times out or loops
without seeing CONS advance.
The window is narrow and it's very difficult to trigger this lockup. Yet, a
subsequent change requires serializing the STE update routines between the
attach_dev path (mutex-ed) and the invalidation path (non-mutexed), where a
spin_lock_irqsave is inevitable. And this would expand the currently narrow
window to a wider range -- arm_smmu_write_ste() as well.
Since we have a cmdq_err_handler, call it when queue_has_space() fails, to
give the CMDQ hardware a chance to advance its CONS.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 7f81fd2e92480..0e4f34ed036c6 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -723,6 +723,13 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu,
while (!queue_has_space(&llq, n + sync)) {
local_irq_restore(flags);
+ /*
+ * If the CMDQ is nearly full, it's possible that the HW
+ * is stalled by an unhandled GERROR_CMDQ_ERR. Thus give
+ * cmdq_err_handler a chance before each poll.
+ */
+ if (cmdq->cmdq_err_handler)
+ cmdq->cmdq_err_handler(smmu, cmdq);
if (arm_smmu_cmdq_poll_until_not_full(smmu, cmdq, &llq))
dev_err_ratelimited(smmu->dev, "CMDQ timeout\n");
local_irq_save(flags);
--
2.43.0
^ permalink raw reply related
* [PATCH v4 20/24] iommu/arm-smmu-v3: Introduce arm_smmu_cmdq_batch_issue() wrapper
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Both arm_smmu_cmdq_batch_submit() and arm_smmu_cmdq_batch_add_cmd_p() call
arm_smmu_cmdq_issue_cmdlist() to flush batches. A future change will retry
the issued commands on -EIO, using the arm_smmu_invs carried in the batch.
So, a single hook point is preferred.
Introduce an arm_smmu_cmdq_batch_issue() wrapper, so a retry logic will be
simply filled into the wrapper.
No functional changes.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index a31f8b1a94979..4f2b23b1e8163 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -901,6 +901,14 @@ static void arm_smmu_cmdq_batch_init_cmd(struct arm_smmu_device *smmu,
cmds->has_ats = false;
}
+static int arm_smmu_cmdq_batch_issue(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq_batch *cmds,
+ bool sync)
+{
+ return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
+ cmds->num, sync);
+}
+
static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
struct arm_smmu_cmdq_batch *cmds,
struct arm_smmu_cmd *cmd)
@@ -911,14 +919,12 @@ static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
unsupported_cmd = !arm_smmu_cmdq_supports_cmd(cmds->cmdq, cmd);
if (force_sync || unsupported_cmd) {
- arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
- cmds->num, true);
+ arm_smmu_cmdq_batch_issue(smmu, cmds, true);
arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd, cmds->invs);
}
if (cmds->num == CMDQ_BATCH_ENTRIES) {
- arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
- cmds->num, false);
+ arm_smmu_cmdq_batch_issue(smmu, cmds, false);
arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd, cmds->invs);
}
@@ -936,8 +942,7 @@ static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
static int arm_smmu_cmdq_batch_submit(struct arm_smmu_device *smmu,
struct arm_smmu_cmdq_batch *cmds)
{
- return arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
- cmds->num, true);
+ return arm_smmu_cmdq_batch_issue(smmu, cmds, true);
}
static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused,
--
2.43.0
^ permalink raw reply related
* [PATCH v4 19/24] iommu/arm-smmu-v3: Add invs and has_ats to struct arm_smmu_cmdq_batch
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
The arm_smmu_cmdq_batch_add_cmd_p() might flush a sub-batch mid-way, when
the ARM_SMMU_OPT_CMDQ_FORCE_SYNC is set or when a batch is full. To allow
a future change to retry these sub-batch flushes on a timeout and identify
the broken master, the batch needs to carry both the per-domain invs and
a per-batch indicator of whether the batch contains an ATC_INV.
Add an "invs" pointer to record the per-domain invalidation array (set via
a new arm_smmu_cmdq_batch_init_cmd() parameter), and a "has_ats" flag set
in arm_smmu_cmdq_batch_add_cmd_p() when an ATC_INV command is queued. Any
caller that does not associate a batch with an invs array can pass NULL.
No functional changes.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 ++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 +++++++++++------
2 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 44956daf83dfa..2074814534fef 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -721,6 +721,10 @@ static inline bool arm_smmu_cmdq_supports_cmd(struct arm_smmu_cmdq *cmdq,
struct arm_smmu_cmdq_batch {
struct arm_smmu_cmd cmds[CMDQ_BATCH_ENTRIES];
struct arm_smmu_cmdq *cmdq;
+ /* Per-domain invalidation array, for sub-batch retry-on-EIO lookup */
+ struct arm_smmu_invs *invs;
+ /* Set when an ATC_INV is queued; gates the retry-aware sync decision */
+ bool has_ats;
int num;
};
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 638956e2535b4..a31f8b1a94979 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -892,10 +892,13 @@ static int arm_smmu_cmdq_issue_cmd_p(struct arm_smmu_device *smmu,
static void arm_smmu_cmdq_batch_init_cmd(struct arm_smmu_device *smmu,
struct arm_smmu_cmdq_batch *cmds,
- struct arm_smmu_cmd *cmd)
+ struct arm_smmu_cmd *cmd,
+ struct arm_smmu_invs *invs)
{
cmds->num = 0;
cmds->cmdq = arm_smmu_get_cmdq(smmu, cmd);
+ cmds->invs = invs;
+ cmds->has_ats = false;
}
static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
@@ -910,15 +913,17 @@ static void arm_smmu_cmdq_batch_add_cmd_p(struct arm_smmu_device *smmu,
if (force_sync || unsupported_cmd) {
arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
cmds->num, true);
- arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd);
+ arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd, cmds->invs);
}
if (cmds->num == CMDQ_BATCH_ENTRIES) {
arm_smmu_cmdq_issue_cmdlist(smmu, cmds->cmdq, cmds->cmds,
cmds->num, false);
- arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd);
+ arm_smmu_cmdq_batch_init_cmd(smmu, cmds, cmd, cmds->invs);
}
+ if (FIELD_GET(CMDQ_0_OP, cmd->data[0]) == CMDQ_OP_ATC_INV)
+ cmds->has_ats = true;
cmds->cmds[cmds->num++] = *cmd;
}
@@ -1486,7 +1491,7 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master,
struct arm_smmu_device *smmu = master->smmu;
struct arm_smmu_cmd cmd = arm_smmu_make_cmd_cfgi_cd(0, ssid, leaf);
- arm_smmu_cmdq_batch_init_cmd(smmu, &cmds, &cmd);
+ arm_smmu_cmdq_batch_init_cmd(smmu, &cmds, &cmd, NULL);
for (i = 0; i < master->num_streams; i++)
arm_smmu_cmdq_batch_add_cmd(
smmu, &cmds,
@@ -2434,7 +2439,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
return 0;
cmd = arm_smmu_make_cmd_atc_inv_all(0, IOMMU_NO_PASID);
- arm_smmu_cmdq_batch_init_cmd(master->smmu, &cmds, &cmd);
+ arm_smmu_cmdq_batch_init_cmd(master->smmu, &cmds, &cmd, NULL);
for (i = 0; i < master->num_streams; i++)
arm_smmu_cmdq_batch_add_cmd(
master->smmu, &cmds,
@@ -2630,7 +2635,7 @@ static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
struct arm_smmu_inv *next;
if (!cmds.num)
- arm_smmu_cmdq_batch_init_cmd(smmu, &cmds, &cmd);
+ arm_smmu_cmdq_batch_init_cmd(smmu, &cmds, &cmd, invs);
switch (cur->type) {
case INV_TYPE_S1_ASID:
--
2.43.0
^ permalink raw reply related
* [PATCH v4 23/24] iommu/arm-smmu-v3: Serialize STE.EATS and ats_broken updates
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
A subsequent change adding an ATS-broken path will set master->ats_broken
flag and overwrite STE.EATS to 0 for a broken master. This will introduce
race conditions:
- A concurrent attachment that read ats_broken=false prior to ats_broken
being set to true could re-enable STE.EATS.
- A concurrent reset_device_done callback could clear master->ats_broken
during the ATS-broken path, leading to iommu_report_device_broken() on
a pre-reset fault.
- When the ATS-broken path reads the old_data[1] and writes (old_data[1]
& ~EATS), a concurrent attachment could write a new_data[1] in-between.
Due to an ATS-broken path can run in an atomic context (invalidation), it
cannot use mutex.
Introduce a per-master spinlock_t ats_broken_lock to fence these cases so
as to guarantee that in concurrent cases:
a) A master->ats_broken update is observed by every concurrent attach
b) STE.EATS is never re-enabled while master->ats_broken is true
c) data[1] writes are not lost to a concurrent ATS-broken path
Note IRQ has to be disabled while holding the lock, because an ATS-broken
path can be entered via a hardirq that might interrupt another caller and
lead to deadlock.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 +++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 ++++++++++++++++++++-
2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 186efcbed1ea9..e3eb4c4a62d3a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1048,6 +1048,11 @@ struct arm_smmu_master {
/* Scratch memory for arm_smmu_atc_inv_master() to build an ATS array */
struct arm_smmu_invs *ats_invs;
struct arm_smmu_vmaster *vmaster; /* use smmu->streams_mutex */
+ /*
+ * Serializes arm_smmu_write_ste(), reset_device_done, and an ATS-broken
+ * path, preventing races on ats_broken flag and STE updates.
+ */
+ spinlock_t ats_broken_lock;
/* Locked by the iommu core using the group mutex */
struct arm_smmu_ctx_desc_cfg cd_table;
unsigned int num_streams;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9591e4ab2b14a..ee864046f0baa 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1791,7 +1791,23 @@ static void arm_smmu_write_ste(struct arm_smmu_master *master, u32 sid,
.sid = sid,
};
- arm_smmu_write_entry(&ste_writer.writer, ste->data, target->data);
+ /*
+ * Fence against the ATS-broken path concurrently overwriting STE.EATS.
+ * It's fine if the ATS-broken path writes after arm_smmu_write_entry.
+ * Otherwise, we must clear STE.EATS before sending a CFGI_STE command.
+ *
+ * Must disable IRQs; otherwise a hardirq-context invalidation path on
+ * this CPU could deadlock at ats_broken_lock on an ATC_INV timeout.
+ */
+ scoped_guard(spinlock_irqsave, &master->ats_broken_lock) {
+ struct arm_smmu_ste local_target = *target;
+
+ if (master->ats_broken)
+ local_target.data[1] &=
+ ~cpu_to_le64(STRTAB_STE_1_EATS);
+ arm_smmu_write_entry(&ste_writer.writer, ste->data,
+ local_target.data);
+ }
/* It's likely that we'll want to use the new STE soon */
if (!(smmu->options & ARM_SMMU_OPT_SKIP_PREFETCH))
@@ -3000,6 +3016,9 @@ static void arm_smmu_reset_device_done(struct device *dev)
if (WARN_ON(!master))
return;
+
+ /* Ensure the device recovery is seen, to flush any pre-reset fault */
+ guard(spinlock_irqsave)(&master->ats_broken_lock);
/* Pair with lockless readers */
WRITE_ONCE(master->ats_broken, false);
}
@@ -4236,6 +4255,7 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
master->dev = dev;
master->smmu = smmu;
+ spin_lock_init(&master->ats_broken_lock);
dev_iommu_priv_set(dev, master);
ret = arm_smmu_insert_master(smmu, master);
--
2.43.0
^ permalink raw reply related
* [PATCH v4 18/24] iommu/arm-smmu-v3: Introduce master->ats_broken flag
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
A subsequent change will set this flag when IOMMU cannot trust device's ATS
function. E.g., when ATC invalidation request to the device times out.
Once the flag is set, unsupport the ATS feature to prevent data corruption,
and skip further ATC invalidation commands to avoid new timeouts.
Unset the flag when the device finishes a reset for recovery.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 29 +++++++++++++++++++++
2 files changed, 30 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index df6e539f75274..44956daf83dfa 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1017,6 +1017,7 @@ struct arm_smmu_master {
/* Locked by the iommu core using the group mutex */
struct arm_smmu_ctx_desc_cfg cd_table;
unsigned int num_streams;
+ bool ats_broken;
bool ats_enabled : 1;
bool ste_ats_enabled : 1;
bool stall_enabled;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index cde2ff2dcc49b..638956e2535b4 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2429,6 +2429,10 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
struct arm_smmu_cmd cmd;
struct arm_smmu_cmdq_batch cmds;
+ /* Do not issue ATC_INV that will definitely time out */
+ if (READ_ONCE(master->ats_broken))
+ return 0;
+
cmd = arm_smmu_make_cmd_atc_inv_all(0, IOMMU_NO_PASID);
arm_smmu_cmdq_batch_init_cmd(master->smmu, &cmds, &cmd);
for (i = 0; i < master->num_streams; i++)
@@ -2651,12 +2655,18 @@ static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
cur->id));
break;
case INV_TYPE_ATS:
+ /* Do not issue ATC_INV that will definitely time out */
+ if (READ_ONCE(cur->master->ats_broken))
+ break;
arm_smmu_cmdq_batch_add_cmd(
smmu, &cmds,
arm_smmu_atc_inv_to_cmd(cur->id, cur->ssid,
iova, size));
break;
case INV_TYPE_ATS_FULL:
+ /* Do not issue ATC_INV that will definitely time out */
+ if (READ_ONCE(cur->master->ats_broken))
+ break;
arm_smmu_cmdq_batch_add_cmd(
smmu, &cmds,
arm_smmu_make_cmd_atc_inv_all(cur->id,
@@ -2995,6 +3005,16 @@ void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master,
}
}
+static void arm_smmu_reset_device_done(struct device *dev)
+{
+ struct arm_smmu_master *master = dev_iommu_priv_get(dev);
+
+ if (WARN_ON(!master))
+ return;
+ /* Pair with lockless readers */
+ WRITE_ONCE(master->ats_broken, false);
+}
+
static bool arm_smmu_ats_supported(struct arm_smmu_master *master)
{
struct device *dev = master->dev;
@@ -3007,6 +3027,14 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master)
if (!(fwspec->flags & IOMMU_FWSPEC_PCI_RC_ATS))
return false;
+ /*
+ * Do not enable ATS if master->ats_broken is set. The PCI device should
+ * go through a recovery (reset) that shall notify the SMMUv3 driver via
+ * a reset_device_done callback.
+ */
+ if (READ_ONCE(master->ats_broken))
+ return false;
+
return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev));
}
@@ -4345,6 +4373,7 @@ static const struct iommu_ops arm_smmu_ops = {
.domain_alloc_paging_flags = arm_smmu_domain_alloc_paging_flags,
.probe_device = arm_smmu_probe_device,
.release_device = arm_smmu_release_device,
+ .reset_device_done = arm_smmu_reset_device_done,
.device_group = arm_smmu_device_group,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
--
2.43.0
^ permalink raw reply related
* [PATCH v4 22/24] iommu/arm-smmu-v3: Introduce master->ats_invs
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Similar to master->build_invs used by a per-domain invalidation, add a new
master->ats_invs to be used by arm_smmu_atc_inv_master().
Since arm_smmu_cmdq_batch_init_cmd() now takes an invs pointer, pass it in.
This will be useful by arm_smmu_cmdq_batch_issue() to backtrack the master
pointer from a timed out ATC invalidation command in a subsequent change.
Also replace the streams loop with arm_smmu_invs_for_each_entry() as it is
initialized (except ssid) upon allocation.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 46 ++++++++++++++++++---
2 files changed, 43 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index b5ace01c05a5d..186efcbed1ea9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -1045,6 +1045,8 @@ struct arm_smmu_master {
* iommu_group mutex.
*/
struct arm_smmu_invs *build_invs;
+ /* Scratch memory for arm_smmu_atc_inv_master() to build an ATS array */
+ struct arm_smmu_invs *ats_invs;
struct arm_smmu_vmaster *vmaster; /* use smmu->streams_mutex */
/* Locked by the iommu core using the group mutex */
struct arm_smmu_ctx_desc_cfg cd_table;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index c95297acf2cfe..9591e4ab2b14a 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2407,21 +2407,28 @@ arm_smmu_atc_inv_to_cmd(u32 sid, int ssid, unsigned long iova, size_t size)
static int arm_smmu_atc_inv_master(struct arm_smmu_master *master,
ioasid_t ssid)
{
- int i;
+ struct arm_smmu_invs *invs = master->ats_invs;
struct arm_smmu_cmd cmd;
struct arm_smmu_cmdq_batch cmds;
+ struct arm_smmu_inv *inv;
+ size_t i;
+
+ /* No concurrent user on master->ats_invs */
+ iommu_group_mutex_assert(master->dev);
/* Do not issue ATC_INV that will definitely time out */
if (READ_ONCE(master->ats_broken))
return 0;
cmd = arm_smmu_make_cmd_atc_inv_all(0, IOMMU_NO_PASID);
- arm_smmu_cmdq_batch_init_cmd(master->smmu, &cmds, &cmd, NULL);
- for (i = 0; i < master->num_streams; i++)
+ arm_smmu_cmdq_batch_init_cmd(master->smmu, &cmds, &cmd, invs);
+
+ arm_smmu_invs_for_each_entry(invs, i, inv) {
+ inv->ssid = ssid;
arm_smmu_cmdq_batch_add_cmd(
master->smmu, &cmds,
- arm_smmu_make_cmd_atc_inv_all(master->streams[i].id,
- ssid));
+ arm_smmu_make_cmd_atc_inv_all(inv->id, ssid));
+ }
return arm_smmu_cmdq_batch_submit(master->smmu, &cmds);
}
@@ -4087,6 +4094,18 @@ static int arm_smmu_stream_id_cmp(const void *_l, const void *_r)
return cmp_int(*l, *r);
}
+static void arm_smmu_master_init_ats_inv(struct arm_smmu_master *master,
+ struct arm_smmu_inv *inv, u32 sid)
+{
+ inv->id = sid;
+ inv->users = 1;
+ inv->master = master;
+ inv->smmu = master->smmu;
+ inv->type = INV_TYPE_ATS;
+ inv->size_opcode = CMDQ_OP_ATC_INV;
+ inv->nsize_opcode = CMDQ_OP_ATC_INV;
+}
+
static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
struct arm_smmu_master *master)
{
@@ -4105,11 +4124,19 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
/* Base case has 1 ASID entry or maximum 2 VMID entries */
master->build_invs = arm_smmu_invs_alloc(2);
} else {
+ master->ats_invs = arm_smmu_invs_alloc(fwspec->num_ids);
+ if (!master->ats_invs) {
+ kfree(master->streams);
+ return -ENOMEM;
+ }
+ master->ats_invs->has_ats = true;
+
/* ATS case adds num_ids of entries, on top of the base case */
master->build_invs = arm_smmu_invs_alloc(2 + fwspec->num_ids);
}
if (!master->build_invs) {
kfree(master->streams);
+ kfree(master->ats_invs);
return -ENOMEM;
}
@@ -4125,6 +4152,13 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
sizeof(master->streams[0]), arm_smmu_stream_id_cmp,
NULL);
+ if (master->ats_invs) {
+ for (i = 0; i < fwspec->num_ids; i++)
+ arm_smmu_master_init_ats_inv(master,
+ &master->ats_invs->inv[i],
+ master->streams[i].id);
+ }
+
mutex_lock(&smmu->streams_mutex);
for (i = 0; i < fwspec->num_ids; i++) {
struct arm_smmu_stream *new_stream = &master->streams[i];
@@ -4159,6 +4193,7 @@ static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
for (i--; i >= 0; i--)
rb_erase(&master->streams[i].node, &smmu->streams);
kfree(master->streams);
+ kfree(master->ats_invs);
kfree(master->build_invs);
}
mutex_unlock(&smmu->streams_mutex);
@@ -4261,6 +4296,7 @@ static void arm_smmu_release_device(struct device *dev)
*/
synchronize_rcu();
kfree(master->streams);
+ kfree(master->ats_invs);
kfree(master->build_invs);
kfree(master);
}
--
2.43.0
^ permalink raw reply related
* [PATCH v4 05/24] iommu: Add reset_device_done callback for hardware fault recovery
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
When an IOMMU hardware detects an error due to a faulty device (e.g. an ATS
invalidation timeout), IOMMU drivers may quarantine the device by disabling
specific hardware features or dropping translation capabilities.
To recover from these states, the IOMMU driver needs a reliable signal that
the underlying physical hardware has been cleanly reset (e.g., via PCIe AER
or a sysfs Function Level Reset) so as to lift the quarantine.
Introduce a reset_device_done callback in struct iommu_ops. Trigger it from
the existing pci_dev_reset_iommu_done() path to notify the underlying IOMMU
driver that the device's internal state has been sanitized, when the result
indicates a successful physical reset.
As the initial use case, this will be used by ATS-capable PCI devices.
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
include/linux/iommu.h | 4 ++++
drivers/iommu/iommu.c | 12 ++++++++++++
2 files changed, 16 insertions(+)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index e191d30d228ac..6c124e9e9af8b 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -629,6 +629,9 @@ __iommu_copy_struct_to_user(const struct iommu_user_data *dst_data,
* @release_device: Remove device from iommu driver handling
* @probe_finalize: Do final setup work after the device is added to an IOMMU
* group and attached to the groups domain
+ * @reset_device_done: Notify the driver that a device has reset successfully.
+ * Note that the core invokes the callback function while
+ * holding the group->mutex
* @device_group: find iommu group for a particular device
* @get_resv_regions: Request list of reserved regions for a device
* @of_xlate: add OF master IDs to iommu grouping
@@ -686,6 +689,7 @@ struct iommu_ops {
struct iommu_device *(*probe_device)(struct device *dev);
void (*release_device)(struct device *dev);
void (*probe_finalize)(struct device *dev);
+ void (*reset_device_done)(struct device *dev);
struct iommu_group *(*device_group)(struct device *dev);
/* Request/Free a list of reserved regions for a device */
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 6c92b7a2b14cc..e68c7b142ad5a 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -4182,12 +4182,14 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev, int reset_result)
{
struct iommu_group *group = pdev->dev.iommu_group;
enum gdev_blocked old_gdev_blocked;
+ const struct iommu_ops *ops;
struct group_device *gdev;
unsigned long pasid;
void *entry;
if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev))
return;
+ ops = dev_iommu_ops(&pdev->dev);
guard(mutex)(&group->mutex);
@@ -4249,6 +4251,16 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev, int reset_result)
"DMA-aliased sibling may be prematurely unblocked\n");
}
+ /*
+ * A PCI device might have been in an error state, so the IOMMU driver
+ * had to quarantine the device by disabling specific hardware features
+ * or dropping translation capability. Here notify the IOMMU driver as
+ * a reliable signal that the faulty PCI device has been cleanly reset
+ * so now it can lift its quarantine and restore full functionality.
+ */
+ if (ops->reset_device_done)
+ ops->reset_device_done(&pdev->dev);
+
/*
* Re-attach RID domain back to group->domain
*
--
2.43.0
^ permalink raw reply related
* [PATCH v4 13/24] iommu/arm-smmu-v3: Skip remaining GERROR causes on SFM
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
When the SMMU enters Service Failure Mode (SFM), arm_smmu_device_disable()
clears CR0 and the SMMU stops processing requests entirely. The remaining
GERROR causes (MSI write aborts, PRIQ/EVTQ aborts, CMDQ_ERR) are moot at
that point: the cmdq is dead so arm_smmu_cmdq_skip_err() would just twiddle
bookkeeping for a queue nobody's reading, and the per-cause dev_warn lines
add little diagnostic value beyond the SFM message itself.
After arm_smmu_device_disable(), ack GERRORN and return. This intentionally
duplicates the last two lines of the function because this SFM path will be
slightly different.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 1065301a54eeb..d9fe48989fcd7 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2268,8 +2268,11 @@ static irqreturn_t arm_smmu_gerror_handler(int irq, void *dev)
active);
if (active & GERROR_SFM_ERR) {
+ /* SMMU is being disabled, so other errors don't matter */
+ writel(gerror, smmu->base + ARM_SMMU_GERRORN);
dev_err(smmu->dev, "device has entered Service Failure Mode!\n");
arm_smmu_device_disable(smmu);
+ return IRQ_HANDLED;
}
if (active & GERROR_MSI_GERROR_ABT_ERR)
--
2.43.0
^ permalink raw reply related
* [PATCH v4 10/24] iommu: Add __iommu_group_block_device helper
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Move the RID/PASID blocking routine into a separate helper, which will be
reused by a new function to quarantine the device but does not bother the
gdev->reset_depth counter.
Also, document the severity ordering at enum gdev_blocked.
No functional changes.
Suggested-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/iommu.c | 106 ++++++++++++++++++++++++------------------
1 file changed, 60 insertions(+), 46 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index f745083c032d6..b150d22d8015f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -77,6 +77,7 @@ struct iommu_group {
#define dev_iommu_group_rcu(dev) \
(*((struct iommu_group __rcu __force **)&(dev)->iommu_group))
+/* A bigger number indicates a higher severity */
enum gdev_blocked {
BLOCKED_NO = 0, /* Not blocked */
BLOCKED_RESETTING, /* PCI reset in flight */
@@ -4053,6 +4054,62 @@ int iommu_replace_group_handle(struct iommu_group *group,
}
EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IOMMUFD_INTERNAL");
+/* Caller can use this function on a blocked @gdev just to update the @reason */
+static int __iommu_group_block_device(struct group_device *gdev,
+ enum gdev_blocked reason)
+{
+ struct iommu_group *group = gdev->group;
+ unsigned long pasid;
+ void *entry;
+ int ret;
+
+ lockdep_assert_held(&group->mutex);
+
+ /* Device might be already blocked for a quarantine */
+ if (gdev->blocked) {
+ /* Escalate the severity */
+ gdev->blocked = max(gdev->blocked, reason);
+ return 0;
+ }
+
+ ret = __iommu_group_alloc_blocking_domain(group);
+ if (ret)
+ return ret;
+
+ /* Stage RID domain at blocking_domain while retaining group->domain */
+ if (group->domain != group->blocking_domain) {
+ ret = __iommu_attach_device(group->blocking_domain, gdev->dev,
+ group->domain);
+ if (ret)
+ return ret;
+ }
+
+ /*
+ * Update gdev->blocked upon the domain change, as it is used to return
+ * the correct domain in iommu_driver_get_domain_for_dev() that might be
+ * called in a set_dev_pasid callback function.
+ */
+ gdev->blocked = reason;
+
+ /*
+ * Stage PASID domains at blocking_domain while retaining pasid_array.
+ *
+ * The pasid_array is mostly fenced by group->mutex, except one reader
+ * in iommu_attach_handle_get(), so it's safe to read without xa_lock.
+ */
+ if (gdev->dev->iommu->max_pasids > 0) {
+ xa_for_each_start(&group->pasid_array, pasid, entry, 1) {
+ struct iommu_domain *pasid_dom =
+ pasid_array_entry_to_domain(entry);
+
+ iommu_remove_dev_pasid(gdev->dev, pasid, pasid_dom);
+ }
+ }
+
+ group->recovery_cnt++;
+ return 0;
+}
+
/**
* pci_dev_reset_iommu_prepare() - Block IOMMU to prepare for a PCI device reset
* @pdev: PCI device that is going to enter a reset routine
@@ -4086,8 +4143,6 @@ int pci_dev_reset_iommu_prepare(struct pci_dev *pdev)
{
struct iommu_group *group = pdev->dev.iommu_group;
struct group_device *gdev;
- unsigned long pasid;
- void *entry;
int ret;
if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev))
@@ -4102,49 +4157,9 @@ int pci_dev_reset_iommu_prepare(struct pci_dev *pdev)
if (gdev->reset_depth++)
return 0;
- /* Device might be already blocked for a quarantine */
- if (gdev->blocked)
- return 0;
-
- ret = __iommu_group_alloc_blocking_domain(group);
- if (ret) {
+ ret = __iommu_group_block_device(gdev, BLOCKED_RESETTING);
+ if (ret)
gdev->reset_depth--;
- return ret;
- }
-
- /* Stage RID domain at blocking_domain while retaining group->domain */
- if (group->domain != group->blocking_domain) {
- ret = __iommu_attach_device(group->blocking_domain, &pdev->dev,
- group->domain);
- if (ret) {
- gdev->reset_depth--;
- return ret;
- }
- }
-
- /*
- * Update gdev->blocked upon the domain change, as it is used to return
- * the correct domain in iommu_driver_get_domain_for_dev() that might be
- * called in a set_dev_pasid callback function.
- */
- gdev->blocked = BLOCKED_RESETTING;
-
- /*
- * Stage PASID domains at blocking_domain while retaining pasid_array.
- *
- * The pasid_array is mostly fenced by group->mutex, except one reader
- * in iommu_attach_handle_get(), so it's safe to read without xa_lock.
- */
- if (pdev->dev.iommu->max_pasids > 0) {
- xa_for_each_start(&group->pasid_array, pasid, entry, 1) {
- struct iommu_domain *pasid_dom =
- pasid_array_entry_to_domain(entry);
-
- iommu_remove_dev_pasid(&pdev->dev, pasid, pasid_dom);
- }
- }
-
- group->recovery_cnt++;
return ret;
}
EXPORT_SYMBOL_GPL(pci_dev_reset_iommu_prepare);
@@ -4256,8 +4271,7 @@ void pci_dev_reset_iommu_done(struct pci_dev *pdev, int reset_result)
dev_err_ratelimited(
&pdev->dev,
"Reset failed. Keep it blocked to protect memory\n");
- if (gdev->blocked == BLOCKED_RESETTING)
- gdev->blocked = BLOCKED_RESET_FAILED;
+ WARN_ON(__iommu_group_block_device(gdev, BLOCKED_RESET_FAILED));
return;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v4 21/24] iommu/arm-smmu-v3: Move arm_smmu_invs_for_each_entry to header
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
A subsequent change will use this helper in a lockless context. Since this
this is a macro, move it to the header file so other functions can use it.
Also, add READ_ONCE to invs->num_invs for lockless use.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 +++++++++++++++++++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 ---------------------
2 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 2074814534fef..b5ace01c05a5d 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -803,6 +803,34 @@ struct arm_smmu_invs {
struct arm_smmu_inv inv[] __counted_by(max_invs);
};
+/* Invalidation array manipulation functions */
+static inline struct arm_smmu_inv *
+arm_smmu_invs_iter_next(struct arm_smmu_invs *invs, size_t next, size_t *idx)
+{
+ while (true) {
+ if (next >= READ_ONCE(invs->num_invs)) {
+ *idx = next;
+ return NULL;
+ }
+ if (!READ_ONCE(invs->inv[next].users)) {
+ next++;
+ continue;
+ }
+ *idx = next;
+ return &invs->inv[next];
+ }
+}
+
+/**
+ * arm_smmu_invs_for_each_entry - Iterate over all non-trash entries in invs
+ * @invs: the base invalidation array
+ * @idx: a stack variable of 'size_t', to store the array index
+ * @cur: a stack variable of 'struct arm_smmu_inv *'
+ */
+#define arm_smmu_invs_for_each_entry(invs, idx, cur) \
+ for (cur = arm_smmu_invs_iter_next(invs, 0, &(idx)); cur; \
+ cur = arm_smmu_invs_iter_next(invs, idx + 1, &(idx)))
+
static inline struct arm_smmu_invs *arm_smmu_invs_alloc(size_t num_invs)
{
struct arm_smmu_invs *new_invs;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 4f2b23b1e8163..c95297acf2cfe 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -979,34 +979,6 @@ static void arm_smmu_page_response(struct device *dev, struct iopf_fault *unused
*/
}
-/* Invalidation array manipulation functions */
-static inline struct arm_smmu_inv *
-arm_smmu_invs_iter_next(struct arm_smmu_invs *invs, size_t next, size_t *idx)
-{
- while (true) {
- if (next >= invs->num_invs) {
- *idx = next;
- return NULL;
- }
- if (!READ_ONCE(invs->inv[next].users)) {
- next++;
- continue;
- }
- *idx = next;
- return &invs->inv[next];
- }
-}
-
-/**
- * arm_smmu_invs_for_each_entry - Iterate over all non-trash entries in invs
- * @invs: the base invalidation array
- * @idx: a stack variable of 'size_t', to store the array index
- * @cur: a stack variable of 'struct arm_smmu_inv *'
- */
-#define arm_smmu_invs_for_each_entry(invs, idx, cur) \
- for (cur = arm_smmu_invs_iter_next(invs, 0, &(idx)); cur; \
- cur = arm_smmu_invs_iter_next(invs, idx + 1, &(idx)))
-
static int arm_smmu_inv_cmp(const struct arm_smmu_inv *inv_l,
const struct arm_smmu_inv *inv_r)
{
--
2.43.0
^ permalink raw reply related
* [PATCH v4 12/24] iommu/arm-smmu-v3: Mark ATC invalidate timeouts via lockless bitmap
From: Nicolin Chen @ 2026-05-19 3:38 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
An ATC invalidation timeout is a fatal error. While the SMMUv3 hardware is
aware of the timeout via a GERROR interrupt, the driver thread issuing the
commands lacks a direct mechanism to verify whether its specific batch was
the cause or not, as polling the CMD_SYNC status doesn't natively return a
failure code, making it very difficult to coordinate per-device recovery.
Introduce an atc_sync_timeouts bitmap in the cmdq structure to bridge this
gap. When the ISR detects an ATC timeout, set the bit corresponding to the
physical CMDQ index of the faulting CMD_SYNC command.
On the issuer side, after polling completes (or times out), test and clear
its dedicated bit. If set, return -EIO to trigger device quarantine.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 ++++++++++++++++++++-
2 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 16353596e08ad..46f9e292a1cc8 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -700,6 +700,7 @@ struct arm_smmu_cmdq {
atomic_long_t *valid_map;
atomic_t owner_prod;
atomic_t lock;
+ unsigned long *atc_sync_timeouts;
bool (*supports_cmd)(struct arm_smmu_cmd *cmd);
};
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9be589d14a3bd..1065301a54eeb 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -343,7 +343,10 @@ void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu,
* at the CMD_SYNC. Attempt to complete other pending commands
* by repeating the CMD_SYNC, though we might well end up back
* here since the ATC invalidation may still be pending.
+ *
+ * Mark the faulty batch in the bitmap for the issuer to match.
*/
+ set_bit(Q_IDX(&q->llq, cons), cmdq->atc_sync_timeouts);
return;
case CMDQ_ERR_CERROR_ILL_IDX:
default:
@@ -750,6 +753,14 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu,
queue_write(Q_ENT(&cmdq->q, prod), cmd_sync.data,
ARRAY_SIZE(cmd_sync.data));
+ /*
+ * Clear any stale ATC-timeout bit left in the slot from a prior
+ * wraparound, before the slot becomes visible to the SMMU. Must
+ * do this prior to step 3 to prevent potentially races with the
+ * GERROR ISR calling set_bit() for our own CMD_SYNC.
+ */
+ clear_bit(Q_IDX(&llq, prod), cmdq->atc_sync_timeouts);
+
/*
* In order to determine completion of our CMD_SYNC, we must
* ensure that the queue can't wrap twice without us noticing.
@@ -796,9 +807,33 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu,
/* 5. If we are inserting a CMD_SYNC, we must wait for it to complete */
if (sync) {
+ u32 sync_prod;
+
llq.prod = queue_inc_prod_n(&llq, n);
+ sync_prod = llq.prod;
ret = arm_smmu_cmdq_poll_until_sync(smmu, cmdq, &llq);
- if (ret) {
+
+ /*
+ * Test atc_sync_timeouts first and see if there is ATC timeout
+ * resulted from this cmdlist. Return -EIO to separate from the
+ * ARM_SMMU_POLL_TIMEOUT_US software timeout.
+ *
+ * FIXME possible unhandled ATC invalidation timeout scenario:
+ * PCI Completion Timeout can be set to a range longer than the
+ * ARM_SMMU_POLL_TIMEOUT_US software timeout. -ETIMEDOUT can be
+ * returned by arm_smmu_cmdq_poll_until_sync() while the ATC_INV
+ * is still pending and not yet reflected in GERROR, so the bit
+ * on atc_sync_timeouts is not set. In this case, we can hardly
+ * do anything here, since the command queue HW is still pending
+ * on the ATC command.
+ */
+ if (test_and_clear_bit(Q_IDX(&llq, sync_prod),
+ cmdq->atc_sync_timeouts)) {
+ dev_err_ratelimited(smmu->dev,
+ "CMD_SYNC for ATC_INV timeout at prod=0x%08x\n",
+ sync_prod);
+ ret = -EIO;
+ } else if (ret) {
dev_err_ratelimited(smmu->dev,
"CMD_SYNC timeout at 0x%08x [hwprod 0x%08x, hwcons 0x%08x]\n",
llq.prod,
@@ -4332,6 +4367,11 @@ int arm_smmu_cmdq_init(struct arm_smmu_device *smmu,
if (!cmdq->valid_map)
return -ENOMEM;
+ cmdq->atc_sync_timeouts =
+ devm_bitmap_zalloc(smmu->dev, nents, GFP_KERNEL);
+ if (!cmdq->atc_sync_timeouts)
+ return -ENOMEM;
+
return 0;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v4 17/24] iommu/arm-smmu-v3: Add master in arm_smmu_inv for ATS entries
From: Nicolin Chen @ 2026-05-19 3:39 UTC (permalink / raw)
To: Will Deacon, Robin Murphy, Joerg Roedel, Bjorn Helgaas,
Jason Gunthorpe
Cc: Rafael J . Wysocki, Len Brown, Pranjal Shrivastava, Mostafa Saleh,
Lu Baolu, Kevin Tian, linux-arm-kernel, iommu, linux-kernel,
linux-acpi, linux-pci, vsethi, Shuai Xue
In-Reply-To: <cover.1779161849.git.nicolinc@nvidia.com>
Storing the master pointer allows backtracking it from an ATS invalidation
entry, which will be useful when handling ATC invalidation timeouts.
Don't simply swap the "smmu" pointer for the "master": a non-ATS entry may
be shared across multiple devices (masters). An ATS entry is okay since it
is tied to a unique SID.
Master must outlive any concurrent RCU reader iterating the domain->invs,
because inv->master is dereferenced inside the read-side critical section.
Add a synchronize_rcu() in arm_smmu_release_device() before freeing master.
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 14 +++++++++++---
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 604f7edf54158..df6e539f75274 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -738,6 +738,7 @@ enum arm_smmu_inv_type {
struct arm_smmu_inv {
struct arm_smmu_device *smmu;
+ struct arm_smmu_master *master; /* INV_TYPE_ATS* */
u8 type;
u8 size_opcode;
u8 nsize_opcode;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0e4f34ed036c6..cde2ff2dcc49b 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3211,6 +3211,7 @@ arm_smmu_master_build_inv(struct arm_smmu_master *master,
case INV_TYPE_ATS_FULL:
cur->size_opcode = cur->nsize_opcode = CMDQ_OP_ATC_INV;
cur->ssid = ssid;
+ cur->master = master;
break;
}
@@ -4168,9 +4169,6 @@ static void arm_smmu_remove_master(struct arm_smmu_master *master)
for (i = 0; i < fwspec->num_ids; i++)
rb_erase(&master->streams[i].node, &smmu->streams);
mutex_unlock(&smmu->streams_mutex);
-
- kfree(master->streams);
- kfree(master->build_invs);
}
static struct iommu_device *arm_smmu_probe_device(struct device *dev)
@@ -4244,6 +4242,16 @@ static void arm_smmu_release_device(struct device *dev)
arm_smmu_remove_master(master);
if (arm_smmu_cdtab_allocated(&master->cd_table))
arm_smmu_free_cd_tables(master);
+
+ /*
+ * The iommu core detaches @dev from every iommu domain before invoking
+ * release_device. So the updated domain->invs no longer references the
+ * @master; IOW, new RCU readers cannot reach it. Wait one grace period
+ * for in-flight readers to drop their references.
+ */
+ synchronize_rcu();
+ kfree(master->streams);
+ kfree(master->build_invs);
kfree(master);
}
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox