From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8E7223DC9 for ; Thu, 26 Feb 2026 13:39:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772113178; cv=none; b=DeJ4XvNkHL67NBBL3LtRlR/cBmeYZQ9qIL+BwbGjKmiSF8Lq7MPEwhlIOjlmD52geeZtECmW7SGj0SlKqWpGpqXep16ph2fCyhLk+kSeufvMd9mm8Xhm6N9rkoTYvBxspSS/ANtvURThGZwSavPiZIp6DybBrVUZZX79AzOAJGE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772113178; c=relaxed/simple; bh=Aeva4/z21UF2gn+bR0z9srxN2YezkuTE8ZtlSFpOGrg=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=mQ6e1gBQNqg+y3abIvgkGscli2v10CjooYVE/vyPNxEsOczUtQXG2XURtk84IZgZThcOY7Etdwi0RLzd7VDjjtDH34jPp4q9S3N3EgU7XUOXfdCGjkj5x3HEBE2EH/4jr6faS5bBzQ1dDN6WIkeO3L7NHE5raNMune1aLs+aEVY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FsFrcXte; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FsFrcXte" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3C01C116C6; Thu, 26 Feb 2026 13:39:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772113177; bh=Aeva4/z21UF2gn+bR0z9srxN2YezkuTE8ZtlSFpOGrg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=FsFrcXte6q41Ri/0hu/T3ZFOqd2FACAnbQmow2dQE+7QZRJUVGzEBGEkHBJ4yXht4 ctRz7W5YogxcZypuZoNIjEkGeuvk4Wrbviz6PC99K3nDRGHslQoAVj6byYmuEmTsGQ rIi6KpJcPjcQ0H5D6j8yT38q0+RWw1ax74zTD2hAiuZDJwz+KM236zKutNzQw7do3I jCv55jbEICJdz4B2aygWzSt5m3du1IWpCd3m4LI8THB+fnUH1l2QMS140k4D49JcAh hMlKzVJwLi3yf36YUSNJz+7BAYILa7DNwfaxp4IsqAQifClotGe9L/vlWTUaKvAhbt eMOBKw+lM9D+g== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vvbaZ-0000000E4Au-3EXq; Thu, 26 Feb 2026 13:39:35 +0000 Date: Thu, 26 Feb 2026 13:39:35 +0000 Message-ID: <865x7jaajs.wl-maz@kernel.org> From: Marc Zyngier To: Qiang Yu Cc: Manivannan Sadhasivam , tglx@linutronix.de, linux-kernel@vger.kernel.org Subject: Re: MSIs not freed in GICv3 ITS driver In-Reply-To: References: <20240708153933.GC5745@thinkpad> <865xtf4woi.wl-maz@kernel.org> <20240709173708.GA44420@thinkpad> <877cdupdvu.wl-maz@kernel.org> <20240721085032.GL1908@thinkpad> <4pdu25dnnqegnd67zf4ftfvwc57bn67kp7mj2gk2cywc3hdcvr@eydar5gvuwtu> <86wm08ad2y.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: qiang.yu@oss.qualcomm.com, mani@kernel.org, tglx@linutronix.de, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Wed, 25 Feb 2026 09:34:41 +0000, Qiang Yu wrote: > > On Thu, Feb 19, 2026 at 04:54:29PM +0000, Marc Zyngier wrote: > > On Fri, 16 Jan 2026 15:03:33 +0000, > > Manivannan Sadhasivam wrote: > > > > > > Hi Marc, > > > > > > Looks like this has fallen through the cracks and my colleage internally > > > reported a warning during the removal of a PCI driver and it seems to be related > > > to the issue we were discussing in this thread: > > > > > > [ 54.727284] WARNING: drivers/irqchip/irq-gic-v3-its.c:3639 at its_msi_teardown+0x11c/0x13c, CPU#4: kworker/u73:1/115 > > > [ 54.738366] Modules linked in: mhi_pci_generic mhi nvme_core usb_f_fs libcomposite sm3_ce nvmem_qcom_spmi_sdam qcom_pon rtc_pm8xxx qcom_spmi_temp_alarm qcom_stats dispcc_glymur gpi llcc_qcom phy_qcom_qmp_pcie qcom_cpucp_mbox qcom_wdt socinfo > > > [ 54.760588] CPU: 4 UID: 0 PID: 115 Comm: kworker/u73:1 Tainted: G W 6.18.0-next-20251210-14099-gc20082c23661-dirty #2 PREEMPT > > > [ 54.774067] Tainted: [W]=WARN > > > [ 54.777412] Hardware name: Qualcomm MTP/Qualcomm Test Device, BIOS 7.0.251121.BOOT.OSSUEFI.3.1-00008-GLYMUR-1 11/21/2025 > > > [ 54.788849] Workqueue: async async_run_entry_fn > > > [ 54.793791] pstate: 21400009 (nzCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > > > [ 54.801230] pc : its_msi_teardown+0x11c/0x13c > > > [ 54.805997] lr : its_msi_teardown+0x54/0x13c > > > [ 54.810675] sp : ffff8000837cb710 > > > [ 54.814373] x29: ffff8000837cb710 x28: ffff00080190e410 x27: ffff0008085ba390 > > > [ 54.821985] x26: ffff000808629bf0 x25: 0000000000000000 x24: 0000000000000066 > > > [ 54.829602] x23: 0000000000000007 x22: 0000000000000020 x21: ffff000800059608 > > > [ 54.837209] x20: ffff000800059607 x19: ffff000800a4a300 x18: 00000000ffffffff > > > [ 54.844819] x17: ffff00080ec65400 x16: ffff00080ec65200 x15: ffff00080ec65000 > > > [ 54.852429] x14: 0000000000000004 x13: ffff0008000b8810 x12: 0000000000000000 > > > [ 54.860046] x11: ffff0008007798e8 x10: 0000000000000002 x9 : 0000000000000001 > > > [ 54.867661] x8 : ffff0008007796f8 x7 : 000000000000001f x6 : ffff8000837cb640 > > > [ 54.875277] x5 : ffff000801918f40 x4 : 0000000000000007 x3 : 0000000000000000 > > > [ 54.882891] x2 : ffff000800a037c0 x1 : 0000000000000020 x0 : 0000000000000007 > > > [ 54.890509] Call trace: > > > [ 54.893320] its_msi_teardown+0x11c/0x13c (P) > > > [ 54.898082] its_msi_teardown+0x34/0x44 > > > [ 54.902316] msi_remove_device_irq_domain+0x70/0x114 > > > [ 54.907701] msi_device_data_release+0x20/0x64 > > > [ 54.912551] devres_release_all+0xa4/0x104 > > > > That's nowhere near enough information for me to do anything about it. > > > > Unless you describe exactly what device this is, its allocation > > requirements, the topology of the system and finally reproduce it on a > > vanilla kernel and not something that I have no access to, I can't do > > much for you. > > Hi Marc, > > Thanks for the feedback. I can reproduce this issue with latest linux-next > tag next-20260224. Please don't test on -next. Pick the latest tag from Linus. As far as I am concerned, -next bears no relevance whatsoever. > > The host is Glymur (Qualcomm compute platform) with an SDX75 modem > connected via PCIe. The SDX75 driver requests 7 MSI IRQs, and the warning > triggers during driver removal. > > I think this is actually a common problem with how we handle > MSI allocation vs freeing. Here's what I'm seeing: > > When allocating, irq_domain_alloc_irqs_hierarchy() makes one call to > domain->ops->alloc() with nr_irqs=7. The MSI controller (ITS in this case > but DWC-MSI has similar behavior) finds a power-of-2 bits in its bitmap > region, so it allocates 8 contiguous bits to satisfy the 7 IRQ request. Well, it's not like the ITS has a choice. Given that the ITT size is expressed in a number of bits, you get the choice between a power of two or absolutely nothing. I'm not going to comment on the DWC stuff, as it has been bitrotting for the best part of two decades. > > But when freeing, irq_domain_free_irqs_hierarchy() loops and calls > domain->ops->free() seven times, each with nr_irqs=1. So we end up freeing > 7 individual bits instead of the original 8 bits that was allocated. > > This allocation/free mismatch seems to corrupt the bitmap tracking, which > is what triggers the warning in its_msi_teardown(). > > I suspect this would happen with any PCIe device that requests a > non-power-of-2 number of MSI IRQs on systems using ITS or DWC-MSI. Is this device doing Multi-MSI or MSI-X? Please post an 'lspci -vv' so that we know what we are up against. Thanks, M. -- Without deviation from the norm, progress is not possible.