* [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM
@ 2024-04-03 5:37 Aleksandr Aprelkov
2024-04-09 10:43 ` Mostafa Saleh
2024-04-19 15:32 ` Will Deacon
0 siblings, 2 replies; 6+ messages in thread
From: Aleksandr Aprelkov @ 2024-04-03 5:37 UTC (permalink / raw)
To: Will Deacon
Cc: Aleksandr Aprelkov, Robin Murphy, Joerg Roedel, Jason Gunthorpe,
Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier,
linux-arm-kernel, iommu, linux-kernel
If devm_add_action() returns ENOMEM, then MSIs allocated but
not freed on teardown.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Fixes: 166bdbd23161 ("iommu/arm-smmu: Add support for MSI on SMMUv3")
Signed-off-by: Aleksandr Aprelkov <aaprelkov@usergate.com>
---
v2: Use appropriate function for registration failure as
Jonathan Cameron <Jonathan.Cameron@Huawei.com> suggested.
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 41f93c3ab160..8800af041e5f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3402,7 +3402,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu)
smmu->priq.q.irq = msi_get_virq(dev, PRIQ_MSI_INDEX);
/* Add callback to free MSIs on teardown */
- devm_add_action(dev, arm_smmu_free_msis, dev);
+ ret = devm_add_action_or_reset(dev, arm_smmu_free_msis, dev);
+ if (ret)
+ dev_warn(dev, "failed to add free MSIs callback - falling back to wired irqs\n");
}
static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu)
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM 2024-04-03 5:37 [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM Aleksandr Aprelkov @ 2024-04-09 10:43 ` Mostafa Saleh 2024-04-09 11:17 ` Robin Murphy 2024-04-19 15:32 ` Will Deacon 1 sibling, 1 reply; 6+ messages in thread From: Mostafa Saleh @ 2024-04-09 10:43 UTC (permalink / raw) To: Aleksandr Aprelkov Cc: Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier, linux-arm-kernel, iommu, linux-kernel Hi Aleksandr, On Wed, Apr 03, 2024 at 12:37:59PM +0700, Aleksandr Aprelkov wrote: > If devm_add_action() returns ENOMEM, then MSIs allocated but > not freed on teardown. > > Found by Linux Verification Center (linuxtesting.org) with SVACE. > > Fixes: 166bdbd23161 ("iommu/arm-smmu: Add support for MSI on SMMUv3") > Signed-off-by: Aleksandr Aprelkov <aaprelkov@usergate.com> > --- > v2: Use appropriate function for registration failure as > Jonathan Cameron <Jonathan.Cameron@Huawei.com> suggested. > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > index 41f93c3ab160..8800af041e5f 100644 > --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > @@ -3402,7 +3402,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) > smmu->priq.q.irq = msi_get_virq(dev, PRIQ_MSI_INDEX); > > /* Add callback to free MSIs on teardown */ > - devm_add_action(dev, arm_smmu_free_msis, dev); > + ret = devm_add_action_or_reset(dev, arm_smmu_free_msis, dev); > + if (ret) > + dev_warn(dev, "failed to add free MSIs callback - falling back to wired irqs\n"); I am not sure that is the right fix, as allowing the driver to probe without MSIs, seems worse than leaking MSI memory. IMHO, we can just add something like: dev_err(smmu->dev, “Can’t allocate devm action, MSIs are never freed! !\n”) ; Also, we can’t unconditionally fallback to wired irqs if MSI exists, according to the user manual: An implementation must support one of, or optionally both of, wired interrupts and MSIs ... The discovery of support for wired interrupts is IMPLEMENTATION DEFINED. We can add some logic, to check dt/acpi irqs and to choose to fallback or not based on that, but, if we get -ENOMEM, (especially early at probe) something really went wrong, so I am not sure it’s worth the complexity. > } > > static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu) > -- > 2.34.1 > Thanks, Mostafa _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM 2024-04-09 10:43 ` Mostafa Saleh @ 2024-04-09 11:17 ` Robin Murphy 2024-04-09 11:31 ` Mostafa Saleh 0 siblings, 1 reply; 6+ messages in thread From: Robin Murphy @ 2024-04-09 11:17 UTC (permalink / raw) To: Mostafa Saleh, Aleksandr Aprelkov Cc: Will Deacon, Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier, linux-arm-kernel, iommu, linux-kernel On 09/04/2024 11:43 am, Mostafa Saleh wrote: > Hi Aleksandr, > > On Wed, Apr 03, 2024 at 12:37:59PM +0700, Aleksandr Aprelkov wrote: >> If devm_add_action() returns ENOMEM, then MSIs allocated but >> not freed on teardown. >> >> Found by Linux Verification Center (linuxtesting.org) with SVACE. >> >> Fixes: 166bdbd23161 ("iommu/arm-smmu: Add support for MSI on SMMUv3") >> Signed-off-by: Aleksandr Aprelkov <aaprelkov@usergate.com> >> --- >> v2: Use appropriate function for registration failure as >> Jonathan Cameron <Jonathan.Cameron@Huawei.com> suggested. >> >> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >> index 41f93c3ab160..8800af041e5f 100644 >> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >> @@ -3402,7 +3402,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) >> smmu->priq.q.irq = msi_get_virq(dev, PRIQ_MSI_INDEX); >> >> /* Add callback to free MSIs on teardown */ >> - devm_add_action(dev, arm_smmu_free_msis, dev); >> + ret = devm_add_action_or_reset(dev, arm_smmu_free_msis, dev); >> + if (ret) >> + dev_warn(dev, "failed to add free MSIs callback - falling back to wired irqs\n"); > > I am not sure that is the right fix, as allowing the driver to probe > without MSIs, seems worse than leaking MSI memory. > > IMHO, we can just add something like: > dev_err(smmu->dev, “Can’t allocate devm action, MSIs are never freed! !\n”) ; Honestly I don't think this matters. If we ever really did fail to allocate 16 bytes, SLUB would already be screaming and spewing stacktraces, and the system is dead already. > Also, we can’t unconditionally fallback to wired irqs if MSI exists, > according to the user manual: > An implementation must support one of, or optionally both of, > wired interrupts and MSIs > ... > The discovery of support for wired interrupts is IMPLEMENTATION DEFINED. > > We can add some logic, to check dt/acpi irqs and to choose to fallback > or not based on that, but, if we get -ENOMEM, (especially early at > probe) something really went wrong, so I am not sure it’s worth > the complexity. That logic already exists in arm_smmu_setup_unique_irqs() - the messages here are in the sense of "we're giving up on MSIs and falling back to trying whatever wired IRQs we may or may not have." The critical point is that we're not using MSIs for some potentially actionable reason, i.e. if the user does expect the system to be MSI-capable, then it could be an indication of perhaps a wrong or missing msi-parent, for which they may pursue a firmware fix. In other cases it's normal and expected not to use MSIs though (e.g. the system just doesn't have an ITS), so we don't want to be *too* noisy about it. Thanks, Robin. > >> } >> >> static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu) >> -- >> 2.34.1 >> > Thanks, > Mostafa _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM 2024-04-09 11:17 ` Robin Murphy @ 2024-04-09 11:31 ` Mostafa Saleh 2024-04-09 11:46 ` Robin Murphy 0 siblings, 1 reply; 6+ messages in thread From: Mostafa Saleh @ 2024-04-09 11:31 UTC (permalink / raw) To: Robin Murphy Cc: Aleksandr Aprelkov, Will Deacon, Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier, linux-arm-kernel, iommu, linux-kernel Hi Robin, On Tue, Apr 09, 2024 at 12:17:54PM +0100, Robin Murphy wrote: > On 09/04/2024 11:43 am, Mostafa Saleh wrote: > > Hi Aleksandr, > > > > On Wed, Apr 03, 2024 at 12:37:59PM +0700, Aleksandr Aprelkov wrote: > > > If devm_add_action() returns ENOMEM, then MSIs allocated but > > > not freed on teardown. > > > > > > Found by Linux Verification Center (linuxtesting.org) with SVACE. > > > > > > Fixes: 166bdbd23161 ("iommu/arm-smmu: Add support for MSI on SMMUv3") > > > Signed-off-by: Aleksandr Aprelkov <aaprelkov@usergate.com> > > > --- > > > v2: Use appropriate function for registration failure as > > > Jonathan Cameron <Jonathan.Cameron@Huawei.com> suggested. > > > > > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++- > > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > > index 41f93c3ab160..8800af041e5f 100644 > > > --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > > @@ -3402,7 +3402,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) > > > smmu->priq.q.irq = msi_get_virq(dev, PRIQ_MSI_INDEX); > > > /* Add callback to free MSIs on teardown */ > > > - devm_add_action(dev, arm_smmu_free_msis, dev); > > > + ret = devm_add_action_or_reset(dev, arm_smmu_free_msis, dev); > > > + if (ret) > > > + dev_warn(dev, "failed to add free MSIs callback - falling back to wired irqs\n"); > > > > I am not sure that is the right fix, as allowing the driver to probe > > without MSIs, seems worse than leaking MSI memory. > > > > IMHO, we can just add something like: > > dev_err(smmu->dev, “Can’t allocate devm action, MSIs are never freed! !\n”) ; > > Honestly I don't think this matters. If we ever really did fail to allocate > 16 bytes, SLUB would already be screaming and spewing stacktraces, and the > system is dead already. > > > Also, we can’t unconditionally fallback to wired irqs if MSI exists, > > according to the user manual: > > An implementation must support one of, or optionally both of, > > wired interrupts and MSIs > > ... > > The discovery of support for wired interrupts is IMPLEMENTATION DEFINED. > > > > We can add some logic, to check dt/acpi irqs and to choose to fallback > > or not based on that, but, if we get -ENOMEM, (especially early at > > probe) something really went wrong, so I am not sure it’s worth > > the complexity. > > That logic already exists in arm_smmu_setup_unique_irqs() - the messages > here are in the sense of "we're giving up on MSIs and falling back to trying > whatever wired IRQs we may or may not have." The critical point is that > we're not using MSIs for some potentially actionable reason, i.e. if the > user does expect the system to be MSI-capable, then it could be an > indication of perhaps a wrong or missing msi-parent, for which they may > pursue a firmware fix. In other cases it's normal and expected not to use > MSIs though (e.g. the system just doesn't have an ITS), so we don't want to > be *too* noisy about it. The case I am worried about in this patch, is for systems with MSIs only. With this patch, that means, we fallback to wired irqs which don't exist, so the driver will probe with no interrupts at all, which in my opinion worse than leaking the memory. Thanks, Mostafa > > Thanks, > Robin. > > > > > > } > > > static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu) > > > -- > > > 2.34.1 > > > > > Thanks, > > Mostafa _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM 2024-04-09 11:31 ` Mostafa Saleh @ 2024-04-09 11:46 ` Robin Murphy 0 siblings, 0 replies; 6+ messages in thread From: Robin Murphy @ 2024-04-09 11:46 UTC (permalink / raw) To: Mostafa Saleh Cc: Aleksandr Aprelkov, Will Deacon, Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier, linux-arm-kernel, iommu, linux-kernel On 09/04/2024 12:31 pm, Mostafa Saleh wrote: > Hi Robin, > > On Tue, Apr 09, 2024 at 12:17:54PM +0100, Robin Murphy wrote: >> On 09/04/2024 11:43 am, Mostafa Saleh wrote: >>> Hi Aleksandr, >>> >>> On Wed, Apr 03, 2024 at 12:37:59PM +0700, Aleksandr Aprelkov wrote: >>>> If devm_add_action() returns ENOMEM, then MSIs allocated but >>>> not freed on teardown. >>>> >>>> Found by Linux Verification Center (linuxtesting.org) with SVACE. >>>> >>>> Fixes: 166bdbd23161 ("iommu/arm-smmu: Add support for MSI on SMMUv3") >>>> Signed-off-by: Aleksandr Aprelkov <aaprelkov@usergate.com> >>>> --- >>>> v2: Use appropriate function for registration failure as >>>> Jonathan Cameron <Jonathan.Cameron@Huawei.com> suggested. >>>> >>>> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++- >>>> 1 file changed, 3 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >>>> index 41f93c3ab160..8800af041e5f 100644 >>>> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >>>> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >>>> @@ -3402,7 +3402,9 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) >>>> smmu->priq.q.irq = msi_get_virq(dev, PRIQ_MSI_INDEX); >>>> /* Add callback to free MSIs on teardown */ >>>> - devm_add_action(dev, arm_smmu_free_msis, dev); >>>> + ret = devm_add_action_or_reset(dev, arm_smmu_free_msis, dev); >>>> + if (ret) >>>> + dev_warn(dev, "failed to add free MSIs callback - falling back to wired irqs\n"); >>> >>> I am not sure that is the right fix, as allowing the driver to probe >>> without MSIs, seems worse than leaking MSI memory. >>> >>> IMHO, we can just add something like: >>> dev_err(smmu->dev, “Can’t allocate devm action, MSIs are never freed! !\n”) ; >> >> Honestly I don't think this matters. If we ever really did fail to allocate >> 16 bytes, SLUB would already be screaming and spewing stacktraces, and the >> system is dead already. >> >>> Also, we can’t unconditionally fallback to wired irqs if MSI exists, >>> according to the user manual: >>> An implementation must support one of, or optionally both of, >>> wired interrupts and MSIs >>> ... >>> The discovery of support for wired interrupts is IMPLEMENTATION DEFINED. >>> >>> We can add some logic, to check dt/acpi irqs and to choose to fallback >>> or not based on that, but, if we get -ENOMEM, (especially early at >>> probe) something really went wrong, so I am not sure it’s worth >>> the complexity. >> >> That logic already exists in arm_smmu_setup_unique_irqs() - the messages >> here are in the sense of "we're giving up on MSIs and falling back to trying >> whatever wired IRQs we may or may not have." The critical point is that >> we're not using MSIs for some potentially actionable reason, i.e. if the >> user does expect the system to be MSI-capable, then it could be an >> indication of perhaps a wrong or missing msi-parent, for which they may >> pursue a firmware fix. In other cases it's normal and expected not to use >> MSIs though (e.g. the system just doesn't have an ITS), so we don't want to >> be *too* noisy about it. > > The case I am worried about in this patch, is for systems with > MSIs only. > With this patch, that means, we fallback to wired irqs which don't > exist, so the driver will probe with no interrupts at all, which in my > opinion worse than leaking the memory. True, the logic looks a bit off at first glance - I was halfway through writing a reply to that effect - but then if you look past the reality that this is all academic since it's never really going to happen anyway, if we *did* fail to allocate 16 bytes here, there's an incredibly high chance that immediately proceeding into iommu_device_sysfs_add() is also going to result in another (larger) allocation failure which ends up aborting the whole probe anyway. Plus the chance of subsequently being able to allocate any domains/pagetables/etc. for any meaningful IOMMU usage would seem slim. Honestly I'd be inclined to do nothing more than add the _or_reset to shut the static checkers up, and not waste code and data on a useless message for a theoretical condition at all. Cheers, Robin. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM 2024-04-03 5:37 [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM Aleksandr Aprelkov 2024-04-09 10:43 ` Mostafa Saleh @ 2024-04-19 15:32 ` Will Deacon 1 sibling, 0 replies; 6+ messages in thread From: Will Deacon @ 2024-04-19 15:32 UTC (permalink / raw) To: Aleksandr Aprelkov Cc: catalin.marinas, kernel-team, Will Deacon, Robin Murphy, Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Michael Shavit, Lu Baolu, Marc Zyngier, linux-arm-kernel, iommu, linux-kernel On Wed, 03 Apr 2024 12:37:59 +0700, Aleksandr Aprelkov wrote: > If devm_add_action() returns ENOMEM, then MSIs allocated but > not freed on teardown. > > Found by Linux Verification Center (linuxtesting.org) with SVACE. > > Applied to will (for-joerg/arm-smmu/updates), thanks! [1/1] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM https://git.kernel.org/will/c/80fea979dd9d Cheers, -- Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-04-19 15:33 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-04-03 5:37 [PATCH v2] iommu/arm-smmu-v3: Free MSIs in case of ENOMEM Aleksandr Aprelkov 2024-04-09 10:43 ` Mostafa Saleh 2024-04-09 11:17 ` Robin Murphy 2024-04-09 11:31 ` Mostafa Saleh 2024-04-09 11:46 ` Robin Murphy 2024-04-19 15:32 ` Will Deacon
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).