From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ACFE1C433F5 for ; Fri, 29 Apr 2022 06:59:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:To:Subject:MIME-Version: Date:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=TJBLI+IyUB9W7zKAQBVZs7Xjn57k+CEvpe8nkoy7fLY=; b=R3b9cTcOqN36n+fwuoTJjhnWHL j/CwzSmyHCLWlgQ2MJQr5NjmyAreu/sXk41aEb+gyehwSAhucT7XCrkRuCidMcxsewcuvKYPR7q45 aEgSQ0GxeubdPP6ZuoBse14Vov21irUCTawIQXkitqnhSwsK3+ZLGu46AEwvpZpgtE91L2EfO4Mti paxi3KEISjbn3oAyJegWDpdzLZCV+Gv6M8NQHMrf1MFbOMEZGMOic6uj2mkTmWn2gGfdGeVhTqGAC naQxSx6/rNjbzQIjAtwx+O7a0toHVfP4f6iBRIHkkYyDzOwubGfsTm8GhxvoPFxWCEmhlRcltUO5U VVonoAAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkKZo-009t3H-O1; Fri, 29 Apr 2022 06:58:04 +0000 Received: from mga12.intel.com ([192.55.52.136]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkKZl-009t18-9p for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 06:58:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651215481; x=1682751481; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=7IYIalnzyIdxUGK/p8/irKiOiXWCGEyooF3P66WCsLY=; b=lM0A+RZ6NkyDbWAKtzexPaR9V3/XJ6jhpLF50d6UzNgY6AL3iT6YsDH5 kd09Z7dwy3/GkZCzsZHufNv6SxoKS5Dn38PFpkziFvx0+dDDdXSNVBINO 1emUsQWuj60oaLHekggLuhKkDFpy2YG4oUHIRyaxos70LuvI6fPzi+ZH7 GgcJf8fivNAMnCAgYqk0p+bodH4Nz7zNXh/nx9e+Q19ciIvSZUXthP8Ok zfMuPYg9PuWyD1g85OwACrws1Wx+5ag538a2WQ3l9OxF1F6GCbalEuhVn R7HsXDC6VsaAqwXq+PMyYn5qpSpcqVitddwLz2tzhqvQMvVtmtJ3+gOW3 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10331"; a="246470198" X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="246470198" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2022 23:57:54 -0700 X-IronPort-AV: E=Sophos;i="5.91,297,1647327600"; d="scan'208";a="581937812" Received: from lye4-mobl.ccr.corp.intel.com (HELO [10.249.170.95]) ([10.249.170.95]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Apr 2022 23:57:48 -0700 Message-ID: <0e459e6e-f236-7a58-970a-a47677a23b44@linux.intel.com> Date: Fri, 29 Apr 2022 14:57:46 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH v2 03/14] iommu: Move bus setup to IOMMU device registration Content-Language: en-US To: Robin Murphy , joro@8bytes.org, will@kernel.org References: <1faba5b5c094379df3d99b8fec924ab50ad75482.1650890638.git.robin.murphy@arm.com> From: Baolu Lu In-Reply-To: <1faba5b5c094379df3d99b8fec924ab50ad75482.1650890638.git.robin.murphy@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220428_235801_385034_0A68C60C X-CRM114-Status: GOOD ( 26.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe@linaro.org, mjrosato@linux.ibm.com, sven@svenpeter.dev, zhang.lyra@gmail.com, robdclark@gmail.com, linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, thierry.reding@gmail.com, linux-arm-kernel@lists.infradead.org, gerald.schaefer@linux.ibm.com, yong.wu@mediatek.com, m.szyprowski@samsung.com Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Robin, On 2022/4/28 21:18, Robin Murphy wrote: > Move the bus setup to iommu_device_register(). This should allow > bus_iommu_probe() to be correctly replayed for multiple IOMMU instances, > and leaves bus_set_iommu() as a glorified no-op to be cleaned up next. I re-fetched the latest patches on https://gitlab.arm.com/linux-arm/linux-rm/-/commits/iommu/bus and rolled back the head to "iommu: Cleanup bus_set_iommu". The test machine still hangs during boot. I went through the code. It seems that the .probe_device for Intel IOMMU driver can't handle the probe replay well. It always assumes that the device has never been probed. Best regards, baolu > > At this point we can also handle cleanup better than just rolling back > the most-recently-touched bus upon failure - which may release devices > owned by other already-registered instances, and still leave devices on > other buses with dangling pointers to the failed instance. Now it's easy > to clean up the exact footprint of a given instance, no more, no less. > > Tested-by: Marek Szyprowski > Signed-off-by: Robin Murphy > --- > drivers/iommu/iommu.c | 51 +++++++++++++++++++++++-------------------- > 1 file changed, 27 insertions(+), 24 deletions(-) > > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c > index 6c4621afc8cf..c89af4dc54c2 100644 > --- a/drivers/iommu/iommu.c > +++ b/drivers/iommu/iommu.c > @@ -175,6 +175,14 @@ static int __init iommu_subsys_init(void) > } > subsys_initcall(iommu_subsys_init); > > +static int remove_iommu_group(struct device *dev, void *data) > +{ > + if (dev->iommu && dev->iommu->iommu_dev == data) > + iommu_release_device(dev); > + > + return 0; > +} > + > /** > * iommu_device_register() - Register an IOMMU hardware instance > * @iommu: IOMMU handle for the instance > @@ -197,12 +205,29 @@ int iommu_device_register(struct iommu_device *iommu, > spin_lock(&iommu_device_lock); > list_add_tail(&iommu->list, &iommu_device_list); > spin_unlock(&iommu_device_lock); > + > + for (int i = 0; i < ARRAY_SIZE(iommu_buses); i++) { > + struct bus_type *bus = iommu_buses[i]; > + int err; > + > + WARN_ON(bus->iommu_ops && bus->iommu_ops != ops); > + bus->iommu_ops = ops; > + err = bus_iommu_probe(bus); > + if (err) { > + iommu_device_unregister(iommu); > + return err; > + } > + } > + > return 0; > } > EXPORT_SYMBOL_GPL(iommu_device_register); > > void iommu_device_unregister(struct iommu_device *iommu) > { > + for (int i = 0; i < ARRAY_SIZE(iommu_buses); i++) > + bus_for_each_dev(iommu_buses[i], NULL, iommu, remove_iommu_group); > + > spin_lock(&iommu_device_lock); > list_del(&iommu->list); > spin_unlock(&iommu_device_lock); > @@ -1655,13 +1680,6 @@ static int probe_iommu_group(struct device *dev, void *data) > return ret; > } > > -static int remove_iommu_group(struct device *dev, void *data) > -{ > - iommu_release_device(dev); > - > - return 0; > -} > - > static int iommu_bus_notifier(struct notifier_block *nb, > unsigned long action, void *data) > { > @@ -1884,27 +1902,12 @@ static int iommu_bus_init(struct bus_type *bus) > */ > int bus_set_iommu(struct bus_type *bus, const struct iommu_ops *ops) > { > - int err; > - > - if (ops == NULL) { > - bus->iommu_ops = NULL; > - return 0; > - } > - > - if (bus->iommu_ops != NULL) > + if (bus->iommu_ops && ops && bus->iommu_ops != ops) > return -EBUSY; > > bus->iommu_ops = ops; > > - /* Do IOMMU specific setup for this bus-type */ > - err = bus_iommu_probe(bus); > - if (err) { > - /* Clean up */ > - bus_for_each_dev(bus, NULL, NULL, remove_iommu_group); > - bus->iommu_ops = NULL; > - } > - > - return err; > + return 0; > } > EXPORT_SYMBOL_GPL(bus_set_iommu); > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel