From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5582FE776C5 for ; Mon, 2 Oct 2023 21:23:09 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=kV+Au7MU; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4Rzv7c04sMz3vZT for ; Tue, 3 Oct 2023 08:23:08 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=kV+Au7MU; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linaro.org (client-ip=2607:f8b0:4864:20::1135; helo=mail-yw1-x1135.google.com; envelope-from=dmitry.baryshkov@linaro.org; receiver=lists.ozlabs.org) Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Rzv6c5jSPz3c2k for ; Tue, 3 Oct 2023 08:22:15 +1100 (AEDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-5a229ac185aso3292087b3.1 for ; Mon, 02 Oct 2023 14:22:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696281731; x=1696886531; darn=lists.ozlabs.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=KY1yTwg5kkSQ1PPrVUqg1E2GU9pyZpek6FOp/omImxs=; b=kV+Au7MUpwXKfZsRSQA7ZyQrojudMDy7WLyelGemfRxb2K9q/vr9SoZV9lqs4ugz3L XkxO+hVxOQ/NTXvlEesuihDnXhdK/ue3V35Z4DKBU9Q1m4Gb+eSb9zhA9qgY/e5dLUjt KvhUiRxRSK0cwJFrfwhJJeDBdnDXThjAEULO92EabhCY8h9NAIseHpaDsf2SlaQvudXP X+mM9X7CJsAYQRQXH6+sIRLbrXaZ6/ZrhV8gynROJyKLIfiRFs+bdMo3ybufoiVNaH+d qquGOk/h0OGqqDLDWP1zPLyB1LpaujZnjAPZgbn3A0xv+zEka5f37GcxgsFaOBEjjrht QtSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696281731; x=1696886531; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KY1yTwg5kkSQ1PPrVUqg1E2GU9pyZpek6FOp/omImxs=; b=dBy6DTaJNcYMQgUwdfQZb6wj+UoWBVqyztR/I2/MKsgrtYmpRT3SZv+GtWXkqDUDdt KlTu1iL5iO6BV/gl8Ts8PdmOuD0QZ/vGeM+ZAPNipdkALb/ScWAUoIi+2xy7dgFiJJTL g758iBy63CoaU0NVceenwoNYmHUkk5W679VT1VMfrbwFI1ZZN9JQcJWOSzfuxVWuzCNE gRlpGkPwcFultSV7dU+1g16VgCOAVSa/xGrb6S2BwDXgot9aZr4+tojThiwvz4aEn0zK 6fQcQA9QD9M0gYb+i6AthD0ZBnUCDKMQfvLLu0lQgIVd/MH28ik/iihg5R6JfjwyaDRM JnpA== X-Gm-Message-State: AOJu0Yx6fM9W1dGdiX17FHiw4N/s4yILEu2fsvDAscOD9zz7Y4V15MQ8 3e3oD4DwKQUcjQAW83BBQ/rUXSDxskx6x0C9DZBqqQ== X-Google-Smtp-Source: AGHT+IFaRCfqm0SlfcT/sMeXMK7dbF3zxTUUIPxu3awMhgeV/37hJ6olOIew2lBKO7L65qL9ql+xhT0CDDmED9Aaft4= X-Received: by 2002:a0d:c906:0:b0:595:9135:83c7 with SMTP id l6-20020a0dc906000000b00595913583c7mr11373199ywd.47.1696281730897; Mon, 02 Oct 2023 14:22:10 -0700 (PDT) MIME-Version: 1.0 References: <0-v8-81230027b2fa+9d-iommu_all_defdom_jgg@nvidia.com> <20-v8-81230027b2fa+9d-iommu_all_defdom_jgg@nvidia.com> In-Reply-To: <20-v8-81230027b2fa+9d-iommu_all_defdom_jgg@nvidia.com> From: Dmitry Baryshkov Date: Tue, 3 Oct 2023 00:21:59 +0300 Message-ID: Subject: Re: [PATCH v8 20/24] iommu: Require a default_domain for all iommu drivers To: Jason Gunthorpe , Rob Clark Content-Type: text/plain; charset="UTF-8" X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Heiko Stuebner , Matthew Rosato , Jerry Snitselaar , Matthias Brugger , Thierry Reding , Jernej Skrabec , Alim Akhtar , Dmitry Osipenko , Steven Price , Will Deacon , Marek Szyprowski , linux-s390@vger.kernel.org, linux-samsung-soc@vger.kernel.org, Samuel Holland , Joerg Roedel , Russell King , Jonathan Hunter , linux-rockchip@lists.infradead.org, iommu@lists.linux.dev, Andy Gross , Nicolin Chen , Yong Wu , Orson Zhai , Gerald Schaefer , Thierry Reding , linux-sunxi@lists.linux.dev, Kevin Tian , Niklas S chnelle , linux-arm-msm@vger.kernel.org, Nicholas Piggin , Krishna Reddy , linux-mediatek@lists.infradead.org, Baolin Wang , linux-tegra@vger.kernel.org, Chen-Yu Tsai , linux-arm-kernel@lists.infradead.org, AngeloGioacchino Del Regno , Robin Murphy , Bjorn Andersson , Konrad Dybcio , Krzysztof Kozlowski , Chunyan Zhang , linuxppc-dev@lists.ozlabs.org, Lu Baolu Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, 13 Sept 2023 at 16:45, Jason Gunthorpe wrote: > > At this point every iommu driver will cause a default_domain to be > selected, so we can finally remove this gap from the core code. > > The following table explains what each driver supports and what the > resulting default_domain will be: > > ops->defaut_domain > IDENTITY DMA PLATFORM v ARM32 dma-iommu ARCH > amd/iommu.c Y Y N/A either > apple-dart.c Y Y N/A either > arm-smmu.c Y Y IDENTITY either > qcom_iommu.c G Y IDENTITY either > arm-smmu-v3.c Y Y N/A either > exynos-iommu.c G Y IDENTITY either > fsl_pamu_domain.c Y Y N/A N/A PLATFORM > intel/iommu.c Y Y N/A either > ipmmu-vmsa.c G Y IDENTITY either > msm_iommu.c G IDENTITY N/A Unfortunately this patch breaks msm_iommu platforms. This driver doesn't select ARM_DMA_USE_IOMMU, so iommu_get_default_domain_type() returns 0, bus_iommu_probe() fails with -ENODEV. If I make MSM_IOMMU select ARM_DMA_USE_IOMMU, then GPU probing fails with -EBUSY. > mtk_iommu.c G Y IDENTITY either > mtk_iommu_v1.c G IDENTITY N/A > omap-iommu.c G IDENTITY N/A > rockchip-iommu.c G Y IDENTITY either > s390-iommu.c Y Y N/A N/A PLATFORM > sprd-iommu.c Y N/A DMA > sun50i-iommu.c G Y IDENTITY either > tegra-smmu.c G Y IDENTITY IDENTITY > virtio-iommu.c Y Y N/A either > spapr Y Y N/A N/A PLATFORM > * G means ops->identity_domain is used > * N/A means the driver will not compile in this configuration > > ARM32 drivers select an IDENTITY default domain through either the > ops->identity_domain or directly requesting an IDENTIY domain through > alloc_domain(). > > In ARM64 mode tegra-smmu will still block the use of dma-iommu.c and > forces an IDENTITY domain. > > S390 uses a PLATFORM domain to represent when the dma_ops are set to the > s390 iommu code. > > fsl_pamu uses an PLATFORM domain. > > POWER SPAPR uses PLATFORM and blocking to enable its weird VFIO mode. > > The x86 drivers continue unchanged. > > After this patch group->default_domain is only NULL for a short period > during bus iommu probing while all the groups are constituted. Otherwise > it is always !NULL. > > This completes changing the iommu subsystem driver contract to a system > where the current iommu_domain always represents some form of translation > and the driver is continuously asserting a definable translation mode. > > It resolves the confusion that the original ops->detach_dev() caused > around what translation, exactly, is the IOMMU performing after > detach. There were at least three different answers to that question in > the tree, they are all now clearly named with domain types. > > Tested-by: Heiko Stuebner > Tested-by: Niklas Schnelle > Tested-by: Steven Price > Tested-by: Marek Szyprowski > Tested-by: Nicolin Chen > Reviewed-by: Lu Baolu > Reviewed-by: Jerry Snitselaar > Signed-off-by: Jason Gunthorpe > --- > drivers/iommu/iommu.c | 22 +++++++--------------- > 1 file changed, 7 insertions(+), 15 deletions(-) > > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c > index 42a4585dd76da6..cfb597751f5bad 100644 > --- a/drivers/iommu/iommu.c > +++ b/drivers/iommu/iommu.c > @@ -1865,7 +1865,6 @@ static int iommu_get_def_domain_type(struct iommu_group *group, > static int iommu_get_default_domain_type(struct iommu_group *group, > int target_type) > { > - const struct iommu_ops *ops = group_iommu_ops(group); > struct device *untrusted = NULL; > struct group_device *gdev; > int driver_type = 0; > @@ -1876,11 +1875,13 @@ static int iommu_get_default_domain_type(struct iommu_group *group, > * ARM32 drivers supporting CONFIG_ARM_DMA_USE_IOMMU can declare an > * identity_domain and it will automatically become their default > * domain. Later on ARM_DMA_USE_IOMMU will install its UNMANAGED domain. > - * Override the selection to IDENTITY if we are sure the driver supports > - * it. > + * Override the selection to IDENTITY. > */ > - if (IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) && ops->identity_domain) > + if (IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU)) { > + static_assert(!(IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) && > + IS_ENABLED(CONFIG_IOMMU_DMA))); > driver_type = IOMMU_DOMAIN_IDENTITY; > + } > > for_each_group_device(group, gdev) { > driver_type = iommu_get_def_domain_type(group, gdev->dev, > @@ -3016,18 +3017,9 @@ static int iommu_setup_default_domain(struct iommu_group *group, > if (req_type < 0) > return -EINVAL; > > - /* > - * There are still some drivers which don't support default domains, so > - * we ignore the failure and leave group->default_domain NULL. > - */ > dom = iommu_group_alloc_default_domain(group, req_type); > - if (!dom) { > - /* Once in default_domain mode we never leave */ > - if (group->default_domain) > - return -ENODEV; > - group->default_domain = NULL; > - return 0; > - } > + if (!dom) > + return -ENODEV; > > if (group->default_domain == dom) > return 0; > -- > 2.42.0 > -- With best wishes Dmitry