From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 227C1C38145 for ; Thu, 8 Sep 2022 00:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5wC8Br9U0PqfY0UeddvYlOOVbCGjJMZeR35diULGDCc=; b=RC2eSbGypmdKXe evW/YgHLLMSyvWmXc+rRG8EnWeJCt6DEcaIoRIfpvE1YqZZy7gQUg/sKhjKvKNmXgOnNUf8q8VOgz Ti1BlOLVl5xESXTihrn2h1qrveFpTI+UsiFOectY9I4wNI+AejT5fV8VXmKyNj7MSLurAFzUbfOaq 77QCI0AIMlXHz9+s5topIFQwf5laPUQDQ6w8l6jR4RoG82h0lXZH0XjSfhdAX0j7VDtlQ9J3qx+P6 ilQPLzsFue+IJTuPx3Wxobc2CsqobdXmH9DL3wAs6xeeMJEGlP74x55ZMh2dQ7CtFEOGJaZekChWn EwxbeeAJsXy4rEUvpPCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oW5dq-00DsiU-DZ; Thu, 08 Sep 2022 00:43:38 +0000 Received: from mail-dm6nam11on2062.outbound.protection.outlook.com ([40.107.223.62] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oW5dm-00DsU3-9q for linux-arm-kernel@lists.infradead.org; Thu, 08 Sep 2022 00:43:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I5o3TZxYnm40o3i4bFh7m2h9FKcbEaPahMkA4G3XjtuFq/dJldww7xDgLm3iMuSx0QeOWMdkXi0HItPMBb0tcrLm6kiDf/uyO8PG2tfCwVRM5kT3MvLHAqewSa5TuxyB1SNR62jIszTqnTPLxWw/H+qsPuF4/DOOiDfPcCMCDuK7aCB3PDccTqzyX/QP8ORIsmeSSoug3zvb+1oXXBYBld5QGxQZesjpVAghn28DDNX5YKbnm08nIbz5FCTxyRnedtuvV1oVQuQpNPS9aPDmNtOdWkr85mAkltoajQW8Kl13o6ikiTavNr9b/Z6rwR94XfHtH7GHYNvUU0wymd4Jfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KfOcDBEgVXO7M7Msk8gAo6LefoAGPdujDTjf4tJltk4=; b=ig/BO1z1f5lNABM9/xw1qcrzt2XJfc56MhJrK3H+MBYZ039OytmbZDzsPynJ52+yQ/D00kXT4j3eyWqNiKe5Eme4AJYfJAxzmVYVmUK+eEFxr+5LO7WEoTNyxcHF5dDMBexHd+9AJ0A+1snt11P3YQccyxyX+q8B0wR+SS1VTbQeuKIdwyVUDEooHjr3Do89Xsj1nkrigI1aAr7nhUsevUu7spMoPtmYor26xrd2j6yS6FaX9aByEBgI1HnvLkaS5URRiyde68TOFXDdNTLJ6e4DhyY+dX/RUz8+pBPZTRZ7qNaovJE4G/bS+lskOZFuyV2KXqDQUbqxkTpZAqMg6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KfOcDBEgVXO7M7Msk8gAo6LefoAGPdujDTjf4tJltk4=; b=nK9esuXDTHF86bApkiRgPNu0pus9/seJ8Qjd1935mZtMyhMayF3f3snT20FO04xSBJ/XhIVsOM9oJAPnpbL7UoL6fu3LLS3JHsMCiiYfC00Z227OQor5eh0y76SBTDhRUA/aJ/T4TpNKny7b2tbb1CkHHJ27lZS3rPJLJ0agAMFPwfkFMTvrOxU1HXrnxrFpAv8W3Yozmpyjnt2PbTlLSNdLAF3PTpTQJPjP+PRpERYPfiGDa4NvsF1vu4fx0/FfqWoqGDJ+UJ5Ox+9E2vpXJVsqS9C2hBQkomVYz2mlElLxUeXOZN1bEocRrOjUiauGvs3yks4/j8eKyeDVcjEr0g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by PH7PR12MB6786.namprd12.prod.outlook.com (2603:10b6:510:1ac::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.11; Thu, 8 Sep 2022 00:43:30 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::462:7fe:f04f:d0d5]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::462:7fe:f04f:d0d5%7]) with mapi id 15.20.5588.018; Thu, 8 Sep 2022 00:43:30 +0000 Date: Wed, 7 Sep 2022 21:43:29 -0300 From: Jason Gunthorpe To: Robin Murphy , Joerg Roedel Subject: Re: [PATCH v6 1/5] iommu: Return -EMEDIUMTYPE for incompatible domain and device/group Message-ID: References: <20220815181437.28127-1-nicolinc@nvidia.com> <20220815181437.28127-2-nicolinc@nvidia.com> <9f91f187-2767-13f9-68a2-a5458b888f00@arm.com> <0b466705-3a17-1bbc-7ef2-5adadc22d1ae@arm.com> Content-Disposition: inline In-Reply-To: <0b466705-3a17-1bbc-7ef2-5adadc22d1ae@arm.com> X-ClientProxiedBy: BLAP220CA0024.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:32c::29) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bcaf40b5-a886-4748-bbb4-08da913326a4 X-MS-TrafficTypeDiagnostic: PH7PR12MB6786:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /2e7yqB3vrvU9ruu4a/VRCXep31ycUJNwiCKVB7GmD5m+ARbCa5NAfOVO2y0LT6wcMnK3eCFSQnqRowPqZiVzyYUe9kuv8pt6kFmgc+sILmHCpKmCMeBle1rn5RTWv/ncjtNUIOp1PQcfxeOCC+VZI6ceYOLmZiwsJAj5RSPfLo/fxtI108qL6BgQxH0h1+SK+FR4wQkVgWDNCsDjX6q3LexmtAOy9PcGZzJ92lRXilhUwDq/YNNpul2aSM1N9nMuP7VaPCP7d+shRxpYFtNrAEeaFJzp28w/QPCeHdMiKvKERGM3ZMOQHzSCCvR8lOJ6Mf8N+H3Oq3dBP0iZ34e9Bigq6A/SQh6AT0A8PVXDpd8aOmtKcttXfbwm48rZKtuDorq+isDlQn04OS5I++DTz7UvIG8UlFGtGNu+7efS8V2nmk0x1uwSoCnpp+DGlx0gL5oKLkEGWamIQ5qhK2yLx64u2WhLihJr9vMJeKcYd0fM2JkDw6QzJ7ZKjY8aIGV9dTWD5oWCbDv9pX8lSnMBJ5OhAYOJgIkf3vUBdtKF21MhjS53INr50bwx3nKJupeayIwH9pJSeasPqfLpc/fHcq1/29Z8lNJo11CVqDLLyE6/3OF9t73NA2XHd1wgJKWTGifS0Z2IVsS1kMMG4DcFCwn+N+MOdwxTJvJ3PiSqu9bxHWdOFb3UilnCXa4S1oBcny8PAR9XHfDhak0/xL3WA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(366004)(346002)(376002)(396003)(136003)(6506007)(2906002)(66946007)(6512007)(2616005)(8936002)(4326008)(8676002)(86362001)(26005)(66476007)(66556008)(186003)(6486002)(7406005)(7416002)(41300700001)(478600001)(5660300002)(110136005)(316002)(36756003)(83380400001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?PFGu8UtqXI4CTRN5ajnfKpn1Vjy1pzzQgVwHDc8FcrTsP7tHO4aGDTdXr0Cr?= =?us-ascii?Q?OpEbtPL/AkKZ//1h3u8V48nxpN+DkORFk0DrmKIKqpXE5/9DoyDksAgwXH5w?= =?us-ascii?Q?Yb9aKoK8BQV0aZHxZf4reJPUx+UmK9hc4HoDjaVDZMar6h6h93UX4uOGvsXQ?= =?us-ascii?Q?qVh5FeoSMBKiveKJs8ivvb4syTH5oz6ZjoeJ8NVt1nk4iG4i6vXiFksHi8+a?= =?us-ascii?Q?Fy53OgWBLhLY9HXL/F1dESRTtIsWw7ERpoI2XeBgbQrxnjL2XJxPxIz1Cn99?= =?us-ascii?Q?uJ11k4bpPiBbAAd9xjfRb8YIRdWDbzBn281xC7o2XPLu0CeA+xeC4d32LWNx?= =?us-ascii?Q?UlCssla2VH/Ina6gxXeCCcBxYYMF7ZoTu3V8S5OYsnZRq6dA5Js/3HZkv15l?= =?us-ascii?Q?vLFDfqMI5Lxw4WFuA0YnuZWKutvU2fTfk+imbuv9V1esY6dnZH2gAWCeO9To?= =?us-ascii?Q?lIf/mVdqLxPGIk+VUL16Fp4ZpBgVpL2ahknWUGyMaOM0mb8fLqaMdERSStHe?= =?us-ascii?Q?/GM4D4SPiWd3UmvzkGiE411bua2nZJcNRYwtdTUamBfamU6NLy/vwlFYJsBB?= =?us-ascii?Q?40SzHnnPYj7J3yA2xS18+bXXJot8V0H4u+DrNFEom4uVe1zAz5TTVjpOF01h?= =?us-ascii?Q?fQ/bM961O3W2UvjrZUENO2uMQcC0Hbbfe9HThnOdPJpwzCZ3Jk+k44oDsNSH?= =?us-ascii?Q?l0+DkTWyjyPaEy/l70DQxYq7Bdy3PYllZ5EBEiGc8mMb3PdWrFIHPcyvygHk?= =?us-ascii?Q?NBUJzTbA3LSoFBsoKB5E19kTU2LFgQGsZ8CKenSXoHSTOAn/GzqXZ4ggdwo7?= =?us-ascii?Q?YYcv7tt8Ikg7IfvTKW45hX6jTR6Id72PVQmP18gNzvJx7FeJnzM1sd8fiF8L?= =?us-ascii?Q?1JaeDL1FxA2CsHm2uA2UdILZ7IBPLOTKI1f0fpHhRZeYduN5s6HJtcIyvE8p?= =?us-ascii?Q?zTOQEGc4rqM2kbc6U89wrj+07Ok8HOJGYolNamX5oTPBQBUHGU2LIMLb+4eK?= =?us-ascii?Q?NlqKanjHL88OVFI3OHHkw6+RWcyxdbG7PYDoSfBPpO76O8PwfqwJLgl5UZ5n?= =?us-ascii?Q?vpb3mG0ZE/EMDftiHfiqe26++7hZN98ssmvLfEyqLslP5iRlix1ltNlVwynA?= =?us-ascii?Q?1xxiJKDmUlib5cTS8NCqnXw5gQjFYa19EpKPkJVJzDYV6hPnU1ytPU4hyhJI?= =?us-ascii?Q?+T2vfo717F811kRebId/pdt8yqHneeLE4sDaxDlQD+gTLVXhxkCU0yDlNxyu?= =?us-ascii?Q?gWzQnr1X1WYiAE+gCEbn8G0xVA0W8CHBojz4Ugzqz+NfgcoL83HQELO4kfy9?= =?us-ascii?Q?GN41sGO4nOlMxJeMhzWbN2ndPvpMjKkTO2kRpW6QowBKwdTjC/arWl85Ju6c?= =?us-ascii?Q?5/5Xj1M0cG6C4MYFkbePisFioC4JaFv3bZrZQ52DF/t1XKsg6Y2ilcb7zU8/?= =?us-ascii?Q?qXD0ZkZ8BBFTbzcDTluV2xDyDCNBFO5UUv72z4dxxvPaGH8fwEX+192fSphD?= =?us-ascii?Q?YRbJ626YyuKUcsjH68tUbZd5Y+0OOPCNV31dZMv857WLvCC8KZKPc46lyQ2c?= =?us-ascii?Q?OdJYLd12m/zDJ4zXT7oVV55TIyuptxrJHqcZ2ayk?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: bcaf40b5-a886-4748-bbb4-08da913326a4 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2022 00:43:30.6535 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: A7f3il3fx2U/DyBc4XD+wKamRNl7Vp4SRqWpgvaNuk7fhKufGl1Ch0WCNR725EHq X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6786 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220907_174334_448287_1076C5AF X-CRM114-Status: GOOD ( 36.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marcan@marcan.st, mjrosato@linux.ibm.com, linux-kernel@vger.kernel.org, thierry.reding@gmail.com, will@kernel.org, alyssa@rosenzweig.io, jean-philippe@linaro.org, kvm@vger.kernel.org, zhang.lyra@gmail.com, jon@solid-run.com, jonathanh@nvidia.com, iommu@lists.linux.dev, Nicolin Chen , yangyingliang@huawei.com, orsonzhai@gmail.com, gerald.schaefer@linux.ibm.com, kevin.tian@intel.com, sven@svenpeter.dev, linux-arm-msm@vger.kernel.org, christophe.jaillet@wanadoo.fr, baolin.wang@linux.alibaba.com, thunder.leizhen@huawei.com, linux-tegra@vger.kernel.org, tglx@linutronix.de, virtualization@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, cohuck@redhat.com, alex.williamson@redhat.com, shameerali.kolothum.thodi@huawei.com, robdclark@gmail.com, asahi@lists.linux.dev, suravee.suthikulpanit@amd.com, dwmw2@infradead.org, baolu.lu@linux.intel.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Sep 07, 2022 at 08:41:13PM +0100, Robin Murphy wrote: > > > FWIW, we're now very close to being able to validate dev->iommu against > > > where the domain came from in core code, and so short-circuit ->attach_dev > > > entirely if they don't match. > > > > I don't think this is a long term direction. We have systems now with > > a number of SMMU blocks and we really are going to see a need that > > they share the iommu_domains so we don't have unncessary overheads > > from duplicated io page table memory. > > > > So ultimately I'd expect to pass the iommu_domain to the driver and > > the driver will decide if the page table memory it represents is > > compatible or not. Restricting to only the same iommu instance isn't > > good.. > > Who said IOMMU instance? Ah, I completely misunderstood what 'dev->iommu' was referring too, OK I see. > Again, not what I was suggesting. In fact the nature of iommu_attach_group() > already rules out bogus devices getting this far, so all a driver currently > has to worry about is compatibility of a device that it definitely probed > with a domain that it definitely allocated. Therefore, from a caller's point > of view, if attaching to an existing domain returns -EINVAL, try another > domain; multiple different existing domains can be tried, and may also > return -EINVAL for the same or different reasons; the final attempt is to > allocate a fresh domain and attach to that, which should always be nominally > valid and *never* return -EINVAL. If any attempt returns any other error, > bail out down the usual "this should have worked but something went wrong" > path. Even if any driver did have a nonsensical "nothing went wrong, I just > can't attach my device to any of my domains" case, I don't think it would > really need distinguishing from any other general error anyway. The algorithm you described is exactly what this series does, it just used EMEDIUMTYPE instead of EINVAL. Changing it to EINVAL is not a fundamental problem, just a bit more work. Looking at Nicolin's series there is a bunch of existing errnos that would still need converting, ie EXDEV, EBUSY, EOPNOTSUPP, EFAULT, and ENXIO are all returned as codes for 'domain incompatible with device' in various drivers. So the patch would still look much the same, just changing them to EINVAL instead of EMEDIUMTYPE. That leaves the question of the remaining EINVAL's that Nicolin did not convert to EMEDIUMTYPE. eg in the AMD driver: if (!check_device(dev)) return -EINVAL; iommu = rlookup_amd_iommu(dev); if (!iommu) return -EINVAL; These are all cases of 'something is really wrong with the device or iommu, everything will fail'. Other drivers are using ENODEV for this already, so we'd probably have an additional patch changing various places like that to ENODEV. This mixture of error codes is the basic reason why a new code was used, because none of the existing codes are used with any consistency. But OK, I'm on board, lets use more common errnos with specific meaning, that can be documented in a comment someplace: ENOMEM - out of memory ENODEV - no domain can attach, device or iommu is messed up EINVAL - the domain is incompatible with the device - Same behavior as ENODEV, use is discouraged. I think achieving consistency of error codes is a generally desirable goal, it makes the error code actually useful. Joerg this is a good bit of work, will you be OK with it? > Thus as long as we can maintain that basic guarantee that attaching > a group to a newly allocated domain can only ever fail for resource > allocation reasons and not some spurious "incompatibility", then we > don't need any obscure trickery, and a single, clear, error code is > in fact enough to say all that needs to be said. As above, this is not the case, drivers do seem to have error paths that are unconditional on the domain. Perhaps they are just protective assertions and never happen. Regardless, it doesn't matter. If they return ENODEV or EINVAL the VFIO side algorithm will continue to work fine, it just does alot more work if EINVAL is permanently returned. Thanks, Jason _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel