From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 505A7D5B158 for ; Mon, 28 Oct 2024 21:42:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:CC:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5/94DjP8AZN+nyxAsn4CFaOPl50B8/B4vzqoL8xHL1I=; b=2QhlgR8OzEWljJTdsSFww0VpiU swgFaVKr04fI58IQItHAZqkpwYK9IvMjTjQSunpyQZRNscYoU8cC1MQ3ov1NuqfhA63IM+qqIRIYq D/RN6Br4sAOb9mcMEbPD9pwWoqC3WwX/DiJbEYsFnQ2pHoFTylP9GQJ/QUxQrUAegE7sbjDRg/D8O kMmlCFwVmIfM17j2Jp1UwVOdaIVycS0RfVDRSGxCVgY4Ux2Y0cdc/RLiD0HUvYQM5ybMH9qZNuQuV hbR9uQNqlx4fZ2L3L/IXBjm3CKni7/zq/jWanICOOfkko6dFvYxjnDYz3BCJgrMQ+jkyqh5UbCG8K QBAlc/gA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t5XV4-0000000CNCs-0qbD; Mon, 28 Oct 2024 21:42:10 +0000 Received: from mail-mw2nam12on2061d.outbound.protection.outlook.com ([2a01:111:f403:200a::61d] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t5Wyw-0000000CIBs-10rs for linux-arm-kernel@lists.infradead.org; Mon, 28 Oct 2024 21:09:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BIIRmxnipvJTmlFIBVFxbjSP8s1hGRi5a5GJJV16aV7Dic4EKJ/etU2GIIZnX2+gqIMzLLWu96qS+tMeTudbkllakmtcqz8KuJo9OLNgNLnDUyIVeLLMWtC1It2GK46X+JfP6xGCv6fM60hZF0t0RB8+I5DCRpMP/2AsdNccOp3N0ywcesbRFpLo+pZtcTw9TJR1FjrespKGyv2oidAjIDeoM0Osxu0eWvKalA7Cu/LOtSFVzsr3ckAzaFurgPP8ag3aG1fz2oohmZaYlMuSbbRXRrngT1bYivL9n9cbRAUZ7mDm9yZKm8RBU2Qqa4Nk2ehraS25MBFqG2yVTbQAPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5/94DjP8AZN+nyxAsn4CFaOPl50B8/B4vzqoL8xHL1I=; b=GhlmGo1zABUSZy7qCK/7o347gIEwPqQBSKvsx8/IVz9QX4T/C3ow22+oSVuOJgvltj92iokmUYGztWHWtbpACqlY3xfHD4rUYJDabSkRqrKw1xc4xN4UTe6WL8gAEA+SStMOouJ7/W4DBW03S9bN+6f40JhQQI4Ah1S6lqOJ4UHU4H7BdbUq/KwV0nk5wZblYJPljMkq5rc6/gGA2LfVUsVqwMLHHwDkVb8gj9+T2ZwGgwGIKn7l7ODaPTZSPqnD0GGWK3BCPPkQ/Ff13OA7SI7mpyX3jnenlxY+IcmrVVBwlKg5dkyvLXD1LrYDKHvVE6NJWTVeLHSZWASTTgfXmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5/94DjP8AZN+nyxAsn4CFaOPl50B8/B4vzqoL8xHL1I=; b=UseCCBb3Ynh/0a60uQYL41jI8CiYaAq/XNH+TqH5XAzOqFv3cyQxeSLGSRAO75uuSt9GtafIsousDLtltoudPIo5oQ/MJUC0JUwW95y2Tp+ka9xoWLCyOmg+2HFjCDczTQ5UP07lAarc8EcJvT/M5sIPnXylNXmFAFtuC1GVsLR5KqUuFV5zrrDThQIOyht1TsphiAVl0hptKw7S1OEgJ6aUngjGJvn9eS3yc7q1yQ84Gz16THgA4u7iG+vlhIDMFlcAda/5dOIqrD35DW5IQOXcFAumA4YlhdQxEA+zOY1QUFVgH9b9EKrd2764RoVSbredrRhAi5TO+0km5x46BQ== Received: from MN2PR14CA0024.namprd14.prod.outlook.com (2603:10b6:208:23e::29) by IA1PR12MB7685.namprd12.prod.outlook.com (2603:10b6:208:423::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.27; Mon, 28 Oct 2024 21:08:51 +0000 Received: from BL02EPF00021F6D.namprd02.prod.outlook.com (2603:10b6:208:23e:cafe::86) by MN2PR14CA0024.outlook.office365.com (2603:10b6:208:23e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.26 via Frontend Transport; Mon, 28 Oct 2024 21:08:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF00021F6D.mail.protection.outlook.com (10.167.249.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Mon, 28 Oct 2024 21:08:50 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 28 Oct 2024 14:08:26 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 28 Oct 2024 14:08:26 -0700 Received: from Asurada-Nvidia (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4 via Frontend Transport; Mon, 28 Oct 2024 14:08:24 -0700 Date: Mon, 28 Oct 2024 14:08:23 -0700 From: Nicolin Chen To: Jason Gunthorpe , CC: Zhangfei Gao , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v5 06/13] iommufd: Allow pt_id to carry viommu_id for IOMMU_HWPT_ALLOC Message-ID: References: <4f5ed9d75c23a503ffe672a85cf9010569652794.1729897352.git.nicolinc@nvidia.com> <20241028130309.GP6956@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6D:EE_|IA1PR12MB7685:EE_ X-MS-Office365-Filtering-Correlation-Id: e781456a-75ff-4c26-1f23-08dcf794b8f4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DoJwqzvTgm2R8rlc3zAX8aGQGcr83niqJfAyedc4SaNz7L3tmPsoZ5dLgOR+?= =?us-ascii?Q?YxN5eAHAVeMCb8oK3knO7Axk7QXDCY67AX7jwQproCjXCbyDZFJgg+bnWOgc?= =?us-ascii?Q?kNa1szTERW7/WU+2bOjsJu26K64D7OuMszRiXOw4bcbsPjMeb9z/h6z/C6m9?= =?us-ascii?Q?/Qntgy4e++YRuLGYilMNFIqBPzYpYoXZAxbjW/kVvB9bK9aUuZrSUq13/xyu?= =?us-ascii?Q?lefUaqJVuO0RBAWHuEqWPcx4NXttIliW+ZWb//ClFCB2ZD9DZVbUg8VxVdxN?= =?us-ascii?Q?L+vUzCvgsJs9jMpwrg0HGN7WeGC6tSO/YTalJmS9ZW7RdHapIvHO6mrmINc7?= =?us-ascii?Q?4lh8p+Lq7mz5XjveLBH0gpbYnCHNmz55HdvRuqICr91Uk0yYvYlfIhD9yqVk?= =?us-ascii?Q?aNigz2uN9+/+OH8UG2JUYgnMaOtCgW0WxrADJOq9vxF+2TgEq7JeDeEFAPcV?= =?us-ascii?Q?a6OF5O+CKy1hmGXJNvUNdEKyQDeM1KWDI38w9RsKGskiKXVJXwNLYT1vcr5r?= =?us-ascii?Q?sLsdhjfbVcgUvxR61W3jMxxb2BJSVHWxBHmSCoD2fTiseAegn/oFk+F8ACjc?= =?us-ascii?Q?5OtQ9NVmwbpIyYndCdM3bjq9/bk9/ZxFsjjnX6T3hb5tLa3tbWSimhUI5WYv?= =?us-ascii?Q?DYCaQFltSlzRnEZgReLTUs8zVj/qNM+8Vw85OCGPjJEPt0UPQYb13Bt4g1Ph?= =?us-ascii?Q?yxWApkChoqrp/jdtXqPQOv/4xA1kWV9t8qrt/y3TF8o5qGYZ7qsXVSgY0H6y?= =?us-ascii?Q?BnJLzDuOZsq47y1MK7JTT+DqaU/8s62hhuC86dTYrMlDwC93q4arY2bxF23d?= =?us-ascii?Q?v8bB76RAjGoxkl4AjNMjLtitPA6jiLVoPcoUx+YL2NfXY+awxy3iNohqd1pW?= =?us-ascii?Q?mbZsZEFqb104v6qNyuJc4FmLxmmysx0sI7F2MzQ25XidgpZsSqfupJTG+s24?= =?us-ascii?Q?Wa9LzZ4wXCeoCl+8bGJXYTSd90frigYgS1ABtqnFUzbnfBDcpLLVS9BrL9+l?= =?us-ascii?Q?dAqi+/YdfT9nIGs82/zga+kb1PoB5DHCeCXBvivpDlJ0TZaAVNgeeG8zyF89?= =?us-ascii?Q?/m6oDVdKOiDg6VdQKrbUH+t9F/WhpsinjqUSXgiFnowBxJeiY/zqVYdKrxSl?= =?us-ascii?Q?ocacrSdKfEWQuUlI8Rt+I1fnCY7q6adaWwme62aAe1Z8QgdcmKyAOgUGEpZT?= =?us-ascii?Q?8F+rz5gEqTrm+qyLWgKqC1VeH0vgAjP80y6WiTprf5vTZgT2mx1FsK8wr44Z?= =?us-ascii?Q?zT4xVUXIo5PD9yYa9xuJ2SqoWjji6JihtGK3BS2Wvk0gX8uvzverc9618ZQS?= =?us-ascii?Q?X/vhhGr0cYgjCwABBqITxr0BYCdjF4Ru7qcv2WxKmKgEIPfeuC+m9qnRXHTu?= =?us-ascii?Q?YZhYQIfRmKDDTx5d4o7MOcG3LjHRbL+6bKEmp5i7oJha8Vjeag=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2024 21:08:50.7026 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e781456a-75ff-4c26-1f23-08dcf794b8f4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7685 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241028_140858_294642_B093C2CA X-CRM114-Status: GOOD ( 23.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Oct 28, 2024 at 07:52:10AM -0700, Nicolin Chen wrote: > On Mon, Oct 28, 2024 at 10:03:09AM -0300, Jason Gunthorpe wrote: > > On Mon, Oct 28, 2024 at 11:24:10AM +0800, Zhangfei Gao wrote: > > > > > > +/** > > > > + * iommufd_viommu_alloc_hwpt_nested() - Get a hwpt_nested for a vIOMMU > > > > + * @viommu: vIOMMU ojbect to associate the hwpt_nested/domain with > > > > + * @user_data: user_data pointer. Must be valid > > > > + * > > > > + * Allocate a new IOMMU_DOMAIN_NESTED for a vIOMMU and return it as a NESTED > > > > + * hw_pagetable. > > > > + */ > > > > +static struct iommufd_hwpt_nested * > > > > +iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu *viommu, u32 flags, > > > > + const struct iommu_user_data *user_data) > > > > +{ > > > > + struct iommufd_hwpt_nested *hwpt_nested; > > > > + struct iommufd_hw_pagetable *hwpt; > > > > + int rc; > > > > + > > > > + if (flags) > > > > + return ERR_PTR(-EOPNOTSUPP); > > > > > > This check should be removed. > > > > > > When a user page fault is required, IOMMU_HWPT_FAULT_ID_VALID is set. > > > if (cmd->flags & IOMMU_HWPT_FAULT_ID_VALID) { > > > > It can't just be removed.. > > > > I suspect that IOMMU_HWPT_FAULT_ID_VALID should not be set on the > > nested domain but on the parent? > > By giving another look, > > In iommufd_hwpt_paging_alloc(), we reject IOMMU_HWPT_FAULT_ID_VALID: > const u32 valid_flags = IOMMU_HWPT_ALLOC_NEST_PARENT | > IOMMU_HWPT_ALLOC_DIRTY_TRACKING; > ... > if (flags & ~valid_flags) > return ERR_PTR(-EOPNOTSUPP); > > In iommufd_hwpt_nested_alloc(), we mask the flag away: > if ((flags & ~IOMMU_HWPT_FAULT_ID_VALID) || > !user_data->len || !ops->domain_alloc_user) > return ERR_PTR(-EOPNOTSUPP); > ... > hwpt->domain = ops->domain_alloc_user(idev->dev, > flags & ~IOMMU_HWPT_FAULT_ID_VALID, > parent->common.domain, user_data); > > Then, in the common function it has a section of > if (cmd->flags & IOMMU_HWPT_FAULT_ID_VALID) { > ... > > It seems that this IOMMU_HWPT_FAULT_ID_VALID is for nested domains? > > So, aligning with that, here we need: > if ((flags & ~IOMMU_HWPT_FAULT_ID_VALID) || !user_data->len) > return ERR_PTR(-EOPNOTSUPP); I added a TEST_F for the coverage: +TEST_F(iommufd_viommu, viommu_alloc_nested_iopf) +{ + struct iommu_hwpt_selftest data = { + .iotlb = IOMMU_TEST_IOTLB_DEFAULT, + }; + uint32_t viommu_id = self->viommu_id; + uint32_t dev_id = self->device_id; + uint32_t iopf_hwpt_id; + uint32_t fault_id; + uint32_t fault_fd; + + if (self->device_id) { + test_ioctl_fault_alloc(&fault_id, &fault_fd); + test_err_hwpt_alloc_iopf( + ENOENT, dev_id, viommu_id, UINT32_MAX, + IOMMU_HWPT_FAULT_ID_VALID, &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_err_hwpt_alloc_iopf( + EOPNOTSUPP, dev_id, viommu_id, fault_id, + IOMMU_HWPT_FAULT_ID_VALID | (1 << 31), &iopf_hwpt_id, + IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_cmd_hwpt_alloc_iopf( + dev_id, viommu_id, fault_id, IOMMU_HWPT_FAULT_ID_VALID, + &iopf_hwpt_id, IOMMU_HWPT_DATA_SELFTEST, &data, + sizeof(data)); + + test_cmd_mock_domain_replace(self->stdev_id, iopf_hwpt_id); + EXPECT_ERRNO(EBUSY, + _test_ioctl_destroy(self->fd, iopf_hwpt_id)); + test_cmd_trigger_iopf(dev_id, fault_fd); + + test_cmd_mock_domain_replace(self->stdev_id, self->ioas_id); + test_ioctl_destroy(iopf_hwpt_id); + close(fault_fd); + test_ioctl_destroy(fault_id); + } +} Thanks Nicolin