From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1A92D3C526 for ; Thu, 17 Oct 2024 18:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uuumkGx4ZqXhWWqsbZdCKrFfR0QqlOWV+/hPWY6qiII=; b=Q5NZtTm7b/3x8UdbkdZ4NoyfGl 7+Q5TyeCHHnpv+shkKGfTjZGj5xSdMYP0qjpeDQkfixJEyPeSsjtr+08IkPHGrK64nNoLBoir7C2Q Btp5k0XAa4hzlJtpauDyFTKE+zvVqPdQFaVYwVHdt7OvHNSaTc3A24ZHZYHfjCbb+KGMRl17yfld3 6nSPK2tCjMCvcqPFS+9bBkdm6h4mMRMyzSJ3CH8/6CM0u1yzzC+G8aMUrdkBXiBadEe0sWbwFzsus db44sYi2W5MDAUIfS6iQeB7XO2DcuNOHDpWp6hhAGxlkFi/Otxgg8vAELyHmf+77bUlRnwWW2I0Cj V6guMGXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Vdi-0000000FwQe-1Rt6; Thu, 17 Oct 2024 18:54:26 +0000 Received: from mail-mw2nam12on20625.outbound.protection.outlook.com ([2a01:111:f403:200a::625] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t1Vbx-0000000FvwA-2mWm for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 18:52:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dpUpuipmsnz7oK7Azh9H6UaCjhvdI7uM8s4G91PttNeyZWAbxxlPlQ8UCidFBkH011L5TZIO308OZbHk/G5ZC6XzXKD2pqRp9qMQ38qzCQqnFn7HtbUKAqmANOOGbEE2JmWshGxphUWawB8roZzGHnSFcU/eOQ3ZE7X9U8ZVhdkJhR5u+ObFD1T97nT2QcYJdmlzbj4e2dw72sJrLwadmgQg4afSBHQHbtuGg6towKtXKKZxKniYaPU5xbMN/hQhXFknZkwO2gjMrUgQ+GCXHjs+XR614+28cdcQmJb8hP6Z/IXpWMkCxLebdkkTodE7nUyrGzEijm1M+DI8ZTuYAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uuumkGx4ZqXhWWqsbZdCKrFfR0QqlOWV+/hPWY6qiII=; b=dZUrI5302xSS7+bYUzxlCj2/KxeyAkGsD4+usukr6OA2twTSZcS2c7icBBGq7ylGBa/8Q6PxGrp/e4cHa3rYJQGiIKCK9OZjtJNXr2o2UNIbxn0vPLCifnQCOKYFTXOdw1eIhkoACp08FUOPBjj2AkCAWV/9kRcNrGVWNPOb+vh1m7T9k1ny1QuTPYpNs+9WbuUqsM9TwkvWC438PndjQ3BsDy9BapkRCXb/sr0UPRFJ0UJavznjRuC4MCAUtpdXtPOlVH+sXpTcs3RyBf3IqsIeOqfVgFSacLRXo2aq43St+iHfg+bgp+hmSG47XMmNR+Go4jhtNNV0EfLovogGHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uuumkGx4ZqXhWWqsbZdCKrFfR0QqlOWV+/hPWY6qiII=; b=nRTsqgsofBwsiWbheCllUtkA5L0JqxHTYUU1b3WxeSy1LgZuRd91znfy86ue/23wSs8p7zYi9psoJTi/x/YJ2T4QDYW9YL09O8YbCeGi0AbyGPAjjQMW1pCeWpHafo902jekb4S7A0M27KLDQh66UxqtqKB785YWjCo+gm+rf6MadsmNxbRSeFboJ4zRSPTdr8ZqDQ+EzXvyzDTFrX0BcKp80zenr4M1o6PgxQ2N0JY/8nxA25fbym8JJTgt3FxuZmSC80BKDBZGR6TYnJLDEEbp8tXQ8VoNUt4HtNBSxDFcw28jRLiIgqgtjWLEfvfzS4YSFGjaW/LtdA83Pkg/Lw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by DS7PR12MB6213.namprd12.prod.outlook.com (2603:10b6:8:97::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.27; Thu, 17 Oct 2024 18:52:32 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8069.016; Thu, 17 Oct 2024 18:52:31 +0000 Date: Thu, 17 Oct 2024 15:52:30 -0300 From: Jason Gunthorpe To: Nicolin Chen Cc: kevin.tian@intel.com, will@kernel.org, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, shuah@kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, eric.auger@redhat.com, jean-philippe@linaro.org, mdf@kernel.org, mshavit@google.com, shameerali.kolothum.thodi@huawei.com, smostafa@google.com, yi.l.liu@intel.com, aik@amd.com, patches@lists.linux.dev Subject: Re: [PATCH v3 03/16] iommufd/viommu: Add IOMMU_VDEVICE_ALLOC ioctl Message-ID: <20241017185230.GS3559746@nvidia.com> References: Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1PR13CA0194.namprd13.prod.outlook.com (2603:10b6:208:2be::19) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|DS7PR12MB6213:EE_ X-MS-Office365-Filtering-Correlation-Id: 33dd42a2-bc60-4a8e-dc79-08dceedcdb23 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?kseFXxEV4zjcRgLCLDeCEmrNy19Drqm1yks6+WbAYW1OPS+H0DDBNxgb32tV?= =?us-ascii?Q?MOEv8dkS8SGZGpK5qTlZkEiowa6UCI/iy0GK220lTFN8VQdpCd6Quuv7GGwU?= =?us-ascii?Q?6Bw9c4GQE5HQmG8rOtrcjbLBxNOaeBlYryzV4GLQV4lCkW8ESNzou8lWfpkJ?= =?us-ascii?Q?rg+Q7mQgPa8bBM5A8RgP1CZHZ9QdTogVfM10JbFkR8UBl0TFwxT4zQb3ArZ3?= =?us-ascii?Q?qcubP89yKZUT/C1ygXTf1k621s/afvXrqrIc/P3uXqV6NAwjgsPlGaLZE02Q?= =?us-ascii?Q?/+gdWGTO7iQx5XMmA7n26+hQznZ9aA1waSHzAIRLwZTFo4Mef/eqhV5wnVoD?= =?us-ascii?Q?rNfflAmagSkT+cvAJCDOP0scRQJDGBF4zShJcSrPNHPqjaoEYF1v9vSXfLoE?= =?us-ascii?Q?IDRiC4DKcA0rFYf3woZECCto0AQM2eCRIKNOWy6jiiyCXb0G7A1qOX2KmFyn?= =?us-ascii?Q?92EagvE4VOEPS/jRk8bX0qezH36dzMxzpBk+vwYX0H79Z/0SYcWjtbbzgIFM?= =?us-ascii?Q?LrLJ3XUcciHHT6Id0KRBZvUvOmijlpSiYcJLbI1MgILe4hatRaCLHrL+GIWK?= =?us-ascii?Q?w0kCgbHJITKcb911463LYuarAjqGIYS2HO2OXWLFEkFX6c2TLtA88FU3BZ8b?= =?us-ascii?Q?kcnUm1yAUId55eOvVFs6b/nwMzg+VxaZWiqZIvXwR7tIQi7+gZzg6ReyzSxZ?= =?us-ascii?Q?cbi+CFwHsJz+UxNfOku9kmtn3AdYy14n4YnVhswk94KWN1qCzOLnz8RLiwZZ?= =?us-ascii?Q?dNZbswOGcAEL1cLDdt5dLLi7dacSDIKWiSz4KqMTcVN5Q7Bpd2gDsI9UdWtj?= =?us-ascii?Q?+qLSQEC9aj/XT16A3D2oZSc8w9dYYUoosSaRI+cNW5tL1Gtwl/J7+dMi7aAI?= =?us-ascii?Q?iRmgN7anJuJ5vlOXs6bxK05HFNuOOTz9H8q3UKLTCuYHQDokanGepSbTZa9R?= =?us-ascii?Q?+QefxSSbpYWouKrXF5oY823vc9S1fzLibJeecj+iTlDxchgBsXq8ekR1FW+g?= =?us-ascii?Q?oU61s/ebwcv1RRZUHF2NOki0NNXQVUpcAv77T4dX8PrrpURQ0L2Vo0+VGmSI?= =?us-ascii?Q?qDL00UbzxsYiDpa5znrYYdJUcUoSKTfH38feGsZf65Yln1Nw/K+tU0J5gMwj?= =?us-ascii?Q?+ifnm8zir7io38ZN7DSIpoGn7XqEEnStzYRYaNRM+wbZAWjmdNzL9QiLmS5d?= =?us-ascii?Q?J+IJzZYFp2y+YXtTdZXOUC2MaBUJIRu73HSt02pQ7yeKbodXI1MwZsHvMFTs?= =?us-ascii?Q?rU2ePH8fy1a3KwQ/8X7fTXFFsaFffQuCaekLmYu1x//95HuBUwcbcANvllLa?= =?us-ascii?Q?ClImk71dzSo5VGeSDJxr/nBw?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?waHG3JLhlwj2P+1qQrpPiwOhvPA5TkPKiJiD1U0/HT/UFTjTl5o8DbGI2Vlx?= =?us-ascii?Q?gteqt2OGxt/67blTGK4AGTEWu/I6GDqkc2M7xgEIqzPfiC3XJgbn9WplfcVT?= =?us-ascii?Q?NU7piJ0JFvnqOZDc1vVCnDMufYTIAGRBJFN7oCf/jqSgV22Dd6ggfci7tfph?= =?us-ascii?Q?bGux7sN1F0N2z75JXRyCc4SjAXMqFZmv6SfSu5kYAwM5XRdA6aorTCnM6phQ?= =?us-ascii?Q?r5pEog/acrOtrc6ulO7yTwmexX2nRsswO/5g9R/BY/qG6hkm4B2azZJr19BT?= =?us-ascii?Q?SOLEF0ay2EDoHVrnPvRrBVq23alWcU9lenlL96sABkekFSAOJ8heLKrume0P?= =?us-ascii?Q?Kl9NFH7lW7/uC7tsu/sRQjZmEngH+3KHXL6CnozEBSh0vUY7yP9fwbwfy+aq?= =?us-ascii?Q?ZQ+tYRtONmOcmemcsv525FHtIa34TsaZ5lTMi/rMbAy0B54zm5a4uiCJbv+T?= =?us-ascii?Q?EeXd8BPOzH8iKtmoPZYMvZpKrBu0m2RDXvhFnTjUzL80QvkV7+Wm6KTAyZKn?= =?us-ascii?Q?WS8QxwpZIFv8JlIyw5OA06hhWcXWshAJWkYycA5WjIEzFudzqrnev1zDoeyS?= =?us-ascii?Q?mFfUwUV3/gYYFaaPafG9/t/fdCbB+5+GtAXwDV9PI3PmHV+ttib5S70qizW1?= =?us-ascii?Q?XIezh1AjvqzQSGR8BOhIFGJm4/TiPNJdP+m8KT34Ruqod7KcL6areqcU+gJ7?= =?us-ascii?Q?OOFjHcbKDTGPaKr8nho1lwNgfxZCCRa6HzLn8kt2S+H9q3kNaiKcAF7QTHed?= =?us-ascii?Q?xj2DNQu0mPAYA87tOdL28uA3dzdVPNfEs9GHg0PnajBreTogDhZv2y7+bjhs?= =?us-ascii?Q?VbqBbqGVx+zdAnqQ53iAxc+m3JhdmBp5McYDCOLHo66zpVjjbNQoLaoAhbJh?= =?us-ascii?Q?8LFpqwT0YAEPDXD3LmLAJY6QeailIB3zte0nb0XD0jMXRUjsNeECRa9Gv7ZS?= =?us-ascii?Q?q5iKaRLIgWx9EkzF3ImSTsNzU6NC/IJ3ZxKEmbYPQwMAb6Vox8c7HJGNvMzy?= =?us-ascii?Q?V0ifdi0cYggUy0RqwwQ7DnpMd2ZuGu4y1Abrl9olNGPd+P0ywrK9c+6F7Od1?= =?us-ascii?Q?RxjS5dQnQ9jggkONJegQ1ZbOyekSTM/SzMAubc1S62bRK+27TyprIXbflvV7?= =?us-ascii?Q?tcOwGuIdURt/dWN86cyXyaSmMnPC83Ez4qnXcaU/vx5iwrOodQRGuC6m6wyg?= =?us-ascii?Q?gki0s4iZQRCAH/yF4gqEKbgwy9nTU0F3tTuNGG8iiPDmY+jcKMq3CGoTJk9c?= =?us-ascii?Q?gO1JiryqEXxaYpF7Ms3r3BnLxJEgw+nDlT9IM+1myAfDUXArL6A/bUGhl2//?= =?us-ascii?Q?eSEjIV+/LiuiLBvb4Z2MMesHFvuDcbAVyWc0RdkEvFqTiCbB9lpQcR4dqXkW?= =?us-ascii?Q?vd4hNT38dGkTDN8jij32cxj46tEJ7B9beCodz92b8vxID8xksG/uebRmu1be?= =?us-ascii?Q?TF6SFt2knTsSjI6WN98oOnwzM4n8b/qPtKY9OLqX9fCsnRZ7Ez5GMSfL5+df?= =?us-ascii?Q?8h191/ihFuxA6QIxzComc6tBdQ2FIwPNIGFFqUabk/jTqDtgoFdbXl/SOipU?= =?us-ascii?Q?uz/0yFDz0gmUuM16Amw=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33dd42a2-bc60-4a8e-dc79-08dceedcdb23 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 18:52:31.8045 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RbL74IqCvHbM8rRonjs9Bckos7y5S6rd+DPJpktEKW/zLb3m5tfQwx0JggCR626m X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6213 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_115237_853890_0FA45E35 X-CRM114-Status: GOOD ( 13.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Oct 09, 2024 at 09:38:15AM -0700, Nicolin Chen wrote: > +void iommufd_vdevice_destroy(struct iommufd_object *obj) > +{ > + struct iommufd_vdevice *old, *vdev = > + container_of(obj, struct iommufd_vdevice, obj); > + struct iommufd_viommu *viommu = vdev->viommu; > + struct iommufd_device *idev = vdev->idev; > + > + if (viommu->ops && viommu->ops->vdevice_free) > + viommu->ops->vdevice_free(vdev); > + > + old = xa_cmpxchg(&viommu->vdevs, vdev->id, vdev, NULL, GFP_KERNEL); > + if (old) > + WARN_ON(old != vdev); > + > + refcount_dec(&viommu->obj.users); > + refcount_dec(&idev->obj.users); > + idev->vdev = NULL; This should hold the igroup lock when touching vdev? > +int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) > +{ > + struct iommu_vdevice_alloc *cmd = ucmd->cmd; > + struct iommufd_vdevice *vdev, *curr; > + struct iommufd_viommu *viommu; > + struct iommufd_device *idev; > + u64 virt_id = cmd->virt_id; > + int rc = 0; > + > + if (virt_id > ULONG_MAX) > + return -EINVAL; > + > + viommu = iommufd_get_viommu(ucmd, cmd->viommu_id); > + if (IS_ERR(viommu)) > + return PTR_ERR(viommu); > + > + idev = iommufd_get_device(ucmd, cmd->dev_id); > + if (IS_ERR(idev)) { > + rc = PTR_ERR(idev); > + goto out_put_viommu; > + } > + > + mutex_lock(&idev->igroup->lock); > + if (idev->vdev) { > + rc = -EEXIST; > + goto out_unlock_igroup; > + } Otherwise this won't work right > + if (viommu->ops && viommu->ops->vdevice_alloc) > + vdev = viommu->ops->vdevice_alloc(viommu, idev->dev, virt_id); > + else > + vdev = __iommufd_vdevice_alloc(ucmd->ictx, sizeof(*vdev)); > + if (IS_ERR(vdev)) { > + rc = PTR_ERR(vdev); > + goto out_unlock_igroup; > + } > + > + vdev->idev = idev; > + vdev->id = virt_id; > + vdev->viommu = viommu; > + > + idev->vdev = vdev; > + refcount_inc(&idev->obj.users); > + refcount_inc(&viommu->obj.users); > + > + curr = xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); > + if (curr) { > + rc = xa_err(curr) ? : -EBUSY; > + goto out_abort; > + } > + > + cmd->out_vdevice_id = vdev->obj.id; > + rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd)); > + if (rc) > + goto out_abort; > + iommufd_object_finalize(ucmd->ictx, &vdev->obj); > + goto out_unlock_igroup; > + > +out_abort: > + iommufd_object_abort_and_destroy(ucmd->ictx, &vdev->obj); But be mindful of this abort, it doesn't want to be inside the lock if it also gets the lock.. fail_nth should be updated to cover these new ioctls to look for tricky things like that But the design looks OK Jason