From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2059.outbound.protection.outlook.com [40.107.220.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE0251494C9; Wed, 26 Feb 2025 00:49:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.59 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740530963; cv=fail; b=J5gc8IWITtnda7XuRbZj/9Gea8n2DRZGNoCfryt6ZCg/VeQkkp3NlFXq3xTVUTYh61Xv4EO2H2Btw7Mc8XnofPmyiv9UANKhedmTY6Gbzijdd4C3WPlXDF1IyPV7qtHPxNk6hgoU9h99dlm6igB9DXvpuYbMEyheepR+1cuqCOc= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740530963; c=relaxed/simple; bh=16Ekmce5l7CYWBlJYPXMNzBWqvf4q6pSdG0zXZWu2cA=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=TKJuOgoRupK1xwl+DX0nyBMzdQn7AqKjarE2bs62IQJYZTqwZB5HV9b8g98gGmZUiqe3vtDJOFIQR4STXeW2RvEne95yymCw3le3aLCxcGHbnKyf0foI46X4XTjEGExvvvkIceNNJSii1Ht0bFBOnqPtHHylKpzha3h+WCnv7Sc= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=TspGTA46; arc=fail smtp.client-ip=40.107.220.59 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="TspGTA46" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mUwKXMJHYY3CQEZ8Qh6eAixxJYB2HoZpVR+c88FuwftxyU9rJ4jY+mBOvIZzK1hIk7dIcRDmvvnJt5AK0pXUNiSYMIEH0fKob6Nqz8Yb0QlKnfLwNzvMQ4oQsqMs+JFadRK+7WPGMOhnpP0xAoPsdWObOA4+woTIY3OPnGhVW+6/+PclU4YRdc8FkfuPZlIPOZGDIZkNGRtJNoE0zF/RY2UiiRcnLjpK0+AVVbN9FMm0t99H7jk8Ky7Cs0yYS/cCOPWXb7FywBJvL1ESQi8A6/gjeftBqGHKbL9rm7yoTfxMbJz2YCFw5xKiuDHvK6LqrpXpqNrjhlssdWTlqsTxMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Wdi8QPj5+ubJssuLU8x8rt1XbSugnTdGgUIASxYtJz8=; b=AhE/geW009tPNppNL1y0v3B+nPBvTJ1OmuJaMVBZmqMUZb7z88z6GDWah9imxmSnJ6B6JBzs7VovfTHAsvWy0ehSgIST5E6YNjNdTpqiSQbTOLhQ7NdUWl3+FTxh8l+2qhyXVV4Tr+iRWsLkdQ9NUbqUKTed1ZbIN4Kf1Uxw1U15gB/udlnKtv9NgI/PDDk5BN25RycOe2MgoWPVQvVCfAyQDXdyzCVhO47yQ3GEVCN5EkHNx/RUCWSBj+OguFhFeqoF/a0qwCA2DkWxc+4asWBWkv1J92ytHTe8LAelQh7JfSXyWjPVxKYTOjp+bgLclkEtncoRmdTpvgZsGUzRMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wdi8QPj5+ubJssuLU8x8rt1XbSugnTdGgUIASxYtJz8=; b=TspGTA469cV0n7qLvoDYBI6yHMBX6jUJH0Y7GIGH6H77gyWju6C74IfZjwcNwR4tAQpywPdovWPVnWWWIrIkl5U+i7/kuXosVrzUJiF9tvfN2MiR2V6zfJ3Rv3wvoopxyPkYitBpTxQBIOwIwf++1AveI2ZZtVHro+ecRaIuHydqmcl9NhlsxswaW62kbrYrh+EBCRONhqknQQ7QXtn9QlxCmrgSBBlk7Y3vKVJ1KxUGFqmaFasL2wXLSTR3JzZF3k0mKH9lgh1W4qaZpMmknd51zPn8BqOF+usdSTolt7H35h9TnTfynaL62iiSV9N0JWVJ+P4p3ytXY8dZkz3yTw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by MN2PR12MB4357.namprd12.prod.outlook.com (2603:10b6:208:262::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.21; Wed, 26 Feb 2025 00:49:18 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8466.016; Wed, 26 Feb 2025 00:49:18 +0000 Date: Tue, 25 Feb 2025 20:49:16 -0400 From: Jason Gunthorpe To: Danilo Krummrich Cc: Joel Fernandes , Alexandre Courbot , Dave Airlie , Gary Guo , Joel Fernandes , Boqun Feng , John Hubbard , Ben Skeggs , linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, paulmck@kernel.org Subject: Re: [RFC PATCH 0/3] gpu: nova-core: add basic timer subdevice implementation Message-ID: <20250226004916.GB4959@nvidia.com> References: <20250224184502.GA1599486@joelnvbox> <2f062199-8d69-48a2-baa6-abb755479a16@nvidia.com> <20250225210228.GA1801922@joelnvbox> <20250225225756.GA4959@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1PR13CA0158.namprd13.prod.outlook.com (2603:10b6:208:2bd::13) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|MN2PR12MB4357:EE_ X-MS-Office365-Filtering-Correlation-Id: e337b819-834d-4b76-6984-08dd55ff664c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?gvStVip5vhW1tIL1kYZaDR+cNNOgsxReRR0n3GdQqpVNt5Y18l6sq+h4mN1W?= =?us-ascii?Q?UDc907Jo93rPCyMuYZ+jPYbZJJHzmmXw58kGHYHUhKt6uhUbjU3/f2Zx/zyT?= =?us-ascii?Q?+spcbDYrzzyaZG0gWdaXdPMCIRStA8uQgKzXQ+bVKwoGtCCPCcVvQnZgrN5D?= =?us-ascii?Q?N3iSHeoKXsuLQlNfLBx/nC/CR1GrBAiqgTo7NGu+SyVOLVJ3OVXitDCHUbyK?= =?us-ascii?Q?ydieaY9JZTtPRmMv9S+UJvn1aJmxqT2iRgT0ceMXrtZ5zNUY13kVAqtkfPhP?= =?us-ascii?Q?wPFFT3ihM+pF3VM3BEMKtOCTXGhGtCL8kyN7Ebhtt8d4ncArgIlLdxBWav1+?= =?us-ascii?Q?m7Vu7cpaCquRARHx8dvL09diSMLpAVYjBtRhX6zLgqxv1gRL7Y/axOAiyTNa?= =?us-ascii?Q?xsoKfYGjIB77MN2m4xSZawG/b42WT8lVW0Gdi5hWDC4bmvlQ6JFfVxTTZs8v?= =?us-ascii?Q?Tu05xTQWcwqKmlRJSnFTuj59ALE86chd4kXeLQ1AXg3ywodlCweZY3z3mMqU?= =?us-ascii?Q?mRELVAUVv79gpWA/91X1sQf1HADlHThNd6YYloauZ16fv30M2FZ7lwTDipdY?= =?us-ascii?Q?imJJsovNXSdi8h4u2nbtHFqhMmnN4pgeXPrdV3vRh5ri16Py0gcCZVlrKNkF?= =?us-ascii?Q?4o9AkfgUBwi+3WPVbhhIK+elSPk8YmfTPJoKMb81IlNwtjXWn24icFTAZPhI?= =?us-ascii?Q?k4lKCf+n+ylbKsFOSMgd68HUU3FO0MWhvTES4QxHqwfBwqCaiyYBS9cm2HPQ?= =?us-ascii?Q?zLP0zeLmP603U7HjSOF/qrOqo/rmSkx94kQKc/BrcrntoW0u4+2JF68m7YcY?= =?us-ascii?Q?JCoZ+V0uzQIoojJ+/yqzCmvyt1bZAzw/5OD4AZ4hiD9SCbJFwtQa7oLrlsds?= =?us-ascii?Q?04r3ayo3uJAqQmXA9XTQieB/K2rrkykpNruBj3c6sC/G+pFqXCUgHypHHWsu?= =?us-ascii?Q?DdBv2aTlXY6jhQpX1IPd4yGpM0Q4gZaHSX7nM9wpvnrVIQnJCi2pptP3seRa?= =?us-ascii?Q?eDVssQHbnLeQxcLYHq7cimSd0NjoqsxK5tqxN24Zy3ytWGO6jwwhSnBb61rK?= =?us-ascii?Q?w9Ikf+jE6u3e11BZPEtDHnCwruWLePWbF9uQTVxR1ficyz+NdZg05VH/z/3+?= =?us-ascii?Q?FJTqzT1QxWLAZmMPkqU4/POSiU9Zbolc17GWHDtlVOdDsFZPzizSmSH0m1CP?= =?us-ascii?Q?b+BmAQOmE3c7UNVUSYiTw/7zMVzsnVL0dQccZcp5m0uT9Ju12D4U/9YdN1Kl?= =?us-ascii?Q?uJQfdcJ5TzPVRNtePLTdVC7QQ7KSmnlAxw9O9rO6Po6b1LNFu2s95BYGCbET?= =?us-ascii?Q?j8BjOZ88ajZvgAW1yX1mfHDyENWCR1q26grg0qHBCy67FDaylP8gH3NMv1bm?= =?us-ascii?Q?Y0ZS1klzUkd6q/0RFQvyE1Yt7vs8?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?b40s9TV4vVXZsmWtbX4sP2pWDKA9fG0gnw8P/M6yUuFSAlknOa1sGyXbO5aT?= =?us-ascii?Q?Q3MFvQoIo4tdjzXfO/2er0bwUUFmAq4gNQ+agL7iMM8ztYICv7ZQmSTXE1qZ?= =?us-ascii?Q?Drj/znAzmDq16mhUGcAf5oliBE+rkPfTKNWGVbUJNwVijVhwPJPvcP3+zFnH?= =?us-ascii?Q?ZAEw2TnvF6uR4WXCWyEF/225DIBt1kDXt5Qszvc3Xsh1A9NPQhIG3ZqeA3rO?= =?us-ascii?Q?1xHvluRu4pOzSyPg+W2Gh2zecg2+/xJ9ksVBuV5miYz0jAgwewCNHurL8zzS?= =?us-ascii?Q?valR60BCELy0bhHjkpRr2N9jOtwcLXyY5/y04x5EsVnmOljhFL9L8QekyZS4?= =?us-ascii?Q?N4rZTnJyW2O6uDOXevyanYaahjjFOs3Gw289vLjkWydbK5ZvoIyh+5KWCU80?= =?us-ascii?Q?cqsEbtaGYTo7j8VPq7SMcKxYnS3JYydrmmt5oTsARLH8KbKYq4v6/7Kyv/C+?= =?us-ascii?Q?Z/EK+R8c6hbtbVFajOkyFyNZ92O9HsdWLMpu25cEcKiyk472HCZzD7EZWDzr?= =?us-ascii?Q?AUkq2ihSWf/ra0vTMwtodJWlNYsdB0IQwoziH3ChKYGZqT8Znc5E9S/6abpw?= =?us-ascii?Q?/vS8Ucwo0zwRQlFE2h+6NTrICta6zCK/uZ9dxEFbRN6tAH7jMm4QUA1ePHde?= =?us-ascii?Q?82kaMbsxz71a4dkFHTMEs85K6zn8r6/85eXTjrNk2fNix13YvvMm8jffd3wE?= =?us-ascii?Q?wxzQEctqT/25jjSrW6u6f41V+8w/efeYE3P5AdzqefioZyA7eoY4iZIOOovr?= =?us-ascii?Q?CUs6Vjh66EZ//N6n8NMt0Wi2TAOmsaq0iTx1KbubfqEhmMYOSj71TGugzU7y?= =?us-ascii?Q?0naS+rmWofJQ5jdWBtYLQEGE5oDP8lLs0wtyoVZroNaNKdgenXUSZ7bMa6am?= =?us-ascii?Q?AU6LYNTXbhGRD1scc5IlChD6NHSCdm8Pg2J63Gs213bO2XBcBw1dHFC1440O?= =?us-ascii?Q?ULJLNmYx9jEIMGjl7kQiMGfyDrQxwj5xYy1RrevWXoJ2nuZGPWjUHjK2USvl?= =?us-ascii?Q?wn+rZlk/DnvmcurkLiHqtmzWsjX8gD05UrV+xExrFr/9mjIQ4DOJhChABWf7?= =?us-ascii?Q?eU4hLT6rRVlKbju7qwvzr1Hdi67h7E6y4pW0OVH6sOvM7EpjFbAkuTObCOqH?= =?us-ascii?Q?mdyTcomYDwVLCLtP7jVhoSh9OQAtSwzB0B9XFFjiuAmQJBAmc3pS1aSrzSG/?= =?us-ascii?Q?79qNzsRgPvIAvniglDvyhtMrC6wKLWx2b2mElaoHwanka0nMJJF2JZI7CD1a?= =?us-ascii?Q?k61rd6c1G+tBnehu+S7aaQVEsgewf6MR3O7mjicT9l8rg3SQ0QnWcO1I8ZXI?= =?us-ascii?Q?yKdwE1OejIER5/zLWq/KrgaUgeSA9hZ90cRqE67Fhd10qiHu1BwLphjDEskl?= =?us-ascii?Q?kQnLyeXNuA/X96Qo46xIKSrqWWM5k0twnn6k5j+bZn8ozAxr+H2Yb8NIV6T7?= =?us-ascii?Q?jAzBOYiNv6Lg736mS1xGSMXuPgOirX4UDqo39ornNeBPBSucL5YrPhUUIN2S?= =?us-ascii?Q?2k4fJZxJPn0Uj1JDsKmVbjZlAE8DpKsmia5J6mLKy6XYfOJPDQju1F3EMSwC?= =?us-ascii?Q?fi0EHonzs9CL5Phq6fQ=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e337b819-834d-4b76-6984-08dd55ff664c X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2025 00:49:17.9283 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o1hwaWSDySjGx9LkrXehwbBj07AHZV1pyKpQcK5D4RZS24I7+lKW1O7OIduXCtuN X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4357 On Wed, Feb 26, 2025 at 12:45:45AM +0100, Danilo Krummrich wrote: > On Tue, Feb 25, 2025 at 06:57:56PM -0400, Jason Gunthorpe wrote: > > The common driver shutdown process in the kernel, that is well tested > > and copied, makes the driver single threaded during the remove() > > callback. > > All devres callbacks run in the same callchain, __device_release_driver() first > calls remove() and then all the devres callbacks, where we revoke the pci::Bar, > by which gets dropped and hence the bar is unmapped and resource regions are > free. It's not different to C drivers. Except that in C you don't lose access to > the void pointer that still points to the (unmapped) MMIO address. I understand how devm works. I'm pointing out the fundamental different in approachs. The typical widely used pattern results in __device_release_driver() completing with no concurrent driver code running. DRM achieves this, in part, by using drm_dev_unplug(). The Rust approach ends up with __device_release_driver() completing and leaving driver code still running in other threads. This is a significant different outcome that must be mitigated somehow to prevent an Execute After Free failure for Rust. DRM happens to be safe because it ends up linking the driver module refcount to FD lifetime. This also prevents unloading the driver (bad!!). However, for instance, you can't rely on the module reference count with work queues, so this scheme is not generally applicable. Jason