From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0EBBDCF9C71 for ; Mon, 23 Sep 2024 18:36:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=J8R5Od5rvsKSJk6uXOVYY0tYItqL7C8XTc0L7RdFc20=; b=aawVPbSGSSvxtzxQJ/q9BORe4m d0zDw8P98Tk9g/3ZeIMrwCLsTLRa9meG4XaHqqVtqdLJ1OGl7xhBvlH8GPziiI35/J8ZwOuXMS3DD jrCmIACyzSZ2qaaTUDoWLDJvFb7czJ/zuCF+8YIfng6v5Qi+YnHPHcgjckVWOQE6O2Nlyuh3a1tlV 3ckRnXEI0hq/hioGfJqupdumq03+FYBPrLQYZSqd/a2VbIc5FuDSEzbp4WMgReFoUfSxtaCuGwU6o okpxMViwTVt0lYZqJogOQhUWDE+Zj1q7zWN4qNnq/yr2Lqbz9L4NDERJxKZa6RFxtxN6l86WLjhfq hqLsL5oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1ssnur-00000000HMq-1sD2; Mon, 23 Sep 2024 18:36:09 +0000 Received: from mail-dm6nam10on20600.outbound.protection.outlook.com ([2a01:111:f403:2413::600] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1ssntj-00000000H69-1tyX for linux-arm-kernel@lists.infradead.org; Mon, 23 Sep 2024 18:35:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vYq30kMnK3a9TKhykXLLc+dngvo0+Tu88ghnoVqXSzQFCL9ZwTVLVCAf52rBD6IwhcoBBpmnyU/RYlOKcRLW2puzMvfG4k2j9DtxHeFtNyyJWz44rRAwlvJeDqm9GDCi1bS37WOwUgo73ysBA4hIRNmS71ZQNOws6QueAFZOs8tWJZkluCV6vyrgcELAofYOdvFHzO855nT1ukQ6vML/F/2tRyLrn0fAiG/dut7fjMn7fbK5J1E+PwJLotBU9877VhGqS/LI/0nInMF+unFp9HHmJgCtSgSP0wJgVKFtsOch4DYWZoy1TcFuTrxG4hkaPfNMWzkAnpacH9++XnOQ5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J8R5Od5rvsKSJk6uXOVYY0tYItqL7C8XTc0L7RdFc20=; b=sp69+YQAJS+riW9ZJVWZuBeDAmicTmrHmHp0lRxHiARvRo0mDKXQcIL+VHrq0tRwuqyNPoFuoq7RGw3tIAvTKlfFQwRaFr36+0zME8tlhfgos2X9lrv5+iMzw5FAZr5j8wWoGIWJkDp7XTEVYt/rlpb0TKxnBidI6SneoXnjNlGSmirbadeYkpF7f6d8oBnpuHiRrTrcg4Pq4hRJRBB8+RA/H8tD/LNuRlf7nG5vb/qc+YoINqLSXx0z93tRzOSCUq4YHjZ1mGOtjngWcbzm4JMyeu3K5FZpuR/ID0KZ1ecKOJ++tCtLCQCepkODhR+hXdrbENF8tJ/WoKnC300KsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J8R5Od5rvsKSJk6uXOVYY0tYItqL7C8XTc0L7RdFc20=; b=NFVTxVkwhmg4ScUbNskDGmfYwHBiMQU23SVqtg/0e+LmtYNe07WnBwtQ1jd2hMaZwlR9bx4mtodX6lXA0mS+GIbQRT3mt0CfnUQnapDyFkbiHEjlHvgG6tHGEdz2fEfMwMQgac1Zp+EgoXsVt9fALTG5qMV1Yg36zLIocKaqtjyx5tFIsVXJj3A3IecomIEkqySY1TwFL5ay5dMhX9fvFN6E3OvW8uO8whe3V2PeOvUZP7pSHeugg39Pf9sM//g/koHR3F7Kvo3yiig3A0MK/AUJ/mh2HFXN68CRJomKdvtcWPuk6NdxprvnqLTEdyAqx6U6R+J0X044M/NLgdy3LQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by SA1PR12MB7200.namprd12.prod.outlook.com (2603:10b6:806:2bb::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.25; Mon, 23 Sep 2024 18:34:48 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.7982.022; Mon, 23 Sep 2024 18:34:48 +0000 Date: Mon, 23 Sep 2024 15:34:47 -0300 From: Jason Gunthorpe To: "Tian, Kevin" Cc: Nicolin Chen , "will@kernel.org" , "joro@8bytes.org" , "suravee.suthikulpanit@amd.com" , "robin.murphy@arm.com" , "dwmw2@infradead.org" , "baolu.lu@linux.intel.com" , "shuah@kernel.org" , "linux-kernel@vger.kernel.org" , "iommu@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , "linux-kselftest@vger.kernel.org" , "eric.auger@redhat.com" , "jean-philippe@linaro.org" , "mdf@kernel.org" , "mshavit@google.com" , "shameerali.kolothum.thodi@huawei.com" , "smostafa@google.com" , "Liu, Yi L" Subject: Re: [PATCH v2 17/19] iommu/arm-smmu-v3: Add arm_smmu_viommu_cache_invalidate Message-ID: <20240923183447.GH9417@nvidia.com> References: <20240905162039.GT1358970@nvidia.com> <20240905182148.GA1358970@nvidia.com> <20240911230735.GQ58321@nvidia.com> <20240914145057.GV58321@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL6PEPF00013E10.NAMP222.PROD.OUTLOOK.COM (2603:10b6:22e:400:0:1001:0:15) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|SA1PR12MB7200:EE_ X-MS-Office365-Filtering-Correlation-Id: f561e264-0e23-4958-7366-08dcdbfe6746 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?vc/jdC1cFfJcTlu3FNnzpk6VCgpxwzugMa27nvjUchgFGjQ4hXGVEd+c4ghl?= =?us-ascii?Q?PTj54Lq94LyXMqFUUS8QKT2Jwgl28Zb4bkY4LzfV/UMTAfySz/qY4Hhs/1g9?= =?us-ascii?Q?BBs6tJmMOGTgh4BeeNc4aMy0+Wnua8t/hwgHHQJ27TQXAHEQT8FavRMr49+w?= =?us-ascii?Q?nDJ1113hW6Yxx+4CE8zOMXWcU3i6mvVWVGmzaRm+zxaf9TiCUSi7FExpJiXN?= =?us-ascii?Q?4HA82f4LHHFY++s6Dr/bkbadvCfvJih4/pRw9ZHGABMQVXdXZIMO7U+bnVpc?= =?us-ascii?Q?3ekd0vpluP7wt1cH2PNC2tITVn7/GE8LC8scsIGDM9lCqlfv94DHYNeacN9y?= =?us-ascii?Q?11326pdd0vv6Fstjybi+JyrkQRKHfHGX+TDpsg7UTk17bNygccEHFORCDg92?= =?us-ascii?Q?NCEStKi0wsLKXVgrh4yVV0Zn7uyRwYv6RO9F8uJEvkD9Ncnb+yeKQ8crW2UJ?= =?us-ascii?Q?Cbe0ZYzoRCnKr6imSiJRjOfE80qREZRoyMe7JHopel+lbczMaqEI5aS3j7X4?= =?us-ascii?Q?+o1bOAxd9tVODiwVZJaosL/A3SYQUb3sK+tk2nZkAcpWT9mnoXC1yrGPrSMD?= =?us-ascii?Q?1y6f94cRlBVXVgn/V0AvWRUTXZaxhmCbtWhjlqz715PMMcfU8J7woPOsPbK3?= =?us-ascii?Q?vxYnlu/z6uYx/b0oZa24aShnDBjDrsadLVu/1gF8HIE112JGkQbBJ6nh1DYs?= =?us-ascii?Q?ESO8iLcm4WWiJJdAVNzTf9GMiX0fCeJmcCDrZDptqe3Z7jUXrvaIulea/j5l?= =?us-ascii?Q?OWRx3o2EfJwd6HItIbd2mWUH3zDi76mPG1jRJgm1tW9/0baCgfT8nJ73ljs1?= =?us-ascii?Q?rXA2jyoiNcfi+8NX3HD0kZprsDzXV1cxrPySrBGcM19oDiv6Ft5E7PcVq/2r?= =?us-ascii?Q?yxx7tv2g2a6bCIQHvOnShpCe3Y7RxmVssUprPBdIUpSPeTWb6yGUNZjE+nWQ?= =?us-ascii?Q?RO0HytspgqBCuAjirlQ9+zYav2Lng4IRRg89LT1sJYGLJOuFH1aK+H+q/+NZ?= =?us-ascii?Q?uZgQkC0OVUSHPyCk9fPJiXYis1dFgGBIplZ/9Wr9ZKvBylgGNDMujvx2WTnS?= =?us-ascii?Q?cw3uoMVVayrSkj0AeNdkJBS0c4feSWG/NbpemrgmNq3o/KNfZ6BtbY/KmW68?= =?us-ascii?Q?3Nyt/A04kEsCVJujEQ0lrHgRuR3Aj/UnYkGOgouGbAHBX+ERSQF1Cg7U9VmN?= =?us-ascii?Q?1VIEZOFaOq/lbN4IrVdVo038EpWZUX31TQ/KaF0TLBFtRFrTvlkUo3IWfh4G?= =?us-ascii?Q?0oOiNyvn+roGu3tMAB7ihKC76GHU74PyVemY4FGLHY7cMKnfVFZY59fNibi7?= =?us-ascii?Q?UscUKLwHF4pAyDneRXO/RMPJ/uM1dGOEbwtzsYCj/Kd8cA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?i1GvRv6kdNmdjLjaLi1N2waAfor9/H9RuGdIO6c9htyYMFB5zThUbbBgg8pm?= =?us-ascii?Q?8xqjrsk9WP461dzN5GFB2QB9fAwDwbtXvAsg76kp3Ule9293/vbb5VtQw1dw?= =?us-ascii?Q?p3jSziwcewfFXEqG1Hj/yTFyR4bp4s/JLxGpn0IoUW+VmXADx45NNR9zcSEP?= =?us-ascii?Q?ZQyLk9thI5cy+MHiqHVylJ3l7+t+G+S5uHHyXy01sYrvWXSjzY4jkCNgEg3y?= =?us-ascii?Q?zNNsrdLR5eyHEGaXT8XpS7lJIg4m2NUUWld3Lvo6CPCZWd7F1VC98iGDyeyN?= =?us-ascii?Q?viVDt54ddko+Z8jJbCQqKe/SgI5xRXO7rd7KNSOpt4gyY9xl7M0vqbiM41aT?= =?us-ascii?Q?u0T/ltNzO0ANPzRbXqhlXfX+o0dBPKThCiMSilXSoH933lRerU6K16XbHYMN?= =?us-ascii?Q?o6LigcWrgzgPwrIHHuRBzp69EqAQL5iM0pXL2fwAQPig0pqGKYZVVpLU9lx2?= =?us-ascii?Q?Dn7O7GJsVNz1IPCbh3OZy0mESBPqr3sRfTneH6AsEBDgc/R7XTzuvTifz9wl?= =?us-ascii?Q?N8KByNMNrB1dqFB7ErRKgBnWyUArIMHEGP6K8yV3OlZI13bk7Hps2mJhDX/4?= =?us-ascii?Q?se2Tr/s40QD8u3/MvRB/zchcn2GLY/nD3vt3XVSOBNcLFZZfOAWbvqWdkf6z?= =?us-ascii?Q?LExRXQKWLswqfKzkOQkUNpoMdi9qdCGPSvfSmgsMlHbnLhuSpZhVxsn4eLbh?= =?us-ascii?Q?wYsjIVAM6jRZ4KbXzJvdF2J5APd7d6BN7v7BA/eYk1ZI0sSHg0q4hamFV3Mt?= =?us-ascii?Q?tKUtaVh5PTIU23gfu7ZXWi9m8yqDNqFclxMM1mHGYva0UzMm6izI3h7HW4RO?= =?us-ascii?Q?SR4SXU3wLuCiK/e/U2TjeqQWf8MBYZPhoKrJFtN7Gg7eegzeUAmywZiocFiP?= =?us-ascii?Q?qOwMCvkmk5t2megJNbsVe6liJG88oOJzQ0+Q1xMZYK8uivJpQ1om3mCaKpxW?= =?us-ascii?Q?r8clluBwq0tLQO3BnCa9uLXhkcW4pjnAnZoBG2lJ1OV4RKRXf0PrmCdXMjb/?= =?us-ascii?Q?wQuClZxiwb1FH8KvR9Ewc5zxIL89ocSEJDlgTqkwEfNKAnhGMGKevFJ7jGn3?= =?us-ascii?Q?39d5bHOrJZTfq83lTHIwU3EM6DkE94htwQr2hm+lmambmZWI9Wn3/gOMNQUG?= =?us-ascii?Q?C87ksKv+C5fdyopVQRfT2vxJTefDyw/yyrjgZ6+Fb+XEVuL6Nlw8DYcCAyEb?= =?us-ascii?Q?Ikg6RMC5vqkl7kaJTdG86FiYW3xiPfv5K9bLD+uG+hZIifTb3JUR6EyU25cn?= =?us-ascii?Q?ftIrdpaBbyfPlVc9Kr8p9dSJXn9Ke1VDAGfCFy5yQEwpRGMeioUV0gxiyNh9?= =?us-ascii?Q?t+rcHJUnMTTql2jdM7fjkvZCimyiTMsb5r3OKdluO5ho6Y7NwR+DA7Zt9nLi?= =?us-ascii?Q?cqncQHy1YPLkGAP0k8tHalV68Q3igfB245EEJVHGFVmhuIi2WqHiCPqFnp5j?= =?us-ascii?Q?FGNmgfxUj1U6cEWu+eTKb+gx8G0eTjyoYhTQ7bUpG5X65wy7N0/ubgBR/Cou?= =?us-ascii?Q?DZ1ETKR78ND3wXXcm00dT8pf6prMnuNe01HGwsGsO7/c8FOWiOAFw1ZhaQBd?= =?us-ascii?Q?sptL5On11L6bgvk7o34=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f561e264-0e23-4958-7366-08dcdbfe6746 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2024 18:34:48.2552 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pyeQ+EHmAG2DAYsjuEUpcuNCBs07zjUNDB92krDjHB9VhwECgEt8F/q00fzM+a94 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7200 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240923_113459_502709_205E2EC1 X-CRM114-Status: GOOD ( 23.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Sep 18, 2024 at 08:10:52AM +0000, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Saturday, September 14, 2024 10:51 PM > > > > On Fri, Sep 13, 2024 at 02:33:59AM +0000, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > > Sent: Thursday, September 12, 2024 7:08 AM > > > > > > > > On Wed, Sep 11, 2024 at 08:13:01AM +0000, Tian, Kevin wrote: > > > > > > > > > Probably there is a good reason e.g. for simplification or better > > > > > aligned with hw accel stuff. But it's not explained clearly so far. > > > > > > > > Probably the most concrete thing is if you have a direct assignment > > > > invalidation queue (ie DMA'd directly by HW) then it only applies to a > > > > single pIOMMU and invalidation commands placed there are unavoidably > > > > limited in scope. > > > > > > > > This creates a representation problem, if we have a vIOMMU that spans > > > > many pIOMMUs but invalidations do some subset how to do we model > > > > that. Just saying the vIOMMU is linked to the pIOMMU solves this > > > > nicely. > > > > > > > > > > yes that is a good reason. > > > > > > btw do we expect the VMM to try-and-fail when deciding whether a > > > new vIOMMU object is required when creating a new vdev? > > > > I think there was some suggestion the getinfo could return this, but > > also I think qemu needs to have a command line that matches physical > > so maybe it needs some sysfs? > > > > My impression was that Qemu is moving away from directly accessing > sysfs (e.g. as the reason behind allowing Libvirt to pass in an opened > cdev fd to Qemu). So probably getinfo makes more sense... Yes, but I think libvirt needs this information before it invokes qemu.. The physical and virtual iommus need to sort of match, something should figure this out automatically I would guess. Jason