From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2049.outbound.protection.outlook.com [40.107.96.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F5C72FB3 for ; Thu, 13 Jul 2023 16:41:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gd/o5A2HFvVCNY95uDaWu4vdDwiGDWV8Vfh3DD87BwcJxM7DF9hLkVc6ZcBqdrjv7JVp/ZPE5Cjc+OwkathI8SUKNyAKYzOxK5ogPuC1xfies835NY8gTbI6mZhNbx8C6ndGar7uPiNY6drG/G8AqimKLl6LyOWCmNV3d+fdtUhlj8Eh4EAfZsOa+K9ZO/2lNn25AucIJXSy+OQmR3Td0RxRCoNAR+aKYxmlVUQmuNgH1S0gUH8KdSNsM+TIzDzMl1jlD1wezmdPZHlLdEwHWzzfWO5N/KTtm4ehH0Uv1ec3Ve4OyfAiolPtqEZ9LZxnf+FsUpjPHEvzxJYEL9jA2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wAX1gAMd6RtSGwZVDkd8gTl8FDOGfMZiT73c0ebwRbA=; b=FDLw9g+734FXLJzifyt/URgd8lZYU72D+svjMaZqytH11Ui1lxG4RsWNP3DR4hOfu0aalhxkFqpmR9w3Ymsq2R/5bpeYu44+Bbqx3yrdLMUh/Jwcg6GBuTTfo4mNu4s5Y76J5oqObR/CGK8iZ1fwgLRUiTEjNiRsg/QreWHHEwgZPO+j49t65/WbW3Tt/h07KE3vAO9jvBxbLS4pmtuZD4SUw0FDSeOeKFOTME38k0vtdhhyPrFYwmXU/RHFQtLrz8R5w6LoyjYyAVBgpOr3WmOZdXv0N2+1Eef1ELThVObXknHrj4oR3oJ3zBxTZaHo6XTjrypE1tmGq6u//GI6yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wAX1gAMd6RtSGwZVDkd8gTl8FDOGfMZiT73c0ebwRbA=; b=aSkgw1sCoIuREou++6EUh8cnlhpH9kvzOB9h7kMZr8BIWZqGtxckaYGkq/oGJM04OLGNP2rZsD1BL2Q2JtGBBe+/MhUtwtJ2Rx92tFgO43Aoou/5SUkGd2LcusDgGI1OLt1xIzt3gTDBCUo28HSxnNX8aFumYqZQKpE9Vuoj8D/NJLu3MYbRw3FiBSuxEuFOetSURrv6gLpbEggbuETOXGuyPfkdV2uQub4sVuK1CtMnDunx/s7BMw5OB+GMHJPdRy0LzVcyMlaEF+9iQrt0C5nyl+rXASG8uWSYjdAZMgytqHyBj5QV7+p2AGhiS7O4xdXn6wVua9ke18eljyP7lg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SJ1PR12MB6290.namprd12.prod.outlook.com (2603:10b6:a03:457::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.20; Thu, 13 Jul 2023 16:41:40 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::69c1:5d87:c73c:cc55]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::69c1:5d87:c73c:cc55%4]) with mapi id 15.20.6565.028; Thu, 13 Jul 2023 16:41:39 +0000 Date: Thu, 13 Jul 2023 13:41:38 -0300 From: Jason Gunthorpe To: Michael Shavit Cc: Nicolin Chen , Will Deacon , Robin Murphy , Joerg Roedel , jean-philippe@linaro.org, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v4 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master Message-ID: References: <20230621063825.268890-1-mshavit@google.com> <20230621063825.268890-3-mshavit@google.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: YT4PR01CA0008.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:d1::16) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SJ1PR12MB6290:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e484b36-0d94-4abe-3457-08db83c00819 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lPBNLRPTzBAMQ6y8uAEeHbNcuu1v7YOUZs0dvH1IqUMqKLmVorZIpQ2BraZ1lbt6IFsqMZ9k+9FpjsTGf+3ikc3dh7YDirYg9gh3Xx9jXeDrh3fML96PfAsRR9BUGTG3Q+fxhKHkpzrDRTOVL8NRJ8Txs1dEw71/8Nxxcv8KSQPOu9DsH935KUJWawEB2lkbivBwNTrJimEhLSyCmtMP7GC2FCcTXPFe2WAoVUfUSkCTStVpZjtBLdwjRwzcuXQ3LBoM/Qo1CS/kF4FpiSZajXxuaDAW0aFwYcTlPsu8LFJezTdIi5MJpE6IDCnH0jhPi1acx6+A1jqAQQxZJPbaLW665FKl/Q7KDcp2ugmfLnTbdAzsbZhXLKXYMOImAa2kVwwrtaCI2o+X1v3nCjeUOKT2WSeY3//ZI8+3rvRJRkbqySfGGXpi6UbsbvRJCkGgng65NqThEPXYB3Aui0zH1jfSKSmZC9DYXXt3ivc/FgJhEzun9LZwYskIKHKwa7lwFdlIjM6ZexH1WM5HCCEFBs7QzHxOXqfKLADtm0OJYALY1xiBcEKy2vh3S+LIQLnU X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(366004)(136003)(451199021)(4326008)(6916009)(66476007)(66946007)(66556008)(316002)(2906002)(41300700001)(478600001)(8676002)(8936002)(5660300002)(54906003)(6486002)(6512007)(6506007)(53546011)(36756003)(186003)(38100700002)(26005)(86362001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bHlvOXdORk41VTMwQmIwNDhPaUZIRUlVUVJQczVjWHpVVk5IMWc0WVZlRS8v?= =?utf-8?B?S0NGR3FlcFY3elVzZ2xibURxSEJmU2gxWFhSKzBpYzUrQjRNT0I1NHVQQXNj?= =?utf-8?B?d2dBbVE5RUVNVlJsc1lxeXRFeHdlWHlIeUZnV3Y2UDdFVmNvVG1NRHlIaFBC?= =?utf-8?B?MEltWFMzb2MybHVmRmkxMysrTXdIaDVDd0s3MTM2S1g5RTRjYzlYZ1hFMmRq?= =?utf-8?B?cEFJNC85YVk4a0tYVUt6TzRNcDh5NlgveHpILzlYQmd0ejU1TGR4RlUyaENH?= =?utf-8?B?R0xoRXl0dkh6c1dId0d5cFVRYm1GZkFhRDE5T3Z2S0JSWlk1bTBBRmxBOVZ4?= =?utf-8?B?M1VsSUxOdWNBL01ud0NQU1Fhb1NjTG5OUXNib1ovYjNRL3FRcy9BbCtraUtv?= =?utf-8?B?UXZFc2gzaXlpb0pkVFRoaDY3SlBLN0owU0RPemM2ZXpnY3JHMWF4S3IyWTN3?= =?utf-8?B?VUdwWEcrTTVmM1JrTlpLcFBpb1BoazJBbk16eWJhcGFsUVpBV0t2dys5dEYx?= =?utf-8?B?Nzg5aW1iR2FUeXdMYk03SjVuL2F5SHk3RVJoeWQ2bTlSZDFQa2E3eTFLbGV3?= =?utf-8?B?ckRNRmExMUZIMDU1L3NITmZKcjRscDV6VlVlNjBGMmNHRURYNExXdG81SzR5?= =?utf-8?B?b3pXYm44YlVtTDByeWJZK25rcThDenlmaEhER3puekF2eWVZRXp3RnN3MVRP?= =?utf-8?B?bEd6VUZ1RnpqSWwzWisraGhvVDZHL2tPRUY4Zm5sdVY0M0t0ZTFJeHp5d2lw?= =?utf-8?B?czk4R2RCM3hkaDk1eTN0a0NxNkxjNjh5L0lyTjdVZHI0Rkt6bW8wVThqdjdL?= =?utf-8?B?bFFPZ2lKNGJYUEEwQXFGbTN0TEFsZDZxTXc5VFZZMjYxdHFJK2U2Z1lhZWJF?= =?utf-8?B?OW9NeGI5N0cyU1phTlBuT1gxVVUxbEx3aGljWURiNkplSGxlMGZIVE9zZ2xn?= =?utf-8?B?M092c2hlWnpoN3JVNEJtS0ZoaWVHcnZYc2dhNTM4Yk9YN3dQaS9ickhVdG5B?= =?utf-8?B?cUppc1VKQUhUMjVyN1pkY1N6c2xIU0lIQk5tZVJQQ2ZpcEFTbEs4RDNkSHJ6?= =?utf-8?B?aTkwK09LYWVNbEtHNWxrcGMxTGpXcVk4QVpOMlpwdEZwSlZJdkRCQlB1Zzlp?= =?utf-8?B?bG5qOUlzS0J2bFhCUThydXNoTzduZEErdThvcy9rckxLbWlKL0lYcDM4T1Rz?= =?utf-8?B?d2lDNDNVeFlCRGd2dnZ1Q3NLenJQLzhQTHY1VWdoVXBBMTBGQXpBL0xBNi83?= =?utf-8?B?NkVuSVZCSHlmbVo5WGZUTTNhQ0FpM2VqRjhXVFc3VGZ4QjhRZ3htRy9BTDhF?= =?utf-8?B?WExOczYyYnRZK3JzMGl1dkhqVlRNL1Zvdm9Id0FjWFN5SGszVEZHdkJZelNC?= =?utf-8?B?YTZ0OWtsT3BZNUlhV1ZTVnBvY2UzeGRVc2VQRDBndC9IcnJwNVgvS3hZVDU3?= =?utf-8?B?YWxiejNJd2EzQ1RmQTF0NXUrSnVESTF1NWJvZFJuY25FMTMzTWUzV2JDQm9G?= =?utf-8?B?OUdpKzFoYy8yNGt4Y3VxRkl4OTJ6d2dRSFB1NHFBelJPVWtpUWlGZzlrUGVB?= =?utf-8?B?TUdGRWVZZ3k1S1pJdExnK3YzZWpQUnZ1dmN4SGYvUFBDM0dGTzhDcVRiSHJw?= =?utf-8?B?eTRRRnJXcW5jVzVreWNzSGhIT3BpVTlFT2s2Mkk4ZGIwQ0g3UHNYZzJ6d0g1?= =?utf-8?B?WS8yZ2sxUmJQU2t0eTNqa2dMcXUxVEYrb3VNc0ZQMng4aUpmME5LTEJWNlRH?= =?utf-8?B?bWdZRXUvN2g1QlBmck5LY1RRNzdWQVZHekV6ajJTYXdLazVvRDdTQkR3OG9K?= =?utf-8?B?eDNRdmNNRElJd0RiaUFoMmdVZ2Evc3dZM1krVlNZeVdpTDkwd2NPTUsxdndx?= =?utf-8?B?UnU5MVdUdGV4Sk00TnZ4SklDUnh1ZWo2K0xtOC9OeWFtNWV1dzRVRHI5MnRI?= =?utf-8?B?Q2NoUFJBNE1kS3luNHVydEpxQVI3VEpMbkJHNzJkcFJiSGFHTWNmUGVNNWZ0?= =?utf-8?B?bTgyakM0MXBtblBVODRQZjBuVW5SNGlLTitzM1RKOFRCL0k5ZTRhTUljRFp2?= =?utf-8?B?QUpnVjdTRklOb3hBdnpiV05XUjJDcjVpZEkyOFl3QytKcTEvTGMwZXhrbWo0?= =?utf-8?Q?i0aKty0utyfmzbqgIfeE6HHKy?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e484b36-0d94-4abe-3457-08db83c00819 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jul 2023 16:41:39.7851 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pUPDa1J+zO2vbCsebidCUcH1psHBqq9XXpWaQ1zRz56tAYucQv96hfzj0i1cwZ78 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6290 On Fri, Jul 14, 2023 at 12:16:16AM +0800, Michael Shavit wrote: > On Thu, Jul 13, 2023 at 10:29 PM Jason Gunthorpe wrote: > > It would make alot more sense if the STE value used by an unmanaged S1 > > domain was located in/near the unmanaged domain or called 'unmanaged > > S1 STE' or something if it really has to be in the master. Why does > > this even need to be stored, can't we compute it? > > struct s1_cfg* and struct s2_cfg* are precisely what is used to > compute an STE. For example, when s1_cfg is set, arm_smmu_write_strtab > will write the s1_cfg's CD table dma_pointer into the STE's > STRTAB_STE_0_CFG field. When neither are set, the STE fields are > written to enable bypass (or abort depending on the config). I guess I never really understood why these were precomputed and stored at all. Even more confusing is why we need to keep pointers to them anywhere? Compute the STE and CDE directly from the source data when you need it? eg If I want to install an IDENITY domain into a STE then I compute the STE for identity and go ahead and do it. > > I'd think the basic mental model should be to extract the STE from the > > thing you intend to install. Either the default CD table, or from the > > iommu_domain. ie some 'get STE from iommu_domain' function? > > I don't follow this. When we attach a domain with pasid (whether > through SVA or the set_dev_pasid API) , we don't want to install an > entirely new CD table. The master object owns an optional CD table. If it is exsists it is used by every domain that is attached to that master. In the code flow there are two entry points to attach a domain, attach to a PASID or attach to a RID. For attach to PASID the code should always force the master to have a CD table and then attach the domain to the CD table. For attach to RID the code should do a bunch of checks and decide if it should force the master to have a CD table and attach the domain to that, or directly attach the domain to the STE. When the master gains a CD table then the CD table object becomes attached to the STE. In all cases we should be able to point to the object the STE points at and don't need a cfg or pointer to cfg since the object itself can provide the cfg. In all cases when you go to compute a STE you figure out what object is attached to it (CD or domain), compute the correct STE for that object, the set it. Same for he CDE, query the correct CDE from the iommu_domain when you attach it to the table. There should be no such thing as a "default" STE, and I question if it makes sense to even precompute the s1/s2_cfg values during finalize at all.. > We want to write something (page-table pointer) to a common CD > table. Where should the s1_cfg which owns that common table live? I would suggest a 'cd table struct' that as all the stuff related to the CD table, including an API to cacluate the STE this CD table requires. If not in actual code with a real struct, then in a logical sense in that a chunk of the master struct is the "CD table". > I thought we concluded that it should be owned by the > arm_smmu_master rather than any domain (to avoid dependencies > between domains a-la aux-domain). Yes, I'm not saying anything against that, just how and where the STE and CDE values flow around. Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89EBBC001DE for ; Thu, 13 Jul 2023 16:42:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VHxJR43Hi6TNm6RAV1coW43ssGQAfRxxuSgb84lwPMg=; b=lewfsjm5NCpxb0 ciSHDgajD1fkf3sK/nlSwBktf3YC/NNwNgfQCQH7QOZjmq29qecSBGNEapWtRLumMxPiJKdmQgrff tc6pdw36Au8A8uhcJVmOLKDZeOCGnIuZYuA5cAUpuoGeSBKVAhD1JHIZluda/pcDP2PscHg6NYb+L 4MmpsnsTrJBlcKfMSTQokvH0vgSBBjq1sTqqmaoC1IB9WTq58FBkTGqZVHPeajXfGxETpwNIhtKx5 Dk7fnwmmuNy4EQuCaULpGQ+/MFl4TzGYEuyZ3wSCJOtLIdKCfPbxA3xnXH3pCMhlIbwdp2WPb/reR HD/umMX1ZyGbK/HKNYtg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qJzNz-003vbF-0s; Thu, 13 Jul 2023 16:41:47 +0000 Received: from mail-dm6nam10on20609.outbound.protection.outlook.com ([2a01:111:f400:7e88::609] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qJzNw-003vaZ-18 for linux-arm-kernel@lists.infradead.org; Thu, 13 Jul 2023 16:41:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gd/o5A2HFvVCNY95uDaWu4vdDwiGDWV8Vfh3DD87BwcJxM7DF9hLkVc6ZcBqdrjv7JVp/ZPE5Cjc+OwkathI8SUKNyAKYzOxK5ogPuC1xfies835NY8gTbI6mZhNbx8C6ndGar7uPiNY6drG/G8AqimKLl6LyOWCmNV3d+fdtUhlj8Eh4EAfZsOa+K9ZO/2lNn25AucIJXSy+OQmR3Td0RxRCoNAR+aKYxmlVUQmuNgH1S0gUH8KdSNsM+TIzDzMl1jlD1wezmdPZHlLdEwHWzzfWO5N/KTtm4ehH0Uv1ec3Ve4OyfAiolPtqEZ9LZxnf+FsUpjPHEvzxJYEL9jA2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wAX1gAMd6RtSGwZVDkd8gTl8FDOGfMZiT73c0ebwRbA=; b=FDLw9g+734FXLJzifyt/URgd8lZYU72D+svjMaZqytH11Ui1lxG4RsWNP3DR4hOfu0aalhxkFqpmR9w3Ymsq2R/5bpeYu44+Bbqx3yrdLMUh/Jwcg6GBuTTfo4mNu4s5Y76J5oqObR/CGK8iZ1fwgLRUiTEjNiRsg/QreWHHEwgZPO+j49t65/WbW3Tt/h07KE3vAO9jvBxbLS4pmtuZD4SUw0FDSeOeKFOTME38k0vtdhhyPrFYwmXU/RHFQtLrz8R5w6LoyjYyAVBgpOr3WmOZdXv0N2+1Eef1ELThVObXknHrj4oR3oJ3zBxTZaHo6XTjrypE1tmGq6u//GI6yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wAX1gAMd6RtSGwZVDkd8gTl8FDOGfMZiT73c0ebwRbA=; b=aSkgw1sCoIuREou++6EUh8cnlhpH9kvzOB9h7kMZr8BIWZqGtxckaYGkq/oGJM04OLGNP2rZsD1BL2Q2JtGBBe+/MhUtwtJ2Rx92tFgO43Aoou/5SUkGd2LcusDgGI1OLt1xIzt3gTDBCUo28HSxnNX8aFumYqZQKpE9Vuoj8D/NJLu3MYbRw3FiBSuxEuFOetSURrv6gLpbEggbuETOXGuyPfkdV2uQub4sVuK1CtMnDunx/s7BMw5OB+GMHJPdRy0LzVcyMlaEF+9iQrt0C5nyl+rXASG8uWSYjdAZMgytqHyBj5QV7+p2AGhiS7O4xdXn6wVua9ke18eljyP7lg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SJ1PR12MB6290.namprd12.prod.outlook.com (2603:10b6:a03:457::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.20; Thu, 13 Jul 2023 16:41:40 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::69c1:5d87:c73c:cc55]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::69c1:5d87:c73c:cc55%4]) with mapi id 15.20.6565.028; Thu, 13 Jul 2023 16:41:39 +0000 Date: Thu, 13 Jul 2023 13:41:38 -0300 From: Jason Gunthorpe To: Michael Shavit Cc: Nicolin Chen , Will Deacon , Robin Murphy , Joerg Roedel , jean-philippe@linaro.org, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v4 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master Message-ID: References: <20230621063825.268890-1-mshavit@google.com> <20230621063825.268890-3-mshavit@google.com> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: YT4PR01CA0008.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:d1::16) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SJ1PR12MB6290:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e484b36-0d94-4abe-3457-08db83c00819 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lPBNLRPTzBAMQ6y8uAEeHbNcuu1v7YOUZs0dvH1IqUMqKLmVorZIpQ2BraZ1lbt6IFsqMZ9k+9FpjsTGf+3ikc3dh7YDirYg9gh3Xx9jXeDrh3fML96PfAsRR9BUGTG3Q+fxhKHkpzrDRTOVL8NRJ8Txs1dEw71/8Nxxcv8KSQPOu9DsH935KUJWawEB2lkbivBwNTrJimEhLSyCmtMP7GC2FCcTXPFe2WAoVUfUSkCTStVpZjtBLdwjRwzcuXQ3LBoM/Qo1CS/kF4FpiSZajXxuaDAW0aFwYcTlPsu8LFJezTdIi5MJpE6IDCnH0jhPi1acx6+A1jqAQQxZJPbaLW665FKl/Q7KDcp2ugmfLnTbdAzsbZhXLKXYMOImAa2kVwwrtaCI2o+X1v3nCjeUOKT2WSeY3//ZI8+3rvRJRkbqySfGGXpi6UbsbvRJCkGgng65NqThEPXYB3Aui0zH1jfSKSmZC9DYXXt3ivc/FgJhEzun9LZwYskIKHKwa7lwFdlIjM6ZexH1WM5HCCEFBs7QzHxOXqfKLADtm0OJYALY1xiBcEKy2vh3S+LIQLnU X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(366004)(136003)(451199021)(4326008)(6916009)(66476007)(66946007)(66556008)(316002)(2906002)(41300700001)(478600001)(8676002)(8936002)(5660300002)(54906003)(6486002)(6512007)(6506007)(53546011)(36756003)(186003)(38100700002)(26005)(86362001)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bHlvOXdORk41VTMwQmIwNDhPaUZIRUlVUVJQczVjWHpVVk5IMWc0WVZlRS8v?= =?utf-8?B?S0NGR3FlcFY3elVzZ2xibURxSEJmU2gxWFhSKzBpYzUrQjRNT0I1NHVQQXNj?= =?utf-8?B?d2dBbVE5RUVNVlJsc1lxeXRFeHdlWHlIeUZnV3Y2UDdFVmNvVG1NRHlIaFBC?= =?utf-8?B?MEltWFMzb2MybHVmRmkxMysrTXdIaDVDd0s3MTM2S1g5RTRjYzlYZ1hFMmRq?= =?utf-8?B?cEFJNC85YVk4a0tYVUt6TzRNcDh5NlgveHpILzlYQmd0ejU1TGR4RlUyaENH?= =?utf-8?B?R0xoRXl0dkh6c1dId0d5cFVRYm1GZkFhRDE5T3Z2S0JSWlk1bTBBRmxBOVZ4?= =?utf-8?B?M1VsSUxOdWNBL01ud0NQU1Fhb1NjTG5OUXNib1ovYjNRL3FRcy9BbCtraUtv?= =?utf-8?B?UXZFc2gzaXlpb0pkVFRoaDY3SlBLN0owU0RPemM2ZXpnY3JHMWF4S3IyWTN3?= =?utf-8?B?VUdwWEcrTTVmM1JrTlpLcFBpb1BoazJBbk16eWJhcGFsUVpBV0t2dys5dEYx?= =?utf-8?B?Nzg5aW1iR2FUeXdMYk03SjVuL2F5SHk3RVJoeWQ2bTlSZDFQa2E3eTFLbGV3?= =?utf-8?B?ckRNRmExMUZIMDU1L3NITmZKcjRscDV6VlVlNjBGMmNHRURYNExXdG81SzR5?= =?utf-8?B?b3pXYm44YlVtTDByeWJZK25rcThDenlmaEhER3puekF2eWVZRXp3RnN3MVRP?= =?utf-8?B?bEd6VUZ1RnpqSWwzWisraGhvVDZHL2tPRUY4Zm5sdVY0M0t0ZTFJeHp5d2lw?= =?utf-8?B?czk4R2RCM3hkaDk1eTN0a0NxNkxjNjh5L0lyTjdVZHI0Rkt6bW8wVThqdjdL?= =?utf-8?B?bFFPZ2lKNGJYUEEwQXFGbTN0TEFsZDZxTXc5VFZZMjYxdHFJK2U2Z1lhZWJF?= =?utf-8?B?OW9NeGI5N0cyU1phTlBuT1gxVVUxbEx3aGljWURiNkplSGxlMGZIVE9zZ2xn?= =?utf-8?B?M092c2hlWnpoN3JVNEJtS0ZoaWVHcnZYc2dhNTM4Yk9YN3dQaS9ickhVdG5B?= =?utf-8?B?cUppc1VKQUhUMjVyN1pkY1N6c2xIU0lIQk5tZVJQQ2ZpcEFTbEs4RDNkSHJ6?= =?utf-8?B?aTkwK09LYWVNbEtHNWxrcGMxTGpXcVk4QVpOMlpwdEZwSlZJdkRCQlB1Zzlp?= =?utf-8?B?bG5qOUlzS0J2bFhCUThydXNoTzduZEErdThvcy9rckxLbWlKL0lYcDM4T1Rz?= =?utf-8?B?d2lDNDNVeFlCRGd2dnZ1Q3NLenJQLzhQTHY1VWdoVXBBMTBGQXpBL0xBNi83?= =?utf-8?B?NkVuSVZCSHlmbVo5WGZUTTNhQ0FpM2VqRjhXVFc3VGZ4QjhRZ3htRy9BTDhF?= =?utf-8?B?WExOczYyYnRZK3JzMGl1dkhqVlRNL1Zvdm9Id0FjWFN5SGszVEZHdkJZelNC?= =?utf-8?B?YTZ0OWtsT3BZNUlhV1ZTVnBvY2UzeGRVc2VQRDBndC9IcnJwNVgvS3hZVDU3?= =?utf-8?B?YWxiejNJd2EzQ1RmQTF0NXUrSnVESTF1NWJvZFJuY25FMTMzTWUzV2JDQm9G?= =?utf-8?B?OUdpKzFoYy8yNGt4Y3VxRkl4OTJ6d2dRSFB1NHFBelJPVWtpUWlGZzlrUGVB?= =?utf-8?B?TUdGRWVZZ3k1S1pJdExnK3YzZWpQUnZ1dmN4SGYvUFBDM0dGTzhDcVRiSHJw?= =?utf-8?B?eTRRRnJXcW5jVzVreWNzSGhIT3BpVTlFT2s2Mkk4ZGIwQ0g3UHNYZzJ6d0g1?= =?utf-8?B?WS8yZ2sxUmJQU2t0eTNqa2dMcXUxVEYrb3VNc0ZQMng4aUpmME5LTEJWNlRH?= =?utf-8?B?bWdZRXUvN2g1QlBmck5LY1RRNzdWQVZHekV6ajJTYXdLazVvRDdTQkR3OG9K?= =?utf-8?B?eDNRdmNNRElJd0RiaUFoMmdVZ2Evc3dZM1krVlNZeVdpTDkwd2NPTUsxdndx?= =?utf-8?B?UnU5MVdUdGV4Sk00TnZ4SklDUnh1ZWo2K0xtOC9OeWFtNWV1dzRVRHI5MnRI?= =?utf-8?B?Q2NoUFJBNE1kS3luNHVydEpxQVI3VEpMbkJHNzJkcFJiSGFHTWNmUGVNNWZ0?= =?utf-8?B?bTgyakM0MXBtblBVODRQZjBuVW5SNGlLTitzM1RKOFRCL0k5ZTRhTUljRFp2?= =?utf-8?B?QUpnVjdTRklOb3hBdnpiV05XUjJDcjVpZEkyOFl3QytKcTEvTGMwZXhrbWo0?= =?utf-8?Q?i0aKty0utyfmzbqgIfeE6HHKy?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e484b36-0d94-4abe-3457-08db83c00819 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jul 2023 16:41:39.7851 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pUPDa1J+zO2vbCsebidCUcH1psHBqq9XXpWaQ1zRz56tAYucQv96hfzj0i1cwZ78 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6290 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230713_094144_395792_2ADB5B19 X-CRM114-Status: GOOD ( 30.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCBKdWwgMTQsIDIwMjMgYXQgMTI6MTY6MTZBTSArMDgwMCwgTWljaGFlbCBTaGF2aXQg d3JvdGU6Cj4gT24gVGh1LCBKdWwgMTMsIDIwMjMgYXQgMTA6MjnigK9QTSBKYXNvbiBHdW50aG9y cGUgPGpnZ0BudmlkaWEuY29tPiB3cm90ZToKPiA+IEl0IHdvdWxkIG1ha2UgYWxvdCBtb3JlIHNl bnNlIGlmIHRoZSBTVEUgdmFsdWUgdXNlZCBieSBhbiB1bm1hbmFnZWQgUzEKPiA+IGRvbWFpbiB3 YXMgbG9jYXRlZCBpbi9uZWFyIHRoZSB1bm1hbmFnZWQgZG9tYWluIG9yIGNhbGxlZCAndW5tYW5h Z2VkCj4gPiBTMSBTVEUnIG9yIHNvbWV0aGluZyBpZiBpdCByZWFsbHkgaGFzIHRvIGJlIGluIHRo ZSBtYXN0ZXIuIFdoeSBkb2VzCj4gPiB0aGlzIGV2ZW4gbmVlZCB0byBiZSBzdG9yZWQsIGNhbid0 IHdlIGNvbXB1dGUgaXQ/Cj4gCj4gc3RydWN0IHMxX2NmZyogYW5kIHN0cnVjdCBzMl9jZmcqIGFy ZSBwcmVjaXNlbHkgd2hhdCBpcyB1c2VkIHRvCj4gY29tcHV0ZSBhbiBTVEUuIEZvciBleGFtcGxl LCB3aGVuIHMxX2NmZyBpcyBzZXQsIGFybV9zbW11X3dyaXRlX3N0cnRhYgo+IHdpbGwgd3JpdGUg dGhlIHMxX2NmZydzIENEIHRhYmxlIGRtYV9wb2ludGVyIGludG8gdGhlIFNURSdzCj4gU1RSVEFC X1NURV8wX0NGRyBmaWVsZC4gV2hlbiBuZWl0aGVyIGFyZSBzZXQsIHRoZSBTVEUgZmllbGRzIGFy ZQo+IHdyaXR0ZW4gdG8gZW5hYmxlIGJ5cGFzcyAob3IgYWJvcnQgZGVwZW5kaW5nIG9uIHRoZSBj b25maWcpLgoKSSBndWVzcyBJIG5ldmVyIHJlYWxseSB1bmRlcnN0b29kIHdoeSB0aGVzZSB3ZXJl IHByZWNvbXB1dGVkIGFuZApzdG9yZWQgYXQgYWxsLiBFdmVuIG1vcmUgY29uZnVzaW5nIGlzIHdo eSB3ZSBuZWVkIHRvIGtlZXAgcG9pbnRlcnMgdG8KdGhlbSBhbnl3aGVyZT8gQ29tcHV0ZSB0aGUg U1RFIGFuZCBDREUgZGlyZWN0bHkgZnJvbSB0aGUgc291cmNlIGRhdGEKd2hlbiB5b3UgbmVlZCBp dD8KCmVnIElmIEkgd2FudCB0byBpbnN0YWxsIGFuIElERU5JVFkgZG9tYWluIGludG8gYSBTVEUg dGhlbiBJIGNvbXB1dGUKdGhlIFNURSBmb3IgaWRlbnRpdHkgYW5kIGdvIGFoZWFkIGFuZCBkbyBp dC4KCj4gPiBJJ2QgdGhpbmsgdGhlIGJhc2ljIG1lbnRhbCBtb2RlbCBzaG91bGQgYmUgdG8gZXh0 cmFjdCB0aGUgU1RFIGZyb20gdGhlCj4gPiB0aGluZyB5b3UgaW50ZW5kIHRvIGluc3RhbGwuIEVp dGhlciB0aGUgZGVmYXVsdCBDRCB0YWJsZSwgb3IgZnJvbSB0aGUKPiA+IGlvbW11X2RvbWFpbi4g aWUgc29tZSAnZ2V0IFNURSBmcm9tIGlvbW11X2RvbWFpbicgZnVuY3Rpb24/Cj4gCj4gSSBkb24n dCBmb2xsb3cgdGhpcy4gV2hlbiB3ZSBhdHRhY2ggYSBkb21haW4gd2l0aCBwYXNpZCAod2hldGhl cgo+IHRocm91Z2ggU1ZBIG9yIHRoZSBzZXRfZGV2X3Bhc2lkIEFQSSkgLCB3ZSBkb24ndCB3YW50 IHRvIGluc3RhbGwgYW4KPiBlbnRpcmVseSBuZXcgQ0QgdGFibGUuIAoKVGhlIG1hc3RlciBvYmpl Y3Qgb3ducyBhbiBvcHRpb25hbCBDRCB0YWJsZS4gSWYgaXQgaXMgZXhzaXN0cyBpdCBpcwp1c2Vk IGJ5IGV2ZXJ5IGRvbWFpbiB0aGF0IGlzIGF0dGFjaGVkIHRvIHRoYXQgbWFzdGVyLgoKSW4gdGhl IGNvZGUgZmxvdyB0aGVyZSBhcmUgdHdvIGVudHJ5IHBvaW50cyB0byBhdHRhY2ggYSBkb21haW4s IGF0dGFjaAp0byBhIFBBU0lEIG9yIGF0dGFjaCB0byBhIFJJRC4KCkZvciBhdHRhY2ggdG8gUEFT SUQgdGhlIGNvZGUgc2hvdWxkIGFsd2F5cyBmb3JjZSB0aGUgbWFzdGVyIHRvIGhhdmUgYQpDRCB0 YWJsZSBhbmQgdGhlbiBhdHRhY2ggdGhlIGRvbWFpbiB0byB0aGUgQ0QgdGFibGUuCgpGb3IgYXR0 YWNoIHRvIFJJRCB0aGUgY29kZSBzaG91bGQgZG8gYSBidW5jaCBvZiBjaGVja3MgYW5kIGRlY2lk ZSBpZgppdCBzaG91bGQgZm9yY2UgdGhlIG1hc3RlciB0byBoYXZlIGEgQ0QgdGFibGUgYW5kIGF0 dGFjaCB0aGUgZG9tYWluIHRvCnRoYXQsIG9yIGRpcmVjdGx5IGF0dGFjaCB0aGUgZG9tYWluIHRv IHRoZSBTVEUuCgpXaGVuIHRoZSBtYXN0ZXIgZ2FpbnMgYSBDRCB0YWJsZSB0aGVuIHRoZSBDRCB0 YWJsZSBvYmplY3QgYmVjb21lcwphdHRhY2hlZCB0byB0aGUgU1RFLiBJbiBhbGwgY2FzZXMgd2Ug c2hvdWxkIGJlIGFibGUgdG8gcG9pbnQgdG8gdGhlCm9iamVjdCB0aGUgU1RFIHBvaW50cyBhdCBh bmQgZG9uJ3QgbmVlZCBhIGNmZyBvciBwb2ludGVyIHRvIGNmZyBzaW5jZQp0aGUgb2JqZWN0IGl0 c2VsZiBjYW4gcHJvdmlkZSB0aGUgY2ZnLgoKSW4gYWxsIGNhc2VzIHdoZW4geW91IGdvIHRvIGNv bXB1dGUgYSBTVEUgeW91IGZpZ3VyZSBvdXQgd2hhdCBvYmplY3QKaXMgYXR0YWNoZWQgdG8gaXQg KENEIG9yIGRvbWFpbiksIGNvbXB1dGUgdGhlIGNvcnJlY3QgU1RFIGZvciB0aGF0Cm9iamVjdCwg dGhlIHNldCBpdC4gU2FtZSBmb3IgaGUgQ0RFLCBxdWVyeSB0aGUgY29ycmVjdCBDREUgZnJvbSB0 aGUKaW9tbXVfZG9tYWluIHdoZW4geW91IGF0dGFjaCBpdCB0byB0aGUgdGFibGUuCgpUaGVyZSBz aG91bGQgYmUgbm8gc3VjaCB0aGluZyBhcyBhICJkZWZhdWx0IiBTVEUsIGFuZCBJIHF1ZXN0aW9u IGlmIGl0Cm1ha2VzIHNlbnNlIHRvIGV2ZW4gcHJlY29tcHV0ZSB0aGUgczEvczJfY2ZnIHZhbHVl cyBkdXJpbmcgZmluYWxpemUgYXQKYWxsLi4KCj4gV2Ugd2FudCB0byB3cml0ZSBzb21ldGhpbmcg KHBhZ2UtdGFibGUgcG9pbnRlcikgdG8gYSBjb21tb24gQ0QKPiB0YWJsZS4gV2hlcmUgc2hvdWxk IHRoZSBzMV9jZmcgd2hpY2ggb3ducyB0aGF0IGNvbW1vbiB0YWJsZSBsaXZlPyAKCkkgd291bGQg c3VnZ2VzdCBhICdjZCB0YWJsZSBzdHJ1Y3QnIHRoYXQgYXMgYWxsIHRoZSBzdHVmZiByZWxhdGVk IHRvCnRoZSBDRCB0YWJsZSwgaW5jbHVkaW5nIGFuIEFQSSB0byBjYWNsdWF0ZSB0aGUgU1RFIHRo aXMgQ0QgdGFibGUKcmVxdWlyZXMuIElmIG5vdCBpbiBhY3R1YWwgY29kZSB3aXRoIGEgcmVhbCBz dHJ1Y3QsIHRoZW4gaW4gYSBsb2dpY2FsCnNlbnNlIGluIHRoYXQgYSBjaHVuayBvZiB0aGUgbWFz dGVyIHN0cnVjdCBpcyB0aGUgIkNEIHRhYmxlIi4KCj4gSSB0aG91Z2h0IHdlIGNvbmNsdWRlZCB0 aGF0IGl0IHNob3VsZCBiZSBvd25lZCBieSB0aGUKPiBhcm1fc21tdV9tYXN0ZXIgcmF0aGVyIHRo YW4gYW55IGRvbWFpbiAodG8gYXZvaWQgZGVwZW5kZW5jaWVzCj4gYmV0d2VlbiBkb21haW5zIGEt bGEgYXV4LWRvbWFpbikuIAoKWWVzLCBJJ20gbm90IHNheWluZyBhbnl0aGluZyBhZ2FpbnN0IHRo YXQsIGp1c3QgaG93IGFuZCB3aGVyZSB0aGUgU1RFCmFuZCBDREUgdmFsdWVzIGZsb3cgYXJvdW5k LgoKSmFzb24KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5m cmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xp bnV4LWFybS1rZXJuZWwK