From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2088.outbound.protection.outlook.com [40.107.220.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20F443C46E for ; Thu, 16 Nov 2023 16:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="V/FP7jLB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n+S3yP/JQx4ytCYBgtYDHJNSJEB3GhSH9I3TjBXsF3pMB2bBkVG/MD9nqhaqaV9l2qNXdfmHRm9IJd+x+n/KXN4754hgsLlU11OFY+vYNOJCn0bkOR6XSWoHbXHRqilW6E7VvZaVKIMgAr6Q/8O0P38YpdtjY+exVg/woRW/gKgmt73/qn33uiuJJNzg9goDLopdENr/tBOpdSLPVxufMt4TaHfkA2h0nvMWvhLg+tjjp0LtNPfuFJrG/qN74KKU23KFhLnrIcWHEpPiz7Scp09+l0XYUUn5yRTrobBDiJVE7syUGVyYMw9ivpbZycthDGpiDL0dN4JaBAfeBv4oRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/f6fE7yjN4xO+VRX6JvQlb6tNVXWfos1ddwKZyf6BEM=; b=grHwFkHqc44JTK3BLHbV/msQTp7QUOue6vTUGDphQo/vcHJtiBx4iEVqePnxzclDPsKRnKoKaTq2L1dIZjd2kR1yQX8Otqc3NKXyPXxBUvMM1BoR9M29Ip4Z78rQXJtsUDcw8ahlu2P14ivPt5le/VbyQfhy5S/WmfF0M1eObljtyfz7m9LDibswHZIZ62zDwr2JKWoVoa+Z5cCU4zLZtWssdfGxdPLS84cHIx9D6hIGT/I+out5ZmlJtfInpj/WaYIHiCAm9r7xSQReqFLS1oR1AM96TJbWtud04YHfGXOwRU2zfxgi1v2GAf0Qqie8Smrs0bcLtL569BAOgHoJwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/f6fE7yjN4xO+VRX6JvQlb6tNVXWfos1ddwKZyf6BEM=; b=V/FP7jLBYVdhmNZzD6VW4orsckjmu5RdFrR8fwujiZ1Nz6qr/irDBDDwi9qZpvIedWtxAORt/5dsIvbmP17V7zOk/EBkefLu+t/z6LxIQ0n09xP8Jsqr4s59eKe4Ol/HCUxOO5zLAfDXkU7u3HDzc9x2SfIDWb+W5pXyLFJbRNQ24V9CBE8mA2OqaO8JKsHFBALvPiLySYBdDAkUU+hLlS0VOk7hxsEiQ/seHal1k1t63UlQIDCleRryXa6q/RZhEYGmc0L/K7teXKqoU7QkV6m/pziWV2dzobkBo5DfFFewIgaOtrEH8qZWhPd6BpaJbcpb+FNdxCjDYlhmzNmIlA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by CH3PR12MB8187.namprd12.prod.outlook.com (2603:10b6:610:125::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.21; Thu, 16 Nov 2023 16:28:10 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93%4]) with mapi id 15.20.7002.021; Thu, 16 Nov 2023 16:28:10 +0000 Date: Thu, 16 Nov 2023 12:28:09 -0400 From: Jason Gunthorpe To: Michael Shavit Cc: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon , Nicolin Chen , Shameerali Kolothum Thodi Subject: Re: [PATCH v2 11/19] iommu/arm-smmu-v3: Do not change the STE twice during arm_smmu_attach_dev() Message-ID: References: <0-v2-de8b10590bf5+400-smmuv3_newapi_p1_jgg@nvidia.com> <11-v2-de8b10590bf5+400-smmuv3_newapi_p1_jgg@nvidia.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: MN2PR01CA0005.prod.exchangelabs.com (2603:10b6:208:10c::18) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|CH3PR12MB8187:EE_ X-MS-Office365-Filtering-Correlation-Id: 52b6776a-3bae-41d5-129d-08dbe6c10585 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xSG1fw+X+zBMH8rFHpoYGzF256kmNzn36YU9Y8yzw03vj0xWG4L9S4IrIeAlRi0n3bIiOade8h41lWtCBJMP67r3L+6efj0fFudviFeOJpmalDS5nqQRD6kEyf1J17TuFuBEKu+E0ISFqRlWeDg0h6K1eoADZZnJulZGNr+rMEpV1VaqagVE29znF/bRUrsde1S2HKkI2KWb6/qpk1j0WAdwFa4mAKHfWPiI+LMOCAMdEbgXATOYZVUTKp868AU0fA7rq6nhwr7RC8sMELp0tr8mNwtE0QAWF4PA/wzm4ahpvcxacW7PDNdcnd/WpuGy4YYd12IQU2pIB3Sht8RXiFv9Ygbss49dbnfNpk9tpCeo2KuYJA7lHGuKcsW7ozCSUMMayQlMpiqv3umrrVXN/zKsj+j5K47SWWGgjF0RJIut1BA64l346NWfyPWJ0PTfrsLx4BTpCG4A5ZIqtDBqXHyZhxER/v+yG5jER7k9BvSiHeGWmVzzU1iJlZONg/sD063/UP35eaLf4DbGcrEwuzSJl0Nu91QRt/CwCicyh15pLe62/O6eNDn2KY1TjEdv X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(376002)(39860400002)(366004)(396003)(346002)(230922051799003)(1800799009)(186009)(64100799003)(451199024)(38100700002)(2906002)(36756003)(86362001)(83380400001)(2616005)(5660300002)(26005)(53546011)(55236004)(6512007)(6506007)(41300700001)(8936002)(478600001)(4326008)(8676002)(6486002)(6916009)(54906003)(66556008)(66476007)(66946007)(316002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dS9XRGxRSW5xSjZxVUYveUgxaEorWVlXa1EzY1VlSVNISEtnUDYxMDZ3dmJv?= =?utf-8?B?NmhaRURibTc0V1hMYWVzTGh1YjVIWkJUZFZqUzZicjJLR3Z6L2R5NUd1Nmxr?= =?utf-8?B?YUYvME5UZUNOb3psK2Zra083YUpWb3JVZTFPVTh5dFhkVTJ2M3VhOXJidzQr?= =?utf-8?B?Zll1OU0zOWNTSUZ2MklsRXU2WW90YlVqUUNDSHNDT3pOMTA4L050VDlieFNv?= =?utf-8?B?Wis0L2Y2bG5sUE9UUG9KQ2NTN0V3ZU9zLzNkZFovM204QXhCTmhPYm1wQ09w?= =?utf-8?B?RDZIRGg2T2FjRGNxL0xaTVRMNzdLUWRBWVFjUU0wRWpUWThBK0pXdVBkUEdX?= =?utf-8?B?clpIeGlZV0l6NXhpU1lKdEo1TlB4MUNGa3dzL08xaVcxUWMrRDZQMER4MURC?= =?utf-8?B?UStJT3V1OEYrcUovem83WW41TnhwOGtsT0ExZ1QyUEcvb0dEMWJrQThkWHBE?= =?utf-8?B?K3BhMlJpL0VUSTZ4WDk5a001emtuQjlnU0hLK0o4alg3SjZtUHJYTVVOYWxJ?= =?utf-8?B?b3pvRTJrMVg5V2xXdlRwSWtJYlRPaWpUOEMxWitRSTR1RmVCRmJ1ZUJuSzBM?= =?utf-8?B?Nkl5eE5NT2J1MjQzbHFBNitGOHlCNjdRN3BsZGEydTJ0eGJ3WUZBaStoWnJO?= =?utf-8?B?MEVYcS8yUFM2TDZoYWdGZjB3OXFscFZoUHYrNXV0K2FuTURNTTEvM2M0MFJV?= =?utf-8?B?RmVtVk13NXdIdTRSNnRjTk1XNU4ySm9tTVpOR3NLaER1Qjd2cVUvOXJWRWUy?= =?utf-8?B?N1RHTVVKTjZTbHhlc2pvTnQrTXl1ZTdOZnYvZDlyaFJCUDh6Z0Q1S0lTS2NO?= =?utf-8?B?T3ZSSjBOQmpJdmF2Ry9SR1RGSitjYWtYMWZCdHdKSFBTZnptdHZwSGUzUU5S?= =?utf-8?B?QzAxdlJqR0NGRVJoVzFsQ2FoTDdCU3RuZE1yc2lIZVFSVFBvRXhhUUtuU3Ix?= =?utf-8?B?QXFYVmdCV29NSnhNVDEvS1AxQ2VrdG1NNEM3aGRFV2dyWHhEV0xmSmdRcFpN?= =?utf-8?B?MEE0bzVMZUkwMytWNjQrbkNscVdWZVdReE9xell0NFpXSkJPZm4rMHVzMldr?= =?utf-8?B?N2paVmxpUlRXR2JuK3J5YkplU2lZbHdYVjNCOFU0eDJqOWdVVlpDSXo3SE1D?= =?utf-8?B?TTUzU2xKdkZjb2pLdk4yWC9zQTVHeXhnOGMwbXhydjI3a2dkbEdIT0pEK0Rk?= =?utf-8?B?UDh1THFyVng2STVqcndSYjNHTklaWFlMTWg3QmU0ZU9PemZ4bXFyQUVvQ3hh?= =?utf-8?B?R1loS2VUd0R3Y0dWY0JCa0ZsRlhvQUtOVkdQSWRPQURUTk5OOFl6a0NobXRx?= =?utf-8?B?OFNtRm9oZThCV3hucGdLMytPVVpkUzZ1aGt2akVrbWFKYkxib3FIU28xWGFL?= =?utf-8?B?dDBTQWlBbHFLMXpXNTVneWJHcG1uTDhNV2VNRU93TTcycytqRVN2L3NTQVZQ?= =?utf-8?B?REZ6NFV6OW9xODkzdllhV2YzWm8zQ2pmN0xHQmI3WGZLR0VoNHoxMGY1cFpF?= =?utf-8?B?WHArWDE5Mks1cmNYMjYwbGxpSi8xWWFDcysyQmdjN090K2wzV2VVTzIva0Ix?= =?utf-8?B?NWtwTTJJejRkVW9QSVJOSmlHbURwZ0RuRHNkVCtmYStVZlNMcUFIcUd6a25L?= =?utf-8?B?MTJLbnNVN01PNTMrUG1ZckJrdTFab3FXdVNJZWRvK2dOUDY3SFdyQ0h4ZnJk?= =?utf-8?B?eFlRVzdnS25kN2NCMnZHeEFMMVVOVkNpaGNBRzlOeE1aNEh5VXY1Z0NWVS9P?= =?utf-8?B?VzNyeHo2Z3lGTWlyc25ra3hvQW51enBrRTlVOXFjd3REaVoxRWRmcmxtd2cw?= =?utf-8?B?Zk44VFpRaktvVlZrajhFeXFzaDV0ZmNYSHZYZlZXcFlVemNmNk5yeUhZcElB?= =?utf-8?B?dkRJZ3JndUVDamJNYkN1TDBSMitlL1B3eUU4T3JQTHdoUkNjaGpUaFgva3Rp?= =?utf-8?B?ZTV5WDZ5SnZPUytSQjhRRE9pbzlRT00zQndaVUIxWnlWVjdkRVZBc3BGbkxy?= =?utf-8?B?aEgvYndMaGRvZXFnTC90cndBQ2RmK3FHTFU2ditVbHNwSGQ5V0xwd2FYUkxC?= =?utf-8?B?SVhSNTFoLzFEaXBuQkM0c2U5Z1lXSXRZbEZCd21vVWxIVEdYb2cwU0FSOUVK?= =?utf-8?Q?8YS96pfYJ278yeI2WyJ/LmPf2?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 52b6776a-3bae-41d5-129d-08dbe6c10585 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Nov 2023 16:28:09.9549 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QDIf2AekaoQQYqC4mXhVWX9ikSfZrSgt7sZF1m8mNbQaLU//h6p2zoIgcSAAic1R X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8187 On Wed, Nov 15, 2023 at 11:15:23PM +0800, Michael Shavit wrote: > On Tue, Nov 14, 2023 at 1:53 AM Jason Gunthorpe wrote: > > > > This was needed because the STE code required the STE to be in > > ABORT/BYPASS inorder to program a cdtable or S2 STE. Now that the STE code > > can automatically handle all transitions we can remove this step > > from the attach_dev flow. > > > > A few small bugs exist because of this: > > > > 1) If the core code does BLOCKED -> UNMANAGED with disable_bypass=false > > then there will be a moment where the STE points at BYPASS. Since > > this can be done by VFIO/IOMMUFD it is a small security race. > > > > 2) If the core code does IDENTITY -> DMA then any IOMMU_RESV_DIRECT > > regions will temporarily become BLOCKED. We'd like drivers to > > work in a way that allows IOMMU_RESV_DIRECT to be continuously > > functional during these transitions. > > > > Make arm_smmu_release_device() put the STE back to the correct > > ABORT/BYPASS setting. Fix a bug where a IOMMU_RESV_DIRECT was ignored on > > this path. > > > > Notice this subtly depends on the prior arm_smmu_asid_lock change as the > > STE must be put to non-paging before removing the device for the linked > > list to avoid races with arm_smmu_share_asid(). > > I'm a little confused by this comment. Is this suggesting that > arm_smmu_detach_dev had a race condition before the arm_smmu_asid_lock > changes, since it deletes the list entry before deactivating the STE > that uses the domain and without grabbing the asid_lock, thus allowing > a gap where the ASID might be re-acquired by an SVA domain while an > STE with that ASID is still live on this device? Wouldn't that belong > on the asid_lock patch instead if so? I wasn't intending to say there is an existing bug, this was more to point out why it was organized like this, and why it is OK to remove the detach manipulation of the STE considering races with share_asid. However, I agree that the code in rc1 is troubled and fixed in the prior patch: spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->domain_head); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); ^^^^ Prevents arm_smmu_update_ctx_desc_devices() from storing to the STE However the STE is still pointing at the ASID master->domain = NULL; master->ats_enabled = false; arm_smmu_install_ste_for_dev(master); ^^^^ Now the STE is gone, so the CD becomes unreferenced if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && master->cd_table.cdtab) arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); ^^^^ Now the CD is non-valid I was primarily concerned with corrupting the CD, ie that share_asid would race and un-clear the write_ctx_desc(). That is prevented by the ordering above. However, I agree the above is still problematic because there is a short time window where the ASID can be installed in two CDs with two different translations. I suppose there is a security issue where this could corrupt the IOTLB. This is all fixed in this series too by having more robust locking. So this does deserve a note in the commit message for the earlier patch about this issue. > > @@ -2852,9 +2846,18 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) > > static void arm_smmu_release_device(struct device *dev) > > { > > struct arm_smmu_master *master = dev_iommu_priv_get(dev); > > + struct arm_smmu_ste target; > > > > if (WARN_ON(arm_smmu_master_sva_enabled(master))) > > iopf_queue_remove_device(master->smmu->evtq.iopf, dev); > > + > > + /* Put the STE back to what arm_smmu_init_strtab() sets */ > > Hmmmm, it seems like checking iommu->require_direct may put STEs in > bypass in scenarios where arm_smmu_init_strtab() wouldn't have. > arm_smmu_init_strtab is calling iort_get_rmr_sids to pick streams to > put into bypass, but IIUC iommu->require_direct also applies to > dts-based reserved-memory regions, not just iort. Indeed, that actually looks like a little bug as the DT should technicaly be the same behavior as the iort.. I'm going to ignore it :) > I'm not very familiar with the history behind disable_bypass; why is > putting an entire stream into bypass the correct behavior if a > reserved-memory (which may be for a small finite region) exists? This specific reserved memory region is requesting a 1:1 translation for a chunk of IOVA. This translation is being used by some agent outside Linux's knowledge and the desire is for the translation to always be in effect. So, if we put the STE to ABORT then the translation will stop working with unknown side effects. This is also why we install the translation in the DMA domain and block use of VIFO if these are set - to ensure the 1:1 translation is always there. Thanks, Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ADDCCC197A0 for ; Thu, 16 Nov 2023 16:28:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XgKrZV/K0+fxrYNLkRaLPhFIGh4wZu/lijyBkZaLl3A=; b=4vpbLygd0qXHUo mZAR0getL5hxUhfAksLGLUR8bXA4Ag2VN2RLkmTLuOIHdvzEnJLWVhxsia0mC0i0LRS4tiO7UdU4U gG/B13Wl5iirfi6mt/gHV8W4sgyfmGtabmNNjrA+3q4DQy1W1HDOOvZ0t8QsnPNOAMM57uFY4heaj gpJksk8pS/biU6kXHS1uOFhY4vLWzxrISkL7FDX+BacXZohSpYaecTcJKwGNpd1tb21NUqXSqIp3i pWYXzfG1HBF37Li7leiu/t90q37mfpI46y2GOcfozShLm5Qt1cwnO5Dzy7qUGomeUy8OStquPx1ON pAzlNbml2AU/GPUhdioQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r3fE7-003xmV-1l; Thu, 16 Nov 2023 16:28:23 +0000 Received: from mail-co1nam11on20621.outbound.protection.outlook.com ([2a01:111:f400:7eab::621] helo=NAM11-CO1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r3fE4-003xl7-2o for linux-arm-kernel@lists.infradead.org; Thu, 16 Nov 2023 16:28:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n+S3yP/JQx4ytCYBgtYDHJNSJEB3GhSH9I3TjBXsF3pMB2bBkVG/MD9nqhaqaV9l2qNXdfmHRm9IJd+x+n/KXN4754hgsLlU11OFY+vYNOJCn0bkOR6XSWoHbXHRqilW6E7VvZaVKIMgAr6Q/8O0P38YpdtjY+exVg/woRW/gKgmt73/qn33uiuJJNzg9goDLopdENr/tBOpdSLPVxufMt4TaHfkA2h0nvMWvhLg+tjjp0LtNPfuFJrG/qN74KKU23KFhLnrIcWHEpPiz7Scp09+l0XYUUn5yRTrobBDiJVE7syUGVyYMw9ivpbZycthDGpiDL0dN4JaBAfeBv4oRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/f6fE7yjN4xO+VRX6JvQlb6tNVXWfos1ddwKZyf6BEM=; b=grHwFkHqc44JTK3BLHbV/msQTp7QUOue6vTUGDphQo/vcHJtiBx4iEVqePnxzclDPsKRnKoKaTq2L1dIZjd2kR1yQX8Otqc3NKXyPXxBUvMM1BoR9M29Ip4Z78rQXJtsUDcw8ahlu2P14ivPt5le/VbyQfhy5S/WmfF0M1eObljtyfz7m9LDibswHZIZ62zDwr2JKWoVoa+Z5cCU4zLZtWssdfGxdPLS84cHIx9D6hIGT/I+out5ZmlJtfInpj/WaYIHiCAm9r7xSQReqFLS1oR1AM96TJbWtud04YHfGXOwRU2zfxgi1v2GAf0Qqie8Smrs0bcLtL569BAOgHoJwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/f6fE7yjN4xO+VRX6JvQlb6tNVXWfos1ddwKZyf6BEM=; b=V/FP7jLBYVdhmNZzD6VW4orsckjmu5RdFrR8fwujiZ1Nz6qr/irDBDDwi9qZpvIedWtxAORt/5dsIvbmP17V7zOk/EBkefLu+t/z6LxIQ0n09xP8Jsqr4s59eKe4Ol/HCUxOO5zLAfDXkU7u3HDzc9x2SfIDWb+W5pXyLFJbRNQ24V9CBE8mA2OqaO8JKsHFBALvPiLySYBdDAkUU+hLlS0VOk7hxsEiQ/seHal1k1t63UlQIDCleRryXa6q/RZhEYGmc0L/K7teXKqoU7QkV6m/pziWV2dzobkBo5DfFFewIgaOtrEH8qZWhPd6BpaJbcpb+FNdxCjDYlhmzNmIlA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by CH3PR12MB8187.namprd12.prod.outlook.com (2603:10b6:610:125::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.21; Thu, 16 Nov 2023 16:28:10 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93%4]) with mapi id 15.20.7002.021; Thu, 16 Nov 2023 16:28:10 +0000 Date: Thu, 16 Nov 2023 12:28:09 -0400 From: Jason Gunthorpe To: Michael Shavit Cc: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon , Nicolin Chen , Shameerali Kolothum Thodi Subject: Re: [PATCH v2 11/19] iommu/arm-smmu-v3: Do not change the STE twice during arm_smmu_attach_dev() Message-ID: References: <0-v2-de8b10590bf5+400-smmuv3_newapi_p1_jgg@nvidia.com> <11-v2-de8b10590bf5+400-smmuv3_newapi_p1_jgg@nvidia.com> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MN2PR01CA0005.prod.exchangelabs.com (2603:10b6:208:10c::18) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|CH3PR12MB8187:EE_ X-MS-Office365-Filtering-Correlation-Id: 52b6776a-3bae-41d5-129d-08dbe6c10585 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xSG1fw+X+zBMH8rFHpoYGzF256kmNzn36YU9Y8yzw03vj0xWG4L9S4IrIeAlRi0n3bIiOade8h41lWtCBJMP67r3L+6efj0fFudviFeOJpmalDS5nqQRD6kEyf1J17TuFuBEKu+E0ISFqRlWeDg0h6K1eoADZZnJulZGNr+rMEpV1VaqagVE29znF/bRUrsde1S2HKkI2KWb6/qpk1j0WAdwFa4mAKHfWPiI+LMOCAMdEbgXATOYZVUTKp868AU0fA7rq6nhwr7RC8sMELp0tr8mNwtE0QAWF4PA/wzm4ahpvcxacW7PDNdcnd/WpuGy4YYd12IQU2pIB3Sht8RXiFv9Ygbss49dbnfNpk9tpCeo2KuYJA7lHGuKcsW7ozCSUMMayQlMpiqv3umrrVXN/zKsj+j5K47SWWGgjF0RJIut1BA64l346NWfyPWJ0PTfrsLx4BTpCG4A5ZIqtDBqXHyZhxER/v+yG5jER7k9BvSiHeGWmVzzU1iJlZONg/sD063/UP35eaLf4DbGcrEwuzSJl0Nu91QRt/CwCicyh15pLe62/O6eNDn2KY1TjEdv X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(376002)(39860400002)(366004)(396003)(346002)(230922051799003)(1800799009)(186009)(64100799003)(451199024)(38100700002)(2906002)(36756003)(86362001)(83380400001)(2616005)(5660300002)(26005)(53546011)(55236004)(6512007)(6506007)(41300700001)(8936002)(478600001)(4326008)(8676002)(6486002)(6916009)(54906003)(66556008)(66476007)(66946007)(316002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dS9XRGxRSW5xSjZxVUYveUgxaEorWVlXa1EzY1VlSVNISEtnUDYxMDZ3dmJv?= =?utf-8?B?NmhaRURibTc0V1hMYWVzTGh1YjVIWkJUZFZqUzZicjJLR3Z6L2R5NUd1Nmxr?= =?utf-8?B?YUYvME5UZUNOb3psK2Zra083YUpWb3JVZTFPVTh5dFhkVTJ2M3VhOXJidzQr?= =?utf-8?B?Zll1OU0zOWNTSUZ2MklsRXU2WW90YlVqUUNDSHNDT3pOMTA4L050VDlieFNv?= =?utf-8?B?Wis0L2Y2bG5sUE9UUG9KQ2NTN0V3ZU9zLzNkZFovM204QXhCTmhPYm1wQ09w?= =?utf-8?B?RDZIRGg2T2FjRGNxL0xaTVRMNzdLUWRBWVFjUU0wRWpUWThBK0pXdVBkUEdX?= =?utf-8?B?clpIeGlZV0l6NXhpU1lKdEo1TlB4MUNGa3dzL08xaVcxUWMrRDZQMER4MURC?= =?utf-8?B?UStJT3V1OEYrcUovem83WW41TnhwOGtsT0ExZ1QyUEcvb0dEMWJrQThkWHBE?= =?utf-8?B?K3BhMlJpL0VUSTZ4WDk5a001emtuQjlnU0hLK0o4alg3SjZtUHJYTVVOYWxJ?= =?utf-8?B?b3pvRTJrMVg5V2xXdlRwSWtJYlRPaWpUOEMxWitRSTR1RmVCRmJ1ZUJuSzBM?= =?utf-8?B?Nkl5eE5NT2J1MjQzbHFBNitGOHlCNjdRN3BsZGEydTJ0eGJ3WUZBaStoWnJO?= =?utf-8?B?MEVYcS8yUFM2TDZoYWdGZjB3OXFscFZoUHYrNXV0K2FuTURNTTEvM2M0MFJV?= =?utf-8?B?RmVtVk13NXdIdTRSNnRjTk1XNU4ySm9tTVpOR3NLaER1Qjd2cVUvOXJWRWUy?= =?utf-8?B?N1RHTVVKTjZTbHhlc2pvTnQrTXl1ZTdOZnYvZDlyaFJCUDh6Z0Q1S0lTS2NO?= =?utf-8?B?T3ZSSjBOQmpJdmF2Ry9SR1RGSitjYWtYMWZCdHdKSFBTZnptdHZwSGUzUU5S?= =?utf-8?B?QzAxdlJqR0NGRVJoVzFsQ2FoTDdCU3RuZE1yc2lIZVFSVFBvRXhhUUtuU3Ix?= =?utf-8?B?QXFYVmdCV29NSnhNVDEvS1AxQ2VrdG1NNEM3aGRFV2dyWHhEV0xmSmdRcFpN?= =?utf-8?B?MEE0bzVMZUkwMytWNjQrbkNscVdWZVdReE9xell0NFpXSkJPZm4rMHVzMldr?= =?utf-8?B?N2paVmxpUlRXR2JuK3J5YkplU2lZbHdYVjNCOFU0eDJqOWdVVlpDSXo3SE1D?= =?utf-8?B?TTUzU2xKdkZjb2pLdk4yWC9zQTVHeXhnOGMwbXhydjI3a2dkbEdIT0pEK0Rk?= =?utf-8?B?UDh1THFyVng2STVqcndSYjNHTklaWFlMTWg3QmU0ZU9PemZ4bXFyQUVvQ3hh?= =?utf-8?B?R1loS2VUd0R3Y0dWY0JCa0ZsRlhvQUtOVkdQSWRPQURUTk5OOFl6a0NobXRx?= =?utf-8?B?OFNtRm9oZThCV3hucGdLMytPVVpkUzZ1aGt2akVrbWFKYkxib3FIU28xWGFL?= =?utf-8?B?dDBTQWlBbHFLMXpXNTVneWJHcG1uTDhNV2VNRU93TTcycytqRVN2L3NTQVZQ?= =?utf-8?B?REZ6NFV6OW9xODkzdllhV2YzWm8zQ2pmN0xHQmI3WGZLR0VoNHoxMGY1cFpF?= =?utf-8?B?WHArWDE5Mks1cmNYMjYwbGxpSi8xWWFDcysyQmdjN090K2wzV2VVTzIva0Ix?= =?utf-8?B?NWtwTTJJejRkVW9QSVJOSmlHbURwZ0RuRHNkVCtmYStVZlNMcUFIcUd6a25L?= =?utf-8?B?MTJLbnNVN01PNTMrUG1ZckJrdTFab3FXdVNJZWRvK2dOUDY3SFdyQ0h4ZnJk?= =?utf-8?B?eFlRVzdnS25kN2NCMnZHeEFMMVVOVkNpaGNBRzlOeE1aNEh5VXY1Z0NWVS9P?= =?utf-8?B?VzNyeHo2Z3lGTWlyc25ra3hvQW51enBrRTlVOXFjd3REaVoxRWRmcmxtd2cw?= =?utf-8?B?Zk44VFpRaktvVlZrajhFeXFzaDV0ZmNYSHZYZlZXcFlVemNmNk5yeUhZcElB?= =?utf-8?B?dkRJZ3JndUVDamJNYkN1TDBSMitlL1B3eUU4T3JQTHdoUkNjaGpUaFgva3Rp?= =?utf-8?B?ZTV5WDZ5SnZPUytSQjhRRE9pbzlRT00zQndaVUIxWnlWVjdkRVZBc3BGbkxy?= =?utf-8?B?aEgvYndMaGRvZXFnTC90cndBQ2RmK3FHTFU2ditVbHNwSGQ5V0xwd2FYUkxC?= =?utf-8?B?SVhSNTFoLzFEaXBuQkM0c2U5Z1lXSXRZbEZCd21vVWxIVEdYb2cwU0FSOUVK?= =?utf-8?Q?8YS96pfYJ278yeI2WyJ/LmPf2?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 52b6776a-3bae-41d5-129d-08dbe6c10585 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Nov 2023 16:28:09.9549 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QDIf2AekaoQQYqC4mXhVWX9ikSfZrSgt7sZF1m8mNbQaLU//h6p2zoIgcSAAic1R X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8187 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231116_082820_911672_2BCD8BBE X-CRM114-Status: GOOD ( 41.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBOb3YgMTUsIDIwMjMgYXQgMTE6MTU6MjNQTSArMDgwMCwgTWljaGFlbCBTaGF2aXQg d3JvdGU6Cj4gT24gVHVlLCBOb3YgMTQsIDIwMjMgYXQgMTo1M+KAr0FNIEphc29uIEd1bnRob3Jw ZSA8amdnQG52aWRpYS5jb20+IHdyb3RlOgo+ID4KPiA+IFRoaXMgd2FzIG5lZWRlZCBiZWNhdXNl IHRoZSBTVEUgY29kZSByZXF1aXJlZCB0aGUgU1RFIHRvIGJlIGluCj4gPiBBQk9SVC9CWVBBU1Mg aW5vcmRlciB0byBwcm9ncmFtIGEgY2R0YWJsZSBvciBTMiBTVEUuIE5vdyB0aGF0IHRoZSBTVEUg Y29kZQo+ID4gY2FuIGF1dG9tYXRpY2FsbHkgaGFuZGxlIGFsbCB0cmFuc2l0aW9ucyB3ZSBjYW4g cmVtb3ZlIHRoaXMgc3RlcAo+ID4gZnJvbSB0aGUgYXR0YWNoX2RldiBmbG93Lgo+ID4KPiA+IEEg ZmV3IHNtYWxsIGJ1Z3MgZXhpc3QgYmVjYXVzZSBvZiB0aGlzOgo+ID4KPiA+IDEpIElmIHRoZSBj b3JlIGNvZGUgZG9lcyBCTE9DS0VEIC0+IFVOTUFOQUdFRCB3aXRoIGRpc2FibGVfYnlwYXNzPWZh bHNlCj4gPiAgICB0aGVuIHRoZXJlIHdpbGwgYmUgYSBtb21lbnQgd2hlcmUgdGhlIFNURSBwb2lu dHMgYXQgQllQQVNTLiBTaW5jZQo+ID4gICAgdGhpcyBjYW4gYmUgZG9uZSBieSBWRklPL0lPTU1V RkQgaXQgaXMgYSBzbWFsbCBzZWN1cml0eSByYWNlLgo+ID4KPiA+IDIpIElmIHRoZSBjb3JlIGNv ZGUgZG9lcyBJREVOVElUWSAtPiBETUEgdGhlbiBhbnkgSU9NTVVfUkVTVl9ESVJFQ1QKPiA+ICAg IHJlZ2lvbnMgd2lsbCB0ZW1wb3JhcmlseSBiZWNvbWUgQkxPQ0tFRC4gV2UnZCBsaWtlIGRyaXZl cnMgdG8KPiA+ICAgIHdvcmsgaW4gYSB3YXkgdGhhdCBhbGxvd3MgSU9NTVVfUkVTVl9ESVJFQ1Qg dG8gYmUgY29udGludW91c2x5Cj4gPiAgICBmdW5jdGlvbmFsIGR1cmluZyB0aGVzZSB0cmFuc2l0 aW9ucy4KPiA+Cj4gPiBNYWtlIGFybV9zbW11X3JlbGVhc2VfZGV2aWNlKCkgcHV0IHRoZSBTVEUg YmFjayB0byB0aGUgY29ycmVjdAo+ID4gQUJPUlQvQllQQVNTIHNldHRpbmcuIEZpeCBhIGJ1ZyB3 aGVyZSBhIElPTU1VX1JFU1ZfRElSRUNUIHdhcyBpZ25vcmVkIG9uCj4gPiB0aGlzIHBhdGguCj4g Pgo+ID4gTm90aWNlIHRoaXMgc3VidGx5IGRlcGVuZHMgb24gdGhlIHByaW9yIGFybV9zbW11X2Fz aWRfbG9jayBjaGFuZ2UgYXMgdGhlCj4gPiBTVEUgbXVzdCBiZSBwdXQgdG8gbm9uLXBhZ2luZyBi ZWZvcmUgcmVtb3ZpbmcgdGhlIGRldmljZSBmb3IgdGhlIGxpbmtlZAo+ID4gbGlzdCB0byBhdm9p ZCByYWNlcyB3aXRoIGFybV9zbW11X3NoYXJlX2FzaWQoKS4KPiAKPiBJJ20gYSBsaXR0bGUgY29u ZnVzZWQgYnkgdGhpcyBjb21tZW50LiBJcyB0aGlzIHN1Z2dlc3RpbmcgdGhhdAo+IGFybV9zbW11 X2RldGFjaF9kZXYgaGFkIGEgcmFjZSBjb25kaXRpb24gYmVmb3JlIHRoZSBhcm1fc21tdV9hc2lk X2xvY2sKPiBjaGFuZ2VzLCBzaW5jZSBpdCBkZWxldGVzIHRoZSBsaXN0IGVudHJ5IGJlZm9yZSBk ZWFjdGl2YXRpbmcgdGhlIFNURQo+IHRoYXQgdXNlcyB0aGUgZG9tYWluIGFuZCB3aXRob3V0IGdy YWJiaW5nIHRoZSBhc2lkX2xvY2ssIHRodXMgYWxsb3dpbmcKPiBhIGdhcCB3aGVyZSB0aGUgQVNJ RCBtaWdodCBiZSByZS1hY3F1aXJlZCBieSBhbiBTVkEgZG9tYWluIHdoaWxlIGFuCj4gU1RFIHdp dGggdGhhdCBBU0lEIGlzIHN0aWxsIGxpdmUgb24gdGhpcyBkZXZpY2U/IFdvdWxkbid0IHRoYXQg YmVsb25nCj4gb24gdGhlIGFzaWRfbG9jayBwYXRjaCBpbnN0ZWFkIGlmIHNvPwoKSSB3YXNuJ3Qg aW50ZW5kaW5nIHRvIHNheSB0aGVyZSBpcyBhbiBleGlzdGluZyBidWcsIHRoaXMgd2FzIG1vcmUg dG8KcG9pbnQgb3V0IHdoeSBpdCB3YXMgb3JnYW5pemVkIGxpa2UgdGhpcywgYW5kIHdoeSBpdCBp cyBPSyB0byByZW1vdmUKdGhlIGRldGFjaCBtYW5pcHVsYXRpb24gb2YgdGhlIFNURSBjb25zaWRl cmluZyByYWNlcyB3aXRoIHNoYXJlX2FzaWQuCgpIb3dldmVyLCBJIGFncmVlIHRoYXQgdGhlIGNv ZGUgaW4gcmMxIGlzIHRyb3VibGVkIGFuZCBmaXhlZCBpbiB0aGUKcHJpb3IgcGF0Y2g6CgoJc3Bp bl9sb2NrX2lycXNhdmUoJnNtbXVfZG9tYWluLT5kZXZpY2VzX2xvY2ssIGZsYWdzKTsKCWxpc3Rf ZGVsKCZtYXN0ZXItPmRvbWFpbl9oZWFkKTsKCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJnNtbXVf ZG9tYWluLT5kZXZpY2VzX2xvY2ssIGZsYWdzKTsKCl5eXl4gUHJldmVudHMgYXJtX3NtbXVfdXBk YXRlX2N0eF9kZXNjX2RldmljZXMoKSBmcm9tIHN0b3JpbmcgdG8gdGhlIFNURQogICAgIEhvd2V2 ZXIgdGhlIFNURSBpcyBzdGlsbCBwb2ludGluZyBhdCB0aGUgQVNJRAoKCW1hc3Rlci0+ZG9tYWlu ID0gTlVMTDsKCW1hc3Rlci0+YXRzX2VuYWJsZWQgPSBmYWxzZTsKCWFybV9zbW11X2luc3RhbGxf c3RlX2Zvcl9kZXYobWFzdGVyKTsKCl5eXl4gTm93IHRoZSBTVEUgaXMgZ29uZSwgc28gdGhlIENE IGJlY29tZXMgdW5yZWZlcmVuY2VkCgoJaWYgKHNtbXVfZG9tYWluLT5zdGFnZSA9PSBBUk1fU01N VV9ET01BSU5fUzEgJiYgbWFzdGVyLT5jZF90YWJsZS5jZHRhYikKCQlhcm1fc21tdV93cml0ZV9j dHhfZGVzYyhtYXN0ZXIsIElPTU1VX05PX1BBU0lELCBOVUxMKTsKCl5eXl4gTm93IHRoZSBDRCBp cyBub24tdmFsaWQKCkkgd2FzIHByaW1hcmlseSBjb25jZXJuZWQgd2l0aCBjb3JydXB0aW5nIHRo ZSBDRCwgaWUgdGhhdCBzaGFyZV9hc2lkCndvdWxkIHJhY2UgYW5kIHVuLWNsZWFyIHRoZSB3cml0 ZV9jdHhfZGVzYygpLiBUaGF0IGlzIHByZXZlbnRlZCBieSB0aGUKb3JkZXJpbmcgYWJvdmUuCgpI b3dldmVyLCBJIGFncmVlIHRoZSBhYm92ZSBpcyBzdGlsbCBwcm9ibGVtYXRpYyBiZWNhdXNlIHRo ZXJlIGlzIGEKc2hvcnQgdGltZSB3aW5kb3cgd2hlcmUgdGhlIEFTSUQgY2FuIGJlIGluc3RhbGxl ZCBpbiB0d28gQ0RzIHdpdGggdHdvCmRpZmZlcmVudCB0cmFuc2xhdGlvbnMuIEkgc3VwcG9zZSB0 aGVyZSBpcyBhIHNlY3VyaXR5IGlzc3VlIHdoZXJlIHRoaXMKY291bGQgY29ycnVwdCB0aGUgSU9U TEIuCgpUaGlzIGlzIGFsbCBmaXhlZCBpbiB0aGlzIHNlcmllcyB0b28gYnkgaGF2aW5nIG1vcmUg cm9idXN0IGxvY2tpbmcuIFNvCnRoaXMgZG9lcyBkZXNlcnZlIGEgbm90ZSBpbiB0aGUgY29tbWl0 IG1lc3NhZ2UgZm9yIHRoZSBlYXJsaWVyIHBhdGNoCmFib3V0IHRoaXMgaXNzdWUuCgo+ID4gQEAg LTI4NTIsOSArMjg0NiwxOCBAQCBzdGF0aWMgc3RydWN0IGlvbW11X2RldmljZSAqYXJtX3NtbXVf cHJvYmVfZGV2aWNlKHN0cnVjdCBkZXZpY2UgKmRldikKPiA+ICBzdGF0aWMgdm9pZCBhcm1fc21t dV9yZWxlYXNlX2RldmljZShzdHJ1Y3QgZGV2aWNlICpkZXYpCj4gPiAgewo+ID4gICAgICAgICBz dHJ1Y3QgYXJtX3NtbXVfbWFzdGVyICptYXN0ZXIgPSBkZXZfaW9tbXVfcHJpdl9nZXQoZGV2KTsK PiA+ICsgICAgICAgc3RydWN0IGFybV9zbW11X3N0ZSB0YXJnZXQ7Cj4gPgo+ID4gICAgICAgICBp ZiAoV0FSTl9PTihhcm1fc21tdV9tYXN0ZXJfc3ZhX2VuYWJsZWQobWFzdGVyKSkpCj4gPiAgICAg ICAgICAgICAgICAgaW9wZl9xdWV1ZV9yZW1vdmVfZGV2aWNlKG1hc3Rlci0+c21tdS0+ZXZ0cS5p b3BmLCBkZXYpOwo+ID4gKwo+ID4gKyAgICAgICAvKiBQdXQgdGhlIFNURSBiYWNrIHRvIHdoYXQg YXJtX3NtbXVfaW5pdF9zdHJ0YWIoKSBzZXRzICovCj4gCj4gSG1tbW0sIGl0IHNlZW1zIGxpa2Ug Y2hlY2tpbmcgaW9tbXUtPnJlcXVpcmVfZGlyZWN0IG1heSBwdXQgU1RFcyBpbgo+IGJ5cGFzcyBp biBzY2VuYXJpb3Mgd2hlcmUgYXJtX3NtbXVfaW5pdF9zdHJ0YWIoKSB3b3VsZG4ndCBoYXZlLgo+ IGFybV9zbW11X2luaXRfc3RydGFiIGlzIGNhbGxpbmcgaW9ydF9nZXRfcm1yX3NpZHMgdG8gcGlj ayBzdHJlYW1zIHRvCj4gcHV0IGludG8gYnlwYXNzLCBidXQgSUlVQyBpb21tdS0+cmVxdWlyZV9k aXJlY3QgYWxzbyBhcHBsaWVzIHRvCj4gZHRzLWJhc2VkIHJlc2VydmVkLW1lbW9yeSByZWdpb25z LCBub3QganVzdCBpb3J0LgoKSW5kZWVkLCB0aGF0IGFjdHVhbGx5IGxvb2tzIGxpa2UgYSBsaXR0 bGUgYnVnIGFzIHRoZSBEVCBzaG91bGQKdGVjaG5pY2FseSBiZSB0aGUgc2FtZSBiZWhhdmlvciBh cyB0aGUgaW9ydC4uIEknbSBnb2luZyB0byBpZ25vcmUgaXQKOikKCj4gSSdtIG5vdCB2ZXJ5IGZh bWlsaWFyIHdpdGggdGhlIGhpc3RvcnkgYmVoaW5kIGRpc2FibGVfYnlwYXNzOyB3aHkgaXMKPiBw dXR0aW5nIGFuIGVudGlyZSBzdHJlYW0gaW50byBieXBhc3MgdGhlIGNvcnJlY3QgYmVoYXZpb3Ig aWYgYQo+IHJlc2VydmVkLW1lbW9yeSAod2hpY2ggbWF5IGJlIGZvciBhIHNtYWxsIGZpbml0ZSBy ZWdpb24pIGV4aXN0cz8KClRoaXMgc3BlY2lmaWMgcmVzZXJ2ZWQgbWVtb3J5IHJlZ2lvbiBpcyBy ZXF1ZXN0aW5nIGEgMToxIHRyYW5zbGF0aW9uCmZvciBhIGNodW5rIG9mIElPVkEuIFRoaXMgdHJh bnNsYXRpb24gaXMgYmVpbmcgdXNlZCBieSBzb21lIGFnZW50Cm91dHNpZGUgTGludXgncyBrbm93 bGVkZ2UgYW5kIHRoZSBkZXNpcmUgaXMgZm9yIHRoZSB0cmFuc2xhdGlvbiB0bwphbHdheXMgYmUg aW4gZWZmZWN0LgoKU28sIGlmIHdlIHB1dCB0aGUgU1RFIHRvIEFCT1JUIHRoZW4gdGhlIHRyYW5z bGF0aW9uIHdpbGwgc3RvcCB3b3JraW5nCndpdGggdW5rbm93biBzaWRlIGVmZmVjdHMuCgpUaGlz IGlzIGFsc28gd2h5IHdlIGluc3RhbGwgdGhlIHRyYW5zbGF0aW9uIGluIHRoZSBETUEgZG9tYWlu IGFuZApibG9jayB1c2Ugb2YgVklGTyBpZiB0aGVzZSBhcmUgc2V0IC0gdG8gZW5zdXJlIHRoZSAx OjEgdHJhbnNsYXRpb24gaXMKYWx3YXlzIHRoZXJlLgoKVGhhbmtzLApKYXNvbgoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBt YWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9s aXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=