From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2047.outbound.protection.outlook.com [40.107.96.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B81E86FB8 for ; Mon, 21 Aug 2023 14:26:31 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JsVRJ2Y5+u+0kmMfcBx068LypdYSOs190RdAFoKfNyz3dQCzngoJUOUDGLTadP95IKHEWxi0GAw1uIWBMqwIJ728hXnqct2HSX6Dh2HjGN8pY8GGt+AKXNc6pISyCDkgZqAQ1yfrQxDmtwEb8S6Pn+V4QAtMmOumtJZzZ0lJTo1ehm5lMe34+a3G492U6b/TALyScNWdFiy4A6bZ+mg9cO3v+q5wkfPgys5FtFvyWuNaF5wRMISMP0Yt/pjbQetF1AwSFW5KrDd3Li9cq5+NKC+2dBXbFXHNoElrwLsqBBukBnYqwu1pMms2gLCd3MgSx/IkPZS5/7vX9UvWSkjBEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a4xk5kk2YwWKd/cnrmpXzM65kinrEyFD+IQw7mfyBjE=; b=n8aFGtz2Xigd3aL0HOmHRwX2U4vHfueIvTuLDa69fogR7y+9q0ekSvNpz7NDuiXZuV5PxsTmZRbWcZMAqDbETUloKpG3pt7sy9itTOqLq0hNhOdCnEDJvmzuRBaeNatZxUz4xx7QcyrlrLdYkK5YV4hi88700noWtQayGuYtF1Eje8zfaT1mAtXmd+Hesyv38MxI8s5bAfmK51K1VDMZSYCqs8IFaaYVDFhaecW83OIMJhuSJa5xqVSih93OGbSaJ/mQJCETX2WcDg0ArtilGr3zOivlCaETTBu/b9VJUn1X+sCbOfvHEnUXo/WprqgNDuRcJEsmP18MDUagt7l59w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a4xk5kk2YwWKd/cnrmpXzM65kinrEyFD+IQw7mfyBjE=; b=qAOm79DJCU5yJivMVcDypmnFrf9Ed7c9c1uuU3COlOGOew3eFmHk8bs/epz9h3Igfl3/sfeSrFHqWAH+/bYgJ6gSsHvRR8AIODujyC89FHf2X0RSLPJE5lK9xhNnIn5bl79fVpWK4Erpw5WSfGdbWMXfMallMVvt3eJiZcuNmnWD+1S5t9ju/B2L2UkyxMExUVteEPzWq7v3mN8J3vkcK0WZgpJUxDN99LQZfeUCvDiePa1PfSoIbwQYlnoW0XxTW5d1321/D9V2qll7bOhoC1BkcoVGYZ8hxdDuoGyvEzYsvBPVLo5YBalptawiLZT7RDnoBAcvSXvKz2bOPEmUmA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by CY8PR12MB7414.namprd12.prod.outlook.com (2603:10b6:930:5e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.24; Mon, 21 Aug 2023 14:26:29 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6699.020; Mon, 21 Aug 2023 14:26:29 +0000 Date: Mon, 21 Aug 2023 11:26:27 -0300 From: Jason Gunthorpe To: Michael Shavit Cc: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, will@kernel.org, nicolinc@nvidia.com, tina.zhang@intel.com, jean-philippe@linaro.org, robin.murphy@arm.com Subject: Re: [RFC PATCH v1 3/8] iommu/arm-smmu-v3-sva: Allocate new ASID from installed_smmus Message-ID: References: <20230817182055.1770180-1-mshavit@google.com> <20230818021629.RFC.v1.3.I326c62dc062aed8d901d319aa665dbe983c7904c@changeid> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: BL1P221CA0016.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::29) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|CY8PR12MB7414:EE_ X-MS-Office365-Filtering-Correlation-Id: b2a1e8e3-8f55-42f9-aa9a-08dba2529bcd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yvjmjkyPZT7y8FlN16qGN6OT6HIsT8fMKtw9kq6P1pNzleOoqjzA5QK3AR+LXZBoAOc4QxWZmQBDaYQaTK/NaPqnCTVn0P5O4H6cWzsLoaD6dXzHj4Z6P7fVsNfCgA2z3O+ajrQ5iUvygtdl/Ls3an7Ouh75SAsbJyRwStW0/BgDByP3iDubWanZDgkHg674G4MuNt9R+JVKvZD3svFz8iHWpporK6atwxIAtTJuW2dsP+nNtgMpvOzKCCzlvzRvWlzpWtHx1cfCncFJSISn3Ew7FvJW47dhaCkA+IcflnRBrpQkO0fdDc5fXvc4pHo7hXN9/p5uLup8Y+G8HlQa/MkTd33/3Wy/TMDdt9mxMMcuqjkGgEEAhuhNlO6H9YF1/CSTg2d7B4ToS0VI4CJAo8Ap/hykkS9iTmQWzB2YhxnKlw+hhfCRTtzy+fBKXVFA6eAJvTIRWrI1i9vBQKa1Lcfrbp9DrAGD9WoE/VSupYI8uj08/j9xBwGTc/WSnIXrt0YzCM8tI1B6xBRZNmBW1ewGuqINYWUjJLa8BX315NPEKyXACQbgjXdgGuniwSOL X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(39860400002)(376002)(136003)(396003)(451199024)(186009)(1800799009)(2906002)(53546011)(38100700002)(6506007)(6486002)(5660300002)(26005)(86362001)(8676002)(2616005)(8936002)(4326008)(316002)(66946007)(6512007)(6916009)(66556008)(66476007)(478600001)(36756003)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RmE4WW9IemdDU01GS3ZNV0xtZXk3c2Z4akgxWUllMzRtbkFkQlBlNmoybGUx?= =?utf-8?B?cmI0TjU5a0ZKcmYvVldpaVBqQ1R1OC9MVHBZVHUrV2NJY2tUSGFzb1JpeDhH?= =?utf-8?B?M3hBdEk4VXU4NG9HeU1vb0N0ajdKNUVDcE9zVEtYcTNSeEdxZnNtaEFRcnZx?= =?utf-8?B?akhuU1pobkJXM04wdy9zUjJNVXNsNFBCTFhROU5vSUF0WXBTTlRiSmRGcFpn?= =?utf-8?B?ZWpsdWYzMWZqNVVrRTNuWUpZUHpqcHQ4SW8wb1ZJeGRLZUI0Q2pUa0hKWGhW?= =?utf-8?B?NVQrSFhrbjJ1cHJGSGpYR2s4NXpReW5oZnk0c2NOUmtxOHBGRHhBL0JjcENZ?= =?utf-8?B?Nk9MZy9XdUlydThhcEdRL2VjcGFmZEF5cEpwVmx0aVVjVm5JUjRsbmRXai9Q?= =?utf-8?B?Y082ekVDdzcxaEt4azU3ajF2N2hxV1hYWktFN1NabEZQVVpMbi9mRDFmZEt6?= =?utf-8?B?T0JRNFJIeWdISWVqTEp3R3FiM1dtdkU4dFpjSU84blBkNW10bllFaVBxTzI0?= =?utf-8?B?UFIzT1MzTFpNczFqY1NmOG9ZQTlQWW95dnA4SGhtSUdNVDA5MWt1am8xcE05?= =?utf-8?B?Y0UyRHJFSGNLR2J5aHB3WlM3Vk9DOFFFcGR6d2pvcGI3b2NUdWtHQ3dXbWV5?= =?utf-8?B?OWM0bG4zSTdHSyt1S21XcG85YitMZ3hleUZlQnBFdDVFSUhMaC9FUlV3NVVP?= =?utf-8?B?MXpHamovRWZFb2FmVENmanpHRnl3Y0J3dGx5Sm1NQ3dsTXpiOHFrU1Q0MnMy?= =?utf-8?B?eXVzK1lsajZrMlZ4c29Bb0k4Zm1NeTl2T2d3Y0M1Q1lhUFBIaXFPNHNJdDFK?= =?utf-8?B?V0g0TkxDaDR5Z2FuaGRXazhEa25ia3VYc2tKbGRxSkdlaW5Nayt3V2VVak9W?= =?utf-8?B?NFdMb2JPQXpUcmYwU1FLdnJPa2ZGZERiZU9oOE01T0VRT1AyOG92RGlOcEpI?= =?utf-8?B?dVRvdjVNK0VrSXdDd3dPeGQzVFk3U0c4a1JOcWF6UTM2dldrN1FjRFRzMjZ0?= =?utf-8?B?NHAweUpVNHFQVDJlYjE4QzlsL2piell5KzVXVG5KQzJOM0orczZlNjhwczE4?= =?utf-8?B?cGdjbk4rbjVMb2crVFRNdkh3bnNDZlh0MDFFdFBCVVZJc2N3R3V3ZjFWNWQ1?= =?utf-8?B?SWN4enBqRWlQWU5lSFBQdnBtM0JOc2RiQklPVnVyNngvWitmVlZQODFxME1V?= =?utf-8?B?VEhXbGROUEdIaVdnRUVDMEdZdGUvQjVCdjdnL0ZlOTlnZUI0a29hQVNmYkI1?= =?utf-8?B?ZzQ3M2p4RHBWWG9rR1NtS2ZuRXd2Q0JIakNMWmczYUxzUXNDKzJXSkRTeFAz?= =?utf-8?B?aHJUcldEOUtndG5obTJTNDdCNHBjUW9iemd0SFBpdEZTYnNMS1c5NjRXU3NO?= =?utf-8?B?MzJvQmI5Uk1FZVZJcGU5aHEvMzdIVTNPaTVoWHJhUElCaXhVSk5LbjJRZEVk?= =?utf-8?B?TGQvT0hnZjcrWVVBTWsrbGNDVDBQaVVSelR3Q0RlUzBNUDVwQVBENGFhRXox?= =?utf-8?B?V1FYSTlTK3J1MG85YVhxaWFkc2Y4S0pQMUVORHprZ3hZQ2Y4T0djOG1mbk1F?= =?utf-8?B?dUlsS3VOSXdTS0E3c2cxeFJnVDVIcTE3SmpuN2FGVGJ0WCtONzFQZEh4bjJy?= =?utf-8?B?dDR5R01PZ3FjTE43NUZFelNSQjJoNUJMRDhRRlJWVXo2cS9kdGZCaWp5WVJJ?= =?utf-8?B?b3ZPeUYzb3JwZzhHRFV2SFVzUzUzSkRXSVJpc3dsellMYUw3eWd5alN0QTNy?= =?utf-8?B?OXRvUVhXTGgyVW12SjlDeCsvZzlRdlpLV3R2Z1VYTSsyZVNMOG9sYk9GZ2kz?= =?utf-8?B?U1hJci9kdE5qRVd0elhrUncrMXZDcmFQNEJWMGF6VlpMcnFWZTc0SUdCaWVR?= =?utf-8?B?dEdUMFdjT3UzdytIZEpCS1VodmFDSU5JWVdZY0R1SUNiNitlWEJiQlQvYSs5?= =?utf-8?B?Ky94cGJlK3N2dlYxNVNZM1JQZHlOM0dtRGxBaU5KU2tkdHBvR2IvREo4ZEVp?= =?utf-8?B?bFpwTEwrYXNtLy9NUm1haDVrdFNQVFVLeEZGWVN1WTFRTkV6TVFpQnFIbi9J?= =?utf-8?B?T0t5dHhLdldLV3ZBZGdSdTl2VTMrSDBrVlVJRFQrSFBZZCtwWlZod2ZNWDNP?= =?utf-8?Q?Z7g65cmq1/IZ2bcREE3aszh2/?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b2a1e8e3-8f55-42f9-aa9a-08dba2529bcd X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2023 14:26:28.9767 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eXn21Zd93kplklbVw/To4p6Vf+KxzrSUbmmEmjq09+L9SAABMcLc3MLc/xn9GGoK X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7414 On Mon, Aug 21, 2023 at 10:16:54PM +0800, Michael Shavit wrote: > On Mon, Aug 21, 2023 at 9:50 PM Jason Gunthorpe wrote: > > > > On Mon, Aug 21, 2023 at 09:38:40PM +0800, Michael Shavit wrote: > > > On Mon, Aug 21, 2023 at 7:54 PM Jason Gunthorpe wrote: > > > > > > > > On Mon, Aug 21, 2023 at 05:31:23PM +0800, Michael Shavit wrote: > > > > > On Fri, Aug 18, 2023 at 2:38 AM Jason Gunthorpe wrote: > > > > > > > > > > > > On Fri, Aug 18, 2023 at 02:16:25AM +0800, Michael Shavit wrote: > > > > > > > Pick an ASID that is within the supported range of all SMMUs that the > > > > > > > domain is installed to. > > > > > > > > > > > > > > Signed-off-by: Michael Shavit > > > > > > > --- > > > > > > > > > > > > This seems like a pretty niche scenario, maybe we should just keep a > > > > > > global for the max ASID? > > > > > > > > > > > > Otherwise we need a code to change the ASID, even for non-SVA domains, > > > > > > when the domain is installed in different devices if the current ASID > > > > > > is over the instance max.. > > > > > > > > > > This RFC took the other easy way out for this problem by rejecting > > > > > attaching a domain if its currently assigned ASID/VMID > > > > > is out of range when attaching to a new SMMU. But I'm not sure > > > > > which of the two options is the right trade-off. > > > > > Especially if we move VMID to a global allocator (which I plan to add > > > > > for v2), setting a global maximum for VMID of 256 sounds small. > > > > > > > > IMHO the simplest and best thing is to make both vmid and asid as > > > > local allocators. Then alot of these problems disappear > > > > > > Well that does sound like the most flexible, but IMO quite a lot more > > > complicated. > > > > > > I'll post a v2 RFC that removes the `iommu/arm-smmu-v3: Add list of > > > installed_smmus` patch and uses a flat master list in smmu_domain as > > > suggested by Robin, for comparison with the v1. But at a glance using a > > > local allocator would require: > > > > > 1. Keeping that patch so we can track the asid/vmid for a domain on a > > > per smmu instance > > > > You'd have to store the cache tag in the per-master struct on that > > list and take it out of the domain struct. > > > > Ie the list of attached masters contains the per-master cache tag > > instead of a global cache tag. > > > > The only place you need the cache tag is when iterating over the list > > of masters, so it is OK. > > > > If the list of masters is sorted by smmu then the first master of each > > smmu can be used to perform the cache tag invalidation, then the rest > > of the list is the ATC invalidation. > > > > The looping code will be a bit ugly. > > I suppose that could work.... but I'm worried it's gonna be messy, > especially if we think about how the PASID feature would interact. > With PASID, there could be multiple domains attached to a master. So > we won't be able to store a single cache tag/asid for the currently > attached domain on the arm_smmu_master. I wasn't suggesting to store it in the arm_smmu_master, I was suggesting to store it in the same place you store the per-master PASID. eg I expect that on attach the domain will allocate new memory to store the pasid/cache tag/master/domain and thread that memory on a list of attached masters. > > > (on a loop over every smmu the domain in arm_smmu_mmu_notifier_get is > > > attached to, which just at a glance looks headache inducing because of > > > sva's piggybacking on the rid domain.) > > > > Not every smmu, just the one you are *currently* attaching to. We > > don't care if the *other* smmu's have different ASIDs, maybe they are > > not using BTM, or won't use SVA. > > I mean because the domain in arm_smmu_mmu_notifier_get is the RID > domain (not the SVA domain, same issue we discussed in previous > thread) , which can be attached to multiple SMMUs. Oh that is totally nonsensical. I expect you will need to fix that sooner than later. Once the CD table is moved and there is a proper way to track the PASID it should not be needed. It shouldn't fall into the decision making about where to put the ASID xarray. Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66F9EEE4996 for ; Mon, 21 Aug 2023 14:26:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lvQ4o7wimJUMEI3sUhE8PVwdjROl8MASzpO46LH5PZU=; b=1LIB7IEMu0XKzE kazszg9FBH1tj+NTvX3rxgiir8zBPtfC0kbMzQOCpvKVZsVWSU9nt6TeBLeu2yPdt+4hOmbRZfpu7 t1lAil7cT5frrgbshni7hdtkSzOj1xQZjRDyl/TS7kjGPuC01gKVsgA5W8OpdvhL2oV4jSTjWReJF +oEWNFqOPPT+o8VpDXlZpxCHWiC5ekLTDozqxMhqbRCjhGESvG+9cuLxAQA+5896HM0cE2UdDqglN quIaI7SQ2riidGsO8nUALhWTMhjC2tCEK1COnJnbGA5Ql+w1QnOcjj/gH57PY+0ku6tkXohU6ca3o GwpTng0j2tPd2PWd6ebA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qY5rb-00EAlB-1Q; Mon, 21 Aug 2023 14:26:39 +0000 Received: from mail-sn1nam02on20630.outbound.protection.outlook.com ([2a01:111:f400:7ea9::630] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qY5rY-00EAjz-1z for linux-arm-kernel@lists.infradead.org; Mon, 21 Aug 2023 14:26:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JsVRJ2Y5+u+0kmMfcBx068LypdYSOs190RdAFoKfNyz3dQCzngoJUOUDGLTadP95IKHEWxi0GAw1uIWBMqwIJ728hXnqct2HSX6Dh2HjGN8pY8GGt+AKXNc6pISyCDkgZqAQ1yfrQxDmtwEb8S6Pn+V4QAtMmOumtJZzZ0lJTo1ehm5lMe34+a3G492U6b/TALyScNWdFiy4A6bZ+mg9cO3v+q5wkfPgys5FtFvyWuNaF5wRMISMP0Yt/pjbQetF1AwSFW5KrDd3Li9cq5+NKC+2dBXbFXHNoElrwLsqBBukBnYqwu1pMms2gLCd3MgSx/IkPZS5/7vX9UvWSkjBEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a4xk5kk2YwWKd/cnrmpXzM65kinrEyFD+IQw7mfyBjE=; b=n8aFGtz2Xigd3aL0HOmHRwX2U4vHfueIvTuLDa69fogR7y+9q0ekSvNpz7NDuiXZuV5PxsTmZRbWcZMAqDbETUloKpG3pt7sy9itTOqLq0hNhOdCnEDJvmzuRBaeNatZxUz4xx7QcyrlrLdYkK5YV4hi88700noWtQayGuYtF1Eje8zfaT1mAtXmd+Hesyv38MxI8s5bAfmK51K1VDMZSYCqs8IFaaYVDFhaecW83OIMJhuSJa5xqVSih93OGbSaJ/mQJCETX2WcDg0ArtilGr3zOivlCaETTBu/b9VJUn1X+sCbOfvHEnUXo/WprqgNDuRcJEsmP18MDUagt7l59w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a4xk5kk2YwWKd/cnrmpXzM65kinrEyFD+IQw7mfyBjE=; b=qAOm79DJCU5yJivMVcDypmnFrf9Ed7c9c1uuU3COlOGOew3eFmHk8bs/epz9h3Igfl3/sfeSrFHqWAH+/bYgJ6gSsHvRR8AIODujyC89FHf2X0RSLPJE5lK9xhNnIn5bl79fVpWK4Erpw5WSfGdbWMXfMallMVvt3eJiZcuNmnWD+1S5t9ju/B2L2UkyxMExUVteEPzWq7v3mN8J3vkcK0WZgpJUxDN99LQZfeUCvDiePa1PfSoIbwQYlnoW0XxTW5d1321/D9V2qll7bOhoC1BkcoVGYZ8hxdDuoGyvEzYsvBPVLo5YBalptawiLZT7RDnoBAcvSXvKz2bOPEmUmA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by CY8PR12MB7414.namprd12.prod.outlook.com (2603:10b6:930:5e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.24; Mon, 21 Aug 2023 14:26:29 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6699.020; Mon, 21 Aug 2023 14:26:29 +0000 Date: Mon, 21 Aug 2023 11:26:27 -0300 From: Jason Gunthorpe To: Michael Shavit Cc: iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, will@kernel.org, nicolinc@nvidia.com, tina.zhang@intel.com, jean-philippe@linaro.org, robin.murphy@arm.com Subject: Re: [RFC PATCH v1 3/8] iommu/arm-smmu-v3-sva: Allocate new ASID from installed_smmus Message-ID: References: <20230817182055.1770180-1-mshavit@google.com> <20230818021629.RFC.v1.3.I326c62dc062aed8d901d319aa665dbe983c7904c@changeid> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1P221CA0016.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::29) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|CY8PR12MB7414:EE_ X-MS-Office365-Filtering-Correlation-Id: b2a1e8e3-8f55-42f9-aa9a-08dba2529bcd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yvjmjkyPZT7y8FlN16qGN6OT6HIsT8fMKtw9kq6P1pNzleOoqjzA5QK3AR+LXZBoAOc4QxWZmQBDaYQaTK/NaPqnCTVn0P5O4H6cWzsLoaD6dXzHj4Z6P7fVsNfCgA2z3O+ajrQ5iUvygtdl/Ls3an7Ouh75SAsbJyRwStW0/BgDByP3iDubWanZDgkHg674G4MuNt9R+JVKvZD3svFz8iHWpporK6atwxIAtTJuW2dsP+nNtgMpvOzKCCzlvzRvWlzpWtHx1cfCncFJSISn3Ew7FvJW47dhaCkA+IcflnRBrpQkO0fdDc5fXvc4pHo7hXN9/p5uLup8Y+G8HlQa/MkTd33/3Wy/TMDdt9mxMMcuqjkGgEEAhuhNlO6H9YF1/CSTg2d7B4ToS0VI4CJAo8Ap/hykkS9iTmQWzB2YhxnKlw+hhfCRTtzy+fBKXVFA6eAJvTIRWrI1i9vBQKa1Lcfrbp9DrAGD9WoE/VSupYI8uj08/j9xBwGTc/WSnIXrt0YzCM8tI1B6xBRZNmBW1ewGuqINYWUjJLa8BX315NPEKyXACQbgjXdgGuniwSOL X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(39860400002)(376002)(136003)(396003)(451199024)(186009)(1800799009)(2906002)(53546011)(38100700002)(6506007)(6486002)(5660300002)(26005)(86362001)(8676002)(2616005)(8936002)(4326008)(316002)(66946007)(6512007)(6916009)(66556008)(66476007)(478600001)(36756003)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RmE4WW9IemdDU01GS3ZNV0xtZXk3c2Z4akgxWUllMzRtbkFkQlBlNmoybGUx?= =?utf-8?B?cmI0TjU5a0ZKcmYvVldpaVBqQ1R1OC9MVHBZVHUrV2NJY2tUSGFzb1JpeDhH?= =?utf-8?B?M3hBdEk4VXU4NG9HeU1vb0N0ajdKNUVDcE9zVEtYcTNSeEdxZnNtaEFRcnZx?= =?utf-8?B?akhuU1pobkJXM04wdy9zUjJNVXNsNFBCTFhROU5vSUF0WXBTTlRiSmRGcFpn?= =?utf-8?B?ZWpsdWYzMWZqNVVrRTNuWUpZUHpqcHQ4SW8wb1ZJeGRLZUI0Q2pUa0hKWGhW?= =?utf-8?B?NVQrSFhrbjJ1cHJGSGpYR2s4NXpReW5oZnk0c2NOUmtxOHBGRHhBL0JjcENZ?= =?utf-8?B?Nk9MZy9XdUlydThhcEdRL2VjcGFmZEF5cEpwVmx0aVVjVm5JUjRsbmRXai9Q?= =?utf-8?B?Y082ekVDdzcxaEt4azU3ajF2N2hxV1hYWktFN1NabEZQVVpMbi9mRDFmZEt6?= =?utf-8?B?T0JRNFJIeWdISWVqTEp3R3FiM1dtdkU4dFpjSU84blBkNW10bllFaVBxTzI0?= =?utf-8?B?UFIzT1MzTFpNczFqY1NmOG9ZQTlQWW95dnA4SGhtSUdNVDA5MWt1am8xcE05?= =?utf-8?B?Y0UyRHJFSGNLR2J5aHB3WlM3Vk9DOFFFcGR6d2pvcGI3b2NUdWtHQ3dXbWV5?= =?utf-8?B?OWM0bG4zSTdHSyt1S21XcG85YitMZ3hleUZlQnBFdDVFSUhMaC9FUlV3NVVP?= =?utf-8?B?MXpHamovRWZFb2FmVENmanpHRnl3Y0J3dGx5Sm1NQ3dsTXpiOHFrU1Q0MnMy?= =?utf-8?B?eXVzK1lsajZrMlZ4c29Bb0k4Zm1NeTl2T2d3Y0M1Q1lhUFBIaXFPNHNJdDFK?= =?utf-8?B?V0g0TkxDaDR5Z2FuaGRXazhEa25ia3VYc2tKbGRxSkdlaW5Nayt3V2VVak9W?= =?utf-8?B?NFdMb2JPQXpUcmYwU1FLdnJPa2ZGZERiZU9oOE01T0VRT1AyOG92RGlOcEpI?= =?utf-8?B?dVRvdjVNK0VrSXdDd3dPeGQzVFk3U0c4a1JOcWF6UTM2dldrN1FjRFRzMjZ0?= =?utf-8?B?NHAweUpVNHFQVDJlYjE4QzlsL2piell5KzVXVG5KQzJOM0orczZlNjhwczE4?= =?utf-8?B?cGdjbk4rbjVMb2crVFRNdkh3bnNDZlh0MDFFdFBCVVZJc2N3R3V3ZjFWNWQ1?= =?utf-8?B?SWN4enBqRWlQWU5lSFBQdnBtM0JOc2RiQklPVnVyNngvWitmVlZQODFxME1V?= =?utf-8?B?VEhXbGROUEdIaVdnRUVDMEdZdGUvQjVCdjdnL0ZlOTlnZUI0a29hQVNmYkI1?= =?utf-8?B?ZzQ3M2p4RHBWWG9rR1NtS2ZuRXd2Q0JIakNMWmczYUxzUXNDKzJXSkRTeFAz?= =?utf-8?B?aHJUcldEOUtndG5obTJTNDdCNHBjUW9iemd0SFBpdEZTYnNMS1c5NjRXU3NO?= =?utf-8?B?MzJvQmI5Uk1FZVZJcGU5aHEvMzdIVTNPaTVoWHJhUElCaXhVSk5LbjJRZEVk?= =?utf-8?B?TGQvT0hnZjcrWVVBTWsrbGNDVDBQaVVSelR3Q0RlUzBNUDVwQVBENGFhRXox?= =?utf-8?B?V1FYSTlTK3J1MG85YVhxaWFkc2Y4S0pQMUVORHprZ3hZQ2Y4T0djOG1mbk1F?= =?utf-8?B?dUlsS3VOSXdTS0E3c2cxeFJnVDVIcTE3SmpuN2FGVGJ0WCtONzFQZEh4bjJy?= =?utf-8?B?dDR5R01PZ3FjTE43NUZFelNSQjJoNUJMRDhRRlJWVXo2cS9kdGZCaWp5WVJJ?= =?utf-8?B?b3ZPeUYzb3JwZzhHRFV2SFVzUzUzSkRXSVJpc3dsellMYUw3eWd5alN0QTNy?= =?utf-8?B?OXRvUVhXTGgyVW12SjlDeCsvZzlRdlpLV3R2Z1VYTSsyZVNMOG9sYk9GZ2kz?= =?utf-8?B?U1hJci9kdE5qRVd0elhrUncrMXZDcmFQNEJWMGF6VlpMcnFWZTc0SUdCaWVR?= =?utf-8?B?dEdUMFdjT3UzdytIZEpCS1VodmFDSU5JWVdZY0R1SUNiNitlWEJiQlQvYSs5?= =?utf-8?B?Ky94cGJlK3N2dlYxNVNZM1JQZHlOM0dtRGxBaU5KU2tkdHBvR2IvREo4ZEVp?= =?utf-8?B?bFpwTEwrYXNtLy9NUm1haDVrdFNQVFVLeEZGWVN1WTFRTkV6TVFpQnFIbi9J?= =?utf-8?B?T0t5dHhLdldLV3ZBZGdSdTl2VTMrSDBrVlVJRFQrSFBZZCtwWlZod2ZNWDNP?= =?utf-8?Q?Z7g65cmq1/IZ2bcREE3aszh2/?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b2a1e8e3-8f55-42f9-aa9a-08dba2529bcd X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2023 14:26:28.9767 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eXn21Zd93kplklbVw/To4p6Vf+KxzrSUbmmEmjq09+L9SAABMcLc3MLc/xn9GGoK X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7414 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230821_072636_655074_43E6D952 X-CRM114-Status: GOOD ( 45.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBBdWcgMjEsIDIwMjMgYXQgMTA6MTY6NTRQTSArMDgwMCwgTWljaGFlbCBTaGF2aXQg d3JvdGU6Cj4gT24gTW9uLCBBdWcgMjEsIDIwMjMgYXQgOTo1MOKAr1BNIEphc29uIEd1bnRob3Jw ZSA8amdnQG52aWRpYS5jb20+IHdyb3RlOgo+ID4KPiA+IE9uIE1vbiwgQXVnIDIxLCAyMDIzIGF0 IDA5OjM4OjQwUE0gKzA4MDAsIE1pY2hhZWwgU2hhdml0IHdyb3RlOgo+ID4gPiBPbiBNb24sIEF1 ZyAyMSwgMjAyMyBhdCA3OjU04oCvUE0gSmFzb24gR3VudGhvcnBlIDxqZ2dAbnZpZGlhLmNvbT4g d3JvdGU6Cj4gPiA+ID4KPiA+ID4gPiBPbiBNb24sIEF1ZyAyMSwgMjAyMyBhdCAwNTozMToyM1BN ICswODAwLCBNaWNoYWVsIFNoYXZpdCB3cm90ZToKPiA+ID4gPiA+IE9uIEZyaSwgQXVnIDE4LCAy MDIzIGF0IDI6MzjigK9BTSBKYXNvbiBHdW50aG9ycGUgPGpnZ0BudmlkaWEuY29tPiB3cm90ZToK PiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gT24gRnJpLCBBdWcgMTgsIDIwMjMgYXQgMDI6MTY6MjVB TSArMDgwMCwgTWljaGFlbCBTaGF2aXQgd3JvdGU6Cj4gPiA+ID4gPiA+ID4gUGljayBhbiBBU0lE IHRoYXQgaXMgd2l0aGluIHRoZSBzdXBwb3J0ZWQgcmFuZ2Ugb2YgYWxsIFNNTVVzIHRoYXQgdGhl Cj4gPiA+ID4gPiA+ID4gZG9tYWluIGlzIGluc3RhbGxlZCB0by4KPiA+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IE1pY2hhZWwgU2hhdml0IDxtc2hhdml0QGdvb2dsZS5j b20+Cj4gPiA+ID4gPiA+ID4gLS0tCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IFRoaXMgc2VlbXMg bGlrZSBhIHByZXR0eSBuaWNoZSBzY2VuYXJpbywgbWF5YmUgd2Ugc2hvdWxkIGp1c3Qga2VlcCBh Cj4gPiA+ID4gPiA+IGdsb2JhbCBmb3IgdGhlIG1heCBBU0lEPwo+ID4gPiA+ID4gPgo+ID4gPiA+ ID4gPiBPdGhlcndpc2Ugd2UgbmVlZCBhIGNvZGUgdG8gY2hhbmdlIHRoZSBBU0lELCBldmVuIGZv ciBub24tU1ZBIGRvbWFpbnMsCj4gPiA+ID4gPiA+IHdoZW4gdGhlIGRvbWFpbiBpcyBpbnN0YWxs ZWQgaW4gZGlmZmVyZW50IGRldmljZXMgaWYgdGhlIGN1cnJlbnQgQVNJRAo+ID4gPiA+ID4gPiBp cyBvdmVyIHRoZSBpbnN0YW5jZSBtYXguLgo+ID4gPiA+ID4KPiA+ID4gPiA+IFRoaXMgUkZDIHRv b2sgdGhlIG90aGVyIGVhc3kgd2F5IG91dCBmb3IgdGhpcyBwcm9ibGVtIGJ5IHJlamVjdGluZwo+ ID4gPiA+ID4gYXR0YWNoaW5nIGEgZG9tYWluIGlmIGl0cyBjdXJyZW50bHkgYXNzaWduZWQgQVNJ RC9WTUlECj4gPiA+ID4gPiBpcyBvdXQgb2YgcmFuZ2Ugd2hlbiBhdHRhY2hpbmcgdG8gYSBuZXcg U01NVS4gQnV0IEknbSBub3Qgc3VyZQo+ID4gPiA+ID4gd2hpY2ggb2YgdGhlIHR3byBvcHRpb25z IGlzIHRoZSByaWdodCB0cmFkZS1vZmYuCj4gPiA+ID4gPiBFc3BlY2lhbGx5IGlmIHdlIG1vdmUg Vk1JRCB0byBhIGdsb2JhbCBhbGxvY2F0b3IgKHdoaWNoIEkgcGxhbiB0byBhZGQKPiA+ID4gPiA+ IGZvciB2MiksIHNldHRpbmcgYSBnbG9iYWwgbWF4aW11bSBmb3IgVk1JRCBvZiAyNTYgc291bmRz IHNtYWxsLgo+ID4gPiA+Cj4gPiA+ID4gSU1ITyB0aGUgc2ltcGxlc3QgYW5kIGJlc3QgdGhpbmcg aXMgdG8gbWFrZSBib3RoIHZtaWQgYW5kIGFzaWQgYXMKPiA+ID4gPiBsb2NhbCBhbGxvY2F0b3Jz LiBUaGVuIGFsb3Qgb2YgdGhlc2UgcHJvYmxlbXMgZGlzYXBwZWFyCj4gPiA+Cj4gPiA+IFdlbGwg dGhhdCBkb2VzIHNvdW5kIGxpa2UgdGhlIG1vc3QgZmxleGlibGUsIGJ1dCBJTU8gcXVpdGUgYSBs b3QgbW9yZQo+ID4gPiBjb21wbGljYXRlZC4KPiA+ID4KPiA+ID4gSSdsbCBwb3N0IGEgdjIgUkZD IHRoYXQgcmVtb3ZlcyB0aGUgYGlvbW11L2FybS1zbW11LXYzOiBBZGQgbGlzdCBvZgo+ID4gPiBp bnN0YWxsZWRfc21tdXNgIHBhdGNoIGFuZCB1c2VzIGEgZmxhdCBtYXN0ZXIgbGlzdCBpbiBzbW11 X2RvbWFpbiBhcwo+ID4gPiBzdWdnZXN0ZWQgYnkgUm9iaW4sIGZvciBjb21wYXJpc29uIHdpdGgg dGhlIHYxLiBCdXQgYXQgYSBnbGFuY2UgdXNpbmcgYQo+ID4gPiBsb2NhbCBhbGxvY2F0b3Igd291 bGQgcmVxdWlyZToKPiA+Cj4gPiA+IDEuIEtlZXBpbmcgdGhhdCBwYXRjaCBzbyB3ZSBjYW4gdHJh Y2sgdGhlIGFzaWQvdm1pZCBmb3IgYSBkb21haW4gb24gYQo+ID4gPiBwZXIgc21tdSBpbnN0YW5j ZQo+ID4KPiA+IFlvdSdkIGhhdmUgdG8gc3RvcmUgdGhlIGNhY2hlIHRhZyBpbiB0aGUgcGVyLW1h c3RlciBzdHJ1Y3Qgb24gdGhhdAo+ID4gbGlzdCBhbmQgdGFrZSBpdCBvdXQgb2YgdGhlIGRvbWFp biBzdHJ1Y3QuCj4gPgo+ID4gSWUgdGhlIGxpc3Qgb2YgYXR0YWNoZWQgbWFzdGVycyBjb250YWlu cyB0aGUgcGVyLW1hc3RlciBjYWNoZSB0YWcKPiA+IGluc3RlYWQgb2YgYSBnbG9iYWwgY2FjaGUg dGFnLgo+ID4KPiA+IFRoZSBvbmx5IHBsYWNlIHlvdSBuZWVkIHRoZSBjYWNoZSB0YWcgaXMgd2hl biBpdGVyYXRpbmcgb3ZlciB0aGUgbGlzdAo+ID4gb2YgbWFzdGVycywgc28gaXQgaXMgT0suCj4g Pgo+ID4gSWYgdGhlIGxpc3Qgb2YgbWFzdGVycyBpcyBzb3J0ZWQgYnkgc21tdSB0aGVuIHRoZSBm aXJzdCBtYXN0ZXIgb2YgZWFjaAo+ID4gc21tdSBjYW4gYmUgdXNlZCB0byBwZXJmb3JtIHRoZSBj YWNoZSB0YWcgaW52YWxpZGF0aW9uLCB0aGVuIHRoZSByZXN0Cj4gPiBvZiB0aGUgbGlzdCBpcyB0 aGUgQVRDIGludmFsaWRhdGlvbi4KPiA+Cj4gPiBUaGUgbG9vcGluZyBjb2RlIHdpbGwgYmUgYSBi aXQgdWdseS4KPiAKPiBJIHN1cHBvc2UgdGhhdCBjb3VsZCB3b3JrLi4uLiBidXQgSSdtIHdvcnJp ZWQgaXQncyBnb25uYSBiZSBtZXNzeSwKPiBlc3BlY2lhbGx5IGlmIHdlIHRoaW5rIGFib3V0IGhv dyB0aGUgUEFTSUQgZmVhdHVyZSB3b3VsZCBpbnRlcmFjdC4KPiBXaXRoIFBBU0lELCB0aGVyZSBj b3VsZCBiZSBtdWx0aXBsZSBkb21haW5zIGF0dGFjaGVkIHRvIGEgbWFzdGVyLiBTbwo+IHdlIHdv bid0IGJlIGFibGUgdG8gc3RvcmUgYSBzaW5nbGUgY2FjaGUgdGFnL2FzaWQgZm9yIHRoZSBjdXJy ZW50bHkKPiBhdHRhY2hlZCBkb21haW4gb24gdGhlIGFybV9zbW11X21hc3Rlci4gCgpJIHdhc24n dCBzdWdnZXN0aW5nIHRvIHN0b3JlIGl0IGluIHRoZSBhcm1fc21tdV9tYXN0ZXIsIEkgd2FzCnN1 Z2dlc3RpbmcgdG8gc3RvcmUgaXQgaW4gdGhlIHNhbWUgcGxhY2UgeW91IHN0b3JlIHRoZSBwZXIt bWFzdGVyClBBU0lELgoKZWcgSSBleHBlY3QgdGhhdCBvbiBhdHRhY2ggdGhlIGRvbWFpbiB3aWxs IGFsbG9jYXRlIG5ldyBtZW1vcnkgdG8Kc3RvcmUgdGhlIHBhc2lkL2NhY2hlIHRhZy9tYXN0ZXIv ZG9tYWluIGFuZCB0aHJlYWQgdGhhdCBtZW1vcnkgb24gYQpsaXN0IG9mIGF0dGFjaGVkIG1hc3Rl cnMuCgo+ID4gPiAob24gYSBsb29wIG92ZXIgZXZlcnkgc21tdSB0aGUgZG9tYWluIGluIGFybV9z bW11X21tdV9ub3RpZmllcl9nZXQgaXMKPiA+ID4gYXR0YWNoZWQgdG8sIHdoaWNoIGp1c3QgYXQg YSBnbGFuY2UgbG9va3MgaGVhZGFjaGUgaW5kdWNpbmcgYmVjYXVzZSBvZgo+ID4gPiBzdmEncyBw aWdneWJhY2tpbmcgb24gdGhlIHJpZCBkb21haW4uKQo+ID4KPiA+IE5vdCBldmVyeSBzbW11LCBq dXN0IHRoZSBvbmUgeW91IGFyZSAqY3VycmVudGx5KiBhdHRhY2hpbmcgdG8uIFdlCj4gPiBkb24n dCBjYXJlIGlmIHRoZSAqb3RoZXIqIHNtbXUncyBoYXZlIGRpZmZlcmVudCBBU0lEcywgbWF5YmUg dGhleSBhcmUKPiA+IG5vdCB1c2luZyBCVE0sIG9yIHdvbid0IHVzZSBTVkEuCj4gCj4gSSBtZWFu IGJlY2F1c2UgdGhlIGRvbWFpbiBpbiBhcm1fc21tdV9tbXVfbm90aWZpZXJfZ2V0IGlzIHRoZSBS SUQKPiBkb21haW4gKG5vdCB0aGUgU1ZBIGRvbWFpbiwgc2FtZSBpc3N1ZSB3ZSBkaXNjdXNzZWQg aW4gcHJldmlvdXMKPiB0aHJlYWQpICwgd2hpY2ggY2FuIGJlIGF0dGFjaGVkIHRvIG11bHRpcGxl IFNNTVVzLgoKT2ggdGhhdCBpcyB0b3RhbGx5IG5vbnNlbnNpY2FsLiBJIGV4cGVjdCB5b3Ugd2ls bCBuZWVkIHRvIGZpeCB0aGF0CnNvb25lciB0aGFuIGxhdGVyLiBPbmNlIHRoZSBDRCB0YWJsZSBp cyBtb3ZlZCBhbmQgdGhlcmUgaXMgYSBwcm9wZXIKd2F5IHRvIHRyYWNrIHRoZSBQQVNJRCBpdCBz aG91bGQgbm90IGJlIG5lZWRlZC4gSXQgc2hvdWxkbid0IGZhbGwgaW50bwp0aGUgZGVjaXNpb24g bWFraW5nIGFib3V0IHdoZXJlIHRvIHB1dCB0aGUgQVNJRCB4YXJyYXkuCgpKYXNvbgoKCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJu ZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRw Oi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK