From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30525127E18 for ; Fri, 10 May 2024 15:11:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715353883; cv=none; b=HXW/RGPtS+oE2yBSRtsujW7zjotyHeJGhQKwF61e0oK77IvvmM0X1C9FkB9kU/e4ObZIS/xrx4Du+O1SKvRlbn6ws5XtvpIK7zve/6mZTEEH5h7mHs3litMlJR204COwCg2SqUQ++UeUgQGsZW7l8ApKw7Lqd0xhgjBKumlaXGg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715353883; c=relaxed/simple; bh=ztkUBcLcIUpZv28LSxi8b0OilCPoVz6sTePjYkYJv/s=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID: MIME-Version:Content-Type; b=ppIIj1s3uw0qCnSwJhs3zJ10SbcNOZDGGI4ONBmycrDwzhshtRraEphqHbgOhkwUDj/BmXJf+6uDJoz/zpJQH+Zn3xSjxcMc0c15awCjYb0YHKyYHtKfWtxtliJTmYfUtgdAnjJQ6zkqE6Mau5KCk0aT552HAisnNlyxN2lq0hI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KBAWrew0; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KBAWrew0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715353881; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ztkUBcLcIUpZv28LSxi8b0OilCPoVz6sTePjYkYJv/s=; b=KBAWrew0axtr3M2O1prJ6cI++wUJwVlR1rlJ943iJKtx9ZEbd+EUmUoSb7lylElP3TwxN4 4zM+dpn79WnZlZ5QWb9sjA1OIeee0sFajCjomNeSnHhisoHDmP6icNMMrdWFcwiff9TOpp jTxAWDpPwz1K0yknG9H3t+6QyXo6/ck= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-669-MT0tNmU0M2mHagOT3kVcWQ-1; Fri, 10 May 2024 11:11:11 -0400 X-MC-Unique: MT0tNmU0M2mHagOT3kVcWQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7BAAA1C0151F; Fri, 10 May 2024 15:11:10 +0000 (UTC) Received: from localhost (dhcp-192-239.str.redhat.com [10.33.192.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F0ABC20898CA; Fri, 10 May 2024 15:11:09 +0000 (UTC) From: Cornelia Huck To: Oliver Upton Cc: "Russell King (Oracle)" , Marc Zyngier , Catalin Marinas , Will Deacon , James Morse , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Subject: Re: [PATCH RFC] KVM: arm64: allow ID_MMFR4_EL1 to be writable In-Reply-To: Organization: "Red Hat GmbH, Sitz: Werner-von-Siemens-Ring 12, D-85630 Grasbrunn, Handelsregister: Amtsgericht =?utf-8?Q?M=C3=BCnchen=2C?= HRB 153243, =?utf-8?Q?Gesch=C3=A4ftsf=C3=BChrer=3A?= Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross" References: <87jzk4h5ir.fsf@redhat.com> User-Agent: Notmuch/0.38.3 (https://notmuchmail.org) Date: Fri, 10 May 2024 17:11:09 +0200 Message-ID: <875xvlhfci.fsf@redhat.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Wed, May 08 2024, Oliver Upton wrote: > Hi Cornelia, > > On Wed, May 08, 2024 at 02:06:36PM +0200, Cornelia Huck wrote: >> On Thu, May 02 2024, Oliver Upton wrote: >> > I think (1) should only be expected of VMMs that want rollback safety, >> > i.e. the ability to migrate state back to an older kernel. Our userspa= ce >> > initializes vCPUs from a fixed set of feature ID register values that >> > prevents VMs on new kernels from picking up new CPU features. >> > >> > It is quite tedious, but necessary as rollback safety is very much a >> > non-goal of the KVM UAPI. >>=20 >> Depending on your use case, rollback safety might be quite >> important... have we ever stated exactly which guarantees the KVM UAPI >> is giving? IOW, can someone implementing a VMM look at a doc and see >> "oh, if I want to be able to go backwards, I need to be able to deal >> with x, y, and z coming up on the new kernel"? > > The behavior of KVM/arm64 has always been that new VMs get the maximum > set of vCPU features supported by KVM / hardware modulo the ones we > require explicit opt-in from userspace (e.g. SVE, vPMU). This behavior > is described in the arm64 vCPU feature documentation [1]. > > The biggest benefit of this approach is that new vCPU features can be > added without a VMM change, as userspace can just treat the registers in > KVM_GET_REG_LIST as an opaque blob of data that needs to be migrated. It also needs to actively turn off everything it does not know how to handle, if migration between different hardware is supposed to be supported. > > I'm willing to wager that the set of users who want to migrate state > from kernel N -> (N - 1) know the exact CPU definition they want to > expose to the guest, and in that case should be using a static set of > feature register values matching their intent. I think the trouble starts when we introduce additional ranges of registers that can be controled via that interface -- old userspace will be able to figure out that there are more ranges available than what it is aware of, but will have no idea how to handle those additional ranges to get into a defined state (what is the actual range, for example?) > > Beyond the CPU architecture, KVM presents hypercall features to the VM > which userspace can _opt-out_ of on a per-feature basis using the > feature bitmap registers described in [2]. Like the feature ID > registers, we've preallocated a range of indices to be used for > hypercall bitmaps. So if an unexpected bitmap register appears on a new > kernel, userspace should write 0 to it to prevent new features from > silently creeping in. Hm, the doc says: "The features for the registers that are untouched, probably because userspace isn=E2=80=99t aware of them, will be exposed as = is to the guest." Seems to indicate that it is not too hard to get this wrong :) > > Prescribing the exact combination of these UAPIs to achieve a > rollback-safe feature set is beyond the scope of the KVM documentation > and should be determined based on the minimum kernel version that needs > to work. > > [1]: https://docs.kernel.org/virt/kvm/arm/vcpu-features.html#the-id-regis= ters > [2]: https://docs.kernel.org/virt/kvm/arm/hypercalls.html > >> >> I've been bitten before with KVM differences between kernel versions >> >> in the past - where the number of registers that userspace sees has >> >> changed despite being on the same hardware. >> > >> > This is intended behavior, as VMs are initialized to the maximum featu= re >> > set KVM is able to support. Forward-compatibility for the set of expos= ed >> > registers is tested, see the get-reg-list selftest. >>=20 >> I've seen this problem come up as well; if it is clear that this is >> something that KVM expects the VMM to handle if needed, that is fine; >> should we consider "it's tested in a selftest" as a canonical indicator >> of "this is what KVM supports compatibility wise"? > > It certainly is the level of compatibility that gets actively tested :) :) > > The canonical reason for this behavior, though, is that KVM/arm64 > defaults to the maximum-possible feature set as discussed above. /me contemplating "reverse" features, but too tired to think this through on a Friday afternoon. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 33D3AC25B10 for ; Fri, 10 May 2024 15:11:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=V+XVhvE6yujrQ+oci+X/aVetBbuMHSgwUPCu6oPJNMM=; b=06CB9Q7bZs+cN5 iyhbNNKC1KNUzHp9bwJz73RD7gR5CMF5oLUQ2JTBdtb7hb5Lh7e4zA0CYBUCswnQ4qQdGn9cXAz+t 668zEiHChq0MgFWVvI2yoUPN66debUen7XP+rtklZHppQQ9jSglywL2FziMBN5JXe3IBdpKbIm2RF vIX2FkxprPxFAznoO+zm/xzF4d7AngaRUwwmZtkA5ebNmXw1gZSlxtO8UZ802DIb8y8zp3UhOd879 e4vPspNkUKhn3i7Lsj/tgJkyUNI1sX4oEDeignnAPR4XyFa7r2frmneUjhzSYwQ7k8HuzzC8T/2CJ RwvNKUaBjuDyU706ONdQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5RuA-00000005faq-0B7L; Fri, 10 May 2024 15:11:26 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s5Ru6-00000005fYS-3g77 for linux-arm-kernel@lists.infradead.org; Fri, 10 May 2024 15:11:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715353879; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ztkUBcLcIUpZv28LSxi8b0OilCPoVz6sTePjYkYJv/s=; b=h+M8aSVihZP6Y/T47swODE5J2qD0g6yq5GHanqQLXh8HfUjMHfd4NXMuNhyugpISr5VhgE rqoXAisgt2x1ljdXk25xJKpYuko8JZSZV/c9UXtjXRJ9DUkVx2ARcS9uypORvlGfXrSpG6 zLiIPAFUodqu7huPb+W1CQj0Flv6OR0= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-669-MT0tNmU0M2mHagOT3kVcWQ-1; Fri, 10 May 2024 11:11:11 -0400 X-MC-Unique: MT0tNmU0M2mHagOT3kVcWQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7BAAA1C0151F; Fri, 10 May 2024 15:11:10 +0000 (UTC) Received: from localhost (dhcp-192-239.str.redhat.com [10.33.192.239]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F0ABC20898CA; Fri, 10 May 2024 15:11:09 +0000 (UTC) From: Cornelia Huck To: Oliver Upton Cc: "Russell King (Oracle)" , Marc Zyngier , Catalin Marinas , Will Deacon , James Morse , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Subject: Re: [PATCH RFC] KVM: arm64: allow ID_MMFR4_EL1 to be writable In-Reply-To: Organization: "Red Hat GmbH, Sitz: Werner-von-Siemens-Ring 12, D-85630 Grasbrunn, Handelsregister: Amtsgericht =?utf-8?Q?M=C3=BCnchen=2C?= HRB 153243, =?utf-8?Q?Gesch=C3=A4ftsf=C3=BChrer=3A?= Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross" References: <87jzk4h5ir.fsf@redhat.com> User-Agent: Notmuch/0.38.3 (https://notmuchmail.org) Date: Fri, 10 May 2024 17:11:09 +0200 Message-ID: <875xvlhfci.fsf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240510_081123_021139_1FA860E1 X-CRM114-Status: GOOD ( 37.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBNYXkgMDggMjAyNCwgT2xpdmVyIFVwdG9uIDxvbGl2ZXIudXB0b25AbGludXguZGV2 PiB3cm90ZToKCj4gSGkgQ29ybmVsaWEsCj4KPiBPbiBXZWQsIE1heSAwOCwgMjAyNCBhdCAwMjow NjozNlBNICswMjAwLCBDb3JuZWxpYSBIdWNrIHdyb3RlOgo+PiBPbiBUaHUsIE1heSAwMiAyMDI0 LCBPbGl2ZXIgVXB0b24gPG9saXZlci51cHRvbkBsaW51eC5kZXY+IHdyb3RlOgo+PiA+IEkgdGhp bmsgKDEpIHNob3VsZCBvbmx5IGJlIGV4cGVjdGVkIG9mIFZNTXMgdGhhdCB3YW50IHJvbGxiYWNr IHNhZmV0eSwKPj4gPiBpLmUuIHRoZSBhYmlsaXR5IHRvIG1pZ3JhdGUgc3RhdGUgYmFjayB0byBh biBvbGRlciBrZXJuZWwuIE91ciB1c2Vyc3BhY2UKPj4gPiBpbml0aWFsaXplcyB2Q1BVcyBmcm9t IGEgZml4ZWQgc2V0IG9mIGZlYXR1cmUgSUQgcmVnaXN0ZXIgdmFsdWVzIHRoYXQKPj4gPiBwcmV2 ZW50cyBWTXMgb24gbmV3IGtlcm5lbHMgZnJvbSBwaWNraW5nIHVwIG5ldyBDUFUgZmVhdHVyZXMu Cj4+ID4KPj4gPiBJdCBpcyBxdWl0ZSB0ZWRpb3VzLCBidXQgbmVjZXNzYXJ5IGFzIHJvbGxiYWNr IHNhZmV0eSBpcyB2ZXJ5IG11Y2ggYQo+PiA+IG5vbi1nb2FsIG9mIHRoZSBLVk0gVUFQSS4KPj4g Cj4+IERlcGVuZGluZyBvbiB5b3VyIHVzZSBjYXNlLCByb2xsYmFjayBzYWZldHkgbWlnaHQgYmUg cXVpdGUKPj4gaW1wb3J0YW50Li4uIGhhdmUgd2UgZXZlciBzdGF0ZWQgZXhhY3RseSB3aGljaCBn dWFyYW50ZWVzIHRoZSBLVk0gVUFQSQo+PiBpcyBnaXZpbmc/IElPVywgY2FuIHNvbWVvbmUgaW1w bGVtZW50aW5nIGEgVk1NIGxvb2sgYXQgYSBkb2MgYW5kIHNlZQo+PiAib2gsIGlmIEkgd2FudCB0 byBiZSBhYmxlIHRvIGdvIGJhY2t3YXJkcywgSSBuZWVkIHRvIGJlIGFibGUgdG8gZGVhbAo+PiB3 aXRoIHgsIHksIGFuZCB6IGNvbWluZyB1cCBvbiB0aGUgbmV3IGtlcm5lbCI/Cj4KPiBUaGUgYmVo YXZpb3Igb2YgS1ZNL2FybTY0IGhhcyBhbHdheXMgYmVlbiB0aGF0IG5ldyBWTXMgZ2V0IHRoZSBt YXhpbXVtCj4gc2V0IG9mIHZDUFUgZmVhdHVyZXMgc3VwcG9ydGVkIGJ5IEtWTSAvIGhhcmR3YXJl IG1vZHVsbyB0aGUgb25lcyB3ZQo+IHJlcXVpcmUgZXhwbGljaXQgb3B0LWluIGZyb20gdXNlcnNw YWNlIChlLmcuIFNWRSwgdlBNVSkuIFRoaXMgYmVoYXZpb3IKPiBpcyBkZXNjcmliZWQgaW4gdGhl IGFybTY0IHZDUFUgZmVhdHVyZSBkb2N1bWVudGF0aW9uIFsxXS4KPgo+IFRoZSBiaWdnZXN0IGJl bmVmaXQgb2YgdGhpcyBhcHByb2FjaCBpcyB0aGF0IG5ldyB2Q1BVIGZlYXR1cmVzIGNhbiBiZQo+ IGFkZGVkIHdpdGhvdXQgYSBWTU0gY2hhbmdlLCBhcyB1c2Vyc3BhY2UgY2FuIGp1c3QgdHJlYXQg dGhlIHJlZ2lzdGVycyBpbgo+IEtWTV9HRVRfUkVHX0xJU1QgYXMgYW4gb3BhcXVlIGJsb2Igb2Yg ZGF0YSB0aGF0IG5lZWRzIHRvIGJlIG1pZ3JhdGVkLgoKSXQgYWxzbyBuZWVkcyB0byBhY3RpdmVs eSB0dXJuIG9mZiBldmVyeXRoaW5nIGl0IGRvZXMgbm90IGtub3cgaG93IHRvCmhhbmRsZSwgaWYg bWlncmF0aW9uIGJldHdlZW4gZGlmZmVyZW50IGhhcmR3YXJlIGlzIHN1cHBvc2VkIHRvIGJlCnN1 cHBvcnRlZC4KCj4KPiBJJ20gd2lsbGluZyB0byB3YWdlciB0aGF0IHRoZSBzZXQgb2YgdXNlcnMg d2hvIHdhbnQgdG8gbWlncmF0ZSBzdGF0ZQo+IGZyb20ga2VybmVsIE4gLT4gKE4gLSAxKSBrbm93 IHRoZSBleGFjdCBDUFUgZGVmaW5pdGlvbiB0aGV5IHdhbnQgdG8KPiBleHBvc2UgdG8gdGhlIGd1 ZXN0LCBhbmQgaW4gdGhhdCBjYXNlIHNob3VsZCBiZSB1c2luZyBhIHN0YXRpYyBzZXQgb2YKPiBm ZWF0dXJlIHJlZ2lzdGVyIHZhbHVlcyBtYXRjaGluZyB0aGVpciBpbnRlbnQuCgpJIHRoaW5rIHRo ZSB0cm91YmxlIHN0YXJ0cyB3aGVuIHdlIGludHJvZHVjZSBhZGRpdGlvbmFsIHJhbmdlcyBvZgpy ZWdpc3RlcnMgdGhhdCBjYW4gYmUgY29udHJvbGVkIHZpYSB0aGF0IGludGVyZmFjZSAtLSBvbGQg dXNlcnNwYWNlIHdpbGwKYmUgYWJsZSB0byBmaWd1cmUgb3V0IHRoYXQgdGhlcmUgYXJlIG1vcmUg cmFuZ2VzIGF2YWlsYWJsZSB0aGFuIHdoYXQgaXQKaXMgYXdhcmUgb2YsIGJ1dCB3aWxsIGhhdmUg bm8gaWRlYSBob3cgdG8gaGFuZGxlIHRob3NlIGFkZGl0aW9uYWwgcmFuZ2VzCnRvIGdldCBpbnRv IGEgZGVmaW5lZCBzdGF0ZSAod2hhdCBpcyB0aGUgYWN0dWFsIHJhbmdlLCBmb3IgZXhhbXBsZT8p Cgo+Cj4gQmV5b25kIHRoZSBDUFUgYXJjaGl0ZWN0dXJlLCBLVk0gcHJlc2VudHMgaHlwZXJjYWxs IGZlYXR1cmVzIHRvIHRoZSBWTQo+IHdoaWNoIHVzZXJzcGFjZSBjYW4gX29wdC1vdXRfIG9mIG9u IGEgcGVyLWZlYXR1cmUgYmFzaXMgdXNpbmcgdGhlCj4gZmVhdHVyZSBiaXRtYXAgcmVnaXN0ZXJz IGRlc2NyaWJlZCBpbiBbMl0uIExpa2UgdGhlIGZlYXR1cmUgSUQKPiByZWdpc3RlcnMsIHdlJ3Zl IHByZWFsbG9jYXRlZCBhIHJhbmdlIG9mIGluZGljZXMgdG8gYmUgdXNlZCBmb3IKPiBoeXBlcmNh bGwgYml0bWFwcy4gU28gaWYgYW4gdW5leHBlY3RlZCBiaXRtYXAgcmVnaXN0ZXIgYXBwZWFycyBv biBhIG5ldwo+IGtlcm5lbCwgdXNlcnNwYWNlIHNob3VsZCB3cml0ZSAwIHRvIGl0IHRvIHByZXZl bnQgbmV3IGZlYXR1cmVzIGZyb20KPiBzaWxlbnRseSBjcmVlcGluZyBpbi4KCkhtLCB0aGUgZG9j IHNheXM6ICJUaGUgZmVhdHVyZXMgZm9yIHRoZSByZWdpc3RlcnMgdGhhdCBhcmUgdW50b3VjaGVk LApwcm9iYWJseSBiZWNhdXNlIHVzZXJzcGFjZSBpc27igJl0IGF3YXJlIG9mIHRoZW0sIHdpbGwg YmUgZXhwb3NlZCBhcyBpcyB0bwp0aGUgZ3Vlc3QuIiBTZWVtcyB0byBpbmRpY2F0ZSB0aGF0IGl0 IGlzIG5vdCB0b28gaGFyZCB0byBnZXQgdGhpcyB3cm9uZyA6KQoKPgo+IFByZXNjcmliaW5nIHRo ZSBleGFjdCBjb21iaW5hdGlvbiBvZiB0aGVzZSBVQVBJcyB0byBhY2hpZXZlIGEKPiByb2xsYmFj ay1zYWZlIGZlYXR1cmUgc2V0IGlzIGJleW9uZCB0aGUgc2NvcGUgb2YgdGhlIEtWTSBkb2N1bWVu dGF0aW9uCj4gYW5kIHNob3VsZCBiZSBkZXRlcm1pbmVkIGJhc2VkIG9uIHRoZSBtaW5pbXVtIGtl cm5lbCB2ZXJzaW9uIHRoYXQgbmVlZHMKPiB0byB3b3JrLgo+Cj4gWzFdOiBodHRwczovL2RvY3Mu a2VybmVsLm9yZy92aXJ0L2t2bS9hcm0vdmNwdS1mZWF0dXJlcy5odG1sI3RoZS1pZC1yZWdpc3Rl cnMKPiBbMl06IGh0dHBzOi8vZG9jcy5rZXJuZWwub3JnL3ZpcnQva3ZtL2FybS9oeXBlcmNhbGxz Lmh0bWwKPgo+PiA+PiBJJ3ZlIGJlZW4gYml0dGVuIGJlZm9yZSB3aXRoIEtWTSBkaWZmZXJlbmNl cyBiZXR3ZWVuIGtlcm5lbCB2ZXJzaW9ucwo+PiA+PiBpbiB0aGUgcGFzdCAtIHdoZXJlIHRoZSBu dW1iZXIgb2YgcmVnaXN0ZXJzIHRoYXQgdXNlcnNwYWNlIHNlZXMgaGFzCj4+ID4+IGNoYW5nZWQg ZGVzcGl0ZSBiZWluZyBvbiB0aGUgc2FtZSBoYXJkd2FyZS4KPj4gPgo+PiA+IFRoaXMgaXMgaW50 ZW5kZWQgYmVoYXZpb3IsIGFzIFZNcyBhcmUgaW5pdGlhbGl6ZWQgdG8gdGhlIG1heGltdW0gZmVh dHVyZQo+PiA+IHNldCBLVk0gaXMgYWJsZSB0byBzdXBwb3J0LiBGb3J3YXJkLWNvbXBhdGliaWxp dHkgZm9yIHRoZSBzZXQgb2YgZXhwb3NlZAo+PiA+IHJlZ2lzdGVycyBpcyB0ZXN0ZWQsIHNlZSB0 aGUgZ2V0LXJlZy1saXN0IHNlbGZ0ZXN0Lgo+PiAKPj4gSSd2ZSBzZWVuIHRoaXMgcHJvYmxlbSBj b21lIHVwIGFzIHdlbGw7IGlmIGl0IGlzIGNsZWFyIHRoYXQgdGhpcyBpcwo+PiBzb21ldGhpbmcg dGhhdCBLVk0gZXhwZWN0cyB0aGUgVk1NIHRvIGhhbmRsZSBpZiBuZWVkZWQsIHRoYXQgaXMgZmlu ZTsKPj4gc2hvdWxkIHdlIGNvbnNpZGVyICJpdCdzIHRlc3RlZCBpbiBhIHNlbGZ0ZXN0IiBhcyBh IGNhbm9uaWNhbCBpbmRpY2F0b3IKPj4gb2YgInRoaXMgaXMgd2hhdCBLVk0gc3VwcG9ydHMgY29t cGF0aWJpbGl0eSB3aXNlIj8KPgo+IEl0IGNlcnRhaW5seSBpcyB0aGUgbGV2ZWwgb2YgY29tcGF0 aWJpbGl0eSB0aGF0IGdldHMgYWN0aXZlbHkgdGVzdGVkIDopCgo6KQoKPgo+IFRoZSBjYW5vbmlj YWwgcmVhc29uIGZvciB0aGlzIGJlaGF2aW9yLCB0aG91Z2gsIGlzIHRoYXQgS1ZNL2FybTY0Cj4g ZGVmYXVsdHMgdG8gdGhlIG1heGltdW0tcG9zc2libGUgZmVhdHVyZSBzZXQgYXMgZGlzY3Vzc2Vk IGFib3ZlLgoKL21lIGNvbnRlbXBsYXRpbmcgInJldmVyc2UiIGZlYXR1cmVzLCBidXQgdG9vIHRp cmVkIHRvIHRoaW5rIHRoaXMKdGhyb3VnaCBvbiBhIEZyaWRheSBhZnRlcm5vb24uCgoKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5l bCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6 Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=