From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=3.0 tests=DATE_IN_PAST_06_12, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF0FAC5B578 for ; Wed, 3 Jul 2019 07:14:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7FC012187F for ; Wed, 3 Jul 2019 07:14:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727025AbfGCHOE (ORCPT ); Wed, 3 Jul 2019 03:14:04 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:42908 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726236AbfGCHOD (ORCPT ); Wed, 3 Jul 2019 03:14:03 -0400 Received: by mail-pf1-f193.google.com with SMTP id q10so760331pff.9; Wed, 03 Jul 2019 00:14:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=B1kQvrQoHFPsOT1yXoo85QIU6lm9lrhuL5yM9gcXKZ0=; b=JxLUAig19YnjUmw6ALYfzgLrHOVjeo17HvEdychfmCmtybaGadTrbQYN3LyWwk7Wh8 gNgPlRn+3SP41Nx1QXz+LUkeHziuOMY/GbX7QH5wfBJcMR7AcfmJii3hvOLuUnjZWIdZ ho0UBvwZZA8BLZnx3v3PwhWbM7Wms5kfZqbMjV+Z5pmz7oSMEf9gO53yPQrP1hY9jcbY ptvL0DN5BR51wI6MivzqiXjslUKqLzmo4cpeIlda7hpPYHqS9Po7lOhS8Uz1tlYSpCn6 Y2/7CN1uHo+4kFdzY1Uo7Nkfz4Wl0PST7h0XzUv+0oyFZJk5p96mUAzGShaOkFFGuR+j DKQQ== X-Gm-Message-State: APjAAAW0I2skziGMJK8aBQE6bg1PWdHIwyLy1hAuwsC5cAbG4spf9Nv5 utTeYli6jEnyy/qq675++68= X-Google-Smtp-Source: APXvYqwUyamaUZpHybk6nfj2wr4zAu1x6OPqmfFZEMCShOUOWld0YXW8Rb4mD/UbT+YdTI4jAfzbyg== X-Received: by 2002:a17:90a:22ef:: with SMTP id s102mr11149274pjc.2.1562138042451; Wed, 03 Jul 2019 00:14:02 -0700 (PDT) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id j21sm1256593pfh.86.2019.07.03.00.14.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 03 Jul 2019 00:14:01 -0700 (PDT) From: Nadav Amit To: Andy Lutomirski , Dave Hansen Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Nadav Amit , Borislav Petkov , Boris Ostrovsky , Haiyang Zhang , Josh Poimboeuf , Juergen Gross , "K. Y. Srinivasan" , Paolo Bonzini , Rik van Riel , Sasha Levin , Stephen Hemminger , kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org Subject: [PATCH v2 0/9] x86: Concurrent TLB flushes Date: Tue, 2 Jul 2019 16:51:42 -0700 Message-Id: <20190702235151.4377-1-namit@vmware.com> X-Mailer: git-send-email 2.17.1 Sender: linux-hyperv-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-hyperv@vger.kernel.org Currently, local and remote TLB flushes are not performed concurrently, which introduces unnecessary overhead - each INVLPG can take 100s of cycles. This patch-set allows TLB flushes to be run concurrently: first request the remote CPUs to initiate the flush, then run it locally, and finally wait for the remote CPUs to finish their work. In addition, there are various small optimizations to avoid unwarranted false-sharing and atomic operations. The proposed changes should also improve the performance of other invocations of on_each_cpu(). Hopefully, no one has relied on this behavior of on_each_cpu() that invoked functions first remotely and only then locally [Peter says he remembers someone might do so, but without further information it is hard to know how to address it]. Running sysbench on dax/ext4 w/emulated-pmem, write-cache disabled on 2-socket, 48-logical-cores (24+SMT) Haswell-X, 5 repetitions: sysbench fileio --file-total-size=3G --file-test-mode=rndwr \ --file-io-mode=mmap --threads=X --file-fsync-mode=fdatasync run Th. tip-jun28 avg (stdev) +patch-set avg (stdev) change --- --------------------- ---------------------- ------ 1 1267765 (14146) 1299253 (5715) +2.4% 2 1734644 (11936) 1799225 (19577) +3.7% 4 2821268 (41184) 2919132 (40149) +3.4% 8 4171652 (31243) 4376925 (65416) +4.9% 16 5590729 (24160) 5829866 (8127) +4.2% 24 6250212 (24481) 6522303 (28044) +4.3% 32 3994314 (26606) 4077543 (10685) +2.0% 48 4345177 (28091) 4417821 (41337) +1.6% (Note that on configurations with up to 24 threads numactl was used to set all threads on socket 1, which explains the drop in performance when going to 32 threads). Running the same benchmark with security mitigations disabled (PTI, Spectre, MDS): Th. tip-jun28 avg (stdev) +patch-set avg (stdev) change --- --------------------- ---------------------- ------ 1 1598896 (5174) 1607903 (4091) +0.5% 2 2109472 (17827) 2224726 (4372) +5.4% 4 3448587 (11952) 3668551 (30219) +6.3% 8 5425778 (29641) 5606266 (33519) +3.3% 16 6931232 (34677) 7054052 (27873) +1.7% 24 7612473 (23482) 7783138 (13871) +2.2% 32 4296274 (18029) 4283279 (32323) -0.3% 48 4770029 (35541) 4764760 (13575) -0.1% Presumably, PTI requires two invalidations of each mapping, which allows to get higher benefits from concurrency when PTI is on. At the same time, when mitigations are on, other overheads reduce the potential speedup. I tried to reduce the size of the code of the main patch, which required restructuring of the series. v1 -> v2: * Removing the patches that Thomas took [tglx] * Adding hyper-v, Xen compile-tested implementations [Dave] * Removing UV [Andy] * Adding lazy optimization, removing inline keyword [Dave] * Restructuring patch-set RFCv2 -> v1: * Fix comment on flush_tlb_multi [Juergen] * Removing async invalidation optimizations [Andy] * Adding KVM support [Paolo] Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Boris Ostrovsky Cc: Dave Hansen Cc: Haiyang Zhang Cc: Ingo Molnar Cc: Josh Poimboeuf Cc: Juergen Gross Cc: "K. Y. Srinivasan" Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Rik van Riel Cc: Sasha Levin Cc: Stephen Hemminger Cc: Thomas Gleixner Cc: kvm@vger.kernel.org Cc: linux-hyperv@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: virtualization@lists.linux-foundation.org Cc: x86@kernel.org Cc: xen-devel@lists.xenproject.org Nadav Amit (9): smp: Run functions concurrently in smp_call_function_many() x86/mm/tlb: Remove reason as argument for flush_tlb_func_local() x86/mm/tlb: Open-code on_each_cpu_cond_mask() for tlb_is_not_lazy() x86/mm/tlb: Flush remote and local TLBs concurrently x86/mm/tlb: Privatize cpu_tlbstate x86/mm/tlb: Do not make is_lazy dirty for no reason cpumask: Mark functions as pure x86/mm/tlb: Remove UV special case x86/mm/tlb: Remove unnecessary uses of the inline keyword arch/x86/hyperv/mmu.c | 13 ++- arch/x86/include/asm/paravirt.h | 6 +- arch/x86/include/asm/paravirt_types.h | 4 +- arch/x86/include/asm/tlbflush.h | 48 +++++---- arch/x86/include/asm/trace/hyperv.h | 2 +- arch/x86/kernel/kvm.c | 11 +- arch/x86/kernel/paravirt.c | 2 +- arch/x86/mm/init.c | 2 +- arch/x86/mm/tlb.c | 147 ++++++++++++++++---------- arch/x86/xen/mmu_pv.c | 20 ++-- include/linux/cpumask.h | 6 +- include/linux/smp.h | 27 +++-- include/trace/events/xen.h | 2 +- kernel/smp.c | 133 +++++++++++------------ 14 files changed, 245 insertions(+), 178 deletions(-) -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=3.0 tests=DATE_IN_PAST_06_12, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBC60C5B578 for ; Wed, 3 Jul 2019 07:14:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DCD22187F for ; Wed, 3 Jul 2019 07:14:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DCD22187F Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=vmware.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiZSz-0006el-DC; Wed, 03 Jul 2019 07:14:09 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiZSx-0006eZ-Rz for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 07:14:07 +0000 X-Inumbo-ID: 230a4250-9d62-11e9-89ec-63de079547d6 Received: from mail-pg1-f193.google.com (unknown [209.85.215.193]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 230a4250-9d62-11e9-89ec-63de079547d6; Wed, 03 Jul 2019 07:14:03 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id g15so729486pgi.4 for ; Wed, 03 Jul 2019 00:14:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=B1kQvrQoHFPsOT1yXoo85QIU6lm9lrhuL5yM9gcXKZ0=; b=r5QDRqq7z6pdTeVbOwIENp9VCFvlbNhi8rZ2HR64fPLudxpEdwKwwXEHGDcebnJ2fC WpkQsq2mKbkzIUH5i2ZUO/XdqHpkR6VUOjay9VVEpYuQXUBt4AFVLbNXD9FEVU8UiL5+ zf+TIAptpUklf7Up64S3LXK4MjmLenEmpIPdSWFoOeU/VAZrOyDtAw6RmScU7A9UXNN1 rXHbSESU93k2qAAZUPmzz3Lj55pJr3eXzxvSz9oC43OE6G45l64arq7sogdoa8WVkGap FNCAgCcgtxVshbW3AuTdmNjsEZVXz3W1JC4Le+GIm7nW8qjuvi5rHls/W65E24luFGzs /fCA== X-Gm-Message-State: APjAAAWGzfmdMiYxumAzPhSawDVQNtgDBfxAtW6vVYoV+J3A1jfg3xFw kcKAwZTT3b/Xb/h9KrcsCAk= X-Google-Smtp-Source: APXvYqwUyamaUZpHybk6nfj2wr4zAu1x6OPqmfFZEMCShOUOWld0YXW8Rb4mD/UbT+YdTI4jAfzbyg== X-Received: by 2002:a17:90a:22ef:: with SMTP id s102mr11149274pjc.2.1562138042451; Wed, 03 Jul 2019 00:14:02 -0700 (PDT) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id j21sm1256593pfh.86.2019.07.03.00.14.00 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 03 Jul 2019 00:14:01 -0700 (PDT) From: Nadav Amit To: Andy Lutomirski , Dave Hansen Date: Tue, 2 Jul 2019 16:51:42 -0700 Message-Id: <20190702235151.4377-1-namit@vmware.com> X-Mailer: git-send-email 2.17.1 Subject: [Xen-devel] [PATCH v2 0/9] x86: Concurrent TLB flushes X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Sasha Levin , linux-hyperv@vger.kernel.org, Stephen Hemminger , kvm@vger.kernel.org, Paolo Bonzini , Rik van Riel , Peter Zijlstra , Haiyang Zhang , x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, Ingo Molnar , Nadav Amit , Josh Poimboeuf , Borislav Petkov , Thomas Gleixner , "K. Y. Srinivasan" , Boris Ostrovsky MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Q3VycmVudGx5LCBsb2NhbCBhbmQgcmVtb3RlIFRMQiBmbHVzaGVzIGFyZSBub3QgcGVyZm9ybWVk IGNvbmN1cnJlbnRseSwKd2hpY2ggaW50cm9kdWNlcyB1bm5lY2Vzc2FyeSBvdmVyaGVhZCAtIGVh Y2ggSU5WTFBHIGNhbiB0YWtlIDEwMHMgb2YKY3ljbGVzLiBUaGlzIHBhdGNoLXNldCBhbGxvd3Mg VExCIGZsdXNoZXMgdG8gYmUgcnVuIGNvbmN1cnJlbnRseTogZmlyc3QKcmVxdWVzdCB0aGUgcmVt b3RlIENQVXMgdG8gaW5pdGlhdGUgdGhlIGZsdXNoLCB0aGVuIHJ1biBpdCBsb2NhbGx5LCBhbmQK ZmluYWxseSB3YWl0IGZvciB0aGUgcmVtb3RlIENQVXMgdG8gZmluaXNoIHRoZWlyIHdvcmsuCgpJ biBhZGRpdGlvbiwgdGhlcmUgYXJlIHZhcmlvdXMgc21hbGwgb3B0aW1pemF0aW9ucyB0byBhdm9p ZCB1bndhcnJhbnRlZApmYWxzZS1zaGFyaW5nIGFuZCBhdG9taWMgb3BlcmF0aW9ucy4KClRoZSBw cm9wb3NlZCBjaGFuZ2VzIHNob3VsZCBhbHNvIGltcHJvdmUgdGhlIHBlcmZvcm1hbmNlIG9mIG90 aGVyCmludm9jYXRpb25zIG9mIG9uX2VhY2hfY3B1KCkuIEhvcGVmdWxseSwgbm8gb25lIGhhcyBy ZWxpZWQgb24gdGhpcwpiZWhhdmlvciBvZiBvbl9lYWNoX2NwdSgpIHRoYXQgaW52b2tlZCBmdW5j dGlvbnMgZmlyc3QgcmVtb3RlbHkgYW5kIG9ubHkKdGhlbiBsb2NhbGx5IFtQZXRlciBzYXlzIGhl IHJlbWVtYmVycyBzb21lb25lIG1pZ2h0IGRvIHNvLCBidXQgd2l0aG91dApmdXJ0aGVyIGluZm9y bWF0aW9uIGl0IGlzIGhhcmQgdG8ga25vdyBob3cgdG8gYWRkcmVzcyBpdF0uCgpSdW5uaW5nIHN5 c2JlbmNoIG9uIGRheC9leHQ0IHcvZW11bGF0ZWQtcG1lbSwgd3JpdGUtY2FjaGUgZGlzYWJsZWQg b24KMi1zb2NrZXQsIDQ4LWxvZ2ljYWwtY29yZXMgKDI0K1NNVCkgSGFzd2VsbC1YLCA1IHJlcGV0 aXRpb25zOgoKIHN5c2JlbmNoIGZpbGVpbyAtLWZpbGUtdG90YWwtc2l6ZT0zRyAtLWZpbGUtdGVz dC1tb2RlPXJuZHdyIFwKICAtLWZpbGUtaW8tbW9kZT1tbWFwIC0tdGhyZWFkcz1YIC0tZmlsZS1m c3luYy1tb2RlPWZkYXRhc3luYyBydW4KCiAgVGguCXRpcC1qdW4yOCBhdmcgKHN0ZGV2KQkrcGF0 Y2gtc2V0IGF2ZyAoc3RkZXYpCWNoYW5nZQogIC0tLQktLS0tLS0tLS0tLS0tLS0tLS0tLS0JLS0t LS0tLS0tLS0tLS0tLS0tLS0tLQktLS0tLS0KICAxCTEyNjc3NjUgKDE0MTQ2KQkJMTI5OTI1MyAo NTcxNSkJCSsyLjQlCiAgMgkxNzM0NjQ0ICgxMTkzNikJCTE3OTkyMjUgKDE5NTc3KQkJKzMuNyUK ICA0CTI4MjEyNjggKDQxMTg0KQkJMjkxOTEzMiAoNDAxNDkpCQkrMy40JQogIDgJNDE3MTY1MiAo MzEyNDMpCQk0Mzc2OTI1ICg2NTQxNikJCSs0LjklCiAgMTYJNTU5MDcyOSAoMjQxNjApCQk1ODI5 ODY2ICg4MTI3KQkJKzQuMiUKICAyNAk2MjUwMjEyICgyNDQ4MSkJCTY1MjIzMDMgKDI4MDQ0KQkJ KzQuMyUKICAzMgkzOTk0MzE0ICgyNjYwNikJCTQwNzc1NDMgKDEwNjg1KQkJKzIuMCUKICA0OAk0 MzQ1MTc3ICgyODA5MSkJCTQ0MTc4MjEgKDQxMzM3KQkJKzEuNiUKCihOb3RlIHRoYXQgb24gY29u ZmlndXJhdGlvbnMgd2l0aCB1cCB0byAyNCB0aHJlYWRzIG51bWFjdGwgd2FzIHVzZWQgdG8Kc2V0 IGFsbCB0aHJlYWRzIG9uIHNvY2tldCAxLCB3aGljaCBleHBsYWlucyB0aGUgZHJvcCBpbiBwZXJm b3JtYW5jZSB3aGVuCmdvaW5nIHRvIDMyIHRocmVhZHMpLgoKUnVubmluZyB0aGUgc2FtZSBiZW5j aG1hcmsgd2l0aCBzZWN1cml0eSBtaXRpZ2F0aW9ucyBkaXNhYmxlZCAoUFRJLApTcGVjdHJlLCBN RFMpOgoKICBUaC4JdGlwLWp1bjI4IGF2ZyAoc3RkZXYpCStwYXRjaC1zZXQgYXZnIChzdGRldikJ Y2hhbmdlCiAgLS0tCS0tLS0tLS0tLS0tLS0tLS0tLS0tLQktLS0tLS0tLS0tLS0tLS0tLS0tLS0t CS0tLS0tLQogIDEJMTU5ODg5NiAoNTE3NCkJCTE2MDc5MDMgKDQwOTEpCQkrMC41JQogIDIJMjEw OTQ3MiAoMTc4MjcpCQkyMjI0NzI2ICg0MzcyKQkJKzUuNCUKICA0CTM0NDg1ODcgKDExOTUyKQkJ MzY2ODU1MSAoMzAyMTkpCQkrNi4zJQogIDgJNTQyNTc3OCAoMjk2NDEpCQk1NjA2MjY2ICgzMzUx OSkJCSszLjMlCiAgMTYJNjkzMTIzMiAoMzQ2NzcpCQk3MDU0MDUyICgyNzg3MykJCSsxLjclCiAg MjQJNzYxMjQ3MyAoMjM0ODIpCQk3NzgzMTM4ICgxMzg3MSkJCSsyLjIlCiAgMzIJNDI5NjI3NCAo MTgwMjkpCQk0MjgzMjc5ICgzMjMyMykJCS0wLjMlCiAgNDgJNDc3MDAyOSAoMzU1NDEpCQk0NzY0 NzYwICgxMzU3NSkJCS0wLjElCgpQcmVzdW1hYmx5LCBQVEkgcmVxdWlyZXMgdHdvIGludmFsaWRh dGlvbnMgb2YgZWFjaCBtYXBwaW5nLCB3aGljaCBhbGxvd3MKdG8gZ2V0IGhpZ2hlciBiZW5lZml0 cyBmcm9tIGNvbmN1cnJlbmN5IHdoZW4gUFRJIGlzIG9uLiBBdCB0aGUgc2FtZQp0aW1lLCB3aGVu IG1pdGlnYXRpb25zIGFyZSBvbiwgb3RoZXIgb3ZlcmhlYWRzIHJlZHVjZSB0aGUgcG90ZW50aWFs CnNwZWVkdXAuCgpJIHRyaWVkIHRvIHJlZHVjZSB0aGUgc2l6ZSBvZiB0aGUgY29kZSBvZiB0aGUg bWFpbiBwYXRjaCwgd2hpY2ggcmVxdWlyZWQKcmVzdHJ1Y3R1cmluZyBvZiB0aGUgc2VyaWVzLgoK djEgLT4gdjI6CiogUmVtb3ZpbmcgdGhlIHBhdGNoZXMgdGhhdCBUaG9tYXMgdG9vayBbdGdseF0K KiBBZGRpbmcgaHlwZXItdiwgWGVuIGNvbXBpbGUtdGVzdGVkIGltcGxlbWVudGF0aW9ucyBbRGF2 ZV0KKiBSZW1vdmluZyBVViBbQW5keV0KKiBBZGRpbmcgbGF6eSBvcHRpbWl6YXRpb24sIHJlbW92 aW5nIGlubGluZSBrZXl3b3JkIFtEYXZlXQoqIFJlc3RydWN0dXJpbmcgcGF0Y2gtc2V0CgpSRkN2 MiAtPiB2MToKKiBGaXggY29tbWVudCBvbiBmbHVzaF90bGJfbXVsdGkgW0p1ZXJnZW5dCiogUmVt b3ZpbmcgYXN5bmMgaW52YWxpZGF0aW9uIG9wdGltaXphdGlvbnMgW0FuZHldCiogQWRkaW5nIEtW TSBzdXBwb3J0IFtQYW9sb10KCkNjOiBBbmR5IEx1dG9taXJza2kgPGx1dG9Aa2VybmVsLm9yZz4K Q2M6IEJvcmlzbGF2IFBldGtvdiA8YnBAYWxpZW44LmRlPgpDYzogQm9yaXMgT3N0cm92c2t5IDxi b3Jpcy5vc3Ryb3Zza3lAb3JhY2xlLmNvbT4KQ2M6IERhdmUgSGFuc2VuIDxkYXZlLmhhbnNlbkBs aW51eC5pbnRlbC5jb20+CkNjOiBIYWl5YW5nIFpoYW5nIDxoYWl5YW5nekBtaWNyb3NvZnQuY29t PgpDYzogSW5nbyBNb2xuYXIgPG1pbmdvQHJlZGhhdC5jb20+CkNjOiBKb3NoIFBvaW1ib2V1ZiA8 anBvaW1ib2VAcmVkaGF0LmNvbT4KQ2M6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT4K Q2M6ICJLLiBZLiBTcmluaXZhc2FuIiA8a3lzQG1pY3Jvc29mdC5jb20+CkNjOiBQYW9sbyBCb256 aW5pIDxwYm9uemluaUByZWRoYXQuY29tPgpDYzogUGV0ZXIgWmlqbHN0cmEgPHBldGVyekBpbmZy YWRlYWQub3JnPgpDYzogUmlrIHZhbiBSaWVsIDxyaWVsQHN1cnJpZWwuY29tPgpDYzogU2FzaGEg TGV2aW4gPHNhc2hhbEBrZXJuZWwub3JnPgpDYzogU3RlcGhlbiBIZW1taW5nZXIgPHN0aGVtbWlu QG1pY3Jvc29mdC5jb20+CkNjOiBUaG9tYXMgR2xlaXhuZXIgPHRnbHhAbGludXRyb25peC5kZT4K Q2M6IGt2bUB2Z2VyLmtlcm5lbC5vcmcKQ2M6IGxpbnV4LWh5cGVydkB2Z2VyLmtlcm5lbC5vcmcK Q2M6IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmcKQ2M6IHZpcnR1YWxpemF0aW9uQGxpc3Rz LmxpbnV4LWZvdW5kYXRpb24ub3JnCkNjOiB4ODZAa2VybmVsLm9yZwpDYzogeGVuLWRldmVsQGxp c3RzLnhlbnByb2plY3Qub3JnCgpOYWRhdiBBbWl0ICg5KToKICBzbXA6IFJ1biBmdW5jdGlvbnMg Y29uY3VycmVudGx5IGluIHNtcF9jYWxsX2Z1bmN0aW9uX21hbnkoKQogIHg4Ni9tbS90bGI6IFJl bW92ZSByZWFzb24gYXMgYXJndW1lbnQgZm9yIGZsdXNoX3RsYl9mdW5jX2xvY2FsKCkKICB4ODYv bW0vdGxiOiBPcGVuLWNvZGUgb25fZWFjaF9jcHVfY29uZF9tYXNrKCkgZm9yIHRsYl9pc19ub3Rf bGF6eSgpCiAgeDg2L21tL3RsYjogRmx1c2ggcmVtb3RlIGFuZCBsb2NhbCBUTEJzIGNvbmN1cnJl bnRseQogIHg4Ni9tbS90bGI6IFByaXZhdGl6ZSBjcHVfdGxic3RhdGUKICB4ODYvbW0vdGxiOiBE byBub3QgbWFrZSBpc19sYXp5IGRpcnR5IGZvciBubyByZWFzb24KICBjcHVtYXNrOiBNYXJrIGZ1 bmN0aW9ucyBhcyBwdXJlCiAgeDg2L21tL3RsYjogUmVtb3ZlIFVWIHNwZWNpYWwgY2FzZQogIHg4 Ni9tbS90bGI6IFJlbW92ZSB1bm5lY2Vzc2FyeSB1c2VzIG9mIHRoZSBpbmxpbmUga2V5d29yZAoK IGFyY2gveDg2L2h5cGVydi9tbXUuYyAgICAgICAgICAgICAgICAgfCAgMTMgKystCiBhcmNoL3g4 Ni9pbmNsdWRlL2FzbS9wYXJhdmlydC5oICAgICAgIHwgICA2ICstCiBhcmNoL3g4Ni9pbmNsdWRl L2FzbS9wYXJhdmlydF90eXBlcy5oIHwgICA0ICstCiBhcmNoL3g4Ni9pbmNsdWRlL2FzbS90bGJm bHVzaC5oICAgICAgIHwgIDQ4ICsrKysrLS0tLQogYXJjaC94ODYvaW5jbHVkZS9hc20vdHJhY2Uv aHlwZXJ2LmggICB8ICAgMiArLQogYXJjaC94ODYva2VybmVsL2t2bS5jICAgICAgICAgICAgICAg ICB8ICAxMSArLQogYXJjaC94ODYva2VybmVsL3BhcmF2aXJ0LmMgICAgICAgICAgICB8ICAgMiAr LQogYXJjaC94ODYvbW0vaW5pdC5jICAgICAgICAgICAgICAgICAgICB8ICAgMiArLQogYXJjaC94 ODYvbW0vdGxiLmMgICAgICAgICAgICAgICAgICAgICB8IDE0NyArKysrKysrKysrKysrKysrLS0t LS0tLS0tLQogYXJjaC94ODYveGVuL21tdV9wdi5jICAgICAgICAgICAgICAgICB8ICAyMCArKy0t CiBpbmNsdWRlL2xpbnV4L2NwdW1hc2suaCAgICAgICAgICAgICAgIHwgICA2ICstCiBpbmNsdWRl L2xpbnV4L3NtcC5oICAgICAgICAgICAgICAgICAgIHwgIDI3ICsrKy0tCiBpbmNsdWRlL3RyYWNl L2V2ZW50cy94ZW4uaCAgICAgICAgICAgIHwgICAyICstCiBrZXJuZWwvc21wLmMgICAgICAgICAg ICAgICAgICAgICAgICAgIHwgMTMzICsrKysrKysrKysrLS0tLS0tLS0tLS0tCiAxNCBmaWxlcyBj aGFuZ2VkLCAyNDUgaW5zZXJ0aW9ucygrKSwgMTc4IGRlbGV0aW9ucygtKQoKLS0gCjIuMTcuMQoK Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClhlbi1kZXZl bCBtYWlsaW5nIGxpc3QKWGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnCmh0dHBzOi8vbGlz dHMueGVucHJvamVjdC5vcmcvbWFpbG1hbi9saXN0aW5mby94ZW4tZGV2ZWw=