From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0AE64C433F5 for ; Thu, 2 Dec 2021 12:39:02 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-348--AUMpFolNISYbO2RIKeU1A-1; Thu, 02 Dec 2021 07:38:58 -0500 X-MC-Unique: -AUMpFolNISYbO2RIKeU1A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C2567801B24; Thu, 2 Dec 2021 12:38:55 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A0B8E19EF9; Thu, 2 Dec 2021 12:38:55 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 71A904BB7C; Thu, 2 Dec 2021 12:38:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1B22PDUY029941 for ; Wed, 1 Dec 2021 21:25:13 -0500 Received: by smtp.corp.redhat.com (Postfix) id 20F6C1402406; Thu, 2 Dec 2021 02:25:13 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast06.extmail.prod.ext.rdu2.redhat.com [10.11.55.22]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1C93C141DED3 for ; Thu, 2 Dec 2021 02:25:13 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ED0E8185A7B4 for ; Thu, 2 Dec 2021 02:25:12 +0000 (UTC) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-424-lrILtUoTMz-5vkYf4JungQ-1; Wed, 01 Dec 2021 21:25:09 -0500 X-MC-Unique: lrILtUoTMz-5vkYf4JungQ-1 Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4J4KY23sCPzcbcW; Thu, 2 Dec 2021 10:24:54 +0800 (CST) Received: from dggpemm500023.china.huawei.com (7.185.36.83) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 2 Dec 2021 10:25:01 +0800 Received: from dggpemm500023.china.huawei.com ([7.185.36.83]) by dggpemm500023.china.huawei.com ([7.185.36.83]) with mapi id 15.01.2308.020; Thu, 2 Dec 2021 10:25:01 +0800 From: "zhaozixuan (C)" To: Paul Moore Subject: Re: [PATCH] audit: accelerate audit rule filter Thread-Topic: [PATCH] audit: accelerate audit rule filter Thread-Index: AdfnI7zBpG+k05BPVEqO6SQrGWmNIA== Date: Thu, 2 Dec 2021 02:25:01 +0000 Message-ID: <10e0cc00a83d4214b7bcfda62c36bb20@huawei.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.174.176.92] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-MIME-Autoconverted: from base64 to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 1B22PDUY029941 X-loop: linux-audit@redhat.com X-Mailman-Approved-At: Thu, 02 Dec 2021 07:37:52 -0500 Cc: "linux-audit@redhat.com" , "linux-kernel@vger.kernel.org" , "eparis@redhat.com" X-BeenThere: linux-audit@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Linux Audit Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=linux-audit-bounces@redhat.com X-Mimecast-Spam-Score: 1 X-Mimecast-Originator: redhat.com Content-Language: zh-CN Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit > On Mon, Nov 29, 2021 at 2:35 AM zhaozixuan (C) wrote: > > >On Tue, Nov 23, 2021 at 2:50 AM Zixuan Zhao wrote: > > >> We used lat_syscall of lmbench3 to test the performance impact of > > >> this patch. We changed the number of rules and run lat_syscall with > > >> 1000 repetitions at each test. Syscalls measured by lat_syscall are > > >> not monitored by rules. > > >> > > >> Before this optimization: > > >> > > >> null read write stat fstat open > > >> 0 rules 1.87ms 2.74ms 2.56ms 26.31ms 4.13ms 69.66ms > > >> 10 rules 2.15ms 3.13ms 3.32ms 26.99ms 4.16ms 74.70ms > > >> 20 rules 2.45ms 3.97ms 3.82ms 27.05ms 4.60ms 76.35ms > > >> 30 rules 2.64ms 4.52ms 3.95ms 30.30ms 4.94ms 78.94ms > > >> 40 rules 2.83ms 4.97ms 4.23ms 32.16ms 5.40ms 81.88ms > > >> 50 rules 3.00ms 5.30ms 4.84ms 33.49ms 5.79ms 83.20ms > > >> 100 rules 4.24ms 9.75ms 7.42ms 37.68ms 6.55ms 93.70ms > > >> 160 rules 5.50ms 16.89ms 12.18ms 51.53ms 17.45ms 155.40ms > > >> > > >> After this optimization: > > >> > > >> null read write stat fstat open > > >> 0 rules 1.81ms 2.84ms 2.42ms 27.70ms 4.15ms 69.10ms > > >> 10 rules 1.97ms 2.83ms 2.69ms 27.70ms 4.15ms 69.30ms > > >> 20 rules 1.72ms 2.91ms 2.41ms 26.49ms 3.91ms 71.19ms > > >> 30 rules 1.85ms 2.94ms 2.48ms 26.27ms 3.97ms 71.43ms > > >> 40 rules 1.88ms 2.94ms 2.78ms 26.85ms 4.08ms 69.79ms > > >> 50 rules 1.86ms 3.17ms 3.08ms 26.25ms 4.03ms 72.32ms > > >> 100 rules 1.84ms 3.00ms 2.81ms 26.25ms 3.98ms 70.25ms > > >> 160 rules 1.92ms 3.32ms 3.06ms 26.81ms 4.57ms 71.41ms > > >> > > >> As the result shown above, the syscall latencies increase as the > > >> number of rules increases, while with the patch the latencies remain stable. > > >> This could help when a user adds many audit rules for purposes > > >> (such as attack tracing or process behavior recording) but suffers > > >> from low performance. > > > > > >I have general concerns about trading memory and complexity for performance gains, but beyond that the numbers you posted above don't yet make sense to me. > > > > Thanks for your reply. > > > > The memory cost of this patch is less than 4KB (1820 bytes on x64 and > > 3640 bytes on compatible x86_64) which is trivial in many cases. > > Besides, syscalls are called frequently on a system so a small > > optimization could bring a good income. > > The tradeoff still exists, even though you feel it is worthwhile. > > > >Why are the latency increases due to rule count not similar across the different syscalls? For example, I would think that if the increase in syscall latency was > >directly attributed to the audit rule processing then the increase on the "open" syscall should be similar to that of the "null" syscall. In other phrasing, if we > >can process 160 rules in ~4ms in the "null" case, why does it take us ~86ms in the "open" case? > > > > As to the test result, we did some investigations and concluded two > > reasons: > > 1. The chosen rule sets were not very suitable. Though they were not > > hit by syscalls being measured, some of them were hit by other > > processes, which reduced the system performance and affected the test > > result; 2. The routine of lat_syscall is much more complicated than we > > thought. It called many other syscalls during the test, which may > > cause the result not to be linear. > > > > Due to the reasons above, we did another test. We modified audit rule > > sets and made sure they wouldn't be hit at runtime. Then, we added > > ktime_get_real_ts64 to auditsc.c to record the time of executing > > __audit_syscall_exit. We ran "stat" syscall 10000 times for each rule > > set and recorded the time interval. The result is shown below: > > > > Before this optimization: > > > > rule set time > > 0 rules 3843.96ns > > 1 rules 13119.08ns > > 10 rules 14003.13ns > > 20 rules 15420.18ns > > 30 rules 17284.84ns > > 40 rules 19010.67ns > > 50 rules 21112.63ns > > 100 rules 25815.02ns > > 130 rules 29447.09ns > > > > After this optimization: > > > > rule set time > > 0 rules 3597.78ns > > 1 rules 13498.73ns > > 10 rules 13122.57ns > > 20 rules 12874.88ns > > 30 rules 14351.99ns > > 40 rules 14181.07ns > > 50 rules 13806.45ns > > 100 rules 13890.85ns > > 130 rules 14441.45ns > > > > As the result showed, the interval is linearly increased before > > optimization while the interval remains stable after optimization. > > Note that audit skips some operations if there are no rules, so there > > is a gap between 0 rule and 1 rule set. > > It looks like a single rule like the one below could effectively disable this optimization, is that correct? > > % auditctl -a exit,always -F uid=1001 > % auditctl -l > -a always,exit -S all -F uid=1001 Yes, rules like this one which monitors all syscalls could disable the optimization. The number of the global array could exponentially increase if we want to handle more audit fields. However, we don't that kind of rule is practical because they might generate a great number of logs and even lead to log loss. -- Linux-audit mailing list Linux-audit@redhat.com https://listman.redhat.com/mailman/listinfo/linux-audit From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 313A4C433F5 for ; Thu, 2 Dec 2021 02:26:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354892AbhLBC3Q (ORCPT ); Wed, 1 Dec 2021 21:29:16 -0500 Received: from szxga01-in.huawei.com ([45.249.212.187]:32864 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354811AbhLBC22 (ORCPT ); Wed, 1 Dec 2021 21:28:28 -0500 Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4J4KY23sCPzcbcW; Thu, 2 Dec 2021 10:24:54 +0800 (CST) Received: from dggpemm500023.china.huawei.com (7.185.36.83) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 2 Dec 2021 10:25:01 +0800 Received: from dggpemm500023.china.huawei.com ([7.185.36.83]) by dggpemm500023.china.huawei.com ([7.185.36.83]) with mapi id 15.01.2308.020; Thu, 2 Dec 2021 10:25:01 +0800 From: "zhaozixuan (C)" To: Paul Moore CC: "eparis@redhat.com" , "linux-audit@redhat.com" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH] audit: accelerate audit rule filter Thread-Topic: [PATCH] audit: accelerate audit rule filter Thread-Index: AdfnI7zBpG+k05BPVEqO6SQrGWmNIA== Date: Thu, 2 Dec 2021 02:25:01 +0000 Message-ID: <10e0cc00a83d4214b7bcfda62c36bb20@huawei.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.174.176.92] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org PiAgT24gTW9uLCBOb3YgMjksIDIwMjEgYXQgMjozNSBBTSB6aGFveml4dWFuIChDKSA8emhhb3pp eHVhbjJAaHVhd2VpLmNvbT4gd3JvdGU6DQo+ID4gPk9uIFR1ZSwgTm92IDIzLCAyMDIxIGF0IDI6 NTAgQU0gWml4dWFuIFpoYW8gPHpoYW96aXh1YW4yQGh1YXdlaS5jb20+IHdyb3RlOg0KPiA+ID4+ IFdlIHVzZWQgbGF0X3N5c2NhbGwgb2YgbG1iZW5jaDMgdG8gdGVzdCB0aGUgcGVyZm9ybWFuY2Ug aW1wYWN0IG9mIA0KPiA+ID4+IHRoaXMgcGF0Y2guIFdlIGNoYW5nZWQgdGhlIG51bWJlciBvZiBy dWxlcyBhbmQgcnVuIGxhdF9zeXNjYWxsIHdpdGggDQo+ID4gPj4gMTAwMCByZXBldGl0aW9ucyBh dCBlYWNoIHRlc3QuIFN5c2NhbGxzIG1lYXN1cmVkIGJ5IGxhdF9zeXNjYWxsIGFyZSANCj4gPiA+ PiBub3QgbW9uaXRvcmVkIGJ5IHJ1bGVzLg0KPiA+ID4+DQo+ID4gPj4gQmVmb3JlIHRoaXMgb3B0 aW1pemF0aW9uOg0KPiA+ID4+DQo+ID4gPj4gICAgICAgICAgICAgIG51bGwgICAgIHJlYWQgICAg d3JpdGUgICAgIHN0YXQgICAgZnN0YXQgICAgICBvcGVuDQo+ID4gPj4gICAwIHJ1bGVzICAxLjg3 bXMgICAyLjc0bXMgICAyLjU2bXMgICAyNi4zMW1zICA0LjEzbXMgICA2OS42Nm1zDQo+ID4gPj4g IDEwIHJ1bGVzICAyLjE1bXMgICAzLjEzbXMgICAzLjMybXMgICAyNi45OW1zICA0LjE2bXMgICA3 NC43MG1zDQo+ID4gPj4gIDIwIHJ1bGVzICAyLjQ1bXMgICAzLjk3bXMgICAzLjgybXMgICAyNy4w NW1zICA0LjYwbXMgICA3Ni4zNW1zDQo+ID4gPj4gIDMwIHJ1bGVzICAyLjY0bXMgICA0LjUybXMg ICAzLjk1bXMgICAzMC4zMG1zICA0Ljk0bXMgICA3OC45NG1zDQo+ID4gPj4gIDQwIHJ1bGVzICAy LjgzbXMgICA0Ljk3bXMgICA0LjIzbXMgICAzMi4xNm1zICA1LjQwbXMgICA4MS44OG1zDQo+ID4g Pj4gIDUwIHJ1bGVzICAzLjAwbXMgICA1LjMwbXMgICA0Ljg0bXMgICAzMy40OW1zICA1Ljc5bXMg ICA4My4yMG1zDQo+ID4gPj4gMTAwIHJ1bGVzICA0LjI0bXMgICA5Ljc1bXMgICA3LjQybXMgICAz Ny42OG1zICA2LjU1bXMgICA5My43MG1zDQo+ID4gPj4gMTYwIHJ1bGVzICA1LjUwbXMgICAxNi44 OW1zICAxMi4xOG1zICA1MS41M21zICAxNy40NW1zICAxNTUuNDBtcw0KPiA+ID4+DQo+ID4gPj4g QWZ0ZXIgdGhpcyBvcHRpbWl6YXRpb246DQo+ID4gPj4NCj4gPiA+PiAgICAgICAgICAgICAgbnVs bCAgICAgcmVhZCAgICB3cml0ZSAgICAgc3RhdCAgICBmc3RhdCAgICAgIG9wZW4NCj4gPiA+PiAg IDAgcnVsZXMgIDEuODFtcyAgIDIuODRtcyAgIDIuNDJtcyAgMjcuNzBtcyAgIDQuMTVtcyAgIDY5 LjEwbXMNCj4gPiA+PiAgMTAgcnVsZXMgIDEuOTdtcyAgIDIuODNtcyAgIDIuNjltcyAgMjcuNzBt cyAgIDQuMTVtcyAgIDY5LjMwbXMNCj4gPiA+PiAgMjAgcnVsZXMgIDEuNzJtcyAgIDIuOTFtcyAg IDIuNDFtcyAgMjYuNDltcyAgIDMuOTFtcyAgIDcxLjE5bXMNCj4gPiA+PiAgMzAgcnVsZXMgIDEu ODVtcyAgIDIuOTRtcyAgIDIuNDhtcyAgMjYuMjdtcyAgIDMuOTdtcyAgIDcxLjQzbXMNCj4gPiA+ PiAgNDAgcnVsZXMgIDEuODhtcyAgIDIuOTRtcyAgIDIuNzhtcyAgMjYuODVtcyAgIDQuMDhtcyAg IDY5Ljc5bXMNCj4gPiA+PiAgNTAgcnVsZXMgIDEuODZtcyAgIDMuMTdtcyAgIDMuMDhtcyAgMjYu MjVtcyAgIDQuMDNtcyAgIDcyLjMybXMNCj4gPiA+PiAxMDAgcnVsZXMgIDEuODRtcyAgIDMuMDBt cyAgIDIuODFtcyAgMjYuMjVtcyAgIDMuOThtcyAgIDcwLjI1bXMNCj4gPiA+PiAxNjAgcnVsZXMg IDEuOTJtcyAgIDMuMzJtcyAgIDMuMDZtcyAgMjYuODFtcyAgIDQuNTdtcyAgIDcxLjQxbXMNCj4g PiA+Pg0KPiA+ID4+IEFzIHRoZSByZXN1bHQgc2hvd24gYWJvdmUsIHRoZSBzeXNjYWxsIGxhdGVu Y2llcyBpbmNyZWFzZSBhcyAgdGhlIA0KPiA+ID4+IG51bWJlciAgb2YgcnVsZXMgaW5jcmVhc2Vz LCB3aGlsZSB3aXRoIHRoZSBwYXRjaCB0aGUgbGF0ZW5jaWVzIHJlbWFpbiBzdGFibGUuDQo+ID4g Pj4gIFRoaXMgY291bGQgaGVscCB3aGVuIGEgdXNlciBhZGRzIG1hbnkgYXVkaXQgcnVsZXMgZm9y IHB1cnBvc2VzIA0KPiA+ID4+IChzdWNoIGFzICBhdHRhY2sgdHJhY2luZyBvciBwcm9jZXNzIGJl aGF2aW9yIHJlY29yZGluZykgYnV0IHN1ZmZlcnMgDQo+ID4gPj4gZnJvbSBsb3cgcGVyZm9ybWFu Y2UuDQo+ID4gPg0KPiA+ID5JIGhhdmUgZ2VuZXJhbCBjb25jZXJucyBhYm91dCB0cmFkaW5nIG1l bW9yeSBhbmQgY29tcGxleGl0eSBmb3IgcGVyZm9ybWFuY2UgZ2FpbnMsIGJ1dCBiZXlvbmQgdGhh dCB0aGUgbnVtYmVycyB5b3UgcG9zdGVkIGFib3ZlIGRvbid0IHlldCBtYWtlIHNlbnNlIHRvIG1l Lg0KPiA+DQo+ID4gVGhhbmtzIGZvciB5b3VyIHJlcGx5Lg0KPiA+DQo+ID4gVGhlIG1lbW9yeSBj b3N0IG9mIHRoaXMgcGF0Y2ggaXMgbGVzcyB0aGFuIDRLQiAoMTgyMCBieXRlcyBvbiB4NjQgYW5k DQo+ID4gIDM2NDAgYnl0ZXMgb24gY29tcGF0aWJsZSB4ODZfNjQpIHdoaWNoIGlzIHRyaXZpYWwg aW4gbWFueSBjYXNlcy4NCj4gPiAgQmVzaWRlcywgc3lzY2FsbHMgYXJlIGNhbGxlZCBmcmVxdWVu dGx5IG9uIGEgc3lzdGVtIHNvIGEgc21hbGwgIA0KPiA+IG9wdGltaXphdGlvbiBjb3VsZCBicmlu ZyBhIGdvb2QgaW5jb21lLg0KPiANCj4gVGhlIHRyYWRlb2ZmIHN0aWxsIGV4aXN0cywgZXZlbiB0 aG91Z2ggeW91IGZlZWwgaXQgaXMgd29ydGh3aGlsZS4NCj4gDQo+ID4gPldoeSBhcmUgdGhlIGxh dGVuY3kgaW5jcmVhc2VzIGR1ZSB0byBydWxlIGNvdW50IG5vdCBzaW1pbGFyIGFjcm9zcyB0aGUg ZGlmZmVyZW50IHN5c2NhbGxzPyBGb3IgZXhhbXBsZSwgSSB3b3VsZCB0aGluayB0aGF0IGlmIHRo ZSBpbmNyZWFzZSBpbiBzeXNjYWxsIGxhdGVuY3kgd2FzID4gPmRpcmVjdGx5IGF0dHJpYnV0ZWQg dG8gdGhlIGF1ZGl0IHJ1bGUgcHJvY2Vzc2luZyB0aGVuIHRoZSBpbmNyZWFzZSBvbiB0aGUgIm9w ZW4iIHN5c2NhbGwgc2hvdWxkIGJlIHNpbWlsYXIgdG8gdGhhdCBvZiB0aGUgIm51bGwiIHN5c2Nh bGwuICBJbiBvdGhlciBwaHJhc2luZywgaWYgd2UgPiA+Y2FuIHByb2Nlc3MgMTYwIHJ1bGVzIGlu IH40bXMgaW4gdGhlICJudWxsIiBjYXNlLCB3aHkgZG9lcyBpdCB0YWtlIHVzIH44Nm1zIGluIHRo ZSAib3BlbiIgY2FzZT8NCj4gPg0KPiA+IEFzIHRvIHRoZSB0ZXN0IHJlc3VsdCwgd2UgZGlkIHNv bWUgaW52ZXN0aWdhdGlvbnMgYW5kIGNvbmNsdWRlZCB0d28NCj4gPiAgcmVhc29uczoNCj4gPiAx LiBUaGUgY2hvc2VuIHJ1bGUgc2V0cyB3ZXJlIG5vdCB2ZXJ5IHN1aXRhYmxlLiBUaG91Z2ggdGhl eSB3ZXJlIG5vdCANCj4gPiBoaXQgIGJ5IHN5c2NhbGxzIGJlaW5nIG1lYXN1cmVkLCBzb21lIG9m IHRoZW0gd2VyZSBoaXQgYnkgb3RoZXIgDQo+ID4gcHJvY2Vzc2VzLCAgd2hpY2ggcmVkdWNlZCB0 aGUgc3lzdGVtIHBlcmZvcm1hbmNlIGFuZCBhZmZlY3RlZCB0aGUgdGVzdCANCj4gPiByZXN1bHQ7 IDIuIFRoZSByb3V0aW5lIG9mIGxhdF9zeXNjYWxsIGlzIG11Y2ggbW9yZSBjb21wbGljYXRlZCB0 aGFuIHdlIA0KPiA+IHRob3VnaHQuIEl0ICBjYWxsZWQgbWFueSBvdGhlciBzeXNjYWxscyBkdXJp bmcgdGhlIHRlc3QsIHdoaWNoIG1heSANCj4gPiBjYXVzZSB0aGUgcmVzdWx0ICBub3QgdG8gYmUg bGluZWFyLg0KPiA+DQo+ID4gRHVlIHRvIHRoZSByZWFzb25zIGFib3ZlLCB3ZSBkaWQgYW5vdGhl ciB0ZXN0LiBXZSBtb2RpZmllZCBhdWRpdCBydWxlIA0KPiA+IHNldHMgIGFuZCBtYWRlIHN1cmUg dGhleSB3b3VsZG4ndCBiZSBoaXQgYXQgcnVudGltZS4gVGhlbiwgd2UgYWRkZWQNCj4gPiAga3Rp bWVfZ2V0X3JlYWxfdHM2NCB0byBhdWRpdHNjLmMgdG8gcmVjb3JkIHRoZSB0aW1lIG9mIGV4ZWN1 dGluZyAgDQo+ID4gX19hdWRpdF9zeXNjYWxsX2V4aXQuIFdlIHJhbiAic3RhdCIgc3lzY2FsbCAx MDAwMCB0aW1lcyBmb3IgZWFjaCBydWxlIA0KPiA+IHNldCAgYW5kIHJlY29yZGVkIHRoZSB0aW1l IGludGVydmFsLiBUaGUgcmVzdWx0IGlzIHNob3duIGJlbG93Og0KPiA+DQo+ID4gQmVmb3JlIHRo aXMgb3B0aW1pemF0aW9uOg0KPiA+DQo+ID4gcnVsZSBzZXQgICAgICAgICAgdGltZQ0KPiA+ICAg MCBydWxlcyAgICAgMzg0My45Nm5zDQo+ID4gICAxIHJ1bGVzICAgIDEzMTE5LjA4bnMNCj4gPiAg MTAgcnVsZXMgICAgMTQwMDMuMTNucw0KPiA+ICAyMCBydWxlcyAgICAxNTQyMC4xOG5zDQo+ID4g IDMwIHJ1bGVzICAgIDE3Mjg0Ljg0bnMNCj4gPiAgNDAgcnVsZXMgICAgMTkwMTAuNjducw0KPiA+ ICA1MCBydWxlcyAgICAyMTExMi42M25zDQo+ID4gMTAwIHJ1bGVzICAgIDI1ODE1LjAybnMNCj4g PiAxMzAgcnVsZXMgICAgMjk0NDcuMDlucw0KPiA+DQo+ID4gQWZ0ZXIgdGhpcyBvcHRpbWl6YXRp b246DQo+ID4NCj4gPiAgcnVsZSBzZXQgICAgICAgICAgdGltZQ0KPiA+ICAgMCBydWxlcyAgICAg MzU5Ny43OG5zDQo+ID4gICAxIHJ1bGVzICAgIDEzNDk4LjczbnMNCj4gPiAgMTAgcnVsZXMgICAg MTMxMjIuNTducw0KPiA+ICAyMCBydWxlcyAgICAxMjg3NC44OG5zDQo+ID4gIDMwIHJ1bGVzICAg IDE0MzUxLjk5bnMNCj4gPiAgNDAgcnVsZXMgICAgMTQxODEuMDducw0KPiA+ICA1MCBydWxlcyAg ICAxMzgwNi40NW5zDQo+ID4gMTAwIHJ1bGVzICAgIDEzODkwLjg1bnMNCj4gPiAxMzAgcnVsZXMg ICAgMTQ0NDEuNDVucw0KPiA+DQo+ID4gQXMgdGhlIHJlc3VsdCBzaG93ZWQsIHRoZSBpbnRlcnZh bCBpcyBsaW5lYXJseSBpbmNyZWFzZWQgYmVmb3JlICANCj4gPiBvcHRpbWl6YXRpb24gd2hpbGUg dGhlIGludGVydmFsIHJlbWFpbnMgc3RhYmxlIGFmdGVyIG9wdGltaXphdGlvbi4gDQo+ID4gTm90 ZSAgdGhhdCBhdWRpdCBza2lwcyBzb21lIG9wZXJhdGlvbnMgaWYgdGhlcmUgYXJlIG5vIHJ1bGVz LCBzbyB0aGVyZSANCj4gPiBpcyBhIGdhcCAgYmV0d2VlbiAwIHJ1bGUgYW5kIDEgcnVsZSBzZXQu DQo+IA0KPiBJdCBsb29rcyBsaWtlIGEgc2luZ2xlIHJ1bGUgbGlrZSB0aGUgb25lIGJlbG93IGNv dWxkIGVmZmVjdGl2ZWx5IGRpc2FibGUgdGhpcyBvcHRpbWl6YXRpb24sIGlzIHRoYXQgY29ycmVj dD8NCj4gDQo+ICAgJSBhdWRpdGN0bCAtYSBleGl0LGFsd2F5cyAtRiB1aWQ9MTAwMQ0KPiAgICUg YXVkaXRjdGwgLWwNCj4gICAtYSBhbHdheXMsZXhpdCAtUyBhbGwgLUYgdWlkPTEwMDENCg0KWWVz LCBydWxlcyBsaWtlIHRoaXMgb25lIHdoaWNoIG1vbml0b3JzIGFsbCBzeXNjYWxscyBjb3VsZCBk aXNhYmxlIHRoZQ0KIG9wdGltaXphdGlvbi4gVGhlIG51bWJlciBvZiB0aGUgZ2xvYmFsIGFycmF5 IGNvdWxkIGV4cG9uZW50aWFsbHkgaW5jcmVhc2UNCiBpZiB3ZSB3YW50IHRvIGhhbmRsZSBtb3Jl IGF1ZGl0IGZpZWxkcy4gSG93ZXZlciwgd2UgZG9uJ3QgdGhhdCBraW5kIG9mDQogcnVsZSBpcyBw cmFjdGljYWwgYmVjYXVzZSB0aGV5IG1pZ2h0IGdlbmVyYXRlIGEgZ3JlYXQgbnVtYmVyIG9mIGxv Z3MgYW5kDQogZXZlbiBsZWFkIHRvIGxvZyBsb3NzLg0K