From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7E6FC388F7 for ; Sat, 31 Oct 2020 09:45:10 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01FC72076D for ; Sat, 31 Oct 2020 09:45:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01FC72076D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 902B58716A; Sat, 31 Oct 2020 09:45:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bV7G94ZLKaqg; Sat, 31 Oct 2020 09:45:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id A070A87169; Sat, 31 Oct 2020 09:45:07 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9558CC088B; Sat, 31 Oct 2020 09:45:07 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3FCAAC0051 for ; Sat, 31 Oct 2020 09:45:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 2439D87730 for ; Sat, 31 Oct 2020 09:45:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aUvNDa-PSa67 for ; Sat, 31 Oct 2020 09:45:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3D1398772F for ; Sat, 31 Oct 2020 09:45:02 +0000 (UTC) Received: from DGGEMM403-HUB.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4CNZ636BKXzQm6n; Sat, 31 Oct 2020 17:44:59 +0800 (CST) Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by DGGEMM403-HUB.china.huawei.com (10.3.20.211) with Microsoft SMTP Server (TLS) id 14.3.487.0; Sat, 31 Oct 2020 17:44:58 +0800 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by dggemi761-chm.china.huawei.com (10.1.198.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Sat, 31 Oct 2020 17:44:58 +0800 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.1913.007; Sat, 31 Oct 2020 17:44:58 +0800 From: "Song Bao Hua (Barry Song)" To: Robin Murphy , "iommu@lists.linux-foundation.org" , "hch@lst.de" , "m.szyprowski@samsung.com" Subject: RE: [PATCH 1/2] dma-mapping: add benchmark support for streaming DMA APIs Thread-Topic: [PATCH 1/2] dma-mapping: add benchmark support for streaming DMA APIs Thread-Index: AQHWrBVIy05LyAA+tkyd0F6o0ziCMamud8IAgACcg7CAALWOgIABrebg Date: Sat, 31 Oct 2020 09:44:58 +0000 Message-ID: References: <20201027035330.29612-1-song.bao.hua@hisilicon.com> <20201027035330.29612-2-song.bao.hua@hisilicon.com> <8fe90795064d4373b4af32959c4e9781@hisilicon.com> <472cf21a-5196-dbb5-caef-c1c0d982fe1c@arm.com> In-Reply-To: <472cf21a-5196-dbb5-caef-c1c0d982fe1c@arm.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.203.244] MIME-Version: 1.0 X-CFilter-Loop: Reflected Cc: "shuah@kernel.org" , "will@kernel.org" , Linuxarm , "linux-kselftest@vger.kernel.org" X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" > -----Original Message----- > From: Robin Murphy [mailto:robin.murphy@arm.com] > Sent: Saturday, October 31, 2020 4:48 AM > To: Song Bao Hua (Barry Song) ; > iommu@lists.linux-foundation.org; hch@lst.de; m.szyprowski@samsung.com > Cc: joro@8bytes.org; will@kernel.org; shuah@kernel.org; Linuxarm > ; linux-kselftest@vger.kernel.org > Subject: Re: [PATCH 1/2] dma-mapping: add benchmark support for streaming > DMA APIs > > On 2020-10-29 21:39, Song Bao Hua (Barry Song) wrote: > [...] > >>> +struct map_benchmark { > >>> + __u64 map_nsec; > >>> + __u64 unmap_nsec; > >>> + __u32 threads; /* how many threads will do map/unmap in parallel > */ > >>> + __u32 seconds; /* how long the test will last */ > >>> + int node; /* which numa node this benchmark will run on */ > >>> + __u64 expansion[10]; /* For future use */ > >>> +}; > >> > >> I'm no expert on userspace ABIs (and what little experience I do have > >> is mostly of Win32...), so hopefully someone else will comment if > >> there's anything of concern here. One thing I wonder is that there's > >> a fair likelihood of functionality evolving here over time, so might > >> it be appropriate to have some sort of explicit versioning parameter > >> for robustness? > > > > I copied that from gup_benchmark. There is no this kind of code to > > compare version. > > I believe there is a likelihood that kernel module is changed but > > users are still using old userspace tool, this might lead to the > > incompatible data structure. > > But not sure if it is a big problem :-) > > Yeah, like I say I don't really have a good feeling for what would be best here, > I'm just thinking of what I do know and wary of the potential for a "640 bits > ought to be enough for anyone" issue ;) > > >>> +struct map_benchmark_data { > >>> + struct map_benchmark bparam; > >>> + struct device *dev; > >>> + struct dentry *debugfs; > >>> + atomic64_t total_map_nsecs; > >>> + atomic64_t total_map_loops; > >>> + atomic64_t total_unmap_nsecs; > >>> + atomic64_t total_unmap_loops; > >>> +}; > >>> + > >>> +static int map_benchmark_thread(void *data) { > >>> + struct page *page; > >>> + dma_addr_t dma_addr; > >>> + struct map_benchmark_data *map = data; > >>> + int ret = 0; > >>> + > >>> + page = alloc_page(GFP_KERNEL); > >>> + if (!page) > >>> + return -ENOMEM; > >>> + > >>> + while (!kthread_should_stop()) { > >>> + ktime_t map_stime, map_etime, unmap_stime, unmap_etime; > >>> + > >>> + map_stime = ktime_get(); > >>> + dma_addr = dma_map_page(map->dev, page, 0, PAGE_SIZE, > >> DMA_BIDIRECTIONAL); > >> > >> Note that for a non-coherent device, this will give an underestimate > >> of the real-world overhead of BIDIRECTIONAL or TO_DEVICE mappings, > >> since the page will never be dirty in the cache (except possibly the > >> very first time through). > > > > Agreed. I'd like to add a DIRECTION parameter like "-d 0", "-d 1" > > after we have this basic framework. > > That wasn't so much about the direction itself, just that if it's anything other > than FROM_DEVICE, we should probably do something to dirty the buffer by a > reasonable amount before each map. Otherwise the measured performance is > going to be unrealistic on many systems. Maybe put a memset(buf, 0, PAGE_SIZE) before dma_map will help ? > > [...] > >>> + atomic64_add((long long)ktime_to_ns(ktime_sub(unmap_etime, > >> unmap_stime)), > >>> + &map->total_unmap_nsecs); > >>> + atomic64_inc(&map->total_map_loops); > >>> + atomic64_inc(&map->total_unmap_loops); > >> > >> I think it would be worth keeping track of the variances as well - it > >> can be hard to tell if a reasonable-looking average is hiding > >> terrible worst-case behaviour. > > > > This is a sensible requirement. I believe it is better to be handled > > by the existing kernel tracing method. > > > > Maybe we need a histogram like: > > Delay sample count > > 1-2us 1000 *** > > 2-3us 2000 ******* > > 3-4us 100 * > > ..... > > This will be more precise than the maximum latency in the worst case. > > > > I'd believe this can be handled by: > > tracepoint A > > Map > > Tracepoint B > > > > Tracepoint C > > Unmap > > Tracepoint D > > > > Let the userspace ebpf to draw the histogram for the delta of B-A and D-C. > > > > So I am planning to put this requirement into todo list and write an > > userspace ebpf/bcc script for histogram and put in tools/ directory. > > > > Please give your comments on this. > > Right, I wasn't suggesting trying to homebrew a full data collection system here > - I agree there are better tools for that already - just that it's basically free to > track a sum of squares alongside a sum, so that we can trivially calculate a > useful variance (or standard > deviation) figure alongside the mean at the end. For this case, I am not sure if it is true. Unless we expose more data such as min, max etc. to userspace, it makes no difference whether total_(un)map_nsecs and total_(un)map_loops are exposed or not. As total loops = seconds / (avg_map_latency + avg_unmap_latency); total_map_nsecs = total loop count * avg_map_latency total_unmap_nsecs = total loop count * avg_unmap_latency all of seconds, avg_unmap_latency, avg_unmap_latency are known by userspace tool. > > [...] > >>> + for (i = 0; i < threads; i++) { > >>> + tsk[i] = kthread_create_on_node(map_benchmark_thread, map, > >>> + map->bparam.node, "dma-map-benchmark/%d", i); > >>> + if (IS_ERR(tsk[i])) { > >>> + dev_err(map->dev, "create dma_map thread failed\n"); > >>> + return PTR_ERR(tsk[i]); > >>> + } > >>> + > >>> + if (node != NUMA_NO_NODE && node_online(node)) > >>> + kthread_bind_mask(tsk[i], cpu_mask); > >>> + > >>> + wake_up_process(tsk[i]); > >> > >> Might it be better to create all the threads first, *then* start > >> kicking them? > > > > The difficulty is that we don't know how many threads we should create > > as the thread number is a parameter to test the contention of IOMMU driver. > > In my test case, I'd like to test things like One thread Two threads > > .... > > 8 threads > > 12 threads > > 16 threads... > > > > On the other hand, I think it is better to drop the memory of > > task_struct of those test threads while we are not testing dma map. > > I simply meant splitting the loop here into two - one to create the threads and > set their affinity, then another to wake them all up - so we don't start > unnecessarily thrashing the system while we're still trying to set up the rest of > the test ;) Agreed. > > Robin. Thanks Barry _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12742C388F7 for ; Sat, 31 Oct 2020 09:45:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BFFA42076D for ; Sat, 31 Oct 2020 09:45:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726589AbgJaJpC (ORCPT ); Sat, 31 Oct 2020 05:45:02 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:2485 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726451AbgJaJpB (ORCPT ); Sat, 31 Oct 2020 05:45:01 -0400 Received: from DGGEMM403-HUB.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4CNZ636BKXzQm6n; Sat, 31 Oct 2020 17:44:59 +0800 (CST) Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by DGGEMM403-HUB.china.huawei.com (10.3.20.211) with Microsoft SMTP Server (TLS) id 14.3.487.0; Sat, 31 Oct 2020 17:44:58 +0800 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by dggemi761-chm.china.huawei.com (10.1.198.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Sat, 31 Oct 2020 17:44:58 +0800 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.1913.007; Sat, 31 Oct 2020 17:44:58 +0800 From: "Song Bao Hua (Barry Song)" To: Robin Murphy , "iommu@lists.linux-foundation.org" , "hch@lst.de" , "m.szyprowski@samsung.com" CC: "joro@8bytes.org" , "will@kernel.org" , "shuah@kernel.org" , Linuxarm , "linux-kselftest@vger.kernel.org" Subject: RE: [PATCH 1/2] dma-mapping: add benchmark support for streaming DMA APIs Thread-Topic: [PATCH 1/2] dma-mapping: add benchmark support for streaming DMA APIs Thread-Index: AQHWrBVIy05LyAA+tkyd0F6o0ziCMamud8IAgACcg7CAALWOgIABrebg Date: Sat, 31 Oct 2020 09:44:58 +0000 Message-ID: References: <20201027035330.29612-1-song.bao.hua@hisilicon.com> <20201027035330.29612-2-song.bao.hua@hisilicon.com> <8fe90795064d4373b4af32959c4e9781@hisilicon.com> <472cf21a-5196-dbb5-caef-c1c0d982fe1c@arm.com> In-Reply-To: <472cf21a-5196-dbb5-caef-c1c0d982fe1c@arm.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.203.244] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogUm9iaW4gTXVycGh5IFtt YWlsdG86cm9iaW4ubXVycGh5QGFybS5jb21dDQo+IFNlbnQ6IFNhdHVyZGF5LCBPY3RvYmVyIDMx LCAyMDIwIDQ6NDggQU0NCj4gVG86IFNvbmcgQmFvIEh1YSAoQmFycnkgU29uZykgPHNvbmcuYmFv Lmh1YUBoaXNpbGljb24uY29tPjsNCj4gaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmc7 IGhjaEBsc3QuZGU7IG0uc3p5cHJvd3NraUBzYW1zdW5nLmNvbQ0KPiBDYzogam9yb0A4Ynl0ZXMu b3JnOyB3aWxsQGtlcm5lbC5vcmc7IHNodWFoQGtlcm5lbC5vcmc7IExpbnV4YXJtDQo+IDxsaW51 eGFybUBodWF3ZWkuY29tPjsgbGludXgta3NlbGZ0ZXN0QHZnZXIua2VybmVsLm9yZw0KPiBTdWJq ZWN0OiBSZTogW1BBVENIIDEvMl0gZG1hLW1hcHBpbmc6IGFkZCBiZW5jaG1hcmsgc3VwcG9ydCBm b3Igc3RyZWFtaW5nDQo+IERNQSBBUElzDQo+IA0KPiBPbiAyMDIwLTEwLTI5IDIxOjM5LCBTb25n IEJhbyBIdWEgKEJhcnJ5IFNvbmcpIHdyb3RlOg0KPiBbLi4uXQ0KPiA+Pj4gK3N0cnVjdCBtYXBf YmVuY2htYXJrIHsNCj4gPj4+ICsJX191NjQgbWFwX25zZWM7DQo+ID4+PiArCV9fdTY0IHVubWFw X25zZWM7DQo+ID4+PiArCV9fdTMyIHRocmVhZHM7IC8qIGhvdyBtYW55IHRocmVhZHMgd2lsbCBk byBtYXAvdW5tYXAgaW4gcGFyYWxsZWwNCj4gKi8NCj4gPj4+ICsJX191MzIgc2Vjb25kczsgLyog aG93IGxvbmcgdGhlIHRlc3Qgd2lsbCBsYXN0ICovDQo+ID4+PiArCWludCBub2RlOyAvKiB3aGlj aCBudW1hIG5vZGUgdGhpcyBiZW5jaG1hcmsgd2lsbCBydW4gb24gKi8NCj4gPj4+ICsJX191NjQg ZXhwYW5zaW9uWzEwXTsJLyogRm9yIGZ1dHVyZSB1c2UgKi8NCj4gPj4+ICt9Ow0KPiA+Pg0KPiA+ PiBJJ20gbm8gZXhwZXJ0IG9uIHVzZXJzcGFjZSBBQklzIChhbmQgd2hhdCBsaXR0bGUgZXhwZXJp ZW5jZSBJIGRvIGhhdmUNCj4gPj4gaXMgbW9zdGx5IG9mIFdpbjMyLi4uKSwgc28gaG9wZWZ1bGx5 IHNvbWVvbmUgZWxzZSB3aWxsIGNvbW1lbnQgaWYNCj4gPj4gdGhlcmUncyBhbnl0aGluZyBvZiBj b25jZXJuIGhlcmUuIE9uZSB0aGluZyBJIHdvbmRlciBpcyB0aGF0IHRoZXJlJ3MNCj4gPj4gYSBm YWlyIGxpa2VsaWhvb2Qgb2YgZnVuY3Rpb25hbGl0eSBldm9sdmluZyBoZXJlIG92ZXIgdGltZSwg c28gbWlnaHQNCj4gPj4gaXQgYmUgYXBwcm9wcmlhdGUgdG8gaGF2ZSBzb21lIHNvcnQgb2YgZXhw bGljaXQgdmVyc2lvbmluZyBwYXJhbWV0ZXINCj4gPj4gZm9yIHJvYnVzdG5lc3M/DQo+ID4NCj4g PiBJIGNvcGllZCB0aGF0IGZyb20gZ3VwX2JlbmNobWFyay4gVGhlcmUgaXMgbm8gdGhpcyBraW5k IG9mIGNvZGUgdG8NCj4gPiBjb21wYXJlIHZlcnNpb24uDQo+ID4gSSBiZWxpZXZlIHRoZXJlIGlz IGEgbGlrZWxpaG9vZCB0aGF0IGtlcm5lbCBtb2R1bGUgaXMgY2hhbmdlZCBidXQNCj4gPiB1c2Vy cyBhcmUgc3RpbGwgdXNpbmcgb2xkIHVzZXJzcGFjZSB0b29sLCB0aGlzIG1pZ2h0IGxlYWQgdG8g dGhlDQo+ID4gaW5jb21wYXRpYmxlIGRhdGEgc3RydWN0dXJlLg0KPiA+IEJ1dCBub3Qgc3VyZSBp ZiBpdCBpcyBhIGJpZyBwcm9ibGVtIDotKQ0KPiANCj4gWWVhaCwgbGlrZSBJIHNheSBJIGRvbid0 IHJlYWxseSBoYXZlIGEgZ29vZCBmZWVsaW5nIGZvciB3aGF0IHdvdWxkIGJlIGJlc3QgaGVyZSwN Cj4gSSdtIGp1c3QgdGhpbmtpbmcgb2Ygd2hhdCBJIGRvIGtub3cgYW5kIHdhcnkgb2YgdGhlIHBv dGVudGlhbCBmb3IgYSAiNjQwIGJpdHMNCj4gb3VnaHQgdG8gYmUgZW5vdWdoIGZvciBhbnlvbmUi IGlzc3VlIDspDQo+IA0KPiA+Pj4gK3N0cnVjdCBtYXBfYmVuY2htYXJrX2RhdGEgew0KPiA+Pj4g KwlzdHJ1Y3QgbWFwX2JlbmNobWFyayBicGFyYW07DQo+ID4+PiArCXN0cnVjdCBkZXZpY2UgKmRl djsNCj4gPj4+ICsJc3RydWN0IGRlbnRyeSAgKmRlYnVnZnM7DQo+ID4+PiArCWF0b21pYzY0X3Qg dG90YWxfbWFwX25zZWNzOw0KPiA+Pj4gKwlhdG9taWM2NF90IHRvdGFsX21hcF9sb29wczsNCj4g Pj4+ICsJYXRvbWljNjRfdCB0b3RhbF91bm1hcF9uc2VjczsNCj4gPj4+ICsJYXRvbWljNjRfdCB0 b3RhbF91bm1hcF9sb29wczsNCj4gPj4+ICt9Ow0KPiA+Pj4gKw0KPiA+Pj4gK3N0YXRpYyBpbnQg bWFwX2JlbmNobWFya190aHJlYWQodm9pZCAqZGF0YSkgew0KPiA+Pj4gKwlzdHJ1Y3QgcGFnZSAq cGFnZTsNCj4gPj4+ICsJZG1hX2FkZHJfdCBkbWFfYWRkcjsNCj4gPj4+ICsJc3RydWN0IG1hcF9i ZW5jaG1hcmtfZGF0YSAqbWFwID0gZGF0YTsNCj4gPj4+ICsJaW50IHJldCA9IDA7DQo+ID4+PiAr DQo+ID4+PiArCXBhZ2UgPSBhbGxvY19wYWdlKEdGUF9LRVJORUwpOw0KPiA+Pj4gKwlpZiAoIXBh Z2UpDQo+ID4+PiArCQlyZXR1cm4gLUVOT01FTTsNCj4gPj4+ICsNCj4gPj4+ICsJd2hpbGUgKCFr dGhyZWFkX3Nob3VsZF9zdG9wKCkpICB7DQo+ID4+PiArCQlrdGltZV90IG1hcF9zdGltZSwgbWFw X2V0aW1lLCB1bm1hcF9zdGltZSwgdW5tYXBfZXRpbWU7DQo+ID4+PiArDQo+ID4+PiArCQltYXBf c3RpbWUgPSBrdGltZV9nZXQoKTsNCj4gPj4+ICsJCWRtYV9hZGRyID0gZG1hX21hcF9wYWdlKG1h cC0+ZGV2LCBwYWdlLCAwLCBQQUdFX1NJWkUsDQo+ID4+IERNQV9CSURJUkVDVElPTkFMKTsNCj4g Pj4NCj4gPj4gTm90ZSB0aGF0IGZvciBhIG5vbi1jb2hlcmVudCBkZXZpY2UsIHRoaXMgd2lsbCBn aXZlIGFuIHVuZGVyZXN0aW1hdGUNCj4gPj4gb2YgdGhlIHJlYWwtd29ybGQgb3ZlcmhlYWQgb2Yg QklESVJFQ1RJT05BTCBvciBUT19ERVZJQ0UgbWFwcGluZ3MsDQo+ID4+IHNpbmNlIHRoZSBwYWdl IHdpbGwgbmV2ZXIgYmUgZGlydHkgaW4gdGhlIGNhY2hlIChleGNlcHQgcG9zc2libHkgdGhlDQo+ ID4+IHZlcnkgZmlyc3QgdGltZSB0aHJvdWdoKS4NCj4gPg0KPiA+IEFncmVlZC4gSSdkIGxpa2Ug dG8gYWRkIGEgRElSRUNUSU9OIHBhcmFtZXRlciBsaWtlICItZCAwIiwgIi1kIDEiDQo+ID4gYWZ0 ZXIgd2UgaGF2ZSB0aGlzIGJhc2ljIGZyYW1ld29yay4NCj4gDQo+IFRoYXQgd2Fzbid0IHNvIG11 Y2ggYWJvdXQgdGhlIGRpcmVjdGlvbiBpdHNlbGYsIGp1c3QgdGhhdCBpZiBpdCdzIGFueXRoaW5n IG90aGVyDQo+IHRoYW4gRlJPTV9ERVZJQ0UsIHdlIHNob3VsZCBwcm9iYWJseSBkbyBzb21ldGhp bmcgdG8gZGlydHkgdGhlIGJ1ZmZlciBieSBhDQo+IHJlYXNvbmFibGUgYW1vdW50IGJlZm9yZSBl YWNoIG1hcC4gT3RoZXJ3aXNlIHRoZSBtZWFzdXJlZCBwZXJmb3JtYW5jZSBpcw0KPiBnb2luZyB0 byBiZSB1bnJlYWxpc3RpYyBvbiBtYW55IHN5c3RlbXMuDQoNCk1heWJlIHB1dCBhIG1lbXNldChi dWYsIDAsIFBBR0VfU0laRSkgYmVmb3JlIGRtYV9tYXAgd2lsbCBoZWxwID8NCg0KPiANCj4gWy4u Ll0NCj4gPj4+ICsJCWF0b21pYzY0X2FkZCgobG9uZyBsb25nKWt0aW1lX3RvX25zKGt0aW1lX3N1 Yih1bm1hcF9ldGltZSwNCj4gPj4gdW5tYXBfc3RpbWUpKSwNCj4gPj4+ICsJCQkJJm1hcC0+dG90 YWxfdW5tYXBfbnNlY3MpOw0KPiA+Pj4gKwkJYXRvbWljNjRfaW5jKCZtYXAtPnRvdGFsX21hcF9s b29wcyk7DQo+ID4+PiArCQlhdG9taWM2NF9pbmMoJm1hcC0+dG90YWxfdW5tYXBfbG9vcHMpOw0K PiA+Pg0KPiA+PiBJIHRoaW5rIGl0IHdvdWxkIGJlIHdvcnRoIGtlZXBpbmcgdHJhY2sgb2YgdGhl IHZhcmlhbmNlcyBhcyB3ZWxsIC0gaXQNCj4gPj4gY2FuIGJlIGhhcmQgdG8gdGVsbCBpZiBhIHJl YXNvbmFibGUtbG9va2luZyBhdmVyYWdlIGlzIGhpZGluZw0KPiA+PiB0ZXJyaWJsZSB3b3JzdC1j YXNlIGJlaGF2aW91ci4NCj4gPg0KPiA+IFRoaXMgaXMgYSBzZW5zaWJsZSByZXF1aXJlbWVudC4g SSBiZWxpZXZlIGl0IGlzIGJldHRlciB0byBiZSBoYW5kbGVkDQo+ID4gYnkgdGhlIGV4aXN0aW5n IGtlcm5lbCB0cmFjaW5nIG1ldGhvZC4NCj4gPg0KPiA+IE1heWJlIHdlIG5lZWQgYSBoaXN0b2dy YW0gbGlrZToNCj4gPiBEZWxheSAgIHNhbXBsZSBjb3VudA0KPiA+IDEtMnVzICAgMTAwMCAgICAg ICAgICAgICAgKioqDQo+ID4gMi0zdXMgICAyMDAwICAgICAgICAgICAgICAqKioqKioqDQo+ID4g My00dXMgICAxMDAgICAgICAgICAgICAgICAqDQo+ID4gLi4uLi4NCj4gPiBUaGlzIHdpbGwgYmUg bW9yZSBwcmVjaXNlIHRoYW4gdGhlIG1heGltdW0gbGF0ZW5jeSBpbiB0aGUgd29yc3QgY2FzZS4N Cj4gPg0KPiA+IEknZCBiZWxpZXZlIHRoaXMgY2FuIGJlIGhhbmRsZWQgYnk6DQo+ID4gdHJhY2Vw b2ludCAgQQ0KPiA+IE1hcA0KPiA+IFRyYWNlcG9pbnQgIEINCj4gPg0KPiA+IFRyYWNlcG9pbnQg ICBDDQo+ID4gVW5tYXANCj4gPiBUcmFjZXBvaW50ICAgRA0KPiA+DQo+ID4gTGV0IHRoZSB1c2Vy c3BhY2UgZWJwZiB0byBkcmF3IHRoZSBoaXN0b2dyYW0gZm9yIHRoZSBkZWx0YSBvZiBCLUEgYW5k IEQtQy4NCj4gPg0KPiA+IFNvIEkgYW0gcGxhbm5pbmcgdG8gcHV0IHRoaXMgcmVxdWlyZW1lbnQg aW50byB0b2RvIGxpc3QgYW5kIHdyaXRlIGFuDQo+ID4gdXNlcnNwYWNlIGVicGYvYmNjIHNjcmlw dCBmb3IgaGlzdG9ncmFtIGFuZCBwdXQgaW4gdG9vbHMvIGRpcmVjdG9yeS4NCj4gPg0KPiA+IFBs ZWFzZSBnaXZlIHlvdXIgY29tbWVudHMgb24gdGhpcy4NCj4gDQo+IFJpZ2h0LCBJIHdhc24ndCBz dWdnZXN0aW5nIHRyeWluZyB0byBob21lYnJldyBhIGZ1bGwgZGF0YSBjb2xsZWN0aW9uIHN5c3Rl bSBoZXJlDQo+IC0gSSBhZ3JlZSB0aGVyZSBhcmUgYmV0dGVyIHRvb2xzIGZvciB0aGF0IGFscmVh ZHkgLSBqdXN0IHRoYXQgaXQncyBiYXNpY2FsbHkgZnJlZSB0bw0KPiB0cmFjayBhIHN1bSBvZiBz cXVhcmVzIGFsb25nc2lkZSBhIHN1bSwgc28gdGhhdCB3ZSBjYW4gdHJpdmlhbGx5IGNhbGN1bGF0 ZSBhDQo+IHVzZWZ1bCB2YXJpYW5jZSAob3Igc3RhbmRhcmQNCj4gZGV2aWF0aW9uKSBmaWd1cmUg YWxvbmdzaWRlIHRoZSBtZWFuIGF0IHRoZSBlbmQuDQoNCkZvciB0aGlzIGNhc2UsIEkgYW0gbm90 IHN1cmUgaWYgaXQgaXMgdHJ1ZS4gVW5sZXNzIHdlIGV4cG9zZSBtb3JlIGRhdGEgc3VjaCBhcw0K bWluLCBtYXggZXRjLiB0byB1c2Vyc3BhY2UsIGl0IG1ha2VzIG5vIGRpZmZlcmVuY2Ugd2hldGhl ciB0b3RhbF8odW4pbWFwX25zZWNzDQphbmQgdG90YWxfKHVuKW1hcF9sb29wcyBhcmUgZXhwb3Nl ZCBvciBub3QuDQoNCkFzIA0KdG90YWwgbG9vcHMgPSBzZWNvbmRzIC8gKGF2Z19tYXBfbGF0ZW5j eSArIGF2Z191bm1hcF9sYXRlbmN5KTsNCnRvdGFsX21hcF9uc2VjcyA9IHRvdGFsIGxvb3AgY291 bnQgKiBhdmdfbWFwX2xhdGVuY3kNCnRvdGFsX3VubWFwX25zZWNzID0gdG90YWwgbG9vcCBjb3Vu dCAqIGF2Z191bm1hcF9sYXRlbmN5DQoNCmFsbCBvZiBzZWNvbmRzLCBhdmdfdW5tYXBfbGF0ZW5j eSwgYXZnX3VubWFwX2xhdGVuY3kgYXJlIGtub3duIGJ5DQp1c2Vyc3BhY2UgdG9vbC4NCg0KPiAN Cj4gWy4uLl0NCj4gPj4+ICsJZm9yIChpID0gMDsgaSA8IHRocmVhZHM7IGkrKykgew0KPiA+Pj4g KwkJdHNrW2ldID0ga3RocmVhZF9jcmVhdGVfb25fbm9kZShtYXBfYmVuY2htYXJrX3RocmVhZCwg bWFwLA0KPiA+Pj4gKwkJCQltYXAtPmJwYXJhbS5ub2RlLCAiZG1hLW1hcC1iZW5jaG1hcmsvJWQi LCBpKTsNCj4gPj4+ICsJCWlmIChJU19FUlIodHNrW2ldKSkgew0KPiA+Pj4gKwkJCWRldl9lcnIo bWFwLT5kZXYsICJjcmVhdGUgZG1hX21hcCB0aHJlYWQgZmFpbGVkXG4iKTsNCj4gPj4+ICsJCQly ZXR1cm4gUFRSX0VSUih0c2tbaV0pOw0KPiA+Pj4gKwkJfQ0KPiA+Pj4gKw0KPiA+Pj4gKwkJaWYg KG5vZGUgIT0gTlVNQV9OT19OT0RFICYmIG5vZGVfb25saW5lKG5vZGUpKQ0KPiA+Pj4gKwkJCWt0 aHJlYWRfYmluZF9tYXNrKHRza1tpXSwgY3B1X21hc2spOw0KPiA+Pj4gKw0KPiA+Pj4gKwkJd2Fr ZV91cF9wcm9jZXNzKHRza1tpXSk7DQo+ID4+DQo+ID4+IE1pZ2h0IGl0IGJlIGJldHRlciB0byBj cmVhdGUgYWxsIHRoZSB0aHJlYWRzIGZpcnN0LCAqdGhlbiogc3RhcnQNCj4gPj4ga2lja2luZyB0 aGVtPw0KPiA+DQo+ID4gVGhlIGRpZmZpY3VsdHkgaXMgdGhhdCB3ZSBkb24ndCBrbm93IGhvdyBt YW55IHRocmVhZHMgd2Ugc2hvdWxkIGNyZWF0ZQ0KPiA+IGFzIHRoZSB0aHJlYWQgbnVtYmVyIGlz IGEgcGFyYW1ldGVyIHRvIHRlc3QgdGhlIGNvbnRlbnRpb24gb2YgSU9NTVUgZHJpdmVyLg0KPiA+ IEluIG15IHRlc3QgY2FzZSwgSSdkIGxpa2UgdG8gdGVzdCB0aGluZ3MgbGlrZSBPbmUgdGhyZWFk IFR3byB0aHJlYWRzDQo+ID4gLi4uLg0KPiA+IDggdGhyZWFkcw0KPiA+IDEyIHRocmVhZHMNCj4g PiAxNiB0aHJlYWRzLi4uDQo+ID4NCj4gPiBPbiB0aGUgb3RoZXIgaGFuZCwgSSB0aGluayBpdCBp cyBiZXR0ZXIgdG8gZHJvcCB0aGUgbWVtb3J5IG9mDQo+ID4gdGFza19zdHJ1Y3Qgb2YgdGhvc2Ug dGVzdCB0aHJlYWRzIHdoaWxlIHdlIGFyZSBub3QgdGVzdGluZyBkbWEgbWFwLg0KPiANCj4gSSBz aW1wbHkgbWVhbnQgc3BsaXR0aW5nIHRoZSBsb29wIGhlcmUgaW50byB0d28gLSBvbmUgdG8gY3Jl YXRlIHRoZSB0aHJlYWRzIGFuZA0KPiBzZXQgdGhlaXIgYWZmaW5pdHksIHRoZW4gYW5vdGhlciB0 byB3YWtlIHRoZW0gYWxsIHVwIC0gc28gd2UgZG9uJ3Qgc3RhcnQNCj4gdW5uZWNlc3NhcmlseSB0 aHJhc2hpbmcgdGhlIHN5c3RlbSB3aGlsZSB3ZSdyZSBzdGlsbCB0cnlpbmcgdG8gc2V0IHVwIHRo ZSByZXN0IG9mDQo+IHRoZSB0ZXN0IDspDQoNCkFncmVlZC4NCg0KPiANCj4gUm9iaW4uDQoNClRo YW5rcw0KQmFycnkNCg0K