From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dmitry Osipenko Subject: Re: [RFC] Host1x/TegraDRM UAPI (sync points) Date: Mon, 29 Jun 2020 22:42:46 +0300 Message-ID: References: <9b06b7ec-f952-2561-7afb-5653514cd5d3@kapsi.fi> <5b1edaad-ba36-7b0f-5b02-457ae5b6d91e@gmail.com> <62859775-514c-2941-75ed-6905e9282a6f@kapsi.fi> <623c1eaa-31fb-8dff-f6c0-d8cd0be60070@gmail.com> <827c92a6-7fed-a81c-ba8e-6c69416c4ab9@kapsi.fi> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: <827c92a6-7fed-a81c-ba8e-6c69416c4ab9-/1wQRMveznE@public.gmane.org> Content-Language: en-US Sender: linux-tegra-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Mikko Perttunen , Thierry Reding , Jon Hunter , David Airlie , Daniel Vetter , sumit.semwal-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org, gustavo-THi1TnShQwVAfugRpC6u6w@public.gmane.org Cc: "linux-tegra-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , dri-devel , talho-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org, bhuntsman-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org, Erik Faye-Lund List-Id: linux-tegra@vger.kernel.org 29.06.2020 13:27, Mikko Perttunen пишет: ... >>>> 4. The job's sync point can't be re-used after job's submission (UAPI >>>> constraint!). Userspace must free sync point and allocate a new one for >>>> the next job submission. And now we: >>>> >>>>     - Know that job's sync point is always in a healthy state! >>>> >>>>     - We're not limited by a number of physically available hardware >>>> sync >>>> points! Allocation should block until free sync point is available. >>>> >>>>     - The logical number of job's sync point increments matches the SP >>>> hardware state! Which is handy for a job's debugging. >>>> >>>> Optionally, the job's sync point could be auto-removed from the DRM's >>>> context after job's submission, avoiding a need for an extra SYNCPT_PUT >>>> IOCTL invocation to be done by userspace after the job's submission. >>>> Could be a job's flag. >>> >>> I think this would cause problems where after a job completes but before >>> the fence has been waited, the syncpoint is already recycled (especially >>> if the syncpoint is reset into some clean state). >> >> Exactly, good point! The dma-fence shouldn't be hardwired to the sync >> point in order to avoid this situation :) >> >> Please take a look at the fence implementation that I made for the >> grate-driver [3]. The host1x-fence is a dma-fence [4] that is attached >> to a sync point by host1x_fence_create(). Once job is completed, the >> host1x-fence is detached from the sync point [5][6] and sync point could >> be recycled safely! > > What if the fence has been programmed as a prefence to another channel > (that is getting delayed), or to the GPU, or some other accelerator like > DLA, or maybe some other VM? Those don't know the dma_fence has been > signaled, they can only rely on the syncpoint ID/threshold pair. The explicit job's fence is always just a dma-fence, it's not tied to a host1x-fence and it should be waited for a signal on CPU. If you want to make job to wait for a sync point on hardware, then you should use the drm_tegra_submit_command wait-command. Again, please notice that DRM scheduler supports the job-submitted fence! This dma-fence will signal once job is pushed to hardware for execution, so it shouldn't be a problem to maintain jobs order for a complex jobs without much hassle. We'll need to write some userspace to check how it works in practice :) For now I really had experience with a simple jobs only. Secondly, I suppose neither GPU, nor DLA could wait on a host1x sync point, correct? Or are they integrated with Host1x HW? Anyways, it shouldn't be difficult to resolve dma-fence into host1x-fence, get SP ID and maintain the SP's liveness. Please see more below. In the grate-driver I made all sync points refcounted, so sync point won't be recycled while it has active users [1][2][3] in kernel (or userspace). [1] https://github.com/grate-driver/linux/blob/master/include/linux/host1x.h#L428 [2] https://github.com/grate-driver/linux/blob/master/include/linux/host1x.h#L1206 [3] https://github.com/grate-driver/linux/blob/master/drivers/gpu/host1x/soc/syncpoints.c#L163 Now, grate-kernel isn't a 100% complete implementation, as I already mentioned before. The host1x-fence doesn't have a reference to a sync point as you may see in the code, this is because the userspace sync points are not implemented in the grate-driver. But nothing stops us to add that SP reference and then we could simply do the following in the code: struct dma_fence *host1x_fence_create(syncpt,...) { ... fence->sp = syncpt; ... return &fence->base; } void host1x_syncpt_signal_fence(struct host1x_fence *fence) { ... fence->sp = NULL; } irqreturn_t host1x_hw_syncpt_isr() { spin_lock(&host1x_syncpts_lock); ... host1x_syncpt_signal_fence(sp->fence); ... spin_unlock(&host1x_syncpts_lock); } void host1x_submit_job(job) { ... spin_lock_irqsave(&host1x_syncpts_lock); sp = host1x_syncpt_get(host1x_fence->sp); spin_unlock_irqrestore(&host1x_syncpts_lock); ... if (sp) { push(WAIT(sp->id, host1x_fence->threshold)); job->sync_points = sp; } } void host1x_free_job(job) { host1x_syncpt_put(job->sync_points); ... } For example: if you'll share host1-fence (dma-fence) with a GPU context, then the fence's SP won't be released until GPU's context will be done with the SP. The GPU's job will be timed out if shared SP won't get incremented, and this is totally okay situation. Does this answer yours question? === I'm not familiar with the Host1x VMs, so please let me use my imagination here: In a case of VM we could have a special VM-shared sync point type. The userspace will allocate this special VM SP using ALLOCATE_SYNCPOINT and this SP won't be used for a job(!). This is the case where job will need to increment multiple sync points, its own SP + VM's SP. If job hangs, then there should be a way to tell VM to release the SP and try again next time with a freshly-allocated SP. The shared SP should stay alive as long as VM uses it, so there should be a way for VM to tell that it's done with SP. Alternatively, we could add a SP recovery (or whatever is needed) for the VM, but this should be left specific to T194+. Older Tegras shouldn't ever need this complexity if I'm not missing anything. Please provide a detailed information about the VM's workflow if the above doesn't sound good. Perhaps we shouldn't focus on the VM support for now, but may left some room for a potential future expansion if necessary. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1CF7C433E0 for ; Tue, 30 Jun 2020 07:34:55 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A3EDA20759 for ; Tue, 30 Jun 2020 07:34:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fwmO3R82" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3EDA20759 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DB7BD89C93; Tue, 30 Jun 2020 07:34:54 +0000 (UTC) Received: from mail-lj1-x241.google.com (mail-lj1-x241.google.com [IPv6:2a00:1450:4864:20::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9723089C8F for ; Mon, 29 Jun 2020 19:42:49 +0000 (UTC) Received: by mail-lj1-x241.google.com with SMTP id q7so6353016ljm.1 for ; Mon, 29 Jun 2020 12:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=JwU5FAidBhh7oWU6Uy7kMSsgdAXDeeK9feLjdd/Qaf0=; b=fwmO3R82y/VwNHM6Xf4kIhyv5VlwZzLnZvKbyEEk7Xbfj8+uyEKgYJVIrLmrF/O51o Ve82L9fKHd9HMW2Yd0Ppc2NkY9U8EGtULeWT9mzC/r/kluxUNzOT4h0wxHn7F2xkJrlH uz1nl/NaQN8mb5lGGEmmv9bKksBSwGlm6h8Gu9cpJ0wls5eC8peHXhnK0znj+iQk23jK zfGxOl0EyRWSPLdd5cuD/ybnk5qcPTA4pUYi1/tsMZCq1/phfNvkvmfpFnFgpQH47DJE dcxmX3PkLMjZ1xDIYgfDdvCmiJAB/v6vti9Ic0r0BxF+f0rVdBrrkk1O88PAwtIfpfvP FnNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=JwU5FAidBhh7oWU6Uy7kMSsgdAXDeeK9feLjdd/Qaf0=; b=IA1NtsQT0mR6+r1HiBTtrwb/++/0omo0jYBf3fqzJe1n7jvT+YF6nDdEZRDZagouJa JanLjN43XtMcIWk5Priyo3NfWWNlnSLgFiDVjZVaHlChLfe4QaBNBjgJDpJV1BRHpQsx E1SXYzv228v4Z2YwOF4Lo+9r8lU1hWSlz5y6fDcAaFFaRS9eSEyRXU7kWg++BE0An7+y SytirxNMyH+CITihP49+Vn8XDn9uNtgSNEs/7rNoRdTTXdvMuhsMaBpn0CBung3NRIQL lqry4iATIXCOMeCraBYLK8IQ9EIeYwgFsYSbpeVhlWXIn5qaIhIJ4NzE/aqVutwckuI5 S8BQ== X-Gm-Message-State: AOAM533FkGDQW+Z/FrGRVnRINwmsWk3srHUWlfzN1+AbROTFF4HXwnPW snsAduzVBmpb271HLzV9iWk= X-Google-Smtp-Source: ABdhPJzuj8qxkBynZYUQiQa5xWwpxasOwB9Z1EakkpTWInuTuxOwvcLLbbHwYwieH0IJs1dBvDjlpA== X-Received: by 2002:a2e:580f:: with SMTP id m15mr8345152ljb.357.1593459767830; Mon, 29 Jun 2020 12:42:47 -0700 (PDT) Received: from [192.168.2.145] (79-139-237-54.dynamic.spd-mgts.ru. [79.139.237.54]) by smtp.googlemail.com with ESMTPSA id q1sm156339lji.71.2020.06.29.12.42.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 29 Jun 2020 12:42:47 -0700 (PDT) Subject: Re: [RFC] Host1x/TegraDRM UAPI (sync points) To: Mikko Perttunen , Thierry Reding , Jon Hunter , David Airlie , Daniel Vetter , sumit.semwal@linaro.org, gustavo@padovan.org References: <9b06b7ec-f952-2561-7afb-5653514cd5d3@kapsi.fi> <5b1edaad-ba36-7b0f-5b02-457ae5b6d91e@gmail.com> <62859775-514c-2941-75ed-6905e9282a6f@kapsi.fi> <623c1eaa-31fb-8dff-f6c0-d8cd0be60070@gmail.com> <827c92a6-7fed-a81c-ba8e-6c69416c4ab9@kapsi.fi> From: Dmitry Osipenko Message-ID: Date: Mon, 29 Jun 2020 22:42:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <827c92a6-7fed-a81c-ba8e-6c69416c4ab9@kapsi.fi> Content-Language: en-US X-Mailman-Approved-At: Tue, 30 Jun 2020 07:34:54 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-tegra@vger.kernel.org" , talho@nvidia.com, bhuntsman@nvidia.com, dri-devel , Erik Faye-Lund Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" MjkuMDYuMjAyMCAxMzoyNywgTWlra28gUGVydHR1bmVuINC/0LjRiNC10YI6Ci4uLgo+Pj4+IDQu IFRoZSBqb2IncyBzeW5jIHBvaW50IGNhbid0IGJlIHJlLXVzZWQgYWZ0ZXIgam9iJ3Mgc3VibWlz c2lvbiAoVUFQSQo+Pj4+IGNvbnN0cmFpbnQhKS4gVXNlcnNwYWNlIG11c3QgZnJlZSBzeW5jIHBv aW50IGFuZCBhbGxvY2F0ZSBhIG5ldyBvbmUgZm9yCj4+Pj4gdGhlIG5leHQgam9iIHN1Ym1pc3Np b24uIEFuZCBub3cgd2U6Cj4+Pj4KPj4+PiDCoMKgwqAgLSBLbm93IHRoYXQgam9iJ3Mgc3luYyBw b2ludCBpcyBhbHdheXMgaW4gYSBoZWFsdGh5IHN0YXRlIQo+Pj4+Cj4+Pj4gwqDCoMKgIC0gV2Un cmUgbm90IGxpbWl0ZWQgYnkgYSBudW1iZXIgb2YgcGh5c2ljYWxseSBhdmFpbGFibGUgaGFyZHdh cmUKPj4+PiBzeW5jCj4+Pj4gcG9pbnRzISBBbGxvY2F0aW9uIHNob3VsZCBibG9jayB1bnRpbCBm cmVlIHN5bmMgcG9pbnQgaXMgYXZhaWxhYmxlLgo+Pj4+Cj4+Pj4gwqDCoMKgIC0gVGhlIGxvZ2lj YWwgbnVtYmVyIG9mIGpvYidzIHN5bmMgcG9pbnQgaW5jcmVtZW50cyBtYXRjaGVzIHRoZSBTUAo+ Pj4+IGhhcmR3YXJlIHN0YXRlISBXaGljaCBpcyBoYW5keSBmb3IgYSBqb2IncyBkZWJ1Z2dpbmcu Cj4+Pj4KPj4+PiBPcHRpb25hbGx5LCB0aGUgam9iJ3Mgc3luYyBwb2ludCBjb3VsZCBiZSBhdXRv LXJlbW92ZWQgZnJvbSB0aGUgRFJNJ3MKPj4+PiBjb250ZXh0IGFmdGVyIGpvYidzIHN1Ym1pc3Np b24sIGF2b2lkaW5nIGEgbmVlZCBmb3IgYW4gZXh0cmEgU1lOQ1BUX1BVVAo+Pj4+IElPQ1RMIGlu dm9jYXRpb24gdG8gYmUgZG9uZSBieSB1c2Vyc3BhY2UgYWZ0ZXIgdGhlIGpvYidzIHN1Ym1pc3Np b24uCj4+Pj4gQ291bGQgYmUgYSBqb2IncyBmbGFnLgo+Pj4KPj4+IEkgdGhpbmsgdGhpcyB3b3Vs ZCBjYXVzZSBwcm9ibGVtcyB3aGVyZSBhZnRlciBhIGpvYiBjb21wbGV0ZXMgYnV0IGJlZm9yZQo+ Pj4gdGhlIGZlbmNlIGhhcyBiZWVuIHdhaXRlZCwgdGhlIHN5bmNwb2ludCBpcyBhbHJlYWR5IHJl Y3ljbGVkIChlc3BlY2lhbGx5Cj4+PiBpZiB0aGUgc3luY3BvaW50IGlzIHJlc2V0IGludG8gc29t ZSBjbGVhbiBzdGF0ZSkuCj4+Cj4+IEV4YWN0bHksIGdvb2QgcG9pbnQhIFRoZSBkbWEtZmVuY2Ug c2hvdWxkbid0IGJlIGhhcmR3aXJlZCB0byB0aGUgc3luYwo+PiBwb2ludCBpbiBvcmRlciB0byBh dm9pZCB0aGlzIHNpdHVhdGlvbiA6KQo+Pgo+PiBQbGVhc2UgdGFrZSBhIGxvb2sgYXQgdGhlIGZl bmNlIGltcGxlbWVudGF0aW9uIHRoYXQgSSBtYWRlIGZvciB0aGUKPj4gZ3JhdGUtZHJpdmVyIFsz XS4gVGhlIGhvc3QxeC1mZW5jZSBpcyBhIGRtYS1mZW5jZSBbNF0gdGhhdCBpcyBhdHRhY2hlZAo+ PiB0byBhIHN5bmMgcG9pbnQgYnkgaG9zdDF4X2ZlbmNlX2NyZWF0ZSgpLiBPbmNlIGpvYiBpcyBj b21wbGV0ZWQsIHRoZQo+PiBob3N0MXgtZmVuY2UgaXMgZGV0YWNoZWQgZnJvbSB0aGUgc3luYyBw b2ludCBbNV1bNl0gYW5kIHN5bmMgcG9pbnQgY291bGQKPj4gYmUgcmVjeWNsZWQgc2FmZWx5IQo+ IAo+IFdoYXQgaWYgdGhlIGZlbmNlIGhhcyBiZWVuIHByb2dyYW1tZWQgYXMgYSBwcmVmZW5jZSB0 byBhbm90aGVyIGNoYW5uZWwKPiAodGhhdCBpcyBnZXR0aW5nIGRlbGF5ZWQpLCBvciB0byB0aGUg R1BVLCBvciBzb21lIG90aGVyIGFjY2VsZXJhdG9yIGxpa2UKPiBETEEsIG9yIG1heWJlIHNvbWUg b3RoZXIgVk0/IFRob3NlIGRvbid0IGtub3cgdGhlIGRtYV9mZW5jZSBoYXMgYmVlbgo+IHNpZ25h bGVkLCB0aGV5IGNhbiBvbmx5IHJlbHkgb24gdGhlIHN5bmNwb2ludCBJRC90aHJlc2hvbGQgcGFp ci4KClRoZSBleHBsaWNpdCBqb2IncyBmZW5jZSBpcyBhbHdheXMganVzdCBhIGRtYS1mZW5jZSwg aXQncyBub3QgdGllZCB0byBhCmhvc3QxeC1mZW5jZSBhbmQgaXQgc2hvdWxkIGJlIHdhaXRlZCBm b3IgYSBzaWduYWwgb24gQ1BVLgoKSWYgeW91IHdhbnQgdG8gbWFrZSBqb2IgdG8gd2FpdCBmb3Ig YSBzeW5jIHBvaW50IG9uIGhhcmR3YXJlLCB0aGVuIHlvdQpzaG91bGQgdXNlIHRoZSBkcm1fdGVn cmFfc3VibWl0X2NvbW1hbmQgd2FpdC1jb21tYW5kLgoKQWdhaW4sIHBsZWFzZSBub3RpY2UgdGhh dCBEUk0gc2NoZWR1bGVyIHN1cHBvcnRzIHRoZSBqb2Itc3VibWl0dGVkCmZlbmNlISBUaGlzIGRt YS1mZW5jZSB3aWxsIHNpZ25hbCBvbmNlIGpvYiBpcyBwdXNoZWQgdG8gaGFyZHdhcmUgZm9yCmV4 ZWN1dGlvbiwgc28gaXQgc2hvdWxkbid0IGJlIGEgcHJvYmxlbSB0byBtYWludGFpbiBqb2JzIG9y ZGVyIGZvciBhCmNvbXBsZXggam9icyB3aXRob3V0IG11Y2ggaGFzc2xlLiBXZSdsbCBuZWVkIHRv IHdyaXRlIHNvbWUgdXNlcnNwYWNlIHRvCmNoZWNrIGhvdyBpdCB3b3JrcyBpbiBwcmFjdGljZSA6 KSBGb3Igbm93IEkgcmVhbGx5IGhhZCBleHBlcmllbmNlIHdpdGggYQpzaW1wbGUgam9icyBvbmx5 LgoKU2Vjb25kbHksIEkgc3VwcG9zZSBuZWl0aGVyIEdQVSwgbm9yIERMQSBjb3VsZCB3YWl0IG9u IGEgaG9zdDF4IHN5bmMKcG9pbnQsIGNvcnJlY3Q/IE9yIGFyZSB0aGV5IGludGVncmF0ZWQgd2l0 aCBIb3N0MXggSFc/CgpBbnl3YXlzLCBpdCBzaG91bGRuJ3QgYmUgZGlmZmljdWx0IHRvIHJlc29s dmUgZG1hLWZlbmNlIGludG8KaG9zdDF4LWZlbmNlLCBnZXQgU1AgSUQgYW5kIG1haW50YWluIHRo ZSBTUCdzIGxpdmVuZXNzLiBQbGVhc2Ugc2VlIG1vcmUKYmVsb3cuCgpJbiB0aGUgZ3JhdGUtZHJp dmVyIEkgbWFkZSBhbGwgc3luYyBwb2ludHMgcmVmY291bnRlZCwgc28gc3luYyBwb2ludAp3b24n dCBiZSByZWN5Y2xlZCB3aGlsZSBpdCBoYXMgYWN0aXZlIHVzZXJzIFsxXVsyXVszXSBpbiBrZXJu ZWwgKG9yCnVzZXJzcGFjZSkuCgpbMV0KaHR0cHM6Ly9naXRodWIuY29tL2dyYXRlLWRyaXZlci9s aW51eC9ibG9iL21hc3Rlci9pbmNsdWRlL2xpbnV4L2hvc3QxeC5oI0w0MjgKWzJdCmh0dHBzOi8v Z2l0aHViLmNvbS9ncmF0ZS1kcml2ZXIvbGludXgvYmxvYi9tYXN0ZXIvaW5jbHVkZS9saW51eC9o b3N0MXguaCNMMTIwNgpbM10KaHR0cHM6Ly9naXRodWIuY29tL2dyYXRlLWRyaXZlci9saW51eC9i bG9iL21hc3Rlci9kcml2ZXJzL2dwdS9ob3N0MXgvc29jL3N5bmNwb2ludHMuYyNMMTYzCgpOb3cs IGdyYXRlLWtlcm5lbCBpc24ndCBhIDEwMCUgY29tcGxldGUgaW1wbGVtZW50YXRpb24sIGFzIEkg YWxyZWFkeQptZW50aW9uZWQgYmVmb3JlLiBUaGUgaG9zdDF4LWZlbmNlIGRvZXNuJ3QgaGF2ZSBh IHJlZmVyZW5jZSB0byBhIHN5bmMKcG9pbnQgYXMgeW91IG1heSBzZWUgaW4gdGhlIGNvZGUsIHRo aXMgaXMgYmVjYXVzZSB0aGUgdXNlcnNwYWNlIHN5bmMKcG9pbnRzIGFyZSBub3QgaW1wbGVtZW50 ZWQgaW4gdGhlIGdyYXRlLWRyaXZlci4KCkJ1dCBub3RoaW5nIHN0b3BzIHVzIHRvIGFkZCB0aGF0 IFNQIHJlZmVyZW5jZSBhbmQgdGhlbiB3ZSBjb3VsZCBzaW1wbHkKZG8gdGhlIGZvbGxvd2luZyBp biB0aGUgY29kZToKCnN0cnVjdCBkbWFfZmVuY2UgKmhvc3QxeF9mZW5jZV9jcmVhdGUoc3luY3B0 LC4uLikKewoJLi4uCglmZW5jZS0+c3AgPSBzeW5jcHQ7CgkuLi4KCXJldHVybiAmZmVuY2UtPmJh c2U7Cn0KCnZvaWQgaG9zdDF4X3N5bmNwdF9zaWduYWxfZmVuY2Uoc3RydWN0IGhvc3QxeF9mZW5j ZSAqZmVuY2UpCnsKCS4uLgoJZmVuY2UtPnNwID0gTlVMTDsKfQoKaXJxcmV0dXJuX3QgaG9zdDF4 X2h3X3N5bmNwdF9pc3IoKQp7CglzcGluX2xvY2soJmhvc3QxeF9zeW5jcHRzX2xvY2spOwoJLi4u Cglob3N0MXhfc3luY3B0X3NpZ25hbF9mZW5jZShzcC0+ZmVuY2UpOwoJLi4uCglzcGluX3VubG9j aygmaG9zdDF4X3N5bmNwdHNfbG9jayk7Cn0KCnZvaWQgaG9zdDF4X3N1Ym1pdF9qb2Ioam9iKQp7 CgkuLi4KCXNwaW5fbG9ja19pcnFzYXZlKCZob3N0MXhfc3luY3B0c19sb2NrKTsKCXNwID0gaG9z dDF4X3N5bmNwdF9nZXQoaG9zdDF4X2ZlbmNlLT5zcCk7CglzcGluX3VubG9ja19pcnFyZXN0b3Jl KCZob3N0MXhfc3luY3B0c19sb2NrKTsKCS4uLgoJaWYgKHNwKSB7CgkJcHVzaChXQUlUKHNwLT5p ZCwgaG9zdDF4X2ZlbmNlLT50aHJlc2hvbGQpKTsKCQlqb2ItPnN5bmNfcG9pbnRzID0gc3A7Cgl9 Cn0KCnZvaWQgaG9zdDF4X2ZyZWVfam9iKGpvYikKewoJaG9zdDF4X3N5bmNwdF9wdXQoam9iLT5z eW5jX3BvaW50cyk7CgkuLi4KfQoKRm9yIGV4YW1wbGU6IGlmIHlvdSdsbCBzaGFyZSBob3N0MS1m ZW5jZSAoZG1hLWZlbmNlKSB3aXRoIGEgR1BVIGNvbnRleHQsCnRoZW4gdGhlIGZlbmNlJ3MgU1Ag d29uJ3QgYmUgcmVsZWFzZWQgdW50aWwgR1BVJ3MgY29udGV4dCB3aWxsIGJlIGRvbmUKd2l0aCB0 aGUgU1AuIFRoZSBHUFUncyBqb2Igd2lsbCBiZSB0aW1lZCBvdXQgaWYgc2hhcmVkIFNQIHdvbid0 IGdldAppbmNyZW1lbnRlZCwgYW5kIHRoaXMgaXMgdG90YWxseSBva2F5IHNpdHVhdGlvbi4KCkRv ZXMgdGhpcyBhbnN3ZXIgeW91cnMgcXVlc3Rpb24/Cgo9PT0KCkknbSBub3QgZmFtaWxpYXIgd2l0 aCB0aGUgSG9zdDF4IFZNcywgc28gcGxlYXNlIGxldCBtZSB1c2UgbXkKaW1hZ2luYXRpb24gaGVy ZToKCkluIGEgY2FzZSBvZiBWTSB3ZSBjb3VsZCBoYXZlIGEgc3BlY2lhbCBWTS1zaGFyZWQgc3lu YyBwb2ludCB0eXBlLiBUaGUKdXNlcnNwYWNlIHdpbGwgYWxsb2NhdGUgdGhpcyBzcGVjaWFsIFZN IFNQIHVzaW5nIEFMTE9DQVRFX1NZTkNQT0lOVCBhbmQKdGhpcyBTUCB3b24ndCBiZSB1c2VkIGZv ciBhIGpvYighKS4gVGhpcyBpcyB0aGUgY2FzZSB3aGVyZSBqb2Igd2lsbCBuZWVkCnRvIGluY3Jl bWVudCBtdWx0aXBsZSBzeW5jIHBvaW50cywgaXRzIG93biBTUCArIFZNJ3MgU1AuIElmIGpvYiBo YW5ncywKdGhlbiB0aGVyZSBzaG91bGQgYmUgYSB3YXkgdG8gdGVsbCBWTSB0byByZWxlYXNlIHRo ZSBTUCBhbmQgdHJ5IGFnYWluCm5leHQgdGltZSB3aXRoIGEgZnJlc2hseS1hbGxvY2F0ZWQgU1Au IFRoZSBzaGFyZWQgU1Agc2hvdWxkIHN0YXkgYWxpdmUKYXMgbG9uZyBhcyBWTSB1c2VzIGl0LCBz byB0aGVyZSBzaG91bGQgYmUgYSB3YXkgZm9yIFZNIHRvIHRlbGwgdGhhdCBpdCdzCmRvbmUgd2l0 aCBTUC4KCkFsdGVybmF0aXZlbHksIHdlIGNvdWxkIGFkZCBhIFNQIHJlY292ZXJ5IChvciB3aGF0 ZXZlciBpcyBuZWVkZWQpIGZvcgp0aGUgVk0sIGJ1dCB0aGlzIHNob3VsZCBiZSBsZWZ0IHNwZWNp ZmljIHRvIFQxOTQrLiBPbGRlciBUZWdyYXMKc2hvdWxkbid0IGV2ZXIgbmVlZCB0aGlzIGNvbXBs ZXhpdHkgaWYgSSdtIG5vdCBtaXNzaW5nIGFueXRoaW5nLgoKUGxlYXNlIHByb3ZpZGUgYSBkZXRh aWxlZCBpbmZvcm1hdGlvbiBhYm91dCB0aGUgVk0ncyB3b3JrZmxvdyBpZiB0aGUKYWJvdmUgZG9l c24ndCBzb3VuZCBnb29kLgoKUGVyaGFwcyB3ZSBzaG91bGRuJ3QgZm9jdXMgb24gdGhlIFZNIHN1 cHBvcnQgZm9yIG5vdywgYnV0IG1heSBsZWZ0IHNvbWUKcm9vbSBmb3IgYSBwb3RlbnRpYWwgZnV0 dXJlIGV4cGFuc2lvbiBpZiBuZWNlc3NhcnkuCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2RyaS1kZXZlbAo=