From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qLk7Pqh4" Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A278A5 for ; Tue, 5 Dec 2023 11:39:30 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-5c65e666609so3701877a12.1 for ; Tue, 05 Dec 2023 11:39:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701805170; x=1702409970; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=xfq67iLmriDqfduTRY0M8dju8QhNwul2zcs5mKn+Qdk=; b=qLk7Pqh4Pd9QsKsbP60577NqraCUiybzAGtjqVrmb3yTT0fZZDlCA6TsvS3vuT2iOB hky+24GulvywVn0x1HL0oSBsjpNVQ04MJOgVctAytLuBKCC0A+y/Pt0g3bt4ATD3Vxaj IcbSQMiQdvW4I+bTkBHWzxKZH+AsfGiDp0HuTB9q4TGD6VMcyVNzi/Goy5jxmvjqGVXx kxC+Po70jB5erw0iT79pdQ4U6jxwvQ2TenwMOfhXivYl6F34MyiOQ7l76PMAKah8nfri p66f0Nqqyb4v0l4v+r1q7uTuDTZij81VAvwIiwgqeG280+UTiohhLakX5684GNT0y4w1 Po8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701805170; x=1702409970; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=xfq67iLmriDqfduTRY0M8dju8QhNwul2zcs5mKn+Qdk=; b=tBLJcdQ9P68NO4WNQZYiBXDIoiMD6EQ/4kzol36eeLYKdFu9AjJy1r8OzXnVzIWQnF 8qYP01q0SbVby0R+1W+rK8TOpc/glXuljT0yoqohnJ3ptc9O+F1cKq89mAIGfRVna/gF g0c2VJPdyZiYCHnqpINfv6X62/dM9gZPOirLqFeC7urcFum9+U/xIZe6C4XDSOn8a5AH YYfpT24deGAqb9ZZFJjnPiwthaHR9OWUs3awSZ+/4/CdKTEei7Kik22fOre5GTQRNnnx 9HAsz9zNee7Kla/QzVfljHr6+3TGsH/RH4r44KfLTsSAom0++95BpftnwSXUSps1vd8I NvCQ== X-Gm-Message-State: AOJu0Yz6yGyPT1wS7IJMyjmHL5biUS2eOXJOwfLIsVz+pLodCczWL1Z6 WTR7rn/undO3pRl9sRcM5w7voUw= X-Google-Smtp-Source: AGHT+IFIztxj2eUKXZ2GS1Ca6Xtk1G+ZaZVM3GaQJJGqcaBfxYyvfWdWIYuJqHVvsmWQlJgRFVSGrCQ= X-Received: from sdf.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5935]) (user=sdf job=sendgmr) by 2002:a63:1220:0:b0:5c2:2f9:c374 with SMTP id h32-20020a631220000000b005c202f9c374mr4433763pgl.9.1701805169961; Tue, 05 Dec 2023 11:39:29 -0800 (PST) Date: Tue, 5 Dec 2023 11:39:28 -0800 In-Reply-To: <656f66023f7bd_3dd6422942a@willemb.c.googlers.com.notmuch> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231203165129.1740512-1-yoong.siang.song@intel.com> <20231203165129.1740512-3-yoong.siang.song@intel.com> <43b01013-e78b-417e-b169-91909c7309b1@kernel.org> <656de830e8d70_2e983e294ca@willemb.c.googlers.com.notmuch> <5a0faf8cc9ec3ab0d5082c66b909c582c8f1eae6.camel@siemens.com> <656f66023f7bd_3dd6422942a@willemb.c.googlers.com.notmuch> Message-ID: Subject: Re: [xdp-hints] Re: [PATCH bpf-next v3 2/3] net: stmmac: add Launch Time support to XDP ZC From: Stanislav Fomichev To: Willem de Bruijn Cc: Florian Bezdeka , yoong.siang.song@intel.com, Jesper Dangaard Brouer , davem@davemloft.net, Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Bjorn Topel , magnus.karlsson@intel.com, maciej.fijalkowski@intel.com, Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Lorenzo Bianconi , Tariq Toukan , Willem de Bruijn , Maxime Coquelin , Andrii Nakryiko , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-doc@vger.kernel.org" , "bpf@vger.kernel.org" , "xdp-hints@xdp-project.net" , "linux-stm32@st-md-mailman.stormreply.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kselftest@vger.kernel.org" Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 12/05, Willem de Bruijn wrote: > Stanislav Fomichev wrote: > > On Tue, Dec 5, 2023 at 7:34=E2=80=AFAM Florian Bezdeka > > wrote: > > > > > > On Tue, 2023-12-05 at 15:25 +0000, Song, Yoong Siang wrote: > > > > On Monday, December 4, 2023 10:55 PM, Willem de Bruijn wrote: > > > > > Jesper Dangaard Brouer wrote: > > > > > > > > > > > > > > > > > > On 12/3/23 17:51, Song Yoong Siang wrote: > > > > > > > This patch enables Launch Time (Time-Based Scheduling) suppor= t to XDP zero > > > > > > > copy via XDP Tx metadata framework. > > > > > > > > > > > > > > Signed-off-by: Song Yoong Siang > > > > > > > --- > > > > > > > drivers/net/ethernet/stmicro/stmmac/stmmac.h | 2 ++ > > > > > > > > > > > > As requested before, I think we need to see another driver impl= ementing > > > > > > this. > > > > > > > > > > > > I propose driver igc and chip i225. > > > > > > > > Sure. I will include igc patches in next version. > > > > > > > > > > > > > > > > The interesting thing for me is to see how the LaunchTime max 1= second > > > > > > into the future[1] is handled code wise. One suggestion is to a= dd a > > > > > > section to Documentation/networking/xsk-tx-metadata.rst per dri= ver that > > > > > > mentions/documents these different hardware limitations. It is= natural > > > > > > that different types of hardware have limitations. This is a c= lose-to > > > > > > hardware-level abstraction/API, and IMHO as long as we document= the > > > > > > limitations we can expose this API without too many limitations= for more > > > > > > capable hardware. > > > > > > > > Sure. I will try to add hardware limitations in documentation. > > > > > > > > > > > > > > I would assume that the kfunc will fail when a value is passed th= at > > > > > cannot be programmed. > > > > > > > > > > > > > In current design, the xsk_tx_metadata_request() dint got return va= lue. > > > > So user won't know if their request is fail. > > > > It is complex to inform user which request is failing. > > > > Therefore, IMHO, it is good that we let driver handle the error sil= ently. > > > > > > > > > > If the programmed value is invalid, the packet will be "dropped" / wi= ll > > > never make it to the wire, right? >=20 > Programmable behavior is to either drop or cap to some boundary > value, such as the farthest programmable time in the future: the > horizon. In fq: >=20 > /* Check if packet timestamp is too far in the future. */ > if (fq_packet_beyond_horizon(skb, q, now)) { > if (q->horizon_drop) { > q->stat_horizon_drops++; > return qdisc_drop(skb, sch, to_fr= ee); > } > q->stat_horizon_caps++; > skb->tstamp =3D now + q->horizon; > } > fq_skb_cb(skb)->time_to_send =3D skb->tstamp; >=20 > Drop is the more obviously correct mode. >=20 > Programming with a clock source that the driver does not support will > then be a persistent failure. >=20 > Preferably, this driver capability can be queried beforehand (rather > than only through reading error counters afterwards). >=20 > Perhaps it should not be a driver task to convert from possibly > multiple clock sources to the device native clock. Right now, we do > use per-device timecounters for this, implemented in the driver. >=20 > As for which clocks are relevant. For PTP, I suppose the device PHC, > converted to nsec. For pacing offload, TCP uses CLOCK_MONOTONIC. Do we need to expose some generic netdev netlink apis to query/adjust nic clock sources (or maybe there is something existing already)? Then the userspace can be responsible for syncing/converting the timestamps to the internal nic clocks. +1 to trying to avoid doing this in the drivers. > > > That is clearly a situation that the user should be informed about. F= or > > > RT systems this normally means that something is really wrong regardi= ng > > > timing / cycle overflow. Such systems have to react on that situation= . > >=20 > > In general, af_xdp is a bit lacking in this 'notify the user that they > > somehow messed up' area :-( > > For example, pushing a tx descriptor with a wrong addr/len in zc mode > > will not give any visible signal back (besides driver potentially > > spilling something into dmesg as it was in the mlx case). > > We can probably start with having some counters for these events? >=20 > This is because the AF_XDP completion queue descriptor format is only > a u64 address? Yeah. XDP_COPY mode has the descriptor validation which is exported via recvmsg errno, but zerocopy path seems to be too deep in the stack to report something back. And there is no place, as you mention, in the completion ring to report the status. > Could error conditions be reported on tx completion in the metadata, > using xsk_tx_metadata_complete? That would be one way to do it, yes. But then the error reporting depends on the metadata opt-in. Having a separate ring to export the errors, or having a v2 tx-completions layout with extra 'status' field would also work. But this seems like something that should be handled separately? Because we'd have to teach all existing zc drivers to report those errors back instead of dropping these descriptors.. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECD82C4167B for ; Tue, 5 Dec 2023 19:39:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=B9P6b+uSS1NgPM2YuV0L/C5qDH1JcqTsuhfPzYxkUmI=; b=IkD0F2VzNCHjdih+qdtlJOEbKc EpQR8XIFfdrF3QTs6zpkBbtBCkXpxveroakct/0TJvakVVTRx6hmteoD+aPKFrpja2XEFttr5HgOd xV51R5VDhsO9MFQ0vwCanGOp2V9+XUFkjxSPuARqjzayJnRSl1YGFytR2ppKIYBdERg1tLWrFzmJG f91nT1LU2VIQTzmsTEC/sjS627t8vi5zHELzigpK2NfSx3FSfGhBBDBSLhxMuIxj9ZxX/fqY/69Lw QqnoCdvj9E4tmIwqe/jPeREZTI4dzowWqDqXE3iNjJSPiPTgqpS7SWxC1y5cCRnE0Tos8n2yCmC+U 5E90R9lw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rAbGZ-008Iff-2m; Tue, 05 Dec 2023 19:39:35 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rAbGV-008IfE-2v for linux-arm-kernel@lists.infradead.org; Tue, 05 Dec 2023 19:39:33 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-5c668dc7f7bso3350835a12.3 for ; Tue, 05 Dec 2023 11:39:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701805170; x=1702409970; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=xfq67iLmriDqfduTRY0M8dju8QhNwul2zcs5mKn+Qdk=; b=iSkl5vXA3UaRdCwrJJArnFt4RcLOTwrFr68erzvoIJzoqYsxS6JiudNkC3Adr7Bf5T vHaxhSvAkr+NNRGFS2nkEGvqq3VDlRvaq2DPCLhrek9FPwodRbgNOFff2mF096wIJJ3+ 6So0XlcrpuSyZYhHre4ao1QY//Az3dR0D8EvUa+35d6FBAKf4uJEajfPaqlCTpwtBZpd AQwLyu41Hjj3cV5ZiTIZdgS+LJzPIvWX8kTQntI/WVcU+MIvRlCHb302VlhOjAr4UYE8 iwC4Nb/VTX1+upufAl+rtPMhMvkS47EN8vg37TVEuN6IR8GJPjmXmgGt/sNA3gxtWtet hUag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701805170; x=1702409970; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=xfq67iLmriDqfduTRY0M8dju8QhNwul2zcs5mKn+Qdk=; b=m9HotvtR5hKkhnTLbydGHrRFz/tCob7YIuhYvOQ6LUkwhLtA9BqYlB64g0FScyglBk jYgDqyzMbP/ds4rNkWlSf6hj7mMWdzRqK5M9ibQEHZU4PhIrv5WF+MdXo5azbo4kdw8g cXSzQXEnk9h7I6kGWEqPbpoulCO9nOpmkB6t3cdBGChU8YlYS8BSodOkltkHyS7lK6fx V9+1h3bN3oFb2T7ABXre/Qsh0q43Wgm6TSX2oD/w0Ll0NTbhjTSxf1eOfrlTBBG6SRiq VvoehcWsff/o8P+ktd+QYGL8Lz39Y+n/cm357K6S40ZkyVHRgj9Vkts544Q3vGyiV5uu rUEA== X-Gm-Message-State: AOJu0YwtapF2/pl2PSWvxngwZINU5Ax9VSS2Y9ZspGSX4qN362rFU2pa GyqkZKzqyPiSWAx8YmWws8rNddo= X-Google-Smtp-Source: AGHT+IFIztxj2eUKXZ2GS1Ca6Xtk1G+ZaZVM3GaQJJGqcaBfxYyvfWdWIYuJqHVvsmWQlJgRFVSGrCQ= X-Received: from sdf.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5935]) (user=sdf job=sendgmr) by 2002:a63:1220:0:b0:5c2:2f9:c374 with SMTP id h32-20020a631220000000b005c202f9c374mr4433763pgl.9.1701805169961; Tue, 05 Dec 2023 11:39:29 -0800 (PST) Date: Tue, 5 Dec 2023 11:39:28 -0800 In-Reply-To: <656f66023f7bd_3dd6422942a@willemb.c.googlers.com.notmuch> Mime-Version: 1.0 References: <20231203165129.1740512-1-yoong.siang.song@intel.com> <20231203165129.1740512-3-yoong.siang.song@intel.com> <43b01013-e78b-417e-b169-91909c7309b1@kernel.org> <656de830e8d70_2e983e294ca@willemb.c.googlers.com.notmuch> <5a0faf8cc9ec3ab0d5082c66b909c582c8f1eae6.camel@siemens.com> <656f66023f7bd_3dd6422942a@willemb.c.googlers.com.notmuch> Message-ID: Subject: Re: [xdp-hints] Re: [PATCH bpf-next v3 2/3] net: stmmac: add Launch Time support to XDP ZC From: Stanislav Fomichev To: Willem de Bruijn Cc: Florian Bezdeka , yoong.siang.song@intel.com, Jesper Dangaard Brouer , davem@davemloft.net, Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Bjorn Topel , magnus.karlsson@intel.com, maciej.fijalkowski@intel.com, Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Lorenzo Bianconi , Tariq Toukan , Willem de Bruijn , Maxime Coquelin , Andrii Nakryiko , Mykola Lysenko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Shuah Khan , Alexandre Torgue , Jose Abreu , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-doc@vger.kernel.org" , "bpf@vger.kernel.org" , "xdp-hints@xdp-project.net" , "linux-stm32@st-md-mailman.stormreply.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kselftest@vger.kernel.org" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231205_113931_946526_F1F101AE X-CRM114-Status: GOOD ( 40.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMTIvMDUsIFdpbGxlbSBkZSBCcnVpam4gd3JvdGU6Cj4gU3RhbmlzbGF2IEZvbWljaGV2IHdy b3RlOgo+ID4gT24gVHVlLCBEZWMgNSwgMjAyMyBhdCA3OjM04oCvQU0gRmxvcmlhbiBCZXpkZWth Cj4gPiA8Zmxvcmlhbi5iZXpkZWthQHNpZW1lbnMuY29tPiB3cm90ZToKPiA+ID4KPiA+ID4gT24g VHVlLCAyMDIzLTEyLTA1IGF0IDE1OjI1ICswMDAwLCBTb25nLCBZb29uZyBTaWFuZyB3cm90ZToK PiA+ID4gPiBPbiBNb25kYXksIERlY2VtYmVyIDQsIDIwMjMgMTA6NTUgUE0sIFdpbGxlbSBkZSBC cnVpam4gd3JvdGU6Cj4gPiA+ID4gPiBKZXNwZXIgRGFuZ2FhcmQgQnJvdWVyIHdyb3RlOgo+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBPbiAxMi8zLzIzIDE3OjUxLCBTb25nIFlv b25nIFNpYW5nIHdyb3RlOgo+ID4gPiA+ID4gPiA+IFRoaXMgcGF0Y2ggZW5hYmxlcyBMYXVuY2gg VGltZSAoVGltZS1CYXNlZCBTY2hlZHVsaW5nKSBzdXBwb3J0IHRvIFhEUCB6ZXJvCj4gPiA+ID4g PiA+ID4gY29weSB2aWEgWERQIFR4IG1ldGFkYXRhIGZyYW1ld29yay4KPiA+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IFNvbmcgWW9vbmcgU2lhbmc8eW9vbmcuc2lhbmcu c29uZ0BpbnRlbC5jb20+Cj4gPiA+ID4gPiA+ID4gLS0tCj4gPiA+ID4gPiA+ID4gICBkcml2ZXJz L25ldC9ldGhlcm5ldC9zdG1pY3JvL3N0bW1hYy9zdG1tYWMuaCAgICAgIHwgIDIgKysKPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gQXMgcmVxdWVzdGVkIGJlZm9yZSwgSSB0aGluayB3ZSBuZWVkIHRv IHNlZSBhbm90aGVyIGRyaXZlciBpbXBsZW1lbnRpbmcKPiA+ID4gPiA+ID4gdGhpcy4KPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gSSBwcm9wb3NlIGRyaXZlciBpZ2MgYW5kIGNoaXAgaTIyNS4KPiA+ ID4gPgo+ID4gPiA+IFN1cmUuIEkgd2lsbCBpbmNsdWRlIGlnYyBwYXRjaGVzIGluIG5leHQgdmVy c2lvbi4KPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBUaGUgaW50ZXJlc3RpbmcgdGhp bmcgZm9yIG1lIGlzIHRvIHNlZSBob3cgdGhlIExhdW5jaFRpbWUgbWF4IDEgc2Vjb25kCj4gPiA+ ID4gPiA+IGludG8gdGhlIGZ1dHVyZVsxXSBpcyBoYW5kbGVkIGNvZGUgd2lzZS4gT25lIHN1Z2dl c3Rpb24gaXMgdG8gYWRkIGEKPiA+ID4gPiA+ID4gc2VjdGlvbiB0byBEb2N1bWVudGF0aW9uL25l dHdvcmtpbmcveHNrLXR4LW1ldGFkYXRhLnJzdCBwZXIgZHJpdmVyIHRoYXQKPiA+ID4gPiA+ID4g bWVudGlvbnMvZG9jdW1lbnRzIHRoZXNlIGRpZmZlcmVudCBoYXJkd2FyZSBsaW1pdGF0aW9ucy4g IEl0IGlzIG5hdHVyYWwKPiA+ID4gPiA+ID4gdGhhdCBkaWZmZXJlbnQgdHlwZXMgb2YgaGFyZHdh cmUgaGF2ZSBsaW1pdGF0aW9ucy4gIFRoaXMgaXMgYSBjbG9zZS10bwo+ID4gPiA+ID4gPiBoYXJk d2FyZS1sZXZlbCBhYnN0cmFjdGlvbi9BUEksIGFuZCBJTUhPIGFzIGxvbmcgYXMgd2UgZG9jdW1l bnQgdGhlCj4gPiA+ID4gPiA+IGxpbWl0YXRpb25zIHdlIGNhbiBleHBvc2UgdGhpcyBBUEkgd2l0 aG91dCB0b28gbWFueSBsaW1pdGF0aW9ucyBmb3IgbW9yZQo+ID4gPiA+ID4gPiBjYXBhYmxlIGhh cmR3YXJlLgo+ID4gPiA+Cj4gPiA+ID4gU3VyZS4gSSB3aWxsIHRyeSB0byBhZGQgaGFyZHdhcmUg bGltaXRhdGlvbnMgaW4gZG9jdW1lbnRhdGlvbi4KPiA+ID4gPgo+ID4gPiA+ID4KPiA+ID4gPiA+ IEkgd291bGQgYXNzdW1lIHRoYXQgdGhlIGtmdW5jIHdpbGwgZmFpbCB3aGVuIGEgdmFsdWUgaXMg cGFzc2VkIHRoYXQKPiA+ID4gPiA+IGNhbm5vdCBiZSBwcm9ncmFtbWVkLgo+ID4gPiA+ID4KPiA+ ID4gPgo+ID4gPiA+IEluIGN1cnJlbnQgZGVzaWduLCB0aGUgeHNrX3R4X21ldGFkYXRhX3JlcXVl c3QoKSBkaW50IGdvdCByZXR1cm4gdmFsdWUuCj4gPiA+ID4gU28gdXNlciB3b24ndCBrbm93IGlm IHRoZWlyIHJlcXVlc3QgaXMgZmFpbC4KPiA+ID4gPiBJdCBpcyBjb21wbGV4IHRvIGluZm9ybSB1 c2VyIHdoaWNoIHJlcXVlc3QgaXMgZmFpbGluZy4KPiA+ID4gPiBUaGVyZWZvcmUsIElNSE8sIGl0 IGlzIGdvb2QgdGhhdCB3ZSBsZXQgZHJpdmVyIGhhbmRsZSB0aGUgZXJyb3Igc2lsZW50bHkuCj4g PiA+ID4KPiA+ID4KPiA+ID4gSWYgdGhlIHByb2dyYW1tZWQgdmFsdWUgaXMgaW52YWxpZCwgdGhl IHBhY2tldCB3aWxsIGJlICJkcm9wcGVkIiAvIHdpbGwKPiA+ID4gbmV2ZXIgbWFrZSBpdCB0byB0 aGUgd2lyZSwgcmlnaHQ/Cj4gCj4gUHJvZ3JhbW1hYmxlIGJlaGF2aW9yIGlzIHRvIGVpdGhlciBk cm9wIG9yIGNhcCB0byBzb21lIGJvdW5kYXJ5Cj4gdmFsdWUsIHN1Y2ggYXMgdGhlIGZhcnRoZXN0 IHByb2dyYW1tYWJsZSB0aW1lIGluIHRoZSBmdXR1cmU6IHRoZQo+IGhvcml6b24uIEluIGZxOgo+ IAo+ICAgICAgICAgICAgICAgICAvKiBDaGVjayBpZiBwYWNrZXQgdGltZXN0YW1wIGlzIHRvbyBm YXIgaW4gdGhlIGZ1dHVyZS4gKi8KPiAgICAgICAgICAgICAgICAgaWYgKGZxX3BhY2tldF9iZXlv bmRfaG9yaXpvbihza2IsIHEsIG5vdykpIHsKPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAo cS0+aG9yaXpvbl9kcm9wKSB7Cj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHEtPnN0YXRfaG9yaXpvbl9kcm9wcysrOwo+ICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICByZXR1cm4gcWRpc2NfZHJvcChza2IsIHNjaCwgdG9fZnJlZSk7Cj4gICAg ICAgICAgICAgICAgICAgICAgICAgfQo+ICAgICAgICAgICAgICAgICAgICAgICAgIHEtPnN0YXRf aG9yaXpvbl9jYXBzKys7Cj4gICAgICAgICAgICAgICAgICAgICAgICAgc2tiLT50c3RhbXAgPSBu b3cgKyBxLT5ob3Jpem9uOwo+ICAgICAgICAgICAgICAgICB9Cj4gICAgICAgICAgICAgICAgIGZx X3NrYl9jYihza2IpLT50aW1lX3RvX3NlbmQgPSBza2ItPnRzdGFtcDsKPiAKPiBEcm9wIGlzIHRo ZSBtb3JlIG9idmlvdXNseSBjb3JyZWN0IG1vZGUuCj4gCj4gUHJvZ3JhbW1pbmcgd2l0aCBhIGNs b2NrIHNvdXJjZSB0aGF0IHRoZSBkcml2ZXIgZG9lcyBub3Qgc3VwcG9ydCB3aWxsCj4gdGhlbiBi ZSBhIHBlcnNpc3RlbnQgZmFpbHVyZS4KPiAKPiBQcmVmZXJhYmx5LCB0aGlzIGRyaXZlciBjYXBh YmlsaXR5IGNhbiBiZSBxdWVyaWVkIGJlZm9yZWhhbmQgKHJhdGhlcgo+IHRoYW4gb25seSB0aHJv dWdoIHJlYWRpbmcgZXJyb3IgY291bnRlcnMgYWZ0ZXJ3YXJkcykuCj4gCj4gUGVyaGFwcyBpdCBz aG91bGQgbm90IGJlIGEgZHJpdmVyIHRhc2sgdG8gY29udmVydCBmcm9tIHBvc3NpYmx5Cj4gbXVs dGlwbGUgY2xvY2sgc291cmNlcyB0byB0aGUgZGV2aWNlIG5hdGl2ZSBjbG9jay4gUmlnaHQgbm93 LCB3ZSBkbwo+IHVzZSBwZXItZGV2aWNlIHRpbWVjb3VudGVycyBmb3IgdGhpcywgaW1wbGVtZW50 ZWQgaW4gdGhlIGRyaXZlci4KPiAKPiBBcyBmb3Igd2hpY2ggY2xvY2tzIGFyZSByZWxldmFudC4g Rm9yIFBUUCwgSSBzdXBwb3NlIHRoZSBkZXZpY2UgUEhDLAo+IGNvbnZlcnRlZCB0byBuc2VjLiBG b3IgcGFjaW5nIG9mZmxvYWQsIFRDUCB1c2VzIENMT0NLX01PTk9UT05JQy4KCkRvIHdlIG5lZWQg dG8gZXhwb3NlIHNvbWUgZ2VuZXJpYyBuZXRkZXYgbmV0bGluayBhcGlzIHRvIHF1ZXJ5L2FkanVz dApuaWMgY2xvY2sgc291cmNlcyAob3IgbWF5YmUgdGhlcmUgaXMgc29tZXRoaW5nIGV4aXN0aW5n IGFscmVhZHkpPwpUaGVuIHRoZSB1c2Vyc3BhY2UgY2FuIGJlIHJlc3BvbnNpYmxlIGZvciBzeW5j aW5nL2NvbnZlcnRpbmcgdGhlCnRpbWVzdGFtcHMgdG8gdGhlIGludGVybmFsIG5pYyBjbG9ja3Mu ICsxIHRvIHRyeWluZyB0byBhdm9pZCBkb2luZwp0aGlzIGluIHRoZSBkcml2ZXJzLgoKPiA+ID4g VGhhdCBpcyBjbGVhcmx5IGEgc2l0dWF0aW9uIHRoYXQgdGhlIHVzZXIgc2hvdWxkIGJlIGluZm9y bWVkIGFib3V0LiBGb3IKPiA+ID4gUlQgc3lzdGVtcyB0aGlzIG5vcm1hbGx5IG1lYW5zIHRoYXQg c29tZXRoaW5nIGlzIHJlYWxseSB3cm9uZyByZWdhcmRpbmcKPiA+ID4gdGltaW5nIC8gY3ljbGUg b3ZlcmZsb3cuIFN1Y2ggc3lzdGVtcyBoYXZlIHRvIHJlYWN0IG9uIHRoYXQgc2l0dWF0aW9uLgo+ ID4gCj4gPiBJbiBnZW5lcmFsLCBhZl94ZHAgaXMgYSBiaXQgbGFja2luZyBpbiB0aGlzICdub3Rp ZnkgdGhlIHVzZXIgdGhhdCB0aGV5Cj4gPiBzb21laG93IG1lc3NlZCB1cCcgYXJlYSA6LSgKPiA+ IEZvciBleGFtcGxlLCBwdXNoaW5nIGEgdHggZGVzY3JpcHRvciB3aXRoIGEgd3JvbmcgYWRkci9s ZW4gaW4gemMgbW9kZQo+ID4gd2lsbCBub3QgZ2l2ZSBhbnkgdmlzaWJsZSBzaWduYWwgYmFjayAo YmVzaWRlcyBkcml2ZXIgcG90ZW50aWFsbHkKPiA+IHNwaWxsaW5nIHNvbWV0aGluZyBpbnRvIGRt ZXNnIGFzIGl0IHdhcyBpbiB0aGUgbWx4IGNhc2UpLgo+ID4gV2UgY2FuIHByb2JhYmx5IHN0YXJ0 IHdpdGggaGF2aW5nIHNvbWUgY291bnRlcnMgZm9yIHRoZXNlIGV2ZW50cz8KPiAKPiBUaGlzIGlz IGJlY2F1c2UgdGhlIEFGX1hEUCBjb21wbGV0aW9uIHF1ZXVlIGRlc2NyaXB0b3IgZm9ybWF0IGlz IG9ubHkKPiBhIHU2NCBhZGRyZXNzPwoKWWVhaC4gWERQX0NPUFkgbW9kZSBoYXMgdGhlIGRlc2Ny aXB0b3IgdmFsaWRhdGlvbiB3aGljaCBpcyBleHBvcnRlZCB2aWEKcmVjdm1zZyBlcnJubywgYnV0 IHplcm9jb3B5IHBhdGggc2VlbXMgdG8gYmUgdG9vIGRlZXAgaW4gdGhlIHN0YWNrCnRvIHJlcG9y dCBzb21ldGhpbmcgYmFjay4gQW5kIHRoZXJlIGlzIG5vIHBsYWNlLCBhcyB5b3UgbWVudGlvbiwK aW4gdGhlIGNvbXBsZXRpb24gcmluZyB0byByZXBvcnQgdGhlIHN0YXR1cy4KCj4gQ291bGQgZXJy b3IgY29uZGl0aW9ucyBiZSByZXBvcnRlZCBvbiB0eCBjb21wbGV0aW9uIGluIHRoZSBtZXRhZGF0 YSwKPiB1c2luZyB4c2tfdHhfbWV0YWRhdGFfY29tcGxldGU/CgpUaGF0IHdvdWxkIGJlIG9uZSB3 YXkgdG8gZG8gaXQsIHllcy4gQnV0IHRoZW4gdGhlIGVycm9yIHJlcG9ydGluZyBkZXBlbmRzCm9u IHRoZSBtZXRhZGF0YSBvcHQtaW4uIEhhdmluZyBhIHNlcGFyYXRlIHJpbmcgdG8gZXhwb3J0IHRo ZSBlcnJvcnMsCm9yIGhhdmluZyBhIHYyIHR4LWNvbXBsZXRpb25zIGxheW91dCB3aXRoIGV4dHJh ICdzdGF0dXMnIGZpZWxkIHdvdWxkIGFsc28Kd29yay4KCkJ1dCB0aGlzIHNlZW1zIGxpa2Ugc29t ZXRoaW5nIHRoYXQgc2hvdWxkIGJlIGhhbmRsZWQgc2VwYXJhdGVseT8gQmVjYXVzZQp3ZSdkIGhh dmUgdG8gdGVhY2ggYWxsIGV4aXN0aW5nIHpjIGRyaXZlcnMgdG8gcmVwb3J0IHRob3NlIGVycm9y cyBiYWNrCmluc3RlYWQgb2YgZHJvcHBpbmcgdGhlc2UgZGVzY3JpcHRvcnMuLgoKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBt YWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9s aXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=