From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ozlabs.org (ozlabs.org [IPv6:2401:3900:2:1::2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xWBzR3mp0zDqvZ for ; Mon, 14 Aug 2017 20:43:31 +1000 (AEST) Message-ID: <1502707410.9844.10.camel@neuling.org> Subject: Re: [PATCH v6 17/17] powerpc/vas: Document FTW API/usage From: Michael Neuling To: Sukadev Bhattiprolu , Michael Ellerman Cc: Benjamin Herrenschmidt , stewart@linux.vnet.ibm.com, apopple@au1.ibm.com, hbabu@us.ibm.com, oohall@gmail.com, linuxppc-dev@ozlabs.org, linux-kernel@vger.kernel.org Date: Mon, 14 Aug 2017 20:43:30 +1000 In-Reply-To: <1502233622-9330-18-git-send-email-sukadev@linux.vnet.ibm.com> References: <1502233622-9330-1-git-send-email-sukadev@linux.vnet.ibm.com> <1502233622-9330-18-git-send-email-sukadev@linux.vnet.ibm.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , T24gVHVlLCAyMDE3LTA4LTA4IGF0IDE2OjA3IC0wNzAwLCBTdWthZGV2IEJoYXR0aXByb2x1IHdy b3RlOgo+IERvY3VtZW50IHRoZSB1c2FnZSBvZiB0aGUgVkFTIEZhc3QgdGhyZWFkLXdha2V1cCBB UEkuCj4gCj4gVGhhbmtzIGZvciBpbnB1dC9jb21tZW50cyBmcm9tIEJlbmphbWluIEhlcnJlbnNj aG1pZHQsIE1pY2hhZWwgTmV1bGluZywKPiBNaWNoYWVsIEVsbGVybWFuLCBSb2JlcnQgQmxhY2tt b3JlLCBJYW4gTXVuc2llLCBIYXJlbiBNeW5lbmksIFBhdWwgTWFja2VycmFzLgo+IAo+IENjOklh biBNdW5zaWUgPGltdW5zaWVAYXUxLmlibS5jb20+Cj4gQ2M6UGF1bCBNYWNrZXJyYXMgPHBhdWx1 c0BvemxhYnMub3JnPgo+IFNpZ25lZC1vZmYtYnk6IFN1a2FkZXYgQmhhdHRpcHJvbHUgPHN1a2Fk ZXZAbGludXgudm5ldC5pYm0uY29tPgo+IC0tLQo+IMKgRG9jdW1lbnRhdGlvbi9wb3dlcnBjL2Z0 dy1hcGkudHh0IHwgMzczCj4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysK PiDCoDEgZmlsZSBjaGFuZ2VkLCAzNzMgaW5zZXJ0aW9ucygrKQo+IMKgY3JlYXRlIG1vZGUgMTAw NjQ0IERvY3VtZW50YXRpb24vcG93ZXJwYy9mdHctYXBpLnR4dAo+IAo+IGRpZmYgLS1naXQgYS9E b2N1bWVudGF0aW9uL3Bvd2VycGMvZnR3LWFwaS50eHQgYi9Eb2N1bWVudGF0aW9uL3Bvd2VycGMv ZnR3LQo+IGFwaS50eHQKPiBuZXcgZmlsZSBtb2RlIDEwMDY0NAo+IGluZGV4IDAwMDAwMDAuLjBi M2YxNmYKPiAtLS0gL2Rldi9udWxsCj4gKysrIGIvRG9jdW1lbnRhdGlvbi9wb3dlcnBjL2Z0dy1h cGkudHh0Cj4gQEAgLTAsMCArMSwzNzMgQEAKPiArVmlydHVhbCBBY2NlbGVyYXRvciBTd2l0Y2hi b2FyZCBhbmQgRmFzdCBUaHJlYWQtV2FrZXVwIEFQSQo+ICsKPiArwqDCoMKgwqBQb3dlcjkgcHJv Y2Vzc29yIHN1cHBvcnRzIGEgaGFyZHdhcmUgc3VieXN0ZW0ga25vd24gYXMgdGhlIFZpcnR1YWwK PiArwqDCoMKgwqBBY2NlbGVyYXRvciBTd2l0Y2hib2FyZCAoVkFTKSB3aGljaCBhbGxvd3MgdHdv IGVudGl0aWVzIGluIHRoZSBQb3dlcjkKPiArwqDCoMKgwqBzeXN0ZW0gdG8gZWZmaWNpZW50bHkg ZXhjaGFuZ2UgbWVzc2FnZXMuIE1lc3NhZ2VzIG11c3QgYmUgZm9ybWF0dGVkIGFzCj4gK8KgwqDC oMKgQ29wcm9jZXNzb3IgUmVxZXVzdCBCbG9ja3MgKENSQikgYW5kIGJlIHN1Ym1pdHRlZCB1c2lu ZyB0aGUgQ09QWS9QQVNURQo+ICvCoMKgwqDCoGluc3RydWN0aW9ucyAobmV3IGluIFBvd2VyOSku Cj4gKwo+ICvCoMKgwqDCoFVzYWdlIG9mIFZBUyBkZXBlbmRzIG9uIHRoZSBlbnRpdGllcyBleGNo YW5naW5nIHRoZSBtZXNzYWdlcyBhbmQKPiArwqDCoMKgwqBjdXJyZW50bHkgdHdvIHVzYWdlcyBo YXZlIGJlZW4gaWRlbnRpZmllZC4KPiArCj4gK8KgwqDCoMKgRmlyc3QgdXNhZ2Ugb2YgVkFTLCBy ZWZlcnJlZCB0byBhcyBWQVMvTlggaW52b2x2ZXMgYSBzb2Z0d2FyZSB0aHJlYWQKPiArwqDCoMKg wqBzdWJtaXR0aW5nIGRhdGEgY29tcHJlc3Npb24gcmVxdWVzdHMgdG8gYSBjby1wcm9jZXNzb3Ig KGhhcmR3YXJlL25lc3QKPiArwqDCoMKgwqBhY2NlbGVyYXRvcikgYWthIE5YIGVuZ2luZS4gVGhl IEFQSSBmb3IgdGhpcyB1c2FnZSBpcyBkZXNjcmliZWQgaW4gdGhlCj4gK8KgwqDCoMKgVkFTL05Y IEFQSSBkb2N1bWVudC4KPiArCj4gK8KgwqDCoMKgQWx0ZXJuYXRpdmVseSwgVkFTIGNhbiBiZSB1 c2VkIGJ5IHR3byBzb2Z0d2FyZSB0aHJlYWRzIHRvIGVmZmljaWVudGx5Cj4gK8KgwqDCoMKgZXhj aGFuZ2UgbWVzc2FnZXMuIEluaXRpYWxseSwgdGhpcyBtZWNoYW5pc20gaXMgaW50ZW5kZWQgdG8g d2FrZSB1cCBhCj4gK8KgwqDCoMKgd2FpdGluZyB0aHJlYWQgcXVpY2tseSAtIGkuZSAiZmFzdCB0 aHJlYWQgd2FrZS11cCAoRlRXKSIuIFRoaXMgZG9jdW1lbnQKPiArwqDCoMKgwqBkZXNjcmliZXMg dGhlIHVzZXIgQVBJIGZvciB0aGlzIFZBUy9GVFcgbWVjaGFuaXNtLgo+ICsKPiArwqDCoMKgwqBB cHBsaWNhdGlvbiBhY2Nlc3MgdG8gdGhlIEZUVyBtZWNoYW5pc20gaXMgcHJvdmlkZWQgdGhyb3Vn aCB0aGUgTlgtRlRXCj4gK8KgwqDCoMKgZGV2aWNlIG5vZGUgKC9kZXYvY3J5cHRvL254LWZ0dykg aW1wbGVtZW50ZWQgYnkgdGhlIFZBUy9GVFcgZGV2aWNlCj4gK8KgwqDCoMKgZHJpdmVyLgoKY3J5 cHRvPwoKPiArCj4gK8KgwqDCoMKgQSBzb2Z0d2FyZSB0aHJlYWQgVDEgdGhhdCBpbnRlbmRzIHRv IHdhaXQgZm9yIGFuIGV2ZW50IG11c3QgZmlyc3Qgc2V0dXAKPiArwqDCoMKgwqBhIHJlY2VpdmUg d2luZG93LCBieSBvcGVuaW5nIHRoZSBOWC1GVFcgZGV2aWNlIGFuZCB1c2luZyB0aGUKPiArwqDC oMKgwqBWQVNfUlhfV0lOX09QRU4gaW9jdGwuIFVwb24gc3VjY2Vzc2Z1bCByZXR1cm4gZnJvbSB0 aGUgVkFTX1JYX1dJTl9PUEVOCj4gK8KgwqDCoMKgaW9jdGwsIGFuIHJ4X3dpbl9oYW5kbGUgaXMg cmV0dXJuZWQuCgpJIHJlYWxpc2UgdGhlcmUgaXMgYSB3aW5kb3cgaGVyZSBhcyBwYXJ0IG9mIHRo ZSBoYXJkd2FyZSBpbXBsZW1lbnRhdGlvbiwgYnV0IHRoZQp1c2VycyBkb24ndCBjYXJlIGFib3V0 IHRoZSB3aW5kb3cgb24gdGhlIHJlY2VpdmUgc2lkZS4gSXQncyBoaWRkZW4gZnJvbSB0aGVtLiAK SXQncyBqdXN0IGFuIHJ4IGhhbmRsZSBJTUhPLgoKVGhlIHNlbmRlciBjZXJ0YWlubHkgaGFzIGEg d2luZG93IHRoYXQgdXNlcnMgY2FyZSBhYm91dCBzaW5jZSB0aGV5IGhhdmUgdG8gbW1hcAppdC4K Cj4gKwo+ICvCoMKgwqDCoEEgc29mdHdhcmUgdGhyZWFkIFQyIHRoYXQgaW50ZW5kcyB0byB3YWtl IHVwIFQxIGF0IHNvbWUgcG9pbnQsIG11c3QgZmlyc3QKPiArwqDCoMKgwqBzZXQgdXAgYSAic2Vu ZCB3aW5kb3ciIHVzaW5nIHRoZSBWQVNfVFhfV0lOX09QRU4gaW9jdGwgYW5kIHNwZWNpZnkgdGhl Cj4gK8KgwqDCoMKgcnhfd2luX2hhbmRsZSBvYnRhaW5lZCBieSBUMS4gQWZ0ZXIgYSBzdWNjZXNz ZnVsIFZBU19UWF9XSU5fT1BFTiBpb2N0bAo+IHRoZQo+ICvCoMKgwqDCoHNlbmQgd2luZG93IG9m IFQyIGlzIGNvbnNpZGVyZWQgcGFpcmVkIHdpdGggdGhlIHJlY2VpdmUgd2luZG93IG9mIFQxLiBU aGUKPiArwqDCoMKgwqB0aHJlYWQgVDIgbXVzdCB0aGVuIHVzZSBtbWFwKCkgdG8gb2J0YWluIGEg InBhc3RlIGFkZHJlc3MiIGZvciB0aGUgc2VuZAo+ICvCoMKgwqDCoHdpbmRvdy4KCgo+ICvCoMKg wqDCoFdpdGggdGhpcyBzZXQgdXAsIHRocmVhZCBUMSBjYW4gd2FpdCBmb3IgYW4gZXZlbnQgdXNp bmcgdGhlIFdBSVQKPiArwqDCoMKgwqBpbnN0cnVjdGlvbi4KPiArCj4gK8KgwqDCoMKgVGhyZWFk IFQyIGNhbiB3YWtlIHVwIFQxIGJ5IHVzaW5nIHRoZSAiQ09QWS9QQVNURSIgaW5zdHJ1Y3Rpb25z IGFuZAo+ICvCoMKgwqDCoHN1Ym1pdHRpbmcgYW4gZW1wdHkvTlVMTCBDUkIgdG8gdGhlIHNlbmQg d2luZG93J3MgcGFzdGUgYWRkcmVzcy4gVGhlCj4gK8KgwqDCoMKgd2FpdC93YWtlIHVwIHByb2Nl c3MgY2FuIGJlIHJlcGVhdGVkIGFzIGxvbmcgYXMgdGhlIHRocmVhZHMgaGF2ZSB0aGUKPiArwqDC oMKgwqBzZW5kL3JlY2VpdmUgd2luZG93cyBvcGVuLgoKCgo+ICsxLiBOWC1GVFcgRGV2aWNlIE5v ZGUKPiArCj4gK8KgwqDCoMKgVGhlcmUgaXMgb25lIC9kZXYvY3J5cHRvL254LWZ0dyBub2RlIGlu IHRoZSBzeXN0ZW0gYW5kIGl0IHByb3ZpZGVzCj4gK8KgwqDCoMKgYWNjZXNzIHRvIHRoZSBWQVMv RlRXIGZ1bmN0aW9uYWxpdHkuCgoKPiArwqDCoMKgwqBUaGUgb25seSB2YWxpZCBvcGVyYXRpb25z IG9uIHRoZSBOWC1GVFcgbm9kZSBhcmU6Cj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgLSBvcGVuKCkg dGhlIGRldmljZSBmb3IgcmVhZCBhbmQgd3JpdGUuCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgLSBp c3N1ZSBlaXRoZXIgVkFTX1JYX1dJTl9PUEVOIG9yIFZBU19UWF9XSU5fT1BFTiBpb2N0bHMgdG8g c2V0IHVwCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgcmVjZWl2ZSBvciBzZW5kIChvbmx5IG9uZSBv ZiB0aGVtIHBlciBvcGVuKS4KPiArCj4gK8KgwqDCoMKgwqDCoMKgwqAtIGlmIHRoZSBvcGVuIGlz IGFzc29jaWF0ZWQgd2l0aCBzZW5kIHdpbmRvdyAoaS5lIFZBU19UWF9XSU5fT1BFTgo+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoGlvY3RsIHdhcyBpc3N1ZWQpIG1tYXAoKSB0aGUgc2VuZCB3aW5kb3cg aW50byB0aGUgYXBwbGljYXRpb24ncwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoHZpcnR1YWwgYWRk cmVzcyBzcGFjZS4gKGkuZSBnZXQgYSAncGFzdGVfYWRkcmVzcycgZm9yIHRoZSBzZW5kCj4gK8Kg wqDCoMKgwqDCoMKgwqDCoMKgd2luZG93KS4KPiArCj4gK8KgwqDCoMKgwqDCoMKgwqAtIGNsb3Nl IHRoZSBkZXZpY2Ugbm9kZS4KPiArCj4gK8KgwqDCoMKgT3RoZXIgZmlsZSBvcGVyYXRpb25zIG9u IHRoZSBOWC1GVFcgbm9kZSBhcmUgdW5kZWZpbmVkLgo+ICsKPiArwqDCoMKgwqBOb3RlIHRIQVQg dGhlIENPUFkgYW5kIFBBU1RFIG9wZXJhdGlvbnMgZ28gZGlyZWN0bHkgdG8gdGhlIGhhcmR3YXJl Cj4gK8KgwqDCoMKgYW5kIG5vdCBnbyB0aHJvdWdoIHRoZSBOWC1GVFcgZGV2aWNlLgoKSSBkb24n dCB1bmRlcnN0YW5kIHRoaXMgc3RhdGVtZW50Cgo+ICsKPiArwqDCoMKgwqBBbHRob3VnaCBhIHN5 c3RlbSBtYXkgaGF2ZSBzZXZlcmFsIGluc3RhbmNlcyBvZiB0aGUgVkFTIGluIHRoZSBzeXN0ZW0K PiArwqDCoMKgwqAodHlwaWNhbGx5LCBvbmUgcGVyIFA5IGNoaXApIHRoZXJlIGlzIGp1c3Qgb25l IE5YLUZUVyBkZXZpY2Ugbm9kZSBpbgo+ICvCoMKgwqDCoHRoZSBzeXN0ZW0uCgo+ICsJV2hlbiB0 aGUgTlgtRlRXIGRldmljZSBub2RlIGlzIG9wZW5lZCwgdGhlIGtlcm5lbCBhc3NpZ25zIGEgc3Vp dGFibGUKPiArCWluc3RhbmNlIG9mIFZBUyB0byB0aGUgcHJvY2Vzcy4gS2VybmVsIHdpbGwgbWFr ZSBhIGJlc3QtZWZmb3J0Cj4gYXR0ZW1wdAo+ICsJdG8gYXNzaWduIGFuIG9wdGltYWwgaW5zdGFu Y2Ugb2YgVkFTIGZvciB0aGUgcHJvY2Vzcy4gSW4gdGhlIGluaXRpYWwKPiArwqDCoMKgwqByZWxl YXNlLCB0aGUga2VybmVsIGRvZXMgbm90IHN1cHBvcnQgbWlncmF0aW5nIHRoZSBWQVMgaW5zdGFu Y2UgaWYgdGhlCj4gK8KgwqDCoMKgcHJvY2VzcyBtaWdyYXRlcyBmcm9tIGEgcHJvY2Vzc29yIG9u IG9uZSBjaGlwIHRvIGEgcHJvY2Vzc29yIG9uIGFub3RoZXIKPiArwqDCoMKgwqBjaGlwLgoKSG93 IGlzIGl0ICJvcHRpbWFsIj8KCj4gK8KgwqDCoMKgQXBwbGljYXRpb25zIG1heSBjaG9zZSBhIHNw ZWNpZmljIGluc3RhbmNlIG9mIHRoZSBWQVMgdXNpbmcgdGhlICd2YXNfaWQnCj4gK8KgwqDCoMKg ZmllbGQgaW4gdGhlIFZBU19UWF9XSU5fT1BFTiBhbmQgVkFTX1JYX1dJTl9PUEVOIGlvY3RscyBh cyBkZXRhaWxlZAo+IGJlbG93LgoKCgoKPiArMi4gT3BlbiBOWC1GVFcgbm9kZQo+ICsKPiArwqDC oMKgwqBUaGUgZGV2aWNlIHNob3VsZCBiZSBvcGVuZWQgZm9yIHJlYWQgYW5kIHdyaXRlLiBObyBz cGVjaWFsIHByaXZpbGVnZXMKPiArwqDCoMKgwqBhcmUgbmVlZGVkIHRvIG9wZW4gdGhlIGRldmlj ZS4gVGhlIGRldmljZSBtYXkgYmUgb3BlbmVkIG11bHRpcGxlIHRpbWVzLgo+ICsKPiArwqDCoMKg wqBFYWNoIG9wZW4oKSBvZiB0aGUgTlgtRlRXIGRldmljZSBtYXkgYmUgYXNzb2NpYXRlZCB3aXRo IGVpdGhlciBhIHNlbmQKPiArwqDCoMKgwqB3aW5kb3cgb3IgcmVjZWl2ZSB3aW5kb3cgYnV0IG5v dCBib3RoLgo+ICsKPiArwqDCoMKgwqBTZWUgb3BlbigyKSBzeXN0ZW0gY2FsbCBtYW4gcGFnZXMg Zm9yIG90aGVyIGRldGFpbHMgc3VjaCBhcyByZXR1cm4KPiArwqDCoMKgwqB2YWx1ZXMsIGVycm9y IGNvZGVzIGFuZCByZXN0cmljdGlvbnMuCj4gKwo+ICszLiBTZXR1cCBSZWNlaXZlIHdpbmRvdyAo VkFTX1JYX1dJTl9PUEVOIGlvY3RsKQo+ICsKPiArwqDCoMKgwqBBIHRocmVhZCB0aGF0IGV4cGVj dHMgdG8gd2FpdCBmb3IgZXZlbnRzIGFuZCBiZSB3b2tlbiB1cCB1c2luZyBDT1BZL1BBU1RFCj4g K8KgwqDCoMKgbXVzdCBmaXJzdCBzZXQgdXAgYSByZWNlaXZlIHdpbmRvdyBieSBpc3N1aW5nIHRo ZSBWQVNfUlhfV0lOX09QRU4gaW9jdGwuCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgI2luY2x1ZGUg PGFzbS92YXMuaD4KPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBzdHJ1Y3QgdmFzX3J4X3dpbl9vcGVu X2F0dHIgcnhhdHRyOwo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoHJjID0gaW9jdGwoZmQsIFZBU19S WF9XSU5fT1BFTiwgJnJ4YXR0cik7Cj4gKwo+ICvCoMKgwqDCoFRoZSBhdHRyaWJ1dGVzIG9mIHJ4 YXR0ciBhcmUgYXMgZm9sbG93czoKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBzdHJ1Y3QgdmFzX3J4 X3dpbl9vcGVuX2F0dHIgewo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDE2 X3TCoMKgwqDCoMKgwqDCoHZlcnNpb247Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgaW50MTZfdMKgwqDCoMKgwqDCoMKgdmFzX2lkOwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoGludDMyX3TCoMKgwqDCoMKgwqDCoHJ4X3dpbl9oYW5kbGU7wqDCoMKgwqAvKiBv dXRwdXQgZmllbGQgKi8KPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqBpbnQ2NF90 wqDCoMKgwqDCoMKgwqByZXNlcnZlZFs4XTsKPiArwqDCoMKgwqDCoMKgwqDCoH07Cj4gKwo+ICvC oMKgwqDCoFRoZSB2ZXJzaW9uIGZpZWxkIGlkZW50aWZpZXMgdGhlIHZlcnNpb24gb2YgdGhlIEFQ SSBhbmQgbXVzdCBjdXJyZW50bHkKPiArwqDCoMKgwqBiZSBzZXQgdG8gMS4KPiArCj4gK8KgwqDC oMKgVGhlIHZhc19pZCBmaWVsZCBpZGVudGlmaWVzIGEgc3BlY2lmaWMgaW5zdGFuY2Ugb2YgdGhl IFZBUyB0aGF0IHRoZQo+ICvCoMKgwqDCoGFwcGxpY2F0aW9uIHdpc2hlcyB0byBhY2Nlc3MuIFNl ZSBzZWN0aW9uIG9uIFZBUyBJRCBiZWxvdy4KPiArCj4gK8KgwqDCoMKgVGhlIHJlc2VydmVkIGZp ZWxkIG11c3QgYmUgc2V0IHRvIGFsbCB6ZXJvZXMuCj4gKwo+ICvCoMKgwqDCoFVwb24gc3VjY2Vz c2Z1bCByZXR1cm4gZnJvbSB0aGUgaW9jdGwsIHRoZSByeF93aW5faGFuZGxlIGZpZWxkIGNvbnRh aW5zCj4gK8KgwqDCoMKgYW4gaWRlbnRpZmllciBmb3IgdGhlIFZBUyB3aW5kb3cgYXNzb2NpYXRl ZCB3aXRoIHRoaXMgInNsZWVwaW5nIiB0aHJlYWQuCj4gKwo+ICvCoMKgwqDCoFRoaXMgcnhfd2lu X2hhbmRsZSBmaWVsZCBpcyB1c2VkIHRvICJwYWlyIiB0aGlzIHJlY2VpdmUgd2luZG93IHdpdGgg YQo+ICvCoMKgwqDCoHNlbmQgd2luZG93IGFuZCBtdXN0IGJlIHNwZWNpZmllZCB3aGVuIG9wZW5p bmcgdGhlIGNvcnJlc3BvbmRpbmcgc2VuZAo+ICvCoMKgwqDCoHdpbmRvdyAoc2VlIHN0cnVjdCB2 YXNfdHhfd2luX29wZW5fYXR0ciBiZWxvdykuCj4gKwo+ICvCoMKgwqDCoFJldHVybiB2YWx1ZToK PiArCj4gK8KgwqDCoMKgVGhlIFZBU19SWF9XSU5fT1BFTiBpb2N0bCByZXR1cm5zIDAgb24gc3Vj Y2Vzcy4gT24gZXJyb3IsIGl0IHJldHVybnMgLTEKPiArwqDCoMKgwqBhbmQgc2V0cyB0aGUgZXJy bm8gdmFyaWFibGUgdG8gaW5kaWNhdGUgdGhlIGVycm9yLgo+ICsKPiArwqDCoMKgwqBFcnJvciBj b2RlczoKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBFSU5WQUzCoMKgwqDCoMKgwqB2ZXJzaW9uIGlz IGludmFsaWQKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBFSU5WQUzCoMKgwqDCoMKgwqB2YXNfaWQg aXMgaW52YWxpZAo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoEVJTlZBTMKgwqDCoMKgwqDCoHJlc2Vy dmVkIGZpZWxkIGlzIG5vdCBzZXQgdG8gemVyb2VzCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgRUlO VkFMwqDCoMKgwqDCoMKgZmQgaXMgYWxyZWFkeSBhc3NvY2lhdGVkIHdpdGggYSBzZW5kIHdpbmRv dwo+ICsKPiArCj4gKzMuIFNldCB1cCBhIFNlbmQgd2luZG93IChWQVNfVFhfV0lOX09QRU4gaW9j dGwpCj4gKwo+ICvCoMKgwqDCoEFuIGFwcGxpY2F0aW9uIHRocmVhZCB0aGF0IGV4cGVjdHMgdG8g d2FrZSB1cCBhIHdhaXRpbmcgdGhyZWFkIHVzaW5nCj4gK8KgwqDCoMKgY29weS9wYXN0ZSwgbXVz dCBmaXJzdCBzZXQgdXAgYSBzZW5kIHdpbmRvdyB0aGF0IGlzIHBhaXJlZCB3aXRoIHRoZQo+ICvC oMKgwqDCoHJlY2VpdmUgd2luZG93IG9mIHRoZSB3YWl0aW5nIHRocmVhZC4gVGhpcyBpcyBhY2Nv bXBsaXNoZWQgdXNpbmcgdGhlCj4gK8KgwqDCoMKgVkFTX1RYX1dJTl9PUEVOIGlvY3RsLgo+ICsK PiArwqDCoMKgwqDCoMKgwqDCoCNpbmNsdWRlIDxhc20vdmFzLmg+Cj4gKwo+ICvCoMKgwqDCoMKg wqDCoMKgc3RydWN0IHZhc190eF93aW5fb3Blbl9hdHRyIHR4YXR0cjsKPiArCj4gK8KgwqDCoMKg wqDCoMKgwqByYyA9IGlvY3RsKGZkLCBWQVNfVFhfV0lOX09QRU4sICZ0eGF0dHIpOwoKU28gd2Ug dGFsa2VkIGFib3V0IHRoaXMgb2ZmbGluZSBiZWZvcmUuLi4uIHRoZSBmZCBoZXJlIHNob3VsZCBu b3QgYmUgZnJvbSB0aGUKL2RldiBkZXZpY2UgYnV0IHNob3VsZCBiZSB0aGUgZmQgZnJvbSByeF93 aW5fb3BlbiBpb2N0bC4gCgpBcyB5b3UgaGF2ZSBpdCBoZXJlIHlvdSBwYXNzIHRoZSBoYW5kbGUg aW4gYXMgYSBwYXJhbWV0ZXIgb2YgaW9jdGwuICBUaGlzIG1lYW5zCmFsbCB0aGUgcGVybWlzc2lv bnMgY2hlY2tzIGhhdmUgdG8gYmUgZG9uZSBieSB5b3UgYXMgdG8gaWYgdGhlc2UgdHdvIHdpbmRv d3MgY2FuCmJlIGxpbmtlZC4gIElmIHlvdSB1c2UgdGhlIGZkIGZyb20gYmVmb3JlLCB5b3UgY2Fu IGFzc3VtZSBpZiB0aGUgcmVjZWl2ZXIgaGFzCmdpdmVuIHRoaXMgZmQgdG8gdGhlIHNlbmRlciwg aXQgaGFzIHRoZSByaWdodCBwZXJtaXNzaW9ucy4KCkkgaGF2ZSBzb21lIHBzZXVkbyBjb2RlIGF0 IHRoZSBlbmQgc2hvd3MgdGhpcy4KCj4gK8KgwqDCoMKgVGhlIGF0dHJpYnV0ZXMgJ3R4YXR0cicg Zm9yIHRoZSBWQVNfVFhfV0lOX09QRU4gaW9jdGwgYXJlIGRlZmluZWQgYXMKPiArwqDCoMKgwqBm b2xsb3dzOgo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoHN0cnVjdCB2YXNfdHhfd2luX29wZW5fYXR0 ciB7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDMyX3TCoMKgwqDCoMKgwqDCoHZlcnNp b247Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDE2X3TCoMKgwqDCoMKgwqDCoHZhc19p ZDsKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgdWludDMyX3TCoMKgwqDCoMKgwqByeF93aW5f aGFuZGxlOwo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgaW50NjRfdMKgwqDCoMKgwqDC oMKgcmVzZXJ2ZWQxOwo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgaW50NjRfdMKgwqDC oMKgwqDCoMKgZmxhZ3M7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDY0X3TCoMKgwqDC oMKgwqDCoHJlc2VydmVkMjsKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDMyX3TC oMKgwqDCoMKgwqDCoHRjX21vZGU7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDMyX3TC oMKgwqDCoMKgwqDCoHJzdmRfdHhidWY7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGludDY0 X3TCoMKgwqDCoMKgwqDCoHJlc2VydmVkM1s2XTsKPiArwqDCoMKgwqDCoMKgwqDCoH07Cj4gKwo+ ICvCoMKgwqDCoFRoZSB2ZXJzaW9uIGZpZWxkIG11c3QgY3VycmVudGx5IGJlIHNldCB0byAxLgo+ ICsKPiArwqDCoMKgwqBUaGUgdmFzX2lkIGZpZWxkIGlkZW50aWZpZXMgYSBzcGVjaWZpYyBpbnN0 YW5jZSBvZiB0aGUgVkFTIHRoYXQgdGhlCj4gK8KgwqDCoMKgYXBwbGljYXRpb24gd2lzaGVzIHRv IGFjY2Vzcy4gU2VlIHNlY3Rpb24gb24gVkFTIElEIGJlbG93LgoKQ2FuIHRoaXMgYmUgZGlmZmVy ZW50IHRvIHRoZSByeD8KCj4gK8KgwqDCoMKgVGhlIHJ4X3dpbl9oYW5kbGUgZmllbGQgbXVzdCBi ZSBzZXQgdG8gdGhlIHJ4X3dpbl9oYW5kbGUgcmV0dXJuZWQgYnkKPiArwqDCoMKgwqBhIHByaW9y IHN1Y2Nlc3NmdWwgY2FsbCB0byBWQVNfUlhfV0lOX09QRU4gaW9jdGwgKHNlZSBhYm92ZSkuIFRo aXMKPiArwqDCoMKgwqBmaWVsZCBpcyB1c2VkIHRvIHBhaXIgdGhpcyBzZW5kIHdpbmRvdyB3aXRo IGEgcmVjZWl2ZSB3aW5kb3cuIFRoZQo+ICvCoMKgwqDCoHByb2Nlc3MgbXVzdCBoYXZlIHN1ZmZp Y2llbnQgcGVybWlzc2lvbnMgdG8gY29tbXVuaWNhdGUgd2l0aCB0aGUKPiArwqDCoMKgwqBwcm9j ZXNzIG93bmluZyB0aGUgcmVjZWl2ZSB3aW5kb3cgaWRlbnRpZmllZCBieSByeF93aW5faGFuZGxl LgoKQXMgYWJvdmUsIHRoaXMgc2hvdWxkIGJlIHBhcnQgb2YgdGhlIEZEIG90aGVyd2lzZSB1c2Vy cyBjb3VsZCBzcGVjaWZ5IGFueXRoaW5nCmhlcmUgYW5kIHBhc3RlIHRvIGFueW9uZS4KCj4gK8Kg wqDCoMKgVGhlIHRjX21vZGUgYW5kwqDCoHJzdmRfdHhidWYgZmllbGRzIGFyZSBjdXJyZW50bHkg dW51c2VkIGFuZCBtdXN0IGJlCj4gK8KgwqDCoMKgc2V0IHRvIDAKPiArCj4gK8KgwqDCoMKgVGhl IGZsYWdzIGZpZWxkIHNwZWNpZmllcyBhZGRpdGlvbmFsIGF0dHJpYnV0ZXMgdG8gdGhlIHdpbmRv dy4gVGhlCj4gK8KgwqDCoMKgb25seSB2YWxpZCBiaXQgaW4gdGhlIGZsYWcgYXJlIGZvciBGVFcg d2luZG93cyBpczoKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBWQVNfRkxBR1NfUElOX1dJTkRPV8Kg wqDCoMKgaWYgc2V0LCBpbmRpY2F0ZXMgdGhlIGEgd2luZG93IHNob3VsZCBiZQo+ICvCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg cGlubmVkIGluIGNhY2hlLiBUaGlzIGZsYWcgaXMgcmVzdHJpY3RlZAo+ICvCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgdG8gcHJp dmlsZWdlZCB1c2Vycy4gU2VlIFBpbm5pbmcgd2luZG93cwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgYmVsb3cuCj4gKwo+ ICvCoMKgwqDCoEFsbCB0aGUgb3RoZXIgYml0cyBpbiB0aGUgZmxhZ3MgZmllbGQgbXVzdCBiZSBz ZXQgdG8gMC4KPiArCj4gK8KgwqDCoMKgVGhlIGZpZWxkcyByZXNlcnZlZDEsIHJlc2VydmVkMiBh bmQgcmVzZXJ2ZWQzIGFyZSBmb3IgZnV0dXJlIGV4dGVuc2lvbgo+ICvCoMKgwqDCoGFuZCBtdXN0 IGJlIHNldCB0byAwLgo+ICsKPiArwqDCoMKgwqBSZXR1cm4gdmFsdWU6Cj4gKwo+ICvCoMKgwqDC oFRoZSBWQVNfVFhfV0lOX09QRU4gaW9jdGwgcmV0dXJucyAwIG9uIHN1Y2Nlc3MuIE9uIGVycm9y LCBpdCByZXR1cm5zIC0xCj4gK8KgwqDCoMKgYW5kIHNldHMgdGhlIGVycm5vIHZhcmlhYmxlIHRv IGluZGljYXRlIHRoZSBlcnJvci4KPiArCj4gK8KgwqDCoMKgRXJyb3IgY29uZGl0aW9uczoKPiAr Cj4gK8KgwqDCoMKgwqDCoMKgwqBFSU5WQUzCoMKgwqDCoMKgwqB2ZXJzaW9uLCB2YXNfaWQgb3Ig cnhfd2luX2hhbmRsZSBmaWVsZHMgYXJlIGludmFsaWQKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBF SU5WQUzCoMKgwqDCoMKgwqBmZCBkb2VzIG5vdCByZWZlciB0byBhIHZhbGlkIFZBUyBkZXZpY2Uu Cj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgRUlOVkFMwqDCoMKgwqDCoMKgZmQgaXMgYWxyZWFkeSBh c3NvY2lhdGVkIHdpdGggYSByZWNlaXZlIHdpbmRvdwo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoEVO T1NQQ8KgwqDCoMKgwqDCoFN5c3RlbSBoYXMgdG9vIG1hbnkgYWN0aXZlIHdpbmRvd3MgKGNvbm5l Y3Rpb25zKSBvcGVuLAo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoEVJTlZBTMKgwqDCoMKgwqDCoEZv ciBGVFcgd2luZG93cywgcnN2ZF90eGJ1ZiBpcyBub3QgMC4KPiArCj4gK8KgwqDCoMKgwqDCoMKg wqBFSU5WQUzCoMKgwqDCoMKgwqBGb3IgRlRXIHdpbmRvd3MsIHRjX21vZGUgaXMgbm90IFZBU19U SFJFU0hfRElTQUJMRUQuCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgRVBFUk3CoMKgwqDCoMKgwqDC oFZBU19GTEFHU19QSU5fV0lORE9XIGlzIHNldCBpbiAnZmxhZ3MnIGZpZWxkIGFuZCBwcm9jZXNz Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqBpcyBub3QgcHJpdmls ZWdlZC4KPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBFUEVSTcKgwqDCoMKgwqDCoMKgVkFTX0ZMQUdT X0hJR0hfUFJJIGlzIHNldCBpbiAnZmxhZ3MnIGZpZWxkIGFuZCBwcm9jZXNzCj4gK8KgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqBpcyBub3QgcHJpdmlsZWdlZC4KPiArCj4g K8KgwqDCoMKgwqDCoMKgwqBFSU5WQUzCoMKgwqDCoMKgwqBhbiBpbnZhbGlkIGZsYWcgaXMgc2V0 IGluIHRoZSAnZmxhZ3MnIGZpZWxkLiAoRm9yIEZUVwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgd2luZG93cywgVkFTX0ZMQUdTX0hJR0hfUFJJIGlzIGFsc28gaW52 YWxpZCkuCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgRUlOVkFMwqDCoMKgwqDCoMKgcmVzZXJ2ZWQg ZmllbGRzIGFyZSBub3Qgc2V0IHRvIDAuCj4gKwo+ICvCoMKgwqDCoFNlZSB0aGUgaW9jdGwoMikg bWFuIHBhZ2UgZm9yIG1vcmUgZGV0YWlscywgZXJyb3IgY29kZXMgYW5kIHJlc3RyaWN0aW9ucy4K PiArCj4gKzQuIG1tYXAoKSBOWC1GVFcgZGV2aWNlIGZkCj4gKwo+ICvCoMKgwqDCoFRoZSBtbWFw KCkgc3lzdGVtIGNhbGwgZm9yIGEgTlgtRlRXIGRldmljZSBmZCByZXR1cm5zIGEgInBhc3RlIGFk ZHJlc3MiCj4gK8KgwqDCoMKgdGhhdCB0aGUgYXBwbGljYXRpb24gY2FuIHVzZSB0byBDT1BZL1BB U1RFIGEgQ1JCIHRvIHRoZSB3YWl0aW5nIHRocmVhZC4KPiArCj4gK8KgwqDCoMKgwqDCoMKgwqBw YXN0ZV9hZGRyID0gbW1hcChOVUxMLCBzaXplLCBwcm90LCBmbGFncywgZmQsIG9mZnNldCk7Cj4g Kwo+ICvCoMKgwqDCoFRoZSBtbWFwKCkgb3BlcmF0aW9uIGlzIG9ubHkgdmFsaWQgb24gYSBmaWxl IGRlc2NyaXB0b3IgYXNzb2NpYXRlZAo+ICvCoMKgwqDCoHdpdGggYSBzZW5kIHdpbmRvdy4KPiAr Cj4gK8KgwqDCoMKgT25seSByZXN0cmljdGlvbnMgb24gbW1hcCBmb3IgYSBOWC1GVFcgZGV2aWNl IGZkIGFyZToKPiArCj4gK8KgwqDCoMKgwqDCoMKgwqAtIHNpemUgcGFyYW1ldGVyIHNob3VsZCBi ZSBvbmUgcGFnZSBzaXplCj4gKwo+ICvCoMKgwqDCoMKgwqDCoMKgLSBvZmZzZXQgcGFyYW1ldGVy IHNob3VsZCBiZSAwVUxMLgo+ICsKPiArwqDCoMKgwqBSZWZlciB0byBtbWFwKDIpIG1hbiBwYWdl IGZvciBhZGRpdGlvbmFsIGRldGFpbHMvcmVzdHJpY3Rpb25zLgo+ICsKPiArwqDCoMKgwqBJbiBh ZGRpdGlvbiB0byB0aGUgZXJyb3IgY29uZGl0aW9ucyBsaXN0ZWQgb24gdGhlIG1tYXAoMikgbWFu IHBhZ2UsCj4gK8KgwqDCoMKgbW1hcCgpIGNhbiBhbHNvIGZhaWwgd2l0aCBvbmUgb2YgZm9sbG93 aW5nIGVycm9yIGNvZGVzOgo+ICsKPiArwqDCoMKgwqDCoMKgwqDCoEVJTlZBTMKgwqDCoMKgwqDC oGZkIGlzIG5vdCBhc3NvY2lhdGVkIHdpdGggYW4gb3BlbiBzZW5kIHdpbmRvdyAoaS5lIG1tYXAo KQo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgZG9lcyBub3QgZm9s bG93IGEgc3VjY2Vzc2Z1bCBjYWxsIHRvIHRoZSBWQVNfVFhfV0lOX09QRU4KPiArwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoGlvY3RsKS4KPiArCj4gK8KgwqDCoMKgwqDC oMKgwqBFSU5WQUzCoMKgwqDCoMKgwqBvZmZzZXQgZmllbGQgaXMgbm90IDBVTEwuCj4gKwo+ICsK PiArNS4gVkFTIElECj4gKwo+ICvCoMKgwqDCoEEgc3lzdGVtIG1heSBoYXZlIHNldmVyYWwgaW5z dGFuY2VzIG9mIFZBUyBpbiB0aGUgaGFyZHdhcmUsIHR5cGljYWxseQo+ICvCoMKgwqDCoG9uZSBw ZXIgUE9XRVIgOSBjaGlwLiBUaGUgY2hvaWNlIG9mIGEgc3BlY2lmaWMgaW5zdGFuY2Ugb2YgVkFT IGNhbiBoYXZlCj4gK8KgwqDCoMKgc2lnbmlmaWNhbnQgaW1wYWN0IG9uIHRoZSBwZXJmb3JtYW5j ZSwgc3BlY2lhbGx5IGlmIHRoZSBhcHBsaWNhdGlvbgo+ICvCoMKgwqDCoG1pZ3JhdGVzIGZyb20g b25lIENQVSB0byBhbm90aGVyLiBBcHBsaWNhdGlvbnMgY2FuIHNwZWNpZnkgYSB2YXNfaWQKPiAr wqDCoMKgwqB1c2luZyB0aGUgVkFTX1RYX1dJTl9PUEVOIGFuZCBWQVNfUlhfV0lOX09QRU4gaW9j dGxzIGFuZCBzaG91bGQgYmUKPiArwqDCoMKgwqBwcnVkZW50IGluIGNob29zaW5nIGFuIGluc3Rh bmNlIG9mIFZBUy4KPiArCj4gK8KgwqDCoMKgVGhlIHZhc19pZCBmb3IgZWFjaCBpbnN0YW5jZSBv ZiBWQVMgaXMgbGlzdGVkIGFzIHRoZSBkZXZpY2UgdHJlZQo+ICvCoMKgwqDCoHByb3BlcnR5ICdp Ym0sdmFzLWlkJy4gRGV0ZXJtaW5pbmcgdGhlIHNwZWNpZmljIHZhc19pZCB0byB1c2UgZm9yCj4g K8KgwqDCoMKgYSBzcGVjaWZpYyBhcHBsaWNhdGlvbiB0aHJlYWQgaXMgYmV5b25kIHRoZSBzY29w ZSBvZiB0aGlzIEFQSS4KCkkgd291bGQgbGVhbiB0b3dhcmRzIGhhdmluZyAxIGRldmljZSBwZXIg dmFzL2NoaXAgYnV0IEknbGwgZGVmZXIgdG8gbXBlIGFuZCBiZW5oCm9uIHRoZSBiZXN0IG9wdGlv biBoZXJlLgoKeW91IHBsYW5uaW5nIGEgbGliZnR3IHRvIGRvIHRoaXM/Cgo+ICsKPiArwqDCoMKg wqBJZiB0aGUgYXBwbGljYXRpb24gaGFzIG5vIHByZWZlcmVuY2UsIHRoZSB2YXNfaWQgZmllbGQg bWF5IGJlIHNldCB0bwo+ICvCoMKgwqDCoC0xIGFuZCB0aGUga2VybmVsIHdpbGwgY2hvb3NlIGEg c3VpdGFibGUgaW5zdGFuY2Ugb2YgdGhlIFZBUyBlbmdpbmUuCgorMSAKCj4gKzYuIENPUFkvUEFT VEUgb3BlcmF0aW9uczoKPiArCj4gK8KgwqDCoMKgQXBwbGljYXRpb25zIHNob3VsZCB1c2UgdGhl IENPUFkgYW5kIFBBU1RFIGluc3RydWN0aW9ucyBkZWZpbmVkIGluCj4gK8KgwqDCoMKgdGhlIFJG QyB0byBjb3B5L3Bhc3RlIHRoZSBDUkIuIEZvciBWQVMvRlRXIHVzYWdlLCB0aGUgY29udGVudHMg b2YKPiArwqDCoMKgwqBDUkIgaWYgYW55LCBhcmUgaWdub3JlZC4gQ1JCIGNhbiBiZSBOVUxMLgo+ ICsKPiArNy4gSW50ZXJydXB0IGNvbXBsZXRpb24gYW5kIHNpZ25hbCBoYW5kbGluZwo+ICsKPiAr wqDCoMKgwqBObyBWQVMtc3BlY2lmaWMgc2lnbmFscyB3aWxsIGJlIGdlbmVyYXRlZCB0byB0aGUg YXBwbGljYXRpb24gdGhyZWFkcwo+ICvCoMKgwqDCoHdpdGggdGhlIFZBUy9GVFcgdXNhZ2UuCgor MQoKPiArCj4gKwo+ICs4LiBFeGFtcGxlL1Byb3Bvc2VkIHVzYWdlIG9mIHRoZSBWQVMvRlRXIEFQ SQo+ICsKPiArwqDCoMKgwqBJbiB0aGUgZm9sbG93aW5nIGV4YW1wbGUgd2UgdXNlIHR3byB0aHJl YWRzIHRoYXQgdXNlIHRoZSBWQVMvRlRXIEFQSS4KPiArwqDCoMKgwqBUaHJlYWQgVDEgdXNlcyB0 aGUgV0FJVCBpbnN0cnVjdGlvbiB0byB3YWl0IGZvciBhbiBldmVudC4gVGhyZWFkIFQyCj4gK8Kg wqDCoMKgdXNlcyBjb3B5L3Bhc3RlIGluc3RydWN0aW9ucyB0byB3YWtlIHVwIFQxLgoKU28gaGVy ZSdzIGhvdyBwc2V1ZG8gY29kZSBmb3IgbXkgaWRlYSB3b3VsZCBsb29rIHdpdGggcHRocmVhZHMu IMKgCgpJJ3ZlIGFsc28gYWRkZWQgc29tZSBtZW1vcnkgYmFycmllcnMuIFRoZSBJU0Egc3VnZ2Vz dHMgdGhhdCBjb3B5L3Bhc3RlIGhhcyBubwpvcmRlcmluZyBhc3NvY2lhdGVkIHdpdGggaXQsIHNv IHlvdSBhcmUgZ29pbmcgdG8gbmVlZCB0aGVtIEkgdGhpbmsuIEknbSBub3Qgc3VyZQpvZiB0aGUg Zmxhdm91ciB0aG91Z2guCgotLS0KYm9vbCBkb25lID0gZmFsc2U7CmludCByeGZkOwoKc3RhdGlj IHZvaWQgcmVjaWV2ZXIodm9pZCkKewoJZG8gewrCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoGFzbSgid2FpdCIpOwoJCXNtcF9tYigpOyAvKiBuZWVkZWQgZm9yIHdhaXQgLT4gbWVtb3J5 wqDCoCovCgl9IHdoaWxlICghZG9uZSk7IC8qIGNoZWNrIGZvciBzcHVyaW91cyB3YWtldXAgKi8K CS8qIHdva2VuIHVwISAqLwp9CgpzdGF0aWMgdm9pZCBzZW5kZXIodm9pZCkKewoJdm9pZCAqcGFz dGVfYWRkcjsKCgkvKiBtbWFwIHRoZSByeCBmaWxlIGRlc2NyaXB0b3IgKi8KCXBhc3RlX2FkZHIg PSBtbWFwKE5VTEwsIGdldHBhZ2VzaXplKCksIHByb3QsIE1BUF9TSEFSRUQsIHJ4ZmQsIDApOwoK CWRvbmUgPSB0cnVlOwoJc21wX21iKCk7IC8qIG5lZWRlZCBmb3IgbWVtb3J5IC0+IHBhc3RlICov CsKgwqDCoMKgwqDCoMKgwqB3cml0ZV9jcmIocGFzdGVfYWRkcik7Cn0KCmludCBtYWluKCkKewoJ cHRocmVhZF90IHRocmVhZDsKCWludCBkZXZmZDsKCsKgwqDCoMKgwqDCoMKgwqBkZXZmZCA9IG9w ZW4oIi9kZXYvdmFzLWZ0dyIsIE9fUkRXUik7CgoJLyogY3JlYXRlIGEgbmV3IHJ4IGZpbGUgZGVz Y3JpcHRvciBhc3NvY2lhdGVkIHdpdGggdGhpcyBMUElEL1BJRC9USUQgKi8KwqDCoMKgwqDCoMKg wqDCoHJ4ZmQgPSBpb2N0bChkZXZmZCwgVkFTX1JYX0NSRUFURSk7CgoJcHRocmVhZF9jcmVhdGUo JnRocmVhZCwgTlVMTCwgc2VuZGVyLCBOVUxMKTsKCgkvKiBSZWNpZXZlciBtdXN0ICpub3QqIGJl IGEgbmV3IHRocmVhZCBzaW5jZSBWQVNfUlhfQ1JFQVRFCgnCoMKgwqBpb2N0bCBpcyBhc3NvY2lh dGVkIHdpdGggdGhpcyBMUElEL1BJRC9USUTCoAoJKi8KCXJlY2lldmVyKCk7Cn0KCl== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752830AbdHNKne (ORCPT ); Mon, 14 Aug 2017 06:43:34 -0400 Received: from ozlabs.org ([103.22.144.67]:60143 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752666AbdHNKnd (ORCPT ); Mon, 14 Aug 2017 06:43:33 -0400 Message-ID: <1502707410.9844.10.camel@neuling.org> Subject: Re: [PATCH v6 17/17] powerpc/vas: Document FTW API/usage From: Michael Neuling To: Sukadev Bhattiprolu , Michael Ellerman Cc: Benjamin Herrenschmidt , stewart@linux.vnet.ibm.com, apopple@au1.ibm.com, hbabu@us.ibm.com, oohall@gmail.com, linuxppc-dev@ozlabs.org, linux-kernel@vger.kernel.org Date: Mon, 14 Aug 2017 20:43:30 +1000 In-Reply-To: <1502233622-9330-18-git-send-email-sukadev@linux.vnet.ibm.com> References: <1502233622-9330-1-git-send-email-sukadev@linux.vnet.ibm.com> <1502233622-9330-18-git-send-email-sukadev@linux.vnet.ibm.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6-1ubuntu1 Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v7EAhdVF026038 On Tue, 2017-08-08 at 16:07 -0700, Sukadev Bhattiprolu wrote: > Document the usage of the VAS Fast thread-wakeup API. > > Thanks for input/comments from Benjamin Herrenschmidt, Michael Neuling, > Michael Ellerman, Robert Blackmore, Ian Munsie, Haren Myneni, Paul Mackerras. > > Cc:Ian Munsie > Cc:Paul Mackerras > Signed-off-by: Sukadev Bhattiprolu > --- >  Documentation/powerpc/ftw-api.txt | 373 > ++++++++++++++++++++++++++++++++++++++ >  1 file changed, 373 insertions(+) >  create mode 100644 Documentation/powerpc/ftw-api.txt > > diff --git a/Documentation/powerpc/ftw-api.txt b/Documentation/powerpc/ftw- > api.txt > new file mode 100644 > index 0000000..0b3f16f > --- /dev/null > +++ b/Documentation/powerpc/ftw-api.txt > @@ -0,0 +1,373 @@ > +Virtual Accelerator Switchboard and Fast Thread-Wakeup API > + > +    Power9 processor supports a hardware subystem known as the Virtual > +    Accelerator Switchboard (VAS) which allows two entities in the Power9 > +    system to efficiently exchange messages. Messages must be formatted as > +    Coprocessor Reqeust Blocks (CRB) and be submitted using the COPY/PASTE > +    instructions (new in Power9). > + > +    Usage of VAS depends on the entities exchanging the messages and > +    currently two usages have been identified. > + > +    First usage of VAS, referred to as VAS/NX involves a software thread > +    submitting data compression requests to a co-processor (hardware/nest > +    accelerator) aka NX engine. The API for this usage is described in the > +    VAS/NX API document. > + > +    Alternatively, VAS can be used by two software threads to efficiently > +    exchange messages. Initially, this mechanism is intended to wake up a > +    waiting thread quickly - i.e "fast thread wake-up (FTW)". This document > +    describes the user API for this VAS/FTW mechanism. > + > +    Application access to the FTW mechanism is provided through the NX-FTW > +    device node (/dev/crypto/nx-ftw) implemented by the VAS/FTW device > +    driver. crypto? > + > +    A software thread T1 that intends to wait for an event must first setup > +    a receive window, by opening the NX-FTW device and using the > +    VAS_RX_WIN_OPEN ioctl. Upon successful return from the VAS_RX_WIN_OPEN > +    ioctl, an rx_win_handle is returned. I realise there is a window here as part of the hardware implementation, but the users don't care about the window on the receive side. It's hidden from them. It's just an rx handle IMHO. The sender certainly has a window that users care about since they have to mmap it. > + > +    A software thread T2 that intends to wake up T1 at some point, must first > +    set up a "send window" using the VAS_TX_WIN_OPEN ioctl and specify the > +    rx_win_handle obtained by T1. After a successful VAS_TX_WIN_OPEN ioctl > the > +    send window of T2 is considered paired with the receive window of T1. The > +    thread T2 must then use mmap() to obtain a "paste address" for the send > +    window. > +    With this set up, thread T1 can wait for an event using the WAIT > +    instruction. > + > +    Thread T2 can wake up T1 by using the "COPY/PASTE" instructions and > +    submitting an empty/NULL CRB to the send window's paste address. The > +    wait/wake up process can be repeated as long as the threads have the > +    send/receive windows open. > +1. NX-FTW Device Node > + > +    There is one /dev/crypto/nx-ftw node in the system and it provides > +    access to the VAS/FTW functionality. > +    The only valid operations on the NX-FTW node are: > + > +        - open() the device for read and write. > + > +        - issue either VAS_RX_WIN_OPEN or VAS_TX_WIN_OPEN ioctls to set up > +          receive or send (only one of them per open). > + > +        - if the open is associated with send window (i.e VAS_TX_WIN_OPEN > +          ioctl was issued) mmap() the send window into the application's > +          virtual address space. (i.e get a 'paste_address' for the send > +          window). > + > +        - close the device node. > + > +    Other file operations on the NX-FTW node are undefined. > + > +    Note tHAT the COPY and PASTE operations go directly to the hardware > +    and not go through the NX-FTW device. I don't understand this statement > + > +    Although a system may have several instances of the VAS in the system > +    (typically, one per P9 chip) there is just one NX-FTW device node in > +    the system. > + When the NX-FTW device node is opened, the kernel assigns a suitable > + instance of VAS to the process. Kernel will make a best-effort > attempt > + to assign an optimal instance of VAS for the process. In the initial > +    release, the kernel does not support migrating the VAS instance if the > +    process migrates from a processor on one chip to a processor on another > +    chip. How is it "optimal"? > +    Applications may chose a specific instance of the VAS using the 'vas_id' > +    field in the VAS_TX_WIN_OPEN and VAS_RX_WIN_OPEN ioctls as detailed > below. > +2. Open NX-FTW node > + > +    The device should be opened for read and write. No special privileges > +    are needed to open the device. The device may be opened multiple times. > + > +    Each open() of the NX-FTW device may be associated with either a send > +    window or receive window but not both. > + > +    See open(2) system call man pages for other details such as return > +    values, error codes and restrictions. > + > +3. Setup Receive window (VAS_RX_WIN_OPEN ioctl) > + > +    A thread that expects to wait for events and be woken up using COPY/PASTE > +    must first set up a receive window by issuing the VAS_RX_WIN_OPEN ioctl. > + > +        #include > + > +        struct vas_rx_win_open_attr rxattr; > + > +        rc = ioctl(fd, VAS_RX_WIN_OPEN, &rxattr); > + > +    The attributes of rxattr are as follows: > + > +        struct vas_rx_win_open_attr { > +                int16_t       version; > +                int16_t       vas_id; > +                int32_t       rx_win_handle;    /* output field */ > +                int64_t       reserved[8]; > +        }; > + > +    The version field identifies the version of the API and must currently > +    be set to 1. > + > +    The vas_id field identifies a specific instance of the VAS that the > +    application wishes to access. See section on VAS ID below. > + > +    The reserved field must be set to all zeroes. > + > +    Upon successful return from the ioctl, the rx_win_handle field contains > +    an identifier for the VAS window associated with this "sleeping" thread. > + > +    This rx_win_handle field is used to "pair" this receive window with a > +    send window and must be specified when opening the corresponding send > +    window (see struct vas_tx_win_open_attr below). > + > +    Return value: > + > +    The VAS_RX_WIN_OPEN ioctl returns 0 on success. On error, it returns -1 > +    and sets the errno variable to indicate the error. > + > +    Error codes: > + > +        EINVAL      version is invalid > + > +        EINVAL      vas_id is invalid > + > +        EINVAL      reserved field is not set to zeroes > + > +        EINVAL      fd is already associated with a send window > + > + > +3. Set up a Send window (VAS_TX_WIN_OPEN ioctl) > + > +    An application thread that expects to wake up a waiting thread using > +    copy/paste, must first set up a send window that is paired with the > +    receive window of the waiting thread. This is accomplished using the > +    VAS_TX_WIN_OPEN ioctl. > + > +        #include > + > +        struct vas_tx_win_open_attr txattr; > + > +        rc = ioctl(fd, VAS_TX_WIN_OPEN, &txattr); So we talked about this offline before.... the fd here should not be from the /dev device but should be the fd from rx_win_open ioctl. As you have it here you pass the handle in as a parameter of ioctl. This means all the permissions checks have to be done by you as to if these two windows can be linked. If you use the fd from before, you can assume if the receiver has given this fd to the sender, it has the right permissions. I have some pseudo code at the end shows this. > +    The attributes 'txattr' for the VAS_TX_WIN_OPEN ioctl are defined as > +    follows: > + > +        struct vas_tx_win_open_attr { > +            int32_t       version; > +            int16_t       vas_id; > +            uint32_t      rx_win_handle; > + > +            int64_t       reserved1; > + > +            int64_t       flags; > +            int64_t       reserved2; > + > +            int32_t       tc_mode; > +            int32_t       rsvd_txbuf; > +            int64_t       reserved3[6]; > +        }; > + > +    The version field must currently be set to 1. > + > +    The vas_id field identifies a specific instance of the VAS that the > +    application wishes to access. See section on VAS ID below. Can this be different to the rx? > +    The rx_win_handle field must be set to the rx_win_handle returned by > +    a prior successful call to VAS_RX_WIN_OPEN ioctl (see above). This > +    field is used to pair this send window with a receive window. The > +    process must have sufficient permissions to communicate with the > +    process owning the receive window identified by rx_win_handle. As above, this should be part of the FD otherwise users could specify anything here and paste to anyone. > +    The tc_mode and  rsvd_txbuf fields are currently unused and must be > +    set to 0 > + > +    The flags field specifies additional attributes to the window. The > +    only valid bit in the flag are for FTW windows is: > + > +        VAS_FLAGS_PIN_WINDOW    if set, indicates the a window should be > +                                pinned in cache. This flag is restricted > +                                to privileged users. See Pinning windows > +                                below. > + > +    All the other bits in the flags field must be set to 0. > + > +    The fields reserved1, reserved2 and reserved3 are for future extension > +    and must be set to 0. > + > +    Return value: > + > +    The VAS_TX_WIN_OPEN ioctl returns 0 on success. On error, it returns -1 > +    and sets the errno variable to indicate the error. > + > +    Error conditions: > + > +        EINVAL      version, vas_id or rx_win_handle fields are invalid > + > +        EINVAL      fd does not refer to a valid VAS device. > + > +        EINVAL      fd is already associated with a receive window > + > +        ENOSPC      System has too many active windows (connections) open, > + > +        EINVAL      For FTW windows, rsvd_txbuf is not 0. > + > +        EINVAL      For FTW windows, tc_mode is not VAS_THRESH_DISABLED. > + > +        EPERM       VAS_FLAGS_PIN_WINDOW is set in 'flags' field and process > +                    is not privileged. > + > +        EPERM       VAS_FLAGS_HIGH_PRI is set in 'flags' field and process > +                    is not privileged. > + > +        EINVAL      an invalid flag is set in the 'flags' field. (For FTW > +                    windows, VAS_FLAGS_HIGH_PRI is also invalid). > + > +        EINVAL      reserved fields are not set to 0. > + > +    See the ioctl(2) man page for more details, error codes and restrictions. > + > +4. mmap() NX-FTW device fd > + > +    The mmap() system call for a NX-FTW device fd returns a "paste address" > +    that the application can use to COPY/PASTE a CRB to the waiting thread. > + > +        paste_addr = mmap(NULL, size, prot, flags, fd, offset); > + > +    The mmap() operation is only valid on a file descriptor associated > +    with a send window. > + > +    Only restrictions on mmap for a NX-FTW device fd are: > + > +        - size parameter should be one page size > + > +        - offset parameter should be 0ULL. > + > +    Refer to mmap(2) man page for additional details/restrictions. > + > +    In addition to the error conditions listed on the mmap(2) man page, > +    mmap() can also fail with one of following error codes: > + > +        EINVAL      fd is not associated with an open send window (i.e mmap() > +                    does not follow a successful call to the VAS_TX_WIN_OPEN > +                    ioctl). > + > +        EINVAL      offset field is not 0ULL. > + > + > +5. VAS ID > + > +    A system may have several instances of VAS in the hardware, typically > +    one per POWER 9 chip. The choice of a specific instance of VAS can have > +    significant impact on the performance, specially if the application > +    migrates from one CPU to another. Applications can specify a vas_id > +    using the VAS_TX_WIN_OPEN and VAS_RX_WIN_OPEN ioctls and should be > +    prudent in choosing an instance of VAS. > + > +    The vas_id for each instance of VAS is listed as the device tree > +    property 'ibm,vas-id'. Determining the specific vas_id to use for > +    a specific application thread is beyond the scope of this API. I would lean towards having 1 device per vas/chip but I'll defer to mpe and benh on the best option here. you planning a libftw to do this? > + > +    If the application has no preference, the vas_id field may be set to > +    -1 and the kernel will choose a suitable instance of the VAS engine. +1 > +6. COPY/PASTE operations: > + > +    Applications should use the COPY and PASTE instructions defined in > +    the RFC to copy/paste the CRB. For VAS/FTW usage, the contents of > +    CRB if any, are ignored. CRB can be NULL. > + > +7. Interrupt completion and signal handling > + > +    No VAS-specific signals will be generated to the application threads > +    with the VAS/FTW usage. +1 > + > + > +8. Example/Proposed usage of the VAS/FTW API > + > +    In the following example we use two threads that use the VAS/FTW API. > +    Thread T1 uses the WAIT instruction to wait for an event. Thread T2 > +    uses copy/paste instructions to wake up T1. So here's how pseudo code for my idea would look with pthreads.   I've also added some memory barriers. The ISA suggests that copy/paste has no ordering associated with it, so you are going to need them I think. I'm not sure of the flavour though. --- bool done = false; int rxfd; static void reciever(void) { do {                 asm("wait"); smp_mb(); /* needed for wait -> memory  */ } while (!done); /* check for spurious wakeup */ /* woken up! */ } static void sender(void) { void *paste_addr; /* mmap the rx file descriptor */ paste_addr = mmap(NULL, getpagesize(), prot, MAP_SHARED, rxfd, 0); done = true; smp_mb(); /* needed for memory -> paste */         write_crb(paste_addr); } int main() { pthread_t thread; int devfd;         devfd = open("/dev/vas-ftw", O_RDWR); /* create a new rx file descriptor associated with this LPID/PID/TID */         rxfd = ioctl(devfd, VAS_RX_CREATE); pthread_create(&thread, NULL, sender, NULL); /* Reciever must *not* be a new thread since VAS_RX_CREATE    ioctl is associated with this LPID/PID/TID  */ reciever(); }