From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-4468-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [66.179.20.138]) by lists.oasis-open.org (Postfix) with ESMTP id EB3C8581810F for ; Thu, 21 Jun 2018 11:14:31 -0700 (PDT) Date: Thu, 21 Jun 2018 21:14:19 +0300 From: "Michael S. Tsirkin" Message-ID: <20180621211359-mutt-send-email-mst@kernel.org> References: <1525734594-11134-1-git-send-email-sridhar.samudrala@intel.com> <20180605152344-mutt-send-email-mst@kernel.org> <20180612144743-mutt-send-email-mst@kernel.org> <5718de66-74c9-1362-a87b-2eba02475b48@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: [virtio-dev] Re: [PATCH] qemu: Introduce VIRTIO_NET_F_STANDBY feature bit to virtio_net To: Jason Wang Cc: "Samudrala, Sridhar" , virtualization@lists.linux-foundation.org, virtio-dev@lists.oasis-open.org, jesse.brandeburg@intel.com, alexander.h.duyck@intel.com, qemu-devel@nongnu.org List-ID: On Wed, Jun 13, 2018 at 01:40:59PM +0800, Jason Wang wrote: > > > On 2018年06月13日 12:24, Samudrala, Sridhar wrote: > > On 6/12/2018 7:38 PM, Jason Wang wrote: > > > > > > > > > On 2018年06月12日 19:54, Michael S. Tsirkin wrote: > > > > On Wed, Jun 06, 2018 at 10:29:03AM +0800, Jason Wang wrote: > > > > > > > > > > On 2018年06月05日 20:33, Michael S. Tsirkin wrote: > > > > > > I don't think this is sufficient. > > > > > > > > > > > > If both primary and standby devices are present, a > > > > > > legacy guest without > > > > > > support for the feature might see two devices with same mac and get > > > > > > confused. > > > > > > > > > > > > I think that we should only make primary visible after > > > > > > guest acked the > > > > > > backup feature bit. > > > > > I think we want exactly the reverse? E.g fail the > > > > > negotiation when guest > > > > > does not ack backup feature. > > > > > > > > > > Otherwise legacy guest won't even have the chance to see > > > > > primary device in > > > > > the guest. > > > > That's by design. > > > > > > So management needs to know the capability of guest to set the > > > backup feature. This looks a chicken or egg problem to me. > > > > I don't think so. If the tenant requests 'accelerated datapath feature', > > the management > > will set 'standby' feature bit on virtio-net interface and if the guest > > virtio-net driver > > supports this feature, then the tenant VM will get that capability via a > > hot-plugged > > primary device. > > Ok, I thought exactly the reverse because of the commit title is "enable > virtio_net to act as a standby for a passthru device". But re-read the > commit log content, I understand the case a little bit. Btw, VF is not > necessarily faster than virtio-net, especially consider virtio-net may have > a lot of queues. Don't do that then, right? > > > > > > > > > > > > > > > > > And on reset or when backup is cleared in some other way, unplug the > > > > > > primary. > > > > > What if guest does not support hotplug? > > > > It shouldn't ack the backup feature then and it's a good point. > > > > We should both document this and check kernel config has > > > > hotplug support. Sridhar could you take a look pls? > > > > > > > > > > Something like the below will do the job: > > > > > > > > > > > > Primary device is added with a special "primary-failover" flag. > > > > > > A virtual machine is then initialized with just a standby virtio > > > > > > device. Primary is not yet added. > > > > > A question is how to do the matching? Qemu knows nothing about e.g mac > > > > > address of a pass-through device I believe? > > > > Supply a flag to the VFIO when it's added, this way QEMU will know. > > > > > > > > > > Later QEMU detects that guest driver device set DRIVER_OK. > > > > > > It then exposes the primary device to the guest, and triggers > > > > > > a device addition event (hot-plug event) for it. > > > > > Do you mean we won't have primary device in the initial qemu cli? > > > > No, that's not what I mean. > > > > > > > > I mean management will supply a flag to VFIO and then > > > > > > > > > > > > - VFIO defers exposing > > > > primary to guest until guest acks the feature bit. > > > > - When we see guest ack, initiate hotplug. > > > > - On reboot, hide it again. > > > > - On reset without reboot, request hot-unplug and on eject hide > > > > it again. > > > > > > This sounds much like a kind of bonding in qemu. > > > > > > > > > > > > > If QEMU detects guest driver removal, it initiates a > > > > > > hot-unplug sequence > > > > > > to remove the primary driver.  In particular, if QEMU detects guest > > > > > > re-initialization (e.g. by detecting guest reset) it > > > > > > immediately removes > > > > > > the primary device. > > > > > I believe guest failover module should handle this gracefully? > > > > It can't control enumeration order, if primary is enumerated before > > > > standby then guest will load its driver and it's too late > > > > when failover driver is loaded. > > > > > > Well, even if we can delay the visibility of primary until > > > DRIVER_OK, there still be a race I think? And it looks to me it's > > > still a bug of guest: > > > > > > E.g primary could be probed before failover_register() in guest. > > > Then we will miss the enslaving of primary forever. > > > > That is not an issue. Even if the primary is probed before failover > > driver, it will > > enslave the primary via the call to failover_existing_slave_register() > > as part of > > failover_register() routine. > > Aha I get it. So the enumeration order is not an issue. > > Consider primary may still be seen by guest kernel even if we delay its > visibility, I wonder whether we can control the lifecycle of primary through > driver but not qemu. This can simplify a lot of things. > > Thanks > > > > > > > > > Thanks > > > > > > > > > > > > Thanks > > > > > > > > > > > We can move some of this code to management as well, > > > > > > architecturally it > > > > > > does not make too much sense but it might be easier > > > > > > implementation-wise. > > > > > > > > > > > > HTH > > > > > > > > > > > > On Mon, Jun 04, 2018 at 06:41:48PM -0700, Samudrala, Sridhar wrote: > > > > > > > Ping on this patch now that the kernel patches are > > > > > > > accepted into davem's net-next tree. > > > > > > > https://patchwork.ozlabs.org/cover/920005/ > > > > > > > > > > > > > > > > > > > > > On 5/7/2018 4:09 PM, Sridhar Samudrala wrote: > > > > > > > > This feature bit can be used by hypervisor to > > > > > > > > indicate virtio_net device to > > > > > > > > act as a standby for another device with the same MAC address. > > > > > > > > > > > > > > > > I tested this with a small change to the patch > > > > > > > > to mark the STANDBY feature 'true' > > > > > > > > by default as i am using libvirt to start the VMs. > > > > > > > > Is there a way to pass the newly added feature > > > > > > > > bit 'standby' to qemu via libvirt > > > > > > > > XML file? > > > > > > > > > > > > > > > > Signed-off-by: Sridhar Samudrala > > > > > > > > --- > > > > > > > >     hw/net/virtio-net.c                         | 2 ++ > > > > > > > >     include/standard-headers/linux/virtio_net.h | 3 +++ > > > > > > > >     2 files changed, 5 insertions(+) > > > > > > > > > > > > > > > > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c > > > > > > > > index 90502fca7c..38b3140670 100644 > > > > > > > > --- a/hw/net/virtio-net.c > > > > > > > > +++ b/hw/net/virtio-net.c > > > > > > > > @@ -2198,6 +2198,8 @@ static Property virtio_net_properties[] = { > > > > > > > >                          true), > > > > > > > >         DEFINE_PROP_INT32("speed", VirtIONet, > > > > > > > > net_conf.speed, SPEED_UNKNOWN), > > > > > > > >         DEFINE_PROP_STRING("duplex", VirtIONet, > > > > > > > > net_conf.duplex_str), > > > > > > > > +    DEFINE_PROP_BIT64("standby", VirtIONet, > > > > > > > > host_features, VIRTIO_NET_F_STANDBY, > > > > > > > > +                      false), > > > > > > > >         DEFINE_PROP_END_OF_LIST(), > > > > > > > >     }; > > > > > > > > diff --git > > > > > > > > a/include/standard-headers/linux/virtio_net.h > > > > > > > > b/include/standard-headers/linux/virtio_net.h > > > > > > > > index e9f255ea3f..01ec09684c 100644 > > > > > > > > --- a/include/standard-headers/linux/virtio_net.h > > > > > > > > +++ b/include/standard-headers/linux/virtio_net.h > > > > > > > > @@ -57,6 +57,9 @@ > > > > > > > >                          * Steering */ > > > > > > > >     #define VIRTIO_NET_F_CTRL_MAC_ADDR 23    /* Set MAC address */ > > > > > > > > +#define VIRTIO_NET_F_STANDBY      62    /* Act > > > > > > > > as standby for another device > > > > > > > > +                                         * with the same MAC. > > > > > > > > +                                         */ > > > > > > > >     #define VIRTIO_NET_F_SPEED_DUPLEX 63    /* > > > > > > > > Device set linkspeed and duplex */ > > > > > > > >     #ifndef VIRTIO_NET_NO_LEGACY > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCH] qemu: Introduce VIRTIO_NET_F_STANDBY feature bit to virtio_net Date: Thu, 21 Jun 2018 21:14:19 +0300 Message-ID: <20180621211359-mutt-send-email-mst@kernel.org> References: <1525734594-11134-1-git-send-email-sridhar.samudrala@intel.com> <20180605152344-mutt-send-email-mst@kernel.org> <20180612144743-mutt-send-email-mst@kernel.org> <5718de66-74c9-1362-a87b-2eba02475b48@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: Jason Wang Cc: alexander.h.duyck@intel.com, virtio-dev@lists.oasis-open.org, "Samudrala, Sridhar" , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org List-Id: virtualization@lists.linuxfoundation.org T24gV2VkLCBKdW4gMTMsIDIwMTggYXQgMDE6NDA6NTlQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiAKPiBPbiAyMDE45bm0MDbmnIgxM+aXpSAxMjoyNCwgU2FtdWRyYWxhLCBTcmlkaGFy IHdyb3RlOgo+ID4gT24gNi8xMi8yMDE4IDc6MzggUE0sIEphc29uIFdhbmcgd3JvdGU6Cj4gPiA+ IAo+ID4gPiAKPiA+ID4gT24gMjAxOOW5tDA25pyIMTLml6UgMTk6NTQsIE1pY2hhZWwgUy4gVHNp cmtpbiB3cm90ZToKPiA+ID4gPiBPbiBXZWQsIEp1biAwNiwgMjAxOCBhdCAxMDoyOTowM0FNICsw ODAwLCBKYXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gCj4gPiA+ID4gPiBPbiAyMDE45bm0MDbm nIgwNeaXpSAyMDozMywgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+ID4gPiA+ID4gPiBJIGRv bid0IHRoaW5rIHRoaXMgaXMgc3VmZmljaWVudC4KPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+IElm IGJvdGggcHJpbWFyeSBhbmQgc3RhbmRieSBkZXZpY2VzIGFyZSBwcmVzZW50LCBhCj4gPiA+ID4g PiA+IGxlZ2FjeSBndWVzdCB3aXRob3V0Cj4gPiA+ID4gPiA+IHN1cHBvcnQgZm9yIHRoZSBmZWF0 dXJlIG1pZ2h0IHNlZSB0d28gZGV2aWNlcyB3aXRoIHNhbWUgbWFjIGFuZCBnZXQKPiA+ID4gPiA+ ID4gY29uZnVzZWQuCj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiBJIHRoaW5rIHRoYXQgd2Ugc2hv dWxkIG9ubHkgbWFrZSBwcmltYXJ5IHZpc2libGUgYWZ0ZXIKPiA+ID4gPiA+ID4gZ3Vlc3QgYWNr ZWQgdGhlCj4gPiA+ID4gPiA+IGJhY2t1cCBmZWF0dXJlIGJpdC4KPiA+ID4gPiA+IEkgdGhpbmsg d2Ugd2FudCBleGFjdGx5IHRoZSByZXZlcnNlPyBFLmcgZmFpbCB0aGUKPiA+ID4gPiA+IG5lZ290 aWF0aW9uIHdoZW4gZ3Vlc3QKPiA+ID4gPiA+IGRvZXMgbm90IGFjayBiYWNrdXAgZmVhdHVyZS4K PiA+ID4gPiA+IAo+ID4gPiA+ID4gT3RoZXJ3aXNlIGxlZ2FjeSBndWVzdCB3b24ndCBldmVuIGhh dmUgdGhlIGNoYW5jZSB0byBzZWUKPiA+ID4gPiA+IHByaW1hcnkgZGV2aWNlIGluCj4gPiA+ID4g PiB0aGUgZ3Vlc3QuCj4gPiA+ID4gVGhhdCdzIGJ5IGRlc2lnbi4KPiA+ID4gCj4gPiA+IFNvIG1h bmFnZW1lbnQgbmVlZHMgdG8ga25vdyB0aGUgY2FwYWJpbGl0eSBvZiBndWVzdCB0byBzZXQgdGhl Cj4gPiA+IGJhY2t1cCBmZWF0dXJlLiBUaGlzIGxvb2tzIGEgY2hpY2tlbiBvciBlZ2cgcHJvYmxl bSB0byBtZS4KPiA+IAo+ID4gSSBkb24ndCB0aGluayBzby4gSWYgdGhlIHRlbmFudCByZXF1ZXN0 cyAnYWNjZWxlcmF0ZWQgZGF0YXBhdGggZmVhdHVyZScsCj4gPiB0aGUgbWFuYWdlbWVudAo+ID4g d2lsbCBzZXQgJ3N0YW5kYnknIGZlYXR1cmUgYml0IG9uIHZpcnRpby1uZXQgaW50ZXJmYWNlIGFu ZCBpZiB0aGUgZ3Vlc3QKPiA+IHZpcnRpby1uZXQgZHJpdmVyCj4gPiBzdXBwb3J0cyB0aGlzIGZl YXR1cmUsIHRoZW4gdGhlIHRlbmFudCBWTSB3aWxsIGdldCB0aGF0IGNhcGFiaWxpdHkgdmlhIGEK PiA+IGhvdC1wbHVnZ2VkCj4gPiBwcmltYXJ5IGRldmljZS4KPiAKPiBPaywgSSB0aG91Z2h0IGV4 YWN0bHkgdGhlIHJldmVyc2UgYmVjYXVzZSBvZiB0aGUgY29tbWl0IHRpdGxlIGlzICJlbmFibGUK PiB2aXJ0aW9fbmV0IHRvIGFjdCBhcyBhIHN0YW5kYnkgZm9yIGEgcGFzc3RocnUgZGV2aWNlIi4g QnV0IHJlLXJlYWQgdGhlCj4gY29tbWl0IGxvZyBjb250ZW50LCBJIHVuZGVyc3RhbmQgdGhlIGNh c2UgYSBsaXR0bGUgYml0LiBCdHcsIFZGIGlzIG5vdAo+IG5lY2Vzc2FyaWx5IGZhc3RlciB0aGFu IHZpcnRpby1uZXQsIGVzcGVjaWFsbHkgY29uc2lkZXIgdmlydGlvLW5ldCBtYXkgaGF2ZQo+IGEg bG90IG9mIHF1ZXVlcy4KCkRvbid0IGRvIHRoYXQgdGhlbiwgcmlnaHQ/Cgo+ID4gCj4gPiAKPiA+ ID4gCj4gPiA+ID4gCj4gPiA+ID4gPiA+IEFuZCBvbiByZXNldCBvciB3aGVuIGJhY2t1cCBpcyBj bGVhcmVkIGluIHNvbWUgb3RoZXIgd2F5LCB1bnBsdWcgdGhlCj4gPiA+ID4gPiA+IHByaW1hcnku Cj4gPiA+ID4gPiBXaGF0IGlmIGd1ZXN0IGRvZXMgbm90IHN1cHBvcnQgaG90cGx1Zz8KPiA+ID4g PiBJdCBzaG91bGRuJ3QgYWNrIHRoZSBiYWNrdXAgZmVhdHVyZSB0aGVuIGFuZCBpdCdzIGEgZ29v ZCBwb2ludC4KPiA+ID4gPiBXZSBzaG91bGQgYm90aCBkb2N1bWVudCB0aGlzIGFuZCBjaGVjayBr ZXJuZWwgY29uZmlnIGhhcwo+ID4gPiA+IGhvdHBsdWcgc3VwcG9ydC4gU3JpZGhhciBjb3VsZCB5 b3UgdGFrZSBhIGxvb2sgcGxzPwo+ID4gPiA+IAo+ID4gPiA+ID4gPiBTb21ldGhpbmcgbGlrZSB0 aGUgYmVsb3cgd2lsbCBkbyB0aGUgam9iOgo+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gUHJpbWFy eSBkZXZpY2UgaXMgYWRkZWQgd2l0aCBhIHNwZWNpYWwgInByaW1hcnktZmFpbG92ZXIiIGZsYWcu Cj4gPiA+ID4gPiA+IEEgdmlydHVhbCBtYWNoaW5lIGlzIHRoZW4gaW5pdGlhbGl6ZWQgd2l0aCBq dXN0IGEgc3RhbmRieSB2aXJ0aW8KPiA+ID4gPiA+ID4gZGV2aWNlLiBQcmltYXJ5IGlzIG5vdCB5 ZXQgYWRkZWQuCj4gPiA+ID4gPiBBIHF1ZXN0aW9uIGlzIGhvdyB0byBkbyB0aGUgbWF0Y2hpbmc/ IFFlbXUga25vd3Mgbm90aGluZyBhYm91dCBlLmcgbWFjCj4gPiA+ID4gPiBhZGRyZXNzIG9mIGEg cGFzcy10aHJvdWdoIGRldmljZSBJIGJlbGlldmU/Cj4gPiA+ID4gU3VwcGx5IGEgZmxhZyB0byB0 aGUgVkZJTyB3aGVuIGl0J3MgYWRkZWQsIHRoaXMgd2F5IFFFTVUgd2lsbCBrbm93Lgo+ID4gPiA+ IAo+ID4gPiA+ID4gPiBMYXRlciBRRU1VIGRldGVjdHMgdGhhdCBndWVzdCBkcml2ZXIgZGV2aWNl IHNldCBEUklWRVJfT0suCj4gPiA+ID4gPiA+IEl0IHRoZW4gZXhwb3NlcyB0aGUgcHJpbWFyeSBk ZXZpY2UgdG8gdGhlIGd1ZXN0LCBhbmQgdHJpZ2dlcnMKPiA+ID4gPiA+ID4gYSBkZXZpY2UgYWRk aXRpb24gZXZlbnQgKGhvdC1wbHVnIGV2ZW50KSBmb3IgaXQuCj4gPiA+ID4gPiBEbyB5b3UgbWVh biB3ZSB3b24ndCBoYXZlIHByaW1hcnkgZGV2aWNlIGluIHRoZSBpbml0aWFsIHFlbXUgY2xpPwo+ ID4gPiA+IE5vLCB0aGF0J3Mgbm90IHdoYXQgSSBtZWFuLgo+ID4gPiA+IAo+ID4gPiA+IEkgbWVh biBtYW5hZ2VtZW50IHdpbGwgc3VwcGx5IGEgZmxhZyB0byBWRklPIGFuZCB0aGVuCj4gPiA+ID4g Cj4gPiA+ID4gCj4gPiA+ID4gLSBWRklPIGRlZmVycyBleHBvc2luZwo+ID4gPiA+IHByaW1hcnkg dG8gZ3Vlc3QgdW50aWwgZ3Vlc3QgYWNrcyB0aGUgZmVhdHVyZSBiaXQuCj4gPiA+ID4gLSBXaGVu IHdlIHNlZSBndWVzdCBhY2ssIGluaXRpYXRlIGhvdHBsdWcuCj4gPiA+ID4gLSBPbiByZWJvb3Qs IGhpZGUgaXQgYWdhaW4uCj4gPiA+ID4gLSBPbiByZXNldCB3aXRob3V0IHJlYm9vdCwgcmVxdWVz dCBob3QtdW5wbHVnIGFuZCBvbiBlamVjdCBoaWRlCj4gPiA+ID4gaXQgYWdhaW4uCj4gPiA+IAo+ ID4gPiBUaGlzIHNvdW5kcyBtdWNoIGxpa2UgYSBraW5kIG9mIGJvbmRpbmcgaW4gcWVtdS4KPiA+ ID4gCj4gPiA+ID4gCj4gPiA+ID4gPiA+IElmIFFFTVUgZGV0ZWN0cyBndWVzdCBkcml2ZXIgcmVt b3ZhbCwgaXQgaW5pdGlhdGVzIGEKPiA+ID4gPiA+ID4gaG90LXVucGx1ZyBzZXF1ZW5jZQo+ID4g PiA+ID4gPiB0byByZW1vdmUgdGhlIHByaW1hcnkgZHJpdmVyLsKgIEluIHBhcnRpY3VsYXIsIGlm IFFFTVUgZGV0ZWN0cyBndWVzdAo+ID4gPiA+ID4gPiByZS1pbml0aWFsaXphdGlvbiAoZS5nLiBi eSBkZXRlY3RpbmcgZ3Vlc3QgcmVzZXQpIGl0Cj4gPiA+ID4gPiA+IGltbWVkaWF0ZWx5IHJlbW92 ZXMKPiA+ID4gPiA+ID4gdGhlIHByaW1hcnkgZGV2aWNlLgo+ID4gPiA+ID4gSSBiZWxpZXZlIGd1 ZXN0IGZhaWxvdmVyIG1vZHVsZSBzaG91bGQgaGFuZGxlIHRoaXMgZ3JhY2VmdWxseT8KPiA+ID4g PiBJdCBjYW4ndCBjb250cm9sIGVudW1lcmF0aW9uIG9yZGVyLCBpZiBwcmltYXJ5IGlzIGVudW1l cmF0ZWQgYmVmb3JlCj4gPiA+ID4gc3RhbmRieSB0aGVuIGd1ZXN0IHdpbGwgbG9hZCBpdHMgZHJp dmVyIGFuZCBpdCdzIHRvbyBsYXRlCj4gPiA+ID4gd2hlbiBmYWlsb3ZlciBkcml2ZXIgaXMgbG9h ZGVkLgo+ID4gPiAKPiA+ID4gV2VsbCwgZXZlbiBpZiB3ZSBjYW4gZGVsYXkgdGhlIHZpc2liaWxp dHkgb2YgcHJpbWFyeSB1bnRpbAo+ID4gPiBEUklWRVJfT0ssIHRoZXJlIHN0aWxsIGJlIGEgcmFj ZSBJIHRoaW5rPyBBbmQgaXQgbG9va3MgdG8gbWUgaXQncwo+ID4gPiBzdGlsbCBhIGJ1ZyBvZiBn dWVzdDoKPiA+ID4gCj4gPiA+IEUuZyBwcmltYXJ5IGNvdWxkIGJlIHByb2JlZCBiZWZvcmUgZmFp bG92ZXJfcmVnaXN0ZXIoKSBpbiBndWVzdC4KPiA+ID4gVGhlbiB3ZSB3aWxsIG1pc3MgdGhlIGVu c2xhdmluZyBvZiBwcmltYXJ5IGZvcmV2ZXIuCj4gPiAKPiA+IFRoYXQgaXMgbm90IGFuIGlzc3Vl LiBFdmVuIGlmIHRoZSBwcmltYXJ5IGlzIHByb2JlZCBiZWZvcmUgZmFpbG92ZXIKPiA+IGRyaXZl ciwgaXQgd2lsbAo+ID4gZW5zbGF2ZSB0aGUgcHJpbWFyeSB2aWEgdGhlIGNhbGwgdG8gZmFpbG92 ZXJfZXhpc3Rpbmdfc2xhdmVfcmVnaXN0ZXIoKQo+ID4gYXMgcGFydCBvZgo+ID4gZmFpbG92ZXJf cmVnaXN0ZXIoKSByb3V0aW5lLgo+IAo+IEFoYSBJIGdldCBpdC4gU28gdGhlIGVudW1lcmF0aW9u IG9yZGVyIGlzIG5vdCBhbiBpc3N1ZS4KPiAKPiBDb25zaWRlciBwcmltYXJ5IG1heSBzdGlsbCBi ZSBzZWVuIGJ5IGd1ZXN0IGtlcm5lbCBldmVuIGlmIHdlIGRlbGF5IGl0cwo+IHZpc2liaWxpdHks IEkgd29uZGVyIHdoZXRoZXIgd2UgY2FuIGNvbnRyb2wgdGhlIGxpZmVjeWNsZSBvZiBwcmltYXJ5 IHRocm91Z2gKPiBkcml2ZXIgYnV0IG5vdCBxZW11LiBUaGlzIGNhbiBzaW1wbGlmeSBhIGxvdCBv ZiB0aGluZ3MuCj4gCj4gVGhhbmtzCj4gCj4gPiAKPiA+ID4gCj4gPiA+IFRoYW5rcwo+ID4gPiAK PiA+ID4gPiAKPiA+ID4gPiA+IFRoYW5rcwo+ID4gPiA+ID4gCj4gPiA+ID4gPiA+IFdlIGNhbiBt b3ZlIHNvbWUgb2YgdGhpcyBjb2RlIHRvIG1hbmFnZW1lbnQgYXMgd2VsbCwKPiA+ID4gPiA+ID4g YXJjaGl0ZWN0dXJhbGx5IGl0Cj4gPiA+ID4gPiA+IGRvZXMgbm90IG1ha2UgdG9vIG11Y2ggc2Vu c2UgYnV0IGl0IG1pZ2h0IGJlIGVhc2llcgo+ID4gPiA+ID4gPiBpbXBsZW1lbnRhdGlvbi13aXNl Lgo+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gSFRICj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiBP biBNb24sIEp1biAwNCwgMjAxOCBhdCAwNjo0MTo0OFBNIC0wNzAwLCBTYW11ZHJhbGEsIFNyaWRo YXIgd3JvdGU6Cj4gPiA+ID4gPiA+ID4gUGluZyBvbiB0aGlzIHBhdGNoIG5vdyB0aGF0IHRoZSBr ZXJuZWwgcGF0Y2hlcyBhcmUKPiA+ID4gPiA+ID4gPiBhY2NlcHRlZCBpbnRvIGRhdmVtJ3MgbmV0 LW5leHQgdHJlZS4KPiA+ID4gPiA+ID4gPiBodHRwczovL3BhdGNod29yay5vemxhYnMub3JnL2Nv dmVyLzkyMDAwNS8KPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiBP biA1LzcvMjAxOCA0OjA5IFBNLCBTcmlkaGFyIFNhbXVkcmFsYSB3cm90ZToKPiA+ID4gPiA+ID4g PiA+IFRoaXMgZmVhdHVyZSBiaXQgY2FuIGJlIHVzZWQgYnkgaHlwZXJ2aXNvciB0bwo+ID4gPiA+ ID4gPiA+ID4gaW5kaWNhdGUgdmlydGlvX25ldCBkZXZpY2UgdG8KPiA+ID4gPiA+ID4gPiA+IGFj dCBhcyBhIHN0YW5kYnkgZm9yIGFub3RoZXIgZGV2aWNlIHdpdGggdGhlIHNhbWUgTUFDIGFkZHJl c3MuCj4gPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiA+IEkgdGVzdGVkIHRoaXMgd2l0aCBh IHNtYWxsIGNoYW5nZSB0byB0aGUgcGF0Y2gKPiA+ID4gPiA+ID4gPiA+IHRvIG1hcmsgdGhlIFNU QU5EQlkgZmVhdHVyZSAndHJ1ZScKPiA+ID4gPiA+ID4gPiA+IGJ5IGRlZmF1bHQgYXMgaSBhbSB1 c2luZyBsaWJ2aXJ0IHRvIHN0YXJ0IHRoZSBWTXMuCj4gPiA+ID4gPiA+ID4gPiBJcyB0aGVyZSBh IHdheSB0byBwYXNzIHRoZSBuZXdseSBhZGRlZCBmZWF0dXJlCj4gPiA+ID4gPiA+ID4gPiBiaXQg J3N0YW5kYnknIHRvIHFlbXUgdmlhIGxpYnZpcnQKPiA+ID4gPiA+ID4gPiA+IFhNTCBmaWxlPwo+ ID4gPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+ID4gPiBTaWduZWQtb2ZmLWJ5OiBTcmlkaGFyIFNh bXVkcmFsYSA8c3JpZGhhci5zYW11ZHJhbGFAaW50ZWwuY29tPgo+ID4gPiA+ID4gPiA+ID4gLS0t Cj4gPiA+ID4gPiA+ID4gPiDCoMKgwqAgaHcvbmV0L3ZpcnRpby1uZXQuY8KgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8IDIgKysKPiA+ID4gPiA+ID4gPiA+ IMKgwqDCoCBpbmNsdWRlL3N0YW5kYXJkLWhlYWRlcnMvbGludXgvdmlydGlvX25ldC5oIHwgMyAr KysKPiA+ID4gPiA+ID4gPiA+IMKgwqDCoCAyIGZpbGVzIGNoYW5nZWQsIDUgaW5zZXJ0aW9ucygr KQo+ID4gPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvaHcvbmV0L3Zp cnRpby1uZXQuYyBiL2h3L25ldC92aXJ0aW8tbmV0LmMKPiA+ID4gPiA+ID4gPiA+IGluZGV4IDkw NTAyZmNhN2MuLjM4YjMxNDA2NzAgMTAwNjQ0Cj4gPiA+ID4gPiA+ID4gPiAtLS0gYS9ody9uZXQv dmlydGlvLW5ldC5jCj4gPiA+ID4gPiA+ID4gPiArKysgYi9ody9uZXQvdmlydGlvLW5ldC5jCj4g PiA+ID4gPiA+ID4gPiBAQCAtMjE5OCw2ICsyMTk4LDggQEAgc3RhdGljIFByb3BlcnR5IHZpcnRp b19uZXRfcHJvcGVydGllc1tdID0gewo+ID4gPiA+ID4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHRydWUpLAo+ID4gPiA+ID4gPiA+ID4gwqDC oMKgwqDCoMKgwqAgREVGSU5FX1BST1BfSU5UMzIoInNwZWVkIiwgVmlydElPTmV0LAo+ID4gPiA+ ID4gPiA+ID4gbmV0X2NvbmYuc3BlZWQsIFNQRUVEX1VOS05PV04pLAo+ID4gPiA+ID4gPiA+ID4g wqDCoMKgwqDCoMKgwqAgREVGSU5FX1BST1BfU1RSSU5HKCJkdXBsZXgiLCBWaXJ0SU9OZXQsCj4g PiA+ID4gPiA+ID4gPiBuZXRfY29uZi5kdXBsZXhfc3RyKSwKPiA+ID4gPiA+ID4gPiA+ICvCoMKg wqAgREVGSU5FX1BST1BfQklUNjQoInN0YW5kYnkiLCBWaXJ0SU9OZXQsCj4gPiA+ID4gPiA+ID4g PiBob3N0X2ZlYXR1cmVzLCBWSVJUSU9fTkVUX0ZfU1RBTkRCWSwKPiA+ID4gPiA+ID4gPiA+ICvC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZmFsc2UpLAo+ID4gPiA+ ID4gPiA+ID4gwqDCoMKgwqDCoMKgwqAgREVGSU5FX1BST1BfRU5EX09GX0xJU1QoKSwKPiA+ID4g PiA+ID4gPiA+IMKgwqDCoCB9Owo+ID4gPiA+ID4gPiA+ID4gZGlmZiAtLWdpdAo+ID4gPiA+ID4g PiA+ID4gYS9pbmNsdWRlL3N0YW5kYXJkLWhlYWRlcnMvbGludXgvdmlydGlvX25ldC5oCj4gPiA+ ID4gPiA+ID4gPiBiL2luY2x1ZGUvc3RhbmRhcmQtaGVhZGVycy9saW51eC92aXJ0aW9fbmV0LmgK PiA+ID4gPiA+ID4gPiA+IGluZGV4IGU5ZjI1NWVhM2YuLjAxZWMwOTY4NGMgMTAwNjQ0Cj4gPiA+ ID4gPiA+ID4gPiAtLS0gYS9pbmNsdWRlL3N0YW5kYXJkLWhlYWRlcnMvbGludXgvdmlydGlvX25l dC5oCj4gPiA+ID4gPiA+ID4gPiArKysgYi9pbmNsdWRlL3N0YW5kYXJkLWhlYWRlcnMvbGludXgv dmlydGlvX25ldC5oCj4gPiA+ID4gPiA+ID4gPiBAQCAtNTcsNiArNTcsOSBAQAo+ID4gPiA+ID4g PiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICog U3RlZXJpbmcgKi8KPiA+ID4gPiA+ID4gPiA+IMKgwqDCoCAjZGVmaW5lIFZJUlRJT19ORVRfRl9D VFJMX01BQ19BRERSIDIzwqDCoMKgIC8qIFNldCBNQUMgYWRkcmVzcyAqLwo+ID4gPiA+ID4gPiA+ ID4gKyNkZWZpbmUgVklSVElPX05FVF9GX1NUQU5EQlnCoMKgwqDCoMKgIDYywqDCoMKgIC8qIEFj dAo+ID4gPiA+ID4gPiA+ID4gYXMgc3RhbmRieSBmb3IgYW5vdGhlciBkZXZpY2UKPiA+ID4gPiA+ ID4gPiA+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAqIHdpdGggdGhlIHNhbWUgTUFDLgo+ID4g PiA+ID4gPiA+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICovCj4gPiA+ID4gPiA+ID4gPiDC oMKgwqAgI2RlZmluZSBWSVJUSU9fTkVUX0ZfU1BFRURfRFVQTEVYIDYzwqDCoMKgIC8qCj4gPiA+ ID4gPiA+ID4gPiBEZXZpY2Ugc2V0IGxpbmtzcGVlZCBhbmQgZHVwbGV4ICovCj4gPiA+ID4gPiA+ ID4gPiDCoMKgwqAgI2lmbmRlZiBWSVJUSU9fTkVUX05PX0xFR0FDWQo+ID4gPiAKPiA+IApfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpWaXJ0dWFsaXphdGlv biBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcK aHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vdmlydHVh bGl6YXRpb24=