From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 938AFC47089 for ; Thu, 27 May 2021 05:48:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 74603613CA for ; Thu, 27 May 2021 05:48:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234485AbhE0FuW (ORCPT ); Thu, 27 May 2021 01:50:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21395 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234399AbhE0FuW (ORCPT ); Thu, 27 May 2021 01:50:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622094529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hrDilVfh/qqPx2XPO3E6zbGSUURQs7c8ZBAfRTr1MU=; b=cYfJ+fwKqWjb5ctKpfS33VmGuEQ+CG6J9ojbpJdGNHlYp+6llAEgjdRcsUR7fJA9WTHjoR xmg5QPVBCnFqMeMqyG7o8JQsm5o4dWDKDIEmCbu44ams+sXfDIPOVjbLOVZpQBiWf4ig9M 7kBjL6caxyN+DDh8OVHnL0MspLA632I= Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-166-vy2dQNzVN8q4fmyNbmKRsw-1; Thu, 27 May 2021 01:48:47 -0400 X-MC-Unique: vy2dQNzVN8q4fmyNbmKRsw-1 Received: by mail-pg1-f199.google.com with SMTP id 1-20020a6306010000b0290215c617f0f8so2304303pgg.8 for ; Wed, 26 May 2021 22:48:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=0hrDilVfh/qqPx2XPO3E6zbGSUURQs7c8ZBAfRTr1MU=; b=KVVUj6KtdYHj8Vp4yKX8nj6Ho54j6P2LvuDgHjvtfd5kuLTHnwXFRPEfpCY9VwvWrs VfqAMt6C8qfEWUljDKbT+N5OYbQxAFTTrVI7uroEOj32SxwvwGvKDXdJM9YMlBUBdzjI jizf9X6Am1QIA6BX8SH1wKPgkoVY8JX1bg0KssKqU9b+8J3mlaXr/AhH1VkI4QNreaIt Qp5wnUZPHVCmagvmGCs7EKN+fWv9ra8MEG1IxKf82BXLg/hyluczBYJ7k7Uk81m8Ml3l 6qBFQjTirYQ4Ic3x6BxS04LiPCBr/hNKdIRHiXTiM3rXBrWdvPFq+UaX+zF1k52E5Ep/ d8Ug== X-Gm-Message-State: AOAM531EZ+7fh3Sx2rgSUuOkG6tOcLtG6tkG58CuQI3lLAkD/oxLcHBz Xw2KmefKKydGXmXGfwJcD8U5oXNhVyD4Iz+TMs2mLZpLqIC/3khze6XeFJVCvpmBNOeevhzlbzz 5ivEsnIWuREJnyU3tUenL5hE= X-Received: by 2002:a63:6cc1:: with SMTP id h184mr2135506pgc.367.1622094525878; Wed, 26 May 2021 22:48:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhiC4pUyikKRVg/fGg/BEWY/NKv/DsEJoI/3n+vZ8sn2JoDl+AANg2WvJ/mb0S+nQ+EAS4Ag== X-Received: by 2002:a63:6cc1:: with SMTP id h184mr2135485pgc.367.1622094525472; Wed, 26 May 2021 22:48:45 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id o186sm873098pfg.170.2021.05.26.22.48.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 May 2021 22:48:44 -0700 (PDT) Subject: Re: [PATCH 3/3] virtio_blk: implement blk_mq_ops->poll() To: Stefan Hajnoczi Cc: virtualization@lists.linux-foundation.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Paolo Bonzini , Jens Axboe , slp@redhat.com, sgarzare@redhat.com, "Michael S. Tsirkin" References: <20210520141305.355961-1-stefanha@redhat.com> <20210520141305.355961-4-stefanha@redhat.com> From: Jason Wang Message-ID: Date: Thu, 27 May 2021 13:48:36 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org 在 2021/5/25 下午4:59, Stefan Hajnoczi 写道: > On Tue, May 25, 2021 at 11:21:41AM +0800, Jason Wang wrote: >> 在 2021/5/20 下午10:13, Stefan Hajnoczi 写道: >>> Request completion latency can be reduced by using polling instead of >>> irqs. Even Posted Interrupts or similar hardware support doesn't beat >>> polling. The reason is that disabling virtqueue notifications saves >>> critical-path CPU cycles on the host by skipping irq injection and in >>> the guest by skipping the irq handler. So let's add blk_mq_ops->poll() >>> support to virtio_blk. >>> >>> The approach taken by this patch differs from the NVMe driver's >>> approach. NVMe dedicates hardware queues to polling and submits >>> REQ_HIPRI requests only on those queues. This patch does not require >>> exclusive polling queues for virtio_blk. Instead, it switches between >>> irqs and polling when one or more REQ_HIPRI requests are in flight on a >>> virtqueue. >>> >>> This is possible because toggling virtqueue notifications is cheap even >>> while the virtqueue is running. NVMe cqs can't do this because irqs are >>> only enabled/disabled at queue creation time. >>> >>> This toggling approach requires no configuration. There is no need to >>> dedicate queues ahead of time or to teach users and orchestration tools >>> how to set up polling queues. >>> >>> Possible drawbacks of this approach: >>> >>> - Hardware virtio_blk implementations may find virtqueue_disable_cb() >>> expensive since it requires DMA. >> >> Note that it's probably not related to the behavior of the driver but the >> design of the event suppression mechanism. >> >> Device can choose to ignore the suppression flag and keep sending >> interrupts. > Yes, it's the design of the event suppression mechanism. > > If we use dedicated polling virtqueues then the hardware doesn't need to > check whether interrupts are enabled for each notification. However, > there's no mechanism to tell the device that virtqueue interrupts are > permanently disabled. This means that as of today, even dedicated > virtqueues cannot suppress interrupts without the device checking for > each notification. This can be detected via a transport specific way. E.g in the case of MSI, VIRTIO_MSI_NO_VECTOR could be a hint. > >>> If such devices become popular then >>> the virtio_blk driver could use a similar approach to NVMe when >>> VIRTIO_F_ACCESS_PLATFORM is detected in the future. >>> >>> - If a blk_poll() thread is descheduled it not only hurts polling >>> performance but also delays completion of non-REQ_HIPRI requests on >>> that virtqueue since vq notifications are disabled. >> >> Can we poll only when only high pri requests are pending? > Yes, that's what this patch does. > >> If the backend is a remote one, I think the polling may cause more cpu >> cycles. > Right, but polling is only done when userspace sets the RWF_HIPRI > request flag. Most applications don't support it and for those that do > it's probably an option that the user needs to enable explicitly. I see. > > Stefan > >>> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c >>> index fc0fb1dcd399..f0243dcd745a 100644 >>> --- a/drivers/block/virtio_blk.c >>> +++ b/drivers/block/virtio_blk.c >>> @@ -29,6 +29,16 @@ static struct workqueue_struct *virtblk_wq; >>> struct virtio_blk_vq { >>> struct virtqueue *vq; >>> spinlock_t lock; >>> + >>> + /* Number of non-REQ_HIPRI requests in flight. Protected by lock. */ >>> + unsigned int num_lopri; >>> + >>> + /* Number of REQ_HIPRI requests in flight. Protected by lock. */ >>> + unsigned int num_hipri; >>> + >>> + /* Are vq notifications enabled? Protected by lock. */ >>> + bool cb_enabled; >> >> We had event_flag_shadow, is it sufficient to introduce a new helper >> virtqueue_cb_is_enabled()? > Yes, I'll try that in the next revision. > >>> + >>> char name[VQ_NAME_LEN]; >>> } ____cacheline_aligned_in_smp; >>> @@ -171,33 +181,67 @@ static inline void virtblk_request_done(struct request *req) >>> blk_mq_end_request(req, virtblk_result(vbr)); >>> } >>> -static void virtblk_done(struct virtqueue *vq) >>> +/* Returns true if one or more requests completed */ >>> +static bool virtblk_complete_requests(struct virtqueue *vq) >>> { >>> struct virtio_blk *vblk = vq->vdev->priv; >>> struct virtio_blk_vq *vbq = &vblk->vqs[vq->index]; >>> bool req_done = false; >>> + bool last_hipri_done = false; >>> struct virtblk_req *vbr; >>> unsigned long flags; >>> unsigned int len; >>> spin_lock_irqsave(&vbq->lock, flags); >>> + >>> do { >>> - virtqueue_disable_cb(vq); >>> + if (vbq->cb_enabled) >>> + virtqueue_disable_cb(vq); >>> while ((vbr = virtqueue_get_buf(vq, &len)) != NULL) { >>> struct request *req = blk_mq_rq_from_pdu(vbr); >>> + if (req->cmd_flags & REQ_HIPRI) { >>> + if (--vbq->num_hipri == 0) >>> + last_hipri_done = true; >>> + } else >>> + vbq->num_lopri--; >>> + >>> if (likely(!blk_should_fake_timeout(req->q))) >>> blk_mq_complete_request(req); >>> req_done = true; >>> } >>> if (unlikely(virtqueue_is_broken(vq))) >>> break; >>> - } while (!virtqueue_enable_cb(vq)); >>> + >>> + /* Enable vq notifications if non-polled requests remain */ >>> + if (last_hipri_done && vbq->num_lopri > 0) { >>> + last_hipri_done = false; >>> + vbq->cb_enabled = true; >>> + } >>> + } while (vbq->cb_enabled && !virtqueue_enable_cb(vq)); >>> /* In case queue is stopped waiting for more buffers. */ >>> if (req_done) >>> blk_mq_start_stopped_hw_queues(vblk->disk->queue, true); >>> spin_unlock_irqrestore(&vbq->lock, flags); >>> + >>> + return req_done; >>> +} >>> + >>> +static int virtblk_poll(struct blk_mq_hw_ctx *hctx) >>> +{ >>> + struct virtio_blk *vblk = hctx->queue->queuedata; >>> + struct virtqueue *vq = vblk->vqs[hctx->queue_num].vq; >>> + >>> + if (!virtqueue_more_used(vq)) >> >> I'm not familiar with block polling but what happens if a buffer is made >> available after virtqueue_more_used() returns false here? > Can you explain the scenario, I'm not sure I understand? "buffer is made > available" -> are you thinking about additional requests being submitted > by the driver or an in-flight request being marked used by the device? Something like that: 1) requests are submitted 2) poll but virtqueue_more_used() return false 3) device make buffer used In this case, will poll() be triggered again by somebody else? (I think interrupt is disabled here). Thanks > > Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7552C47089 for ; Thu, 27 May 2021 05:48:55 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 834EF613DA for ; Thu, 27 May 2021 05:48:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 834EF613DA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 322EC40365; Thu, 27 May 2021 05:48:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P4AYC_uLXAvv; Thu, 27 May 2021 05:48:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTP id 5A0B0401D9; Thu, 27 May 2021 05:48:53 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 32603C000D; Thu, 27 May 2021 05:48:53 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 095DAC0001 for ; Thu, 27 May 2021 05:48:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C6663600CA for ; Thu, 27 May 2021 05:48:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lG3XwYJwr_V9 for ; Thu, 27 May 2021 05:48:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 700EC60733 for ; Thu, 27 May 2021 05:48:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622094529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hrDilVfh/qqPx2XPO3E6zbGSUURQs7c8ZBAfRTr1MU=; b=cYfJ+fwKqWjb5ctKpfS33VmGuEQ+CG6J9ojbpJdGNHlYp+6llAEgjdRcsUR7fJA9WTHjoR xmg5QPVBCnFqMeMqyG7o8JQsm5o4dWDKDIEmCbu44ams+sXfDIPOVjbLOVZpQBiWf4ig9M 7kBjL6caxyN+DDh8OVHnL0MspLA632I= Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-509-N-XIlHG9MuuEyRu_M-R-qg-1; Thu, 27 May 2021 01:48:47 -0400 X-MC-Unique: N-XIlHG9MuuEyRu_M-R-qg-1 Received: by mail-pg1-f200.google.com with SMTP id b17-20020a63eb510000b029021a1da627beso2294294pgk.12 for ; Wed, 26 May 2021 22:48:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=0hrDilVfh/qqPx2XPO3E6zbGSUURQs7c8ZBAfRTr1MU=; b=fcVb8taOe/nid7J5a92jfJZDCBFqmCnzIdiktlRKewYMbeyfHiFfQCWXrU2qncGPQ7 GsoFmJs3HZid7eRfBO5vAnE0DUHs6LvwzFjulqCb6RMBcB0SVKQ/j77xmIP+oTwbvnvn iUMksLhlkO6PxLpJ77uE3ARuk2mZiYupVPI9dsJ6sPDQfKACMDFJOW9+N8213Xkzib3s zuCgvjFP0Ata5cU+irfNmHaus7jLmEVFVdqnlIZ68ZgEbFzR33U3o79TUETldgghQ1EX YhKUFlpEiQWyKwzA3UxqO3geUI0kkHukksmcDFVZyIDDQo8e0ydHlaS2d0UNjnPOIu56 uCgw== X-Gm-Message-State: AOAM531/dNL8feZd7hs7aYh6XJ9gbmjSF9HaeOX1foqNs8DCDpLcTBIl zyP7/A5QSxMMXCKXv+zEuO8c/S1lg+roxbwR2aVxlyx1hllzVQt4Qc+XnudkfrQtfDVDpMP89Mo bQLgE+Qz29P1DBTNpYL46fRXebKTX4OPSoP2ZbHwGvA== X-Received: by 2002:a63:6cc1:: with SMTP id h184mr2135512pgc.367.1622094525882; Wed, 26 May 2021 22:48:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhiC4pUyikKRVg/fGg/BEWY/NKv/DsEJoI/3n+vZ8sn2JoDl+AANg2WvJ/mb0S+nQ+EAS4Ag== X-Received: by 2002:a63:6cc1:: with SMTP id h184mr2135485pgc.367.1622094525472; Wed, 26 May 2021 22:48:45 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id o186sm873098pfg.170.2021.05.26.22.48.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 May 2021 22:48:44 -0700 (PDT) Subject: Re: [PATCH 3/3] virtio_blk: implement blk_mq_ops->poll() To: Stefan Hajnoczi References: <20210520141305.355961-1-stefanha@redhat.com> <20210520141305.355961-4-stefanha@redhat.com> From: Jason Wang Message-ID: Date: Thu, 27 May 2021 13:48:36 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Jens Axboe , "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-block@vger.kernel.org, Paolo Bonzini , Christoph Hellwig X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIxLzUvMjUg5LiL5Y2INDo1OSwgU3RlZmFuIEhham5vY3ppIOWGmemBkzoKPiBPbiBU dWUsIE1heSAyNSwgMjAyMSBhdCAxMToyMTo0MUFNICswODAwLCBKYXNvbiBXYW5nIHdyb3RlOgo+ PiDlnKggMjAyMS81LzIwIOS4i+WNiDEwOjEzLCBTdGVmYW4gSGFqbm9jemkg5YaZ6YGTOgo+Pj4g UmVxdWVzdCBjb21wbGV0aW9uIGxhdGVuY3kgY2FuIGJlIHJlZHVjZWQgYnkgdXNpbmcgcG9sbGlu ZyBpbnN0ZWFkIG9mCj4+PiBpcnFzLiBFdmVuIFBvc3RlZCBJbnRlcnJ1cHRzIG9yIHNpbWlsYXIg aGFyZHdhcmUgc3VwcG9ydCBkb2Vzbid0IGJlYXQKPj4+IHBvbGxpbmcuIFRoZSByZWFzb24gaXMg dGhhdCBkaXNhYmxpbmcgdmlydHF1ZXVlIG5vdGlmaWNhdGlvbnMgc2F2ZXMKPj4+IGNyaXRpY2Fs LXBhdGggQ1BVIGN5Y2xlcyBvbiB0aGUgaG9zdCBieSBza2lwcGluZyBpcnEgaW5qZWN0aW9uIGFu ZCBpbgo+Pj4gdGhlIGd1ZXN0IGJ5IHNraXBwaW5nIHRoZSBpcnEgaGFuZGxlci4gU28gbGV0J3Mg YWRkIGJsa19tcV9vcHMtPnBvbGwoKQo+Pj4gc3VwcG9ydCB0byB2aXJ0aW9fYmxrLgo+Pj4KPj4+ IFRoZSBhcHByb2FjaCB0YWtlbiBieSB0aGlzIHBhdGNoIGRpZmZlcnMgZnJvbSB0aGUgTlZNZSBk cml2ZXIncwo+Pj4gYXBwcm9hY2guIE5WTWUgZGVkaWNhdGVzIGhhcmR3YXJlIHF1ZXVlcyB0byBw b2xsaW5nIGFuZCBzdWJtaXRzCj4+PiBSRVFfSElQUkkgcmVxdWVzdHMgb25seSBvbiB0aG9zZSBx dWV1ZXMuIFRoaXMgcGF0Y2ggZG9lcyBub3QgcmVxdWlyZQo+Pj4gZXhjbHVzaXZlIHBvbGxpbmcg cXVldWVzIGZvciB2aXJ0aW9fYmxrLiBJbnN0ZWFkLCBpdCBzd2l0Y2hlcyBiZXR3ZWVuCj4+PiBp cnFzIGFuZCBwb2xsaW5nIHdoZW4gb25lIG9yIG1vcmUgUkVRX0hJUFJJIHJlcXVlc3RzIGFyZSBp biBmbGlnaHQgb24gYQo+Pj4gdmlydHF1ZXVlLgo+Pj4KPj4+IFRoaXMgaXMgcG9zc2libGUgYmVj YXVzZSB0b2dnbGluZyB2aXJ0cXVldWUgbm90aWZpY2F0aW9ucyBpcyBjaGVhcCBldmVuCj4+PiB3 aGlsZSB0aGUgdmlydHF1ZXVlIGlzIHJ1bm5pbmcuIE5WTWUgY3FzIGNhbid0IGRvIHRoaXMgYmVj YXVzZSBpcnFzIGFyZQo+Pj4gb25seSBlbmFibGVkL2Rpc2FibGVkIGF0IHF1ZXVlIGNyZWF0aW9u IHRpbWUuCj4+Pgo+Pj4gVGhpcyB0b2dnbGluZyBhcHByb2FjaCByZXF1aXJlcyBubyBjb25maWd1 cmF0aW9uLiBUaGVyZSBpcyBubyBuZWVkIHRvCj4+PiBkZWRpY2F0ZSBxdWV1ZXMgYWhlYWQgb2Yg dGltZSBvciB0byB0ZWFjaCB1c2VycyBhbmQgb3JjaGVzdHJhdGlvbiB0b29scwo+Pj4gaG93IHRv IHNldCB1cCBwb2xsaW5nIHF1ZXVlcy4KPj4+Cj4+PiBQb3NzaWJsZSBkcmF3YmFja3Mgb2YgdGhp cyBhcHByb2FjaDoKPj4+Cj4+PiAtIEhhcmR3YXJlIHZpcnRpb19ibGsgaW1wbGVtZW50YXRpb25z IG1heSBmaW5kIHZpcnRxdWV1ZV9kaXNhYmxlX2NiKCkKPj4+ICAgICBleHBlbnNpdmUgc2luY2Ug aXQgcmVxdWlyZXMgRE1BLgo+Pgo+PiBOb3RlIHRoYXQgaXQncyBwcm9iYWJseSBub3QgcmVsYXRl ZCB0byB0aGUgYmVoYXZpb3Igb2YgdGhlIGRyaXZlciBidXQgdGhlCj4+IGRlc2lnbiBvZiB0aGUg ZXZlbnQgc3VwcHJlc3Npb24gbWVjaGFuaXNtLgo+Pgo+PiBEZXZpY2UgY2FuIGNob29zZSB0byBp Z25vcmUgdGhlIHN1cHByZXNzaW9uIGZsYWcgYW5kIGtlZXAgc2VuZGluZwo+PiBpbnRlcnJ1cHRz Lgo+IFllcywgaXQncyB0aGUgZGVzaWduIG9mIHRoZSBldmVudCBzdXBwcmVzc2lvbiBtZWNoYW5p c20uCj4KPiBJZiB3ZSB1c2UgZGVkaWNhdGVkIHBvbGxpbmcgdmlydHF1ZXVlcyB0aGVuIHRoZSBo YXJkd2FyZSBkb2Vzbid0IG5lZWQgdG8KPiBjaGVjayB3aGV0aGVyIGludGVycnVwdHMgYXJlIGVu YWJsZWQgZm9yIGVhY2ggbm90aWZpY2F0aW9uLiBIb3dldmVyLAo+IHRoZXJlJ3Mgbm8gbWVjaGFu aXNtIHRvIHRlbGwgdGhlIGRldmljZSB0aGF0IHZpcnRxdWV1ZSBpbnRlcnJ1cHRzIGFyZQo+IHBl cm1hbmVudGx5IGRpc2FibGVkLiBUaGlzIG1lYW5zIHRoYXQgYXMgb2YgdG9kYXksIGV2ZW4gZGVk aWNhdGVkCj4gdmlydHF1ZXVlcyBjYW5ub3Qgc3VwcHJlc3MgaW50ZXJydXB0cyB3aXRob3V0IHRo ZSBkZXZpY2UgY2hlY2tpbmcgZm9yCj4gZWFjaCBub3RpZmljYXRpb24uCgoKVGhpcyBjYW4gYmUg ZGV0ZWN0ZWQgdmlhIGEgdHJhbnNwb3J0IHNwZWNpZmljIHdheS4KCkUuZyBpbiB0aGUgY2FzZSBv ZiBNU0ksIFZJUlRJT19NU0lfTk9fVkVDVE9SIGNvdWxkIGJlIGEgaGludC4KCgo+Cj4+PiAgICBJ ZiBzdWNoIGRldmljZXMgYmVjb21lIHBvcHVsYXIgdGhlbgo+Pj4gICAgIHRoZSB2aXJ0aW9fYmxr IGRyaXZlciBjb3VsZCB1c2UgYSBzaW1pbGFyIGFwcHJvYWNoIHRvIE5WTWUgd2hlbgo+Pj4gICAg IFZJUlRJT19GX0FDQ0VTU19QTEFURk9STSBpcyBkZXRlY3RlZCBpbiB0aGUgZnV0dXJlLgo+Pj4K Pj4+IC0gSWYgYSBibGtfcG9sbCgpIHRocmVhZCBpcyBkZXNjaGVkdWxlZCBpdCBub3Qgb25seSBo dXJ0cyBwb2xsaW5nCj4+PiAgICAgcGVyZm9ybWFuY2UgYnV0IGFsc28gZGVsYXlzIGNvbXBsZXRp b24gb2Ygbm9uLVJFUV9ISVBSSSByZXF1ZXN0cyBvbgo+Pj4gICAgIHRoYXQgdmlydHF1ZXVlIHNp bmNlIHZxIG5vdGlmaWNhdGlvbnMgYXJlIGRpc2FibGVkLgo+Pgo+PiBDYW4gd2UgcG9sbCBvbmx5 IHdoZW4gb25seSBoaWdoIHByaSByZXF1ZXN0cyBhcmUgcGVuZGluZz8KPiBZZXMsIHRoYXQncyB3 aGF0IHRoaXMgcGF0Y2ggZG9lcy4KPgo+PiBJZiB0aGUgYmFja2VuZCBpcyBhIHJlbW90ZSBvbmUs IEkgdGhpbmsgdGhlIHBvbGxpbmcgbWF5IGNhdXNlIG1vcmUgY3B1Cj4+IGN5Y2xlcy4KPiBSaWdo dCwgYnV0IHBvbGxpbmcgaXMgb25seSBkb25lIHdoZW4gdXNlcnNwYWNlIHNldHMgdGhlIFJXRl9I SVBSSQo+IHJlcXVlc3QgZmxhZy4gTW9zdCBhcHBsaWNhdGlvbnMgZG9uJ3Qgc3VwcG9ydCBpdCBh bmQgZm9yIHRob3NlIHRoYXQgZG8KPiBpdCdzIHByb2JhYmx5IGFuIG9wdGlvbiB0aGF0IHRoZSB1 c2VyIG5lZWRzIHRvIGVuYWJsZSBleHBsaWNpdGx5LgoKCkkgc2VlLgoKCj4KPiBTdGVmYW4KPgo+ Pj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvYmxvY2svdmlydGlvX2Jsay5jIGIvZHJpdmVycy9ibG9j ay92aXJ0aW9fYmxrLmMKPj4+IGluZGV4IGZjMGZiMWRjZDM5OS4uZjAyNDNkY2Q3NDVhIDEwMDY0 NAo+Pj4gLS0tIGEvZHJpdmVycy9ibG9jay92aXJ0aW9fYmxrLmMKPj4+ICsrKyBiL2RyaXZlcnMv YmxvY2svdmlydGlvX2Jsay5jCj4+PiBAQCAtMjksNiArMjksMTYgQEAgc3RhdGljIHN0cnVjdCB3 b3JrcXVldWVfc3RydWN0ICp2aXJ0YmxrX3dxOwo+Pj4gICAgc3RydWN0IHZpcnRpb19ibGtfdnEg ewo+Pj4gICAgCXN0cnVjdCB2aXJ0cXVldWUgKnZxOwo+Pj4gICAgCXNwaW5sb2NrX3QgbG9jazsK Pj4+ICsKPj4+ICsJLyogTnVtYmVyIG9mIG5vbi1SRVFfSElQUkkgcmVxdWVzdHMgaW4gZmxpZ2h0 LiBQcm90ZWN0ZWQgYnkgbG9jay4gKi8KPj4+ICsJdW5zaWduZWQgaW50IG51bV9sb3ByaTsKPj4+ ICsKPj4+ICsJLyogTnVtYmVyIG9mIFJFUV9ISVBSSSByZXF1ZXN0cyBpbiBmbGlnaHQuIFByb3Rl Y3RlZCBieSBsb2NrLiAqLwo+Pj4gKwl1bnNpZ25lZCBpbnQgbnVtX2hpcHJpOwo+Pj4gKwo+Pj4g KwkvKiBBcmUgdnEgbm90aWZpY2F0aW9ucyBlbmFibGVkPyBQcm90ZWN0ZWQgYnkgbG9jay4gKi8K Pj4+ICsJYm9vbCBjYl9lbmFibGVkOwo+Pgo+PiBXZSBoYWQgZXZlbnRfZmxhZ19zaGFkb3csIGlz IGl0IHN1ZmZpY2llbnQgdG8gaW50cm9kdWNlIGEgbmV3IGhlbHBlcgo+PiB2aXJ0cXVldWVfY2Jf aXNfZW5hYmxlZCgpPwo+IFllcywgSSdsbCB0cnkgdGhhdCBpbiB0aGUgbmV4dCByZXZpc2lvbi4K Pgo+Pj4gKwo+Pj4gICAgCWNoYXIgbmFtZVtWUV9OQU1FX0xFTl07Cj4+PiAgICB9IF9fX19jYWNo ZWxpbmVfYWxpZ25lZF9pbl9zbXA7Cj4+PiBAQCAtMTcxLDMzICsxODEsNjcgQEAgc3RhdGljIGlu bGluZSB2b2lkIHZpcnRibGtfcmVxdWVzdF9kb25lKHN0cnVjdCByZXF1ZXN0ICpyZXEpCj4+PiAg ICAJYmxrX21xX2VuZF9yZXF1ZXN0KHJlcSwgdmlydGJsa19yZXN1bHQodmJyKSk7Cj4+PiAgICB9 Cj4+PiAtc3RhdGljIHZvaWQgdmlydGJsa19kb25lKHN0cnVjdCB2aXJ0cXVldWUgKnZxKQo+Pj4g Ky8qIFJldHVybnMgdHJ1ZSBpZiBvbmUgb3IgbW9yZSByZXF1ZXN0cyBjb21wbGV0ZWQgKi8KPj4+ ICtzdGF0aWMgYm9vbCB2aXJ0YmxrX2NvbXBsZXRlX3JlcXVlc3RzKHN0cnVjdCB2aXJ0cXVldWUg KnZxKQo+Pj4gICAgewo+Pj4gICAgCXN0cnVjdCB2aXJ0aW9fYmxrICp2YmxrID0gdnEtPnZkZXYt PnByaXY7Cj4+PiAgICAJc3RydWN0IHZpcnRpb19ibGtfdnEgKnZicSA9ICZ2YmxrLT52cXNbdnEt PmluZGV4XTsKPj4+ICAgIAlib29sIHJlcV9kb25lID0gZmFsc2U7Cj4+PiArCWJvb2wgbGFzdF9o aXByaV9kb25lID0gZmFsc2U7Cj4+PiAgICAJc3RydWN0IHZpcnRibGtfcmVxICp2YnI7Cj4+PiAg ICAJdW5zaWduZWQgbG9uZyBmbGFnczsKPj4+ICAgIAl1bnNpZ25lZCBpbnQgbGVuOwo+Pj4gICAg CXNwaW5fbG9ja19pcnFzYXZlKCZ2YnEtPmxvY2ssIGZsYWdzKTsKPj4+ICsKPj4+ICAgIAlkbyB7 Cj4+PiAtCQl2aXJ0cXVldWVfZGlzYWJsZV9jYih2cSk7Cj4+PiArCQlpZiAodmJxLT5jYl9lbmFi bGVkKQo+Pj4gKwkJCXZpcnRxdWV1ZV9kaXNhYmxlX2NiKHZxKTsKPj4+ICAgIAkJd2hpbGUgKCh2 YnIgPSB2aXJ0cXVldWVfZ2V0X2J1Zih2cSwgJmxlbikpICE9IE5VTEwpIHsKPj4+ICAgIAkJCXN0 cnVjdCByZXF1ZXN0ICpyZXEgPSBibGtfbXFfcnFfZnJvbV9wZHUodmJyKTsKPj4+ICsJCQlpZiAo cmVxLT5jbWRfZmxhZ3MgJiBSRVFfSElQUkkpIHsKPj4+ICsJCQkJaWYgKC0tdmJxLT5udW1faGlw cmkgPT0gMCkKPj4+ICsJCQkJCWxhc3RfaGlwcmlfZG9uZSA9IHRydWU7Cj4+PiArCQkJfSBlbHNl Cj4+PiArCQkJCXZicS0+bnVtX2xvcHJpLS07Cj4+PiArCj4+PiAgICAJCQlpZiAobGlrZWx5KCFi bGtfc2hvdWxkX2Zha2VfdGltZW91dChyZXEtPnEpKSkKPj4+ICAgIAkJCQlibGtfbXFfY29tcGxl dGVfcmVxdWVzdChyZXEpOwo+Pj4gICAgCQkJcmVxX2RvbmUgPSB0cnVlOwo+Pj4gICAgCQl9Cj4+ PiAgICAJCWlmICh1bmxpa2VseSh2aXJ0cXVldWVfaXNfYnJva2VuKHZxKSkpCj4+PiAgICAJCQli cmVhazsKPj4+IC0JfSB3aGlsZSAoIXZpcnRxdWV1ZV9lbmFibGVfY2IodnEpKTsKPj4+ICsKPj4+ ICsJCS8qIEVuYWJsZSB2cSBub3RpZmljYXRpb25zIGlmIG5vbi1wb2xsZWQgcmVxdWVzdHMgcmVt YWluICovCj4+PiArCQlpZiAobGFzdF9oaXByaV9kb25lICYmIHZicS0+bnVtX2xvcHJpID4gMCkg ewo+Pj4gKwkJCWxhc3RfaGlwcmlfZG9uZSA9IGZhbHNlOwo+Pj4gKwkJCXZicS0+Y2JfZW5hYmxl ZCA9IHRydWU7Cj4+PiArCQl9Cj4+PiArCX0gd2hpbGUgKHZicS0+Y2JfZW5hYmxlZCAmJiAhdmly dHF1ZXVlX2VuYWJsZV9jYih2cSkpOwo+Pj4gICAgCS8qIEluIGNhc2UgcXVldWUgaXMgc3RvcHBl ZCB3YWl0aW5nIGZvciBtb3JlIGJ1ZmZlcnMuICovCj4+PiAgICAJaWYgKHJlcV9kb25lKQo+Pj4g ICAgCQlibGtfbXFfc3RhcnRfc3RvcHBlZF9od19xdWV1ZXModmJsay0+ZGlzay0+cXVldWUsIHRy dWUpOwo+Pj4gICAgCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJnZicS0+bG9jaywgZmxhZ3MpOwo+ Pj4gKwo+Pj4gKwlyZXR1cm4gcmVxX2RvbmU7Cj4+PiArfQo+Pj4gKwo+Pj4gK3N0YXRpYyBpbnQg dmlydGJsa19wb2xsKHN0cnVjdCBibGtfbXFfaHdfY3R4ICpoY3R4KQo+Pj4gK3sKPj4+ICsJc3Ry dWN0IHZpcnRpb19ibGsgKnZibGsgPSBoY3R4LT5xdWV1ZS0+cXVldWVkYXRhOwo+Pj4gKwlzdHJ1 Y3QgdmlydHF1ZXVlICp2cSA9IHZibGstPnZxc1toY3R4LT5xdWV1ZV9udW1dLnZxOwo+Pj4gKwo+ Pj4gKwlpZiAoIXZpcnRxdWV1ZV9tb3JlX3VzZWQodnEpKQo+Pgo+PiBJJ20gbm90IGZhbWlsaWFy IHdpdGggYmxvY2sgcG9sbGluZyBidXQgd2hhdCBoYXBwZW5zIGlmIGEgYnVmZmVyIGlzIG1hZGUK Pj4gYXZhaWxhYmxlIGFmdGVyIHZpcnRxdWV1ZV9tb3JlX3VzZWQoKSByZXR1cm5zIGZhbHNlIGhl cmU/Cj4gQ2FuIHlvdSBleHBsYWluIHRoZSBzY2VuYXJpbywgSSdtIG5vdCBzdXJlIEkgdW5kZXJz dGFuZD8gImJ1ZmZlciBpcyBtYWRlCj4gYXZhaWxhYmxlIiAtPiBhcmUgeW91IHRoaW5raW5nIGFi b3V0IGFkZGl0aW9uYWwgcmVxdWVzdHMgYmVpbmcgc3VibWl0dGVkCj4gYnkgdGhlIGRyaXZlciBv ciBhbiBpbi1mbGlnaHQgcmVxdWVzdCBiZWluZyBtYXJrZWQgdXNlZCBieSB0aGUgZGV2aWNlPwoK ClNvbWV0aGluZyBsaWtlIHRoYXQ6CgoxKSByZXF1ZXN0cyBhcmUgc3VibWl0dGVkCjIpIHBvbGwg YnV0IHZpcnRxdWV1ZV9tb3JlX3VzZWQoKSByZXR1cm4gZmFsc2UKMykgZGV2aWNlIG1ha2UgYnVm ZmVyIHVzZWQKCkluIHRoaXMgY2FzZSwgd2lsbCBwb2xsKCkgYmUgdHJpZ2dlcmVkIGFnYWluIGJ5 IHNvbWVib2R5IGVsc2U/IChJIHRoaW5rIAppbnRlcnJ1cHQgaXMgZGlzYWJsZWQgaGVyZSkuCgpU aGFua3MKCgoKPgo+IFN0ZWZhbgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KVmlydHVhbGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxp c3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL3ZpcnR1YWxpemF0aW9u