From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F067FC6FD1C for ; Fri, 24 Mar 2023 09:06:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8430D81A5C; Fri, 24 Mar 2023 09:06:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8430D81A5C Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TOpLJMiF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24bxnbWjOkYP; Fri, 24 Mar 2023 09:06:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8AB0981886; Fri, 24 Mar 2023 09:06:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8AB0981886 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3E74BC0071; Fri, 24 Mar 2023 09:06:10 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26C17C0032 for ; Fri, 24 Mar 2023 09:06:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0C6D941BB9 for ; Fri, 24 Mar 2023 09:06:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0C6D941BB9 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TOpLJMiF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TdWPfSp39VTM for ; Fri, 24 Mar 2023 09:06:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4F3BE41BB1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 4F3BE41BB1 for ; Fri, 24 Mar 2023 09:06:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679648765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7+GqAWmz15rP7oViR7OeAok2f70x1pUo5xW0EoL4Usg=; b=TOpLJMiFX+Lg0OC7HuCkF9Bz/OjfIYqDhkafqc9LVSBZmtIyuXtUKV/UEKdXjBCzVBOd81 C2hGcvwXvqjmy2Pt9Y94Ws8XTGfCEqPM8q32OrWa2gZYlTFwUWwIr80pa/Lc9HsQZpBL4J wa3s6kGIpQVJtAzyZkLbs11TLzhE5hw= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-501-2HCUcVG0O8iaSooKz3-Uow-1; Fri, 24 Mar 2023 05:06:03 -0400 X-MC-Unique: 2HCUcVG0O8iaSooKz3-Uow-1 Received: by mail-wr1-f72.google.com with SMTP id k16-20020adfd230000000b002cfe7555486so88398wrh.13 for ; Fri, 24 Mar 2023 02:06:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679648762; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7+GqAWmz15rP7oViR7OeAok2f70x1pUo5xW0EoL4Usg=; b=HXXj6wSZXl+eVvSV5j/vqmuTr+/SIKLsH3zyd7LOvP1ps7Gap1BdCwK55HMQo1hUmm 6oF1cRvopwlC+D3jmOqMNaS0DqKRiNXVsceMDuOQcmYK3XzRxKm9eqaJ67yP9HDbkmzP Rcv6pyjwF9JmR97jpP3LyjzNuDPe9EDuGOmzZ8twqcN/ctxmNGTEQhYqm5rS4inQoWE/ jJGcVlWV0uG4BNrmOaQT7Dz35QaALNhKEil6OYj8ybBRdYCUZhC+0FkqIIZ02JhL4sYd BW+CnW02FMaXbYdxBh27xnR/VeaJw40HbMpFXGhnRmT2hSBdmI4IA6ZeHA+2Rlz9mb/g 2wKQ== X-Gm-Message-State: AO0yUKUcIJhTxSLZ+i6Qi7codpa22ohGvdC8vlDvsMROZZ2b90FIz5qK fJ9KwmGvsyE0AsQO//1LyCnTjmjVssxM4seT7RXVk7XD35//NolVluq2QOfRMta+7braloxuyTx qC/f+AoDRpro4mA3rQaccT9pphySszCVeHISQnZkHUQ== X-Received: by 2002:a1c:4b13:0:b0:3ed:33a1:ba8e with SMTP id y19-20020a1c4b13000000b003ed33a1ba8emr1829893wma.1.1679648762355; Fri, 24 Mar 2023 02:06:02 -0700 (PDT) X-Google-Smtp-Source: AK7set+tRNl1cyqajon+Q3sDihZIdsLxHbeZ/aojwtRsQlg0XR4tPIAi4yaR5/T6R667BcDUr6RZPw== X-Received: by 2002:a1c:4b13:0:b0:3ed:33a1:ba8e with SMTP id y19-20020a1c4b13000000b003ed33a1ba8emr1829875wma.1.1679648761970; Fri, 24 Mar 2023 02:06:01 -0700 (PDT) Received: from redhat.com ([2.52.12.190]) by smtp.gmail.com with ESMTPSA id d5-20020adfef85000000b002cfed482e9asm17997062wro.61.2023.03.24.02.06.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Mar 2023 02:06:01 -0700 (PDT) Date: Fri, 24 Mar 2023 05:05:58 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [External] Re: [PATCH] virtio_ring: Suppress tx interrupt when napi_tx disable Message-ID: <20230324050424-mutt-send-email-mst@kernel.org> References: <20230321085953.24949-1-huangjie.albert@bytedance.com> <20230324013805-mutt-send-email-mst@kernel.org> <20230324024155-mutt-send-email-mst@kernel.org> <20230324025937-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: =?utf-8?B?6buE5p2w?= , open list , "open list:VIRTIO CORE AND NET DRIVERS" X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gRnJpLCBNYXIgMjQsIDIwMjMgYXQgMDM6Mzc6MDRQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiBPbiBGcmksIE1hciAyNCwgMjAyMyBhdCAzOjAw4oCvUE0gTWljaGFlbCBTLiBUc2lya2lu IDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPgo+ID4gT24gRnJpLCBNYXIgMjQsIDIwMjMgYXQg MDI6NDc6MDJQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ID4gT24gRnJpLCBNYXIgMjQs IDIwMjMgYXQgMjo0MuKAr1BNIE1pY2hhZWwgUy4gVHNpcmtpbiA8bXN0QHJlZGhhdC5jb20+IHdy b3RlOgo+ID4gPiA+Cj4gPiA+ID4gT24gRnJpLCBNYXIgMjQsIDIwMjMgYXQgMDI6MzI6NDBQTSAr MDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ID4gPiA+IE9uIEZyaSwgTWFyIDI0LCAyMDIzIGF0 IDE6NTnigK9QTSBNaWNoYWVsIFMuIFRzaXJraW4gPG1zdEByZWRoYXQuY29tPiB3cm90ZToKPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4gT24gRnJpLCBNYXIgMjQsIDIwMjMgYXQgMTE6NDE6MTJBTSAr MDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ID4gPiA+ID4gPiBPbiBUaHUsIE1hciAyMywgMjAy MyBhdCA0OjAx4oCvUE0g6buE5p2wIDxodWFuZ2ppZS5hbGJlcnRAYnl0ZWRhbmNlLmNvbT4gd3Jv dGU6Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gSmFzb24gV2FuZyA8amFzb3dhbmdA cmVkaGF0LmNvbT4g5LqOMjAyM+W5tDPmnIgyMuaXpeWRqOS4iSAxMDozN+WGmemBk++8mgo+ID4g PiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiBPbiBUdWUsIE1hciAyMSwgMjAyMyBhdCA1 OjAw4oCvUE0gQWxiZXJ0IEh1YW5nCj4gPiA+ID4gPiA+ID4gPiA+IDxodWFuZ2ppZS5hbGJlcnRA Ynl0ZWRhbmNlLmNvbT4gd3JvdGU6Cj4gPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ ID4gPiBGcm9tOiAiaHVhbmdqaWUuYWxiZXJ0IiA8aHVhbmdqaWUuYWxiZXJ0QGJ5dGVkYW5jZS5j b20+Cj4gPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiBmaXggY29tbWl0IDhk NjIyZDIxZDI0OCAoInZpcnRpbzogZml4IHVwIHZpcnRpb19kaXNhYmxlX2NiIikKPiA+ID4gPiA+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiA+IGlmIHdlIGRpc2FibGUgdGhlIG5hcGlfdHgu IHdoZW4gd2UgdHJpZ2VyIGEgdHggaW50ZXJydXB0LCB0aGUKPiA+ID4gPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gPiA+ID4gdHlwbyBzaG91bGQgYmUgInRyaWdnZXIiCj4gPiA+ID4gPiA+ID4gPiA+ Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gT0ssIHRoYW5rcyBmb3IgdGhpcy4gSSB3 aWxsIGNvcnJlY3QgaXQgaW4gdGhlIG5leHQgdmVyc2lvbgo+ID4gPiA+ID4gPiA+ID4KPiA+ID4g PiA+ID4gPiA+ID4gPiB2cS0+ZXZlbnRfdHJpZ2dlcmVkIHdpbGwgYmUgc2V0IHRvIHRydWUuIEl0 IHdpbGwgbm8gbG9uZ2VyIGJlCj4gPiA+ID4gPiA+ID4gPiA+ID4gc2V0IHRvIGZhbHNlLiBVbmxl c3Mgd2UgZXhwbGljaXRseSBjYWxsIHZpcnRxdWV1ZV9lbmFibGVfY2JfZGVsYXllZAo+ID4gPiA+ ID4gPiA+ID4gPiA+IG9yIHZpcnRxdWV1ZV9lbmFibGVfY2JfcHJlcGFyZQo+ID4gPiA+ID4gPiA+ ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiA+ID4gaWYgd2UgZGlzYWJsZSB0aGUgbmFwaV90eCwgSXQg d2lsbCBvbmx5IGJlIGNhbGxlZCB3aGVuIHRoZSB0eCByaW5nCj4gPiA+ID4gPiA+ID4gPiA+ID4g YnVmZmVyIGlzIHJlbGF0aXZlbHkgc21hbGw6Cj4gPiA+ID4gPiA+ID4gPiA+ID4gdmlydGlvX25l dC0+c3RhcnRfeG1pdDoKPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgICAgIGlmIChzcS0+dnEtPm51 bV9mcmVlIDwgMitNQVhfU0tCX0ZSQUdTKSB7Cj4gPiA+ID4gPiA+ID4gPiA+ID4gICAgICAgICAg ICAgICAgIG5ldGlmX3N0b3Bfc3VicXVldWUoZGV2LCBxbnVtKTsKPiA+ID4gPiA+ID4gPiA+ID4g PiAgICAgICAgICAgICAgICAgaWYgKCF1c2VfbmFwaSAmJgo+ID4gPiA+ID4gPiA+ID4gPiA+ICAg ICAgICAgICAgICAgICAgICAgdW5saWtlbHkoIXZpcnRxdWV1ZV9lbmFibGVfY2JfZGVsYXllZChz cS0+dnEpKSkgewo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgIC8q IE1vcmUganVzdCBnb3QgdXNlZCwgZnJlZSB0aGVtIHRoZW4gcmVjaGVjay4gKi8KPiA+ID4gPiA+ ID4gPiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBmcmVlX29sZF94bWl0X3NrYnMoc3Es IGZhbHNlKTsKPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICBpZiAo c3EtPnZxLT5udW1fZnJlZSA+PSAyK01BWF9TS0JfRlJBR1MpIHsKPiA+ID4gPiA+ID4gPiA+ID4g PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5ldGlmX3N0YXJ0X3N1YnF1ZXVlKGRl diwgcW51bSk7Cj4gPiA+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICB2aXJ0cXVldWVfZGlzYWJsZV9jYihzcS0+dnEpOwo+ID4gPiA+ID4gPiA+ID4gPiA+ICAg ICAgICAgICAgICAgICAgICAgICAgIH0KPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ ID4gVGhlIGNvZGUgZXhhbXBsZSBoZXJlIGlzIG91dCBvZiBkYXRlLCBtYWtlIHN1cmUgeW91ciB0 cmVlIGhhcyB0aGlzOgo+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+IGFsc28sIEkgd2ls bCBjb3JyZWN0IGl0IGluIHRoZSBuZXh0IHZlcnNpb27vvIx0aGlzIGlzIGZyb20ga2VybmVsIDUu MTUuCj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiBj b21taXQgZDcxZWJlODExNGI0YmY2MjI4MDRiODEwZjVlMjc0MDY5MDYwYTE3NAo+ID4gPiA+ID4g PiA+ID4gPiBBdXRob3I6IEphc29uIFdhbmcgPGphc293YW5nQHJlZGhhdC5jb20+Cj4gPiA+ID4g PiA+ID4gPiA+IERhdGU6ICAgVHVlIEphbiAxNyAxMTo0NzowNyAyMDIzICswODAwCj4gPiA+ID4g PiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiA+ICAgICB2aXJ0aW8tbmV0OiBjb3JyZWN0bHkgZW5h YmxlIGNhbGxiYWNrIGR1cmluZyBzdGFydF94bWl0Cj4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4g PiA+ID4gPiA+ID4gICAgICAgICAgICAgICAgIH0KPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgICAg IH0KPiA+ID4gPiA+ID4gPiA+ID4gPiBCZWNhdXNlIGV2ZW50X3RyaWdnZXJlZCBpcyB0cnVlLlRo ZXJlZm9yZSwgVlJJTkdfQVZBSUxfRl9OT19JTlRFUlJVUFQgb3IKPiA+ID4gPiA+ID4gPiA+ID4g PiBWUklOR19QQUNLRURfRVZFTlRfRkxBR19ESVNBQkxFIHdpbGwgbm90IGJlIHNldC5TbyB3ZSB1 cGRhdGUKPiA+ID4gPiA+ID4gPiA+ID4gPiB2cmluZ191c2VkX2V2ZW50KCZ2cS0+c3BsaXQudnJp bmcpIG9yIHZxLT5wYWNrZWQudnJpbmcuZHJpdmVyLT5vZmZfd3JhcAo+ID4gPiA+ID4gPiA+ID4g PiA+IGV2ZXJ5IHRpbWUgd2UgY2FsbCB2aXJ0cXVldWVfZ2V0X2J1Zl9jdHguVGhpcyB3aWxsIGJy aW5nIG1vcmUgaW50ZXJydXB0aW9ucy4KPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ ID4gQ2FuIHlvdSBwbGVhc2UgcG9zdCBob3cgdG8gdGVzdCB3aXRoIHRoZSBwZXJmb3JtYW5jZSBu dW1iZXJzPwo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ IGlwZXJmMyB0Y3Agc3RyZWFtOgo+ID4gPiA+ID4gPiA+ID4gdm0xIC0tLS0tLS0tLS0tLS0tLS0t PiB2bTIKPiA+ID4gPiA+ID4gPiA+IHZtMiBqdXN0IHJlY2VpdmUgdGNwIGRhdGEgc3RyZWFtIGZy b20gdm0xLCBhbmQgc2VuZCB0aGUgYWNrIHRvIHZtMSwKPiA+ID4gPiA+ID4gPiA+IHRoZXJlIGFy ZSBzbwo+ID4gPiA+ID4gPiA+ID4gbWFueSB0eCBpbnRlcnJ1cHRpb25zICBpbiB2bTIuCj4gPiA+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gYnV0IHdpdGhvdXQgZXZlbnRfdHJpZ2dlcmVkIHRo ZXJlIGFyZSBqdXN0IGEgZmV3IHR4IGludGVycnVwdGlvbnMuCj4gPiA+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiA+ID4gaWYgZXZlbnRfdHJpZ2dlcmVkIGlz IHNldCB0byB0cnVlLCBkbyBub3QgdXBkYXRlIHZyaW5nX3VzZWRfZXZlbnQoJnZxLT5zcGxpdC52 cmluZykKPiA+ID4gPiA+ID4gPiA+ID4gPiBvciB2cS0+cGFja2VkLnZyaW5nLmRyaXZlci0+b2Zm X3dyYXAKPiA+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiA+IFNpZ25lZC1vZmYt Ynk6IGh1YW5namllLmFsYmVydCA8aHVhbmdqaWUuYWxiZXJ0QGJ5dGVkYW5jZS5jb20+Cj4gPiA+ ID4gPiA+ID4gPiA+ID4gLS0tCj4gPiA+ID4gPiA+ID4gPiA+ID4gIGRyaXZlcnMvdmlydGlvL3Zp cnRpb19yaW5nLmMgfCA2ICsrKystLQo+ID4gPiA+ID4gPiA+ID4gPiA+ICAxIGZpbGUgY2hhbmdl ZCwgNCBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQo+ID4gPiA+ID4gPiA+ID4gPiA+Cj4g PiA+ID4gPiA+ID4gPiA+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvdmlydGlvL3ZpcnRpb19yaW5n LmMgYi9kcml2ZXJzL3ZpcnRpby92aXJ0aW9fcmluZy5jCj4gPiA+ID4gPiA+ID4gPiA+ID4gaW5k ZXggMzA3ZTEzOWNiMTFkLi5mNDg2Y2NjYWRiZWIgMTAwNjQ0Cj4gPiA+ID4gPiA+ID4gPiA+ID4g LS0tIGEvZHJpdmVycy92aXJ0aW8vdmlydGlvX3JpbmcuYwo+ID4gPiA+ID4gPiA+ID4gPiA+ICsr KyBiL2RyaXZlcnMvdmlydGlvL3ZpcnRpb19yaW5nLmMKPiA+ID4gPiA+ID4gPiA+ID4gPiBAQCAt Nzk1LDcgKzc5NSw4IEBAIHN0YXRpYyB2b2lkICp2aXJ0cXVldWVfZ2V0X2J1Zl9jdHhfc3BsaXQo c3RydWN0IHZpcnRxdWV1ZSAqX3ZxLAo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgLyogSWYg d2UgZXhwZWN0IGFuIGludGVycnVwdCBmb3IgdGhlIG5leHQgZW50cnksIHRlbGwgaG9zdAo+ID4g PiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICogYnkgd3JpdGluZyBldmVudCBpbmRleCBhbmQgZmx1 c2ggb3V0IHRoZSB3cml0ZSBiZWZvcmUKPiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAqIHRo ZSByZWFkIGluIHRoZSBuZXh0IGdldF9idWYgY2FsbC4gKi8KPiA+ID4gPiA+ID4gPiA+ID4gPiAt ICAgICAgIGlmICghKHZxLT5zcGxpdC5hdmFpbF9mbGFnc19zaGFkb3cgJiBWUklOR19BVkFJTF9G X05PX0lOVEVSUlVQVCkpCj4gPiA+ID4gPiA+ID4gPiA+ID4gKyAgICAgICBpZiAoISh2cS0+c3Bs aXQuYXZhaWxfZmxhZ3Nfc2hhZG93ICYgVlJJTkdfQVZBSUxfRl9OT19JTlRFUlJVUFQpCj4gPiA+ ID4gPiA+ID4gPiA+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgJiYgKHZxLT5ldmVudF90cmln Z2VyZWQgPT0gZmFsc2UpKQo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiBJJ20g bm90IHN1cmUgdGhpcyBjYW4gd29yaywgd2hlbiBldmVudF90cmlnZ2VyZWQgaXMgdHJ1ZSBpdCBt ZWFucwo+ID4gPiA+ID4gPiA+ID4gPiB3ZSd2ZSBnb3QgYW4gaW50ZXJydXB0LCBpbiB0aGlzIGNh c2UgaWYgd2Ugd2FudCBhbm90aGVyIGludGVycnVwdCBmb3IKPiA+ID4gPiA+ID4gPiA+ID4gdGhl IG5leHQgZW50cnksIHdlIHNob3VsZCB1cGRhdGUgdXNlZF9ldmVudCBvdGhlcndpc2Ugd2Ugd2ls bCBsb3NlCj4gPiA+ID4gPiA+ID4gPiA+IHRoYXQgaW50ZXJydXB0Pwo+ID4gPiA+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPiA+ID4gPiBUaGFua3MKPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4g PiBOb3JtYWxseSwgaWYgd2UgcmVjZWl2ZSBhbiBpbnRlcnJ1cHQsIHdlIHNob3VsZCBkaXNhYmxl IHRoZSBpbnRlcnJ1cHQKPiA+ID4gPiA+ID4gPiA+IGluIHRoZSBpbnRlcnJ1cHQgY2FsbGJhY2sg aGFuZGxlci4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+IFNvIHRoZSBwcm9ibGVtIGlzOgo+ ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gMSkgZXZlbnRfdHJpZ2dlcmVkIHdhcyBzZXQgdG8g dHJ1ZSBpbiB2cmluZ19pbnRlcnJ1cHQoKQo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gMikg YWZ0ZXIgdGhpcyBub3RoaW5nIHdpbGwgaGFwcGVuIGZvciB2aXJ0cXVldWVfZGlzYWJsZV9jYigp IHNvCj4gPiA+ID4gPiA+ID4gVlJJTkdfQVZBSUxfRl9OT19JTlRFUlJVUFQgaXMgbm90IHNldCBp biBhdmFpbF9mbGFnc19zaGFkb3cKPiA+ID4gPiA+ID4gPiAzKSB2aXJ0cXVldWVfZ2V0X2J1Zl9j dHhfc3BsaXQoKSB3aWxsIHN0aWxsIHRoaW5rIHRoZSBjYiBpcyBlbmFibGVkCj4gPiA+ID4gPiA+ ID4gdGhlbiBpdCB0cmllcyB0byBwdWJsaXNoIG5ldyBldmVudAo+ID4gPiA+ID4gPgo+ID4gPiA+ ID4gPiBPaC4gR29vZCBwb2ludCEgSSB0aGluayB3aGVuIEkgd3JvdGUgdXAKPiA+ID4gPiA+ID4g OGQ2MjJkMjFkMjQ4ICgidmlydGlvOiBmaXggdXAgdmlydGlvX2Rpc2FibGVfY2IiKQo+ID4gPiA+ ID4gPiBJIG1pc3NlZCB0aGlzIGNvcm5lciBjYXNlLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+IFRoaXMgbWFrZXMgbWUgdGhpbmsgYWJvdXQgd2hldGhl ciBvciBub3Qgd2UgcmVhbGx5IG5lZWQKPiA+ID4gPiA+ID4gPiBldmVudF90cmlnZ2VyZWQuIFRo ZSBhc3N1bXB0aW9uIGluIHRoZSB2aXJ0cXVldWVfZGlzYWJsZV9jYigpIHNlZW1zCj4gPiA+ID4g PiA+ID4gd3Jvbmc6Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiAvKiBJZiBkZXZpY2UgdHJp Z2dlcmVkIGFuIGV2ZW50IGFscmVhZHkgaXQgd29uJ3QgdHJpZ2dlciBvbmUgYWdhaW46Cj4gPiA+ ID4gPiA+ID4gICogbm8gbmVlZCB0byBkaXNhYmxlLgo+ID4gPiA+ID4gPiA+ICAqLwo+ID4gPiA+ ID4gPiA+IGlmICh2cS0+ZXZlbnRfdHJpZ2dlcmVkKQo+ID4gPiA+ID4gPiA+ICAgICAgICAgICAg ICAgICByZXR1cm47Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBUaGlzIGlzIHdyb25nIGlm IHRoZXJlJ3Mgbm8gZXZlbnQgaW5kZXggc3VwcG9ydC4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4K PiA+ID4gPiA+ID4gSSBkb24ndCBnZXQgaXQuICBob3cgZG9lcyB0aGlzIGdldCB0cmlnZ2VyZWQ/ Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IFlvdSBhcmUgdGFsa2luZyBhYm91dCBkZXZpY2Ugd2l0 aG91dCBldmVudCBpbmRleD8KPiA+ID4gPiA+ID4gSGVyZSdzIGNvZGUgZnJvbSB2cmluZ19pbnRl cnJ1cHQoKToKPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gICAgICAgICAvKiBKdXN0IGEgaGludCBm b3IgcGVyZm9ybWFuY2U6IHNvIGl0J3Mgb2sgdGhhdCB0aGlzIGNhbiBiZSByYWN5ISAqLwo+ID4g PiA+ID4gPiAgICAgICAgIGlmICh2cS0+ZXZlbnQpCj4gPiA+ID4gPiA+ICAgICAgICAgICAgICAg ICB2cS0+ZXZlbnRfdHJpZ2dlcmVkID0gdHJ1ZTsKPiA+ID4gPiA+Cj4gPiA+ID4gPiBCdXQgd2Ug aGF2ZSB0aGUgZm9sbG93aW5nIGluIHZpcnRxdWV1ZV9kaXNhYmxlX2NiKCk6Cj4gPiA+ID4gPgo+ ID4gPiA+ID4gICAgICAgICAvKiBJZiBkZXZpY2UgdHJpZ2dlcmVkIGFuIGV2ZW50IGFscmVhZHkg aXQgd29uJ3QgdHJpZ2dlciBvbmUgYWdhaW46Cj4gPiA+ID4gPiAgICAgICAgICAqIG5vIG5lZWQg dG8gZGlzYWJsZS4KPiA+ID4gPiA+ICAgICAgICAgICovCj4gPiA+ID4gPiAgICAgICAgIGlmICh2 cS0+ZXZlbnRfdHJpZ2dlcmVkKQo+ID4gPiA+ID4gICAgICAgICAgICAgICAgIHJldHVybjsKPiA+ ID4gPiA+Cj4gPiA+ID4gPiAgICAgICAgIGlmICh2cS0+cGFja2VkX3JpbmcpCj4gPiA+ID4gPiAg ICAgICAgICAgICAgICAgdmlydHF1ZXVlX2Rpc2FibGVfY2JfcGFja2VkKF92cSk7Cj4gPiA+ID4g PiAgICAgICAgIGVsc2UKPiA+ID4gPiA+ICAgICAgICAgICAgICAgICB2aXJ0cXVldWVfZGlzYWJs ZV9jYl9zcGxpdChfdnEpOwo+ID4gPiA+ID4KPiA+ID4gPiA+IFRoaXMgbWVhbnMsIHdpdGhvdXQg YW4gZXZlbnQgaW5kZXgsIHdlIGRvbid0IHNldCBhdmFpbCBmbGFncy4gU28gdGhlCj4gPiA+ID4g PiBpbnRlcnJ1cHQgaXMgbm90IGRpc2FibGVkIGFjdHVhbGx5IGluIHRoaXMgY2FzZS4KPiA+ID4g PiA+Cj4gPiA+ID4gPiBUaGFua3MKPiA+ID4gPgo+ID4gPiA+IE9ubHkgaWYgZXZlbnRfdHJpZ2dl cmVkIGlzIHRydWUsIHdoaWNoIHdpdGhvdXQgZXZlbnQgaW5kZXggaXQgbmV2ZXIgaXMuCj4gPiA+ Cj4gPiA+IEknbSBub3Qgc3VyZSBJIHdpbGwgZ2V0IGhlcmUuIEkgbWVhbnQgZm9yIGV4YW1wbGUg dGhlIGNvbW1pdAo+ID4gPiBzdXBwcmVzc2VzIHRoZSBlZmZvcnQgb2Ygc2tiX3htaXRfZG9uZSgp Ogo+ID4gPgo+ID4gPiBzdGF0aWMgdm9pZCBza2JfeG1pdF9kb25lKHN0cnVjdCB2aXJ0cXVldWUg KnZxKQo+ID4gPiB7Cj4gPiA+ICAgICAgICAgc3RydWN0IHZpcnRuZXRfaW5mbyAqdmkgPSB2cS0+ dmRldi0+cHJpdjsKPiA+ID4gICAgICAgICBzdHJ1Y3QgbmFwaV9zdHJ1Y3QgKm5hcGkgPSAmdmkt PnNxW3ZxMnR4cSh2cSldLm5hcGk7Cj4gPiA+Cj4gPiA+ICAgICAgICAgLyogU3VwcHJlc3MgZnVy dGhlciBpbnRlcnJ1cHRzLiAqLwo+ID4gPiAgICAgICAgIHZpcnRxdWV1ZV9kaXNhYmxlX2NiKHZx KTsKPiA+ID4KPiA+ID4gVGhlIHZpcnRxdWV1ZV9kaXNhYmxlX2NiKCkgZG9lc24ndCBkaXNhYmxl IGZ1cnRoZXIgaW50ZXJydXB0cyB3aGVuIHRoZQo+ID4gPiBldmVudCBpbmRleCBpcyBub3QgdGhl cmUuCj4gPiA+Cj4gPiA+IFRoYW5rcwo+ID4KPiA+IENoZWNrIHdoYXQgY2FuIHNldCBldmVudF90 cmlnZ2VyZWQsIHlvdSB3aWxsIHNlZS4KPiAKPiBTZXQgdG8gdHJ1dGggYnkgdnJpbmdfaW50ZXJy dXB0KCkKCnZyaW5nX2ludGVycnVwdCBvbmx5IHNldHMgaXQgdG8gdHJ1ZSBpZiB2cS0+ZXZlbnQg aXMgdHJ1ZQoKCj4gU2V0IHRvIGZhbHNlIGJ5IHZpcnRxdWV1ZV9pbml0KCksIHZpcnRxdWV1ZV9l bmFibGVfY2JfcHJlcGFyZSgpLAo+IHZpcnRxdWV1ZV9lbmFibGVfY2JfZGVsYXllZCgpCj4gCj4g QXNzdW1pbmcgTkFQSSBUWCBpcyBlbmFibGVkIGFuZCB0aGUgZGV2aWNlIGRvZXNuJ3Qgc3VwcG9y dCBldmVudCBpbmRleC4KPiAKPiAxKSBkcml2ZXIgc2VuZHMgcGFja2V0cyAxLTEwCj4gMikgdGhl IHN0YXJ0X3htaXQoKSBmb3IgdGhlIGxhc3QgcGFja2V0IHdpbGwgY2FsbAo+IHZpcnRxdWV1ZV9l bmFibGVfY2JfZGVsYXllZCgpIHdoaWNoIHNldCBldmVudF90cmlnZ2VyZWQgPSBmYWxzZQo+IDMp IDFzdCBwYWNrZXQgd2VyZSBzZW50LCB2cmluZ19pbnRlcnJ1cHQgc2V0IGV2ZW50X3RyaWdnZXJl ZCA9IHRydWUKPiA0KSBza2JfeG1pdF9kb25lKCkgd29uJ3QgZGlzYWJsZSB2aXJ0cXVldWVfZGlz YWJsZV9jYigpIGluIHRoaXMgY2FzZQo+IDUpIHNvIHdlIHdpbGwgZ2V0IHRoZSBpbnRlcnJ1cHRz IGZvciAybmQgdG8gMTB0aCBwYWNrZXQKPiAKPiBBbnl0aGluZyBJIG1pc3NlZCBoZXJlPwoKMyBk b2VzIG5vdCBoYXBwZW4gaWYgZXZlbnQgaW5kZXggaXMgb2ZmLgoKPiBOb3RlIHRoZSBjb21tZW50 IHNhaWQgaXQncyB1c2VkIGZvciBldmVudCBpbmRleDoKPiAKPiAgICAgICAgIC8qIEhpbnQgZm9y IGV2ZW50IGlkeDogYWxyZWFkeSB0cmlnZ2VyZWQgbm8gbmVlZCB0byBkaXNhYmxlLiAqLwo+ICAg ICAgICAgYm9vbCBldmVudF90cmlnZ2VyZWQ7Cj4gCj4gSSBndWVzcyB3aGF0IHlvdSBtZWFudCBp cyB0aGF0IGlmIHdlIGRvbid0IHB1Ymxpc2ggYSBuZXcgZXZlbnQsIHdlCj4gd2lsbCBnZXQgYXQg bW9zdCAxIGludGVycnVwdCBmb3IgZXZlcnl0aGluZyAkcXVldWVfc2l6ZSB1c2VkIGJ1ZmZlcnMu Cj4gQnV0IHRoaXMgaXMgbm90IHRoZSBjYXNlIHdpdGhvdXQgZXZlbnQgaW5kZXguIEJ0dywgaXQg bWF5IHN1cHJlc3MgdGhlCj4gZWZmb3J0IG9mOgo+IAo+IHZyaW5nX3VzZWRfZXZlbnQoJnZxLT5z cGxpdC52cmluZykgPSAweDA7Cj4gCj4gVGhhbmtzCgoKQmVjYXVzZSBpdCdzIG5vdCBuZWNlc3Nh cnkgdGhlbi4KCj4gPgo+ID4KPiA+Cj4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBBbmQgdGhlCj4gPiA+ID4gPiA+ID4g ZXZlbnRfdHJpZ2dlcmVkIGlzIHNvbWVob3cgZHVwbGljYXRlZCB3aXRoIHRoZQo+ID4gPiA+ID4g PiA+IFZSSU5HX0FWQUlMX0ZfTk9fSU5URVJSVVBUIGluIHRoZSBjYXNlIG9mIGV2ZW50IGluZGV4 LiBUaGUgY29ycmVjdCBmaXgKPiA+ID4gPiA+ID4gPiBtaWdodCBiZToKPiA+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPiA+IDEpIHJlbW92ZSBldmVudF90cmlnZ2VyZWQKPiA+ID4gPiA+ID4gPiAyKSBz ZXQgVlJJTkdfQVZBSUxfRl9OT19JTlRFUlJVUFQgaW4gYXZhaWxfZmxhZ3Nfc2hhZG93IGluCj4g PiA+ID4gPiA+ID4gdnJpbmdfaW50ZXJycnVwdCBpZiBldmVudCBpbmRleCBpcyBzdXBwb3J0ZWQK PiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID8KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ IFRoYW5rcwo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBJIGFtIG5vdCBzdXJlIGFsbCB0aGlzIGlz IHJpZ2h0IGFuZCBJJ2QgcmF0aGVyIHdlIGZvY3VzZWQKPiA+ID4gPiA+ID4gcGVyZm9ybWFuY2Uv Y29ycmVjdG5lc3MgYW5kIGNsZWFudXBzIHNlcGFyYXRlbHkuCj4gPiA+ID4gPiA+Cj4gPiA+ID4g PiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ IEJ1dCBiZWNhdXNlIG9mIHRoZSBpbnRyb2R1Y3Rpb24gb2YgZXZlbnRfdHJpZ2dlcmVkLCBoZXJl LAo+ID4gPiA+ID4gPiA+ID4gdmlydHF1ZXVlX2dldF9idWZfY3R4X3NwbGl0ICBjYW5ub3QgYmUg cmVjb2duaXplZAo+ID4gPiA+ID4gPiA+ID4gdGhhdCB0aGUgaW50ZXJydXB0IGhhcyBiZWVuIHR1 cm5lZCBvZmYuCj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gaWYgd2Ugd2FudCAgYW5v dGhlciBpbnRlcnJ1cHQgZm9yIHRoZSBuZXh0IGVudHJ5LCBXZSBzaG91bGQgcHJvYmFibHkKPiA+ ID4gPiA+ID4gPiA+IGNhbGwgdmlydHF1ZXVlX2VuYWJsZV9jYu+8nwo+ID4gPiA+ID4gPiA+ID4K PiA+ID4gPiA+ID4gPiA+IFRoYW5rcwo+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4K PiA+ID4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgICAgICAgdmlydGlvX3N0b3JlX21iKHZxLT53 ZWFrX2JhcnJpZXJzLAo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgJnZyaW5nX3VzZWRfZXZlbnQoJnZxLT5zcGxpdC52cmluZyksCj4gPiA+ID4gPiA+ ID4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjcHVfdG9fdmlydGlvMTYo X3ZxLT52ZGV2LCB2cS0+bGFzdF91c2VkX2lkeCkpOwo+ID4gPiA+ID4gPiA+ID4gPiA+IEBAIC0x NTI5LDcgKzE1MzAsOCBAQCBzdGF0aWMgdm9pZCAqdmlydHF1ZXVlX2dldF9idWZfY3R4X3BhY2tl ZChzdHJ1Y3QgdmlydHF1ZXVlICpfdnEsCj4gPiA+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgKiBi eSB3cml0aW5nIGV2ZW50IGluZGV4IGFuZCBmbHVzaCBvdXQgdGhlIHdyaXRlIGJlZm9yZQo+ID4g PiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICogdGhlIHJlYWQgaW4gdGhlIG5leHQgZ2V0X2J1ZiBj YWxsLgo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICovCj4gPiA+ID4gPiA+ID4gPiA+ID4g LSAgICAgICBpZiAodnEtPnBhY2tlZC5ldmVudF9mbGFnc19zaGFkb3cgPT0gVlJJTkdfUEFDS0VE X0VWRU5UX0ZMQUdfREVTQykKPiA+ID4gPiA+ID4gPiA+ID4gPiArICAgICAgIGlmICh2cS0+cGFj a2VkLmV2ZW50X2ZsYWdzX3NoYWRvdyA9PSBWUklOR19QQUNLRURfRVZFTlRfRkxBR19ERVNDCj4g PiA+ID4gPiA+ID4gPiA+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgJiYgKHZxLT5ldmVudF90 cmlnZ2VyZWQgPT0gZmFsc2UpKQo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgICB2 aXJ0aW9fc3RvcmVfbWIodnEtPndlYWtfYmFycmllcnMsCj4gPiA+ID4gPiA+ID4gPiA+ID4gICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmdnEtPnBhY2tlZC52cmluZy5kcml2ZXItPm9m Zl93cmFwLAo+ID4gPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgY3B1X3RvX2xlMTYodnEtPmxhc3RfdXNlZF9pZHgpKTsKPiA+ID4gPiA+ID4gPiA+ID4gPiAt LQo+ID4gPiA+ID4gPiA+ID4gPiA+IDIuMzEuMQo+ID4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4g PiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+Cj4gPgoKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVmlydHVhbGl6YXRpb24g bWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0 dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3ZpcnR1YWxp emF0aW9u From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 693E4C6FD1C for ; Fri, 24 Mar 2023 09:07:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231860AbjCXJG7 (ORCPT ); Fri, 24 Mar 2023 05:06:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230111AbjCXJG4 (ORCPT ); Fri, 24 Mar 2023 05:06:56 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AC14136FB for ; Fri, 24 Mar 2023 02:06:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679648765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7+GqAWmz15rP7oViR7OeAok2f70x1pUo5xW0EoL4Usg=; b=TOpLJMiFX+Lg0OC7HuCkF9Bz/OjfIYqDhkafqc9LVSBZmtIyuXtUKV/UEKdXjBCzVBOd81 C2hGcvwXvqjmy2Pt9Y94Ws8XTGfCEqPM8q32OrWa2gZYlTFwUWwIr80pa/Lc9HsQZpBL4J wa3s6kGIpQVJtAzyZkLbs11TLzhE5hw= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-636-46fVMd5wN1e6CNJ8x4S6Zw-1; Fri, 24 Mar 2023 05:06:03 -0400 X-MC-Unique: 46fVMd5wN1e6CNJ8x4S6Zw-1 Received: by mail-wm1-f70.google.com with SMTP id u6-20020a05600c19c600b003ed7cb98a8dso527630wmq.9 for ; Fri, 24 Mar 2023 02:06:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679648762; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7+GqAWmz15rP7oViR7OeAok2f70x1pUo5xW0EoL4Usg=; b=BC2TmUfvhe1lT3+YOnwKZemys+lfsI+Vk2vEEw3v12ekZM8aVeRVZpitOkBIKk94ch 43YRZPZ8x20MJ9SXb/qNaSM7yxz7sA8yUae/X9xpaJQpR1osMbtd3ewQV9W32jT2Ww16 bn8XYnopaJTE2KIe0qfaJE5Bfu9O62gCmfONVkBCZ6XTgYtAdl1Z/UpCB988Ufvd+Ygn nbHjSLnDOEsevl2W9hU/4ubecJbYdQdhRZcZZF7Wm4gpNpcEX8ogxmopNV9fT+3xPfPH ixXHZFOVFa0WyjVutETvdS3E8nvvvqEHqbhAfuycTjw9ACCM/y8BXbSa/SBrI7SWQYlX eyig== X-Gm-Message-State: AO0yUKWchijuLjLbmhUYJC73XRl/oSFf/tghXXoVF6Md+Q/ORKjdH0AE yx7uSUdTy1TAsmFMXMFDGCuoI4P6kqyhqm54Xi1YLp9iQmiceDhS5j0Y38kvZs7UnoRL5Jf+Kp/ I7Tm9COcEEuWFitB287nOKsbP X-Received: by 2002:a1c:4b13:0:b0:3ed:33a1:ba8e with SMTP id y19-20020a1c4b13000000b003ed33a1ba8emr1829894wma.1.1679648762355; Fri, 24 Mar 2023 02:06:02 -0700 (PDT) X-Google-Smtp-Source: AK7set+tRNl1cyqajon+Q3sDihZIdsLxHbeZ/aojwtRsQlg0XR4tPIAi4yaR5/T6R667BcDUr6RZPw== X-Received: by 2002:a1c:4b13:0:b0:3ed:33a1:ba8e with SMTP id y19-20020a1c4b13000000b003ed33a1ba8emr1829875wma.1.1679648761970; Fri, 24 Mar 2023 02:06:01 -0700 (PDT) Received: from redhat.com ([2.52.12.190]) by smtp.gmail.com with ESMTPSA id d5-20020adfef85000000b002cfed482e9asm17997062wro.61.2023.03.24.02.06.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Mar 2023 02:06:01 -0700 (PDT) Date: Fri, 24 Mar 2023 05:05:58 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: =?utf-8?B?6buE5p2w?= , "open list:VIRTIO CORE AND NET DRIVERS" , open list Subject: Re: [External] Re: [PATCH] virtio_ring: Suppress tx interrupt when napi_tx disable Message-ID: <20230324050424-mutt-send-email-mst@kernel.org> References: <20230321085953.24949-1-huangjie.albert@bytedance.com> <20230324013805-mutt-send-email-mst@kernel.org> <20230324024155-mutt-send-email-mst@kernel.org> <20230324025937-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Mar 24, 2023 at 03:37:04PM +0800, Jason Wang wrote: > On Fri, Mar 24, 2023 at 3:00 PM Michael S. Tsirkin wrote: > > > > On Fri, Mar 24, 2023 at 02:47:02PM +0800, Jason Wang wrote: > > > On Fri, Mar 24, 2023 at 2:42 PM Michael S. Tsirkin wrote: > > > > > > > > On Fri, Mar 24, 2023 at 02:32:40PM +0800, Jason Wang wrote: > > > > > On Fri, Mar 24, 2023 at 1:59 PM Michael S. Tsirkin wrote: > > > > > > > > > > > > On Fri, Mar 24, 2023 at 11:41:12AM +0800, Jason Wang wrote: > > > > > > > On Thu, Mar 23, 2023 at 4:01 PM 黄杰 wrote: > > > > > > > > > > > > > > > > Jason Wang 于2023年3月22日周三 10:37写道： > > > > > > > > > > > > > > > > > > On Tue, Mar 21, 2023 at 5:00 PM Albert Huang > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > From: "huangjie.albert" > > > > > > > > > > > > > > > > > > > > fix commit 8d622d21d248 ("virtio: fix up virtio_disable_cb") > > > > > > > > > > > > > > > > > > > > if we disable the napi_tx. when we triger a tx interrupt, the > > > > > > > > > > > > > > > > > > typo should be "trigger" > > > > > > > > > > > > > > > > > > > > > > > > > OK, thanks for this. I will correct it in the next version > > > > > > > > > > > > > > > > > > vq->event_triggered will be set to true. It will no longer be > > > > > > > > > > set to false. Unless we explicitly call virtqueue_enable_cb_delayed > > > > > > > > > > or virtqueue_enable_cb_prepare > > > > > > > > > > > > > > > > > > > > if we disable the napi_tx, It will only be called when the tx ring > > > > > > > > > > buffer is relatively small: > > > > > > > > > > virtio_net->start_xmit: > > > > > > > > > > if (sq->vq->num_free < 2+MAX_SKB_FRAGS) { > > > > > > > > > > netif_stop_subqueue(dev, qnum); > > > > > > > > > > if (!use_napi && > > > > > > > > > > unlikely(!virtqueue_enable_cb_delayed(sq->vq))) { > > > > > > > > > > /* More just got used, free them then recheck. */ > > > > > > > > > > free_old_xmit_skbs(sq, false); > > > > > > > > > > if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) { > > > > > > > > > > netif_start_subqueue(dev, qnum); > > > > > > > > > > virtqueue_disable_cb(sq->vq); > > > > > > > > > > } > > > > > > > > > > > > > > > > > > The code example here is out of date, make sure your tree has this: > > > > > > > > > > > > > > > > also, I will correct it in the next version，this is from kernel 5.15. > > > > > > > > > > > > > > > > > > > > > > > > > > commit d71ebe8114b4bf622804b810f5e274069060a174 > > > > > > > > > Author: Jason Wang > > > > > > > > > Date: Tue Jan 17 11:47:07 2023 +0800 > > > > > > > > > > > > > > > > > > virtio-net: correctly enable callback during start_xmit > > > > > > > > > > > > > > > > > > > } > > > > > > > > > > } > > > > > > > > > > Because event_triggered is true.Therefore, VRING_AVAIL_F_NO_INTERRUPT or > > > > > > > > > > VRING_PACKED_EVENT_FLAG_DISABLE will not be set.So we update > > > > > > > > > > vring_used_event(&vq->split.vring) or vq->packed.vring.driver->off_wrap > > > > > > > > > > every time we call virtqueue_get_buf_ctx.This will bring more interruptions. > > > > > > > > > > > > > > > > > > Can you please post how to test with the performance numbers? > > > > > > > > > > > > > > > > > > > > > > > > > iperf3 tcp stream: > > > > > > > > vm1 -----------------> vm2 > > > > > > > > vm2 just receive tcp data stream from vm1, and send the ack to vm1, > > > > > > > > there are so > > > > > > > > many tx interruptions in vm2. > > > > > > > > > > > > > > > > but without event_triggered there are just a few tx interruptions. > > > > > > > > > > > > > > > > > > > > > > > > > > > > if event_triggered is set to true, do not update vring_used_event(&vq->split.vring) > > > > > > > > > > or vq->packed.vring.driver->off_wrap > > > > > > > > > > > > > > > > > > > > Signed-off-by: huangjie.albert > > > > > > > > > > --- > > > > > > > > > > drivers/virtio/virtio_ring.c | 6 ++++-- > > > > > > > > > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > > > > > > > > > index 307e139cb11d..f486cccadbeb 100644 > > > > > > > > > > --- a/drivers/virtio/virtio_ring.c > > > > > > > > > > +++ b/drivers/virtio/virtio_ring.c > > > > > > > > > > @@ -795,7 +795,8 @@ static void *virtqueue_get_buf_ctx_split(struct virtqueue *_vq, > > > > > > > > > > /* If we expect an interrupt for the next entry, tell host > > > > > > > > > > * by writing event index and flush out the write before > > > > > > > > > > * the read in the next get_buf call. */ > > > > > > > > > > - if (!(vq->split.avail_flags_shadow & VRING_AVAIL_F_NO_INTERRUPT)) > > > > > > > > > > + if (!(vq->split.avail_flags_shadow & VRING_AVAIL_F_NO_INTERRUPT) > > > > > > > > > > + && (vq->event_triggered == false)) > > > > > > > > > > > > > > > > > > I'm not sure this can work, when event_triggered is true it means > > > > > > > > > we've got an interrupt, in this case if we want another interrupt for > > > > > > > > > the next entry, we should update used_event otherwise we will lose > > > > > > > > > that interrupt? > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > Normally, if we receive an interrupt, we should disable the interrupt > > > > > > > > in the interrupt callback handler. > > > > > > > > > > > > > > So the problem is: > > > > > > > > > > > > > > 1) event_triggered was set to true in vring_interrupt() > > > > > > > > > > > > > > 2) after this nothing will happen for virtqueue_disable_cb() so > > > > > > > VRING_AVAIL_F_NO_INTERRUPT is not set in avail_flags_shadow > > > > > > > 3) virtqueue_get_buf_ctx_split() will still think the cb is enabled > > > > > > > then it tries to publish new event > > > > > > > > > > > > Oh. Good point! I think when I wrote up > > > > > > 8d622d21d248 ("virtio: fix up virtio_disable_cb") > > > > > > I missed this corner case. > > > > > > > > > > > > > > > > > > > > > > > > > This makes me think about whether or not we really need > > > > > > > event_triggered. The assumption in the virtqueue_disable_cb() seems > > > > > > > wrong: > > > > > > > > > > > > > > /* If device triggered an event already it won't trigger one again: > > > > > > > * no need to disable. > > > > > > > */ > > > > > > > if (vq->event_triggered) > > > > > > > return; > > > > > > > > > > > > > > This is wrong if there's no event index support. > > > > > > > > > > > > > > > > > > I don't get it. how does this get triggered? > > > > > > > > > > > > You are talking about device without event index? > > > > > > Here's code from vring_interrupt(): > > > > > > > > > > > > /* Just a hint for performance: so it's ok that this can be racy! */ > > > > > > if (vq->event) > > > > > > vq->event_triggered = true; > > > > > > > > > > But we have the following in virtqueue_disable_cb(): > > > > > > > > > > /* If device triggered an event already it won't trigger one again: > > > > > * no need to disable. > > > > > */ > > > > > if (vq->event_triggered) > > > > > return; > > > > > > > > > > if (vq->packed_ring) > > > > > virtqueue_disable_cb_packed(_vq); > > > > > else > > > > > virtqueue_disable_cb_split(_vq); > > > > > > > > > > This means, without an event index, we don't set avail flags. So the > > > > > interrupt is not disabled actually in this case. > > > > > > > > > > Thanks > > > > > > > > Only if event_triggered is true, which without event index it never is. > > > > > > I'm not sure I will get here. I meant for example the commit > > > suppresses the effort of skb_xmit_done(): > > > > > > static void skb_xmit_done(struct virtqueue *vq) > > > { > > > struct virtnet_info *vi = vq->vdev->priv; > > > struct napi_struct *napi = &vi->sq[vq2txq(vq)].napi; > > > > > > /* Suppress further interrupts. */ > > > virtqueue_disable_cb(vq); > > > > > > The virtqueue_disable_cb() doesn't disable further interrupts when the > > > event index is not there. > > > > > > Thanks > > > > Check what can set event_triggered, you will see. > > Set to truth by vring_interrupt() vring_interrupt only sets it to true if vq->event is true > Set to false by virtqueue_init(), virtqueue_enable_cb_prepare(), > virtqueue_enable_cb_delayed() > > Assuming NAPI TX is enabled and the device doesn't support event index. > > 1) driver sends packets 1-10 > 2) the start_xmit() for the last packet will call > virtqueue_enable_cb_delayed() which set event_triggered = false > 3) 1st packet were sent, vring_interrupt set event_triggered = true > 4) skb_xmit_done() won't disable virtqueue_disable_cb() in this case > 5) so we will get the interrupts for 2nd to 10th packet > > Anything I missed here? 3 does not happen if event index is off. > Note the comment said it's used for event index: > > /* Hint for event idx: already triggered no need to disable. */ > bool event_triggered; > > I guess what you meant is that if we don't publish a new event, we > will get at most 1 interrupt for everything $queue_size used buffers. > But this is not the case without event index. Btw, it may supress the > effort of: > > vring_used_event(&vq->split.vring) = 0x0; > > Thanks Because it's not necessary then. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > And the > > > > > > > event_triggered is somehow duplicated with the > > > > > > > VRING_AVAIL_F_NO_INTERRUPT in the case of event index. The correct fix > > > > > > > might be: > > > > > > > > > > > > > > 1) remove event_triggered > > > > > > > 2) set VRING_AVAIL_F_NO_INTERRUPT in avail_flags_shadow in > > > > > > > vring_interrrupt if event index is supported > > > > > > > > > > > > > > ? > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > I am not sure all this is right and I'd rather we focused > > > > > > performance/correctness and cleanups separately. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But because of the introduction of event_triggered, here, > > > > > > > > virtqueue_get_buf_ctx_split cannot be recognized > > > > > > > > that the interrupt has been turned off. > > > > > > > > > > > > > > > > if we want another interrupt for the next entry, We should probably > > > > > > > > call virtqueue_enable_cb？ > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > virtio_store_mb(vq->weak_barriers, > > > > > > > > > > &vring_used_event(&vq->split.vring), > > > > > > > > > > cpu_to_virtio16(_vq->vdev, vq->last_used_idx)); > > > > > > > > > > @@ -1529,7 +1530,8 @@ static void *virtqueue_get_buf_ctx_packed(struct virtqueue *_vq, > > > > > > > > > > * by writing event index and flush out the write before > > > > > > > > > > * the read in the next get_buf call. > > > > > > > > > > */ > > > > > > > > > > - if (vq->packed.event_flags_shadow == VRING_PACKED_EVENT_FLAG_DESC) > > > > > > > > > > + if (vq->packed.event_flags_shadow == VRING_PACKED_EVENT_FLAG_DESC > > > > > > > > > > + && (vq->event_triggered == false)) > > > > > > > > > > virtio_store_mb(vq->weak_barriers, > > > > > > > > > > &vq->packed.vring.driver->off_wrap, > > > > > > > > > > cpu_to_le16(vq->last_used_idx)); > > > > > > > > > > -- > > > > > > > > > > 2.31.1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >