From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36F5BC04FF3 for ; Mon, 24 May 2021 09:10:17 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4889610C7 for ; Mon, 24 May 2021 09:10:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4889610C7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 7AFE8400FB; Mon, 24 May 2021 09:10:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fBAu27C-kRzL; Mon, 24 May 2021 09:10:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTP id D86D3400A7; Mon, 24 May 2021 09:10:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A1AB1C000E; Mon, 24 May 2021 09:10:14 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 566E0C0001 for ; Mon, 24 May 2021 09:10:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 303C082EB6 for ; Mon, 24 May 2021 09:10:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lf5sIN34lzAq for ; Mon, 24 May 2021 09:10:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 31BB482E54 for ; Mon, 24 May 2021 09:10:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621847408; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L3wSqyBZrZuWAaUg/+zzbLh1KHxQeq+UhWJvXbU1jvM=; b=AlKYOTit7QdKTH4xp0I2aUzjmLumLBa9NekYBeAk0C1QQOHlle9RWRIM9lYBM5RzKtl48I 24E6TH0UAfx/w1JSR0Uu1jKejbFTkjDODNIBw4gCUypFEdY6zpZu4gZOsKhe5bwSh68/pj RAF0nL1qEsT+3rbv2hf0aHUQ8AuCe7U= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-373-bv57qokMMu29F3oTz4sb2g-1; Mon, 24 May 2021 05:10:04 -0400 X-MC-Unique: bv57qokMMu29F3oTz4sb2g-1 Received: by mail-wm1-f70.google.com with SMTP id g9-20020a1c39090000b029017545f2da89so3760718wma.8 for ; Mon, 24 May 2021 02:10:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=L3wSqyBZrZuWAaUg/+zzbLh1KHxQeq+UhWJvXbU1jvM=; b=Vi8S51xCBTvYk5uIBSlznHjARopImddh0NJmJhSDGG5ArivMRGStt16akhl2mJKSga eBBuggUAEGJuAefQpWvxHkDDPt9y6h9HZ9lGA2wR/p7j4YDXHhXVdztqPcY++km4c2Gw BdqVxS/fbv1Kd2ZWVtL3vd3azmFG+YZkDHPvE0tsLqHUQYlbrXsedtpAH4ZCFBLwFZFv 9xbDI93QWWP/DO6wb7BSZoepT0oNWSqMz+hULJ32qHi1wCt8ShsdTKk+3qu/l+qXPfSZ JyprFypGbfCl+g+l6VU5wqOn91odZ3mMQOs/+MM1Qu8wi79YS0zZlVTFQTVN34jULG9b sHfw== X-Gm-Message-State: AOAM53391Y5ABgLVXkrlSyB+ZKkxbqyjDGtE/SV/F/ylpl9QQT8ENTxu H+DssrkS1ijBa2gRBgOrKVJPgTkL8OVy1WghqZODnXQJ+YdnlEo4p60OeJa47PjsbUAh7cDtFuo ZhXZ0kKGcENxmsUitXHIABpOdQJD0yV9KeAs2yj/ALQ== X-Received: by 2002:adf:f04f:: with SMTP id t15mr20315411wro.377.1621847403769; Mon, 24 May 2021 02:10:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJytbecMYMOBl3tQdN/Kb27iSiGCSiu57Cvca61WWGiciWSVF/D85dYw2RXxPRqM5ZPPEiFVYQ== X-Received: by 2002:adf:f04f:: with SMTP id t15mr20315393wro.377.1621847403598; Mon, 24 May 2021 02:10:03 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id w25sm7397924wmk.25.2021.05.24.02.10.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 May 2021 02:10:02 -0700 (PDT) Date: Mon, 24 May 2021 05:10:00 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: virtio_net: BQL? Message-ID: <20210524050840-mutt-send-email-mst@kernel.org> References: <56270996-33a6-d71b-d935-452dad121df7@linux.alibaba.com> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: Willem de Bruijn , Xianting Tian , Linux Kernel Network Developers , Dave Taht , virtualization , bloat , Jakub Kicinski , "David S. Miller" , LKML X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gTW9uLCBNYXkgMjQsIDIwMjEgYXQgMTA6NTM6MDhBTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiDlnKggMjAyMS81LzE4IOS4iuWNiDU6NDgsIERhdmUgVGFodCDlhpnpgZM6Cj4gPiBP biBNb24sIE1heSAxNywgMjAyMSBhdCAxOjIzIFBNIFdpbGxlbSBkZSBCcnVpam4KPiA+IDx3aWxs ZW1kZWJydWlqbi5rZXJuZWxAZ21haWwuY29tPiB3cm90ZToKPiA+ID4gT24gTW9uLCBNYXkgMTcs IDIwMjEgYXQgMjo0NCBQTSBEYXZlIFRhaHQgPGRhdmUudGFodEBnbWFpbC5jb20+IHdyb3RlOgo+ ID4gPiA+IE5vdCByZWFsbHkgcmVsYXRlZCB0byB0aGlzIHBhdGNoLCBidXQgaXMgdGhlcmUgc29t ZSByZWFzb24gd2h5IHZpcnRpbwo+ID4gPiA+IGhhcyBubyBzdXBwb3J0IGZvciBCUUw/Cj4gPiA+ IFRoZXJlIGhhdmUgYmVlbiBhIGZldyBhdHRlbXB0cyB0byBhZGQgaXQgb3ZlciB0aGUgeWVhcnMu Cj4gPiA+IAo+ID4gPiBNb3N0IHJlY2VudGx5LCBodHRwczovL2xvcmUua2VybmVsLm9yZy9sa21s LzIwMTgxMjA1MjI1MzIzLjEyNTU1LTItbXN0QHJlZGhhdC5jb20vCj4gPiA+IAo+ID4gPiBUaGF0 IHRocmVhZCBoYXMgYSBsb25nIGRpc2N1c3Npb24uIEkgdGhpbmsgdGhlIGtleSBvcGVuIGlzc3Vl IHJlbWFpbnMKPiA+ID4gCj4gPiA+ICJUaGUgdHJpY2t5IHBhcnQgaXMgdGhlIG1vZGUgc3dpdGNo aW5nIGJldHdlZW4gbmFwaSBhbmQgbm8gbmFwaS4iCj4gPiBPeSwgdmV5Lgo+ID4gCj4gPiBJIGRp ZG4ndCBwYXkgYW55IGF0dGVudGlvbiB0byB0aGF0IGRpc2N1c3Npb24sIHNhZGx5IGVub3VnaC4K PiA+IAo+ID4gSXQncyBiZWVuIGFib3V0IHRoYXQgbG9uZyAoMjAxOCkgc2luY2UgSSBwYWlkIGFu eSBhdHRlbnRpb24gdG8KPiA+IGJ1ZmZlcmJsb2F0IGluIHRoZSBjbG91ZCBhbmQgbXkgY2xvdWR5 IHByb3ZpZGVyIChsaW5vZGUpIHN3aXRjaGVkIHRvCj4gPiB1c2luZyB2aXJ0aW8gd2hlbiBJIHdh c24ndCBsb29raW5nLiBGb3Igb3ZlciBhIHllYXIgbm93LCBJJ2QgYmVlbgo+ID4gZ2V0dGluZyBy ZXBvcnRzIHNheWluZyB0aGF0IGNvbWNhc3QncyBwaWUgcm9sbG91dCB3YXNuJ3Qgd29ya2luZyBh cwo+ID4gd2VsbCBhcyBleHBlY3RlZCwgdGhhdCBldmVucm91dGUncyBpbXBsZW1lbnRhdGlvbiBv ZiBzY2hfY2FrZSBhbmQgc3FtCj4gPiBvbiBpbmJvdW5kIHdhc24ndCB3b3JraW5nIHJpZ2h0LCBu b3IgcGZfc2Vuc2UncyBhbmQgbnVtZXJvdXMgb3RoZXIKPiA+IGlzc3VlcyBhdCBJbnRlcm5ldCBz Y2FsZS4KPiA+IAo+ID4gTGFzdCB3ZWVrIEkgcmFuIGEgc3RyaW5nIG9mIGJlbmNobWFya3MgYWdh aW5zdCBzdGFybGluaydzIG5ldyBzZXJ2aWNlcwo+ID4gYW5kIHdhcyByZWFsbHkgYWdoYXN0IGF0 IHdoYXQgSSBmb3VuZCB0aGVyZSwgdG9vLiBidXQgdGhlIHByb2JsZW0KPiA+IHNlZW1lZCBkZWVw ZXIgdGhhbiBpbiBqdXN0IHRoZSBkaXNoeS4uLgo+ID4gCj4gPiBXaXRob3V0IEJRTCwgdGhlcmUn cyBubyBiYWNrcHJlc3N1cmUgZm9yIGZxX2NvZGVsIHRvIGRvIGl0cyB0aGluZy4KPiA+IE5vbmUu IE15IG1lYXN1cmVtZW50IHNlcnZlcnMgYXJlbid0IEZRLWNvZGVsaW5nCj4gPiBubyBtYXR0ZXIg aG93IG11Y2ggbG9hZCBJIHB1dCBvbiB0aGVtLiBTaW5jZSB0aGF0IHFkaXNjIGlzIHRoZSBkZWZh dWx0Cj4gPiBub3cgaW4gbW9zdCBsaW51eCBkaXN0cmlidXRpb25zLCBJIGltYWdpbmUgdGhhdCB0 aGUgYnVsayBvZiB0aGUgY2xvdWQKPiA+IGlzIG5vdyBiZWhhdmluZyBhcyBlcnJhdGljYWxseSBh cyBsaW51eCB3YXMgaW4gMjAxMSB3aXRoIGVub3Jtb3VzCj4gPiBzd2luZ3MgaW4gdGhyb3VnaHB1 dCBhbmQgbGF0ZW5jeSBmcm9tIEdTTy9UU08gaGl0dGluZyBvdmVybGFyZ2UgcngvdHgKPiA+IHJp bmdzLCBbMV0sIGJyZWFraW5nIHZhcmlvdXMgcmF0ZSBlc3RpbWF0b3JzIGluIGNvZGVsLCBwaWUg YW5kIHRoZSB0Y3AKPiA+IHN0YWNrIGl0c2VsZi4KPiA+IAo+ID4gU2VlOgo+ID4gCj4gPiBodHRw Oi8vZnJlbW9udC5zdGFybGluay50YWh0Lm5ldC9+ZC92aXJ0aW9fbm9icWwvcnJ1bF8tX2V2ZW5y b3V0ZV92M19zZXJ2ZXJfZnFfY29kZWwucG5nCj4gPiAKPiA+IFNlZSB0aGUgc3dpbmdzIGluIGxh dGVuY3kgdGhlcmU/IHRoYXQncyBzeW1wdG9tYXRpYyBvZiB0eC9yeCByaW5ncwo+ID4gZmlsbGlu ZyBhbmQgZW1wdHlpbmcuCj4gPiAKPiA+IGl0IHdhc24ndCB1bnRpbCBJIHN3aXRjaGVkIG15IG1l YXN1cmVtZW50IHNlcnZlciB0ZW1wb3JhcmlseSBvdmVyIHRvCj4gPiBzY2hfZnEgdGhhdCBJIGdv dCBhIHJydWwgcmVzdWx0IHRoYXQgd2FzIGNsb3NlIHRvIHRoZSByZXN1bHRzIHdlIHVzZWQKPiA+ IHRvIGdldCBmcm9tIHRoZSB2aXJ0dWFsaXplZCBlMTAwMGUgZHJpdmVycyB3ZSB3ZXJlIHVzaW5n IGluIDIwMTQuCj4gPiAKPiA+IGh0dHA6Ly9mcmVtb250LnN0YXJsaW5rLnRhaHQubmV0L35kL3Zp cnRpb19ub2JxbC9ycnVsXy1fZXZlbnJvdXRlX3YzX3NlcnZlcl9mcS5wbmcKPiA+IAo+ID4gV2hp bGUgSSBoYXZlIGxvbmcgc3VwcG9ydGVkIHRoZSB1c2Ugb2Ygc2NoX2ZxIGZvciB0Y3AtaGVhdnkg d29ya2xvYWRzLAo+ID4gaXQgc3RpbGwgYmVoYXZlcyBiZXR0ZXIgd2l0aCBicWwgaW4gcGxhY2Us IGFuZCBmcV9jb2RlbCBpcyBiZXR0ZXIgZm9yCj4gPiBnZW5lcmljIHdvcmtsb2Fkcy4uLiBidXQg bmVlZHMgYnFsIGJhc2VkIGJhY2twcmVzc3VyZSB0byBraWNrIGluLgo+ID4gCj4gPiBbMV0gSSBy ZWFsbHkgaG9wZSBJJ20gb3ZlcnJlYWN0aW5nIGJ1dCwgdW0sIGVyLCBjb3VsZCBzb21lb25lKHMp IHNwaW4KPiA+IHVwIGEgbmV3IHBhdGNoIHRoYXQgZG9lcyBicWwgaW4gc29tZSB3YXkgZXZlbiBo YWxmIHJpZ2h0IGZvciB0aGlzCj4gPiBkcml2ZXIgYW5kIGhlbHAgdGVzdCBpdD8gSSBoYXZlbid0 IGJ1aWx0IGEga2VybmVsIGluIGEgd2hpbGUuCj4gCj4gCj4gSSB0aGluayBpdCdzIHRpbWUgdG8g b2Jzb2xldGUgc2tiX29ycGhhbigpIGZvciB2aXJ0aW8tbmV0IHRvIGdldCByaWQgb2YgYQo+IGJy dW5jaCBvZiB0cmlja3kgY29kZXMgaW4gdGhlIGN1cnJlbnQgdmlydGlvLW5ldCBkcml2ZXIuCj4g Cj4gVGhlbiB3ZSBjYW4gZG8gQlFMIG9uIHRvcC4KPiAKPiBJIHdpbGwgcHJlcGFyZSBzb21lIHBh dGNoZXMgdG8gZG8gdGhpcyAocHJvYmFibHkgd2l0aCBNaWNoYWVsJ3MgQlFMIHBhdGNoKS4KPiAK PiBUaGFua3MKCkZpcnN0IHN0ZXAgd291bGQgYmUgdG8gZml4IHVwIGFuZCB0ZXN0IHRoZSBCUUwg cGFydC4KSUlSQyBpdCBkaWRuJ3Qgc2VlbSB0byBoZWxwIHBlcmZvcm1hbmNlIGluIG91ciBiZW5j aG1hcmtpbmcsCmFuZCBFcmljIHNlZW1zIHRvIHNheSB0aGF0J3MgZXhwZWN0ZWQgLi4uCgoKPiAK PiA+IAo+ID4gCj4gPiA+ID4gT24gTW9uLCBNYXkgMTcsIDIwMjEgYXQgMTE6NDEgQU0gWGlhbnRp bmcgVGlhbgo+ID4gPiA+IDx4aWFudGluZy50aWFuQGxpbnV4LmFsaWJhYmEuY29tPiB3cm90ZToK PiA+ID4gPiA+IEJVR19PTigpIHVzZXMgdW5saWtlbHkgaW4gaWYoKSwgd2hpY2ggY2FuIGJlIG9w dGltaXplZCBhdCBjb21waWxlIHRpbWUuCj4gPiA+ID4gPiAKPiA+ID4gPiA+IFNpZ25lZC1vZmYt Ynk6IFhpYW50aW5nIFRpYW4gPHhpYW50aW5nLnRpYW5AbGludXguYWxpYmFiYS5jb20+Cj4gPiA+ ID4gPiAtLS0KPiA+ID4gPiA+ICAgIGRyaXZlcnMvbmV0L3ZpcnRpb19uZXQuYyB8IDUgKystLS0K PiA+ID4gPiA+ICAgIDEgZmlsZSBjaGFuZ2VkLCAyIGluc2VydGlvbnMoKyksIDMgZGVsZXRpb25z KC0pCj4gPiA+ID4gPiAKPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9kcml2ZXJzL25ldC92aXJ0aW9f bmV0LmMgYi9kcml2ZXJzL25ldC92aXJ0aW9fbmV0LmMKPiA+ID4gPiA+IGluZGV4IGM5MjFlYmYz YWU4Mi4uMjEyZDUyMjA0ODg0IDEwMDY0NAo+ID4gPiA+ID4gLS0tIGEvZHJpdmVycy9uZXQvdmly dGlvX25ldC5jCj4gPiA+ID4gPiArKysgYi9kcml2ZXJzL25ldC92aXJ0aW9fbmV0LmMKPiA+ID4g PiA+IEBAIC0xNjQ2LDEwICsxNjQ2LDkgQEAgc3RhdGljIGludCB4bWl0X3NrYihzdHJ1Y3Qgc2Vu ZF9xdWV1ZSAqc3EsIHN0cnVjdAo+ID4gPiA+ID4gc2tfYnVmZiAqc2tiKQo+ID4gPiA+ID4gICAg ICAgICAgZWxzZQo+ID4gPiA+ID4gICAgICAgICAgICAgICAgICBoZHIgPSBza2Jfdm5ldF9oZHIo c2tiKTsKPiA+ID4gPiA+IAo+ID4gPiA+ID4gLSAgICAgICBpZiAodmlydGlvX25ldF9oZHJfZnJv bV9za2Ioc2tiLCAmaGRyLT5oZHIsCj4gPiA+ID4gPiArICAgICAgIEJVR19PTih2aXJ0aW9fbmV0 X2hkcl9mcm9tX3NrYihza2IsICZoZHItPmhkciwKPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICB2aXJ0aW9faXNfbGl0dGxlX2VuZGlhbih2aS0+dmRldiksIGZh bHNlLAo+ID4gPiA+ID4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMCkpCj4g PiA+ID4gPiAtICAgICAgICAgICAgICAgQlVHKCk7Cj4gPiA+ID4gPiArICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAwKSk7Cj4gPiA+ID4gPiAKPiA+ID4gPiA+ICAgICAgICAgIGlm ICh2aS0+bWVyZ2VhYmxlX3J4X2J1ZnMpCj4gPiA+ID4gPiAgICAgICAgICAgICAgICAgIGhkci0+ bnVtX2J1ZmZlcnMgPSAwOwo+ID4gPiA+ID4gLS0KPiA+ID4gPiA+IDIuMTcuMQo+ID4gPiA+ID4g Cj4gPiA+ID4gCj4gPiA+ID4gLS0KPiA+ID4gPiBMYXRlc3QgUG9kY2FzdDoKPiA+ID4gPiBodHRw czovL3d3dy5saW5rZWRpbi5jb20vZmVlZC91cGRhdGUvdXJuOmxpOmFjdGl2aXR5OjY3OTEwMTQy ODQ5MzY3ODU5MjAvCj4gPiA+ID4gCj4gPiA+ID4gRGF2ZSBUw6RodCBDVE8sIFRla0xpYnJlLCBM TEMKPiA+IAo+ID4gCj4gPiAtLQo+ID4gTGF0ZXN0IFBvZGNhc3Q6Cj4gPiBodHRwczovL3d3dy5s aW5rZWRpbi5jb20vZmVlZC91cGRhdGUvdXJuOmxpOmFjdGl2aXR5OjY3OTEwMTQyODQ5MzY3ODU5 MjAvCj4gPiAKPiA+IERhdmUgVMOkaHQgQ1RPLCBUZWtMaWJyZSwgTExDCj4gPiAKCl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1h aWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRw czovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXph dGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D4D1C04FF3 for ; Mon, 24 May 2021 09:10:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 77F23610C7 for ; Mon, 24 May 2021 09:10:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232459AbhEXJLj (ORCPT ); Mon, 24 May 2021 05:11:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21622 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232318AbhEXJLh (ORCPT ); Mon, 24 May 2021 05:11:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621847409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L3wSqyBZrZuWAaUg/+zzbLh1KHxQeq+UhWJvXbU1jvM=; b=DF4cohUZdrdaoZODfPdElc4Rs7iPcxHC6MyFNzEeEp0hi+6Hho3OaSRzrv35FJeJgefNz9 r33Dc5KO8vSQVKoZ8Ico8xzTIOYBKcOqS5VdMXngqjuATf5kNKyQwzCUSXE10NlR7m81/m u0Ba+8ktXow1vJlBkr4FydtAb9sMQnM= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-269-_fCclypfP3aSLIZFpz2Psw-1; Mon, 24 May 2021 05:10:05 -0400 X-MC-Unique: _fCclypfP3aSLIZFpz2Psw-1 Received: by mail-wr1-f72.google.com with SMTP id i102-20020adf90ef0000b029010dfcfc46c0so12826095wri.1 for ; Mon, 24 May 2021 02:10:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=L3wSqyBZrZuWAaUg/+zzbLh1KHxQeq+UhWJvXbU1jvM=; b=fx5yZl6PsfI1bfDq/H+ISn4nodl4Av38Ofl4fHwJ0VQ5VJA2vGH6fHTJ67YYU0PXfK hkWpRpAkYIcp1AHLsNH4F3tqKD9C5oGXuUWq1nmhLeOHl5KLnYCRtjhKI0TmwPkx/rL9 56teyviU2TyhO8KJLRNqHuKn8l6hmMMVYWCwUpajk4hox5SqhW1dId/I2ciSDaEzXOA0 F+dmsPrhsKFQD+mmkIBDN8Unwxrk8G+gZSd+i+PSb2V+Xr408iqIijRUy8+n7SzCm0GN BmiHFsjk0M/AzC2pJ2o3mBCawvWn97fPO3oHbsKjs+8zhi95d5VU7LJncyOkjqie4HEh Yfww== X-Gm-Message-State: AOAM531L1N+P+TAGwbSG6w4jRjlkrcm8z9PuvZchjOmtSFoE/TYltmBP Wl8HpKs1plnWybFW4qXEJjZZGKhWZIcRyyQX/ZJJIBqHkoqZP6T5V1aJRNGuN02FWcSFWDwpyBN pANYIYO5TJQtd+8/G+4B5e/hj X-Received: by 2002:adf:f04f:: with SMTP id t15mr20315417wro.377.1621847403795; Mon, 24 May 2021 02:10:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJytbecMYMOBl3tQdN/Kb27iSiGCSiu57Cvca61WWGiciWSVF/D85dYw2RXxPRqM5ZPPEiFVYQ== X-Received: by 2002:adf:f04f:: with SMTP id t15mr20315393wro.377.1621847403598; Mon, 24 May 2021 02:10:03 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id w25sm7397924wmk.25.2021.05.24.02.10.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 May 2021 02:10:02 -0700 (PDT) Date: Mon, 24 May 2021 05:10:00 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: Dave Taht , Willem de Bruijn , Xianting Tian , "David S. Miller" , Jakub Kicinski , virtualization , Linux Kernel Network Developers , LKML , bloat Subject: Re: virtio_net: BQL? Message-ID: <20210524050840-mutt-send-email-mst@kernel.org> References: <56270996-33a6-d71b-d935-452dad121df7@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 24, 2021 at 10:53:08AM +0800, Jason Wang wrote: > > 在 2021/5/18 上午5:48, Dave Taht 写道: > > On Mon, May 17, 2021 at 1:23 PM Willem de Bruijn > > wrote: > > > On Mon, May 17, 2021 at 2:44 PM Dave Taht wrote: > > > > Not really related to this patch, but is there some reason why virtio > > > > has no support for BQL? > > > There have been a few attempts to add it over the years. > > > > > > Most recently, https://lore.kernel.org/lkml/20181205225323.12555-2-mst@redhat.com/ > > > > > > That thread has a long discussion. I think the key open issue remains > > > > > > "The tricky part is the mode switching between napi and no napi." > > Oy, vey. > > > > I didn't pay any attention to that discussion, sadly enough. > > > > It's been about that long (2018) since I paid any attention to > > bufferbloat in the cloud and my cloudy provider (linode) switched to > > using virtio when I wasn't looking. For over a year now, I'd been > > getting reports saying that comcast's pie rollout wasn't working as > > well as expected, that evenroute's implementation of sch_cake and sqm > > on inbound wasn't working right, nor pf_sense's and numerous other > > issues at Internet scale. > > > > Last week I ran a string of benchmarks against starlink's new services > > and was really aghast at what I found there, too. but the problem > > seemed deeper than in just the dishy... > > > > Without BQL, there's no backpressure for fq_codel to do its thing. > > None. My measurement servers aren't FQ-codeling > > no matter how much load I put on them. Since that qdisc is the default > > now in most linux distributions, I imagine that the bulk of the cloud > > is now behaving as erratically as linux was in 2011 with enormous > > swings in throughput and latency from GSO/TSO hitting overlarge rx/tx > > rings, [1], breaking various rate estimators in codel, pie and the tcp > > stack itself. > > > > See: > > > > http://fremont.starlink.taht.net/~d/virtio_nobql/rrul_-_evenroute_v3_server_fq_codel.png > > > > See the swings in latency there? that's symptomatic of tx/rx rings > > filling and emptying. > > > > it wasn't until I switched my measurement server temporarily over to > > sch_fq that I got a rrul result that was close to the results we used > > to get from the virtualized e1000e drivers we were using in 2014. > > > > http://fremont.starlink.taht.net/~d/virtio_nobql/rrul_-_evenroute_v3_server_fq.png > > > > While I have long supported the use of sch_fq for tcp-heavy workloads, > > it still behaves better with bql in place, and fq_codel is better for > > generic workloads... but needs bql based backpressure to kick in. > > > > [1] I really hope I'm overreacting but, um, er, could someone(s) spin > > up a new patch that does bql in some way even half right for this > > driver and help test it? I haven't built a kernel in a while. > > > I think it's time to obsolete skb_orphan() for virtio-net to get rid of a > brunch of tricky codes in the current virtio-net driver. > > Then we can do BQL on top. > > I will prepare some patches to do this (probably with Michael's BQL patch). > > Thanks First step would be to fix up and test the BQL part. IIRC it didn't seem to help performance in our benchmarking, and Eric seems to say that's expected ... > > > > > > > > > On Mon, May 17, 2021 at 11:41 AM Xianting Tian > > > > wrote: > > > > > BUG_ON() uses unlikely in if(), which can be optimized at compile time. > > > > > > > > > > Signed-off-by: Xianting Tian > > > > > --- > > > > > drivers/net/virtio_net.c | 5 ++--- > > > > > 1 file changed, 2 insertions(+), 3 deletions(-) > > > > > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > > > > index c921ebf3ae82..212d52204884 100644 > > > > > --- a/drivers/net/virtio_net.c > > > > > +++ b/drivers/net/virtio_net.c > > > > > @@ -1646,10 +1646,9 @@ static int xmit_skb(struct send_queue *sq, struct > > > > > sk_buff *skb) > > > > > else > > > > > hdr = skb_vnet_hdr(skb); > > > > > > > > > > - if (virtio_net_hdr_from_skb(skb, &hdr->hdr, > > > > > + BUG_ON(virtio_net_hdr_from_skb(skb, &hdr->hdr, > > > > > virtio_is_little_endian(vi->vdev), false, > > > > > - 0)) > > > > > - BUG(); > > > > > + 0)); > > > > > > > > > > if (vi->mergeable_rx_bufs) > > > > > hdr->num_buffers = 0; > > > > > -- > > > > > 2.17.1 > > > > > > > > > > > > > -- > > > > Latest Podcast: > > > > https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ > > > > > > > > Dave Täht CTO, TekLibre, LLC > > > > > > -- > > Latest Podcast: > > https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ > > > > Dave Täht CTO, TekLibre, LLC > >