From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80A3FC47084 for ; Tue, 25 May 2021 06:48:48 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C95B611C9 for ; Tue, 25 May 2021 06:48:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C95B611C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 70BB0404E0; Tue, 25 May 2021 06:48:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UNaezbEmMkKn; Tue, 25 May 2021 06:48:46 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTP id ABAAB4021C; Tue, 25 May 2021 06:48:45 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8CD4CC000D; Tue, 25 May 2021 06:48:45 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 86E06C0024 for ; Tue, 25 May 2021 06:48:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7610F83B72 for ; Tue, 25 May 2021 06:48:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iGH-9Iotixh2 for ; Tue, 25 May 2021 06:48:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3659983300 for ; Tue, 25 May 2021 06:48:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621925322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=U+MXAtJ+92tLWvy1A8WClR1rTAynn2EprzDjAnUev7Pn7nRzpZpgbQvJeBNH6wSf2T7Urr hoUj/ihCg6QAkkNPhNv11/Gfi7rWhot4ndvyeO7BeNa4gYKNDsLcfvcDooPOVBBs2bTzxs hQ3nIjYS1hajEqoS3YTYhwMgtOE6vC8= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-339-lrsHBuQmMwCrj3LX4xs4Dw-1; Tue, 25 May 2021 02:48:38 -0400 X-MC-Unique: lrsHBuQmMwCrj3LX4xs4Dw-1 Received: by mail-wr1-f70.google.com with SMTP id p11-20020adfc38b0000b0290111f48b8adfso10631996wrf.7 for ; Mon, 24 May 2021 23:48:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=hgalmB0gUUmj/doUwlKNOIl1E6Zefu67EDQIKlevWonWheb9CAfD31pbiY3Lw4CmAn Y0WZRy0uuQpp9500PjmGHT/C8i6CY2st3vp1qJsCi/uR4i7SktycMcwcOXCx0CNRoNW6 2cw+5n06OzDUriBx5OPYpCOn3Dz9CPTXMnUASlqn+TTheC3kyg9FcTZLzEBdgWlHFcc6 TwCC2qGWjO1k+R1CuWsBTi0j497pUiWeOq6zwWfcyA0BIt5xFMbkxWbYg1S+HH9KNoft mj7T2Nc4t6jgmFbp8dYufaufzUGk2v1nuZEVQGsieJXx3GHdknFTq4a+MXsCEn1O63wW aG6w== X-Gm-Message-State: AOAM530cp8Y05p6SwchrczDOq+npZLXabSpjUgQWLkVU2Bbk7q6rjRX3 8iiq6BEQqjuHGoXTGMj9VXwinw1aVdFgT6zrqLn6QyG5CDVvyHG6F+cLNJxUoKhI8ITjNCxRZY8 DdAWtfBHZfbKe/d+LtZ9xsB46s4iAZQ== X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535511wrf.420.1621925317003; Mon, 24 May 2021 23:48:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwHrJfvA/haiL2kDkU2Sc824s/VhxDUz9WKfrjZUVALV+rMusOmrmXdB7xQ8MMFAx4z9xMC5w== X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535496wrf.420.1621925316796; Mon, 24 May 2021 23:48:36 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id g78sm1839253wme.27.2021.05.24.23.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 May 2021 23:48:34 -0700 (PDT) Date: Tue, 25 May 2021 02:48:30 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [PATCH v7 00/12] Introduce VDUSE - vDPA Device in Userspace Message-ID: <20210525024500-mutt-send-email-mst@kernel.org> References: <20210517095513.850-1-xieyongji@bytedance.com> <20210520014349-mutt-send-email-mst@kernel.org> <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> MIME-Version: 1.0 In-Reply-To: <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: Jens Axboe , linux-kernel , kvm , Jonathan Corbet , netdev@vger.kernel.org, Randy Dunlap , iommu@lists.linux-foundation.org, Matthew Wilcox , virtualization , Christoph Hellwig , Yongji Xie , Christian Brauner , bcrl@kvack.org, Al Viro , Stefan Hajnoczi , linux-fsdevel@vger.kernel.org, Dan Carpenter , Mika =?iso-8859-1?Q?Penttil=E4?= , Stefano Garzarella X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gVHVlLCBNYXkgMjUsIDIwMjEgYXQgMDI6NDA6NTdQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiDlnKggMjAyMS81LzIwIOS4i+WNiDU6MDYsIFlvbmdqaSBYaWUg5YaZ6YGTOgo+ID4g T24gVGh1LCBNYXkgMjAsIDIwMjEgYXQgMjowNiBQTSBNaWNoYWVsIFMuIFRzaXJraW4gPG1zdEBy ZWRoYXQuY29tPiB3cm90ZToKPiA+ID4gT24gTW9uLCBNYXkgMTcsIDIwMjEgYXQgMDU6NTU6MDFQ TSArMDgwMCwgWGllIFlvbmdqaSB3cm90ZToKPiA+ID4gPiBUaGlzIHNlcmllcyBpbnRyb2R1Y2Vz IGEgZnJhbWV3b3JrLCB3aGljaCBjYW4gYmUgdXNlZCB0byBpbXBsZW1lbnQKPiA+ID4gPiB2RFBB IERldmljZXMgaW4gYSB1c2Vyc3BhY2UgcHJvZ3JhbS4gVGhlIHdvcmsgY29uc2lzdCBvZiB0d28g cGFydHM6Cj4gPiA+ID4gY29udHJvbCBwYXRoIGZvcndhcmRpbmcgYW5kIGRhdGEgcGF0aCBvZmZs b2FkaW5nLgo+ID4gPiA+IAo+ID4gPiA+IEluIHRoZSBjb250cm9sIHBhdGgsIHRoZSBWRFVTRSBk cml2ZXIgd2lsbCBtYWtlIHVzZSBvZiBtZXNzYWdlCj4gPiA+ID4gbWVjaG5pc20gdG8gZm9yd2Fy ZCB0aGUgY29uZmlnIG9wZXJhdGlvbiBmcm9tIHZkcGEgYnVzIGRyaXZlcgo+ID4gPiA+IHRvIHVz ZXJzcGFjZS4gVXNlcnNwYWNlIGNhbiB1c2UgcmVhZCgpL3dyaXRlKCkgdG8gcmVjZWl2ZS9yZXBs eQo+ID4gPiA+IHRob3NlIGNvbnRyb2wgbWVzc2FnZXMuCj4gPiA+ID4gCj4gPiA+ID4gSW4gdGhl IGRhdGEgcGF0aCwgdGhlIGNvcmUgaXMgbWFwcGluZyBkbWEgYnVmZmVyIGludG8gVkRVU0UKPiA+ ID4gPiBkYWVtb24ncyBhZGRyZXNzIHNwYWNlLCB3aGljaCBjYW4gYmUgaW1wbGVtZW50ZWQgaW4g ZGlmZmVyZW50IHdheXMKPiA+ID4gPiBkZXBlbmRpbmcgb24gdGhlIHZkcGEgYnVzIHRvIHdoaWNo IHRoZSB2RFBBIGRldmljZSBpcyBhdHRhY2hlZC4KPiA+ID4gPiAKPiA+ID4gPiBJbiB2aXJ0aW8t dmRwYSBjYXNlLCB3ZSBpbXBsZW1lbnRzIGEgTU1VLWJhc2VkIG9uLWNoaXAgSU9NTVUgZHJpdmVy IHdpdGgKPiA+ID4gPiBib3VuY2UtYnVmZmVyaW5nIG1lY2hhbmlzbSB0byBhY2hpZXZlIHRoYXQu IEFuZCBpbiB2aG9zdC12ZHBhIGNhc2UsIHRoZSBkbWEKPiA+ID4gPiBidWZmZXIgaXMgcmVzaWRl IGluIGEgdXNlcnNwYWNlIG1lbW9yeSByZWdpb24gd2hpY2ggY2FuIGJlIHNoYXJlZCB0byB0aGUK PiA+ID4gPiBWRFVTRSB1c2Vyc3BhY2UgcHJvY2Vzc3MgdmlhIHRyYW5zZmVycmluZyB0aGUgc2ht ZmQuCj4gPiA+ID4gCj4gPiA+ID4gVGhlIGRldGFpbHMgYW5kIG91ciB1c2VyIGNhc2UgaXMgc2hv d24gYmVsb3c6Cj4gPiA+ID4gCj4gPiA+ID4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tICAgIC0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gICAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tCj4gPiA+ID4gfCAgICAgICAgICAgIENvbnRhaW5lciB8ICAgIHwgICAg ICAgICAgICAgIFFFTVUoVk0pIHwgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZE VVNFIGRhZW1vbiB8Cj4gPiA+ID4gfCAgICAgICAtLS0tLS0tLS0gICAgICB8ICAgIHwgIC0tLS0t LS0tLS0tLS0tLS0tLS0gIHwgICB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gLS0tLS0tLS0t LS0tLS0tLSB8Cj4gPiA+ID4gfCAgICAgICB8ZGV2L3ZkeHwgICAgICB8ICAgIHwgIHwvZGV2L3Zo b3N0LXZkcGEteHwgIHwgICB8IHwgdkRQQSBkZXZpY2UgZW11bGF0aW9uIHwgfCBibG9jayBkcml2 ZXIgfCB8Cj4gPiA+ID4gLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAtLS0tLS0tLS0tLSst LS0tLS0tLS0tLS0gICAtLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0t LS0tCj4gPiA+ID4gICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfAo+ID4gPiA+ ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgIHwKPiA+ID4gPiAtLS0tLS0tLS0t LS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0KPiA+ID4gPiB8ICAgIHwgYmxvY2sg ZGV2aWNlIHwgICAgICAgICAgIHwgIHZob3N0IGRldmljZSB8ICAgICAgICAgICAgfCB2ZHVzZSBk cml2ZXIgfCAgICAgICAgICB8IFRDUC9JUCB8ICAgIHwKPiA+ID4gPiB8ICAgIC0tLS0tLS0rLS0t LS0tLS0gICAgICAgICAgIC0tLS0tLS0tKy0tLS0tLS0tICAgICAgICAgICAgLS0tLS0tLSstLS0t LS0tLSAgICAgICAgICAtLS0tLSstLS0tICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICB8ICAgICAg ICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IC0tLS0tLS0tLS0rLS0tLS0tLS0t LSAgICAgICAtLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAgICAgLS0tLS0tLSstLS0tLS0tICAg ICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IHwgdmlydGlvLWJsayBkcml2ZXIgfCAg ICAgICB8ICB2aG9zdC12ZHBhIGRyaXZlciB8ICAgICAgICAgfCB2ZHBhIGRldmljZSB8ICAgICAg ICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IC0tLS0tLS0tLS0rLS0tLS0tLS0tLSAgICAg ICAtLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAgICAgLS0tLS0tLSstLS0tLS0tICAgICAgICAg ICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICB8ICAgICAgdmlydGlvIGJ1cyAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg ICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgLS0tLS0tLS0rLS0tLSstLS0tLS0tLS0tLSAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg IHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwg ICAgICAgIHwKPiA+ID4gPiB8ICAgICAgLS0tLS0tLS0tLSstLS0tLS0tLS0tICAgICAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAg ICAgIHwKPiA+ID4gPiB8ICAgICAgfCB2aXJ0aW8tYmxrIGRldmljZSB8ICAgICAgICAgICAgfCAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAg IHwKPiA+ID4gPiB8ICAgICAgLS0tLS0tLS0tLSstLS0tLS0tLS0tICAgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwK PiA+ID4gPiB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ ID4gPiB8ICAgICAtLS0tLS0tLS0tLSstLS0tLS0tLS0tLSAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4g PiB8ICAgICB8ICB2aXJ0aW8tdmRwYSBkcml2ZXIgfCAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8 ICAgICAtLS0tLS0tLS0tLSstLS0tLS0tLS0tLSAgICAgICAgICAgfCAgICAgICAgICAgICAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAg ICAgICAgIHwgICAgdmRwYSBidXMgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAt LS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0tLS0tLS0gICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIC0tLSstLS0gICAgIHwKPiA+ID4gPiAtLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLXwgTklDIHwtLS0tLS0KPiA+ID4gPiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAtLS0rLS0tCj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgfAo+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0tLS0tLS0t LSstLS0tLS0tLS0KPiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8IFJlbW90ZSBT dG9yYWdlcyB8Cj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLS0tLS0tLS0tLS0t LS0tLS0tLQo+ID4gPiA+IAo+ID4gPiA+IFdlIG1ha2UgdXNlIG9mIGl0IHRvIGltcGxlbWVudCBh IGJsb2NrIGRldmljZSBjb25uZWN0aW5nIHRvCj4gPiA+ID4gb3VyIGRpc3RyaWJ1dGVkIHN0b3Jh Z2UsIHdoaWNoIGNhbiBiZSB1c2VkIGJvdGggaW4gY29udGFpbmVycyBhbmQKPiA+ID4gPiBWTXMu IFRodXMsIHdlIGNhbiBoYXZlIGFuIHVuaWZpZWQgdGVjaG5vbG9neSBzdGFjayBpbiB0aGlzIHR3 byBjYXNlcy4KPiA+ID4gPiAKPiA+ID4gPiBUbyB0ZXN0IGl0IHdpdGggbnVsbC1ibGs6Cj4gPiA+ ID4gCj4gPiA+ID4gICAgJCBxZW11LXN0b3JhZ2UtZGFlbW9uIFwKPiA+ID4gPiAgICAgICAgLS1j aGFyZGV2IHNvY2tldCxpZD1jaGFybW9uaXRvcixwYXRoPS90bXAvcW1wLnNvY2ssc2VydmVyLG5v d2FpdCBcCj4gPiA+ID4gICAgICAgIC0tbW9uaXRvciBjaGFyZGV2PWNoYXJtb25pdG9yIFwKPiA+ ID4gPiAgICAgICAgLS1ibG9ja2RldiBkcml2ZXI9aG9zdF9kZXZpY2UsY2FjaGUuZGlyZWN0PW9u LGFpbz1uYXRpdmUsZmlsZW5hbWU9L2Rldi9udWxsYjAsbm9kZS1uYW1lPWRpc2swIFwKPiA+ID4g PiAgICAgICAgLS1leHBvcnQgdHlwZT12ZHVzZS1ibGssaWQ9dGVzdCxub2RlLW5hbWU9ZGlzazAs d3JpdGFibGU9b24sbmFtZT12ZHVzZS1udWxsLG51bS1xdWV1ZXM9MTYscXVldWUtc2l6ZT0xMjgK PiA+ID4gPiAKPiA+ID4gPiBUaGUgcWVtdS1zdG9yYWdlLWRhZW1vbiBjYW4gYmUgZm91bmQgYXQg aHR0cHM6Ly9naXRodWIuY29tL2J5dGVkYW5jZS9xZW11L3RyZWUvdmR1c2UKPiA+ID4gPiAKPiA+ ID4gPiBUbyBtYWtlIHRoZSB1c2Vyc3BhY2UgVkRVU0UgcHJvY2Vzc2VzIHN1Y2ggYXMgcWVtdS1z dG9yYWdlLWRhZW1vbiBhYmxlIHRvCj4gPiA+ID4gcnVuIHVucHJpdmlsZWdlZC4gV2UgZGlkIHNv bWUgd29ya3Mgb24gdmlydGlvIGRyaXZlciB0byBhdm9pZCB0cnVzdGluZwo+ID4gPiA+IGRldmlj ZSwgaW5jbHVkaW5nOgo+ID4gPiA+IAo+ID4gPiA+ICAgIC0gdmFsaWRhdGluZyB0aGUgZGV2aWNl IHN0YXR1czoKPiA+ID4gPiAKPiA+ID4gPiAgICAgICogaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcv bGttbC8yMDIxMDUxNzA5MzQyOC42NzAtMS14aWV5b25namlAYnl0ZWRhbmNlLmNvbS8KPiA+ID4g PiAKPiA+ID4gPiAgICAtIHZhbGlkYXRpbmcgdGhlIHVzZWQgbGVuZ3RoOgo+ID4gPiA+IAo+ID4g PiA+ICAgICAgKiBodHRwczovL2xvcmUua2VybmVsLm9yZy9sa21sLzIwMjEwNTE3MDkwODM2LjUz My0xLXhpZXlvbmdqaUBieXRlZGFuY2UuY29tLwo+ID4gPiA+IAo+ID4gPiA+ICAgIC0gdmFsaWRh dGluZyB0aGUgZGV2aWNlIGNvbmZpZzoKPiA+ID4gPiAKPiA+ID4gPiAgICAgICogcGF0Y2ggNCAo InZpcnRpby1ibGs6IEFkZCB2YWxpZGF0aW9uIGZvciBibG9jayBzaXplIGluIGNvbmZpZyBzcGFj ZSIpCj4gPiA+ID4gCj4gPiA+ID4gICAgLSB2YWxpZGF0aW5nIHRoZSBkZXZpY2UgcmVzcG9uc2U6 Cj4gPiA+ID4gCj4gPiA+ID4gICAgICAqIHBhdGNoIDUgKCJ2aXJ0aW9fc2NzaTogQWRkIHZhbGlk YXRpb24gZm9yIHJlc2lkdWFsIGJ5dGVzIGZyb20gcmVzcG9uc2UiKQo+ID4gPiA+IAo+ID4gPiA+ IFNpbmNlIEknbSBub3Qgc3VyZSBpZiBJIG1pc3Npbmcgc29tZXRoaW5nIGR1cmluZyBhdWRpdGlu ZywgZXNwZWNpYWxseSBvbiBzb21lCj4gPiA+ID4gdmlydGlvIGRldmljZSBkcml2ZXJzIHRoYXQg SSdtIG5vdCBmYW1pbGlhciB3aXRoLCBub3cgd2Ugb25seSBzdXBwb3J0IGVtdWFsdGluZwo+ID4g PiA+IGEgZmV3IHZEUEEgZGV2aWNlcyBieSBkZWZhdWx0LCBpbmNsdWRpbmc6IHZpcnRpby1uZXQg ZGV2aWNlLCB2aXJ0aW8tYmxrIGRldmljZSwKPiA+ID4gPiB2aXJ0aW8tc2NzaSBkZXZpY2UgYW5k IHZpcnRpby1mcyBkZXZpY2UuIFRoaXMgbGltaXRhdGlvbiBjYW4gaGVscCB0byByZWR1Y2UKPiA+ ID4gPiBzZWN1cml0eSByaXNrcy4KPiA+ID4gSSBzdXNwZWN0IHRoZXJlIGFyZSBhIGxvdCBvZiBh c3N1bXB0aW9ucyBldmVuIHdpdGggdGhlc2UgNC4KPiA+ID4gSnVzdCB3aGF0IGFyZSB0aGUgc2Vj dXJpdHkgYXNzdW1wdGlvbnMgYW5kIGd1YXJhbnRlZXMgaGVyZT8KPiAKPiAKPiBOb3RlIHRoYXQg VkRVU0UgaXMgbm90IHRoZSBvbmx5IGRldmljZSB0aGF0IG1heSBzdWZmZXIgZnJvbSB0aGlzLCBo ZXJlJ3JlCj4gdHdvIG90aGVyczoKPiAKPiAxKSBFbmNyeXB0ZWQgVk0KCkVuY3J5cHRlZCBWTXMg YXJlIGdlbmVyYWxseSB1bmRlcnN0b29kIG5vdCB0byBiZSBmdWxseQpwcm90ZWN0ZWQgZnJvbSBh dHRhY2tzIGJ5IGEgbWFsaWNpb3VzIGh5cGVydmlzb3IuIEZvciBleGFtcGxlCmEgRG9TIGJ5IGEg aHlwZXJ2aXNvciBpcyBjdXJyZW50bHkgdHJpdmlhbC4KCj4gMikgU21hcnQgTklDcwoKTW9yZSBv ciBsZXNzIHRoZSBzYW1lIHRoaW5nLgoKCj4gCj4gPiBUaGUgYXR0YWNrIHN1cmZhY2UgZnJvbSBh IHZpcnRpbyBkZXZpY2UgaXMgbGltaXRlZCB3aXRoIElPTU1VIGVuYWJsZWQuCj4gPiBJdCBzaG91 bGQgYmUgYWJsZSB0byBhdm9pZCBzZWN1cml0eSByaXNrIGlmIHdlIGNhbiB2YWxpZGF0ZSBhbGwg ZGF0YQo+ID4gc3VjaCBhcyBjb25maWcgc3BhY2UgYW5kIHVzZWQgbGVuZ3RoIGZyb20gZGV2aWNl IGluIGRldmljZSBkcml2ZXIuCj4gPiAKPiA+ID4gRS5nLiBpdCBzZWVtcyBwcmV0dHkgY2xlYXIg dGhhdCBleHBvc2luZyBhIG1hbGZvcm1lZCBGUwo+ID4gPiB0byBhIHJhbmRvbSBrZXJuZWwgY29u ZmlnIGNhbiBjYXVzZSB1bnRvbGQgbWlzY2hpZWYuCj4gPiA+IAo+ID4gPiBUaGluZ3MgbGlrZSB2 aXJ0bmV0X3NlbmRfY29tbWFuZCBhcmUgYWxzbyBhbiBlYXN5IHdheSBmb3IKPiA+ID4gdGhlIGRl dmljZSB0byBET1MgdGhlIGtlcm5lbC4KPiAKPiAKPiBJIHRoaW5rIHRoZSB2aXJ0bmV0X3NlbmRf Y29tbWFuZCgpIG5lZWRzIHRvIHVzZSBpbnRlcnJ1cHQgaW5zdGVhZCBvZgo+IHBvbGxpbmcuCj4g Cj4gVGhhbmtzCj4gCj4gCj4gPiA+IEFuZCBiZWZvcmUgeW91IHRyeSB0byBhZGQKPiA+ID4gYW4g YXJiaXRyYXJ5IHRpbWVvdXQgdGhlcmUgLSBwbGVhc2UgZG9uJ3QsCj4gPiA+IHRoZSBmaXggaXMg bW92aW5nIHRoaW5ncyB0aGF0IG11c3QgYmUgZ3VhcmFudGVlZCBpbnRvIGtlcm5lbAo+ID4gPiBh bmQgbWFraW5nIHRoaW5ncyB0aGF0IGFyZSBub3QgZ3VhcmFudGVlZCBhc3luY2hyb25vdXMuCj4g PiA+IFJpZ2h0IG5vdyB0aGVyZSBhcmUgc29tZSB0aGluZ3MgdGhhdCBoYXBwZW4gd2l0aCBsb2Nr cyB0YWtlbiwKPiA+ID4gd2hlcmUgaWYgd2UgZG9uJ3Qgd2FpdCBmb3IgZGV2aWNlIHdlIGxvc2Ug dGhlIGFiaWxpdHkgdG8gcmVwb3J0IGZhaWx1cmVzCj4gPiA+IHRvIHVzZXJzcGFjZS4gRS5nLiBh bGwga2luZCBvZiBuZXRsaW5rIHRoaW5ncyBhcmUgbGlrZSB0aGlzLgo+ID4gPiBPbmUgY2FuIHRo aW5rIG9mIGEgYnVuY2ggb2Ygd2F5cyB0byBhZGRyZXNzIHRoaXMsIHRoaXMKPiA+ID4gbmVlZHMg dG8gYmUgZGlzY3Vzc2VkIHdpdGggdGhlIHJlbGV2YW50IHN1YnN5c3RlbSBtYWludGFpbmVycy4K PiA+ID4gCj4gPiA+IAo+ID4gPiBJZiBJIHdlcmUgeW91IEkgd291bGQgc3RhcnQgd2l0aCBvbmUg dHlwZSBvZiBkZXZpY2UsIGFuZCBhcyBzaW1wbGUgb25lCj4gPiA+IGFzIHBvc3NpYmxlLgo+ID4g PiAKPiA+IE1ha2Ugc2Vuc2UgdG8gbWUuIFRoZSB2aXJ0aW8tYmxrIGRldmljZSBtaWdodCBiZSBh IGdvb2Qgc3RhcnQuIFdlCj4gPiBhbHJlYWR5IGhhdmUgc29tZSBleGlzdGluZyBpbnRlcmZhY2Ug bGlrZSBOQkQgdG8gZG8gc2ltaWxhciB0aGluZ3MuCj4gPiAKPiA+ID4gCj4gPiA+ID4gV2hlbiBh IHN5c2FkbWluIHRydXN0cyB0aGUgdXNlcnNwYWNlIHByb2Nlc3MgZW5vdWdoLCBpdCBjYW4gcmVs YXgKPiA+ID4gPiB0aGUgbGltaXRhdGlvbiB3aXRoIGEgJ2FsbG93X3Vuc2FmZV9kZXZpY2VfZW11 bGF0aW9uJyBtb2R1bGUgcGFyYW1ldGVyLgo+ID4gPiBUaGF0J3Mgbm90IGEgZ3JlYXQgc2VjdXJp dHkgaW50ZXJmYWNlLiBJdCdzIGEgZ2xvYmFsIG1vZHVsZSBzcGVjaWZpYyBrbm9iCj4gPiA+IHRo YXQganVzdCBhbGxvd3MgYW55IHVzZXJzcGFjZSB0byBlbXVsYXRlIGFueXRoaW5nIGF0IGFsbC4K PiA+ID4gQ29taW5nIHVwIHdpdGggYSByZWFzb25hYmxlIGludGVyZmFjZSBpc24ndCBnb2luZyB0 byBiZSBlYXN5Lgo+ID4gPiBGb3Igbm93IG1heWJlIGp1c3QgaGF2ZSBwZW9wbGUgcGF0Y2ggdGhl aXIga2VybmVscyBpZiB0aGV5IHdhbnQgdG8KPiA+ID4gbW92ZSBmYXN0IGFuZCBicmVhayB0aGlu Z3MuCj4gPiA+IAo+ID4gT0suIEEgcmVhc29uYWJsZSBpbnRlcmZhY2UgY2FuIGJlIGFkZGVkIGlm IHdlIG5lZWQgaXQgaW4gdGhlIGZ1dHVyZS4KPiA+IAo+ID4gVGhhbmtzLAo+ID4gWW9uZ2ppCgpf X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBtYWls aW5nIGxpc3QKaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5s aW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD941C2B9F8 for ; Tue, 25 May 2021 06:48:46 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E8D261413 for ; Tue, 25 May 2021 06:48:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E8D261413 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4554E40187; Tue, 25 May 2021 06:48:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QqIqLF9w1qvA; Tue, 25 May 2021 06:48:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTP id 813744010E; Tue, 25 May 2021 06:48:44 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 58F43C000D; Tue, 25 May 2021 06:48:44 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 41A54C0001 for ; Tue, 25 May 2021 06:48:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 231F140187 for ; Tue, 25 May 2021 06:48:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mT1i3nwgVBmH for ; Tue, 25 May 2021 06:48:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id EC7FB4010E for ; Tue, 25 May 2021 06:48:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621925320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=MH3+bMFzvyVKEJIrrqwy0liIA1OeDGOZ6p13LDotaHU67k49XxGwAjuwCzjIGkUvyB/NqF ByHhfqR9PmhqpTyqSx83ll59tRKRxLkCqlF7y8WyeuG7wmK2dqMyX7uHQIQpTrJug5eEND SXnCFoewWC3IN+RwURN9QB6O6rDaxS4= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-400-zorvrpw0NiyPgRNCE4muEA-1; Tue, 25 May 2021 02:48:38 -0400 X-MC-Unique: zorvrpw0NiyPgRNCE4muEA-1 Received: by mail-wr1-f72.google.com with SMTP id 2-20020adf94020000b0290110481f75ddso14135998wrq.21 for ; Mon, 24 May 2021 23:48:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=bgsbzKlv62hpr8zhuerEIFsBSinMHL1nlquEyAT4hLcZpbllj6pQbqNPxKPFl7Ng1n I6OKAyS2yp1ZJ1g1zodfJJP5WXtFcrvBDaVdBrpmv4s0gZUHlrT9M93SAqCtcDOl5f7a BgPMkGb2Xvc8+RLGxieImFNtHZC5sEkI0c+TnLwlrTQBAkNutDGHYxG3eO+uC5GBUC2H zLx1PIowUoa9zG5EI5XYl1QPoizd/ceYXRgPvox2G12Fd8OctuaLAPdOhLdepwDJV31O DfzBzTnFcwg8fjzEjapWJVEnNSFlc+Dpj5xOQ9G3CaaUbf+MSxcC3AGksCBLArm2SJq8 ROaA== X-Gm-Message-State: AOAM533xNV3GsWEsR90YYV48JrGZsLDP/pU55IXlqwO6ycZZQK5AjBSs XSsu2+rRpNnaD8UDsV9zGAXeaYcoIDJVqDUh0n4CNkqcpIwn5JVD5UR6jkizupikT4pTuicM7TN RcaYRbLjNZ//sTDod7T8MAGil6MbixZrPByjIeARzRg== X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535533wrf.420.1621925317064; Mon, 24 May 2021 23:48:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwHrJfvA/haiL2kDkU2Sc824s/VhxDUz9WKfrjZUVALV+rMusOmrmXdB7xQ8MMFAx4z9xMC5w== X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535496wrf.420.1621925316796; Mon, 24 May 2021 23:48:36 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id g78sm1839253wme.27.2021.05.24.23.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 May 2021 23:48:34 -0700 (PDT) Date: Tue, 25 May 2021 02:48:30 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [PATCH v7 00/12] Introduce VDUSE - vDPA Device in Userspace Message-ID: <20210525024500-mutt-send-email-mst@kernel.org> References: <20210517095513.850-1-xieyongji@bytedance.com> <20210520014349-mutt-send-email-mst@kernel.org> <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> MIME-Version: 1.0 In-Reply-To: <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: Jens Axboe , linux-kernel , kvm , Jonathan Corbet , netdev@vger.kernel.org, joro@8bytes.org, Randy Dunlap , iommu@lists.linux-foundation.org, Matthew Wilcox , virtualization , Christoph Hellwig , Yongji Xie , Christian Brauner , bcrl@kvack.org, Al Viro , Stefan Hajnoczi , linux-fsdevel@vger.kernel.org, Dan Carpenter , Mika =?iso-8859-1?Q?Penttil=E4?= X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gVHVlLCBNYXkgMjUsIDIwMjEgYXQgMDI6NDA6NTdQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiDlnKggMjAyMS81LzIwIOS4i+WNiDU6MDYsIFlvbmdqaSBYaWUg5YaZ6YGTOgo+ID4g T24gVGh1LCBNYXkgMjAsIDIwMjEgYXQgMjowNiBQTSBNaWNoYWVsIFMuIFRzaXJraW4gPG1zdEBy ZWRoYXQuY29tPiB3cm90ZToKPiA+ID4gT24gTW9uLCBNYXkgMTcsIDIwMjEgYXQgMDU6NTU6MDFQ TSArMDgwMCwgWGllIFlvbmdqaSB3cm90ZToKPiA+ID4gPiBUaGlzIHNlcmllcyBpbnRyb2R1Y2Vz IGEgZnJhbWV3b3JrLCB3aGljaCBjYW4gYmUgdXNlZCB0byBpbXBsZW1lbnQKPiA+ID4gPiB2RFBB IERldmljZXMgaW4gYSB1c2Vyc3BhY2UgcHJvZ3JhbS4gVGhlIHdvcmsgY29uc2lzdCBvZiB0d28g cGFydHM6Cj4gPiA+ID4gY29udHJvbCBwYXRoIGZvcndhcmRpbmcgYW5kIGRhdGEgcGF0aCBvZmZs b2FkaW5nLgo+ID4gPiA+IAo+ID4gPiA+IEluIHRoZSBjb250cm9sIHBhdGgsIHRoZSBWRFVTRSBk cml2ZXIgd2lsbCBtYWtlIHVzZSBvZiBtZXNzYWdlCj4gPiA+ID4gbWVjaG5pc20gdG8gZm9yd2Fy ZCB0aGUgY29uZmlnIG9wZXJhdGlvbiBmcm9tIHZkcGEgYnVzIGRyaXZlcgo+ID4gPiA+IHRvIHVz ZXJzcGFjZS4gVXNlcnNwYWNlIGNhbiB1c2UgcmVhZCgpL3dyaXRlKCkgdG8gcmVjZWl2ZS9yZXBs eQo+ID4gPiA+IHRob3NlIGNvbnRyb2wgbWVzc2FnZXMuCj4gPiA+ID4gCj4gPiA+ID4gSW4gdGhl IGRhdGEgcGF0aCwgdGhlIGNvcmUgaXMgbWFwcGluZyBkbWEgYnVmZmVyIGludG8gVkRVU0UKPiA+ ID4gPiBkYWVtb24ncyBhZGRyZXNzIHNwYWNlLCB3aGljaCBjYW4gYmUgaW1wbGVtZW50ZWQgaW4g ZGlmZmVyZW50IHdheXMKPiA+ID4gPiBkZXBlbmRpbmcgb24gdGhlIHZkcGEgYnVzIHRvIHdoaWNo IHRoZSB2RFBBIGRldmljZSBpcyBhdHRhY2hlZC4KPiA+ID4gPiAKPiA+ID4gPiBJbiB2aXJ0aW8t dmRwYSBjYXNlLCB3ZSBpbXBsZW1lbnRzIGEgTU1VLWJhc2VkIG9uLWNoaXAgSU9NTVUgZHJpdmVy IHdpdGgKPiA+ID4gPiBib3VuY2UtYnVmZmVyaW5nIG1lY2hhbmlzbSB0byBhY2hpZXZlIHRoYXQu IEFuZCBpbiB2aG9zdC12ZHBhIGNhc2UsIHRoZSBkbWEKPiA+ID4gPiBidWZmZXIgaXMgcmVzaWRl IGluIGEgdXNlcnNwYWNlIG1lbW9yeSByZWdpb24gd2hpY2ggY2FuIGJlIHNoYXJlZCB0byB0aGUK PiA+ID4gPiBWRFVTRSB1c2Vyc3BhY2UgcHJvY2Vzc3MgdmlhIHRyYW5zZmVycmluZyB0aGUgc2ht ZmQuCj4gPiA+ID4gCj4gPiA+ID4gVGhlIGRldGFpbHMgYW5kIG91ciB1c2VyIGNhc2UgaXMgc2hv d24gYmVsb3c6Cj4gPiA+ID4gCj4gPiA+ID4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tICAgIC0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gICAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tCj4gPiA+ID4gfCAgICAgICAgICAgIENvbnRhaW5lciB8ICAgIHwgICAg ICAgICAgICAgIFFFTVUoVk0pIHwgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZE VVNFIGRhZW1vbiB8Cj4gPiA+ID4gfCAgICAgICAtLS0tLS0tLS0gICAgICB8ICAgIHwgIC0tLS0t LS0tLS0tLS0tLS0tLS0gIHwgICB8IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gLS0tLS0tLS0t LS0tLS0tLSB8Cj4gPiA+ID4gfCAgICAgICB8ZGV2L3ZkeHwgICAgICB8ICAgIHwgIHwvZGV2L3Zo b3N0LXZkcGEteHwgIHwgICB8IHwgdkRQQSBkZXZpY2UgZW11bGF0aW9uIHwgfCBibG9jayBkcml2 ZXIgfCB8Cj4gPiA+ID4gLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAtLS0tLS0tLS0tLSst LS0tLS0tLS0tLS0gICAtLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0t LS0tCj4gPiA+ID4gICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfAo+ID4gPiA+ ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgICAgICAgIHwKPiA+ID4gPiAtLS0tLS0tLS0t LS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0rLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0KPiA+ID4gPiB8ICAgIHwgYmxvY2sg ZGV2aWNlIHwgICAgICAgICAgIHwgIHZob3N0IGRldmljZSB8ICAgICAgICAgICAgfCB2ZHVzZSBk cml2ZXIgfCAgICAgICAgICB8IFRDUC9JUCB8ICAgIHwKPiA+ID4gPiB8ICAgIC0tLS0tLS0rLS0t LS0tLS0gICAgICAgICAgIC0tLS0tLS0tKy0tLS0tLS0tICAgICAgICAgICAgLS0tLS0tLSstLS0t LS0tLSAgICAgICAgICAtLS0tLSstLS0tICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICB8ICAgICAg ICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAg ICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IC0tLS0tLS0tLS0rLS0tLS0tLS0t LSAgICAgICAtLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAgICAgLS0tLS0tLSstLS0tLS0tICAg ICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IHwgdmlydGlvLWJsayBkcml2ZXIgfCAg ICAgICB8ICB2aG9zdC12ZHBhIGRyaXZlciB8ICAgICAgICAgfCB2ZHBhIGRldmljZSB8ICAgICAg ICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8IC0tLS0tLS0tLS0rLS0tLS0tLS0tLSAgICAg ICAtLS0tLS0tLS0tKy0tLS0tLS0tLS0tICAgICAgICAgLS0tLS0tLSstLS0tLS0tICAgICAgICAg ICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICB8ICAgICAgdmlydGlvIGJ1cyAg ICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg ICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgLS0tLS0tLS0rLS0tLSstLS0tLS0tLS0tLSAgICAg ICAgICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAg IHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAg ICAgfCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwg ICAgICAgIHwKPiA+ID4gPiB8ICAgICAgLS0tLS0tLS0tLSstLS0tLS0tLS0tICAgICAgICAgICAg fCAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAg ICAgIHwKPiA+ID4gPiB8ICAgICAgfCB2aXJ0aW8tYmxrIGRldmljZSB8ICAgICAgICAgICAgfCAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAg IHwKPiA+ID4gPiB8ICAgICAgLS0tLS0tLS0tLSstLS0tLS0tLS0tICAgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwK PiA+ID4gPiB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ ID4gPiB8ICAgICAtLS0tLS0tLS0tLSstLS0tLS0tLS0tLSAgICAgICAgICAgfCAgICAgICAgICAg ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4g PiB8ICAgICB8ICB2aXJ0aW8tdmRwYSBkcml2ZXIgfCAgICAgICAgICAgfCAgICAgICAgICAgICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8 ICAgICAtLS0tLS0tLS0tLSstLS0tLS0tLS0tLSAgICAgICAgICAgfCAgICAgICAgICAgICAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAg ICAgICAgICAgICAgIHwgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICAgICAg ICAgICAgIHwgICAgdmRwYSBidXMgICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAt LS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLSstLS0tLS0tLS0tLS0gICAgICAgICAgIHwgICAgICAgIHwKPiA+ID4gPiB8ICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIC0tLSstLS0gICAgIHwKPiA+ID4gPiAtLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLXwgTklDIHwtLS0tLS0KPiA+ID4gPiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAtLS0rLS0tCj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgfAo+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0tLS0tLS0t LSstLS0tLS0tLS0KPiA+ID4gPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8IFJlbW90ZSBT dG9yYWdlcyB8Cj4gPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLS0tLS0tLS0tLS0t LS0tLS0tLQo+ID4gPiA+IAo+ID4gPiA+IFdlIG1ha2UgdXNlIG9mIGl0IHRvIGltcGxlbWVudCBh IGJsb2NrIGRldmljZSBjb25uZWN0aW5nIHRvCj4gPiA+ID4gb3VyIGRpc3RyaWJ1dGVkIHN0b3Jh Z2UsIHdoaWNoIGNhbiBiZSB1c2VkIGJvdGggaW4gY29udGFpbmVycyBhbmQKPiA+ID4gPiBWTXMu IFRodXMsIHdlIGNhbiBoYXZlIGFuIHVuaWZpZWQgdGVjaG5vbG9neSBzdGFjayBpbiB0aGlzIHR3 byBjYXNlcy4KPiA+ID4gPiAKPiA+ID4gPiBUbyB0ZXN0IGl0IHdpdGggbnVsbC1ibGs6Cj4gPiA+ ID4gCj4gPiA+ID4gICAgJCBxZW11LXN0b3JhZ2UtZGFlbW9uIFwKPiA+ID4gPiAgICAgICAgLS1j aGFyZGV2IHNvY2tldCxpZD1jaGFybW9uaXRvcixwYXRoPS90bXAvcW1wLnNvY2ssc2VydmVyLG5v d2FpdCBcCj4gPiA+ID4gICAgICAgIC0tbW9uaXRvciBjaGFyZGV2PWNoYXJtb25pdG9yIFwKPiA+ ID4gPiAgICAgICAgLS1ibG9ja2RldiBkcml2ZXI9aG9zdF9kZXZpY2UsY2FjaGUuZGlyZWN0PW9u LGFpbz1uYXRpdmUsZmlsZW5hbWU9L2Rldi9udWxsYjAsbm9kZS1uYW1lPWRpc2swIFwKPiA+ID4g PiAgICAgICAgLS1leHBvcnQgdHlwZT12ZHVzZS1ibGssaWQ9dGVzdCxub2RlLW5hbWU9ZGlzazAs d3JpdGFibGU9b24sbmFtZT12ZHVzZS1udWxsLG51bS1xdWV1ZXM9MTYscXVldWUtc2l6ZT0xMjgK PiA+ID4gPiAKPiA+ID4gPiBUaGUgcWVtdS1zdG9yYWdlLWRhZW1vbiBjYW4gYmUgZm91bmQgYXQg aHR0cHM6Ly9naXRodWIuY29tL2J5dGVkYW5jZS9xZW11L3RyZWUvdmR1c2UKPiA+ID4gPiAKPiA+ ID4gPiBUbyBtYWtlIHRoZSB1c2Vyc3BhY2UgVkRVU0UgcHJvY2Vzc2VzIHN1Y2ggYXMgcWVtdS1z dG9yYWdlLWRhZW1vbiBhYmxlIHRvCj4gPiA+ID4gcnVuIHVucHJpdmlsZWdlZC4gV2UgZGlkIHNv bWUgd29ya3Mgb24gdmlydGlvIGRyaXZlciB0byBhdm9pZCB0cnVzdGluZwo+ID4gPiA+IGRldmlj ZSwgaW5jbHVkaW5nOgo+ID4gPiA+IAo+ID4gPiA+ICAgIC0gdmFsaWRhdGluZyB0aGUgZGV2aWNl IHN0YXR1czoKPiA+ID4gPiAKPiA+ID4gPiAgICAgICogaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcv bGttbC8yMDIxMDUxNzA5MzQyOC42NzAtMS14aWV5b25namlAYnl0ZWRhbmNlLmNvbS8KPiA+ID4g PiAKPiA+ID4gPiAgICAtIHZhbGlkYXRpbmcgdGhlIHVzZWQgbGVuZ3RoOgo+ID4gPiA+IAo+ID4g PiA+ICAgICAgKiBodHRwczovL2xvcmUua2VybmVsLm9yZy9sa21sLzIwMjEwNTE3MDkwODM2LjUz My0xLXhpZXlvbmdqaUBieXRlZGFuY2UuY29tLwo+ID4gPiA+IAo+ID4gPiA+ICAgIC0gdmFsaWRh dGluZyB0aGUgZGV2aWNlIGNvbmZpZzoKPiA+ID4gPiAKPiA+ID4gPiAgICAgICogcGF0Y2ggNCAo InZpcnRpby1ibGs6IEFkZCB2YWxpZGF0aW9uIGZvciBibG9jayBzaXplIGluIGNvbmZpZyBzcGFj ZSIpCj4gPiA+ID4gCj4gPiA+ID4gICAgLSB2YWxpZGF0aW5nIHRoZSBkZXZpY2UgcmVzcG9uc2U6 Cj4gPiA+ID4gCj4gPiA+ID4gICAgICAqIHBhdGNoIDUgKCJ2aXJ0aW9fc2NzaTogQWRkIHZhbGlk YXRpb24gZm9yIHJlc2lkdWFsIGJ5dGVzIGZyb20gcmVzcG9uc2UiKQo+ID4gPiA+IAo+ID4gPiA+ IFNpbmNlIEknbSBub3Qgc3VyZSBpZiBJIG1pc3Npbmcgc29tZXRoaW5nIGR1cmluZyBhdWRpdGlu ZywgZXNwZWNpYWxseSBvbiBzb21lCj4gPiA+ID4gdmlydGlvIGRldmljZSBkcml2ZXJzIHRoYXQg SSdtIG5vdCBmYW1pbGlhciB3aXRoLCBub3cgd2Ugb25seSBzdXBwb3J0IGVtdWFsdGluZwo+ID4g PiA+IGEgZmV3IHZEUEEgZGV2aWNlcyBieSBkZWZhdWx0LCBpbmNsdWRpbmc6IHZpcnRpby1uZXQg ZGV2aWNlLCB2aXJ0aW8tYmxrIGRldmljZSwKPiA+ID4gPiB2aXJ0aW8tc2NzaSBkZXZpY2UgYW5k IHZpcnRpby1mcyBkZXZpY2UuIFRoaXMgbGltaXRhdGlvbiBjYW4gaGVscCB0byByZWR1Y2UKPiA+ ID4gPiBzZWN1cml0eSByaXNrcy4KPiA+ID4gSSBzdXNwZWN0IHRoZXJlIGFyZSBhIGxvdCBvZiBh c3N1bXB0aW9ucyBldmVuIHdpdGggdGhlc2UgNC4KPiA+ID4gSnVzdCB3aGF0IGFyZSB0aGUgc2Vj dXJpdHkgYXNzdW1wdGlvbnMgYW5kIGd1YXJhbnRlZXMgaGVyZT8KPiAKPiAKPiBOb3RlIHRoYXQg VkRVU0UgaXMgbm90IHRoZSBvbmx5IGRldmljZSB0aGF0IG1heSBzdWZmZXIgZnJvbSB0aGlzLCBo ZXJlJ3JlCj4gdHdvIG90aGVyczoKPiAKPiAxKSBFbmNyeXB0ZWQgVk0KCkVuY3J5cHRlZCBWTXMg YXJlIGdlbmVyYWxseSB1bmRlcnN0b29kIG5vdCB0byBiZSBmdWxseQpwcm90ZWN0ZWQgZnJvbSBh dHRhY2tzIGJ5IGEgbWFsaWNpb3VzIGh5cGVydmlzb3IuIEZvciBleGFtcGxlCmEgRG9TIGJ5IGEg aHlwZXJ2aXNvciBpcyBjdXJyZW50bHkgdHJpdmlhbC4KCj4gMikgU21hcnQgTklDcwoKTW9yZSBv ciBsZXNzIHRoZSBzYW1lIHRoaW5nLgoKCj4gCj4gPiBUaGUgYXR0YWNrIHN1cmZhY2UgZnJvbSBh IHZpcnRpbyBkZXZpY2UgaXMgbGltaXRlZCB3aXRoIElPTU1VIGVuYWJsZWQuCj4gPiBJdCBzaG91 bGQgYmUgYWJsZSB0byBhdm9pZCBzZWN1cml0eSByaXNrIGlmIHdlIGNhbiB2YWxpZGF0ZSBhbGwg ZGF0YQo+ID4gc3VjaCBhcyBjb25maWcgc3BhY2UgYW5kIHVzZWQgbGVuZ3RoIGZyb20gZGV2aWNl IGluIGRldmljZSBkcml2ZXIuCj4gPiAKPiA+ID4gRS5nLiBpdCBzZWVtcyBwcmV0dHkgY2xlYXIg dGhhdCBleHBvc2luZyBhIG1hbGZvcm1lZCBGUwo+ID4gPiB0byBhIHJhbmRvbSBrZXJuZWwgY29u ZmlnIGNhbiBjYXVzZSB1bnRvbGQgbWlzY2hpZWYuCj4gPiA+IAo+ID4gPiBUaGluZ3MgbGlrZSB2 aXJ0bmV0X3NlbmRfY29tbWFuZCBhcmUgYWxzbyBhbiBlYXN5IHdheSBmb3IKPiA+ID4gdGhlIGRl dmljZSB0byBET1MgdGhlIGtlcm5lbC4KPiAKPiAKPiBJIHRoaW5rIHRoZSB2aXJ0bmV0X3NlbmRf Y29tbWFuZCgpIG5lZWRzIHRvIHVzZSBpbnRlcnJ1cHQgaW5zdGVhZCBvZgo+IHBvbGxpbmcuCj4g Cj4gVGhhbmtzCj4gCj4gCj4gPiA+IEFuZCBiZWZvcmUgeW91IHRyeSB0byBhZGQKPiA+ID4gYW4g YXJiaXRyYXJ5IHRpbWVvdXQgdGhlcmUgLSBwbGVhc2UgZG9uJ3QsCj4gPiA+IHRoZSBmaXggaXMg bW92aW5nIHRoaW5ncyB0aGF0IG11c3QgYmUgZ3VhcmFudGVlZCBpbnRvIGtlcm5lbAo+ID4gPiBh bmQgbWFraW5nIHRoaW5ncyB0aGF0IGFyZSBub3QgZ3VhcmFudGVlZCBhc3luY2hyb25vdXMuCj4g PiA+IFJpZ2h0IG5vdyB0aGVyZSBhcmUgc29tZSB0aGluZ3MgdGhhdCBoYXBwZW4gd2l0aCBsb2Nr cyB0YWtlbiwKPiA+ID4gd2hlcmUgaWYgd2UgZG9uJ3Qgd2FpdCBmb3IgZGV2aWNlIHdlIGxvc2Ug dGhlIGFiaWxpdHkgdG8gcmVwb3J0IGZhaWx1cmVzCj4gPiA+IHRvIHVzZXJzcGFjZS4gRS5nLiBh bGwga2luZCBvZiBuZXRsaW5rIHRoaW5ncyBhcmUgbGlrZSB0aGlzLgo+ID4gPiBPbmUgY2FuIHRo aW5rIG9mIGEgYnVuY2ggb2Ygd2F5cyB0byBhZGRyZXNzIHRoaXMsIHRoaXMKPiA+ID4gbmVlZHMg dG8gYmUgZGlzY3Vzc2VkIHdpdGggdGhlIHJlbGV2YW50IHN1YnN5c3RlbSBtYWludGFpbmVycy4K PiA+ID4gCj4gPiA+IAo+ID4gPiBJZiBJIHdlcmUgeW91IEkgd291bGQgc3RhcnQgd2l0aCBvbmUg dHlwZSBvZiBkZXZpY2UsIGFuZCBhcyBzaW1wbGUgb25lCj4gPiA+IGFzIHBvc3NpYmxlLgo+ID4g PiAKPiA+IE1ha2Ugc2Vuc2UgdG8gbWUuIFRoZSB2aXJ0aW8tYmxrIGRldmljZSBtaWdodCBiZSBh IGdvb2Qgc3RhcnQuIFdlCj4gPiBhbHJlYWR5IGhhdmUgc29tZSBleGlzdGluZyBpbnRlcmZhY2Ug bGlrZSBOQkQgdG8gZG8gc2ltaWxhciB0aGluZ3MuCj4gPiAKPiA+ID4gCj4gPiA+ID4gV2hlbiBh IHN5c2FkbWluIHRydXN0cyB0aGUgdXNlcnNwYWNlIHByb2Nlc3MgZW5vdWdoLCBpdCBjYW4gcmVs YXgKPiA+ID4gPiB0aGUgbGltaXRhdGlvbiB3aXRoIGEgJ2FsbG93X3Vuc2FmZV9kZXZpY2VfZW11 bGF0aW9uJyBtb2R1bGUgcGFyYW1ldGVyLgo+ID4gPiBUaGF0J3Mgbm90IGEgZ3JlYXQgc2VjdXJp dHkgaW50ZXJmYWNlLiBJdCdzIGEgZ2xvYmFsIG1vZHVsZSBzcGVjaWZpYyBrbm9iCj4gPiA+IHRo YXQganVzdCBhbGxvd3MgYW55IHVzZXJzcGFjZSB0byBlbXVsYXRlIGFueXRoaW5nIGF0IGFsbC4K PiA+ID4gQ29taW5nIHVwIHdpdGggYSByZWFzb25hYmxlIGludGVyZmFjZSBpc24ndCBnb2luZyB0 byBiZSBlYXN5Lgo+ID4gPiBGb3Igbm93IG1heWJlIGp1c3QgaGF2ZSBwZW9wbGUgcGF0Y2ggdGhl aXIga2VybmVscyBpZiB0aGV5IHdhbnQgdG8KPiA+ID4gbW92ZSBmYXN0IGFuZCBicmVhayB0aGlu Z3MuCj4gPiA+IAo+ID4gT0suIEEgcmVhc29uYWJsZSBpbnRlcmZhY2UgY2FuIGJlIGFkZGVkIGlm IHdlIG5lZWQgaXQgaW4gdGhlIGZ1dHVyZS4KPiA+IAo+ID4gVGhhbmtzLAo+ID4gWW9uZ2ppCgpf X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpWaXJ0dWFsaXph dGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRhdGlvbi5v cmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vdmly dHVhbGl6YXRpb24= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 078C0C47085 for ; Tue, 25 May 2021 06:48:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E359361431 for ; Tue, 25 May 2021 06:48:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231410AbhEYGuN (ORCPT ); Tue, 25 May 2021 02:50:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26970 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231396AbhEYGuL (ORCPT ); Tue, 25 May 2021 02:50:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621925322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=U+MXAtJ+92tLWvy1A8WClR1rTAynn2EprzDjAnUev7Pn7nRzpZpgbQvJeBNH6wSf2T7Urr hoUj/ihCg6QAkkNPhNv11/Gfi7rWhot4ndvyeO7BeNa4gYKNDsLcfvcDooPOVBBs2bTzxs hQ3nIjYS1hajEqoS3YTYhwMgtOE6vC8= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-395-YLPGeeXvMEiGWcREdz5_AQ-1; Tue, 25 May 2021 02:48:38 -0400 X-MC-Unique: YLPGeeXvMEiGWcREdz5_AQ-1 Received: by mail-wr1-f72.google.com with SMTP id i15-20020a5d558f0000b029011215b1cf5cso7735580wrv.22 for ; Mon, 24 May 2021 23:48:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=ul3miHPA+1C0mVf5IsSNMuGBmo+BqIM3N/sQ86Z+ItM=; b=QXk0LymVX/EM85bEvEl513PmfmgSW2lACI0rk/31uRn/rZs6HTs+Rd78KZvhq62R2X Cb8FpJFwUguFzCQ5sej2cjLpyPL+y1WENoGl85YqHN3sCQQri0yFQeNp/9XKG854v5Wt 36cLKwQcLwWkdRZKgKlb6qVyVBkunZqiCVajQUBpu4i0gcNQsiYUpFpOsi1i+L5QCmFS HuHYzM16ng9Ur0UuClQB3dSMMuvvpmC8f9FCk4x56JM/g80C8cpPoQZD31Y9uuT9aJub O8o/Sn7t3lUEf6C0SBoyAy99BCPCYe/ukYkj/+Sjcr8xZkNydIM6XECjcTN38jjv8Thc SvyQ== X-Gm-Message-State: AOAM530sfSiEf8lFbN+RMGOWjIxbyuReA+0nY+jZunaOsEhMGHMOOtPJ psVEaMCneM2tovH2TVKj3rGeVAbkMMnLkjSDfwmjoSb07CVmdgdTwQG2MzKUxDzeQ6Rs4j6enge tGvj3nUYH20b7 X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535516wrf.420.1621925317003; Mon, 24 May 2021 23:48:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwHrJfvA/haiL2kDkU2Sc824s/VhxDUz9WKfrjZUVALV+rMusOmrmXdB7xQ8MMFAx4z9xMC5w== X-Received: by 2002:a5d:58d0:: with SMTP id o16mr25535496wrf.420.1621925316796; Mon, 24 May 2021 23:48:36 -0700 (PDT) Received: from redhat.com ([2a10:8006:fcda:0:90d:c7e7:9e26:b297]) by smtp.gmail.com with ESMTPSA id g78sm1839253wme.27.2021.05.24.23.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 May 2021 23:48:34 -0700 (PDT) Date: Tue, 25 May 2021 02:48:30 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: Yongji Xie , Stefan Hajnoczi , Stefano Garzarella , Parav Pandit , Christoph Hellwig , Christian Brauner , Randy Dunlap , Matthew Wilcox , Al Viro , Jens Axboe , bcrl@kvack.org, Jonathan Corbet , Mika =?iso-8859-1?Q?Penttil=E4?= , Dan Carpenter , joro@8bytes.org, virtualization , netdev@vger.kernel.org, kvm , linux-fsdevel@vger.kernel.org, iommu@lists.linux-foundation.org, linux-kernel Subject: Re: [PATCH v7 00/12] Introduce VDUSE - vDPA Device in Userspace Message-ID: <20210525024500-mutt-send-email-mst@kernel.org> References: <20210517095513.850-1-xieyongji@bytedance.com> <20210520014349-mutt-send-email-mst@kernel.org> <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2a79fa0f-352d-b8e9-f60a-181960d054ec@redhat.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, May 25, 2021 at 02:40:57PM +0800, Jason Wang wrote: > > 在 2021/5/20 下午5:06, Yongji Xie 写道: > > On Thu, May 20, 2021 at 2:06 PM Michael S. Tsirkin wrote: > > > On Mon, May 17, 2021 at 05:55:01PM +0800, Xie Yongji wrote: > > > > This series introduces a framework, which can be used to implement > > > > vDPA Devices in a userspace program. The work consist of two parts: > > > > control path forwarding and data path offloading. > > > > > > > > In the control path, the VDUSE driver will make use of message > > > > mechnism to forward the config operation from vdpa bus driver > > > > to userspace. Userspace can use read()/write() to receive/reply > > > > those control messages. > > > > > > > > In the data path, the core is mapping dma buffer into VDUSE > > > > daemon's address space, which can be implemented in different ways > > > > depending on the vdpa bus to which the vDPA device is attached. > > > > > > > > In virtio-vdpa case, we implements a MMU-based on-chip IOMMU driver with > > > > bounce-buffering mechanism to achieve that. And in vhost-vdpa case, the dma > > > > buffer is reside in a userspace memory region which can be shared to the > > > > VDUSE userspace processs via transferring the shmfd. > > > > > > > > The details and our user case is shown below: > > > > > > > > ------------------------ ------------------------- ---------------------------------------------- > > > > | Container | | QEMU(VM) | | VDUSE daemon | > > > > | --------- | | ------------------- | | ------------------------- ---------------- | > > > > | |dev/vdx| | | |/dev/vhost-vdpa-x| | | | vDPA device emulation | | block driver | | > > > > ------------+----------- -----------+------------ -------------+----------------------+--------- > > > > | | | | > > > > | | | | > > > > ------------+---------------------------+----------------------------+----------------------+--------- > > > > | | block device | | vhost device | | vduse driver | | TCP/IP | | > > > > | -------+-------- --------+-------- -------+-------- -----+---- | > > > > | | | | | | > > > > | ----------+---------- ----------+----------- -------+------- | | > > > > | | virtio-blk driver | | vhost-vdpa driver | | vdpa device | | | > > > > | ----------+---------- ----------+----------- -------+------- | | > > > > | | virtio bus | | | | > > > > | --------+----+----------- | | | | > > > > | | | | | | > > > > | ----------+---------- | | | | > > > > | | virtio-blk device | | | | | > > > > | ----------+---------- | | | | > > > > | | | | | | > > > > | -----------+----------- | | | | > > > > | | virtio-vdpa driver | | | | | > > > > | -----------+----------- | | | | > > > > | | | | vdpa bus | | > > > > | -----------+----------------------+---------------------------+------------ | | > > > > | ---+--- | > > > > -----------------------------------------------------------------------------------------| NIC |------ > > > > ---+--- > > > > | > > > > ---------+--------- > > > > | Remote Storages | > > > > ------------------- > > > > > > > > We make use of it to implement a block device connecting to > > > > our distributed storage, which can be used both in containers and > > > > VMs. Thus, we can have an unified technology stack in this two cases. > > > > > > > > To test it with null-blk: > > > > > > > > $ qemu-storage-daemon \ > > > > --chardev socket,id=charmonitor,path=/tmp/qmp.sock,server,nowait \ > > > > --monitor chardev=charmonitor \ > > > > --blockdev driver=host_device,cache.direct=on,aio=native,filename=/dev/nullb0,node-name=disk0 \ > > > > --export type=vduse-blk,id=test,node-name=disk0,writable=on,name=vduse-null,num-queues=16,queue-size=128 > > > > > > > > The qemu-storage-daemon can be found at https://github.com/bytedance/qemu/tree/vduse > > > > > > > > To make the userspace VDUSE processes such as qemu-storage-daemon able to > > > > run unprivileged. We did some works on virtio driver to avoid trusting > > > > device, including: > > > > > > > > - validating the device status: > > > > > > > > * https://lore.kernel.org/lkml/20210517093428.670-1-xieyongji@bytedance.com/ > > > > > > > > - validating the used length: > > > > > > > > * https://lore.kernel.org/lkml/20210517090836.533-1-xieyongji@bytedance.com/ > > > > > > > > - validating the device config: > > > > > > > > * patch 4 ("virtio-blk: Add validation for block size in config space") > > > > > > > > - validating the device response: > > > > > > > > * patch 5 ("virtio_scsi: Add validation for residual bytes from response") > > > > > > > > Since I'm not sure if I missing something during auditing, especially on some > > > > virtio device drivers that I'm not familiar with, now we only support emualting > > > > a few vDPA devices by default, including: virtio-net device, virtio-blk device, > > > > virtio-scsi device and virtio-fs device. This limitation can help to reduce > > > > security risks. > > > I suspect there are a lot of assumptions even with these 4. > > > Just what are the security assumptions and guarantees here? > > > Note that VDUSE is not the only device that may suffer from this, here're > two others: > > 1) Encrypted VM Encrypted VMs are generally understood not to be fully protected from attacks by a malicious hypervisor. For example a DoS by a hypervisor is currently trivial. > 2) Smart NICs More or less the same thing. > > > The attack surface from a virtio device is limited with IOMMU enabled. > > It should be able to avoid security risk if we can validate all data > > such as config space and used length from device in device driver. > > > > > E.g. it seems pretty clear that exposing a malformed FS > > > to a random kernel config can cause untold mischief. > > > > > > Things like virtnet_send_command are also an easy way for > > > the device to DOS the kernel. > > > I think the virtnet_send_command() needs to use interrupt instead of > polling. > > Thanks > > > > > And before you try to add > > > an arbitrary timeout there - please don't, > > > the fix is moving things that must be guaranteed into kernel > > > and making things that are not guaranteed asynchronous. > > > Right now there are some things that happen with locks taken, > > > where if we don't wait for device we lose the ability to report failures > > > to userspace. E.g. all kind of netlink things are like this. > > > One can think of a bunch of ways to address this, this > > > needs to be discussed with the relevant subsystem maintainers. > > > > > > > > > If I were you I would start with one type of device, and as simple one > > > as possible. > > > > > Make sense to me. The virtio-blk device might be a good start. We > > already have some existing interface like NBD to do similar things. > > > > > > > > > When a sysadmin trusts the userspace process enough, it can relax > > > > the limitation with a 'allow_unsafe_device_emulation' module parameter. > > > That's not a great security interface. It's a global module specific knob > > > that just allows any userspace to emulate anything at all. > > > Coming up with a reasonable interface isn't going to be easy. > > > For now maybe just have people patch their kernels if they want to > > > move fast and break things. > > > > > OK. A reasonable interface can be added if we need it in the future. > > > > Thanks, > > Yongji