From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06FDCC04A6B for ; Fri, 10 May 2019 16:30:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C3A57208C3 for ; Fri, 10 May 2019 16:30:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727660AbfEJQaL (ORCPT ); Fri, 10 May 2019 12:30:11 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:35845 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbfEJQaI (ORCPT ); Fri, 10 May 2019 12:30:08 -0400 Received: by mail-qk1-f194.google.com with SMTP id c14so4047451qke.3 for ; Fri, 10 May 2019 09:30:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=QYV9CSnlMGLW/+YNa74BGVcmOFFVV0pQn9SNX0XEZO8=; b=qEKqHcb3nGaubVdSFR8wnvvug5OsLYkQrC0fd3cMs/A9DcpX1bRmpyizIcT6Hp96NF wgogZSqehTAzaGmFA7sa2Othicww5FLfm+kRAKISf9XWPLV+EO+gPprGAi55i129fd/z RsJ3fcUvv2pZEr5jkv52jHyaN8KQqIt2c2dmn6aEfIUcPuQaDccEzc0W49OxhtTgwjXL oF02fbCfyEAHySjYrAVKgtx8xTWuV3skBjElDNBm5FhLSfr+sSgVhrpMsfPQgvhSe7wA A9+Lg4pm93dvPbKHXakzy+u7mxzoh5O0vpV2psPPTMcFqbwzhoAav0GgOqkwzrqP4AOF 6aJw== X-Gm-Message-State: APjAAAXpJUMVMSQ2xQO81WeTLPGZ1y4pXboHig5cpVwMITRZDQ0OhP2R p5mGVE3SW5ipy+uoEzHUtK/HUQ== X-Google-Smtp-Source: APXvYqwqGw0tLN1kosrPHP/TgqH7/k5vTToiHq0KOvopgumJI2PV5SFzvIHY16911D9Xeql9Y6Q8tA== X-Received: by 2002:a37:4f95:: with SMTP id d143mr9608061qkb.253.1557505806901; Fri, 10 May 2019 09:30:06 -0700 (PDT) Received: from redhat.com (pool-173-76-105-71.bstnma.fios.verizon.net. [173.76.105.71]) by smtp.gmail.com with ESMTPSA id j4sm3556590qti.49.2019.05.10.09.30.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 10 May 2019 09:30:05 -0700 (PDT) Date: Fri, 10 May 2019 12:30:02 -0400 From: "Michael S. Tsirkin" To: Pankaj Gupta Cc: linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-acpi@vger.kernel.org, qemu-devel@nongnu.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, dan.j.williams@intel.com, zwisler@kernel.org, vishal.l.verma@intel.com, dave.jiang@intel.com, jasowang@redhat.com, willy@infradead.org, rjw@rjwysocki.net, hch@infradead.org, lenb@kernel.org, jack@suse.cz, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com, lcapitulino@redhat.com, kwolf@redhat.com, imammedo@redhat.com, jmoyer@redhat.com, nilal@redhat.com, riel@surriel.com, stefanha@redhat.com, aarcange@redhat.com, david@redhat.com, david@fromorbit.com, cohuck@redhat.com, xiaoguangrong.eric@gmail.com, pbonzini@redhat.com, kilobyte@angband.pl, yuval.shaia@oracle.com, jstaron@google.com Subject: Re: [PATCH v8 0/6] virtio pmem driver Message-ID: <20190510122935-mutt-send-email-mst@kernel.org> References: <20190510155202.14737-1-pagupta@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190510155202.14737-1-pagupta@redhat.com> Sender: linux-acpi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org On Fri, May 10, 2019 at 09:21:56PM +0530, Pankaj Gupta wrote: > Hi Michael & Dan, > > Please review/ack the patch series from LIBNVDIMM & VIRTIO side. Thanks! Hope to do this early next week. > We have ack on ext4, xfs patches(4, 5 & 6) patch 2. Still need > your ack on nvdimm patches(1 & 3) & virtio patch 2. > > Changes done from v7 are only in patch(2 & 3) and not > affecting existing reviews. Request to please review. > ---- > > This patch series has implementation for "virtio pmem". > "virtio pmem" is fake persistent memory(nvdimm) in guest > which allows to bypass the guest page cache. This also > implements a VIRTIO based asynchronous flush mechanism. > > Sharing guest kernel driver in this patchset with the > changes suggested in v4. Tested with Qemu side device > emulation [6] for virtio-pmem. Documented the impact of > possible page cache side channel attacks with suggested > countermeasures. > > Details of project idea for 'virtio pmem' flushing interface > is shared [3] & [4]. > > Implementation is divided into two parts: > New virtio pmem guest driver and qemu code changes for new > virtio pmem paravirtualized device. > > 1. Guest virtio-pmem kernel driver > --------------------------------- > - Reads persistent memory range from paravirt device and > registers with 'nvdimm_bus'. > - 'nvdimm/pmem' driver uses this information to allocate > persistent memory region and setup filesystem operations > to the allocated memory. > - virtio pmem driver implements asynchronous flushing > interface to flush from guest to host. > > 2. Qemu virtio-pmem device > --------------------------------- > - Creates virtio pmem device and exposes a memory range to > KVM guest. > - At host side this is file backed memory which acts as > persistent memory. > - Qemu side flush uses aio thread pool API's and virtio > for asynchronous guest multi request handling. > > David Hildenbrand CCed also posted a modified version[7] of > qemu virtio-pmem code based on updated Qemu memory device API. > > Virtio-pmem security implications and countermeasures: > ----------------------------------------------------- > > In previous posting of kernel driver, there was discussion [9] > on possible implications of page cache side channel attacks with > virtio pmem. After thorough analysis of details of known side > channel attacks, below are the suggestions: > > - Depends entirely on how host backing image file is mapped > into guest address space. > > - virtio-pmem device emulation, by default shared mapping is used > to map host backing file. It is recommended to use separate > backing file at host side for every guest. This will prevent > any possibility of executing common code from multiple guests > and any chance of inferring guest local data based based on > execution time. > > - If backing file is required to be shared among multiple guests > it is recommended to don't support host page cache eviction > commands from the guest driver. This will avoid any possibility > of inferring guest local data or host data from another guest. > > - Proposed device specification [8] for virtio-pmem device with > details of possible security implications and suggested > countermeasures for device emulation. > > Virtio-pmem errors handling: > ---------------------------------------- > Checked behaviour of virtio-pmem for below types of errors > Need suggestions on expected behaviour for handling these errors? > > - Hardware Errors: Uncorrectable recoverable Errors: > a] virtio-pmem: > - As per current logic if error page belongs to Qemu process, > host MCE handler isolates(hwpoison) that page and send SIGBUS. > Qemu SIGBUS handler injects exception to KVM guest. > - KVM guest then isolates the page and send SIGBUS to guest > userspace process which has mapped the page. > > b] Existing implementation for ACPI pmem driver: > - Handles such errors with MCE notifier and creates a list > of bad blocks. Read/direct access DAX operation return EIO > if accessed memory page fall in bad block list. > - It also starts backgound scrubbing. > - Similar functionality can be reused in virtio-pmem with MCE > notifier but without scrubbing(no ACPI/ARS)? Need inputs to > confirm if this behaviour is ok or needs any change? > > Changes from PATCH v7: [1] > - Corrected pending request queue logic (patch 2) - Jakub Staroń > - Used unsigned long flags for passing DAXDEV_F_SYNC (patch 3) - Dan > - Fixed typo => vma 'flag' to 'vm_flag' (patch 4) > - Added rob in patch 6 & patch 2 > > Changes from PATCH v6: [1] > - Corrected comment format in patch 5 & patch 6. [Dave] > - Changed variable declaration indentation in patch 6 [Darrick] > - Add Reviewed-by tag by 'Jan Kara' in patch 4 & patch 5 > > Changes from PATCH v5: [2] > Changes suggested in by - [Cornelia, Yuval] > - Remove assignment chaining in virtio driver > - Better error message and remove not required free > - Check nd_region before use > > Changes suggested by - [Jan Kara] > - dax_synchronous() for !CONFIG_DAX > - Correct 'daxdev_mapping_supported' comment and non-dax implementation > > Changes suggested by - [Dan Williams] > - Pass meaningful flag 'DAXDEV_F_SYNC' to alloc_dax > - Gate nvdimm_flush instead of additional async parameter > - Move block chaining logic to flush callback than common nvdimm_flush > - Use NULL flush callback for generic flush for better readability [Dan, Jan] > > - Use virtio device id 27 from 25(already used) - [MST] > > Changes from PATCH v4: > - Factor out MAP_SYNC supported functionality to a common helper > [Dave, Darrick, Jan] > - Comment, indentation and virtqueue_kick failure handle - Yuval Shaia > > Changes from PATCH v3: > - Use generic dax_synchronous() helper to check for DAXDEV_SYNC > flag - [Dan, Darrick, Jan] > - Add 'is_nvdimm_async' function > - Document page cache side channel attacks implications & > countermeasures - [Dave Chinner, Michael] > > Changes from PATCH v2: > - Disable MAP_SYNC for ext4 & XFS filesystems - [Dan] > - Use name 'virtio pmem' in place of 'fake dax' > > Changes from PATCH v1: > - 0-day build test for build dependency on libnvdimm > > Changes suggested by - [Dan Williams] > - Split the driver into two parts virtio & pmem > - Move queuing of async block request to block layer > - Add "sync" parameter in nvdimm_flush function > - Use indirect call for nvdimm_flush > - Don’t move declarations to common global header e.g nd.h > - nvdimm_flush() return 0 or -EIO if it fails > - Teach nsio_rw_bytes() that the flush can fail > - Rename nvdimm_flush() to generic_nvdimm_flush() > - Use 'nd_region->provider_data' for long dereferencing > - Remove virtio_pmem_freeze/restore functions > - Remove BSD license text with SPDX license text > > - Add might_sleep() in virtio_pmem_flush - [Luiz] > - Make spin_lock_irqsave() narrow > > Pankaj Gupta (6): > libnvdimm: nd_region flush callback support > virtio-pmem: Add virtio-pmem guest driver > libnvdimm: add nd_region buffered dax_dev flag > dax: check synchronous mapping is supported > ext4: disable map_sync for virtio pmem > xfs: disable map_sync for virtio pmem > > [1] https://lkml.org/lkml/2019/4/26/36 > [2] https://lkml.org/lkml/2019/4/23/1092 > [3] https://www.spinics.net/lists/kvm/msg149761.html > [4] https://www.spinics.net/lists/kvm/msg153095.html > [5] https://lkml.org/lkml/2018/8/31/413 > [6] https://marc.info/?l=linux-kernel&m=153572228719237&w=2 > [7] https://marc.info/?l=qemu-devel&m=153555721901824&w=2 > [8] https://lists.oasis-open.org/archives/virtio-dev/201903/msg00083.html > [9] https://lkml.org/lkml/2019/1/9/1191 > > drivers/acpi/nfit/core.c | 4 - > drivers/dax/bus.c | 2 > drivers/dax/super.c | 13 +++ > drivers/md/dm.c | 3 > drivers/nvdimm/Makefile | 1 > drivers/nvdimm/claim.c | 6 + > drivers/nvdimm/nd.h | 1 > drivers/nvdimm/nd_virtio.c | 129 +++++++++++++++++++++++++++++++++++++++ > drivers/nvdimm/pmem.c | 18 +++-- > drivers/nvdimm/region_devs.c | 33 +++++++++ > drivers/nvdimm/virtio_pmem.c | 117 +++++++++++++++++++++++++++++++++++ > drivers/virtio/Kconfig | 10 +++ > fs/ext4/file.c | 10 +-- > fs/xfs/xfs_file.c | 9 +- > include/linux/dax.h | 25 ++++++- > include/linux/libnvdimm.h | 9 ++ > include/linux/virtio_pmem.h | 60 ++++++++++++++++++ > include/uapi/linux/virtio_ids.h | 1 > include/uapi/linux/virtio_pmem.h | 10 +++ > 19 files changed, 436 insertions(+), 25 deletions(-) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 6B3C221268FAC for ; Fri, 10 May 2019 09:30:07 -0700 (PDT) Received: by mail-qk1-f195.google.com with SMTP id c14so4047453qke.3 for ; Fri, 10 May 2019 09:30:07 -0700 (PDT) Date: Fri, 10 May 2019 12:30:02 -0400 From: "Michael S. Tsirkin" Subject: Re: [PATCH v8 0/6] virtio pmem driver Message-ID: <20190510122935-mutt-send-email-mst@kernel.org> References: <20190510155202.14737-1-pagupta@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190510155202.14737-1-pagupta@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Pankaj Gupta Cc: cohuck@redhat.com, jack@suse.cz, kvm@vger.kernel.org, david@redhat.com, jasowang@redhat.com, david@fromorbit.com, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, adilger.kernel@dilger.ca, zwisler@kernel.org, aarcange@redhat.com, jstaron@google.com, linux-nvdimm@lists.01.org, willy@infradead.org, hch@infradead.org, linux-acpi@vger.kernel.org, linux-ext4@vger.kernel.org, lenb@kernel.org, kilobyte@angband.pl, riel@surriel.com, yuval.shaia@oracle.com, stefanha@redhat.com, pbonzini@redhat.com, lcapitulino@redhat.com, kwolf@redhat.com, nilal@redhat.com, tytso@mit.edu, xiaoguangrong.eric@gmail.com, darrick.wong@oracle.com, rjw@rjwysocki.net, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, imammedo@redhat.com List-ID: T24gRnJpLCBNYXkgMTAsIDIwMTkgYXQgMDk6MjE6NTZQTSArMDUzMCwgUGFua2FqIEd1cHRhIHdy b3RlOgo+ICBIaSBNaWNoYWVsICYgRGFuLAo+IAo+ICBQbGVhc2UgcmV2aWV3L2FjayB0aGUgcGF0 Y2ggc2VyaWVzIGZyb20gTElCTlZESU1NICYgVklSVElPIHNpZGUuCgpUaGFua3MhCkhvcGUgdG8g ZG8gdGhpcyBlYXJseSBuZXh0IHdlZWsuCgo+ICBXZSBoYXZlIGFjayBvbiBleHQ0LCB4ZnMgcGF0 Y2hlcyg0LCA1ICYgNikgcGF0Y2ggMi4gU3RpbGwgbmVlZAo+ICB5b3VyIGFjayBvbiBudmRpbW0g cGF0Y2hlcygxICYgMykgJiB2aXJ0aW8gcGF0Y2ggMi4gCj4gCj4gIENoYW5nZXMgZG9uZSBmcm9t IHY3IGFyZSBvbmx5IGluIHBhdGNoKDIgJiAzKSBhbmQgbm90Cj4gIGFmZmVjdGluZyBleGlzdGlu ZyByZXZpZXdzLiBSZXF1ZXN0IHRvIHBsZWFzZSByZXZpZXcuCj4gIC0tLS0KPiAKPiAgVGhpcyBw YXRjaCBzZXJpZXMgaGFzIGltcGxlbWVudGF0aW9uIGZvciAidmlydGlvIHBtZW0iLiAKPiAgInZp cnRpbyBwbWVtIiBpcyBmYWtlIHBlcnNpc3RlbnQgbWVtb3J5KG52ZGltbSkgaW4gZ3Vlc3QgCj4g IHdoaWNoIGFsbG93cyB0byBieXBhc3MgdGhlIGd1ZXN0IHBhZ2UgY2FjaGUuIFRoaXMgYWxzbwo+ ICBpbXBsZW1lbnRzIGEgVklSVElPIGJhc2VkIGFzeW5jaHJvbm91cyBmbHVzaCBtZWNoYW5pc20u ICAKPiAgCj4gIFNoYXJpbmcgZ3Vlc3Qga2VybmVsIGRyaXZlciBpbiB0aGlzIHBhdGNoc2V0IHdp dGggdGhlIAo+ICBjaGFuZ2VzIHN1Z2dlc3RlZCBpbiB2NC4gVGVzdGVkIHdpdGggUWVtdSBzaWRl IGRldmljZSAKPiAgZW11bGF0aW9uIFs2XSBmb3IgdmlydGlvLXBtZW0uIERvY3VtZW50ZWQgdGhl IGltcGFjdCBvZgo+ICBwb3NzaWJsZSBwYWdlIGNhY2hlIHNpZGUgY2hhbm5lbCBhdHRhY2tzIHdp dGggc3VnZ2VzdGVkCj4gIGNvdW50ZXJtZWFzdXJlcy4KPiAKPiAgRGV0YWlscyBvZiBwcm9qZWN0 IGlkZWEgZm9yICd2aXJ0aW8gcG1lbScgZmx1c2hpbmcgaW50ZXJmYWNlIAo+ICBpcyBzaGFyZWQg WzNdICYgWzRdLgo+IAo+ICBJbXBsZW1lbnRhdGlvbiBpcyBkaXZpZGVkIGludG8gdHdvIHBhcnRz Ogo+ICBOZXcgdmlydGlvIHBtZW0gZ3Vlc3QgZHJpdmVyIGFuZCBxZW11IGNvZGUgY2hhbmdlcyBm b3IgbmV3IAo+ICB2aXJ0aW8gcG1lbSBwYXJhdmlydHVhbGl6ZWQgZGV2aWNlLgo+IAo+IDEuIEd1 ZXN0IHZpcnRpby1wbWVtIGtlcm5lbCBkcml2ZXIKPiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0KPiAgICAtIFJlYWRzIHBlcnNpc3RlbnQgbWVtb3J5IHJhbmdlIGZyb20gcGFyYXZp cnQgZGV2aWNlIGFuZCAKPiAgICAgIHJlZ2lzdGVycyB3aXRoICdudmRpbW1fYnVzJy4gIAo+ICAg IC0gJ252ZGltbS9wbWVtJyBkcml2ZXIgdXNlcyB0aGlzIGluZm9ybWF0aW9uIHRvIGFsbG9jYXRl IAo+ICAgICAgcGVyc2lzdGVudCBtZW1vcnkgcmVnaW9uIGFuZCBzZXR1cCBmaWxlc3lzdGVtIG9w ZXJhdGlvbnMgCj4gICAgICB0byB0aGUgYWxsb2NhdGVkIG1lbW9yeS4gCj4gICAgLSB2aXJ0aW8g cG1lbSBkcml2ZXIgaW1wbGVtZW50cyBhc3luY2hyb25vdXMgZmx1c2hpbmcgCj4gICAgICBpbnRl cmZhY2UgdG8gZmx1c2ggZnJvbSBndWVzdCB0byBob3N0Lgo+IAo+IDIuIFFlbXUgdmlydGlvLXBt ZW0gZGV2aWNlCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gICAgLSBDcmVh dGVzIHZpcnRpbyBwbWVtIGRldmljZSBhbmQgZXhwb3NlcyBhIG1lbW9yeSByYW5nZSB0byAKPiAg ICAgIEtWTSBndWVzdC4gCj4gICAgLSBBdCBob3N0IHNpZGUgdGhpcyBpcyBmaWxlIGJhY2tlZCBt ZW1vcnkgd2hpY2ggYWN0cyBhcyAKPiAgICAgIHBlcnNpc3RlbnQgbWVtb3J5LiAKPiAgICAtIFFl bXUgc2lkZSBmbHVzaCB1c2VzIGFpbyB0aHJlYWQgcG9vbCBBUEkncyBhbmQgdmlydGlvIAo+ICAg ICAgZm9yIGFzeW5jaHJvbm91cyBndWVzdCBtdWx0aSByZXF1ZXN0IGhhbmRsaW5nLiAKPiAKPiAg ICBEYXZpZCBIaWxkZW5icmFuZCBDQ2VkIGFsc28gcG9zdGVkIGEgbW9kaWZpZWQgdmVyc2lvbls3 XSBvZiAKPiAgICBxZW11IHZpcnRpby1wbWVtIGNvZGUgYmFzZWQgb24gdXBkYXRlZCBRZW11IG1l bW9yeSBkZXZpY2UgQVBJLiAKPiAKPiAgVmlydGlvLXBtZW0gc2VjdXJpdHkgaW1wbGljYXRpb25z IGFuZCBjb3VudGVybWVhc3VyZXM6Cj4gIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gCj4gIEluIHByZXZpb3VzIHBvc3Rpbmcgb2Yga2VybmVs IGRyaXZlciwgdGhlcmUgd2FzIGRpc2N1c3Npb24gWzldCj4gIG9uIHBvc3NpYmxlIGltcGxpY2F0 aW9ucyBvZiBwYWdlIGNhY2hlIHNpZGUgY2hhbm5lbCBhdHRhY2tzIHdpdGggCj4gIHZpcnRpbyBw bWVtLiBBZnRlciB0aG9yb3VnaCBhbmFseXNpcyBvZiBkZXRhaWxzIG9mIGtub3duIHNpZGUgCj4g IGNoYW5uZWwgYXR0YWNrcywgYmVsb3cgYXJlIHRoZSBzdWdnZXN0aW9uczoKPiAKPiAgLSBEZXBl bmRzIGVudGlyZWx5IG9uIGhvdyBob3N0IGJhY2tpbmcgaW1hZ2UgZmlsZSBpcyBtYXBwZWQgCj4g ICAgaW50byBndWVzdCBhZGRyZXNzIHNwYWNlLiAKPiAKPiAgLSB2aXJ0aW8tcG1lbSBkZXZpY2Ug ZW11bGF0aW9uLCBieSBkZWZhdWx0IHNoYXJlZCBtYXBwaW5nIGlzIHVzZWQKPiAgICB0byBtYXAg aG9zdCBiYWNraW5nIGZpbGUuIEl0IGlzIHJlY29tbWVuZGVkIHRvIHVzZSBzZXBhcmF0ZQo+ICAg IGJhY2tpbmcgZmlsZSBhdCBob3N0IHNpZGUgZm9yIGV2ZXJ5IGd1ZXN0LiBUaGlzIHdpbGwgcHJl dmVudAo+ICAgIGFueSBwb3NzaWJpbGl0eSBvZiBleGVjdXRpbmcgY29tbW9uIGNvZGUgZnJvbSBt dWx0aXBsZSBndWVzdHMKPiAgICBhbmQgYW55IGNoYW5jZSBvZiBpbmZlcnJpbmcgZ3Vlc3QgbG9j YWwgZGF0YSBiYXNlZCBiYXNlZCBvbiAKPiAgICBleGVjdXRpb24gdGltZS4KPiAKPiAgLSBJZiBi YWNraW5nIGZpbGUgaXMgcmVxdWlyZWQgdG8gYmUgc2hhcmVkIGFtb25nIG11bHRpcGxlIGd1ZXN0 cyAKPiAgICBpdCBpcyByZWNvbW1lbmRlZCB0byBkb24ndCBzdXBwb3J0IGhvc3QgcGFnZSBjYWNo ZSBldmljdGlvbiAKPiAgICBjb21tYW5kcyBmcm9tIHRoZSBndWVzdCBkcml2ZXIuIFRoaXMgd2ls bCBhdm9pZCBhbnkgcG9zc2liaWxpdHkKPiAgICBvZiBpbmZlcnJpbmcgZ3Vlc3QgbG9jYWwgZGF0 YSBvciBob3N0IGRhdGEgZnJvbSBhbm90aGVyIGd1ZXN0LiAKPiAKPiAgLSBQcm9wb3NlZCBkZXZp Y2Ugc3BlY2lmaWNhdGlvbiBbOF0gZm9yIHZpcnRpby1wbWVtIGRldmljZSB3aXRoIAo+ICAgIGRl dGFpbHMgb2YgcG9zc2libGUgc2VjdXJpdHkgaW1wbGljYXRpb25zIGFuZCBzdWdnZXN0ZWQgCj4g ICAgY291bnRlcm1lYXN1cmVzIGZvciBkZXZpY2UgZW11bGF0aW9uLgo+IAo+ICBWaXJ0aW8tcG1l bSBlcnJvcnMgaGFuZGxpbmc6Cj4gIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0KPiAgIENoZWNrZWQgYmVoYXZpb3VyIG9mIHZpcnRpby1wbWVtIGZvciBiZWxvdyB0eXBl cyBvZiBlcnJvcnMKPiAgIE5lZWQgc3VnZ2VzdGlvbnMgb24gZXhwZWN0ZWQgYmVoYXZpb3VyIGZv ciBoYW5kbGluZyB0aGVzZSBlcnJvcnM/Cj4gCj4gICAtIEhhcmR3YXJlIEVycm9yczogVW5jb3Jy ZWN0YWJsZSByZWNvdmVyYWJsZSBFcnJvcnM6IAo+ICAgYV0gdmlydGlvLXBtZW06IAo+ICAgICAt IEFzIHBlciBjdXJyZW50IGxvZ2ljIGlmIGVycm9yIHBhZ2UgYmVsb25ncyB0byBRZW11IHByb2Nl c3MsIAo+ICAgICAgIGhvc3QgTUNFIGhhbmRsZXIgaXNvbGF0ZXMoaHdwb2lzb24pIHRoYXQgcGFn ZSBhbmQgc2VuZCBTSUdCVVMuIAo+ICAgICAgIFFlbXUgU0lHQlVTIGhhbmRsZXIgaW5qZWN0cyBl eGNlcHRpb24gdG8gS1ZNIGd1ZXN0LiAKPiAgICAgLSBLVk0gZ3Vlc3QgdGhlbiBpc29sYXRlcyB0 aGUgcGFnZSBhbmQgc2VuZCBTSUdCVVMgdG8gZ3Vlc3QgCj4gICAgICAgdXNlcnNwYWNlIHByb2Nl c3Mgd2hpY2ggaGFzIG1hcHBlZCB0aGUgcGFnZS4gCj4gICAKPiAgIGJdIEV4aXN0aW5nIGltcGxl bWVudGF0aW9uIGZvciBBQ1BJIHBtZW0gZHJpdmVyOiAKPiAgICAgLSBIYW5kbGVzIHN1Y2ggZXJy b3JzIHdpdGggTUNFIG5vdGlmaWVyIGFuZCBjcmVhdGVzIGEgbGlzdCAKPiAgICAgICBvZiBiYWQg YmxvY2tzLiBSZWFkL2RpcmVjdCBhY2Nlc3MgREFYIG9wZXJhdGlvbiByZXR1cm4gRUlPIAo+ICAg ICAgIGlmIGFjY2Vzc2VkIG1lbW9yeSBwYWdlIGZhbGwgaW4gYmFkIGJsb2NrIGxpc3QuCj4gICAg IC0gSXQgYWxzbyBzdGFydHMgYmFja2dvdW5kIHNjcnViYmluZy4gIAo+ICAgICAtIFNpbWlsYXIg ZnVuY3Rpb25hbGl0eSBjYW4gYmUgcmV1c2VkIGluIHZpcnRpby1wbWVtIHdpdGggTUNFIAo+ICAg ICAgIG5vdGlmaWVyIGJ1dCB3aXRob3V0IHNjcnViYmluZyhubyBBQ1BJL0FSUyk/IE5lZWQgaW5w dXRzIHRvIAo+ICAgICAgIGNvbmZpcm0gaWYgdGhpcyBiZWhhdmlvdXIgaXMgb2sgb3IgbmVlZHMg YW55IGNoYW5nZT8KPiAKPiBDaGFuZ2VzIGZyb20gUEFUQ0ggdjc6IFsxXQo+ICAtIENvcnJlY3Rl ZCBwZW5kaW5nIHJlcXVlc3QgcXVldWUgbG9naWMgKHBhdGNoIDIpIC0gSmFrdWIgU3Rhcm/FhAo+ ICAtIFVzZWQgdW5zaWduZWQgbG9uZyBmbGFncyBmb3IgcGFzc2luZyBEQVhERVZfRl9TWU5DIChw YXRjaCAzKSAtIERhbgo+ICAtIEZpeGVkIHR5cG8gPT4gIHZtYSAnZmxhZycgdG8gJ3ZtX2ZsYWcn IChwYXRjaCA0KQo+ICAtIEFkZGVkIHJvYiBpbiBwYXRjaCA2ICYgcGF0Y2ggMgo+IAo+IENoYW5n ZXMgZnJvbSBQQVRDSCB2NjogWzFdCj4gIC0gQ29ycmVjdGVkIGNvbW1lbnQgZm9ybWF0IGluIHBh dGNoIDUgJiBwYXRjaCA2LiBbRGF2ZV0KPiAgLSBDaGFuZ2VkIHZhcmlhYmxlIGRlY2xhcmF0aW9u IGluZGVudGF0aW9uIGluIHBhdGNoIDYgW0RhcnJpY2tdCj4gIC0gQWRkIFJldmlld2VkLWJ5IHRh ZyBieSAnSmFuIEthcmEnIGluIHBhdGNoIDQgJiBwYXRjaCA1Cj4gCj4gQ2hhbmdlcyBmcm9tIFBB VENIIHY1OiBbMl0KPiAgIENoYW5nZXMgc3VnZ2VzdGVkIGluIGJ5IC0gW0Nvcm5lbGlhLCBZdXZh bF0KPiAtIFJlbW92ZSBhc3NpZ25tZW50IGNoYWluaW5nIGluIHZpcnRpbyBkcml2ZXIKPiAtIEJl dHRlciBlcnJvciBtZXNzYWdlIGFuZCByZW1vdmUgbm90IHJlcXVpcmVkIGZyZWUKPiAtIENoZWNr IG5kX3JlZ2lvbiBiZWZvcmUgdXNlCj4gCj4gICBDaGFuZ2VzIHN1Z2dlc3RlZCBieSAtIFtKYW4g S2FyYV0KPiAtIGRheF9zeW5jaHJvbm91cygpIGZvciAhQ09ORklHX0RBWAo+IC0gQ29ycmVjdCAn ZGF4ZGV2X21hcHBpbmdfc3VwcG9ydGVkJyBjb21tZW50IGFuZCBub24tZGF4IGltcGxlbWVudGF0 aW9uCj4gCj4gICBDaGFuZ2VzIHN1Z2dlc3RlZCBieSAtIFtEYW4gV2lsbGlhbXNdCj4gLSBQYXNz IG1lYW5pbmdmdWwgZmxhZyAnREFYREVWX0ZfU1lOQycgdG8gYWxsb2NfZGF4Cj4gLSBHYXRlIG52 ZGltbV9mbHVzaCBpbnN0ZWFkIG9mIGFkZGl0aW9uYWwgYXN5bmMgcGFyYW1ldGVyCj4gLSBNb3Zl IGJsb2NrIGNoYWluaW5nIGxvZ2ljIHRvIGZsdXNoIGNhbGxiYWNrIHRoYW4gY29tbW9uIG52ZGlt bV9mbHVzaAo+IC0gVXNlIE5VTEwgZmx1c2ggY2FsbGJhY2sgZm9yIGdlbmVyaWMgZmx1c2ggZm9y IGJldHRlciByZWFkYWJpbGl0eSBbRGFuLCBKYW5dCj4gCj4gLSBVc2UgdmlydGlvIGRldmljZSBp ZCAyNyBmcm9tIDI1KGFscmVhZHkgdXNlZCkgLSBbTVNUXQo+IAo+IENoYW5nZXMgZnJvbSBQQVRD SCB2NDoKPiAtIEZhY3RvciBvdXQgTUFQX1NZTkMgc3VwcG9ydGVkIGZ1bmN0aW9uYWxpdHkgdG8g YSBjb21tb24gaGVscGVyCj4gCQkJCVtEYXZlLCBEYXJyaWNrLCBKYW5dCj4gLSBDb21tZW50LCBp bmRlbnRhdGlvbiBhbmQgdmlydHF1ZXVlX2tpY2sgZmFpbHVyZSBoYW5kbGUgLSBZdXZhbCBTaGFp YQo+IAo+IENoYW5nZXMgZnJvbSBQQVRDSCB2MzogCj4gLSBVc2UgZ2VuZXJpYyBkYXhfc3luY2hy b25vdXMoKSBoZWxwZXIgdG8gY2hlY2sgZm9yIERBWERFVl9TWU5DIAo+ICAgZmxhZyAtIFtEYW4s IERhcnJpY2ssIEphbl0KPiAtIEFkZCAnaXNfbnZkaW1tX2FzeW5jJyBmdW5jdGlvbgo+IC0gRG9j dW1lbnQgcGFnZSBjYWNoZSBzaWRlIGNoYW5uZWwgYXR0YWNrcyBpbXBsaWNhdGlvbnMgJiAKPiAg IGNvdW50ZXJtZWFzdXJlcyAtIFtEYXZlIENoaW5uZXIsIE1pY2hhZWxdCj4gCj4gQ2hhbmdlcyBm cm9tIFBBVENIIHYyOiAKPiAtIERpc2FibGUgTUFQX1NZTkMgZm9yIGV4dDQgJiBYRlMgZmlsZXN5 c3RlbXMgLSBbRGFuXSAKPiAtIFVzZSBuYW1lICd2aXJ0aW8gcG1lbScgaW4gcGxhY2Ugb2YgJ2Zh a2UgZGF4JyAKPiAKPiBDaGFuZ2VzIGZyb20gUEFUQ0ggdjE6IAo+IC0gMC1kYXkgYnVpbGQgdGVz dCBmb3IgYnVpbGQgZGVwZW5kZW5jeSBvbiBsaWJudmRpbW0gCj4gCj4gIENoYW5nZXMgc3VnZ2Vz dGVkIGJ5IC0gW0RhbiBXaWxsaWFtc10KPiAtIFNwbGl0IHRoZSBkcml2ZXIgaW50byB0d28gcGFy dHMgdmlydGlvICYgcG1lbSAgCj4gLSBNb3ZlIHF1ZXVpbmcgb2YgYXN5bmMgYmxvY2sgcmVxdWVz dCB0byBibG9jayBsYXllcgo+IC0gQWRkICJzeW5jIiBwYXJhbWV0ZXIgaW4gbnZkaW1tX2ZsdXNo IGZ1bmN0aW9uCj4gLSBVc2UgaW5kaXJlY3QgY2FsbCBmb3IgbnZkaW1tX2ZsdXNoCj4gLSBEb27i gJl0IG1vdmUgZGVjbGFyYXRpb25zIHRvIGNvbW1vbiBnbG9iYWwgaGVhZGVyIGUuZyBuZC5oCj4g LSBudmRpbW1fZmx1c2goKSByZXR1cm4gMCBvciAtRUlPIGlmIGl0IGZhaWxzCj4gLSBUZWFjaCBu c2lvX3J3X2J5dGVzKCkgdGhhdCB0aGUgZmx1c2ggY2FuIGZhaWwKPiAtIFJlbmFtZSBudmRpbW1f Zmx1c2goKSB0byBnZW5lcmljX252ZGltbV9mbHVzaCgpCj4gLSBVc2UgJ25kX3JlZ2lvbi0+cHJv dmlkZXJfZGF0YScgZm9yIGxvbmcgZGVyZWZlcmVuY2luZwo+IC0gUmVtb3ZlIHZpcnRpb19wbWVt X2ZyZWV6ZS9yZXN0b3JlIGZ1bmN0aW9ucwo+IC0gUmVtb3ZlIEJTRCBsaWNlbnNlIHRleHQgd2l0 aCBTUERYIGxpY2Vuc2UgdGV4dAo+IAo+IC0gQWRkIG1pZ2h0X3NsZWVwKCkgaW4gdmlydGlvX3Bt ZW1fZmx1c2ggLSBbTHVpel0KPiAtIE1ha2Ugc3Bpbl9sb2NrX2lycXNhdmUoKSBuYXJyb3cKPiAK PiBQYW5rYWogR3VwdGEgKDYpOgo+ICAgIGxpYm52ZGltbTogbmRfcmVnaW9uIGZsdXNoIGNhbGxi YWNrIHN1cHBvcnQKPiAgICB2aXJ0aW8tcG1lbTogQWRkIHZpcnRpby1wbWVtIGd1ZXN0IGRyaXZl cgo+ICAgIGxpYm52ZGltbTogYWRkIG5kX3JlZ2lvbiBidWZmZXJlZCBkYXhfZGV2IGZsYWcKPiAg ICBkYXg6IGNoZWNrIHN5bmNocm9ub3VzIG1hcHBpbmcgaXMgc3VwcG9ydGVkCj4gICAgZXh0NDog ZGlzYWJsZSBtYXBfc3luYyBmb3IgdmlydGlvIHBtZW0KPiAgICB4ZnM6IGRpc2FibGUgbWFwX3N5 bmMgZm9yIHZpcnRpbyBwbWVtCj4gCj4gWzFdIGh0dHBzOi8vbGttbC5vcmcvbGttbC8yMDE5LzQv MjYvMzYKPiBbMl0gaHR0cHM6Ly9sa21sLm9yZy9sa21sLzIwMTkvNC8yMy8xMDkyCj4gWzNdIGh0 dHBzOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2t2bS9tc2cxNDk3NjEuaHRtbAo+IFs0XSBodHRw czovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9rdm0vbXNnMTUzMDk1Lmh0bWwgIAo+IFs1XSBodHRw czovL2xrbWwub3JnL2xrbWwvMjAxOC84LzMxLzQxMwo+IFs2XSBodHRwczovL21hcmMuaW5mby8/ bD1saW51eC1rZXJuZWwmbT0xNTM1NzIyMjg3MTkyMzcmdz0yIAo+IFs3XSBodHRwczovL21hcmMu aW5mby8/bD1xZW11LWRldmVsJm09MTUzNTU1NzIxOTAxODI0Jnc9Mgo+IFs4XSBodHRwczovL2xp c3RzLm9hc2lzLW9wZW4ub3JnL2FyY2hpdmVzL3ZpcnRpby1kZXYvMjAxOTAzL21zZzAwMDgzLmh0 bWwKPiBbOV0gaHR0cHM6Ly9sa21sLm9yZy9sa21sLzIwMTkvMS85LzExOTEKPiAKPiAgZHJpdmVy cy9hY3BpL25maXQvY29yZS5jICAgICAgICAgfCAgICA0IC0KPiAgZHJpdmVycy9kYXgvYnVzLmMg ICAgICAgICAgICAgICAgfCAgICAyIAo+ICBkcml2ZXJzL2RheC9zdXBlci5jICAgICAgICAgICAg ICB8ICAgMTMgKysrCj4gIGRyaXZlcnMvbWQvZG0uYyAgICAgICAgICAgICAgICAgIHwgICAgMyAK PiAgZHJpdmVycy9udmRpbW0vTWFrZWZpbGUgICAgICAgICAgfCAgICAxIAo+ICBkcml2ZXJzL252 ZGltbS9jbGFpbS5jICAgICAgICAgICB8ICAgIDYgKwo+ICBkcml2ZXJzL252ZGltbS9uZC5oICAg ICAgICAgICAgICB8ICAgIDEgCj4gIGRyaXZlcnMvbnZkaW1tL25kX3ZpcnRpby5jICAgICAgIHwg IDEyOSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysKPiAgZHJpdmVycy9u dmRpbW0vcG1lbS5jICAgICAgICAgICAgfCAgIDE4ICsrKy0tCj4gIGRyaXZlcnMvbnZkaW1tL3Jl Z2lvbl9kZXZzLmMgICAgIHwgICAzMyArKysrKysrKysKPiAgZHJpdmVycy9udmRpbW0vdmlydGlv X3BtZW0uYyAgICAgfCAgMTE3ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrCj4g IGRyaXZlcnMvdmlydGlvL0tjb25maWcgICAgICAgICAgIHwgICAxMCArKysKPiAgZnMvZXh0NC9m aWxlLmMgICAgICAgICAgICAgICAgICAgfCAgIDEwICstLQo+ICBmcy94ZnMveGZzX2ZpbGUuYyAg ICAgICAgICAgICAgICB8ICAgIDkgKy0KPiAgaW5jbHVkZS9saW51eC9kYXguaCAgICAgICAgICAg ICAgfCAgIDI1ICsrKysrKy0KPiAgaW5jbHVkZS9saW51eC9saWJudmRpbW0uaCAgICAgICAgfCAg ICA5ICsrCj4gIGluY2x1ZGUvbGludXgvdmlydGlvX3BtZW0uaCAgICAgIHwgICA2MCArKysrKysr KysrKysrKysrKysKPiAgaW5jbHVkZS91YXBpL2xpbnV4L3ZpcnRpb19pZHMuaCAgfCAgICAxIAo+ ICBpbmNsdWRlL3VhcGkvbGludXgvdmlydGlvX3BtZW0uaCB8ICAgMTAgKysrCj4gIDE5IGZpbGVz IGNoYW5nZWQsIDQzNiBpbnNlcnRpb25zKCspLCAyNSBkZWxldGlvbnMoLSkKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXgtbnZkaW1tIG1haWxpbmcg bGlzdApMaW51eC1udmRpbW1AbGlzdHMuMDEub3JnCmh0dHBzOi8vbGlzdHMuMDEub3JnL21haWxt YW4vbGlzdGluZm8vbGludXgtbnZkaW1tCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6CDCC04A6B for ; Fri, 10 May 2019 16:33:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9A46A21479 for ; Fri, 10 May 2019 16:33:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A46A21479 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([127.0.0.1]:46388 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hP8SC-0003bl-SQ for qemu-devel@archiver.kernel.org; Fri, 10 May 2019 12:33:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43710) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hP8Pi-0001tR-LF for qemu-devel@nongnu.org; Fri, 10 May 2019 12:30:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hP8Pg-0004Mn-NL for qemu-devel@nongnu.org; Fri, 10 May 2019 12:30:26 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:33617) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hP8Pg-00041E-Go for qemu-devel@nongnu.org; Fri, 10 May 2019 12:30:24 -0400 Received: by mail-qk1-f195.google.com with SMTP id k189so4062472qkc.0 for ; Fri, 10 May 2019 09:30:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=QYV9CSnlMGLW/+YNa74BGVcmOFFVV0pQn9SNX0XEZO8=; b=Wt6ewA9vqn2w9LMLt26ILwvXvwxyhJ5YYGA1gpS/6Jwvx+d9J3UU3bLA0V4buuHjAd l+AMCAfeW/lW+mum/skc5wEfOURzGePJEFOIDJEJBr16pJTwgV5eeO+jB3Hkl1jK00SS g2OP0CxkcbTmZ6MOVd9Ss/rvKDXpZZ03JfMkoODL8sUhbOlZsRJVTv8P2eK58Hx3EU4b k3iT3f0pKZB9o73d4qPgY76Q1sboywyo/93QdAUsTwNdEVI24bQufk6evVV8ZjvhVKe2 56piVIlU4pvukKATslkdcL4YSwVxAiKsCGtkDFKg9OVtEXaaeqAVUnNYh/ztl6spibky fqqw== X-Gm-Message-State: APjAAAV3j8G/xlPaJ2KHo4KDPpvByB6tPpnRmbYtcSg1OY5E8yCPKFHq /TSDDegCxtagnDl+5fMqIbtFyA== X-Google-Smtp-Source: APXvYqwqGw0tLN1kosrPHP/TgqH7/k5vTToiHq0KOvopgumJI2PV5SFzvIHY16911D9Xeql9Y6Q8tA== X-Received: by 2002:a37:4f95:: with SMTP id d143mr9608061qkb.253.1557505806901; Fri, 10 May 2019 09:30:06 -0700 (PDT) Received: from redhat.com (pool-173-76-105-71.bstnma.fios.verizon.net. [173.76.105.71]) by smtp.gmail.com with ESMTPSA id j4sm3556590qti.49.2019.05.10.09.30.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 10 May 2019 09:30:05 -0700 (PDT) Date: Fri, 10 May 2019 12:30:02 -0400 From: "Michael S. Tsirkin" To: Pankaj Gupta Message-ID: <20190510122935-mutt-send-email-mst@kernel.org> References: <20190510155202.14737-1-pagupta@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190510155202.14737-1-pagupta@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.222.195 Subject: Re: [Qemu-devel] [PATCH v8 0/6] virtio pmem driver X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cohuck@redhat.com, jack@suse.cz, kvm@vger.kernel.org, david@redhat.com, jasowang@redhat.com, david@fromorbit.com, qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, adilger.kernel@dilger.ca, zwisler@kernel.org, aarcange@redhat.com, dave.jiang@intel.com, jstaron@google.com, linux-nvdimm@lists.01.org, vishal.l.verma@intel.com, willy@infradead.org, hch@infradead.org, linux-acpi@vger.kernel.org, jmoyer@redhat.com, linux-ext4@vger.kernel.org, lenb@kernel.org, kilobyte@angband.pl, riel@surriel.com, yuval.shaia@oracle.com, stefanha@redhat.com, pbonzini@redhat.com, dan.j.williams@intel.com, lcapitulino@redhat.com, kwolf@redhat.com, nilal@redhat.com, tytso@mit.edu, xiaoguangrong.eric@gmail.com, darrick.wong@oracle.com, rjw@rjwysocki.net, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, imammedo@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Fri, May 10, 2019 at 09:21:56PM +0530, Pankaj Gupta wrote: > Hi Michael & Dan, > > Please review/ack the patch series from LIBNVDIMM & VIRTIO side. Thanks! Hope to do this early next week. > We have ack on ext4, xfs patches(4, 5 & 6) patch 2. Still need > your ack on nvdimm patches(1 & 3) & virtio patch 2. > > Changes done from v7 are only in patch(2 & 3) and not > affecting existing reviews. Request to please review. > ---- > > This patch series has implementation for "virtio pmem". > "virtio pmem" is fake persistent memory(nvdimm) in guest > which allows to bypass the guest page cache. This also > implements a VIRTIO based asynchronous flush mechanism. > > Sharing guest kernel driver in this patchset with the > changes suggested in v4. Tested with Qemu side device > emulation [6] for virtio-pmem. Documented the impact of > possible page cache side channel attacks with suggested > countermeasures. > > Details of project idea for 'virtio pmem' flushing interface > is shared [3] & [4]. > > Implementation is divided into two parts: > New virtio pmem guest driver and qemu code changes for new > virtio pmem paravirtualized device. > > 1. Guest virtio-pmem kernel driver > --------------------------------- > - Reads persistent memory range from paravirt device and > registers with 'nvdimm_bus'. > - 'nvdimm/pmem' driver uses this information to allocate > persistent memory region and setup filesystem operations > to the allocated memory. > - virtio pmem driver implements asynchronous flushing > interface to flush from guest to host. > > 2. Qemu virtio-pmem device > --------------------------------- > - Creates virtio pmem device and exposes a memory range to > KVM guest. > - At host side this is file backed memory which acts as > persistent memory. > - Qemu side flush uses aio thread pool API's and virtio > for asynchronous guest multi request handling. > > David Hildenbrand CCed also posted a modified version[7] of > qemu virtio-pmem code based on updated Qemu memory device API. > > Virtio-pmem security implications and countermeasures: > ----------------------------------------------------- > > In previous posting of kernel driver, there was discussion [9] > on possible implications of page cache side channel attacks with > virtio pmem. After thorough analysis of details of known side > channel attacks, below are the suggestions: > > - Depends entirely on how host backing image file is mapped > into guest address space. > > - virtio-pmem device emulation, by default shared mapping is used > to map host backing file. It is recommended to use separate > backing file at host side for every guest. This will prevent > any possibility of executing common code from multiple guests > and any chance of inferring guest local data based based on > execution time. > > - If backing file is required to be shared among multiple guests > it is recommended to don't support host page cache eviction > commands from the guest driver. This will avoid any possibility > of inferring guest local data or host data from another guest. > > - Proposed device specification [8] for virtio-pmem device with > details of possible security implications and suggested > countermeasures for device emulation. > > Virtio-pmem errors handling: > ---------------------------------------- > Checked behaviour of virtio-pmem for below types of errors > Need suggestions on expected behaviour for handling these errors? > > - Hardware Errors: Uncorrectable recoverable Errors: > a] virtio-pmem: > - As per current logic if error page belongs to Qemu process, > host MCE handler isolates(hwpoison) that page and send SIGBUS. > Qemu SIGBUS handler injects exception to KVM guest. > - KVM guest then isolates the page and send SIGBUS to guest > userspace process which has mapped the page. > > b] Existing implementation for ACPI pmem driver: > - Handles such errors with MCE notifier and creates a list > of bad blocks. Read/direct access DAX operation return EIO > if accessed memory page fall in bad block list. > - It also starts backgound scrubbing. > - Similar functionality can be reused in virtio-pmem with MCE > notifier but without scrubbing(no ACPI/ARS)? Need inputs to > confirm if this behaviour is ok or needs any change? > > Changes from PATCH v7: [1] > - Corrected pending request queue logic (patch 2) - Jakub Staroń > - Used unsigned long flags for passing DAXDEV_F_SYNC (patch 3) - Dan > - Fixed typo => vma 'flag' to 'vm_flag' (patch 4) > - Added rob in patch 6 & patch 2 > > Changes from PATCH v6: [1] > - Corrected comment format in patch 5 & patch 6. [Dave] > - Changed variable declaration indentation in patch 6 [Darrick] > - Add Reviewed-by tag by 'Jan Kara' in patch 4 & patch 5 > > Changes from PATCH v5: [2] > Changes suggested in by - [Cornelia, Yuval] > - Remove assignment chaining in virtio driver > - Better error message and remove not required free > - Check nd_region before use > > Changes suggested by - [Jan Kara] > - dax_synchronous() for !CONFIG_DAX > - Correct 'daxdev_mapping_supported' comment and non-dax implementation > > Changes suggested by - [Dan Williams] > - Pass meaningful flag 'DAXDEV_F_SYNC' to alloc_dax > - Gate nvdimm_flush instead of additional async parameter > - Move block chaining logic to flush callback than common nvdimm_flush > - Use NULL flush callback for generic flush for better readability [Dan, Jan] > > - Use virtio device id 27 from 25(already used) - [MST] > > Changes from PATCH v4: > - Factor out MAP_SYNC supported functionality to a common helper > [Dave, Darrick, Jan] > - Comment, indentation and virtqueue_kick failure handle - Yuval Shaia > > Changes from PATCH v3: > - Use generic dax_synchronous() helper to check for DAXDEV_SYNC > flag - [Dan, Darrick, Jan] > - Add 'is_nvdimm_async' function > - Document page cache side channel attacks implications & > countermeasures - [Dave Chinner, Michael] > > Changes from PATCH v2: > - Disable MAP_SYNC for ext4 & XFS filesystems - [Dan] > - Use name 'virtio pmem' in place of 'fake dax' > > Changes from PATCH v1: > - 0-day build test for build dependency on libnvdimm > > Changes suggested by - [Dan Williams] > - Split the driver into two parts virtio & pmem > - Move queuing of async block request to block layer > - Add "sync" parameter in nvdimm_flush function > - Use indirect call for nvdimm_flush > - Don’t move declarations to common global header e.g nd.h > - nvdimm_flush() return 0 or -EIO if it fails > - Teach nsio_rw_bytes() that the flush can fail > - Rename nvdimm_flush() to generic_nvdimm_flush() > - Use 'nd_region->provider_data' for long dereferencing > - Remove virtio_pmem_freeze/restore functions > - Remove BSD license text with SPDX license text > > - Add might_sleep() in virtio_pmem_flush - [Luiz] > - Make spin_lock_irqsave() narrow > > Pankaj Gupta (6): > libnvdimm: nd_region flush callback support > virtio-pmem: Add virtio-pmem guest driver > libnvdimm: add nd_region buffered dax_dev flag > dax: check synchronous mapping is supported > ext4: disable map_sync for virtio pmem > xfs: disable map_sync for virtio pmem > > [1] https://lkml.org/lkml/2019/4/26/36 > [2] https://lkml.org/lkml/2019/4/23/1092 > [3] https://www.spinics.net/lists/kvm/msg149761.html > [4] https://www.spinics.net/lists/kvm/msg153095.html > [5] https://lkml.org/lkml/2018/8/31/413 > [6] https://marc.info/?l=linux-kernel&m=153572228719237&w=2 > [7] https://marc.info/?l=qemu-devel&m=153555721901824&w=2 > [8] https://lists.oasis-open.org/archives/virtio-dev/201903/msg00083.html > [9] https://lkml.org/lkml/2019/1/9/1191 > > drivers/acpi/nfit/core.c | 4 - > drivers/dax/bus.c | 2 > drivers/dax/super.c | 13 +++ > drivers/md/dm.c | 3 > drivers/nvdimm/Makefile | 1 > drivers/nvdimm/claim.c | 6 + > drivers/nvdimm/nd.h | 1 > drivers/nvdimm/nd_virtio.c | 129 +++++++++++++++++++++++++++++++++++++++ > drivers/nvdimm/pmem.c | 18 +++-- > drivers/nvdimm/region_devs.c | 33 +++++++++ > drivers/nvdimm/virtio_pmem.c | 117 +++++++++++++++++++++++++++++++++++ > drivers/virtio/Kconfig | 10 +++ > fs/ext4/file.c | 10 +-- > fs/xfs/xfs_file.c | 9 +- > include/linux/dax.h | 25 ++++++- > include/linux/libnvdimm.h | 9 ++ > include/linux/virtio_pmem.h | 60 ++++++++++++++++++ > include/uapi/linux/virtio_ids.h | 1 > include/uapi/linux/virtio_pmem.h | 10 +++ > 19 files changed, 436 insertions(+), 25 deletions(-)