From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 7DDFB20945C0A for ; Mon, 13 Aug 2018 07:29:11 -0700 (PDT) Date: Mon, 13 Aug 2018 10:29:06 -0400 From: Jerome Glisse Subject: Re: [PATCH V3 3/4] mm: add a function to differentiate the pages is from DAX device memory Message-ID: <20180813142906.GA3451@redhat.com> References: <2b7856596e519130946c834d5d61b00b7f592770.1533811181.git.yi.z.zhang@linux.intel.com> <872818364.892078.1533806608252.JavaMail.zimbra@redhat.com> <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: "Zhang,Yi" Cc: jack@suse.cz, yu c zhang , kvm@vger.kernel.org, linux-nvdimm@lists.01.org, rkrcmar@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, pbonzini@redhat.com, hch@lst.de, yi z zhang List-ID: T24gVHVlLCBBdWcgMTQsIDIwMTggYXQgMDE6NDE6NDBBTSArMDgwMCwgWmhhbmcsWWkgd3JvdGU6 Cj4gCj4gCj4gT24gMjAxOOW5tDA45pyIMDnml6UgMTc6MjMsIFBhbmthaiBHdXB0YSB3cm90ZToK PiA+PiBEQVggZHJpdmVyIGhvdHBsdWcgdGhlIGRldmljZSBtZW1vcnkgYW5kIG1vdmUgaXQgdG8g bWVtb3J5IHpvbmUsIHRoZXNlCj4gPj4gcGFnZXMgd2lsbCBiZSBtYXJrZWQgcmVzZXJ2ZWQgZmxh ZywgaG93ZXZlciwgc29tZSBvdGhlciBrZXJuZWwgY29tcG9uZXQKPiA+PiB3aWxsIG1pc2NvbmNl aXZlIHRoZXNlIHBhZ2VzIGFyZSByZXNlcnZlZCBtbWlvIChleDogd2UgbWFwIHRoZXNlIGRldl9k YXgKPiA+PiBvciBmc19kYXggcGFnZXMgdG8ga3ZtIGZvciBESU1NL05WRElNTSBiYWNrZW5kKS4g VG9nZXRoZXIgd2l0aCB0aGUgdHlwZQo+ID4+IE1FTU9SWV9ERVZJQ0VfRlNfREFYLCB3ZSBjYW4g dXNlIGlzX2RheF9wYWdlKCkgdG8gZGlmZmVyZW50aWF0ZSB0aGUgcGFnZXMKPiA+PiBpcyBEQVgg ZGV2aWNlIG1lbW9yeSBvciBub3QuCj4gPj4KPiA+PiBTaWduZWQtb2ZmLWJ5OiBaaGFuZyBZaSA8 eWkuei56aGFuZ0BsaW51eC5pbnRlbC5jb20+Cj4gPj4gU2lnbmVkLW9mZi1ieTogWmhhbmcgWXUg PHl1LmMuemhhbmdAbGludXguaW50ZWwuY29tPgo+ID4+IC0tLQo+ID4+ICBpbmNsdWRlL2xpbnV4 L21tLmggfCAxMiArKysrKysrKysrKysKPiA+PiAgMSBmaWxlIGNoYW5nZWQsIDEyIGluc2VydGlv bnMoKykKPiA+Pgo+ID4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRl L2xpbnV4L21tLmgKPiA+PiBpbmRleCA2OGE1MTIxLi5kZTVjYmMzIDEwMDY0NAo+ID4+IC0tLSBh L2luY2x1ZGUvbGludXgvbW0uaAo+ID4+ICsrKyBiL2luY2x1ZGUvbGludXgvbW0uaAo+ID4+IEBA IC04ODksNiArODg5LDEzIEBAIHN0YXRpYyBpbmxpbmUgYm9vbCBpc19kZXZpY2VfcHVibGljX3Bh Z2UoY29uc3Qgc3RydWN0Cj4gPj4gcGFnZSAqcGFnZSkKPiA+PiAgCQlwYWdlLT5wZ21hcC0+dHlw ZSA9PSBNRU1PUllfREVWSUNFX1BVQkxJQzsKPiA+PiAgfQo+ID4+ICAKPiA+PiArc3RhdGljIGlu bGluZSBib29sIGlzX2RheF9wYWdlKGNvbnN0IHN0cnVjdCBwYWdlICpwYWdlKQo+ID4+ICt7Cj4g Pj4gKwlyZXR1cm4gaXNfem9uZV9kZXZpY2VfcGFnZShwYWdlKSAmJgo+ID4+ICsJCShwYWdlLT5w Z21hcC0+dHlwZSA9PSBNRU1PUllfREVWSUNFX0ZTX0RBWCB8fAo+ID4+ICsJCXBhZ2UtPnBnbWFw LT50eXBlID09IE1FTU9SWV9ERVZJQ0VfREVWX0RBWCk7Cj4gPj4gK30KPiA+IEkgdGhpbmsgcXVl c3Rpb24gZnJvbSBEYW4gZm9yIEtWTSBWTSB3aXRoICdNRU1PUllfREVWSUNFX1BVQkxJQycgc3Rp bGwgaG9sZHM/Cj4gPiBJIGFtIGFsc28gaW50ZXJlc3RlZCB0byBrbm93IGlmIHRoZXJlIGlzIGFu eSB1c2UtY2FzZS4KPiA+Cj4gPiBUaGFua3MsCj4gPiBQYW5rYWoKPiBZZXMsIGl0IGlzLCB0aGFu a3MgZm9yIHlvdXIgcmVtaW5kLCBQYW5rYWouCj4gQWRkaW5nIEplcm9tZSBmb3IgRGFuJ3MgcXVl c3Rpb25zIG9uIFYxOgo+IFtEYW5dOgo+IAo+IEplcm9tZSwgbWlnaHQgdGhlcmUgYmUgYW55IHVz ZSBjYXNlIHRvIHBhc3MgTUVNT1JZX0RFVklDRV9QVUJMSUMKPiBtZW1vcnkgdG8gYSBndWVzdCB2 bT8KClllcyBhbmQgbm8sIGkgYW0gbm90IHN1cmUgaG93IHdlIGFyZSBnb2luZyB0byBkbyBpdC4g QnV0IGJlaW5nIGFibGUgdG8Kc2hhcmUgR1BVIGFtb25nIG11bHRpcGxlIFZNIGlzIG9uIFRPRE8g bGlzdCBhbmQgdGhvc2UgR1BVIHdpbGwgaGF2ZQpNRU1PUllfREVWSUNFX1BVQkxJQ3xQUklWQVRF IGRlcGVuZGluZyBvbiB0aGUgcGxhdGZvcm0uIFNvIGVpdGhlciB3ZQpwYXNzIGRvd24gdGhlIHJl YWwgdW5kZXJseWluZyByZXNvdXJjZSB0byB0aGUgZ3Vlc3QsIG9yIHdlIHdpbGwgcGFzcwpkb3du IGEgZmFrZSBvbmUgYW5kIGhhdmUgZ3Vlc3QgYW5kIGhvc3QgZHJpdmVyIHRhbGsgdG8gZWFjaCBv dGhlciBzbwp0aGF0IHRoZSBob3N0IGRyaXZlciBjYW4gZG8gb3ZlcmFsbCByZXNvdXJjZSBtYW5h Z2VtZW50IGFjY3Jvc3MgbXVsdGlwbGUKZ3Vlc3RzLgoKU28gaSB3b3VsZCBzYXkgdGhhdCBmb3Ig bm93IHlvdSBjYW4gaWdub3JlIE1FTU9SWV9ERVZJQ0VfUFVCTElDIGFuZCB3aGVuCndlIGdldCB0 byB0aGUgS1ZNIGd1ZXN0IHNoYXJpbmcgb2YgdGhvc2UgYW5kIGRlY2lkZSBob3cgd2Ugd2FudCB0 byBkbwppdCB0aGVuIHdlIGNhbiB1cGRhdGUga3ZtIHRvIHByb3Blcmx5IGludGVycHJldCB0aG9z ZS4KCkNoZWVycywKSsOpcsO0bWUKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KTGludXgtbnZkaW1tIG1haWxpbmcgbGlzdApMaW51eC1udmRpbW1AbGlzdHMu MDEub3JnCmh0dHBzOi8vbGlzdHMuMDEub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbnZkaW1t Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [PATCH V3 3/4] mm: add a function to differentiate the pages is from DAX device memory Date: Mon, 13 Aug 2018 10:29:06 -0400 Message-ID: <20180813142906.GA3451@redhat.com> References: <2b7856596e519130946c834d5d61b00b7f592770.1533811181.git.yi.z.zhang@linux.intel.com> <872818364.892078.1533806608252.JavaMail.zimbra@redhat.com> <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Cc: Pankaj Gupta , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan j williams , jack@suse.cz, hch@lst.de, yu c zhang , linux-mm@kvack.org, rkrcmar@redhat.com, yi z zhang To: "Zhang,Yi" Return-path: Content-Disposition: inline In-Reply-To: <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> Sender: linux-kernel-owner@vger.kernel.org List-Id: kvm.vger.kernel.org On Tue, Aug 14, 2018 at 01:41:40AM +0800, Zhang,Yi wrote: > > > On 2018年08月09日 17:23, Pankaj Gupta wrote: > >> DAX driver hotplug the device memory and move it to memory zone, these > >> pages will be marked reserved flag, however, some other kernel componet > >> will misconceive these pages are reserved mmio (ex: we map these dev_dax > >> or fs_dax pages to kvm for DIMM/NVDIMM backend). Together with the type > >> MEMORY_DEVICE_FS_DAX, we can use is_dax_page() to differentiate the pages > >> is DAX device memory or not. > >> > >> Signed-off-by: Zhang Yi > >> Signed-off-by: Zhang Yu > >> --- > >> include/linux/mm.h | 12 ++++++++++++ > >> 1 file changed, 12 insertions(+) > >> > >> diff --git a/include/linux/mm.h b/include/linux/mm.h > >> index 68a5121..de5cbc3 100644 > >> --- a/include/linux/mm.h > >> +++ b/include/linux/mm.h > >> @@ -889,6 +889,13 @@ static inline bool is_device_public_page(const struct > >> page *page) > >> page->pgmap->type == MEMORY_DEVICE_PUBLIC; > >> } > >> > >> +static inline bool is_dax_page(const struct page *page) > >> +{ > >> + return is_zone_device_page(page) && > >> + (page->pgmap->type == MEMORY_DEVICE_FS_DAX || > >> + page->pgmap->type == MEMORY_DEVICE_DEV_DAX); > >> +} > > I think question from Dan for KVM VM with 'MEMORY_DEVICE_PUBLIC' still holds? > > I am also interested to know if there is any use-case. > > > > Thanks, > > Pankaj > Yes, it is, thanks for your remind, Pankaj. > Adding Jerome for Dan's questions on V1: > [Dan]: > > Jerome, might there be any use case to pass MEMORY_DEVICE_PUBLIC > memory to a guest vm? Yes and no, i am not sure how we are going to do it. But being able to share GPU among multiple VM is on TODO list and those GPU will have MEMORY_DEVICE_PUBLIC|PRIVATE depending on the platform. So either we pass down the real underlying resource to the guest, or we will pass down a fake one and have guest and host driver talk to each other so that the host driver can do overall resource management accross multiple guests. So i would say that for now you can ignore MEMORY_DEVICE_PUBLIC and when we get to the KVM guest sharing of those and decide how we want to do it then we can update kvm to properly interpret those. Cheers, Jérôme From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id 405396B0003 for ; Mon, 13 Aug 2018 10:29:12 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id 99-v6so17229022qkr.14 for ; Mon, 13 Aug 2018 07:29:12 -0700 (PDT) Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id r41-v6si4374658qtj.307.2018.08.13.07.29.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 07:29:10 -0700 (PDT) Date: Mon, 13 Aug 2018 10:29:06 -0400 From: Jerome Glisse Subject: Re: [PATCH V3 3/4] mm: add a function to differentiate the pages is from DAX device memory Message-ID: <20180813142906.GA3451@redhat.com> References: <2b7856596e519130946c834d5d61b00b7f592770.1533811181.git.yi.z.zhang@linux.intel.com> <872818364.892078.1533806608252.JavaMail.zimbra@redhat.com> <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5ea50e63-b55a-c1e1-50be-6e2d951c04cf@linux.intel.com> Sender: owner-linux-mm@kvack.org List-ID: To: "Zhang,Yi" Cc: Pankaj Gupta , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan j williams , jack@suse.cz, hch@lst.de, yu c zhang , linux-mm@kvack.org, rkrcmar@redhat.com, yi z zhang On Tue, Aug 14, 2018 at 01:41:40AM +0800, Zhang,Yi wrote: > > > On 2018a1'08ae??09ae?JPY 17:23, Pankaj Gupta wrote: > >> DAX driver hotplug the device memory and move it to memory zone, these > >> pages will be marked reserved flag, however, some other kernel componet > >> will misconceive these pages are reserved mmio (ex: we map these dev_dax > >> or fs_dax pages to kvm for DIMM/NVDIMM backend). Together with the type > >> MEMORY_DEVICE_FS_DAX, we can use is_dax_page() to differentiate the pages > >> is DAX device memory or not. > >> > >> Signed-off-by: Zhang Yi > >> Signed-off-by: Zhang Yu > >> --- > >> include/linux/mm.h | 12 ++++++++++++ > >> 1 file changed, 12 insertions(+) > >> > >> diff --git a/include/linux/mm.h b/include/linux/mm.h > >> index 68a5121..de5cbc3 100644 > >> --- a/include/linux/mm.h > >> +++ b/include/linux/mm.h > >> @@ -889,6 +889,13 @@ static inline bool is_device_public_page(const struct > >> page *page) > >> page->pgmap->type == MEMORY_DEVICE_PUBLIC; > >> } > >> > >> +static inline bool is_dax_page(const struct page *page) > >> +{ > >> + return is_zone_device_page(page) && > >> + (page->pgmap->type == MEMORY_DEVICE_FS_DAX || > >> + page->pgmap->type == MEMORY_DEVICE_DEV_DAX); > >> +} > > I think question from Dan for KVM VM with 'MEMORY_DEVICE_PUBLIC' still holds? > > I am also interested to know if there is any use-case. > > > > Thanks, > > Pankaj > Yes, it is, thanks for your remind, Pankaj. > Adding Jerome for Dan's questions on V1: > [Dan]: > > Jerome, might there be any use case to pass MEMORY_DEVICE_PUBLIC > memory to a guest vm? Yes and no, i am not sure how we are going to do it. But being able to share GPU among multiple VM is on TODO list and those GPU will have MEMORY_DEVICE_PUBLIC|PRIVATE depending on the platform. So either we pass down the real underlying resource to the guest, or we will pass down a fake one and have guest and host driver talk to each other so that the host driver can do overall resource management accross multiple guests. So i would say that for now you can ignore MEMORY_DEVICE_PUBLIC and when we get to the KVM guest sharing of those and decide how we want to do it then we can update kvm to properly interpret those. Cheers, JA(C)rA'me