From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35671) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1as0Ct-0007Nq-FX for qemu-devel@nongnu.org; Sun, 17 Apr 2016 23:50:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1as0Cq-0001fi-8B for qemu-devel@nongnu.org; Sun, 17 Apr 2016 23:50:39 -0400 Received: from mga03.intel.com ([134.134.136.65]:3444) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1as0Cp-0001er-Sc for qemu-devel@nongnu.org; Sun, 17 Apr 2016 23:50:36 -0400 From: "Li, Tianyou" Date: Mon, 18 Apr 2016 03:50:12 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_1A93516D63C74545A87B6EF0983F2B1F052E897CSHSMSX103ccrcor_" MIME-Version: 1.0 Subject: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "qemu-devel@nongnu.org" , "Li, Tianyou" --_000_1A93516D63C74545A87B6EF0983F2B1F052E897CSHSMSX103ccrcor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Currently we are trying to implement below functionalities in QEmu: main me= mory in guest can be logically viewed as persistent and its content can be = survived through reboot or shutdown/powerup. I have looked into the QEmu memory management code include memory.c, exec.c= and other related source, unfortunately I do not have the chance to get cl= ue of how to make QEmu main memory persistent. I found that pmemsave could dump physical memory of guest, but I could not fi= nd how to restore the dump file before VM startup to execution. Could anyone provide some hints to me? Thanks in advance! Regards, Tianyou --_000_1A93516D63C74545A87B6EF0983F2B1F052E897CSHSMSX103ccrcor_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

Currently we are trying to implement below functiona= lities in QEmu: main memory in guest can be logically viewed as persistent = and its content can be survived through reboot or shutdown/powerup.

I have looked into the QEmu memory management code i= nclude memory.c, exec.c and other related source, unfortunately I do not ha= ve the chance to get clue of how to make QEmu main memory persistent. I fou= nd that pmemsave could dump physical memory of guest, but I could not find how = to restore the dump file before VM startup to execution.

Could anyone provide some hints to me? Thanks in adv= ance!

Regards,

Tianyou

--_000_1A93516D63C74545A87B6EF0983F2B1F052E897CSHSMSX103ccrcor_-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60495) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asFeG-0003dp-Ot for qemu-devel@nongnu.org; Mon, 18 Apr 2016 16:19:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asFeF-0003ZZ-Ko for qemu-devel@nongnu.org; Mon, 18 Apr 2016 16:19:56 -0400 Received: from mail-lf0-x241.google.com ([2a00:1450:4010:c07::241]:36682) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asFeF-0003ZH-Cr for qemu-devel@nongnu.org; Mon, 18 Apr 2016 16:19:55 -0400 Received: by mail-lf0-x241.google.com with SMTP id j10so8423688lfg.3 for ; Mon, 18 Apr 2016 13:19:54 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> From: Artyom Tarasenko Date: Mon, 18 Apr 2016 22:19:34 +0200 Message-ID: Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Tianyou" Cc: "qemu-devel@nongnu.org" Hi Tianyou, On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou wrote: > Currently we are trying to implement below functionalities in QEmu: main > memory in guest can be logically viewed as persistent and its content can be > survived through reboot or shutdown/powerup. > > I have looked into the QEmu memory management code include memory.c, exec.c > and other related source, unfortunately I do not have the chance to get clue > of how to make QEmu main memory persistent. I found that pmemsave could dump > physical memory of guest, but I could not find how to restore the dump file > before VM startup to execution. > > > > Could anyone provide some hints to me? Thanks in advance! Is the option "-mem-path=/path/to/mem-file" what you are looking for? Stefan wrote a nice post about QEMU RAM internals: http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html Regards, Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35438) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asLmM-0004Rn-2m for qemu-devel@nongnu.org; Mon, 18 Apr 2016 22:52:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asLmI-0005Sn-RB for qemu-devel@nongnu.org; Mon, 18 Apr 2016 22:52:41 -0400 Received: from mga02.intel.com ([134.134.136.20]:65509) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asLmI-0005Sh-GX for qemu-devel@nongnu.org; Mon, 18 Apr 2016 22:52:38 -0400 From: "Li, Tianyou" Date: Tue, 19 Apr 2016 02:52:32 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EA750@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Artyom Tarasenko Cc: "qemu-devel@nongnu.org" SGkgQXJ0eW9tLA0KDQpUaGFua3MgZm9yIHlvdXIgcG9pbnRlciEgSSBoYXZlIHRyaWVkIHRoZSAt bWVtLXBhdGggb3B0aW9uLCBhbmQgcmlnaHQgbm93IGxvb2tpbmcgaW50byB0aGUgY29kZSB0byBz ZWUgaWYgdGhlIGNvbnRlbnQgb2YgdGhlIGZpbGUgd2lsbCBiZSB1c2VkIGR1cmluZyB0aGUgZ3Vl c3QgTGludXggcnVubmluZyBuZXh0IHRpbWUuIFdpbGwgbGV0IHlvdSBrbm93IHRoZSByZXN1bHQu IA0KDQpTdGVmYW4ncyBwb3N0IGlzIGRlZmluaXRlbHkgaGVscGZ1bCwgdGhhbmtzIGZvciBsZXR0 aW5nIG1lIGtub3cuIENvdWxkIHlvdSBwbGVhc2UgbGV0IG1lIGtub3cgaWYgeW91IGhhdmUgbW9y ZSBRRW11IG1lbW9yeSBtYW5hZ2VtZW50IGRvY3VtZW50YXRpb24gYWJvdXQgaW50ZXJuYWxzPyBJ IGhhdmUgZ29vZ2xlZCBhcm91bmQsIGJ1dCBtb3JlIGxpa2VseSB0byBoZWFyIGZyb20geW91ciBh ZHZpY2UuIFRoYW5rcy4NCg0KUmVnYXJkcywNClRpYW55b3UNCg0KLS0tLS1PcmlnaW5hbCBNZXNz YWdlLS0tLS0NCkZyb206IEFydHlvbSBUYXJhc2Vua28gW21haWx0bzphdGFyNHFlbXVAZ21haWwu Y29tXSANClNlbnQ6IFR1ZXNkYXksIEFwcmlsIDE5LCAyMDE2IDQ6MjAgQU0NClRvOiBMaSwgVGlh bnlvdSA8dGlhbnlvdS5saUBpbnRlbC5jb20+DQpDYzogcWVtdS1kZXZlbEBub25nbnUub3JnDQpT dWJqZWN0OiBSZTogW1FlbXUtZGV2ZWxdIFBlcnNpc3RlbnQgTWFpbiBNZW1vcnkgaW4gUUVtdQ0K DQpIaSBUaWFueW91LA0KDQpPbiBNb24sIEFwciAxOCwgMjAxNiBhdCA1OjUwIEFNLCBMaSwgVGlh bnlvdSA8dGlhbnlvdS5saUBpbnRlbC5jb20+IHdyb3RlOg0KPiBDdXJyZW50bHkgd2UgYXJlIHRy eWluZyB0byBpbXBsZW1lbnQgYmVsb3cgZnVuY3Rpb25hbGl0aWVzIGluIFFFbXU6IA0KPiBtYWlu IG1lbW9yeSBpbiBndWVzdCBjYW4gYmUgbG9naWNhbGx5IHZpZXdlZCBhcyBwZXJzaXN0ZW50IGFu ZCBpdHMgDQo+IGNvbnRlbnQgY2FuIGJlIHN1cnZpdmVkIHRocm91Z2ggcmVib290IG9yIHNodXRk b3duL3Bvd2VydXAuDQo+DQo+IEkgaGF2ZSBsb29rZWQgaW50byB0aGUgUUVtdSBtZW1vcnkgbWFu YWdlbWVudCBjb2RlIGluY2x1ZGUgbWVtb3J5LmMsIA0KPiBleGVjLmMgYW5kIG90aGVyIHJlbGF0 ZWQgc291cmNlLCB1bmZvcnR1bmF0ZWx5IEkgZG8gbm90IGhhdmUgdGhlIA0KPiBjaGFuY2UgdG8g Z2V0IGNsdWUgb2YgaG93IHRvIG1ha2UgUUVtdSBtYWluIG1lbW9yeSBwZXJzaXN0ZW50LiBJIGZv dW5kIA0KPiB0aGF0IHBtZW1zYXZlIGNvdWxkIGR1bXAgcGh5c2ljYWwgbWVtb3J5IG9mIGd1ZXN0 LCBidXQgSSBjb3VsZCBub3QgDQo+IGZpbmQgaG93IHRvIHJlc3RvcmUgdGhlIGR1bXAgZmlsZSBi ZWZvcmUgVk0gc3RhcnR1cCB0byBleGVjdXRpb24uDQo+DQo+DQo+DQo+IENvdWxkIGFueW9uZSBw cm92aWRlIHNvbWUgaGludHMgdG8gbWU/IFRoYW5rcyBpbiBhZHZhbmNlIQ0KDQpJcyB0aGUgb3B0 aW9uICItbWVtLXBhdGg9L3BhdGgvdG8vbWVtLWZpbGUiIHdoYXQgeW91IGFyZSBsb29raW5nIGZv cj8NCg0KU3RlZmFuIHdyb3RlIGEgbmljZSBwb3N0IGFib3V0IFFFTVUgUkFNIGludGVybmFsczoN Cmh0dHA6Ly9ibG9nLnZtc3BsaWNlLm5ldC8yMDE2LzAxL3FlbXUtaW50ZXJuYWxzLWhvdy1ndWVz dC1waHlzaWNhbC1yYW0uaHRtbA0KDQpSZWdhcmRzLA0KQXJ0eW9tDQoNCg0KLS0NClJlZ2FyZHMs DQpBcnR5b20gVGFyYXNlbmtvDQoNClNQQVJDIGFuZCBQUEMgUFJlUCB1bmRlciBxZW11IGJsb2c6 IGh0dHA6Ly90eW9tLmJsb2dzcG90LmNvbS9zZWFyY2gvbGFiZWwvcWVtdQ0K From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38469) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asncP-0007id-R4 for qemu-devel@nongnu.org; Wed, 20 Apr 2016 04:36:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asncM-0008K9-Km for qemu-devel@nongnu.org; Wed, 20 Apr 2016 04:36:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41354) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asncM-0008Jx-FE for qemu-devel@nongnu.org; Wed, 20 Apr 2016 04:36:14 -0400 Date: Wed, 20 Apr 2016 09:36:10 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20160420083609.GD2263@work-vm> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Tianyou" Cc: "qemu-devel@nongnu.org" * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi, > > Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup. > > I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution. Can you explain what you mean by 'persistent' - where do you intend to store the guests memory? Also, remember that you'll need to save/load the device state as well as the rest of RAM. If you've got a way to preserve RAM then maybe hte xen-save-devices-state qemu command could be used to store the rest of devices. Dave > > Could anyone provide some hints to me? Thanks in advance! > > Regards, > Tianyou -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46969) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1at2yV-0008Fe-9a for qemu-devel@nongnu.org; Wed, 20 Apr 2016 21:00:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1at2yS-0007Bg-2l for qemu-devel@nongnu.org; Wed, 20 Apr 2016 21:00:07 -0400 Received: from mga09.intel.com ([134.134.136.24]:41364) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1at2yR-0007A0-Py for qemu-devel@nongnu.org; Wed, 20 Apr 2016 21:00:04 -0400 From: "Li, Tianyou" Date: Thu, 21 Apr 2016 00:59:11 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EB19A@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <20160420083609.GD2263@work-vm> In-Reply-To: <20160420083609.GD2263@work-vm> Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: "qemu-devel@nongnu.org" Hi Dave, Thanks for your response. Below are my explanations: > Can you explain what you mean by 'persistent' - where do you intend to st= ore the guests memory? There could be a file or memory region that can survive across guest shutdo= wn/reboot. Seems Artyom has pointed out the right direction and I have veri= fied by looking into the code throughout the call stack from pc_memory_init= to qemu_ram_alloc_from_file. I plan to write something like kernel module = to verify the persistency characteristics from guest point of view. > Also, remember that you'll need to save/load the device state as well as = the rest of RAM. =20 Device state handling could be done from two different aspects: 1. From hos= t perspective or, 2. From guest perspective. From host, qemu will always kn= ow the state of guest devices so that we can use qemu command to checkpoint= states, as you point out (very appreciated that, I do not know the command= xen-save-devices-state before). From guest, it can be something like suspe= nd to RAM or S3 for PC to checkpoint the current state of PC and restore th= em when wakeup. Currently I will prefer the #2.=20 In summary, I'd like to have the functionality in qemu that can save & rest= ore PC main memory at shutdown/power-on phase. Thanks. Regards, Tianyou -----Original Message----- From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com]=20 Sent: Wednesday, April 20, 2016 4:36 PM To: Li, Tianyou Cc: qemu-devel@nongnu.org Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi, >=20 > Currently we are trying to implement below functionalities in QEmu: main = memory in guest can be logically viewed as persistent and its content can b= e survived through reboot or shutdown/powerup. >=20 > I have looked into the QEmu memory management code include memory.c, exec= .c and other related source, unfortunately I do not have the chance to get = clue of how to make QEmu main memory persistent. I found that pmemsave could dump physical memory of guest, but I could not = find how to restore the dump file before VM startup to execution. Can you explain what you mean by 'persistent' - where do you intend to stor= e the guests memory? Also, remember that you'll need to save/load the device state as well as th= e rest of RAM. If you've got a way to preserve RAM then maybe hte xen-save= -devices-state qemu command could be used to store the rest of devices. Dave >=20 > Could anyone provide some hints to me? Thanks in advance! >=20 > Regards, > Tianyou -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35081) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1at9da-0002uo-2h for qemu-devel@nongnu.org; Thu, 21 Apr 2016 04:07:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1at9dW-0000Ve-SZ for qemu-devel@nongnu.org; Thu, 21 Apr 2016 04:06:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49776) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1at9dW-0000VU-L7 for qemu-devel@nongnu.org; Thu, 21 Apr 2016 04:06:54 -0400 Date: Thu, 21 Apr 2016 09:06:50 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20160421080649.GA2268@work-vm> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <20160420083609.GD2263@work-vm> <1A93516D63C74545A87B6EF0983F2B1F052EB19A@SHSMSX103.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1A93516D63C74545A87B6EF0983F2B1F052EB19A@SHSMSX103.ccr.corp.intel.com> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Tianyou" Cc: "qemu-devel@nongnu.org" * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi Dave, > > Thanks for your response. Below are my explanations: > > > Can you explain what you mean by 'persistent' - where do you intend to store the guests memory? > > There could be a file or memory region that can survive across guest shutdown/reboot. Seems Artyom has pointed out the right direction and I have verified by looking into the code throughout the call stack from pc_memory_init to qemu_ram_alloc_from_file. I plan to write something like kernel module to verify the persistency characteristics from guest point of view. Maybe it's worth checking the stuff in docs/memory-hotplug.txt - that shows how to create a memory region backed by a file (in that case using a hugepagefs - but I think it's general). Note, I don't think there's a way to use that at the moment for main PC memory. > > > Also, remember that you'll need to save/load the device state as well as the rest of RAM. > > Device state handling could be done from two different aspects: 1. From host perspective or, 2. From guest perspective. From host, qemu will always know the state of guest devices so that we can use qemu command to checkpoint states, as you point out (very appreciated that, I do not know the command xen-save-devices-state before). From guest, it can be something like suspend to RAM or S3 for PC to checkpoint the current state of PC and restore them when wakeup. Currently I will prefer the #2. > > In summary, I'd like to have the functionality in qemu that can save & restore PC main memory at shutdown/power-on phase. Thanks. Dave > > Regards, > Tianyou > > > -----Original Message----- > From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com] > Sent: Wednesday, April 20, 2016 4:36 PM > To: Li, Tianyou > Cc: qemu-devel@nongnu.org > Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu > > * Li, Tianyou (tianyou.li@intel.com) wrote: > > Hi, > > > > Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup. > > > > I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution. > > Can you explain what you mean by 'persistent' - where do you intend to store the guests memory? > Also, remember that you'll need to save/load the device state as well as the rest of RAM. If you've got a way to preserve RAM then maybe hte xen-save-devices-state qemu command could be used to store the rest of devices. > > Dave > > > > > Could anyone provide some hints to me? Thanks in advance! > > > > Regards, > > Tianyou > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38486) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atC9T-0002oF-9K for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:48:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atC9P-0006TS-CT for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:48:03 -0400 Received: from mga11.intel.com ([192.55.52.93]:29042) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atC9P-0006TJ-2I for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:47:59 -0400 From: "Li, Tianyou" Date: Thu, 21 Apr 2016 10:47:50 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Artyom Tarasenko , "Dr. David Alan Gilbert" Cc: "qemu-devel@nongnu.org" SGkgQXJ0eW9tLCBEYXZlICYgb3RoZXJzLA0KDQpBbiB1cGRhdGU6IEkgaGF2ZSB0cmllZCB0byB1 c2UgLW1lbS1wYXRoIG9wdGlvbiB0byBtYWtlIGEgZmlsZS1iYWNrZWQgbWVtb3J5LiBJIGhhdmUg dHJpZWQgdG8gd3JpdGUgYSBwYXJ0aWN1bGFyIHBoeXNpY2FsIGFkZHJlc3MgaW4gZ3Vlc3QgTGlu dXggT1Mgd2l0aCBzcGVjaWZpYyB2YWx1ZSB0byB2ZXJpZnkgdGhlIHBlcnNpc3RlbmN5IGNoYXJh Y3RlcmlzdGljcy4gVGhlIHJlc3VsdCBpcyBub3QgYXMgd2UgZXhwZWN0ZWQuIEJlbG93IGlzIG1v cmUgZGV0YWlscy4NCg0KDQoxLiBndWVzdCBvcyBsYXVuY2g6IHFlbXUtc3lzdGVtLXg4Nl82NCAt bW9uaXRvciBzdGRpbyAtbWFjaGluZSBwYyAtZW5hYmxlLWt2bSAtc21wIDEgIC1tIDhHICAtbWVt LXBhdGggJHtSQU1fRklMRX0gIC1kZXZpY2UgZTEwMDAsbmV0ZGV2PXVzZXIuMCAtbmV0ZGV2IHVz ZXIsaWQ9dXNlci4wLGhvc3Rmd2Q9dGNwOjo1NTU1LToyMiAke1NSQ19ESVJ9Ly4uL2ltYWdlcy9z bmFwc2hvdC9hcmNoLXNuYXBzaG90MS5pbWcNCjIuIGluc3RhbGwgZm1lbSBrZXJuZWwgbW9kdWxl IGluIGd1ZXN0IGxpbnV4LCB0aGUgY29kZSBpcyBoZXJlIChxdWljayAmIGRpcnR5IHdvcmthcm91 bmQgZm9yIG1lbV93cml0ZSwgcGxlYXNlIGlnbm9yZSk6IGh0dHBzOi8vZ2l0aHViLmNvbS9UaWFu eW91TGkvZm1lbQ0KMy4gcnVuIHByaW50ZiAid29ybGRoZWxsbyIgfCBkZCBvZj0vZGV2L2ZtZW0g YnM9MSBjb3VudD0xMCBzZWVrPTYyOTQ5NjcyOTYgY29udj1ub3RydW5jDQo0LiBydW4gZGQgaWY9 L2Rldi9mbWVtIGJzPTEgY291bnQ9MTAgc2Vlaz02Mjk0OTY3Mjk2IG9mPW91dA0KNS4gY2F0IG91 dCAgIC0tLS0tLS0+IGhlcmUgdGhlIHJlc3VsdCBpcyAid29ybGRoZWxsbyINCjYuIHBvd2Vyb2Zm IC0tLT4gZ3Vlc3Qgc2h1dGRvd24NCjcuIHJlLWxhdW5jaCBndWVzdCB1c2UgIzENCjguIHJ1biAj NCwgZ2V0IG1lbW9yeSBjb250ZW50IG9mIHNhbWUgYWRkcmVzcw0KOS4gdGhlIHJlc3VsdCBpcyBu b3QgYXMgZXhwZWN0ZWQsIGFsbCB6ZXJvIHRoZXJlDQoNCg0KSSBhbSBsb29raW5nIGludG8gdGhl IGNvZGUgaW4gbnVtYS5jIG9mIGZ1bmN0aW9uIGFsbG9jYXRlX3N5c3RlbV9tZW1vcnlfbm9ubnVt YSwgd2hpY2ggZG8gaW5pdCBtZW1vcnkgcmVnaW9uIGFuZCBjYWxsIHFlbXVfcmFtX21tYXAgaW4g bW1hcC1hbGxvYy5jLiBTZWVtcyB0aGUgbW1hcCB3aXRoIGZkIGhhcyBiZWVuIHNldHVwIGNvcnJl Y3RseSBzbyB0aGF0IG1lbW9yeSBkYXRhIHJlYWQvd3JpdGUgc2hvdWxkIGJlIGZsdXNoZWQgdG8g ZmlsZSBhbmQgc3Vydml2ZSBuZXh0IHRpbWUgb2YgYm9vdC4gSSBhbSBub3QgcXVpdGUgY2xlYXIg d2h5IHN0ZXAgIzkgZmFpbGVkIHRvIGdldCB0aGUgdmFsdWUgd2UgcHJldmlvdXMgc2V0IGluIHN0 ZXAgIzQuIFRoZXJlIGNvdWxkIGJlIHRoZSB0ZXN0IG1ldGhvZG9sb2d5IHByb2JsZW0gb3IgY2Fu IGJlIG15IGluLWNvcnJlY3QgdW5kZXJzdGFuZGluZyBvZiBxZW11IGZlYXR1cmUuIENvdWxkIHlv dSBwbGVhc2UgZWxhYm9yYXRlIG1vcmUgZGV0YWlscyBvciBnaXZlIG1lIHNvbWUgaGludHM/IFRo YW5rcy4NCg0KDQpSZWdhcmRzLA0KVGlhbnlvdQ0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0t LQ0KRnJvbTogTGksIFRpYW55b3UgDQpTZW50OiBUdWVzZGF5LCBBcHJpbCAxOSwgMjAxNiAxMDo1 MyBBTQ0KVG86IEFydHlvbSBUYXJhc2Vua28gPGF0YXI0cWVtdUBnbWFpbC5jb20+DQpDYzogcWVt dS1kZXZlbEBub25nbnUub3JnDQpTdWJqZWN0OiBSRTogW1FlbXUtZGV2ZWxdIFBlcnNpc3RlbnQg TWFpbiBNZW1vcnkgaW4gUUVtdQ0KDQpIaSBBcnR5b20sDQoNClRoYW5rcyBmb3IgeW91ciBwb2lu dGVyISBJIGhhdmUgdHJpZWQgdGhlIC1tZW0tcGF0aCBvcHRpb24sIGFuZCByaWdodCBub3cgbG9v a2luZyBpbnRvIHRoZSBjb2RlIHRvIHNlZSBpZiB0aGUgY29udGVudCBvZiB0aGUgZmlsZSB3aWxs IGJlIHVzZWQgZHVyaW5nIHRoZSBndWVzdCBMaW51eCBydW5uaW5nIG5leHQgdGltZS4gV2lsbCBs ZXQgeW91IGtub3cgdGhlIHJlc3VsdC4gDQoNClN0ZWZhbidzIHBvc3QgaXMgZGVmaW5pdGVseSBo ZWxwZnVsLCB0aGFua3MgZm9yIGxldHRpbmcgbWUga25vdy4gQ291bGQgeW91IHBsZWFzZSBsZXQg bWUga25vdyBpZiB5b3UgaGF2ZSBtb3JlIFFFbXUgbWVtb3J5IG1hbmFnZW1lbnQgZG9jdW1lbnRh dGlvbiBhYm91dCBpbnRlcm5hbHM/IEkgaGF2ZSBnb29nbGVkIGFyb3VuZCwgYnV0IG1vcmUgbGlr ZWx5IHRvIGhlYXIgZnJvbSB5b3VyIGFkdmljZS4gVGhhbmtzLg0KDQpSZWdhcmRzLA0KVGlhbnlv dQ0KDQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogQXJ0eW9tIFRhcmFzZW5rbyBb bWFpbHRvOmF0YXI0cWVtdUBnbWFpbC5jb21dDQpTZW50OiBUdWVzZGF5LCBBcHJpbCAxOSwgMjAx NiA0OjIwIEFNDQpUbzogTGksIFRpYW55b3UgPHRpYW55b3UubGlAaW50ZWwuY29tPg0KQ2M6IHFl bXUtZGV2ZWxAbm9uZ251Lm9yZw0KU3ViamVjdDogUmU6IFtRZW11LWRldmVsXSBQZXJzaXN0ZW50 IE1haW4gTWVtb3J5IGluIFFFbXUNCg0KSGkgVGlhbnlvdSwNCg0KT24gTW9uLCBBcHIgMTgsIDIw MTYgYXQgNTo1MCBBTSwgTGksIFRpYW55b3UgPHRpYW55b3UubGlAaW50ZWwuY29tPiB3cm90ZToN Cj4gQ3VycmVudGx5IHdlIGFyZSB0cnlpbmcgdG8gaW1wbGVtZW50IGJlbG93IGZ1bmN0aW9uYWxp dGllcyBpbiBRRW11OiANCj4gbWFpbiBtZW1vcnkgaW4gZ3Vlc3QgY2FuIGJlIGxvZ2ljYWxseSB2 aWV3ZWQgYXMgcGVyc2lzdGVudCBhbmQgaXRzIA0KPiBjb250ZW50IGNhbiBiZSBzdXJ2aXZlZCB0 aHJvdWdoIHJlYm9vdCBvciBzaHV0ZG93bi9wb3dlcnVwLg0KPg0KPiBJIGhhdmUgbG9va2VkIGlu dG8gdGhlIFFFbXUgbWVtb3J5IG1hbmFnZW1lbnQgY29kZSBpbmNsdWRlIG1lbW9yeS5jLCANCj4g ZXhlYy5jIGFuZCBvdGhlciByZWxhdGVkIHNvdXJjZSwgdW5mb3J0dW5hdGVseSBJIGRvIG5vdCBo YXZlIHRoZSANCj4gY2hhbmNlIHRvIGdldCBjbHVlIG9mIGhvdyB0byBtYWtlIFFFbXUgbWFpbiBt ZW1vcnkgcGVyc2lzdGVudC4gSSBmb3VuZCANCj4gdGhhdCBwbWVtc2F2ZSBjb3VsZCBkdW1wIHBo eXNpY2FsIG1lbW9yeSBvZiBndWVzdCwgYnV0IEkgY291bGQgbm90IA0KPiBmaW5kIGhvdyB0byBy ZXN0b3JlIHRoZSBkdW1wIGZpbGUgYmVmb3JlIFZNIHN0YXJ0dXAgdG8gZXhlY3V0aW9uLg0KPg0K Pg0KPg0KPiBDb3VsZCBhbnlvbmUgcHJvdmlkZSBzb21lIGhpbnRzIHRvIG1lPyBUaGFua3MgaW4g YWR2YW5jZSENCg0KSXMgdGhlIG9wdGlvbiAiLW1lbS1wYXRoPS9wYXRoL3RvL21lbS1maWxlIiB3 aGF0IHlvdSBhcmUgbG9va2luZyBmb3I/DQoNClN0ZWZhbiB3cm90ZSBhIG5pY2UgcG9zdCBhYm91 dCBRRU1VIFJBTSBpbnRlcm5hbHM6DQpodHRwOi8vYmxvZy52bXNwbGljZS5uZXQvMjAxNi8wMS9x ZW11LWludGVybmFscy1ob3ctZ3Vlc3QtcGh5c2ljYWwtcmFtLmh0bWwNCg0KUmVnYXJkcywNCkFy dHlvbQ0KDQoNCi0tDQpSZWdhcmRzLA0KQXJ0eW9tIFRhcmFzZW5rbw0KDQpTUEFSQyBhbmQgUFBD IFBSZVAgdW5kZXIgcWVtdSBibG9nOiBodHRwOi8vdHlvbS5ibG9nc3BvdC5jb20vc2VhcmNoL2xh YmVsL3FlbXUNCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39062) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atCDE-0008Tc-Gp for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:51:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atCDA-0007FS-FL for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:51:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39255) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atCDA-0007FN-84 for qemu-devel@nongnu.org; Thu, 21 Apr 2016 06:51:52 -0400 Date: Thu, 21 Apr 2016 11:51:47 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20160421105146.GC2268@work-vm> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Tianyou" Cc: Artyom Tarasenko , "qemu-devel@nongnu.org" * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi Artyom, Dave & others, > > An update: I have tried to use -mem-path option to make a file-backed memory. I have tried to write a particular physical address in guest Linux OS with specific value to verify the persistency characteristics. The result is not as we expected. Below is more details. > > > 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc -enable-kvm -smp 1 -m 8G -mem-path ${RAM_FILE} -device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22 ${SRC_DIR}/../images/snapshot/arch-snapshot1.img > 2. install fmem kernel module in guest linux, the code is here (quick & dirty workaround for mem_write, please ignore): https://github.com/TianyouLi/fmem > 3. run printf "worldhello" | dd of=/dev/fmem bs=1 count=10 seek=6294967296 conv=notrunc > 4. run dd if=/dev/fmem bs=1 count=10 seek=6294967296 of=out > 5. cat out -------> here the result is "worldhello" > 6. poweroff ---> guest shutdown > 7. re-launch guest use #1 > 8. run #4, get memory content of same address > 9. the result is not as expected, all zero there > > > I am looking into the code in numa.c of function allocate_system_memory_nonnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c. Seems the mmap with fd has been setup correctly so that memory data read/write should be flushed to file and survive next time of boot. I am not quite clear why step #9 failed to get the value we previous set in step #4. There could be the test methodology problem or can be my in-correct understanding of qemu feature. Could you please elaborate more details or give me some hints? Thanks. I wonder if QEMU or the guest (BIOS? Kernel?) is zeroing the memory ? For normal memory I'd expect it to zero it. Dave > > > Regards, > Tianyou > > -----Original Message----- > From: Li, Tianyou > Sent: Tuesday, April 19, 2016 10:53 AM > To: Artyom Tarasenko > Cc: qemu-devel@nongnu.org > Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu > > Hi Artyom, > > Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. > > Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks. > > Regards, > Tianyou > > -----Original Message----- > From: Artyom Tarasenko [mailto:atar4qemu@gmail.com] > Sent: Tuesday, April 19, 2016 4:20 AM > To: Li, Tianyou > Cc: qemu-devel@nongnu.org > Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu > > Hi Tianyou, > > On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou wrote: > > Currently we are trying to implement below functionalities in QEmu: > > main memory in guest can be logically viewed as persistent and its > > content can be survived through reboot or shutdown/powerup. > > > > I have looked into the QEmu memory management code include memory.c, > > exec.c and other related source, unfortunately I do not have the > > chance to get clue of how to make QEmu main memory persistent. I found > > that pmemsave could dump physical memory of guest, but I could not > > find how to restore the dump file before VM startup to execution. > > > > > > > > Could anyone provide some hints to me? Thanks in advance! > > Is the option "-mem-path=/path/to/mem-file" what you are looking for? > > Stefan wrote a nice post about QEMU RAM internals: > http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html > > Regards, > Artyom > > > -- > Regards, > Artyom Tarasenko > > SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40070) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atRTG-0007qM-Jr for qemu-devel@nongnu.org; Thu, 21 Apr 2016 23:09:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atRTD-0001f4-Cl for qemu-devel@nongnu.org; Thu, 21 Apr 2016 23:09:30 -0400 Received: from mga01.intel.com ([192.55.52.88]:26410) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atRTD-0001d9-2x for qemu-devel@nongnu.org; Thu, 21 Apr 2016 23:09:27 -0400 From: "Li, Tianyou" Date: Fri, 22 Apr 2016 03:09:18 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EB895@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> <20160421105146.GC2268@work-vm> In-Reply-To: <20160421105146.GC2268@work-vm> Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: Artyom Tarasenko , "qemu-devel@nongnu.org" > I wonder if QEMU or the guest (BIOS? Kernel?) is zeroing the memory ? F= or normal memory I'd expect it to zero it. Zeroing page probably happens when allocating memory, I am not sure if it d= id happen once release the page. And, I am trying to find specific pattern = of the content I write to the physical address of the guest, in the memory = backed file in host, using bgrep. The result is no that kind of data was wr= itten into the memory backed file specified in -mem-path option. Could some= one help on elaborating more details? Thanks. Regards, Tianyou -----Original Message----- From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com]=20 Sent: Thursday, April 21, 2016 6:52 PM To: Li, Tianyou Cc: Artyom Tarasenko ; qemu-devel@nongnu.org Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi Artyom, Dave & others, >=20 > An update: I have tried to use -mem-path option to make a file-backed mem= ory. I have tried to write a particular physical address in guest Linux OS = with specific value to verify the persistency characteristics. The result i= s not as we expected. Below is more details. >=20 >=20 > 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc=20 > -enable-kvm -smp 1 -m 8G -mem-path ${RAM_FILE} -device=20 > e1000,netdev=3Duser.0 -netdev user,id=3Duser.0,hostfwd=3Dtcp::5555-:22=20 > ${SRC_DIR}/../images/snapshot/arch-snapshot1.img > 2. install fmem kernel module in guest linux, the code is here (quick=20 > & dirty workaround for mem_write, please ignore):=20 > https://github.com/TianyouLi/fmem 3. run printf "worldhello" | dd of=3D/d= ev/fmem bs=3D1 count=3D10 seek=3D6294967296 conv=3Dnotrunc 4. run dd if=3D/= dev/fmem bs=3D1 count=3D10 seek=3D6294967296 of=3Dout > 5. cat out -------> here the result is "worldhello" > 6. poweroff ---> guest shutdown > 7. re-launch guest use #1 > 8. run #4, get memory content of same address 9. the result is not as=20 > expected, all zero there >=20 >=20 > I am looking into the code in numa.c of function allocate_system_memory_n= onnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c.= Seems the mmap with fd has been setup correctly so that memory data read/w= rite should be flushed to file and survive next time of boot. I am not quit= e clear why step #9 failed to get the value we previous set in step #4. The= re could be the test methodology problem or can be my in-correct understand= ing of qemu feature. Could you please elaborate more details or give me som= e hints? Thanks. I wonder if QEMU or the guest (BIOS? Kernel?) is zeroing the memory ? For= normal memory I'd expect it to zero it. Dave >=20 >=20 > Regards, > Tianyou >=20 > -----Original Message----- > From: Li, Tianyou > Sent: Tuesday, April 19, 2016 10:53 AM > To: Artyom Tarasenko > Cc: qemu-devel@nongnu.org > Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu >=20 > Hi Artyom, >=20 > Thanks for your pointer! I have tried the -mem-path option, and right now= looking into the code to see if the content of the file will be used durin= g the guest Linux running next time. Will let you know the result.=20 >=20 > Stefan's post is definitely helpful, thanks for letting me know. Could yo= u please let me know if you have more QEmu memory management documentation = about internals? I have googled around, but more likely to hear from your a= dvice. Thanks. >=20 > Regards, > Tianyou >=20 > -----Original Message----- > From: Artyom Tarasenko [mailto:atar4qemu@gmail.com] > Sent: Tuesday, April 19, 2016 4:20 AM > To: Li, Tianyou > Cc: qemu-devel@nongnu.org > Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu >=20 > Hi Tianyou, >=20 > On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou wrote= : > > Currently we are trying to implement below functionalities in QEmu:=20 > > main memory in guest can be logically viewed as persistent and its=20 > > content can be survived through reboot or shutdown/powerup. > > > > I have looked into the QEmu memory management code include memory.c,=20 > > exec.c and other related source, unfortunately I do not have the=20 > > chance to get clue of how to make QEmu main memory persistent. I=20 > > found that pmemsave could dump physical memory of guest, but I could=20 > > not find how to restore the dump file before VM startup to execution. > > > > > > > > Could anyone provide some hints to me? Thanks in advance! >=20 > Is the option "-mem-path=3D/path/to/mem-file" what you are looking for? >=20 > Stefan wrote a nice post about QEMU RAM internals: > http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram > .html >=20 > Regards, > Artyom >=20 >=20 > -- > Regards, > Artyom Tarasenko >=20 > SPARC and PPC PReP under qemu blog:=20 > http://tyom.blogspot.com/search/label/qemu -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38172) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atd7K-0003vs-Oq for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:35:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atd7G-0007SH-86 for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:35:38 -0400 Received: from mga04.intel.com ([192.55.52.120]:1062) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atd7F-0007RL-N4 for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:35:34 -0400 From: "Li, Tianyou" Date: Fri, 22 Apr 2016 15:35:23 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EBAB4@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> <20160421105146.GC2268@work-vm> Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: Artyom Tarasenko , "qemu-devel@nongnu.org" I found the mmap was invoked in qemu with option MAP_ANONYMOUS, which will = zero pages. After a simple workaround right now pc.ram region will keep its= content through reboot/shutdown.=20 Thanks Dave and Artyom for the help! Regards, Tianyou -----Original Message----- From: Li, Tianyou=20 Sent: Friday, April 22, 2016 11:09 AM To: Dr. David Alan Gilbert Cc: Artyom Tarasenko ; qemu-devel@nongnu.org Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu > I wonder if QEMU or the guest (BIOS? Kernel?) is zeroing the memory ? F= or normal memory I'd expect it to zero it. Zeroing page probably happens when allocating memory, I am not sure if it d= id happen once release the page. And, I am trying to find specific pattern = of the content I write to the physical address of the guest, in the memory = backed file in host, using bgrep. The result is no that kind of data was wr= itten into the memory backed file specified in -mem-path option. Could some= one help on elaborating more details? Thanks. Regards, Tianyou -----Original Message----- From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com] Sent: Thursday, April 21, 2016 6:52 PM To: Li, Tianyou Cc: Artyom Tarasenko ; qemu-devel@nongnu.org Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu * Li, Tianyou (tianyou.li@intel.com) wrote: > Hi Artyom, Dave & others, >=20 > An update: I have tried to use -mem-path option to make a file-backed mem= ory. I have tried to write a particular physical address in guest Linux OS = with specific value to verify the persistency characteristics. The result i= s not as we expected. Below is more details. >=20 >=20 > 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc=20 > -enable-kvm -smp 1 -m 8G -mem-path ${RAM_FILE} -device > e1000,netdev=3Duser.0 -netdev user,id=3Duser.0,hostfwd=3Dtcp::5555-:22 > ${SRC_DIR}/../images/snapshot/arch-snapshot1.img > 2. install fmem kernel module in guest linux, the code is here (quick=20 > & dirty workaround for mem_write, please ignore): > https://github.com/TianyouLi/fmem 3. run printf "worldhello" | dd of=3D/d= ev/fmem bs=3D1 count=3D10 seek=3D6294967296 conv=3Dnotrunc 4. run dd if=3D/= dev/fmem bs=3D1 count=3D10 seek=3D6294967296 of=3Dout > 5. cat out -------> here the result is "worldhello" > 6. poweroff ---> guest shutdown > 7. re-launch guest use #1 > 8. run #4, get memory content of same address 9. the result is not as=20 > expected, all zero there >=20 >=20 > I am looking into the code in numa.c of function allocate_system_memory_n= onnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c.= Seems the mmap with fd has been setup correctly so that memory data read/w= rite should be flushed to file and survive next time of boot. I am not quit= e clear why step #9 failed to get the value we previous set in step #4. The= re could be the test methodology problem or can be my in-correct understand= ing of qemu feature. Could you please elaborate more details or give me som= e hints? Thanks. I wonder if QEMU or the guest (BIOS? Kernel?) is zeroing the memory ? For= normal memory I'd expect it to zero it. Dave >=20 >=20 > Regards, > Tianyou >=20 > -----Original Message----- > From: Li, Tianyou > Sent: Tuesday, April 19, 2016 10:53 AM > To: Artyom Tarasenko > Cc: qemu-devel@nongnu.org > Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu >=20 > Hi Artyom, >=20 > Thanks for your pointer! I have tried the -mem-path option, and right now= looking into the code to see if the content of the file will be used durin= g the guest Linux running next time. Will let you know the result.=20 >=20 > Stefan's post is definitely helpful, thanks for letting me know. Could yo= u please let me know if you have more QEmu memory management documentation = about internals? I have googled around, but more likely to hear from your a= dvice. Thanks. >=20 > Regards, > Tianyou >=20 > -----Original Message----- > From: Artyom Tarasenko [mailto:atar4qemu@gmail.com] > Sent: Tuesday, April 19, 2016 4:20 AM > To: Li, Tianyou > Cc: qemu-devel@nongnu.org > Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu >=20 > Hi Tianyou, >=20 > On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou wrote= : > > Currently we are trying to implement below functionalities in QEmu:=20 > > main memory in guest can be logically viewed as persistent and its=20 > > content can be survived through reboot or shutdown/powerup. > > > > I have looked into the QEmu memory management code include memory.c,=20 > > exec.c and other related source, unfortunately I do not have the=20 > > chance to get clue of how to make QEmu main memory persistent. I=20 > > found that pmemsave could dump physical memory of guest, but I could=20 > > not find how to restore the dump file before VM startup to execution. > > > > > > > > Could anyone provide some hints to me? Thanks in advance! >=20 > Is the option "-mem-path=3D/path/to/mem-file" what you are looking for? >=20 > Stefan wrote a nice post about QEMU RAM internals: > http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram > .html >=20 > Regards, > Artyom >=20 >=20 > -- > Regards, > Artyom Tarasenko >=20 > SPARC and PPC PReP under qemu blog:=20 > http://tyom.blogspot.com/search/label/qemu -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42247) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atdLk-0000qm-Ek for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:50:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atdLj-0003A3-DF for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:50:32 -0400 Received: from mail-lf0-x22e.google.com ([2a00:1450:4010:c07::22e]:33481) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atdLi-00039h-QA for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:50:31 -0400 Received: by mail-lf0-x22e.google.com with SMTP id e190so81789291lfe.0 for ; Fri, 22 Apr 2016 08:50:30 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1A93516D63C74545A87B6EF0983F2B1F052EBAB4@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> <20160421105146.GC2268@work-vm> <1A93516D63C74545A87B6EF0983F2B1F052EBAB4@SHSMSX103.ccr.corp.intel.com> From: Artyom Tarasenko Date: Fri, 22 Apr 2016 17:50:09 +0200 Message-ID: Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Tianyou" Cc: "Dr. David Alan Gilbert" , "qemu-devel@nongnu.org" , Stefan Hajnoczi Hi Tianyou, On Fri, Apr 22, 2016 at 5:35 PM, Li, Tianyou wrote: > I found the mmap was invoked in qemu with option MAP_ANONYMOUS, which will zero pages. After a simple workaround right now pc.ram region will keep its content through reboot/shutdown. Can you publish the workaround? I think it would be nice to implement a persistent RAM as a feature, specified by a command line switch. Regards, Artyom -- Regards, Artyom Tarasenko SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42717) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atdNp-0007ks-Pb for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:52:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atdNl-0003r2-P4 for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:52:41 -0400 Received: from mga04.intel.com ([192.55.52.120]:49751) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atdNl-0003qy-K1 for qemu-devel@nongnu.org; Fri, 22 Apr 2016 11:52:37 -0400 From: "Li, Tianyou" Date: Fri, 22 Apr 2016 15:52:33 +0000 Message-ID: <1A93516D63C74545A87B6EF0983F2B1F052EBAF6@SHSMSX103.ccr.corp.intel.com> References: <1A93516D63C74545A87B6EF0983F2B1F052E897C@SHSMSX103.ccr.corp.intel.com> <1A93516D63C74545A87B6EF0983F2B1F052EB4FB@SHSMSX103.ccr.corp.intel.com> <20160421105146.GC2268@work-vm> <1A93516D63C74545A87B6EF0983F2B1F052EBAB4@SHSMSX103.ccr.corp.intel.com> In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Artyom Tarasenko Cc: "Dr. David Alan Gilbert" , "qemu-devel@nongnu.org" , Stefan Hajnoczi U3VyZSwgdGhhdCB3aWxsIGJlIGdyZWF0ISBIb3cgYWJvdXQgZ2l2ZSBtZSBhIGZldyBkYXlzIHNv IEkgY2FuIGNsZWFudXAgdGhlIGNvZGUgYW5kIHByb3ZpZGUgYSBiZXR0ZXIgd29ya2Fyb3VuZCBh cyBwYXRjaCBmb3IgeW91IGFsbCB0byByZXZpZXc/IA0KDQpSZWdhcmRzLA0KVGlhbnlvdQ0KDQot LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogQXJ0eW9tIFRhcmFzZW5rbyBbbWFpbHRv OmF0YXI0cWVtdUBnbWFpbC5jb21dIA0KU2VudDogRnJpZGF5LCBBcHJpbCAyMiwgMjAxNiAxMTo1 MCBQTQ0KVG86IExpLCBUaWFueW91IDx0aWFueW91LmxpQGludGVsLmNvbT4NCkNjOiBEci4gRGF2 aWQgQWxhbiBHaWxiZXJ0IDxkZ2lsYmVydEByZWRoYXQuY29tPjsgcWVtdS1kZXZlbEBub25nbnUu b3JnOyBTdGVmYW4gSGFqbm9jemkgPHN0ZWZhbmhhQGdtYWlsLmNvbT4NClN1YmplY3Q6IFJlOiBb UWVtdS1kZXZlbF0gUGVyc2lzdGVudCBNYWluIE1lbW9yeSBpbiBRRW11DQoNCkhpICBUaWFueW91 LA0KDQpPbiBGcmksIEFwciAyMiwgMjAxNiBhdCA1OjM1IFBNLCBMaSwgVGlhbnlvdSA8dGlhbnlv dS5saUBpbnRlbC5jb20+IHdyb3RlOg0KPiBJIGZvdW5kIHRoZSBtbWFwIHdhcyBpbnZva2VkIGlu IHFlbXUgd2l0aCBvcHRpb24gTUFQX0FOT05ZTU9VUywgd2hpY2ggd2lsbCB6ZXJvIHBhZ2VzLiBB ZnRlciBhIHNpbXBsZSB3b3JrYXJvdW5kIHJpZ2h0IG5vdyBwYy5yYW0gcmVnaW9uIHdpbGwga2Vl cCBpdHMgY29udGVudCB0aHJvdWdoIHJlYm9vdC9zaHV0ZG93bi4NCg0KQ2FuIHlvdSBwdWJsaXNo IHRoZSB3b3JrYXJvdW5kPyBJIHRoaW5rIGl0IHdvdWxkIGJlIG5pY2UgdG8gaW1wbGVtZW50IGEg cGVyc2lzdGVudCBSQU0gYXMgYSBmZWF0dXJlLCBzcGVjaWZpZWQgYnkgYSBjb21tYW5kIGxpbmUg c3dpdGNoLg0KDQpSZWdhcmRzLA0KQXJ0eW9tDQoNCi0tDQpSZWdhcmRzLA0KQXJ0eW9tIFRhcmFz ZW5rbw0KDQpTUEFSQyBhbmQgUFBDIFBSZVAgdW5kZXIgcWVtdSBibG9nOiBodHRwOi8vdHlvbS5i bG9nc3BvdC5jb20vc2VhcmNoL2xhYmVsL3FlbXUNCg==