From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41612) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXILO-000380-KI for qemu-devel@nongnu.org; Tue, 09 Aug 2016 21:30:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bXILK-0001b4-D0 for qemu-devel@nongnu.org; Tue, 09 Aug 2016 21:30:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56878) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bXILK-0001au-1h for qemu-devel@nongnu.org; Tue, 09 Aug 2016 21:30:02 -0400 Date: Wed, 10 Aug 2016 09:22:58 +0800 From: Fam Zheng Message-ID: <20160810012258.GA11351@al.usersys.redhat.com> References: <20160809191216.GF2025@work-vm> <1470790447-83284-1-git-send-email-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1470790447-83284-1-git-send-email-jiangshanlai@gmail.com> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH V2] add migration capability to bypass the shared memory List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Lai Jiangshan Cc: qemu-devel@nongnu.org, Juan Quintela , Peter Crosthwaite , Markus Armbruster , Amit Shah , Paolo Bonzini , Richard Henderson On Wed, 08/10 08:54, Lai Jiangshan wrote: > When the migration capability 'bypass-shared-memory' > is set, the shared memory will be bypassed when migration. >=20 > It is the key feature to enable several excellent features for > the qemu, such as qemu-local-migration, qemu-live-update, > extremely-fast-save-restore, vm-template, vm-fast-live-clone, > yet-another-post-copy-migration, etc.. >=20 > The philosophy behind this key feature and the advanced > key features is that a part of the memory management is > separated out from the qemu, and let the other toolkits > such as libvirt, runv(https://github.com/hyperhq/runv/) > or the next qemu-cmd directly access to it, manage it, > provide features to it. >=20 > The hyperhq(http://hyper.sh http://hypercontainer.io/) > introduced the feature vm-template(vm-fast-live-clone) > to the hyper container for several months, it works perfect. > (see https://github.com/hyperhq/runv/pull/297) >=20 > The feature vm-template makes the containers(VMs) can > be started in 130ms and save 80M memory for every > container(VM). So that the hyper containers are fast > and high-density as normal containers. >=20 > In current qemu command line, shared memory has > to be configured via memory-object. Anyone can add a > -mem-path-share to the qemu command line for combining > with -mem-path for this feature. This patch doesn=E2=80=99t include > this change of -mem-path-share. >=20 > Advanced features: > 1) qemu-local-migration, qemu-live-update > Set the mem-path on the tmpfs and set share=3Don for it when > start the vm. example: > -object \ > memory-backend-file,id=3Dmem,size=3D128M,mem-path=3D/dev/shm/memory,sha= re=3Don \ > -numa node,nodeid=3D0,cpus=3D0-7,memdev=3Dmem >=20 > when you want to migrate the vm locally (after fixed a security bug > of the qemu-binary, or other reason), you can start a new qemu with > the same command line and -incoming, then you can migrate the > vm from the old qemu to the new qemu with the migration capability > 'bypass-shared-memory' set. The migration will migrate the device-state > *ONLY*, the memory is the origin memory backed by tmpfs file. >=20 > 2) extremely-fast-save-restore > the same above, but the mem-path is on the persistent file system. >=20 > 3) vm-template, vm-fast-live-clone > the template vm is started as 1), and paused when the guest reaches > the template point(example: the guest app is ready), then the template > vm is saved. (the qemu process of the template can be killed now, becau= se > we need only the memory and the device state files (in tmpfs)). >=20 > Then we can launch one or multiple VMs base on the template vm states, > the new VMs are started without the =E2=80=9Cshare=3Don=E2=80=9D, all t= he new VMs share > the initial memory from the memory file, they save a lot of memory. > all the new VMs start from the template point, the guest app can go to > work quickly. >=20 > The new VM booted from template vm can=E2=80=99t become template again, > if you need this unusual chained-template feature, you can write > a cloneable-tmpfs kernel module for it. >=20 > The libvirt toolkit can=E2=80=99t manage vm-template currently, in the > hyperhq/runv, we use qemu wrapper script to do it. I hope someone add > =E2=80=9Clibvrit managed template=E2=80=9D feature to libvirt. >=20 > 4) yet-another-post-copy-migration > It is a possible feature, no toolkit can do it well now. > Using nbd server/client on the memory file is reluctantly Ok but > inconvenient. A special feature for tmpfs might be needed to > fully complete this feature. > No one need yet another post copy migration method, > but it is possible when some crazy man need it. >=20 > Changed from v1: > fix style Thanks for sending the patch and fixing the coding style. Without having = looked at the patch, I have one meta comment here. In the future, please move th= e revision log to below the --- line after the signed-off-by line. This is because that piece of text is not necessary in the git history, so it won= 't be picked up by git-am when maintainers apply it, if put there. Also please remember to always send top level thread for new versions ins= tead of as a reply to v1 (i.e. don't set In-Reply-To in the header). Thanks, Fam