From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50971) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcwyV-0004UG-9M for qemu-devel@nongnu.org; Tue, 31 Mar 2015 10:17:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YcwyR-0000Pc-59 for qemu-devel@nongnu.org; Tue, 31 Mar 2015 10:17:03 -0400 Received: from mail-wi0-x22c.google.com ([2a00:1450:400c:c05::22c]:36424) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcwyQ-0000P6-Rh for qemu-devel@nongnu.org; Tue, 31 Mar 2015 10:16:59 -0400 Received: by wixo5 with SMTP id o5so15480580wix.1 for ; Tue, 31 Mar 2015 07:16:58 -0700 (PDT) Date: Tue, 31 Mar 2015 15:16:55 +0100 From: Stefan Hajnoczi Message-ID: <20150331141655.GC27156@stefanha-thinkpad.redhat.com> References: <55128084.2040304@huawei.com> <87a8z12yot.fsf@neno.neno> <5513793B.6020909@cn.fujitsu.com> <874mp8xd9k.fsf@neno.neno> <20150327085634.GB3304@stefanha-thinkpad.redhat.com> <551A52CB.6040303@cn.fujitsu.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i7F3eY7HS/tUJxUd" Content-Disposition: inline In-Reply-To: <551A52CB.6040303@cn.fujitsu.com> Subject: Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Wen Congyang Cc: Kevin Wolf , hangaohuai@huawei.com, zhanghailiang , Li Zhijian , Juan Quintela , Michael Tsirkin , qemu-devel@nongnu.org, "Dr. David Alan Gilbert (git)" , "Gonglei (Arei)" , Stefan Hajnoczi , Amit Shah , peter.huangpeng@huawei.com, david@gibson.dropbear.id.au --i7F3eY7HS/tUJxUd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 31, 2015 at 03:54:51PM +0800, Wen Congyang wrote: > On 03/27/2015 04:56 PM, Stefan Hajnoczi wrote: > > On Thu, Mar 26, 2015 at 11:29:43AM +0100, Juan Quintela wrote: > >> Wen Congyang wrote: > >>> On 03/25/2015 05:50 PM, Juan Quintela wrote: > >>>> zhanghailiang wrote: > >>>>> Hi all, > >>>>> > >>>>> We found that, sometimes, the content of VM's memory is > >>>>> inconsistent between Source side and Destination side > >>>>> when we check it just after finishing migration but before VM conti= nue to Run. > >>>>> > >>>>> We use a patch like bellow to find this issue, you can find it from= affix, > >>>>> and Steps to reprduce: > >>>>> > >>>>> (1) Compile QEMU: > >>>>> ./configure --target-list=3Dx86_64-softmmu --extra-ldflags=3D"-ls= sl" && make > >>>>> > >>>>> (2) Command and output: > >>>>> SRC: # x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu > >>>>> qemu64,-kvmclock -netdev tap,id=3Dhn0-device > >>>>> virtio-net-pci,id=3Dnet-pci0,netdev=3Dhn0 -boot c -drive > >>>>> file=3D/mnt/sdb/pure_IMG/sles/sles11_sp3.img,if=3Dnone,id=3Ddrive-v= irtio-disk0,cache=3Dunsafe > >>>>> -device > >>>>> virtio-blk-pci,bus=3Dpci.0,addr=3D0x4,drive=3Ddrive-virtio-disk0,id= =3Dvirtio-disk0 > >>>>> -vnc :7 -m 2048 -smp 2 -device piix3-usb-uhci -device usb-tablet > >>>>> -monitor stdio > >>>> > >>>> Could you try to reproduce: > >>>> - without vhost > >>>> - without virtio-net > >>>> - cache=3Dunsafe is going to give you trouble, but trouble should on= ly > >>>> happen after migration of pages have finished. > >>> > >>> If I use ide disk, it doesn't happen. > >>> Even if I use virtio-net with vhost=3Don, it still doesn't happen. I = guess > >>> it is because I migrate the guest when it is booting. The virtio net > >>> device is not used in this case. > >> > >> Kevin, Stefan, Michael, any great idea? > >=20 > > You must use -drive cache=3Dnone if you want to use live migration. It > > should not directly affect memory during migration though. > >=20 > >>>>> We have done further test and found that some pages has been > >>>>> dirtied but its corresponding migration_bitmap is not set. > >>>>> We can't figure out which modules of QEMU has missed setting bitmap > >>>>> when dirty page of VM, > >>>>> it is very difficult for us to trace all the actions of dirtying VM= 's pages. > >>>> > >>>> This seems to point to a bug in one of the devices. > >=20 > > I think you'll need to track down which pages are different. If you are > > lucky, their contents will reveal what the page is used for. >=20 > I have a question about memory dirty log: > If qemu modifies some memory, is this page marked dirty? I think the answer is "no", the dirty page tracking only affects guest mode. Even if dirty page tracking covered QEMU userspace code, remember that QEMU can be run without KVM in TCG mode. In order to support that it's necessary to always mark memory dirty explicitly when written from QEMU code. Stefan --i7F3eY7HS/tUJxUd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVGqxXAAoJEJykq7OBq3PIBbkH/A+GON0YlQ5tR7J6oImmSfC/ my36vL5DBbaS9HeJN1wSsXEShkkA6iQk8xqnKpXVlM0WJWVH4OzROnIY+7Lyev2a No3nierMM1rUiELtFjF32Ju95XiVe2BtPDJjz2I66xOdTFdgoU3HjfJz80U+vzNF m92nuzlw0EAhtYJulfLePhB3hNvJ2vlWllZhImj2tWJ8gqbS6d4vxcRA7VKmJPAZ Nca2EOR8BKAqmbDc9XGraK9nY4XIbHrenDIA6hbzA76qOP2kdnXoYpQmvMGeTlKk L0o66Jf2DITGh6sSBjcvlFAPhBn5HURm5UWLGVsUyIx+s+dqTLdI52XxXxmzqLQ= =zAkY -----END PGP SIGNATURE----- --i7F3eY7HS/tUJxUd--