From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xr9l1-0000To-Cr for qemu-devel@nongnu.org; Wed, 19 Nov 2014 13:13:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xr9kw-0005kE-8G for qemu-devel@nongnu.org; Wed, 19 Nov 2014 13:13:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:17021) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xr9kv-0005kA-W9 for qemu-devel@nongnu.org; Wed, 19 Nov 2014 13:13:30 -0500 Message-ID: <546CDDC7.2070805@redhat.com> Date: Wed, 19 Nov 2014 11:13:27 -0700 From: Eric Blake MIME-Version: 1.0 References: <2A6E6B95B6E5C146ACE8440760E58185BDC31A5E@EXCHANGESERVER.schefczyk.local> <546C782E.9070803@redhat.com> <2A6E6B95B6E5C146ACE8440760E58185BDC31C30@EXCHANGESERVER.schefczyk.local> <546C8200.4070800@redhat.com> <2A6E6B95B6E5C146ACE8440760E58185BDC31C70@EXCHANGESERVER.schefczyk.local> <546C84C1.5000901@redhat.com> <2A6E6B95B6E5C146ACE8440760E58185BDC31CD0@EXCHANGESERVER.schefczyk.local> <546CAF2F.4090103@redhat.com> <546CBAB5.8050202@redhat.com> <2A6E6B95B6E5C146ACE8440760E58185BDC32099@EXCHANGESERVER.schefczyk.local> In-Reply-To: <2A6E6B95B6E5C146ACE8440760E58185BDC32099@EXCHANGESERVER.schefczyk.local> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9bCpupbidN35xhoglGVCv91NQa317SR26" Subject: Re: [Qemu-devel] "File too large" error from "qemu-img snapshot" (was Re: AW: Bug Repoting Directions Request) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Prof. Dr. Michael Schefczyk" , Paolo Bonzini , qemu-devel This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9bCpupbidN35xhoglGVCv91NQa317SR26 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/19/2014 10:32 AM, Prof. Dr. Michael Schefczyk wrote: > Dear Eric, dear all, >=20 > Again, thank you very much. I now gather that I took the wrong path tow= ards nightly backups of running VM. I remain surprised that I did work fo= r a relatively long time. [can you convince your mailer to wrap long lines? also, we tend to frown on top-posting on technical lists] Yeah, you were just getting lucky that two different processes weren't both trying to allocate a cluster for different purposes at the same time. When the collision finally did happen, it had catastrophic results on your image. >=20 > A major book on KVM in German language by Kofler & Spenneberg recommend= s the following approach for online backups (with and without "--disk-onl= y"): >=20 > virsh snapshot-create-as vm XXX --disk-only > qemu-img convert -f qcow2 -s XXX -O qcow2 XXX.img /backup/YYY.img > virsh snapshot-delete vm XXX Yes, virsh is using QMP commands under the hood, so this method is safe. One slight issue is that this sequence is incomplete (it does not shrink the backing file chain after the copy is complete), so if you keep repeating it, you will eventually cause reduced performance when you have a really long chain of multiple qcow2 overlays, or even cause qemu to run into fd exhaustion. Also, this command does not show that unless you clean things up, then the second time you run this you do not want to copy XXX.img into backup, but instead the qcow2 wrapper file that was created by the first snapshot (and which itself wrapped by the second snapshot). With new enough libvirt and qemu, you can shrink the chain back down with an active commit, as in: virsh blockcommit vm XXX --verbose --active --pivot Also, the use of --disk-only means that your disks have a snapshot taken as if at a point in time when you yank the power cord; reverting to such a backup may require fsck, and may suffer from other problems from anything that was depending on I/O that had not yet been flushed to disk. If you add the --quiesce option (which implies that you set up your guest to use qemu-guest-agent, and told libvirt to manage the agent), then you can guarantee that your guest has flushed and frozen all filesystems prior to the point in time where the snapshot is created; and you can even install hooks in the guest to extend that stability to things like databases. Then your backups are much easier to use. If you omit --disk-only, and take a full snapshot, then you have the guest memory state that is necessary to restore all pending I/O, and don't need to worry about freezing the guest file systems; but then you have to remember to back up the memory state in addition to your disk state. >=20 > Would this be any better than my script, because it uses virsh snapshot= -create-as instead of qemu-img snapshot? The second command is still qemu= -img convert which may be problematical. No, remember what I said. qemu-img may not be used on any image that is opened read-write by qemu, but is perfectly safe to do read-only operations on any image that is opened read-only by qemu. That sequence of commands goes from the initial: disk.img [read-write] then the snapshot-create command causes libvirt to issue a QMP command to switch qemu to: disk.img [read-only] <- overlay.qcow2 [read-write] at which point you can do anything read-only to disk.img (whether 'qemu-img convert' or 'cp' or any other command that doesn't alter the contents of the file) finally, the 'virsh blockcommit' command would take you back to: disk.img [read-write] >=20 > The problem I am facing is that the documentation is not easy to unders= tand for the average user/administrator who is not among the kvm develope= rs and experts. I have of course tried to read section 14.2.3 of RHEL 7 V= irtualization Deployment and Administration Guide on backups, but I found= that too cryptic for someone like myself to draw practical consequences = from it. If you are using libvirt to manage your guests, then yes, the qemu documentation is cryptic, but that shouldn't matter - you should be asking the libvirt community how to accomplish a job. And yes, libvirt could probably do a better job of advertising and documenting its level of support for live backups, but that is more a question for the libvirt-users@redhat.com mailing list (the archive of that list show that questions like this come up very regularly; you are not the first to ask about live backups, so perusing those archives may give you some ideas on what works). --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --9bCpupbidN35xhoglGVCv91NQa317SR26 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg iQEcBAEBCAAGBQJUbN3HAAoJEKeha0olJ0Nq8x4H/1+/uXVgAxD18LbGSnh3lTzi ZoKzkCNW9DEXT4AxAdN5oV3UVS8WaoMLu2jUunhBlcOSp55oWDDHixYHuNDn9OdK kvRlHk2rUzp0Ak7w+Jzn0v4wOpFm8BOa/mHvefC0xRqpTYIEg4zWrIAOWddwyHRG HwgE6o0LzBqFgOjNc4edpIwgzz+gp47L1TNALjxbGNSohet5y8yshqM7JiW+fTmq jS2EevWQ4Lzk8puKKWs0dYYfBxr+cfRHmMenrmDufIpl0OtpHfwksoaPv/J71rYV g+S4fYli56tAT7RcCNR2/obhEbm6I0JRybpS0xn1YJKO9V5Cn4PAun3dE1J2Tac= =FN1x -----END PGP SIGNATURE----- --9bCpupbidN35xhoglGVCv91NQa317SR26--