From mboxrd@z Thu Jan 1 00:00:00 1970 From: Lukas Laukamp Subject: Re: [User Question] How to create a backup of an LVM based maschine without wasting space Date: Fri, 12 Oct 2012 12:50:40 +0200 Message-ID: <5077F600.7080809@laukamp.me> References: <5077BE30.5010209@laukamp.me> <20121012084221.GC14822@stefanha-thinkpad.redhat.com> <5077DB9F.6050809@laukamp.me> <5077E03E.2040507@laukamp.me> <5077EDEB.6050703@laukamp.me> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: Stefan Hajnoczi Return-path: Received: from mailer0.lippux.de ([5.9.218.245]:47995 "EHLO mailer0.lippux.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751362Ab2JLKup (ORCPT ); Fri, 12 Oct 2012 06:50:45 -0400 In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: Am 12.10.2012 12:47, schrieb Stefan Hajnoczi: > On Fri, Oct 12, 2012 at 12:16 PM, Lukas Laukamp wrote: >> Am 12.10.2012 12:11, schrieb Stefan Hajnoczi: >> >>> On Fri, Oct 12, 2012 at 11:17 AM, Lukas Laukamp wrote: >>>> Am 12.10.2012 10:58, schrieb Lukas Laukamp: >>>> >>>>> Am 12.10.2012 10:42, schrieb Stefan Hajnoczi: >>>>>> On Fri, Oct 12, 2012 at 08:52:32AM +0200, Lukas Laukamp wrote: >>>>>>> I have a simple user question. I have a few LVM based KVM guests and >>>>>>> wan't to backup them to files. The simple and nasty way would be to >>>>>>> create a complete output file with dd, which wastes very much space. >>>>>>> So I would like to create a backup of the LVM to a file which only >>>>>>> locates the space which is used on the LVM. Would be create when the >>>>>>> output file would be something like a qcow2 file which could be also >>>>>>> simply startet with KVM. >>>>>> If the VM is not running you can use "qemu-img convert": >>>>>> >>>>>> qemu-img convert -f raw -O qcow2 /dev/vg/vm001 vm001-backup.qcow2 >>>>>> >>>>>> Note that cp(1) tries to make the destination file sparse (see the >>>>>> --sparse option in the man page). So you don't need to use qcow2, you >>>>>> can use cp(1) to copy the LVM volume to a raw file. It will not use >>>>>> disk space for zero regions. >>>>>> >>>>>> If the VM is running you need to use LVM snapshots or stop the VM >>>>>> temporarily so a crash-consistent backup can be taken. >>>>>> >>>>>> Stefan >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>>> Hello Stefano, >>>>> >>>>> thanks for the fast reply. I will test this later. In my case now it >>>>> would >>>>> be a offline backup. For the online backup I think about a seperated >>>>> system >>>>> which every day makes incremental backups and once a week a full backup. >>>>> The >>>>> main problem is, that the systems are in a WAN network and I need >>>>> encryption >>>>> between the systems. Would it be possible to do something like this: >>>>> create >>>>> the LVM snapshot for the backup, read this LVM snapshot with the remote >>>>> backup system via ssh tunnel and save the output of this to qcow2 files >>>>> on >>>>> the backup system? And in which format could be the incremental backups >>>>> be >>>>> stored? >>> Since there is a WAN link it's important to use a compact image >>> representation before hitting the network. I would use qemu-img >>> convert -O qcow2 on the host and only transfer the qcow2 output. The >>> qcow2 file does not contain zero regions and will therefore save a lot >>> of network bandwidth compared to accessing the LVM volume over the >>> WAN. >>> >>> If you are using rsync or another tool it's a different story. You >>> could rsync the current LVM volume on the host over the last full >>> backup, it should avoid transferring image data which is already >>> present in the last full backup - the result is that you only transfer >>> changed data plus the rsync metadata. >>> >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> Hello Stefan, >> >> the rsync part I don't have understood fully. So to create a qcow2 on the >> host, transfer this to the backup server will result in the weekly full >> backup. So do you mean I could use rsync to read the LVM from the host, >> compare the LVM data with the data in the qcow2 on the backup server and >> simply transfer the differences to the file? Or does it work on another way? > When using rsync you can skip qcow2. Only two objects are needed: > 1. The LVM volume on the host. > 2. The last full backup on the backup client. > > rsync compares #1 and #2 efficiently over the network and only > transfers data from #1 which has changed. > > After rsync completes your full backup image is identical to the LVM > volume. Next week you can use it as the "last" image to rsync > against. > > Stefan > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html So I simply update the full backup, which is simply a raw file which get mounted while the backup? Best Regards