From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42696) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aTDuo-0002sv-Vh for qemu-devel@nongnu.org; Tue, 09 Feb 2016 14:25:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aTDuk-0004hw-P9 for qemu-devel@nongnu.org; Tue, 09 Feb 2016 14:25:34 -0500 Received: from mx2.parallels.com ([199.115.105.18]:36996) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aTDuk-0004hi-Dn for qemu-devel@nongnu.org; Tue, 09 Feb 2016 14:25:30 -0500 References: <1454151394-52320-1-git-send-email-vsementsov@virtuozzo.com> <20160203081418.GC25746@ad.usersys.redhat.com> <56B45D3A.405@virtuozzo.com> <20160209142100.GA19303@stefanha-x1.localdomain> <56B9F9B0.8010801@virtuozzo.com> <56BA1888.7050308@redhat.com> <56BA1AAB.7040504@virtuozzo.com> <56BA2BFE.7070001@redhat.com> From: "Denis V. Lunev" Message-ID: <56BA3D1E.8090706@virtuozzo.com> Date: Tue, 9 Feb 2016 22:25:18 +0300 MIME-Version: 1.0 In-Reply-To: <56BA2BFE.7070001@redhat.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v2 0/6] external backup api List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: John Snow , Stefan Hajnoczi Cc: kwolf@redhat.com, Vladimir Sementsov-Ogievskiy , Fam Zheng , qemu-devel@nongnu.org, armbru@redhat.com On 02/09/2016 09:12 PM, John Snow wrote: > > On 02/09/2016 11:58 AM, Denis V. Lunev wrote: >> On 02/09/2016 07:49 PM, John Snow wrote: >>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote: >>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote: >>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote: >>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote: >>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote: >>>>>>>> Hi all. >>>>>>>> >>>>>>>> These series which aims to add external backup api. This is needed >>>>>>>> to allow >>>>>>>> backup software use our dirty bitmaps. >>>>>>>> >>>>>>>> Vmware and Parallels Cloud Server have this feature. >>>>>>> What is the advantage of this appraoch over "drive-backup >>>>>>> sync=incremental >>>>>>> ..."? >>>>>> This will allow third-party vendors to backup QEMU VMs into >>>>>> their own formats or to the cloud etc. >>>>> Backup software can implement NBD to receive the incremental blocks >>>>> from >>>>> QEMU. Use drive-backup sync=incremental and the backup appliance as >>>>> the >>>>> NBD target. >>>>> >>>>> It's more complicated to add this QMP command flow to backup software >>>>> than to implement NBD. >>>>> >>>>> Stefan >>>> it can, but this is a matter of problem due to the nature of >>>> how this software is implemented. Usually it is written >>>> in a semi-standard way and it uses "plugins" to actually >>>> collect the data, i.e. the code is written in standard >>>> interface/real implementation pattern and interfaces are >>>> basically the same. >>>> >>>> With this standard approach backup software is working >>>> as an active side of the process, i.e. it performs operations >>>> and controls the flow. >>>> >>>> This means that "non-standard" QEMU technology will be >>>> pain here. >>>> >>>> Den >>> Am I to understand that for e.g. VMWare the backup appliance is >>> literally reading the disk image from storage directly while the VM is >>> running? >>> >>> I'd be a bit surprised if that were true. >> I think that backup software is asking alive VM about the data. >> >>> My biggest concern here is that there is not a safe way, today, to read >>> from a disk image atomically while the VM is running. I think that'd >>> take a lot of work to do and you'll not find a lot of support in >>> implementing it. >>> >>> Of course, while the VM is paused/off is a different story, but for now >>> I still feel like NBD is the right answer for getting block data from >>> QEMU. >>> >>> What am I missing? >>> >>> --js >> In general, in Parallels Server the backup was created using the >> following approach: >> - create external snapshot. In this case the base image (backing store >> in QEMU terminology) >> will be READ-ONLY and could be safely read by any entity >> - backup that read-only disk image (any way) >> - merge snapshots >> > I see. > >> In this process backup software is active while PCS is passive. > PCS? Parallels Cloud Server. Sorry for abbreviation :( >> With QEMU the approach looks the same to me: >> - start a backup >> - ask QEMU to give a data to be backuped (using NBD server in QEMU >> with Fam's image fleecing) >> - finish backup >> >> Important bit here is that dirty bitmap should be provided >> by QEMU by request. This dirty bitmap will be read-only at that >> moment, current active dirty bitmap should be replaced with >> new at backup start operation. >> >> Den > I don't have any problems providing the bitmap data through an external > API, but the part I want to be 100% clear on before I ACK it is the API > portion where we allow an external client to split or merge bitmaps > externally -- that's functionality you don't need if you query the data > from QEMU itself. That is fine :) > If using NBD (either through the incremental backup tool or image > fleecing) that is completely fine, too. > > My only reservations are over making direct reads to image files outside > of QEMU while the VM is running. Perfect :) We are on the same page. I would not going to go something unsafe is this very sensitive area. Den