From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53791) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YSVOQ-0000uX-1A for qemu-devel@nongnu.org; Mon, 02 Mar 2015 13:48:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YSVON-0005bz-A2 for qemu-devel@nongnu.org; Mon, 02 Mar 2015 13:48:37 -0500 Message-ID: <54F4B080.1000406@redhat.com> Date: Mon, 02 Mar 2015 13:48:32 -0500 From: John Snow MIME-Version: 1.0 References: <1425084477-31602-1-git-send-email-jsnow@redhat.com> <1425084477-31602-2-git-send-email-jsnow@redhat.com> <54F4A2BF.8090304@redhat.com> In-Reply-To: <54F4A2BF.8090304@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH RESEND 01/17] docs: incremental backup documentation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Max Reitz , qemu-block@nongnu.org Cc: kwolf@redhat.com, famz@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, vsementsov@parallels.com, stefanha@redhat.com On 03/02/2015 12:49 PM, Max Reitz wrote: > On 2015-02-27 at 19:47, John Snow wrote: >> Signed-off-by: John Snow >> --- >> docs/bitmaps.md | 303 >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 303 insertions(+) >> create mode 100644 docs/bitmaps.md >> >> diff --git a/docs/bitmaps.md b/docs/bitmaps.md >> new file mode 100644 >> index 0000000..ebb6ae8 >> --- /dev/null >> +++ b/docs/bitmaps.md >> @@ -0,0 +1,303 @@ >> +# Dirty Bitmaps >> + >> +* Dirty bitmaps can be created at any time and attached to any node >> +(not just complete drives.) >> + >> +## Dirty Bitmap Names >> + >> +* A dirty bitmap's name is unique to the node, but bitmaps attached >> to different >> +nodes can share the same name. >> + >> +## Bitmap Modes >> + >> +* A Bitmap can be "enabled" (tracking writes, the default) or "disabled" >> +(read-only, I/O is ignored.) This state is currently only changed >> internally >> +for the purposes of migration, and otherwise remains enabled. >> + >> +* A Bitmap can be "frozen," which means that it is currently in-use >> by a backup >> +operation and cannot be deleted, enabled, disabled, renamed, written >> to, reset, >> +etc. >> + >> +## Basic QMP Usage >> + >> +### Supported Commands ### >> + >> +* block-dirty-bitmap-add >> +* block-dirty-bitmap-remove >> +* block-dirty-bitmap-clear >> + >> +### Creation >> + >> +* To create a new bitmap, enabled, on the drive with id=drive0: >> + >> +```json >> +{ "execute": "block-dirty-bitmap-add", >> + "arguments": { >> + "node": "drive0", >> + "name": "bitmap0" >> + } >> +} >> +``` >> + >> +* This bitmap will have a default granularity that matches the >> cluster size of >> +its associated drive, if available, clamped to between [4KiB, 64KiB]. >> +The current default for qcow2 is 64KiB. >> + >> +* To create a new bitmap that tracks changes in 32KiB segments: >> + >> +```json >> +{ "execute": "block-dirty-bitmap-add", >> + "arguments": { >> + "node": "drive0", >> + "name": "bitmap0", >> + "granularity": 32768 >> + } >> +} >> +``` >> + >> +### Deletion >> + >> +* Can be performed on a disabled bitmap, but not a frozen one. >> + >> +* Because bitmaps are only unique to the node to which they are >> attached, >> +you must specify the node/drive name here, too. >> + >> +```json >> +{ "execute": "block-dirty-bitmap-remove", >> + "arguments": { >> + "node": "drive0", >> + "name": "bitmap0" >> + } >> +} >> +``` >> + >> +### Resetting >> + >> +* Resetting a bitmap will clear all information it holds. >> +* An incremental backup created from an empty bitmap will copy no data, >> +as if nothing has changed. >> + >> +```json >> +{ "execute": "block-dirty-bitmap-clear", >> + "arguments": { >> + "node": "drive0", >> + "name": "bitmap0" >> + } >> +} >> +``` >> + >> +## Transactions (Not yet implemented) >> + >> +* Transactional commands are forthcoming in a future version, >> + and are not yet available for use. This section serves as >> + documentation of intent for their design and usage. >> + >> +### Justification >> +Bitmaps can be safely modified when the VM is paused or halted by using >> +the basic QMP commands. For instance, you might perform the following >> actions: >> + >> +1. Boot the VM in a paused state. >> +2. Create a full drive backup of drive0. >> +3. Create a new bitmap attached to drive0. >> +4. Resume execution of the VM. >> +5. Incremental backups are ready to be created. >> + >> +At this point, the bitmap and drive backup would be correctly in sync, >> +and incremental backups made from this point forward would be >> correctly aligned >> +to the full drive backup. >> + >> +This is not particularly useful if we decide we want to start >> incremental >> +backups after the VM has been running for a while, for which we will >> need to >> +perform actions such as the following: >> + >> +1. Boot the VM and begin execution. >> +2. Using a single transaction, perform the following operations: >> + * Create bitmap0. >> + * Create a full drive backup of drive0. >> +3. Incremental backups are now ready to be created. >> + >> +### Supported Bitmap Transactions >> + >> +* block-dirty-bitmap-add >> +* block-dirty-bitmap-clear >> + >> +The usages are identical to their respective QMP commands, but see below >> +for examples. >> + >> +### Example: New Incremental Backup >> + >> +As outlined in the justification, perhaps we want to create a new >> incremental >> +backup chain attached to a drive. >> + >> +```json >> +{ "execute": "transaction", >> + "arguments": { >> + "actions": [ >> + {"type": "block-dirty-bitmap-add", >> + "data": {"node": "drive0", "name": "bitmap0"} }, >> + {"type": "drive-backup", > > Note that this does not do a full drive-backup during the transaction > but only starts the block job. Above you said you'd do a single > transaction in which you'd "Create a full drive backup of drive0" (which > is not possible right now, however). > > So the problem is that any write to the node while the block job is > running will dirty the bitmap although they will be handled by the block > job (which is not bad, your bitmap will simply be dirtier than it needs > to be). I don't know exactly what Stefan proposed in regards to the > callbacks, but I can imagine two solutions to this: > It is my understanding that backup_run will register the backup_before_write_notify handler to back up any bits before they are written to, which means that the full backup makes a coherent backup _before_ any writes, and not a coherent backup _after_ all writes. The dirty bitmap that results from this should describe new writes that occurred during the backup, but aren't reflected in the backup data. > 1. You add transaction support for completing a block job and clearing a > dirty bitmap. Then you'd simply start the drive-backup job, add the > bitmap (both in any order), and then do a transaction of > block-job-complete and block-dirty-bitmap-clear. But making > block-job-complete safe for transactions can be difficult. > 2. You add a command to completely dirty a dirty bitmap (the opposite of > block-dirty-bitmap-clear). This way, you'd simply create a dirty bitmap, > mark everything dirty, and then you can start doing incremental backups > from there (so you'd force a full backup in bitmap mode). > > Maybe that helps. Maybe it doesn't. I don't know. > >> + "data": {"device": "drive0", "target": >> "/path/to/full_backup.img", >> + "sync": "full", "format": "qcow2"} } >> + ] >> + } >> +} >> +``` >> + >> +### Example: New Incremental Backup Anchor Point >> + >> +Maybe we just want to create a new full backup with an existing >> bitmap and >> +want to reset the bitmap to track the new chain. >> + >> +```json >> +{ "execute": "transaction", >> + "arguments": { >> + "actions": [ >> + {"type": "block-dirty-bitmap-clear", >> + "data": {"node": "drive0", "name": "bitmap0"} }, >> + {"type": "drive-backup", >> + "data": {"device": "drive0", "target": >> "/path/to/new_full_backup.img", >> + "sync": "full", "format": "qcow2"} } >> + ] >> + } >> +} >> +``` > > Same here, the dirty bitmap will be dirtier than it needs to be (not > really bad, but with the above suggestion we can do something about this). > >> + >> +## Incremental Backups >> + >> +The star of the show. > > I'm fine with a bit of fun in the documentation, but I'll see whether > others are, too. :-) > > (I fear that maybe just being able to imagine someone not allowing fun > is enough to reinforce the image of the no-fun-allowed German) > >> + >> +**Nota Bene!** Only incremental backups of entire drives are >> supported for now. >> +So despite the fact that you can attach a bitmap to any arbitrary >> node, they are >> +only currently useful when attached to the root node. This is because >> +drive-backup only supports drives/devices instead of arbitrary nodes. > > Well, not a real reason, since there's blockdev-backup, too. I'd just > omit this here. If people see that you're using drive-backup, they > should know that this can only be used for BlockBackends ("full drives"). > > You can leave it, what I don't like is just that it sounds like it'll be > really difficult to implement it for single BDS nodes as well; but it's > just a technical question of drive-backup vs. blockdev-backup. The core > of everything, block/backup.c, should be agnostic of whether you used > drive-backup or blockdev-backup, so support is actually there, it's only > the interface that's missing. > I didn't mean to imply it wouldn't ever be useful, it's just factually not useful yet. I can try to rephrase this to make it clearer why attaching bitmaps to arbitrary nodes is not yet a useful thing to do. >> + >> +### Example: First Incremental Backup >> + >> +1. Create a full backup and sync it to the dirty bitmap, as in the >> transactional >> +examples above; or with the VM offline, manually create a full copy >> and then >> +create a new bitmap before the VM begins execution. >> + >> + * Let's assume the full backup is named 'full_backup.img'. >> + * Let's assume the bitmap you created is 'bitmap0' attached to >> 'drive0'. >> + >> +2. Create a destination image for the incremental backup that >> utilizes the >> +full backup as a backing image. >> + >> + * Let's assume it is named 'incremental.0.img'. >> + >> + ```sh >> + # qemu-img create -f qcow2 incremental.0.img -b full_backup.img >> -F qcow2 >> + ``` > > Aha, using .img for qcow2. *g* > > Also, we really do need some sort of image-create for QMP at some point > in time, I guess... > It would be convenient, but not necessary. I am not sure what permission issues that would create for QEMU. Some installations may have QEMU quite locked down on the file creation front. >> + >> +3. Issue the incremental backup command: >> + >> + ```json >> + { "execute": "drive-backup", >> + "arguments": { >> + "device": "drive0", >> + "bitmap": "bitmap0", >> + "target": "incremental.0.img", >> + "format": "qcow2", >> + "sync": "dirty-bitmap", >> + "mode": "existing" >> + } >> + } >> + ``` >> + >> +### Example: Second Incremental Backup >> + >> +1. Create a new destination image for the incremental backup that >> points to the >> + previous one, e.g.: 'incremental.1.img' >> + >> + ```bash >> + # qemu-img create -f qcow2 incremental.1.img -b incremental.0.img >> -F qcow2 >> + ``` >> + >> +2. Issue a new incremental backup command. The only difference here >> is that we >> + have changed the target image below. >> + >> + ```json >> + { "execute": "drive-backup", >> + "arguments": { >> + "device": "drive0", >> + "bitmap": "bitmap0", >> + "target": "incremental.1.img", >> + "format": "qcow2", >> + "sync": "dirty-bitmap", >> + "mode": "existing" >> + } >> + } >> + ``` >> + >> +## Errors >> + >> +* In the event of an unsuccessful incremental backup, either via QMP >> or a >> + QMP transaction, the user will receive a BLOCK_JOB_COMPLETE event with >> + a failure message. > > Well, except for when the block job did not even start, but you could > argue that this is not an unsuccessful backup, because it isn't a backup > at all. > Worth elaborating, though. If synchronous part completes, you'll get a message. The synchronous portion might fail and you will not get an event. >> + >> +* In this case, the incremental backup data contained within the >> bitmap is >> + safely rolled back, and the data within the bitmap is not lost. >> + >> +* Once the underlying problem is fixed (e.g. more storage space is >> freed up), >> + you can simply retry the incremental backup command with the same >> bitmap. >> + >> +### Example >> + >> +1. Attempt to create an incremental backup. >> + >> + ```sh >> + # qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F >> qcow2 >> + ``` >> + >> + ```json >> + { "execute": "drive-backup", >> + "arguments": { >> + "device": "drive0", >> + "bitmap": "bitmap0", >> + "target": "incremental.0.img", >> + "format": "qcow2", >> + "sync": "dirty-bitmap", >> + "mode": "existing" >> + } >> + } >> + ``` > > This looks funny outside of markdown (like if converted to html). It > looks like you're feeding JSON data to qemu-img. Maybe you should > explicitly note that this is to be used over QMP while the qemu-img > command is supposed to be executed outside of qemu (I know it's not > ambiguous, but it would be more clear nonetheless). > OK. > Oh, and by the way: At least pandoc behaves differently depending on > whether the ``` block is indented or not. If it's indented, line break > will not be present in the output, but the language name ("json", "sh", > etc.) will be present there. > I might just convert to plaintext, then. I'm not familiar enough with markdown flavors to make something that works and looks nice everywhere. I just like it for providing some kind of visual structure. >> + >> +2. Receive an event notifying us of failure: >> + >> + ```json >> + { "timestamp": { "seconds": 1424709442, "microseconds": 844524 }, >> + "data": { "speed": 0, "offset": 0, "len": 67108864, >> + "error": 'No space left on device', > > Single quotes? Did you type this yourself? ;-) > Printed via a different debugging utility :) >> + "device": "drive1", "type": "backup" }, >> + "event": "BLOCK_JOB_COMPLETED" } >> + ``` >> + >> +3. Delete the failed incremental, and re-create the image. >> + >> + ```sh >> + # rm incremental.0.img >> + # qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F >> qcow2 >> + ``` >> + >> +4. Retry the command after fixing the underlaying problem, >> + such as freeing up space on the backup volume: >> + >> + ```json >> + { "execute": "drive-backup", >> + "arguments": { >> + "device": "drive0", >> + "bitmap": "bitmap0", >> + "target": "incremental.0.img", >> + "format": "qcow2", >> + "sync": "dirty-bitmap", >> + "mode": "existing" >> + } >> + } >> + ``` >> + >> + ```json >> + { "timestamp": { "seconds": 1424709668, "microseconds": 526525 }, >> + "data": { "device": "drive1", "type": "backup", >> + "speed": 0, "len": 67108864, "offset": 67108864}, >> + "event": "BLOCK_JOB_COMPLETED" } >> + ``` > > The second time stamp is greater than the first one, good. (I had to > make sure you don't have a time machine or something (but if you did, > that'd be great (do you?))) > > Max You reviewed my hypothetical timestamps? :)