Re: [Qemu-devel] RFC: moving fsfreeze support from the userland guest agent to the guest kernel

[Anthony Liguori <anthony@codemonkey.ws>]

On 07/27/2011 10:24 AM, Andrea Arcangeli wrote:
> Hello everyone,
>
> I've been thinking at the current design of the fsfreeze feature used
> by libvirt.
>
> It currently relays on an userland agent in the guest talking to qemu
> with some vmchannel communication. The guest agent would walk the
> filesystems in the guest and call fsfreeze ioctl on them.
>
> The fsfreeze is an optional feature, it's not required to do safe
> snapshots, after fsfreeze (regardless if available or not) QEMU must
> still block all I/O for all qemu blkdevices before the image is saved,
> to allow safe snapshotting of non-linux guests. Then if a VM is
> restarted in the snapshot it becomes identical to a fault tolerance
> fallback with nfs or drdb in a highly available
> configuration. Fsfreeze just provides some further (minor) benefit on
> top of that (which probably won't be available for non-linux guests
> any time soon).
>
> The benefits this optional fsfreeze feature provides to the snapshot
> are:
>
> 1) more peace of mind by not relaying on the kernel journal reply code
> when snapshotting journaled/cow filesystems like ext4/btrfs/xfs
>
> 2) all dirty outstanding cache is flushed, which reduces the chances
> of running into userland journaling data reply bugs if userland is
> restarted on the snapshot
>
> 3) allows safe live snapshotting of not jorunaled fs like vfat/ext2 on
> linux (not so common, and vfat on non-linux guest won't benefit)
>
> 4) allows to mount the snapshotted image readonly without requiring
> metadata journal reply
>
> Problem is that having a daemon in guest userland is not my
> preference, considering it can be done with a virtio-fsfreeze.ko
> kernel module in guest without requiring any userland modification to
> the guest (and no interprocess communication through vmchannel
> or similar way).
>
> This means a kernel upgrade in the guest that adds the
> virtio-fsfreeze.ko virtio paravirt driver would be enough to be able
> to provide fsfreeze during snapshots.
>
> A virtio-fsfreeze.ko would certainly be more developer friendly, you
> could just build the kernel and even boot it with -kernel bzImage
> (after building it with VIRTIO_FSFREEZE=y). Then it'd just work
> without any daemon or vmchannel or any other change to the guest
> userland.
>
> I could see some advantage in not having to modify qemu if libvirt was
> talking directly to the guest agent, so to avoid any knowledge into
> qemu about FSFREEZE. But it's not even like that, I see FSFREEZE guest
> agent patches floating around. So if qemu has to be modified and be
> aware of the fsfreeze feature in the userland guest agent (and not
> just asked to block all I/O which doesn't require any guest knowledge
> and in turn it'd remain agnostic about fsfreeze) I think it'd be
> better if the fsfreeze qemu code would just go into a virtio backend.

Currently, QEMU doesn't know about fsfreeze.  I don't think it ever will 
either.

> I understand an agent may be needed for other features but I think
> whenever a feature is better suited for not requiring userland guest
> support, it shouldn't. To me requiring modifications to the guest
> userland, looks the least transparent and most intrusive possible way
> to implement a libvirt feature so it should be used when it has
> advantages and I see mostly disadvantages here.

I also dislike having orchestrate all of the freezing stuff because it's 
extremely hard in userspace to do it reliably.

One challenge though is that it's highly desirable to have script hooks 
as part of the freeze process to let other userspace applications 
participate which means you will always need some userspace daemon to 
kick things off.

Instead of having a virtio-fsfreeze, I think it would be better to think 
about if the kernel needs a higher level interface such that the 
userspace operation is dirt-simple.

But I don't see a way to avoid userspace involvement in this set of 
operations unfortunately.

Regards,

Anthony Liguori

> This is just a suggestions, I think the agent should work too.
>
> Thanks a lot,
> Andrea