consistent LVM snapshot of domUs from dom0

consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hello,
I'd like to ask a question regarding LVM snapshots:
normally if You make a snapshot of mounted LVM partition, DM 
infrastructure handles putting fs into consistent state using VFS 
infrastructure. BUT, does it work even if I make snapshot of LV used by 
XEN? 
If not, I guess it should not be that hard to fix, I guess we just need to 
notify domU kernel to call needed VFS function to put fs into consistent 
state prior to creating snapshot.
Can somebody comment on it, eventually correct me if I'm wrong?
thanks a lot in advance
regards
nik


-- 

consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hello,
I'd like to ask a question regarding LVM snapshots:
normally if You make a snapshot of mounted LVM partition, DM
infrastructure handles putting fs into consistent state using VFS
infrastructure. BUT, does it work even if I make snapshot of LV used by
XEN?
If not, I guess it should not be that hard to fix, I guess we just need to
notify domU kernel to call needed VFS function to put fs into consistent
state prior to creating snapshot.
Can somebody comment on it, eventually correct me if I'm wrong?
thanks a lot in advance
regards
nik

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> I'd like to ask a question regarding LVM snapshots:
> normally if You make a snapshot of mounted LVM partition, DM
> infrastructure handles putting fs into consistent state using VFS
> infrastructure. BUT, does it work even if I make snapshot of LV used by
> XEN?

If your LVM is running in dom0 and you snapshot the storage used by a domU 
then, no, it won't snapshot the domU's filesystem automatically.

> If not, I guess it should not be that hard to fix, I guess we just need to
> notify domU kernel to call needed VFS function to put fs into consistent
> state prior to creating snapshot.

Yep.

> Can somebody comment on it, eventually correct me if I'm wrong?

You're quite right.  I've just had a look at it and it looks quite feasible to 
do this.  The obvious simple way to implement it would be to add an xm 
command that causes a domain to freeze its filesystems into a consistent 
state and then return when that is done.

e.g.

xm create mydomain
xm fs_freeze mydomain
<do backup>
xm fs_unfreeze mydomain

Ideally the backup operation would just take a snapshot e.g. using LVM or 
qcow.  With a bit more tools / storage integration we could have an "xm 
safesnapshot" command that would create a coherent snapshot of the disk.

It would be awesome.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Mike Sun]

Thanks for the question Nikola and the answer Mark, I was wondering
the same thing.

Here's another scenario.  What if dom0 has a mounted LVM filesystem,
and a file-backed VBD located on that filesystem is used as the root
filesystem by a domU.  Would LVM snapshots of the mounted LVM
filesystem provide consistent checkpoints?

In looking at how xm suspend works, it seems that only memory state is
saved and nothing is done to ensure the guest domain puts its
filesystem into a consistent state, though I'm not completely certain
about that.

Mike

On Dec 8, 2007 1:26 PM, Mark Williamson <mark.williamson@cl.cam.ac.uk> wrote:
> > I'd like to ask a question regarding LVM snapshots:
> > normally if You make a snapshot of mounted LVM partition, DM
> > infrastructure handles putting fs into consistent state using VFS
> > infrastructure. BUT, does it work even if I make snapshot of LV used by
> > XEN?
>
> If your LVM is running in dom0 and you snapshot the storage used by a domU
> then, no, it won't snapshot the domU's filesystem automatically.
>
> > If not, I guess it should not be that hard to fix, I guess we just need to
> > notify domU kernel to call needed VFS function to put fs into consistent
> > state prior to creating snapshot.
>
> Yep.
>
> > Can somebody comment on it, eventually correct me if I'm wrong?
>
> You're quite right.  I've just had a look at it and it looks quite feasible to
> do this.  The obvious simple way to implement it would be to add an xm
> command that causes a domain to freeze its filesystems into a consistent
> state and then return when that is done.
>
> e.g.
>
> xm create mydomain
> xm fs_freeze mydomain
> <do backup>
> xm fs_unfreeze mydomain
>
> Ideally the backup operation would just take a snapshot e.g. using LVM or
> qcow.  With a bit more tools / storage integration we could have an "xm
> safesnapshot" command that would create a coherent snapshot of the disk.
>
> It would be awesome.
>
> Cheers,
> Mark
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hello Mark,
thanks very very much for a reply, I was beginning to feel a bit sad that
nobody replied :)

On Sat, 8 Dec 2007, Mark Williamson wrote:

> If your LVM is running in dom0 and you snapshot the storage used by a domU
> then, no, it won't snapshot the domU's filesystem automatically.
OK, good to know there is a problem to solve :)

> You're quite right.  I've just had a look at it and it looks quite feasible to
> do this.  The obvious simple way to implement it would be to add an xm
> command that causes a domain to freeze its filesystems into a consistent
> state and then return when that is done.
> e.g.
>
> xm create mydomain
> xm fs_freeze mydomain
> <do backup>
> xm fs_unfreeze mydomain
Yes, I really like this way, and now the question is: do You think, 
that You could have a time to look at it in near future? Or should I try 
to solve it? I do not have knowledge of xen internals to do it (yet?), but 
I'm trying to learn anyways, so maybe I could try to fix, if there would 
be somebody to give me some guidance ;)
What do you think?

Anyways thanks a lot for Your time once more
nik

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi Mike,
I think that this problem can be solved using procedure Mark proposed - 
freeze filesystems of all running domUs using some xm command, then create 
snapshot and unfreeze them all again. I think that You won't need suspend 
at all.
We just now need to implement it :)
n.

  On Sat, 8 Dec 2007, Mike Sun wrote:

> Thanks for the question Nikola and the answer Mark, I was wondering
> the same thing.
>
> Here's another scenario.  What if dom0 has a mounted LVM filesystem,
> and a file-backed VBD located on that filesystem is used as the root
> filesystem by a domU.  Would LVM snapshots of the mounted LVM
> filesystem provide consistent checkpoints?
>
> In looking at how xm suspend works, it seems that only memory state is
> saved and nothing is done to ensure the guest domain puts its
> filesystem into a consistent state, though I'm not completely certain
> about that.
>
> Mike
>
> On Dec 8, 2007 1:26 PM, Mark Williamson <mark.williamson@cl.cam.ac.uk> wrote:
>>> I'd like to ask a question regarding LVM snapshots:
>>> normally if You make a snapshot of mounted LVM partition, DM
>>> infrastructure handles putting fs into consistent state using VFS
>>> infrastructure. BUT, does it work even if I make snapshot of LV used by
>>> XEN?
>>
>> If your LVM is running in dom0 and you snapshot the storage used by a domU
>> then, no, it won't snapshot the domU's filesystem automatically.
>>
>>> If not, I guess it should not be that hard to fix, I guess we just need to
>>> notify domU kernel to call needed VFS function to put fs into consistent
>>> state prior to creating snapshot.
>>
>> Yep.
>>
>>> Can somebody comment on it, eventually correct me if I'm wrong?
>>
>> You're quite right.  I've just had a look at it and it looks quite feasible to
>> do this.  The obvious simple way to implement it would be to add an xm
>> command that causes a domain to freeze its filesystems into a consistent
>> state and then return when that is done.
>>
>> e.g.
>>
>> xm create mydomain
>> xm fs_freeze mydomain
>> <do backup>
>> xm fs_unfreeze mydomain
>>
>> Ideally the backup operation would just take a snapshot e.g. using LVM or
>> qcow.  With a bit more tools / storage integration we could have an "xm
>> safesnapshot" command that would create a coherent snapshot of the disk.
>>
>> It would be awesome.
>>
>> Cheers,
>> Mark
>>
>> --
>> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
>> Mark: To answer a question with a question: What use is a skateboard?
>> Dave: Skateboards have wheels.
>> Mark: My wheel has a wheel!
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Stefan de Konink]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Nikola Ciprich schreef:
> Hi Mike,
> I think that this problem can be solved using procedure Mark proposed -
> freeze filesystems of all running domUs using some xm command, then
> create snapshot and unfreeze them all again. I think that You won't need
> suspend at all.
> We just now need to implement it :)


Just a question about unsafely. I have implemented iscsi and zfs as
combination with Xen on Linux. Is the only safe procedure:

pause
memory snapsnot
[send out disk snapshot]
resume


Stefan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHWxFIYH1+F2Rqwn0RCiT7AKCBAnfW4E0M8pLjHKZUHOOX0mwoZwCfcfY3
Rns2S2AETOMypqLDmRnljx4=
=ADlt
-----END PGP SIGNATURE-----

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi Mike, well, I maybe don't understand where the question is ;) but I 
guess that Your procedure is perfectly correct (for creation of a 
checkpoint of running system), if You want to restore the backup later, 
You restore system into running state, so a filesystem is in state 
consistent with running system....
n.


  On Sat, 8 Dec 2007, Stefan de Konink wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Nikola Ciprich schreef:
>> Hi Mike,
>> I think that this problem can be solved using procedure Mark proposed -
>> freeze filesystems of all running domUs using some xm command, then
>> create snapshot and unfreeze them all again. I think that You won't need
>> suspend at all.
>> We just now need to implement it :)
>
>
> Just a question about unsafely. I have implemented iscsi and zfs as
> combination with Xen on Linux. Is the only safe procedure:
>
> pause
memory snapsnot
> [send out disk snapshot]
> resume
>
>
> Stefan
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.7 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFHWxFIYH1+F2Rqwn0RCiT7AKCBAnfW4E0M8pLjHKZUHOOX0mwoZwCfcfY3
> Rns2S2AETOMypqLDmRnljx4=
> =ADlt
> -----END PGP SIGNATURE-----
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mike Sun]

Ok... I see where you're coming from Nikola... you're only concerned
with safe, consistent snapshots of the filesystem state without
checkpointing the running vm state.  Yes, implementing a special xm
fssnapshot command would be the way to go.  I was just curious if
there were any other pitfalls given my approach of checkpointing the
entire VM with a suspend followed by an lvm snapshot.  Thanks for the
clarifications.  I'm actually considering  an attempt to implement the
COW memory functionality that's been discussed on this list in the
past.

Mike

On Dec 8, 2007 5:17 PM, Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
> Hi Mike, well, I maybe don't understand where the question is ;) but I
> guess that Your procedure is perfectly correct (for creation of a
> checkpoint of running system), if You want to restore the backup later,
> You restore system into running state, so a filesystem is in state
> consistent with running system....
> n.
>
>
>
>   On Sat, 8 Dec 2007, Stefan de Konink wrote:
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > Nikola Ciprich schreef:
> >> Hi Mike,
> >> I think that this problem can be solved using procedure Mark proposed -
> >> freeze filesystems of all running domUs using some xm command, then
> >> create snapshot and unfreeze them all again. I think that You won't need
> >> suspend at all.
> >> We just now need to implement it :)
> >
> >
> > Just a question about unsafely. I have implemented iscsi and zfs as
> > combination with Xen on Linux. Is the only safe procedure:
> >
> > pause
> memory snapsnot
> > [send out disk snapshot]
> > resume
> >
> >
> > Stefan
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.7 (GNU/Linux)
> > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> >
> > iD8DBQFHWxFIYH1+F2Rqwn0RCiT7AKCBAnfW4E0M8pLjHKZUHOOX0mwoZwCfcfY3
> > Rns2S2AETOMypqLDmRnljx4=
> > =ADlt
> > -----END PGP SIGNATURE-----
> >
> >
>
> --
>
>

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

Hi there,

> thanks very very much for a reply, I was beginning to feel a bit sad that
> nobody replied :)

No problem!

> > xm create mydomain
> > xm fs_freeze mydomain
> > <do backup>
> > xm fs_unfreeze mydomain
>

Actually, an even better way (with better tools integration) would be:

xm create mydomain

xm fs_snapshot mydomain
  +---> freezes domain filesystem
  +---> does a block level snapshot with LVM or qcow
  +---> unfreezes domain filesystem

<user now backs up the snapshot at their leisure>

> Yes, I really like this way, and now the question is: do You think,
> that You could have a time to look at it in near future? Or should I try 
> to solve it? I do not have knowledge of xen internals to do it (yet?), but
> I'm trying to learn anyways, so maybe I could try to fix, if there would
> be somebody to give me some guidance ;)

It's mostly a question of Linux kernel hacking, I think, plus adding support 
to the Xen tools.  Xen itself would definitely not need to be touched.

I've actually been having a look at this today and trying to code up a little 
example implementation.  You're welcome to have a go too if you want, 
particularly if you're into kernel hacking.  But I'll put my code online if I 
get anywhere, anyhow.

> What do you think?

Lots of people seem to be interested in this sort of thing, so I think it's 
worth looking into.

Cheers,
Mark



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> Thanks for the question Nikola and the answer Mark, I was wondering
> the same thing.
>
> Here's another scenario.  What if dom0 has a mounted LVM filesystem,
> and a file-backed VBD located on that filesystem is used as the root
> filesystem by a domU.  Would LVM snapshots of the mounted LVM
> filesystem provide consistent checkpoints?

I believe the filesystem on the LVM volume should be consistent (as long as 
you're using one of the common Linux filesystems which can be frozen for a 
snapshot to take place).  However, the guest filesystem in the file-backed 
VBD will not be consistent: if you try to mount that VBD it will be in an 
unclean and possibly inconsistent state.

The only way you can guarantee a consistent view of a guest filesystem is with 
the guest's co-operation, either by using a network or cluster filesystem, or 
(as Nik has suggested) by explicitly asking for the domain to freeze its 
filesystem state for a while.

Cheers,
Mark

> In looking at how xm suspend works, it seems that only memory state is
> saved and nothing is done to ensure the guest domain puts its
> filesystem into a consistent state, though I'm not completely certain
> about that.
>
> Mike
>
> On Dec 8, 2007 1:26 PM, Mark Williamson <mark.williamson@cl.cam.ac.uk> 
wrote:
> > > I'd like to ask a question regarding LVM snapshots:
> > > normally if You make a snapshot of mounted LVM partition, DM
> > > infrastructure handles putting fs into consistent state using VFS
> > > infrastructure. BUT, does it work even if I make snapshot of LV used by
> > > XEN?
> >
> > If your LVM is running in dom0 and you snapshot the storage used by a
> > domU then, no, it won't snapshot the domU's filesystem automatically.
> >
> > > If not, I guess it should not be that hard to fix, I guess we just need
> > > to notify domU kernel to call needed VFS function to put fs into
> > > consistent state prior to creating snapshot.
> >
> > Yep.
> >
> > > Can somebody comment on it, eventually correct me if I'm wrong?
> >
> > You're quite right.  I've just had a look at it and it looks quite
> > feasible to do this.  The obvious simple way to implement it would be to
> > add an xm command that causes a domain to freeze its filesystems into a
> > consistent state and then return when that is done.
> >
> > e.g.
> >
> > xm create mydomain
> > xm fs_freeze mydomain
> > <do backup>
> > xm fs_unfreeze mydomain
> >
> > Ideally the backup operation would just take a snapshot e.g. using LVM or
> > qcow.  With a bit more tools / storage integration we could have an "xm
> > safesnapshot" command that would create a coherent snapshot of the disk.
> >
> > It would be awesome.
> >
> > Cheers,
> > Mark
> >
> > --
> > Dave: Just a question. What use is a unicyle with no seat?  And no
> > pedals! Mark: To answer a question with a question: What use is a
> > skateboard? Dave: Skateboards have wheels.
> > Mark: My wheel has a wheel!
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> Just a question about unsafely. I have implemented iscsi and zfs as
> combination with Xen on Linux. Is the only safe procedure:
>
> pause
> memory snapsnot
> [send out disk snapshot]
> resume

As long as you've got a reasonably recent version of Xen, I think that should 
work fine.

I guess you're using a Solaris as a guest and Linux as a host?  I'm not so 
familiar with Solaris or zfs...

However, I can perhaps be a little speculative.  Since zfs is 
transaction-based you probably don't actually *have* to pause the guest 
before you back it up...  If you were to LVM snapshot the VBD from dom0, then 
the resulting snapshot would probably still be a mountable ZFS volume, it 
would just be as if the power had been cut off to the VM at that point.  You 
would lose cached and in-memory data (I guess there might be incomplete 
writes to files, but I'm not even sure that'd be the case).  There'd be no 
guarantee that the data are consistent at the application level, but there's 
no general way to ensure consistency at application level in a running 
machine anyhow.

In summary: you might be able to get away with not taking the memory snapshot, 
based on my understanding of ZFS (which is a bit vague).  You could 
experiment with this, but really it's not going to take long to do a memory 
snapshot too and that seems the safest (works regardless of FS 
implementation) and most complete (won't lose any data, even cached and 
in-memory state).

I hope that helps a bit, sorry for hand-waving so much but I don't have 
specific experience with what you're doing!

Cheercs,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> Ok... I see where you're coming from Nikola... you're only concerned
> with safe, consistent snapshots of the filesystem state without
> checkpointing the running vm state.  Yes, implementing a special xm
> fssnapshot command would be the way to go.  I was just curious if
> there were any other pitfalls given my approach of checkpointing the
> entire VM with a suspend followed by an lvm snapshot.  Thanks for the
> clarifications.  I'm actually considering  an attempt to implement the
> COW memory functionality that's been discussed on this list in the
> past.

In an earlier e-mail you also said:
>
> In looking at how xm suspend works, it seems that only memory state is
> saved and nothing is done to ensure the guest domain puts its
> filesystem into a consistent state, though I'm not completely certain
> about that.
>

xm suspend doesn't put the filesystem into a consistent state, so the 
filesystem will not be clean and may be inconsistent at the time.  This is 
just because the guest may still be caching information that it hasn't yet 
written back to disk.  One *could* get it to make the filesystem consistent, 
but that would slow down the suspend so I think it's really an orthogonal 
problem.

As long as you have the contents of the block device *and* the suspend image, 
you can resume the domain and it'll still be able to access the FS because it 
has all the cached data needed to make sense of the image.

If you don't keep the suspend image with your backup then you won't 
necessarily be able to mount the filesystem properly (although you can 
probably repair it with fsck - but that isn't really ideal for a backup 
though!).

The ability to freeze the filesystem into a consistent state, create an LVM or 
qcow snapshot and then unfreeze (potentially all without pausing the domain!) 
would be handy for a zero downtime backup operation.

Cheers,
Mark

>
> Mike
>
> On Dec 8, 2007 5:17 PM, Nikola Ciprich <extmaillist@linuxbox.cz> wrote:
> > Hi Mike, well, I maybe don't understand where the question is ;) but I
> > guess that Your procedure is perfectly correct (for creation of a
> > checkpoint of running system), if You want to restore the backup later,
> > You restore system into running state, so a filesystem is in state
> > consistent with running system....
> > n.
> >
> >   On Sat, 8 Dec 2007, Stefan de Konink wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA512
> > >
> > > Nikola Ciprich schreef:
> > >> Hi Mike,
> > >> I think that this problem can be solved using procedure Mark proposed
> > >> - freeze filesystems of all running domUs using some xm command, then
> > >> create snapshot and unfreeze them all again. I think that You won't
> > >> need suspend at all.
> > >> We just now need to implement it :)
> > >
> > > Just a question about unsafely. I have implemented iscsi and zfs as
> > > combination with Xen on Linux. Is the only safe procedure:
> > >
> > > pause
> >
> > memory snapsnot
> >
> > > [send out disk snapshot]
> > > resume
> > >
> > >
> > > Stefan
> > >
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v2.0.7 (GNU/Linux)
> > > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> > >
> > > iD8DBQFHWxFIYH1+F2Rqwn0RCiT7AKCBAnfW4E0M8pLjHKZUHOOX0mwoZwCfcfY3
> > > Rns2S2AETOMypqLDmRnljx4=
> > > =ADlt
> > > -----END PGP SIGNATURE-----
> >
> > --



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[tgh]

hi
if we use the ParallaxFS to do the filesystem snapshot, does it have the 
similar problem ,or not? could you clear me?

Thanks in advance


Mark Williamson 写道:
>> I'd like to ask a question regarding LVM snapshots:
>> normally if You make a snapshot of mounted LVM partition, DM
>> infrastructure handles putting fs into consistent state using VFS
>> infrastructure. BUT, does it work even if I make snapshot of LV used by
>> XEN?
>>     
>
> If your LVM is running in dom0 and you snapshot the storage used by a domU 
> then, no, it won't snapshot the domU's filesystem automatically.
>
>   
>> If not, I guess it should not be that hard to fix, I guess we just need to
>> notify domU kernel to call needed VFS function to put fs into consistent
>> state prior to creating snapshot.
>>     
>
> Yep.
>
>   
>> Can somebody comment on it, eventually correct me if I'm wrong?
>>     
>
> You're quite right.  I've just had a look at it and it looks quite feasible to 
> do this.  The obvious simple way to implement it would be to add an xm 
> command that causes a domain to freeze its filesystems into a consistent 
> state and then return when that is done.
>
> e.g.
>
> xm create mydomain
> xm fs_freeze mydomain
> <do backup>
> xm fs_unfreeze mydomain
>
> Ideally the backup operation would just take a snapshot e.g. using LVM or 
> qcow.  With a bit more tools / storage integration we could have an "xm 
> safesnapshot" command that would create a coherent snapshot of the disk.
>
> It would be awesome.
>
> Cheers,
> Mark
>
>   

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi,
I don't know anything about parallaxfs (and neither does google it seems), 
but I guess the problem will be exactly the same. If it won't give the 
domU command to put fs into consistent state, snapshot won't be 
consistent.
hopefully I'll manage to create a fix Mark proposed...
cheers
n.

  On Sat, 15 Dec 2007, tgh wrote:

> hi
> if we use the ParallaxFS to do the filesystem snapshot, does it have the 
> similar problem ,or not? could you clear me?
>
> Thanks in advance
>
>
> Mark Williamson ??:
>>> I'd like to ask a question regarding LVM snapshots:
>>> normally if You make a snapshot of mounted LVM partition, DM
>>> infrastructure handles putting fs into consistent state using VFS
>>> infrastructure. BUT, does it work even if I make snapshot of LV used by
>>> XEN?
>>> 
>> 
>> If your LVM is running in dom0 and you snapshot the storage used by a domU 
>> then, no, it won't snapshot the domU's filesystem automatically.
>>
>> 
>>> If not, I guess it should not be that hard to fix, I guess we just need to
>>> notify domU kernel to call needed VFS function to put fs into consistent
>>> state prior to creating snapshot.
>>> 
>> 
>> Yep.
>>
>> 
>>> Can somebody comment on it, eventually correct me if I'm wrong?
>>> 
>> 
>> You're quite right.  I've just had a look at it and it looks quite feasible 
>> to do this.  The obvious simple way to implement it would be to add an xm 
>> command that causes a domain to freeze its filesystems into a consistent 
>> state and then return when that is done.
>> 
>> e.g.
>> 
>> xm create mydomain
>> xm fs_freeze mydomain
>> <do backup>
>> xm fs_unfreeze mydomain
>> 
>> Ideally the backup operation would just take a snapshot e.g. using LVM or 
>> qcow.  With a bit more tools / storage integration we could have an "xm 
>> safesnapshot" command that would create a coherent snapshot of the disk.
>> 
>> It would be awesome.
>> 
>> Cheers,
>> Mark
>>
>> 
>
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> I don't know anything about parallaxfs (and neither does google it seems),
> but I guess the problem will be exactly the same. If it won't give the
> domU command to put fs into consistent state, snapshot won't be
> consistent.

There's some discussion of its current status here: 
http://www.xen.org/files/xensummit_fall07/20_DutchMeyer.pdf

It's essentially a distributed block store that's specifically oriented to the 
needs of storing VM disk images.

> hopefully I'll manage to create a fix Mark proposed...

I've been having a look at this too.  Would you like to look into sharing the 
work?  I'd got as far as plumbing some control plane stuff through xm, Xend, 
etc to talk to a stub driver in drivers/xen/fssnap/fssnap.c.  I've also 
identified where I think we'd need to hook in to get the kernel to freeze its 
filesystem.  I've not tied it all together yet though.

Once this feature is available, it'd be cool to get it plugged into various 
other layers of tools - there's lots of potential to do interesting things 
with it.  e.g.
1) manual backups
2) integrate automatically with block device snapshots
3) enable introspection of a domain filesystem e.g. filesystem contents, free 
space, checksum files, measure fragmentation, etc - all these can be 
performed on the consistent snapshot whilst the domain itself continues to 
run on the primary copy
4) etc etc etc

Lots of fun stuff :-)

Cheers,
Mark

> cheers
> n.
>
>   On Sat, 15 Dec 2007, tgh wrote:
> > hi
> > if we use the ParallaxFS to do the filesystem snapshot, does it have the
> > similar problem ,or not? could you clear me?
> >
> > Thanks in advance
> >
> > Mark Williamson ??:
> >>> I'd like to ask a question regarding LVM snapshots:
> >>> normally if You make a snapshot of mounted LVM partition, DM
> >>> infrastructure handles putting fs into consistent state using VFS
> >>> infrastructure. BUT, does it work even if I make snapshot of LV used by
> >>> XEN?
> >>
> >> If your LVM is running in dom0 and you snapshot the storage used by a
> >> domU then, no, it won't snapshot the domU's filesystem automatically.
> >>
> >>> If not, I guess it should not be that hard to fix, I guess we just need
> >>> to notify domU kernel to call needed VFS function to put fs into
> >>> consistent state prior to creating snapshot.
> >>
> >> Yep.
> >>
> >>> Can somebody comment on it, eventually correct me if I'm wrong?
> >>
> >> You're quite right.  I've just had a look at it and it looks quite
> >> feasible to do this.  The obvious simple way to implement it would be to
> >> add an xm command that causes a domain to freeze its filesystems into a
> >> consistent state and then return when that is done.
> >>
> >> e.g.
> >>
> >> xm create mydomain
> >> xm fs_freeze mydomain
> >> <do backup>
> >> xm fs_unfreeze mydomain
> >>
> >> Ideally the backup operation would just take a snapshot e.g. using LVM
> >> or qcow.  With a bit more tools / storage integration we could have an
> >> "xm safesnapshot" command that would create a coherent snapshot of the
> >> disk.
> >>
> >> It would be awesome.
> >>
> >> Cheers,
> >> Mark



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi Mark,
and sorry for not replying for so long.
Parallax looks really promising, and I'll surely keep eye on it...

> I've been having a look at this too.  Would you like to look into sharing the
> work?  I'd got as far as plumbing some control plane stuff through xm, Xend,
> etc to talk to a stub driver in drivers/xen/fssnap/fssnap.c.  I've also
> identified where I think we'd need to hook in to get the kernel to freeze its
> filesystem.  I've not tied it all together yet though.
Well, sure I'd like to share, I didn't have much time to play with it yet, 
I've mostly studied a bit on how the DM stuff works, and I think the 
simplest, yet working solution could be:

$ xm freeze_dev xvda1 myDom
 	- this will tell the domU kernel to call freeze_bdev(...) (see 
fs/buffer.c) for the given device to put it into consistent state, and 
stop using it for a while

then from dom0 You can safely create a snapshot of lv on top of domU's 
device is sitting:
$ lvcreate -s blabla

$ xm unfreeze_dev xvda1 myDom
 	- this will tell the domU kernel to call thaw_bdev(..), which 
will allow domU to continue using device

What do You think about it?

> Once this feature is available, it'd be cool to get it plugged into various
> other layers of tools - there's lots of potential to do interesting things
> with it.  e.g.
> 1) manual backups
> 2) integrate automatically with block device snapshots
> 3) enable introspection of a domain filesystem e.g. filesystem contents, free
> space, checksum files, measure fragmentation, etc - all these can be
> performed on the consistent snapshot whilst the domain itself continues to
> run on the primary copy
> 4) etc etc etc
>
> Lots of fun stuff :-)
Yup, agree :)
Cheers!
nik


>
> Cheers,
> Mark
>
>> cheers
>> n.
>>
>>   On Sat, 15 Dec 2007, tgh wrote:
>>> hi
>>> if we use the ParallaxFS to do the filesystem snapshot, does it have the
>>> similar problem ,or not? could you clear me?
>>>
>>> Thanks in advance
>>>
>>> Mark Williamson ??:
>>>>> I'd like to ask a question regarding LVM snapshots:
>>>>> normally if You make a snapshot of mounted LVM partition, DM
>>>>> infrastructure handles putting fs into consistent state using VFS
>>>>> infrastructure. BUT, does it work even if I make snapshot of LV used by
>>>>> XEN?
>>>>
>>>> If your LVM is running in dom0 and you snapshot the storage used by a
>>>> domU then, no, it won't snapshot the domU's filesystem automatically.
>>>>
>>>>> If not, I guess it should not be that hard to fix, I guess we just need
>>>>> to notify domU kernel to call needed VFS function to put fs into
>>>>> consistent state prior to creating snapshot.
>>>>
>>>> Yep.
>>>>
>>>>> Can somebody comment on it, eventually correct me if I'm wrong?
>>>>
>>>> You're quite right.  I've just had a look at it and it looks quite
>>>> feasible to do this.  The obvious simple way to implement it would be to
>>>> add an xm command that causes a domain to freeze its filesystems into a
>>>> consistent state and then return when that is done.
>>>>
>>>> e.g.
>>>>
>>>> xm create mydomain
>>>> xm fs_freeze mydomain
>>>> <do backup>
>>>> xm fs_unfreeze mydomain
>>>>
>>>> Ideally the backup operation would just take a snapshot e.g. using LVM
>>>> or qcow.  With a bit more tools / storage integration we could have an
>>>> "xm safesnapshot" command that would create a coherent snapshot of the
>>>> disk.
>>>>
>>>> It would be awesome.
>>>>
>>>> Cheers,
>>>> Mark
>
>
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> > I've been having a look at this too.  Would you like to look into sharing
> > the work?  I'd got as far as plumbing some control plane stuff through
> > xm, Xend, etc to talk to a stub driver in drivers/xen/fssnap/fssnap.c. 
> > I've also identified where I think we'd need to hook in to get the kernel
> > to freeze its filesystem.  I've not tied it all together yet though.
>
> Well, sure I'd like to share, I didn't have much time to play with it yet,
> I've mostly studied a bit on how the DM stuff works, and I think the
> simplest, yet working solution could be:
>
> $ xm freeze_dev xvda1 myDom
>  	- this will tell the domU kernel to call freeze_bdev(...) (see
> fs/buffer.c) for the given device to put it into consistent state, and
> stop using it for a while
>
> then from dom0 You can safely create a snapshot of lv on top of domU's
> device is sitting:
> $ lvcreate -s blabla
>
> $ xm unfreeze_dev xvda1 myDom
>  	- this will tell the domU kernel to call thaw_bdev(..), which
> will allow domU to continue using device
>
> What do You think about it?

Yes, I think that's a sensible first step to take.  Later on, we could perhaps 
look into putting some more smarts in, e.g. doing the snapshot creation 
automatically in the case that an LVM volume or QCow virtual disk is already 
being used.

I hadn't come across freeze/thaw_bdev before - I'd found a lock / unlock call 
for filesystem locking somewhere that looked promising but not implemented it 
yet.  I imagine it might boil down to a similar thing in the end.

Either way, being able to freeze specific block devices could be useful for 
backup purposes.  There's no point freezing filesystems that are not used to 
store backed-up data, for instance!

>
> Yup, agree :)
> Cheers!

OK, I'll try and get some code online at some point and let you know when it's 
available.

Cheers,
Mark

> nik
>
> > Cheers,
> > Mark
> >
> >> cheers
> >> n.
> >>
> >>   On Sat, 15 Dec 2007, tgh wrote:
> >>> hi
> >>> if we use the ParallaxFS to do the filesystem snapshot, does it have
> >>> the similar problem ,or not? could you clear me?
> >>>
> >>> Thanks in advance
> >>>
> >>> Mark Williamson ??:
> >>>>> I'd like to ask a question regarding LVM snapshots:
> >>>>> normally if You make a snapshot of mounted LVM partition, DM
> >>>>> infrastructure handles putting fs into consistent state using VFS
> >>>>> infrastructure. BUT, does it work even if I make snapshot of LV used
> >>>>> by XEN?
> >>>>
> >>>> If your LVM is running in dom0 and you snapshot the storage used by a
> >>>> domU then, no, it won't snapshot the domU's filesystem automatically.
> >>>>
> >>>>> If not, I guess it should not be that hard to fix, I guess we just
> >>>>> need to notify domU kernel to call needed VFS function to put fs into
> >>>>> consistent state prior to creating snapshot.
> >>>>
> >>>> Yep.
> >>>>
> >>>>> Can somebody comment on it, eventually correct me if I'm wrong?
> >>>>
> >>>> You're quite right.  I've just had a look at it and it looks quite
> >>>> feasible to do this.  The obvious simple way to implement it would be
> >>>> to add an xm command that causes a domain to freeze its filesystems
> >>>> into a consistent state and then return when that is done.
> >>>>
> >>>> e.g.
> >>>>
> >>>> xm create mydomain
> >>>> xm fs_freeze mydomain
> >>>> <do backup>
> >>>> xm fs_unfreeze mydomain
> >>>>
> >>>> Ideally the backup operation would just take a snapshot e.g. using LVM
> >>>> or qcow.  With a bit more tools / storage integration we could have an
> >>>> "xm safesnapshot" command that would create a coherent snapshot of the
> >>>> disk.
> >>>>
> >>>> It would be awesome.
> >>>>
> >>>> Cheers,
> >>>> Mark
> >
> > --
> > Dave: Just a question. What use is a unicyle with no seat?  And no
> > pedals! Mark: To answer a question with a question: What use is a
> > skateboard? Dave: Skateboards have wheels.
> > Mark: My wheel has a wheel!
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

On Mon, 24 Dec 2007, Mark Williamson wrote:
Hi Mark,
is there some progress regarding this subject? Still, if there is 
something I could help with, please do not hesitate to tell me what could 
I do...
Cheers
Nik

>
> Yes, I think that's a sensible first step to take.  Later on, we could perhaps
> look into putting some more smarts in, e.g. doing the snapshot creation
> automatically in the case that an LVM volume or QCow virtual disk is already
> being used.
>
> I hadn't come across freeze/thaw_bdev before - I'd found a lock / unlock call
> for filesystem locking somewhere that looked promising but not implemented it
> yet.  I imagine it might boil down to a similar thing in the end.
>
> Either way, being able to freeze specific block devices could be useful for
> backup purposes.  There's no point freezing filesystems that are not used to
> store backed-up data, for instance!
>
>>
>> Yup, agree :)
>> Cheers!
>
> OK, I'll try and get some code online at some point and let you know when it's
> available.

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> is there some progress regarding this subject? Still, if there is
> something I could help with, please do not hesitate to tell me what could
> I do...

Hi!

Sorry, I've not been doing Xen coding over the Christmas / New Year period, 
partly due to being away and partly due to being ill.

I'm back in Cambridge and getting back into things now.  I'll be trying to get 
a copy of the code online later today and will point you at it when it's 
there.  Currently it doesn't actually *do* very much ;-) but it's a start on 
what I think could turn into some interesting stuff!

Cheers,
Mark

> > Yes, I think that's a sensible first step to take.  Later on, we could
> > perhaps look into putting some more smarts in, e.g. doing the snapshot
> > creation automatically in the case that an LVM volume or QCow virtual
> > disk is already being used.
> >
> > I hadn't come across freeze/thaw_bdev before - I'd found a lock / unlock
> > call for filesystem locking somewhere that looked promising but not
> > implemented it yet.  I imagine it might boil down to a similar thing in
> > the end.
> >
> > Either way, being able to freeze specific block devices could be useful
> > for backup purposes.  There's no point freezing filesystems that are not
> > used to store backed-up data, for instance!
> >
> >> Yup, agree :)
> >> Cheers!
> >
> > OK, I'll try and get some code online at some point and let you know when
> > it's available.



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

Hi there,

I've started checking code into Xenbits.  Right now all that's there is the 
proof-of-concept control tools work and a stub driver in the guest kernel 
which is able to listen for commands.

The actual mechanics of FS snapshotting need to be implemented, plus UI 
features (such as actually taking arguments - and more importantly only 
returning from xm fsfreeze when guest says the freeze is actually 
complete)...

If you want to have a crack at plumbing this together you'd be more than 
welcome, or I can have a go at it, or we can split it.  Any preference?  I'll 
get paid for any work I do, so I'm happy to do any amount ;-)

Cheers,
Mark

On Friday 04 January 2008, Nikola Ciprich wrote:
> On Mon, 24 Dec 2007, Mark Williamson wrote:
> Hi Mark,
> is there some progress regarding this subject? Still, if there is
> something I could help with, please do not hesitate to tell me what could
> I do...
> Cheers
> Nik
>
> > Yes, I think that's a sensible first step to take.  Later on, we could
> > perhaps look into putting some more smarts in, e.g. doing the snapshot
> > creation automatically in the case that an LVM volume or QCow virtual
> > disk is already being used.
> >
> > I hadn't come across freeze/thaw_bdev before - I'd found a lock / unlock
> > call for filesystem locking somewhere that looked promising but not
> > implemented it yet.  I imagine it might boil down to a similar thing in
> > the end.
> >
> > Either way, being able to freeze specific block devices could be useful
> > for backup purposes.  There's no point freezing filesystems that are not
> > used to store backed-up data, for instance!
> >
> >> Yup, agree :)
> >> Cheers!
> >
> > OK, I'll try and get some code online at some point and let you know when
> > it's available.



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi Mark,
that's a good news! Where can I get the code? I don't see it in 
xen-unstable yet, is it available in some other repo?
When it comes to dividing the work, I could try to finish the kernel part 
and try to glue it all together.. as soon as I can get my hands on the 
code :)
so please let me know where I can get it and I'll have a look at it ASAP
cheers and thanks!
nik

  On Fri, 4 Jan 2008, Mark Williamson wrote:

> Hi there,
>
> I've started checking code into Xenbits.  Right now all that's there is the
> proof-of-concept control tools work and a stub driver in the guest kernel
> which is able to listen for commands.
>
> The actual mechanics of FS snapshotting need to be implemented, plus UI
> features (such as actually taking arguments - and more importantly only
> returning from xm fsfreeze when guest says the freeze is actually
> complete)...
>
> If you want to have a crack at plumbing this together you'd be more than
> welcome, or I can have a go at it, or we can split it.  Any preference?  I'll
> get paid for any work I do, so I'm happy to do any amount ;-)
>
> Cheers,
> Mark
>
> On Friday 04 January 2008, Nikola Ciprich wrote:
>> On Mon, 24 Dec 2007, Mark Williamson wrote:
>> Hi Mark,
>> is there some progress regarding this subject? Still, if there is
>> something I could help with, please do not hesitate to tell me what could
>> I do...
>> Cheers
>> Nik
>>
>>> Yes, I think that's a sensible first step to take.  Later on, we could
>>> perhaps look into putting some more smarts in, e.g. doing the snapshot
>>> creation automatically in the case that an LVM volume or QCow virtual
>>> disk is already being used.
>>>
>>> I hadn't come across freeze/thaw_bdev before - I'd found a lock / unlock
>>> call for filesystem locking somewhere that looked promising but not
>>> implemented it yet.  I imagine it might boil down to a similar thing in
>>> the end.
>>>
>>> Either way, being able to freeze specific block devices could be useful
>>> for backup purposes.  There's no point freezing filesystems that are not
>>> used to store backed-up data, for instance!
>>>
>>>> Yup, agree :)
>>>> Cheers!
>>>
>>> OK, I'll try and get some code online at some point and let you know when
>>> it's available.
>
>
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

Oh I am a silly sausage!  I wrote the whole e-mail and forgot to mention where 
the code actually was :-)

See:
http://xenbits.xensource.com/maw/xen-fssnapshot.hg
and
http://xenbits.xensource.com/maw/xen-linux-fssnapshot.hg

The code is just a template of what's needed to plumb commands from xm, 
through Xend and to the guest domain.  It's probably a bit buggy and ugly, 
but it shows basically what files you need to touch and it might provide a 
base to work from.  With a few tweaks you should be able to get some calls to 
filesystem freezing going, and then add more controls to make it more useful 
to administrators.

I've added the command to the legacy protocol for now because I don't 
understand the XenAPI stuff.  But it ought to be added there too at some 
point...

Cheers,
Mark

On Friday 04 January 2008, Nikola Ciprich wrote:
> Hi Mark,
> that's a good news! Where can I get the code? I don't see it in
> xen-unstable yet, is it available in some other repo?
> When it comes to dividing the work, I could try to finish the kernel part
> and try to glue it all together.. as soon as I can get my hands on the
> code :)
> so please let me know where I can get it and I'll have a look at it ASAP
> cheers and thanks!
> nik
>
>   On Fri, 4 Jan 2008, Mark Williamson wrote:
> > Hi there,
> >
> > I've started checking code into Xenbits.  Right now all that's there is
> > the proof-of-concept control tools work and a stub driver in the guest
> > kernel which is able to listen for commands.
> >
> > The actual mechanics of FS snapshotting need to be implemented, plus UI
> > features (such as actually taking arguments - and more importantly only
> > returning from xm fsfreeze when guest says the freeze is actually
> > complete)...
> >
> > If you want to have a crack at plumbing this together you'd be more than
> > welcome, or I can have a go at it, or we can split it.  Any preference? 
> > I'll get paid for any work I do, so I'm happy to do any amount ;-)
> >
> > Cheers,
> > Mark
> >
> > On Friday 04 January 2008, Nikola Ciprich wrote:
> >> On Mon, 24 Dec 2007, Mark Williamson wrote:
> >> Hi Mark,
> >> is there some progress regarding this subject? Still, if there is
> >> something I could help with, please do not hesitate to tell me what
> >> could I do...
> >> Cheers
> >> Nik
> >>
> >>> Yes, I think that's a sensible first step to take.  Later on, we could
> >>> perhaps look into putting some more smarts in, e.g. doing the snapshot
> >>> creation automatically in the case that an LVM volume or QCow virtual
> >>> disk is already being used.
> >>>
> >>> I hadn't come across freeze/thaw_bdev before - I'd found a lock /
> >>> unlock call for filesystem locking somewhere that looked promising but
> >>> not implemented it yet.  I imagine it might boil down to a similar
> >>> thing in the end.
> >>>
> >>> Either way, being able to freeze specific block devices could be useful
> >>> for backup purposes.  There's no point freezing filesystems that are
> >>> not used to store backed-up data, for instance!
> >>>
> >>>> Yup, agree :)
> >>>> Cheers!
> >>>
> >>> OK, I'll try and get some code online at some point and let you know
> >>> when it's available.
> >
> > --
> > Dave: Just a question. What use is a unicyle with no seat?  And no
> > pedals! Mark: To answer a question with a question: What use is a
> > skateboard? Dave: Skateboards have wheels.
> > Mark: My wheel has a wheel!



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: consistent LVM snapshot of domUs from dom0

[Nikola Ciprich]

Hi Mark,
thanks! I'm already looking at the changes - although the second address 
is linux-xen-fssnapshot.hg, not xen-linux-fssnapshot.hg :)

I'll try to add the code for (un)freeze_bdev and send You the patch..

For the future, do You think we could have it that You can specify which 
device You want to freeze? It could reduce time needed for backup of domUs 
with many (and/or quickly changing) devs, if You want to make snapshot 
only of one of them...

Cheers!
nik

  On Sat, 5 Jan 2008, Mark Williamson wrote:

> Oh I am a silly sausage!  I wrote the whole e-mail and forgot to mention where
> the code actually was :-)
>
> See:
> http://xenbits.xensource.com/maw/xen-fssnapshot.hg
> and
> http://xenbits.xensource.com/maw/xen-linux-fssnapshot.hg
>
> The code is just a template of what's needed to plumb commands from xm,
> through Xend and to the guest domain.  It's probably a bit buggy and ugly,
> but it shows basically what files you need to touch and it might provide a
> base to work from.  With a few tweaks you should be able to get some calls to
> filesystem freezing going, and then add more controls to make it more useful
> to administrators.
>
> I've added the command to the legacy protocol for now because I don't
> understand the XenAPI stuff.  But it ought to be added there too at some
> point...
>
> Cheers,
> Mark
>
> On Friday 04 January 2008, Nikola Ciprich wrote:
>> Hi Mark,
>> that's a good news! Where can I get the code? I don't see it in
>> xen-unstable yet, is it available in some other repo?
>> When it comes to dividing the work, I could try to finish the kernel part
>> and try to glue it all together.. as soon as I can get my hands on the
>> code :)
>> so please let me know where I can get it and I'll have a look at it ASAP
>> cheers and thanks!
>> nik
>>
>>   On Fri, 4 Jan 2008, Mark Williamson wrote:
>>> Hi there,
>>>
>>> I've started checking code into Xenbits.  Right now all that's there is
>>> the proof-of-concept control tools work and a stub driver in the guest
>>> kernel which is able to listen for commands.
>>>
>>> The actual mechanics of FS snapshotting need to be implemented, plus UI
>>> features (such as actually taking arguments - and more importantly only
>>> returning from xm fsfreeze when guest says the freeze is actually
>>> complete)...
>>>
>>> If you want to have a crack at plumbing this together you'd be more than
>>> welcome, or I can have a go at it, or we can split it.  Any preference?
>>> I'll get paid for any work I do, so I'm happy to do any amount ;-)
>>>
>>> Cheers,
>>> Mark
>>>
>>> On Friday 04 January 2008, Nikola Ciprich wrote:
>>>> On Mon, 24 Dec 2007, Mark Williamson wrote:
>>>> Hi Mark,
>>>> is there some progress regarding this subject? Still, if there is
>>>> something I could help with, please do not hesitate to tell me what
>>>> could I do...
>>>> Cheers
>>>> Nik
>>>>
>>>>> Yes, I think that's a sensible first step to take.  Later on, we could
>>>>> perhaps look into putting some more smarts in, e.g. doing the snapshot
>>>>> creation automatically in the case that an LVM volume or QCow virtual
>>>>> disk is already being used.
>>>>>
>>>>> I hadn't come across freeze/thaw_bdev before - I'd found a lock /
>>>>> unlock call for filesystem locking somewhere that looked promising but
>>>>> not implemented it yet.  I imagine it might boil down to a similar
>>>>> thing in the end.
>>>>>
>>>>> Either way, being able to freeze specific block devices could be useful
>>>>> for backup purposes.  There's no point freezing filesystems that are
>>>>> not used to store backed-up data, for instance!
>>>>>
>>>>>> Yup, agree :)
>>>>>> Cheers!
>>>>>
>>>>> OK, I'll try and get some code online at some point and let you know
>>>>> when it's available.
>>>
>>> --
>>> Dave: Just a question. What use is a unicyle with no seat?  And no
>>> pedals! Mark: To answer a question with a question: What use is a
>>> skateboard? Dave: Skateboards have wheels.
>>> Mark: My wheel has a wheel!
>
>
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>
>

-- 

Re: consistent LVM snapshot of domUs from dom0

[Mark Williamson]

> thanks! I'm already looking at the changes - although the second address
> is linux-xen-fssnapshot.hg, not xen-linux-fssnapshot.hg :)

Doh!  I'm a muppet, sorry.  Well done for figuring it out.

> I'll try to add the code for (un)freeze_bdev and send You the patch..

Awesome!

> For the future, do You think we could have it that You can specify which
> device You want to freeze? It could reduce time needed for backup of domUs
> with many (and/or quickly changing) devs, if You want to make snapshot
> only of one of them...

I think it'd be nice eventually to be able to freeze individual partitions 
(should be doable, since this is the normal scope of a filesystem).

I think that might be good for several reasons, including:
1) if you only want to back up certain filesystems (e.g. ones containing user 
data)
2) if you only want to freeze one FS at a time to minimise disruption to guest 
IO

For a proof-of-concept, I think it'd be fine to freeze all 
filesystems /disks - or even just to freeze one particular filesystem / disk 
by default.

Cheers,
Mark

> Cheers!
> nik
>
>   On Sat, 5 Jan 2008, Mark Williamson wrote:
> > Oh I am a silly sausage!  I wrote the whole e-mail and forgot to mention
> > where the code actually was :-)
> >
> > See:
> > http://xenbits.xensource.com/maw/xen-fssnapshot.hg
> > and
> > http://xenbits.xensource.com/maw/xen-linux-fssnapshot.hg
> >
> > The code is just a template of what's needed to plumb commands from xm,
> > through Xend and to the guest domain.  It's probably a bit buggy and
> > ugly, but it shows basically what files you need to touch and it might
> > provide a base to work from.  With a few tweaks you should be able to get
> > some calls to filesystem freezing going, and then add more controls to
> > make it more useful to administrators.
> >
> > I've added the command to the legacy protocol for now because I don't
> > understand the XenAPI stuff.  But it ought to be added there too at some
> > point...
> >
> > Cheers,
> > Mark
> >
> > On Friday 04 January 2008, Nikola Ciprich wrote:
> >> Hi Mark,
> >> that's a good news! Where can I get the code? I don't see it in
> >> xen-unstable yet, is it available in some other repo?
> >> When it comes to dividing the work, I could try to finish the kernel
> >> part and try to glue it all together.. as soon as I can get my hands on
> >> the code :)
> >> so please let me know where I can get it and I'll have a look at it ASAP
> >> cheers and thanks!
> >> nik
> >>
> >>   On Fri, 4 Jan 2008, Mark Williamson wrote:
> >>> Hi there,
> >>>
> >>> I've started checking code into Xenbits.  Right now all that's there is
> >>> the proof-of-concept control tools work and a stub driver in the guest
> >>> kernel which is able to listen for commands.
> >>>
> >>> The actual mechanics of FS snapshotting need to be implemented, plus UI
> >>> features (such as actually taking arguments - and more importantly only
> >>> returning from xm fsfreeze when guest says the freeze is actually
> >>> complete)...
> >>>
> >>> If you want to have a crack at plumbing this together you'd be more
> >>> than welcome, or I can have a go at it, or we can split it.  Any
> >>> preference? I'll get paid for any work I do, so I'm happy to do any
> >>> amount ;-)
> >>>
> >>> Cheers,
> >>> Mark
> >>>
> >>> On Friday 04 January 2008, Nikola Ciprich wrote:
> >>>> On Mon, 24 Dec 2007, Mark Williamson wrote:
> >>>> Hi Mark,
> >>>> is there some progress regarding this subject? Still, if there is
> >>>> something I could help with, please do not hesitate to tell me what
> >>>> could I do...
> >>>> Cheers
> >>>> Nik
> >>>>
> >>>>> Yes, I think that's a sensible first step to take.  Later on, we
> >>>>> could perhaps look into putting some more smarts in, e.g. doing the
> >>>>> snapshot creation automatically in the case that an LVM volume or
> >>>>> QCow virtual disk is already being used.
> >>>>>
> >>>>> I hadn't come across freeze/thaw_bdev before - I'd found a lock /
> >>>>> unlock call for filesystem locking somewhere that looked promising
> >>>>> but not implemented it yet.  I imagine it might boil down to a
> >>>>> similar thing in the end.
> >>>>>
> >>>>> Either way, being able to freeze specific block devices could be
> >>>>> useful for backup purposes.  There's no point freezing filesystems
> >>>>> that are not used to store backed-up data, for instance!
> >>>>>
> >>>>>> Yup, agree :)
> >>>>>> Cheers!
> >>>>>
> >>>>> OK, I'll try and get some code online at some point and let you know
> >>>>> when it's available.
> >>>
> >>> --
> >>> Dave: Just a question. What use is a unicyle with no seat?  And no
> >>> pedals! Mark: To answer a question with a question: What use is a
> >>> skateboard? Dave: Skateboards have wheels.
> >>> Mark: My wheel has a wheel!
> >
> > --
> > Dave: Just a question. What use is a unicyle with no seat?  And no
> > pedals! Mark: To answer a question with a question: What use is a
> > skateboard? Dave: Skateboards have wheels.
> > Mark: My wheel has a wheel!



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!