snapshot, checkpoints

snapshot, checkpoints

[Timothy Webster]

Does reiserfs4 include features to allow
vxfs like checkpoints 
or netapps snapshots?

Thanks in advance.
-tim


	
		
__________________________________
Do you Yahoo!?
Friends.  Fun.  Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/ 

Re: snapshot, checkpoints

[Vladimir Saveliev]

Hello

On Thu, 2004-05-27 at 10:52, Timothy Webster wrote:
> Does reiserfs4 include features to allow
> vxfs like checkpoints 
> or netapps snapshots?
> 
no
> Thanks in advance.
> -tim
> 
> 
> 	
> 		
> __________________________________
> Do you Yahoo!?
> Friends.  Fun.  Try the all-new Yahoo! Messenger.
> http://messenger.yahoo.com/
> 

Re: snapshot, checkpoints

[mjt]

On Thu, May 27, 2004 at 11:03:14AM +0400, Vladimir Saveliev wrote:

>On Thu, 2004-05-27 at 10:52, Timothy Webster wrote:
>> Does reiserfs4 include features to allow
>> vxfs like checkpoints 
>> or netapps snapshots?
>no

I am not familiar with these mechanisms, would someone care to
ellaborate a bit?

I can't really even hazard a guess on what a filesystem snapshot
or checkpoint might be, but it sounds like a backup of the file system.
Or maybe just a journal state...

Thanks

-- 
mjt

Re: snapshot, checkpoints

[Heinz-Josef Claes]

Am Do, den 27.05.2004 schrieb Markus Törnqvist um 11:27:
> On Thu, May 27, 2004 at 11:03:14AM +0400, Vladimir Saveliev wrote:
> 
> >On Thu, 2004-05-27 at 10:52, Timothy Webster wrote:
> >> Does reiserfs4 include features to allow
> >> vxfs like checkpoints 
> >> or netapps snapshots?
> >no
> 
> I am not familiar with these mechanisms, would someone care to
> ellaborate a bit?
> 
> I can't really even hazard a guess on what a filesystem snapshot
> or checkpoint might be, but it sounds like a backup of the file system.
> Or maybe just a journal state...

Hi,
simply read these paragraphs:
http://www.netapp.com/tech_library/3002.html#I34

-- 
Heinz-Josef Claes                     hjclaes@web.de
project: http://sourceforge.net/projects/storebackup
      -> snapshot-like backup to another disk

Re: snapshot, checkpoints

[Paul Wagland]

On Thu, 27 May 2004 12:27:27 +0300, Markus TXrnqvist <mjt@nysv.org> wrote:

> On Thu, May 27, 2004 at 11:03:14AM +0400, Vladimir Saveliev wrote:
>
>> On Thu, 2004-05-27 at 10:52, Timothy Webster wrote:
>>> Does reiserfs4 include features to allow
>>> vxfs like checkpoints
>>> or netapps snapshots?
>> no

Can't the same functionality be created with device mapper though? At least
under linux anyway?

Cheers,
Paul

Re: snapshot, checkpoints

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On Fri, 04 Jun 2004 13:10:05 +0200, Paul Wagland said:

> Can't the same functionality be created with device mapper though? At least
> under linux anyway?

You'd need 2 things:

1) *very* recent patched device mapper (I think patches for snapshot support
went by on LKML just day before yesterday or so).

2) You also need a suitable write-barrier interlock to the filesystem, to
basically force a flush-to-disk of all the incore data buffers, etc (basically,
you need to ensure that at the instant the snapshot is taken, the on-disk copy
is "clean" by fsck standards).

At that point, you can just have a utility that goes "flush; snapshot;" and go on
your way.....


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: snapshot, checkpoints

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Valdis.Kletnieks@vt.edu wrote:
| On Fri, 04 Jun 2004 13:10:05 +0200, Paul Wagland said:
|
|
|>Can't the same functionality be created with device mapper though? At
least
|>under linux anyway?

[...]

| 2) You also need a suitable write-barrier interlock to the filesystem, to
| basically force a flush-to-disk of all the incore data buffers, etc
(basically,
| you need to ensure that at the instant the snapshot is taken, the
on-disk copy
| is "clean" by fsck standards).


My main problem with this is that it seems to literally be a snapshot --
that is, a complete backup to another partition.  I think we should have
something like a version control system -- blocks (or files, or
clusters, or whatever granularity is best/easiest) which change after
the snapshot must be rewritten to somewhere else in the same partition.

This means that the original filesystem is recoverable (in case of
software trouble), and that the recovery takes very little time.  It
also means that things which don't change much don't get copies made.

This is even an advantage when you do want a separate partition/drive to
guard against disk failures, because you can now store many more
versions on a single partition.  I currently do something like this with
hardlinks -- every new backup is a hardlink farm of the latest backup,
which I then rsync with the live filesystem.  (rsync unlinks files
before it overwrites them.)  But that's hardlinks -- it'd be nice to be
able to do it at finer granularity (if there'd be no performance loss)
and on live partitions (without an actual "backup").

Of course, the necessity of a full snapshot is still there, and with
that, I'd rather have the files copied a la "cp" -- less disk activity.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQMFOnHgHNmZLgCUhAQLN5BAAilxfG4Cs+49SFgE3T//L9Iss6nH2Lz2Y
LEZft4uElLRLkGoreAeEMJew1Tz4GV5bGajLSkNZingeefwMawhUKXLYq9z6jO9O
YYhoQYEGBrvQeHhe8rqCSQ+rqOqmk8PZrXZsNbNJJ25RCdCTPTuRw/VTkTvUYCOY
N9MSv2ihJDzWvy58/gfwY6zGYmqYoLowttzRm+CrxIH/KBtFcxhu7bbs5cWXgJ6P
/MDO1BbkdCaK0FV/HZljsk12d38LKAriCmInmp6ZuDQiWmL9Waq/dFu6sYhb9q3P
0RTWURM9eBV/I8E2Xh8VcEuCSLJMHnxjTqCcWU8d17rALrZbdwSPyXNS8zuzsndI
O4EZRbXrWrDlXfWoO7vkA+upkaLKkraYUjEPdx/KQAdgflFHbq0ta76Vc+LMaLfE
Id8+MaDug71cQsngQSujr6FEbKZOfAhIf3VBFf7T5irciaOzsKJeSQxE7Ceo8tME
vz0D1j7d4LDQWtmxpCHXiJuFXVbPR0EbDGJuFz6YKQcRk4gPoUXKntuY3QN8e59n
39Q5zJGw++1SrQkAG9/Bvjq2nvgqIxuxv2WH8nWOmDeQKDlaKq3o9+wxWJYiq8kj
MvFo2CJA1DzRQ510zgsu3k+Mk4RQyXeGG/HdaE3xuT1PnB2BlGug8jCtyrk4i1Bu
xZNBiflMqZw=
=7rR5
-----END PGP SIGNATURE-----

Re: snapshot, checkpoints

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]


On Jun 5, 2004, at 6:39, David Masover wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Valdis.Kletnieks@vt.edu wrote:
> | On Fri, 04 Jun 2004 13:10:05 +0200, Paul Wagland said:
> |
> |
> |>Can't the same functionality be created with device mapper though? At
> least
> |>under linux anyway?
>
> [...]
>
> | 2) You also need a suitable write-barrier interlock to the 
> filesystem, to
> | basically force a flush-to-disk of all the incore data buffers, etc
> (basically,
> | you need to ensure that at the instant the snapshot is taken, the
> on-disk copy
> | is "clean" by fsck standards).
>
>
> My main problem with this is that it seems to literally be a snapshot 
> --
> that is, a complete backup to another partition.  I think we should 
> have
> something like a version control system -- blocks (or files, or
> clusters, or whatever granularity is best/easiest) which change after
> the snapshot must be rewritten to somewhere else in the same partition.

At least with LVM, and therefore I assume with device mapper, the 
"snapshot" only stores changed blocks. From what I understand, this 
means that when you take a snapshot, if you change a block in the 
original, it first copies it to the snapshot, and then writes the new 
block to the original system. Of course, you will run into a problem if 
more changes are made than the snapshot partition size, but AFAIK it 
then just disables the snapshot.

Cheers,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: snapshot, checkpoints

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Paul Wagland wrote:
| At least with LVM, and therefore I assume with device mapper, the
| "snapshot" only stores changed blocks. From what I understand, this
| means that when you take a snapshot, if you change a block in the
| original, it first copies it to the snapshot, and then writes the new
| block to the original system. Of course, you will run into a problem if
| more changes are made than the snapshot partition size, but AFAIK it
| then just disables the snapshot.

Good, then we don't have to immediately rewrite the wheel.  But I still
would rather that we do this eventually.  My concept has the changes
stored on the same partition, seems like it may be faster, and opens the
door for a filesystem-based cvs.

Think about that!  I mean, if copy-on-capture is what I think it is,
then things like 'cp' could be extended to copy using a system call
(similar to how 'ln' works).  Probably a few utilities would have to be
extended, but the net result is, you don't have to say you're doing
something in a cvs dir for it to be cosidered a cvs action (just do 'cp'
instead of 'cvs cp' or whatever).  You don't "check out" different
versions, you just look in different folders.

(I'm using 'cvs' in the generic sense -- could work as easily for
bittorrent or anything else.)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQMeNhngHNmZLgCUhAQLe6w/+I5Emoxp6pLDA+9Ncd7UK0ZSkQsT0lwqu
jofkphMJ6hKZhE4KSB5IYup7Xh2ZqUZu1v+9gj0d/tLHatpCfLLklZJn0gKRA9+f
wUBY5VD18VmuWO0Zhi1R7QGAzijCThvyJTmhlu56E6dXEaQTxu7OufLmwfGDFgeT
LN3I/diPXVPp4AihzeFqT7F3/cYQgK/6wYgx9kB8QWsq9qcRzXY1XghuPgyoWJ9Y
AtIFmHmKLu9vP2S0mh7u6sv9R1UlzwdKZhpdxESytMqOV1m1xQbLdi5wLkS37i9l
Y9oxuhXXjiK+bkEwwdr+7/Ide1CzaRDthlBUMT+jy+zKekQLRLJtiZi3IhZeRb6V
UbO5SmHUqTxBltMEmp5zDMTX7QtT+n663GQCBzepMX55u9VWGTKrGYuSxDeJF5T7
5BT+FZ/UtY9ONL3hFFzMwAtaiI4r7rEIx0t2R1TVhsUhV69KmUXs4A2I0WOXPG4i
wFIQAjyX6BQtP3nn1QKIfeqpTRSHO2wPBC8dEAL8fFiiEBh/HEQdbbWCVCD3hj7G
0iNoLvL6w3qfPG+rRZtRdIn0apyd4zus0vWNpWWT8jh/QUeHGp97kzoEP3MQUlYk
kyVdFgwiX63l1AtJh6MWTDA2iVvegiaCY9Uuna5H77yJkV7uw8vc+TfRGRiq2otU
N7AQvJK8OBU=
=Nr9u
-----END PGP SIGNATURE-----

Re: snapshot, checkpoints

[Pierre Etchemaite]

Le Wed, 09 Jun 2004 17:21:58 -0500, David Masover <ninja@slaphack.com> a
écrit :

> Good, then we don't have to immediately rewrite the wheel.  But I still
> would rather that we do this eventually.  My concept has the changes
> stored on the same partition, seems like it may be faster, and opens the
> door for a filesystem-based cvs.

Look for "copy-on-write links" (cow links for short) in archives, or with
Google; You'll find many hits, including a message from 2000 suggesting the
idea has been around since at least 1997...

Re: snapshot, checkpoints

[Hans Reiser]

Pierre Etchemaite wrote:

>Le Wed, 09 Jun 2004 17:21:58 -0500, David Masover <ninja@slaphack.com> a
>écrit :
>
>  
>
>>Good, then we don't have to immediately rewrite the wheel.  But I still
>>would rather that we do this eventually.  My concept has the changes
>>stored on the same partition, seems like it may be faster, and opens the
>>door for a filesystem-based cvs.
>>    
>>
This phrase made me thing you might want to be aware of 
http://www.namesys.com/blackbox_security.html which got darpa funding 
recently....

>
>Look for "copy-on-write links" (cow links for short) in archives, or with
>Google; You'll find many hits, including a message from 2000 suggesting the
>idea has been around since at least 1997...
>
>
>  
>

Re: snapshot, checkpoints

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Hans Reiser wrote:
| Pierre Etchemaite wrote:
|
|> Le Wed, 09 Jun 2004 17:21:58 -0500, David Masover <ninja@slaphack.com> a
|> écrit :
|>
|>
|>
|>> Good, then we don't have to immediately rewrite the wheel.  But I still
|>> would rather that we do this eventually.  My concept has the changes
|>> stored on the same partition, seems like it may be faster, and opens the
|>> door for a filesystem-based cvs.
|>>
|
| This phrase made me thing you might want to be aware of
| http://www.namesys.com/blackbox_security.html which got darpa funding
| recently....

Looks cool, I sort of understand it, but what's it got to do with what I
said?  What "phrase"?  Filesystem-based cvs?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQMjdc3gHNmZLgCUhAQIBjBAAj1Dul0IWoyvTcyEHgg7471Gl0nEl+A/D
ROukc6UnQ0o/rVHAfjV/5GoqzBFIDXhBUOjMlAnyL908SJleyEt0CHuuBKiswGq/
9Y+Ymac/WDz/+v6oAo1pGHswPui8H17ubEnh8hrK6Y9GKC15nANY2kLtz3f+fzVC
aJAq2pGCbC/gJcRJR7tkM2ADH3tgtwFuWJdJWbnOxJvrSkUF/TVxuysROTmEDqiv
FtH3UMlyAbxxH7ZJEl5HR8pARmqejJAM4L6kFLHEWva577tkwrxhUu+w7A4gSGbE
vjM3zhm95TTS/fuUqJnw7cvr4t/TkFj5Lh3A0+0UQmhSdx/rPEwKYRR6ff8VzWyU
T7dNSsdkQvxO+W6uZf9HDdwZta9+emIakTxq8H25pnBmdOoGGq1PyEQ/DhfuVuTI
3o+idHRs7n+bzUBEkwRhZ2UkWSrpIsCwrq0e2otfosDaA3psTTi/W5rtrOFPOouD
8amdOZv0h8vV6tpD8P4kbIA5JA+aN0UEA4iWhS5Df6XOouJe5ZNAAdOL/plW1Vzq
X6D1LGzwv0RzxKwz0677xCyeuliTY5+heIpXJQ3Wx+TCdHY9akJQpYJsJxkwiSN5
LB5RoMJFDsEfcvMv5sG8j5qqTUJx9zmrzCpZgvRThyxGAMBavb2/ZcVmU1rzf4+G
6LwUWmAi3Gw=
=Wfv6
-----END PGP SIGNATURE-----

Re: snapshot, checkpoints

[Hans Reiser]

David Masover wrote:

>
>
> Hans Reiser wrote:
> | Pierre Etchemaite wrote:
> |
> |> Le Wed, 09 Jun 2004 17:21:58 -0500, David Masover 
> <ninja@slaphack.com> a
> |> écrit :
> |>
> |>
> |>
> |>> Good, then we don't have to immediately rewrite the wheel.  But I 
> still
> |>> would rather that we do this eventually.  My concept has the changes
> |>> stored on the same partition, seems like it may be faster, and 
> opens the
> |>> door for a filesystem-based cvs.
> |>>
> |
> | This phrase made me thing you might want to be aware of
> | http://www.namesys.com/blackbox_security.html which got darpa funding
> | recently....
>
> Looks cool, I sort of understand it, but what's it got to do with what I
> said?  What "phrase"?  Filesystem-based cvs?
>
Yes, views are an important step towards "Filesystem-based cvs", see 
clearcase as an example.  Transaction infrastructure, substantially 
present in reiser4 (see our (long) main page on reiser4 design, 
somewhere near the middle I think), is also an important component.  
Currently we support atomic sets of fs operations, although this is 
immature code (meaning reiser4 works reliably but the api is still in 
progress, etc.).

Re: snapshot, checkpoints

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hans Reiser wrote:
[...]
| Yes, views are an important step towards "Filesystem-based cvs", see
| clearcase as an example.  Transaction infrastructure, substantially

url please?  Or I'll google.

| Currently we support atomic sets of fs operations, although this is
| immature code (meaning reiser4 works reliably but the api is still in
| progress, etc.).

I think this can actually be done in userland, outside (on top of) the
fs.  That's not the better way, just something practical for people who
don't want to wait for (or use) the new api.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQMoecHgHNmZLgCUhAQKIAA/9GWZBmn9rFL4QYfPNr5UpMQz0e6s1Aor/
LDFu/CDKcb2KnRAkGdoxCKr2WdxvPPUxieUNXQAdpEejn1SQ+6mSTzE0HRzab1HJ
+O95bseF3eSb+mLiFC7CepQoviXUb/w1lzRww5v0f4s8biXv60qdkq+ZSK4a8CWP
PD/sB4S0prfK/y/k+mSRAqFBZzxohnmK03Iz3qbaN8MrRsP7Thns+DEIG6RuzKoz
H+fYwvUDUszkHXSG3CbyWUPSIT613lHcDbtfL52wpK+3722BM1hbiIw+aob07GIg
UwFgXD6Okhy6BURNQykbmu2uHQYKQSd+zRZujd/b6QoWGGFe75cUlTUTdxawUUsM
vvVQ6M7FN8BQ5Lj+/Oqtt8A9hjjMVIulJsP4zzxWrLpc8+JPi2WnO/x+beGVSbob
kfqvN8eHMpgwkqVl4wtyIQ+yuDxtLcfYIjyp7BLW82wuXRhMt5jwgzw/FUgmfiAI
O3VD2Bjbb8ndNFl++R29AQKtt9+n8od2vlLXLPK+/7hpSH82ymDXlA22/a8nekgk
ofhwYQYYWI+z7YAS6nyMtXvxK79Cr9ep1lGsc+aEZGeu1mJorJ6WMI3kNsw8a+nT
Mx2ntBJeI8YV7XPjQVKZb7Bz5DipUklrMkfrmrgQwOvwAKpY/BQwa2970myx6hJd
hQlzWDXy27g=
=CA6N
-----END PGP SIGNATURE-----