time-shifting

time-shifting

[Evert Mouw]

Hi,

Yesterday I played a bit with NILFS2 on ubuntu 11.04. It was quite easy 
to use, but finding previous versions of a file is still a bit 
complicated from a user point of view.

I read a bit about ext3cow, which is currently not actively maintained 
anymore. In one of the papers on ext3cow, I read the following:

Other disk file systems provide coarse-grained access to versions through
the creation of namespace tunnels [Hitz et al. 1994] or via mounting 
separate logical volumes
[Strunk et al. 2000; Soules et al. 2003]. Some disk file systems provide 
no interface
to versions, restricting versioning to internal use only [Rosenblum and 
Ousterhout 1992;
Seltzer et al. 1993].

In time-shifting, ext3cow introduces an interface to versioning that 
presents a continuous
view of time. Users or applications specify a file name and any point in 
time, which
ext3cow scopes to the appropriate snapshot or file version. The 
time-shifting interface
allows user-space tools to create snapshots and access versions.

Ext3cow: A Time-Shifting File System for Regulatory Compliance
ZACHARY N. J. PETERSON and RANDAL BURNS
ACM Transactions on Storage, 1(2), May, 2005
http://www.ext3cow.com/ext3cow/Publications.html

A usage scenario is given in a screenshot:
http://www.ext3cow.com/ext3cow/Welcome_files/example1.jpg

I don't think using the "@" symbol for such thinks is a great idea (I 
sometimes use it in my filenames) but the ability to use regular tools 
to access old versions of files is great.

I think it would be really useful to have access to previous versions of 
files without having to go through checkpoints or snapshots of the whole 
filesystem / partition. So having something like lshistory or lsversions 
for specific files would be great. It could output the versions, the 
checkpoints where those versions are available, etc.  Maybe direct 
retrieval of old versions (cathistory?) could also be possible using 
such a path.

Of course the architecture of NILFS is different from ext3cow so I don't 
know whether this is even possible. Something like a data structure 
linking the different verions of some inode must exist before this can 
happen. Just an idea, though.

Thanks for your cool stuff!

Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: time-shifting

[Ryusuke Konishi]

Hi,
On Sat, 06 Aug 2011 09:38:26 +0200, Evert Mouw wrote:
> Hi,
> 
> Yesterday I played a bit with NILFS2 on ubuntu 11.04. It was quite easy 
> to use, but finding previous versions of a file is still a bit 
> complicated from a user point of view.
> 
> I read a bit about ext3cow, which is currently not actively maintained 
> anymore. In one of the papers on ext3cow, I read the following:
> 
> Other disk file systems provide coarse-grained access to versions through
> the creation of namespace tunnels [Hitz et al. 1994] or via mounting 
> separate logical volumes
> [Strunk et al. 2000; Soules et al. 2003]. Some disk file systems provide 
> no interface
> to versions, restricting versioning to internal use only [Rosenblum and 
> Ousterhout 1992;
> Seltzer et al. 1993].
> 
> In time-shifting, ext3cow introduces an interface to versioning that 
> presents a continuous
> view of time. Users or applications specify a file name and any point in 
> time, which
> ext3cow scopes to the appropriate snapshot or file version. The 
> time-shifting interface
> allows user-space tools to create snapshots and access versions.
> 
> Ext3cow: A Time-Shifting File System for Regulatory Compliance
> ZACHARY N. J. PETERSON and RANDAL BURNS
> ACM Transactions on Storage, 1(2), May, 2005
> http://www.ext3cow.com/ext3cow/Publications.html
> 
> A usage scenario is given in a screenshot:
> http://www.ext3cow.com/ext3cow/Welcome_files/example1.jpg
> 
> I don't think using the "@" symbol for such thinks is a great idea (I 
> sometimes use it in my filenames) but the ability to use regular tools 
> to access old versions of files is great.
> 
> I think it would be really useful to have access to previous versions of 
> files without having to go through checkpoints or snapshots of the whole 
> filesystem / partition. So having something like lshistory or lsversions 
> for specific files would be great. It could output the versions, the 
> checkpoints where those versions are available, etc.  Maybe direct 
> retrieval of old versions (cathistory?) could also be possible using 
> such a path.
> 
> Of course the architecture of NILFS is different from ext3cow so I don't 
> know whether this is even possible. Something like a data structure 
> linking the different verions of some inode must exist before this can 
> happen. Just an idea, though.

If memory serves me right, ext3cow allocates an inode every time its
file is changed and a pointer to the previous inode is stored in the
new inode.

At present, nilfs does not have such per-file pointer to past
versions, so actually it's not optimal for file history lookups.

We may be able to do indirectly by adding a new field to on-disk inode
to keep a checkpoint number in which the inode lastly changed.
However, the new field must be updated every time garbage collector
thins out the previous version, -- this may complicate things
unacceptably because the current nilfs is designed so that it never
overwrites in-use blocks except super blocks.

Apart from these implementation issues, I don't feel the "@" symbol
semantics is odd, though it can come under fire from related kernel
developers as a violation of namespace semantics.


With regards,
Ryusuke Konishi


> Thanks for your cool stuff!
> 
> Evert
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: time-shifting

[Evert Mouw]

Hi,

Op 2011-08-07 16:19, Ryusuke Konishi schreef:
> At present, nilfs does not have such per-file pointer to past
> versions, so actually it's not optimal for file history lookups.
>
> We may be able to do indirectly by adding a new field to on-disk inode
> to keep a checkpoint number in which the inode lastly changed.
> However, the new field must be updated every time garbage collector
> thins out the previous version, -- this may complicate things
> unacceptably because the current nilfs is designed so that it never
> overwrites in-use blocks except super blocks.
Very interesting, thanks for your explanation. I agree that updating 
such a field to a previous checkpoint should not be overwritten. A 
solution would be to not update the field when the garbage collector 
runs. The result is that a pointer to a previous checkpoint could point 
to a non-existing checkpoint. That should not be a problem, because it 
is known which checkpoints do exist, so a simple "if ( ( 
previous-checkpoint == X ) and ( X not element of available checkpoints 
) ) then previous-checkpoint = NULL" would do. That would enable easier 
checks for previous versions, although if there is some chechpoint 
series like "1 2 3 4" and some file exists in all four of those, and if 
you remove checkpoint 2, then this system will never find the oldest 
version in checkpoint 1. But such a scenario is unlikely because the 
garbage collector removes the oldest checkpoints first. Also, this 
solution doesn't need to be perfect to be better than nothing at all.

Just my 2 cents, for I don't have any filesystem development experience.

Regards, Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: time-shifting

[Ryusuke Konishi]

On Mon, 08 Aug 2011 00:17:58 +0200, Evert Mouw wrote:
> Hi,
> 
> Op 2011-08-07 16:19, Ryusuke Konishi schreef:
> > At present, nilfs does not have such per-file pointer to past
> > versions, so actually it's not optimal for file history lookups.
> >
> > We may be able to do indirectly by adding a new field to on-disk inode
> > to keep a checkpoint number in which the inode lastly changed.
> > However, the new field must be updated every time garbage collector
> > thins out the previous version, -- this may complicate things
> > unacceptably because the current nilfs is designed so that it never
> > overwrites in-use blocks except super blocks.
> Very interesting, thanks for your explanation. I agree that updating 
> such a field to a previous checkpoint should not be overwritten. A 
> solution would be to not update the field when the garbage collector 
> runs. The result is that a pointer to a previous checkpoint could point 
> to a non-existing checkpoint. That should not be a problem, because it 
> is known which checkpoints do exist, so a simple "if ( ( 
> previous-checkpoint == X ) and ( X not element of available checkpoints 
> ) ) then previous-checkpoint = NULL" would do. That would enable easier 
> checks for previous versions,

I agree with this let-it-be approach.

> although if there is some chechpoint series like "1 2 3 4" and some
> file exists in all four of those, and if you remove checkpoint 2,
> then this system will never find the oldest version in checkpoint
> 1. But such a scenario is unlikely because the garbage collector
> removes the oldest checkpoints first. Also, this solution doesn't
> need to be perfect to be better than nothing at all.
 
Okay, garbage collector may delete newer checkpoints first, and also
intermediate checkpoints can be deleted manually with rmcp command.
But, the let-it-be approach seems to go well in most cases.  If the
previous checkpoint number is lost, we only have to track back to the
last valid checkpoint younger than it and confirm that the inode
contained in the checkpoint is alive with the same generation number.

Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: time-shifting

[Evert Mouw]

Hi,

Op 2011-08-08 14:28, Ryusuke Konishi schreef:
> [...]
> I agree with this let-it-be approach. [...] If the
> previous checkpoint number is lost, we only have to track back to the
> last valid checkpoint younger than it and confirm that the inode
> contained in the checkpoint is alive with the same generation number.
>
> Regards,
> Ryusuke Konishi
Totally cool.

Nice that it would be possible and workable. Maybe the idea can be put 
in the todo list? I saw a one points on the todo list that might benefit 
from this approach:

- Time slider

The time slider for a single file could be implemented by using the 
proposed extra inode field "previous". (Another approach would be the 
one currently employed by the TimeBrowse program.)

Keep up the good work!

Regards, Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html