Encryption

Encryption

[Pierre Abbat]

How is filesystem encryption going to work? Or has anyone figured that out 
yet?

phma
-- 
.i toljundi do .ibabo mi'afra tu'a do
.ibabo damba do .ibabo do jinga
.icu'u la ma'atman.

Re: Encryption

[Yury Umanets]

Pierre Abbat wrote:

>How is filesystem encryption going to work? Or has anyone figured that out 
>yet?
>
>phma
>  
>
Don't worry, it is going up :)

-- 
Yury Umanets

Re: Encryption

[Hans Reiser]

Yury Umanets wrote:

> Pierre Abbat wrote:
>
>> How is filesystem encryption going to work? Or has anyone figured 
>> that out yet?
>>
>> phma
>>  
>>
> Don't worry, it is going up :)
>
Edward, please provide a serious answer including your design documents, 
so that the list can review your design and comment.  Chances are that 
they will make at least one serious improvement to the design..... and 
it might be nice to get that improvement made before you have made the 
time investment required to debug.....

-- 
Hans

Re: Encryption

[Edward Shushkin]

Hans Reiser wrote:
> 
> Yury Umanets wrote:
> 
> > Pierre Abbat wrote:
> >
> >> How is filesystem encryption going to work? Or has anyone figured
> >> that out yet?
> >>
> >> phma
> >>
> >>
> > Don't worry, it is going up :)
> >
> Edward, please provide a serious answer including your design documents,
> so that the list can review your design and comment. 
> Chances are that
> they will make at least one serious improvement to the design...

Ok, it will be very nice.
Edward.

.. and
> it might be nice to get that improvement made before you have made the
> time investment required to debug.....
> 
> --
> Hans

Snapshots a la NetApp?

[kend]

I'm just wondering if there's any development being done on NetApp-like
snapshots.  (This differs from LVM snapshots in that it's file-by-file,
instead of volume based.)  If not, has anyone given it any consideration? 
It would be a huge win for the RAID-in-a-box folks, and, speaking as a
sysadmin, is something about which I dream frequently.

Just curious...

Ken D'Ambrosio
Sr. SysAdmin,
Xanoptix, Inc.

Re: Snapshots a la NetApp?

[Heinz-Josef Claes]

Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com:
> I'm just wondering if there's any development being done on NetApp-like
> snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> instead of volume based.)  If not, has anyone given it any consideration? 
> It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> sysadmin, is something about which I dream frequently.
> 
> Just curious...
> 
> Ken D'Ambrosio
> Sr. SysAdmin,
> Xanoptix, Inc.

Hi,
you can look at the URL below. It's a kind of snapshot inspired by
NetApp, but has nothing to do with LVM or the filesystem. It also has a
different behaviour.

-- 
Heinz-Josef Claes                     hjclaes@web.de
project: http://sourceforge.net/projects/storebackup
      -> snapshot-like backup to another disk

Re: Snapshots a la NetApp?

[Hans Reiser]

Heinz-Josef Claes wrote:

>Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com:
>  
>
>>I'm just wondering if there's any development being done on NetApp-like
>>snapshots.  (This differs from LVM snapshots in that it's file-by-file,
>>instead of volume based.)  If not, has anyone given it any consideration? 
>>It would be a huge win for the RAID-in-a-box folks, and, speaking as a
>>sysadmin, is something about which I dream frequently.
>>
>>Just curious...
>>
>>Ken D'Ambrosio
>>Sr. SysAdmin,
>>Xanoptix, Inc.
>>    
>>
>
>Hi,
>you can look at the URL below. It's a kind of snapshot inspired by
>NetApp, but has nothing to do with LVM or the filesystem. It also has a
>different behaviour.
>
>  
>
I didn't find the magic click that gave me the design doc for 
storebackup, could you post it to the list?

-- 
Hans

Re: Snapshots a la NetApp?

[Heinz-Josef Claes]

Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser:
> Heinz-Josef Claes wrote:
> 
> >Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com:
> >  
> >
> >>I'm just wondering if there's any development being done on NetApp-like
> >>snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> >>instead of volume based.)  If not, has anyone given it any consideration? 
> >>It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> >>sysadmin, is something about which I dream frequently.
> >>
> >>Just curious...
> >>
> >>Ken D'Ambrosio
> >>Sr. SysAdmin,
> >>Xanoptix, Inc.
> >>    
> >>
> >
> >Hi,
> >you can look at the URL below. It's a kind of snapshot inspired by
> >NetApp, but has nothing to do with LVM or the filesystem. It also has a
> >different behaviour.
> >
> >  
> >
> I didn't find the magic click that gave me the design doc for 
> storebackup, could you post it to the list?

There is no design doc, because it's principal design is so simple :-).
Here is an excerpt from the README file include in the tar file at
sourceforge:

GENERAL FUNCTIONALITY
---------------------

storeBackup is a disk-to-disk backup tool for Linux. It should run on
other Unix like machines. You can directly browse through the backuped
files (locally, via NFS, via SAMBA or whatever). This gives the users
the possibility to restore files absolutely easily and fast. He/She
only has to copy (and possibly uncompress) the file. There is also a
tool for easily restoring (sub) trees for the administrator. Every
single backup of a specific time can be deleted without affecting the
other existing backups.


HOW DOES IT WORK?
-----------------

storeBackup makes a backup of a directory to another direct reachable
location. It does not care about where this location is (same disk,
another disc, via NFS over the network). You should use another disk
or better another computer to store the backup. The target directory
must be on a Unix virtual file system which supports hard links. So
backing up to a SAMBA share is not possible. Naturally, you can also
mount the source directory via NFS and backup in a local
filesystem. In this case, it's good to have a fast network.  The
backup(s) can be seen in a directory in the form date_time
(yyyy.mm.dd_hh.mm.ss) which it creates.

Implemented are several optimizations to reduce disk usage:

- The files to backup are compressed (default bzip2) as discrete files
  in the backup. Files with definable suffixes (like .gz, which is part
  of the default value) will not be compressed. It is possible to
  avoid compression in general.

- If a file with the same contents exists in the previous backup, the
  new backup will only be a hard link to the other one. (This
  mechanism depends on the contents, not on a file name or path!) If you
  rename a file or directory or move sub trees around, it will not cost
  additional space in the backup.

- You can also check older backups than the last one for files with
  the same contents. But this is normally not worth the effort. You
  can also check backups of *other* machines (or backup series)
  for files with the same contents, which can be very efficient.

- If a file with the same contents exists in the same tree to back up,
  it will be hard linked to the other one (and naturally to the older
  ones).

As a result, only changes resulting in different files were stored
(compressed) and require disk space. Normally, the required disk space
for one backup a day for 30 days is less then the required space for the
original. But this depends on the number of changes.


There are several optimizations to improve performance. Normally, the
first backup is much slower than the followings, because all the data
has to be compressed and/or copied. storeBackup is able to take
advantage of multiprocessor machines. From the second run, it should not
be a problem to get more than 100 files/sec with a normal machine of
today. This does not depend on the file size.

There is are special files in the root of the created backup called
.md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2
(default). You must not delete these files. They contain all
information about the original files. You can use this information to
write you own tools beside the existing to restore or to analyze the
backups.

When started, storeBackup will read .md5CheckSums and creates its own
databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you
back up a large number of files (some millions), the required space can
be several dozens of megabytes. If you do not have enough memory to
cache the dbm file, I recommend using a separate hard disk (if
available) for better performance.


LIMITATIONS
-----------

- storeBackup can backup normal files, directories, symbolic links and
  named pipes. Other file types are not supported up to now and will
  generate a warning.

- The permissions in the backup tree(s) are equal to the permissions
  in the original directory. Under special rare conditions it is
  possible, that a user cannot read one ore more of own his/her files
  in the backup. With the restore tool - storeBackupRecover.pl -
  everything is restored with the original permissions.

- storeBackup uses hard links to save disk space. Linux with ext2 file
  system supports up to 32000, reiserfs up to 64535 hard links. If
  storeBackup needs more hard links, it will write a warning and store
  a new (compressed) copy of the file. If you use ext2 for the backup,
  you have to reserve enough (static) inodes!



Hope this helps. As I wrote, it's not depending on the filesystem. It's
depending on files, while NetApp is depending on blocks. Making a
snapshot of a big database file with NetApp is a good idea. Making
backups of big database files with storebackup costs you a lot of space.

On the other side, if you change a normal text file, the block related
algorithm of NetApp will not save much blocks. Compressing the whole
file is much more efficient. And duplicated files, with exist in a great
number in a "normal" file system containing the data of many users (I
also didn't believe this :-)) are saved only once in the backup.

If you want to backup big DB files on linux, there's a better tool for
this use case which uses the algorithm of rsync and stores the original
file and a series of deltas. If I remember well it's called rdiff.
(Sorry, but I never used it.)

-- 
Heinz-Josef Claes                     hjclaes@web.de
project: http://sourceforge.net/projects/storebackup
      -> snapshot-like backup to another disk

Re: Snapshots a la NetApp?

[Tom Vier]

fwiw, there's pdumpfs, which does much the same thing. it's much like
plan9's dump (which writes snapshots to worm media). pdumpfs has worked
great for me. i just use it for my home directory. note that many programs
don't handle hardlinks well. specifically tar, cpio, and rsync (the last two
use TONS of memory even if you have just a few months worth of snapshots).

-- 
Tom Vier <tmv@comcast.net>
DSA Key ID 0xE6CB97DA

Re: Snapshots a la NetApp?

[Heinz-Josef Claes]

Am Die, 2003-03-25 um 01.03 schrieb Tom Vier:
> fwiw, there's pdumpfs, which does much the same thing. it's much like
> plan9's dump (which writes snapshots to worm media). pdumpfs has worked
> great for me. i just use it for my home directory. note that many programs
> don't handle hardlinks well. specifically tar, cpio, and rsync (the last two
> use TONS of memory even if you have just a few months worth of snapshots).

Thanks for the hint - I've never heard before about pdumpfs. I wrote storeBackup
because I could not find such a tool.

A very basic idea of both progs are the same: to use hard links. But the
implementation differs.

The difference is, that pdumpfs depends on paths and file names.
StoreBackup depends on the *contents* of the files, compresses optional
and can share files via hard links between independent backups of different
trees (computers).

The parameters are:
        storeBackup.pl -f configFile [-g | --print]
or
        storeBackup.pl -s sourceDir -t targetDir [-T tmpdir] [-L lockFile]
                [--exceptDirs dir1,dir2,dir3] [--exceptDirsSep sep]
                [--precommand job] [--followLinks depth]
                [-c compress] [-u uncompress] [-p postfix]
                [--noCompress number] [--queueCompress number]
                [--noCopy number] [--queueCopy number]
                [--withUserGroupStat] [--userGroupStatFile filename]
                [--exceptSuffix suffixes]
                [--addExceptSuffix suffixes] [--contExceptDirsErr]
                [--compressMD5File yes|no] [--chmodMD5File] [-v] [-d level]
                [--keepAll timescale] [--keepWeekday entry]
                [--keepMinNumber number] [--keepOnlyLastOfDay timescale]
                [--nodelete] [--progressReport number] [--printDepth]
                [--ignoreReadError]
                [-l logFile
                 [--plusLogStdout] [--withTime yes|no] [-m maxFilelen]
                 [[-n noOfOldFiles] | [--saveLogs yes|no]
                 [--compressWith compressprog]]
                [--logInBackupDir yes|no [--compressLogInBackupDir yes|no]
                 [--logInBackupDirFileName logFile]]
                [otherBackupDirs ...]

best regards,

-- 
Heinz-Josef Claes                     hjclaes@web.de
project: http://sourceforge.net/projects/storebackup
      -> snapshot-like backup to another disk

Re: Snapshots a la NetApp?

[Hans Reiser]

Heinz-Josef Claes wrote:

>Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser:
>  
>
>>Heinz-Josef Claes wrote:
>>
>>    
>>
>>>Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com:
>>> 
>>>
>>>      
>>>
>>>>I'm just wondering if there's any development being done on NetApp-like
>>>>snapshots.  (This differs from LVM snapshots in that it's file-by-file,
>>>>instead of volume based.)  If not, has anyone given it any consideration? 
>>>>It would be a huge win for the RAID-in-a-box folks, and, speaking as a
>>>>sysadmin, is something about which I dream frequently.
>>>>
>>>>Just curious...
>>>>
>>>>Ken D'Ambrosio
>>>>Sr. SysAdmin,
>>>>Xanoptix, Inc.
>>>>   
>>>>
>>>>        
>>>>
>>>Hi,
>>>you can look at the URL below. It's a kind of snapshot inspired by
>>>NetApp, but has nothing to do with LVM or the filesystem. It also has a
>>>different behaviour.
>>>
>>> 
>>>
>>>      
>>>
>>I didn't find the magic click that gave me the design doc for 
>>storebackup, could you post it to the list?
>>    
>>
>
>There is no design doc, because it's principal design is so simple :-).
>Here is an excerpt from the README file include in the tar file at
>sourceforge:
>
>GENERAL FUNCTIONALITY
>---------------------
>
>storeBackup is a disk-to-disk backup tool for Linux. It should run on
>other Unix like machines. You can directly browse through the backuped
>files (locally, via NFS, via SAMBA or whatever). This gives the users
>the possibility to restore files absolutely easily and fast. He/She
>only has to copy (and possibly uncompress) the file. There is also a
>tool for easily restoring (sub) trees for the administrator. Every
>single backup of a specific time can be deleted without affecting the
>other existing backups.
>
>
>HOW DOES IT WORK?
>-----------------
>
>storeBackup makes a backup of a directory to another direct reachable
>location. It does not care about where this location is (same disk,
>another disc, via NFS over the network). You should use another disk
>or better another computer to store the backup. The target directory
>must be on a Unix virtual file system which supports hard links. So
>backing up to a SAMBA share is not possible. Naturally, you can also
>mount the source directory via NFS and backup in a local
>filesystem. In this case, it's good to have a fast network.  The
>backup(s) can be seen in a directory in the form date_time
>(yyyy.mm.dd_hh.mm.ss) which it creates.
>
>Implemented are several optimizations to reduce disk usage:
>
>- The files to backup are compressed (default bzip2) as discrete files
>  in the backup. Files with definable suffixes (like .gz, which is part
>  of the default value) will not be compressed. It is possible to
>  avoid compression in general.
>
>- If a file with the same contents exists in the previous backup, the
>  new backup will only be a hard link to the other one. (This
>  mechanism depends on the contents, not on a file name or path!) If you
>  rename a file or directory or move sub trees around, it will not cost
>  additional space in the backup.
>
>- You can also check older backups than the last one for files with
>  the same contents. But this is normally not worth the effort. You
>  can also check backups of *other* machines (or backup series)
>  for files with the same contents, which can be very efficient.
>
>- If a file with the same contents exists in the same tree to back up,
>  it will be hard linked to the other one (and naturally to the older
>  ones).
>
>As a result, only changes resulting in different files were stored
>(compressed) and require disk space. Normally, the required disk space
>for one backup a day for 30 days is less then the required space for the
>original. But this depends on the number of changes.
>
>
>There are several optimizations to improve performance. Normally, the
>first backup is much slower than the followings, because all the data
>has to be compressed and/or copied. storeBackup is able to take
>advantage of multiprocessor machines. From the second run, it should not
>be a problem to get more than 100 files/sec with a normal machine of
>today. This does not depend on the file size.
>
>There is are special files in the root of the created backup called
>.md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2
>(default). You must not delete these files. They contain all
>information about the original files. You can use this information to
>write you own tools beside the existing to restore or to analyze the
>backups.
>
>When started, storeBackup will read .md5CheckSums and creates its own
>databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you
>back up a large number of files (some millions), the required space can
>be several dozens of megabytes. If you do not have enough memory to
>cache the dbm file, I recommend using a separate hard disk (if
>available) for better performance.
>
>
>LIMITATIONS
>-----------
>
>- storeBackup can backup normal files, directories, symbolic links and
>  named pipes. Other file types are not supported up to now and will
>  generate a warning.
>
>- The permissions in the backup tree(s) are equal to the permissions
>  in the original directory. Under special rare conditions it is
>  possible, that a user cannot read one ore more of own his/her files
>  in the backup. With the restore tool - storeBackupRecover.pl -
>  everything is restored with the original permissions.
>
>- storeBackup uses hard links to save disk space. Linux with ext2 file
>  system supports up to 32000, reiserfs up to 64535 hard links. If
>  storeBackup needs more hard links, it will write a warning and store
>  a new (compressed) copy of the file. If you use ext2 for the backup,
>  you have to reserve enough (static) inodes!
>
>
>
>Hope this helps. As I wrote, it's not depending on the filesystem. It's
>depending on files, while NetApp is depending on blocks. Making a
>snapshot of a big database file with NetApp is a good idea. Making
>backups of big database files with storebackup costs you a lot of space.
>
>On the other side, if you change a normal text file, the block related
>algorithm of NetApp will not save much blocks. Compressing the whole
>file is much more efficient. And duplicated files, with exist in a great
>number in a "normal" file system containing the data of many users (I
>also didn't believe this :-)) are saved only once in the backup.
>
>If you want to backup big DB files on linux, there's a better tool for
>this use case which uses the algorithm of rsync and stores the original
>file and a series of deltas. If I remember well it's called rdiff.
>(Sorry, but I never used it.)
>
>  
>
Flx, should we try this for our backups?

-- 
Hans

Re: Snapshots a la NetApp?

[Heinz-Josef Claes]

Am Die, 2003-03-25 um 03.35 schrieb Hans Reiser:
> Heinz-Josef Claes wrote:
> 
> >Am Mon, 2003-03-24 um 21.49 schrieb Hans Reiser:
> >  
> >
> >>Heinz-Josef Claes wrote:
> >>
> >>    
> >>
> >>>Am Son, 2003-03-23 um 04.38 schrieb kend@xanoptix.com:
> >>> 
> >>>
> >>>      
> >>>
> >>>>I'm just wondering if there's any development being done on NetApp-like
> >>>>snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> >>>>instead of volume based.)  If not, has anyone given it any consideration? 
> >>>>It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> >>>>sysadmin, is something about which I dream frequently.
> >>>>
> >>>>Just curious...
> >>>>
> >>>>Ken D'Ambrosio
> >>>>Sr. SysAdmin,
> >>>>Xanoptix, Inc.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Hi,
> >>>you can look at the URL below. It's a kind of snapshot inspired by
> >>>NetApp, but has nothing to do with LVM or the filesystem. It also has a
> >>>different behaviour.
> >>>
> >>> 
> >>>
> >>>      
> >>>
> >>I didn't find the magic click that gave me the design doc for 
> >>storebackup, could you post it to the list?
> >>    
> >>
> >
> >There is no design doc, because it's principal design is so simple :-).
> >Here is an excerpt from the README file include in the tar file at
> >sourceforge:
> >
> >GENERAL FUNCTIONALITY
> >---------------------
> >
> >storeBackup is a disk-to-disk backup tool for Linux. It should run on
> >other Unix like machines. You can directly browse through the backuped
> >files (locally, via NFS, via SAMBA or whatever). This gives the users
> >the possibility to restore files absolutely easily and fast. He/She
> >only has to copy (and possibly uncompress) the file. There is also a
> >tool for easily restoring (sub) trees for the administrator. Every
> >single backup of a specific time can be deleted without affecting the
> >other existing backups.
> >
> >
> >HOW DOES IT WORK?
> >-----------------
> >
> >storeBackup makes a backup of a directory to another direct reachable
> >location. It does not care about where this location is (same disk,
> >another disc, via NFS over the network). You should use another disk
> >or better another computer to store the backup. The target directory
> >must be on a Unix virtual file system which supports hard links. So
> >backing up to a SAMBA share is not possible. Naturally, you can also
> >mount the source directory via NFS and backup in a local
> >filesystem. In this case, it's good to have a fast network.  The
> >backup(s) can be seen in a directory in the form date_time
> >(yyyy.mm.dd_hh.mm.ss) which it creates.
> >
> >Implemented are several optimizations to reduce disk usage:
> >
> >- The files to backup are compressed (default bzip2) as discrete files
> >  in the backup. Files with definable suffixes (like .gz, which is part
> >  of the default value) will not be compressed. It is possible to
> >  avoid compression in general.
> >
> >- If a file with the same contents exists in the previous backup, the
> >  new backup will only be a hard link to the other one. (This
> >  mechanism depends on the contents, not on a file name or path!) If you
> >  rename a file or directory or move sub trees around, it will not cost
> >  additional space in the backup.
> >
> >- You can also check older backups than the last one for files with
> >  the same contents. But this is normally not worth the effort. You
> >  can also check backups of *other* machines (or backup series)
> >  for files with the same contents, which can be very efficient.
> >
> >- If a file with the same contents exists in the same tree to back up,
> >  it will be hard linked to the other one (and naturally to the older
> >  ones).
> >
> >As a result, only changes resulting in different files were stored
> >(compressed) and require disk space. Normally, the required disk space
> >for one backup a day for 30 days is less then the required space for the
> >original. But this depends on the number of changes.
> >
> >
> >There are several optimizations to improve performance. Normally, the
> >first backup is much slower than the followings, because all the data
> >has to be compressed and/or copied. storeBackup is able to take
> >advantage of multiprocessor machines. From the second run, it should not
> >be a problem to get more than 100 files/sec with a normal machine of
> >today. This does not depend on the file size.
> >
> >There is are special files in the root of the created backup called
> >.md5CheckSums.info and .md5CheckSums or .md5CheckSums.bz2
> >(default). You must not delete these files. They contain all
> >information about the original files. You can use this information to
> >write you own tools beside the existing to restore or to analyze the
> >backups.
> >
> >When started, storeBackup will read .md5CheckSums and creates its own
> >databases (dbm file) in $TMPDIR or --tmpdir (default is /tmp). If you
> >back up a large number of files (some millions), the required space can
> >be several dozens of megabytes. If you do not have enough memory to
> >cache the dbm file, I recommend using a separate hard disk (if
> >available) for better performance.
> >
> >
> >LIMITATIONS
> >-----------
> >
> >- storeBackup can backup normal files, directories, symbolic links and
> >  named pipes. Other file types are not supported up to now and will
> >  generate a warning.
> >
> >- The permissions in the backup tree(s) are equal to the permissions
> >  in the original directory. Under special rare conditions it is
> >  possible, that a user cannot read one ore more of own his/her files
> >  in the backup. With the restore tool - storeBackupRecover.pl -
> >  everything is restored with the original permissions.
> >
> >- storeBackup uses hard links to save disk space. Linux with ext2 file
> >  system supports up to 32000, reiserfs up to 64535 hard links. If
> >  storeBackup needs more hard links, it will write a warning and store
> >  a new (compressed) copy of the file. If you use ext2 for the backup,
> >  you have to reserve enough (static) inodes!
> >
> >
> >
> >Hope this helps. As I wrote, it's not depending on the filesystem. It's
> >depending on files, while NetApp is depending on blocks. Making a
> >snapshot of a big database file with NetApp is a good idea. Making
> >backups of big database files with storebackup costs you a lot of space.
> >
> >On the other side, if you change a normal text file, the block related
> >algorithm of NetApp will not save much blocks. Compressing the whole
> >file is much more efficient. And duplicated files, with exist in a great
> >number in a "normal" file system containing the data of many users (I
> >also didn't believe this :-)) are saved only once in the backup.
> >
> >If you want to backup big DB files on linux, there's a better tool for
> >this use case which uses the algorithm of rsync and stores the original
> >file and a series of deltas. If I remember well it's called rdiff.
> >(Sorry, but I never used it.)
> >
> >  
> >
> Flx, should we try this for our backups?

If you test it, I would be glad to hear about your experiences, because
I believe that you must be a stress tester by birth :-)

btw: The only handycap of the used method is that all the links to a
file can have only *one* set of permissions (chmod, chown). Will it be
possible to change this with a loadable module in reiser4?

best regards,
HJC

Re: Snapshots a la NetApp?

[Hans Reiser]

Heinz-Josef Claes wrote:

>
>btw: The only handycap of the used method is that all the links to a
>file can have only *one* set of permissions (chmod, chown). Will it be
>possible to change this with a loadable module in reiser4?
>
>best regards,
>HJC
>
>
>
>  
>
I am not sure if we will support that.  If someone wanted to write the 
code to do it, then I could examine the problem more closely and it 
could be done.....

-- 
Hans

Re: Snapshots a la NetApp?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

On Wed, 26 Mar 2003 21:37:10 +0300, Hans Reiser said:
> Heinz-Josef Claes wrote:
> >btw: The only handycap of the used method is that all the links to a
> >file can have only *one* set of permissions (chmod, chown). Will it be
> >possible to change this with a loadable module in reiser4?

> I am not sure if we will support that.  If someone wanted to write the 
> code to do it, then I could examine the problem more closely and it 
> could be done.....

Please don't.

There is just *WAY* too much Unix/Linux semantics that are dependent on the
concept that an inode describes a file, with all it implies - the fact that all
hard links have the same owner/perms (hint - how would you store 2 inodes that
had different owner/perm/mtime/ctime/etc but the same block list, and still
have it fsck correctly?  If you have a linked list of owners/perms, how do you
determine which one to use? etc etc..)  I'm not even going into the aspects
like programs that creat() and then unlink() in order to get a anonymous temp
file that goes away at last close, etc etc....

You want different permissions, use ACLs.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: unsubscribe

[Voicu Liviu]

unsubscribe

Re: Snapshots a la NetApp?

[Lars Marowsky-Bree]

On 2003-03-22T22:38:01,
   kend@xanoptix.com said:

> I'm just wondering if there's any development being done on NetApp-like
> snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> instead of volume based.)  If not, has anyone given it any consideration? 
> It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> sysadmin, is something about which I dream frequently.

Well, implementing it at the LVM level isn't entirely bad though. What kind of
advantage would have implementing at the filesystem level have? It would need
to compensate at least the disadvantage of being less general.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: Snapshots a la NetApp?

[Ragnar Kjørstad]

On Sun, Mar 23, 2003 at 12:18:31PM +0100, Lars Marowsky-Bree wrote:
> > I'm just wondering if there's any development being done on NetApp-like
> > snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> > instead of volume based.)  If not, has anyone given it any consideration? 
> > It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> > sysadmin, is something about which I dream frequently.
> 
> Well, implementing it at the LVM level isn't entirely bad though. What kind of
> advantage would have implementing at the filesystem level have? It would need
> to compensate at least the disadvantage of being less general.

I think the main advantage of having it in the filesystem is that it
makes the snapshots available to the users. This makes it possible for
regular users to undelete / restore files without having to contact the
sysadmin. 

As for linux implementations of NetApp-like snapshots you should check
out snapfs: 

http://www.mountainviewdata.com/us/product/product_snap_1.html
http://www-mddsp.enel.ucalgary.ca/People/adilger/snapfs/
http://sourcefrog.net/projects/snapfs/


-- 
Ragnar Kjørstad
Zet.no

Re: Snapshots a la NetApp?

[Oleg Drokin]

Hello!

On Sat, Mar 22, 2003 at 10:38:01PM -0500, kend@xanoptix.com wrote:
> I'm just wondering if there's any development being done on NetApp-like
> snapshots.  (This differs from LVM snapshots in that it's file-by-file,
> instead of volume based.)  If not, has anyone given it any consideration? 
> It would be a huge win for the RAID-in-a-box folks, and, speaking as a
> sysadmin, is something about which I dream frequently.

That sounds like file versioning on filesystem, perhaps
filesystem (binary only and stuff) from ClearCase suite (or what was the name)
from Rational have this.
Though I have not tried it myself.
And their http://www.rational.com does not answers. May be related to the fact
that they were bought by IBM recently ;)

Bye,
    Oleg

RE: Snapshots a la NetApp?

[Barry, Christopher]

cruft snippage....

>>>  
>>>
>> Flx, should we try this for our backups?

>If you test it, I would be glad to hear about your experiences, because
>I believe that you must be a stress tester by birth :-)
>
>btw: The only handycap of the used method is that all the links to a
>file can have only *one* set of permissions (chmod, chown). Will it be
>possible to change this with a loadable module in reiser4?
>
>best regards,
>HJC


HJC,


Under what circumstances would this be a handicap? Do you currently have the same file in multiple locations with different perms? Shouldn't this be controllable via groups?


--
Christopher Barry
Manager of Information Systems
InfiniCon Systems
http://www.infiniconsys.com

RE: Snapshots a la NetApp?

[Heinz-Josef Claes]

Am Mit, 2003-03-26 um 18.54 schrieb Barry, Christopher:
> cruft snippage....
> 
> >>>  
> >>>
> >> Flx, should we try this for our backups?
> 
> >If you test it, I would be glad to hear about your experiences, because
> >I believe that you must be a stress tester by birth :-)
> >
> >btw: The only handycap of the used method is that all the links to a
> >file can have only *one* set of permissions (chmod, chown). Will it be
> >possible to change this with a loadable module in reiser4?
> >
> >best regards,
> >HJC
> 
> 
> HJC,
> 
> 
> Under what circumstances would this be a handicap? Do you currently have the same file in multiple locations with different perms? Shouldn't this be controllable via groups?
> 
> 
> --
> Christopher Barry
> Manager of Information Systems
> InfiniCon Systems
> http://www.infiniconsys.com
> 

I never saw the problem in real live. But it can happen:

Users 'tim' and 'bob' have the same file in their directories.
In this example both of them have set the permissions to 0600.
In the backup, only one file exists with permissions 0600 and belongs to
one of them, let's say 'bob'.

So 'tim' is not able to restore the file by simply copying it with a
file browser (or simply cp).

With the tool 'storeBackupRecover.pl', which must run as root and uses a
table for restoring the users and permissions, it's no problem. (btw, it
can also restore hard links.)

It's not controllable via groups because the two users perhaps are not
and must not be in the same group. The file can e.g. be transported via
email. I think, it could be controlled via ACLs, but they are not
standarised. One the other hand, the problem will not pop up often and
can be resolved easily (=restoring the file(s)) with the system
administrator.

-- 
Heinz-Josef Claes                     hjclaes@web.de
project: http://sourceforge.net/projects/storebackup
      -> snapshot-like backup to another disk