My Dad suggests a redundant copies plugin

My Dad suggests a redundant copies plugin

[Hans Reiser]

This would not be (at least in theory) useful for RAID devices, but for
a user with a single disk drive, it might be useful to have a plugin
that creates two (or N) copies, and tries to allocate the two copies at
opposite ends of the disk.  Anyone out there still looking for a plugin
to write?

Hans

Re: My Dad suggests a redundant copies plugin

[Sander]

Hans Reiser wrote (ao):
> This would not be (at least in theory) useful for RAID devices, but for
> a user with a single disk drive, it might be useful to have a plugin
> that creates two (or N) copies, and tries to allocate the two copies at
> opposite ends of the disk. Anyone out there still looking for a plugin
> to write?

That will kill performance badly. First of all the two read/writes
needed, and second because you have to seek from one end to the disk to
the other every time you read/write something.

And what is the advantage? You are not protected against a lot of disk
failures (only against bad blocks, right?).

There is a lot more advantage in buying two disks and do raid over them.
This is (much) cheaper, gives better performance and gives more
protection.

Anyway, no need for a plugin. You can just divide your disk in two
partitions and configure them as a raid1.

Or am I missing something in your suggestion?

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: My Dad suggests a redundant copies plugin

[Hans Reiser]

Sander wrote:

>Hans Reiser wrote (ao):
>  
>
>>This would not be (at least in theory) useful for RAID devices, but for
>>a user with a single disk drive, it might be useful to have a plugin
>>that creates two (or N) copies, and tries to allocate the two copies at
>>opposite ends of the disk. Anyone out there still looking for a plugin
>>to write?
>>    
>>
>
>That will kill performance badly. First of all the two read/writes
>needed, and second because you have to seek from one end to the disk to
>the other every time you read/write something.
>
>And what is the advantage? You are not protected against a lot of disk
>failures (only against bad blocks, right?).
>
>There is a lot more advantage in buying two disks and do raid over them.
>This is (much) cheaper, gives better performance and gives more
>protection.
>
>Anyway, no need for a plugin. You can just divide your disk in two
>partitions and configure them as a raid1.
>
>Or am I missing something in your suggestion?
>
>  
>
It is only for very important files for computers which have only one
hard drive.  Some of the work is with changing fsck.....

Hans

Re: My Dad suggests a redundant copies plugin

[Sander]

Hans Reiser wrote (ao):
> It is only for very important files for computers which have only one
> hard drive. Some of the work is with changing fsck.....

Well, if the files are important, then you should have backups anyway,
whatever raid or similar you have. I still don't see an advantage in
having two versions of a file on one disk.

But then again, that should not hinder anyone :-)

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: My Dad suggests a redundant copies plugin

[Konstantin Münning]

Sander wrote:
> Hans Reiser wrote (ao):
> 
>>It is only for very important files for computers which have only one
>>hard drive. Some of the work is with changing fsck.....
> 
> 
> Well, if the files are important, then you should have backups anyway,
> whatever raid or similar you have. I still don't see an advantage in
> having two versions of a file on one disk.

Here is one: something bad happens with your FS so you need to fsck,
rebuild-tree (or what the corresponding thing for reiser4 is) or
something like this. Having important files duplicated improves the
chance that at least one copy is still intact afterwards as depending on
the grade of corruption rebuilt fs may show the same files but contents
may differ. (verifying which copy is OK is not in the scope of this writing)

Yes, having backups is better but that wouldn't help if it's your
laptop, the backups are 2000 miles away in your office and you have only
some Linux Boot CD?! ;-)

I know, that doesn't happen often but I've had similar situations. As
I've allways managed to help me somehow missing this feature wouldn't
make me cry but I would probably use it. Just my two cents.

> But then again, that should not hinder anyone :-)

Re: My Dad suggests a redundant copies plugin

[Ingo Bormuth]

On 2005-10-25 02:28, Hans Reiser wrote:
>
> >And what is the advantage? You are not protected against a lot of disk
> >failures (only against bad blocks, right?).
>
> It is only for very important files for computers which have only one
> hard drive.  Some of the work is with changing fsck.....
> 

I agree, real backups are the major weappon against classical data loss due 
to hardware failure. 
Other quite anoying and common causes for data loss are accidentally deleted, 
overwritten or modified files. A _simple_ versioning plugin would be very
nice to have (I'd definitly use it in /etc).

Anyway, oun could then easily add the proposed redundancy feature (implemented
as the files version number zero).




-- 
Ingo Bormuth, voicebox & telefax: +49-12125-10226517          '(~o-o~)'
public key 86326EC9, http://ibormuth.efil.de/contact     ---ooO--(.)--Ooo---

Re: My Dad suggests a redundant copies plugin

[David Masover]

Konstantin Münning wrote:
> Sander wrote:

> Yes, having backups is better but that wouldn't help if it's your
> laptop, the backups are 2000 miles away in your office and you have only
> some Linux Boot CD?! ;-)

Many see Internet access as an essential utility, like running water.
If I have Internet, I can get to my backups.  And if I have a laptop
with important stuff on it, I'll have it backup and keep synced whenever
there is Internet -- probably something like drbd.

Much cheaper to engineer that solution, and it protects you from a hard
drive failure -- you boot your Knoppix, ssh to home, and keep working.

Re: My Dad suggests a redundant copies plugin

[Charles P. Wright]

On Tue, 2005-10-25 at 09:25 +0200, Sander wrote:
> Hans Reiser wrote (ao):
> > This would not be (at least in theory) useful for RAID devices, but for
> > a user with a single disk drive, it might be useful to have a plugin
> > that creates two (or N) copies, and tries to allocate the two copies at
> > opposite ends of the disk. Anyone out there still looking for a plugin
> > to write?
> 
> That will kill performance badly. First of all the two read/writes
> needed, and second because you have to seek from one end to the disk to
> the other every time you read/write something.
> 
> And what is the advantage? You are not protected against a lot of disk
> failures (only against bad blocks, right?).
> 
> There is a lot more advantage in buying two disks and do raid over them.
> This is (much) cheaper, gives better performance and gives more
> protection.
> 
> Anyway, no need for a plugin. You can just divide your disk in two
> partitions and configure them as a raid1.
There is an interesting paper from UWISC in SOSP (the one going on right
now) about file system reliability.  They actually claim that there are
quite a few situations in which disks break not as a whole unit.  The
enhanced Ext3 to have clearly defined failure semantics, and be a bit
more tolerant of disk failures.   If anyone wants to tackle this sort of
plugin, their paper is a must read.

Another interesting, but separate, thing in that paper is that they
checksum each journal transaction and write the checksum at the end.
Because they can tell if the journal transaction is valid through the
checksum, they can avoid all synchronous journal writes.

Charles

My Dad suggests a redundant copies plugin

[Gregory Maxwell]

On 10/25/05, Hans Reiser <reiser@namesys.com> wrote:
> This would not be (at least in theory) useful for RAID devices, but for
> a user with a single disk drive, it might be useful to have a plugin
> that creates two (or N) copies, and tries to allocate the two copies at
> opposite ends of the disk.  Anyone out there still looking for a plugin
> to write?

It would be more important to have redundancy for important filesystem
datastructures first, because if you lose your filesystem it becomes
very difficult to read your data.

With modern high density drives bad blocks have become more common
than disk failures, redundancy makes a lot of sense but the cost of
duplicating every file is simply too great for many applications. The
ability to control the level of redundancy and protection from the
filesystem is becoming increasingly important.

Hans, have you read the Iron filesystems paper
(http://www.cs.wisc.edu/wind/Publications/iron-sosp05.ps) that was
previously cited on this list?  You should at least skim it.

Fwd: My Dad suggests a redundant copies plugin

[Gregory Maxwell]

On 10/25/05, Sander <sander@humilis.net> wrote:
> That will kill performance badly. First of all the two read/writes
> needed, and second because you have to seek from one end to the disk to
> the other every time you read/write something.

Kill it worse for writes than a filesytem without wandering logs? I
don't think so, since they double write in any case... Since reiser4
defers flushes you don't end up seeking all over the disk, you will
write out a nice long queue, then a little seeking to ripple up the
transaction(s).

Reading will not be harmed since we will expect the underlying disk to
report read failures.

> And what is the advantage? You are not protected against a lot of disk
> failures (only against bad blocks, right?).

Badblocks have become the most common non-transient failure mode of
disks by far.

> There is a lot more advantage in buying two disks and do raid over them.
> This is (much) cheaper, gives better performance and gives more
> protection.

Cheaper how? Two disks are cheaper than one?

> Anyway, no need for a plugin. You can just divide your disk in two
> partitions and configure them as a raid1.
>
> Or am I missing something in your suggestion?

If I do not want, and can not afford, that level of protection for all
my data, but only some, RAID-1 is a very wasteful solution.

Re: My Dad suggests a redundant copies plugin

[Gregory Maxwell]

On 10/25/05, Ingo Bormuth <ibormuth@efil.de> wrote:
> I agree, real backups are the major weappon against classical data loss due
> to hardware failure.
> Other quite anoying and common causes for data loss are accidentally deleted,
> overwritten or modified files. A _simple_ versioning plugin would be very
> nice to have (I'd definitly use it in /etc).

It would be a sin to implement versioning in resier4 without taking
advantage of how transactions work since done right they can provide
this with almost zero overhead... plus it can be difficult to make
sure that you've got a consistent copy of the file because of how many
applications update files (is a version a write() call? no that won't
work.. so do we only version files that get unlinked and replaced?). 
Between these two things a simple implementation wouldn't be so
simple.

Re: Fwd: My Dad suggests a redundant copies plugin

[Hans Reiser]

Please remember that the idea of the plugin is that it is selectively
used on a few files.

Re: Fwd: My Dad suggests a redundant copies plugin

[Sander]

Gregory Maxwell wrote (ao):
> On 10/25/05, Sander <sander@humilis.net> wrote:
> > That will kill performance badly. First of all the two read/writes
> > needed, and second because you have to seek from one end to the disk
> > to the other every time you read/write something.
> 
> Kill it worse for writes than a filesytem without wandering logs? I
> don't think so, since they double write in any case... Since reiser4
> defers flushes you don't end up seeking all over the disk, you will
> write out a nice long queue, then a little seeking to ripple up the
> transaction(s).
> 
> Reading will not be harmed since we will expect the underlying disk to
> report read failures.

Either way, more reads and writes are need with such a plugin, or am I
missing something?

> > And what is the advantage? You are not protected against a lot of
> > disk failures (only against bad blocks, right?).
> 
> Badblocks have become the most common non-transient failure mode of
> disks by far.

That is not my experience. Where does this knowledge come from?

> > There is a lot more advantage in buying two disks and do raid over
> > them. This is (much) cheaper, gives better performance and gives
> > more protection.
> 
> Cheaper how? Two disks are cheaper than one?

If you opt for a copy of everything, yes. Because of a 500GB drive only
half of it is effectively usable. Two 250GB disks raid1 are much cheaper
than one 500GB disk.

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: Fwd: My Dad suggests a redundant copies plugin

[David Masover]

Sander wrote:
> Gregory Maxwell wrote (ao):
> 
>>On 10/25/05, Sander <sander@humilis.net> wrote:


>>Cheaper how? Two disks are cheaper than one?
> 
> 
> If you opt for a copy of everything, yes. Because of a 500GB drive only
> half of it is effectively usable. Two 250GB disks raid1 are much cheaper
> than one 500GB disk.

But not cheaper than one 300 gig disk.  Remember, out of 500 gigs, how
much of that is essential?

If you're doing some sort of media, like video editing, probably a lot,
so you get the two 250 gig disks.  But if you're an author, say, your
life's work probably fits on a single floppy (if you actually use text
files) or at least fifty times over on a CD (if you use some bloaty
office suite).  A programmer?  My entire kernel directory is less than
300 megs, and I haven't run "make clean" recently.

Metadata shouldn't take more than 5% of the disk.

So, if your disk starts going bad and you need to recover, and you lose
a program, so what?  You can still recover the files.  For that matter,
how many people here would be devastated if they lost a .o file?
Probably very few people here have a .o file without a matching .c file,
so if your .o goes, you run "gcc" and you're fine.

Much cheaper, easier than a full backup, easier than RAID, and still
keeps you from losing everything....

Except, you still are your own worst enemy.  I don't think I've ever
lost data to hardware, it was always my own stupidity.