Re: reiserFS?

Re: reiserFS?

[Xavier Roche]

> It simply the best filesystem for many kinds of usage patterns.

The most frightening too. Reiserfs might be suitable for very specific appliactions, but to use it in production machine, you need to have some guts.

My last reiserfs partition was blown up two days ago, because of a bad sector, plus a fatal oops, looping endlessly. This was the second time, and the last one, as none of my ext3 filesystems *ever* had similar problems, despite numerous other bad sector issues. Not mentionning the funny "recovery" tool, which generally finishes to trash your data.

Jul 14 23:35:29 linux kernel: hdh: dma_intr: status=0x51 { DriveReady SeekComplete Error }
Jul 14 23:35:29 linux kernel: hdh: dma_intr: error=0x40 { UncorrectableError }, LBAsect=12458384, sector=12458383
Jul 14 23:35:29 linux kernel: ide: failed opcode was: unknown
Jul 14 23:35:29 linux kernel: end_request: I/O error, dev hdh, sector 12458383
Jul 14 23:35:29 linux kernel: ------------[ cut here ]------------
Jul 14 23:35:29 linux kernel: kernel BUG at fs/reiserfs/file.c:620!
..
Jul 14 23:35:29 linux kernel:  <0>Fatal exception: panic in 5 seconds

The funny part is that 14 july is the french's fireworks day, generally launched around midnight.

Re: reiserFS?

[Christian Trefzer]

[-- Attachment #1: Type: text/plain, Size: 1397 bytes --]

On Sun, Jul 16, 2006 at 06:16:31PM +0200, Xavier Roche wrote:
> > It simply the best filesystem for many kinds of usage patterns.
> 
> The most frightening too. Reiserfs might be suitable for very specific
> appliactions, but to use it in production machine, you need to have
> some guts.
> 
> My last reiserfs partition was blown up two days ago, because of a bad
> sector, plus a fatal oops, looping endlessly. This was the second
> time, and the last one, as none of my ext3 filesystems *ever* had
> similar problems, despite numerous other bad sector issues. Not
> mentionning the funny "recovery" tool, which generally finishes to
> trash your data.

I don't quite understand. You are supposed to dd_rescue the whole block
device to a working drive and use fsck on the copy.  Whatever is lost in
the process must of course be restored from a recent backup. But, as a
friend of mine put it recently, people don't need backup, they only need
restore ; )

fsck on a faulty drive might cause even more damage - how do you know
that other areas of the device are OK? 

I also oppose the ReiserFS-v3.x design philosophy regarding faulty
storage layer, but in any case where your _live_ data is valuable and
uptime counts, you _really_should_ use a RAID of some sort.

Kind regards,
uziel

PS: Your mail was line-wrapped really bad, you might want to look into
that.

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: reiserFS?

[Theodore Tso]

On Sun, Jul 16, 2006 at 06:28:31PM +0200, Christian Trefzer wrote:
> I don't quite understand. You are supposed to dd_rescue the whole block
> device to a working drive and use fsck on the copy.  Whatever is lost in
> the process must of course be restored from a recent backup. But, as a
> friend of mine put it recently, people don't need backup, they only need
> restore ; )

If the disk is known to be bad, yes, and the number of bad blocks is
growing.  On the other hand, disks can and will have a few bad blocks,
or bad writes that don't mean the disk is going bad, and a modern
filesystem should be robust enough that a single failed sector doesn't
cause the filesystem to go completely kaput.

In fact, one of the scary trends with hard drives is that size is
continuing to grow expoentially, access times linearly (more or less),
and error rates (errors per kilobytes per unit time) are remaining
more or less constant.

The fact that reiserfs uses a single B-tree to store all of its data
means that very entertaining things can happen if you lose a sector
containing a high-level node in the tree.  It's even more entertaining
if you have image files (like initrd files) in reiserfs format stored
in reiserfs, and you run the recovery program on the filesystem.....

Yes, I know that reiserfs4 is alleged to fix this problem, but as far
as I know it is still using a single unitary tree, with all of the
pitfalls that this entails.

Now, that being said, that by itself is not a reason not to decide not
to include reseirfs4 into the mainline sources.  (I might privately
get amused when system administrators use reiserfs and then report
massive data loss, but that's my own failure of chairty; I'm working
on it.)  For the technical reasons why resierfs4 hasn't been
integrated, please see the mailing list archives.

						- Ted

Re: reiserFS?

[Lexington Luthor]

Theodore Tso wrote:
> If the disk is known to be bad, yes, and the number of bad blocks is
> growing.  On the other hand, disks can and will have a few bad blocks,
> or bad writes that don't mean the disk is going bad, and a modern
> filesystem should be robust enough that a single failed sector doesn't
> cause the filesystem to go completely kaput.

I never trust a single hard drive with data that cannot be instantly 
re-generated anyway (eg squid caches). The fact that some people have 
hardware errors should not require every single fs to accommodate random 
bad-sectors. Feel free to use ext3 or other fs which handles this 
situation (and other situations) better, but reiserfs works fine on good 
hardware. It has been my root filesystem on many systems with no 
problems whatsoever, even with cheap SATA drives.

> In fact, one of the scary trends with hard drives is that size is
> continuing to grow expoentially, access times linearly (more or less),
> and error rates (errors per kilobytes per unit time) are remaining
> more or less constant.
> 
> The fact that reiserfs uses a single B-tree to store all of its data
> means that very entertaining things can happen if you lose a sector
> containing a high-level node in the tree.  It's even more entertaining
> if you have image files (like initrd files) in reiserfs format stored
> in reiserfs, and you run the recovery program on the filesystem.....
> 
> Yes, I know that reiserfs4 is alleged to fix this problem, but as far
> as I know it is still using a single unitary tree, with all of the
> pitfalls that this entails.
> 
> Now, that being said, that by itself is not a reason not to decide not
> to include reseirfs4 into the mainline sources.  (I might privately
> get amused when system administrators use reiserfs and then report
> massive data loss, but that's my own failure of chairty; I'm working
> on it.)  For the technical reasons why resierfs4 hasn't been
> integrated, please see the mailing list archives.
> 

I read the archives, and most of the problems pointed out during the 
review were fixed relatively quickly, followed by a flame war due to 
some suggesting that reiser4 should not be able to affect VFS semantics, 
and other such matters (which IMO should be outside of the scope of a 
code review). There has been no follow-up review as far as I can tell. 
The discussion quickly degenerated into a personality argument against 
Mr Reiser, with several developers taking a strong position against 
reiser4 (the exact reasons for which are not specified).

I don't quite know where reiser4 stands at the moment, given that it is 
in -mm and has been for a very long time. I also looked at the patch 
again, and it is indeed quite well isolated from the rest of the kernel 
so I don't understand why it is not being merged as an EXPERIMENTAL option.

Regardless, it is available in -mm for anyone to use, and last I 
checked, works incredibly well leaving other filesystems miles behind in 
terms of speed. I haven't tested it enough to comment on the 
reliability, but if it is as reliable as reiserfs, it is sufficient for 
me and many others who use RAID and a UPS.

Regards,
LL

Re: reiserFS?

[Dr. David Alan Gilbert]

It is a sad reflection that we have these regular 'fs wars'; and that
most of them are driven by peoples bad experiences with particular
filesystems.

That leads me to ask what level of testing is performed on each
filesystem - are there filesystem torture tests that are getting
run by someone (who?) on various filesystems (preferably on large
TB sized ones, preferably with simulated failures and resets)?
The discussions on Ext4 a few weeks ago made me think that the
thing I'd value more than anything else would be a damn good
test regime.

It would be much nicer if the fs wars came down to peoples
particularly good experiences with filesystems rather than people
selecting file systems based on which one has lost them data most
rarely.

Dave

P.S. For the record I use Reiser for large (>500GB) fs since 
I couldn't get Ext3 stable on one a year or so ago and xfs failed
the 'recover from hitting reset' test.   A couple of years
ago I wouldn't touch Reiser because of NFS issues, but it seems
to have grown out of that.
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    | Running GNU/Linux on Alpha,68K| Happy  \ 
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Re: reiserFS?

[Theodore Tso]

On Sun, Jul 16, 2006 at 06:26:03PM +0100, Lexington Luthor wrote:
> I read the archives, and most of the problems pointed out during the 
> review were fixed relatively quickly, followed by a flame war due to 
> some suggesting that reiser4 should not be able to affect VFS semantics, 
> and other such matters (which IMO should be outside of the scope of a 
> code review). There has been no follow-up review as far as I can tell. 

As far as I know not all of the problems were fixed.  And it has been
observed that given the abuse and accusations that were directed at
the people who did decide to review it, that it would not at all
surprising if some (all?) of reviewers may have decided they had
better things to do.  Getting things merged into mainline is not a
right, and the reviewers are volunteers.....

Speaking for myself, since I don't enjoy being accused of partisanship
and being ascribed of having a desire to backstab reiserfs, I have a
personal policy to avoid reiserfs review, and recuse myself from any
votes within program committee discussions regarding Hans Reiser.
Being accused of taking unfair advantage of my volunteer activities is
something I allow myself to get into once.

						- Ted

Re: reiserFS?

[Theodore Tso]

On Sun, Jul 16, 2006 at 06:46:37PM +0100, Dr. David Alan Gilbert wrote:
> That leads me to ask what level of testing is performed on each
> filesystem - are there filesystem torture tests that are getting
> run by someone (who?) on various filesystems (preferably on large
> TB sized ones, preferably with simulated failures and resets)?
> The discussions on Ext4 a few weeks ago made me think that the
> thing I'd value more than anything else would be a damn good
> test regime.

As far as I know ext2/3 is the only filesystem with a fsck tool that
has a regression test suite.  It's always amazed me that filesystem
consistency checkers and repair tools get so little attenion by most
filesystem developers.

As far as random torture testing, Pavel has written a random test tool
that punches random errors into random blocks of a filesystem, and
that was used to uncover a couple of cases that e2fsprogs didn't
handle cleanly.  Those were reported to me, and I fixed them, and the
edge cases were encorprated into the regression test suite.

IIRC it came up in discussion a few weeks ago one LKML or
linux-fsdevel (I can't remember which), and I believe either someone
from XFS or Reiser team was going to take Pavel's torture tester and
adapt it do some robustifying of their filesystem's repair
capabilities.

Finally, the good folks at the Stanford Metacompilation group did some
very interesting work to find bugs in three common Linux filesystems:

	http://keeda.stanford.edu/~junfeng/papers/osdi04/osdi04.html

Regards,

						- Ted

"Why Reuser 4 still is not in" doc (was: 'reiserFS?')

[Diego Calleja]

El Sun, 16 Jul 2006 13:48:04 -0400,
Theodore Tso <tytso@mit.edu> escribió:

> As far as I know not all of the problems were fixed.  And it has been
> observed that given the abuse and accusations that were directed at
> the people who did decide to review it, that it would not at all
> surprising if some (all?) of reviewers may have decided they had
> better things to do.  Getting things merged into mainline is not a
> right, and the reviewers are volunteers.....


Maybe it's too late and reiser 4 will get in in the next release, but
I've written this doc into the kernelnewbies' wiki:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with
something in that doc, edit it or just answer to this mail what you want
to see in it and I'll add it myself.

Re: "Why Reuser 4 still is not in" doc

[Lexington Luthor]

Diego Calleja wrote:
> Maybe it's too late and reiser 4 will get in in the next release, but
> I've written this doc into the kernelnewbies' wiki:
> http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with
> something in that doc, edit it or just answer to this mail what you want
> to see in it and I'll add it myself.

Hi,

Thanks for the doc and for answering the FAQs.

Do you know of a list of current issues or problems with the reiser4 
code that need to be resolved prior to a merge or another review? I have 
looked around and the mailing list archives only contain one review and 
the issues pointed out in there (aside from the VFS integration debate) 
have been resolved as far as I can tell.

I would like to jump in and perhaps help reiser4 along by trying to fix 
any simple issues, but I am unable to find any specifics as to why it is 
not being merged in its current state. Is there any message or page I am 
missing?

Thanks,
LL

Re: "Why Reuser 4 still is not in" doc

[Joshua Hudson]

> (aside from the VFS integration debate)
Anybody know what's in Reiser4 that VFS doesn't like (link please)?

Re: "Why Reuser 4 still is not in" doc

[Lexington Luthor]

Joshua Hudson wrote:
>> (aside from the VFS integration debate)
> Anybody know what's in Reiser4 that VFS doesn't like (link please)?

Reiser4 plug-ins have (had?) the ability to alter the semantics of 
things, like making files into directories inside which you could see 
meta-files like file/uid and file/size which contained meta-data and 
such accessible as normal files to all the unix tools (which is a very 
good idea IMO). You could get things like chmod by just 'echo root 
 >file/owner' or something, very nice.

This was frowned upon by kernel developers who felt that it belonged in 
the kernel VFS (if at all), rather than in reiser4 directly.

Regards,
LL

Re: "Why Reuser 4 still is not in" doc

[Jan Engelhardt]

>> > (aside from the VFS integration debate)
>> Anybody know what's in Reiser4 that VFS doesn't like (link please)?
>
> Reiser4 plug-ins have (had?) the ability to alter the semantics of things, like
> making files into directories inside

Yes, it changes the semantics. Suddenly you can "cd linux-2.6.17.tar.bz2". 
But what will stat() return? S_IFDIR? S_IFREG? S_IFANY? A .tar parser in 
kernelspace is almost never the right thing. And then a cpio parser, 
because that's what initramfs'es are made of. Not to forget .zip, because 
that's omnipresent. Oh of course we'd also need bzip2 and gzip decoder. 
BASE64 and UU anyone?

> which you could see meta-files like
> file/uid and file/size which contained meta-data and such accessible as normal
> files to all the unix tools (which is a very good idea IMO). You could get
> things like chmod by just 'echo root 
>> file/owner' or something, very nice.

I wish you a lot of fun with users in LDAP or other exotic storage methods.
By making Everything possible through echo, you are violating the unix 
philosophy that one tool should do one thing (though echo does just that). 
And in this case, echo would be chown, chmod, tar, bzip2 all at once. This 
sounds familiar, I think I have seen this with explorer.exe (and its 
uncountable DLLs), which lets you change everything within the same 
window.

What I think is promising are the compression/encryption plugins. ext2 
and 3 had an attribute (`lsattr`) for compression but it does not seem like 
ever implemented.

> This was frowned upon by kernel developers who felt that it belonged in the
> kernel VFS (if at all), rather than in reiser4 directly.


Jan Engelhardt
-- 

Re: "Why Reuser 4 still is not in" doc

[Al Boldi]

Jan Engelhardt wrote:
> >> > (aside from the VFS integration debate)
> >>
> >> Anybody know what's in Reiser4 that VFS doesn't like (link please)?
> >
> > Reiser4 plug-ins have (had?) the ability to alter the semantics of
> > things, like making files into directories inside
>
> Yes, it changes the semantics. Suddenly you can "cd linux-2.6.17.tar.bz2".
> But what will stat() return? S_IFDIR? S_IFREG? S_IFANY? A .tar parser in
> kernelspace is almost never the right thing. And then a cpio parser,
> because that's what initramfs'es are made of. Not to forget .zip, because
> that's omnipresent. Oh of course we'd also need bzip2 and gzip decoder.
> BASE64 and UU anyone?

Using this as an argument against plug-ins is a bit strange.  I suppose 
somebody could go overboard and use plug-ins to implement a subKernel.  
Would this then imply that plug-ins are wrong?

> > which you could see meta-files like
> > file/uid and file/size which contained meta-data and such accessible as
> > normal files to all the unix tools (which is a very good idea IMO). You
> > could get things like chmod by just 'echo root
> >
> >> file/owner' or something, very nice.
>
> I wish you a lot of fun with users in LDAP or other exotic storage
> methods. By making Everything possible through echo, you are violating the
> unix philosophy that one tool should do one thing (though echo does just
> that). 

The unix philosophy would hold with plug-ins, as this would aid flexibility.  
Using plug-ins is a form of modularization, much like the 'one tool should 
do one thing' approach.

> And in this case, echo would be chown, chmod, tar, bzip2 all at
> once. This sounds familiar, I think I have seen this with explorer.exe
> (and its uncountable DLLs), which lets you change everything within the
> same window.

Nothing wrong with that, unless you have an allergy against explorer.

> What I think is promising are the compression/encryption plugins. ext2
> and 3 had an attribute (`lsattr`) for compression but it does not seem
> like ever implemented.

Now that's a great example for using a plug-in in the wright place.

> > This was frowned upon by kernel developers who felt that it belonged in
> > the kernel VFS (if at all), rather than in reiser4 directly.

This is really the crux of the issue.  Introducing plug-ins into the FS is 
really the wrong place, when we already have an abstracted VFS that allows 
this to be fanned out to its children.

Thanks!

--
Al

Re: "Why Reuser 4 still is not in" doc

[Jan Engelhardt]

>
>Using this as an argument against plug-ins is a bit strange.  I suppose 
>somebody could go overboard and use plug-ins to implement a subKernel.  
>Would this then imply that plug-ins are wrong?
>
Ok, I've read some other threads too and so this claim should be adjusted:

Writing a plugin (not necessarily r4 specific) that changes the semantics 
of objects based on context (i.e. turning a file into a dir) is a bad idea 
IMHO.
Actually, BSD has this double-semantic to a limited degree: you can call 
`/usr/bin/vi /usr/bin` and get some binary representation of readdir.



Jan Engelhardt
-- 

Re: reiserFS?

[Matthias Andree]

Christian Trefzer schrieb am 2006-07-16:

> fsck on a faulty drive might cause even more damage - how do you know
> that other areas of the device are OK? 

All drives I bought in the recent years shipped with ARRE and ARWE set.
Any halfway decent fsck tool can just rewrite the b0rked sector and the
drive transparently remaps a spare.

If reiser4 crashes the kernel instead, it's not failure resilient and is
certainly not ready for production.

> I also oppose the ReiserFS-v3.x design philosophy regarding faulty
> storage layer, but in any case where your _live_ data is valuable and
> uptime counts, you _really_should_ use a RAID of some sort.

...and certainly a backup system that gives you FAST access to the data.
RAID doesn't protect against accidents with rm(1) or, worse, wipe(1).

-- 
Matthias Andree

Re: "Why Reuser 4 still is not in" doc

[Lexington Luthor]

Jan Engelhardt wrote:
> Yes, it changes the semantics. Suddenly you can "cd linux-2.6.17.tar.bz2". 
> But what will stat() return? S_IFDIR? S_IFREG? S_IFANY? A .tar parser in 
> kernelspace is almost never the right thing. And then a cpio parser, 
> because that's what initramfs'es are made of. Not to forget .zip, because 
> that's omnipresent. Oh of course we'd also need bzip2 and gzip decoder. 
> BASE64 and UU anyone?

Is there any particular reason that the parsers need to be in 
kernel-space. The reiser4 plugins seem like an ideal counterpart to 
FUSE. Imagine being able to automatically FUSE-mount a tar file as a 
filesystem when you cd into it. stat() need not return S_IFDIR since 
everything is a directory anyway (only normal directories need S_IFDIR, 
just like currently). When you cd into a tar file, a FUSE-fs kicks in 
and provides access to the tar file as a normal filesystem inside it - 
from userspace.

> I wish you a lot of fun with users in LDAP or other exotic storage methods.
> By making Everything possible through echo, you are violating the unix 
> philosophy that one tool should do one thing (though echo does just that). 
> And in this case, echo would be chown, chmod, tar, bzip2 all at once. This 
> sounds familiar, I think I have seen this with explorer.exe (and its 
> uncountable DLLs), which lets you change everything within the same 
> window.

And why can meta-data not be accessed as files? To me, a lowly userspace 
developer, it seems even more inline with the UNIX way of things. bzip2 
can be in userspace while still providing data to kernel space via a 
FUSE-like interface.

Regards,
LL

Re: "Why Reuser 4 still is not in" doc

[Horst von Brand]

Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> >Using this as an argument against plug-ins is a bit strange.  I suppose 
> >somebody could go overboard and use plug-ins to implement a subKernel.  
> >Would this then imply that plug-ins are wrong?

> Ok, I've read some other threads too and so this claim should be adjusted:

> Writing a plugin (not necessarily r4 specific) that changes the semantics 
> of objects based on context (i.e. turning a file into a dir) is a bad idea 
> IMHO.

Right.

> Actually, BSD has this double-semantic to a limited degree: you can call 
> `/usr/bin/vi /usr/bin` and get some binary representation of readdir.

How is that useful? read(2) errors out on a directory, and that is fine
with me. If xemacs wants to do funky stuff when opening a directory, it is
free to notice that special case and do something (readdir(3) and its ilk
are quite useful here) about it.

[Yes, I did work on some boxen where you could read directories, but with
 the current variety of filesystems (and corresponding directory formats!)
 in Linux this way lies utter madnes.]
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: reiserFS?

[Helge Hafting]

On Sun, Jul 16, 2006 at 06:28:31PM +0200, Christian Trefzer wrote:
> On Sun, Jul 16, 2006 at 06:16:31PM +0200, Xavier Roche wrote:
> > > It simply the best filesystem for many kinds of usage patterns.
> > 
> > The most frightening too. Reiserfs might be suitable for very specific
> > appliactions, but to use it in production machine, you need to have
> > some guts.
> > 
> > My last reiserfs partition was blown up two days ago, because of a bad
> > sector, plus a fatal oops, looping endlessly. This was the second
> > time, and the last one, as none of my ext3 filesystems *ever* had
> > similar problems, despite numerous other bad sector issues. Not
> > mentionning the funny "recovery" tool, which generally finishes to
> > trash your data.
> 
> I don't quite understand. You are supposed to dd_rescue the whole block
> device to a working drive and use fsck on the copy.  Whatever is lost in
> the process must of course be restored from a recent backup. But, as a
> friend of mine put it recently, people don't need backup, they only need
> restore ; )
>
Well, many a home user simply doesn't have a a spare block device of 
the same size.  The hassle of reinstalling instead of just waiting
out a fsck is something still.  

The ext filesystems are nice in that they have spare superblocks,
if the main one dies from a bad sector, the spares still work
so you don't loose the entire fs to only a few damaged sectors.

> fsck on a faulty drive might cause even more damage - how do you know
> that other areas of the device are OK? 
>
Somehow, that has saved my day quite a few times with ext2.
I only lost a few files, then went shopping for a new disk. :-)

Helge Hafting

Re: "Why Reuser 4 still is not in" doc

[Tilman Schmidt]

On Sun, 16 Jul 2006 22:01:15 +0200, Diego Calleja wrote:
> Maybe it's too late and reiser 4 will get in in the next release, but
> I've written this doc into the kernelnewbies' wiki:
> http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with
> something in that doc, edit it or just answer to this mail what you want
> to see in it and I'll add it myself.

It could be improved in two ways in order to better convince people:

Firstly, mention some (perhaps even all) of the actual technical
issues preventing the inclusion of Reiser4. Right now, you are
giving much space to the political issues (or their non-existence).
This answers a different question ("Why we are fed up with the
Reiser4 discussion") than the one in the title, and in the worst
case might leave a reader with the impression that political
reasons are more important after all.

Secondly, the questions in the FAQ part should probably be put in a
less loaded form. For example, Q1 could just read objectively:
"Why can't Reiser4 be included as an experimental feature?"
As it stands now, I fear too many people's reaction upon reading it
will be: "That's not my question." At least leave off the plaintive
initial "but" on Q1 and Q2. It puts off the readers you want to
convince.

HTH
Tilman