Atomic filesystem or not

Atomic filesystem or not

[Marcel Hilzinger]

I tried to do a crash-test with Reiser4. I copied a 650 MB file from directory 
A to directory B and pressed the reset buttom at approx 300 MB (the 
testmachine has 256 MB RAM). 

After reboot the 300 MB chunk was still in B.
I thougt it should be either there full or not at all? Or did I understand 
something wrong? Does it work only with files < RAM? What's the difference 
then?

Marcel
-- 
Üdvözlettel -- Mit freundlichen Grüssen,
Marcel Hilzinger

Re: Atomic filesystem or not

[Claudio Martins]

On Thursday 15 July 2004 13:34, Marcel Hilzinger wrote:
> I tried to do a crash-test with Reiser4. I copied a 650 MB file from
> directory A to directory B and pressed the reset buttom at approx 300 MB
> (the testmachine has 256 MB RAM).
>
> After reboot the 300 MB chunk was still in B.
> I thougt it should be either there full or not at all? Or did I understand
> something wrong? Does it work only with files < RAM? What's the difference

  You did understand something wrong ;-)

  A copy using the Unix cp command is not an atomic operation per se. When it 
is said that reiser4 is atomic what it means is that each low level operation 
on the filesystem either happens or not at all. That means that each block 
write request issued by cp either happens or not, but the entire 650MB copy 
itself is composed of many separate atomig requests.

  So what you shouldn't see on the half copied file is blocks which contain 
zeros or otherwise garbage data that didn't belong to the original file (as 
somethimes happens on other journalling filesystems, reiserfs 3.6 sometimes 
included). If you check the differences between the files, they should 
contain the same data up to the last block that hit disk surface before you 
pressed the reset button. 

  As far as I understand this is the guarantee that reiser4 gives. For the cp 
to happen entirely or not at all it would have to be one big atomic request 
only, but I think that kind of operation might only be possible using the 
special reiser4 system call in development.

Best regards

Claudio


> then?
>
> Marcel

Re: Atomic filesystem or not

[Marcel Hilzinger]

Donnerstag 15 Juli 2004 14.54 dátummal ezt írta:
> On Thursday 15 July 2004 13:34, Marcel Hilzinger wrote:
> > I tried to do a crash-test with Reiser4. I copied a 650 MB file from
> > directory A to directory B and pressed the reset buttom at approx 300 MB
> > (the testmachine has 256 MB RAM).
> >
> > After reboot the 300 MB chunk was still in B.
> > I thougt it should be either there full or not at all? Or did I
> > understand something wrong? Does it work only with files < RAM? What's
> > the difference
>
>   You did understand something wrong ;-)
I was quite sure about this :-))

>   A copy using the Unix cp command is not an atomic operation per se. When
> it is said that reiser4 is atomic what it means is that each low level
> operation on the filesystem either happens or not at all. That means that
> each block write request issued by cp either happens or not, but the entire
> 650MB copy itself is composed of many separate atomig requests.
>
>   So what you shouldn't see on the half copied file is blocks which contain
> zeros or otherwise garbage data that didn't belong to the original file (as
> somethimes happens on other journalling filesystems, reiserfs 3.6 sometimes
> included). If you check the differences between the files, they should
> contain the same data up to the last block that hit disk surface before you
> pressed the reset button.

OK. Everything clear now. I will then make an octal or a hexa dump of the 
original and the copied file and compare them. If Reiser4 is atomic, there 
must not be any different entry in the copied file.

Thanks.

Marcel

-- 
Üdvözlettel -- Mit freundlichen Grüssen,
Marcel Hilzinger

Re: Atomic filesystem or not

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1




As I understand it:

Claudio Martins wrote:

| is said that reiser4 is atomic what it means is that each low level
operation
| on the filesystem either happens or not at all. That means that each
block

Each low-level request is automatically an atom, but...

|   As far as I understand this is the guarantee that reiser4 gives. For
the cp
| to happen entirely or not at all it would have to be one big atomic
request
| only, but I think that kind of operation might only be possible using the
| special reiser4 system call in development.

Indeed, the "special reiser4 system call" should support building bigger
atoms.

In fact, I suspect that if reiser4 is accepted as a defacto standard,
then it's even likely that someone would implement some library for
doing atomic operations -- because a system call named "reiser4" is not
portable to other filesystems.  For instance, I believe someone's done a
filesystem based on MySQL, which has atomic operations -- but would
probably want a whole different interface for doing them.

If such a library were developed, and had at least one stable backend
(reiser4), then I think it'd be very likely for standard utils such as
'cp' to support it.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPbeSngHNmZLgCUhAQK5nw/+Jb0pRAq2uKHsxAnsVDMjgBW2rJsKry5T
n7J21lEdOF4l7p+ohs1de/x5RJN/PfnJYUTDPn6JooJ1mQngulXtzXbihHdxeccw
9X9rwbWA4YZHSwRaoIlb5SZXWTWTXYezuIXWZYzVAAxmTT9ca2baqWw+96nohHyF
nJ3rsmUjW2tMQpJRqDGc0IkSMllBI3iAcRPaQUWGr072eomMS/5txMZJJDvikRsf
iTFjJoOFadfr41rVpq34WLE03Z0uJH7JeSsFwhS/20Wqx1VKJoUlSoHExQEvt+1P
h5o22WCqkU3W5Djhu9ulNqWCVMDDbGgJf5r6FqEXQmkmSSDRS3tRIengeb1EoMan
bFenwcB401hapuEzkY6cP14eDufmCKcWaEacvWDymRTOHGTHasn4zdztlwaAYtOr
wHCRhFyWTnSt2PTCI5bUgJEqNDA/Zv3OlsGDoDxHW5ZXrFffUXLyLZhIy6kc1mZA
f7NpL3TBmsyvJSN6la8ked6OXs67JekYv951LsCFRCUVm+G2kWrHHqesK8aFZRjH
bUmhT+U/vox7rpAPQxtsI7Ldk7kgxnF5CYYU945r2FvMKNUQNfbRWJlDXVRDFc1D
A8wZT4ABtEdEUVtA9TMRvlEKHt+RLZkkyLqalvgrfMldC+LRKTi96ff6pL0MEWPr
Df3SbhYUp8Y=
=Tvog
-----END PGP SIGNATURE-----

Re: Atomic filesystem or not

[Hans Reiser]

David Masover wrote:

>
> doing atomic operations -- because a system call named "reiser4"

Maybe I should call it sys_reiser and not sys_reiser4? ;-) What about 
the name is not portable? ;-)

> is not
> portable to other filesystems. For instance, I believe someone's done a
> filesystem based on MySQL, which has atomic operations -- but would
> probably want a whole different interface for doing them.
>
> If such a library were developed, and had at least one stable backend
> (reiser4), then I think it'd be very likely for standard utils such as
> 'cp' to support it.
>
>
>

Re: Atomic filesystem or not

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1




Hans Reiser wrote:
| David Masover wrote:
|
|>
|> doing atomic operations -- because a system call named "reiser4"
|
|
| Maybe I should call it sys_reiser and not sys_reiser4? ;-) What about
| the name is not portable? ;-)

Um.  Suppose someone wanted to duplicate it on (say) ext3?  Then the
name sys_reiser becomes confusing.

But I bet your system call does more than just define atoms, right?  Or
it's supposed to?

Is there an advantage to having one system call, instead of many?
(sys_begin_atom, sys_end_atom, sys_keyword_search, and so on...)

If so, you either want to try to turn it into a standard (so all new
filesystems will have a sys_reiser call) or create a library to abstract
the system call away (something like atom_new, atom_end, and so on).  In
fact, if I'm not mistaken, the "atomic" features could even be
implemented entirely as a userland library on top of the filesystem, at
least as long as the power is on ;)

I like your way better, but I also like the idea of making those
features so widespread (even poorly implemented) that programs start
adopting them.

No matter how good reiser4 is, I don't think everyone will start using
it overnight.  And its atomicity means very little to the average user
if it isn't ubiquitous.  I want cp, vim, thunderbird, and so on to all
support reiser4-style atoms.  If they can do that without a reformat for
existing users/developers, it's more likely to happen in such mainstream
apps, and it helps make reiser4 more useful.




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPczb3gHNmZLgCUhAQKkPw//R388NdP+dEpU/Tw4XG4Fpg8BF5gfv1aG
jpa0rExvG0qdfbDUKL6esiWoMIKqsELhxt7UdIl8a3ncs8Grd/XuE62cdUZsWwnn
xCjXtQ3OlHOI33hXwPokVQ4VJ7WvjgGxKFr9de3R7lT5Aw+Y7QomH92oCK3SNBHT
pDOwOqeDUyhnN6bRHIWvBsk+9zq5QiI9IQEaYjMDwCqKUg3vyyM0qLWlcQNVwE9A
cgJgb8xkWdXQbK5+ZZDKbVGM5yws7rt6pqTVdJgc2JhSMk1Rt4i+no2+pBqyAz1d
LvCzsRJ+7yGayzGQbqvNPYMM4ddlJz6FIBXPhbBFSiCFCAR38zb3LUgXhxAIZxIT
g6fkQjWPJhuQkqGGK2zmjKaUQynMfuby1pjw2mPdxuHwVGiSxXv4fdNBnlHkqqkF
bbWLBGOC/gu3RZzzImOKeAKAcmqjItZnIihsERUEf2EijV5Xtz15HeJ8wO+UEchi
GRIYzIkiAhE5PGeCZyCuEhbRPgSe5/78pkQIa1kO8USR1p0xMpEnFPNLpdZtpeah
/TOgTOztuxVZJhDlJnbydnJHD2wr/pj9jWVa7nUoCrDJFCK93LENL0QrsMvXWhp6
/T+xYRPAfee2/rTLV+0xKRSuJNO+jrjWnUAbMDSBGP+gm6Idfhh/RERD4SF0DNH1
l+n/dqCpCKE=
=YPGx
-----END PGP SIGNATURE-----

Re: Atomic filesystem or not

[Hans Reiser]

David Masover wrote:

>
>
>
> Hans Reiser wrote:
> | David Masover wrote:
> |
> |>
> |> doing atomic operations -- because a system call named "reiser4"
> |
> |
> | Maybe I should call it sys_reiser and not sys_reiser4? ;-) What about
> | the name is not portable? ;-)
>
> Um.  Suppose someone wanted to duplicate it on (say) ext3?  Then the
> name sys_reiser becomes confusing.

I think it becomes clear not confusing.  It tells them they are 
following the reiser standard for how to act on their files, even if 
ext3 stores the files and has implemented the methods invoked to 
implement those actions. 

APIs should be defined by one person when possible. Committees are great 
for feedback, but generally do poorly at remembering the spiritual 
purpose of the API when they are given control of it.

>
> But I bet your system call does more than just define atoms, right?  Or
> it's supposed to?

A lot more, and it will expand rapidly over the next few years.

>
> Is there an advantage to having one system call, instead of many?
> (sys_begin_atom, sys_end_atom, sys_keyword_search, and so on...)

Yes, but it is hard to articulate the advantage.  I think the main one 
is that it encourages applications to just pass the semantics straight 
to the user without each app inventing its own atom delimiters, etc.  
This encouragement comes in the form of it being simplest to code it 
that way.  This is important.  I want users to only have to learn one 
way of naming things, not a different way for each application, not 
unless it is so well motivated that a programmer goes to extra effort to 
code it to be different.

>
> If so, you either want to try to turn it into a standard (so all new
> filesystems will have a sys_reiser call) or create a library to abstract
> the system call away (something like atom_new, atom_end, and so on).  In
> fact, if I'm not mistaken, the "atomic" features could even be
> implemented entirely as a userland library on top of the filesystem, at
> least as long as the power is on ;)
>
> I like your way better, but I also like the idea of making those
> features so widespread (even poorly implemented) that programs start
> adopting them.
>
> No matter how good reiser4 is, I don't think everyone will start using
> it overnight.  And its atomicity means very little to the average user
> if it isn't ubiquitous.  I want cp, vim, thunderbird, and so on to all
> support reiser4-style atoms.  If they can do that without a reformat for
> existing users/developers, it's more likely to happen in such mainstream
> apps, and it helps make reiser4 more useful.

David, none of the other linux filesystem development teams have any 
interest in implementing innovative semantics.  If they support 
sys_reiser4 by any name, they will do it sniping at it all the way.  
Designing semantics requires a very serious education in the topic, and 
not a lot of people are suited for such work.  (Dominic Giampaolo of 
Apple is one person, Codd was another, maybe MS has a few such persons 
but I don't know who and hopefully MS does not know either;-).)  Look at 
the  hassle they gave me over xattrs, which they want to do because 
someone else did it for Unix 5 years ago and not because they have the 
slightest clue about namespace design principles.

Apple wants to innovate in the FS semantics, Microsoft wants to innovate 
in the FS semantics, no major linux filesystem development teams 
competing with us want to implement any semantics less than 5 years old, 
and some of them even think and say that Linux should never implement 
things that have not had 5 years to mature in some other operating 
system first.   I cannot comprehend such people.

They are a wet noodle, and it is better to drag them by setting an 
example their users tell them to imitate than to try to push them by 
getting them to do what I say is the right API before the users are 
using it.

We have created a great storage layer, and if they were reasonable they 
would start building on top of it because it is the fastest storage 
layer ever.  It is so easy to add plugins to it. We could use so much 
help in adding new plugins, tweaking performance in corners we have 
neglected, etc. 

The Apple and Microsoft teams are much more serious about semantics than 
our linux competitors, and much better funded (that is to say, they are 
funded....)  than I am.  I worry about them.  Our linux competitors have 
zero architects, and  genuinely think architects have no value.  Sigh.  
There are so many ways to be silly in life.

We need to get some funding for pushing sys_reiser4() semantics out to 
the most commonly used apps by writing patches for them.  That would be 
nice....

Hans

Re: Atomic filesystem or not

[Toby Dickenson]

On Monday 19 July 2004 09:33, Hans Reiser wrote:
> We need to get some funding for pushing sys_reiser4() semantics out to 
> the most commonly used apps by writing patches for them.  That would be 
> nice....

I would appreciate a pointer to any sys_reiser4 documentation.

Thanks,

-- 
Toby Dickenson

reiser acceptance (was Re: Atomic filesystem or not)

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1




Hans Reiser wrote:
[...]
| David, none of the other linux filesystem development teams have any
| interest in implementing innovative semantics.  If they support
[...]
| but I don't know who and hopefully MS does not know either;-).)  Look at
| the  hassle they gave me over xattrs, which they want to do because

Do you know why they bothered you about xattrs?  xattrs already exists,
is already used, and already works.  Not supporting it means that any
apps which use it will have to be rewritten.  I'm all for rewriting apps
to support reiser4, but I doubt the community feels the same way.


| They are a wet noodle, and it is better to drag them by setting an
| example their users tell them to imitate than to try to push them by
| getting them to do what I say is the right API before the users are
| using it.

I hope that it works -- I hope users notice reiser4 and pester everyone
to support it.  I doubt it will, unless the developers feel comfortable
supporting it.  As a developer, I wouldn't feel comfortable with what
could turn into this:

switch (fs_type) {
	case REISER4:		sys_reiser4(...)
	case EXT3:		sys_ext3_atom(...)
	case I_OWN_REISER:	sys_something_different(...)
}

You are fooling yourself if you think we won't need that -- that reiser4
is the only fs that will ever properly support transactions, or that
every other fs will support the full sys_reiser4.  And I think it's
seriously poor design to require such a switch statement in every app,
and have to modify it to support every new and innovative fs that comes
along.

Now, suppose there was a standard library, which contained such a switch
statement -- so that this library has support for reiser4, some random
other FS types, and even fakes it on filesystems like ntfs and fat32.

Or, look at how others have done it.  GNOME did not build in support for
Linux's dnotify, they used FAM -- the File Alteration Monitor -- which
could support Linux's dnotify, but could also support various other
systems, including actual polling of files.  They only had to patch it
once to support FAM, and technically, they could have removed all
support for polling from GNOME and made FAM a requirement.

Based on that, I think GNOME would be much more likely to support
"libfsatom" than "sys_reiser4".

| We have created a great storage layer, and if they were reasonable they
| would start building on top of it because it is the fastest storage

What about JFS + hardware elevator sorting?  Or, more relevantly, can
you afford to be so arrogant when you're trying to sell something?
Being right isn't enough -- being right makes you Tesla, not Edison.

| We need to get some funding for pushing sys_reiser4() semantics out to
| the most commonly used apps by writing patches for them.  That would be
| nice....

Do you intend to have a reiser4 patch repository, so that end-users have
to go and download a patch for each app?  Or are you actually looking
for inclusion in things like Debian?  If the latter, then you really
ought to do this my way -- make it look like you are creating a portable
standard.  Make it so that each app only needs one patch to support any
filesystem transactions, ever.

And even if you do reach your goal of being required by every major
Linux distribution, what's the harm?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPw3MHgHNmZLgCUhAQKthA//bjg0XcODmmZqt8ExN+Br6wxWin/G/S6k
/KaUrvBoxTIk08fCdVFsqbdmhWuNYY8OiUS/ctWEOtbUCOfOLlAgj2zD7UPry2/n
rnIFPl+sum1kOuYVzNzht7LbgWTFT188lqqiI9HizMP9BERKLPe9jQdNarOYi6kI
A3rSY4mXdwv6K3oVwjuBF2F7BvPCtTkCAn3wne+sNaT7RsZpBQyKUGhfHEa5dVVW
3gpApF14d0eC+6yl3UhYLU4GbQozzO92zAiq24oThtcxaI2i12WAnKTKjBft4TxE
k6Xs4SvdTzEJh1qXUO/IMqMrJg/hEiuc3bvEYbetoXy/W9Dw36MlPq0mqHl3u7y3
kfXPycFevtcVEPrKtGD/33DHlm5Q89aM8ApStF4tNE8LEUcC0+EInJ0gKCeKpgO4
oRu1gEE0ZgFd9v+12oivoTyNIN34IvOXCzMWy3as+dMk/W3P97UVvarIisAqZQeX
mkZPiffJGSU7Qdb5+DS5vVIxBoa8WtW1Uk5yAxDOaNfFd0N7tnw4tMhtm2SCD4oh
EltDMEpIhS/61wh7tGBZ+lR5w/HPPwfzLb/cGjDvFHxlhBYU9j/8VjGxKI9H43/E
MRfSteXcflZBlbwRbyaWlNfwShdlo0nzGY+UyeWYdTv5HqfFSQORn2vNuWmehazX
f0kPk4KN5rg=
=N2y2
-----END PGP SIGNATURE-----

Re: reiser acceptance (was Re: Atomic filesystem or not)

[John D. Heintz]

I don't have a problem with a sys_reiser4 system call because this is 
still clearly innovation. Once the system is out there, used, tweaked, 
and refined it will be time to begin standardizing efforts on something 
that should be shared. That could very well be a libfsatom.

Unless there is already a well known file-like atomic operation API out 
there that matches the future semantics and performance targets of 
ReiserFS it's too early to even try for a standard for all linux 
filesystems. Innovate then standardize.

John Heintz


David Masover wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> 
> 
> Hans Reiser wrote:
> [...]
> | David, none of the other linux filesystem development teams have any
> | interest in implementing innovative semantics.  If they support
> [...]
> | but I don't know who and hopefully MS does not know either;-).)  Look at
> | the  hassle they gave me over xattrs, which they want to do because
> 
> Do you know why they bothered you about xattrs?  xattrs already exists,
> is already used, and already works.  Not supporting it means that any
> apps which use it will have to be rewritten.  I'm all for rewriting apps
> to support reiser4, but I doubt the community feels the same way.
> 
> 
> | They are a wet noodle, and it is better to drag them by setting an
> | example their users tell them to imitate than to try to push them by
> | getting them to do what I say is the right API before the users are
> | using it.
> 
> I hope that it works -- I hope users notice reiser4 and pester everyone
> to support it.  I doubt it will, unless the developers feel comfortable
> supporting it.  As a developer, I wouldn't feel comfortable with what
> could turn into this:
> 
> switch (fs_type) {
>     case REISER4:        sys_reiser4(...)
>     case EXT3:        sys_ext3_atom(...)
>     case I_OWN_REISER:    sys_something_different(...)
> }
> 
> You are fooling yourself if you think we won't need that -- that reiser4
> is the only fs that will ever properly support transactions, or that
> every other fs will support the full sys_reiser4.  And I think it's
> seriously poor design to require such a switch statement in every app,
> and have to modify it to support every new and innovative fs that comes
> along.
> 
> Now, suppose there was a standard library, which contained such a switch
> statement -- so that this library has support for reiser4, some random
> other FS types, and even fakes it on filesystems like ntfs and fat32.
> 
> Or, look at how others have done it.  GNOME did not build in support for
> Linux's dnotify, they used FAM -- the File Alteration Monitor -- which
> could support Linux's dnotify, but could also support various other
> systems, including actual polling of files.  They only had to patch it
> once to support FAM, and technically, they could have removed all
> support for polling from GNOME and made FAM a requirement.
> 
> Based on that, I think GNOME would be much more likely to support
> "libfsatom" than "sys_reiser4".
> 
> | We have created a great storage layer, and if they were reasonable they
> | would start building on top of it because it is the fastest storage
> 
> What about JFS + hardware elevator sorting?  Or, more relevantly, can
> you afford to be so arrogant when you're trying to sell something?
> Being right isn't enough -- being right makes you Tesla, not Edison.
> 
> | We need to get some funding for pushing sys_reiser4() semantics out to
> | the most commonly used apps by writing patches for them.  That would be
> | nice....
> 
> Do you intend to have a reiser4 patch repository, so that end-users have
> to go and download a patch for each app?  Or are you actually looking
> for inclusion in things like Debian?  If the latter, then you really
> ought to do this my way -- make it look like you are creating a portable
> standard.  Make it so that each app only needs one patch to support any
> filesystem transactions, ever.
> 
> And even if you do reach your goal of being required by every major
> Linux distribution, what's the harm?
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> 
> iQIVAwUBQPw3MHgHNmZLgCUhAQKthA//bjg0XcODmmZqt8ExN+Br6wxWin/G/S6k
> /KaUrvBoxTIk08fCdVFsqbdmhWuNYY8OiUS/ctWEOtbUCOfOLlAgj2zD7UPry2/n
> rnIFPl+sum1kOuYVzNzht7LbgWTFT188lqqiI9HizMP9BERKLPe9jQdNarOYi6kI
> A3rSY4mXdwv6K3oVwjuBF2F7BvPCtTkCAn3wne+sNaT7RsZpBQyKUGhfHEa5dVVW
> 3gpApF14d0eC+6yl3UhYLU4GbQozzO92zAiq24oThtcxaI2i12WAnKTKjBft4TxE
> k6Xs4SvdTzEJh1qXUO/IMqMrJg/hEiuc3bvEYbetoXy/W9Dw36MlPq0mqHl3u7y3
> kfXPycFevtcVEPrKtGD/33DHlm5Q89aM8ApStF4tNE8LEUcC0+EInJ0gKCeKpgO4
> oRu1gEE0ZgFd9v+12oivoTyNIN34IvOXCzMWy3as+dMk/W3P97UVvarIisAqZQeX
> mkZPiffJGSU7Qdb5+DS5vVIxBoa8WtW1Uk5yAxDOaNfFd0N7tnw4tMhtm2SCD4oh
> EltDMEpIhS/61wh7tGBZ+lR5w/HPPwfzLb/cGjDvFHxlhBYU9j/8VjGxKI9H43/E
> MRfSteXcflZBlbwRbyaWlNfwShdlo0nzGY+UyeWYdTv5HqfFSQORn2vNuWmehazX
> f0kPk4KN5rg=
> =N2y2
> -----END PGP SIGNATURE-----
> 
> 

Re: reiser acceptance (was Re: Atomic filesystem or not)

[Hans Reiser]

John D. Heintz wrote:

 > Innovate then standardize.

Thanks John, I think you understand me.

Trying to innovate in a standards setting way is really hard unless your 
competitors understand your ideas and support them and want to help you 
create a new market.  Ours don't.

David, I understand you mean well.  You just don't understand what 
motivates technology trailers.  They have no desire at all to innovate.  
Their only desire is to keep us from innovating because it makes more 
work for them.  This is not arrogance speaking, this is tired experience 
speaking.  They want to make nice little tweaks one step at a time, 
working a bit every week between their speaking engagements, and none of 
them put in the kind of grueling coding/debugging marathons that last 
for 4-5 years of despair and a pitiful salary before it finally starts 
to work that I put my guys through.  We beat them mostly by working 
harder and taking harder approaches that no one has chosen before 
because it looked like too much work. 

Maybe it WAS too much work.... oh well.... did it anyway....

Hans

Re: reiser acceptance (was Re: Atomic filesystem or not)

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Hans Reiser wrote:

| David, I understand you mean well.  You just don't understand what
| motivates technology trailers.  They have no desire at all to innovate.

This, I do understand, and I thought that was part of why "libfsatom"
would be the way to go.  Not only do they have little desire to innovate
(believe me, it's somewhere above "not at all" but way, way below "let's
change the world"), they also don't like to let innovations in unless
they already look pragmatic.

Still, I bow to your experience.  I've said my piece.  If you still
don't agree with me, saying it again won't change your mind.

And since we're doing things your way, I'll start with Gentoo -- a USE
flag and patches to fileutils.  If/when sys_reiser4 is done/documented,
that is.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPzJgngHNmZLgCUhAQI1kw/+MAVozlRqSLCXDPRy7B24GWCv/xgd0xlm
u8MMZginvhywWPPjJNq+OHo5t0YjjbuS0AR5KAoeEmXQVqNadmg0w2IIQAkelnxg
ocLwmogmJMN+dP+rLsi8n8OnH0fvErMEe1Xl+Gmr1BRKojcoHZwzjXGBXCknHC3V
Wc/VLcJvGhaBL1BzKyj6dI5Po4dThqC+2dZDHTg4+TDo1oSCuYk9q+/pNKUWnBhX
08wa36nSxBe6svBMuv5rgWhUVUuhhFHdkHuUCrSq64gFnwl2NoAPqeOqQIRrTdh1
OFJ0clJ7qsnKbxzkSaBl/N4//dXrIJCaE5MCyvAb2nrtfL9daKyya+THm2KYXdej
7ObdKbSbDJbn2CnMlJxb8+mzChqQSXSiuQIqy38RZUXJgWUtRElxCRS1RYJnFAMp
gyteDQeKnqTsF31Q1VAH2GPzH73NwecI4PTK8WMgxWVvUbDIpSDy9qvqfZvc3X2u
2KsTMTkGvxwDTiVeBh3LdLfLyb2mLQgfDTrsDmkBAJWB0anMddBj6qHUhuwfRcO9
qfgRthRDDIYo3NQQ7tPt9w43AI/vjhsndQUUV4aRJFb6vIJICx3655R993S+7gkT
wYPQHRNwbEfN3kC3jUUYnf0/LXnnELK9Lm6XUsRjYMJ+9Iy15MC1tMo9vPz/R/jJ
nEzIsbynlSk=
=cMxL
-----END PGP SIGNATURE-----

Re: reiser acceptance (was Re: Atomic filesystem or not)

[Hans Reiser]

David Masover wrote:

>
>
> What about JFS + hardware elevator sorting? 

I am not familiar with this, unless you mean disk drive hardware 
elevator sorting, which I suspect you don't mean.

Re: reiser acceptance (was Re: Atomic filesystem or not)

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Hans Reiser wrote:
| David Masover wrote:
|>
|> What about JFS + hardware elevator sorting?
|
| I am not familiar with this, unless you mean disk drive hardware
| elevator sorting, which I suspect you don't mean.

If you suspect that, you must already be faster.  Because that is
exactly what I mean -- that or SCSI controller hardware elevator sorting.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPzEFngHNmZLgCUhAQIueBAAiC/9BPF1n3WbAGVzyjSbzYatLhVZAmck
pG8Sy/5pospByp9TMiT8JxVmoaCVI+ElQa8am/G0gHwXuUezWl0eoW2TAcone5y2
vP5588fVvO4H5z3/KUeVPmYYIAzQQefw5VtBSc3CZ6PPJXFhB9ujMicJN37LHutH
Jz/Q0ptHLWH84v55HzEwDCRLwLxtFlkaUTXg7IguBXSo+A/LF9itVaf4IdVCxX7k
kV7fNsPkegcY2MDM0c62f4HqtIRX/22DE/X/T9rhSis6Cq5T+bD/f2FOZcACIKbM
rpI8KiM7FTxGkDIgUmb+3mgVxOmjaFYK7M3NC8whNkSXYFvdOZ0NyH5RseFrpAH5
4kTjer0JpR7Eq0dCB7GNHeVLbdWaqN705e0F2JUIKKXPFDDJbz7BJLjhm++SRbb9
cvFyxQs8DQXXdArE/2V6e5WlS/VKtj62ocou8BDB18w2Nf6GwjybVLa4zsHBUzyf
Yptgrd8EayQzuiAfDPdweDuXdOrEvLt6xrCQO2Atm4/BiFLOLNEep/5zJE3NYvCb
v5P9OJ/xBv8MIwHOwROSurSRzhseOeCbluMPS8gk5Kd6JrKW1i6+UDqq6dOhb/xX
SXBwkWfpksbcUAyB9tw/Ao1BPQbINwbGO2QAnYvg6dXcvF9B/TdKgh1J6alAidv4
Izik0EWCbJI=
=pscp
-----END PGP SIGNATURE-----

Re: reiser acceptance (was Re: Atomic filesystem or not)

[mjt]

On Mon, Jul 19, 2004 at 04:03:44PM -0500, David Masover wrote:

>You are fooling yourself if you think we won't need that -- that reiser4
>is the only fs that will ever properly support transactions, or that
>every other fs will support the full sys_reiser4.  And I think it's
>seriously poor design to require such a switch statement in every app,
>and have to modify it to support every new and innovative fs that comes
>along.

But hopefully sys_reiser4 will be the standard, yes?
(Also, rename it to sys_reiser, the number makes it a bit more unclear,
more bound to the fs, not to the guy behind it)

I'd say it's the problem of the next innovator after reiser4. If he won't
support the sys_reiser calls, he has to fight for standardization, and if
he wants the basic stuff that's implemented in sys_reiser and then some,
he could just send in extensions to the sys_reiser.

The Namesys guys have the advantage of being the first around here
with something really new. And good.

>Now, suppose there was a standard library, which contained such a switch
>statement -- so that this library has support for reiser4, some random
>other FS types, and even fakes it on filesystems like ntfs and fat32.

This would have to be a small enough library, having an extra dependancy
for every basic GNU tool is not too good. There's already libattr and
libacl in Debian, for example.

>Being right isn't enough -- being right makes you Tesla, not Edison.

I remember hearing this story in school, but it was four years ago :)
Was it something about Edison being the ultimate asshole who sabotaged
Tesla all the way for his own fame, and he won in the end?

>Do you intend to have a reiser4 patch repository, so that end-users have
>to go and download a patch for each app?  Or are you actually looking
>for inclusion in things like Debian?  If the latter, then you really
>ought to do this my way -- make it look like you are creating a portable
>standard.  Make it so that each app only needs one patch to support any
>filesystem transactions, ever.

How does libaal figure into this? Could it provide the libfsatom
functionality? Just compile the tools against libaal as well...

Also, I think Debian inclusion is a bad target, you have to hit upstream
first. Debian maintainers are probably more difficult people than
the upstream maintainers ;)

Anyway, a patch repository is a good idea. Maybe one entirely open to
the public as well, to begin with. Anything that brings forth easy
inclusion. If I remember correctly, Torvalds implemented POSIX
syscalls on demand, he used his kernel and when it would fail, he would
implement the required call. I think something similar could be done
here?

-- 
mjt

Re: reiser acceptance (was Re: Atomic filesystem or not)

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Markus Törnqvist wrote:
| On Mon, Jul 19, 2004 at 04:03:44PM -0500, David Masover wrote:
|
| But hopefully sys_reiser4 will be the standard, yes?
| (Also, rename it to sys_reiser, the number makes it a bit more unclear,
| more bound to the fs, not to the guy behind it)

How many versions of Reiser will there be?  If you can get a separate
system call for each one, go for it.  Then you can always support
sys_reiser4 on newer versions, and people use sys_reiser5 and so on to
support newer features.

Unless, that is, you've already made it so modular that people who call
sys_reiser on reiser4 won't find it broken on reiser5.  If you can do
that without cruft, you are amazing, and the number should go.

|>Being right isn't enough -- being right makes you Tesla, not Edison.
|
| I remember hearing this story in school, but it was four years ago :)
| Was it something about Edison being the ultimate asshole who sabotaged
| Tesla all the way for his own fame, and he won in the end?

Possibly.  My point is Tesla was a genius who died poor and Edison was a
hard worker who won.  My guess is that Edison understood business and
people a bit better than Tesla, who understood electricity a bit better
than Edison.

| Also, I think Debian inclusion is a bad target, you have to hit upstream
| first. Debian maintainers are probably more difficult people than
| the upstream maintainers ;)

What about Gentoo?  (I just mentioned this in an email, probably sent
exactly the same time as yours.)  Gentoo maintainers welcome new,
unstable stuff, as long as it's well isolated -- use the ~x86 flag if
it's a package, or a USE flag if it's a patch.  So we just add a reiser4
USE flag.  As long as they can warn users away from it because of
possible instability (as they did with the gtk2 flag), we're fine.

What one distro does, others are more likely to do.

| Anyway, a patch repository is a good idea. Maybe one entirely open to
| the public as well, to begin with. Anything that brings forth easy

Not really.  Best thing?  A Gentoo SYNC mirror or a Debian source, or
the Fedora equivalent (if there is one).  That is, the next best thing
to a Gentoo USE flag.  A lot of work for us, but users who want the
latest reiser4 can get it, using a package manager, not patch and make.



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPzMT3gHNmZLgCUhAQI3oQ/+N0lxb2P2FJgkNeCJYaLiSXGlVac42XdE
Ei3bR+wq4VXdQef7CqoI/38eljiI8wtZQgchlguSsG5DGp+NVquPnGYNXsX8/Lys
TDsb3I8iE5tXLEHjA7W8LS83TEZVQVWNsoAIRru565dqIuNrySOAdM4zeG4OtBvn
xwZ3E3qWv3VfQCz9t4xty/NsMa8CIuRr+ceF+nBp6UCqTRax2sT5J8VT9fA73hCq
OkN6au90/qmQBVnW8MgVjZCkKiHan7UNqBduaJ+YjxDUfyRJcb4cNaygSNxZ0idp
tdriYR2f+0IbJcBSxe3lqaKLhQxhM6iZJDNsGsA8Dorqv7dx/icMxBABkf24rRyg
oPpwtFal7WEOt9ENZyuMmLXVT4m8gOHshhu+FmyHFbuuvDPE2fJZIgerh/tqt/QH
FfIgrAuQFA64mHCGXmPdZuh6S3xPGlVSxM0vLpclvI8ZT69/lATL66GQ7d7ZZ4id
t75VnQc2LwRk013BTn027840FdExS6q5/QG7ikwIDKAPUcv/Ze5H08Skt5RnN4ca
iPyRXohKB6Wce40w4UekaJievKN91kD5HTh+suyEjfPZtqKXpRjtQj1kb2Z80zXj
/MWfvhY1xQUgG8IBiqqd1+BuHBLTKx3B9SdSnWik/RrM1Q55ucajoR0D3dSCNFPk
tL/mhlt0KtE=
=skcC
-----END PGP SIGNATURE-----

Re: reiser acceptance (was Re: Atomic filesystem or not)

[mjt]

On Tue, Jul 20, 2004 at 02:39:59AM -0500, David Masover wrote:
>
>Unless, that is, you've already made it so modular that people who call
>sys_reiser on reiser4 won't find it broken on reiser5.  If you can do
>that without cruft, you are amazing, and the number should go.

But if the syscalls get extended, we have the same situation in
sys_reiser4, right?

So it has to be somehow forward-compatible, I think. But if it's all
in libaal or some equivalent, it's just enough to upgrade the library?

>Possibly.  My point is Tesla was a genius who died poor and Edison was a
>hard worker who won.  My guess is that Edison understood business and
>people a bit better than Tesla, who understood electricity a bit better
>than Edison.

Yeah, but I'm pretty sure I don't mis-remember Edison using a few
very dirty tricks to prove his superiority...

>What about Gentoo?  (I just mentioned this in an email, probably sent
>exactly the same time as yours.)  Gentoo maintainers welcome new,
>unstable stuff, as long as it's well isolated -- use the ~x86 flag if
>it's a package, or a USE flag if it's a patch.  So we just add a reiser4
>USE flag.  As long as they can warn users away from it because of
>possible instability (as they did with the gtk2 flag), we're fine.

Sure. But it'll be a cold day in hell when I start using Gentoo.

Debian is basically flexible enough for me, and I abhor the notion
of compiling more than what I have to.

Anyway, getting the patches out there, I'd be more than happy to do
things with them in Debian :)

>What one distro does, others are more likely to do.

But still I think upstream penetration would be best.

>Not really.  Best thing?  A Gentoo SYNC mirror or a Debian source, or
>the Fedora equivalent (if there is one).  That is, the next best thing
>to a Gentoo USE flag.  A lot of work for us, but users who want the
>latest reiser4 can get it, using a package manager, not patch and make.

Sure, I could gladly maintain a Debian mirror with reiser4-patched
packages for woody, sarge and sid, but I probably wouldn't have time
to thoroughly test all these setups, which means someone may poison
their stable Debian with these. So maybe just testing in sid
would be enough for starters.

-- 
mjt

Re: reiser acceptance (was Re: Atomic filesystem or not)

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Markus Törnqvist wrote:
| On Tue, Jul 20, 2004 at 02:39:59AM -0500, David Masover wrote:
|
|>Unless, that is, you've already made it so modular that people who call
|>sys_reiser on reiser4 won't find it broken on reiser5.  If you can do
|>that without cruft, you are amazing, and the number should go.
|
|
| But if the syscalls get extended, we have the same situation in
| sys_reiser4, right?

Wrong.  sys_reiser5 could require a totally different set of arguments
than sys_reiser4.  Suppose it's like this:

// It's been awhile since I've done serious C. I hope my syntax is sane.
sys_reiser4 (int, reiser4_struct)
sys_reiser5 (reiser5_struct)

You cannot just do a "s/reiser4/reiser5" on all your source files.  You
have to actually change your code to use the new interface everywhere.
If I didn't have the version number in there, whenever someone upgraded
to reiser5, your program would neither compile nor run until it was updated.

Yet sys_reiser4 could probably still be implemented -- as a separate
system call, or even as a library function which calls sys_reiser5,
passing its first argument inside the reiser5_struct somehow.

Once it becomes a standard, there will probably be one system call, with
one name, forever.  It would be as ubiquitous as open().  When that
happens, no versioning will be needed, and we will be shackled into
backwards compatibility.  And I do mean "shackled".  Feel the asbestos.

| So it has to be somehow forward-compatible, I think. But if it's all

That's better, of course.  Is it forwards compatible enough to call it
sys_reiser?  Anyone?

| in libaal or some equivalent, it's just enough to upgrade the library?

If only, if only.  But we discussed libraries, and it seems Hans wants
apps patched to use sys_reiser4 directly.  Right?

|>Possibly.  My point is Tesla was a genius who died poor and Edison was a
|>hard worker who won.  My guess is that Edison understood business and
|>people a bit better than Tesla, who understood electricity a bit better
|>than Edison.
|
| Yeah, but I'm pretty sure I don't mis-remember Edison using a few
| very dirty tricks to prove his superiority...

Imagine if Tesla had been able to play that game.  This is why democracy
currently sucks -- the honest people won't be able to play the game, so
of course there are mostly dishonest politicians!

Oops, offtopic :X

|>What about Gentoo?  (I just mentioned this in an email, probably sent
[...]
| Debian is basically flexible enough for me, and I abhor the notion
| of compiling more than what I have to.

Why, when it's automated, stable, and ultimately makes your box faster?

And I abhor the notion of having packages excluded because someone else
doesn't like the license.

But I wasn't trying to sell Gentoo as a distro to actually use, only one
to send patches to.

|
|>What one distro does, others are more likely to do.
|
|
| But still I think upstream penetration would be best.

If you can do it.  But this kind of administrivia is not a big deal.
Send it to everyone!  Put patch code on mousepads and teacups!
|
|
|>Not really.  Best thing?  A Gentoo SYNC mirror or a Debian source, or
|>the Fedora equivalent (if there is one).  That is, the next best thing
|>to a Gentoo USE flag.  A lot of work for us, but users who want the
|>latest reiser4 can get it, using a package manager, not patch and make.
|
| Sure, I could gladly maintain a Debian mirror with reiser4-patched
| packages for woody, sarge and sid, but I probably wouldn't have time
| to thoroughly test all these setups, which means someone may poison
| their stable Debian with these. So maybe just testing in sid
| would be enough for starters.

Just tell them that you are a (woody|sarge|sid)-based very unstable
reiser4-powered distro.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQP36sXgHNmZLgCUhAQJQAw/+JDN9Qqls9qNZgTxOoeOmbfdONYghPe8m
U7kTIZb4FfYL3NM7Fwq3Chx7Bh8Ch2uZjBzE0Qw1T+6PrSau8XUWIZdbBbKRd+wb
xzKk/CdaNkTXmRryLzfgpXGImBZkGJO/ueEYriT9pztNOMcdYi+sP68cPhizrCtP
Zgsar1lSBC88DhKbBMhC8ff54vMumQkUtVkdNwsWVtyAUGQnw8YkeC3UZEmqWz97
vbGumcmPiQRExqxf/kchvNZqlGJrRDpmz61L37jhNEaXkcLKTssRjNTERUp2/1IK
mq11s9Nw1FbZAL7RcasSfLYrvrhSe/e2Fm8lsUXmdCmMsDjUNoRe8QkkL68kC4DI
Q6fYwB7zykf5f8cc2A5ZigBmHtuCAy6qYqa3HHIajN8p1grtIRmZVSgSmXKFXcPN
ZchhcaduScTLoOMGI+1yHcNEroA/K1wUwcfYY6/3ReQ+ZVfFZuqkNo1aRS95Weta
xcqw/Xd8L1V0zHbQnTMZ8mBV4irvRu2vwvl1TqQ8i0nTTwWVyrYqONdrF/S3jIiM
hdRug+0JMly91yPZAsSxmzkto7PfYAF/d5ezIaSIKdl8cexBFIckmFja3iveGwRS
eLIJc/6w9ZQ82/fmWL6Mhe5BrUJ/aHVXrb3QCXiy8DPmuAIRA33KUKg29KlhIgT5
nf/b/9wDOgU=
=3goJ
-----END PGP SIGNATURE-----

Re: reiser acceptance (was Re: Atomic filesystem or not)

[mjt]

On Wed, Jul 21, 2004 at 12:10:09AM -0500, David Masover wrote:

>Wrong.  sys_reiser5 could require a totally different set of arguments
>than sys_reiser4.  Suppose it's like this:

You are right.

>If only, if only.  But we discussed libraries, and it seems Hans wants
>apps patched to use sys_reiser4 directly.  Right?

Is there a performance reason behind this?
Also, one less library dependancy is one less library dependancy...
Beside the fact that direct patching is a stronger vote for
standardizing.

>Imagine if Tesla had been able to play that game.  This is why democracy
>currently sucks -- the honest people won't be able to play the game, so
>of course there are mostly dishonest politicians!

So we have the Free Software movement that can't play the corporate
software game. But revolutions have been known to happen.

>Oops, offtopic :X

A strong maybe.

>Why, when it's automated, stable, and ultimately makes your box faster?

The time that goes to compiling may even save the 1% speedup that
the new binaries bring along. Remember, that Debian has different packages
with different optimizations where it actually matters.
And I can make my own packages with apt-get source --build if I need.

So I just don't like to sit on my arse for hours while something compiles
just so I can have a next-to-imaginary speed-up :)

>And I abhor the notion of having packages excluded because someone else
>doesn't like the license.

Amen to that. cf. DFSG/Debian.

>But I wasn't trying to sell Gentoo as a distro to actually use, only one
>to send patches to.

And I think it's a good point, Gentoo users are most likely very good
testers.

>Just tell them that you are a (woody|sarge|sid)-based very unstable
>reiser4-powered distro.

This is a bit like defeating the purpose of Debian stable, and it
takes a bit of time to make all these packages. But sure, there may
be some automatization to build these backports easily; if there isn't
I'd have to invent one, and still no-one would be forced to use it.

So, now we have the strategy, now let's wait for the call to arms.

-- 
mjt

Re: reiser acceptance (was Re: Atomic filesystem or not)

[Hans Reiser]

David Masover wrote:

>
>
> Markus Törnqvist wrote:
> | On Mon, Jul 19, 2004 at 04:03:44PM -0500, David Masover wrote:
> |
> | But hopefully sys_reiser4 will be the standard, yes?
> | (Also, rename it to sys_reiser, the number makes it a bit more unclear,
> | more bound to the fs, not to the guy behind it)
>
> How many versions of Reiser will there be?

One every 3-5 years for ~30 years.

>   If you can get a separate
> system call for each one, go for it.  Then you can always support
> sys_reiser4 on newer versions, and people use sys_reiser5 and so on to
> support newer features.

That was the idea.

>
> Unless, that is, you've already made it so modular that people who call
> sys_reiser on reiser4 won't find it broken on reiser5.  If you can do
> that without cruft, you are amazing, and the number should go.

I am not amazing, and prefer to keep the number.;-)

Re: reiser acceptance

[Hubert Chan]

>>>>> "Markus" == Markus Törnqvist <mjt@nysv.org> writes:

[...]

Markus> Also, I think Debian inclusion is a bad target, you have to hit
Markus> upstream first. Debian maintainers are probably more difficult
Markus> people than the upstream maintainers ;)

It depends a lot on the Debian maintainer and the upstream maintainer.
Some Debian maintainers are happy to include new stuff, and some
upstream maintainers are unresponsive.  And for some packages, neither
of them want to do anything. :(

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.