Re: Reiser4 Inclusion

Re: Reiser4 Inclusion

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

On 25.07.2006 07:15, Chris Siebenmann wrote:
> You write:
> | [...] Therefore an attitude which says "go on developing that
> | code out-of-tree, it's not ready for inclusion yet" is in direct
> | contradiction with the foundations of the no-stable-API policy.
> 
>  I don't think that there's a contradiction, because I believe that what
> the kernel developers are saying in general can be rewritten as:
> 
> 	- we don't care about things that are deliberately kept
> 	  out of the kernel
> *and*	- we also don't care about code that does not meet quality
> 	  or relevance standards

Actually, that *isn't* what I read regularly in lkml. Most statements of
rejection by kernel developers do *not* read "we don't care about that,
go away", but "this needs work here and there before we will accept it",
which in a way is the opposite of "we don't care".

But I am growing tired of this discussion. I tried to help, and instead
drew fire myself. My own fault of course. I misjudged the situation and
the emotional content of the ongoing dispute. I will now keep my tongue.

Regards
Tilman

PS: I was forced to give this answer publicly because your given E-mail
address wouldn't accept my private mail answer. My apologies if this is
not what you wanted.

-- 
Tilman Schmidt                          E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 253 bytes --]

Re: Reiser4 Inclusion

[linux]

>> I have deployed two nearly identical servers in Florida (I live in
>> Washington state) but one difference: one uses ext3 and the other
>> reiser4. The ping time of the reiser4 server is (on average) 20ms faster
>> than the ext3 server.
>
> OK, I'll bite.  What *POSSIBLE* reason is there for the choice of filesystem
> to matter to an ICMP Echo Request/Reply?  I'm suspecting something else,
> like the ext3 server needs to re-ARP before sending the Echo Reply, or some
> such.

Er... I was assuming that was an application-level ping, e.g. "fetch
database-generated web page", and not an ICMP-level ping.

If this *is* talking about ICMP-level ping, I agree completely;
that makes no sense.  Either there is a dire bug in Linux networking,
or the networks aren't the same.  Assuming the two machines are
located together (if they're not, there's your problem right there),
is the same 20 ms visible when you ping them from each other?

Reiser4 Inclusion

[Caleb Gray]

Dear Linux Kernel Developers,

I would like to express my experiences with the reiser4 filesystem and
my reasons for its readiness to be officially included in the Linux kernel.

I have been putting together servers since 2001, all of which are still
operational and serving web sites reliably. The earliest servers I built
used ext3 for their primary filesystems. Overtime I realized that I
needed a faster filesystem for my servers' so I tried reiserfs. Those
servers were, in fact, more responsive but carried several headaches
into my life due to severe unreliability and so I was forced to convert
all of the reiserfs servers to ext3. It wasn't until two years ago that
I read about reiser4 and felt as though I should give the new reiser
filesystem a chance. After two years of reiser4 and five years of ext3,
I can attest to three things that reiser4 does just as well or better
than ext3: speed, responsiveness, and reliability. This is not to say
that reiser4 is _better_ than ext3, this is to simply say that it is as
production ready as ext3 is.

The reliability of reiser4 does _NOT_ compete with ext3 but it doesn't
falter from it either. For every time that I have to run fsck.ext3, I
have to run fsck.reiser4. Every time one of my servers crash, whether
it's ext3 or reiser4, I spend the exact same amount of time recovering
lost/broken files. And to note: the atomic file saving system that
reiser4 implements has never caused me any issues during file recovery.

Reiser4's responsiveness is undoubtedly at least twice as fast as ext3.
I have deployed two nearly identical servers in Florida (I live in
Washington state) but one difference: one uses ext3 and the other
reiser4. The ping time of the reiser4 server is (on average) 20ms faster
than the ext3 server. It has maintained this speed for the past two
years against the ext3 server even with aging hardware and bulking file
and directory structures. (Both of the filesystems have slowed down at a
similar pace for the duration of their lifetime [about 15ms].)

And finally reiser4's speed. I am constantly transferring files between
other servers, and hard drives. The servers are also (obviously) serving
data to the viewers of web sites, dealing with huge email queues (a few
gigabytes every few hours), and handling heavy cron jobs to tarball user
dirs from one drive to another. The reiser4 and ext3 servers deal with
relatively the same amount of data to compress (~190GiB each), and the
reiser4 is and always has been the first to finish. Not only finish
first though, it generally finishes about 45 minutes before the ext3
server. (You can ignore the idea that it's probably the CPUs that can't
handle the compression not the filesystems, because while the backup is
running on both dual core processors the load never surpasses 45%; the
bottleneck comes down to the throughput efficiency of data between drives.)

The purpose of this email is not to bash ext3. As I have said I have a
five year old ext3 server that runs great, and I intend to keep it that
way. The reason that I have sent this is to present real life situations
where reiser4 is reliable, fast, usable, and production ready. It is
both realistic and reasonable to say that reiser4 is prepared to be
officially supported in the Linux kernel.

Please consider the fact that I have patched my servers' kernels time
and time again, with all kinds of patches, and I have never once had an
issue with the reiser4 patched kernels. Thank you for taking time away
from development to read this email (I'm a programmer too), I know how
it is.

Sincerely,
Caleb Gray

Re: Reiser4 Inclusion

[Arjan van de Ven]

On Sun, 2006-07-16 at 20:02 -0700, Caleb Gray wrote:
> Dear Linux Kernel Developers,
> 
> I would like to express my experiences with the reiser4 filesystem and
> my reasons for its readiness to be officially included in the Linux kernel.


Hi,

may I ask why you are sending this? Have you done code audits to the
code? Have you done anything that was on the "these need fixing before
it can go in" list? If not, aren't you just doing campaigning on
non-technical grounds? And isn't that a bad idea?


Arjan van de Ven -- who is starting to smell a directed PR campaign
leading to allergic reactions 

Re: Reiser4 Inclusion

[Grzegorz Kulewski]

Hi Arjan,

On Mon, 17 Jul 2006, Arjan van de Ven wrote:
> On Sun, 2006-07-16 at 20:02 -0700, Caleb Gray wrote:
>> Dear Linux Kernel Developers,
>>
>> I would like to express my experiences with the reiser4 filesystem and
>> my reasons for its readiness to be officially included in the Linux kernel.
>
> Hi,
>
> may I ask why you are sending this? Have you done code audits to the
> code? Have you done anything that was on the "these need fixing before
> it can go in" list?

Well, as I understand he is end user (an advanced one). He does his job as 
a end user: he does testing and reports back the results. This is not that 
common, as many users do not report problems / requests they have.

He even did more: he tested (very hard and extensively) experimental, not 
even in the tree part of the kernel. And he reported problems / ideas he 
found in a very kind and gentle way. This is not so common and makes him a 
valuable person in the users comunity in my opinion.

As I understand you, as a developer, should say "thank you" to him and 
make everything you can to solve the problems he has and help implement 
the parts of the software he needs. No?

That way you build comunity of users that not only are using the software 
but also are giving back in form of bug reports, feature requests, 
continuous testing on variety of setups (that no developer ever can have 
all), reviews, ideas, telling others about what a great software with 
friendly comunity they found and so on.

For me (I am active end user of most open source projects and developer 
on others) the comunity and good contacs between developers and end users 
is the most important part of the software. It gives me security. Even if 
the software is not yet stable it can be fixed by cooperation between 
users and developers. While people are really way harder to fix than 
software.

And he as a end user does not have to (and probably does not even have 
enough knowledge about the kernel internals) make code audits and 
review of new filesystem. So why are you demanding that he does one?


> If not, aren't you just doing campaigning on
> non-technical grounds? And isn't that a bad idea?

Well, his kind message was not very technical. But wasn't completly non 
technical or flamewar either. He tested software, compared and reported 
what he saw. He also expressed wish (that many users have) that Reiser4,
as a usefull and even useable in some production evironments, should be 
integrated into the kernel. Because there are users for it.


> Arjan van de Ven -- who is starting to smell a directed PR campaign
> leading to allergic reactions

Come on. Another conspiracy theory? Why some people just can't understand 
that Reiser4 is not that bad (from end user's point of view)? Some people 
tested it and found it good and want to have it integrated ASAP. Some even 
can't live without it after they used it for a while and saw how good it 
is in something...

I can assure you that it really is not some directed centrally controlled 
campaign. This is just what many users want.

I too tested Reiser4 some time ago. It didn't have any big problems for 
me. But I am not using (or testing) it now. Why? Mainly because of 
security: if Reiser4 is not merged (even as a experminental, subject to 
change, unstable, whatever) it will work with new kernels as long as 
Namesys will release patches. And if something happens to Namesys I will 
have to port it to new kernels (and that is usually trivial for kernel 
developers introducing incompatible internal kernel API changes but not 
for me) myself or will have to use old kernels. And _that_ is a problem 
for me.

(Not to mention that I am regulary applying 4-7 patches, some big ones, 
for every kernel I am building and resolving merge problems in not your 
code is not easy thing to do and takes time. While I can live without 
staircase scheduler or vesafb-tng if my manual merge attempt fails I can 
not do so without my main filesystem. And -mm is a little too unstable for 
me recently.)

It is unfortunate that Hans Reiser pushed Reiser4 the way he did and that 
he got the reaction from some kernel developers he did got. But he and his 
developers did (and are still doing) very hard job to fix problems and 
make Reiser4 better and more suitable into the kernel. And having Reiser4 
out of the kernel is hurting mainly end users. Really.

Arjan, is this really technically impossible to have Reiser4 merged into 
the kernel after fixing some worst problems that touch mm and VFS (in say 
2 months), flagged unofficial-try-merge-for-testing, super-experimental 
and subject-to-change? I would make live of many end users easier and does 
not sound that bad for me especially in the 2.6 forever era...

If someone thinks that Reiser4 is too unstable or evil he can set it to N 
and be happy. And if Reiser4 will be abandoned by Namesys and not fixed 
further it could be maintained by kernel developers at a minimal level 
(porting to new kernel internal APIs as they change) for say 6-12 months 
while flagged for removal and then removed because of 
unofficial-try-merge-for-testing flag. This at least does give some time 
to migrate from it for end users (and maybe even time to fix it for some 
other developers?).


Thanks and sorry for such long post,
wrong as usual,

Grzegorz Kulewski

Re: Reiser4 Inclusion

[Alexander Gran]

[-- Attachment #1: Type: text/plain, Size: 304 bytes --]

Am Montag, 17. Juli 2006 13:48 schrieb Grzegorz Kulewski:
> Thanks and sorry for such long post,
> wrong as usual,

100% ACK, Nevertehless!

regards,
Alex, reiser4 on /, /home and /files

-- 
Encrypted Mails welcome.
PGP-Key at http://zodiac.dnsalias.org/misc/pgpkey.asc | Key-ID: 0x6D7DD291

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Reiser4 Inclusion

[Diego Calleja]

El Mon, 17 Jul 2006 13:48:02 +0200 (CEST),
Grzegorz Kulewski <kangur@polcom.net> escribió:


> If someone thinks that Reiser4 is too unstable or evil he can set it to N 
> and be happy. And if Reiser4 will be abandoned by Namesys and not fixed 


http://wiki.kernelnewbies.org/WhyReiser4IsNotIn

Re: Reiser4 Inclusion

[Grzegorz Kulewski]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3511 bytes --]

On Mon, 17 Jul 2006, Diego Calleja wrote:
> El Mon, 17 Jul 2006 13:48:02 +0200 (CEST),
> Grzegorz Kulewski <kangur@polcom.net> escribió:
>> If someone thinks that Reiser4 is too unstable or evil he can set it to N
>> and be happy. And if Reiser4 will be abandoned by Namesys and not fixed
>
> http://wiki.kernelnewbies.org/WhyReiser4IsNotIn

I already read it when it was posted first. I am reading LKML and 
reiserfs-list for several years and I already read all that arguments, 
flames and so on that were ever pointed here. I think I have enough.

But if we are there:

""But just include it as experimental code regardless of everything, 
reiser programmers will fix all the problems eventually!"

Well, no and yes. As said, nobody expects reiser 4 to be bug-free, but 
there're some important issues that need to be fixed, the problems is 
that reiser 4 is still working in the important ones. Some of the issues 
fixed in the past included severe design issues, BTW. Others are about 
being well integrated with Linux: duplication of kernel's own 
functionality for no reason, etc. Every piece of code submitted needs to 
have some quality - requesting developers to fix severe issues before 
getting it into the main tree helps to have better code. If you ask 
people people to fix those issues "in the future", they'll be lazy and 
there'll be critical issues around all the time - this has happened in 
Linux in the past. Quality is important, specially under a stable 
development phase. Linux is already being critized a lot for merging new 
features during this stable phase - that criticism happens with the 
current quality control. Imagine what would happen if linux started to 
merge things without caring a bit about what gets merged. Also, consider 
what Reiser 4 is. It's a filesystem, once it gets included in the kernel 
many people WILL use it and will DEPEND on it (your disk format is 
reiser4): Linux needs to ensure that things don't blow up everything."

Why do some people think that users are not already depending on it? They 
are. I don't know how much but I am willing to bet that at least 100 
people. I think that there are some drivers in the kernel that have 
smaller user base.

Keeping Reiser4 out of kernel is even worse (for those users, other users 
that could test this filesystem, for Reiser4 developers and whole 
comunity) than accepting it for a try period with a big fat warning that 
it my be removed if Namesys abandons futher fixing of it (after some time 
to let user migrate).

And any arguments like "if Reiser4 is not in the kernel then people will 
not use and depend on it" are fundamentally flawed IMHO. Everything bad 
that could happen with Reiser4 in the kernel can happen with Reiser4 out 
of it.

It may look like some kernel developers are trying hard not to take 
responsibility for Reiser4 saying that there is very huge difference 
between selecting highly experimental kernel feature that is marked so and 
patching the kernel with it. Sorry but I think there is very little 
difference. And that little difference is only hurting users that want to 
try and test something new.


Thanks,

Grzegorz Kulewski


PS. I really don't want to begin World War 4 about Reiser4. I just think 
that curious people asking from time to time about _current_ Reiser4 
status should not be treated bad because that could make them stop 
testing and giving back to the open source projects.

Re: Reiser4 Inclusion

[Matthias Andree]

On Mon, 17 Jul 2006, Grzegorz Kulewski wrote:

> Keeping Reiser4 out of kernel is even worse (for those users, other users 
> that could test this filesystem, for Reiser4 developers and whole 
> comunity) than accepting it for a try period with a big fat warning that 
> it my be removed if Namesys abandons futher fixing of it (after some time 
> to let user migrate).

And what are you going to do in the meanwhile, if reiser4 should be
fundamentally broken because namesys decided reiser11 was so much cooler
to work on, as has happened with 3.6 which still cannot handle hash
collisions properly?

> And any arguments like "if Reiser4 is not in the kernel then people will 
> not use and depend on it" are fundamentally flawed IMHO. Everything bad 
> that could happen with Reiser4 in the kernel can happen with Reiser4 out 
> of it.

Right, but accepting it into baseline is seen as "endorsement" by major
parts of the audience, and this is in fact what they like to see.

> It may look like some kernel developers are trying hard not to take 
> responsibility for Reiser4 saying that there is very huge difference 
> between selecting highly experimental kernel feature that is marked so and 
> patching the kernel with it. Sorry but I think there is very little 
> difference. And that little difference is only hurting users that want to 
> try and test something new.

Any why can those users not download and apply a patch, try it and
report back to Namesys?

-- 
Matthias Andree

Re: Reiser4 Inclusion

[Jan Engelhardt]

>
>Any why can those users not download and apply a patch, try it and
>report back to Namesys?
>
Because that patch pokes into some files outside fs/reiser4, giving a hard time
with rejects for the novice user. The linux kernel is a huge codebase, and 
I have, for example, less clue of mm/ than of fs/.

I was very happy to see that
ftp://ftp.namesys.com/pub/reiser4-for-2.6/2.6.16/reiser4-for-2.6.16-4.patch.gz
worked well on 2.6.17 and 2.6.17.x, but it already stopped patching again 
on 2.6.18-rc1. The next newer version of reiser4 is for -mm, which is not 
so useful for me. If namesys provided reiser4 patches for every vanilla out 
there (possibly including -rc's, but that's just extra sugar), that would 
be great, but I cannot force them to do so; people may have better things 
to do than packaging up r4 whenever there is a linux tarball release.



Jan Engelhardt
-- 

Re: Reiser4 Inclusion

[Matthias Andree]

Jan Engelhardt schrieb am 2006-07-18:

> Because that patch pokes into some files outside fs/reiser4, giving a hard time
> with rejects for the novice user. The linux kernel is a huge codebase, and 
> I have, for example, less clue of mm/ than of fs/.
> 
> I was very happy to see that
> ftp://ftp.namesys.com/pub/reiser4-for-2.6/2.6.16/reiser4-for-2.6.16-4.patch.gz
> worked well on 2.6.17 and 2.6.17.x, but it already stopped patching again 
> on 2.6.18-rc1. The next newer version of reiser4 is for -mm, which is not 

Sorry, none of the kernel's business.

ISTR they have been asked to review/justify the bits outside fs/reiser4
and feed them separately where valuable and otherwise move or drop.

> so useful for me. If namesys provided reiser4 patches for every vanilla out 
> there (possibly including -rc's, but that's just extra sugar), that would 
> be great, but I cannot force them to do so; people may have better things 
> to do than packaging up r4 whenever there is a linux tarball release.

And probably kernel hackers have better things to do than keeping that
code building if they don't mean to support it. This touches the "stable
APIs" can of worms again, so let's stop here before it springs open.

- 
Matthias Andree

Re: Reiser4 Inclusion

[Joshua Hudson]

> And probably kernel hackers have better things to do than keeping that
> code building if they don't mean to support it. This touches the "stable
> APIs" can of worms again, so let's stop here before it springs open.

Hey, at least its stable APIs rather than stable ABIs :-)

Re: Reiser4 Inclusion

[Tilman Schmidt]

On Tue, 18 Jul 2006 22:50:06 +0200, Matthias Andree wrote:

> Jan Engelhardt schrieb am 2006-07-18:
>
>> If namesys provided reiser4 patches for every vanilla out 
>> there [...], that would 
>> be great, but I cannot force them to do so; people may have better things 
>> to do than packaging up r4 whenever there is a linux tarball release.
>
> And probably kernel hackers have better things to do than keeping that
> code building if they don't mean to support it. This touches the "stable
> APIs" can of worms again, so let's stop here before it springs open.

But that's exactly the point. No good sweeping it under the carpet.
The entire concept of not having a stable API hinges on being able
to get code into the main kernel tree, and "kernel hackers keeping
that code building" is explicitly part of the promise. As the
document with the nicely nettling name "stable_api_nonsense.txt"
says:

  You think you want a stable kernel interface, but you really do not, and
  you don't even know it.  What you want is a stable running driver, and
  you get that only if your driver is in the main kernel tree.

Conversely, the fact that for very good reasons there isn't, and
won't be, a stable API is a major source of pressure to get things
into the kernel. It is even touted as such. Each time some
maintainer of an out-of-tree piece of code asks a question about an
incompatible change he or she is told: "it wouldn't be a problem if
your code were in the main kernel tree".

But all the nice arguments turn moot if someone who takes them to
heart and duly submits his or her code for inclusion in the main
kernel tree is turned away at the door. Someone who *cannot* get his
or her driver into the main kernel tree (no matter for what reason)
will quite naturally conclude that stable_api_nonsense.txt is itself
nonsense and a stable API might not be such a bad idea after all.

You can't have it both ways. Either you want everything in the main
kernel tree or you don't. Of course there will always be a limbo of
code waiting for inclusion. But if it has to linger there for too
long (again: no matter for what reason) then it invalidates the
whole concept of not having a stable API. And that would be a pity.

HTH
Tilman

Re: Reiser4 Inclusion

[Krzysztof Halasa]

Tilman Schmidt <tilman@imap.cc> writes:

> You can't have it both ways. Either you want everything in the main
> kernel tree or you don't.

Of course you don't. Only things which improve Linux. Preferably nice,
clean designs + implementations, and if possible, actively maintained
(especially non-trivial ones).
-- 
Krzysztof Halasa

Re: Reiser4 Inclusion

[Pekka Enberg]

On 7/19/06, Tilman Schmidt <tilman@imap.cc> wrote:
> You can't have it both ways. Either you want everything in the main
> kernel tree or you don't. Of course there will always be a limbo of
> code waiting for inclusion. But if it has to linger there for too
> long (again: no matter for what reason) then it invalidates the
> whole concept of not having a stable API. And that would be a pity.

You seem to have this idea of everyone having some sacred right of
having their code either in the kernel or supported by the kernel
developers. That's fine, but last I checked, does not apply to Linux.
It is up to the maintainers to decide what's included and what's not.
We obviously don't want _everything_ in the kernel anyway and don't
want to stagnate the kernel development because of out-of-tree code
either. If you're unhappy about maintainer decisions, feel free to
complain to them or better yet, step up to do the necessary legwork to
get the code included. After all, that's how Linux development works.

Re: Reiser4 Inclusion

[Tilman Schmidt]

Pekka Enberg wrote:

> On 7/19/06, Tilman Schmidt <tilman@imap.cc> wrote:
>> You can't have it both ways. Either you want everything in the main
>> kernel tree or you don't. Of course there will always be a limbo of
>> code waiting for inclusion. But if it has to linger there for too
>> long (again: no matter for what reason) then it invalidates the
>> whole concept of not having a stable API. And that would be a pity.
> 
> You seem to have this idea of everyone having some sacred right of
> having their code either in the kernel or supported by the kernel
> developers.

No, and seriously I cannot see how you could possibly get that
impression from what I wrote.

> It is up to the maintainers to decide what's included and what's not.

Ok. But then it's also up to them to stand by their decision and
take the heat for it. Don't jump from your left foot to your right
foot, saying "submit your code to the kernel" if someone wants to
maintain something off-tree and "maintain it separately" if they
try to submit it. State clearly: "We do not want Reiser4 in Linux"
and defend it, instead of creating a string of ifs and then
complaining that people go on about it.

> We obviously don't want _everything_ in the kernel anyway and don't
> want to stagnate the kernel development because of out-of-tree code
> either.

Well, that doesn't make sense. You can't have your cake and eat it
too. Either you have out-of-tree code or you haven't. Documents
like stable_api_nonsense.txt explicitly discourage out-of-tree code,
which is formally equivalent to saying that all kernel code should
be in-tree. Therefore an attitude which says "go on developing that
code out-of-tree, it's not ready for inclusion yet" is in direct
contradiction with the foundations of the no-stable-API policy.

> If you're unhappy about maintainer decisions, feel free to
> complain to them

Well, isn't that what the present thread is all about?

> or better yet, step up to do the necessary legwork to
> get the code included.

That's completely beside the point. There are enough people already
who are willing to do the legwork. The question is whether they and
the kernel maintainers will be able to bridge their differences
about how it should be done, and that would not be made easier by
another party entering the arena.

Seriously, what do you think would happen if I actually took the
Reiser4 code, modified it according to what I think the kernel
people would like to see, and submitted the result to lkml? I'll
tell you what'd happen: I would get heat from both sides, I would
be blamed both for perceived flaws in the original design and for
any deviation from it, my motives would be questioned, I would be
asked whether I'd commit to maintaining that code for the
foreseeable future, and there would be no right answer to that
question. No matter what I did, it would only make things worse.

(Please note that the scenario above is completely fictitious. I am
neither capable of, nor interested in, taking over the development
of Reiser4 or promoting its admission into the kernel tree.)

> After all, that's how Linux development works.

Obviously there's more to it than that. There are people involved.

HTH
Tilman

Re: Reiser4 Inclusion

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1476 bytes --]

On Wed, 19 Jul 2006 17:32:48 +0200, Tilman Schmidt said:
 
> Well, that doesn't make sense. You can't have your cake and eat it
> too. Either you have out-of-tree code or you haven't. Documents
> like stable_api_nonsense.txt explicitly discourage out-of-tree code,
> which is formally equivalent to saying that all kernel code should
> be in-tree. Therefore an attitude which says "go on developing that
> code out-of-tree, it's not ready for inclusion yet" is in direct
> contradiction with the foundations of the no-stable-API policy.

Which part of "read Documentation/SubmittingPatches.txt and do what it says,
or it doesn't get into the kernel" do you have trouble understanding?

It isn't a case of "out of tree code or you haven't". There's actually
*three* major categories:

1) Code that's already in-tree and maintained.  These guys don't need to
worry about the API, as it will usually get handled free of charge.

2) Code that's out-of-tree, but a potential (after possible rework) candidate
for submission (for instance, the hi-res timers, CKRM, some drivers, etc).
These guys need to forward-port their code for API changes as they work
towards getting their code into the tree.

3) Code that's out-of-tree, but is so far out in left field that there's
no way it will ever go in.  For instance, that guy with the MVS JCL emulator
better not be holding his breath waiting.  And quite frankly, nobody else
really cares whether they forward port their code or not.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4 Inclusion

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 1971 bytes --]

On 19.07.2006 21:04, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 19 Jul 2006 17:32:48 +0200, Tilman Schmidt said:
>  
>>Well, that doesn't make sense. You can't have your cake and eat it
>>too. Either you have out-of-tree code or you haven't. Documents
>>like stable_api_nonsense.txt explicitly discourage out-of-tree code,
>>which is formally equivalent to saying that all kernel code should
>>be in-tree. Therefore an attitude which says "go on developing that
>>code out-of-tree, it's not ready for inclusion yet" is in direct
>>contradiction with the foundations of the no-stable-API policy.
> 
> Which part of "read Documentation/SubmittingPatches.txt and do what it says,
> or it doesn't get into the kernel" do you have trouble understanding?

None. Why do you think I'd have? And what relevance does this have to
the present discussion?

> It isn't a case of "out of tree code or you haven't". There's actually
> *three* major categories:
> 
> 1) Code that's already in-tree and maintained.  These guys don't need to
> worry about the API, as it will usually get handled free of charge.
> 
> 2) Code that's out-of-tree, but a potential (after possible rework) candidate
> for submission (for instance, the hi-res timers, CKRM, some drivers, etc).
> These guys need to forward-port their code for API changes as they work
> towards getting their code into the tree.
> 
> 3) Code that's out-of-tree, but is so far out in left field that there's
> no way it will ever go in.  For instance, that guy with the MVS JCL emulator
> better not be holding his breath waiting.  And quite frankly, nobody else
> really cares whether they forward port their code or not.

Correct. And you could easily subdivide it further. Your point being?

-- 
Tilman Schmidt                          E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 253 bytes --]

Re: Reiser4 Inclusion

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

On Wed, 19 Jul 2006 21:12:21 +0200, Tilman Schmidt said:
> On 19.07.2006 21:04, Valdis.Kletnieks@vt.edu wrote:
> > Which part of "read Documentation/SubmittingPatches.txt and do what it says,
> > or it doesn't get into the kernel" do you have trouble understanding?
> 
> None. Why do you think I'd have? And what relevance does this have to
> the present discussion?

So in other words, you're trying to tell us about what code should or
shouldn't allow into the kernel when you don't have even a clue what's
required on a purely technical level, or how the code gets into the kernel.

> > It isn't a case of "out of tree code or you haven't". There's actually
> > *three* major categories:

> Correct. And you could easily subdivide it further. Your point being?

You're the one who was trying to paint it as a binary "either you have out of
tree code or you don't", even though the actual situation is much more
complicated.  You want to subdivide it even further, go right ahead. But the
more you subdivide, the less binary it is....


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4 Inclusion

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On 19.07.2006 22:09, Valdis.Kletnieks@vt.edu wrote:

> On Wed, 19 Jul 2006 21:12:21 +0200, Tilman Schmidt said:
> 
>>On 19.07.2006 21:04, Valdis.Kletnieks@vt.edu wrote:
>>
>>>Which part of "read Documentation/SubmittingPatches.txt and do what it says,
>>>or it doesn't get into the kernel" do you have trouble understanding?
>>
>>None. Why do you think I'd have? And what relevance does this have to
>>the present discussion?
> 
> So in other words, you're trying to tell us about what code should or
> shouldn't allow into the kernel when you don't have even a clue what's
> required on a purely technical level, or how the code gets into the kernel.

That's entirely uncalled for. If you deem it necessary to resort to
insults when you are out of arguments then please go looking for another
victim. I consider the discussion closed.

-- 
Tilman Schmidt                          E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 253 bytes --]

Re: Reiser4 Inclusion

[Jeff V. Merkey]

A big ego thing I guess. 

Keeping a lot of FS's and code out of tree has a lot of advatanges (like 
not getting targeted by folks
claiming IP issues suing Linus and Co.).    I am still distributing NWFS 
out of the tree, and there's still 2000+ downloads per month from
three separate locations -- code I have not touched in three years.   

Projects I have taken commercial are big money makers and obviously will 
not be submitted to the tree.  Reiser is shipped in Suse
Linux as the default, so who cares whether its in the tree or not -- it 
still has succeeded.  You guys should keep the stuff out of the tree
(in fact most of the FS's and other code should not be in the tree).

The whole Linux base is too big as it is and becoming larger.  Only 
minimal drivers and FS's should be in the tree.  Less broken stuff
and dependencies.  I have been watching development cycles take longer 
and longer to get Linux kernels out -- Red Hat is STILL
shipping 2.6.9-22 with ES4 (update kernels have various issues of 
stability).

The smaller the better.    You have more control if you keep it out, and 
greater opporutnities for serious folks to do business
deals to promote your stuff.

Jeff

Valdis.Kletnieks@vt.edu wrote:

>On Wed, 19 Jul 2006 17:32:48 +0200, Tilman Schmidt said:
> 
>  
>
>>Well, that doesn't make sense. You can't have your cake and eat it
>>too. Either you have out-of-tree code or you haven't. Documents
>>like stable_api_nonsense.txt explicitly discourage out-of-tree code,
>>which is formally equivalent to saying that all kernel code should
>>be in-tree. Therefore an attitude which says "go on developing that
>>code out-of-tree, it's not ready for inclusion yet" is in direct
>>contradiction with the foundations of the no-stable-API policy.
>>    
>>
>
>Which part of "read Documentation/SubmittingPatches.txt and do what it says,
>or it doesn't get into the kernel" do you have trouble understanding?
>
>It isn't a case of "out of tree code or you haven't". There's actually
>*three* major categories:
>
>1) Code that's already in-tree and maintained.  These guys don't need to
>worry about the API, as it will usually get handled free of charge.
>
>2) Code that's out-of-tree, but a potential (after possible rework) candidate
>for submission (for instance, the hi-res timers, CKRM, some drivers, etc).
>These guys need to forward-port their code for API changes as they work
>towards getting their code into the tree.
>
>3) Code that's out-of-tree, but is so far out in left field that there's
>no way it will ever go in.  For instance, that guy with the MVS JCL emulator
>better not be holding his breath waiting.  And quite frankly, nobody else
>really cares whether they forward port their code or not.
>  
>

Re: Reiser4 Inclusion

[Matthias Andree]

On Wed, 19 Jul 2006, Jeff V. Merkey wrote:

> Projects I have taken commercial are big money makers and obviously will 
> not be submitted to the tree.

So let's hope they're in compliance with the GPL...

> Reiser is shipped in Suse Linux as the default, so who cares whether
> its in the tree or not

That is Reiserfs 3.6 which was officially discontinued by its
maintainers, and has rather little relevance to the discussion: no
reiser4 or dropping 3.X support was on the horizon when reiserfs 3.5 was
merged (and messed up horribly when NFS exported).

-- 
Matthias Andree

Re: Reiser4 Inclusion

[Helge Hafting]

On Wed, Jul 19, 2006 at 05:32:48PM +0200, Tilman Schmidt wrote:
> Pekka Enberg wrote:
> 
> > On 7/19/06, Tilman Schmidt <tilman@imap.cc> wrote:
> >> You can't have it both ways. Either you want everything in the main
> >> kernel tree or you don't. Of course there will always be a limbo of
> >> code waiting for inclusion. But if it has to linger there for too
> >> long (again: no matter for what reason) then it invalidates the
> >> whole concept of not having a stable API. And that would be a pity.
> > 
> > You seem to have this idea of everyone having some sacred right of
> > having their code either in the kernel or supported by the kernel
> > developers.
> 
> No, and seriously I cannot see how you could possibly get that
> impression from what I wrote.
> 
> > It is up to the maintainers to decide what's included and what's not.
> 
> Ok. But then it's also up to them to stand by their decision and
> take the heat for it. Don't jump from your left foot to your right
> foot, saying "submit your code to the kernel" if someone wants to
> maintain something off-tree and "maintain it separately" if they
> try to submit it. State clearly: "We do not want Reiser4 in Linux"
> and defend it, instead of creating a string of ifs and then
> complaining that people go on about it.
> 
It really is "_clean up_ and submit your code to the kernel, and then
we _will_ include it _if_ we find it useful!"

Reiserfs4 doesn't go in currently, because  nobody bothered cleaning
it sufficiently up.  Any popular filesystem is considered useful,
so reiserfs4 goes right in as soon as:
1) it gets cleaned up just the way the kernel maintainers wants it. 
   (And yes, that could be some work.  Code with known problems in
    it doesn't go in until they are fixed)
2) Some maintainer have the will and time to look at it and see that it
   is indeed of good quality.  That will is worn thin by reiserfs people
   quarreling with the maintainers, but I guess this could be rectified
   with some apologies and a serious attempt at the above mentioned
   cleanup.

If you want reiserfs4 in - either help the reiserfs team with the
cleanup, or bully _them_ into doing it.  Complaining here won't do the
trick at all, as you are not in a position to put pressure on
the developers. (I.e. you are neither their boss nor their paying
customer.) Linux development isn't market-driven.

> > We obviously don't want _everything_ in the kernel anyway and don't
> > want to stagnate the kernel development because of out-of-tree code
> > either.
> 
> Well, that doesn't make sense. You can't have your cake and eat it
> too. Either you have out-of-tree code or you haven't. Documents
> like stable_api_nonsense.txt explicitly discourage out-of-tree code,
> which is formally equivalent to saying that all kernel code should
> be in-tree. Therefore an attitude which says "go on developing that
> code out-of-tree, it's not ready for inclusion yet" is in direct
> contradiction with the foundations of the no-stable-API policy.
> 
If you think you caught anyone in a self-contradiction, it won't help
you.  It only means the real rules is a bit more complicated:

1. We want in-tree code, and so no attempt is made to make it easy to
have out of tree code
2. High quality code is more important than quantity, so code with
problems don't go in!

It is up to whoever wants a piece of code in, to clean it up so well
that can be added to the kernel tree.  Sometimes a kernel maintainer will
do that job - because he want the stuff in himself.  So he volunteers.
At other times, some piece of code "could be nice to have" but no
maintainer thinks it is so nice that they clean it up themselves.
Reiserfs4 is in this category.  It is then up to reidser4 fans to get
the job done - by doing it themselves or raising enough money to pay
someone to do it.  

> > If you're unhappy about maintainer decisions, feel free to
> > complain to them
> 
> Well, isn't that what the present thread is all about?
> 
> > or better yet, step up to do the necessary legwork to
> > get the code included.
> 
> That's completely beside the point. There are enough people already
> who are willing to do the legwork. The question is whether they and
> the kernel maintainers will be able to bridge their differences
> about how it should be done, and that would not be made easier by
> another party entering the arena.
>
Actually, there is nothing wrong in taking over maintainership of
reiserfs4.  Sure, you will annoy the namesys people but such is the
nature of open source projects - good stuff badly maintained can be
taken over anytime by a better maintainer.  That has happened several
times, look at the history of gcc for example.

If there really was enough people willing to do the legwork - it'd be
done or at least an ongoing effort now.  How to do it is simple - the
way the kernel maintainers tell them to.  There is little use
disagreeing about that.  One can complain to Linus, other than that,
there is no way around the maintainers.

> Seriously, what do you think would happen if I actually took the
> Reiser4 code, modified it according to what I think the kernel
> people would like to see, and submitted the result to lkml? I'll
> tell you what'd happen: I would get heat from both sides, I would
> be blamed both for perceived flaws in the original design and for
> any deviation from it, my motives would be questioned, I would be
> asked whether I'd commit to maintaining that code for the
> foreseeable future, and there would be no right answer to that
> question. No matter what I did, it would only make things worse.
>
The kernel side would not flame you for this - if you did good work and
managed to remain polite.  Of course you wouldn't get it in immediately,
you'd expect to have several iterations of fixing stuff, submitting, get
feedback about more wrong things to fix - but eventually you'd get it
in!

This process can take time, especially with something as big as
reiserfs.  The general opinion here is that the reiserfs developers
didn't do this part properly, when they couldn't get reiserfs4 in
quickly, they argued instead of fixing more.  But that approach doesn't
work.  A "no" or "fix that too" tend to be final around here, and
arguing about it is only considered rude.  And yes - there is the
real risk of annoying a maintainer so bad he won't look at the stuff
again.  The kernel effort is not a company, people here are allowed to
take things personally and bear grudges.  Anyone who want code into the
kernel should therefore consider this be polite at all time - even if 
they get told to "fix more stuff" for the umpteenth time.

Arguing don't work.  Those who try tend to find that clever arguments
fail, and then that resorting to nastier stuff fail even worse.

> (Please note that the scenario above is completely fictitious. I am
> neither capable of, nor interested in, taking over the development
> of Reiser4 or promoting its admission into the kernel tree.)
> 
> > After all, that's how Linux development works.
> 
> Obviously there's more to it than that. There are people involved.

Indeed!

Helge Hafting

Re: Reiser4 Inclusion

[gmu 2k6]

the last time I've tried Reiser4 in -mm (4 - 6 weeks ago
approxamitely) I managed to oops by doing:
$ tar xfjv linux-2.6.17.tar.bz2
$ sync

I was calling sync because I was curious as to why XFS seemed to sync
all the time and look and feel slow when compared to ext3. So I
thought let's see what Reiser4 does and the moment I ran sync it
oopsed. Yes this could be a problem with that particular -mm version.

I'd be interested to hear whether this still happens.

no, I'm not making any statements about code quality just curious to
hear as you seem to be using Reiser4 successfully.

On 7/17/06, Grzegorz Kulewski <kangur@polcom.net> wrote:
> On Mon, 17 Jul 2006, Diego Calleja wrote:
> > El Mon, 17 Jul 2006 13:48:02 +0200 (CEST),
> > Grzegorz Kulewski <kangur@polcom.net> escribió:
> >> If someone thinks that Reiser4 is too unstable or evil he can set it to N
> >> and be happy. And if Reiser4 will be abandoned by Namesys and not fixed
> >
> > http://wiki.kernelnewbies.org/WhyReiser4IsNotIn
>
> I already read it when it was posted first. I am reading LKML and
> reiserfs-list for several years and I already read all that arguments,
> flames and so on that were ever pointed here. I think I have enough.
>
> But if we are there:
>
> ""But just include it as experimental code regardless of everything,
> reiser programmers will fix all the problems eventually!"
>
> Well, no and yes. As said, nobody expects reiser 4 to be bug-free, but
> there're some important issues that need to be fixed, the problems is
> that reiser 4 is still working in the important ones. Some of the issues
> fixed in the past included severe design issues, BTW. Others are about
> being well integrated with Linux: duplication of kernel's own
> functionality for no reason, etc. Every piece of code submitted needs to
> have some quality - requesting developers to fix severe issues before
> getting it into the main tree helps to have better code. If you ask
> people people to fix those issues "in the future", they'll be lazy and
> there'll be critical issues around all the time - this has happened in
> Linux in the past. Quality is important, specially under a stable
> development phase. Linux is already being critized a lot for merging new
> features during this stable phase - that criticism happens with the
> current quality control. Imagine what would happen if linux started to
> merge things without caring a bit about what gets merged. Also, consider
> what Reiser 4 is. It's a filesystem, once it gets included in the kernel
> many people WILL use it and will DEPEND on it (your disk format is
> reiser4): Linux needs to ensure that things don't blow up everything."
>
> Why do some people think that users are not already depending on it? They
> are. I don't know how much but I am willing to bet that at least 100
> people. I think that there are some drivers in the kernel that have
> smaller user base.
>
> Keeping Reiser4 out of kernel is even worse (for those users, other users
> that could test this filesystem, for Reiser4 developers and whole
> comunity) than accepting it for a try period with a big fat warning that
> it my be removed if Namesys abandons futher fixing of it (after some time
> to let user migrate).
>
> And any arguments like "if Reiser4 is not in the kernel then people will
> not use and depend on it" are fundamentally flawed IMHO. Everything bad
> that could happen with Reiser4 in the kernel can happen with Reiser4 out
> of it.
>
> It may look like some kernel developers are trying hard not to take
> responsibility for Reiser4 saying that there is very huge difference
> between selecting highly experimental kernel feature that is marked so and
> patching the kernel with it. Sorry but I think there is very little
> difference. And that little difference is only hurting users that want to
> try and test something new.
>
>
> Thanks,
>
> Grzegorz Kulewski
>
>
> PS. I really don't want to begin World War 4 about Reiser4. I just think
> that curious people asking from time to time about _current_ Reiser4
> status should not be treated bad because that could make them stop
> testing and giving back to the open source projects.
>
>

Re: Reiser4 Inclusion

[Alexander Gran]

[-- Attachment #1: Type: text/plain, Size: 780 bytes --]

Am Montag, 17. Juli 2006 17:52 schrieb gmu 2k6:
> thought let's see what Reiser4 does and the moment I ran sync it
> oopsed. Yes this could be a problem with that particular -mm version.

Yep.

> I'd be interested to hear whether this still happens.

No, We had quite some trouble for some mms (I had to recreate my root-fs 
*grr*). Current versions are running fine (older version too)

> no, I'm not making any statements about code quality just curious to
> hear as you seem to be using Reiser4 successfully.

Yes. Runs on my main machine on all partiions. Except the releases mentioned 
above no problems so far (for 1-2 years IIRC).

regards
Alex

-- 
Encrypted Mails welcome.
PGP-Key at http://zodiac.dnsalias.org/misc/pgpkey.asc | Key-ID: 0x6D7DD291

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Reiser4 Inclusion

[Horst von Brand]

Grzegorz Kulewski <kangur@polcom.net> wrote:

[...]

> Why do some people think that users are not already depending on it? 
> They are.

Foolishly, perhaps...

>           I don't know how much but I am willing to bet that at least
> 100 people. I think that there are some drivers in the kernel that
> have smaller user base.

Feel free to suggest deleting said drivers.

> Keeping Reiser4 out of kernel is even worse (for those users, other
> users that could test this filesystem, for Reiser4 developers and
> whole comunity) than accepting it for a try period with a big fat
> warning that it my be removed if Namesys abandons futher fixing of it
> (after some time to let user migrate).

I just won't work that way. Using that logic, Reiser 3 should be gone long
ago...

> And any arguments like "if Reiser4 is not in the kernel then people
> will not use and depend on it" are fundamentally flawed
> IMHO. Everything bad that could happen with Reiser4 in the kernel can
> happen with Reiser4 out of it.

Right. But this way it is /not/ the kernel crowd's job to look after it.

> It may look like some kernel developers are trying hard not to take
> responsibility for Reiser4

Why should they? They don't feel confortable with the code, believe it has
fatal design (and other) problems. Its maintainers have shown a tendency to
disregard data safety, and just dump the code when something new fancies
them. The kernel hackers can't just take over any random piece of complex
code, the only scalable way of integrating new stuff that will stay for a
/long/ time is to have reasonable expectations that whoever proposes the
code will stay around maintaining it.

>                            saying that there is very huge difference
> between selecting highly experimental kernel feature that is marked so
> and patching the kernel with it. Sorry but I think there is very
> little difference. And that little difference is only hurting users
> that want to try and test something new.

If it is in the kernel, the expectation is that it will stay in the kernel
for the foreseeable future. For a filesystem, something like the next 10
years. Thus letting in a new filesystem is a /huge/ commitment.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Reiser4 Inclusion

[Jeff Anderson-Lee]

Grzegorz Kulewski wrote:

>I too tested Reiser4 some time ago. It didn't have any big problems for me.

>But I am not using (or testing) it now. Why? Mainly because of security: 
>if Reiser4 is not merged (even as a experminental, subject to change, 
>unstable, whatever) it will work with new kernels as long as Namesys will 
>release patches. And if something happens to Namesys I will have to port it

>to new kernels (and that is usually trivial for kernel developers 
>introducing incompatible internal kernel API changes but not for me) myself

>or will have to use old kernels. And _that_ is a problem for me. 

>(Not to mention that I am regulary applying 4-7 patches, some big ones, for

>every kernel I am building and resolving merge problems in not your code is

>not easy thing to do and takes time. While I can live without staircase 
>scheduler or vesafb-tng if my manual merge attempt fails I can not do so 
>without my main filesystem. And -mm is a little too unstable for me
recently.)
 
While I cannot speak directly to the Reiser4 component of this issue, I
would like to comment on this related issue.

In the past I've wondered why so many experimental FS projects die this
death of obscurity in that they only work under FreeBSD or some ancient
version of Linux.  I'm beginning to see why that is so:  the Linux core
simply changes too fast for it to be a decent FS R&D environment!

I have been looking at implementing a COW archival file system for Linux on
and off for some time now.  While I had hoped to develop these new FS ideas
under Linux, so that they could have a longer life-time and wider exposure,
that seems to be a pipe dream with the current situation.  The file system,
VFS, and mm code has been changing so much lately it would be like trying to
build on quicksand.  The LKML has such a high volume that I cannot afford
the time to follow it 100%, but issues that would affect FS development are
often raised there, instead of in linux-fsdevel.  linux-mm often contains
issues that would affect linux-fsdevel without cross posting.  The overhead
of following all of these lists is a huge burden of time that subtracts for
the time available for development (and the rest of my job).

I saw a log-structured file system being developed as a Google summer
project recently.  It's likely doomed to obscurity by the fs-related
code-churning in the Linux kernel.  Since it is "experimental" it won't be
included in the kernel distribution and hence won't get the benefit of
kernel developers making sweeping changes that touch all the file system
dependent code.  You practically need it to be your full-time job in order
to do any research or development work under Linux with this kind of
environment.

The frequent chant of LKML is "don't write a new f/s, make changes to an
existing FS".   While there is much merit to this approach it limits the
ideas that can be tried to small incremental changes.  Also, since every
existing f/s is essentially considered as "production", each change must be
vetted by the LKML -- not ideal for "experimentation".

Things that could make Linux a better environment for FS development might
include:

1) Create a F/S "sandbox" where experimental FS can be added that will be
benefit from sweeping changes that affect f/s specific code, or

2) A lessening (moratorium?) on sweeping changes for a while, so that FS
developers would have a chance to try new ideas without being flooded with
changes needed just to keep up with the latest kernel, or

3) Better isolation of the FS dependent and FS independent code, so that
fewer sweeping changes are needed.

Of these: (1) is likely impractical, as it imposes an additional burden on
kernel developers to support obscure or experimental f/s.  (2) is only a
stop-gap, as at some point sweeping changes might again be made that would
out-date most experimental f/s.  (3) seems the most logical course: work
towards a better interface between the FS dependent and independent layers
(e.g. VFS, mm) that does a better job of isolating the layers from each
other.

Without that, *BSD (and now possibly OpenSolaris) will be preferred over
Linux for FS research, which typically means that few if any people benefit
from the results: a loss for both Linux and the community at large.

Jeff Anderson-Lee
Petabyte Storage Infrastructure Project
UC Berkeley

Re: Reiser4 Inclusion

[Matthias Andree]

On Mon, 17 Jul 2006, Jeff Anderson-Lee wrote:

> I saw a log-structured file system being developed as a Google summer
> project recently.  It's likely doomed to obscurity by the fs-related
> code-churning in the Linux kernel.  Since it is "experimental" it won't be
> included in the kernel distribution and hence won't get the benefit of
> kernel developers making sweeping changes that touch all the file system
> dependent code.  You practically need it to be your full-time job in order
> to do any research or development work under Linux with this kind of
> environment.

> 2) A lessening (moratorium?) on sweeping changes for a while, so that FS
> developers would have a chance to try new ideas without being flooded with
> changes needed just to keep up with the latest kernel, or

Other suggestions:

- try it on 2.6.16.X which is supposed to be longer-lived, and forward
  port as that system is discontinued.

- see if you can implement your concepts in user space (FUSE). If there
  are sufficient advantages, port it to the kernel.

> Of these: (1) is likely impractical, as it imposes an additional burden on
> kernel developers to support obscure or experimental f/s.  (2) is only a
> stop-gap, as at some point sweeping changes might again be made that would
> out-date most experimental f/s.  (3) seems the most logical course: work
> towards a better interface between the FS dependent and independent layers
> (e.g. VFS, mm) that does a better job of isolating the layers from each
> other.

Linux developers have often uttered their unwillingness for stable
interfaces.

> Without that, *BSD (and now possibly OpenSolaris) will be preferred over
> Linux for FS research, which typically means that few if any people benefit
> from the results: a loss for both Linux and the community at large.

If the file system is so crucial for the intended application, would not
its choice alone dominate over all other criteria and demote the weight
of the OS rather "convenience" or "it's just a name"?

-- 
Matthias Andree

Re: Reiser4 Inclusion

[Erik Mouw]

On Mon, Jul 17, 2006 at 08:13:04AM -0700, Jeff Anderson-Lee wrote:
> In the past I've wondered why so many experimental FS projects die this
> death of obscurity in that they only work under FreeBSD or some ancient
> version of Linux.? I'm beginning to see why that is so:? the Linux core
> simply changes too fast for it to be a decent FS R&D environment!

That hasn't been a problem for OCFS2 and FUSE (recently merged), and
also doesn't seem to be a problem for GFS.

> I have been looking at implementing a COW archival file system for Linux on
> and off for some time now.? While I had hoped to develop these new FS ideas
> under Linux, so that they could have a longer life-time and wider exposure,
> that seems to be a pipe dream with the current situation.? The file system,
> VFS, and mm code has been changing so much lately it would be like trying to
> build on quicksand.? The LKML has such a high volume that I cannot afford
> the time to follow it 100%, but issues that would affect FS development are
> often raised there, instead of in linux-fsdevel.? linux-mm often contains
> issues that would affect linux-fsdevel without cross posting.? The overhead
> of following all of these lists is a huge burden of time that subtracts for
> the time available for development (and the rest of my job).

A good MUA is very important to follow high volume lists (I use Mutt
and Sylpheed, anything that supports threading should do): skim the
subjects, delete any thread you're not interested in. The result is
quite a small amount of useful posts. If you still think linux-kernel
is too much, linux-fsdevel should do. In my experience the important
changes go there first (or are CC'ed from linux-kernel).

> I saw a log-structured file system being developed as a Google summer
> project recently.? It's likely doomed to obscurity by the fs-related
> code-churning in the Linux kernel.? Since it is "experimental" it won't be
> included in the kernel distribution and hence won't get the benefit of
> kernel developers making sweeping changes that touch all the file system
> dependent code.? You practically need it to be your full-time job in order
> to do any research or development work under Linux with this kind of
> environment.

FYI the amount of vfs changes is quite small. Use git to figure out:

  git-log v2.6.12..v2.6.17 | git-shortlog | grep -i vfs | grep -vi devfs | wc -l
  57
  git-log v2.6.17..v2.6.18-rc2 | git-shortlog | grep -i vfs | grep -vi devfs | wc -l
  15

There's some clutter in those numbers, so the real amount is even
smaller. Your claim about a high amount of changes in the vfs code
doesn't seem to be backed by evidence from the kernel changelog.

> The frequent chant of LKML is "don't write a new f/s, make changes to an
> existing FS".?? While there is much merit to this approach it limits the
> ideas that can be tried to small incremental changes.? Also, since every
> existing f/s is essentially considered as "production", each change must be
> vetted by the LKML -- not ideal for "experimentation".
> 
> Things that could make Linux a better environment for FS development might
> include:
> 
> 1) Create a F/S "sandbox" where experimental FS can be added that will be
> benefit from sweeping changes that affect f/s specific code, or

You can use FUSE for that: your fs lives in userspace where you can
experiment at will. FUSE will take care of the kernel specific things.
 
> 2) A lessening (moratorium?) on sweeping changes for a while, so that FS
> developers would have a chance to try new ideas without being flooded with
> changes needed just to keep up with the latest kernel, or

See Documentation/stable_api_nonsense.txt.

> 3) Better isolation of the FS dependent and FS independent code, so that
> fewer sweeping changes are needed.
> 
> Of these: (1) is likely impractical, as it imposes an additional burden on
> kernel developers to support obscure or experimental f/s.? (2) is only a
> stop-gap, as at some point sweeping changes might again be made that would
> out-date most experimental f/s.? (3) seems the most logical course: work
> towards a better interface between the FS dependent and independent layers
> (e.g. VFS, mm) that does a better job of isolating the layers from each
> other.

I opt for option (4): accept that the number of changes are low enough
to follow for any filesystem.

> Without that, *BSD (and now possibly OpenSolaris) will be preferred over
> Linux for FS research, which typically means that few if any people benefit
> from the results: a loss for both Linux and the community at large.

IMHO that's just FUD. High rates of change haven't obstructed Linux in
any way to get a larger installed base than Solaris and *BSD.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: Reiser4 Inclusion

[Jeff Dike]

On Mon, Jul 17, 2006 at 10:48:04PM +0200, Erik Mouw wrote:
> On Mon, Jul 17, 2006 at 08:13:04AM -0700, Jeff Anderson-Lee wrote:
> > In the past I've wondered why so many experimental FS projects die this
> > death of obscurity in that they only work under FreeBSD or some ancient
> > version of Linux.? I'm beginning to see why that is so:? the Linux core
> > simply changes too fast for it to be a decent FS R&D environment!
> 
> That hasn't been a problem for OCFS2 and FUSE (recently merged), and
> also doesn't seem to be a problem for GFS.

I'm been maintaining a couple (for now) out-of-tree filesystems for
UML, and have seen only minor updates needed over the course of 2.6.

Complaints about interface churn for filesystems (or anything else,
actually, since an architecture, such as UML, is exposed to nearly the
entire kernel) are imcomprehensible to me.

				Jeff

Re: Reiser4 Inclusion

[Christoph Hellwig]

On Mon, Jul 17, 2006 at 08:49:31PM -0400, Jeff Dike wrote:
> On Mon, Jul 17, 2006 at 10:48:04PM +0200, Erik Mouw wrote:
> > On Mon, Jul 17, 2006 at 08:13:04AM -0700, Jeff Anderson-Lee wrote:
> > > In the past I've wondered why so many experimental FS projects die this
> > > death of obscurity in that they only work under FreeBSD or some ancient
> > > version of Linux.? I'm beginning to see why that is so:? the Linux core
> > > simply changes too fast for it to be a decent FS R&D environment!
> > 
> > That hasn't been a problem for OCFS2 and FUSE (recently merged), and
> > also doesn't seem to be a problem for GFS.
> 
> I'm been maintaining a couple (for now) out-of-tree filesystems for
> UML, and have seen only minor updates needed over the course of 2.6.
> 
> Complaints about interface churn for filesystems (or anything else,
> actually, since an architecture, such as UML, is exposed to nearly the
> entire kernel) are imcomprehensible to me.

Yes, in 2.6 there were very little changes to the filesystem interface.
I count that as a good sign, it means our VFS and common fs helper code
has become pretty mature.

Re: Reiser4 Inclusion

[Patrick McFarland]

On Sunday 16 July 2006 23:02, Caleb Gray wrote:
> The reason that I have sent this is to present real life situations
> where reiser4 is reliable, fast, usable, and production ready. It is
> both realistic and reasonable to say that reiser4 is prepared to be
> officially supported in the Linux kernel.

I don't agree. I think reiser4 is a good development platform, and ideas 
introduced with it can be included in a later file system (such as ext4), but 
I do not think it is ready for inclusion in the Linux kernel at this time, if 
ever.

-- 
Patrick McFarland || www.AdTerrasPerAspera.com
"Computer games don't affect kids; I mean if Pac-Man affected us as kids,
we'd all be running around in darkened rooms, munching magic pills and
listening to repetitive electronic music." -- Kristian Wilson, Nintendo,
Inc, 1989

Re: Reiser4 Inclusion

[Diego Calleja]

El Sun, 16 Jul 2006 20:02:15 -0700,
Caleb Gray <caleb@calebgray.com> escribió:

> Dear Linux Kernel Developers,
> 
> I would like to express my experiences with the reiser4 filesystem and
> my reasons for its readiness to be officially included in the Linux kernel.

http://wiki.kernelnewbies.org/WhyReiser4IsNotIn

Re: Reiser4 Inclusion

[Alex Riesen]

On 7/17/06, Caleb Gray <caleb@calebgray.com> wrote:
> Reiser4's responsiveness is undoubtedly at least twice as fast as ext3.
> I have deployed two nearly identical servers in Florida (I live in
> Washington state) but one difference: one uses ext3 and the other
> reiser4. The ping time of the reiser4 server is (on average) 20ms faster
> than the ext3 server.

While it is possible (by reiser4 loading the system less) that filesystem
affects network subsystem, it is highly unlikely and usually a sign of
problems elsewhere.

> It has maintained this speed for the past two
> years against the ext3 server even with aging hardware and bulking file
> and directory structures. (Both of the filesystems have slowed down at a
> similar pace for the duration of their lifetime [about 15ms].)

especially so if the difference lies in 5 ms range.

Re: Reiser4 Inclusion

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

On Sun, 16 Jul 2006 20:02:15 PDT, Caleb Gray said:
> Reiser4's responsiveness is undoubtedly at least twice as fast as ext3.
> I have deployed two nearly identical servers in Florida (I live in
> Washington state) but one difference: one uses ext3 and the other
> reiser4. The ping time of the reiser4 server is (on average) 20ms faster
> than the ext3 server.

OK, I'll bite.  What *POSSIBLE* reason is there for the choice of filesystem
to matter to an ICMP Echo Request/Reply?  I'm suspecting something else,
like the ext3 server needs to re-ARP before sending the Echo Reply, or some
such.

> and directory structures. (Both of the filesystems have slowed down at a
> similar pace for the duration of their lifetime [about 15ms].)

Unclear why *that* should matter to ICMP either.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4 Inclusion

[Nix]

On 17 Jul 2006, Valdis Kletnieks prattled cheerily:
> On Sun, 16 Jul 2006 20:02:15 PDT, Caleb Gray said:
>> and directory structures. (Both of the filesystems have slowed down at a
>> similar pace for the duration of their lifetime [about 15ms].)
> 
> Unclear why *that* should matter to ICMP either.

I bet his network topology changed, or the extra ARPs turned up later,
assuming he's still talking about ping time and not seek time.

-- 
`We're sysadmins. We deal with the inconceivable so often I can clearly 
 see the need to define levels of inconceivability.' --- Rik Steenwinkel