reiserfs on redhat advanced server?

reiserfs on redhat advanced server?

[Jure Pecar]

Hi all,

does anyone have some expirience with the reiserfs provided in redhat's
2.4.9-e.3 kernel from its Advanced Server?

Judging by the version number and the general lack of interest for
reiserfs redhat shows, it is not that well supported.
/proc/fs/reiserfs/version says it's a 3.6.25 and reiserfs-utils are
3.x.0j-3.

Now, what are the known issues with these versions? Anything critical? 
I would like to stay with that well tested kernel, if this version of
reiserfs wont eat my data (or cause other great pain).

This box (a pair of them, actually) will be used as a HA cyrus mailstore.
Both are connected to the same disks via fibrechannel. For a start they'll
serve 200k users from a 500gb raid10.

Thanks for suggestions,

--

Jure Pecar

Re: reiserfs on redhat advanced server?

[Vitaly Fertman]

On Thursday 30 January 2003 19:35, Jure Pecar wrote:
> Hi all,
>
> does anyone have some expirience with the reiserfs provided in redhat's
> 2.4.9-e.3 kernel from its Advanced Server?
>
> Judging by the version number and the general lack of interest for
> reiserfs redhat shows, it is not that well supported.
> /proc/fs/reiserfs/version says it's a 3.6.25 and reiserfs-utils are
> 3.x.0j-3.

Use the latest reiserfsprogs from ftp.namesys.com/pub/reiserfsprogs.

> Now, what are the known issues with these versions? Anything critical?
> I would like to stay with that well tested kernel, if this version of
> reiserfs wont eat my data (or cause other great pain).
>
> This box (a pair of them, actually) will be used as a HA cyrus mailstore.
> Both are connected to the same disks via fibrechannel. For a start they'll
> serve 200k users from a 500gb raid10.
>
> Thanks for suggestions,

-- 

Thanks,
Vitaly Fertman

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Jure Pecar wrote:

>Hi all,
>
>does anyone have some expirience with the reiserfs provided in redhat's
>2.4.9-e.3 kernel from its Advanced Server?
>
>Judging by the version number and the general lack of interest for
>reiserfs redhat shows, it is not that well supported.
>/proc/fs/reiserfs/version says it's a 3.6.25 and reiserfs-utils are
>3.x.0j-3.
>
>Now, what are the known issues with these versions? Anything critical? 
>I would like to stay with that well tested kernel, if this version of
>reiserfs wont eat my data (or cause other great pain).
>
>This box (a pair of them, actually) will be used as a HA cyrus mailstore.
>Both are connected to the same disks via fibrechannel. For a start they'll
>serve 200k users from a 500gb raid10.
>
>Thanks for suggestions,
>
>--
>
>Jure Pecar
>
>
>  
>
Linux 2.4.x did not become stable until 2.4.18.  I strongly advise not 
to use anything before 2.4.18 for a machine you care about.

Well tested is not a substitute for stable.;-)

-- 
Hans

Re: reiserfs on redhat advanced server?

[Ragnar Kjørstad]

On Thu, Jan 30, 2003 at 10:16:41PM +0300, Hans Reiser wrote:
> >does anyone have some expirience with the reiserfs provided in redhat's
> >2.4.9-e.3 kernel from its Advanced Server?
>
> Linux 2.4.x did not become stable until 2.4.18.  I strongly advise not 
> to use anything before 2.4.18 for a machine you care about.
> 
> Well tested is not a substitute for stable.;-)

I don't know this kernel in particular, but in general RH's kernels are
heavily patched, and 2.4.9-e.3 may actually be closer to 2.4.18 than
2.4.9 from Linus.



-- 
Ragnar Kjørstad

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Ragnar Kjørstad wrote:

>On Thu, Jan 30, 2003 at 10:16:41PM +0300, Hans Reiser wrote:
>  
>
>>>does anyone have some expirience with the reiserfs provided in redhat's
>>>2.4.9-e.3 kernel from its Advanced Server?
>>>      
>>>
>>Linux 2.4.x did not become stable until 2.4.18.  I strongly advise not 
>>to use anything before 2.4.18 for a machine you care about.
>>
>>Well tested is not a substitute for stable.;-)
>>    
>>
>
>I don't know this kernel in particular, but in general RH's kernels are
>heavily patched, and 2.4.9-e.3 may actually be closer to 2.4.18 than
>2.4.9 from Linus.
>
>
>
>  
>
Heavily patched plus RedHat famous QA process without millions of end 
users testing it like Linus/Marcelo has (err, without them testing it 
before it ships on the CD I should say....), and you think it will be 
more stable than 2.4.9?

If you want a stable kernel for a mission critical server, the only 
thing that works is to ask around and find out what Marcelo kernel has 
been known stable for at least 6 weeks.

QA can do a lot to avoid wasting the time of testers with  bugs they 
needn't have gone through the pain of hitting, but it can only do so much.

Marcelo is very responsible and effective in his management of the 
stable kernel series.  He is unusually talented.  RedHat doesn't have 
anyone in his league doing QA --- you could throw a dart in a room of 
professional release managers, and chances are low that you would not do 
better than RedHat did.  

I would never choose to use a RedHat kernel over a Marcelo kernel.

What I find really interesting is that RedHat marketing manages to make 
a lot of people think that a RedHat kernel is less adventurous than a 
Marcelo kernel, when it is very much the other way around.

-- 
Hans

Re: reiserfs on redhat advanced server?

[Lars Marowsky-Bree]

On 2003-01-31T14:39:29,
   Hans Reiser <reiser@namesys.com> said:

> If you want a stable kernel for a mission critical server, the only 
> thing that works is to ask around and find out what Marcelo kernel has 
> been known stable for at least 6 weeks.

And lose all support from RH or SuSE because you installed a non-supported
kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if RH
had one), etc pp.

Just something you can't do in some environments.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: reiserfs on redhat advanced server?

[Alexander Lyamin]

Fri, Jan 31, 2003 at 12:53:34PM +0100, Lars Marowsky-Bree wrote:
> On 2003-01-31T14:39:29,
>    Hans Reiser <reiser@namesys.com> said:
> 
> > If you want a stable kernel for a mission critical server, the only 
> > thing that works is to ask around and find out what Marcelo kernel has 
> > been known stable for at least 6 weeks.
> 
> And lose all support from RH or SuSE because you installed a non-supported
> kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if RH
> had one), etc pp.
> 
> Just something you can't do in some environments.

O.k. Its purely my private oppinion. and my own expirince talking here:

Cant say that i benefited from SuSE support even once :)
that's something you dont actually need on some expertise level.

Oracle support might be valuable(since they really do kernel-tweaking for this monster), but its not an issue there, right? Even if NO, i made Oracle running on standart issue kernel once. Probably with some perfomance loss, have'nt tested it yet.

The issue is high load (200k users) on this two puny workhorses.

My expirience and Corban Dallas telling me "If you have to something done - do it yourself".

2.4.19 without doubts.
reiserfsprogs version - lets ask Vitaly Fertman.


P.S.
most of tech.supports i bumped (even famous Cisco TAC) suck badly anyways, so who cares ? especially when you Open Source :)

> Sincerely,
>     Lars Marowsky-Br?e <lmb@suse.de>
-- 
"Cache remedies via multi-variable logic shorts will leave you crying."(cl)
Lex Lyamin

Re: reiserfs on redhat advanced server?

[Hans Reiser]

You are not a typical sysadmin, you work for an organization with ~10 
full-time kernel hackers, and the percentage of companies out there who 
can say that their average computer users are as skilled as any of 
SuSE's staff is not very high you know....

Most companies do indeed have a real need for tech support.  If the 
distros don't support official stable linux kernels, it is a serious 
social problem for our community, as it will substantially reduce the 
user base for those kernels in corporate environments.  It is especially 
a problem if the largest vendor does it.


Hans

Alexander Lyamin wrote:

>Fri, Jan 31, 2003 at 12:53:34PM +0100, Lars Marowsky-Bree wrote:
>  
>
>>On 2003-01-31T14:39:29,
>>   Hans Reiser <reiser@namesys.com> said:
>>
>>    
>>
>>>If you want a stable kernel for a mission critical server, the only 
>>>thing that works is to ask around and find out what Marcelo kernel has 
>>>been known stable for at least 6 weeks.
>>>      
>>>
>>And lose all support from RH or SuSE because you installed a non-supported
>>kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if RH
>>had one), etc pp.
>>
>>Just something you can't do in some environments.
>>    
>>
>
>O.k. Its purely my private oppinion. and my own expirince talking here:
>
>Cant say that i benefited from SuSE support even once :)
>that's something you dont actually need on some expertise level.
>
>Oracle support might be valuable(since they really do kernel-tweaking for this monster), but its not an issue there, right? Even if NO, i made Oracle running on standart issue kernel once. Probably with some perfomance loss, have'nt tested it yet.
>
>The issue is high load (200k users) on this two puny workhorses.
>
>My expirience and Corban Dallas telling me "If you have to something done - do it yourself".
>
>2.4.19 without doubts.
>reiserfsprogs version - lets ask Vitaly Fertman.
>
>
>P.S.
>most of tech.supports i bumped (even famous Cisco TAC) suck badly anyways, so who cares ? especially when you Open Source :)
>
>  
>
>>Sincerely,
>>    Lars Marowsky-Br?e <lmb@suse.de>
>>    
>>


-- 
Hans

Re: reiserfs on redhat advanced server?

[Lars Marowsky-Bree]

On 2003-01-31T15:20:55,
   Hans Reiser <reiser@namesys.com> said:

> Most companies do indeed have a real need for tech support.  If the 
> distros don't support official stable linux kernels, it is a serious 
> social problem for our community, as it will substantially reduce the 
> user base for those kernels in corporate environments.  It is especially 
> a problem if the largest vendor does it.

Distros can't support multiple trees; too much support load. If you run a
vanilla kernel, use the vanilla channels, most of the time. And notice how the
_community_ doesn't support the distro kernels either, strange isn't it? ;-)

Just the way it is: You ship it, you support it. You don't ship it, you don't
support it (at least not without higher fees).

(And the kernel hackers of the vendors _do_ support such kernels on LKML etc,
and you know that: Just not as part of the 'official' support, maintenance and
certification contracts)

And I don't see the social problem. I recall that a certain filesystem got
quite a bit of exposure and testing this way, and in the end was merged into
the vanilla kernel tree. What was it called again? 

So it seems to work out quite well in the end for both sides, even if it would
be great if we didn't have to; trust me, we would have better use for our
kernel hackers.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: reiserfs on redhat advanced server?

[Alexander Lyamin]

Fri, Jan 31, 2003 at 03:20:55PM +0300, Hans Reiser wrote:
> You are not a typical sysadmin, you work for an organization with ~10 
> full-time kernel hackers, and the percentage of companies out there who 
> can say that their average computer users are as skilled as any of 
> SuSE's staff is not very high you know....

Some linux-kernel hackers still do some stupid things userland :) sometimes...
Wont go in details.

> 
> Most companies do indeed have a real need for tech support.  If the 
> distros don't support official stable linux kernels, it is a serious 
> social problem for our community, as it will substantially reduce the 
> user base for those kernels in corporate environments.  It is especially 
> a problem if the largest vendor does it.
But thats quite a GOOD point. Its excellent point actually, worth broughting
it in LKML. Its a problem and its a solution. indeed.

All the idea of open-source and linux is that you could take a turn from
"railroad" anywhere you wont, and while its an effort theres ussually a benefits
which are worthy of effort. 

In propietary software if something gone wrong (and its ussually does when you
leaving "railroad") you left to reverce engeneering (effort*10, and DMCA wont like that) or merciless tech.support (merciless people quite often stupid also).

> Hans
> 
> Alexander Lyamin wrote:
> 
> >Fri, Jan 31, 2003 at 12:53:34PM +0100, Lars Marowsky-Bree wrote:
> > 
> >
> >>On 2003-01-31T14:39:29,
> >>  Hans Reiser <reiser@namesys.com> said:
> >>
> >>   
> >>
> >>>If you want a stable kernel for a mission critical server, the only 
> >>>thing that works is to ask around and find out what Marcelo kernel has 
> >>>been known stable for at least 6 weeks.
> >>>     
> >>>
> >>And lose all support from RH or SuSE because you installed a non-supported
> >>kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if 
> >>RH
> >>had one), etc pp.
> >>
> >>Just something you can't do in some environments.

-- 
"Cache remedies via multi-variable logic shorts will leave you crying."(cl)
Lex Lyamin

Re: reiserfs on redhat advanced server?

[Yury Umanets]

Alexander Lyamin wrote:

>Fri, Jan 31, 2003 at 12:53:34PM +0100, Lars Marowsky-Bree wrote:
>  
>
>>On 2003-01-31T14:39:29,
>>   Hans Reiser <reiser@namesys.com> said:
>>
>>    
>>
>>>If you want a stable kernel for a mission critical server, the only 
>>>thing that works is to ask around and find out what Marcelo kernel has 
>>>been known stable for at least 6 weeks.
>>>      
>>>
>>And lose all support from RH or SuSE because you installed a non-supported
>>kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if RH
>>had one), etc pp.
>>
>>Just something you can't do in some environments.
>>    
>>
>
>O.k. Its purely my private oppinion. and my own expirince talking here:
>
>Cant say that i benefited from SuSE support even once :)
>that's something you dont actually need on some expertise level.
>
>Oracle support might be valuable(since they really do kernel-tweaking for this monster), but its not an issue there, right? Even if NO, i made Oracle running on standart issue kernel once. Probably with some perfomance loss, have'nt tested it yet.
>
>The issue is high load (200k users) on this two puny workhorses.
>
>My expirience and Corban Dallas telling me "If you have to something done - do it yourself".
>
Actually it was saif not by Corban Dallas but rather by mr. Zorg. But, 
in general I'm agreed :)

>
>2.4.19 without doubts.
>reiserfsprogs version - lets ask Vitaly Fertman.
>
>
>P.S.
>most of tech.supports i bumped (even famous Cisco TAC) suck badly anyways, so who cares ? especially when you Open Source :)
>
>  
>
>>Sincerely,
>>    Lars Marowsky-Br?e <lmb@suse.de>
>>    
>>

Re: reiserfs on redhat advanced server?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

On Fri, 31 Jan 2003 15:08:52 +0300, Alexander Lyamin said:

> Cant say that i benefited from SuSE support even once :)
> that's something you dont actually need on some expertise level.

Some sites have this really silly notion that if you're running a production
workload on something, that *everything* be covered by a service/maintenance
contract, Just In Case.

> My expirience and Corban Dallas telling me "If you have to something done - d
o it yourself".

See above - does the CIO want to bet revenue on the resident kernel wizard
being able to find the bug themselves?  Remember there's a planning issue here
as well - if the CIO has a support contract with RedHat, he can be fairly
sure that 9 months from now, if he calls RedHat somebody will answer the
phone.  The resident kernel wizard can always get a better job offer and be
gone 9 months from now.

*YOU* may know enough to get Oracle to run well on a stock kernel.  But your
boss wants to know *for sure* that if you leave, that your 200K users
won't be left unsupported....
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Lars Marowsky-Bree wrote:

>On 2003-01-31T14:39:29,
>   Hans Reiser <reiser@namesys.com> said:
>
>  
>
>>If you want a stable kernel for a mission critical server, the only 
>>thing that works is to ask around and find out what Marcelo kernel has 
>>been known stable for at least 6 weeks.
>>    
>>
>
>And lose all support from RH or SuSE because you installed a non-supported
>kernel image ;-) And thus violate the Oracle cert, the SAP cert (well, if RH
>had one), etc pp.
>
>Just something you can't do in some environments.
>
>
>Sincerely,
>    Lars Marowsky-Brée <lmb@suse.de>
>
>  
>
The distros are really doing their best to undermine the open source 
community testing process that works so well.... 

for them to not certify/support a stable Marcelo kernel is.... well it 
makes me angry quite honestly....

-- 
Hans

Re: reiserfs on redhat advanced server?

[Lars Marowsky-Bree]

On 2003-01-31T15:10:22,
   Hans Reiser <reiser@namesys.com> said:

> The distros are really doing their best to undermine the open source 
> community testing process that works so well.... 
> 
> for them to not certify/support a stable Marcelo kernel is.... well it 
> makes me angry quite honestly....

Tell that to the ISVs. And the plain kernel, when it comes out, is usually
simply not really useable as a distro kernel until, oh, dot twenty something
or so, for a variety of reasons.

From "buzzwords" like 'enterprise features missing' or missing support for
hardware to different schedulers, accomodations for IHVs/ISVs which mainline
doesn't want (yet) etc. The way it works seems to be that distros test drive a
lot of code before it gets merged into mainline, at the expense of mainline
being slightly behind quite often.

And at the time when distributions hit their respective deadlines, even a
bunch of just plain fixes will have already piled up for the last stable
vanilla kernel, so they get thrown in too, mostly from "pre" kernels.

I'm not saying this is good or bad, but it is just they way it is. I can't
really see a distribution shipping a plain, unpatched kernel. Trust me. Every
distributor would love too, because it would lower cost for them. You know how
much highly qualified _work_ is needed to maintain a distribution kernel?
Nobody does that out of their own free will.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: reiserfs on redhat advanced server?

[Hans Reiser]

I understand and support being pissed at Linus for calling it 2.4.0 when 
it wasn't stable enough before 2.4.18 because VM and VFS were still 
being changed, but Marcelo is pretty stable in all of his official 
releases, and it is easy to get him to take good code.

Lars Marowsky-Bree wrote:

>On 2003-01-31T15:10:22,
>   Hans Reiser <reiser@namesys.com> said:
>
>  
>
>>The distros are really doing their best to undermine the open source 
>>community testing process that works so well.... 
>>
>>for them to not certify/support a stable Marcelo kernel is.... well it 
>>makes me angry quite honestly....
>>    
>>
>
>Tell that to the ISVs. And the plain kernel, when it comes out, is usually
>simply not really useable as a distro kernel until, oh, dot twenty something
>or so, for a variety of reasons.
>
>>From "buzzwords" like 'enterprise features missing' or missing support for
>hardware to different schedulers, accomodations for IHVs/ISVs which mainline
>doesn't want (yet) etc. The way it works seems to be that distros test drive a
>lot of code before it gets merged into mainline, at the expense of mainline
>being slightly behind quite often.
>
So it should be the mainline that is certified....

>
>And at the time when distributions hit their respective deadlines, even a
>bunch of just plain fixes will have already piled up for the last stable
>vanilla kernel, so they get thrown in too, mostly from "pre" kernels.
>
This is legitimate (or at least it is for SuSE because they seem to be 
able to do it well).

>
>I'm not saying this is good or bad, but it is just they way it is. I can't
>really see a distribution shipping a plain, unpatched kernel. Trust me. Every
>distributor would love too, because it would lower cost for them. You know how
>much highly qualified _work_ is needed to maintain a distribution kernel?
>Nobody does that out of their own free will.
>
>
>Sincerely,
>    Lars Marowsky-Brée <lmb@suse.de>
>
>  
>

I am not really opposed to vendors shipping their own kernels and 
supporting them, but I am opposed to them not supporting an official 
stable Marcelo kernel unless they have a specific reason not to.  The 
Marcelo kernels need to be considered the official supported ones by the 
entire community, regardless of what other ones might also be supported 
by parts of the community.

-- 
Hans

Re: reiserfs on redhat advanced server?

[Lars Marowsky-Bree]

On 2003-01-31T15:35:02,
   Hans Reiser <reiser@namesys.com> said:

> I understand and support being pissed at Linus for calling it 2.4.0 when 
> it wasn't stable enough before 2.4.18 because VM and VFS were still 
> being changed, but Marcelo is pretty stable in all of his official 
> releases, and it is easy to get him to take good code.

No wonder, Marcelo came 18 minors later. Those kind of release cycles would be
quite inacceptable.

> >>From "buzzwords" like 'enterprise features missing' or missing support
> >>for
> >hardware to different schedulers, accomodations for IHVs/ISVs which
> >mainline doesn't want (yet) etc. The way it works seems to be that distros
> >test drive a lot of code before it gets merged into mainline, at the
> >expense of mainline being slightly behind quite often.
> So it should be the mainline that is certified....

Woot? Mainline tends to crash under Oracle or SAP, for example, lacks drivers
etc. Yeah, this is a problem, but don't bitch at the distros about it, but at
the driver developers etc.

> I am not really opposed to vendors shipping their own kernels and 
> supporting them, but I am opposed to them not supporting an official 
> stable Marcelo kernel unless they have a specific reason not to. 

This is a pointless discussion. The specific argument is "Support cost". End
of discussion.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: reiserfs on redhat advanced server?

[Oleg Drokin]

Hello!

On Fri, Jan 31, 2003 at 01:39:43PM +0100, Lars Marowsky-Bree wrote:

> Woot? Mainline tends to crash under Oracle or SAP, for example, lacks drivers
> etc. Yeah, this is a problem, but don't bitch at the distros about it, but at
> the driver developers etc.

Distro kernels tend to crash in unusual places, too.
E.g. SuSE 8.1 default kernel breaking under memory pressure.
RedHat 8.0 default kernel with slow block devices and ext3 bugs.
And so on.

> > I am not really opposed to vendors shipping their own kernels and 
> > supporting them, but I am opposed to them not supporting an official 
> > stable Marcelo kernel unless they have a specific reason not to. 
> This is a pointless discussion. The specific argument is "Support cost". End
> of discussion.

That's true.
And these days bosses tend to save on people and buy cheap support from vendors instead.

Bye,
    Oleg

Re: reiserfs on redhat advanced server?

[Chris Mason]

On Fri, 2003-01-31 at 08:06, Oleg Drokin wrote:
> Hello!
> 
> On Fri, Jan 31, 2003 at 01:39:43PM +0100, Lars Marowsky-Bree wrote:
> 
> > Woot? Mainline tends to crash under Oracle or SAP, for example, lacks drivers
> > etc. Yeah, this is a problem, but don't bitch at the distros about it, but at
> > the driver developers etc.
> 
> Distro kernels tend to crash in unusual places, too.
> E.g. SuSE 8.1 default kernel breaking under memory pressure.
> RedHat 8.0 default kernel with slow block devices and ext3 bugs.
> And so on.

The simple truth is that all kernels have bugs.  In general, I believe
the distro kernels are more suitable for production use than a vanilla
kernel.  These companies are betting their support time and future
revenue on their kernels working properly, and the distros take that
very seriously.

The fact that nobody actually ships a vanilla kernel should make it
pretty clear the stock releases are not ready for the tasks our
customers need them for.  The fact that our customers pay us to add
patches instead of using the stock kernel for free is also important.

This doesn't make it a good idea to run rhas 2.4.9 based kernel in
production with reiserfs, they were not horribly focused on reiserfs,
especially when rhas came out.

-chris

Re: reiserfs on redhat advanced server?

[Russell Coker]

On Fri, 31 Jan 2003 14:55, Chris Mason wrote:
> The fact that nobody actually ships a vanilla kernel should make it
> pretty clear the stock releases are not ready for the tasks our
> customers need them for.  The fact that our customers pay us to add
> patches instead of using the stock kernel for free is also important.

The fact that the patched kernels shipped by distributions appear to have 
almost totally disjoint sets of patches is also noteworthy...

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Re: reiserfs on redhat advanced server?

[Chris Mason]

On Fri, 2003-01-31 at 08:58, Russell Coker wrote:
> On Fri, 31 Jan 2003 14:55, Chris Mason wrote:
> > The fact that nobody actually ships a vanilla kernel should make it
> > pretty clear the stock releases are not ready for the tasks our
> > customers need them for.  The fact that our customers pay us to add
> > patches instead of using the stock kernel for free is also important.
> 
> The fact that the patched kernels shipped by distributions appear to have 
> almost totally disjoint sets of patches is also noteworthy...

It certainly is.  Different companies focus on different things, both
based on their expertise and which customer/partner manages to influence
them during the feature planning before a release.  

In the long term, the competition between us is good for linux, in the
short term, we duplicate effort much more than we should.

-chris

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Chris Mason wrote:

>  
>
>In the long term, the competition between us is good for linux, in the
>short term, we duplicate effort much more than we should.
>
>-chris
>
>
>
>  
>
This is a traditional capitalism vs. communism tradeoff.

-- 
Hans

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Russell Coker wrote:

>On Fri, 31 Jan 2003 14:55, Chris Mason wrote:
>  
>
>>The fact that nobody actually ships a vanilla kernel should make it
>>pretty clear the stock releases are not ready for the tasks our
>>customers need them for.  The fact that our customers pay us to add
>>patches instead of using the stock kernel for free is also important.
>>    
>>
>
>The fact that the patched kernels shipped by distributions appear to have 
>almost totally disjoint sets of patches is also noteworthy...
>
>  
>
and truly sad....  politics determines code...  and neither side is 
especially to blame for it --- it is simply herd sociology in action....

Though I must remember not to say this on lkml, the last time I 
suggested that herd sociology determined code acceptance, well....

-- 
Hans

Re: reiserfs on redhat advanced server?

[Russell Coker]

On Fri, 31 Jan 2003 15:20, Hans Reiser wrote:
> >The fact that the patched kernels shipped by distributions appear to have
> >almost totally disjoint sets of patches is also noteworthy...
>
> and truly sad....  politics determines code...  and neither side is
> especially to blame for it --- it is simply herd sociology in action....

There are plenty of good reasons for having different sets of patches as Chris 
pointed out.  Sometimes patches conflict to a degree that it's impossible to 
merge them, and you have to make a choice of which patch you want.  Sometimes 
patch conflict enough that you could merge them if you spent a lot of time 
working on it and you have to decide whether this time (and money) is 
justified.

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Re: reiserfs on redhat advanced server?

[Oleg Drokin]

Hello!

On Fri, Jan 31, 2003 at 08:55:11AM -0500, Chris Mason wrote:
> > > Woot? Mainline tends to crash under Oracle or SAP, for example, lacks drivers
> > > etc. Yeah, this is a problem, but don't bitch at the distros about it, but at
> > > the driver developers etc.
> > Distro kernels tend to crash in unusual places, too.
> > E.g. SuSE 8.1 default kernel breaking under memory pressure.
> > RedHat 8.0 default kernel with slow block devices and ext3 bugs.
> > And so on.
> The simple truth is that all kernels have bugs.  In general, I believe

That's true.

> the distro kernels are more suitable for production use than a vanilla
> kernel.  These companies are betting their support time and future

This is questionable. Distro's QA cannot just test all possible patterns.
So there are some kinds of usage (even in production) where vanilla
kernel is the best thing to use (at least until vendor releases an errata
kernel).

> revenue on their kernels working properly, and the distros take that
> very seriously.

That's true.

> The fact that nobody actually ships a vanilla kernel should make it
> pretty clear the stock releases are not ready for the tasks our

Well, I provided an example of people who ship vanilla kernels.

> customers need them for.  The fact that our customers pay us to add
> patches instead of using the stock kernel for free is also important.

That's true.

> This doesn't make it a good idea to run rhas 2.4.9 based kernel in
> production with reiserfs, they were not horribly focused on reiserfs,
> especially when rhas came out.

Actually, I remember I once reviewed a patchset for Red Hat's
kernel-2.4.x-yy.src.rpm, and it contained patch-2.4.18-acZZ.bz2,
that somehow suggests they had it up to 2.4.18 at least ;)
But of course no warrany on that can be gived for some random kernel
until all of patches applied to its vanilla counterpart were reviewed.

(and, of course, RedHat is actually cares much more about ext3.)

Bye,
    Oleg

Re: reiserfs on redhat advanced server?

[Chris Mason]

On Fri, 2003-01-31 at 09:08, Oleg Drokin wrote:
> Hello!
> 
> On Fri, Jan 31, 2003 at 08:55:11AM -0500, Chris Mason wrote:

> > the distro kernels are more suitable for production use than a vanilla
> > kernel.  These companies are betting their support time and future
> 
> This is questionable. Distro's QA cannot just test all possible patterns.
> So there are some kinds of usage (even in production) where vanilla
> kernel is the best thing to use (at least until vendor releases an errata
> kernel).
> 

What you end up with is a vendor kernel that has been specifically
tested in a given workload, and they tend to work very well in that
config.  The vanilla kernels do have a broader testing base, and they do
get a wider variety of bug fixes.  But they don't get the concentrated
high end certification and testing the vendor kernels do.  And that last
little bit of certification is _really_ hard.

> > revenue on their kernels working properly, and the distros take that
> > very seriously.
> 
> That's true.
> 
> > The fact that nobody actually ships a vanilla kernel should make it
> > pretty clear the stock releases are not ready for the tasks our
> 
> Well, I provided an example of people who ship vanilla kernels.
> 

But not a recent example of someone using them in a high end
configuration.  It's even possible that someone has, but doing it over
time for release after release?  Those are the customers that end up
going to redhat and suse.

-chris

Re: reiserfs on redhat advanced server?

[Brian Tinsley]

Chris Mason wrote:

>But not a recent example of someone using them in a high end
>configuration.  It's even possible that someone has, but doing it over
>time for release after release?  Those are the customers that end up
>going to redhat and suse.
>
>-chris
>
Here's a good case study on this topic:

We started out a couple of years ago with a RedHat distro that we found 
a problem with and could not get a fix for *in any timely manner*, so I 
started using vanilla 2.4.17 kernel with the patch I required. Over 
time, we started seeing numerous other subtle problems that drove our 
service guys and customers crazy because they were difficult to pinpoint 
and caused a lot of down time. So I would eventually track down the 
problem, find a patch, and redeploy a new kernel. I am neither a sys 
admin (in this job) nor a kernel maintainer, so I got sick of this cycle 
quickly. By this time, a new Red Hat distro rolled out that I tested and 
verified that it addressed all of our previous problems. Frankly, I was 
elated, and we brought most of our existing systems up to this new level 
quickly and started loading it on new systems. Well, it didn't take very 
long before I found myself right back in the same boat (those that use 
the linux-ha heartbeat software know what I'm talking about here!). 
AFAIK, this problem is *still* not fixed and a precise bug report was 
posted many months ago. So, here we go to vanilla kernels 2.4.19, and 
soon thereafter, 2.4.20. Guess what happens now? Linux VM problems, 
really *stupid* ones in my opinion, start hitting a select few of our 
systems; of course they are the most critical ones! Thankfully, I 
happened to notice a couple of postings on LKML of others having 
seemingly the same problem, so I start talking to them. It didn't take 
long before one of the VM developers got involved and worked one-on-one 
with me to analyze my problem and within 2 days, I had it fixed (for 
anyone interested, the 2.4.20-aa1 kernel was the silver bullet). So, now 
I spend endless hours in the lab testing this kernel and eventually many 
more hours overseeing the upgrade of numerous sites. Guess what happens 
next? It's good news this time! Not a *single* (kernel based) problem 
from any of the sites running this new kernel! The downside is that I am 
once again exhausted by having to maintain the kernel. So now I've 
extended the opportunity to SuSE to try and make my life easier (but 
haven't heard back from them yet, Lars ;) ). What will happen next, I 
dare not even guess, but having worked on some level with a few people 
at SuSE and being educated about their products offerings recently, I 
have to say that I am confident things will work out.


-- 

Brian Tinsley
Chief Systems Engineer
Emageon

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Chris Mason wrote:

>On Fri, 2003-01-31 at 08:06, Oleg Drokin wrote:
>  
>
>>Hello!
>>
>>On Fri, Jan 31, 2003 at 01:39:43PM +0100, Lars Marowsky-Bree wrote:
>>
>>    
>>
>>>Woot? Mainline tends to crash under Oracle or SAP, for example, lacks drivers
>>>etc. Yeah, this is a problem, but don't bitch at the distros about it, but at
>>>the driver developers etc.
>>>      
>>>
>>Distro kernels tend to crash in unusual places, too.
>>E.g. SuSE 8.1 default kernel breaking under memory pressure.
>>RedHat 8.0 default kernel with slow block devices and ext3 bugs.
>>And so on.
>>    
>>
>
>The simple truth is that all kernels have bugs. 
>
Yes, true.

> In general, I believe
>the distro kernels are more suitable for production use than a vanilla
>kernel.  These companies are betting their support time and future
>revenue on their kernels working properly, and the distros take that
>very seriously.
>
and Marcelo does not have his reputation bet on it?

>
>The fact that nobody actually ships a vanilla kernel should make it
>pretty clear the stock releases are not ready for the tasks our
>customers need them for.  
>
This is not clear to me at all.

>The fact that our customers pay us to add
>patches instead of using the stock kernel for free is also important.
>
There is a difference between using a distro kernel because it has some 
missing feature (desiring untested by the masses features is 
reasonable), and using it because it is more stable (thinking features 
untested by the masses for 6 weeks are stable is unreasonable).

I think our developer community is slowly breaking down into squabbling 
sub-herds....

it is sad to watch....

I think the difficulty of getting good patches into the kernel 
accentuates the trend.

I think most users would be happier if the patches went straight to 
Marcelo rather than through a distro.

I think that most users are better off with a recent stable kernel with 
a good word of mouth than blindly using what the distro ships.  I think 
that most users don't have the time to be bothered with downloading and 
compiling a kernel, but those that do should feel confident about doing 
so.  We probably need a better system of rating and labeling kernels 6 
weeks after they have shipped.

-- 
Hans

Re: reiserfs on redhat advanced server?

[Ookhoi]

Hi Guys and Girls (?),

I think I'm lost. Can somebody point me to the reiserfs mailinglist?

Re: reiserfs on redhat advanced server?

[Chris Mason]

On Fri, 2003-01-31 at 09:15, Hans Reiser wrote:

> > In general, I believe
> >the distro kernels are more suitable for production use than a vanilla
> >kernel.  These companies are betting their support time and future
> >revenue on their kernels working properly, and the distros take that
> >very seriously.
> >
> and Marcelo does not have his reputation bet on it?
> 

Not in the same way.  If there's a bug in 2.4.20, he fixes it in
2.4.21-preX, he might even release 2.4.21 right away with just a single
fix.  If there's a bug in a vendor kernel, it is burnt forever onto a
cd, and we have to support everyone afflicted by it.

When the vendor screws up, they risk their revenue and the paychecks of
every employee.

Clearly we won't convince each other ;-)  The cool part about linux is
that we don't have to.

-chris

Re: reiserfs on redhat advanced server?

[Russell Coker]

On Fri, 31 Jan 2003 15:15, Hans Reiser wrote:
> >The fact that nobody actually ships a vanilla kernel should make it
> >pretty clear the stock releases are not ready for the tasks our
> >customers need them for.
>
> This is not clear to me at all.

I agree.  I have fairly vanilla 2.4.20 kernels running high performance mail 
servers and other tasks on ReiserFS.  The only patches I have are LSM and a 
patch one of the ReiserFS developers pointed me to for performance.

The ReiserFS patch made a significant performance difference which is required 
for my use (the servers would not be able to function properly under peak 
load without it).  However with the possible exception of SUSE no 
distribution supports it.

The only thing I am missing which is in a distro kernel is the aa1 fix for the 
kswapd issue we discussed here (and I decided that it's not enough of a 
problem at the moment to justify replacing kernels on such important machines 
with kernels that have a 300,000 line patch).

> >The fact that our customers pay us to add
> >patches instead of using the stock kernel for free is also important.
>
> There is a difference between using a distro kernel because it has some
> missing feature (desiring untested by the masses features is
> reasonable), and using it because it is more stable (thinking features
> untested by the masses for 6 weeks are stable is unreasonable).

I am more likely to use the aa1 patch because of SUSE shipping it.  The fact 
that the SUSE customers are not complaining suggests that it's good.

Also a problem I have is that I am often forced to upgrade sooner than the 
distro's support.  By the time a distro supports a new kernel then I've 
usually already put it through such a burn-in that having the distro-tested 
one provides little extra benefit.

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Re: reiserfs on redhat advanced server?

[Chris Mason]

On Fri, 2003-01-31 at 10:00, Russell Coker wrote:
> On Fri, 31 Jan 2003 15:15, Hans Reiser wrote:
> > >The fact that nobody actually ships a vanilla kernel should make it
> > >pretty clear the stock releases are not ready for the tasks our
> > >customers need them for.
> >
> > This is not clear to me at all.
> 
> I agree.  I have fairly vanilla 2.4.20 kernels running high performance mail 
> servers and other tasks on ReiserFS.  The only patches I have are LSM and a 
> patch one of the ReiserFS developers pointed me to for performance.
> 

A highly qualified admin can go out and find the small number of patches
their configuration really needs, and this will always be smaller than
the list of patches a distro includes (BTW, I thought you also needed
block-highmem?)

Most people don't have the time or energy to manage that however.

-chris

Re: reiserfs on redhat advanced server?

[Russell Coker]

On Fri, 31 Jan 2003 16:22, Chris Mason wrote:
> > I agree.  I have fairly vanilla 2.4.20 kernels running high performance
> > mail servers and other tasks on ReiserFS.  The only patches I have are
> > LSM and a patch one of the ReiserFS developers pointed me to for
> > performance.
>
> A highly qualified admin can go out and find the small number of patches
> their configuration really needs, and this will always be smaller than
> the list of patches a distro includes (BTW, I thought you also needed
> block-highmem?)

I could do with a patch for kswapd, that's all I really need.  But that isn't 
a major issue as performance dies before kswapd becomes an issue anyway.

> Most people don't have the time or energy to manage that however.

True.  However it must be a PITA for them if they are using SUSE for example 
but the RH kernel has the patches they need...

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

Re: reiserfs on redhat advanced server?

[Dieter Nützel]

Am Freitag, 31. Januar 2003 15:15 schrieb Hans Reiser:
> Chris Mason wrote:

> >The fact that nobody actually ships a vanilla kernel should make it
> >pretty clear the stock releases are not ready for the tasks our
> >customers need them for.
>
> This is not clear to me at all.

Only two little examples (VM not ReiserFS ;-):

Oracle and SAP need -aa VM.
Some pieces missing for ages in the stock kernel.

-ac (base for RH?) has the O(1)-scheduler (-aa now, too) and low-lat.

Nothing in stock.

Peace,
	Dieter

PS Go on with your great ReiserFS 3.x (when will we see data-logging in stock) 
and Reiser4 work.
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)

Re: reiserfs on redhat advanced server?

[Hans Reiser]

Dieter Nützel wrote:

>Am Freitag, 31. Januar 2003 15:15 schrieb Hans Reiser:
>  
>
>>Chris Mason wrote:
>>    
>>
>
>  
>
>>>The fact that nobody actually ships a vanilla kernel should make it
>>>pretty clear the stock releases are not ready for the tasks our
>>>customers need them for.
>>>      
>>>
>>This is not clear to me at all.
>>    
>>
>
>Only two little examples (VM not ReiserFS ;-):
>
>Oracle and SAP need -aa VM.
>Some pieces missing for ages in the stock kernel.
>
>-ac (base for RH?) has the O(1)-scheduler (-aa now, too) and low-lat.
>
>Nothing in stock.
>
>Peace,
>	Dieter
>
>PS Go on with your great ReiserFS 3.x (when will we see data-logging in stock) 
>
Chris is working on that now.

>and Reiser4 work.
>  
>
We are starting to hit the harder bugs now.... sigh....

but we have fixed the performance loss we had in August.... :)

Best,

-- 
Hans

Re: reiserfs on redhat advanced server?

[Oleg Drokin]

Hello!

On Fri, Jan 31, 2003 at 03:35:02PM +0300, Hans Reiser wrote:
> I understand and support being pissed at Linus for calling it 2.4.0 when 
> it wasn't stable enough before 2.4.18 because VM and VFS were still 
> being changed, but Marcelo is pretty stable in all of his official 
> releases, and it is easy to get him to take good code.

Unfortunatelly vanilla 2.4.20 is pretty bad shaped. (and pretty bad mouthed
for that, too).

Bye,
    Oleg

Re: reiserfs on redhat advanced server?

[Alexander Lyamin]

Fri, Jan 31, 2003 at 03:40:18PM +0300, Oleg Drokin wrote:
> Hello!
> 
> On Fri, Jan 31, 2003 at 03:35:02PM +0300, Hans Reiser wrote:
> > I understand and support being pissed at Linus for calling it 2.4.0 when 
> > it wasn't stable enough before 2.4.18 because VM and VFS were still 
> > being changed, but Marcelo is pretty stable in all of his official 
> > releases, and it is easy to get him to take good code.
> 
> Unfortunatelly vanilla 2.4.20 is pretty bad shaped. (and pretty bad mouthed
> for that, too).
Yep.
Would'nt use it.
Bad expirience.

> 
> Bye,
>     Oleg

-- 
"Cache remedies via multi-variable logic shorts will leave you crying."(cl)
Lex Lyamin

Re: reiserfs on redhat advanced server?

[Juan Quintela]

>>>>> "hans" == Hans Reiser <reiser@namesys.com> writes:

hans> I understand and support being pissed at Linus for calling it 2.4.0
hans> when it wasn't stable enough before 2.4.18 because VM and VFS were
hans> still being changed, but Marcelo is pretty stable in all of his
hans> official releases, and it is easy to get him to take good code.

But it is not possible to get Marcelo to adapt his release schedule to
the distro's release schedule :p That is one of the BIG problems.  If
when your release is about to freeze, marcelo kernel is in pre5/pre6
what do you do:

- bet that final kernel will be there by the end of the distro release
  and switch.  And in the proccess, invalidate all the testing that
  you have done so far.

- get the old known stable kernel, and adapt all the bugfixes that you
  found in the pre series?

What strategy do you think that is better?  If you bet (as almost
everybody) that second one is better, you are going to have a heavily
patched kernel.

And that is without taking into account that a lot of the bug fixes
that go to marcelo kernel go the route:

- user find bug
- user blame distro kernel
- distro kernel team found the problem (sometimes with cooperation
  with the subsystem maintainer)
- distro kernel team send the patch to subsystem maintainer
- subsytem maintainer send the patch to marcelo (perhaps after some
  local modification)

hans> I am not really opposed to vendors shipping their own kernels and
hans> supporting them, but I am opposed to them not supporting an official
hans> stable Marcelo kernel unless they have a specific reason not to.  The
hans> Marcelo kernels need to be considered the official supported ones by
hans> the entire community, regardless of what other ones might also be
hans> supported by parts of the community.

Believe me, if it will be possible (not indeed easy) to get that done,
my life will be much, much better :p

Later, Juan.

-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy

when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Juan Quintela wrote:

>>>>>>"hans" == Hans Reiser <reiser@namesys.com> writes:
>>>>>>            
>>>>>>
>
>hans> I understand and support being pissed at Linus for calling it 2.4.0
>hans> when it wasn't stable enough before 2.4.18 because VM and VFS were
>hans> still being changed, but Marcelo is pretty stable in all of his
>hans> official releases, and it is easy to get him to take good code.
>
>But it is not possible to get Marcelo to adapt his release schedule to
>the distro's release schedule :p That is one of the BIG problems.  If
>when your release is about to freeze, marcelo kernel is in pre5/pre6
>what do you do:
>
>- bet that final kernel will be there by the end of the distro release
>  and switch.  And in the proccess, invalidate all the testing that
>  you have done so far.
>
>- get the old known stable kernel, and adapt all the bugfixes that you
>  found in the pre series?
>
This is reasonable, and I am not complaining about it. 

It is different from refusing to support the user who downloads 
Marcelo's kernel after it does ship (after the distro CD went into the 
stamping plant). That is what I am complaining about.  The default 
should be to support all Marcelo kernels unless there is a motivated 
reason not to (e.g. he ships a broken NFS kernel and the user is 
complaining about NFS).  Users should feel that they can download any 
latest official stable kernel (it is okay though to tell them to check a 
website created by the distro to see if it is a known bad/unsupported 
kernel), and everything will be fine with the distro.  When distros 
don't do this, they are not being team players.

>
>What strategy do you think that is better?  If you bet (as almost
>everybody) that second one is better, you are going to have a heavily
>patched kernel.
>
>And that is without taking into account that a lot of the bug fixes
>that go to marcelo kernel go the route:
>
>- user find bug
>- user blame distro kernel
>- distro kernel team found the problem (sometimes with cooperation
>  with the subsystem maintainer)
>
I don't see as many ReiserFS bugs found/fixed by distro kernel teams 
responding to complaints by their users as I would expect.  Perhaps we 
are unusual, I lack the perspective to know.  I would like to see more 
of them, and I don't really understand the lack of them as I would 
expect to see more.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Chris Mason]

On Mon, 2003-02-03 at 09:20, Hans Reiser wrote:

> It is different from refusing to support the user who downloads 
> Marcelo's kernel after it does ship (after the distro CD went into the 
> stamping plant). That is what I am complaining about.  The default 
> should be to support all Marcelo kernels unless there is a motivated 
> reason not to (e.g. he ships a broken NFS kernel and the user is 
> complaining about NFS).  Users should feel that they can download any 
> latest official stable kernel (it is okay though to tell them to check a 
> website created by the distro to see if it is a known bad/unsupported 
> kernel), and everything will be fine with the distro.  When distros 
> don't do this, they are not being team players.

Hans, the vanilla kernels are lacking both bug fixes and features that
are critical to what our users are doing.  Even if the bug fixes all got
in, there are various reasons the features probably won't.  

If there was any vanilla kernel that had everything we needed, we'd
support it, and do a dance around a bonfire made from all of our patch
maintenance scripts and code.

The whole point of buying the distro is that you don't have the time and
energy to collect and compile every application and turn it into
something you can easily install on your personal machine.  The kernel
is one of those applications.  Feel free to replace it, but it doesn't 
make sense to expect us to help you fix the problems when we don't have
control over the configuration, compile or sources.

That would be like switching engines in your car and expecting the
original car company to do a warranty repair on the new engine.

-chris (speaking only for himself and not SuSE)

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

I think it is much more like switching the engine in the car and 
expecting the car company to honor the warranty on the air conditioner, 
or expecting that if I install linux on my dell laptop and the hard 
drive goes bust that the hard drive will be replaced under warranty.  
You do remember that in the US the car companies tried to lock out 
after-market parts suppliers by using the warranty as leverage, yes?  
Fortunately in the car case that was before the courts were packed with 
judges who hate the anti-trust laws by our lovely elected 
representatives, and so it was found to be illegal.  Since we no longer 
have anti-trust laws in America, social pressure is the appropriate 
response in the case of Linux.

I hope you are not saying that recompiling the kernel is more deserving 
of voiding a warranty than, say, creating a custom .bashrc file?

Hans

Chris Mason wrote:

>On Mon, 2003-02-03 at 09:20, Hans Reiser wrote:
>
>  
>
>>It is different from refusing to support the user who downloads 
>>Marcelo's kernel after it does ship (after the distro CD went into the 
>>stamping plant). That is what I am complaining about.  The default 
>>should be to support all Marcelo kernels unless there is a motivated 
>>reason not to (e.g. he ships a broken NFS kernel and the user is 
>>complaining about NFS).  Users should feel that they can download any 
>>latest official stable kernel (it is okay though to tell them to check a 
>>website created by the distro to see if it is a known bad/unsupported 
>>kernel), and everything will be fine with the distro.  When distros 
>>don't do this, they are not being team players.
>>    
>>
>
>Hans, the vanilla kernels are lacking both bug fixes and features that
>are critical to what our users are doing.  Even if the bug fixes all got
>in, there are various reasons the features probably won't.  
>
>If there was any vanilla kernel that had everything we needed, we'd
>support it, and do a dance around a bonfire made from all of our patch
>maintenance scripts and code.
>
>The whole point of buying the distro is that you don't have the time and
>energy to collect and compile every application and turn it into
>something you can easily install on your personal machine.  The kernel
>is one of those applications.  Feel free to replace it, but it doesn't 
>make sense to expect us to help you fix the problems when we don't have
>control over the configuration, compile or sources.
>
>That would be like switching engines in your car and expecting the
>original car company to do a warranty repair on the new engine.
>
>-chris (speaking only for himself and not SuSE)
>
>
>
>
>  
>


-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

On Mon, 03 Feb 2003 21:51:43 +0300, Hans Reiser said:
> I think it is much more like switching the engine in the car and 
> expecting the car company to honor the warranty on the air conditioner, 

On many cars, the air conditioner is powered by a belt that is driven by
the engine, and the air conditioner is assuming that the engine will be
within some range of RPM and torque provided.  If the swapped engine doesn't
stay within those ranges, the air conditioner will fail.

The kernel equivalent is somebody changing an API and not fixing all callers.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Valdis.Kletnieks@vt.edu wrote:

>On Mon, 03 Feb 2003 21:51:43 +0300, Hans Reiser said:
>  
>
>>I think it is much more like switching the engine in the car and 
>>expecting the car company to honor the warranty on the air conditioner, 
>>    
>>
>
>On many cars, the air conditioner is powered by a belt that is driven by
>the engine, and the air conditioner is assuming that the engine will be
>within some range of RPM and torque provided.  If the swapped engine doesn't
>stay within those ranges, the air conditioner will fail.
>
Yes, this was the car manufacturer argument.  Funny how everybody 
somehow manages to cope with all those aftermarket carburetors and 
non-standard batteries bought from unknown sources like Sears, Kragen, 
and Wal-Mart despite all of that.  The world is much better because some 
judge of my father's generation ensured that for a car manufacturer to 
void a warranty they have to have a motivated reason for doing so.  Now 
with people wanting to substitute the Linux Kernel for the RedHat 
Kernel, a new generation of technologists faces an old question. 

>
>The kernel equivalent is somebody changing an API and not fixing all callers.
>  
>
Motivated failure to support is correct.  Failing to support because it 
was not manufactured by the dominant market force is evil.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Chris Mason]

On Mon, 2003-02-03 at 13:51, Hans Reiser wrote:
> I think it is much more like switching the engine in the car and 
> expecting the car company to honor the warranty on the air conditioner, 
> or expecting that if I install linux on my dell laptop and the hard 
> drive goes bust that the hard drive will be replaced under warranty.

Grin, my analogy was that if you replace the engine you can't complain
about engine problems.  If you replace the kernel you can't complain to
us about kernel problems.  The fact that many different aspects of the
system actually translate into kernel issues makes it hard to fix things
when people start playing musical kernels on us.

> 
> I hope you are not saying that recompiling the kernel is more deserving 
> of voiding a warranty than, say, creating a custom .bashrc file?

A customer recently called us to figure out why their benchmark was
spending all it's time in system time instead of actually doing
something useful.

Eventually I figured out they had switched from our scsi driver to one
directly from the hardware vendor (could just as easily have been from
the vanilla kernel though).  They had done everything right in terms of
compiling it, and the replacement driver probably didn't have any
additional bugs over ours, but it also wasn't highmem io enabled.  So it
spent all it's time doing bounce buffer copies.  

At no point in time during the debugging did I ask them if they had
changed their .bashrc.  

Any reasonable distro is going to try working with their customers as
much as possible.  At the same time, we've put huge amounts of time and
energy into making a coherent, fast and reliable product.  The customer
needs to understand that swapping out bits and pieces of that makes it
significantly harder to support the system as a whole.

-chris

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Chris Mason wrote:

>Grin, my analogy was that if you replace the engine you can't complain
>about engine problems.
>
This we agree on.

>  If you replace the kernel you can't complain to
>us about kernel problems.  The fact that many different aspects of the
>system actually translate into kernel issues makes it hard to fix things
>when people start playing musical kernels on us.
>
But you have a civic duty to determine that the modified kernel is at 
fault (or at least could be at fault) before you refuse to support it.  
Otherwise you are no better than Dell.

Also, if you refuse to support an official Marcelo kernel, you are 
undermining our unified social structure.  That is a social structure 
that should be undermined only for good cause.

>
>  
>
>>I hope you are not saying that recompiling the kernel is more deserving 
>>of voiding a warranty than, say, creating a custom .bashrc file?
>>    
>>
>
>A customer recently called us to figure out why their benchmark was
>spending all it's time in system time instead of actually doing
>something useful.
>
>Eventually I figured out they had switched from our scsi driver to one
>directly from the hardware vendor (could just as easily have been from
>the vanilla kernel though).  They had done everything right in terms of
>compiling it, and the replacement driver probably didn't have any
>additional bugs over ours, but it also wasn't highmem io enabled.  So it
>spent all it's time doing bounce buffer copies.  
>
>At no point in time during the debugging did I ask them if they had
>changed their .bashrc. 
>
I used to know sysadmin departments that would not support users with 
customized .bashrc files....

I have always had a pet peeve for keeping users disenfranchised from the 
right to program.  Not everyone agrees with me, particularly the persons 
whose workload is increased by such subversive activity.;-)

> 
>
>Any reasonable distro is going to try working with their customers as
>much as possible.  At the same time, we've put huge amounts of time and
>energy into making a coherent, fast and reliable product.  The customer
>needs to understand that swapping out bits and pieces of that makes it
>significantly harder to support the system as a whole.
>
Kind of like buying a non-IBM hard drive for your IBM PC?

>
>-chris
>
>
>
>
>  
>
The issues you raise are real ones, and important ones, but not 
sufficient ones.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Lars Marowsky-Bree]

On 2003-02-03T22:50:04,
   Hans Reiser <reiser@namesys.com> said:

> >Any reasonable distro is going to try working with their customers as
> >much as possible.  At the same time, we've put huge amounts of time and
> >energy into making a coherent, fast and reliable product.  The customer
> >needs to understand that swapping out bits and pieces of that makes it
> >significantly harder to support the system as a whole.

> Kind of like buying a non-IBM hard drive for your IBM PC?

Yes, just like that. Buying a non-IBM hard drive and expecting IBM to help you
with hard drive problems.

> The issues you raise are real ones, and important ones, but not 
> sufficient ones.

We are all anxiously waiting for your solution and explanation of who should
pay the additional support infrastructure necessary.

This is part of the deal: We support what we deliver. Anything else is not
supported, and there are core pieces which void the entire warranty. Usually,
such pieces include the kernel and glibc.

If you want someone to support your home-brew system - which is an entirely
reasonable request - you are probably much better off paying a local smaller
Linux company or some inspired college students. Or be prepared to pay a
_really_ impressive sum to a commercial distributor ;-)

Can't see how this is negatively affecting the Linux community, seems to be a
big opportunity for CS students...


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthias Andree]

Lars Marowsky-Bree <lmb@suse.de> writes:

> This is part of the deal: We support what we deliver. Anything else is not
> supported, and there are core pieces which void the entire warranty. Usually,
> such pieces include the kernel and glibc.

Well, "support" is something SuSE used to be MUCH better at. Unless you
can talk to a maintainer directly, you're getting less than 33% of
issues to feedback@suse.de past the defense line (aka 1st level support)
through to people who do real work. I know that all costs money, but why
not release less often and offer more updates in-between...

For example, a problem that bothers me right now, and to remain remotely
on-topic, I have a specific application pattern that drives ext3fs on
IDE drives mad on SuSE 8.1 with k_athlon-2.4.19-167 -- it will cause
high block-out rates to the drive, but the same application won't do
that on ext2fs, won't do that on reiserfs and won't do that on ext3fs
when on a SCSI drive. It's just ext3fs + IDE. Given that Andrew Morton
finds the bug which hogs some applications down big time, would SuSE
offer new kernels? That'd be "support".

> If you want someone to support your home-brew system - which is an entirely
> reasonable request - you are probably much better off paying a local smaller
> Linux company or some inspired college students. Or be prepared to pay a
> _really_ impressive sum to a commercial distributor ;-)

The idea is to have common standards, and the vanilla kernel is one
such. The diversification of kernels and their
bugfixes/tuning/... patches doesn't make it easier to maintain the
baseline code, because figuring if it's the distributor's patch, a
combination of patches, this all eats up and binds time that could be
better invested into the base line kernel. I understand it's too
conservative or at the wrong moment when a distro is released, but the
issue is that the "community" gets split up that way.

-- 
Matthias Andree

Re: when distros do not support official Marcelo kernels they are not being team players

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On Tue, 04 Feb 2003 02:55:55 +0100, Matthias Andree <ma+rfs@dt.e-technik.uni-dortmund.de>  said:

> Well, "support" is something SuSE used to be MUCH better at. Unless you
> can talk to a maintainer directly, you're getting less than 33% of
> issues to feedback@suse.de past the defense line (aka 1st level support)
> through to people who do real work. I know that all costs money, but why

The important question is, of course, what the *proper* disposition of the
other 67% should be. Are the other 67% *also* real bugs, or are they
pilot error, RTFM/FAQ, "Oh that's already fixed in the patch we're about
to ship", etc etc?

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: when distros do not support official Marcelo kernels they are not being team players

[Brian Tinsley]

Valdis.Kletnieks@vt.edu wrote:

>On Tue, 04 Feb 2003 02:55:55 +0100, Matthias Andree <ma+rfs@dt.e-technik.uni-dortmund.de>  said:
>
>>Well, "support" is something SuSE used to be MUCH better at. Unless you can talk to a maintainer directly, you're getting less than 33% of
>>issues to feedback@suse.de past the defense line (aka 1st level support) through to people who do real work. I know that all costs money, but why
>>    
>>
>
>The important question is, of course, what the *proper* disposition of the other 67% should be. Are the other 67% *also* real bugs, or are they
>pilot error, RTFM/FAQ, "Oh that's already fixed in the patch we're about
>to ship", etc etc?
>
>  
>
I'll even weigh in here and bet anything that the other 67% can be 
disposed as garbage. As bad as 1st level support can often be in *any* 
organization, they more often than not serve their purpose and weed out 
the riff-raff.

I'm still interested to hear what Chris and Lars (and whoever else) have 
to say to this :)

Re: when distros do not support official Marcelo kernels they are not being team players

[Chris Mason]

On Mon, 2003-02-03 at 21:35, Brian Tinsley wrote:

> I'll even weigh in here and bet anything that the other 67% can be 
> disposed as garbage. As bad as 1st level support can often be in *any* 
> organization, they more often than not serve their purpose and weed out 
> the riff-raff.
> 
> I'm still interested to hear what Chris and Lars (and whoever else) have 
> to say to this :)

I have the luxury of spending my time coding, reading mailing lists and
arguing with Hans, roughly 1 ocean away from the guys who get to read
feedback@suse.de ;-)  So, I'm honestly not sure.

-chris

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthew Johnson]

On Mon, 2003-02-03 at 18:41, Chris Mason wrote:
> On Mon, 2003-02-03 at 21:35, Brian Tinsley wrote:
> 
> > I'll even weigh in here and bet anything that the other 67% can be 
> > disposed as garbage. As bad as 1st level support can often be in *any* 
> > organization, they more often than not serve their purpose and weed out 
> > the riff-raff.
> > 
> > I'm still interested to hear what Chris and Lars (and whoever else) have 
> > to say to this :)
> 
> I have the luxury of spending my time coding, reading mailing lists and
> arguing with Hans, roughly 1 ocean away from the guys who get to read
> feedback@suse.de ;-)  So, I'm honestly not sure.
> 
> -chris

Not sure where feedback e-mail goes to now, but SuSE did change it to:

http://www.suse.de/cgi-bin/feedback.cgi

Its much better this way (I don't work for SuSE but did use that form
once and found it quite useful). Probably did it this way due to the
amount of riff-raff getting through.

Matt

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthias Andree]

Matthew Johnson <matthew@psychohorse.com> writes:

> Not sure where feedback e-mail goes to now, but SuSE did change it to:
>
> http://www.suse.de/cgi-bin/feedback.cgi
>
> Its much better this way (I don't work for SuSE but did use that form
> once and found it quite useful). Probably did it this way due to the
> amount of riff-raff getting through.

I'd think it's there to avoid asking back for missing information.

-- 
Matthias Andree

Re: when distros do not support official Marcelo kernels they are not being team players

[Hubert Mantel]

Hi,

On Mon, Feb 03, Chris Mason wrote:

> > I'll even weigh in here and bet anything that the other 67% can be 
> > disposed as garbage. As bad as 1st level support can often be in *any* 
> > organization, they more often than not serve their purpose and weed out 
> > the riff-raff.
> > 
> > I'm still interested to hear what Chris and Lars (and whoever else) have 
> > to say to this :)
> 
> I have the luxury of spending my time coding, reading mailing lists and
> arguing with Hans, roughly 1 ocean away from the guys who get to read
> feedback@suse.de ;-)  So, I'm honestly not sure.

Also one should note that feedback@suse.de != support@suse.de

Funnily a lot of support requests go to "webmaster@suse.de"...

> -chris
                                                                  -o)
    Hubert Mantel              Goodbye, dots...                   /\\
                                                                 _\_v

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthias Andree]

Brian Tinsley <btinsley@emageon.com> writes:

> I'll even weigh in here and bet anything that the other 67% can be
> disposed as garbage. As bad as 1st level support can often be in *any*
> organization, they more often than not serve their purpose and weed out
> the riff-raff.

67% of what they get in total, likely.

67% of the reports that developers and administrators send them, no.

-- 
Matthias Andree

Re: when distros do not support official Marcelo kernels they are not being team players

[Hans Reiser]

Matthias Andree wrote:

>Brian Tinsley <btinsley@emageon.com> writes:
>
>  
>
>>I'll even weigh in here and bet anything that the other 67% can be
>>disposed as garbage. As bad as 1st level support can often be in *any*
>>organization, they more often than not serve their purpose and weed out
>>the riff-raff.
>>    
>>
>
>67% of what they get in total, likely.
>
>67% of the reports that developers and administrators send them, no.
>
>  
>
Developers are not magical beasts immune to error.  33% being real 
errors would be expectable. 

This is not to say that they have enough staff to handle all their real 
error reports though....  Remember that with open source, there is not a 
lot of money floating around at companies like SuSE, Mandrake, etc.  If 
you aren't offering to pay them to fix the bug, can they really afford 
to fix every bug reported?  Probably not is my guess.

Namesys fixes every bug, but filesystem developers are unusual because 
our product has to be zero defect or no one wants to use it.

Send bug reports to developers listed in the source code, not distros.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthias Andree]

On Tue, 04 Feb 2003, Hans Reiser wrote:

> Developers are not magical beasts immune to error.  33% being real 
> errors would be expectable. 

Depends on the person, if he/she is chary about reporting an issue
he/she is unsure about or just reports anything.

> This is not to say that they have enough staff to handle all their real 
> error reports though....  Remember that with open source, there is not a 
> lot of money floating around at companies like SuSE, Mandrake, etc.  If 
> you aren't offering to pay them to fix the bug, can they really afford 
> to fix every bug reported?  Probably not is my guess.

How do they get to break packages that used to work in older versions
without fixing it within 15 months? NTP broadcastclient used to work
before 4.0.99something.

> Send bug reports to developers listed in the source code, not distros.

That's no guarantee distros will even apply patches...

Re: when distros do not support official Marcelo kernels they are not being team players

[Matthias Andree]

On Mon, 03 Feb 2003, Valdis.Kletnieks@vt.edu wrote:

> The important question is, of course, what the *proper* disposition of the
> other 67% should be. Are the other 67% *also* real bugs, or are they
> pilot error, RTFM/FAQ, "Oh that's already fixed in the patch we're about
> to ship", etc etc?

This isn't really the place to discuss SuSE policy that is unrelated to
kernel/reiserfs or something; on topic, they haven't updated reiserfs to
3.6.4. Besides that, I usually only report real issues and I think that
less than 1/6 of my reports are false in the sense of not being bug
reports.

SuSE seem to have a lot of time to help an individual
who hosed his system by installing a vendor driver compiled with wrong
options -- OTOH, there are problems: gfxboot (security), NTP (critical
regression, bcastclient, reported against SuSE 7.3), leafnode-1.9.19
(critical, locking doesn't work), lpr (critical) are unfixed still --
and numerous issues I haven't bothered to report, because what good is
adding new reports if the old aren't being worked on?

How dare they waste time helping with nonstandard setups when even the
shipped software doesn't work right?

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Chris Mason]

On Mon, 2003-02-03 at 14:50, Hans Reiser wrote:

> Also, if you refuse to support an official Marcelo kernel, you are 
> undermining our unified social structure.  That is a social structure 
> that should be undermined only for good cause.
> 

We do support the Marcelo kernel by fixing bugs and giving out the
resulting binary (and source) with all the fixes/features we believe are
important. 

We contribute back to the social structure as much as we can, because
we're a part of it.

> I have always had a pet peeve for keeping users disenfranchised from the 
> right to program.  Not everyone agrees with me, particularly the persons 
> whose workload is increased by such subversive activity.;-)
> 

Our workload goes down when people modify and contribute to programs,
especially when they find bugs or add cool features, it's highly
encouraged ;-)  But once you go off and hack vi, you're using your vi
and not the suse vi.  Send yourself the bug reports, or reproduce on the
suse vi and send us the bug report.

> > 
> >
> >Any reasonable distro is going to try working with their customers as
> >much as possible.  At the same time, we've put huge amounts of time and
> >energy into making a coherent, fast and reliable product.  The customer
> >needs to understand that swapping out bits and pieces of that makes it
> >significantly harder to support the system as a whole.
> >
> Kind of like buying a non-IBM hard drive for your IBM PC?

No, kind of like breaking the warranty void seal on the IBM hard drive,
replacing the motor and then expecting namesys to find a reiserfs bug
when there's constant drive corruption.  Or how about 'I burnt my own
roms onto my hardware raid controller, my parity checking algorithm is
much faster, but now I get these odd crashes all the time.'

In both cases I'd tell them to come back when they can reproduce the
problem on stable hardware, regardless of what actual problem they were
having. 

> >
> The issues you raise are real ones, and important ones, but not 
> sufficient ones.

Grin, then I don't see how I'll ever convince you.  Feel free to start
the swap-any-package-with-anything-that-you-find-on-the-net distro;
convince me by example.

namesys does a great job of processing bug reports from any random guy
with any random set of patches/apps/distro.  But my personal belief is
that supporting the distro as a whole is different, especially when
trying to keep the support costs within the bounds set by the sale price
on the box.

-chris

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Chris Mason wrote:

>
>namesys does a great job of processing bug reports from any random guy
>with any random set of patches/apps/distro. 
>
A Marcelo kernel is NOT a randomly patched kernel.  It is the OFFICIAL 
kernel that the community has picked to be the official kernel.  SuSE 
and RedHat should be team players, and act accordingly. 

Linebackers should defend the quarterback, or convince the rest of the 
team to get a different quarterback.

> But my personal belief is
>that supporting the distro as a whole is different,
>
How?

Why is what Namesys (and you, since you don't personally seem to 
hesitate to fix bugs wherever you can find one however much you might 
defend SuSE not officially doing so) does so hard?  I don't think it is....

If someone finds a bug in something we don't think will become part of 
the official kernel, then we might charge extra once we are sure it is 
too obscure to be worth supporting.  If they are using not the latest 
reiserfsprogs, then I typically feel torn between thinking that they 
should pay for the support when they don't read the FAQ that tells them 
to use the latest fsck, and not wanting them to go off muttering about 
how ReiserFS fsck is buggy....

Of course, when they don't want to upgrade to a recent stable official 
kernel, and they want to keep with a buggy old distro kernel because 
distro marketing has been speaking to the fillings in their head about 
the need for fear of unsupported Marcelo kernels, oh, this irks me....;-)

> especially when
>trying to keep the support costs within the bounds set by the sale price
>on the box.
>
It doesn't really add that much to our support costs.....

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Chris Mason]

On Mon, 2003-02-03 at 19:46, Hans Reiser wrote:
> Chris Mason wrote:
> 
> >
> >namesys does a great job of processing bug reports from any random guy
> >with any random set of patches/apps/distro. 
> >
> A Marcelo kernel is NOT a randomly patched kernel.  It is the OFFICIAL 
> kernel that the community has picked to be the official kernel.  SuSE 
> and RedHat should be team players, and act accordingly. 
> 

The idea is that since you support randomly patched kernels, you also
support generic kernels well.  I was trying to complement the support
job you currently do, especially important since later comments might be
misunderstood as saying you don't do support well.


> Why is what Namesys (and you, since you don't personally seem to 
> hesitate to fix bugs wherever you can find one however much you might 
> defend SuSE not officially doing so

Every developer (not just in suse) I've worked with tries to fix bug
reports seen on various mailing lists and other unofficial channels. 
It's a major part of all of our jobs.

My argument is that when someone buys the distribution, replaces some
piece and complains about their replacement not working, we suggest they
undo their change and call back.  And yes, the kernel is one of those
pieces.

> ) does so hard?  I don't think it is....
> 

Feel free to start the ultimate distribution where people can purchase a
complete set of cds that results in a coherent, stable and fast install,
where any component could be swapped with it's
generic-recompiled-from-the-net counterpart and still provide support
when people report bugs, all for a reasonable price.

I believe it's a problem of scale, where many components are
exponentially harder to support than a single one.

-chris

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Chris Mason wrote:

>Feel free to start the ultimate distribution where people can purchase a
>complete set of cds that results in a coherent, stable and fast install,
>where any component could be swapped with it's
>generic-recompiled-from-the-net counterpart and still provide support
>when people report bugs, all for a reasonable price.
>
>  
>
If the maintainer of the software has issued it as a stable official 
release, the distros should support it unless they know that particular 
release is at fault.

It comes down to a struggle over social place.  I think it is 
presumptuous of distros, and harmful to the community, for them to 
discourage downloading the latest version by the official maintainer 
because of their desire that the user come to them for everything.  The 
maintainer, not the distro, should be telling the user what to use.  If 
the maintainer makes mistakes, they should live with them.  For the 
kernel, the maintainer is Marcelo.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players

[Juan Quintela]

>>>>> "hans" == Hans Reiser <reiser@namesys.com> writes:

hans> Chris Mason wrote:
>> Feel free to start the ultimate distribution where people can purchase a
>> complete set of cds that results in a coherent, stable and fast install,
>> where any component could be swapped with it's
>> generic-recompiled-from-the-net counterpart and still provide support
>> when people report bugs, all for a reasonable price.
>> 
>> 
hans> If the maintainer of the software has issued it as a stable official
hans> release, the distros should support it unless they know that
hans> particular release is at fault.

hans> It comes down to a struggle over social place.  I think it is
hans> presumptuous of distros, and harmful to the community, for them to
hans> discourage downloading the latest version by the official maintainer
hans> because of their desire that the user come to them for everything.
hans> The maintainer, not the distro, should be telling the user what to
hans> use.  If the maintainer makes mistakes, they should live with them.
hans> For the kernel, the maintainer is Marcelo.

Nope.  Life is more complicated than that.  It just happens that
sometimes for having two applications to work well together, you need
to modify one or both.  That is the labour of the distributions.  You
as user has several options:
- You use the distro version.
- Bug distro maintainer to update (it normally works, at least for the
  development versions of the distros : debian unstable, mandrake
  cooker, and I think that the other distributions have one equivalent).
- You know enough to modify maintainer version to work well with your
  distro.
- You install the maintainer version and pray that it will work with
  your distro.

In my view, 1, 2 or 3 are valid options. 4 is just inaceptable except
for learning purposes :(

Later, Juan.

-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hubert Mantel]

Hi,

On Tue, Feb 04, Hans Reiser wrote:

> A Marcelo kernel is NOT a randomly patched kernel.  It is the OFFICIAL 
> kernel that the community has picked to be the official kernel.  SuSE 
> and RedHat should be team players, and act accordingly. 

SuSE and Red Hat cannot use the official kernel, because it falls apart in 
certain scenarios and is lacking lots of features and drivers.

All the needed fixes and features are available. I would be happy if all 
this stuff would be in the official kernel, so I would not need to 
maintain several hundreds of patches.

> Linebackers should defend the quarterback, or convince the rest of the 
> team to get a different quarterback.

Sorry, I don't understand a single word in this sentence.

> Hans
                                                                  -o)
    Hubert Mantel              Goodbye, dots...                   /\\
                                                                 _\_v

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Tue, 04 Feb 2003 17:36:09 +0100, Hubert Mantel <mantel@suse.de>  said:

> > Linebackers should defend the quarterback, or convince the rest of the 
> > team to get a different quarterback.
> 
> Sorry, I don't understand a single word in this sentence.

I didn't either.  Usually in American football, Linebackers are trying to
tackle whatever person on the opposing team happens to have the ball at
the moment.

Linebackers *do* sometimes cause a change of quarterback, but this usually
involves an injury timeout to remove a recently-flattened quarterback from
the field....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Valdis.Kletnieks@vt.edu wrote:

>On Tue, 04 Feb 2003 17:36:09 +0100, Hubert Mantel <mantel@suse.de>  said:
>
>  
>
>>>Linebackers should defend the quarterback, or convince the rest of the 
>>>team to get a different quarterback.
>>>      
>>>
>>Sorry, I don't understand a single word in this sentence.
>>    
>>
>
>I didn't either.  Usually in American football, Linebackers are trying to
>tackle whatever person on the opposing team happens to have the ball at
>the moment.
>
>Linebackers *do* sometimes cause a change of quarterback, but this usually
>involves an injury timeout to remove a recently-flattened quarterback from
>the field....
>  
>
Oh dear, it seems I haven't played football in 20 years.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Hubert Mantel wrote:

>Hi,
>
>On Tue, Feb 04, Hans Reiser wrote:
>
>  
>
>>A Marcelo kernel is NOT a randomly patched kernel.  It is the OFFICIAL 
>>kernel that the community has picked to be the official kernel.  SuSE 
>>and RedHat should be team players, and act accordingly. 
>>    
>>
>
>SuSE and Red Hat cannot use the official kernel, because it falls apart in 
>certain scenarios and is lacking lots of features and drivers.
>
>All the needed fixes and features are available. I would be happy if all 
>this stuff would be in the official kernel, so I would not need to 
>maintain several hundreds of patches.
>
Has Namesys just been lucky in regards to how responsive Marcelo has 
been for us?  In my interactions with him, he has never once 
unreasonably rejected a patch or ignored a patch.

>
>  
>
>>Linebackers should defend the quarterback, or convince the rest of the 
>>team to get a different quarterback.
>>    
>>
>
>Sorry, I don't understand a single word in this sentence.
>
Sorry about the american football reference (quarterbacks are the team 
strategists and the players who most often have the ball, linebackers 
are guys who among other things keep him from being tackled (being 
tackled while you have the ball is bad)). 

The ability to download the latest official stable kernel is one of the 
great things about Linux.  There are times when the latest official 
kernel is behind SuSE or RedHat, and there are times when it is ahead, 
as a result of different release schedules if nothing else.

It seems that what RedHat puts on its Advanced Server CD is a kernel no 
reiserfs user would want to use if he had a choice.  What SuSE has put 
on its CD is at the moment probably a faster and more stable for 
reiserfs kernel than the official kernel (I hope we will change that 
soon, but....).  ReiserFS is just one kernel feature, and what is best 
for users is going to vary all over the place.  Sometimes the distro 
kernel is more advanced for some feature, sometimes the official kernel 
is more advanced.

If we as a community fragment for the purpose of experimenting, then it 
is healthy.  However, there needs to be one kernel branch that the 
entire community backs up and says "we stand behind this except when we 
are motivated by its specific flaws for the particular user in question 
to not do so". 

Maintainers need to be able to send their patches to Marcelo, and know 
that now they can tell any Linux user of any distro to just upgrade to 
the latest official stable kernel and case closed.

Consider my situation with the user of the RedHat 2.4.9 based kernel, 
because it has the advantage of not being a SuSE related problem.;-)

ReiserFS users should not use a 2.4.9 based kernel, they should use 
2.4.18 or later.  It is wrong that RedHat won't support him if he 
upgrades to the official kernel.  In this case, the official kernel is 
more advanced for his needs.  It removes one of the great advantages of 
the open source methodology compared to proprietary OSes if he can't 
download kernels from the net.  What should I say to him (other than 
switch his whole distro to SuSE;-) )?

The official kernel is our kernel community's only chance to overcome 
its fragmentation.  We need to support it.  All of us, regardless of 
what camp we are in.   It should never be the discouraged officially 
unsupported branch of the kernel (except when specific flaws relevant to 
the usage of the user in question are present, and even then it should 
be with an attitude of "the patch you need isn't yet in the official 
kernel, try the distro kernel and things will work").

Let's all work as a team.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Chris Mason]

On Tue, 2003-02-04 at 16:45, Hans Reiser wrote:

> The official kernel is our kernel community's only chance to overcome 
> its fragmentation.  We need to support it.  

We do support the official kernel, by actively developing and improving
it, and by answering questions on public mailing lists.  When things
work in our kernel (like reiserfs, or andrea's vm etc) we work to get it
into the vanilla kernel.  We're also constantly updating our work to
keep it in line with the current vanilla sources.  It takes a while due
to the volume of patches and testing required, but we're always working
on it.

Of course this is a different issue from debugging customer problems
sent to support@distro.foo hit while running a vanilla kernel.

The entire reason we have our own kernel is because for some reason or
another the vanilla kernel wasn't sufficient to meet our needs.  This
fact is mutually exclusive with supporting someone who uses the vanilla
kernel.

This is the key point that I seem to be failing to convey, so please
explain how we can:

A) modify a kernel because it doesn't work, and then B) help people use
the unmodified kernel that doesn't work without C) telling them to use
our modifications.

> All of us, regardless of 
> what camp we are in.   It should never be the discouraged officially 
> unsupported branch of the kernel 

Marcelo's kernel is a base that we all start from, and all work toward,
but each distro has the right to decide which features and fixes they
believe are critical for their own support contracts, and then expect
the users to use those features and fixes before we start answering
their questions.

Again, this has nothing to do with questions sent to public mailing
lists, where every distro coder will help debug a problem hit on a
vanilla kernel.  It's only about someone sending mail to
support@distro.foo, and expecting help with some random combination of
packages that may or may not have come from us.

-chris (not speaking for suse in any official sense)

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hans Reiser]

Chris Mason wrote:

>
>This is the key point that I seem to be failing to convey, so please
>explain how we can:
>
>A) modify a kernel because it doesn't work, and then B) help people use
>the unmodified kernel that doesn't work without C) telling them to use
>our modifications.
>
It is very rare that a stable official kernel fails at every task.  I 
encourage you to tell them to use your modified kernel when your 
modification is relevant to the bug they have, and to be proud that you 
are able to do so.

When my Dell laptop can't power on, I don't want to hear about how I 
need to install windows before I can get warranty service.

When a Redhat user's C compiler is broken, he should not hear about how 
he needs to get rid of 2.4.18 and go back to some RedHat variant of 
2.4.9 before he can get support for the broken C compiler.   He should 
be able to upgrade the kernel to an official stable kernel in which 
ReiserFS is at its most stable without worrying that he is endangering 
his ability to get C compiler support.

-- 
Hans

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Bernd Schubert]

On Tuesday 04 February 2003 23:22, Chris Mason wrote:
> On Tue, 2003-02-04 at 16:45, Hans Reiser wrote:
> > The official kernel is our kernel community's only chance to overcome
> > its fragmentation.  We need to support it.
>
> We do support the official kernel, by actively developing and improving
> it, and by answering questions on public mailing lists.  When things
> work in our kernel (like reiserfs, or andrea's vm etc) we work to get it
> into the vanilla kernel.  We're also constantly updating our work to
> keep it in line with the current vanilla sources.  It takes a while due
> to the volume of patches and testing required, but we're always working
> on it.
>

Hmm, I know its out of topic, but I want to take my chance and complain about 
the Suse kernel.

We are currently testing one of our servers with non-free software and only 
get a pre-compiled kernel module from this company. Unfortunality they only 
have kernel modules for well knows Distros, such as Suse, RedHat, etc and it 
seems to be very difficult to convince them to compile it for a vanilla 
kernel.
So we are using a Suse kernel on Debian. For some reason we couldn't use the 
binary and had to recompile the kernel from the source Suse provides. Well, 
until it came to the module part everything was fine, but then errors 
orrcured and we had to find the config options to disable those modules. 
Since we first tried to use the config-options Suse had set as default, the 
Suse-people *MUST* have seen themselves that it doesn't work this way. 
The other thing I'm strongly wondering about, is what for nomal home users 
need a kernel that has kdb and other very seldom used patches included. Which 
usual home-user (and that is what I thought to be Suse for) needs kernel 
debugging? IMHO kernel debuggers won't have a problem with patching a kernel 
themselves.

Finally we got it working and could even load the module, but later on we 
figured out that nfs file locking for imports from another server isn't 
working properly (/proc/mounts shows that it is enabled, but e.g. 'man' 
complains that it is not). Well, actually I'm wondering that anything works 
in such a stronlgy patched kernel. 
Of course, the nfs file locking works when we use a vanilla kernel.

So whom shall I blame it is not? Suse, and tell them we are only using their 
kernel (and this not even without some force)? And I believe people from the 
kernel mailinglist won't feel responsible, too.

Just a few reasons why I don't like using distros' kernels !

Best regards,
	Bernd

PS: The kernel is 2.4.19-4GB, I've forgotten which one of Suses subversions it 
was, but it was rather high (174?).

Re: when distros do not support official Marcelo kernels they are not being team players (was Re: reiserfs on redhat advanced server?)

[Hubert Mantel]

Hi Hans,

On Wed, Feb 05, Hans Reiser wrote:

> >>A Marcelo kernel is NOT a randomly patched kernel.  It is the OFFICIAL 
> >>kernel that the community has picked to be the official kernel.  SuSE 
> >>and RedHat should be team players, and act accordingly. 
> >
> >SuSE and Red Hat cannot use the official kernel, because it falls apart in 
> >certain scenarios and is lacking lots of features and drivers.
> >
> >All the needed fixes and features are available. I would be happy if all 
> >this stuff would be in the official kernel, so I would not need to 
> >maintain several hundreds of patches.
> >
> Has Namesys just been lucky in regards to how responsive Marcelo has 
> been for us?  In my interactions with him, he has never once 
> unreasonably rejected a patch or ignored a patch.

Every time an official kernel is released, I can see postings to this list 
asking for new versions of various patches (data-logging, quota for 
reiserfs and so on). So apparently there are users who cannot use the 
vanilla kernel but need a patched one. So this is not really different 
from the situation distributors are in.

This does not imply that Marcelo is doing a bad job. The opposite is true. 
There might be good reasons not to include every new feature into the 
official kernel (if it is only because there are more urgent things that 
get higher priority).

And the distributors really do help the development of the kernel and thus 
the whole community. Take for example the highmem-I/O feature: This is 
something that is strictly needed when you want to run Linux on some 
larger machines. We have been shipping it for about two years now and 
finally it was included in 2.4.20. I assume the fact that it has been in 
use by distributions for quite some time was proof enough that this 
feature is needed and stable, so finally it got in. I'm not sure if this 
had happened if it hadn't been used in distributions before.

Btw, same happened with reiserfs itself. If SuSE had always shipped the 
official kernel, reiserfs probably hadn't made its way into 2.4. So you 
are yourself profiting from distributors shipping patched kernels ;)

And things move on. For example we need the pte-highmem feature. In the 
official kernel ptes cannot live in highmem, which means that Oracle is 
unusable on bigger machines. On a box with way 32 GB of RAM, Oracle simply 
will run out of memory (in reality it "only" runs out of ptes, but the 
result is the same).

There are lots of similar examples.

[...]

> The ability to download the latest official stable kernel is one of the 
> great things about Linux.  There are times when the latest official 
> kernel is behind SuSE or RedHat, and there are times when it is ahead, 
> as a result of different release schedules if nothing else.

Users are always able to download the official kernel. And they are free 
to use it. The question is whether a distributor can officially support 
those kernels. Problem is that nobody is willing to pay the price for this 
service. It _IS_ expensive. As Chris pointed out, this is a great 
opportunity for independent Linux gurus who can sell this kind of service 
to people who are willing to pay for it.

But distributors really need to restrict official support to the kernel 
they ship. I'm in this business for about 10 years now and have seen so 
many cases where people have screwed their system by just recompiling the 
kernel and forgetting some important config option. This simply cannot be 
supported by the price of a CD box.

All this does not make the official kernel useless. It is a great 
reference kernel and THE major synchronisation point. It is very valuable 
for distributors to get a report of the form "I have a problem with your 
distro kernel, but the official kernel works fine", because this restricts 
the source of the problem and makes it easier to fix. And when we update 
our kernel to some new official version, every time lots of patches become 
obsolete, which is great. I'm not really eager to maintain hundreds of 
patches. If we would be able to just use the official kernel, I would be a 
very happy guy.

> It seems that what RedHat puts on its Advanced Server CD is a kernel no 
> reiserfs user would want to use if he had a choice.  What SuSE has put 

Yes, but Red Hat always has made it very clear that they are not a big 
supporter of reiserfs and prefer ext3 instead. It's their default 
filesystem and chances are high that ext3 in RH is better than in the 
standard kernel. If people really care about support they should use what 
the distributor recommends and supports.

You should realize that you (and probably most of the people on this list) 
are not representative for the people just using Linux to get their work 
done. For those people support is some kind of insurance. They are willing 
to pay money just in order to be sure somebody will solve their problems 
when they arise. They will just use what they think is best supported, so 
they should (and most of them will) use ext3 when deploying RHAS.

The sysadmin of some big bank NEVER will use some "inofficial" kernel, 
because his boss will kill him as soon as something breakes. He will not 
even use something different than the default filesystem of the product, 
because he wants to be safe. He gets paid for this.

> on its CD is at the moment probably a faster and more stable for 
> reiserfs kernel than the official kernel (I hope we will change that 
> soon, but....).  ReiserFS is just one kernel feature, and what is best 
> for users is going to vary all over the place.  Sometimes the distro 
> kernel is more advanced for some feature, sometimes the official kernel 
> is more advanced.
> 
> If we as a community fragment for the purpose of experimenting, then it 
> is healthy.  However, there needs to be one kernel branch that the 
> entire community backs up and says "we stand behind this except when we 
> are motivated by its specific flaws for the particular user in question 
> to not do so". 

This is exactly what's happening. The official kernel is the one that 
mandates the release number. It is the major synchronization point every 
distributor bases its kernel on. And the official kernel benefits from the 
work the distributors are doing.

> Maintainers need to be able to send their patches to Marcelo, and know 
> that now they can tell any Linux user of any distro to just upgrade to 
> the latest official stable kernel and case closed.

An user who has spent lots of money for his IT infrastructure WANTS an 
official kernel from the distributor. If a bug is present in the distro 
kernel but not in the official one, they will not use the official kernel, 
but demand that the fix goes into the distro kernel.

> Consider my situation with the user of the RedHat 2.4.9 based kernel, 
> because it has the advantage of not being a SuSE related problem.;-)
> 
> ReiserFS users should not use a 2.4.9 based kernel, they should use 
> 2.4.18 or later.  It is wrong that RedHat won't support him if he 
> upgrades to the official kernel.  In this case, the official kernel is 
> more advanced for his needs.  It removes one of the great advantages of 
> the open source methodology compared to proprietary OSes if he can't 
> download kernels from the net.  What should I say to him (other than 
> switch his whole distro to SuSE;-) )?

There is nothing other that you can say. If the user wants RHAS and wants
the support (hey, he is paying quite some money for it), he really should
use what the distributor recommends, tests and certifies, which is ext3 in
the case of RHAS.

> The official kernel is our kernel community's only chance to overcome 
> its fragmentation.  We need to support it.  All of us, regardless of 
> what camp we are in.   It should never be the discouraged officially 
> unsupported branch of the kernel (except when specific flaws relevant to 
> the usage of the user in question are present, and even then it should 
> be with an attitude of "the patch you need isn't yet in the official 
> kernel, try the distro kernel and things will work").
> 
> Let's all work as a team.

I think this is the case. And it works pretty well.

> Hans
                                                                  -o)
    Hubert Mantel              Goodbye, dots...                   /\\
                                                                 _\_v

Re: when distros do not support official Marcelo kernels they are not being team players

[Juan Quintela]

>>>>> "hubert" == Hubert Mantel <mantel@suse.de> writes:

Hi

hubert> Every time an official kernel is released, I can see postings to this list 
hubert> asking for new versions of various patches (data-logging, quota for 
hubert> reiserfs and so on). So apparently there are users who cannot use the 
hubert> vanilla kernel but need a patched one. So this is not really different 
hubert> from the situation distributors are in.

I agree here.  For a new feature to be included in Marcelo kernel, it
is good to have been for a while in a distro kernel.  If you send a
new feature/patch to Marcelo, and you add that this patch has been
included for a year in <put distriubtion here> and found stable, you
get more posibilities of having it included.

hubert> All this does not make the official kernel useless. It is a great 
hubert> reference kernel and THE major synchronisation point. It is very valuable 
hubert> for distributors to get a report of the form "I have a problem with your 
hubert> distro kernel, but the official kernel works fine", because this restricts 
hubert> the source of the problem and makes it easier to fix. And when we update 
hubert> our kernel to some new official version, every time lots of patches become 
hubert> obsolete, which is great. 

Completely agree.

hubert> I'm not really eager to maintain hundreds of 
hubert> patches. If we would be able to just use the official kernel, I would be a 
hubert> very happy guy.

/me founds that making happy distro kernels maintainers is easy :)

Later, Juan.


-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy

Re: reiserfs on redhat advanced server?

[Oleg Drokin]

Hello!

On Fri, Jan 31, 2003 at 01:21:48PM +0100, Lars Marowsky-Bree wrote:
> I'm not saying this is good or bad, but it is just they way it is. I can't
> really see a distribution shipping a plain, unpatched kernel. Trust me. Every

http://www.slackware.com
(Don't read it as if I propose to use slackware ;) )
Some of their releases were shipped with vanilla kernels.

> distributor would love too, because it would lower cost for them. You know how
> much highly qualified _work_ is needed to maintain a distribution kernel?
> Nobody does that out of their own free will.

This only confirms that linux kernel development model is somewhat flawed at general.

Bye,
    Oleg