Re: Loadable drivers [was SMP/cc Cluster description ]

Re: Loadable drivers [was SMP/cc Cluster description ]

[Andi Kleen]

erich@uruk.org writes:
> 
> MS with Windows (and even DOS) went the right direction here.  In fact,
> they have been hurting themselves by what lack of driver interoperability
> there is even between Windows NT/2K/XP.  Admittedly they didn't have much
> of a choice with their closed-source scheme, but it still is a better
> solution from a usability and stability point of view in general.

I remember some quote from a Microsoft manager when they released Win2k.
(paraphrased) "A significant percentage of the blue screens in NT4 were 
caused by buggy third party drivers." (and then how they will try to avoid
it in the future by having more stringent windows logo tests etc.

The experience in recemt Linux is basically similar. Single Linux has
gained vendor support in drivers it has gotten a lot more unstable
than it used to be (sad but true). There are first a lot more and more
complex drivers than there used to be.  A lot of drivers both writen
by individuals and also companies for their are simply crappy buggy
code. I could give you numerous examples here; names withheld to
protect the guilty. For hardware companies it might be because driver
writing is not a profit center, but a cost. There might be other
reasons. There are just a lot of bad drivers around.

[This is not a general insult on driver writers; there are some very well
written drivers, but also unfortunately a lot of bad ones.]

Now when a driver crashes guess who is blamed? Not the driver author
but the Linux kernel is seen as unstable and it effectively is as 
a end result - it doesn't work for the user. All just because of bad drivers. 

The solution that is tried in Linux land to avoid the "buggy drivers" ->
"linux going to windows levels of stability" trap is to keep drivers in tree
and aggressively auditing them; trying to fix their bugs.

A lot of driver code is actually cut'n'pasted or based
on other driver code (or you can say they often use common design patterns)
Now when a bug/race/.. is found and fixed in one driver having the majority
of drivers in tree makes it actually possible to fix the bug who is likely
in other drivers who use similar code there too. With having drivers in external
trees that crashing/angry user/debugging/etc. would likely need to be done once 
per driver; overall causing much more pain for everybody. 

Your scheme would make this strategy of tight quality control of
drivers much harder, and I think it wouldn't do good to the long term
stability of linux.

There are other reasons why linux has really no stable driver interface.
Sometimes it is just needed to break drivers to fix some bugs cleanly.
Getting rid of this possibility (fixing bugs cleanly even if it requires
changes in other kernel code) would also cause more bugs in the long term.

-Andi

Re: Loadable drivers [was SMP/cc Cluster description ]

[erich]

Andi Kleen <ak@suse.de> wrote:

> erich@uruk.org writes:
> 
> > MS with Windows (and even DOS) went the right direction here.  In fact,
> > they have been hurting themselves by what lack of driver interoperability
> > there is even between Windows NT/2K/XP.  Admittedly they didn't have much
> > of a choice with their closed-source scheme, but it still is a better
> > solution from a usability and stability point of view in general.
> 
> I remember some quote from a Microsoft manager when they released Win2k.
> (paraphrased) "A significant percentage of the blue screens in NT4 were 
> caused by buggy third party drivers." (and then how they will try to avoid
> it in the future by having more stringent windows logo tests etc.

Sure.  Though if they really cared enough, they could have a
"safe-driver" mode that would run the drivers in protected address
spaces or something.  There is a performance loss associated with this
obviously, but it would be a great way to run drivers while testing them
out, or if you had a critical need and a driver would eat itself
occasionally.


> The experience in recemt Linux is basically similar. Single Linux has
> gained vendor support in drivers it has gotten a lot more unstable
> than it used to be (sad but true). There are first a lot more and more
> complex drivers than there used to be.  A lot of drivers both writen
> by individuals and also companies for their are simply crappy buggy
> code. I could give you numerous examples here; names withheld to
> protect the guilty. For hardware companies it might be because driver
> writing is not a profit center, but a cost. There might be other
> reasons. There are just a lot of bad drivers around.

[see above comment about a "safe-driver" mode...  I'm already working
 on this for my own OS project, and have been moderately tempted to do
 it for Linux (as well as an automatic buffer-overflow protector) ]

I'm not disagreeing with you, in fact this is the major reason I'd want
such a framework.

You can pick the drivers that way relatively easily, and stick to the ones
you know work!!  Right now, that is very difficult (and effectively
impossible for most) if you rev anything kernel/driver-related.

There could even be a website that maintained info about drivers and
known-working versions.  Having such info for the kernel tree wouldn't
be a bad idea as is.

The distribution folks could work with the general community on this
pretty easily I'd guess.


> Now when a driver crashes guess who is blamed? Not the driver author
> but the Linux kernel is seen as unstable and it effectively is as
> a end result - it doesn't work for the user. All just because of bad
> drivers.
> 
> The solution that is tried in Linux land to avoid the "buggy drivers" ->
> "linux going to windows levels of stability" trap is to keep drivers in
> tree and aggressively auditing them; trying to fix their bugs.

This works kind of fine when you're talking about developers (though the
problems I mention below aren't completely aleviated either, just not as
bad).  For random people or even IT folks trying to maintain some systems
I would say this is not a feature.

Their only alternative right now is to try out things that work for them
in a very coarse-grained manner ("try out this kernel" : "hmm, didn't work").

A good example is if you need a driver for some new bit of hardware...
you often have to rev your entire kernel/driver tree to get it.  Most would
have no choice but to do so unless hand-held by a kernel hacker.

If it breaks something else (which happens often enough) then you're just
SOL.

I.e. I think too many things are tied together for non-hackers.


> A lot of driver code is actually cut'n'pasted or based
> on other driver code (or you can say they often use common design patterns)
> Now when a bug/race/.. is found and fixed in one driver having the majority
> of drivers in tree makes it actually possible to fix the bug who is likely
> in other drivers who use similar code there too. With having drivers in
> external trees that crashing/angry user/debugging/etc. would likely need
> to be done once per driver; overall causing much more pain for everybody. 
> 
> Your scheme would make this strategy of tight quality control of
> drivers much harder, and I think it wouldn't do good to the long term
> stability of linux.

Not necessarily at all.

You're talking about 2 things:

Driver maintenance:  This mostly need not change, so your example of fixing
problems is the same.

Driver/kernel error reporting:  I say the same thing as I said about MS:
that if we cared enough, we'd have a "safe-driver" framework and driver/
module versioning so that we can tell pretty much exactly where the
problem occured.  You don't use it all the time by any means, but if you
want to get a report answered, there would be some trivial way to turn it
on.

[The "safe-driver" thing obviously may not be the identical footprint so
 a crash may not happen the same way or at all, but if it doesn't happen,
 then hey, you have a way to run stably, which for most people that is
 enough]

Note that MS has been getting a lot of pushback from people about the
inclusion of third-party drivers (to the point where they've partially
disabled the "signed drivers" thing by default, falling back from an
earlier position in the Win XP beta of not accepting them without some
major gyrations on the part of the poor person trying to just run their
hardware).  Just saying "no" isn't that supportable a position.

For example, I think the "module tainted" thing is not terribly useful
along the lines of really fixing bugs.  It may be an indicator and tell
the core kernel folks when not to "waste their time on some binary-only
bug", but I'm not sure what else it does.


> There are other reasons why linux has really no stable driver interface.
> Sometimes it is just needed to break drivers to fix some bugs cleanly.
> Getting rid of this possibility (fixing bugs cleanly even if it requires
> changes in other kernel code) would also cause more bugs in the long term.

This is of course a different problem, but it's unclear if it's one that
would cause more bugs.

Unstable interfaces between major subsystems are often a sign of lack of
design.  [ok, I'm flamebait, but at least I'll be warm-n-toasty now that
it's colder outside]


--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Re: Loadable drivers [was SMP/cc Cluster description ]

[Alan Cox]

> The solution that is tried in Linux land to avoid the "buggy drivers" ->
> "linux going to windows levels of stability" trap is to keep drivers in tree
> and aggressively auditing them; trying to fix their bugs.

We need to teach Linus about "taste" in drivers. His core code taste is
impeccable, but I'm not fond of his driver taste ;)

Alan (currently cleaning up NCR5380)

Re: SMP/cc Cluster description [was Linux/Pro]

[Alan Cox]

> For example, there should be no reason that most drivers or any other
> random kernel module should know anything about SMP.  Under Linux, it
> annoys me to no end that I have to ever know (and yes, I count compiling
> against "SMP" configuration having to know)...  more and more sources of
> error.

Unfortunately the progression with processors seems to be going the
wrong way. Spin locks are getting more not less expensive. That makes
CONFIG_SMP=n a meaningful optimisation for the 99%+ of people not running
SMP


Alan

Loadable drivers [was SMP/cc Cluster description ]

[erich]

Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > For example, there should be no reason that most drivers or any other
> > random kernel module should know anything about SMP.  Under Linux, it
> > annoys me to no end that I have to ever know (and yes, I count compiling
> > against "SMP" configuration having to know)...  more and more sources of
> > error.
> 
> Unfortunately the progression with processors seems to be going the
> wrong way. Spin locks are getting more not less expensive. That makes
> CONFIG_SMP=n a meaningful optimisation for the 99%+ of people not running
> SMP

I appreciate the costs (having worked in the Intel x86 arch team) and don't
like that part either, but the question at hand is whether you need the SMP
locks in most of the compiled drivers/modules themselves.  That is in part
an artifact of the old architecture before they were loadable.


This really goes into a side-topic, but plainly:

The general driver/random module framework in Linux really needs to get
separated from the core kernel, and made so that it doesn't need to be
recompiled between minor kernel versions.  Perhaps even pulling the
drivers in general apart from each other, just aggregating releases for
convenience.

MS with Windows (and even DOS) went the right direction here.  In fact,
they have been hurting themselves by what lack of driver interoperability
there is even between Windows NT/2K/XP.  Admittedly they didn't have much
of a choice with their closed-source scheme, but it still is a better
solution from a usability and stability point of view in general.


Don't get me wrong, I only run MS Software on 2 of my 8 machines (and
have been trying to remove one of those with Wine/WineX), but I appreciate
a better overall solution when I see one.


I will go so far as to say, for the long term this is necessary for the
survival of Linux, and would help it's health even in the server arena.

For example, we need the ability to easily:
  --  Install random drivers on top of your kernel version and not
	have them disappear when changing kernels unless that is desired.

	This is just a royal pain for most people I've ever dealt with
	who are not kernel hackers, and rightly so.

  --  Install random kernels while retaining the same drivers for
	testing/stability purposes.

	I know that on my Linux servers, it bugs me that it's hard/nigh
	impossible sometimes to change the core kernel and still trust
	the drivers haven't drifted.


Those can usually be done by someone who is willing to/capable of
performing sleazy tricks or detailed hacking, but it's a burden.  It
gets prohibitive often enough to be very frustrating, and I'm more patient
than most I know with random kernel hacking.

Stability/control/compatibility is in general of far more concern to most
than tweaked performance of the core kernel against the drivers.  That
wouldn't be given up either, it just becomes an option.


A lesser solution, but moderately workable in the short-term:

Build a framework for an external "drivers" source/binary tree that can be
(via a single command) rebuilt for a different Linux kernel version.  This
is arguably a Linux distribution thing.


--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Re: Loadable drivers [was SMP/cc Cluster description ]

[erich]

I wrote:

> This really goes into a side-topic, but plainly:
> 
> The general driver/random module framework in Linux really needs to get
> separated from the core kernel, and made so that it doesn't need to be
> recompiled between minor kernel versions.  Perhaps even pulling the
> drivers in general apart from each other, just aggregating releases for
> convenience.

...and licensing issues aside (probably only GPL for the moment), this
kind of thing could form the basis for an open-source kernel/OS-independent
set of drivers.

I know that I have been intending on doing a Linux driver compatibility
layer of sorts for my OS project (going to be GPL'd) I'm working on.

Maybe there are some others interested in this kind of project?

--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Re: Loadable drivers [was SMP/cc Cluster description ]

[Stephan von Krawczynski]

On Wed, 05 Dec 2001 11:40:07 -0800
erich@uruk.org wrote:

> This really goes into a side-topic, but plainly:
> 
> The general driver/random module framework in Linux really needs to get
> separated from the core kernel, and made so that it doesn't need to be
> recompiled between minor kernel versions.  Perhaps even pulling the
> drivers in general apart from each other, just aggregating releases for
> convenience.

You have just expressed my wildest nightmares. Just go ahead ...

> MS with Windows (and even DOS) went the right direction here.  In fact,
> they have been hurting themselves by what lack of driver interoperability
> there is even between Windows NT/2K/XP.  Admittedly they didn't have much
> of a choice with their closed-source scheme, but it still is a better
> solution from a usability and stability point of view in general.

You can only be plain kidding with this statement. If you split the drivers
from the rest of the kernel, you managed to get rid of this nice (yes, I really
meant nice) monolithic design, where I only need a simple config file to
_update_ to a new kernel revision (_up_, not _down_) and be happy (including
all the drivers). Obviously you just prove yourself wrong - mentioning not
working driver interoperability between NT/2K/XP whatever - with the idea that
it is indeed possible to make major new kernel versions (which should be
getting better btw) without _any_ changes in the driver framework that will
break your nice and stable but _old_ drivers. What's the use of this? You are
not talking about releasing driver-plugin-/framework-patches or the like just
to load _old_ drivers into _new_ kernel-environment?
Is this what they do at MS? Well if, they have not come that far.

> Don't get me wrong, I only run MS Software on 2 of my 8 machines (and
> have been trying to remove one of those with Wine/WineX), but I appreciate
> a better overall solution when I see one.

This brings up memories about reading Peter Nortons' latest hardcore books some
years ago, brrrr.

Reading between your lines, I can well see that you are most probably talking
about closed-source linux-drivers breaking with permanently released new kernel
revisions. But, in fact, this is the closed-source phenomenon, and _not_ linux.

> I will go so far as to say, for the long term this is necessary for the
> survival of Linux, and would help it's health even in the server arena.

You just invented the circle with edges: you are talking about "long-term" and
"MS drivers solution" at the same time. Remember, this company is currently
saying the product cycle is 3 years. I know pretty big companies that haven't
even managed to get the NT drivers really working under W2K, and were just shot
down by XP release.
I tend to believe we could just wait another two MS cycles to have even the
biggest MS-fans converted to kernel-hackers, only because of being real fed up
with the brilliant, long term driver design.
 
Regards,
Stephan

Re: Loadable drivers [was SMP/cc Cluster description ]

[erich]

Stephan von Krawczynski <skraw@ithnet.com> wrote:

...
> You can only be plain kidding with this statement. If you split the
> drivers from the rest of the kernel, you managed to get rid of this
> nice (yes, I really meant nice) monolithic design, where I only need
> a simple config file to _update_ to a new kernel revision (_up_, not
> _down_) and be happy (including all the drivers).

Hmm.  There is little fundamentally incompatible with having splits in
the core kernel and sets of drivers, and getting most of what you want
here.

However, the comment about being "happy (including all the drivers)"
seems a bit naive in my experience.  That makes the assumption that the
drivers in the new/old/whatever kernel you change to are necessarily of
the same caliber as the ones you came from, and that is not always true
by any means.

Also, in a world where one values stability, being able to rev
backward is quite important also.  The lack of software package tools
like rpm being able to "downgrade" software easily is a serious one in
my mind.  What do you do when you install something major and it
breaks, but it has interdependencies on multiple other things?

I don't object to producing well-tested full sets of driver source that
go with kernel versions, I just don't want them to be tied if I have
a need to pull something apart (a driver from one version and from
another are the only ones that run stably), which frankly happens too
often for my taste.  Even if it didn't I'd still want it as much as
possibly for the fall-back options it provides.


...
> working driver interoperability between NT/2K/XP whatever - with the
> idea that it is indeed possible to make major new kernel versions (which
> should be getting better btw) without _any_ changes in the driver
> framework that will break your nice and stable but _old_ drivers. What's
> the use of this? You are not talking about releasing driver-plugin-/
> framework-patches or the like just to load _old_ drivers into _new_
> kernel-environment?
> Is this what they do at MS? Well if, they have not come that far.

I didn't go far enough with my comment.  I should have said that the
lack of a common driver framework base that works between *all* (or nearly
all, some obsolescence is ok) their Windows versions is a problem.

In the consumer line (win3.1/95/98/me) where they were keeping very good
compatibility across some kinds of drivers (it had it's problems, to be
sure), they were trying the hardest because they recognized that the
most important thing was to just have things work in the first place.

NOTE:  I'm not endorsing their overall API convenience (driver or
Win16/Win32), stability, suitability, merchantability, whatever, I'm
just talking about drivers and their distribution model at the moment.


...
> Reading between your lines, I can well see that you are most probably
> talking about closed-source linux-drivers breaking with permanently
> released new kernel revisions. But, in fact, this is the closed-source
> phenomenon, and _not_ linux.

No, though I don't object to closed-source in general per se.  I hate it
for myself and businesses I've worked for because I like to be able to
fix/improve/whatever code, but I recognize that the majority of users
out there would never touch code.

My general feeling is that binary drivers are ok/should be supported well
across versions since that is the thing you load in at boot/bring-system-
up time.  Having separate (and usually many) step(s) to getting a driver
and having it load on your system is a major and I'm thinking artificial
pain.


In general my argument stems from the same basis that the kernel/user-level
interface does:  keeping interfaces stable and removing packaging/bundling
requirements across major boundaries almost always yield a win somewhere.

In the case of MS and drivers, the win they have in convenience to end-users
of all types, and the ability to mix and match drivers forward or backward
up to some limitations in API revisions.


...
> I tend to believe we could just wait another two MS cycles to have even
> the biggest MS-fans converted to kernel-hackers, only because of being
> real fed up with the brilliant, long term driver design.

Most people will never touch code or a compiler, and just want a simple
obvious formula or even to have the system automagically do the Right
Things for you.  Even many programmers have limitations of curiousity or
energy, and it isn't a bad thing to organize even core system things to
be easy.

MS may be stupid/annoying in other ways, but they know this and have
moved toward it, albeit sluggishly at times.


--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Re: Loadable drivers [was SMP/cc Cluster description ]

[Stephan von Krawczynski]

On Wed, 05 Dec 2001 13:17:47 -0800
erich@uruk.org wrote:

> Stephan von Krawczynski <skraw@ithnet.com> wrote:
> > You can only be plain kidding with this statement. If you split the
> > drivers from the rest of the kernel, you managed to get rid of this
> > nice (yes, I really meant nice) monolithic design, where I only need
> > a simple config file to _update_ to a new kernel revision (_up_, not
> > _down_) and be happy (including all the drivers).
> 
> Hmm.  There is little fundamentally incompatible with having splits in
> the core kernel and sets of drivers, and getting most of what you want
> here.

There is. You double the administrational work. Most updates are performed
because you want the latest bugfixes. If you get more bugfixes than you
intended - lucky.
Currently you draw the latest kernel-tree (or the latest _you_ like to use),
use your old .config and it works out - as long as you do not use closed-source
drivers.
If you split things up, you draw _two_ archives, compile and install both.

> However, the comment about being "happy (including all the drivers)"
> seems a bit naive in my experience.  That makes the assumption that the
> drivers in the new/old/whatever kernel you change to are necessarily of
> the same caliber as the ones you came from, and that is not always true
> by any means.

Oops. Then you have a severe problem in understanding how linux _should_ be
worked with. Obviously you can go and buy some distribution and it may work out
pretty well. But being a serious administrator you _will_ do kernel updates
apart from the distro (sooner or later). If there are drivers in a newer kernel
release, that do not work any longer, compared to an older release, you should
say so. Best place would be the listed maintainer. Because if it doesn't do the
job for your configuration any longer, chances are high others were hit, too.
The maintainer cannot know, if you do not tell him.
I guess we have the mutual agreement here, that maintained drivers should get
better not worse through the revisions. This means configurations once
successful are not meant to break later on.
This basically is what I intended to say with "being happy" :-)

> Also, in a world where one values stability, being able to rev
> backward is quite important also.  The lack of software package tools
> like rpm being able to "downgrade" software easily is a serious one in
> my mind.  What do you do when you install something major and it
> breaks, but it has interdependencies on multiple other things?

This isn't kernel-related, is it? You can always boot the previous kernel (and
drivers), if you updated correctly. I am talking about revision-updates, not
major updates (like from 2.0 to 2.2). Major update will break mostly. But -
frankly spoken - it should. If you are using a bunch of very old binaries, you
should be driven to update these anyway.

> I don't object to producing well-tested full sets of driver source that
> go with kernel versions, I just don't want them to be tied if I have
> a need to pull something apart (a driver from one version and from
> another are the only ones that run stably), which frankly happens too
> often for my taste.  Even if it didn't I'd still want it as much as
> possibly for the fall-back options it provides.

Can you give a striking example for this? (old driver ok, new driver broken,
both included in kernel releases)

> I didn't go far enough with my comment.  I should have said that the
> lack of a common driver framework base that works between *all* (or nearly
> all, some obsolescence is ok) their Windows versions is a problem.

Which means: it does not work there.

> In the consumer line (win3.1/95/98/me) where they were keeping very good
> compatibility across some kinds of drivers (it had it's problems, to be
> sure), they were trying the hardest because they recognized that the
> most important thing was to just have things work in the first place.

I honour they were _trying_, only it was not successful. I wrote drivers for
all noted versions above and can tell you: it is a _mess_.

> NOTE:  I'm not endorsing their overall API convenience (driver or
> Win16/Win32), stability, suitability, merchantability, whatever, I'm
> just talking about drivers and their distribution model at the moment.

Ok, lets put it this way: they use a completely split up model of driver
maintenance to get the most out of the market (anybody can release anything at
anytime and any quality and interoperability), and therefore everything tends
to be severly broken and complex in administration. You have to draw the latest
drivers for your graphics card, scsi card, scanner, printer, USB equipment,
even monitor from the respective source (manufacturer), install it seperately
and pray it does not shoot your last driver installation from different
hardware component - which it does in a significant percentage. And if you
upgrade from (e.g.) win95 to win98, you will draw all drivers again and
completely reinstall everything.
To tell the pure truth: nobody cares about anything on w*indoze.

> > Reading between your lines, I can well see that you are most probably
> > talking about closed-source linux-drivers breaking with permanently
> > released new kernel revisions. But, in fact, this is the closed-source
> > phenomenon, and _not_ linux.
> 
> No, though I don't object to closed-source in general per se.  I hate it
> for myself and businesses I've worked for because I like to be able to
> fix/improve/whatever code, but I recognize that the majority of users
> out there would never touch code.

They _should_ not do that. And they _need_ not do that today. The distros
really got very far on the path to the real lusers, that don't know much and
don't want to learn anything. This is ok.
I mean have you had a look at the latest distros, I found it amazing how far
things have already come. You can install client systems under linux in 20% of
the time you would need for _any_ windows.

> My general feeling is that binary drivers are ok/should be supported well
> across versions since that is the thing you load in at boot/bring-system-
> up time.  Having separate (and usually many) step(s) to getting a driver
> and having it load on your system is a major and I'm thinking artificial
> pain.

I have learned something over the recent years: I guess RMS pointed in the
right direction. I _don't_ think binary drivers are ok. I want to control my
environment, and don't let _anybody_ control it _for_ me. And if something goes
wrong, I have a look. And if I am too dumb, I can ask somebody who isn't. And
there may be a lot of those.

> In general my argument stems from the same basis that the kernel/user-level
> interface does:  keeping interfaces stable and removing packaging/bundling
> requirements across major boundaries almost always yield a win somewhere.

Carl Sagan "The Demon Haunted World":
"If there is a chain of argument every link in the chain must work."

> In the case of MS and drivers, the win they have in convenience to end-users
> of all types, and the ability to mix and match drivers forward or backward
> up to some limitations in API revisions.

Ah, yes. Indeed I am one of those thinking that a driver should _work_, I do
not measure its quality on the number of versions available - and loadable in
my environment. If I have to this, the driver is _broken_.
But yes, you are right: 99% of all w-users obviously think internet is designed
for downloading the latest w-drivers, and it is a definitive must to have all
revisions to find one working.

> > I tend to believe we could just wait another two MS cycles to have even
> > the biggest MS-fans converted to kernel-hackers, only because of being
> > real fed up with the brilliant, long term driver design.
> 
> Most people will never touch code or a compiler, and just want a simple
> obvious formula or even to have the system automagically do the Right
> Things for you.  Even many programmers have limitations of curiousity or
> energy, and it isn't a bad thing to organize even core system things to
> be easy.

I talked about the developers, but even talking about users, I do believe that
...

> MS may be stupid/annoying in other ways, but they know this and have
> moved toward it, albeit sluggishly at times.

... it is not their ultimate goal to let MS control their lives. You think this
is over-estimation? They are already handing out electronic _passports_, sorry,
but in my understanding of democracy this is not done by commercial companies.
This is what a state is all about - knowing and identifying its citizens - and
no one _else_.

MS is not stupid. I won't tell you what it is though, find out yourself.

Regards,
Stephan

Re: Loadable drivers [was SMP/cc Cluster description ]

[Kai Henningsen]

skraw@ithnet.com (Stephan von Krawczynski)  wrote on 06.12.01 in <20011206173455.104b6a02.skraw@ithnet.com>:

> I have learned something over the recent years: I guess RMS pointed in the
> right direction. I _don't_ think binary drivers are ok. I want to control my
> environment, and don't let _anybody_ control it _for_ me. And if something
> goes wrong, I have a look. And if I am too dumb, I can ask somebody who
> isn't. And there may be a lot of those.

And it is absolutely amazing what you *can* do, if you have the soure,  
what you never expected to be able to do, but because you *do* have the  
source, you just *have* to look at the problem area ... and poof! there's  
something that certainly doesn't look right, maybe if you just try to  
change this little bit ...

And thus you learn something new which you wouldn't even have tried with  
closed source.

It's not only this. I remember when I first tried to get PPP to work - no,  
wait, back then it was SLIP. (Slightly before Linux 1.0, I think.) It just  
wouldn't work, and I had no idea why. Obviously I was doing something  
wrong, but what?

So I looked at the relevant kernel part. Still rather unclear, but the  
data has to go through *here* ... now suppose I insert a printk there, and  
one there, and then reboot and retry and watch syslog ... aha! (Well,  
actually, it took me several passes, and I don't remember what the problem  
turned out to be except it wasn't a kernel bug.)

MfG Kai

Re: Loadable drivers [was SMP/cc Cluster description ]

[erich]

Stephan von Krawczynski <skraw@ithnet.com> wrote:

> erich@uruk.org wrote:
> 
> > Hmm.  There is little fundamentally incompatible with having splits in
> > the core kernel and sets of drivers, and getting most of what you want
> > here.
> 
> There is. You double the administrational work. Most updates are
> performed because you want the latest bugfixes. If you get more bugfixes
> than you intended - lucky.
> Currently you draw the latest kernel-tree (or the latest _you_ like to
> use), use your old .config and it works out - as long as you do not use
> closed-source drivers.

And if something breaks out of it - unlucky?

I again think your statement that "...and it works out" is naive.

See below for more.


> If you split things up, you draw _two_ archives, compile and install
> both.

Actually, that's something of my point.

I think the control to install one or the other, certainly for the
systems I want to be most reliable (firewalls/other servers), is of
high benefit, hence my proposing this in the first place.

All these things tied together means they all stand or fall together
for most people, it's only people like many on this list or other
kernel hackers that end up being able to do anything about it.

There are many important reasons why those in charge of so many
production computing environments around the world try to, when
faced with some broken component, want to *only* change that one
and not all or even a large subset of them.  It is risky.

Even if you want to upgrade wholesale, then the ability to rev
backward something that doesn't work in the new set, but you know
works in the old form, is very valuable.


> Oops. Then you have a severe problem in understanding how linux _should_
> be worked with. Obviously you can go and buy some distribution and it
> may work out pretty well. But being a serious administrator you _will_
> do kernel updates apart from the distro (sooner or later). If there are
> drivers in a newer kernel release, that do not work any longer, compared
> to an older release, you should say so. Best place would be the listed
> maintainer. Because if it doesn't do the job for your configuration any
> longer, chances are high others were hit, too.
> The maintainer cannot know, if you do not tell him.
> I guess we have the mutual agreement here, that maintained drivers
> should get better not worse through the revisions. This means
> configurations once successful are not meant to break later on.
> This basically is what I intended to say with "being happy" :-)

OK, you're telling me that people "should fix it", but my point, and
one I haven't heard any contradiction to yet, is that often enough
people don't.

So, the response has been to hold the drivers close in so when
interfaces change/a bug is found, they can all be changed together.
I think there is some merit to that when you find a bug, but
otherwise this becomes a statistical shooting gallery.

My reasoning?  Each time driver code is touched, in particular without
testing done afterward, there is some probability of another bug
being induced.


> > I don't object to producing well-tested full sets of driver source that
> > go with kernel versions, I just don't want them to be tied if I have
> > a need to pull something apart (a driver from one version and from
> > another are the only ones that run stably), which frankly happens too
> > often for my taste.  Even if it didn't I'd still want it as much as
> > possibly for the fall-back options it provides.
> 
> Can you give a striking example for this? (old driver ok, new driver
> broken, both included in kernel releases)

Not really striking per se, but 2 off of the top of my head I've had to
deal with:

  --  integrated 10/100 MBit Enet network in a SiS735 motherboard,
      effectively SiS900 chipset.  Worked fine in later RH 7.1
      kernel (2.4.3-rhat), but not with 7.2 kernel(s) (2.4.7-rhat &
      2.4.9-rhat).
  --  drivers for the Lucent pcmcia Orinoco series of cards work/
      don't work across different revisions from 2.4.0-ish -> present.


Yes, I've reported various problems with other bits before, and
debugged some of my own, submitted patches, etc.

You can make the argument that some of them are young drivers/etc., but
that just proves my point that I want the control, not just for me, but
for others to easily load across drivers across different kernel
versions.

My issue with the current process is that it's developer-centric, and
though I've been able to work past these problems, none of the non-
kernel hackers I've known would generally care to.

In fact, several others who I know with similar hardware just gave up
when I told them they had to install a kernel/hacked driver patches.


> Ok, lets put it this way: they use a completely split up model of
> driver maintenance to get the most out of the market (anybody can
...
> hardware component - which it does in a significant percentage. And if
> you upgrade from (e.g.) win95 to win98, you will draw all drivers again
> and completely reinstall everything.
> 
> To tell the pure truth: nobody cares about anything on w*indoze.

I can't believe you just said that.  Maybe nobody *on this list*, but
in general, this makes it hard to take you at all seriously.


...
> > My general feeling is that binary drivers are ok/should be supported well
> > across versions since that is the thing you load in at boot/bring-system-
> > up time.  Having separate (and usually many) step(s) to getting a driver
> > and having it load on your system is a major and I'm thinking artificial
> > pain.
> 
> I have learned something over the recent years: I guess RMS pointed in the
> right direction. I _don't_ think binary drivers are ok. I want to control
> my environment, and don't let _anybody_ control it _for_ me. And if
> something goes wrong, I have a look. And if I am too dumb, I can ask
> somebody who isn't. And there may be a lot of those.

Huh?  I didn't ask you to use them.

I'm just pointing out that binary drivers are the end-product (you don't
"insmod driver.c"), so it's kind of hard to just hand-wave them out of
existence.  Not everyone has a fully self-hosted system with compilers
just sitting around to do your bidding.

Also, the model you present above is only suitable for a small fraction
of users.  It is irrelevant to most users if the code is fixable.  If
it doesn't work as is and there is no easy other thing they can try,
it is effectively broken.


--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Re: Loadable drivers [was SMP/cc Cluster description ]

[Stephan von Krawczynski]

On Thu, 06 Dec 2001 16:37:36 -0800
erich@uruk.org wrote:

> > If you split things up, you draw _two_ archives, compile and install
> > both.
> 
> Actually, that's something of my point.
> 
> I think the control to install one or the other, certainly for the
> systems I want to be most reliable (firewalls/other servers), is of
> high benefit, hence my proposing this in the first place.

Only in an environment with little evolution going on. But whats the use of
updating anyway, then?
Another thing that really jumps right into my face from your arguments:
you say "administrator needs to have a way to change core kernel, without
changing any drivers, and is not able to read and understand the source of both
core and drivers, so it is best for him to change as little as possible."
But wait: how does _he_ know then, that a standard update (kernel _and_
drivers, as currently) will not work. I mean, you trust the work of some people
building one set of core and drivers, but do not trust the same people to give
you a newer set of core and drivers? And all this, although you are not able or
willing to have a look at the _sources_ to really make sure.
I am definitely not fond of the proposed MS idea, that anybody can administrate
any server, he only needs to know how to perform mouse-clicks. If you can't
drive a car, you should probably not take the seat at the drivers side.

> All these things tied together means they all stand or fall together
> for most people, it's only people like many on this list or other
> kernel hackers that end up being able to do anything about it.

What is wrong with this idea? If you are joe-user, then take your preferred
distro and live with it (quite well nowadays). If you are an experienced user,
then tune your setup according to your needs. If you are really good at it,
then get yourself a job as an administrator for servers people work and rely on
- and read sources. And last, if you have some ideas, go ahead and write source
and give it to the public to verify and tune its quality. And if your real good
it may well be, that god will take your code and you suddenly become one of the
kernel-driver-maintainers. This is a pretty simple and straight-forward world.
And there is noone coming round the corner and tell you: "btw, we changed the
world just yesterday, everything is easy, only you should throw away your old
code."

> There are many important reasons why those in charge of so many
> production computing environments around the world try to, when
> faced with some broken component, want to *only* change that one
> and not all or even a large subset of them.  It is risky.

I really don't get it: if your system works, why upgrade? if your system does
not work: why did you give it to the people in the first place?

> Even if you want to upgrade wholesale, then the ability to rev
> backward something that doesn't work in the new set, but you know
> works in the old form, is very valuable.

Ever heard of lilo? It can boot multiple kernel-images. And guess what: they
are all finding their corresponding modules if you do it right.

> OK, you're telling me that people "should fix it", but my point, and
> one I haven't heard any contradiction to yet, is that often enough
> people don't.

Interesting. I in fact never heard of anyone complaining about a driver (that
is maintained) that does not get fixed if you tell the people about a problem.
If they don't have the hardware to test, simply give it to them.

> My reasoning?  Each time driver code is touched, in particular without
> testing done afterward, there is some probability of another bug
> being induced.

There is always another bug. This is fundamental in software development. This
is why it is really important to talk about bugs you came across. If you do not
use new code, no code will ever become better.

> > Can you give a striking example for this? (old driver ok, new driver
> > broken, both included in kernel releases)
> 
> Not really striking per se, but 2 off of the top of my head I've had to
> deal with:
> 
>   --  integrated 10/100 MBit Enet network in a SiS735 motherboard,
>       effectively SiS900 chipset.  Worked fine in later RH 7.1
>       kernel (2.4.3-rhat), but not with 7.2 kernel(s) (2.4.7-rhat &
>       2.4.9-rhat).

Okay, this is my personal tip for administrators: do not use distro-kernels, do
use linus-kernels, and do not update to very fresh ones. If you wanna be
brilliant, read LKML to know whats going on. This is no hint for joe-user.

>   --  drivers for the Lucent pcmcia Orinoco series of cards work/
>       don't work across different revisions from 2.4.0-ish -> present.

Haha. This is a real good example for real life not matching my ideas, perhaps
david hinds and linus should find some solution for it. If you want to have a
working set of lucent pcmcia you need to draw the latest pcmcia-package and
tools from david and install them on top of standard kernel. This has worked
since the civil war, believe me, I use it everyday with _lots_ of different
kernels and hardware setups. Only annoying part: it is not part of the kernel.
Therefore it is pretty interesting to hear you complaining about a _split away_
drivers' issue. :-)

> My issue with the current process is that it's developer-centric, and
> though I've been able to work past these problems, none of the non-
> kernel hackers I've known would generally care to.

Tell you what, this happened about 2 months ago:
I build a nice and simple hardware for a desktop. It contained Asus Board,
PIII-500, 512MB, ATI graphics card, IDE-10G-hd and ATAPI cdrom, keyboard,
PS/2-mouse and soundblaster . It didn't look all that non-standard.
I tried to install:

win95-orig: 
install fails right away, win95 cannot CD-boot

win98:
doesn't install with 512 MB, install fails.

win ME:
doesn't even get recognised by the BIOS as bootable CD

win 2K:
hangs in hardware detection cycle, I switched it off after 3 hours of continous
hd seeking at about 50%.

SuSE 7.2 (Linux):
install takes about 15 minutes, everything works.

What do we learn of this: win scales everybody down to joe-user-level, and you
cannot do a thing against it. I guess it is definitely not developer-centric. 

> > To tell the pure truth: nobody cares about anything on w*indoze.
> 
> I can't believe you just said that.  Maybe nobody *on this list*, but
> in general, this makes it hard to take you at all seriously.

I have seen to many things on windows to come to a different conclusion.

> Also, the model you present above is only suitable for a small fraction
> of users.  It is irrelevant to most users if the code is fixable.  If
> it doesn't work as is and there is no easy other thing they can try,
> it is effectively broken.

Again, you are talking about joe-user here. If joe-user has a problem, he
should call up the distro-hotline he bought, they will help him with
installation problems (mostly for free). He does not need a compiler or
anything.

And the experienced guy (or girl, hey there are some :-) will have a large
number of options available, including the source and his (her) brain.

Regards,
Stephan

Re: Loadable drivers [was SMP/cc Cluster description ]

[Keith Owens]

On Wed, 05 Dec 2001 11:40:07 -0800, 
erich@uruk.org wrote:
>Build a framework for an external "drivers" source/binary tree that can be
>(via a single command) rebuilt for a different Linux kernel version.  This
>is arguably a Linux distribution thing.

kbuild 2.5.  Already done.

Re: Loadable drivers [was SMP/cc Cluster description ]

[erich]

Keith Owens <kaos@ocs.com.au> wrote:

> On Wed, 05 Dec 2001 11:40:07 -0800, 
> erich@uruk.org wrote:
> >Build a framework for an external "drivers" source/binary tree that can be
> >(via a single command) rebuilt for a different Linux kernel version.  This
> >is arguably a Linux distribution thing.
> 
> kbuild 2.5.  Already done.

Cool.  Thanks for the good work guys!

[dang, now I have to go off and learn all about kbuild 2.5...]

--
    Erich Stefan Boleyn     <erich@uruk.org>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"