Linux/Pro [was Re: Coding style - a non-issue]

Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Fri, Nov 30, 2001 at 02:17:33PM -0800, Andrew Morton wrote:
> Larry McVoy wrote:
> > 
> > Linux isn't there yet
> > and unless the development model changes somewhat, I'll stand behind my
> > belief that it is unlikely to ever get there.
> 
> I am (genuinely) interested in what changes you think are needed.

Well I have an opinion, not sure if it will be well received or not,
but here goes.

There is a choice which needs to be made up front, and that is this:

    do you want to try and turn the Linux kernel hackers into Sun style
    hackers or do you want to try something else?

This assumes we have agreement that there is a difference between the
two camps.  I'd rather not get into a "this way is better than that way"
discussion, let's just postulate that the Sun way has some pros/cons
and so do the Linux way.

If you want to try and make Linux people work like Sun people, I think
that's going to be tough.  First of all, Sun has a pretty small kernel
group, they work closely with each other, and they are full time,
highly paid, professionals working with a culture that is intolerant of
anything but the best.  It's a cool place to be, to learn, but I think
it is virtually impossible to replicate in a distributed team, with way
more people, who are not paid the same or working in the same way.

Again, let's not argue the point, let's postulate for the time being
that the Linux guys aren't going to work like the Sun guys any time soon.

So what's the problem?  The Sun guys fix more bugs, handle more corner
cases, and scale up better (Linux is still better on the uniprocessors
but the gap has narrowed considerably; Sun is getting better faster than
Linux is getting better, performance wise.  That's a bit unfair because
Linux had, and has, better absolute numbers so there was less room to
improve, but the point is that Sun is catching up fast.)

As the source base increases in size, handles more devices, runs on more
platforms, etc., it gets harder and harder to make the OS be reliable.
Anyone can make a small amount of code work well, it's exponentially
more difficult to make a large amount of code work well.  There are lots
of studies which show this to be true, the mythical man month is a good
starting place.

OK, so it sounds like I'm saying that the Linux guys are lame, Sun is
great, and there isn't any chance that Linux is going to be as good
as Solaris.  That's not quite what I'm saying.  *If* you want to play
by the same rules as Sun, i.e., develop and build things the same way,
then that is what I'm saying.  The Linux team will never be as good
as the Sun team unless the Sun team gets a lot worse.  I think that's
a fact of life, Sun has 100s of millions of dollars a year going into
software development.  ESR can spout off all he likes, but there is no
way a team of people working for fun is going to compete with that.

On the other hand, there is perhaps a way Linux could be better.  But it
requires changing the rules quite a bit.

Instead of trying to make the Linux hackers compete with the Sun hackers,
what would happen if you architected your way around the problem?
For example, suppose I said we need to have a smaller, faster, more
reliable uniprocessor kernel.  Suppose I could wave a magic wand and
make SMP go away (I can't, but bear with me for a second).  The problem
space just got at least an order of magnitude less complex than what Sun
deals with.  I think they are up to 32-64 way SMP and you can imagine,
I hope, the additional complexity that added.  OK, so *if* uniprocessor
was the only thing we had to worry about, or say 2-4 way SMP with just
a handful of locks, then can we agree that it is a lot more likely that
we could produce a kernel which was in every way as good or better than
Sun's kernel, on the same platform?  If the answer is yes, keep reading,
if the answer is no, then we're done because I don't know what to do if
we can't get that far.

For the sake of discussion, let's assume that you buy what I am saying
so far.  And let's say that we agree that if you were to toss the SMP
stuff then we have a good chance at making a reliable kernel, albeit
an uninteresting one when compared to big boxes.  If you want me to go
into what I think it would take to do that, I will.

The problem is that we can't ignore the SMP issues, it drives hardware
sales, gets press, it's cool, etc.  We have to have both but the problem
is that if we have both we really need Sun's level of professionalism
to make it work, and it isn't realistic to expect that from a bunch of
underpaid (or not at all paid) Linux hackers.

Here's how you get both.  Fork the development efforts into the SMP part
and the uniprocessor part.  The uniprocessor focus is on reliability,
stability, performance.  The SMP part is a whole new development effort,
which is architected in such a way as to avoid the complexity of a
traditional SMP implementation.

The uniprocessor team owns the core architecture of the system.  The
abstractions provided, the baseline code, etc., that's all uni.  The
focus there is a small, fast, stable kernel.

The SMP team doesn't get to touch the core code, or at least there is 
a very strong feedback loop against.  In private converstations, we've
started talking about the "punch in the nose" feedback loop, which means
while it may be necessary to touch generic code for the benefit of SMP,
someone has to feel strongly enough about it that they well sacrifice
their nose.

It would seem like I haven't solved anything here, just painted a nice
but impossible picture.  Maybe.  I've actually thought long and hard 
about the approach needed to scale up without touching all the code
and it is radically different from the traditional way (i.e., how
Sun, SGI, Sequent, etc., did it).  If you are interested in that, I'll
talk about it but I'll wait to see if anyone cares.

The summary over all of this is:

    If you want to solve all the problems that Sun does, run on the same
    range of UP to big SMP, Linux is never going to be as reliable as 
    Solaris.  My opinion, of course, but one that is starting to gain
    some traction as the OS becomes more complex.

    That leaves you with a choice: either give up on some things,
    magically turn the Linux hackers into Sun hackers, or
    architect your way around the problem.

My vote is the last one, it fits better with the Linux effort, the answer
is way more cool than anything Sun or anyone else has done, and it lets
you have a simple mainline kernel source base.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Davide Libenzi]

On Fri, 30 Nov 2001, Larry McVoy wrote:

[ A lot of stuff Pro-Sun ]

Wait a minute.
Wasn't it you that were screaming against Sun, leaving their team because
their SMP decisions about scaling sucked, because their OS was bloated
like hell with sync spinlocks, saying that they tried to make it scale but
they failed miserably ?
What is changed now to make Solaris, a fairly vanishing OS, to be the
reference OS/devmodel for every kernel developer ?
Wasn't it you that were saying that Linux will never scale with more than
2 CPUs ?
Isn't Linux SMP improved from the very first implementation ?
Isn't Linux SMP improved from the very first implementation w/out losing
reliability ?
Why you don't try to compare 2.0.36 with 2.4.x let's say on a 8 way SMP ?
Why should it stop improving ?
Because you know that adding fine grained spinlocks will make the OS
complex to maintain and bloated ... like it was Solaris before you
suddendly changed your mind.


<YOUR QUOTE>
>     Then people want more performance.  So they thread some more and now
>     the locks aren't 1:1 to the objects.  What a lock covers starts to
>     become fuzzy.  Thinks break down quickly after this because what
>     happens is that it becomes unclear if you are covered or not and
>     it's too much work to figure it out, so each time a thing is added
>     to the kernel, it comes with a lock.  Before long, your 10 or 20
>     locks are 3000 or more like what Solaris has.  This is really bad,
>     it hurts performance in far reaching ways and it is impossible to
>     undo.
</YOUR QUOTE>

I kindly agree with this, just curious to understand which kind of amazing
architectural solution Solaris took to be a reference for SMP
development/scaling.




- Davide

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Fri, Nov 30, 2001 at 05:13:38PM -0800, Davide Libenzi wrote:
> On Fri, 30 Nov 2001, Larry McVoy wrote:
> Wait a minute.
> Wasn't it you that were screaming against Sun, leaving their team because
> their SMP decisions about scaling sucked, because their OS was bloated
> like hell with sync spinlocks, saying that they tried to make it scale but
> they failed miserably ?

Yup, that's me, guilty on all charges.

> What is changed now to make Solaris, a fairly vanishing OS, to be the
> reference OS/devmodel for every kernel developer ?

It's not.  I never said that we should solve the same problems the same
way that Sun did, go back and read the posting.

> Wasn't it you that were saying that Linux will never scale with more than
> 2 CPUs ?

No, that wasn't me.  I said it shouldn't scale beyond 4 cpus.  I'd be pretty
lame if I said it couldn't scale with more than 2.  Should != could.

> Because you know that adding fine grained spinlocks will make the OS
> complex to maintain and bloated ... like it was Solaris before you
> suddendly changed your mind.

Sorry it came out like that, I haven't changed my mind one bit.

> <YOUR QUOTE>
> >     Then people want more performance.  So they thread some more and now
> >     the locks aren't 1:1 to the objects.  What a lock covers starts to
> >     become fuzzy.  Thinks break down quickly after this because what
> >     happens is that it becomes unclear if you are covered or not and
> >     it's too much work to figure it out, so each time a thing is added
> >     to the kernel, it comes with a lock.  Before long, your 10 or 20
> >     locks are 3000 or more like what Solaris has.  This is really bad,
> >     it hurts performance in far reaching ways and it is impossible to
> >     undo.
> </YOUR QUOTE>
> 
> I kindly agree with this, just curious to understand which kind of amazing
> architectural solution Solaris took to be a reference for SMP
> development/scaling.

OK, so you got the wrong message.  I do _not_ like the approach Sun took,
it's a minor miracle that they are able to make Solaris work as well as
it works given the design decisions they made.

What I do like is Sun's engineering culture.  They work hard, they don't
back away from the corner cases, they have high standards.  All of which
and more are, in my opinion, a requirement to try and solve the problems
the way they solved them.

So the problem I've been stewing on is how you go about scaling the OS
in a way that doesn't require all those hot shot sun engineers to make
it work and maintain it.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Andrew Morton]

Davide Libenzi wrote:
> 
> On Fri, 30 Nov 2001, Larry McVoy wrote:
> 
> [ A lot of stuff Pro-Sun ]
> 
> Wait a minute.

umm..   Let's try to keep moving forward here.

Larry appears to be referring to the proposal sometimes
known as ccClusters.  I'd ask him a couple of followup questions:

Is there any precedent for the ccCluster idea, which would
increase confidence that it'll actually work?

Will it run well on existing hardware, or are changes needed?

You're assuming that our current development processes are
sufficient for development of a great 1-to-4-way kernel, and
that the biggest force against that is the introduction of
fine-grained locking.   Are you sure about this?  Do you see
ways in which the uniprocessor team can improve?

My take is that there's a sort of centralism in the kernel where
key people get atracted into mm/*.c, fs/*.c, net/most_everything
and kernel/*.c while other great wilderness of the tree (with
honourable exceptions) get moldier.  How to address that?

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Stephan von Krawczynski]

On Fri, 30 Nov 2001 15:57:40 -0800
Larry McVoy <lm@bitmover.com> wrote:

> [...]
> Here's how you get both.  Fork the development efforts into the SMP part
> and the uniprocessor part.  The uniprocessor focus is on reliability,
> stability, performance.  The SMP part is a whole new development effort,
> which is architected in such a way as to avoid the complexity of a
> traditional SMP implementation.
> 
> The uniprocessor team owns the core architecture of the system.  The
> abstractions provided, the baseline code, etc., that's all uni.  The
> focus there is a small, fast, stable kernel.
> 
> The SMP team doesn't get to touch the core code, or at least there is 
> a very strong feedback loop against.  In private converstations, we've
> started talking about the "punch in the nose" feedback loop, which means
> while it may be necessary to touch generic code for the benefit of SMP,
> someone has to feel strongly enough about it that they well sacrifice
> their nose.

Hi Larry,

let me first tell you this: I am only stating my very personal opinion here and
am probably no guy of your experience or insight in high-tech delevopment
groups. But I saw the whole business for quite some years now, so my thinking:

Your proposal is the wrong way, because:
1) You split up "the crew". Whatever you may call "the crew" here, they all
have one thing in common: they are working on the _same_ project _and_ (very
important either) _one_ man has the final decision. If you look at _any_ other
OS developed by quite a number of completely different people you have to admit
one thing: everything was busted when they began to split up in different
"branches". That simply does not work out. I am only referring to simple human
interaction and communication here, nothing to do with the technical issues.
One of the basic reasons for the success of Linux so far is the collaborated
work of a damn lot of people on the _same_ project.

2) I cannot see the _need_ for such a "team-splitup". If you say one team (core
team) has the "last word" about the baseline code, what do you think will
happen if "necessary" code changes for the second team will be refused? Simple:
they will fork a completely new linux-tree. And this is _the_ end. If you were
to write a driver, what tree will you choose after that? I mean you are dealing
with the main reason why people like linux, here. And not: SCO Unix 3.x,
Unixware, Solaris, Minix, AT&T Unix, Xenix, HPUX, AIX, BSD, NetBSD, BSDi,
Solaris-for-Intel, make-my-day ... (all registered trademark of their
respective owners, which have all low interaction capabilities)
I don't want that, do we want that?

3) Your SMP team (you are talking of true high scaled SMP here) has a very big
problem: it has _very_ few users (compared to UP) and even less developers with
the _hardware_ you need for something like that. I mean you are not talking
about buying an Asus Board and 2 PIII here, you talk about serious, expensive
hardware here. How many people have this at home, or at work for free playing?
Well, see what I mean?

4) Warning, this is the hard stuff!
Ok, so you are fond of SUN. Well, me too. But I am not completely blind, not
yet :-) So I must tell you, if Solaris were the real big hit, then why its
Intel-Version is virtualy been eaten up on the market (the _buying_ market out
there) by linux? The last time I met a guy seriously talking about Solaris
(x86) being "better" than Linux was about three years ago. And btw, this was
_the_ last guy talking about it at all. So lets simply assume this: the Solaris
UP market is already gone, if you are talking about the _mass_ market. This
parrot is _deceased_! It is plain dead.
Now you are dealing with another problem: SUN (being kind of the last resort of
a widespread and really working commercial unix) will have a lot of work to do
in the future to keep up with the simple mass of software and application
coming in for linux, only because it is even _more_ widespread today than
Solaris has ever been. And it is _real_ cheap, and you get it _everywhere_. And
everybody owns a box on which you can use it.
This is going to get very hard for SUN, if they do not enter a stage of
complete rethinking what is going on out there.
To make that clear: _nobody_ here _fights_ against SUN or Solaris. Quite a few
of us like it very much. But this is a commercial product. It needs to be sold
to survive - and that is going to be a future problem. SUN will not survive
only building the high-power low-mass computer. CRAY did not either. So maybe
the real good choice would be this: let the good parts of Solaris (and maybe
its SMP features) migrate into linux. This does not mean they should lay off
the staff, just on contrary: these are brilliant people, let them do what they
can do best, but keep an eye on the market. We are in the state of: the market
_demands_ linux. It has already become a well-known trademark, I tend to
believe better-known than Solaris. Somehow one has the feeling they indeed know
whats coming (fs), but have not come to the final (and hard to take)
conclusion.
And to clarify: I am not taking any drugs. This is serious. I have the strong
feeling, that there is already a big player out there, that has learnt a hard
lesson: IBM - and the lesson named OS/2.
When OS/2 came out, I told the people: if they are not going to give it away
for free, they will not make it. And they didn't. Indeed I did not expect them
to learn at all (simply because big companies are mostly not quick-movers), but
they do really astonishing things nowadays. I have the strong feeling the
management is at least as brilliant as the Solaris developpers and worth every
buck, too.

But this is only my small voice, and quite possibly only few are listening, if
any ...

Regards,
Stephan

PS: Is this really a topic for a kernel-mailing-list?

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Fri, Nov 30, 2001 at 06:17:43PM -0800, Davide Libenzi wrote:
> > It's not.  I never said that we should solve the same problems the same
> > way that Sun did, go back and read the posting.
> 
> This is your quote Larry :
> 
> <>
> If you want to try and make Linux people work like Sun people, I think
> that's going to be tough.  First of all, Sun has a pretty small kernel
> group, they work closely with each other, and they are full time,
> highly paid, professionals working with a culture that is intolerant of
> anything but the best.  It's a cool place to be, to learn, but I think
> it is virtually impossible to replicate in a distributed team, with way
> more people, who are not paid the same or working in the same way.
> <>

I'm sorry, I'm lost.  You are quoting what I said, that's what I said
said, so what's the point?  yes, for the third time, that's what I said
and that's what I meant.

> So, if these guys are smart, work hard and are professionals, why did they
> take bad design decisions ?
> Why didn't they implemented different solutions like, let's say "multiple
> independent OSs running on clusters of 4 CPUs" ?

Because, just like the prevailing wisdom in the Linux hackers, they thought
it would be relatively straightforward to get SMP to work.  They started at
2, went to 4, etc., etc.  Noone ever asked them to go from 1 to 100 in one
shot.  It was always incremental.

I recently talked over the approach I have in mind with the architect of
Sun's multithreaded kernel, the guy who started and guided that effort at
Sun.  He agreed that if he had thought of my approach he would have done
it that way.  We both agreed it was unlikely that anyone would think of
that approach without first having done it the hard way.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Davide Libenzi]

On Fri, 30 Nov 2001, Larry McVoy wrote:

> On Fri, Nov 30, 2001 at 05:13:38PM -0800, Davide Libenzi wrote:
> > On Fri, 30 Nov 2001, Larry McVoy wrote:
> > Wait a minute.
> > Wasn't it you that were screaming against Sun, leaving their team because
> > their SMP decisions about scaling sucked, because their OS was bloated
> > like hell with sync spinlocks, saying that they tried to make it scale but
> > they failed miserably ?
>
> Yup, that's me, guilty on all charges.
>
> > What is changed now to make Solaris, a fairly vanishing OS, to be the
> > reference OS/devmodel for every kernel developer ?
>
> It's not.  I never said that we should solve the same problems the same
> way that Sun did, go back and read the posting.

This is your quote Larry :

<>
If you want to try and make Linux people work like Sun people, I think
that's going to be tough.  First of all, Sun has a pretty small kernel
group, they work closely with each other, and they are full time,
highly paid, professionals working with a culture that is intolerant of
anything but the best.  It's a cool place to be, to learn, but I think
it is virtually impossible to replicate in a distributed team, with way
more people, who are not paid the same or working in the same way.
<>

So, if these guys are smart, work hard and are professionals, why did they
take bad design decisions ?
Why didn't they implemented different solutions like, let's say "multiple
independent OSs running on clusters of 4 CPUs" ?
What we really have to like about Sun ?
Me personally, if I've to choose, I'll take the logo.




- Davide

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Mike Fedyk]

On Sat, Dec 01, 2001 at 02:21:57AM +0100, Stephan von Krawczynski wrote:
> _the_ last guy talking about it at all. So lets simply assume this: the Solaris
> UP market is already gone, if you are talking about the _mass_ market. This
> parrot is _deceased_! It is plain dead.

Not completely.  Many people who *need* to develop for solaris on sun
hardware will run solaris x86 at home (or maybe on a corporate laptop) to be
able to work at home and test the software there too.  I know one such
developer myself, there have to be more.

> So maybe
> the real good choice would be this: let the good parts of Solaris (and maybe
> its SMP features) migrate into linux. 

Before 2.3 and 2.4 there probably would've been much more contention against
something like this.  Even now with large SMP scalability goals, I think it
would be hard to get something like this to be accepted into Linux.

mf

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Mike Fedyk]

On Fri, Nov 30, 2001 at 03:57:40PM -0800, Larry McVoy wrote:
> Here's how you get both.  Fork the development efforts into the SMP part
> and the uniprocessor part.  

Basically with linux, and enough #ifdef's you end up with both in one.  IIUC

What would be nice is UP only drivers for initial release. Write a driver
module that says "I'm made for UP and haven't been tested with SMP/HighMEM"
so if you try to compile it with those features it would break with a
helpful error message.

What would be interesting would be SMP with support for UP.  The UP only
module would be inserted into a SMP kernel, but would only work inside one
processor, and would have source compatibility with both UP ans SMP kernels.
No non-UP locking required.

Is something like this possible?

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Gérard Roudier]

On Sat, 1 Dec 2001, Alan Cox wrote:

> > > Wasn't it you that were saying that Linux will never scale with more than
> > > 2 CPUs ?
> >
> > No, that wasn't me.  I said it shouldn't scale beyond 4 cpus.  I'd be pretty
> > lame if I said it couldn't scale with more than 2.  Should != could.
>
> Question: What happens when people stick 8 threads of execution on a die with
> a single L2 cache ?

As long as we will not have clean asynchronous mechanisms available from
user land, some applications will have to use more threads of execution
than needed, even with programmers that aren't thread-maniac.

Response to your question: If the problem is to optimize IOs against 8
slow devices using synchronous IO APIs , you will get far better
performances. :-)

  Gérard.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> sufficient for development of a great 1-to-4-way kernel, and
> that the biggest force against that is the introduction of
> fine-grained locking.   Are you sure about this?  Do you see
> ways in which the uniprocessor team can improve?

ccCluster seems a sane idea to me. I don't by Larry's software engineering
thesis but ccCluster makes sense simply because when you want an efficient
system in computing you get it by not pretending one thing is another.
SMP works best when the processors are not doing anything that interacts
with another CPU.

> key people get atracted into mm/*.c, fs/*.c, net/most_everything
> and kernel/*.c while other great wilderness of the tree (with
> honourable exceptions) get moldier.  How to address that?

Actually there are lots of people who work on the driver code nowdays
notably the janitors. The biggest problem there IMHO is that when it comes
to driver code Linus has no taste, so he keeps accepting driver patches
which IMHO are firmly at the hamburger end of "taste"

Another thing for 2.5 is going to be to weed out the unused and unmaintained
drivers, and either someone fixes them or they go down the comsic toilet and
we pull the flush handle before 2.6 comes out.

Thankfully the scsi layer breakage is going to help no end in the area of 
clockwork 8 bit scsi controllers, which is major culprit number 1. number 2
is probably the audio which is hopefully going to go away with ALSA based
code.

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> > Wasn't it you that were saying that Linux will never scale with more than
> > 2 CPUs ?
> 
> No, that wasn't me.  I said it shouldn't scale beyond 4 cpus.  I'd be pretty
> lame if I said it couldn't scale with more than 2.  Should != could.

Question: What happens when people stick 8 threads of execution on a die with
a single L2 cache ?

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Rik van Riel]

On Fri, 30 Nov 2001, Larry McVoy wrote:
> On Fri, Nov 30, 2001 at 06:17:43PM -0800, Davide Libenzi wrote:

> > So, if these guys are smart, work hard and are professionals, why did they
> > take bad design decisions ?
> > Why didn't they implemented different solutions like, let's say "multiple
> > independent OSs running on clusters of 4 CPUs" ?
>
> Because, just like the prevailing wisdom in the Linux hackers, they
> thought it would be relatively straightforward to get SMP to work.
> They started at 2, went to 4, etc., etc.  Noone ever asked them to go
> from 1 to 100 in one shot.  It was always incremental.

Incremental stuff always breaks and misses out on the corner
cases. The same seems to be true for stuff coming out of
random mutation and biological selection ... natural selection
really doesn't care about corner cases.

This, for example, has always resulted in a VM subsystem which
works nicely under low load but falls apart under high load.
Any efforts to fix that corner case have been met with nothing
but resistance.

Lets face it, if you want to deal with corner cases you don't
want to deal with Linus.

regards,

Rik
-- 
Shortwave goes a long way:  irc.starchat.net  #swl

http://www.surriel.com/		http://distro.conectiva.com/

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Mark Frazer]

Stephan von Krawczynski <skraw@ithnet.com> [01/11/30 20:27]:
> 4) Warning, this is the hard stuff!
> Ok, so you are fond of SUN. Well, me too. But I am not completely blind, not
> yet :-) So I must tell you, if Solaris were the real big hit, then why its
> Intel-Version is virtualy been eaten up on the market (the _buying_ market out
> there) by linux?

I can't say for the O/S buying market.  But I do embedded (pretty large
embedded systems but embedded nonetheless) development and we walked away
from Solaris after comparing the complexity of our first network drivers.

STREAMS:  just say no.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Sat, Dec 01, 2001 at 11:04:30AM -0500, Mark Frazer wrote:
> Stephan von Krawczynski <skraw@ithnet.com> [01/11/30 20:27]:
> > 4) Warning, this is the hard stuff!
> > Ok, so you are fond of SUN. Well, me too. But I am not completely blind, not
> > yet :-) So I must tell you, if Solaris were the real big hit, then why its
> > Intel-Version is virtualy been eaten up on the market (the _buying_ market out
> > there) by linux?
> 
> I can't say for the O/S buying market.  But I do embedded (pretty large
> embedded systems but embedded nonetheless) development and we walked away
> from Solaris after comparing the complexity of our first network drivers.
> 
> STREAMS:  just say no.

Amen to that.  STREAMS would be one of the strongest arguments in favor
of Linus' theory that evolution takes care of it.  STREAMS were done at
Sun by some "architects" who thought they would be better than sockets.
Linus is dead right, on this sort of issue, Linux responds quickly and 
weeds out the crap.  We still have some room for discussion on the 
design issues, but yeah, Sun blew it on this one and refused to admit it.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Victor Yodaiken]

On Sat, Dec 01, 2001 at 10:05:55AM +0000, Alan Cox wrote:
> > sufficient for development of a great 1-to-4-way kernel, and
> > that the biggest force against that is the introduction of
> > fine-grained locking.   Are you sure about this?  Do you see
> > ways in which the uniprocessor team can improve?
> 
> ccCluster seems a sane idea to me. I don't by Larry's software engineering
> thesis but ccCluster makes sense simply because when you want an efficient
> system in computing you get it by not pretending one thing is another.
> SMP works best when the processors are not doing anything that interacts
> with another CPU.

Careful Alan. That sounds suspiciously like a "design principle", and
true macho Linux developers don't need no theoretical stuff like that.
They just slop that code together and see what explodes - pulling their
alchemists hats over their eyes for protection.

> 
> > key people get atracted into mm/*.c, fs/*.c, net/most_everything
> > and kernel/*.c while other great wilderness of the tree (with
> > honourable exceptions) get moldier.  How to address that?
> 
> Actually there are lots of people who work on the driver code nowdays
> notably the janitors. The biggest problem there IMHO is that when it comes
> to driver code Linus has no taste, so he keeps accepting driver patches
> which IMHO are firmly at the hamburger end of "taste"

"Taste" ? Now you want aesthetics as well as theory. I'm horrified.

Technical content: does anyone know the max spinlock depth in Linux 2.5
?

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Ricardo Galli]

> Amen to that. STREAMS would be one of the strongest arguments in favor
> of Linus' theory that evolution takes care of it. STREAMS were done at
> Sun by some "architects" who thought they would be better than sockets.

Wasn't Dennis Ritchie one of the "architects" of STREAMS at Bell Labs?

  A Stream Input-Output System : AT&T Bell Laboratories Technical Journal 
  63,  No. 8 Part 2 (October, 1984)

  (http://cm.bell-labs.com/cm/cs/who/dmr/st.pdf)

--ricardo

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Sat, Dec 01, 2001 at 07:38:42PM +0100, Ricardo Galli wrote:
> > Amen to that. STREAMS would be one of the strongest arguments in favor
> > of Linus' theory that evolution takes care of it. STREAMS were done at
> > Sun by some "architects" who thought they would be better than sockets.
> 
> Wasn't Dennis Ritchie one of the "architects" of STREAMS at Bell Labs?
> 
>   A Stream Input-Output System : AT&T Bell Laboratories Technical Journal 
>   63,  No. 8 Part 2 (October, 1984)

Dennis wrote something called "streams" not be confused with "STREAMS".
He was trying to provide a more flexible mechanism for tty line 
disciplines.  The Sys III guys grabbed it and decided to promote it as 
a networking infrastructure, because System III et al did not have
networking.  Dennis doesn't like the result any better than any of us.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Victor Yodaiken]

On Sat, Dec 01, 2001 at 07:38:42PM +0100, Ricardo Galli wrote:
> > Amen to that. STREAMS would be one of the strongest arguments in favor
> > of Linus' theory that evolution takes care of it. STREAMS were done at
> > Sun by some "architects" who thought they would be better than sockets.
> 
> Wasn't Dennis Ritchie one of the "architects" of STREAMS at Bell Labs?

Different thing. Same name.

> 
>   A Stream Input-Output System : AT&T Bell Laboratories Technical Journal 
>   63,  No. 8 Part 2 (October, 1984)
> 
>   (http://cm.bell-labs.com/cm/cs/who/dmr/st.pdf)
> 
> --ricardo
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Kai Henningsen]

mfedyk@matchmail.com (Mike Fedyk)  wrote on 30.11.01 in <20011130210155.B489@mikef-linux.matchmail.com>:

> On Sat, Dec 01, 2001 at 02:21:57AM +0100, Stephan von Krawczynski wrote:

> > So maybe
> > the real good choice would be this: let the good parts of Solaris (and
> > maybe its SMP features) migrate into linux.
>
> Before 2.3 and 2.4 there probably would've been much more contention against
> something like this.  Even now with large SMP scalability goals, I think it
> would be hard to get something like this to be accepted into Linux.

It works for SGI/Irix/XFS, why would it not work for SUN/Solaris/whatever?

MfG Kai

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Horst von Brand]

Larry McVoy <lm@bitmover.com> said:

[...]

> Because, just like the prevailing wisdom in the Linux hackers, they thought
> it would be relatively straightforward to get SMP to work.  They started at
> 2, went to 4, etc., etc.  Noone ever asked them to go from 1 to 100 in one
> shot.  It was always incremental.

Maybe that is because 128 CPU machines aren't exactly common... just as
SPARC, m68k, S/390 development lags behind ia32 just because there are
many, many more of the later around.

Just as Linus said, the development is shaped by its environment.
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Davide Libenzi]

On Sat, 1 Dec 2001, Alan Cox wrote:

> > > Wasn't it you that were saying that Linux will never scale with more than
> > > 2 CPUs ?
> >
> > No, that wasn't me.  I said it shouldn't scale beyond 4 cpus.  I'd be pretty
> > lame if I said it couldn't scale with more than 2.  Should != could.
>
> Question: What happens when people stick 8 threads of execution on a die with
> a single L2 cache ?

Or for example the new HP PARISC design ( Mako ) with two on-die cores,
3-4 Mb L1 I-Cache per core, 3-4 Mb L1 D-Cache per core and up to 32 Mb
external L2 ( with on-die tagging ).
Anyway it's still better 8 on-die threads of execution sharing an L2
instead of 8 CPU with 1 thread of execution shring through the external
bus.




- Davide

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin Dalecki]

Alan Cox wrote:
> Another thing for 2.5 is going to be to weed out the unused and unmaintained
> drivers, and either someone fixes them or they go down the comsic toilet and
> we pull the flush handle before 2.6 comes out.
> 
> Thankfully the scsi layer breakage is going to help no end in the area of
> clockwork 8 bit scsi controllers, which is major culprit number 1. number 2
> is probably the audio which is hopefully going to go away with ALSA based
> code.

Please consider the following wipe out candidates as well:

1. ftape <- it should really really go away
2. proprietary CD-ROM
3. xd.c (ridiculous isn't it?)
4. old ide driver...
5. old directory reading syscalls.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin Dalecki]

Alan Cox wrote:
> 
> > > Wasn't it you that were saying that Linux will never scale with more than
> > > 2 CPUs ?
> >
> > No, that wasn't me.  I said it shouldn't scale beyond 4 cpus.  I'd be pretty
> > lame if I said it couldn't scale with more than 2.  Should != could.
> 
> Question: What happens when people stick 8 threads of execution on a die with
> a single L2 cache ?

That had been already researched. Gogin bejoind 2 threads on a single
CPU
engine doesn't give you very much... The first step is giving about 25%
the second only about 5%. There are papers in the IBM research magazine
on
this topic in context of the PowerPC.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> > Question: What happens when people stick 8 threads of execution on a die with
> > a single L2 cache ?
> 
> That had been already researched. Gogin bejoind 2 threads on a single
> CPU
> engine doesn't give you very much... The first step is giving about 25%
> the second only about 5%. There are papers in the IBM research magazine
> on

The IBM papers make certain architectural assumptions. With some of the
tiny modern CPU cores its going to perfectly viable to put 4 or 8 of them
on one die. At that point cccluster still has to have cluster nodes scaling
to 8 way

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> Please consider the following wipe out candidates as well:
> 
> 2. proprietary CD-ROM
> 3. xd.c (ridiculous isn't it?)
> 4. old ide driver...

I know people using all 3 of those, while bugs in some of the old scsi 8bit
drivers went unnoticed for a year.

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Oliver Xymoron]

On Sun, 2 Dec 2001, Alan Cox wrote:

> > Please consider the following wipe out candidates as well:
> >
> > 2. proprietary CD-ROM
> > 3. xd.c (ridiculous isn't it?)
> > 4. old ide driver...
>
> I know people using all 3 of those, while bugs in some of the old scsi 8bit
> drivers went unnoticed for a year.

We need a 'prompt for unmaintained drivers' trailing-edge option in the
build process so people will know when something's been orphaned and pick
it up.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Jeff Garzik]

Oliver Xymoron wrote:
> 
> On Sun, 2 Dec 2001, Alan Cox wrote:
> 
> > > Please consider the following wipe out candidates as well:
> > >
> > > 2. proprietary CD-ROM
> > > 3. xd.c (ridiculous isn't it?)
> > > 4. old ide driver...
> >
> > I know people using all 3 of those, while bugs in some of the old scsi 8bit
> > drivers went unnoticed for a year.
> 
> We need a 'prompt for unmaintained drivers' trailing-edge option in the
> build process so people will know when something's been orphaned and pick
> it up.

There's already CONFIG_OBSOLETE...

-- 
Jeff Garzik      | Only so many songs can be sung
Building 1024    | with two lips, two lungs, and one tongue.
MandrakeSoft     |         - nomeansno

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Oliver Xymoron]

On Sun, 2 Dec 2001, Jeff Garzik wrote:

> Oliver Xymoron wrote:
> >
> > On Sun, 2 Dec 2001, Alan Cox wrote:
> >
> > > > Please consider the following wipe out candidates as well:
> > > >
> > > > 2. proprietary CD-ROM
> > > > 3. xd.c (ridiculous isn't it?)
> > > > 4. old ide driver...
> > >
> > > I know people using all 3 of those, while bugs in some of the old scsi 8bit
> > > drivers went unnoticed for a year.
> >
> > We need a 'prompt for unmaintained drivers' trailing-edge option in the
> > build process so people will know when something's been orphaned and pick
> > it up.
>
> There's already CONFIG_OBSOLETE...

And it's practically obsolete itself, outside of the ARM directory. What
I'm proposing is something in the Code Maturity menu that's analogous to
EXPERIMENTAL along with a big (UNMAINTAINED) marker next to unmaintained
drivers. Obsolete and unmaintained and deprecated all mean slightly
different things, by the way. So the config option would probably say
'Show obsolete, unmaintained, or deprecated items?' and mark each item
appropriately. Anything that no one made a fuss about by 2.7 would be
candidates for removal.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Jeff Garzik]

Oliver Xymoron wrote:
> 
> On Sun, 2 Dec 2001, Jeff Garzik wrote:
> 
> > Oliver Xymoron wrote:
> > >
> > > On Sun, 2 Dec 2001, Alan Cox wrote:
> > >
> > > > > Please consider the following wipe out candidates as well:
> > > > >
> > > > > 2. proprietary CD-ROM
> > > > > 3. xd.c (ridiculous isn't it?)
> > > > > 4. old ide driver...
> > > >
> > > > I know people using all 3 of those, while bugs in some of the old scsi 8bit
> > > > drivers went unnoticed for a year.
> > >
> > > We need a 'prompt for unmaintained drivers' trailing-edge option in the
> > > build process so people will know when something's been orphaned and pick
> > > it up.
> >
> > There's already CONFIG_OBSOLETE...
> 
> And it's practically obsolete itself, outside of the ARM directory. What
> I'm proposing is something in the Code Maturity menu that's analogous to
> EXPERIMENTAL along with a big (UNMAINTAINED) marker next to unmaintained
> drivers. Obsolete and unmaintained and deprecated all mean slightly
> different things, by the way. So the config option would probably say
> 'Show obsolete, unmaintained, or deprecated items?' and mark each item
> appropriately. Anything that no one made a fuss about by 2.7 would be
> candidates for removal.

The idea behind CONFIG_OBSOLETE is supposed to be that it does not
actually appear as a Y/N option.  You enclose a Config.in option with
that, and it disappears 

But I'm all for removing old stuff.  There is no reason to keep
something that flat out doesn't work and hasn't for a long long time... 
if somebody wants to pick it up they can grab linux-2.2 or linux-2.0
from any FTP mirror.

	Jeff


-- 
Jeff Garzik      | Only so many songs can be sung
Building 1024    | with two lips, two lungs, and one tongue.
MandrakeSoft     |         - nomeansno

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Oliver Xymoron]

On Sun, 2 Dec 2001, Jeff Garzik wrote:

> Oliver Xymoron wrote:
> >
> > And it's practically obsolete itself, outside of the ARM directory. What
> > I'm proposing is something in the Code Maturity menu that's analogous to
> > EXPERIMENTAL along with a big (UNMAINTAINED) marker next to unmaintained
> > drivers. Obsolete and unmaintained and deprecated all mean slightly
> > different things, by the way. So the config option would probably say
> > 'Show obsolete, unmaintained, or deprecated items?' and mark each item
> > appropriately. Anything that no one made a fuss about by 2.7 would be
> > candidates for removal.
>
> The idea behind CONFIG_OBSOLETE is supposed to be that it does not
> actually appear as a Y/N option.  You enclose a Config.in option with
> that, and it disappears

Which works for stuff that is really known broken. It doesn't work for
stuff that you'd like to get rid of but you suspect people may still be
using (sbpcd) or stuff that you want to phase out (initrd).

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: Linux/Pro [was Re: Coding style - a non-issue]

[jeff millar]

----- Original Message -----
From: "Alan Cox" <alan@lxorguk.ukuu.org.uk>
To: <dalecki@evision.ag>
Cc: "Alan Cox" <alan@lxorguk.ukuu.org.uk>; "Larry McVoy" <lm@bitmover.com>;
"Davide Libenzi" <davidel@xmailserver.org>; "Andrew Morton"
<akpm@zip.com.au>; "Daniel Phillips" <phillips@bonn-fries.net>; "Henning
Schmiedehausen" <hps@intermeta.de>; "Jeff Garzik"
<jgarzik@mandrakesoft.com>; <linux-kernel@vger.kernel.org>
Sent: Sunday, December 02, 2001 11:42 AM
Subject: Re: Linux/Pro [was Re: Coding style - a non-issue]


> > > Question: What happens when people stick 8 threads of execution on a
die with
> > > a single L2 cache ?
> >
> > That had been already researched. Gogin bejoind 2 threads on a single
> > CPU
> > engine doesn't give you very much... The first step is giving about 25%
> > the second only about 5%. There are papers in the IBM research magazine
> > on
>
> The IBM papers make certain architectural assumptions. With some of the
> tiny modern CPU cores its going to perfectly viable to put 4 or 8 of them
> on one die. At that point cccluster still has to have cluster nodes
scaling
> to 8 way

Semiconductor technology will push this way because it's no longer possible
to propagate a signal across the die in one clock cycle.  This means
pipeline
interlocking becomes vastly more complicated.  The simple solution puts
several CPU cores on the die, each able to interlock in one clock but
sharing
memory over several clocks.

RE: Linux/Pro [was Re: Coding style - a non-issue]

[M. Edward Borasky]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Alan Cox
> Sent: Saturday, December 01, 2001 2:06 AM
> To: Andrew Morton
> Cc: Davide Libenzi; Larry McVoy; Daniel Phillips; Henning
> Schmiedehausen; Jeff Garzik; linux-kernel@vger.kernel.org
> Subject: Re: Linux/Pro [was Re: Coding style - a non-issue]

> Another thing for 2.5 is going to be to weed out the unused and
> unmaintained
> drivers, and either someone fixes them or they go down the comsic
> toilet and
> we pull the flush handle before 2.6 comes out.
>
> Thankfully the scsi layer breakage is going to help no end in the area of
> clockwork 8 bit scsi controllers, which is major culprit number
> 1. number 2
> is probably the audio which is hopefully going to go away with ALSA based
> code.

I am *completely* underwhelmed by ALSA, *especially* in the areas of
usability and documentation! I have an M-Audio Delta 66 sound card and I was
unable to get ALSA to work with it. At the time I attempted this, I did not
have the free time to do c-code-level integration work; I needed something
that was plug-and-play with usable documentation. I ended up buying the
closed-source OSS/Linux driver, which at least installs and boots. Their
documentation isn't much better, and I finally was forced to dual-boot
Windows 2000 to get a usable audio tool set for this device.

My point here is that just because a composer is *capable* of doing
integration work and building or repairing tools (and I am) does *not* mean
he (or she :-) has either the time or the willingness to do so (and I
don't).
--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach

znmeb@borasky-research.net
http://www.meta-trading-coach.com
http://groups.yahoo.com/group/meta-trading-coach

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> And it's practically obsolete itself, outside of the ARM directory. What

Thats just history. Doesn't mean it won't do the job. Probably when we get
to say 2.5.4 or so someone should do a build all as modules and anything
that doesn't build gets an obsolete tag until someone fixes it.

Anything not fixed for 2.6 will then be nicely labelled

Re: Linux/Pro [was Re: Coding style - a non-issue]

[[MOc]cda*mirabilos]

> But I'm all for removing old stuff.  There is no reason to keep
> something that flat out doesn't work and hasn't for a long long
time...
> if somebody wants to pick it up they can grab linux-2.2 or linux-2.0
> from any FTP mirror.

By the way, what happened to xiafs?
Back to 2.0.33 it even didn't work (complaints after newfs).

Just an interest...
Thorsten

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Sat, Dec 01, 2001 at 08:05:59PM -0300, Horst von Brand wrote:
> Larry McVoy <lm@bitmover.com> said:
> 
> [...]
> 
> > Because, just like the prevailing wisdom in the Linux hackers, they thought
> > it would be relatively straightforward to get SMP to work.  They started at
> > 2, went to 4, etc., etc.  Noone ever asked them to go from 1 to 100 in one
> > shot.  It was always incremental.
> 
> Maybe that is because 128 CPU machines aren't exactly common... just as
> SPARC, m68k, S/390 development lags behind ia32 just because there are
> many, many more of the later around.
> 
> Just as Linus said, the development is shaped by its environment.

Really?  So then people should be designing for 128 CPU machines, right?
So why is it that 100% of the SMP patches are incremental?  Linux is 
following exactly the same path taken by every other OS, 1->2, then 2->4,
then 4->8, etc.  By your logic, someone should be sitting down and saying
here is how you get to 128.  Other than myself, noone is doing that and
I'm not really a Linux kernel hack, so I don't count.  

So why is it that the development is just doing what has been done before?
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Rik van Riel]

On Sun, 2 Dec 2001, Larry McVoy wrote:

> So why is it that the development is just doing what has been done before?

No matter how often you try to reinvent the wheel, you'll
end up converging to a round shape.

The sad thing is that many of the people here have to try
the octagonal wheel first (because it's "simpler to build
and more maintainable" ?) and then get confused when they
run into problems.

cheers,

Rik
-- 
Shortwave goes a long way:  irc.starchat.net  #swl

http://www.surriel.com/		http://distro.conectiva.com/

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Eric W. Biederman]

Larry McVoy <lm@bitmover.com> writes:

> On Sat, Dec 01, 2001 at 08:05:59PM -0300, Horst von Brand wrote:
> > Larry McVoy <lm@bitmover.com> said:
> > 
> > [...]
> > 
> > > Because, just like the prevailing wisdom in the Linux hackers, they thought
> > > it would be relatively straightforward to get SMP to work.  They started at
> > > 2, went to 4, etc., etc.  Noone ever asked them to go from 1 to 100 in one
> > > shot.  It was always incremental.
> > 
> > Maybe that is because 128 CPU machines aren't exactly common... just as
> > SPARC, m68k, S/390 development lags behind ia32 just because there are
> > many, many more of the later around.
> > 
> > Just as Linus said, the development is shaped by its environment.
> 
> Really?  So then people should be designing for 128 CPU machines, right?
> So why is it that 100% of the SMP patches are incremental?  Linux is 
> following exactly the same path taken by every other OS, 1->2, then 2->4,
> then 4->8, etc.  By your logic, someone should be sitting down and saying
> here is how you get to 128.  Other than myself, noone is doing that and
> I'm not really a Linux kernel hack, so I don't count.  
> 
> So why is it that the development is just doing what has been done before?

Hmm.  Meanwhile linux appears to be the platform of choice for new
clusters.  And if you are working on a cluster you can by incremental
improvements reach your design Larry.  In truth a 128 CPU machine is a
modest cluster.  The typical goal for a large system I have seen is a
1000 Node cluster.  Which could mean anywhere from 1000 to 4000
processors.  And I have seen a few serious proposals for even larger
systems.

So with a cluster you start by looking for something that scales, on a
relatively slow interconnect.  Which is simply lots, and lots of
kernels, that don't know each other at all.  And then you figure out
how to get them all talking to each other.

As has been pointed out exporting an interface that looks to the
programmer like a giant SMP.  Tends to make programs share too much
data, and thus get high degrees of lock contention.

The next incremental step is to get some good distributed and parallel
file systems.  So you can share one filesystem across the cluster.
And there is some work going on in those areas.  luster, gfs,
intermezzo.

Eric

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Davide Libenzi]

On Sun, 2 Dec 2001, Larry McVoy wrote:

> On Sat, Dec 01, 2001 at 08:05:59PM -0300, Horst von Brand wrote:
> > Larry McVoy <lm@bitmover.com> said:
> >
> > [...]
> >
> > > Because, just like the prevailing wisdom in the Linux hackers, they thought
> > > it would be relatively straightforward to get SMP to work.  They started at
> > > 2, went to 4, etc., etc.  Noone ever asked them to go from 1 to 100 in one
> > > shot.  It was always incremental.
> >
> > Maybe that is because 128 CPU machines aren't exactly common... just as
> > SPARC, m68k, S/390 development lags behind ia32 just because there are
> > many, many more of the later around.
> >
> > Just as Linus said, the development is shaped by its environment.
>
> Really?  So then people should be designing for 128 CPU machines, right?
> So why is it that 100% of the SMP patches are incremental?  Linux is
> following exactly the same path taken by every other OS, 1->2, then 2->4,
> then 4->8, etc.  By your logic, someone should be sitting down and saying
> here is how you get to 128.  Other than myself, noone is doing that and
> I'm not really a Linux kernel hack, so I don't count.
>
> So why is it that the development is just doing what has been done before?

That's exactly the Linus point: no long term preventive design.
That means short term design on what we actually have or on what we can
think available in a very near future.
Because, again, if you start designing now an SMP solution for 128 CPUs,
you're going to model it on the current technology that won't probably fit
in future SMP architectures and you'll be screwed up.
There should be a reason why software evolves in this way, think about it.
Either everyone else is fool or you're the prophet.
And if you're the prophet and you think that the future of multiprocessing
is UP on clusters, why instead of spreading your word between us poor
kernel fans don't you pull out money from your pocket ( or investors ) and
start a new Co. that will have that solution has primary and unique goal ?
You could be the J.C. Maxwell of the next century.



- Davide

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Andrew Morton]

Larry McVoy wrote:
> 
> Really?  So then people should be designing for 128 CPU machines, right?

Linux only supports 99 CPUs.  At 100, "ksoftirqd_CPU100" overflows
task_struct.comm[].

Just thought I'd sneak in that helpful observation.

-

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> > Just as Linus said, the development is shaped by its environment.
> 
> Really?  So then people should be designing for 128 CPU machines, right?

Linux environment is a single athlon/pii type processor with 128-256Mb of
RAM, because quite simply thats what most developers have.

> So why is it that 100% of the SMP patches are incremental?  Linux is 
> following exactly the same path taken by every other OS, 1->2, then 2->4,
> then 4->8, etc.  By your logic, someone should be sitting down and saying
> here is how you get to 128.  Other than myself, noone is doing that and
> I'm not really a Linux kernel hack, so I don't count.  

The problem being that unless you happen to have a load of people with 128
processor boxes you can't verify your work is even useful or correct. 

You can look at Linux development and the areas which have been heavily
funded rather than written for fun/need by people and you'll see a heavy
slant to the enterprise end. J Random Hacker doesn't have an IA64, an 8 way
SMP box or 2Tb of disk so J Random Hacker is much more interested in making
sure his/her webcam works (which is why we pee all over Solaris X86 on device 
support).

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> The next incremental step is to get some good distributed and parallel
> file systems.  So you can share one filesystem across the cluster.
> And there is some work going on in those areas.  luster, gfs,
> intermezzo.

gfs went proprietary - you want opengfs

A lot of good work on the rest of that multi-node clustering is going on
already - take a look at the compaq open source site.

cccluster is more for numa boxes, but it needs the management and SSI views
that the compaq stuff offers simply because most programmers won't program
for a cccluster or manage one.

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Dave Jones]

On Sun, 2 Dec 2001, Andrew Morton wrote:

> > Really?  So then people should be designing for 128 CPU machines, right?
> Linux only supports 99 CPUs.  At 100, "ksoftirqd_CPU100" overflows
> task_struct.comm[].
> Just thought I'd sneak in that helpful observation.

Wasn't someone looking at fixing that problem so it didn't need a per-cpu
thread ? 128 kernel threads sitting around waiting for a problem that
rarely happens seems a little.. strange. (for want of a better word).

Dave.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > The next incremental step is to get some good distributed and parallel
> > file systems.  So you can share one filesystem across the cluster.
> > And there is some work going on in those areas.  luster, gfs,
> > intermezzo.
> 
> gfs went proprietary - you want opengfs

Right.
 
> A lot of good work on the rest of that multi-node clustering is going on
> already - take a look at the compaq open source site.

Basically my point.
 
> cccluster is more for numa boxes, but it needs the management and SSI views
> that the compaq stuff offers simply because most programmers won't program
> for a cccluster or manage one.

I've seen a fair number of mpi programs, and if you have a program
that takes weeks to run on a single system.  There is a lot of
incentive to work it out.  Plus I have read about a lot of web sites
that are running on a farm of servers.  Admittedly the normal
architecture has one fat database server behind the web servers, but
that brings me back to needing a good distributed storage techniques.

And I really don't care if most programmers won't program for a
cccluster.  Most programmers don't have one or a problem that needs
one to solve.  So you really only need those people interested in the
problem to work on it.

But single system image type projects are useful, but need to be
watched.  You really need to standardize on how a cluster is put
together (software wise), and making things easier always helps.  But
you also need to be very careful because you can easily write code
that does not scale.  And people doing cluster have wild notions of
scaling o.k. 64 Nodes worked let's try a thousand...

As far as I can tell the only real difference between a numa box, and
a normal cluster of machines running connected with fast ethernet is
that a numa interconnect is a blazingly fast interconnect.  So if you
can come up with a single system image solution over fast ethernet a
ccNuma machine just magically works.

Eric

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Andrew Morton]

Dave Jones wrote:
> 
> On Sun, 2 Dec 2001, Andrew Morton wrote:
> 
> > > Really?  So then people should be designing for 128 CPU machines, right?
> > Linux only supports 99 CPUs.  At 100, "ksoftirqd_CPU100" overflows
> > task_struct.comm[].
> > Just thought I'd sneak in that helpful observation.
> 
> Wasn't someone looking at fixing that problem so it didn't need a per-cpu
> thread ?

Not to my knowledge.

> 128 kernel threads sitting around waiting for a problem that
> rarely happens seems a little.. strange. (for want of a better word).

I've kinda lost the plot on ksoftirqd.  Never really understood
why a thread was needed for this, nor why it runs at nice +20.
But things seem to be working now.

-

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Stephan von Krawczynski]

> On Sat, Dec 01, 2001 at 08:05:59PM -0300, Horst von Brand wrote:    
> > Just as Linus said, the development is shaped by its environment. 
>                                                                     
> Really?  So then people should be designing for 128 CPU machines,   
right?                                                                
> So why is it that 100% of the SMP patches are incremental?  Linux is
                                                                      
> following exactly the same path taken by every other OS, 1->2, then 
2->4,                                                                 
> then 4->8, etc.  By your logic, someone should be sitting down and  
saying                                                                
> here is how you get to 128.  Other than myself, noone is doing that 
and                                                                   
> I'm not really a Linux kernel hack, so I don't count.               
>                                                                     
> So why is it that the development is just doing what has been done  
before?                                                               
                                                                      
Please Larry, have a look at the environment: nobody here owns a box  
with 128 CPUs. Most of the people here take care of things they either
- own themselves                                                      
- have their hands own at work                                        
- get paid for                                                        
                                                                      
You will not find _any_ match with 128 CPUs here.                     
                                                                      
_Obviously_ you are completely right if this were a company _building_
these boxes. Then your question is the right one, as they would get   
paid for the job.                                                     
But this is a different environment. As long as you cannot buy these  
boxes at some local store for a buck and a bit, you will have no      
chance to find willing people for your approach. Therefore it is      
absolutely clear, that it will (again) walk the line from 1,2,4,8 ... 
CPUs, because the boxes will be available along this line.            
                                                                      
I give you this advice: if you _really_ want to move something in this
area, find someone who should care about this specific topic, and has 
the money _and_ the will to pay for development of critical GPL code  
like this.                                                            
Take the _first_ step: create the environment. _Then_ people will come
and follow your direction.                                            
                                                                      
Regards,                                                              
Stephan                                                               
                                                                      
                                                                      

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

> Please Larry, have a look at the environment: nobody here owns a box  
> with 128 CPUs. Most of the people here take care of things they either
> - own themselves                                                      
> - have their hands own at work                                        
> - get paid for                                                        
>                                                                       
> You will not find _any_ match with 128 CPUs here.                     

Nor will you find any match with 4 or 8 CPU systems, except in very rare
cases.  Yet changes go into the system for 8 way and 16 way performance.
That's a mistake.

I'd be ecstatic if the hackers limited themselves to what was commonly 
available, that is essentially what I'm arguing for.  
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[H. Peter Anvin]

Followup to:  <000b01c17b68$2ff846e0$30d8fea9@ecce>
By author:    "[MOc]cda*mirabilos" <mirabilos@netcologne.de>
In newsgroup: linux.dev.kernel
> 
> By the way, what happened to xiafs?
> Back to 2.0.33 it even didn't work (complaints after newfs).
> 

It got ripped out because the vfs changed and noone ported it.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Horst von Brand]

"M. Edward Borasky" <znmeb@aracnet.com> said:

[...]

> My point here is that just because a composer is *capable* of doing
> integration work and building or repairing tools (and I am) does *not* mean
> he (or she :-) has either the time or the willingness to do so (and I
> don't).

So band together with some others with your same problem, and pay somebody
to fix it. What you saved on propietary OS lease should make up for it.
Amply.

Oh wait, you are just a troll, right?
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Davide Libenzi]

On Sun, 2 Dec 2001, Davide Libenzi wrote:

> That's exactly the Linus point: no long term preventive design.

And now for the ones that don't speak Italish :

s/preventive/prior/




- Davide

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Horst von Brand]

Larry McVoy <lm@bitmover.com> said:
> skraw@ithnet.com on Sun, Dec 02, 2001 at 11:52:32PM +0100 said:

[...]

> > You will not find _any_ match with 128 CPUs here.                     
> 
> Nor will you find any match with 4 or 8 CPU systems, except in very rare
> cases.  Yet changes go into the system for 8 way and 16 way performance.
> That's a mistake.

And you are proposing a fork for handling exactly this? I don't get it...
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

RE: Linux/Pro [was Re: Coding style - a non-issue]

[M. Edward Borasky]

> -----Original Message-----
> From: Horst von Brand [mailto:vonbrand@sleipnir.valparaiso.cl]
> Sent: Sunday, December 02, 2001 7:23 PM
> To: M. Edward Borasky
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: Linux/Pro [was Re: Coding style - a non-issue]
>
>
> "M. Edward Borasky" <znmeb@aracnet.com> said:
>
> [...]
>
> > My point here is that just because a composer is *capable* of doing
> > integration work and building or repairing tools (and I am)
> does *not* mean
> > he (or she :-) has either the time or the willingness to do so (and I
> > don't).
>
> So band together with some others with your same problem, and pay somebody
> to fix it. What you saved on propietary OS lease should make up for it.
> Amply.

What I spent on Windows 2000 is $300 US. This converted my $400
top-of-the-line sound card from a useless space-taker on my desk to a
functioning musical device. As for banding together with some others, well,
they are even *more* frustrated than I am, because most of them are *purely*
musicians and *can't* program. Nor do they have the money to spend on
programmers. I'm on a number of musical mailing lists, and their
overwhelming complaint is that they spend most of their time being system
administrators rather than musicians/composers. And these are people using
*commercial* tools -- some *quite* expensive -- on Windows and Macs.

> Oh wait, you are just a troll, right?

Not really ... if you'd like I can be, though. Eventually, when I run out of
other projects, I'll sit down and force ALSA to work with my sound card if
someone hasn't done it already. Of course, now that I have the sound card
running and Windows 2000, why would I need to? So much of Linux is
plug-and-play right now, at least the Red Hat Linux that I'm using. I bought
a sound card unsupported by Red Hat because I knew of two drivers for it --
OSS/Linux and ALSA. I tried ALSA first and gave up on it after a week of
agony on the ALSA mailing list. Then I bought OSS/Linux, which installed
fine but didn't generate any sound. When I sent e-mail to the support desk,
I got a very fast response -- RTFM. The FM in this case consists of a single
page ASCII document which is less than helpful.

What I'm trying to establish here is that if ALSA is to become the
main-stream Linux sound driver set, it's going to need to support -- *fully*
support -- the top-of-the-line sound cards like my M-Audio Delta 66. It
isn't enough to just support the Envy chip inside -- it has to support the
whole card with interfaces to all the sound tools that come with KDE and
Gnome! It has to install flawlessly, boot flawlessly and understand
everything that is in the card. I haven't checked recently to see if the
ALSA situation has changed any -- too busy making music on my Windows
machine :-).
--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach

znmeb@borasky-research.net
http://www.meta-trading-coach.com
http://groups.yahoo.com/group/meta-trading-coach

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Henning P. Schmiedehausen]

Larry McVoy <lm@bitmover.com> writes:

>then 4->8, etc.  By your logic, someone should be sitting down and saying
>here is how you get to 128.  Other than myself, noone is doing that and
>I'm not really a Linux kernel hack, so I don't count.  

"No one that you know". I'm always surprised that you're able to speak
with such confidence. There may be lots of things going on that don't
daily report to you.

	Regards
		Henning


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

>> Really?  So then people should be designing for 128 CPU machines, right?
> 
> Linux only supports 99 CPUs.  At 100, "ksoftirqd_CPU100" overflows
> task_struct.comm[].
> 
> Just thought I'd sneak in that helpful observation.

For machines that are 99bit architectures or more, maybe. For 32 bit machines,
your limit is 32, for 64 bit, 64.

M.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

> As far as I can tell the only real difference between a numa box, and
> a normal cluster of machines running connected with fast ethernet is
> that a numa interconnect is a blazingly fast interconnect.  

Plus some fairly hairy cache coherency hardware.

> So if you
> can come up with a single system image solution over fast ethernet a
> ccNuma machine just magically works.

it's not cc if you just use fast ethernet.

Martin.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

>> Please Larry, have a look at the environment: nobody here owns a box  
>> with 128 CPUs. Most of the people here take care of things they either
>> - own themselves                                                      
>> - have their hands own at work                                        
>> - get paid for                                                        
>>                                                                       
>> You will not find _any_ match with 128 CPUs here.                     
> 
> Nor will you find any match with 4 or 8 CPU systems, except in very rare
> cases.  Yet changes go into the system for 8 way and 16 way performance.
> That's a mistake.
> 
> I'd be ecstatic if the hackers limited themselves to what was commonly 
> available, that is essentially what I'm arguing for.  

We need a *little* bit of foresight. If 4 ways are common now, and 8 ways 
and 16 ways are available, then in a year or two 8 ways (at least) will be
commonplace. On the other hand 128 cpu machines are a way off, and 
I'd agree we shouldn't spend too much time on them right now.

Martin.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

On Mon, Dec 03, 2001 at 11:01:24PM +0000, Henning P. Schmiedehausen wrote:
> Larry McVoy <lm@bitmover.com> writes:
> 
> >then 4->8, etc.  By your logic, someone should be sitting down and saying
> >here is how you get to 128.  Other than myself, noone is doing that and
> >I'm not really a Linux kernel hack, so I don't count.  
> 
> "No one that you know". I'm always surprised that you're able to speak
> with such confidence. There may be lots of things going on that don't
> daily report to you.

Right you are, but...  There is also piles of information that I absorb
on a daily basis, and I'm always willing to be educated.  For example,
you could educate me about all those 128 processor Linux boxes in the
world and fill in a hole in my knowledge.  I'm listening...
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

>> Larry McVoy <lm@bitmover.com> writes:
>> 
>> > then 4->8, etc.  By your logic, someone should be sitting down and saying
>> > here is how you get to 128.  Other than myself, noone is doing that and
>> > I'm not really a Linux kernel hack, so I don't count.  
>> 
>> "No one that you know". I'm always surprised that you're able to speak
>> with such confidence. There may be lots of things going on that don't
>> daily report to you.
> 
> Right you are, but...  There is also piles of information that I absorb
> on a daily basis, and I'm always willing to be educated.  For example,
> you could educate me about all those 128 processor Linux boxes in the
> world and fill in a hole in my knowledge.  I'm listening...

SGI has machines bigger than 128 (if memory serves, 1024??) that I thought
had booted Linux. The Sequent/IBM NUMA-Q archictecture now supports Linux 
and would go to 64 procs if we removed a few bitmap limitations from the kernel 
that have been patched out before (well, actually 60 is easy, 64 would require
some more apic modifications). 

Anyway, bigger than 8 way is no pipedream. I'll admit few people could afford
such machines, but they do exist.

Martin.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> you could educate me about all those 128 processor Linux boxes in the
> world and fill in a hole in my knowledge.  I'm listening...

There are at least two sets of people working on NUMA machines of that
order, as well as the IBM work on smaller NUMA systems.

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> > can come up with a single system image solution over fast ethernet a
> > ccNuma machine just magically works.
> 
> it's not cc if you just use fast ethernet.

Thats a matter for handwaving and distributed shared memory algorithms. The
general point is still true - if you assume your NUMA interconnects are
utter crap when performance and latency issues come up - you'll get the
right results.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Stefan Smietanowski]

Larry McVoy wrote:

>>Please Larry, have a look at the environment: nobody here owns a box  
>>with 128 CPUs. Most of the people here take care of things they either
>>- own themselves                                                      
>>- have their hands own at work                                        
>>- get paid for                                                        
>>                                                                      
>>You will not find _any_ match with 128 CPUs here.                     
>>
> 
> Nor will you find any match with 4 or 8 CPU systems, except in very rare
> cases.  Yet changes go into the system for 8 way and 16 way performance.
> That's a mistake.
> 
> I'd be ecstatic if the hackers limited themselves to what was commonly 
> available, that is essentially what I'm arguing for.  

"There are only black cars, we don't make any other color. Noone has 
ordered a car with a different color cause we don't accept those orders. 
And since noone orders why add colors? Unnecessary cruft."

In essence, People don't run big boxes due to scalability issues, fixing 
those might get someone to install a 16-Way.

They sure as hell aren't gonna buy one and then wait around 3 years for 
Linux to support it.

// Stefan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Lars Brinkhoff]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
> > you could educate me about all those 128 processor Linux boxes in the
> > world and fill in a hole in my knowledge.  I'm listening...
> There are at least two sets of people working on NUMA machines of that
> order, as well as the IBM work on smaller NUMA systems.

Are you NUMA people listening?  What do you think of Larry's ideas?

-- 
Lars Brinkhoff          http://lars.nocrew.org/     Linux, GCC, PDP-10
Brinkhoff Consulting    http://www.brinkhoff.se/    programming

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> What I'm trying to establish here is that if ALSA is to become the
> main-stream Linux sound driver set, it's going to need to support -- *fully*
> support -- the top-of-the-line sound cards like my M-Audio Delta 66. It

Not really. The number of people who actually care about such cards is close
to nil. What matters is that the API can cleanly express what the Delta66
can do, and that you can write a driver for it under ALSA without hacking up
the ALSA core.

I'm happy both of those are true.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Henning P. Schmiedehausen]

Horst von Brand <vonbrand@inf.utfsm.cl> writes:

Hi,

>> Nor will you find any match with 4 or 8 CPU systems, except in very rare
>> cases.  Yet changes go into the system for 8 way and 16 way performance.
>> That's a mistake.

>And you are proposing a fork for handling exactly this? I don't get it...

[Ah, and I've sworn that I won't participate in this thread again...]

99.99% of your userbase have one or two processors. 99.999% of your
userbase have less than 8 processors.

It simply doesn't make sense to design something for 128+ Processors
if noone uses it. Especially if it puts a penalty on the user base
with one or two processors. Then such a design is actively hurting
your user base. And this is nonsense. That's why the design goes
1->2->4->8. We got SMP once a core developer got a sponsored SMP
board. Not because Linus designed his OS with "I want to scale to lots
of processors I can only dream of". Along that road lies madness (and
maybe GNU Hurd ;-) ).

I actually have five (at a customers site) eight-way machines. 
Unfortunately now they all run Windows 2000 (Datacenter Edition
(TM)). But if the "eight-way" support would hurt the "two way"
machines, I'd advocate to put that code at least under a compile
switch.

Actually I do like the idea in Larrys' paper. It doesn't hurt beyond
eight way, because the next eight processors are their own
"system". Just because it doesn't hurt the smaller boxes (much).

	Regards
		Henning


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> In essence, People don't run big boxes due to scalability issues, fixing 
> those might get someone to install a 16-Way.

Hackers don't run Linux on 16 way boxes because they cost $100,000 each

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Stefan Smietanowski]

Alan Cox wrote:

>>In essence, People don't run big boxes due to scalability issues, fixing 
>>those might get someone to install a 16-Way.
>>
> 
> Hackers don't run Linux on 16 way boxes because they cost $100,000 each
> 
> Alan
> 


And here I thought all linux hackers were millionaires :)

// Stefan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[David Weinehall]

On Tue, Dec 04, 2001 at 09:28:49AM +0000, Alan Cox wrote:
> On Mon, Dec 03, 2001 at 06:31:38AM -0800, M. Edward Borasky wrote:
> > What I'm trying to establish here is that if ALSA is to become the
> > main-stream Linux sound driver set, it's going to need to support --
> > *fully* support -- the top-of-the-line sound cards like my M-Audio
> > Delta 66.
> 
> Not really. The number of people who actually care about such cards is
> close to nil. What matters is that the API can cleanly express what
> the Delta66 can do, and that you can write a driver for it under ALSA
> without hacking up the ALSA core.

Indeed. And I'm sure the ALSA-team would be delighted and fully willing
to write a working driver, if mr Borasky donated an M-Audio Delta 66
together with full documentation to them...


/David
  _                                                                 _
 // David Weinehall <tao@acc.umu.se> /> Northern lights wander      \\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\>  http://www.acc.umu.se/~tao/    </   Full colour fire           </

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Jamie Lokier]

Andrew Morton wrote:
> > 128 kernel threads sitting around waiting for a problem that
> > rarely happens seems a little.. strange. (for want of a better word).
> 
> I've kinda lost the plot on ksoftirqd.  Never really understood
> why a thread was needed for this, nor why it runs at nice +20.
> But things seem to be working now.

Me no idea either.  It wasn't to work around the problem of losing
softirqs on syscall return was it?  Because there was a patch for that
in the low-latency set that fixed that without a thread, and without a
delay...

-- Jamie

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Dan Hollis]

On Tue, 4 Dec 2001, Alan Cox wrote:
> > What I'm trying to establish here is that if ALSA is to become the
> > main-stream Linux sound driver set, it's going to need to support -- *fully*
> > support -- the top-of-the-line sound cards like my M-Audio Delta 66. It
> Not really. The number of people who actually care about such cards is close
> to nil. What matters is that the API can cleanly express what the Delta66
> can do, and that you can write a driver for it under ALSA without hacking up
> the ALSA core.
> I'm happy both of those are true.

ALSA has supported ice1712 chipset for some time now.

BTW Delta 66 isnt top of the line -- Delta 1010 is. And ALSA supports it
too.

-Dan
-- 
[-] Omae no subete no kichi wa ore no mono da. [-]

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Dan Hollis]

On Tue, 4 Dec 2001, David Weinehall wrote:
> Indeed. And I'm sure the ALSA-team would be delighted and fully willing
> to write a working driver, if mr Borasky donated an M-Audio Delta 66
> together with full documentation to them...

ALSA already has a working driver...!

-Dan
-- 
[-] Omae no subete no kichi wa ore no mono da. [-]

Re: Linux/Pro [was Re: Coding style - a non-issue]

[David Weinehall]

On Tue, Dec 04, 2001 at 11:35:11AM -0800, Dan Hollis wrote:
> On Tue, 4 Dec 2001, David Weinehall wrote:
> > Indeed. And I'm sure the ALSA-team would be delighted and fully willing
> > to write a working driver, if mr Borasky donated an M-Audio Delta 66
> > together with full documentation to them...
> 
> ALSA already has a working driver...!

The point I was trying to make was just "stop complaining about lack
of drivers, contribute one or help someone else create one. I wasn't
criticizing ALSA, rather the opposite. Now, if I could just find
someone willing to program a driver for that old 8-bit, totally sucky,
IBM ACPA/A I have (the only MCA sound adapter I have managed to get
hold of...)


/David
  _                                                                 _
 // David Weinehall <tao@acc.umu.se> /> Northern lights wander      \\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\>  http://www.acc.umu.se/~tao/    </   Full colour fire           </

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Pavel Machek]

Hi!

> Another thing for 2.5 is going to be to weed out the unused and unmaintained
> drivers, and either someone fixes them or they go down the comsic toilet and
> we pull the flush handle before 2.6 comes out.

Hey, I still have 8-bit isa scsi card somewhere.... Last time I fixed
that was just before 2.4 because that was when I got it... Don't flush
drivers too fast, please... If you kill drivers during 2.5, people
will not notice, and even interesting drivers will get killed. Killing
them during 2.6.2 might be better.
								Pavel
-- 
"I do not steal MS software. It is not worth it."
                                -- Pavel Kankovsky

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

>> > you could educate me about all those 128 processor Linux boxes in the
>> > world and fill in a hole in my knowledge.  I'm listening...
>> There are at least two sets of people working on NUMA machines of that
>> order, as well as the IBM work on smaller NUMA systems.
> 
> Are you NUMA people listening?  What do you think of Larry's ideas?

I presume we're talking about what he calls ccClusters or SMP clusters.
I did a little background reading and found a couple of his old posts.
I'm not an expert on this, though I've done some work in the NUMA area. 
So I'll stick my neck out for people to chop off - I'm not sure I'd agree with 
some of his premises:

> Premise 1: SMP scaling is a bad idea beyond a very small number processors. 
>    The reasoning for this is that when you start out threading a kernel, 
>    it's just a few locks. That quickly evolves into more locks, and 
>    for a short time, there is a 1:1 mapping between each sort of object 
>    in the system (file, file system, device, process, etc) and a lock. 
>    So there can be a lot of locks, but there is only one reader/writer 
>    lock per object instance. This is a pretty nice place to be - it's 
>    understandable, explainable, and maintainable. 
>
>   Then people want more performance. So they thread some more and now 
>    the locks aren't 1:1 to the objects. What a lock covers starts to 
>    become fuzzy. Thinks break down quickly after this because what 
>    happens is that it becomes unclear if you are covered or not and 
>    it's too much work to figure it out, so each time a thing is added 
>    to the kernel, it comes with a lock. Before long, your 10 or 20 
>    locks are 3000 or more like what Solaris has. This is really bad, 
>    it hurts performance in far reaching ways and it is impossible to 
>    undo. 

OK, apart from the fact that there's some leaps of faith here (mostly
due to a lack of detail, I need to go and read some more of his papers),
the obvious objection to this is that just because he's seen it done badly
before, even that it's easy to do badly, it doesn't mean it's impossible to
do it well (it being scalability to many processors).

We will try to make it scale without breaking the low end systems. If we
can, all well and good. If we can't then our patches will get rejected
and we'll all be publicly flogged. Fair enough. And, yes, it's hard. And,
yes, it'll take a while. 

But whilst we gradually make scalability better, NUMA boxes will still
work in the meantime - just not quite as fast. I currently have a NUMA 
box that thinks it an SMP box ... it still works, just not particularly efficiently.
It will get better.

> Premise 3: it is far easier to take a bunch of operating system images 
>    and make them share the parts they need to share (i.e., the page 
>    cache), than to take a single image and pry it apart so that it 
>    runs well on N processors. 

Of course it's easier. But it seems like you're left with much more work to 
reiterate in each application you write to run on this thing. Do you want to 
do the work once in the kernel, or repeatedly in each application? I'd say
it's a damned sight easier to make an application work well on one big
machine than on a cluster.

I like Linus' opinion on the subject, which I'd boil down to "implement
both, see what works". We must have the most massivly parallel 
software engineering team for any OS ever - let's use it ;-)

Martin.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Rik van Riel]

On Tue, 4 Dec 2001, Martin J. Bligh wrote:

> > Premise 3: it is far easier to take a bunch of operating system images
> >    and make them share the parts they need to share (i.e., the page
> >    cache), than to take a single image and pry it apart so that it
> >    runs well on N processors.
>
> Of course it's easier. But it seems like you're left with much more
> work to reiterate in each application you write to run on this thing.
> Do you want to do the work once in the kernel, or repeatedly in each
> application?

There seems to be a little misunderstanding here; from what
I gathered when talking to Larry, the idea behind ccClusters
is that they provide a single system image in a NUMA box, but
with separated operating system kernels.

Of course, this is close to what a "single" NUMA kernel often
ends up doing with much ugliness, so I think Larry's idea to
construct NUMA OSes by putting individual kernels of nodes to
work together makes a lot of sense.

regards,

Rik
-- 
Shortwave goes a long way:  irc.starchat.net  #swl

http://www.surriel.com/		http://distro.conectiva.com/

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Martin J. Bligh]

>> > Premise 3: it is far easier to take a bunch of operating system images
>> >    and make them share the parts they need to share (i.e., the page
>> >    cache), than to take a single image and pry it apart so that it
>> >    runs well on N processors.
>> 
>> Of course it's easier. But it seems like you're left with much more
>> work to reiterate in each application you write to run on this thing.
>> Do you want to do the work once in the kernel, or repeatedly in each
>> application?
> 
> There seems to be a little misunderstanding here; from what
> I gathered when talking to Larry, the idea behind ccClusters
> is that they provide a single system image in a NUMA box, but
> with separated operating system kernels.

OK, then I've partially misunderstood this ... can people provide some 
more reference material? Please email to me, and I'll collate the results
back to the list (should save some traffic).

I have already the following:

http://www.bitmover.com/talks/cliq/slide01.html 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0001.2/1172.html 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0001.3/0236.html 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0007.3/1222.html 

Thanks,

Martin.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Mike Fedyk]

On Sun, Dec 02, 2001 at 12:05:00AM +0200, Kai Henningsen wrote:
> mfedyk@matchmail.com (Mike Fedyk)  wrote on 30.11.01 in <20011130210155.B489@mikef-linux.matchmail.com>:
> 
> > On Sat, Dec 01, 2001 at 02:21:57AM +0100, Stephan von Krawczynski wrote:
> 
> > > So maybe
> > > the real good choice would be this: let the good parts of Solaris (and
> > > maybe its SMP features) migrate into linux.
> >
> > Before 2.3 and 2.4 there probably would've been much more contention against
> > something like this.  Even now with large SMP scalability goals, I think it
> > would be hard to get something like this to be accepted into Linux.
> 
> It works for SGI/Irix/XFS, why would it not work for SUN/Solaris/whatever?
>

I meant actually transplanting Solaris's SMP into Linux.  You don't see SGI
doing anything like this...

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Alan Cox]

> that was just before 2.4 because that was when I got it... Don't flush
> drivers too fast, please... If you kill drivers during 2.5, people
> will not notice, and even interesting drivers will get killed. Killing
> them during 2.6.2 might be better.

They need to die before 2.6. I'm all for leaving the code present and the
ability to select

	Expert
		Drivers that need fixing
			Clockwork scsi controller, windup mark 2

in 2.6 so that people do fix them

Alan

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Pavel Machek]

Hi!

> > cases.  Yet changes go into the system for 8 way and 16 way performance.
> > That's a mistake.
> > 
> > I'd be ecstatic if the hackers limited themselves to what was commonly 
> > available, that is essentially what I'm arguing for.  
> 
> We need a *little* bit of foresight. If 4 ways are common now, and 8 ways 
> and 16 ways are available, then in a year or two 8 ways (at least) will be
> commonplace. On the other hand 128 cpu machines are a way off, and 
> I'd agree we shouldn't spend too much time on them right now.

90% developers have more than one machine to play with... So maybe there's
time for mosix to be merged...?

Then we can create memnet (netdevice over shared memory), and Larry's dream
can come true...
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Pavel Machek]

Hi!

> > Premise 3: it is far easier to take a bunch of operating system images 
> >    and make them share the parts they need to share (i.e., the page 
> >    cache), than to take a single image and pry it apart so that it 
> >    runs well on N processors. 
> 
> Of course it's easier. But it seems like you're left with much more work to 
> reiterate in each application you write to run on this thing. Do you want to 
> do the work once in the kernel, or repeatedly in each application? I'd say
> it's a damned sight easier to make an application work well on one big
> machine than on a cluster.

With mosix, it is exactly as hard. You just run it. I can do that today.
Larry's ideas should look same w.r.t. user.. Or maybe you'll see 128x
boot messages, but that's it.

								Pavel 
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Larry McVoy]

> Then we can create memnet (netdevice over shared memory), and Larry's dream
> can come true...

I'm hoping, but my dreams do not include shared memory over a network.
That's just way too slow.  It's been done a pile of times, every time
people say that the caching will make it fast enough and those people
are wrong every time.

People who think DSM is a good idea are the same people who think a
millisecond is OK for a cache miss (current cache miss times are well
under .0002 milliseconds).
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Linux/Pro [was Re: Coding style - a non-issue]

[Wilson]

----- Original Message -----
From: "Larry McVoy" <lm@bitmover.com>
To: "Pavel Machek" <pavel@suse.cz>
Cc: "Martin J. Bligh" <Martin.Bligh@us.ibm.com>; "Larry McVoy"
<lm@bitmover.com>; "Stephan von Krawczynski" <skraw@ithnet.com>; "Horst von
Brand" <vonbrand@sleipnir.valparaiso.cl>; "lkml"
<linux-kernel@vger.kernel.org>
Sent: Thursday, December 06, 2001 3:50 PM
Subject: Re: Linux/Pro [was Re: Coding style - a non-issue]


> > Then we can create memnet (netdevice over shared memory), and Larry's
dream
> > can come true...
>
> I'm hoping, but my dreams do not include shared memory over a network.

He's talking about net over shared memory, not shared memory over net.