the state of the linuxppc-dev community

the state of the linuxppc-dev community

[Dan Bethe]

> So what exactly is the problem? The 2.2.12 source? The .config? The fact
> that LinuxPPC developers appear to break the GPL by not dropping a fresh
> source tarball along each kernel binary?

	I happen to know the problem shared by Gabe, me, and anyone else primarily
concerned with the stability, robustness, and ease of use of the whole LinuxPPC
distribution and community authority.
	The problem is that LinuxPPC.org has a development environment that is
splintered, unstable, undocumented, obscure, nonintuitive, and basically
impenetrable even to people who are fully dedicated to it such as Gabe and
myself.  It is run almost exclusively on tribal knowledge that is only in the
heads of the people who are writing the ppc-specific core of the OS.  There are
no procedures and no interface.  And there is very little respect toward people
like me and Gabe who have the time, dedication, and skills to clean it all up.
	First, obviously, we have to map out how everything currently is, before we
could even suggest anything better.  But because 90% of the responses we get
are either dead silence or asinine arguments like yours, we are slowly getting
nowhere.  The remaining 10% are the responses of a mob cheering us on just for
having pointed out the silly state of the maintenance of LinuxPPC.
	We have definately produced deeds rather than just words.  Gabe is a primary
maintainer of Stampede Linux (www.stampede.org), and is working on porting
Stampede to PPC; and I've accepted jhaas's request to be the primary maintainer
of LinuxPPC security and of LinuxPPC Powerbook support.  We both use LinuxPPC
on Powerbook for 99% of our daily work.
	We are capable of contributing to organizing the random mess that is the
LinuxPPC dev community, and tremendously boosting the quality of the product by
making the community accessible to a lot more people.  There are dozens more
likeminded individuals in the linuxppc-user community, who just haven't yet
addressed the root of the issue -- the developers and the distribution
maintainers who are simply incredibly brilliant but disorganized.  I get
private emails from end users, cheering on anyone who makes an organizing move.
	BUT until I hear from the Head Honchos -- jcarr, jhaas, and at least some of
linuxppc's main kickass developers -- I'm not going to fork from linuxppc.org
just to have an organized environment.  We're professionals.  This is an
extremely high quality product who deserves better documentation than random
bulk mailing list searches, and better distribution than a hundred random files
in random ftp sites all over the world.

=====
"Don't expect your own messiah; this neverworld which you desire is
only in your mind." -- http://www.dreamtheater.net/songb4.htm#IV5

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Tom Rini]

On Tue, 8 Feb 2000, Dan Bethe wrote:

> 	The problem is that LinuxPPC.org has a development environment that is
> splintered, unstable, undocumented, obscure, nonintuitive, and basically
> impenetrable even to people who are fully dedicated to it such as Gabe and
> myself.  It is run almost exclusively on tribal knowledge that is only in the
> heads of the people who are writing the ppc-specific core of the OS.  There are
> no procedures and no interface.  And there is very little respect toward people
> like me and Gabe who have the time, dedication, and skills to clean it all up.

Well, yes.  A lot of it comes from everything else being splintered.
Userland is atleast somewhat unified.  Everybody (well, almost) uses at
least a common compiler/glibc.  Thats where the reference release came
in/out of.  We needed to hit glibc 2.1 and we needed everybody else to as
well.  Check out http://www.crashing.org/pressrel.txt (No, the 1.1 update
wasn't as widely published as 1.0, but it still hit lwn).  The kernel is a
whole 'nother ball of wax.

> 	First, obviously, we have to map out how everything currently is, before we
> could even suggest anything better.  But because 90% of the responses we get
> are either dead silence or asinine arguments like yours, we are slowly getting
> nowhere.  The remaining 10% are the responses of a mob cheering us on just for
> having pointed out the silly state of the maintenance of LinuxPPC.

By and large, there aren't many problems, directly related to userland.
compiler and libc bugs get pointed out on the proper lists (ie
gcc-*@gcc.gnu.org, or wherever the glibc bug program points you).  The
problem lies in that the two listed MAINTAINERS for Linux/PPC are either
busy and quite or just quite.  2.3.x still doesn't run on most machines
iirc, and 2.4 is close.

---
Tom Rini (TR1265)
http://gate.crashing.org/~trini/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Tony Mantler]

At 5:37 PM -0600 2/8/2000, Dan Bethe wrote:
[...]
>	The problem is that LinuxPPC.org has a development environment that is
>splintered, unstable, undocumented, obscure, nonintuitive, and basically
>impenetrable even to people who are fully dedicated to it such as Gabe and
>myself.  It is run almost exclusively on tribal knowledge that is only in the
>heads of the people who are writing the ppc-specific core of the OS.
>There are
>no procedures and no interface.  And there is very little respect toward
>people
>like me and Gabe who have the time, dedication, and skills to clean it all up.
[...]

I've noticed that it can be a bit of a task trying to keep up with what's
happening in the PPC-Linux kernel. I found myself in a somewhat similar
position a year or so ago on the Mac68k-Linux project.

That problem was solved when we (Mac68k) moved the development to a CVS server.

Mind you, the PPC-Linux rsync server makes it very easy to get the
latest-ish kernel sources, which I used quite successfully recently to
compile a kernel for my friend's PBG3. (well, mostly successful. got some
missing symbols from some unimportant modules that I didn't care to track
down. used the benh sources)

However, I find that CVS makes it a lot easier to keep track of what's
going on, especially with a CVS mailing list, which reports all updates to
the repository.


All in all I'm not really complaining, though it would be nice if there was
a centralized place where all the updates relating to the (various) PPC
kernel tree(s) were logged.

That's my 2c.


Cheers - Tony :)


--
Tony Mantler         Renaissance Nerd Extraordinaire         eek@escape.ca
Winnipeg, Manitoba, Canada                       http://www.escape.ca/~eek


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Tom Rini]

On Tue, 8 Feb 2000, Tony Mantler wrote:

> Mind you, the PPC-Linux rsync server makes it very easy to get the
> latest-ish kernel sources, which I used quite successfully recently to
> compile a kernel for my friend's PBG3. (well, mostly successful. got some
> missing symbols from some unimportant modules that I didn't care to track
> down. used the benh sources)

Except that the server is never current, yes.  The _only_ place right now
with any sort of current sources is in bitkeeper trees.  So once bk is
released, things should look a bit better.

---
Tom Rini (TR1265)
http://gate.crashing.org/~trini/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Larry McVoy]

: On Tue, 8 Feb 2000, Tony Mantler wrote:
:
: > Mind you, the PPC-Linux rsync server makes it very easy to get the
: > latest-ish kernel sources, which I used quite successfully recently to
: > compile a kernel for my friend's PBG3. (well, mostly successful. got some
: > missing symbols from some unimportant modules that I didn't care to track
: > down. used the benh sources)
:
: Except that the server is never current, yes.  The _only_ place right now
: with any sort of current sources is in bitkeeper trees.  So once bk is
: released, things should look a bit better.

In the it-doesn't-help-at-all department, the reason BK hasn't been publicly
released is that we _know_ of bad rename problems.  We've been working with
Cort to get those resolved and you'll all be happy to know that the BK
resolve process (the thing that shleps in new work and figures out all the
renames, conflicts, etc) has been rewritten and is passing all regressions.
We're gonna spring it on you by the end of the week.  If Cort and the other
BK users bless it, we'll do a public release.

--lm

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Tom Rini]

On Tue, 8 Feb 2000, Larry McVoy wrote:

> : On Tue, 8 Feb 2000, Tony Mantler wrote:
> :
> : > Mind you, the PPC-Linux rsync server makes it very easy to get the
> : > latest-ish kernel sources, which I used quite successfully recently to
> : > compile a kernel for my friend's PBG3. (well, mostly successful. got some
> : > missing symbols from some unimportant modules that I didn't care to track
> : > down. used the benh sources)
> :
> : Except that the server is never current, yes.  The _only_ place right now
> : with any sort of current sources is in bitkeeper trees.  So once bk is
> : released, things should look a bit better.
>
> In the it-doesn't-help-at-all department, the reason BK hasn't been publicly
> released is that we _know_ of bad rename problems.  We've been working with
> Cort to get those resolved and you'll all be happy to know that the BK
> resolve process (the thing that shleps in new work and figures out all the
> renames, conflicts, etc) has been rewritten and is passing all regressions.
> We're gonna spring it on you by the end of the week.  If Cort and the other
> BK users bless it, we'll do a public release.

ooh ooh ooh, you mean it might finally be out?  It has _not_ helped any
that the only up to date trees have been hidden from the public.
Duplication of work, general annoyance, etc.

---
Tom Rini (TR1265)
http://gate.crashing.org/~trini/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Larry McVoy]

: > BK users bless it, we'll do a public release.
:
: ooh ooh ooh, you mean it might finally be out?  It has _not_ helped any
: that the only up to date trees have been hidden from the public.
: Duplication of work, general annoyance, etc.

Yeah.  Sorry it's taken so long.  BK is fairly complicated and the problem
is that it is a really poor match to the open style of development, in
my opinion.  If you guys all use it, it isn't really acceptable to have it
break.  Supporting Cort and his buddies on it turned into a big job and it
became clear that we needed to do the renames rewrite before a public
release.

It's minorly amazing that a group of people who don't have tools that handle
renames quickly grow to like having tools that handle renames and use that
feature enough to promptly break it.

The breakage was fairly subtle - stuff like

	start with A, B, C
	# rotate left 1
	mv A tmp
	mv B A
	mv C B
	mv tmp C

actually showed up.  Among others.  So we now have an architecturally
clean rename processing alg, as well as an idempotent resolver (you can
bail out in the middle and start again and its happy), a more robust
resolver (it saves everything and if there is an error, it restores
everything), as well as a rewritten reader/writer global locking scheme,
etc.  It was a lot of work but I think it will pay off in two ways:
less support overhead, and when you guys find a case we missed, we have
a much cleaner way of handling it.

Again, my apologies for the delays.  If it helps - I'm sure it doesn't
help much - we're working on this 7 days a week, usually pulling 12-14
hour days.  My wife pretty much hates me.  I suspect the other guy's
wives are none too thrilled either...

Light at the end of the tunnel, though.

--lm

P.S.  Andrew (awc@bitmover.com) is rolling all the code into one executable.
Initial tests shows this reduces the x86 installation size from about 14MB
to 340KB...  Busy, busy...

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Dan Bethe]

> Except that the server is never current, yes.  The _only_ place right now
> with any sort of current sources is in bitkeeper trees.  So once bk is
> released, things should look a bit better.

	Hi, Tom.  Freshmeat.net tells me that bitkeeper at
http://www.bitmover.com/bitkeeper/.  This is the official maintenance tool for
the generic Linux kernel, right?  If I understand you correctly, it sounds like
LinuxPPC is moving to bitkeeper as well, once bitkeeper 1.0 is out.
	Does this mean that the intermediary unofficial LinuxPPC-specific code will be
also located on the same Bitkeeper server as the generic kernel all the way?
So that even if there is some PPC code that is not [yet] community-tested and
Linus-approved, we will still check it out from an alternate "line of
development" on the same bitkeeper server?
	It sounds to me like, in general, bitkeeper intends to supercede or obsolete
CVS.
	Thanks for the info.  And many thanks to everyone who's responded today.  This
is the kind of cooperation we're looking for.  Gabe and I are going to take the
facts that everyone's painstakingly outlined (thanks, BenH) and make unifying
docs, to be served from linuxppc.org, hopefully to everyone's approval.
	We were also planning on setting up a CVS server for linuxppc.org, but I'm
gathering that that is unnecessary due to the upcoming use of bitkeeper.  Any
confirmation or comments on that?
	We'll still make sure that linuxppc.org has a good internal server setup, and
to help manage the userspace stuff even better.  We can install a bugtracking
system.  I've only extensively used GNATS, but I'm thinking about trying
Bugzilla (www.bugzilla.org) at my current workplace.
	And finally, I'm glad people responded positively and optimistically.  That's
my main attitude.  Within an hour, I got at least 5 private responses that
basically say "Amen".  Just like each of the several times I've already sent
out varying degrees of interrogation observation to the linuxppc.org community.
	I don't ever mean to be harsh, no matter how many non-positives I may ever
point out to anyone about anything.  We just need for the bazaar to be less
bizarre  ;-)

=====
"Don't expect your own messiah; this neverworld which you desire is
only in your mind." -- http://www.dreamtheater.net/songb4.htm#IV5

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Larry McVoy]

As an aside, Dan, it would be really nice if you made your mail fit in 80
columns.  Lots and lots of people live in 80 column terminals and your mail
is so poorly formatted that I reformat it just to read it.  It also makes it
hard to reply to.

I use a little macro in vi

	map , !}fmt

which says "take the next paragraph and run it through fmt".  See if your
mailer/editor can't do something like that, please.

: > Yeah.  Sorry it's taken so long.  BK is fairly complicated and the problem
: > is that it is a really poor match to the open style of development, in
: > my opinion.  If you guys all use it, it isn't really acceptable to have it
: > break.  Supporting Cort and his buddies on it turned into a big job and it
:
: 	Why is it a poor match?  Is that just because it's currently
: incomplete/unstable, and would spontaneously kill us all?

I wouldn't say that.  We have been using BK to develop BK for about 2.5
years.  It works quite well, actually.

The reason it is a poor match is that a tool like BK is a lot like a file
system - you live on top of it, and when it doesn't work, you don't work.
So having lots of slightly incompatible versions of BK out there would do
nothing but slow down the development.

: 	Is it suitable for fulltime hardcore production use, like
:The Linux Kernel Tree, like your web site says it's intended for?

That's the intent.  And, yeah, I think it is.  It has some rough edges,
you ping Cort to hear about those, but we're getting rid of them.
The architecture of it is very good, we're quite pleased with it and more
than a little proud of it.  It's a lot like stuff you get from Sun - the
first pass shows that the architecture is solid and then over time the
implementation lives up to the architecture.  We're quite close already...

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Michael Schmitz]

> > So what exactly is the problem? The 2.2.12 source? The .config? The fact
> > that LinuxPPC developers appear to break the GPL by not dropping a fresh
> > source tarball along each kernel binary?
>
> 	I happen to know the problem shared by Gabe, me, and anyone else primarily
> concerned with the stability, robustness, and ease of use of the whole LinuxPPC
> distribution and community authority.
> 	The problem is that LinuxPPC.org has a development environment that is
> splintered, unstable, undocumented, obscure, nonintuitive, and basically
> impenetrable even to people who are fully dedicated to it such as Gabe and
> myself.  It is run almost exclusively on tribal knowledge that is only in the
> heads of the people who are writing the ppc-specific core of the OS.  There are
> no procedures and no interface.  And there is very little respect toward people
> like me and Gabe who have the time, dedication, and skills to clean it all up.

With all due respect: if that's what you wanted to direct our attention
to, you should have said so up front. I'm a bit over sensitzited to people
that wave the GPL argument in my face (being a Debian developer for m68k,
and seeing all the tribal infighting there). That's why I jumped all over
Gabe (your name sounds familiar from c.o.l.powerpc or this list, his
wasn't. Sorry for assuming this was just some moron posting flame bait.)

On the issue at hand: when I started working on PPC last year, I also
noticed that information about patches, kernels, people to talk and send
patches to was hard to find (I remember posting a complaint to that effect
to the NG). Having had some contact with Paul on m68k/PPC code sharing
issues before, I knew where to start asking at least, and sent a few
patches to Paul and BenH. Listening on this list for a while since gave me
a better overview on who's doing what, at least where kernel devlopment is
concerned.

I do see the problem that information is splintered over too many web
pages and FTP servers (I fail to see what this has to do with the GPL). I
would welcome having a central dsitribution independent site gather what's
out on the net, I'm too lazy to dig around for bits of information most of
the time myself. I fail to see how lack of response to your suggestions
(that I see voiced by you and Gabe for the first time now on this list)
constitutes lack of respect for you. I imagine keeping up with the project
at hand may leave little time for developers to think about procedures.
That doesn't mean nobody cares. It's not a high priority for some.

> 	First, obviously, we have to map out how everything currently is, before we
> could even suggest anything better.  But because 90% of the responses we get
> are either dead silence or asinine arguments like yours, we are slowly getting
> nowhere.  The remaining 10% are the responses of a mob cheering us on just for
> having pointed out the silly state of the maintenance of LinuxPPC.

The asinine argument was in reply to a post I perceived as asinine. I hope
you accept that as explanation, and let us move on. The resources I know
of are the FAQ-o-matic, mailing list and archive, c.o.l.powerpc (and
maybe deja archive), ftp.dev.linuxppc.org for all sorts of kernel and user
space updates, rsync servers for stable and development trees, and the
linuxppc.org web site that links to few of the above in no particularly
logical fashion.

I could imagine a reworked web site, plus CVS or other access to the
sources would be a good start. I agree with Tony Mantler on the importance
of CVS - having run the m68k Mac port almost as one-man-show for a year,
switching to CVS encouraged at least five more people to start working on
various parts. Getting the latest sources without waiting for me
to build a new diff, checking in patches directly and improved
communication were instrumental in advancing development.

> 	We have definately produced deeds rather than just words.  Gabe is a primary
> maintainer of Stampede Linux (www.stampede.org), and is working on porting
> Stampede to PPC; and I've accepted jhaas's request to be the primary maintainer
> of LinuxPPC security and of LinuxPPC Powerbook support.  We both use LinuxPPC
> on Powerbook for 99% of our daily work.

More power to you. I hope you can get some of your suggestions on
improving the communication and infrastructure and organizing the
PPC port adopted here. Please keep in mind that not all of us are
'professionals' though. I use LinuxPPC (on a Powerbook) for my daily work,
but I can work on the kernel source only in my spare time, as an amateur.

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Jan Nieuwenhuizen]

BenH writes:

>  - The primary source for the "current" up-to-date kernel source tree for
> powermac
>    is Paul Mackerras rsync tree. It can be retreived with rsync:
>
>       rsync -arvz linucare.com.au::linux-pmac-stable <dest_dir>
>
>    This tree contains all the latest fixes, features, etc... as long as
> they are
>    considered stable.

Similarly, I heard some time ago, that

    rsync -auvz linuxcare.com.au::linux-pmac-devel <dest_dir>

would give me a fairly recent development (2.3.x) kernel.
Jan 14, and Feb 1, I succeeded in building a kernel (2.3.39) from that,
jippie!  But since then, I'm getting silly permission errors:

    [root@appel linux]# rsync -auvz linuxcare.com.au::linux-pmac-devel .
    Welcome to the Linuxcare Australia rsync server
    For information about Linuxcare see http://linuxcare.com.au/

    receiving file list ... done
    ./
    arch/ppc/
    send_files failed to open Documentation/kernel-docs.txt: Permission denied
    send_files failed to open Documentation/sound/NM256: Permission denied
    wrote 123 bytes  read 110685 bytes  4345.41 bytes/sec
    total size is 71388166  speedup is 644.25

Greetings,

Jan.

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond - The music typesetter
http://www.xs4all.nl/~jantien       | http://www.lilypond.org

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Michael Schmitz]

> > I do see the problem that information is splintered over too many
> > web pages and FTP servers (I fail to see what this has to do with
> > the GPL).
>
> I have, in the past, been unable to find sources for kernel
> binaries...  Out of date binaries, mind you, for less than
> run-of-the-mill systems, but still, it's frustrating.  Let's see what
> can be fixed and improvied though, and move on.

Short of using a CVS server, tagging each kernel build as a separate
branch, or posting a diff along with each kernel, that would be
impossible. I agree that a release kernel (:= kernel binary used in some
distribution) should have a corresponding diff and .config - demanding
'show me the source' in each case where someone posts a kernel someplace
would be asking too much, in my book. YMMV; I don't post kernels anyway;
I'm going to shut up now. Thank you.

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: the state of the linuxppc-dev community

[Tom Rini]

On Wed, 9 Feb 2000, David N. Welton wrote:

> I have, in the past, been unable to find sources for kernel
> binaries...  Out of date binaries, mind you, for less than
> run-of-the-mill systems, but still, it's frustrating.  Let's see what
> can be fixed and improvied though, and move on.

Not only for you trying to support 'em, but for people that would like to
get those fixes, and put them into the main kernel sources so they aren't
needed in the future.

---
Tom Rini (TR1265)
http://gate.crashing.org/~trini/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/