(no subject)

[Keith Owens]

Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.  I
want to do this in separate steps to make it easier for architectures
that have not been converted yet.

2.5.1           Semi-stable kernel, after bio is working.

2.5.2-pre1      Add the kbuild 2.5 and CML2 code, still using
                Makefile-2.5, supporting both CML1 and CML2.
                i386, sparc, sparc64 can use either kbuild 2.4 or 2.5,
                2.5 is recommended.
                ia64 can only use kbuild 2.5.
                Other architectures continue to use kbuild 2.4.
                Wait 24 hours for any major problems then -

2.5.2-pre2      Remove kbuild 2.4 code, rename Makefile-2.5 to Makefile.
		Still supporting both CML1 and CML2.
                i386, ia64, sparc, sparc64 can compile using kbuild 2.5.
                Other architectures cannot compile until they convert
                to kbuild 2.5.  The kbuild group can help with the
                conversion but without access to a machine we cannot
                test other architectures.  Until the other archs have
                been converted, they can stay on 2.5.2-pre1.
                Wait 24 hours for any major problems then -

2.5.2-pre3      Remove CML1 support.

Doing the change in steps provides a platform where both kbuild 2.4 and
2.5 work and both CML1 and CML2 are available.  This allows other
architectures to parallel test the old and new kbuild and CML during
their conversion, I found that ability was very useful during
conversion.

Linus, is this acceptable?  When do you want the kbuild 2.5 and CML2
patches?

Re: [kbuild-devel] (no subject)

[Tom Rini]

On Fri, Dec 07, 2001 at 03:17:53PM +1100, Keith Owens wrote:
> Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.  I
> want to do this in separate steps to make it easier for architectures
> that have not been converted yet.
> 
> 2.5.1           Semi-stable kernel, after bio is working.
> 
> 2.5.2-pre1      Add the kbuild 2.5 and CML2 code, still using
>                 Makefile-2.5, supporting both CML1 and CML2.
>                 i386, sparc, sparc64 can use either kbuild 2.4 or 2.5,
>                 2.5 is recommended.
>                 ia64 can only use kbuild 2.5.
>                 Other architectures continue to use kbuild 2.4.
>                 Wait 24 hours for any major problems then -

Could we wait longer here?  Maybe 48 or 72 to give other arches time to
convert and attempt to sync again?  Or at least show it to Keith so he
can try and sync it up. :)

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: [kbuild-devel] (no subject)

[Keith Owens]

On Thu, 6 Dec 2001 21:30:59 -0700, 
Tom Rini <trini@kernel.crashing.org> wrote:
>On Fri, Dec 07, 2001 at 03:17:53PM +1100, Keith Owens wrote:
>> 2.5.2-pre1      Add the kbuild 2.5 and CML2 code, still using
>>                 Makefile-2.5, supporting both CML1 and CML2.
>>                 i386, sparc, sparc64 can use either kbuild 2.4 or 2.5,
>>                 2.5 is recommended.
>>                 ia64 can only use kbuild 2.5.
>>                 Other architectures continue to use kbuild 2.4.
>>                 Wait 24 hours for any major problems then -
>
>Could we wait longer here?  Maybe 48 or 72 to give other arches time to
>convert and attempt to sync again?  Or at least show it to Keith so he
>can try and sync it up. :)

We will not get all architectures converted in 48 hours or even 72.
kbuild 2.5 has been available for months and only i386, ia64, sparc32
(I did all those) and sparc64 (Ben Collins) have been converted.  Alpha
is in progress.  Unconverted architectures stay on 2.5.2-pre1 until
they do the conversion, but there is no need to hold up everybody else.

Re: [kbuild-devel] (no subject)

[Tom Rini]

On Fri, Dec 07, 2001 at 03:36:13PM +1100, Keith Owens wrote:
> On Thu, 6 Dec 2001 21:30:59 -0700, 
> Tom Rini <trini@kernel.crashing.org> wrote:
> >On Fri, Dec 07, 2001 at 03:17:53PM +1100, Keith Owens wrote:
> >> 2.5.2-pre1      Add the kbuild 2.5 and CML2 code, still using
> >>                 Makefile-2.5, supporting both CML1 and CML2.
> >>                 i386, sparc, sparc64 can use either kbuild 2.4 or 2.5,
> >>                 2.5 is recommended.
> >>                 ia64 can only use kbuild 2.5.
> >>                 Other architectures continue to use kbuild 2.4.
> >>                 Wait 24 hours for any major problems then -
> >
> >Could we wait longer here?  Maybe 48 or 72 to give other arches time to
> >convert and attempt to sync again?  Or at least show it to Keith so he
> >can try and sync it up. :)
> 
> We will not get all architectures converted in 48 hours or even 72.
> kbuild 2.5 has been available for months and only i386, ia64, sparc32
> (I did all those) and sparc64 (Ben Collins) have been converted.  Alpha
> is in progress.  Unconverted architectures stay on 2.5.2-pre1 until
> they do the conversion, but there is no need to hold up everybody else.

I think at least part of it is a time thing.  It's not official so it's
not on the urgent to do list.  Once it gets in I can imagine other
people picking it up again.  I know rmk mentioned he played with it, 6
months ago.  Which is about when I played with PPC too.  I imagine once
it gets in other people/arches will start to try too, and probably have
a few more suggestions for things that i386/ia64/sparc* haven't run
into.

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: [kbuild-devel] (no subject)

[Russell King]

On Fri, Dec 07, 2001 at 03:36:13PM +1100, Keith Owens wrote:
> We will not get all architectures converted in 48 hours or even 72.
> kbuild 2.5 has been available for months and only i386, ia64, sparc32
> (I did all those) and sparc64 (Ben Collins) have been converted.  Alpha
> is in progress.  Unconverted architectures stay on 2.5.2-pre1 until
> they do the conversion, but there is no need to hold up everybody else.

It's a shame that you were too busy to discuss the bugs I found in
kbuild 2.5 6 months ago when I tried it on ARM, despite me following
it up since.

My sympathies are with Tom here, and I think there should be longer than
24 hours for this.

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: [kbuild-devel] (no subject)

[Keith Owens]

On Fri, 7 Dec 2001 11:01:58 +0000, 
Russell King <rmk@arm.linux.org.uk> wrote:
>On Fri, Dec 07, 2001 at 03:36:13PM +1100, Keith Owens wrote:
>> We will not get all architectures converted in 48 hours or even 72.
>> kbuild 2.5 has been available for months and only i386, ia64, sparc32
>> (I did all those) and sparc64 (Ben Collins) have been converted.  Alpha
>> is in progress.  Unconverted architectures stay on 2.5.2-pre1 until
>> they do the conversion, but there is no need to hold up everybody else.
>
>It's a shame that you were too busy to discuss the bugs I found in
>kbuild 2.5 6 months ago when I tried it on ARM, despite me following
>it up since.

What bugs?  Details please.

Re: your mail

[Linus Torvalds]

On Fri, 7 Dec 2001, Keith Owens wrote:
>
> Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.

We're getting the block IO layer in shape first, the time has not come for
_anything_ else before that.

		Linus

Re: [kbuild-devel] Re: your mail

[Eric S. Raymond]

Linus Torvalds <torvalds@transmeta.com>:
> > Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.
> 
> We're getting the block IO layer in shape first, the time has not come for
> _anything_ else before that.

Right, that is more important.  We'll be ready when you're done.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The kind of charity you can force out of people nourishes about as much as
the kind of love you can buy --- and spreads even nastier diseases.

Re: Converting the 2.5 kernel to kbuild 2.5

[Keith Owens]

On Thu, 6 Dec 2001 21:10:11 -0800 (PST), 
Linus Torvalds <torvalds@transmeta.com> wrote:
>
>On Fri, 7 Dec 2001, Keith Owens wrote:
>>
>> Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.
>
>We're getting the block IO layer in shape first, the time has not come for
>_anything_ else before that.

That is what I said ....

2.5.1           Semi-stable kernel, after bio is working.
2.5.2-pre1      Add the kbuild 2.5 and CML2 code, still using
                Makefile-2.5, supporting both CML1 and CML2.
2.5.2-pre2      Remove kbuild 2.4 code, rename Makefile-2.5 to Makefile.
		Still supporting both CML1 and CML2.
2.5.2-pre3      Remove CML1 support.

Is that timetable acceptable?

Re: your mail

[Andre Hedrick]

On Thu, 6 Dec 2001, Linus Torvalds wrote:

> 
> On Fri, 7 Dec 2001, Keith Owens wrote:
> >
> > Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.
> 
> We're getting the block IO layer in shape first, the time has not come for
> _anything_ else before that.
> 
> 		Linus

Lots of luck ... please pass your crack pipe arounds so the rest of us
idiots can see your vision or lack of ...

Regards,

Andre Hedrick
CEO/President, LAD Storage Consulting Group
Linux ATA Development
Linux Disk Certification Project

Re: your mail

[Linus Torvalds]

On Thu, 27 Dec 2001, Andre Hedrick wrote:
>
> Lots of luck ... please pass your crack pipe arounds so the rest of us
> idiots can see your vision or lack of ...

Heh. I think I must have passed it on to you long ago, and you never gave
it back, you sneaky bastard ;)

The vision, btw, is to get the request layer in good enough shape that we
can dispense with the mid-layer approaches of SCSI/IDE, and block devices
turn into _just_ device drivers.

For example, ide-scsi is heading for that big scrap-yard in the sky: it's
not the SCSI layer that handles special ioctl requests any more, because
the upper layers are going to be flexible enough that you can just pass
the requests down the regular pipe.

(Right now you can see this in block_ioctl.c - while only a few of the
ioctl's have been converted, you get the idea. I'm actually surprised that
nobody seems to have commented on that part).

The final end result of this (I sincerely hope) is that we can get rid of
some of the couplings that we've had in the block layer. ide-scsi is just
the most obvious strange coupling - things like "sg.c" in general are
rather horrible. There's very little _SCSI_ in sg.c - it's really about
sending commands down to the block devices.

The reason I want to get rid of the couplings is that they end up being
big anchors holding down development: you can create a clean driver that
isn't dependent on the SCSI layer overheads (and people do, for things
like DAC etc), but when you do that you lose _all_ of the support
infrastructure, not just the bloat. Which is sad.

(And which is why things like ide-scsi exist - IDE didn't really want to
be a SCSI driver, but people _did_ want to be able to use some of the
generic support routines that the SCSI layer offers. You couldn't just
cherry-pick the parts you wanted).

The other part of the bio rewrite has been to get rid of another coupling:
the coupling between "struct buffer_head" (which is used for a limited
kind of memory management by a number of filesystems) and the act of
actually just doing IO.

I used to think that we could just relegate "struct buffer_head" to _be_
the IO entity, but it turns out to be much easier to just split off the IO
part, which is why you now have a separate "bio" structure for the block
IO part, and the buffer_head stuff uses that to get the work done.

Andre, I know that you're worried about the low-level drivers, but:

 - I've long since noticed that we cannot communicate, which is why Jens
   is the block level driver person. You'll have to live with it.

 - I personally don't think you _can_ make a good driver without having
   reasonable interfaces, and we didn't have them.

   For example, the network drivers have improved a lot and do not have
   _nearly_ the amount of problems block drivers have. That's obviously
   partly just because it is a simpler problem, but because it was simpler
   it was also possible to change them. The infrastructure changes in the
   networking during 2.3.x really did help drivers.

And note that the "Jens" and "communication" part is important. If you
have patches, please talk to Jens, tell him what the issues, are, and I
know I can communicate with him.

			Linus

Re: your mail

[Andrew Morton]

Linus Torvalds wrote:
> 
> The other part of the bio rewrite has been to get rid of another coupling:
> the coupling between "struct buffer_head" (which is used for a limited
> kind of memory management by a number of filesystems) and the act of
> actually just doing IO.
> 
> I used to think that we could just relegate "struct buffer_head" to _be_
> the IO entity, but it turns out to be much easier to just split off the IO
> part, which is why you now have a separate "bio" structure for the block
> IO part, and the buffer_head stuff uses that to get the work done.
> 

So... would it be correct to say that there won't be any large
changes to the buffer_head concept in 2.5?

Re: your mail

[Martin Dalecki]

Linus Torvalds wrote:

>(Right now you can see this in block_ioctl.c - while only a few of the
>ioctl's have been converted, you get the idea. I'm actually surprised that
>nobody seems to have commented on that part).
>

That was just too obvious, at least for me... However I don't see why 
you just don't start killing of constructs like:

swtch  (ioctrl)

    BLASH:
BLAHHH:
 BLASHH:
 BLAASS:
     BLAH:
    default:
            return -ENOVAL;
}

There are ton' s of them out there in the block drivers..

Re: [kbuild-devel] Converting the 2.5 kernel to kbuild 2.5

[Eric S. Raymond]

Matthias Andree <matthias.andree@stud.uni-dortmund.de>:
> Seriously: what do you fear? Losing the efforts you put into mconfig?
> Linux 2.2 and 2.4 will be around for quite some time (not sure about
> mconfig on 2.0, I don't use 2.0.x ATM).

Oops.  I wasn't going to tell anyone this yet, but since you've made
this argument I feel I must be up front here....

After CML2 has proven itself in 2.5, I do plan to go back to Marcelo
and lobby for him accepting it into 2.4, on the grounds that doing so
will simplify his maintainance task no end.  That's why I'm tracking
both sides of the fork in the rulebase, so it will be an easy drop-in
replacement for Marcelo as well as Linus.

> What are the precise issues with Python? Just claiming it is an issue is
> not useful for discussing this. Archive pointers are welcome.

The issues can be divided into two groups: silly and serious.  The 
representative silly objection was "Python is evil because significant
whitespace sucks!".  Cristoph's objection to the use of a binary pickle 
as an intermediate format is in this category also.

I heard two serious objections: 

(1) The overhead of learning a new config language is too high.

(2) Requiring Python introduces another tool into the requisites list for
kernel building.  

As to (1), the very people who maintained the in-kernel CML1 tools
judged that the overhead of sticking with what they wrote was
forseeably going to be higher than that of putting a new language in
place.  Otherwise they would not have encouraged me to replace it when
I offered.

As to (2), I could make all kinds of elaborate defensive technical 
arguments, or I could point at Greg Bank's CML2-in-C project, but 
screw that.  I'm just going to say "Today's problems, today's tools."
Progress happens. If you don't like it, feel free to go back to 
writing Autocoder on your 1401.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"The state calls its own violence `law', but that of the individual `crime'"
	-- Max Stirner

Re: [kbuild-devel] Converting the 2.5 kernel to kbuild 2.5

[Alan Cox]

> After CML2 has proven itself in 2.5, I do plan to go back to Marcelo
> and lobby for him accepting it into 2.4, on the grounds that doing so
> will simplify his maintainance task no end.  That's why I'm tracking
> both sides of the fork in the rulebase, so it will be an easy drop-in
> replacement for Marcelo as well as Linus.

Thats somewhat impractical. You will break all the existing additional
configuration tools for the 2.4 stable tree that people expect to continue
to work

Breaking them in 2.5 isnt a big issue, but breaking stable kernel trees
is a complete nono

Re: Converting the 2.5 kernel to kbuild 2.5

[Cameron Simpson]

On Tue, Dec 04, 2001 at 05:44:19PM +0000, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
| > After CML2 has proven itself in 2.5, I do plan to go back to Marcelo
| > and lobby for him accepting it into 2.4, on the grounds that doing so
| > will simplify his maintainance task no end.  That's why I'm tracking
| > both sides of the fork in the rulebase, so it will be an easy drop-in
| > replacement for Marcelo as well as Linus.
| 
| Thats somewhat impractical. You will break all the existing additional
| configuration tools for the 2.4 stable tree that people expect to continue
| to work
| 
| Breaking them in 2.5 isnt a big issue, but breaking stable kernel trees
| is a complete nono

Folks, have you forgotten that you're programmers?

ESR, is it practical to have CML2 transcribe a CML1 config file?
Then as part of the build-the-kernel-src-tarball, Marcelo or whoever's
make target runs the transcriber.

This would let people fetch a kernel and build with the old tools
for personal hacking purposes which keeping the source config in CML2
which is cleans and more powerful. Kernel code _authors_ would need to
write in CML2, but not kernel end users.
-- 
Cameron Simpson, DoD#743        cs@zip.com.au    http://www.zip.com.au/~cs/

A motorcycle is like a toothbrush.  Everyone should have their own.
	- roserunr@noller.com

Re: Converting the 2.5 kernel to kbuild 2.5

[Eric S. Raymond]

Cameron Simpson <cs@zip.com.au>:
> ESR, is it practical to have CML2 transcribe a CML1 config file?

No, alas.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Government should be weak, amateurish and ridiculous. At present, it
fulfills only a third of the role.
	-- Edward Abbey

Converting the 2.5 kernel to kbuild 2.5

[Keith Owens]

Linus, the time has come to convert the 2.5 kernel to kbuild 2.5.  I
want to do this in separate steps to make it easier for architectures
that have not been converted yet.

2.5.1           Semi-stable kernel, after bio is working.

2.5.2-pre1      Add the kbuild 2.5 code, still using Makefile-2.5.
                i386, sparc, sparc64 can use either kbuild 2.4 or 2.5,
                2.5 is recommended.
                ia64 can only use kbuild 2.5.
                Other architectures continue to use kbuild 2.4.
                Wait 24 hours for any major problems then -

2.5.2-pre2      Remove kbuild 2.4 code, rename Makefile-2.5 to Makefile.
                i386, ia64, sparc, sparc64 can compile using kbuild 2.5.
                Other architectures cannot compile until they convert
                to kbuild 2.5.  The kbuild group can help with the
                conversion but without access to a machine we cannot
                test other architectures.  Until the other archs have
                been converted, they can stay on 2.5.2-pre1.

Doing the change in two steps provides a platform where both kbuild 2.4
and 2.5 work.  This allows other architectures to parallel test the old
and new kbuild during their conversion, I found that ability was very
useful during conversion.

The CML1 to CML2 conversion comes later, either in 2.5.3 or 2.5.4.

Linus, is this acceptable?