linux-next: Requirements and process

linux-next: Requirements and process

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 4566 bytes --]

Hi all,

This email will outline the way I think that I should run linux-next.
People are welcome (in fact encouraged) to comment and add their thoughts.

Definitions:
"current" - this is the release we are working on now (i.e. at the moment
	2.6.26)
"next" - this is the release after "current" (i.e. at the moment 2.6.27)

Changesets I will accept into linux-next (currently being abused a bit):

	1) things destined for "current" i.e. fixes that have not made it
into Linus' tree yet. These will obviously have been posted on lkml and
reviewed and tested.

	2) things expected to be in "next".  These will have been posted
on appropriate mailing lists (most often lkml) and reviewed and tested.

Linux-next is not for experimental patches (but see below).

During the merge window, there is clearly a cross over between these types.

I would expect subsystem owners who use git would have two separate
branches in their published trees (or two separate trees) for these two
types of commits  (e.g. the powerpc tree has a "merge" branch for type 1
and a powerpc-next branch for 2).  Quilt users may have two series files
or just keep the type 1 patches at the start (or something similar).

I would expect the flow of changes to be something like this:

	During the entire cycle, the number type 1 changes would vary up
and down but never be very high.  These changes would probably stay in
linux-next only for short periods and then be integrated into Linus' tree.

	The number type 2 changes should be at its highest just before
the merge window opens and be close to zero at rc1 (or maybe rc2). After
that it should grow again until the next merge window.  Things entering
linux-next should generally stay there (though they may be modified due
to conflicts or bugs).

I currently have 9 trees that are of type 1 above:

x86-fixes	sched-fixes	powerpc-merge
scsi-rc-fixes	net-current	sparc-current
sound-current	arm-current	pci-current

I currently have 54 trees that (I assume) are of type 2 above:

driver-core	usb		x86
sched		pci		device-mapper
hid		i2c		kernel-doc
avr32		v4l-dvb		s390
sh		jfs		kbuild
ide		libata		nfs
xfs		infiniband	acpi
blackfin	nfsd		ieee1394
hwmon		ubi		kvm
dlm		scsi		ia64
tests		ocfs2		selinux
m68k		powerpc		hrt
lblnet		ext4		4xx
async_tx	udf		security-testing
net		sparc		galak
mtd		wireless	crypto
vfs		sound		arm
cpufreq		semaphore	semaphore-removal

And three trees that I am not sure about:

x86-latest	sched-latest	ldp

Most of these trees (apart from the last three) are now small or empty
relative to Linus' tree.

Process (so you all know what I am up to):

Each day (or so) I start with Linus' tree and merge all the above trees
one at a time.  I will attempt to fixup merge conflicts and notify the
appropriate tree/change owners of what I have needed to do.  If things
are too bad, I will not merge a particular tree for that day (I have only
had to do that a couple of times).  Between each merge (including before
the first) I do two builds of the tree (currently a powerpc
ppc64_defconfig and an x86_64 allmodconfig).  If the build fails, I
either revert offending changes or add small fixup patches.  Again I will
notify the appropriate tree/change owners.

Occasionally, I will pick up single patches that are needed to make the
tree build for some architectures/configs.  Andrew tells me that I need
to take ownership of these patches and make sure someone adds them to a
tree destined for Linus - that makes sense to me.

After all the above, I put the tree into our build farm and make sure it
builds a few configs before I release it.

Experimental stuff:

I am currently integrating the Linux Driver Project tree from Greg KH on
the understanding that anything in it that causes a problem gets dropped
i.e. I generally don't even try to figure out what went wrong, just let
him know.  I am beginning to feel that this may need to be in some way
separated from linux-next proper.  Ideas welcome.

Where from here:

Andrew is currently rebasing the -mm tree on linux-next.  He intends to
also feed some of the trees he hosts into linux-next (hopefully avoiding
circular dependancies :-))

The following architectures are not in linux-next (and should be):

alpha		cris		frv
h8300		m32r		m68knommu
mips		mn10300		parisc
um		v850		xtensa

See Andrew's mail for a list of other subsystems.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: linux-next: Requirements and process

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

On Sat, 3 May 2008 15:45:42 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> The following architectures are not in linux-next (and should be):
> 
> alpha		cris		frv
> h8300		m32r		m68knommu
> mips		mn10300		parisc
> um		v850		xtensa
> 
> See Andrew's mail for a list of other subsystems.

In case it isn't clear, I need to be told where I can get a particular
tree from and who should be contacted in case of problems (I do not troll
through the MAINTAINERS file).  I can currently handle git trees and
quilt series.  I am willing to consider other formats/scms.  I also do
not mind having multiple trees (or branches or series files) for a single
subsystem/maintainer.

Series files must be annotated with an indication of the place in Linus'
tree that they are based on using the following format:

# BASE <some identification of a place in Linus' tree>

This base can be a SHA1 or a release tag or one of the ...-git<n> or the
output from "git describe".  Alternatively, you can be based on another
tree in linux-next by using:

# NEXT_BASE <name of a tree in linux-next>

The names are in the Next/Trees file in each release and normally do not
change.

Also you may (if you like) delimit the parts of the series you want in
linux-next by using:

# NEXT_PATCHES_START
.
.
# NEXT_PATCHES_END

There can be multiple such sections.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: linux-next: Requirements and process

[Andrew Morton]

On Sat, 3 May 2008 15:45:42 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> The following architectures are not in linux-next (and should be):
> 
> alpha		cris		frv
> h8300		m32r		m68knommu
> mips		mn10300		parisc
> um		v850		xtensa

mips, m32r, parisc and xtensa do have git trees.  The rest are mastered as
discrete patches in -mm.

Except for m68knommu, which pops unexpectedly out of the woodwork during
the merge window.  I've asked that this be altered ;)

Re: linux-next: Requirements and process

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On Fri, 2 May 2008 23:35:19 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Sat, 3 May 2008 15:45:42 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> 
> > The following architectures are not in linux-next (and should be):
> > 
> > alpha		cris		frv
> > h8300		m32r		m68knommu
> > mips		mn10300		parisc
> > um		v850		xtensa
> 
> mips, m32r, parisc and xtensa do have git trees.  The rest are mastered as
> discrete patches in -mm.

So, I was wondering if it would be worth while having subsections to a
series file like:

# NEXT_PATCHES_START [<label> [<base>]]

# NEXT_PATCHES_END

With <label> sections being logically separate enough that we can talk
about them/drop them/merge them at different places etc.

Or am I over engineering? :-)

> Except for m68knommu, which pops unexpectedly out of the woodwork during
> the merge window.  I've asked that this be altered ;)

Yeah, I saw that, thanks.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: linux-next: Requirements and process

[Andrew Morton]

On Sat, 3 May 2008 17:45:57 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> On Fri, 2 May 2008 23:35:19 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Sat, 3 May 2008 15:45:42 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> > 
> > > The following architectures are not in linux-next (and should be):
> > > 
> > > alpha		cris		frv
> > > h8300		m32r		m68knommu
> > > mips		mn10300		parisc
> > > um		v850		xtensa
> > 
> > mips, m32r, parisc and xtensa do have git trees.  The rest are mastered as
> > discrete patches in -mm.
> 
> So, I was wondering if it would be worth while having subsections to a
> series file like:
> 
> # NEXT_PATCHES_START [<label> [<base>]]
> 
> # NEXT_PATCHES_END
> 
> With <label> sections being logically separate enough that we can talk
> about them/drop them/merge them at different places etc.
> 
> Or am I over engineering? :-)

That sounds good.  I once started to think about how to do this but
accidentally fell asleep.  I was thinking along the lines of the above,
only it drives an akpm script which spits out separate quilt (or git) trees
for linux-next.

The problem is that I then need to "drop" the patches so that I can merge
linux-next.  That's where I fell asleep.  I suppose that putting the
well-baked ones into a git tree and mastering them there solves the
problem.  But juggling 100-odd git branches on top of everything else
doesn't sound fun.

I need to think about this some more - it'll come.

What does "[<base>]" do?

Re: linux-next: Requirements and process

[Greg KH]

On Sat, May 03, 2008 at 03:45:42PM +1000, Stephen Rothwell wrote:
> I currently have 54 trees that (I assume) are of type 2 above:
> 
> driver-core	usb

Internally I have both of these trees split into "current" and "next"
sections, so that I know what to merge when.

If it would make things easier for you, I'd be glad to split both of
these trees up in such a manner to make it more visable.

Perhaps:
	driver-core-next
	usb-next
	driver-core
	usb

would be sufficient for you?

> And three trees that I am not sure about:
> 
> x86-latest	sched-latest	ldp

ldp you have properly named as something that will probably go into
next-next or most likely next-next-next, but does build and run properly
today, yet has work to go in the "must clean up this crap" arena.

thanks,

greg k-h

Re: linux-next: Requirements and process

[Randy Dunlap]

On Sat, 3 May 2008 16:07:24 +1000 Stephen Rothwell wrote:

> In case it isn't clear, I need to be told where I can get a particular
> tree from and who should be contacted in case of problems (I do not troll
> through the MAINTAINERS file).  I can currently handle git trees and
> quilt series.  I am willing to consider other formats/scms.  I also do
> not mind having multiple trees (or branches or series files) for a single
> subsystem/maintainer.
> 
> Series files must be annotated with an indication of the place in Linus'
> tree that they are based on using the following format:
> 
> # BASE <some identification of a place in Linus' tree>
> 
> This base can be a SHA1 or a release tag or one of the ...-git<n> or the
> output from "git describe".  Alternatively, you can be based on another
> tree in linux-next by using:
> 
> # NEXT_BASE <name of a tree in linux-next>

What do we use for BASE or NEXT_BASE if the base tree is linux-next?
Just don't use these annotations?


> The names are in the Next/Trees file in each release and normally do not
> change.
> 
> Also you may (if you like) delimit the parts of the series you want in
> linux-next by using:
> 
> # NEXT_PATCHES_START
> .
> .
> # NEXT_PATCHES_END
> 
> There can be multiple such sections.


---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

Re: linux-next: Requirements and process

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

Hi Randy,

On Tue, 22 Jul 2008 15:31:14 -0700 Randy Dunlap <randy.dunlap@oracle.com> wrote:
>
> > output from "git describe".  Alternatively, you can be based on another
> > tree in linux-next by using:
> > 
> > # NEXT_BASE <name of a tree in linux-next>
> 
> What do we use for BASE or NEXT_BASE if the base tree is linux-next?
> Just don't use these annotations?

The name from the Next/Trees file in any linux-next release.  You should
be very careful doing this if you don't control the other tree, though.
And you should let me know so that I can order the trees appropriately.

This was mainly added for Greg who has 4 quilt series in linux-next that
depend on each other.  If you depend on some random tree that changes day
to day your quilt series may no longer apply cleanly on top of it.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: linux-next: Requirements and process

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

On Fri, 2 May 2008 23:35:19 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Sat, 3 May 2008 15:45:42 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> 
> > The following architectures are not in linux-next (and should be):
> > 
> > alpha		cris		frv
> > h8300		m32r		m68knommu
> > mips		mn10300		parisc
> > um		v850		xtensa
> 
> mips, m32r, parisc and xtensa do have git trees.  The rest are mastered as
> discrete patches in -mm.
> 
> Except for m68knommu, which pops unexpectedly out of the woodwork during
> the merge window.  I've asked that this be altered ;)

So (from the above list) cris, m68knommu and mips have joined the
linux-next fray ...

(cc's some maintainers)
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]