maintaining DRM and using bitkeeper..

maintaining DRM and using bitkeeper..

[Dave Airlie]

Okay I want to ask other groups of developers how they are maintaining
subsystems away from linux-kernel list and how they get things merged?

The biggest problem we are facing with the DRM is getting things tested
before submitting them to Linus as the last thing we want to happen is to
destabilise the DRM code in the kernel, the two testing paths are
a) DRM CVS, this gets picked up in DRI snapshots and tested quite heavily
- this is by far the most successful testing path for the DRM
b) the -mm tree is more useful for picking out non-x86 (sparc usually)
users,

At the moment I put patches into the BK CVS first, we stabilise them, I
take the CVS changes and patch them into bitkeeper, Andrew picks them up,
we find more bugs I fix em in bitkeeper and put em back in CVS, now
however the bk tree hasn't got a unique patch for a fix, it has a bunch of
changesets that are the initial patch plus the fixes for it,

Now if I submit to Linus via the -bk tree I'm screwed when he either a)
rejects an idea in the tree or b) doesn't respond at all, as I can't just
have him pull up the changesets that matter as a lot of them have been
crossed over by core kernel cleanup patches and bk is getting very shirty
with me about undo and excluding things...if I submit patches to Linus and
he puts them into his tree then I'll be continually dumping my bk trees
and starting again - it takes most of an evening to clone a tree to my
machine at home and I have time to work on this 2 or 3 evenings.. so
should I abandon bk and go it the old way, this means I submit every patch
to Linus and the DRM doesn't stay stable, (if we had 2.7 I would be happy
with this but we don't so .. :-(

So I'm asking is there a better way, considering at this stage I'm not
*trusted*, the DRM is an unholy mess and I'm the only one stupid enough to
step up and do anything about it ... (you should see the DRM in CVS at the
moment it is so much easier to work with without a lot of macros,
unfortunately it won't go into the kernel anytime soon....)

Dave.

-- 
David Airlie, Software Engineer
http://www.skynet.ie/~airlied / airlied at skynet.ie
pam_smb / Linux DECstation / Linux VAX / ILUG person

Re: maintaining DRM and using bitkeeper..

[David S. Miller]

On Fri, 27 Aug 2004 00:57:56 +0100 (IST)
Dave Airlie <airlied@linux.ie> wrote:

> it takes most of an evening to clone a tree to my
> machine at home

Why?

You should always have a current vanilla linus-2.6 BK tree
locally, and just pull into it occaisionally.  Then when
you want to do work just clone it using links:

	bk clone -l linus-2.6 drm-2.6

and fire away. This is the fastest way.

If you're going:

	bk clone bk://linux.bkbits.net/linux-2.5 drm-2.6

then no wonder it takes all evening :-)

When I rebase I just go:

	cd linus-2.6
	bk pull
	cd ..
	mv tree treework
	bk clone -l linus-2.6 tree
	cd tree
	bk pull ../treework

and that takes less than 10 minutes even on my super slow
UltraSPARC machines :-)

Re: maintaining DRM and using bitkeeper..

[Dave Airlie]

> Why?

Thanks Dave, that sorts out the I'm stupid and don't know how to use
bitkeeper issue :-), I'm still not in the bk paradigm...

I'll give it a blast this evening...

Dave.

>
> You should always have a current vanilla linus-2.6 BK tree
> locally, and just pull into it occaisionally.  Then when
> you want to do work just clone it using links:
>
> 	bk clone -l linus-2.6 drm-2.6
>
> and fire away. This is the fastest way.
>
> If you're going:
>
> 	bk clone bk://linux.bkbits.net/linux-2.5 drm-2.6
>
> then no wonder it takes all evening :-)
>
> When I rebase I just go:
>
> 	cd linus-2.6
> 	bk pull
> 	cd ..
> 	mv tree treework
> 	bk clone -l linus-2.6 tree
> 	cd tree
> 	bk pull ../treework
>
> and that takes less than 10 minutes even on my super slow
> UltraSPARC machines :-)
>

-- 
David Airlie, Software Engineer
http://www.skynet.ie/~airlied / airlied at skynet.ie
pam_smb / Linux DECstation / Linux VAX / ILUG person

Re: maintaining DRM and using bitkeeper..

[Sam Ravnborg]

On Fri, Aug 27, 2004 at 12:57:56AM +0100, Dave Airlie wrote:
> 
> Okay I want to ask other groups of developers how they are maintaining
> subsystems away from linux-kernel list and how they get things merged?
> 
> The biggest problem we are facing with the DRM is getting things tested
> before submitting them to Linus as the last thing we want to happen is to
> destabilise the DRM code in the kernel, the two testing paths are
> a) DRM CVS, this gets picked up in DRI snapshots and tested quite heavily
> - this is by far the most successful testing path for the DRM
> b) the -mm tree is more useful for picking out non-x86 (sparc usually)
> users,
> 
> At the moment I put patches into the BK CVS first, we stabilise them, I
> take the CVS changes and patch them into bitkeeper, Andrew picks them up,
> we find more bugs I fix em in bitkeeper and put em back in CVS, now
> however the bk tree hasn't got a unique patch for a fix, it has a bunch of
> changesets that are the initial patch plus the fixes for it,
> 
> Now if I submit to Linus via the -bk tree I'm screwed when he either a)
> rejects an idea in the tree or b) doesn't respond at all, as I can't just
> have him pull up the changesets that matter as a lot of them have been
> crossed over by core kernel cleanup patches and bk is getting very shirty
> with me about undo and excluding things...if I submit patches to Linus and
> he puts them into his tree then I'll be continually dumping my bk trees
> and starting again
The bk way-of-working is to have several bk trees around each with independent
patches.


Consider the following:

linux-2.6
(bk://linux.bkbits.net/linux-2.6)
      |
      +- clone/pull --> (edit) DRM remove macros
      |                        |
      |                        +- clone/pull --> (edit) DRM add driver1
      |                        |
      |                        +- clone/pull --> (edit) DRM fix driver2
      |
      +-clone/pull --> (edit) DRM add driver3
      |
      +- clone/pull --> (wild editing) DRM sandbox
      |
      |
      |
      |
      +- clone/pull --> (pull) DRM-for-linus

DRM-for-linus is the tree you push to bk://drm.bkbits.net/for-linus
When Linus (or lkml) respond you have to do something dramatically different
you may need to drop the DRM-for-linus tree and start all over.
But you should realise that the tree named DRM-for-linus you _never_ edit in that tree.
It is only used to pull from the edited tree to collect changes to send off til Linus.

The aboce to some degree reflect branches in CVS - you just have several individual
trees in bk.

[Larry has saind many times that when you start to use several trees you have started
to understand how to use Bitkeeper efficiently].

You may also be able to convince Andrew to suck from the for-linus tree at bkbits,
so all changes you push will go into next -mm.
That works very well for kbuild for example.


Another thing you should realise is that flag-day patches are NOT welcome.
The ongoing effort removing macors are better included in the kernel as 20
small steps rather than 1 big patch.

In other words it is so much easier to get 10 small patches touching 2000 lines
in total accepted, compared to one big patch.
With smaller individual patches you also keep the history etc.


 
> So I'm asking is there a better way, considering at this stage I'm not
> *trusted*, the DRM is an unholy mess and I'm the only one stupid enough to
> step up and do anything about it ... 
s/stupid/smart/

>(you should see the DRM in CVS at the
> moment it is so much easier to work with without a lot of macros,
> unfortunately it won't go into the kernel anytime soon....)

May I ask why not?

	Sam

Re: maintaining DRM and using bitkeeper..

[Dave Airlie]

>
> Another thing you should realise is that flag-day patches are NOT welcome.
> The ongoing effort removing macors are better included in the kernel as 20
> small steps rather than 1 big patch.

This is my main issue with using bitkeeper vs separate patches, if I check
the macro conversion into bitkeeper as separate patches each removing a
couple of inter-related macros (this is my intention... it is what I did
in the DRM CVS tree) and I ask Linus to pull it from a tree, will it not
look like one big patch (albeit in a number of changesets...) - so the
patch itself will look like a flag day but the bk pull will look like a
set of patches... whereas without bk I send 20 patches to LK in 20 mails
they are nice and separated out ..

>
> May I ask why not?

well they are down the patch queue and will take me a bit of time to sort
out if the patches in front don't make it in ... though I might get to it
soon..

Dave.

-- 
David Airlie, Software Engineer
http://www.skynet.ie/~airlied / airlied at skynet.ie
pam_smb / Linux DECstation / Linux VAX / ILUG person

Re: maintaining DRM and using bitkeeper..

[David S. Miller]

On Sat, 28 Aug 2004 00:54:06 +0100 (IST)
Dave Airlie <airlied@linux.ie> wrote:

> This is my main issue with using bitkeeper vs separate patches, if I check
> the macro conversion into bitkeeper as separate patches each removing a
> couple of inter-related macros (this is my intention... it is what I did
> in the DRM CVS tree) and I ask Linus to pull it from a tree, will it not
> look like one big patch (albeit in a number of changesets...)

No, this will look perfectly fine.  This is exactly how things are
supposed to work.

When you ask for a pull, you can just concatenate the individual changesets
as patches, and it looks really nice.  That is what I do when I send a pull
request to Marcelo or Linus.