A question of workflow

A question of workflow

[Matthew Palmer]

[I've never used monotone for a project before, so this may be something of
a general monotone question, but please bear with me]

I've hacked on the Ruby bitbake spec so it can build commonly-used
extensions like zlib, socket, and openssl.  However, I'm not sure how to go
about committing, submitting, and merging this patch with the main OE tree.

What I've done so far:

* Gotten a checkout of the org.openembedded.dev branch;
* Made my modifications;
* Created a key;
* Committed my changes to my working copy.

Now I can't even manage to retrieve the diff of my changes (the 'obvious'
step of running "mtn diff -r <SHA1>" didn't do anything useful) to submit
them to the bugzilla.  Of course, submitting a context-free patch when
you've got a full distributed RCS at your bidding seems a bit archaic
anyway.

So, my questions:

* How do I get my diff back?
* How do I submit my changes to the OE project for merging?
* Will I have problems when the change gets committed to the trunk?
* (The biggie) is there a guide on the wiki or somewhere describing the
  preferred workflow for making changes and working effectively with the rest
  of the OE developer community?  (Google wasn't at all illuminating)

Thanks,
- Matt

Re: A question of workflow

[Richard Purdie]

On Sat, 2006-12-30 at 16:16 +1100, Matthew Palmer wrote:
> I've hacked on the Ruby bitbake spec so it can build commonly-used
> extensions like zlib, socket, and openssl.  However, I'm not sure how to go
> about committing, submitting, and merging this patch with the main OE tree.
> 
> What I've done so far:
> 
> * Gotten a checkout of the org.openembedded.dev branch;
> * Made my modifications;
> * Created a key;
> * Committed my changes to my working copy.
> 
> Now I can't even manage to retrieve the diff of my changes (the 'obvious'
> step of running "mtn diff -r <SHA1>" didn't do anything useful) to submit
> them to the bugzilla.  

I'm not sure what the right command it but you should be able to extract
that diff somehow, I don't know offhand what the command would be.

> Of course, submitting a context-free patch when
> you've got a full distributed RCS at your bidding seems a bit archaic
> anyway.
> 
> So, my questions:
> 
> * How do I get my diff back?
> * How do I submit my changes to the OE project for merging?
> * Will I have problems when the change gets committed to the trunk?
> * (The biggie) is there a guide on the wiki or somewhere describing the
>   preferred workflow for making changes and working effectively with the rest
>   of the OE developer community?  (Google wasn't at all illuminating)

What you've described is how I or any other OE developer with an
authorised monotone key would work with OE. The problem is we can't just
give access to anyone as that wouldn't make any sense for obvious
reasons.

Before we give out such access, we need to have some idea of a persons
capabilities and trust them enough not to break things (too badly ;-).
We generally ask you start submitting patches via the bugzilla and then
when we're happy you might get direct commit access. This doesn't fit in
with the SCM (monotone) that well.

It does mean you shouldn't be committing changes locally via monotone.
If you do this you will have to pull and then merge every time. That
isn't a problem in itself but if you do get direct commit access, we
will not be happy adding hundreds of extra merges to the main
repository.

Richard

Re: A question of workflow

[Matthew Palmer]

On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> On Sat, 2006-12-30 at 16:16 +1100, Matthew Palmer wrote:
> > I've hacked on the Ruby bitbake spec so it can build commonly-used
> > extensions like zlib, socket, and openssl.  However, I'm not sure how to go
> > about committing, submitting, and merging this patch with the main OE tree.
> > 
> > What I've done so far:
> > 
> > * Gotten a checkout of the org.openembedded.dev branch;
> > * Made my modifications;
> > * Created a key;
> > * Committed my changes to my working copy.
> > 
> > Now I can't even manage to retrieve the diff of my changes (the 'obvious'
> > step of running "mtn diff -r <SHA1>" didn't do anything useful) to submit
> > them to the bugzilla.  
> 
> I'm not sure what the right command it but you should be able to extract
> that diff somehow, I don't know offhand what the command would be.

Marcin Juszkiewicz gave me the answer in the bugzilla issue I opened
(#1725).  It was... non-trivial.  And I thought tla had a difficult UI...

> > Of course, submitting a context-free patch when
> > you've got a full distributed RCS at your bidding seems a bit archaic
> > anyway.
> > 
> > So, my questions:
> > 
> > * How do I get my diff back?
> > * How do I submit my changes to the OE project for merging?
> > * Will I have problems when the change gets committed to the trunk?
> > * (The biggie) is there a guide on the wiki or somewhere describing the
> >   preferred workflow for making changes and working effectively with the rest
> >   of the OE developer community?  (Google wasn't at all illuminating)
> 
> What you've described is how I or any other OE developer with an
> authorised monotone key would work with OE. The problem is we can't just
> give access to anyone as that wouldn't make any sense for obvious
> reasons.

Yep, you're naturally not going to give trunk access to random people, but
I'm not looking for trunk commit rights.  You're using a distributed
revision control system, in which I should be able to work pretty much
entirely independently, just feeding changesets back for the core devs to
merge.  I don't want, and shouldn't need, core commit rights in order to be
able to do what I want -- that's the beauty of a DRCS.  However, being
unfamiliar with monotone, I need assistance with knowing exactly which
commands I need to be running to do what I need to do.

> It does mean you shouldn't be committing changes locally via monotone.
> If you do this you will have to pull and then merge every time. That
> isn't a problem in itself but if you do get direct commit access, we
> will not be happy adding hundreds of extra merges to the main
> repository.

WTF?  Why are you using a distributed revision control system, if I'm not
supposed to be committing locally?  If everything I do is supposed to be
bundled up into a patch and sent to the bugzilla, how am I meant to maintain
my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
That might make sense if I'm stuck interacting with SVN, but with a DRCS in
the mix I expect *it* to be able to take on that role.

- Matt

Re: A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthew Palmer schreef:
> On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:

>> It does mean you shouldn't be committing changes locally via monotone.
>> If you do this you will have to pull and then merge every time. That
>> isn't a problem in itself but if you do get direct commit access, we
>> will not be happy adding hundreds of extra merges to the main
>> repository.
> 
> WTF?  Why are you using a distributed revision control system, if I'm not
> supposed to be committing locally?  If everything I do is supposed to be
> bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> the mix I expect *it* to be able to take on that role.

Do what virtually every DSCM does to support that: make a branch
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFluM9MkyGM64RGpERApWCAJ4vGC91DRHlu2CiBN+Zjl+u0YQx3gCgrdfF
kcLE9jQGwMRuafhSB8gUs3o=
=svf1
-----END PGP SIGNATURE-----

Re: A question of workflow

[Erik Hovland]

On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
> > On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> 
> >> It does mean you shouldn't be committing changes locally via monotone.
> >> If you do this you will have to pull and then merge every time. That
> >> isn't a problem in itself but if you do get direct commit access, we
> >> will not be happy adding hundreds of extra merges to the main
> >> repository.
> > 
> > WTF?  Why are you using a distributed revision control system, if I'm not
> > supposed to be committing locally?  If everything I do is supposed to be
> > bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> > my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> > That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> > the mix I expect *it* to be able to take on that role.
> 
> Do what virtually every DSCM does to support that: make a branch

Strange. Git nor mercucial require you to branch.

E

-- 
Erik Hovland
mail: erik@hovland.org
web: http://hovland.org/
PGP/GPG public key available on request

Re: A question of workflow

[Matthew Palmer]

On Sat, Dec 30, 2006 at 03:08:34PM -0800, Erik Hovland wrote:
> On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
> > > On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> > 
> > >> It does mean you shouldn't be committing changes locally via monotone.
> > >> If you do this you will have to pull and then merge every time. That
> > >> isn't a problem in itself but if you do get direct commit access, we
> > >> will not be happy adding hundreds of extra merges to the main
> > >> repository.
> > > 
> > > WTF?  Why are you using a distributed revision control system, if I'm not
> > > supposed to be committing locally?  If everything I do is supposed to be
> > > bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> > > my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> > > That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> > > the mix I expect *it* to be able to take on that role.
> > 
> > Do what virtually every DSCM does to support that: make a branch
> 
> Strange. Git nor mercucial require you to branch.

Add darcs and bzr to the "no need to explicitly branch" list.  But then
again, it's not a fundamental failing of the tool if you need to say "Please
branch now", especially when, like monotone, you've got the ability to have
this "micro-branch" thing with the multiple "heads".

- Matt

-- 
"Ah, the beauty of OSS. Hundreds of volunteers worldwide volunteering their
time inventing and implementing new, exciting ways for software to suck."
		-- Toni Lassila, in the Monastery

Re: A question of workflow

[Richard Purdie]

On Sun, 2006-12-31 at 10:40 +1100, Matthew Palmer wrote:
> On Sat, Dec 30, 2006 at 03:08:34PM -0800, Erik Hovland wrote:
> > On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
> > > > On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> > > 
> > > >> It does mean you shouldn't be committing changes locally via monotone.
> > > >> If you do this you will have to pull and then merge every time. That
> > > >> isn't a problem in itself but if you do get direct commit access, we
> > > >> will not be happy adding hundreds of extra merges to the main
> > > >> repository.
> > > > 
> > > > WTF?  Why are you using a distributed revision control system, if I'm not
> > > > supposed to be committing locally?  If everything I do is supposed to be
> > > > bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> > > > my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> > > > That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> > > > the mix I expect *it* to be able to take on that role.
> > > 
> > > Do what virtually every DSCM does to support that: make a branch
> > 
> > Strange. Git nor mercucial require you to branch.
> 
> Add darcs and bzr to the "no need to explicitly branch" list.  But then
> again, it's not a fundamental failing of the tool if you need to say "Please
> branch now", especially when, like monotone, you've got the ability to have
> this "micro-branch" thing with the multiple "heads".

You don't have to branch. It simply means you will have to merge every
time you pull from upstream. This is the same as git, I'm not familiar
with the others. All I've tried to do it make you aware of what you're
doing and which a) you will need to always merge after pulling and b)
why you won't be allowed to push to trunk with such a repository. If
neither thing concerns you, please go ahead.

This isn't a unique problem to monotone. git handles this by allowing
you to rebase commits against a new base revision. I'm not sure if
monotone has a way to handle this or not. If it doesn't, it should have
but that is something for the monotone people to consider.

As for the problem of patch interchange using the actual SCM without
trunk write access, this is something that needs careful thought. You
can share a repository and I guess we'd pull into a local branch, then
merge with trunk. I'd be interested in the monotone people's thoughts on
how to handle this tbh. Currently we have no good solution but it would
be nice to have one. Its probably an education problem on both ends.
FWIW, the linux kernel has similar issues with git and developers still
exchange patches rather than git repositories. 

Richard

Re: A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Erik Hovland schreef:
> On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
>>> On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
>>>> It does mean you shouldn't be committing changes locally via monotone.
>>>> If you do this you will have to pull and then merge every time. That
>>>> isn't a problem in itself but if you do get direct commit access, we
>>>> will not be happy adding hundreds of extra merges to the main
>>>> repository.
>>> WTF?  Why are you using a distributed revision control system, if I'm not
>>> supposed to be committing locally?  If everything I do is supposed to be
>>> bundled up into a patch and sent to the bugzilla, how am I meant to maintain
>>> my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
>>> That might make sense if I'm stuck interacting with SVN, but with a DRCS in
>>> the mix I expect *it* to be able to take on that role.
>> Do what virtually every DSCM does to support that: make a branch
> 
> Strange. Git nor mercucial require you to branch.

Actually git branches the moment when you pull, it just hides that.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFl4SsMkyGM64RGpERAurNAKCjkLjRb0r5CvV+r4ZlinBXqnP5FACfeAFn
ybhy4X43hRaAfqafOSnzhPA=
=HtD6
-----END PGP SIGNATURE-----

Re: A question of workflow

[Matthew Palmer]

On Sun, Dec 31, 2006 at 10:36:44AM +0100, Koen Kooi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Erik Hovland schreef:
> > On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
> >>> On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> >>>> It does mean you shouldn't be committing changes locally via monotone.
> >>>> If you do this you will have to pull and then merge every time. That
> >>>> isn't a problem in itself but if you do get direct commit access, we
> >>>> will not be happy adding hundreds of extra merges to the main
> >>>> repository.
> >>> WTF?  Why are you using a distributed revision control system, if I'm not
> >>> supposed to be committing locally?  If everything I do is supposed to be
> >>> bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> >>> my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> >>> That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> >>> the mix I expect *it* to be able to take on that role.
> >> Do what virtually every DSCM does to support that: make a branch
> > 
> > Strange. Git nor mercucial require you to branch.
> 
> Actually git branches the moment when you pull, it just hides that.

That's what bzr and darcs do, too.  You can't really have a DRCS without
either implicit or explicit branching on retrieval.  In reality, monotone is
branching implicitly just like git, bzr, and darcs.  It's just that monotone
has two levels of branching: "heads", which are nothing more than unnamed
microbranches, and the more coarse-grained named branch.

- Matt

Re: A question of workflow

[Matthew Palmer]

On Sat, Dec 30, 2006 at 11:07:57PM +0100, Koen Kooi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Matthew Palmer schreef:
> > On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
> 
> >> It does mean you shouldn't be committing changes locally via monotone.
> >> If you do this you will have to pull and then merge every time. That
> >> isn't a problem in itself but if you do get direct commit access, we
> >> will not be happy adding hundreds of extra merges to the main
> >> repository.
> > 
> > WTF?  Why are you using a distributed revision control system, if I'm not
> > supposed to be committing locally?  If everything I do is supposed to be
> > bundled up into a patch and sent to the bugzilla, how am I meant to maintain
> > my own tree of fixes while I wait for them all to be applied to dev?  Quilt? 
> > That might make sense if I'm stuck interacting with SVN, but with a DRCS in
> > the mix I expect *it* to be able to take on that role.
> 
> Do what virtually every DSCM does to support that: make a branch

You seem right on with that "virtually" bit:

$ mtn help |grep -i branch
$

Oh, commit with a --branch option.  Cute, if not exactly discoverable. 
Monotone certainly seems to have read from the Tom Lord manual of Interface
Design.

Note also that Richard said that I shouldn't be doing *any* commits locally
-- no mention of branching or any other way to take advantage of the
benefits of a DRCS.  (Not having a go at you, Richard, just explaining my
thought processes.)

So having solved The Mystery Of The Branch, how do I make the completed
branch available for the core devs to merge back into the trunk?  (If it
involves copying a huge monotone database to my webserver and running
mtnserve, I'll just give up now -- I don't have the resources to spare for
that).  What's the naming policy for branches -- presumably, since my branch
names will end up "polluting" the main database if you propagate my changes,
I can't just go naming my branches any old thing (since it appears that it's
a single global namespace for branches).  How do I manage separate feature
branches (so that each branch is kept clean for merging) while still having
a comglomerate branch I can use for building from?  I suppose I can
propagate each feature branch back into a dirty work branch, but that just
seems... labour-intensive.

Do you understand my frustration?  I know what I've got to do, but I can't
find any docs in the wiki that cater to anyone other than OE core devs
(MonotonePhraseBook) -- which I'm certainly not going to be any time soon --
so I'm left either working everything out from first principles (wasting
time that I could be spending fixing things, not to mention making a pile of
screwups -- like committing to the main branch, creating a second head -- in
the meantime), writing slightly pissy e-mails on mailing lists (and getting
equally pissy responses back), or just giving up, converting the dev branch
to darcs, and not bothering to contribute my fixes back in any meaningful
manner because it's just Too Damn Hard.

In short, how do *you* want to be worked with?  I'm happy to work in
whatever reasonable way works best for you, but I need to know what that
process is first.

- Matt

-- 
It fsck's the volume or it gets the format again.
		-- Don Quixote, in the Monastery

Re: A question of workflow

[Richard Purdie]

On Sun, 2006-12-31 at 10:59 +1100, Matthew Palmer wrote:
> Note also that Richard said that I shouldn't be doing *any* commits locally
> -- no mention of branching or any other way to take advantage of the
> benefits of a DRCS.  (Not having a go at you, Richard, just explaining my
> thought processes.)

This is simply because we have no established way of receiving data
through the SCM for merging into trunk without direct write access. As
my other email says, I'd like to find a way of supporting that but its
probably an education issue on both sides.

Richard

Re: A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:
> On Sun, 2006-12-31 at 10:59 +1100, Matthew Palmer wrote:
>> Note also that Richard said that I shouldn't be doing *any* commits locally
>> -- no mention of branching or any other way to take advantage of the
>> benefits of a DRCS.  (Not having a go at you, Richard, just explaining my
>> thought processes.)
> 
> This is simply because we have no established way of receiving data
> through the SCM for merging into trunk without direct write access. As
> my other email says, I'd like to find a way of supporting that but its
> probably an education issue on both sides.

And the fact that we usually have to tweak 80% if the patches we receive. Attaching them
to the bugtracker makes it easy to search for, view and comment on patches, and most
importantly, view their status.
It doesn't matter which SCM we use with that workflow.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFl4bHMkyGM64RGpERAqP/AJ4p9m4FAbXgtVOiACLMQEKScGYqmwCgqC/4
9Oee1LelN+CNy3/fWLqDRn8=
=fv68
-----END PGP SIGNATURE-----

Re: A question of workflow

[Matthew Palmer]

On Sun, Dec 31, 2006 at 10:45:43AM +0100, Koen Kooi wrote:
> Richard Purdie schreef:
> > On Sun, 2006-12-31 at 10:59 +1100, Matthew Palmer wrote:
> >> Note also that Richard said that I shouldn't be doing *any* commits locally
> >> -- no mention of branching or any other way to take advantage of the
> >> benefits of a DRCS.  (Not having a go at you, Richard, just explaining my
> >> thought processes.)
> > 
> > This is simply because we have no established way of receiving data
> > through the SCM for merging into trunk without direct write access. As
> > my other email says, I'd like to find a way of supporting that but its
> > probably an education issue on both sides.
> 
> And the fact that we usually have to tweak 80% if the patches we receive.

So you propagate the contributed branch to the dev tree, then make the
changes that are needed to fix it up as a separate changeset.  That
maintains full history for everyone.

> Attaching them to the bugtracker makes it easy to search for, view and
> comment on patches, and most importantly, view their status.

The problem with just dealing in raw patches is that when I'm maintaining my
local tree, a branch of the upstream repo, my patches come back to me after
getting applied upstream without any context, so that I get all manner of
spurious conflicts.  It looks as though all I can do to try and avoid this
is disapprove patches, I can't "unpropagate" a whole branch to prevent the
conflict -- and that assumes that I know in advance that the patch is coming
back down.  Far better to have the SCM do that all for me.

> It doesn't matter which SCM we use with that workflow.

Which does make me wonder why you went with monotone instead of the
lowest-common-denominator, since you don't appear to be using any of
monotone's more interesting features?

- Matt

Re: A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthew Palmer schreef:
> On Sun, Dec 31, 2006 at 10:45:43AM +0100, Koen Kooi wrote:

>> It doesn't matter which SCM we use with that workflow.
> 
> Which does make me wonder why you went with monotone instead of the
> lowest-common-denominator, since you don't appear to be using any of
> monotone's more interesting features?

Offline commit, diff and history as well as peer-to-peer syncing. We've had 2 periods
(when our bk license ran out and when mtn 0.26 changed the network protocol) were we
tested different SCMs (hg, git, svn, svk, mtn) and each time monotone came out on top.
BTW, most of the questions you asked are answered in http://venge.net/monotone/monotone.html.


regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFl5qpMkyGM64RGpERAkihAKCJqU1kRwEbtMp7LbufGFn1sMJ8KACgiY3k
QVLw7nVwscXhLBam6Mrktjg=
=tgwY
-----END PGP SIGNATURE-----

Re: A question of workflow

[Patrick Ohly]

On So, 2006-12-31 at 10:45 +0100, Koen Kooi wrote:
> > This is simply because we have no established way of receiving data
> > through the SCM for merging into trunk without direct write access.
> As
> > my other email says, I'd like to find a way of supporting that but
> its
> > probably an education issue on both sides.
> 
> And the fact that we usually have to tweak 80% if the patches we
> receive.

If you do that, can you commit the patch from the original author
literally on a branch, then apply your own changes which fix it on top
of the original patch and then merge back into the main trunk? I did an
experiment and found that this considerably eases the pain for external
contributors.

The experiment simulated a "worst case" scenario:
      * external author creates a new file
      * OE developer modifies and renames the file before committing to
        the trunk
      * external author updates the trunk and merges automatically
      * => file exists twice with different content; without renaming
        there would be a merge conflict

If external contributor and core OE developers adhere to the following
procedures, then the merge conflict is avoided. The external developer
should:
      * pull official OE monotone database
      * checkout, make changes
      * commit on a personal branch ("mtn commit -b
        personal.branch.name")
      * generate a patch (e.g. "mtn diff -r org.openembedded.dev", be
        more specific if necessary)
      * propose to merge the patch by submitting it in the OE tracker
      * continue working on the personal branch
      * follow upstream changes by propagating them ("mtn propagate
        org.openembedded.dev personal.branch.name" + "mtn update -b
        personal.branch.name")

To accept this contribution, a core OE developer should:
      * apply the original patch ("patch <foo.patch" + "mtn add <new
        files>")
      * commit on a new branch _before_ making any changes to it ("mtn
        commit -b org.openembedded.<developer>.merging")
      * make any required changes on that branch
      * commit, then propagate to org.openembedded.dev

This worked for me in the scenario above where the external developer
had not made further changes on his branch. During propagating the trunk
with patch + changes applied monotone reported that no merging was
necessary and directly made the current head part of the personal
branch. After that, updating the personal branch had all changes. I
suppose it would have worked just as well if merging had been needed
because monotone had all the required information.

One of the disadvantages is the need for another branch. I don't think
it has to be pushed into the official monotone DB, so only the developer
doing the merging needs to know about it. I should even be possible to
reuse the branch in multiple patch/commit/fix/propagate cycles by
propagating the trunk to it before applying changes.

Clearly this puts some extra work on the OE developers, but in exchange
you not only encourage external contributions, you also better document
inside monotone what the original patch was and how you modified it.

What do you think? Did I miss something?

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Paul Sokolovsky]

Hello Patrick,

Sunday, January 7, 2007, 10:04:24 PM, you wrote:

> On So, 2006-12-31 at 10:45 +0100, Koen Kooi wrote:
>> > This is simply because we have no established way of receiving data
>> > through the SCM for merging into trunk without direct write access.
>> As
>> > my other email says, I'd like to find a way of supporting that but
>> its
>> > probably an education issue on both sides.
>> 
>> And the fact that we usually have to tweak 80% if the patches we
>> receive.

> If you do that, can you commit the patch from the original author
> literally on a branch, then apply your own changes which fix it on top
> of the original patch and then merge back into the main trunk? I did an
> experiment and found that this considerably eases the pain for external
> contributors.

[]

> What do you think? Did I miss something?

  Well, why would OE core developers (err, what about non-core
developers - we also care for contributed patches - after all, to get
commit access to OE, you first contribute bunch of patches) - well, why
would they do that? OE is open-source project, there's no position of
"patch processor", and people do what they find useful, leaving some
share of their for patch processing. Any complex and artificial rules
won't work. Just in case, there's always another alternative - contributors
just should submit patches in the perfect form, suitable for merging w/o
tweaks. Obviously, it's just the same not attainable, nor it's really
*that* important - we want people contribute more than either us or them
to follow strict rules.

  Just my IMHO - I'm not a core developer ;-).




-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: A question of workflow

[Patrick Ohly]

On So, 2007-01-07 at 23:16 +0200, Paul Sokolovsky wrote:
> Hello Patrick,
> > What do you think? Did I miss something?
> 
>   Well, why would OE core developers (err, what about non-core
> developers - we also care for contributed patches - after all, to get
> commit access to OE, you first contribute bunch of patches) - well, why
> would they do that?

Perhaps because they care enough about external contributions to make
life easier for the people preparing patches? The fact that their own
changes would also be tracked by monotone is probably less important.

I'm aware that this is not something that can be requested from
volunteers, but at least it can be suggested and perhaps someone will
find it useful and (more important) doable.

> Just in case, there's always another alternative - contributors
> just should submit patches in the perfect form, suitable for merging w/o
> tweaks.

I'm not so sure that this will work - As Koen said, having to tweak a
patch is the rule, not the exception. Just doing those tweaks instead of
striving for perfect patches might be less work also for the core
developers.

>  Obviously, it's just the same not attainable, nor it's really
> *that* important - we want people contribute more than either us or them
> to follow strict rules.

Of course, these shouldn't be strict rules, but having some guidelines
wouldn't hurt either. I think I have seen questions about "how to
contribute patches" several times on this list already.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Matthew Palmer]

On Sun, Jan 07, 2007 at 11:16:32PM +0200, Paul Sokolovsky wrote:
>   Well, why would OE core developers (err, what about non-core
> developers - we also care for contributed patches - after all, to get
> commit access to OE, you first contribute bunch of patches) - well, why
> would they do that? OE is open-source project, there's no position of
> "patch processor", and people do what they find useful, leaving some
> share of their for patch processing. Any complex and artificial rules
> won't work.

The only reason I can think of for the core devs to follow a particular
workflow is to make it ultimately easier for external contributors to
produce and submit patches.  The more external patches that come in, the
less work the core devs have to do (or the more 'other' things they can do). 
For example, a few days ago Marcin asked for a patch to make a build of a
newer version of fakeroot; I'd have whipped that patch up by now except that
I know that it's going to cause a conflict when the change comes back down,
so I'm loathe to make the change until I work out how I can manage the patch
flow, because it's going to cause me pain without any benefit.  I'm just not
that masochistic.

- Matt

Re: A question of workflow

[Justin Patrin]

On 1/7/07, Matthew Palmer <mpalmer@hezmatt.org> wrote:
> On Sun, Jan 07, 2007 at 11:16:32PM +0200, Paul Sokolovsky wrote:
> >   Well, why would OE core developers (err, what about non-core
> > developers - we also care for contributed patches - after all, to get
> > commit access to OE, you first contribute bunch of patches) - well, why
> > would they do that? OE is open-source project, there's no position of
> > "patch processor", and people do what they find useful, leaving some
> > share of their for patch processing. Any complex and artificial rules
> > won't work.
>
> The only reason I can think of for the core devs to follow a particular
> workflow is to make it ultimately easier for external contributors to
> produce and submit patches.  The more external patches that come in, the
> less work the core devs have to do (or the more 'other' things they can do).
> For example, a few days ago Marcin asked for a patch to make a build of a
> newer version of fakeroot; I'd have whipped that patch up by now except that
> I know that it's going to cause a conflict when the change comes back down,
> so I'm loathe to make the change until I work out how I can manage the patch
> flow, because it's going to cause me pain without any benefit.  I'm just not
> that masochistic.
>

You know, instead of adding extra branches and making the workflow mor
complicated for core devs you, as an external contributor, could
follow your patch as it makes it into OE and update your local version
accordingly. That way you also have no conflicts when you propagate or
merge.

One thing I could see us doing is possibly comitting a patch as-is and
then making changes and comitting that. (Of course, we wouldn't push
until it's all finished) but I don't know how well that will merge.
Monotone uses the revision graph to deal with merging and your local
commit still wouldn't likely be the same revision as what the dev
comitted.

Hmmm, now that I think of it this workflow might work:

Contributor:
1) notes current (.dev) revision they are using in their workspace
2) makes changes
3) commits a new revision with those changes
4) creates a patch and opens a bug
OE dev (assuming patch and change is ok):
5) updates their workspace to revision from 1)
6) commits the contributor's patch
7) makes changes to "fix up" patch
8) commits fixed up revision
9) merges with head (if needed)

This way, the revision from 3) is the same as the revision from 6) and
your DB will realize this (from the rev#, which is a SHA1 hash) and
you don't have to merge at all.

This only works if the patch is accepted as-is (if it requires
revision before an OE dev initially commits it you still will have to
manually update your own revision to allow it to merge cleanly) and
assumes that you make all of your changes *directly* on a .dev
revision and not in your own branch. Monotone tracks revisions, not
patches, remember.

It should also be noted that it doesn't take too much to get to be an
OE dev yourself. You can keep your own branch and deal with a few
merge conflicts, then get commit access to OE and commit/push
directly.

-- 
Justin Patrin

Re: A question of workflow

[Matthew Palmer]

On Sun, Jan 07, 2007 at 02:46:38PM -0800, Justin Patrin wrote:
> You know, instead of adding extra branches and making the workflow mor
> complicated for core devs you, as an external contributor, could
> follow your patch as it makes it into OE and update your local version
> accordingly. That way you also have no conflicts when you propagate or
> merge.

Or I could just not submit the patch to OE.  That way I would also have no
conflicts.  Not so good for OE, but the same end result for me.  Keeping
track of a large number of patches in any particular state wouldn't be a
whole load of fun, either, especially when there's a tool which, in theory,
should be doing that work for me.

> One thing I could see us doing is possibly comitting a patch as-is and
> then making changes and comitting that. (Of course, we wouldn't push
> until it's all finished) but I don't know how well that will merge.
> Monotone uses the revision graph to deal with merging and your local
> commit still wouldn't likely be the same revision as what the dev
> comitted.

Or you could merge the patch directly from the user's tree, which keeps all
the metadata intact.  Monotone seems particularly poor in this regard,
requiring you to be running your own mtnserve, though.

> It should also be noted that it doesn't take too much to get to be an
> OE dev yourself. You can keep your own branch and deal with a few
> merge conflicts, then get commit access to OE and commit/push
> directly.

That shouldn't be necessary.

- Matt

-- 
If Alan Turing was alive today, the homosexuality would be OK but he'd be in    
trouble for codebreaking.
		-- Martin Bacon

Re: A question of workflow

[Justin Patrin]

On 1/7/07, Matthew Palmer <mpalmer@hezmatt.org> wrote:
> On Sun, Jan 07, 2007 at 02:46:38PM -0800, Justin Patrin wrote:
> > You know, instead of adding extra branches and making the workflow mor
> > complicated for core devs you, as an external contributor, could
> > follow your patch as it makes it into OE and update your local version
> > accordingly. That way you also have no conflicts when you propagate or
> > merge.
>
> Or I could just not submit the patch to OE.  That way I would also have no
> conflicts.  Not so good for OE, but the same end result for me.  Keeping
> track of a large number of patches in any particular state wouldn't be a
> whole load of fun, either, especially when there's a tool which, in theory,
> should be doing that work for me.
>

Then don't keep track of them. Here's another perfectly valid idea.
You submit patches and, as you submit more and listen to the list you
come to understand how your patches should be contributed. Your
patches get better and better and you have less and less merge
conflicts. Be glad we're using such a great system and you have these
possibilities. We could still be using CVS like so many other
projects. We don't try to create conflicts. We want your patches to be
able to be just dropped in.

> > One thing I could see us doing is possibly comitting a patch as-is and
> > then making changes and comitting that. (Of course, we wouldn't push
> > until it's all finished) but I don't know how well that will merge.
> > Monotone uses the revision graph to deal with merging and your local
> > commit still wouldn't likely be the same revision as what the dev
> > comitted.
>
> Or you could merge the patch directly from the user's tree, which keeps all
> the metadata intact.  Monotone seems particularly poor in this regard,
> requiring you to be running your own mtnserve, though.

How would you suggest we do this? The only way to get your revision
*directly* into our DB is for you to push it or for us to pull it.

It is also possible to get the same revision using a patch as I
outlined in my previous mail, which you seem to have left out of this
reply. That is the only possible way I can think of right now for
monotone and OE to somewhat easily deal with this so that you don't
have to do as much work keeping up to date with patches. Of course, it
puts constraints on both the contributor and the OE developers, but it
would work.

-- 
Justin Patrin

Re: A question of workflow

[Patrick Ohly]

On Sun, 2007-01-07 at 14:46 -0800, Justin Patrin wrote:
> One thing I could see us doing is possibly comitting a patch as-is and
> then making changes and comitting that. (Of course, we wouldn't push
> until it's all finished) but I don't know how well that will merge.

I think it would merge as well (or bad) as committing to a branch first.
The reason why I didn't mention this possibility is that it means that
the core developer's revision of the trunk gets temporarily "polluted"
with an unchecked and possibly incomplete patch. That might prevent
finishing and pushing some other, more important work. I also don't know
how easy it would be to back out the patch completely.

> Monotone uses the revision graph to deal with merging and your local
> commit still wouldn't likely be the same revision as what the dev
> comitted.

I'm on thin ice here because I don't know monotone well enough, but
isn't a common ancestor enough to do the three-way merge that has a much
better chance of avoiding merge conflicts? There is guaranteed to be a
common ancestor for external and core OE developer even if both work
with branches because both share the same OE trunk.

> Hmmm, now that I think of it this workflow might work:
> 
> Contributor:
> 1) notes current (.dev) revision they are using in their workspace
> 2) makes changes

As you said, this assumes that the contributor always develops the patch
against the oe.dev branch. The more likely situation with an active
contributor is that he has multiple pending patches that he needs to
continue with his work. He might store them using quilt/patcher or in a
monotone branch, but in either case the base revision of a patch is not
something found on the oe.dev branch.

> OE dev (assuming patch and change is ok):
> 5) updates their workspace to revision from 1)

Even if the contributor is able to provide a revision, isn't that going
to be a lot slower for the core developer? It implies going back to an
older revision and then recompiling all sources necessary to check the
patch (okay, that could be skipped, but that's not desirable).

In my "temporarily branch on commit" proposal the developer would be
able to keep working with the currently checked out and compiled
revision of the meta data, although admittedly reusing the same branch
name requires some more thought - I'm not exactly sure how to do that
with monotone.

Anyway, all of this is idle speculation unless one or more of the core
developers are willing to adopt such a method.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Justin Patrin]

On 1/8/07, Patrick Ohly <Patrick.Ohly@gmx.de> wrote:
> On Sun, 2007-01-07 at 14:46 -0800, Justin Patrin wrote:
> > One thing I could see us doing is possibly comitting a patch as-is and
> > then making changes and comitting that. (Of course, we wouldn't push
> > until it's all finished) but I don't know how well that will merge.
>
> I think it would merge as well (or bad) as committing to a branch first.
> The reason why I didn't mention this possibility is that it means that
> the core developer's revision of the trunk gets temporarily "polluted"
> with an unchecked and possibly incomplete patch. That might prevent
> finishing and pushing some other, more important work. I also don't know
> how easy it would be to back out the patch completely.

I'm not sure what making another branch realy buys here. In fact, I'm
not even sure how monotone is happily merging your conflicts seeing as
the patch would still be coming from 2 places. If a contributor makes
a change in their own branch and then it's comitted seperately to .dev
then they're 2 different revisions.

>
> > Monotone uses the revision graph to deal with merging and your local
> > commit still wouldn't likely be the same revision as what the dev
> > comitted.
>
> I'm on thin ice here because I don't know monotone well enough, but
> isn't a common ancestor enough to do the three-way merge that has a much
> better chance of avoiding merge conflicts? There is guaranteed to be a
> common ancestor for external and core OE developer even if both work
> with branches because both share the same OE trunk.

With my workflow you have a common *revision* and this means that
monotone can realy do the merge work for you.

>
> > Hmmm, now that I think of it this workflow might work:
> >
> > Contributor:
> > 1) notes current (.dev) revision they are using in their workspace
> > 2) makes changes
>
> As you said, this assumes that the contributor always develops the patch
> against the oe.dev branch. The more likely situation with an active
> contributor is that he has multiple pending patches that he needs to
> continue with his work. He might store them using quilt/patcher or in a
> monotone branch, but in either case the base revision of a patch is not
> something found on the oe.dev branch.

This is easily solveable. The contributor can use the latest official
.dev rev and make their patch, then commit. Once this is done they can
merge/propagate to their own branch.

>
> > OE dev (assuming patch and change is ok):
> > 5) updates their workspace to revision from 1)
>
> Even if the contributor is able to provide a revision, isn't that going
> to be a lot slower for the core developer? It implies going back to an
> older revision and then recompiling all sources necessary to check the
> patch (okay, that could be skipped, but that's not desirable).

Updating to an older revision isn't hard. It's about the same thing as
doing a new checkout or creating a new branch.

>
> In my "temporarily branch on commit" proposal the developer would be
> able to keep working with the currently checked out and compiled
> revision of the meta data, although admittedly reusing the same branch
> name requires some more thought - I'm not exactly sure how to do that
> with monotone.
>

As long as we create the same revision, merging is going to be much
simpler. Even if a dev has to go "back" to commit the patch then they
can simply merge with head and be back up to current.

> Anyway, all of this is idle speculation unless one or more of the core
> developers are willing to adopt such a method.
>

Some comments from monotone developers would be nice. Nathaniel, are
you listening?

-- 
Justin Patrin

Re: A question of workflow

[Patrick Ohly]

On Mon, 2007-01-08 at 11:11 -0800, Justin Patrin wrote:
> On 1/8/07, Patrick Ohly <Patrick.Ohly@gmx.de> wrote:
> > On Sun, 2007-01-07 at 14:46 -0800, Justin Patrin wrote:
> > > One thing I could see us doing is possibly comitting a patch as-is and
> > > then making changes and comitting that. (Of course, we wouldn't push
> > > until it's all finished) but I don't know how well that will merge.
> >
> > I think it would merge as well (or bad) as committing to a branch first.
> > The reason why I didn't mention this possibility is that it means that
> > the core developer's revision of the trunk gets temporarily "polluted"
> > with an unchecked and possibly incomplete patch. That might prevent
> > finishing and pushing some other, more important work. I also don't know
> > how easy it would be to back out the patch completely.
> 
> I'm not sure what making another branch realy buys here.

The main point which makes life easier for external contributors is that
the patch is applied as-is and then modified, with all changes recorded
in monotone. However, you are right: the change must be recorded on
trunk. Making it on a branch and then only propagating the result fails,
see below.

>  In fact, I'm
> not even sure how monotone is happily merging your conflicts seeing as
> the patch would still be coming from 2 places.

I don't know exactly why it worked, but it did in a case where the
normal "reapply your patches" method would have failed.

I also tested another scenario that we have mentioned:
      * the contributor creates two patches changing the same file and
        submits them separately because they fix different problems
        while at the same time commiting them to a private branch
      * a developer commits the first patch literally on the trunk
      * the contributor pulls trunk and propagates to his branch (no
        conflicts)
      * developer commits second patch literally on trunk
      * pulling and propagating again works without conflicts

I think the key reason why this works is that the developer recreates a
revision as it exists on the contributors branch and therefore monotone
doesn't really have to do anything.

Let's try a more complicated scenario:
      * the contributor makes a change to one file, submits patch
      * he makes another change to another file, submits patch
      * OE developer accepts first patch into trunk, then modifies it
      * pulling and propagating to private branch works without
        conflicts, but this time monotone created a merged version that
        the contributor can update to
      * OE developer commits second patch
      * contributor can pull + propagate and his private branch is
        identical with trunk again

Finally a last one:
      * the contributor makes a change to one file, submits patch
      * he makes another change to another file, submits patch
      * OE developer commits the second patch first, on a branch
      * he modifies the same file to fix it
      * finally propagates to trunk
      * contributor pulls and propagates to private branch => now there
        is a conflict which needs to be resolved manually

Okay, so if there is something to be learned from this, then I suppose
it is this:
      * Contributors, keep your changes in a private branch by using
        "monotone commit -b <your branch name>" the first time you
        commit changes. The branch names is remembered for the current
        working copy, so there is no need to always use the "-b". Use
        "mtn pull && mtn propagate org.openembedded.dev <your branch
        name> && mtn update" to follow the main development.
      * OE developers, please, be kind to your contributors and apply
        patches literally, commit them to org.openembedded.dev and only
        then modify them.

> > > OE dev (assuming patch and change is ok):
> > > 5) updates their workspace to revision from 1)
> >
> > Even if the contributor is able to provide a revision, isn't that going
> > to be a lot slower for the core developer? It implies going back to an
> > older revision and then recompiling all sources necessary to check the
> > patch (okay, that could be skipped, but that's not desirable).
> 
> Updating to an older revision isn't hard. It's about the same thing as
> doing a new checkout or creating a new branch.

But recompiling everything that was modified by going to the other
revision is going to take time.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Patrick Ohly schreef:

<snip>

So after every first commmit the tree is in a broken state, where-as if a developers
applied a fixed patch in one go it wouldn't be. Sounds like your concept is badly broken.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFForPqMkyGM64RGpERAhNiAKC0h5k18/9xgqdMpkx1ydjzU/8XQwCglkU1
XeKU9oiSgFx5wjFQMCkHfc0=
=O1dj
-----END PGP SIGNATURE-----

Re: A question of workflow

[Patrick Ohly]

On Mon, 2007-01-08 at 22:13 +0100, Koen Kooi wrote:
> So after every first commmit the tree is in a broken state, where-as if a developers
> applied a fixed patch in one go it wouldn't be.

That's exactly why I had suggested that a developer should commit on a
branch, fix the patch there and then propagate all changes from the
branch. As I learned since then this does not work with monotone because
monotone only propagates the final revision.

What do you think about commiting the original patch, fixing it,
commiting again, and then pushing all these changes? This is the model
suggested by Justin.

>  Sounds like your concept is badly broken.

So what do you suggest should be done instead?

Ignore anything that a revision control system might be able to do and
let external contributors sort it out manually or with quilt/patcher?
Sorry, that doesn't sound very encouraging.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Justin Patrin]

On 1/8/07, Koen Kooi <koen@dominion.kabel.utwente.nl> wrote:
> Patrick Ohly schreef:
>
> <snip>
>
> So after every first commmit the tree is in a broken state, where-as if a developers
> applied a fixed patch in one go it wouldn't be. Sounds like your concept is badly broken.

Apply, commit (note brokenness if needed in commit message), then fix
it, commit, and push. Don't push before fixing.

-- 
Justin Patrin

Re: A question of workflow

[Richard Purdie]

On Mon, 2007-01-08 at 13:53 -0800, Justin Patrin wrote:
> On 1/8/07, Koen Kooi <koen@dominion.kabel.utwente.nl> wrote:
> > Patrick Ohly schreef:
> > So after every first commmit the tree is in a broken state, where-as if a developers
> > applied a fixed patch in one go it wouldn't be. Sounds like your concept is badly broken.
> 
> Apply, commit (note brokenness if needed in commit message), then fix
> it, commit, and push. Don't push before fixing.

The argument against this in other projects like the kernel is that you
just broke the git-bisect method of debugging...

I'm not a fan of broken patches being committed...

Richard

Re: A question of workflow

[Patrick Ohly]

On Tue, 2007-01-09 at 09:42 +0000, Richard Purdie wrote:
> On Mon, 2007-01-08 at 13:53 -0800, Justin Patrin wrote:
> > On 1/8/07, Koen Kooi <koen@dominion.kabel.utwente.nl> wrote:
> > > So after every first commmit the tree is in a broken state, where-as if a developers
> > > applied a fixed patch in one go it wouldn't be. Sounds like your concept is badly broken.
> > 
> > Apply, commit (note brokenness if needed in commit message), then fix
> > it, commit, and push. Don't push before fixing.
> 
> The argument against this in other projects like the kernel is that you
> just broke the git-bisect method of debugging...

That's a valid point, although I am not sure whether the same argument
applies to monotone: an OE developer choosing the next pivot version
manually could be smarter than "git bisect" and always pick the version
which includes the fixes for the immediately preceeding patch
submission.

Anyway, it seems like there are contradicting criteria for what is
desirable when pushing patches, and the ones of the core OE developers
absolutely trump the ones from external contributors. It would have been
nice if there had been a solution which worked for both, but that seems
unlikely.

> I'm not a fan of broken patches being committed...

One last word: perhaps there is a middle ground where functionally
correct patches are applied as they are and then cosmetic changes like
fixing the formating or exchanging lines is applied on top of that.
Obviously, its entirely up to the OE developer importing the patch
whether he wants to do that.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Rolf Leggewie]

Matthew Palmer wrote:
> For example, a few days ago Marcin asked for a patch to make a build of a
> newer version of fakeroot; I'd have whipped that patch up by now except that
> I know that it's going to cause a conflict when the change comes back down,
> so I'm loathe to make the change until I work out how I can manage the patch
> flow

The way I handled this before having rw access was to use quilt and it 
worked nicely for me.  Actually, I still use this for stuff I cannot be 
sure to fix in one setting to commit right after.

Workflow (optimistically assuming no loops and pauses here)

  quilt new some.patch
  quilt edit packages/somepackage/some.bb
  quilt refresh

and then send the patch to the bug tracker.  Before updating monotone, I 
back out all quilt patches (cron does this for me daily)

  quilt pop -a && mtn pull && mtn update && quilt push -a

Maybe this is workable for others as well?

Re: A question of workflow

[Patrick Ohly]

On Mon, 2007-01-08 at 02:18 +0100, Rolf Leggewie wrote:
> The way I handled this before having rw access was to use quilt and it 
> worked nicely for me.  Actually, I still use this for stuff I cannot be 
> sure to fix in one setting to commit right after.
> 
> Workflow (optimistically assuming no loops and pauses here)
> 
>   quilt new some.patch
>   quilt edit packages/somepackage/some.bb
>   quilt refresh
> 
> and then send the patch to the bug tracker.  Before updating monotone, I 
> back out all quilt patches (cron does this for me daily)
> 
>   quilt pop -a && mtn pull && mtn update && quilt push -a
> 
> Maybe this is workable for others as well?

I have done the same before, the only difference was that I used Holger
Schurig's "patcher". The method has some drawbacks:
      * you have to keep track of which patches have been applied, and
        you cannot let tools do it for you
      * you cannot easily(*) find out how your patch was modified after
        you submitted it; this implies that learning how to write better
        patches requires extra effort and thus becomes harder (more
        unlikely?) and that mistakes made by the core developer when
        merging the patch might not be detected by the external
        developer (hey, it might happen, so for the sake of the argument
        bear we me when I mention it ;-)

(*) Instead of one working copy of the sources you would need two, the
upstream version and the last version which you patched yourself. It's
just not the same as having that information in the revision control
system.

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Rolf Leggewie]

Patrick Ohly wrote:
> I have done the same before, the only difference was that I used Holger
> Schurig's "patcher". The method has some drawbacks:
>       * you have to keep track of which patches have been applied, and
>         you cannot let tools do it for you

I am not sure if Holger's patcher can do this, but quilt makes this real 
easy.  "ls -l patches/" or "quilt series" depending on what you want to 
know.

>       * you cannot easily(*) find out how your patch was modified after
>         you submitted it; this implies that learning how to write better
>         patches requires extra effort and thus becomes harder (more
>         unlikely?) and that mistakes made by the core developer when
>         merging the patch might not be detected by the external
>         developer (hey, it might happen, so for the sake of the argument
>         bear we me when I mention it ;-)

This one is a little bit more work but still doable.

mtn can give you the diff between any revision.  Get the one that was 
applied to OE with "mtn diff -r $rev1 -r $rev2", output it to a tmp file 
and compare to your quilt patch.

mtn can also restore the state of your OE tree to any revision you want. 
  "mtn update -r $rev_just_before_your_patch_was_applied;quilt push 
-a;cp $dir_you_were_working_on /tmp;quilt pop -a;mtn update; diff -Naur 
$dir_you_were_working_on /tmp/$dir_you_were_working_on|less"

IMHO this is all easier than the rigid regime you are proposing that no 
one is going to stick to anyways and that the main devs have already 
rejected.  The beauty of it is that you don't have to rely on anyone 
else to behave in a particular way for this work for you.

Re: A question of workflow

[Rolf Leggewie]

Patrick Ohly wrote:
>       * you have to keep track of which patches have been applied, and
>         you cannot let tools do it for you

Did you mean you wanted to know which of your patches have been applied 
in OE?  That one is also easy.  They are the ones that quilt cannot 
apply anymore when doing "quilt push -a" ;-)

Re: A question of workflow

[Patrick Ohly]

On Tue, 2007-01-09 at 13:54 +0100, Rolf Leggewie wrote:
> Patrick Ohly wrote:
> >       * you have to keep track of which patches have been applied, and
> >         you cannot let tools do it for you
> 
> Did you mean you wanted to know which of your patches have been applied 
> in OE?

That's indeed what I meant. Just like quilt, patcher also keeps track of
which patches are currently applied locally.

>   That one is also easy.  They are the ones that quilt cannot 
> apply anymore when doing "quilt push -a" ;-)

Not necessarily. The patch might also fail to apply because unrelated
changes were made upstream, so the external contributor always has to
check these conflicts manually.

On Tue, 2007-01-09 at 13:51 +0100, Rolf Leggewie wrote: 
> >       * you cannot easily(*) find out how your patch was modified after
> >         you submitted it; this implies that learning how to write better
> >         patches requires extra effort and thus becomes harder (more
> >         unlikely?) and that mistakes made by the core developer when
> >         merging the patch might not be detected by the external
> >         developer (hey, it might happen, so for the sake of the argument
> >         bear we me when I mention it ;-)
> 
> This one is a little bit more work but still doable.

Of course, and if there is no better solution I don't mind doing it. But
that doesn't mean that one shouldn't at least search for a better
solution. Too bad there doesn't seem to be any :-/

-- 
Bye, Patrick Ohly
--  
Patrick.Ohly@gmx.de
http://www.estamos.de/

Re: A question of workflow

[Paul Sokolovsky]

Hello Koen,

Sunday, December 31, 2006, 12:07:57 AM, you wrote:

[]
> Do what virtually every DSCM does to support that: make a branch

  Well, that's what I wanted to ask for a long time: if I make my
private branch in the local OE checkout, do I need to do some steps to
ensure it won't be pushed to the upstream repo (assuming I have commit to
.dev branch)?


-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: [Angstrom-devel] A question of workflow

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Sokolovsky schreef:
> Hello Koen,
> 
> Sunday, December 31, 2006, 12:07:57 AM, you wrote:
> 
> []
>> Do what virtually every DSCM does to support that: make a branch
> 
>   Well, that's what I wanted to ask for a long time: if I make my
> private branch in the local OE checkout, do I need to do some steps to
> ensure it won't be pushed to the upstream repo (assuming I have commit to
> .dev branch)?

'mtn push org.openembedded.dev' instead of 'mtn sync', as outlined in the monotone doc:
http://venge.net/monotone/monotone.html

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFmrvMMkyGM64RGpERAn0uAKCB8UWfx2KUJ9zUEzANzDr3FgJM2gCdHnFm
G99bqMalZy6qgUsfDM1JPVs=
=ONFz
-----END PGP SIGNATURE-----

Re: [Angstrom-devel] A question of workflow

[Justin Patrin]

On 1/2/07, Paul Sokolovsky <pmiscml@gmail.com> wrote:
> Hello Koen,
>
> Sunday, December 31, 2006, 12:07:57 AM, you wrote:
>
> []
> > Do what virtually every DSCM does to support that: make a branch
>
>   Well, that's what I wanted to ask for a long time: if I make my
> private branch in the local OE checkout, do I need to do some steps to
> ensure it won't be pushed to the upstream repo (assuming I have commit to
> .dev branch)?
>

What you would do is to create your own branch and propagate from
org.openembedded.dev to your own branch whenever you want the newest
changes. Make changes locally in your branch, put them in the
bugtracker, and when they're applied you get the new versions when you
propagate. Of course, since it's not using your branch revision as the
ancestor and we not always commit your patch as it originally was, you
will have conflicts, but c'est la vie. This is a general trust
problem.

You could also make changes in a local branch and then propagate to
org.openembedded.dev, but in this case you are basically just making
your changes to org.openembedded.dev and if you got push access to our
DB all of a sudden your branch's revisions would be pulled into the
main DB since they are ancestors of revisions in org.openembedded.dev.
The revisions in your branch would also not have branch certs (due to
our limiting of branches you can push) and so would be basically
dangling ancestors. Again, this wouldn't be good for us as it brings
in a lot of revisions that we don't really need.

I suggest option 1 where you propagate from .dev to your own branch.
Yes, you will have conflicts if we alter your patches, but there is
really no way (with monotone at least) to deal with this.

(Essentially what we are talking about here is cherry picking and
knowing where a patch came from. It's a complicated issue and not
somehting that monotone supports directly at the moment.)

-- 
Justin Patrin

Re: A question of workflow

[jack-oe]

[-- Attachment #1: Type: text/plain, Size: 2009 bytes --]

On Sat, Dec 30, 2006 at 07:19:29PM +0000, Richard Purdie wrote:
 
> What you've described is how I or any other OE developer with an
> authorised monotone key would work with OE. The problem is we can't just
> give access to anyone as that wouldn't make any sense for obvious
> reasons.
 
Monotone supports trust relations[0] as well as netsync access controls[1].

Netsync access controls are primarily to prevent someone trashing your
repository. OE restricts write access to the repository on openembedded.org. 

Trust relations are orthogonal to access control. Monotone allows you to only
trust revisions committed by certain users(keys).

In the case of a single trusted repository, that only a certain number of
developers have write access to -- they're the same thing. 

However, in the case where you have multiple repositories, with different
permissions, things can get complicated. There is work ongoing in monotone to
make this simpler[2]. 

> It does mean you shouldn't be committing changes locally via monotone.
> If you do this you will have to pull and then merge every time. That
> isn't a problem in itself but if you do get direct commit access, we
> will not be happy adding hundreds of extra merges to the main
> repository.

Yeah. Or commit your changes to a private branch, and 'propagate' changes there
from the main branch. 

The monotone wiki has a series of (incomplete) best practices[3]. Important are
DaggyFixes[4], and CommitEarlyCommitOften[5]..

Cheers, 

--Jack 

[0] http://venge.net/monotone/wiki/TrustFoundations
[1] section 6.1.3 of http://venge.net/monotone/docs/Hooks.html
[2] http://venge.net/monotone/wiki/VersionedPolicy 
[3] http://venge.net/monotone/wiki/BestPractices
[4] http://venge.net/monotone/wiki/DaggyFixes
[3] http://venge.net/monotone/wiki/CommitEarlyCommitOften

--
Jack (John) Cummings                           http://mudshark.org/
PGP fingerprint: 0774 D073 E386 B70B 6B16  2D2B 1DD8 F8B0 CCF0 FAEE

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: A question of workflow

[Cliff Brake]

On 12/30/06, Matthew Palmer <mpalmer@hezmatt.org> wrote:

> * (The biggie) is there a guide on the wiki or somewhere describing the
>   preferred workflow for making changes and working effectively with the rest
>   of the OE developer community?  (Google wasn't at all illuminating)

Many of us use bbcollections to keep any OE changes in a completely
separate OE tree and don't modify the dev tree.  The components in the
custom tree are given higher priority and in effect replace the
components in the dev tree.  This works very well for the case where
you are working on a project and have quite a bit of custom
development that will never get pushed into dev.  This is far from
ideal, but is the best solution I've found.

Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com