TopGit: problem with patch series generation

TopGit: problem with patch series generation

[martin f krafft]

[-- Attachment #1.1: Type: text/plain, Size: 2454 bytes --]

Hi folks,

I am playing around with TopGit and encountered a (conceptual)
problem. I'd love to hear some input.

I want to use TopGit for distro packaging. Any of my packages have
one or more feature branches, some intended for upstream, some
distro-specific. As I am packaging TopGit for Debian, I encountered
the situation that two branches conflict with each other (they
change the same line), but there is no dependency between the
branches. Thus, when I squash the branches into a series, the
resulting patches will not apply (they both change the same original
line to something else).

Obviously, I can introduce a "fake" dependency to force TopGit to
create one patch based on another. However, this then prevents me
from testing and developing the depending branch in isolation,
meaning that I always have to have the dependent branch applied when
I want to work on the second feature. Furthermore, it's not
trivially possible in this situation to cherry-pick only the second
patch.

I see that this is a hard problem with no obvious solution. The only
thing that comes to my mind is maintaining multiple patches for each
branch. In the above, if B "fake-depends" on A, which depends on
master, then I would have A and B depend on master only, but have
TopGit also manage B2 for me, which is a diff against A.

Doing this for all branches is polynomial, but then again, the
number of independent branches, or rather branch trees, is likely to
be pretty low in most cases.

As an alternative, it may be possible, however, to let TopGit know
about a "fake dependency" from B on A. When serialised, TopGit would
notice that there are multiple paths from master to B (master->B and
master->A->B) and use the longer one.

Do you see any other ways in which the situation could be handled?

Is the above something that TopGit could learn, Petr?

Cheers,

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems
 
"with sufficient thrust, pigs fly just fine. however, this is not
 necessarily a good idea. it is hard to be sure where they are going
 to land, and it could be dangerous sitting under them as they fly
 overhead."
                                                           -- rfc 1925

[-- Attachment #1.2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: TopGit: problem with patch series generation

[Avery Pennarun]

On Tue, Aug 12, 2008 at 12:18 PM, martin f krafft <madduck@debian.org> wrote:
> As I am packaging TopGit for Debian, I encountered
> the situation that two branches conflict with each other (they
> change the same line), but there is no dependency between the
> branches. Thus, when I squash the branches into a series, the
> resulting patches will not apply (they both change the same original
> line to something else).

Isn't this what git-rerere is for?  If TopGit doesn't use rerere,
maybe it would be easy to add...

Have fun,

Avery

Re: TopGit: problem with patch series generation

[Petr Baudis]

  Hi,

On Tue, Aug 12, 2008 at 01:18:54PM -0300, martin f krafft wrote:
> I want to use TopGit for distro packaging. Any of my packages have
> one or more feature branches, some intended for upstream, some
> distro-specific. As I am packaging TopGit for Debian, I encountered
> the situation that two branches conflict with each other (they
> change the same line), but there is no dependency between the
> branches. Thus, when I squash the branches into a series, the
> resulting patches will not apply (they both change the same original
> line to something else).

  yes, that is good point.

> Obviously, I can introduce a "fake" dependency to force TopGit to
> create one patch based on another. However, this then prevents me
> from testing and developing the depending branch in isolation,
> meaning that I always have to have the dependent branch applied when
> I want to work on the second feature. Furthermore, it's not
> trivially possible in this situation to cherry-pick only the second
> patch.

  Well, at least we're not _worse_ off than when using a classical patch
series instead of TopGit, since all the downsides would be the same as
if we had this "fake dependency", right? Though of course, it would be
nice if we could do better here.

> As an alternative, it may be possible, however, to let TopGit know
> about a "fake dependency" from B on A. When serialised, TopGit would
> notice that there are multiple paths from master to B (master->B and
> master->A->B) and use the longer one.

  I'm sorry, I don't follow you here. If there is a dependency, TopGit
will always serialize A first, if there is no dependency, reordering
won't help you since B's base won't accomodate A.

> Do you see any other ways in which the situation could be handled?


  First, let's consider the simplest situation:

	A--C
          /
	B.

(These are branches, not commits!)

  In this case, we _CAN_ actually serialize A,B by doing a "resolution
sling" operation - simply take diff(A,C) instead of diff(B^,B)!

  The trouble is that we will have:

	A1--A2--A3--A4--C
                       /
	B1--B2--B3--B4.

  Here, we would have to squash B1..B4 to B in order to be able to do
this, which is of course undesirable. We want to sling the C resolution
in front of B1, and there seems to be no simple way to do this - or can
anyone see any?

  So we would have to ask the user to propagate it instead. Let's call
these "tie branches":

	A1--A2--A3--A4------------------C
		     \                 /
		      T1--T2--T3--T4  /		tree(T4) == tree(C)
		     /   /   /   /   /
		    B1--B2--B3--B4--'

  I guess this is just a more formalized way of rewording your other
proposal. Then, TopGit, instead of directly setting up C_base by merging
A4+B4, would create the tie branches T1..T4 and use T4 as C_base.

  I'm not really happy about this idea, though. It would complicate
TopGit even much more than it is now, and I'm not sure if this is worth
it instead of just requiring you to maintain your dependencies more
carefully when you intend to serialize your series later.

-- 
				Petr "Pasky" Baudis
The next generation of interesting software will be done
on the Macintosh, not the IBM PC.  -- Bill Gates

Re: TopGit: problem with patch series generation

[martin f krafft]

[-- Attachment #1.1: Type: text/plain, Size: 660 bytes --]

also sprach Avery Pennarun <apenwarr-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> [2008.08.12.1325 -0300]:
> Isn't this what git-rerere is for?  If TopGit doesn't use rerere,
> maybe it would be easy to add...

I'd rather avoid the need to replay conflict resolution and do it in
one place only. It always seemed to me that git-rerere is a solution
to a problem that could have been avoided.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"it is only the modern that ever becomes old-fashioned." 
                                                        -- oscar wilde
 
spamtraps: madduck.bogus-JX/+c5DPh7vR7s880joybQ@public.gmane.org

[-- Attachment #1.2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 207 bytes --]

_______________________________________________
vcs-pkg-discuss mailing list
vcs-pkg-discuss-XbBxUvOt3X2LieD7tvxI8l/i77bcL1HB@public.gmane.org
http://lists.alioth.debian.org/mailman/listinfo/vcs-pkg-discuss

Re: TopGit: problem with patch series generation

[Bert Wesarg]

On Tue, Aug 12, 2008 at 18:25, Avery Pennarun <apenwarr@gmail.com> wrote:
> On Tue, Aug 12, 2008 at 12:18 PM, martin f krafft <madduck@debian.org> wrote:
>> As I am packaging TopGit for Debian, I encountered
>> the situation that two branches conflict with each other (they
>> change the same line), but there is no dependency between the
>> branches. Thus, when I squash the branches into a series, the
>> resulting patches will not apply (they both change the same original
>> line to something else).
>
> Isn't this what git-rerere is for?  If TopGit doesn't use rerere,
> maybe it would be easy to add...
I thought about this too, but have no experience with rerere.

Bert
>
> Have fun,
>
> Avery
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: TopGit: problem with patch series generation

[Santi Béjar]

On Tue, Aug 12, 2008 at 18:18, martin f krafft <madduck@debian.org> wrote:
> Hi folks,
>
> I am playing around with TopGit and encountered a (conceptual)
> problem. I'd love to hear some input.
>
> I want to use TopGit for distro packaging. Any of my packages have
> one or more feature branches, some intended for upstream, some
> distro-specific. As I am packaging TopGit for Debian, I encountered
> the situation that two branches conflict with each other (they
> change the same line), but there is no dependency between the
> branches. Thus, when I squash the branches into a series, the
> resulting patches will not apply (they both change the same original
> line to something else).
>

I don´t know if it fits topgit, but this is what Junio uses:

http://article.gmane.org/gmane.comp.version-control.git/24498

Basically he creates a new feature branch that is the merge of the
conflicting branches (it works for both semantically and explicit
conficts), and there he resolves the conficts. Then if one of the
branches are merged in master (in you case a patch is created) then
the other branch fast-forward to the merge.

HTH,

Santi

Re: TopGit: problem with patch series generation

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 510 bytes --]

also sprach Santi Béjar <sbejar@gmail.com> [2008.08.12.1828 -0300]:
> I don´t know if it fits topgit, but this is what Junio uses:
> 
> http://article.gmane.org/gmane.comp.version-control.git/24498

I think this is definitely something TopGit can automate.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"life is what happens to you while you're busy making other plans."
                                                        -- john lennon
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: TopGit: problem with patch series generation

[Petr Baudis]

On Tue, Aug 12, 2008 at 07:41:55PM -0300, martin f krafft wrote:
> also sprach Santi Béjar <sbejar@gmail.com> [2008.08.12.1828 -0300]:
> > I don´t know if it fits topgit, but this is what Junio uses:
> > 
> > http://article.gmane.org/gmane.comp.version-control.git/24498
> 
> I think this is definitely something TopGit can automate.

This seems to be in principle the same as the tie branches. It might
make sense to have a way to _optionally_ make a tie branch.

How should that work? Maybe there needs to be even an explicit support
for this - should TopGit just check the dependency tree when
sequencing the topic branches and have a step that says:

	"I'm going to sequence branch A. If there is branch T that has
	only already sequenced branches + branch A as dependencies,
	use T's content instead of A."

Would that be satisfactory?

Finding out this information would be very expensive, of course. But for
other reasons, we might want to keep a cache of branch dependencies.

Of course, in the case of

        A1--A2--A3--A4--C
                       /
        B1--B2--B3--B4.

the sequenced branches would still be like

        A1--A2--A3--A4--B1--B2--B3--C

unless you create the T1..T4 branches manually.

-- 
				Petr "Pasky" Baudis
The next generation of interesting software will be done
on the Macintosh, not the IBM PC.  -- Bill Gates

Re: TopGit: problem with patch series generation

[martin f krafft]

[-- Attachment #1.1: Type: text/plain, Size: 660 bytes --]

also sprach Petr Baudis <pasky@suse.cz> [2008.08.12.1432 -0300]:
> I'm not really happy about this idea, though. It would complicate
> TopGit even much more than it is now, and I'm not sure if this is
> worth it instead of just requiring you to maintain your
> dependencies more carefully when you intend to serialize your
> series later.

Sure, but there are no dependencies between some of these branches.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"i worked myself up from nothing to a state of extreme poverty."
                                                       -- groucho marx
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #1.2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: TopGit: problem with patch series generation

[martin f krafft]

[-- Attachment #1.1: Type: text/plain, Size: 1554 bytes --]

also sprach Petr Baudis <pasky-AlSwsSmVLrQ@public.gmane.org> [2008.08.12.1959 -0300]:
> > I think this is definitely something TopGit can automate.
> 
> This seems to be in principle the same as the tie branches. It might
> make sense to have a way to _optionally_ make a tie branch.

Yes, optional would make sense!

> How should that work? Maybe there needs to be even an explicit support
> for this - should TopGit just check the dependency tree when
> sequencing the topic branches and have a step that says:
> 
> 	"I'm going to sequence branch A. If there is branch T that has
> 	only already sequenced branches + branch A as dependencies,
> 	use T's content instead of A."
> 
> Would that be satisfactory?

Yes, that's what I was thinking about, if I read you correctly.

> Of course, in the case of
> 
>         A1--A2--A3--A4--C
>                        /
>         B1--B2--B3--B4.
> 
> the sequenced branches would still be like
> 
>         A1--A2--A3--A4--B1--B2--B3--C
> 
> unless you create the T1..T4 branches manually.

Yes. Or add a dependency. I'd just prefer not to add a dependency
where there is none; instead, I'd prefer if TopGit could be aided
with the serialisation in cases when it cannot possibly make
a proper decision.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"a mathematician is a device for turning coffee into theorems."
                                                         -- paul erdös
 
spamtraps: madduck.bogus-JX/+c5DPh7vR7s880joybQ@public.gmane.org

[-- Attachment #1.2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 207 bytes --]

_______________________________________________
vcs-pkg-discuss mailing list
vcs-pkg-discuss-XbBxUvOt3X2LieD7tvxI8l/i77bcL1HB@public.gmane.org
http://lists.alioth.debian.org/mailman/listinfo/vcs-pkg-discuss

Re: TopGit: problem with patch series generation

[Petr Baudis]

On Tue, Aug 12, 2008 at 08:10:37PM -0300, martin f krafft wrote:
> also sprach Petr Baudis <pasky@suse.cz> [2008.08.12.1959 -0300]:
> > How should that work? Maybe there needs to be even an explicit support
> > for this - should TopGit just check the dependency tree when
> > sequencing the topic branches and have a step that says:
> > 
> > 	"I'm going to sequence branch A. If there is branch T that has
> > 	only already sequenced branches + branch A as dependencies,
> > 	use T's content instead of A."
> > 
> > Would that be satisfactory?
> 
> Yes, that's what I was thinking about, if I read you correctly.

But we _REALLY_ want to do this only for stage branches!

> > Of course, in the case of
> > 
> >         A1--A2--A3--A4--C
> >                        /
> >         B1--B2--B3--B4.
> > 
> > the sequenced branches would still be like
> > 
> >         A1--A2--A3--A4--B1--B2--B3--C
> > 
> > unless you create the T1..T4 branches manually.
> 
> Yes. Or add a dependency. I'd just prefer not to add a dependency
> where there is none; instead, I'd prefer if TopGit could be aided
> with the serialisation in cases when it cannot possibly make
> a proper decision.

Yes, and it should certainly warn you about this at any rate, and give
you a chance to resolve this manually - possibly taking advantage of
rerere.

So, my idea: tg export --quilt should set up and maintain a private
working branch where it merges all the exported branches, one-by-one
(possibly doing the sling as described above first). In case there is a
conflict, it either aborts or gives you a chance to resolve it and go on
with the export. It could also offer you to save your resolution in a
new tie branch for future auto-slinging, but it would be tricky to
figure out exactly what patches does it depend on.

Overally, I'd start simply with implementing the sequential merge and
forget about slinging resolutions from tie branches. The former should
be very simple and solve most of the cases, especially when using rerere.
For the hairy cases, you can just add a dependency for now - sort of
preliminary serialization. :-)

The slinging might be feasible, but would be much more complicated
to implement. I think this can simply wait.

But to be clear, I don't plean to do any of this myself in the near
future anyway, since this case is not that important for me - I now need
TopGit to support remote topic branches instead. So if this is a
priority for you, you need to implement this on your own. But the
sequential merge part should be really easy, just something like

	git checkout -b tmp
	foreach patch_branch
		git merge patch_branch
		cat .topmsg >output/patch.diff
		git diff HEAD^..HEAD >>output/patch.diff

with all the frills. ;-) (Maybe make a special "show diff between X and
Y instead of base and head" flag for tg patch so that we can rely on it
for the frills.)

-- 
				Petr "Pasky" Baudis
The next generation of interesting software will be done
on the Macintosh, not the IBM PC.  -- Bill Gates