[TopGit] Multiple concurrent sets of patches

[TopGit] Multiple concurrent sets of patches

[Jonas Smedegaard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I have run into a challenge that I suspect is a limitation of current 
TopGit. I am hoping you could help enlighten me, as I otherwise feel 
that TopGit is not generally usable for my Debian packaging needs:

How to manage patches against multiple source branches using TopGit?

Let's take netatalk as an example. Upstream only quite infrequently 
release new upstream tarball releases, so I cherry-pick patches from 
upstream VCS. Recently a security-related bug was discovered which 
needed fixing not only in the current packaging development (git master 
branch) but also needed branching off earlier releases of the package 
(those included in the "stable" and "oldstable" Debian distro releases) 
and applying same fix for them.

The actual patch needed for the various branches was obviously not 
identical, as upstream sources were different and my cherry-picked set 
of patches had evolved.

It seems to me that TopGit is incapable of handling this. That it can 
only handle patchset against a single branch, and if the need arise for 
restructuring an additional patchset for e.g. a stable or oldstable 
branch, then quilt needs to be used manually anyway.


   - Jonas

Debian developer

- -- 
* Jonas Smedegaard - idealist og Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

   [x] quote me freely  [ ] ask before reusing  [ ] keep private
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkmtFoUACgkQn7DbMsAkQLgvrACdHfy5K0igPa6Yj/LYyhh3Llyn
jvcAnRfla1QyuUrx8+L4IL9XYY2CB+Su
=B1Kc
-----END PGP SIGNATURE-----

Re: [TopGit] Multiple concurrent sets of patches

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

also sprach Jonas Smedegaard <dr@jones.dk> [2009.03.03.1237 +0100]:
> It seems to me that TopGit is incapable of handling this. That it can 
> only handle patchset against a single branch, and if the need arise for 
> restructuring an additional patchset for e.g. a stable or oldstable 
> branch, then quilt needs to be used manually anyway.

Let me try to understand you: you want TopGit to maintain a single
feature branch against two different source branches? How should
that work? Could you elaborate a bit so that your needs become a bit
more obvious?

Thanks,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"i feel sorry for people who don't drink. when they wake up in the
 morning, that's as good as they're going to feel all day."
                                                    -- frank sinatra
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [TopGit] Multiple concurrent sets of patches

[Jonas Smedegaard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Mar 03, 2009 at 02:03:16PM +0100, martin f krafft wrote:
>also sprach Jonas Smedegaard <dr@jones.dk> [2009.03.03.1237 +0100]:
>> It seems to me that TopGit is incapable of handling this. That it can 
>> only handle patchset against a single branch, and if the need arise 
>> for restructuring an additional patchset for e.g. a stable or 
>> oldstable branch, then quilt needs to be used manually anyway.
>
>Let me try to understand you: you want TopGit to maintain a single
>feature branch against two different source branches? How should
>that work? Could you elaborate a bit so that your needs become a bit
>more obvious?

Not quite. I want TopGit to maintain multiple feature branches, 
preferrably related.

With "related" I mean that I would like to be able to "fork" a chain of 
interdependent feature branches.

TopGit easily support one chain of branches:

upstream + pristine-tar -> master -> build

I want TopGit to additionally support the following:

upstream + pristine-tar -> stable-master -> stable-build

upstream + pristine-tar -> oldstable-master -> oldstable-build


E.g. in addition to TopGit branches t/fix_01 and t/feature_01 I would 
want to fork those branches as t_stable/fix_01 and t_stable/feature_01.


I know that I can create all those TopGit branches one by one, but I 
would then need to explicitly declare a list of TopGit branches to apply 
each time I want to (re)generate a quilt patchlist.


Perhaps what I really am looking for here is something like "tg tag":

    git checkout t/fix_01
    tg tag t/fix_01 master
    git checkout -b t_stable/fix_01 t/fix_01
    tg tag -d master
    tg tag stable
    git checkout stable_build
    ./debian/rules tg-export tags=stable


...or perhaps this is not relevant upstream to TopGit at all but only to 
your packaging of a "tg-export" rule?


    - Jonas

- -- 
* Jonas Smedegaard - idealist og Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

    [x] quote me freely  [ ] ask before reusing  [ ] keep private
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkmtg20ACgkQn7DbMsAkQLhQGQCfQjtfJzzQUu6B0qywpkmxmdGp
66oAnjEtR2Dc/zJ+lMoP3TD3jy1pr1s9
=MwTs
-----END PGP SIGNATURE-----

Re: [TopGit] Multiple concurrent sets of patches

[Uwe Kleine-König]

On Tue, Mar 03, 2009 at 08:22:21PM +0100, Jonas Smedegaard wrote:
> On Tue, Mar 03, 2009 at 02:03:16PM +0100, martin f krafft wrote:
> >also sprach Jonas Smedegaard <dr@jones.dk> [2009.03.03.1237 +0100]:
> >> It seems to me that TopGit is incapable of handling this. That it can 
> >> only handle patchset against a single branch, and if the need arise 
> >> for restructuring an additional patchset for e.g. a stable or 
> >> oldstable branch, then quilt needs to be used manually anyway.
> >
> >Let me try to understand you: you want TopGit to maintain a single
> >feature branch against two different source branches? How should
> >that work? Could you elaborate a bit so that your needs become a bit
> >more obvious?
> 
> Not quite. I want TopGit to maintain multiple feature branches, 
> preferrably related.
> 
> With "related" I mean that I would like to be able to "fork" a chain of 
> interdependent feature branches.
> 
> TopGit easily support one chain of branches:
> 
> upstream + pristine-tar -> master -> build
> 
> I want TopGit to additionally support the following:
> 
> upstream + pristine-tar -> stable-master -> stable-build
> 
> upstream + pristine-tar -> oldstable-master -> oldstable-build
> 
> 
> E.g. in addition to TopGit branches t/fix_01 and t/feature_01 I would 
> want to fork those branches as t_stable/fix_01 and t_stable/feature_01.
> 
> 
> I know that I can create all those TopGit branches one by one, but I 
> would then need to explicitly declare a list of TopGit branches to apply 
> each time I want to (re)generate a quilt patchlist.
For my kernel development I use topgit topic branches that collect
topgit patch branches.  Take this dependency graph:

	linus/master <- t/armmisc/patch1 <- t/armmisc-master <- t/armmisc-pu
	         ^ ^----t/armmisc/patch2 <--'                   |
                 `------t/armmisc/patch3 <----------------------'

So t/armmisc-master collects patches that are ready for upstream, -pu
patches that need some more work.

With etch <- lenny <- squeeze <- sid you could do the same.  The only
Consider you have a security fix that is relevant for all
releases starting at lenny. Then you do

	tg create t/fixes-lenny/CVE-2009-abcd master-lenny
	... apply patch
	... fill .topmsg
	git commit

	git checkout t/fixes-lenny-master
	tg depend add t/fixes-lenny/CVE-2009-abcd

	tg create t/fixes-squeeze/CVE-2009-abcd master-squeeze t/fixes-etch/CVE-2009-abcd
	... no need to apply patch
	... cherry-pick .topmsg from t/fixes-lenny/CVE-2009-abcd
	git commit

	git checkout t/fixes-squeeze-master
	tg depend add t/fixes-squeeze/CVE-2009-abcd
	
And if you notice that etch needs the same fix, you can create
t/fixes-etch/CVE-2009-abcd based on master-etch and then do

	git checkout t/fixes-lenny/CVE-2009-abcd
	tg depend add t/fixes-etch/CVE-2009-abcd

tatata, I think that's it for packaging.

But this doesn't help me for my kernel problem, because here I don't
have that ordering on releases.  I want to manage patches on top of
linux-tip and the ARM tree, but none of these contains the other :-/

Best regards
Uwe

-- 
Pengutronix e.K.                              | Uwe Kleine-König            |
Industrial Linux Solutions                    | http://www.pengutronix.de/  |
Peiner Strasse 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686              | Fax:   +49-5121-206917-5555 |

Re: [TopGit] Multiple concurrent sets of patches

[Jonas Smedegaard]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Mar 03, 2009 at 08:42:32PM +0100, Uwe Kleine-König wrote:
>On Tue, Mar 03, 2009 at 08:22:21PM +0100, Jonas Smedegaard wrote:
>> On Tue, Mar 03, 2009 at 02:03:16PM +0100, martin f krafft wrote:
>> >also sprach Jonas Smedegaard <dr@jones.dk> [2009.03.03.1237 +0100]:
>> >> It seems to me that TopGit is incapable of handling this. That it 
>> >> can only handle patchset against a single branch, and if the need 
>> >> arise for restructuring an additional patchset for e.g. a stable 
>> >> or oldstable branch, then quilt needs to be used manually anyway.
>> >
>> >Let me try to understand you: you want TopGit to maintain a single 
>> >feature branch against two different source branches? How should 
>> >that work? Could you elaborate a bit so that your needs become a bit 
>> >more obvious?
>> 
>> Not quite. I want TopGit to maintain multiple feature branches, 
>> preferrably related.
>> 
>> With "related" I mean that I would like to be able to "fork" a chain 
>> of interdependent feature branches.
>> 
>> TopGit easily support one chain of branches:
>> 
>> upstream + pristine-tar -> master -> build
>> 
>> I want TopGit to additionally support the following:
>> 
>> upstream + pristine-tar -> stable-master -> stable-build
>> 
>> upstream + pristine-tar -> oldstable-master -> oldstable-build
>> 
>> 
>> E.g. in addition to TopGit branches t/fix_01 and t/feature_01 I would 
>> want to fork those branches as t_stable/fix_01 and t_stable/feature_01.
>> 
>> 
>> I know that I can create all those TopGit branches one by one, but I 
>> would then need to explicitly declare a list of TopGit branches to 
>> apply each time I want to (re)generate a quilt patchlist.
>For my kernel development I use topgit topic branches that collect 
>topgit patch branches.  Take this dependency graph:
>
>	linus/master <- t/armmisc/patch1 <- t/armmisc-master <- t/armmisc-pu
>	         ^ ^----t/armmisc/patch2 <--'                   |
>                 `------t/armmisc/patch3 <----------------------'
>
>So t/armmisc-master collects patches that are ready for upstream, -pu
>patches that need some more work.

My head is spinning - the dependency graph in my head was upside down 
compared to your ascii art here.

I think I understand it . Thanks!

I will try it out - I converted ghostscript last week to using TopGit as 
an excercise: good challenge, a nice bunch of patches (but nothing near 
as complex as kernel stuff, I know).


>	tg create t/fixes-squeeze/CVE-2009-abcd master-squeeze t/fixes-etch/CVE-2009-abcd
>	... no need to apply patch
>	... cherry-pick .topmsg from t/fixes-lenny/CVE-2009-abcd
>	git commit

Above puzzles me: Is cherry-picking .topmsg just a lazy way to write 
same description for parallel topicbranch, or is it a trick to cheat 
TopGit into believing that that branch never needs updating?


>But this doesn't help me for my kernel problem, because here I don't 
>have that ordering on releases.  I want to manage patches on top of 
>linux-tip and the ARM tree, but none of these contains the other :-/

I can only begin to imagine the complexities of dealing with parallel 
tracks of kernel patching... Good luck with that!


Thanks a lot for your insight,

  - Jonas

- -- 
* Jonas Smedegaard - idealist og Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

  [x] quote me freely  [ ] ask before reusing  [ ] keep private
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkmtqg0ACgkQn7DbMsAkQLhMFACgkW1FYlWnEP8unk+wVgAkd0i+
UMwAoKT1mkxIRP6XmNKt+Rj3iJn2yUpM
=7x2Z
-----END PGP SIGNATURE-----

Re: [TopGit] Multiple concurrent sets of patches

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1914 bytes --]

also sprach Jonas Smedegaard <dr@jones.dk> [2009.03.03.2022 +0100]:
> I know that I can create all those TopGit branches one by one, but
> I would then need to explicitly declare a list of TopGit branches
> to apply each time I want to (re)generate a quilt patchlist.

There are two ways to achieve what you want with TopGit. Uwe
outlined one way:

- create a new branch depending on all the patch branches you want
  to use. This is what I advocated for packaging in Debian's topgit
  0.3-1 package:
  http://git.debian.org/?p=collab-maint/topgit.git;a=blob;f=debian/README.source;hb=debian/topgit-0.3-1

- declare the list of patches to use, as you suggest. This is what
  the current tg2quilt.mk approach of Debian's topgit 0.5-1 package
  does.

In the context where you have a single debian/rules file to prepare
a quilt series as part of building your package, I think the latter
makes more sense, as it keeps information in debian/rules and
alleviates the user of repetetive steps.

However, in the special context of a security fix, the suggestion
illustrated by Uwe probably makes a lot more sense. One reason for
this is because it is not yet possible to use TopGit patch branches
of the past, meaning that you can only ever use the tip of each, and
that tg-update basically destroys the infrastructure needed to go
back in time. By creating a special depending branch for the
security fix, however, you are preserving the graph at the time,
which is the tag you were alluding to.

This is what I can add to this discussion. I don't think I have
actually fully understood the scope of the problem yet.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"no work of art ever puts forward views.
 views belong to people
 who are not artists."
                                                        -- oscar wilde
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 197 bytes --]