* [Core GIT] Long-term cherrypicking
@ 2005-09-21 16:40 Petr Baudis
2005-09-21 20:57 ` Junio C Hamano
2005-09-22 8:31 ` Greg KH
0 siblings, 2 replies; 8+ messages in thread
From: Petr Baudis @ 2005-09-21 16:40 UTC (permalink / raw)
To: git; +Cc: kkeil
Hello,
I've been working out some workflow possibilities with Karsten Keil at
the SuSE Labs conference, and I'd like to see your opinions. His
position does not seem unique at all, so I'm a bit surprised that noone
solved this before, or at least I didn't find any documentation about
this in the tutorial and howtos (or perhaps I missed it?).
His situation is that he has some patches in the ISDN subsystem, a
public repository, but sends the patches over e-mail to Linus. So he is
something between the subsystem maintainer and individual developer in
the categories listed out in the tutorial. What should be his merging
strategy?
Since he has public repository, he cannot rebase, but he needs to
be cherrypicking. The trouble is, git-cherry really does not work
"through" merges now - it stops at his last merge with Linus. The
obvious solution would be something like this patch:
diff --git a/git-cherry.sh b/git-cherry.sh
--- a/git-cherry.sh
+++ b/git-cherry.sh
@@ -5,18 +5,22 @@
. git-sh-setup || die "Not a git archive."
-usage="usage: $0 "'[-v] <upstream> [<head>]
+usage="usage: $0 "'[-v] <upstream> [<head>] [<base>]
__*__*__*__*__> <upstream>
- /
- fork-point
+ / \
+ <base> \ fork-point
\__+__+__+__+__+__+__+__> <head>
Each commit between the fork-point and <head> is examined, and
-compared against the change each commit between the fork-point and
+compared against the change each commit between the <base> and
<upstream> introduces. If the change seems to be in the upstream,
it is shown on the standard output with prefix "+". Otherwise
it is shown with prefix "-".
+
+If no <base> is specified, it is assumed to be the same as fork-point.
+This is the case if you run git-cherry only before you merge the
+upstream.
'
case "$1" in -v) verbose=t; shift ;; esac
@@ -35,12 +39,16 @@ case "$#" in
2) upstream=`git-rev-parse --verify "$1"` &&
ours=`git-rev-parse --verify "$2"` || exit
;;
+3) upstream=`git-rev-parse --verify "$1"` &&
+ ours=`git-rev-parse --verify "$2"` &&
+ base=`git-rev-parse --verify "$3"` || exit
*) echo >&2 "$usage"; exit 1 ;;
esac
# Note that these list commits in reverse order;
# not that the order in inup matters...
-inup=`git-rev-list ^$ours $upstream` &&
+[ -z "$base" ] && base=$ours
+inup=`git-rev-list ^$base $upstream` &&
ours=`git-rev-list $ours ^$upstream` || exit
tmp=.cherry-tmp$$
Then you would invoke it like
git cherry origin master forkpoint
where forkpoint is a tag pointing at the last known point in Linus'
branch when he had all my (well, Karsten's) patches integrated in his
tree. I tested it briefly and it seems to work - I suppose that I will
leave more testing to him. :-) If GIT wants to be even cooler, it could
remember the last forkpoint and automatically advance it.
Opinions?
PS: Perex hints over my shoulder that the ALSA guys are just manually
recording which patches were merged and which weren't. It's a shame that
GIT has no UI to do that for them if it can. Anyway, I'll be thinking
about if/how to implement the cherrypicking workflow support for Cogito.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-21 16:40 [Core GIT] Long-term cherrypicking Petr Baudis
@ 2005-09-21 20:57 ` Junio C Hamano
2005-09-21 22:19 ` Petr Baudis
2005-09-22 8:31 ` Greg KH
1 sibling, 1 reply; 8+ messages in thread
From: Junio C Hamano @ 2005-09-21 20:57 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
Let me see if I understand what you are trying to solve.
S is subsystem line of development, L is Linus line of
development, M is a merge from L into S.
A' and B' are resulting commits from e-mail patch, that contain
changes introduced to S by A and B commits, respectively.
S: A - B - C - M - D - ...
/ /
L: O- X - A'- Y - Z - ...
After sending off patches A and B, L accepts only A and creates
commit A'. After a while, S pulls from L and merges at M [*1*].
In the meantime, there are other changes on both ends (C and Y).
Also note that I am not drawing commits in S that are not to
be sent to L via e-mail nor pull request -- there may be other
commits on lines between B and C, C and M, etc.
M contains changes B and C which are lacking from L. On top of
that, a commit D is made and S wants to ask L to incorporate
changes in S but not in L (B, C and D), but finding B, C and D
is cumbersome [*2*]. You solved this by 'git cherry' to go
beyond M, which is a valid workaround, but then the problem
becomes how to find where to stop.
I am assuming the reason this is a cherry-pick problem is
because S does not necessarily send everything out, and whenever
S does 'format-patch L' to generate a list of patches, only a
subset of them is actually sent out. If that is the case, how
about maintaining another branch T on the S side, which is
private to S?
T starts out as the same commit as the head of L. Whenever S
has merge-worthy commits, run 'git-cherry-pick' to copy them
over to T, and 'format-patch L' is run in T, not in S, to
generate patches to be sent out.
T being a private tree allows you to rebase every once in a
while to L [*3*], so that already applied patches are dropped
from it.
[Footnote]
*1* I am assuming that merge M would not be made any more
difficult by having A and A' on both sides. Is it a problem in
practice?
*2* Finding B for resend may be debatable -- it is more of an
inter developer communication issue.
*3* For the above workflow to work, I think the current 'git
rebase' needs to be improved much in two aspects. (1) The
current 'cherry' uses 'git-patch-id' which shares the same
strictness with 'git-apply' and would not tolerate much fuzz
between what you send out and what got actually applied. (2)
'rebase' keeps going when a merge is not cleanly done, noting
them to sort them out by hand at the end. This loses the order
of commits in the tree being rebased and very unwieldy. I have
a plan to fix this in the TODO list for some time.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-21 20:57 ` Junio C Hamano
@ 2005-09-21 22:19 ` Petr Baudis
2005-09-21 23:26 ` Junio C Hamano
0 siblings, 1 reply; 8+ messages in thread
From: Petr Baudis @ 2005-09-21 22:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Dear diary, on Wed, Sep 21, 2005 at 10:57:04PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> You solved this by 'git cherry' to go
> beyond M, which is a valid workaround, but then the problem
> becomes how to find where to stop.
When forking your branch, record that in a tag. When running git-cherry
with the tag as base, take the earliest cherrypick found and find the
LCA of the two corresponding commits. Reset the tag to that.
> I am assuming the reason this is a cherry-pick problem is
> because S does not necessarily send everything out, and whenever
> S does 'format-patch L' to generate a list of patches, only a
> subset of them is actually sent out. If that is the case, how
> about maintaining another branch T on the S side, which is
> private to S?
That is not the case, this is a cherry-pick problem since L dropped some
of the patches (if the git's definition of "cherry-pick problem" isn't
something different to what I'd expect)...
> T starts out as the same commit as the head of L. Whenever S
> has merge-worthy commits, run 'git-cherry-pick' to copy them
> over to T, and 'format-patch L' is run in T, not in S, to
> generate patches to be sent out.
>
> T being a private tree allows you to rebase every once in a
> while to L [*3*], so that already applied patches are dropped
> from it.
...nevertheless your solution still seems to apply and sounds good to
me, thanks. I wonder why this didn't occur to me. :-) Perhaps it could
be noted in the tutorial?
The disadvantage is that this seems as a lot of work. You have to
explicitly cherry-pick each commit, and the fact that you have a
floating (continuously rebased) tree means that you can conveniently
keep it only on just one machine. [*1*]
The advantage is that it is easy to "forget" commits which were dropped
altogether or applied modified. You could put together some filter and
grep -v, but that's ugly and doesn't scale.
Well, I guess that if you are going to do this all, it really gets
visible that this rebasing stuff is really quite an ugly hack and that
you are probably much better off with something like StGIT. ;-)
> [Footnote]
>
> *1* I am assuming that merge M would not be made any more
> difficult by having A and A' on both sides. Is it a problem in
> practice?
That's something Karsten Keil would be more qualified to reply to, but
unfortunately you removed him from the Cc list. ;-)
*1* I find the fast-forward restriction more and more annoying in this
context, actually, and I'm seriously considering dropping it from Cogito
altogether. GIT seems to care a lot about it, but Cogito doesn't so far,
it would only have problems fast-forwarding your master to catch up with
the branch you are merging, trying to do a tree merge instead. But
if old_ref == master then master = new_ref
would solve that. Can it ever really hurt?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-21 22:19 ` Petr Baudis
@ 2005-09-21 23:26 ` Junio C Hamano
0 siblings, 0 replies; 8+ messages in thread
From: Junio C Hamano @ 2005-09-21 23:26 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
Petr Baudis <pasky@suse.cz> writes:
> Well, I guess that if you are going to do this all, it really gets
> visible that this rebasing stuff is really quite an ugly hack and that
> you are probably much better off with something like StGIT. ;-)
Yes.
> *1* I find the fast-forward restriction more and more annoying in this
> context, actually, and I'm seriously considering dropping it from Cogito
> altogether.
My "pu" is always floating and I have this in my
.git/remotes/ko:
URL: master.kernel.org:/pub/scm/git/git.git/
Push: master maint rc +pu
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-21 16:40 [Core GIT] Long-term cherrypicking Petr Baudis
2005-09-21 20:57 ` Junio C Hamano
@ 2005-09-22 8:31 ` Greg KH
2005-09-22 9:53 ` Catalin Marinas
1 sibling, 1 reply; 8+ messages in thread
From: Greg KH @ 2005-09-22 8:31 UTC (permalink / raw)
To: Petr Baudis; +Cc: git, kkeil
On Wed, Sep 21, 2005 at 06:40:15PM +0200, Petr Baudis wrote:
> Hello,
>
> I've been working out some workflow possibilities with Karsten Keil at
> the SuSE Labs conference, and I'd like to see your opinions. His
> position does not seem unique at all, so I'm a bit surprised that noone
> solved this before, or at least I didn't find any documentation about
> this in the tutorial and howtos (or perhaps I missed it?).
>
> His situation is that he has some patches in the ISDN subsystem, a
> public repository, but sends the patches over e-mail to Linus. So he is
> something between the subsystem maintainer and individual developer in
> the categories listed out in the tutorial. What should be his merging
> strategy?
Not to use git for this.
Seriously, that's what I have switched to doing, and it's so much
easier. I use quilt to manage patches from the community, and Andrew
pulls them into to the -mm releases, and all users can test them. I
keep them up to date with the git snapshots, which handles the different
merge and fuzz issues very well.
Then, when it's time to merge with Linus, I pick and choose the patches
that I want to send off, create a git tree, add them to the tree, and
send them off.
It's ended up saving me a lot of time (I used to do what Karsten is
trying to do with bitkeeper, and it had the same issues that he is
running into) and would recommend this situation for anyone who wants to
keep patches from being merged immediatly (like he is trying to do.)
thanks,
greg k-h
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-22 8:31 ` Greg KH
@ 2005-09-22 9:53 ` Catalin Marinas
2005-09-22 9:58 ` Petr Baudis
0 siblings, 1 reply; 8+ messages in thread
From: Catalin Marinas @ 2005-09-22 9:53 UTC (permalink / raw)
To: Greg KH; +Cc: Petr Baudis, git, kkeil
Greg KH <greg@kroah.com> wrote:
> On Wed, Sep 21, 2005 at 06:40:15PM +0200, Petr Baudis wrote:
>> His situation is that he has some patches in the ISDN subsystem, a
>> public repository, but sends the patches over e-mail to Linus. So he is
>> something between the subsystem maintainer and individual developer in
>> the categories listed out in the tutorial. What should be his merging
>> strategy?
>
> Not to use git for this.
>
> Seriously, that's what I have switched to doing, and it's so much
> easier. I use quilt to manage patches from the community, and Andrew
> pulls them into to the -mm releases, and all users can test them. I
> keep them up to date with the git snapshots, which handles the different
> merge and fuzz issues very well.
You might want to try StGIT as well, unless you have a huge number of
patches and rebasing them takes too much time. It follows the quilt
ideas but it does a three-way merge instead of patch+fuzz. This allows
it to detect patches accepted by Linus and also see whether they were
not fully applied or were modified. You can run 'stg export' to
generate a patch series to send to Andrew.
> Then, when it's time to merge with Linus, I pick and choose the patches
> that I want to send off, create a git tree, add them to the tree, and
> send them off.
With StGIT, you can either use 'stg mail <patches...>' or re-organise
the stack with push/pop so that you only keep the patches to be merged
by Linus and tell him the HEAD value to merge. Even if you modify the
stack afterwards, the HEAD value still represents the stack state at
that moment (unless you run 'git prune' and the value is not saved in
a file under refs/heads/).
StGIT also support cherry-picking via 'stg import --commit=...' if you
want to create a separate branch.
But what I understand from Petr's e-mail is that re-basing the patches
is not acceptable for a repository which is public. One option would
be to keep a private tree where the patches are re-based and a public
one which periodically pulls from Linus' tree and your private
one. Not sure how complicated the commit graph would look.
> It's ended up saving me a lot of time (I used to do what Karsten is
> trying to do with bitkeeper, and it had the same issues that he is
> running into) and would recommend this situation for anyone who wants to
> keep patches from being merged immediatly (like he is trying to do.)
I only maintain a small set of patches (< 20) and send some of them
upstream via RMK. I would recommend StGIT :-) as an alternative to
quilt (I don't say a better one but some people might like it).
--
Catalin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-22 9:53 ` Catalin Marinas
@ 2005-09-22 9:58 ` Petr Baudis
2005-09-22 10:17 ` Catalin Marinas
0 siblings, 1 reply; 8+ messages in thread
From: Petr Baudis @ 2005-09-22 9:58 UTC (permalink / raw)
To: Catalin Marinas; +Cc: Greg KH, git, kkeil
Dear diary, on Thu, Sep 22, 2005 at 11:53:27AM CEST, I got a letter
where Catalin Marinas <catalin.marinas@gmail.com> told me that...
> But what I understand from Petr's e-mail is that re-basing the patches
> is not acceptable for a repository which is public. One option would
> be to keep a private tree where the patches are re-based and a public
> one which periodically pulls from Linus' tree and your private
> one. Not sure how complicated the commit graph would look.
It would be pretty ugly since the same patches would re-appear in the
commit graph in another incarnations many times.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Core GIT] Long-term cherrypicking
2005-09-22 9:58 ` Petr Baudis
@ 2005-09-22 10:17 ` Catalin Marinas
0 siblings, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2005-09-22 10:17 UTC (permalink / raw)
To: Petr Baudis; +Cc: Greg KH, git, kkeil
Petr Baudis <pasky@suse.cz> wrote:
> Dear diary, on Thu, Sep 22, 2005 at 11:53:27AM CEST, I got a letter
> where Catalin Marinas <catalin.marinas@gmail.com> told me that...
>> But what I understand from Petr's e-mail is that re-basing the patches
>> is not acceptable for a repository which is public. One option would
>> be to keep a private tree where the patches are re-based and a public
>> one which periodically pulls from Linus' tree and your private
>> one. Not sure how complicated the commit graph would look.
>
> It would be pretty ugly since the same patches would re-appear in the
> commit graph in another incarnations many times.
True. And re-basing patches in the public branch would generate the
same complicated graph in people's repositories which were pulling
from it. If this is not desirable, StGIT on the public branch wouldn't
help much (and neither quilt since quilt patches are not visible via
GIT anyway).
As Junio said, using 'git cherry-pick' (or 'stg import --commit') on a
private branch would help organising the patches to be sent
upstream. One problem with this approach is that the conflicts
generated by merging with Linus in the public branch (and already
fixed there) would need to be re-solved in the private one after
cherry-picking.
--
Catalin
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2005-09-22 10:19 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-09-21 16:40 [Core GIT] Long-term cherrypicking Petr Baudis
2005-09-21 20:57 ` Junio C Hamano
2005-09-21 22:19 ` Petr Baudis
2005-09-21 23:26 ` Junio C Hamano
2005-09-22 8:31 ` Greg KH
2005-09-22 9:53 ` Catalin Marinas
2005-09-22 9:58 ` Petr Baudis
2005-09-22 10:17 ` Catalin Marinas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).