* Efficiency and correctness patches for git-svn mergeinfo support
@ 2009-12-19 16:33 Sam Vilain
2009-12-19 16:33 ` [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures Sam Vilain
` (2 more replies)
0 siblings, 3 replies; 13+ messages in thread
From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw)
To: git; +Cc: Eric Wong, Andrew Myrick
This series implements some efficiency enhancements; particularly in
dealing with repositories which have a single trunk branch which
receives many merges of feature branches and/or cherry-picks. It also
fixes a number of corner cases in the merge conversion code.
It would be nice if the people who have experienced slow git-svn
performance in these situations could test that this fixes the
performance issues, and that the resulting repositories seem to have
correct contents.
Eric, I'm using Alex's trick of a single commit which adds failing
tests - marked as _expect_failure - and then marked them as succeeding
as the series progresses.
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures 2009-12-19 16:33 Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain @ 2009-12-19 16:33 ` Sam Vilain 2009-12-19 16:33 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-19 16:42 ` Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain 2009-12-19 22:15 ` Andrew Myrick 2 siblings, 1 reply; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick, Sam Vilain As shown, git-svn has some problems; not all svn merges are correctly detected, and cherry picks may incorrectly be detected as real merges. These test cases will be marked as _success once the relevant fixes are in. Signed-off-by: Sam Vilain <sam@vilain.net> --- t/t9151-svn-mergeinfo.sh | 27 ++- t/t9151/make-svnmerge-dump | 166 ++++++--- t/t9151/svn-mergeinfo.dump | 839 ++++++++++++++++++++++++++++++++++++++------ 3 files changed, 860 insertions(+), 172 deletions(-) diff --git a/t/t9151-svn-mergeinfo.sh b/t/t9151-svn-mergeinfo.sh index f57daf4..dc3478f 100755 --- a/t/t9151-svn-mergeinfo.sh +++ b/t/t9151-svn-mergeinfo.sh @@ -15,12 +15,27 @@ test_expect_success 'load svn dump' " git svn fetch --all " -test_expect_success 'represent svn merges without intervening commits' " - [ `git cat-file commit HEAD^1 | grep parent | wc -l` -eq 2 ] - " +test_expect_failure 'all svn merges became git merge commits' ' + unmarked=$(git rev-list --parents --all --grep=Merge | + grep -v " .* " | cut -f1 -d" ") + [ -z "$unmarked" ] + ' -test_expect_success 'represent svn merges with intervening commits' " - [ `git cat-file commit HEAD | grep parent | wc -l` -eq 2 ] - " +test_expect_failure 'cherry picks did not become git merge commits' ' + bad_cherries=$(git rev-list --parents --all --grep=Cherry | + grep " .* " | cut -f1 -d" ") + [ -z "$bad_cherries" ] + ' + +test_expect_success 'svn non-merge merge commits did not become git merge commits' ' + bad_non_merges=$(git rev-list --parents --all --grep=non-merge | + grep " .* " | cut -f1 -d" ") + [ -z "$bad_non_merges" ] + ' + +test_expect_failure 'everything got merged in the end' ' + unmerged=$(git rev-list --all --not master) + [ -z "$unmerged" ] + ' test_done diff --git a/t/t9151/make-svnmerge-dump b/t/t9151/make-svnmerge-dump index 7e3da75..d917717 100644 --- a/t/t9151/make-svnmerge-dump +++ b/t/t9151/make-svnmerge-dump @@ -11,93 +11,151 @@ mkdir foo.svn svnadmin create foo.svn svn co file://`pwd`/foo.svn foo +commit() { + i=$(( $1 + 1 )) + shift; + svn commit -m "(r$i) $*" >/dev/null || exit 1 + echo $i +} + +say() { + echo "^[[1m * $*^[[0m" +} + +i=0 cd foo mkdir trunk mkdir branches svn add trunk branches -svn commit -m "Setup trunk and branches" -cd trunk +i=$(commit $i "Setup trunk and branches") -git cat-file blob 6683463e:Makefile > Makefile -svn add Makefile +git cat-file blob 6683463e:Makefile > trunk/Makefile +svn add trunk/Makefile -echo "Committing ANCESTOR" -svn commit -m "ancestor" -cd .. +say "Committing ANCESTOR" +i=$(commit $i "ancestor") svn cp trunk branches/left -echo "Committing BRANCH POINT" -svn commit -m "make left branch" +say "Committing BRANCH POINT" +i=$(commit $i "make left branch") svn cp trunk branches/right -echo "Committing other BRANCH POINT" -svn commit -m "make right branch" -cd branches/left/ +say "Committing other BRANCH POINT" +i=$(commit $i "make right branch") -#$sm init -#svn commit -m "init svnmerge" +say "Committing LEFT UPDATE" +git cat-file blob 5873b67e:Makefile > branches/left/Makefile +i=$(commit $i "left update 1") -git cat-file blob 5873b67e:Makefile > Makefile -echo "Committing BRANCH UPDATE 1" -svn commit -m "left update 1" -cd ../.. - -cd trunk -git cat-file blob 75118b13:Makefile > Makefile -echo "Committing TRUNK UPDATE" -svn commit -m "trunk update" +git cat-file blob 75118b13:Makefile > branches/right/Makefile +say "Committing RIGHT UPDATE" +pre_right_update_1=$i +i=$(commit $i "right update 1") -cd ../branches/left -git cat-file blob ff5ebe39:Makefile > Makefile -echo "Committing BRANCH UPDATE 2" -svn commit -m "left update 2" +say "Making more commits on LEFT" +git cat-file blob ff5ebe39:Makefile > branches/left/Makefile +i=$(commit $i "left update 2") +git cat-file blob b5039db6:Makefile > branches/left/Makefile +i=$(commit $i "left update 3") -git cat-file blob b5039db6:Makefile > Makefile -echo "Committing BRANCH UPDATE 3" -svn commit -m "left update 3" +say "Making a LEFT SUB-BRANCH" +svn cp branches/left branches/left-sub +sub_left_make=$i +i=$(commit $i "make left sub-branch") -# merge to trunk +say "Making a commit on LEFT SUB-BRANCH" +echo "crunch" > branches/left-sub/README +svn add branches/left-sub/README +i=$(commit $i "left sub-branch update 1") -cd ../.. +say "Merging LEFT to TRUNK" svn update cd trunk - svn merge ../branches/left --accept postpone - -git cat-file blob b51ad431:Makefile > Makefile - +git cat-file blob b5039db6:Makefile > Makefile svn resolved Makefile +i=$(commit $i "Merge left to trunk 1") +cd .. -svn commit -m "Merge trunk 1" - -# create commits on both branches - -cd ../branches/left -git cat-file blob ff5ebe39:Makefile > Makefile -echo "Committing BRANCH UPDATE 4" -svn commit -m "left update 4" - -cd ../right -git cat-file blob b5039db6:Makefile > Makefile -echo "Committing other BRANCH UPDATE 1" -svn commit -m "right update 1" +say "Making more commits on LEFT and RIGHT" +echo "touche" > branches/left/zlonk +svn add branches/left/zlonk +i=$(commit $i "left update 4") +echo "thwacke" > branches/right/bang +svn add branches/right/bang +i=$(commit $i "right update 2") -# merge to trun again +say "Squash merge of RIGHT tip 2 commits onto TRUNK" +svn update +cd trunk +svn merge -r$pre_right_update_1:$i ../branches/right +i=$(commit $i "Cherry-pick right 2 commits to trunk") +cd .. -cd ../.. +say "Merging RIGHT to TRUNK" svn update cd trunk +svn merge ../branches/right --accept postpone +git cat-file blob b51ad431:Makefile > Makefile +svn resolved Makefile +i=$(commit $i "Merge right to trunk 1") +cd .. -svn merge ../branches/left --accept postpone +say "Making more commits on RIGHT and TRUNK" +echo "whamm" > branches/right/urkkk +svn add branches/right/urkkk +i=$(commit $i "right update 3") +echo "pow" > trunk/vronk +svn add trunk/vronk +i=$(commit $i "trunk update 1") +say "Merging RIGHT to LEFT SUB-BRANCH" +svn update +cd branches/left-sub +svn merge ../right --accept postpone git cat-file blob b51ad431:Makefile > Makefile - svn resolved Makefile +i=$(commit $i "Merge right to left sub-branch") +cd ../.. -svn commit -m "Merge trunk 2" +say "Making more commits on LEFT SUB-BRANCH and LEFT" +echo "zowie" > branches/left-sub/wham_eth +svn add branches/left-sub/wham_eth +pre_sub_left_update_2=$i +i=$(commit $i "left sub-branch update 2") +sub_left_update_2=$i +echo "eee_yow" > branches/left/glurpp +svn add branches/left/glurpp +i=$(commit $i "left update 5") + +say "Cherry pick LEFT SUB-BRANCH commit to LEFT" +svn update +cd branches/left +svn merge -r$pre_sub_left_update_2:$sub_left_update_2 ../left-sub +i=$(commit $i "Cherry-pick left sub-branch commit to left") +cd ../.. +say "Merging LEFT SUB-BRANCH back to LEFT" +svn update +cd branches/left +# it's only a merge because the previous merge cherry-picked the top commit +svn merge -r$sub_left_make:$sub_left_update_2 ../left-sub --accept postpone +i=$(commit $i "Merge left sub-branch to left") cd ../.. +say "Merging EVERYTHING to TRUNK" +svn update +cd trunk +svn merge ../branches/left --accept postpone +svn resolved bang +i=$(commit $i "Merge left to trunk 2") +# this merge, svn happily updates the mergeinfo, but there is actually +# nothing to merge. git-svn will not make a meaningless merge commit. +svn merge ../branches/right --accept postpone +i=$(commit $i "non-merge right to trunk 2") +cd .. + +cd .. svnadmin dump foo.svn > svn-mergeinfo.dump rm -rf foo foo.svn diff --git a/t/t9151/svn-mergeinfo.dump b/t/t9151/svn-mergeinfo.dump index 11a883f..9543e31 100644 --- a/t/t9151/svn-mergeinfo.dump +++ b/t/t9151/svn-mergeinfo.dump @@ -1,6 +1,6 @@ SVN-fs-dump-format-version: 2 -UUID: 1530d5a2-a1dc-4438-8ad5-d95e96db8945 +UUID: 64142547-0943-4db2-836a-d1e1eb2f9924 Revision-number: 0 Prop-content-length: 56 @@ -9,25 +9,25 @@ Content-length: 56 K 8 svn:date V 27 -2009-11-12T20:29:38.812226Z +2009-12-19T16:17:51.232640Z PROPS-END Revision-number: 1 -Prop-content-length: 127 -Content-length: 127 +Prop-content-length: 128 +Content-length: 128 K 7 svn:log -V 24 -Setup trunk and branches +V 29 +(r1) Setup trunk and branches K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:39.045856Z +2009-12-19T16:17:51.831965Z PROPS-END Node-path: branches @@ -49,21 +49,21 @@ PROPS-END Revision-number: 2 -Prop-content-length: 110 -Content-length: 110 +Prop-content-length: 112 +Content-length: 112 K 7 svn:log -V 8 -ancestor +V 13 +(r2) ancestor K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:40.079587Z +2009-12-19T16:17:52.300075Z PROPS-END Node-path: trunk/Makefile @@ -156,21 +156,21 @@ backup: clean Revision-number: 3 -Prop-content-length: 119 -Content-length: 119 +Prop-content-length: 120 +Content-length: 120 K 7 svn:log -V 16 -make left branch +V 21 +(r3) make left branch K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:42.084439Z +2009-12-19T16:17:52.768800Z PROPS-END Node-path: branches/left @@ -190,21 +190,21 @@ Text-copy-source-sha1: 103205ce331f7d64086dba497574734f78439590 Revision-number: 4 -Prop-content-length: 120 -Content-length: 120 +Prop-content-length: 121 +Content-length: 121 K 7 svn:log -V 17 -make right branch +V 22 +(r4) make right branch K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:44.065452Z +2009-12-19T16:17:53.177879Z PROPS-END Node-path: branches/right @@ -224,21 +224,21 @@ Text-copy-source-sha1: 103205ce331f7d64086dba497574734f78439590 Revision-number: 5 -Prop-content-length: 116 -Content-length: 116 +Prop-content-length: 117 +Content-length: 117 K 7 svn:log -V 13 -left update 1 +V 18 +(r5) left update 1 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:45.066262Z +2009-12-19T16:17:53.604691Z PROPS-END Node-path: branches/left/Makefile @@ -329,24 +329,24 @@ backup: clean Revision-number: 6 -Prop-content-length: 115 -Content-length: 115 +Prop-content-length: 118 +Content-length: 118 K 7 svn:log -V 12 -trunk update +V 19 +(r6) right update 1 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:46.278498Z +2009-12-19T16:17:54.063555Z PROPS-END -Node-path: trunk/Makefile +Node-path: branches/right/Makefile Node-kind: file Node-action: change Text-content-length: 2521 @@ -437,21 +437,21 @@ backup: clean Revision-number: 7 -Prop-content-length: 116 -Content-length: 116 +Prop-content-length: 117 +Content-length: 117 K 7 svn:log -V 13 -left update 2 +V 18 +(r7) left update 2 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:47.069090Z +2009-12-19T16:17:54.523904Z PROPS-END Node-path: branches/left/Makefile @@ -542,21 +542,21 @@ backup: clean Revision-number: 8 -Prop-content-length: 116 -Content-length: 116 +Prop-content-length: 117 +Content-length: 117 K 7 svn:log -V 13 -left update 3 +V 18 +(r8) left update 3 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:48.053835Z +2009-12-19T16:17:54.975970Z PROPS-END Node-path: branches/left/Makefile @@ -647,33 +647,285 @@ backup: clean Revision-number: 9 -Prop-content-length: 116 -Content-length: 116 +Prop-content-length: 124 +Content-length: 124 K 7 svn:log -V 13 -Merge trunk 1 +V 25 +(r9) make left sub-branch +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:55.459904Z +PROPS-END + +Node-path: branches/left-sub +Node-kind: dir +Node-action: add +Node-copyfrom-rev: 3 +Node-copyfrom-path: branches/left + + +Node-path: branches/left-sub/Makefile +Node-kind: file +Node-action: delete + +Node-path: branches/left-sub/Makefile +Node-kind: file +Node-action: add +Node-copyfrom-rev: 8 +Node-copyfrom-path: branches/left/Makefile +Text-copy-source-md5: 5ccff689fb290e00b85fe18ee50c54ba +Text-copy-source-sha1: a13de8e23f1483efca3e57b2b64b0ae6f740ce10 + + + + +Revision-number: 10 +Prop-content-length: 129 +Content-length: 129 + +K 7 +svn:log +V 30 +(r10) left sub-branch update 1 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:51.098306Z +2009-12-19T16:17:55.862113Z +PROPS-END + +Node-path: branches/left-sub/README +Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 7 +Text-content-md5: fdbcfb6be9afe1121862143f226b51cf +Text-content-sha1: 1d1f5ea4ceb584337ffe59b8980d92e3b78dfef4 +Content-length: 17 + +PROPS-END +crunch + + +Revision-number: 11 +Prop-content-length: 126 +Content-length: 126 + +K 7 +svn:log +V 27 +(r11) Merge left to trunk 1 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:56.413416Z PROPS-END Node-path: trunk Node-kind: dir Node-action: change -Prop-content-length: 53 -Content-length: 53 +Prop-content-length: 54 +Content-length: 54 K 13 svn:mergeinfo -V 18 -/branches/left:2-8 +V 19 +/branches/left:2-10 +PROPS-END + + +Node-path: trunk/Makefile +Node-kind: file +Node-action: change +Text-content-length: 2593 +Text-content-md5: 5ccff689fb290e00b85fe18ee50c54ba +Text-content-sha1: a13de8e23f1483efca3e57b2b64b0ae6f740ce10 +Content-length: 2593 + +# -DCOLLISION_CHECK if you believe that SHA1's +# 1461501637330902918203684832716283019655932542976 hashes do not give you +# enough guarantees about no collisions between objects ever hapenning. +# +# -DNSEC if you want git to care about sub-second file mtimes and ctimes. +# Note that you need some new glibc (at least >2.2.4) for this, and it will +# BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely randomly +# break unless your underlying filesystem supports those sub-second times +# (my ext3 doesn't). +CFLAGS=-g -O3 -Wall + +CC=gcc + + +PROG= update-cache show-diff init-db write-tree read-tree commit-tree \ + cat-file fsck-cache checkout-cache diff-tree rev-tree show-files \ + check-files ls-tree merge-base + +all: $(PROG) + +install: $(PROG) + install $(PROG) $(HOME)/bin/ + +LIBS= -lssl -lz + +init-db: init-db.o + +update-cache: update-cache.o read-cache.o + $(CC) $(CFLAGS) -o update-cache update-cache.o read-cache.o $(LIBS) + +show-diff: show-diff.o read-cache.o + $(CC) $(CFLAGS) -o show-diff show-diff.o read-cache.o $(LIBS) + +write-tree: write-tree.o read-cache.o + $(CC) $(CFLAGS) -o write-tree write-tree.o read-cache.o $(LIBS) + +read-tree: read-tree.o read-cache.o + $(CC) $(CFLAGS) -o read-tree read-tree.o read-cache.o $(LIBS) + +commit-tree: commit-tree.o read-cache.o + $(CC) $(CFLAGS) -o commit-tree commit-tree.o read-cache.o $(LIBS) + +cat-file: cat-file.o read-cache.o + $(CC) $(CFLAGS) -o cat-file cat-file.o read-cache.o $(LIBS) + +fsck-cache: fsck-cache.o read-cache.o object.o commit.o tree.o blob.o + $(CC) $(CFLAGS) -o fsck-cache fsck-cache.o read-cache.o object.o commit.o tree.o blob.o $(LIBS) + +checkout-cache: checkout-cache.o read-cache.o + $(CC) $(CFLAGS) -o checkout-cache checkout-cache.o read-cache.o $(LIBS) + +diff-tree: diff-tree.o read-cache.o + $(CC) $(CFLAGS) -o diff-tree diff-tree.o read-cache.o $(LIBS) + +rev-tree: rev-tree.o read-cache.o object.o commit.o tree.o blob.o + $(CC) $(CFLAGS) -o rev-tree rev-tree.o read-cache.o object.o commit.o tree.o blob.o $(LIBS) + +show-files: show-files.o read-cache.o + $(CC) $(CFLAGS) -o show-files show-files.o read-cache.o $(LIBS) + +check-files: check-files.o read-cache.o + $(CC) $(CFLAGS) -o check-files check-files.o read-cache.o $(LIBS) + +ls-tree: ls-tree.o read-cache.o + $(CC) $(CFLAGS) -o ls-tree ls-tree.o read-cache.o $(LIBS) + +merge-base: merge-base.o read-cache.o object.o commit.o tree.o blob.o + $(CC) $(CFLAGS) -o merge-base merge-base.o read-cache.o object.o commit.o tree.o blob.o $(LIBS) + +read-cache.o: cache.h +show-diff.o: cache.h + +clean: + rm -f *.o $(PROG) + +backup: clean + cd .. ; tar czvf dircache.tar.gz dir-cache + + +Revision-number: 12 +Prop-content-length: 118 +Content-length: 118 + +K 7 +svn:log +V 19 +(r12) left update 4 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:56.831014Z +PROPS-END + +Node-path: branches/left/zlonk +Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 7 +Text-content-md5: 8b9d8c7c2aaa6167e7d3407a773bbbba +Text-content-sha1: 9716527ebd70a75c27625cacbeb2d897c6e86178 +Content-length: 17 + +PROPS-END +touche + + +Revision-number: 13 +Prop-content-length: 119 +Content-length: 119 + +K 7 +svn:log +V 20 +(r13) right update 2 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:57.341143Z +PROPS-END + +Node-path: branches/right/bang +Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 8 +Text-content-md5: 34c28f1d2dc6a9adeccc4265bf7516cb +Text-content-sha1: 0bc5bb345c0e71d28f784f12e0bd2d384c283062 +Content-length: 18 + +PROPS-END +thwacke + + +Revision-number: 14 +Prop-content-length: 141 +Content-length: 141 + +K 7 +svn:log +V 42 +(r14) Cherry-pick right 2 commits to trunk +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:57.841851Z +PROPS-END + +Node-path: trunk +Node-kind: dir +Node-action: change +Prop-content-length: 75 +Content-length: 75 + +K 13 +svn:mergeinfo +V 40 +/branches/left:2-10 +/branches/right:6-13 PROPS-END @@ -767,31 +1019,147 @@ backup: clean cd .. ; tar czvf dircache.tar.gz dir-cache -Revision-number: 10 -Prop-content-length: 116 -Content-length: 116 +Node-path: trunk/bang +Node-kind: file +Node-action: add +Node-copyfrom-rev: 13 +Node-copyfrom-path: branches/right/bang +Text-copy-source-md5: 34c28f1d2dc6a9adeccc4265bf7516cb +Text-copy-source-sha1: 0bc5bb345c0e71d28f784f12e0bd2d384c283062 + + +Revision-number: 15 +Prop-content-length: 127 +Content-length: 127 K 7 svn:log -V 13 -left update 4 +V 28 +(r15) Merge right to trunk 1 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:52.081644Z +2009-12-19T16:17:58.368520Z PROPS-END -Node-path: branches/left/Makefile +Node-path: trunk +Node-kind: dir +Node-action: change +Prop-content-length: 75 +Content-length: 75 + +K 13 +svn:mergeinfo +V 40 +/branches/left:2-10 +/branches/right:2-14 +PROPS-END + + +Revision-number: 16 +Prop-content-length: 119 +Content-length: 119 + +K 7 +svn:log +V 20 +(r16) right update 3 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:58.779056Z +PROPS-END + +Node-path: branches/right/urkkk +Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 6 +Text-content-md5: 5889c8392e16251b0c80927607a03036 +Text-content-sha1: 3934264d277a0cf886b6b1c7f2b9e56da2525302 +Content-length: 16 + +PROPS-END +whamm + + +Revision-number: 17 +Prop-content-length: 119 +Content-length: 119 + +K 7 +svn:log +V 20 +(r17) trunk update 1 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:59.221851Z +PROPS-END + +Node-path: trunk/vronk Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 4 +Text-content-md5: b2f80fa02a7f1364b9c29d3da44bf9f9 +Text-content-sha1: e994d980c0f2d7a3f76138bf96d57f36f9633828 +Content-length: 14 + +PROPS-END +pow + + +Revision-number: 18 +Prop-content-length: 135 +Content-length: 135 + +K 7 +svn:log +V 36 +(r18) Merge right to left sub-branch +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:17:59.781666Z +PROPS-END + +Node-path: branches/left-sub +Node-kind: dir Node-action: change -Text-content-length: 2529 -Text-content-md5: f6b197cc3f2e89a83e545d4bb003de73 -Text-content-sha1: 2f656677cfec0bceec85e53036ffb63e25126f8e -Content-length: 2529 +Prop-content-length: 55 +Content-length: 55 + +K 13 +svn:mergeinfo +V 20 +/branches/right:2-17 +PROPS-END + + +Node-path: branches/left-sub/Makefile +Node-kind: file +Node-action: change +Text-content-length: 2713 +Text-content-md5: 0afbe34f244cd662b1f97d708c687f90 +Text-content-sha1: 46d9377d783e67a9b581da110352e799517c8a14 +Content-length: 2713 # -DCOLLISION_CHECK if you believe that SHA1's # 1461501637330902918203684832716283019655932542976 hashes do not give you @@ -809,7 +1177,7 @@ CC=gcc PROG= update-cache show-diff init-db write-tree read-tree commit-tree \ cat-file fsck-cache checkout-cache diff-tree rev-tree show-files \ - check-files ls-tree merge-base + check-files ls-tree merge-base merge-cache all: $(PROG) @@ -859,8 +1227,11 @@ check-files: check-files.o read-cache.o ls-tree: ls-tree.o read-cache.o $(CC) $(CFLAGS) -o ls-tree ls-tree.o read-cache.o $(LIBS) -merge-base: merge-base.o read-cache.o - $(CC) $(CFLAGS) -o merge-base merge-base.o read-cache.o $(LIBS) +merge-base: merge-base.o read-cache.o object.o commit.o tree.o blob.o + $(CC) $(CFLAGS) -o merge-base merge-base.o read-cache.o object.o commit.o tree.o blob.o $(LIBS) + +merge-cache: merge-cache.o read-cache.o + $(CC) $(CFLAGS) -o merge-cache merge-cache.o read-cache.o $(LIBS) read-cache.o: cache.h show-diff.o: cache.h @@ -872,31 +1243,165 @@ backup: clean cd .. ; tar czvf dircache.tar.gz dir-cache -Revision-number: 11 -Prop-content-length: 117 -Content-length: 117 +Node-path: branches/left-sub/bang +Node-kind: file +Node-action: add +Node-copyfrom-rev: 17 +Node-copyfrom-path: branches/right/bang +Text-copy-source-md5: 34c28f1d2dc6a9adeccc4265bf7516cb +Text-copy-source-sha1: 0bc5bb345c0e71d28f784f12e0bd2d384c283062 + + +Node-path: branches/left-sub/urkkk +Node-kind: file +Node-action: add +Node-copyfrom-rev: 17 +Node-copyfrom-path: branches/right/urkkk +Text-copy-source-md5: 5889c8392e16251b0c80927607a03036 +Text-copy-source-sha1: 3934264d277a0cf886b6b1c7f2b9e56da2525302 + + +Revision-number: 19 +Prop-content-length: 129 +Content-length: 129 K 7 svn:log -V 14 -right update 1 +V 30 +(r19) left sub-branch update 2 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:53.059636Z +2009-12-19T16:18:00.200531Z PROPS-END -Node-path: branches/right/Makefile +Node-path: branches/left-sub/wham_eth +Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 6 +Text-content-md5: 757bcd5818572ef3f9580052617c1c8b +Text-content-sha1: b165019b005c199237ba822c4404e771e93b654a +Content-length: 16 + +PROPS-END +zowie + + +Revision-number: 20 +Prop-content-length: 118 +Content-length: 118 + +K 7 +svn:log +V 19 +(r20) left update 5 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:18:00.659636Z +PROPS-END + +Node-path: branches/left/glurpp Node-kind: file +Node-action: add +Prop-content-length: 10 +Text-content-length: 8 +Text-content-md5: 14a169f628e0bb59df9c2160649d0a30 +Text-content-sha1: ef7d929e52177767ecfcd28941f6b7f04b4131e3 +Content-length: 18 + +PROPS-END +eee_yow + + +Revision-number: 21 +Prop-content-length: 147 +Content-length: 147 + +K 7 +svn:log +V 48 +(r21) Cherry-pick left sub-branch commit to left +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:18:01.194402Z +PROPS-END + +Node-path: branches/left +Node-kind: dir Node-action: change -Text-content-length: 2593 -Text-content-md5: 5ccff689fb290e00b85fe18ee50c54ba -Text-content-sha1: a13de8e23f1483efca3e57b2b64b0ae6f740ce10 -Content-length: 2593 +Prop-content-length: 56 +Content-length: 56 + +K 13 +svn:mergeinfo +V 21 +/branches/left-sub:19 +PROPS-END + + +Node-path: branches/left/wham_eth +Node-kind: file +Node-action: add +Node-copyfrom-rev: 19 +Node-copyfrom-path: branches/left-sub/wham_eth +Text-copy-source-md5: 757bcd5818572ef3f9580052617c1c8b +Text-copy-source-sha1: b165019b005c199237ba822c4404e771e93b654a + + +Revision-number: 22 +Prop-content-length: 134 +Content-length: 134 + +K 7 +svn:log +V 35 +(r22) Merge left sub-branch to left +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:18:01.679218Z +PROPS-END + +Node-path: branches/left +Node-kind: dir +Node-action: change +Prop-content-length: 79 +Content-length: 79 + +K 13 +svn:mergeinfo +V 44 +/branches/left-sub:4-19 +/branches/right:2-17 +PROPS-END + + +Node-path: branches/left/Makefile +Node-kind: file +Node-action: change +Text-content-length: 2713 +Text-content-md5: 0afbe34f244cd662b1f97d708c687f90 +Text-content-sha1: 46d9377d783e67a9b581da110352e799517c8a14 +Content-length: 2713 # -DCOLLISION_CHECK if you believe that SHA1's # 1461501637330902918203684832716283019655932542976 hashes do not give you @@ -914,7 +1419,7 @@ CC=gcc PROG= update-cache show-diff init-db write-tree read-tree commit-tree \ cat-file fsck-cache checkout-cache diff-tree rev-tree show-files \ - check-files ls-tree merge-base + check-files ls-tree merge-base merge-cache all: $(PROG) @@ -967,6 +1472,9 @@ ls-tree: ls-tree.o read-cache.o merge-base: merge-base.o read-cache.o object.o commit.o tree.o blob.o $(CC) $(CFLAGS) -o merge-base merge-base.o read-cache.o object.o commit.o tree.o blob.o $(LIBS) +merge-cache: merge-cache.o read-cache.o + $(CC) $(CFLAGS) -o merge-cache merge-cache.o read-cache.o $(LIBS) + read-cache.o: cache.h show-diff.o: cache.h @@ -977,34 +1485,141 @@ backup: clean cd .. ; tar czvf dircache.tar.gz dir-cache -Revision-number: 12 -Prop-content-length: 116 -Content-length: 116 +Node-path: branches/left/README +Node-kind: file +Node-action: add +Node-copyfrom-rev: 18 +Node-copyfrom-path: branches/left-sub/README +Text-copy-source-md5: fdbcfb6be9afe1121862143f226b51cf +Text-copy-source-sha1: 1d1f5ea4ceb584337ffe59b8980d92e3b78dfef4 + + +Node-path: branches/left/bang +Node-kind: file +Node-action: add +Node-copyfrom-rev: 18 +Node-copyfrom-path: branches/left-sub/bang +Text-copy-source-md5: 34c28f1d2dc6a9adeccc4265bf7516cb +Text-copy-source-sha1: 0bc5bb345c0e71d28f784f12e0bd2d384c283062 + + +Node-path: branches/left/urkkk +Node-kind: file +Node-action: add +Node-copyfrom-rev: 18 +Node-copyfrom-path: branches/left-sub/urkkk +Text-copy-source-md5: 5889c8392e16251b0c80927607a03036 +Text-copy-source-sha1: 3934264d277a0cf886b6b1c7f2b9e56da2525302 + + +Revision-number: 23 +Prop-content-length: 126 +Content-length: 126 K 7 svn:log -V 13 -Merge trunk 2 +V 27 +(r23) Merge left to trunk 2 K 10 svn:author -V 8 -tallsopp +V 4 +samv K 8 svn:date V 27 -2009-11-12T20:29:56.083003Z +2009-12-19T16:18:02.212349Z PROPS-END Node-path: trunk Node-kind: dir Node-action: change -Prop-content-length: 54 -Content-length: 54 +Prop-content-length: 99 +Content-length: 99 K 13 svn:mergeinfo -V 19 -/branches/left:2-11 +V 64 +/branches/left:2-22 +/branches/left-sub:4-19 +/branches/right:2-17 +PROPS-END + + +Node-path: trunk/README +Node-kind: file +Node-action: add +Node-copyfrom-rev: 22 +Node-copyfrom-path: branches/left/README +Text-copy-source-md5: fdbcfb6be9afe1121862143f226b51cf +Text-copy-source-sha1: 1d1f5ea4ceb584337ffe59b8980d92e3b78dfef4 + + +Node-path: trunk/glurpp +Node-kind: file +Node-action: add +Node-copyfrom-rev: 22 +Node-copyfrom-path: branches/left/glurpp +Text-copy-source-md5: 14a169f628e0bb59df9c2160649d0a30 +Text-copy-source-sha1: ef7d929e52177767ecfcd28941f6b7f04b4131e3 + + +Node-path: trunk/urkkk +Node-kind: file +Node-action: add +Node-copyfrom-rev: 22 +Node-copyfrom-path: branches/left/urkkk +Text-copy-source-md5: 5889c8392e16251b0c80927607a03036 +Text-copy-source-sha1: 3934264d277a0cf886b6b1c7f2b9e56da2525302 + + +Node-path: trunk/wham_eth +Node-kind: file +Node-action: add +Node-copyfrom-rev: 22 +Node-copyfrom-path: branches/left/wham_eth +Text-copy-source-md5: 757bcd5818572ef3f9580052617c1c8b +Text-copy-source-sha1: b165019b005c199237ba822c4404e771e93b654a + + +Node-path: trunk/zlonk +Node-kind: file +Node-action: add +Node-copyfrom-rev: 22 +Node-copyfrom-path: branches/left/zlonk +Text-copy-source-md5: 8b9d8c7c2aaa6167e7d3407a773bbbba +Text-copy-source-sha1: 9716527ebd70a75c27625cacbeb2d897c6e86178 + + +Revision-number: 24 +Prop-content-length: 131 +Content-length: 131 + +K 7 +svn:log +V 32 +(r24) non-merge right to trunk 2 +K 10 +svn:author +V 4 +samv +K 8 +svn:date +V 27 +2009-12-19T16:18:02.672148Z +PROPS-END + +Node-path: trunk +Node-kind: dir +Node-action: change +Prop-content-length: 99 +Content-length: 99 + +K 13 +svn:mergeinfo +V 64 +/branches/left:2-22 +/branches/left-sub:4-19 +/branches/right:2-22 PROPS-END -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges 2009-12-19 16:33 ` [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures Sam Vilain @ 2009-12-19 16:33 ` Sam Vilain 2009-12-19 16:33 ` [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo " Sam Vilain ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick, Sam Vilain Each time the svn mergeinfo ticket changes, we look it up in the rev_map; when there are a lot of merged branches, this will result in many repeated lookups of the same information for subsequent commits. Arrange the slow part of the function so that it may be memoized, and memoize it. The more expensive revision walking operation can be memoized separately. Signed-off-by: Sam Vilain <sam@vilain.net> --- git-svn.perl | 91 ++++++++++++++++++++++++++++++++++----------------------- 1 files changed, 54 insertions(+), 37 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index a4b052c..3b17a83 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -1634,6 +1634,7 @@ use Carp qw/croak/; use File::Path qw/mkpath/; use File::Copy qw/copy/; use IPC::Open3; +use Memoize; # core since 5.8.0, Jul 2002 my ($_gc_nr, $_gc_period); @@ -2967,6 +2968,55 @@ sub find_extra_svk_parents { } } +sub lookup_svn_merge { + my $uuid = shift; + my $url = shift; + my $merge = shift; + + my ($source, $revs) = split ":", $merge; + my $path = $source; + $path =~ s{^/}{}; + my $gs = Git::SVN->find_by_url($url.$source, $url, $path); + if ( !$gs ) { + warn "Couldn't find revmap for $url$source\n"; + next; + } + my @ranges = split ",", $revs; + my ($tip, $tip_commit); + my @merged_commit_ranges; + # find the tip + for my $range ( @ranges ) { + my ($bottom, $top) = split "-", $range; + $top ||= $bottom; + my $bottom_commit = + $gs->rev_map_get($bottom, $uuid) || + $gs->rev_map_get($bottom+1, $uuid); + my $top_commit; + for (; !$top_commit && $top >= $bottom; --$top) { + $top_commit = + $gs->rev_map_get($top, $uuid); + } + + unless ($top_commit and $bottom_commit) { + warn "W:unknown path/rev in svn:mergeinfo " + ."dirprop: $source:$range\n"; + next; + } + + push @merged_commit_ranges, + "$bottom_commit..$top_commit"; + + if ( !defined $tip or $top > $tip ) { + $tip = $top; + $tip_commit = $top_commit; + } + } + return ($tip_commit, @merged_commit_ranges); +} +BEGIN { + memoize 'lookup_svn_merge'; +} + # note: this function should only be called if the various dirprops # have actually changed sub find_extra_svn_parents { @@ -2981,44 +3031,11 @@ sub find_extra_svn_parents { my @merge_tips; my @merged_commit_ranges; my $url = $self->rewrite_root || $self->{url}; + my $uuid = $self->ra_uuid; for my $merge ( @merges ) { - my ($source, $revs) = split ":", $merge; - my $path = $source; - $path =~ s{^/}{}; - my $gs = Git::SVN->find_by_url($url.$source, $url, $path); - if ( !$gs ) { - warn "Couldn't find revmap for $url$source\n"; - next; - } - my @ranges = split ",", $revs; - my ($tip, $tip_commit); - # find the tip - for my $range ( @ranges ) { - my ($bottom, $top) = split "-", $range; - $top ||= $bottom; - my $bottom_commit = - $gs->rev_map_get($bottom, $self->ra_uuid) || - $gs->rev_map_get($bottom+1, $self->ra_uuid); - my $top_commit; - for (; !$top_commit && $top >= $bottom; --$top) { - $top_commit = - $gs->rev_map_get($top, $self->ra_uuid); - } - - unless ($top_commit and $bottom_commit) { - warn "W:unknown path/rev in svn:mergeinfo " - ."dirprop: $source:$range\n"; - next; - } - - push @merged_commit_ranges, - "$bottom_commit..$top_commit"; - - if ( !defined $tip or $top > $tip ) { - $tip = $top; - $tip_commit = $top_commit; - } - } + my ($tip_commit, @ranges) = + lookup_svn_merge( $uuid, $url, $merge ); + push @merged_commit_ranges, @ranges; unless (!$tip_commit or grep { $_ eq $tip_commit } @$parents ) { push @merge_tips, $tip_commit; -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo commit ranges 2009-12-19 16:33 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain @ 2009-12-19 16:33 ` Sam Vilain 2009-12-19 16:33 ` [PATCH 4/5] git-svn: exclude already merged tips using one rev-list call Sam Vilain 2009-12-19 16:37 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-20 21:24 ` Sam Vilain 2 siblings, 1 reply; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick, Sam Vilain SVN's list of commit ranges in mergeinfo tickets is inclusive, whereas git commit ranges are exclusive on the left hand side. Also, the end points of the commit ranges may not exist; they simply delineate ranges of commits which may or may not exist. Fix these two mistakes. Signed-off-by: Sam Vilain <sam@vilain.net> --- git-svn.perl | 12 +++--------- t/t9151-svn-mergeinfo.sh | 2 +- 2 files changed, 4 insertions(+), 10 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index 3b17a83..9cf4a3e 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -2988,14 +2988,8 @@ sub lookup_svn_merge { for my $range ( @ranges ) { my ($bottom, $top) = split "-", $range; $top ||= $bottom; - my $bottom_commit = - $gs->rev_map_get($bottom, $uuid) || - $gs->rev_map_get($bottom+1, $uuid); - my $top_commit; - for (; !$top_commit && $top >= $bottom; --$top) { - $top_commit = - $gs->rev_map_get($top, $uuid); - } + my $bottom_commit = $gs->find_rev_after( $bottom, 1, $top ); + my $top_commit = $gs->find_rev_before( $top, 1, $bottom ); unless ($top_commit and $bottom_commit) { warn "W:unknown path/rev in svn:mergeinfo " @@ -3004,7 +2998,7 @@ sub lookup_svn_merge { } push @merged_commit_ranges, - "$bottom_commit..$top_commit"; + "$bottom_commit^..$top_commit"; if ( !defined $tip or $top > $tip ) { $tip = $top; diff --git a/t/t9151-svn-mergeinfo.sh b/t/t9151-svn-mergeinfo.sh index dc3478f..f6e00ea 100755 --- a/t/t9151-svn-mergeinfo.sh +++ b/t/t9151-svn-mergeinfo.sh @@ -33,7 +33,7 @@ test_expect_success 'svn non-merge merge commits did not become git merge commit [ -z "$bad_non_merges" ] ' -test_expect_failure 'everything got merged in the end' ' +test_expect_success 'everything got merged in the end' ' unmerged=$(git rev-list --all --not master) [ -z "$unmerged" ] ' -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 4/5] git-svn: exclude already merged tips using one rev-list call 2009-12-19 16:33 ` [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo " Sam Vilain @ 2009-12-19 16:33 ` Sam Vilain 2009-12-19 16:33 ` [PATCH 5/5] git-svn: detect cherry-picks correctly Sam Vilain 0 siblings, 1 reply; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick, Sam Vilain The old function would have to check all mentioned merge tips, every time that the mergeinfo ticket changed. This involved 1-2 rev-list operation for each listed mergeinfo line. If there are a lot of feature branches being merged into a trunk, this makes for a very expensive operation for detecting the new parents on every merge. This new version first uses a single 'rev-list' to figure out which commit ranges are already reachable from the parents. This is used to eliminate the already merged branches from the list. Signed-off-by: Sam Vilain <sam@vilain.net> --- git-svn.perl | 52 ++++++++++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 48 insertions(+), 4 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index 9cf4a3e..7a790d7 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -3011,6 +3011,41 @@ BEGIN { memoize 'lookup_svn_merge'; } +sub parents_exclude { + my $parents = shift; + my @commits = @_; + return unless @commits; + + my @excluded; + my $excluded; + do { + my @cmd = ('rev-list', "-1", @commits, "--not", @$parents ); + $excluded = command_oneline(@cmd); + if ( $excluded ) { + my @new; + my $found; + for my $commit ( @commits ) { + if ( $commit eq $excluded ) { + push @excluded, $commit; + $found++; + last; + } + else { + push @new, $commit; + } + } + die "saw commit '$excluded' in rev-list output, " + ."but we didn't ask for that commit (wanted: @commits --not @$parents)" + unless $found; + @commits = @new; + } + } + while ($excluded and @commits); + + return @excluded; +} + + # note: this function should only be called if the various dirprops # have actually changed sub find_extra_svn_parents { @@ -3023,23 +3058,32 @@ sub find_extra_svn_parents { # are now marked as merge, we can add the tip as a parent. my @merges = split "\n", $mergeinfo; my @merge_tips; - my @merged_commit_ranges; my $url = $self->rewrite_root || $self->{url}; my $uuid = $self->ra_uuid; + my %ranges; for my $merge ( @merges ) { my ($tip_commit, @ranges) = lookup_svn_merge( $uuid, $url, $merge ); - push @merged_commit_ranges, @ranges; unless (!$tip_commit or grep { $_ eq $tip_commit } @$parents ) { push @merge_tips, $tip_commit; + $ranges{$tip_commit} = \@ranges; } else { push @merge_tips, undef; } } + + my %excluded = map { $_ => 1 } + parents_exclude($parents, grep { defined } @merge_tips); + + # check merge tips for new parents + my @new_parents; for my $merge_tip ( @merge_tips ) { my $spec = shift @merges; - next unless $merge_tip; + next unless $merge_tip and $excluded{$merge_tip}; + + my $ranges = $ranges{$merge_tip}; + my @cmd = ('rev-list', "-1", $merge_tip, "--not", @$parents ); my ($msg_fh, $ctx) = command_output_pipe(@cmd); @@ -3049,7 +3093,7 @@ sub find_extra_svn_parents { } command_close_pipe($msg_fh, $ctx); if ( $new ) { - push @cmd, @merged_commit_ranges; + push @cmd, @$ranges; my ($msg_fh, $ctx) = command_output_pipe(@cmd); my $unmerged; while ( <$msg_fh> ) { -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 5/5] git-svn: detect cherry-picks correctly. 2009-12-19 16:33 ` [PATCH 4/5] git-svn: exclude already merged tips using one rev-list call Sam Vilain @ 2009-12-19 16:33 ` Sam Vilain 0 siblings, 0 replies; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:33 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick, Sam Vilain The old function was incorrect; in some instances it marks a cherry picked range as a merged branch (because of an incorrect assumption that 'rev-list COMMIT --not RANGE' would work). This is replaced with a function which should detect them correctly, memoized to limit the expense of dealing with branches with many cherry picks to one 'merge-base' call per merge, per branch which used cherry picking. Signed-off-by: Sam Vilain <sam@vilain.net> --- git-svn.perl | 87 +++++++++++++++++++++++++++++++++------------ t/t9151-svn-mergeinfo.sh | 4 +- 2 files changed, 66 insertions(+), 25 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index 7a790d7..f06e535 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -3007,8 +3007,35 @@ sub lookup_svn_merge { } return ($tip_commit, @merged_commit_ranges); } + +sub _rev_list { + my ($msg_fh, $ctx) = command_output_pipe( + "rev-list", @_, + ); + my @rv; + while ( <$msg_fh> ) { + chomp; + push @rv, $_; + } + command_close_pipe($msg_fh, $ctx); + @rv; +} + +sub check_cherry_pick { + my $base = shift; + my $tip = shift; + my @ranges = @_; + my %commits = map { $_ => 1 } + _rev_list("--no-merges", $tip, "--not", $base); + for my $range ( @ranges ) { + delete @commits{_rev_list($range)}; + } + return (keys %commits); +} + BEGIN { memoize 'lookup_svn_merge'; + memoize 'check_cherry_pick'; } sub parents_exclude { @@ -3084,32 +3111,46 @@ sub find_extra_svn_parents { my $ranges = $ranges{$merge_tip}; - my @cmd = ('rev-list', "-1", $merge_tip, - "--not", @$parents ); - my ($msg_fh, $ctx) = command_output_pipe(@cmd); - my $new; - while ( <$msg_fh> ) { - $new=1;last; - } - command_close_pipe($msg_fh, $ctx); - if ( $new ) { - push @cmd, @$ranges; - my ($msg_fh, $ctx) = command_output_pipe(@cmd); - my $unmerged; - while ( <$msg_fh> ) { - $unmerged=1;last; - } - command_close_pipe($msg_fh, $ctx); - if ( $unmerged ) { - warn "W:svn cherry-pick ignored ($spec)\n"; - } else { - warn - "Found merge parent (svn:mergeinfo prop): ", - $merge_tip, "\n"; - push @$parents, $merge_tip; + # check out 'new' tips + my $merge_base = command_oneline( + "merge-base", + @$parents, $merge_tip, + ); + + # double check that there are no missing non-merge commits + my (@incomplete) = check_cherry_pick( + $merge_base, $merge_tip, + @$ranges, + ); + + if ( @incomplete ) { + warn "W:svn cherry-pick ignored ($spec) - missing " + .@incomplete." commit(s) (eg $incomplete[0])\n"; + } else { + warn + "Found merge parent (svn:mergeinfo prop): ", + $merge_tip, "\n"; + push @new_parents, $merge_tip; + } + } + + # cater for merges which merge commits from multiple branches + if ( @new_parents > 1 ) { + for ( my $i = 0; $i <= $#new_parents; $i++ ) { + for ( my $j = 0; $j <= $#new_parents; $j++ ) { + next if $i == $j; + next unless $new_parents[$i]; + next unless $new_parents[$j]; + my $revs = command_oneline( + "rev-list", "-1", "$i..$j", + ); + if ( !$revs ) { + undef($new_parents[$i]); + } } } } + push @$parents, grep { defined } @new_parents; } sub make_log_entry { diff --git a/t/t9151-svn-mergeinfo.sh b/t/t9151-svn-mergeinfo.sh index f6e00ea..359eeaa 100755 --- a/t/t9151-svn-mergeinfo.sh +++ b/t/t9151-svn-mergeinfo.sh @@ -15,13 +15,13 @@ test_expect_success 'load svn dump' " git svn fetch --all " -test_expect_failure 'all svn merges became git merge commits' ' +test_expect_success 'all svn merges became git merge commits' ' unmarked=$(git rev-list --parents --all --grep=Merge | grep -v " .* " | cut -f1 -d" ") [ -z "$unmarked" ] ' -test_expect_failure 'cherry picks did not become git merge commits' ' +test_expect_success 'cherry picks did not become git merge commits' ' bad_cherries=$(git rev-list --parents --all --grep=Cherry | grep " .* " | cut -f1 -d" ") [ -z "$bad_cherries" ] -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges 2009-12-19 16:33 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-19 16:33 ` [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo " Sam Vilain @ 2009-12-19 16:37 ` Sam Vilain 2009-12-20 21:24 ` Sam Vilain 2 siblings, 0 replies; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:37 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick On Sun, 2009-12-20 at 05:33 +1300, Sam Vilain wrote: > Each time the svn mergeinfo ticket changes, we look it up in the rev_map; > when there are a lot of merged branches, this will result in many repeated > lookups of the same information for subsequent commits. Arrange the slow > part of the function so that it may be memoized, and memoize it. The more > expensive revision walking operation can be memoized separately. Sorry, that text was an old revision. Read as: Each time the svn mergeinfo ticket changes, we look it up in the rev_map; when there are a lot of merged branches, this will result in many repeated lookups of the same information for subsequent commits. Arrange that part of the function so that it may be memoized, and memoize it. Sam ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges 2009-12-19 16:33 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-19 16:33 ` [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo " Sam Vilain 2009-12-19 16:37 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain @ 2009-12-20 21:24 ` Sam Vilain 2009-12-21 10:44 ` Eric Wong 2 siblings, 1 reply; 13+ messages in thread From: Sam Vilain @ 2009-12-20 21:24 UTC (permalink / raw) To: git; +Cc: Eric Wong On Sun, 2009-12-20 at 05:33 +1300, Sam Vilain wrote: > Each time the svn mergeinfo ticket changes, we look it up in the rev_map; > when there are a lot of merged branches, this will result in many repeated > lookups of the same information for subsequent commits. Arrange the slow > part of the function so that it may be memoized, and memoize it. The more > expensive revision walking operation can be memoized separately. > > Signed-off-by: Sam Vilain <sam@vilain.net> > --- > git-svn.perl | 91 ++++++++++++++++++++++++++++++++++----------------------- > 1 files changed, 54 insertions(+), 37 deletions(-) > > diff --git a/git-svn.perl b/git-svn.perl > index a4b052c..3b17a83 100755 > --- a/git-svn.perl > +++ b/git-svn.perl > @@ -1634,6 +1634,7 @@ use Carp qw/croak/; > use File::Path qw/mkpath/; > use File::Copy qw/copy/; > use IPC::Open3; > +use Memoize; # core since 5.8.0, Jul 2002 > > my ($_gc_nr, $_gc_period); > > @@ -2967,6 +2968,55 @@ sub find_extra_svk_parents { > } > } > > +sub lookup_svn_merge { > + my $uuid = shift; > + my $url = shift; > + my $merge = shift; > + > + my ($source, $revs) = split ":", $merge; > + my $path = $source; > + $path =~ s{^/}{}; > + my $gs = Git::SVN->find_by_url($url.$source, $url, $path); > + if ( !$gs ) { > + warn "Couldn't find revmap for $url$source\n"; > + next; > + } As mentioned in the other thread, that 'next' should now be 'return'. Sam ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges 2009-12-20 21:24 ` Sam Vilain @ 2009-12-21 10:44 ` Eric Wong 0 siblings, 0 replies; 13+ messages in thread From: Eric Wong @ 2009-12-21 10:44 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, Andrew Myrick, Sam Vilain Sam Vilain <sam@vilain.net> wrote: > On Sun, 2009-12-20 at 05:33 +1300, Sam Vilain wrote: > > +sub lookup_svn_merge { > > + my $uuid = shift; > > + my $url = shift; > > + my $merge = shift; > > + > > + my ($source, $revs) = split ":", $merge; > > + my $path = $source; > > + $path =~ s{^/}{}; > > + my $gs = Git::SVN->find_by_url($url.$source, $url, $path); > > + if ( !$gs ) { > > + warn "Couldn't find revmap for $url$source\n"; > > + next; > > + } > > As mentioned in the other thread, that 'next' should now be 'return'. Thanks Sam and Andrew. I've acked this series and pushed them out along with a release notes update as well as another small fix (inline below). Eric Wong (2): git svn: fix --revision when fetching deleted paths update release notes for git svn in 1.6.6 Sam Vilain (5): git-svn: expand the svn mergeinfo test suite, highlighting some failures git-svn: memoize conversion of SVN merge ticket info to git commit ranges git-svn: fix some mistakes with interpreting SVN mergeinfo commit ranges git-svn: exclude already merged tips using one rev-list call git-svn: detect cherry-picks correctly. >From 577e9fcad2c8968846b365226b89778050496a78 Mon Sep 17 00:00:00 2001 From: Eric Wong <normalperson@yhbt.net> Date: Mon, 21 Dec 2009 02:06:04 -0800 Subject: [PATCH] git svn: fix --revision when fetching deleted paths When using the -r/--revision argument to fetch deleted history, calling SVN::Ra::get_log() from an SVN::Ra object initialized to track the deleted URL will fail. This regression was introduced in: commit 4aacaeb3dc82bb6479e70e120053dc27a399460e "fix shallow clone when upstream revision is too new" We now ignore errors from SVN::Ra::get_log() here because using --revision will always override the value of $head here if (and only if) we're tracking deleted directories. Signed-off-by: Eric Wong <normalperson@yhbt.net> --- git-svn.perl | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index d362de7..a6f5061 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -1741,7 +1741,11 @@ sub fetch_all { my $ra = Git::SVN::Ra->new($url); my $uuid = $ra->get_uuid; my $head = $ra->get_latest_revnum; - $ra->get_log("", $head, 0, 1, 0, 1, sub { $head = $_[1] }); + + # ignore errors, $head revision may not even exist anymore + eval { $ra->get_log("", $head, 0, 1, 0, 1, sub { $head = $_[1] }) }; + warn "W: $@\n" if $@; + my $base = defined $fetch ? $head : 0; # read the max revs for wildcard expansion (branches/*, tags/*) -- Eric Wong ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: Efficiency and correctness patches for git-svn mergeinfo support 2009-12-19 16:33 Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain 2009-12-19 16:33 ` [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures Sam Vilain @ 2009-12-19 16:42 ` Sam Vilain 2009-12-19 22:15 ` Andrew Myrick 2 siblings, 0 replies; 13+ messages in thread From: Sam Vilain @ 2009-12-19 16:42 UTC (permalink / raw) To: git; +Cc: Eric Wong, Andrew Myrick On Sun, 2009-12-20 at 05:33 +1300, Sam Vilain wrote: > This series implements some efficiency enhancements; particularly in > dealing with repositories which have a single trunk branch which > receives many merges of feature branches and/or cherry-picks. It also > fixes a number of corner cases in the merge conversion code. > > It would be nice if the people who have experienced slow git-svn > performance in these situations could test that this fixes the > performance issues, and that the resulting repositories seem to have > correct contents. > > Eric, I'm using Alex's trick of a single commit which adds failing > tests - marked as _expect_failure - and then marked them as succeeding > as the series progresses. More useful info: git-svn.perl | 220 +++++++++---- t/t9151-svn-mergeinfo.sh | 27 ++- t/t9151/make-svnmerge-dump | 166 ++++++--- t/t9151/svn-mergeinfo.dump | 839 ++++++++++++++++++++++++++++++++++++++------ 4 files changed, 1018 insertions(+), 234 deletions(-) And available at git://github.com/samv/git (branch mergeinfo-fixes) Sam ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Efficiency and correctness patches for git-svn mergeinfo support 2009-12-19 16:33 Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain 2009-12-19 16:33 ` [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures Sam Vilain 2009-12-19 16:42 ` Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain @ 2009-12-19 22:15 ` Andrew Myrick 2009-12-20 21:07 ` Sam Vilain 2 siblings, 1 reply; 13+ messages in thread From: Andrew Myrick @ 2009-12-19 22:15 UTC (permalink / raw) To: Sam Vilain; +Cc: git, Eric Wong > It would be nice if the people who have experienced slow git-svn > performance in these situations could test that this fixes the > performance issues, and that the resulting repositories seem to have > correct contents. I tried cloning from a fairly recent revision that I knew was after our switchover to svn 1.5, and I received a number of these errors: Couldn't find revmap for [branch] Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. I'm not sure if this is expected, since I didn't clone from the whole repo, but it did cause a lot of spew. I'm starting a fresh clone now, but it takes a few days to get through the whole repository. I'm fairly new to git, so I would welcome any tips on how I can test this more quickly. -Andrew ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Efficiency and correctness patches for git-svn mergeinfo support 2009-12-19 22:15 ` Andrew Myrick @ 2009-12-20 21:07 ` Sam Vilain 2009-12-20 22:03 ` Andrew Myrick 0 siblings, 1 reply; 13+ messages in thread From: Sam Vilain @ 2009-12-20 21:07 UTC (permalink / raw) To: Andrew Myrick; +Cc: git, Eric Wong On Sat, 2009-12-19 at 14:15 -0800, Andrew Myrick wrote: > I tried cloning from a fairly recent revision that I knew was after > our switchover to svn 1.5, and I received a number of these errors: > > Couldn't find revmap for [branch] > Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. > Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. > Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. > > I'm not sure if this is expected, since I didn't clone from the whole > repo, but it did cause a lot of spew. I'm starting a fresh clone now, > but it takes a few days to get through the whole repository. I'm > fairly new to git, so I would welcome any tips on how I can test this > more quickly. Whoops, no, not expected, I'll post a minor correction. That means that the branch which was merged in does not have git-svn metadata; ie, it's not being tracked explicitly. If people are doing merging of things which aren't roots of branches you would expect this. SVN, like Perforce, supports a confusing amount of flexibility in its merge tracking. If [branch] is a real branch, then you'll want to see why it doesn't have metadata yet. Is it really a sub-tree of a real branch? You could fetch it independently using a separate git-svn remote, or you could ignore the warning; it should be relatively self-evident what happened from the merge message and the contents of the changeset. Note if your repository was significantly re-organized at any point, it will pay to treat each section of history as a separate import project, and stitch the results together afterwards using grafts and filter-branch. This version should be *significantly* faster than the old one. ie, it should not take a minute per commit while importing the heavily merged-into integration branch. Possibly a few seconds at most. Sam ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Efficiency and correctness patches for git-svn mergeinfo support 2009-12-20 21:07 ` Sam Vilain @ 2009-12-20 22:03 ` Andrew Myrick 0 siblings, 0 replies; 13+ messages in thread From: Andrew Myrick @ 2009-12-20 22:03 UTC (permalink / raw) To: Sam Vilain; +Cc: git, Eric Wong On Sun, Dec 20, 2009 at 1:07 PM, Sam Vilain <sam@vilain.net> wrote: > On Sat, 2009-12-19 at 14:15 -0800, Andrew Myrick wrote: >> I tried cloning from a fairly recent revision that I knew was after >> our switchover to svn 1.5, and I received a number of these errors: >> >> Couldn't find revmap for [branch] >> Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. >> Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. >> Exiting subroutine via next at /Users/adm/libexec/git-core/git-svn line 2983. >> >> I'm not sure if this is expected, since I didn't clone from the whole >> repo, but it did cause a lot of spew. I'm starting a fresh clone now, >> but it takes a few days to get through the whole repository. I'm >> fairly new to git, so I would welcome any tips on how I can test this >> more quickly. > > Whoops, no, not expected, I'll post a minor correction. That means that > the branch which was merged in does not have git-svn metadata; ie, it's > not being tracked explicitly. If people are doing merging of things > which aren't roots of branches you would expect this. SVN, like > Perforce, supports a confusing amount of flexibility in its merge > tracking. If [branch] is a real branch, then you'll want to see why it > doesn't have metadata yet. Is it really a sub-tree of a real branch? > You could fetch it independently using a separate git-svn remote, or you > could ignore the warning; it should be relatively self-evident what > happened from the merge message and the contents of the changeset. I think the problem is that I started fetching from a later revision than when the branch was created and reintegrated. There would be no metadata for the branch, so a lookup for it would fail. Fetching the branch explicitly, or fetching from r1 as my current test is doing, should not experience this problem. > Note if your repository was significantly re-organized at any point, it > will pay to treat each section of history as a separate import project, > and stitch the results together afterwards using grafts and > filter-branch. I don't believe it was, but I will keep that in mind if I run into trouble. The import from r1 went fine with git v1.6.5.*, and it's proceeding well now. > This version should be *significantly* faster than the old one. ie, it > should not take a minute per commit while importing the heavily > merged-into integration branch. Possibly a few seconds at most. So far, most commits are fetched in under 3s with your new version. Thanks for your help, Sam. -Andrew ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2009-12-21 10:45 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-12-19 16:33 Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain 2009-12-19 16:33 ` [PATCH 1/5] git-svn: expand the svn mergeinfo test suite, highlighting some failures Sam Vilain 2009-12-19 16:33 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-19 16:33 ` [PATCH 3/5] git-svn: fix some mistakes with interpreting SVN mergeinfo " Sam Vilain 2009-12-19 16:33 ` [PATCH 4/5] git-svn: exclude already merged tips using one rev-list call Sam Vilain 2009-12-19 16:33 ` [PATCH 5/5] git-svn: detect cherry-picks correctly Sam Vilain 2009-12-19 16:37 ` [PATCH 2/5] git-svn: memoize conversion of SVN merge ticket info to git commit ranges Sam Vilain 2009-12-20 21:24 ` Sam Vilain 2009-12-21 10:44 ` Eric Wong 2009-12-19 16:42 ` Efficiency and correctness patches for git-svn mergeinfo support Sam Vilain 2009-12-19 22:15 ` Andrew Myrick 2009-12-20 21:07 ` Sam Vilain 2009-12-20 22:03 ` Andrew Myrick
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).