[PATCH] cvsimport: test case for severe branch import problem

[PATCH] cvsimport: test case for severe branch import problem

[Steffen Prohaska]

The result of importing branches using git-cvsimport depends
on the time of the first commit to the cvs branch. If the first
commit is done after other commits to the cvs trunk the result
of cvsimport may be wrong. git-cvsimport creates a wrong history.

The problem is quite severe because merging such a wrongly imported
branch by git may be successful without reporting any problem but the
results are wrong. The result may differ from what a simple cvs merge
(cvs up -j) yields.

This test script creates two cvs repositories and imports both to
git. The first cvs repository has the 'wrong' order of commits and
yields an error. The result of a merge in cvs and a merge in git
differs. The second cvs repository has the 'right' order and the
import to git runs as expected and merging yields the same results
in cvs and git.

The conclusion is you must not rely on the existing cvsimport for
tracking cvs branches. The history of such branches may be plain wrong.
Git may display different patches than cvs would do. Merging cvs
topic branches may yield completely wrong results. One obvious thing
that may happen is that a merge reverts changes commited to the cvs
trunk before the first commit to the cvs branch. And this would happen
without any indication by git. Everything would seem to run smoothely.

This conclusion should be stated in bold at appropriate places in
the documentation.

It's a pity because merging is what git is especially good at. But
as long as git-cvsimport may create the wrong history you can't use
git to merge cvs topic branches without double checking every detail,
which makes this approach unfeasable.

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
---
 t/t9600-cvsimport.sh |  184 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 184 insertions(+), 0 deletions(-)
 create mode 100755 t/t9600-cvsimport.sh

diff --git a/t/t9600-cvsimport.sh b/t/t9600-cvsimport.sh
new file mode 100755
index 0000000..180cd8a
--- /dev/null
+++ b/t/t9600-cvsimport.sh
@@ -0,0 +1,184 @@
+
+test_description='CVS import'
+
+. ./test-lib.sh
+
+cvs >/dev/null 2>&1
+if test $? -ne 1
+then
+    test_expect_success 'skipping git-cvsimport tests, cvs not found' :
+    test_done
+    exit
+fi
+
+CVSROOT=$(pwd)/cvsrootwrong
+CVSWORK=$(pwd)/cvsworkwrong
+CVSIMPORTED=$(pwd)/cvsimportedwrong
+export CVSROOT CVSWORK
+
+cvspscache="$HOME/.cvsps/$(echo $CVSROOT | sed -e 's%/%#%g')#src"
+rm -f $cvspscache
+
+rm -rf "$CVSROOT" "$CVSWORK"
+mkdir "$CVSROOT" &&
+cvs init &&
+mkdir "$CVSROOT/src"
+cvs -Q co -d "$CVSWORK" src &&
+rm -rf .git || 
+exit 1
+
+# sleeps are needed to fight cvsps' fuzz
+test_expect_success \
+	'initial cvs commits, tag, and branch' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 1" >>a.txt &&
+	   cvs add a.txt &&
+	   cvs commit -m "cvs commit 1" &&
+	   sleep 2 &&
+	   echo "a: line 2" >>a.txt &&
+	   echo "a: line 3" >>a.txt &&
+	   echo "a: line 4" >>a.txt &&
+	   echo "a: line 5" >>a.txt &&
+	   cvs commit -m "cvs commit 2" &&
+	   cvs tag split &&
+	   cvs tag -b branch &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 6" >>a.txt &&
+	   cvs commit -m "cvs commit 2" &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 7" >>a.txt &&
+	   cvs commit -m "cvs commit 3" &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   cvs up -r branch &&
+	   sed -e 's/2/B/' a.txt >t &&
+	   mv t a.txt &&
+	   cvs commit -m "cvs commit on branch" &&
+	   cvs up -A &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'merging' \
+    '( cd "$CVSWORK" &&
+	   cvs up -j branch
+	 ) &&
+	 (
+	   cd "$CVSIMPORTED" &&
+	   git-checkout cvshead &&
+	   git-merge branch
+	 ) &&
+	 diff -q "$CVSWORK/a.txt" "$CVSIMPORTED/a.txt"'
+
+CVSROOT=$(pwd)/cvsrootright
+CVSWORK=$(pwd)/cvsworkright
+CVSIMPORTED=$(pwd)/cvsimportedright
+export CVSROOT CVSWORK
+
+cvspscache="$HOME/.cvsps/$(echo $CVSROOT | sed -e 's%/%#%g')#src"
+rm -f $cvspscache
+
+rm -rf "$CVSROOT" "$CVSWORK"
+mkdir "$CVSROOT" &&
+cvs init &&
+mkdir "$CVSROOT/src"
+cvs -Q co -d "$CVSWORK" src &&
+rm -rf .git || 
+exit 1
+
+# sleeps are needed to fight cvsps' fuzz
+test_expect_success \
+	'initial cvs commits, tag, and branch' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 1" >>a.txt &&
+	   cvs add a.txt &&
+	   cvs commit -m "cvs commit 1" &&
+	   sleep 2 &&
+	   echo "a: line 2" >>a.txt &&
+	   echo "a: line 3" >>a.txt &&
+	   echo "a: line 4" >>a.txt &&
+	   echo "a: line 5" >>a.txt &&
+	   cvs commit -m "cvs commit 2" &&
+	   cvs tag split &&
+	   cvs tag -b branch &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   cvs up -r branch &&
+	   sed -e 's/2/B/' a.txt >t &&
+	   mv t a.txt &&
+	   cvs commit -m "cvs commit on branch" &&
+	   cvs up -A &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 6" >>a.txt &&
+	   cvs commit -m "cvs commit 2" &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'cvs commit' \
+	'( cd "$CVSWORK" &&
+	   echo "a: line 7" >>a.txt &&
+	   cvs commit -m "cvs commit 3" &&
+	   sleep 2
+	)'
+
+test_expect_success \
+	'importing to git' \
+	'git-cvsimport -v -a -i -k -u -z 1 -a -C $CVSIMPORTED -o cvshead src'
+
+test_expect_success \
+	'merging' \
+    '( cd "$CVSWORK" &&
+	   cvs up -j branch
+	 ) &&
+	 (
+	   cd "$CVSIMPORTED" &&
+	   git-checkout cvshead &&
+	   git-merge branch
+	 ) &&
+	 diff -q "$CVSWORK/a.txt" "$CVSIMPORTED/a.txt"'
+
+test_done
-- 
1.5.2.2.315.gc649a

Re: [PATCH] cvsimport: test case for severe branch import problem

[Steffen Prohaska]

On Jun 23, 2007, at 2:51 PM, Steffen Prohaska wrote:

> The conclusion is you must not rely on the existing cvsimport for
> tracking cvs branches. The history of such branches may be plain  
> wrong.
> Git may display different patches than cvs would do. Merging cvs
> topic branches may yield completely wrong results. One obvious thing
> that may happen is that a merge reverts changes commited to the cvs
> trunk before the first commit to the cvs branch. And this would happen
> without any indication by git. Everything would seem to run smoothely.

This is not only a theoretical problem but I am experiencing it on a
real-world repository right now. I have a topic branch that branches
off from the wrong commit in git.

Is there an easy way to fix this? I know the right commit the branch
should have as its parent. How can I move it there? git-rebase is not  
the
right command because the patches derived from my branch are already  
wrong.
I would only need to attach the first commit to a different parent.

	Steffen

[PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

git-transplant.sh <onto> <from> <to>

transplant starts with the contents of <onto> and puts on top of
it the contents of files if they are touched by the series of
commits <from>..<to>.  If a commit touches a file the content of
this file is taken as it is in the commit. No merging is
performed. Original authors, commiters, and commit messages are
preserved.

Warning: this is just a quick hack to solve _my_ problem.
- No error checking is performed.
- Removal of files is not handled.
- Whitespace in filename is not handled.
- The index is left in dirty state.
- No branch is created for the result.
- The script is not integrated with git's shell utilities.

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
---
 git-transplant.sh |   60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)
 create mode 100755 git-transplant.sh

This script seems to solved the problem for me. I can place
the topic branch imported from cvs to the right place.

What do you think? Is this a sane way to handle the situation?

    Steffen

diff --git a/git-transplant.sh b/git-transplant.sh
new file mode 100755
index 0000000..3320071
--- /dev/null
+++ b/git-transplant.sh
@@ -0,0 +1,60 @@
+#!/bin/sh
+
+[[ $# == 3 ]] || { echo "$0 <onto> <from> <to>"; exit 1; }
+onto=$(git-rev-parse $1)
+from=$(git-rev-parse $2)
+to=$(git-rev-parse $3)
+
+# copied from git-filter-branch.sh 
+set_ident () {
+    lid="$(echo "$1" | tr "A-Z" "a-z")"
+    uid="$(echo "$1" | tr "a-z" "A-Z")"
+    pick_id_script='
+        /^'$lid' /{
+            s/'\''/'\''\\'\'\''/g
+            h
+            s/^'$lid' \([^<]*\) <[^>]*> .*$/\1/
+            s/'\''/'\''\'\'\''/g
+            s/.*/export GIT_'$uid'_NAME='\''&'\''/p
+
+            g
+            s/^'$lid' [^<]* <\([^>]*\)> .*$/\1/
+            s/'\''/'\''\'\'\''/g
+            s/.*/export GIT_'$uid'_EMAIL='\''&'\''/p
+
+            g
+            s/^'$lid' [^<]* <[^>]*> \(.*\)$/\1/
+            s/'\''/'\''\'\'\''/g
+            s/.*/export GIT_'$uid'_DATE='\''&'\''/p
+
+            q
+        }
+    '
+
+    LANG=C LC_ALL=C sed -ne "$pick_id_script"
+    # Ensure non-empty id name.
+    echo "[ -n \"\$GIT_${uid}_NAME\" ] || export GIT_${uid}_NAME=\"\${GIT_${uid}_EMAIL%%@*}\""
+}
+
+parent=$onto
+git-read-tree --reset $parent
+
+for commit in $(git-rev-list --reverse $from..$to)
+do
+    echo "rewriting commit $commit..."
+    git-diff-tree -r $commit | grep ^: | cut -b 9-15,57-97,100- |
+    while read mode sha path 
+    do
+        echo " $mode $sha $path"
+        git-update-index --add --cacheinfo $mode $sha $path
+    done
+
+    eval "$(git-cat-file commit $commit |set_ident AUTHOR)"
+    eval "$(git-cat-file commit $commit |set_ident COMMITTER)"
+
+    parent=$(git-cat-file commit $commit |  sed -e '1,/^$/d' | git-commit-tree $(git-write-tree) -p $parent)
+    echo "... new commit $parent"
+done
+
+echo ""
+echo "new head is $parent"
-- 
1.5.2.2.315.gc649a

Re: [PATCH] transplant: move a series of commits to a different parent

[Johannes Schindelin]

Hi,

On Sat, 23 Jun 2007, Steffen Prohaska wrote:

> git-transplant.sh <onto> <from> <to>
> 
> transplant starts with the contents of <onto> and puts on top of
> it the contents of files if they are touched by the series of
> commits <from>..<to>.

This reeks of rebase.

IOW, I suspect that it does almost the same as

	git checkout <to>
	git rebase -s ours --onto <onto> <from>^

Ciao,
Dscho

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 23, 2007, at 10:54 PM, Johannes Schindelin wrote:

>> git-transplant.sh <onto> <from> <to>
>>
>> transplant starts with the contents of <onto> and puts on top of
>> it the contents of files if they are touched by the series of
>> commits <from>..<to>.
>
> This reeks of rebase.
>
> IOW, I suspect that it does almost the same as
>
> 	git checkout <to>
> 	git rebase -s ours --onto <onto> <from>^

It doesn't do anything useful for me. In fact it seems as if it
did nothing.

I tried your proposal:
  - rebase says 'Changes from <onto> to <onto>',
  - then it rewinds to <onto>,
  - next it says several time 'Already applied: ...' with increasing
    patch numbers,
  - then 'All done',
  - The result is the same as if I executed 'git reset --hard <onto>'.

I thought about something similar before I wrote transplant.
Honestly, I didn't understand what rebase would do combined with ours.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

On 6/23/07, Steffen Prohaska <prohaska@zib.de> wrote:
> git-transplant.sh <onto> <from> <to>
>
> transplant starts with the contents of <onto> and puts on top of
> it the contents of files if they are touched by the series of
> commits <from>..<to>.  If a commit touches a file the content of
> this file is taken as it is in the commit. No merging is
> performed. Original authors, commiters, and commit messages are
> preserved.
>
> Warning: this is just a quick hack to solve _my_ problem.
> - No error checking is performed.
> - Removal of files is not handled.
> - Whitespace in filename is not handled.
> - The index is left in dirty state.
> - No branch is created for the result.
> - The script is not integrated with git's shell utilities.

# detached head
git checkout $(git rev-parse onto) && git format-patch --stdout
--full-index from..to|git am -3

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 23, 2007, at 11:04 PM, Alex Riesen wrote:

>> git-transplant.sh <onto> <from> <to>
>>
>> transplant starts with the contents of <onto> and puts on top of
>> it the contents of files if they are touched by the series of
>> commits <from>..<to>.  If a commit touches a file the content of
>> this file is taken as it is in the commit. No merging is
>> performed. Original authors, commiters, and commit messages are
>> preserved.
>>
> [...]
> # detached head
> git checkout $(git rev-parse onto) && git format-patch --stdout
> --full-index from..to|git am -3

No. This one tries to apply the _changes_ between from..to. What I
need is the resulting _content_ of files modified between from..to.

The _changes_ are already wrong because they are relative to the
history. But the history was messed up by git-cvsimport, as I tried to
explaine in my first mail in this thread. So the changes derived
from the wrong history are useless.

transplant only checks if a file is modified by a commit. If it is
it takes the _content_ of the file in that commit. The changes from
the parent commit, which you can find by format-patch, do not matter.

I believe it's more like git-filter-branch, but I wasn't yet abel to
tell git-filter-branch how to do the job.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

Steffen Prohaska, Sun, Jun 24, 2007 09:08:16 +0200:
> 
> On Jun 23, 2007, at 11:04 PM, Alex Riesen wrote:
> 
> >>git-transplant.sh <onto> <from> <to>
> >>
> >>transplant starts with the contents of <onto> and puts on top of
> >>it the contents of files if they are touched by the series of
> >>commits <from>..<to>.  If a commit touches a file the content of
> >>this file is taken as it is in the commit. No merging is
> >>performed. Original authors, commiters, and commit messages are
> >>preserved.
> >>
> >[...]
> ># detached head
> >git checkout $(git rev-parse onto) && git format-patch --stdout
> >--full-index from..to|git am -3
> 
> No. This one tries to apply the _changes_ between from..to. What I
> need is the resulting _content_ of files modified between from..to.
> 

Ach, yes. I should have read your message a bit more closely. There is
a term for that "contents of files", BTW: "repository state".

> I believe it's more like git-filter-branch, but I wasn't yet abel to
> tell git-filter-branch how to do the job.

I suspect git-filter-branch can be both.

Re: [PATCH] transplant: move a series of commits to a different parent

[Johannes Schindelin]

Hi,

On Sun, 24 Jun 2007, Steffen Prohaska wrote:

> On Jun 23, 2007, at 11:04 PM, Alex Riesen wrote:
> 
> > # detached head
> > git checkout $(git rev-parse onto) && git format-patch --stdout
> > --full-index from..to|git am -3
> 
> No. This one tries to apply the _changes_ between from..to. What I need 
> is the resulting _content_ of files modified between from..to.

Oh! But the commit messages do no longer correspond to their patches, do 
they?

Example:

In "onto", you have a sorely needed bugfix in main.c. In "from", you have 
not. Then you do your transplant, and all of a sudden, the 
first transplanted commit _undoes_ that bugfix (because you take the 
contents at face value), but the commit message _cannot_ say so, or even 
why.

IMHO this makes no sense (and that is why I misunderstood it as being a 
rebase).

Ciao,
Dscho

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 24, 2007, at 12:26 PM, Johannes Schindelin wrote:

>
> Oh! But the commit messages do no longer correspond to their  
> patches, do
> they?
>
> Example:
>
> In "onto", you have a sorely needed bugfix in main.c. In "from",  
> you have
> not. Then you do your transplant, and all of a sudden, the
> first transplanted commit _undoes_ that bugfix (because you take the
> contents at face value), but the commit message _cannot_ say so, or  
> even
> why.
>
> IMHO this makes no sense (and that is why I misunderstood it as  
> being a
> rebase).

It doesn't make sense on a sane repository.

I need the script to fix an insane, broken repository that was generated
by git-cvsimport [1]. cvsimport created commits attached to the wrong  
parent
in the first place. So the patches derived from this history are wrong.
They are different from the patches that you'd expect from the cvs  
repository.
The commit messages and the patches did never correspond. My script  
fixes
this relationship. Only after I transplanted the branch the commits and
their messages match.

The following is a different illustration of the same problem.  
Suppose you copy
a file and modify its contents somewhere else. Now you checkout the  
_wrong_
branch, but do not recognize, copy the file back and commit it. You'd be
writing a message as if the right branch would have been checked out.  
This
message describes what you believe you did. But because of the wrong  
branch
you may have done something completely different to the repository.  
You may
for example have reverted earlier changes. Your commit doesn't make  
any sense
before you transplanted it to the right branch. You don't want to  
apply a
patch to the right branch but transplant the content of your file.

	Steffen

[1] http://article.gmane.org/gmane.comp.version-control.git/50736

Re: [PATCH] transplant: move a series of commits to a different parent

[Johannes Sixt]

Steffen Prohaska wrote:
> 
> On Jun 23, 2007, at 11:04 PM, Alex Riesen wrote:
> 
> >> git-transplant.sh <onto> <from> <to>
> >>
> >> transplant starts with the contents of <onto> and puts on top of
> >> it the contents of files if they are touched by the series of
> >> commits <from>..<to>.  If a commit touches a file the content of
> >> this file is taken as it is in the commit. No merging is
> >> performed. Original authors, commiters, and commit messages are
> >> preserved.
> >>
> > [...]
> > # detached head
> > git checkout $(git rev-parse onto) && git format-patch --stdout
> > --full-index from..to|git am -3
> 
> No. This one tries to apply the _changes_ between from..to. What I
> need is the resulting _content_ of files modified between from..to.

Install a graft that points the parent of "from" to "onto", then run
git-filter-branch. Like so:

 $ echo "$(git-rev-parse from^0) $(git-rev-parse onto^0)" \
        > .git/info/grafts
 $ git-filter-branch new-to onto..to

(no filters necessary).

-- Hannes

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 25, 2007, at 9:16 AM, Johannes Sixt wrote:

> Steffen Prohaska wrote:
>>
>> On Jun 23, 2007, at 11:04 PM, Alex Riesen wrote:
>>
>>>> git-transplant.sh <onto> <from> <to>
>>>>
>>>> transplant starts with the contents of <onto> and puts on top of
>>>> it the contents of files if they are touched by the series of
>>>> commits <from>..<to>.  If a commit touches a file the content of
>>>> this file is taken as it is in the commit. No merging is
>>>> performed. Original authors, commiters, and commit messages are
>>>> preserved.
>>>>
>>> [...]
>>> # detached head
>>> git checkout $(git rev-parse onto) && git format-patch --stdout
>>> --full-index from..to|git am -3
>>
>> No. This one tries to apply the _changes_ between from..to. What I
>> need is the resulting _content_ of files modified between from..to.
>
> Install a graft that points the parent of "from" to "onto", then run
> git-filter-branch. Like so:

I stumbled over the grafts yesterday and thought that this may be a  
way...

>  $ echo "$(git-rev-parse from^0) $(git-rev-parse onto^0)" \
>     > .git/info/grafts
>  $ git-filter-branch new-to onto..to

... ok this is interesting: git-filter-branch will take the info from
grafts to create its new commits. The new commits will contain all
information independent of the grafts file, right?

... but, I'm not convinced that this is what I need. I need to mix the
tree of onto with the files _modified_ between from..to. Taking _all_
files from the commits between from..to is _wrong_. And I think that
is what the command you proposed would do.

My feeling is that repairing a repository resulting from a broken
cvsimport is a bit more complex than what the existing tools provide
out-of-the box.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Johannes Sixt]

Steffen Prohaska wrote:
> 
> On Jun 25, 2007, at 9:16 AM, Johannes Sixt wrote:
> >  $ echo "$(git-rev-parse from^0) $(git-rev-parse onto^0)" \
> >     > .git/info/grafts
> >  $ git-filter-branch new-to onto..to
> 
> ... ok this is interesting: git-filter-branch will take the info from
> grafts to create its new commits. The new commits will contain all
> information independent of the grafts file, right?

Yes.

> ... but, I'm not convinced that this is what I need. I need to mix the
> tree of onto with the files _modified_ between from..to. Taking _all_
> files from the commits between from..to is _wrong_. And I think that
> is what the command you proposed would do.

My proposed commands won't do what you need. They just change
parenthood, but not the tree snapshots that they represent.

-- Hannes

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

Steffen Prohaska, Sat, Jun 23, 2007 21:27:57 +0200:
> +for commit in $(git-rev-list --reverse $from..$to)
> +do
> +    echo "rewriting commit $commit..."
> +    git-diff-tree -r $commit | grep ^: | cut -b 9-15,57-97,100- |
> +    while read mode sha path 
> +    do
> +        echo " $mode $sha $path"
> +        git-update-index --add --cacheinfo $mode $sha $path
> +    done

Why not just read-tree for every commit? It is not like you're
modifying the repository in any way, just changing parenthood. That'd
solve the problem with deletions.
So it should be enough to read-tree the repo state for each and every
source commit into the index (and you can just use a temporary index
file for that, see GIT_INDEX_FILE). Than just commit the index.

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 24, 2007, at 10:29 AM, Alex Riesen wrote:

> Steffen Prohaska, Sat, Jun 23, 2007 21:27:57 +0200:
>> +for commit in $(git-rev-list --reverse $from..$to)
>> +do
>> +    echo "rewriting commit $commit..."
>> +    git-diff-tree -r $commit | grep ^: | cut -b 9-15,57-97,100- |
>> +    while read mode sha path
>> +    do
>> +        echo " $mode $sha $path"
>> +        git-update-index --add --cacheinfo $mode $sha $path
>> +    done
>
> Why not just read-tree for every commit? It is not like you're
> modifying the repository in any way, just changing parenthood. That'd
> solve the problem with deletions.
> So it should be enough to read-tree the repo state for each and every
> source commit into the index (and you can just use a temporary index
> file for that, see GIT_INDEX_FILE). Than just commit the index.

I am changing the repository.

I only modify the index for files that have changes in $commit. Their
content gets replaced by the content from the commit. I'm leaving
all other files untouched.

This creates a new series of commits that starts from the repository
state of <onto> and has mixed in files only if they are changed in
the series of commits from..to. These files are just replaced. I'm not
trying to merge changes but just replace the whole file.

Opposed to that, read-tree would modify the content of _all_ files.


Here's the situation before transplant

           o--Y--3
          /
   1--X--2--o--o--o

Say at X the file x.txt got modified. At Y the file y.txt got modified.
3 has both modifications.

Now I do transplant 1 2 3, which yields

     o--Y--4
    /
   1--X--2--o--o--o

y.txt is identical in 3 and 4 but x.txt is identical in 1 (!) and 4.
Hence 3 and 4 are different. The changes to x.txt in commit X got
eliminated from the history. 4 is a mixture of 1 and the repository
state of files at 3 that got modified between 2 and 3. Changes between
1 and 2 got eliminated from the history.

This is exactly what I want to achieve. The content of the files on
branch 3 is correct for all files that were committed after 2. But
because 2 is the wrong branching point all the content originating
from commits between 1 and 2 is wrong. Files committed between
2 and 3 have the right content but the branch needs to be attached
at 1.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

Steffen Prohaska, Sun, Jun 24, 2007 11:05:29 +0200:
> >Steffen Prohaska, Sat, Jun 23, 2007 21:27:57 +0200:
> >>+for commit in $(git-rev-list --reverse $from..$to)
> >>+do
> >>+    echo "rewriting commit $commit..."
> >>+    git-diff-tree -r $commit | grep ^: | cut -b 9-15,57-97,100- |
> >>+    while read mode sha path
> >>+    do
> >>+        echo " $mode $sha $path"
> >>+        git-update-index --add --cacheinfo $mode $sha $path
> >>+    done
> >
> >Why not just read-tree for every commit? It is not like you're
> >modifying the repository in any way, just changing parenthood. That'd
> >solve the problem with deletions.
> >So it should be enough to read-tree the repo state for each and every
> >source commit into the index (and you can just use a temporary index
> >file for that, see GIT_INDEX_FILE). Than just commit the index.
> 
> I am changing the repository.
> 

No, you don't.

> I only modify the index for files that have changes in $commit. Their
> content gets replaced by the content from the commit. I'm leaving
> all other files untouched.

No, you don't modify anything. Ever tried to run git-status after your
script finished? Tried to understand what the output means?

> This creates a new series of commits that starts from the repository
> state of <onto> and has mixed in files only if they are changed in
> the series of commits from..to. These files are just replaced. I'm not
> trying to merge changes but just replace the whole file.
> 
> Opposed to that, read-tree would modify the content of _all_ files.

No, it wouldn't (unless you run git-read-tree -u, and I fail to see
why would you want that). You probably confuse git-read-tree with
git-checkout-index.

> This is exactly what I want to achieve. The content of the files on
> branch 3 is correct for all files that were committed after 2. But
> because 2 is the wrong branching point all the content originating
> from commits between 1 and 2 is wrong. Files committed between
> 2 and 3 have the right content but the branch needs to be attached
> at 1.

This misses merges (see git-rev-list --parents), but does the job for
linear history:

    export GIT_INDEX_FILE="$(git rev-parse --git-dir)/tr.idx"
    parent=$(git rev-parse "$onto")
    git rev-list --reverse "$from..$to" | while read c
    do
	rm -f "$GIT_INDEX_FILE"
	git read-tree $c || break;
	# Authorship information here
	parent=$(git cat-file commit $c | \
	    sed -e '1,/^$/d' | \
	    git commit-tree $(git write-tree) -p "$parent")
	echo "Commit $parent"
    done

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 24, 2007, at 11:30 AM, Alex Riesen wrote:

>> I am changing the repository.
>>
>
> No, you don't.

Magically, the script solved my problem by creating a new, corrected
branch that is different from the original one. I didn't run any
other script. I promise, I ran the script that I sent in the patch.


>> I only modify the index for files that have changes in $commit. Their
>> content gets replaced by the content from the commit. I'm leaving
>> all other files untouched.
>
> No, you don't modify anything. Ever tried to run git-status after your
> script finished?

Yes.

> Tried to understand what the output means?

Not really. I instead reset the index to a controlled state.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

Steffen Prohaska, Sun, Jun 24, 2007 19:13:42 +0200:
> >>I am changing the repository.
> >
> >No, you don't.
> 
> Magically, the script solved my problem by creating a new, corrected
> branch that is different from the original one. I didn't run any
> other script. I promise, I ran the script that I sent in the patch.

I meant: "it does not change the working directory". It is irrelevant,
was a bit of confusion on my part.

Your script works, just it can be made simplier: no need for diff,
it'll only hurt perfomance and complicates things. And you don't have
to care about additions/deletions, and it is trivially extensible to
support merges, and the current index is untouched - your user can
continue working in predictable environment.

As to perfomance: read-tree doesn't actually _read_ the blobs to
populate index, just the trees. And diff-tree has do do the same, but
also _compare_ two trees recursively: more work, more memory needed.

BTW, Johannes moved that ident code you copied from git-filter-branch
into its own shell file, so it can be sourced and trivially reused.

Re: [PATCH] transplant: move a series of commits to a different parent

[Steffen Prohaska]

On Jun 24, 2007, at 8:35 PM, Alex Riesen wrote:

> Your script works, just it can be made simplier: no need for diff,
> it'll only hurt perfomance and complicates things. And you don't have
> to care about additions/deletions, and it is trivially extensible to
> support merges, and the current index is untouched - your user can
> continue working in predictable environment.

I understand that I can leave the default index untouched by using
a different index. I knew that this must be possible somehow but was
too lazy to find out how. Thanks for the details.

I don't see how I can avoid tree diffs. As I pointed out earlier I need
to mix the tree of the base commit of the newly built branch with
files that were changed in the series of commits that I'm transplanting.

Just taking the whole tree from the commits I'm transplanting is  
_wrong_.
I need to only take files that were touched by a commit. The tree of
the tip of the resulting branch can be quite different from the tree
of the tip of the original branch.

> As to perfomance: read-tree doesn't actually _read_ the blobs to
> populate index, just the trees. And diff-tree has do do the same, but
> also _compare_ two trees recursively: more work, more memory needed.

I'm still pretty convinced that I need tree diffs (not file diffs!).

> BTW, Johannes moved that ident code you copied from git-filter-branch
> into its own shell file, so it can be sourced and trivially reused.

I see, thanks.

Anyway, the script worked for me and I still think it may be useful for
fixing broken repositories resulting from a wrong cvsimport. I would
probably improve many details if someone else considered my work useful.
But up to now it seems as if I failed to explain, why the script would
be needed in the first place.

However, the best way would be to fix git-cvsimport to handle branches
correctly independently of the time of the first commit to a branch;
and avoid insane, broken repositories altogether.

	Steffen

Re: [PATCH] transplant: move a series of commits to a different parent

[Alex Riesen]

Steffen Prohaska, Sun, Jun 24, 2007 22:54:21 +0200:
> I don't see how I can avoid tree diffs. As I pointed out earlier I need
> to mix the tree of the base commit of the newly built branch with
> files that were changed in the series of commits that I'm transplanting.

Hmm...

> Just taking the whole tree from the commits I'm transplanting is  
> _wrong_.
> I need to only take files that were touched by a commit. The tree of
> the tip of the resulting branch can be quite different from the tree
> of the tip of the original branch.

I see.

> >As to perfomance: read-tree doesn't actually _read_ the blobs to
> >populate index, just the trees. And diff-tree has do do the same, but
> >also _compare_ two trees recursively: more work, more memory needed.
> 
> I'm still pretty convinced that I need tree diffs (not file diffs!).

"git-read-tree --reset" does an in-index merge (just discards unmerged
entries), so it still is better then git-diff-tree. But remove that
unlink, so that the previuos tree is not discarded and do a
"git-read-tree $onto" before starting the loop.

> Anyway, the script worked for me and I still think it may be useful for
> fixing broken repositories resulting from a wrong cvsimport. I would
> probably improve many details if someone else considered my work useful.
> But up to now it seems as if I failed to explain, why the script would
> be needed in the first place.
> 
> However, the best way would be to fix git-cvsimport to handle branches
> correctly independently of the time of the first commit to a branch;
> and avoid insane, broken repositories altogether.

You still better make it work properly wrt deleted files.
And you have to be careful not to hit a real content conflict.