* Re: [PATCH 2/2] fast-import: treat cat-blob as a delta base hint for next blob
From: Jonathan Nieder @ 2011-08-20 19:17 UTC (permalink / raw)
To: Dmitry Ivankov; +Cc: git, Shawn O. Pearce, David Barr
In-Reply-To: <1313867052-11993-3-git-send-email-divanorama@gmail.com>
Dmitry Ivankov wrote:
> --- a/fast-import.c
> +++ b/fast-import.c
> @@ -2802,7 +2802,12 @@ static void cat_blob(struct object_entry *oe, unsigned char sha1[20])
> strbuf_release(&line);
> cat_blob_write(buf, size);
> cat_blob_write("\n", 1);
> - free(buf);
> + if (oe && oe->pack_id == pack_id) {
> + strbuf_attach(&last_blob.data, buf, size, size);
> + last_blob.offset = oe->idx.offset;
> + last_blob.depth = oe->depth;
> + } else
> + free(buf);
> }
Neat. For what it's worth,
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
^ permalink raw reply
* Re: [PATCH v3 0/4] git-p4: Improve branch support
From: Pete Wyckoff @ 2011-08-20 19:14 UTC (permalink / raw)
To: Vitor Antunes; +Cc: git, Tor Arvid Lund
In-Reply-To: <CAOpHH-WeLWyEZaktYRzVN+dfgqu3==o4bgrfzpvS-QUCQboPgg@mail.gmail.com>
vitor.hda@gmail.com wrote on Fri, 19 Aug 2011 15:30 +0100:
> On Fri, Aug 19, 2011 at 12:53 PM, Pete Wyckoff <pw@padd.com> wrote:
> > Patches 1 and 2 are great. We should have gotten those in way
> > back when you first submitted them. I happily ack those.
> >
> > I'm still a bit hung up on #3, mainly because I don't get branch
> > support. Let me play around with your test. Having this
> > playbook of how it is supposed to work will help to educate me.
>
> (I am probably going in too much detail here. Please skip any
> redundant information.)
>
> In general, you can see branches in P4 as you do in SVN. They are
> simple copies from one directory to another. For example:
>
> p4 integrate //depot/big/directory/path/myproj/version_A \
> //depot/big/directory/path/myproj/version_B
>
> Now the history of "version_B" is tied to "version_A". If you want to
> integrate (merge, in this case) new updates from one of the
> directories to the other you can just rerun the above command
> (version_A and version_B order may change depending on the direction
> of the merge).
>
> P4 also allows you to define "branch specs" that you can use as a
> short hand instead of having to type in the full directory paths. So,
> assuming that you have a branch named "myproj_B" with the following
> view defined:
>
> //depot/big/directory/path/myproj/version_A \
> //depot/big/directory/path/myproj/version_B
>
> When you want to integrate it again you can simply type:
>
> p4 integrate -b myproj_B
>
> or, if you want to integrate from version_B into version_A instead:
>
> p4 integrate -b myproj_B -r
>
> git-p4 is using these branch specs to identify branches. Without this
> extra information it is nearly impossible for it to identify branches
> because an integrate can be done of a single file.
>
> But since P4 does not strictly require branch specs and because most
> of the time integrations are done through P4V, then most of the time
> no branch specs are created. Now, if this happens git-p4 will not be
> able to detect branches! And creating branches in P4 just for git-p4
> does not make much sense, right? This is the main reason behind the
> third patch... :)
Thank you for these useful comments. I have taken a first stab
at documenting the bevy of confusing branch flags and settings
in git-p4. Please correct my errors and add other information
that is missing.
In particular, a clear case of how to set up the branches for
the branchList example would be good. My minimal test did not
exactly work.
-- Pete
--------------8<----------------------
From 36608a6e195041e8738bab24b84fadd0bf386865 Mon Sep 17 00:00:00 2001
From: Pete Wyckoff <pw@padd.com>
Date: Sat, 20 Aug 2011 15:04:09 -0400
Subject: [PATCH 4/4] git-p4: branch detection documentation start
Initial stab at documentation for --detect-branches and the
new branchList config setting.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
contrib/fast-import/git-p4.txt | 48 +++++++++++++++++++++++++++++++++++++++-
1 files changed, 47 insertions(+), 1 deletions(-)
diff --git a/contrib/fast-import/git-p4.txt b/contrib/fast-import/git-p4.txt
index 52003ae..6d21557 100644
--- a/contrib/fast-import/git-p4.txt
+++ b/contrib/fast-import/git-p4.txt
@@ -265,11 +265,57 @@ Only use branch specifications defined by the selected username.
git-p4.branchList
-List of branches to be imported when branch detection is enabled.
+List of branches to be imported when branch detection is enabled. This
+example shows that branchA and branchB are both created off branch main.
git config [--global] git-p4.branchList main:branchA
git config [--global] --add git-p4.branchList main:branchB
+Branch Detection
+================
+
+There are a few options that try to automatically create branches in git
+corresponding to branches in p4.
+
+Unfortunately, the concept of branch in p4 is quite different: a p4
+branch is a mapping from one area of the repository to another. It is
+used as input to "p4 integrate" to specify the source and destination of
+an integration.
+
+If you have a repository where all the branches are below a single
+directory, you can use "--detect-branches" when cloning or syncing to
+have git-p4 automatically look up the p4 branches and use these to
+generate branches in git. Example:
+
+ P4 repository structure:
+
+ //depot/main/...
+ //depot/branch1/...
+
+ View section from "p4 branch -o branch1"
+
+ //depot/main/... //depot/branch1/...
+
+ Git clone command:
+
+ git-p4 clone --detect-branches //depot@all
+
+ Produces a separate branch in remotes/p4 for
+ //depot/main (called "master"), and one for //depot/branch1
+ (called "depot/branch1").
+
+However, it is not necessary to create branches in p4 to be able to
+use them like branches. It is quite difficult to detect this
+automatically, but a git configuration setting "git-p4.branchList" can be
+used to explicitly identify branch relationships. It is a list of
+"source":"destination" pairs, like a simple p4 branch specification,
+where the "source" and "destination" are the path elements in the
+p4 repository. The example above relied on the presence of the p4
+branch. Without that, you can get the same results by setting:
+
+ git config git-p4.branchList main:branch1
+
+
Implementation Details...
=========================
--
1.7.5.4
^ permalink raw reply related
* Re: [PATCH/RFC 2/2] git-p4: Add complex test case for branch import
From: Pete Wyckoff @ 2011-08-20 19:12 UTC (permalink / raw)
To: Vitor Antunes; +Cc: git, Tor Arvid Lund
In-Reply-To: <1313712590-26876-3-git-send-email-vitor.hda@gmail.com>
vitor.hda@gmail.com wrote on Fri, 19 Aug 2011 01:09 +0100:
> Check if branches created from old changelists are correctly imported.
>
> Signed-off-by: Vitor Antunes <vitor.hda@gmail.com>
Style edits.
-- Pete
---------------8<-------------------
>From 5d1103a97b60570423782fd3873a2b413ddb1d5b Mon Sep 17 00:00:00 2001
From: Pete Wyckoff <pw@padd.com>
Date: Sat, 20 Aug 2011 14:31:49 -0400
Subject: [PATCH 3/4] git-p4: complex branch test edits
A few simplifications and style edits here too.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
t/t9800-git-p4.sh | 21 +++++++++------------
1 files changed, 9 insertions(+), 12 deletions(-)
diff --git a/t/t9800-git-p4.sh b/t/t9800-git-p4.sh
index 5bcb8b4..cf3a70e 100755
--- a/t/t9800-git-p4.sh
+++ b/t/t9800-git-p4.sh
@@ -489,11 +489,11 @@ test_expect_success 'git-p4 clone simple branches' '
# `- file3
test_expect_success 'add p4 complex branches' '
cd "$cli" &&
- changelist=$(p4 changes -m1 //depot/... | cut -d \ -f 2) &&
- changelist=$(($changelist - 5)) &&
+ changelist=$(p4 changes -m1 //depot/... | cut -d" " -f2) &&
+ changelist=$((changelist - 5)) &&
p4 integrate //depot/branch1/...@$changelist //depot/branch4/... &&
p4 submit -d "branch4" &&
- changelist=$(($changelist + 2)) &&
+ changelist=$((changelist + 2)) &&
p4 integrate //depot/branch1/...@$changelist //depot/branch5/... &&
p4 submit -d "branch5" &&
cd "$TRASH_DIRECTORY"
@@ -503,15 +503,14 @@ test_expect_success 'add p4 complex branches' '
# to clone the original structure if it is able to detect the origin changelist
# of each branch.
test_expect_success 'git-p4 clone complex branches' '
- git init "$git" &&
+ test_when_finished cleanup_git &&
+ test_create_repo "$git" &&
cd "$git" &&
git config git-p4.branchList branch1:branch2 &&
git config --add git-p4.branchList branch1:branch3 &&
git config --add git-p4.branchList branch1:branch4 &&
git config --add git-p4.branchList branch1:branch5 &&
- cd "$TRASH_DIRECTORY" &&
- "$GITP4" clone --dest="$git" --detect-branches //depot@all &&
- cd "$git" &&
+ "$GITP4" clone --dest=. --detect-branches //depot@all &&
git log --all --graph --decorate --stat &&
git reset --hard p4/depot/branch1 &&
test -f file1 &&
@@ -521,7 +520,7 @@ test_expect_success 'git-p4 clone complex branches' '
git reset --hard p4/depot/branch2 &&
test -f file1 &&
test -f file2 &&
- test \! -f file3 &&
+ test ! -f file3 &&
! grep -q update file2 &&
git reset --hard p4/depot/branch3 &&
test -f file1 &&
@@ -531,15 +530,13 @@ test_expect_success 'git-p4 clone complex branches' '
git reset --hard p4/depot/branch4 &&
test -f file1 &&
test -f file2 &&
- test \! -f file3 &&
+ test ! -f file3 &&
! grep -q update file2 &&
git reset --hard p4/depot/branch5 &&
test -f file1 &&
test -f file2 &&
test -f file3 &&
- ! grep -q update file2 &&
- cd "$TRASH_DIRECTORY" &&
- rm -rf "$git" && mkdir "$git"
+ ! grep -q update file2
'
test_expect_success 'shutdown' '
--
1.7.5.4
^ permalink raw reply related
* Re: [PATCH v3 4/4] git-p4: Add simple test case for branch import
From: Pete Wyckoff @ 2011-08-20 19:11 UTC (permalink / raw)
To: Vitor Antunes; +Cc: git, Tor Arvid Lund
In-Reply-To: <1313711046-23489-5-git-send-email-vitor.hda@gmail.com>
vitor.hda@gmail.com wrote on Fri, 19 Aug 2011 00:44 +0100:
> Create a basic branch structure in P4 and clone it with git-p4.
> Also, make an update on P4 side and check if git-p4 imports it correctly.
> The branch structure is created in such a way that git-p4 will fail to import
> updates if patch "git-p4: Correct branch base depot path detection" is not
> applied.
Here are some minor style edits to this test.
-- Pete
-----------------8<-----------------------
>From 958effaad24d129ad472b48f3a691c7e7de2e918 Mon Sep 17 00:00:00 2001
From: Pete Wyckoff <pw@padd.com>
Date: Sat, 20 Aug 2011 14:24:31 -0400
Subject: [PATCH 2/4] git-p4: simple branch tests edits
More review comments.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
t/t9800-git-p4.sh | 17 +++++++----------
1 files changed, 7 insertions(+), 10 deletions(-)
diff --git a/t/t9800-git-p4.sh b/t/t9800-git-p4.sh
index bbf1485..5bcb8b4 100755
--- a/t/t9800-git-p4.sh
+++ b/t/t9800-git-p4.sh
@@ -412,7 +412,7 @@ test_expect_success 'add simple p4 branches' '
cd branch1 &&
echo file1 >file1 &&
echo file2 >file2 &&
- p4 add file* &&
+ p4 add file1 file2 &&
p4 submit -d "branch1" &&
p4 integrate //depot/branch1/... //depot/branch2/... &&
p4 submit -d "branch2" &&
@@ -432,13 +432,12 @@ test_expect_success 'add simple p4 branches' '
# Finally, make an update to branch1 on P4 side to check if it is imported
# correctly by git-p4.
test_expect_success 'git-p4 clone simple branches' '
- git init "$git" &&
+ test_when_finished cleanup_git &&
+ test_create_repo "$git" &&
cd "$git" &&
git config git-p4.branchList branch1:branch2 &&
git config --add git-p4.branchList branch1:branch3 &&
- cd "$TRASH_DIRECTORY" &&
- "$GITP4" clone --dest="$git" --detect-branches //depot@all &&
- cd "$git" &&
+ "$GITP4" clone --dest=. --detect-branches //depot@all &&
git log --all --graph --decorate --stat &&
git reset --hard p4/depot/branch1 &&
test -f file1 &&
@@ -448,7 +447,7 @@ test_expect_success 'git-p4 clone simple branches' '
git reset --hard p4/depot/branch2 &&
test -f file1 &&
test -f file2 &&
- test \! -z file3 &&
+ test ! -f file3 &&
! grep -q update file2 &&
git reset --hard p4/depot/branch3 &&
test -f file1 &&
@@ -458,14 +457,12 @@ test_expect_success 'git-p4 clone simple branches' '
cd "$cli" &&
cd branch1 &&
p4 edit file2 &&
- echo file2_ >> file2 &&
+ echo file2_ >>file2 &&
p4 submit -d "update file2 in branch3" &&
cd "$git" &&
git reset --hard p4/depot/branch1 &&
"$GITP4" rebase &&
- grep -q file2_ file2 &&
- cd "$TRASH_DIRECTORY" &&
- rm -rf "$git" && mkdir "$git"
+ grep -q file2_ file2
'
# Create a complex branch structure in P4 depot to check if they are correctly
--
1.7.5.4
^ permalink raw reply related
* Re: [PATCH v2 0/4] Support threshold in copy/rename detection
From: Pete Wyckoff @ 2011-08-20 19:09 UTC (permalink / raw)
To: Vitor Antunes; +Cc: git, Tor Arvid Lund
In-Reply-To: <1313706054-11740-1-git-send-email-vitor.hda@gmail.com>
vitor.hda@gmail.com wrote on Thu, 18 Aug 2011 23:20 +0100:
> Second version that includes updates recommended by Pete Wyckoff.
> Now only "true" and "false" arguments are processed, any other argument
> that is no "" is passed directly.
>
> Vitor Antunes (4):
> git-p4: Allow setting rename/copy detection threshold.
> git-p4: Add description of rename/copy detection options.
> git-p4: Add test case for rename detection.
> git-p4: Add test case for copy detection.
Here is a patch of review comments. Some of these are style
and correctness fixes. Others are my annotations to understand
what's going on in the test. You might take the good ones and
send them to Junio.
-- Pete
-----------------------8<--------------------
>From e8c627b08248bc17e8c4e6ca246099b0431a4bfb Mon Sep 17 00:00:00 2001
From: Pete Wyckoff <pw@padd.com>
Date: Sat, 20 Aug 2011 14:10:18 -0400
Subject: [PATCH 1/4] git-p4: copy/rename test case edits
Some review comments. Take the ones in here you like.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
t/t9800-git-p4.sh | 88 ++++++++++++++++++++++++++++++----------------------
1 files changed, 51 insertions(+), 37 deletions(-)
diff --git a/t/t9800-git-p4.sh b/t/t9800-git-p4.sh
index 2a872bc..bbf1485 100755
--- a/t/t9800-git-p4.sh
+++ b/t/t9800-git-p4.sh
@@ -272,24 +272,22 @@ test_expect_success 'initial import time from top change time' '
# Rename a file and confirm that rename is not detected in P4.
# Rename the new file again with detectRenames option enabled and confirm that
# this is detected in P4.
-# Rename the new file again adding an extra blank line, configure a big
-# threshold in detectRenames and confirm that rename is not detected in P4.
-# Rename the new file again adding another extra blank line, configure a small
+# Rename the new file again adding an extra line, configure a big
# threshold in detectRenames and confirm that rename is not detected in P4.
+# Repeat, this time with a smaller threshold.
test_expect_success 'detect renames' '
- git init "$git" &&
- cd "$git" &&
- cd "$TRASH_DIRECTORY" &&
"$GITP4" clone --dest="$git" //depot@all &&
- p4 files //depot/* &&
+ test_when_finished cleanup_git &&
cd "$git" &&
+ git config git-p4.skipSubmitEditCheck true &&
+
git mv file1 file4 &&
git commit -a -m "Rename file1 to file4" &&
git diff-tree -r -M HEAD &&
- git config git-p4.skipSubmitEditCheck true &&
"$GITP4" submit &&
p4 filelog //depot/file4 &&
! p4 filelog //depot/file4 | grep -q "branch from //depot/file1" &&
+
git mv file4 file5 &&
git commit -a -m "Rename file4 to file5" &&
git diff-tree -r -M HEAD &&
@@ -297,26 +295,30 @@ test_expect_success 'detect renames' '
"$GITP4" submit &&
p4 filelog //depot/file5 &&
p4 filelog //depot/file5 | grep -q "branch from //depot/file4" &&
+
git mv file5 file6 &&
- echo update >> file6 &&
+ echo update >>file6 &&
git add file6 &&
git commit -a -m "Rename file5 to file6 with changes" &&
git diff-tree -r -M HEAD &&
- git config git-p4.detectRenames 95 &&
+ level=$(git diff-tree -r -M HEAD | sed 1d | cut -f1 | cut -d" " -f5 | sed "s/R0*//") &&
+ test -n "$level" && test "$level" -gt 0 && test "$level" -lt 98 &&
+ git config git-p4.detectRenames $((level + 2)) &&
"$GITP4" submit &&
p4 filelog //depot/file6 &&
! p4 filelog //depot/file6 | grep -q "branch from //depot/file5" &&
+
git mv file6 file7 &&
- echo update >> file7 &&
+ echo update >>file7 &&
git add file7 &&
git commit -a -m "Rename file6 to file7 with changes" &&
git diff-tree -r -M HEAD &&
- git config git-p4.detectRenames 80 &&
+ level=$(git diff-tree -r -M HEAD | sed 1d | cut -f1 | cut -d" " -f5 | sed "s/R0*//") &&
+ test -n "$level" && test "$level" -gt 2 && test "$level" -lt 100 &&
+ git config git-p4.detectRenames $((level - 2)) &&
"$GITP4" submit &&
p4 filelog //depot/file7 &&
- p4 filelog //depot/file7 | grep -q "branch from //depot/file6" &&
- cd "$TRASH_DIRECTORY" &&
- rm -rf "$git" && mkdir "$git"
+ p4 filelog //depot/file7 | grep -q "branch from //depot/file6"
'
# Copy a file and confirm that copy is not detected in P4.
@@ -326,68 +328,80 @@ test_expect_success 'detect renames' '
# is detected in P4.
# Copy a file with detectCopies and detectCopiesHarder options enabled and
# confirm that copy is detected in P4.
-# Modify and copy a file, configure a big threshold in detectCopies and confirm
-# that copy is not detected in P4.
-# Modify and copy a file, configure a small threshold in detectCopies and
+# Modify and copy a file, configure a bigger threshold in detectCopies and
+# confirm that copy is not detected in P4.
+# Modify and copy a file, configure a smaller threshold in detectCopies and
# confirm that copy is detected in P4.
test_expect_success 'detect copies' '
- git init "$git" &&
- cd "$git" &&
- cd "$TRASH_DIRECTORY" &&
"$GITP4" clone --dest="$git" //depot@all &&
+ test_when_finished cleanup_git &&
cd "$git" &&
+ git config git-p4.skipSubmitEditCheck true &&
+
cp file2 file8 &&
git add file8 &&
git commit -a -m "Copy file2 to file8" &&
- git diff-tree -r -C HEAD
- git config git-p4.skipSubmitEditCheck true &&
+ git diff-tree -r -C HEAD &&
"$GITP4" submit &&
p4 filelog //depot/file8 &&
- ! p4 filelog //depot/file8 | grep -q "branch from //depot/file" &&
+ ! p4 filelog //depot/file8 | grep -q "branch from" &&
+
cp file2 file9 &&
git add file9 &&
git commit -a -m "Copy file2 to file9" &&
- git diff-tree -r -C HEAD
+ git diff-tree -r -C HEAD &&
git config git-p4.detectCopies true &&
"$GITP4" submit &&
p4 filelog //depot/file9 &&
! p4 filelog //depot/file9 | grep -q "branch from //depot/file" &&
- echo file2 >> file2 &&
+
+ echo "file2" >>file2 &&
cp file2 file10 &&
git add file2 file10 &&
git commit -a -m "Modify and copy file2 to file10" &&
- git diff-tree -r -C HEAD
+ git diff-tree -r -C HEAD &&
"$GITP4" submit &&
p4 filelog //depot/file10 &&
p4 filelog //depot/file10 | grep -q "branch from //depot/file" &&
+
cp file2 file11 &&
git add file11 &&
git commit -a -m "Copy file2 to file11" &&
- git diff-tree -r -C --find-copies-harder HEAD
+ git diff-tree -r -C --find-copies-harder HEAD &&
+ src=$(git diff-tree -r -C --find-copies-harder HEAD | sed 1d | cut -f2) &&
+ test "$src" = file10 &&
git config git-p4.detectCopiesHarder true &&
"$GITP4" submit &&
p4 filelog //depot/file11 &&
p4 filelog //depot/file11 | grep -q "branch from //depot/file" &&
+
cp file2 file12 &&
- echo >> file12 &&
+ echo "some text" >>file12 &&
git add file12 &&
git commit -a -m "Copy file2 to file12 with changes" &&
- git diff-tree -r -C --find-copies-harder HEAD
- git config git-p4.detectCopies 98 &&
+ git diff-tree -r -C --find-copies-harder HEAD &&
+ level=$(git diff-tree -r -C --find-copies-harder HEAD | sed 1d | cut -f1 | cut -d" " -f5 | sed "s/C0*//") &&
+ test -n "$level" && test "$level" -gt 0 && test "$level" -lt 98 &&
+ src=$(git diff-tree -r -C --find-copies-harder HEAD | sed 1d | cut -f2) &&
+ test "$src" = file10 &&
+ git config git-p4.detectCopies $((level + 2)) &&
"$GITP4" submit &&
p4 filelog //depot/file12 &&
! p4 filelog //depot/file12 | grep -q "branch from //depot/file" &&
+
cp file2 file13 &&
- echo >> file13 &&
+ echo "different text" >>file13 &&
git add file13 &&
git commit -a -m "Copy file2 to file13 with changes" &&
- git diff-tree -r -C --find-copies-harder HEAD
- git config git-p4.detectCopies 80 &&
+ git diff-tree -r -C --find-copies-harder HEAD &&
+ level=$(git diff-tree -r -C --find-copies-harder HEAD | sed 1d | cut -f1 | cut -d" " -f5 | sed "s/C0*//") &&
+ test -n "$level" && test "$level" -gt 2 && test "$level" -lt 100 &&
+ src=$(git diff-tree -r -C --find-copies-harder HEAD | sed 1d | cut -f2) &&
+ test "$src" = file10 &&
+ git config git-p4.detectCopies $((level - 2)) &&
"$GITP4" submit &&
p4 filelog //depot/file13 &&
- p4 filelog //depot/file13 | grep -q "branch from //depot/file" &&
- cd "$TRASH_DIRECTORY" &&
- rm -rf "$git" && mkdir "$git"
+ p4 filelog //depot/file13 | grep -q "branch from //depot/file"
'
# Create a simple branch structure in P4 depot to check if it is correctly
--
1.7.5.4
^ permalink raw reply related
* [PATCH 2/2] fast-import: treat cat-blob as a delta base hint for next blob
From: Dmitry Ivankov @ 2011-08-20 19:04 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
In-Reply-To: <1313867052-11993-1-git-send-email-divanorama@gmail.com>
Delta base for blobs is chosen as a previously saved blob. If we
treat cat-blob's blob as a delta base for the next blob, nothing
is likely to become worse.
For fast-import stream producer like svn-fe cat-blob is used like
following:
- svn-fe reads file delta in svn format
- to apply it, svn-fe asks cat-blob 'svn delta base'
- applies 'svn delta' to the response
- produces a blob command to store the result
Currently there is no way for svn-fe to give fast-import a hint on
object delta base. While what's requested in cat-blob is most of
the time a best delta base possible. Of course, it could be not a
good delta base, but we don't know any better one anyway.
So do treat cat-blob's result as a delta base for next blob. The
profit is nice: 2x to 7x reduction in pack size AND 1.2x to 3x
time speedup due to diff_delta being faster on good deltas. git gc
--aggressive can compress it even more, by 10% to 70%, utilizing
more cpu time, real time and 3 cpu cores.
Tested on 213M and 2.7G fast-import streams, resulting packs are 22M
and 113M, import time is 7s and 60s, both streams are produced by
svn-fe, sniffed and then used as raw input for fast-import.
For git-fast-export produced streams there is no change as it doesn't
use cat-blob and doesn't try to reorder blobs in some smart way to
make successive deltas small.
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 2b069e3..0480fbf 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -2802,7 +2802,12 @@ static void cat_blob(struct object_entry *oe, unsigned char sha1[20])
strbuf_release(&line);
cat_blob_write(buf, size);
cat_blob_write("\n", 1);
- free(buf);
+ if (oe && oe->pack_id == pack_id) {
+ last_blob.offset = oe->idx.offset;
+ strbuf_attach(&last_blob.data, buf, size, size);
+ last_blob.depth = oe->depth;
+ } else
+ free(buf);
}
static void parse_cat_blob(void)
--
1.7.3.4
^ permalink raw reply related
* [PATCH 0/2] fast-import: improve deltas for blobs
From: Dmitry Ivankov @ 2011-08-20 19:04 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
Currently delta base for blob objects is just a previous blob object
written. This way we just keep the last one in memory and it's cheap
(not too smart though and gains no pack size reduction most of the time).
If we also keep as last blob a response to cat-blob (whose main purpose
is to provide delta bases for a importer), svn-fe imports become faster
and packs produced become smaller.
1/2 adds a diff_delta attemps count as a related and interesting number
2/2 gives a nice performance improvement for svn-fe produced imports
Dmitry Ivankov (2):
fast-import: count and report # of calls to diff_delta in stats
fast-import: treat cat-blob as a delta base hint for next blob
fast-import.c | 17 ++++++++++++-----
1 files changed, 12 insertions(+), 5 deletions(-)
--
1.7.3.4
^ permalink raw reply
* [PATCH 1/2] fast-import: count and report # of calls to diff_delta in stats
From: Dmitry Ivankov @ 2011-08-20 19:04 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
In-Reply-To: <1313867052-11993-1-git-send-email-divanorama@gmail.com>
It's an interesting number, how often do we try to deltify each type of
objects and how often do we succeed. So do add it to stats.
Success doesn't mean much gain in pack size though. As we allow delta to
be as big as (data.len - 20). And delta close to data.len gains nothing
compared to no delta at all even after zlib compression (delta is pretty
much the same as data, just with few modifications).
We should try to make less attempts that result in huge deltas as these
consume more cpu than trivial small deltas. Either by choosing a better
delta base or reducing delta size upper bound or doing less delta attempts
at all.
Currently, delta base for blobs is a waste literally. Each blob delta
base is chosen as a previously stored blob. Disabling deltas for blobs
doesn't increase pack size and reduce import time, or at least doesn't
increase time for all fast-import streams I've tried.
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 7cc2262..2b069e3 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -284,6 +284,7 @@ static uintmax_t marks_set_count;
static uintmax_t object_count_by_type[1 << TYPE_BITS];
static uintmax_t duplicate_count_by_type[1 << TYPE_BITS];
static uintmax_t delta_count_by_type[1 << TYPE_BITS];
+static uintmax_t delta_count_attempts_by_type[1 << TYPE_BITS];
static unsigned long object_count;
static unsigned long branch_count;
static unsigned long branch_load_count;
@@ -1045,6 +1046,7 @@ static int store_object(
}
if (last && last->data.buf && last->depth < max_depth && dat->len > 20) {
+ delta_count_attempts_by_type[type]++;
delta = diff_delta(last->data.buf, last->data.len,
dat->buf, dat->len,
&deltalen, dat->len - 20);
@@ -3338,10 +3340,10 @@ int main(int argc, const char **argv)
fprintf(stderr, "---------------------------------------------------------------------\n");
fprintf(stderr, "Alloc'd objects: %10" PRIuMAX "\n", alloc_count);
fprintf(stderr, "Total objects: %10" PRIuMAX " (%10" PRIuMAX " duplicates )\n", total_count, duplicate_count);
- fprintf(stderr, " blobs : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas)\n", object_count_by_type[OBJ_BLOB], duplicate_count_by_type[OBJ_BLOB], delta_count_by_type[OBJ_BLOB]);
- fprintf(stderr, " trees : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas)\n", object_count_by_type[OBJ_TREE], duplicate_count_by_type[OBJ_TREE], delta_count_by_type[OBJ_TREE]);
- fprintf(stderr, " commits: %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas)\n", object_count_by_type[OBJ_COMMIT], duplicate_count_by_type[OBJ_COMMIT], delta_count_by_type[OBJ_COMMIT]);
- fprintf(stderr, " tags : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas)\n", object_count_by_type[OBJ_TAG], duplicate_count_by_type[OBJ_TAG], delta_count_by_type[OBJ_TAG]);
+ fprintf(stderr, " blobs : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas of %10" PRIuMAX" attempts)\n", object_count_by_type[OBJ_BLOB], duplicate_count_by_type[OBJ_BLOB], delta_count_by_type[OBJ_BLOB], delta_count_attempts_by_type[OBJ_BLOB]);
+ fprintf(stderr, " trees : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas of %10" PRIuMAX" attempts)\n", object_count_by_type[OBJ_TREE], duplicate_count_by_type[OBJ_TREE], delta_count_by_type[OBJ_TREE], delta_count_attempts_by_type[OBJ_TREE]);
+ fprintf(stderr, " commits: %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas of %10" PRIuMAX" attempts)\n", object_count_by_type[OBJ_COMMIT], duplicate_count_by_type[OBJ_COMMIT], delta_count_by_type[OBJ_COMMIT], delta_count_attempts_by_type[OBJ_COMMIT]);
+ fprintf(stderr, " tags : %10" PRIuMAX " (%10" PRIuMAX " duplicates %10" PRIuMAX " deltas of %10" PRIuMAX" attempts)\n", object_count_by_type[OBJ_TAG], duplicate_count_by_type[OBJ_TAG], delta_count_by_type[OBJ_TAG], delta_count_attempts_by_type[OBJ_TAG]);
fprintf(stderr, "Total branches: %10lu (%10lu loads )\n", branch_count, branch_load_count);
fprintf(stderr, " marks: %10" PRIuMAX " (%10" PRIuMAX " unique )\n", (((uintmax_t)1) << marks->shift) * 1024, marks_set_count);
fprintf(stderr, " atoms: %10u\n", atom_cnt);
--
1.7.3.4
^ permalink raw reply related
* Re: Failure cloning from a separate-git-dir repository
From: Richard Tibbitt @ 2011-08-20 18:56 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: git
In-Reply-To: <CACsJy8AavETJb-Lm0Qd6Sz54zYbgvDUWmf+uXYTxO3+iYNHvoA@mail.gmail.com>
On Sun, 21 Aug 2011 01:25:36 +0700, Nguyen Thai Ngoc Duy wrote:
>This should be "git clone repo2/.git clone2", but with that command I got
>
>Cloning into clone2..
>error: '/tmp/repo2/.git' does not look like a v2 bundle file
>fatal: Could not read bundle '/tmp/repo2/.git'.
>
>You just do "git clone repo2git/.git clone2" for now.
Hi - thanks for trying that
Unfortunately I'm getting :-
call git clone repo2git\.git clone2
fatal: repository 'repo2git\.git' does not exist
& similarly
call git clone repo2\.git clone2
fatal: repository 'repo2\.git' does not exist
I guess this must be a peculiarity of msysgit on windows.
^ permalink raw reply
* Re: [PATCH v2] git svn : hook before 'git svn dcommit'
From: Eric Wong @ 2011-08-20 18:41 UTC (permalink / raw)
To: Frédéric Heitzmann; +Cc: git, Junio C Hamano
In-Reply-To: <CALeToSUUJ4egjU_Pduub4VC+CJ3b2YSvPFEfJikRMsm9w4Ad5w@mail.gmail.com>
Frédéric Heitzmann <frederic.heitzmann@gmail.com> wrote:
> 2011/8/17 Eric Wong <normalperson@yhbt.net>:
> >> 2011/8/17 Eric Wong <normalperson@yhbt.net> wrote:
> >> > Perhaps an interactive option for dcommit would be just as useful?
> >
> > 1 and 3 can both implemented, but I think 3 would be easier to
> > use/setup/standardize. I suspect it's also easier to train oneself to
> > always use "dcommit -i". Perhaps even default to interactive mode
> > like git-send-email does nowadays.
> >
> > Unfortunately interactive dcommit requires more effort to implement.
>
> It seems that proposal 3 is somehow equivalent to
> $ git rebase -i --onto remotes/trunk remotes/trunk
> ... check commits, maybe remove some of them, ...
> $ git svn dcommit
No, it would just prompt before making every commit (showing the log
message), like git-send-email can before sending every message.
> note : is the SVN remote branch always named "remote/trunk" ? If not,
> is there a way to guess its name ?
git svn dcommit --dry-run will print the URL from which you can
infer the remote branch. There might be an easier way, but I
can't remember... (I don't see SVN much nowadays, most projects
I've cared about migrated to git)
--
Eric Wong
^ permalink raw reply
* Re: git svn --stdlayout 's little quirk.
From: Eric Wong @ 2011-08-20 18:35 UTC (permalink / raw)
To: Hin-Tak Leung; +Cc: madduck, git
In-Reply-To: <1313859988.60143.YahooMailClassic@web29518.mail.ird.yahoo.com>
Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
> first of all, thanks for a great tool - use it daily!
You're welcome :>
> I think I found a small bug in git 1.7.6. Having "trunk" at the end of
> the url in combination of --stdlayout is wrong, but it looks like that
> git-svn tries to cope, but doesn't go try far enough:
>
> Doing this:
> ----------------
> git svn clone --stdlayout http://quick-settings.googlecode.com/svn/trunk/ android-quick-settings
--stdlayout expects the "root" path of the code you're interested
in (not necessarily the SVN repository root, but in this case they
could be the same).
Try the following instead:
git svn clone --stdlayout \
http://quick-settings.googlecode.com/svn android-quick-settings
--
Eric Wong
^ permalink raw reply
* Re: [PATCH v4] fast-import: do not write bad delta for replaced subtrees
From: Dmitry Ivankov @ 2011-08-20 18:28 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: git, Andreas Schwab, Shawn O. Pearce, David Barr
In-Reply-To: <20110820174812.GD15864@elie.gateway.2wire.net>
On Sat, Aug 20, 2011 at 11:48 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Dmitry Ivankov wrote:
>
>> How about adding a new bit field "no_delta" instead?
>
> Currently the layout of "struct tree_entry_ms" is:
>
> uint16_t mode; two bytes
> unsigned char sha1[20] 20 bytes
>
> which adds up to 22 bytes. Here is "struct tree_entry":
>
> struct tree_entry *tree; one machine word
> struct atom_str *name; one machine word
> struct tree_entry_ms versions[2]; 44 bytes
>
> Although it only looks like it adds one byte per tree entry, in
> practice I suspect your patch adds four. Is that worth it? (The
> answer might be yes. I'm not sure.)
Hm, we can make mode 1 byte in fast-import (only 8 values are used).
But not in this patch of course.
>
>> The patch is
>> smaller this way. Also could 04000 theoretically be S_IFDIR on some
>> platform?
>
> No, these modes are part of the format of objects as written on disk
> and over the wire, so when we meet a platform with S_IFDIR != 040000,
> there will have to be bigger changes (to distinguish between the
> platform's idea of file status and git's idea of modes).
>
>> - switch to a separate no_delta bit in tree_entry
>
> If it doesn't cost too much, this is a good idea.
>
>> - when setting no_delta = 1 don't check for S_ISDIR(versions[0].mode),
>> this is a redundant check and logic duplication. Who knows, maybe some
>> day we'll want to delta a tree against blob. :)
>
> Why? When versions[0] is not a tree, the hack is not needed, since
> versions[0].mode and versions[0].sha1 accurately describe the delta
> base and are not inconsistent with anything.
Oh, right you are. The new check and one in store_tree look the same
but the purpose differs.
>
> Thanks, that was helpful.
>
^ permalink raw reply
* Re: Failure cloning from a separate-git-dir repository
From: Nguyen Thai Ngoc Duy @ 2011-08-20 18:25 UTC (permalink / raw)
To: Richard Tibbitt; +Cc: git
In-Reply-To: <6csv47te4qv41ni53q0kf1fne0a4e1grnm@4ax.com>
On Sun, Aug 21, 2011 at 12:47 AM, Richard Tibbitt <gitlist@lazyprune.com> wrote:
> F:\>call git clone repo2 clone2
> Cloning into clone2...
> fatal: failed to open 'F:/repo2/objects': No such file or directory
This should be "git clone repo2/.git clone2", but with that command I got
Cloning into clone2..
error: '/tmp/repo2/.git' does not look like a v2 bundle file
fatal: Could not read bundle '/tmp/repo2/.git'.
You just do "git clone repo2git/.git clone2" for now.
--
Duy
^ permalink raw reply
* Failure cloning from a separate-git-dir repository
From: Richard Tibbitt @ 2011-08-20 17:47 UTC (permalink / raw)
To: git
Hi
I've been struggling to find an answer to this for days.
Can anyone confirm whether I've got the syntax right for 'Git clone'
when attempting to clone FROM a repository that was initialized using
--separate-git-dir ?
If so, does this work on Linux systems ?
With Git on windows (git version 1.7.6.msysgit.0 installed from
GitExtensions224SetupComplete.msi) I get an error as below, that looks
as though 'Git clone' is expecting the repo to have a 'normal'
integral .git dir
F:\>mkdir repo2
F:\>mkdir repo2git
F:\>cd repo2
F:\repo2>call git init --separate-git-dir="..\repo2git\.git"
Initialized empty Git repository in F:/repo2git/.git/
F:\repo2>echo abc >file1.txt
F:\repo2>call git add . -A
F:\repo2>call git commit -m "first"
[master (root-commit) c7e4766] first
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 file1.txt
F:\repo2>cd ..
F:\>call git clone repo2 clone2
Cloning into clone2...
fatal: failed to open 'F:/repo2/objects': No such file or directory
Many thanks in advance for any insight into this.
Richard
^ permalink raw reply
* Re: [PATCH v4] fast-import: do not write bad delta for replaced subtrees
From: Jonathan Nieder @ 2011-08-20 17:48 UTC (permalink / raw)
To: Dmitry Ivankov; +Cc: git, Andreas Schwab, Shawn O. Pearce, David Barr
In-Reply-To: <1313860946-1596-1-git-send-email-divanorama@gmail.com>
Dmitry Ivankov wrote:
> How about adding a new bit field "no_delta" instead?
Currently the layout of "struct tree_entry_ms" is:
uint16_t mode; two bytes
unsigned char sha1[20] 20 bytes
which adds up to 22 bytes. Here is "struct tree_entry":
struct tree_entry *tree; one machine word
struct atom_str *name; one machine word
struct tree_entry_ms versions[2]; 44 bytes
Although it only looks like it adds one byte per tree entry, in
practice I suspect your patch adds four. Is that worth it? (The
answer might be yes. I'm not sure.)
> The patch is
> smaller this way. Also could 04000 theoretically be S_IFDIR on some
> platform?
No, these modes are part of the format of objects as written on disk
and over the wire, so when we meet a platform with S_IFDIR != 040000,
there will have to be bigger changes (to distinguish between the
platform's idea of file status and git's idea of modes).
> - switch to a separate no_delta bit in tree_entry
If it doesn't cost too much, this is a good idea.
> - when setting no_delta = 1 don't check for S_ISDIR(versions[0].mode),
> this is a redundant check and logic duplication. Who knows, maybe some
> day we'll want to delta a tree against blob. :)
Why? When versions[0] is not a tree, the hack is not needed, since
versions[0].mode and versions[0].sha1 accurately describe the delta
base and are not inconsistent with anything.
Thanks, that was helpful.
^ permalink raw reply
* [PATCH v4] fast-import: do not write bad delta for replaced subtrees
From: Dmitry Ivankov @ 2011-08-20 17:22 UTC (permalink / raw)
To: git
Cc: Jonathan Nieder, Andreas Schwab, Shawn O. Pearce, David Barr,
Dmitry Ivankov
In-Reply-To: <20110820154356.GB15864@elie.gateway.2wire.net>
How about adding a new bit field "no_delta" instead? The patch is
smaller this way. Also could 04000 theoretically be S_IFDIR on some
platform?
Changes:
- switch to a separate no_delta bit in tree_entry, don't assume that it
is used only for trees. Could be useful for blobs too once their delta
base logic changes (now it's just delta against blob that was stored
just before the current one)
- when setting no_delta = 1 don't check for S_ISDIR(versions[0].mode),
this is a redundant check and logic duplication. Who knows, maybe some
day we'll want to delta a tree against blob. :)
- removed the asserts on mode, as now "mode" meaning isn't changed
- removed the duplicated signed-of-by :)
-- >8 --
Subject: fast-import: do not write bad delta for replaced subtrees
To produce deltas for tree objects, fast-import tracks two versions of
each tree entry - a base and the current version. The base version on
a tree stands both for a delta base of this tree, and for a entry
inside the delta base of the parent tree. So care needs to be taken to
keep them consistent.
Unfortunately this all gets forgotten when replacing one subtree by
another using tree_content_set. When writing an entry representing
the new subtree, it keeps the old base sha1, since it is needed by the
parent tree. But the new tree doesn't have the implied base version
entries, and when it is time to write it to pack, git writes an
invalid delta that is declared to have one base (the old tree name)
but actually has another one (the new tree for an "M" command, or the
tree's old base for an "R" or "C" command).
How to fix it? Modifying the new subtree's entries to match the
declared base would be expensive, since it requires reading the tree
corresponding to the declared base from the object db and recursively
rewriting children's base versions to match. Invalidating the parent
trees' bases would involve recursively walking up the tree and
disables deltas for each tree it touches, meaning a larger pack.
Let's just mark the new tree as do-not-delta-me instead. Add a new bit
for tree_entry named no_delta. It is set to 1 when subtree is replaced
and reset back to 0 when we set a new legal delta base, that is when
e->versions[0] is changed.
tree_content_replace is in a similar predicament to tree_content_set,
except that because it is only used to replace the root, just
invalidating the base sha1 there (instead of setting the no-delta bit)
is fine.
Initial hack by Jonathan, test and description by Dmitry.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 27 ++++++++++++++++++++++++++-
t/t9300-fast-import.sh | 40 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 66 insertions(+), 1 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 7cc2262..3bae498 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -221,6 +221,7 @@ struct tree_entry {
uint16_t mode;
unsigned char sha1[20];
} versions[2];
+ unsigned no_delta : 1;
};
struct tree_content {
@@ -1368,6 +1369,7 @@ static void load_tree(struct tree_entry *root)
if (!c)
die("Corrupt mode in %s", sha1_to_hex(sha1));
e->versions[0].mode = e->versions[1].mode;
+ e->no_delta = 0;
e->name = to_atom(c, strlen(c));
c += e->name->str_len + 1;
hashcpy(e->versions[0].sha1, (unsigned char *)c);
@@ -1437,7 +1439,10 @@ static void store_tree(struct tree_entry *root)
store_tree(t->entries[i]);
}
- le = find_object(root->versions[0].sha1);
+ if (root->no_delta)
+ le = NULL;
+ else
+ le = find_object(root->versions[0].sha1);
if (S_ISDIR(root->versions[0].mode) && le && le->pack_id == pack_id) {
mktree(t, 0, &old_tree);
lo.data = old_tree;
@@ -1453,6 +1458,7 @@ static void store_tree(struct tree_entry *root)
struct tree_entry *e = t->entries[i];
if (e->versions[1].mode) {
e->versions[0].mode = e->versions[1].mode;
+ e->no_delta = 0;
hashcpy(e->versions[0].sha1, e->versions[1].sha1);
t->entries[j++] = e;
} else {
@@ -1471,6 +1477,7 @@ static void tree_content_replace(
{
if (!S_ISDIR(mode))
die("Root cannot be a non-directory");
+ hashclr(root->versions[0].sha1);
hashcpy(root->versions[1].sha1, sha1);
if (root->tree)
release_tree_content_recursive(root->tree);
@@ -1515,11 +1522,28 @@ static int tree_content_set(
if (e->tree)
release_tree_content_recursive(e->tree);
e->tree = subtree;
+
+ /*
+ * We need to leave e->versions[0].sha1 alone
+ * to avoid modifying the preimage tree used
+ * when writing out the parent directory.
+ * But after replacing the subdir with a
+ * completely different one, e->versions[0]
+ * is not a good delta base any more, and
+ * besides, we've thrown away the tree
+ * entries needed to make a delta against it.
+ *
+ * Let's just disable deltas when the time
+ * comes to write this subtree to pack.
+ */
+ e->no_delta = 1;
+
hashclr(root->versions[1].sha1);
return 1;
}
if (!S_ISDIR(e->versions[1].mode)) {
e->tree = new_tree_content(8);
+ e->no_delta = 1;
e->versions[1].mode = S_IFDIR;
}
if (!e->tree)
@@ -1537,6 +1561,7 @@ static int tree_content_set(
e = new_tree_entry();
e->name = to_atom(p, n);
e->versions[0].mode = 0;
+ e->no_delta = 0;
hashclr(e->versions[0].sha1);
t->entries[t->entry_count++] = e;
if (slash1) {
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f256475..04fa70d 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -734,6 +734,46 @@ test_expect_success \
git diff-tree --abbrev --raw L^ L >output &&
test_cmp expect output'
+cat >input <<INPUT_END
+blob
+mark :1
+data <<EOF
+the data
+EOF
+
+commit refs/heads/L2
+committer C O Mitter <committer@example.com> 1112912473 -0700
+data <<COMMIT
+init L2
+COMMIT
+M 644 :1 a/b/c
+M 644 :1 a/b/d
+M 644 :1 a/e/f
+
+commit refs/heads/L2
+committer C O Mitter <committer@example.com> 1112912473 -0700
+data <<COMMIT
+update L2
+COMMIT
+C a g
+C a/e g/b
+M 644 :1 g/b/h
+INPUT_END
+
+cat <<EOF >expect
+g/b/f
+g/b/h
+EOF
+
+test_expect_success \
+ 'L: modifying a copied tree does not produce a corrupt pack' \
+ 'test_when_finished "git update-ref -d refs/heads/L2" &&
+ git fast-import <input &&
+ git ls-tree L2 g/b/ >tmp &&
+ cut -f 2 <tmp >actual &&
+ test_cmp expect actual &&
+ git fsck L2'
+
###
### series M
###
--
1.7.3.4
^ permalink raw reply related
* git svn --stdlayout 's little quirk.
From: Hin-Tak Leung @ 2011-08-20 17:06 UTC (permalink / raw)
To: madduck, normalperson, git
first of all, thanks for a great tool - use it daily!
I think I found a small bug in git 1.7.6. Having "trunk" at the end of the url in combination of --stdlayout is wrong, but it looks like that git-svn tries to cope, but doesn't go try far enough:
Doing this:
----------------
git svn clone --stdlayout http://quick-settings.googlecode.com/svn/trunk/ android-quick-settings
----------------
Gives this message and stops:
----------------
Initialized empty Git repository in /home/Hin-Tak/tmp-git/svn-imports/android-quick-settings/.git/
Using higher level of URL: http://quick-settings.googlecode.com/svn/trunk => http://quick-settings.googlecode.com/svn
----------------
When I saw the message I thought it is striping the ending 'trunk' for my convenience, but when I look at .git/config:
--------------------
$ more .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[svn-remote "svn"]
url = http://quick-settings.googlecode.com/svn
fetch = trunk/trunk:refs/remotes/trunk
branches = trunk/branches/*:refs/remotes/*
tags = trunk/tags/*:refs/remotes/tags/*
------------------
It is doing 'trunk/trunk', etc, which is why it stopped.
Can it either clone correctly despite the wrong instruction, or fail with a better message than 'Using higher level...' which suggests it tried?
^ permalink raw reply
* Re: [PATCH v4 4/4] Accept tags in HEAD or MERGE_HEAD
From: Nguyen Thai Ngoc Duy @ 2011-08-20 16:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vaab52m2r.fsf@alter.siamese.dyndns.org>
2011/8/20 Junio C Hamano <gitster@pobox.com>:
> That's de-reference, not deference ;-). You may want to be more explicit
> about what kind of de-reference you are talking about.
>
> /*
> * Get a commit object for the given sha1, unwrapping a tag object that
> * point at a commit while at it. ref_name is only used when the result
> * is not a commit in the error message to report where we got the sha1
> * from.
> */
>
> I actually was hoping that you would have this comment in commit.h to help
> people who want to add callers of this function, not next to the
> implementation.
OK. It's because I tend to go straight to implementation instead of
the declaration when I want to know how to use it.
> As I said earlier, I do not think updating sha1[] here is necessary. The
> caller should be updated to use c->object.sha1 instead.
The usual pattern is
get_sha1(ref, sha1);
commit = lookup_commit_or_die(sha1, ref);
From a quick look, it's very easy to assume "sha1" is safe to use
afterwards while it may be different from commit->object.sha1. I'm
tempted to make lookup_commit_or_die() resolve ref to sha1 internally,
no temporary sha1 variable will be hanging around, the pattern becomes
commit = lookup_commit_or_die(ref);
The only problem is MERGE_HEAD is not usual ref and cannot be treated this way.
--
Duy
^ permalink raw reply
* Re: [PATCH v3] fast-import: do not write bad delta for replaced subtrees
From: Jonathan Nieder @ 2011-08-20 15:43 UTC (permalink / raw)
To: Andreas Schwab; +Cc: Dmitry Ivankov, git, Shawn O. Pearce, David Barr
In-Reply-To: <m262lspi0h.fsf@linux-m68k.org>
Andreas Schwab wrote:
> $ git grep S_ISUID
> compat/mingw.h:#define S_ISUID 0
Good catch, thanks. Fix below --- despite appearances, this "mode"
field is about tree objects in the repository and there is no need to
match the current platform's file status conventions.
Another worrisome detail is that the patch changes the semantics of
tree_entry.versions[0].mode (by introducing the NO_DELTA bit) without
changing its name. It would be safer to rename it to te_mode or
something.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
diff --git i/fast-import.c w/fast-import.c
index 95919b63..45e2128a 100644
--- i/fast-import.c
+++ w/fast-import.c
@@ -173,7 +173,7 @@ Format of STDIN stream:
/*
* We abuse the setuid bit on directories to mean "do not delta".
*/
-#define NO_DELTA S_ISUID
+#define NO_DELTA 04000
struct object_entry {
struct pack_idx_entry idx;
--
^ permalink raw reply related
* Re: [PATCH v2 1/4] git-p4: Allow setting rename/copy detection threshold.
From: Pete Wyckoff @ 2011-08-20 11:19 UTC (permalink / raw)
To: Vitor Antunes; +Cc: Junio C Hamano, git, Tor Arvid Lund
In-Reply-To: <CAOpHH-WJWxeTSO5Dy6UR9pJh3w9hQSBuE7O1mY2KDAEsS9OAPw@mail.gmail.com>
vitor.hda@gmail.com wrote on Fri, 19 Aug 2011 14:51 +0100:
> On Fri, Aug 19, 2011 at 12:47 PM, Pete Wyckoff <pw@padd.com> wrote:
> > I think you have to decide that 1 means 1 in this case.
> > Everything else can mean true. That may suggest that using
> > --bool or --bool-or-int isn't possible in this case.
>
> But doing that kind of post-processing would require me to call
> git-config (at least) twice: first to check if it is a number with a
> possible "." in the middle or "%" at the end and a second time with
> the --bool option. I have no problem in doing this, but I think it
> increases the complexity without bringing major advantages.
>
> I will use --bool for detectCopiesHarder and will send you the new set
> of patches tonight, unless we decide to also start using it for
> detectCopies and detectRenames.
I was imagining you would not use "--bool", and parse everything
yourself. But I see your point, and letting "1" mean "-M1" as
opposed to "-M" is probably okay too. I'm not particularly
worked up about which way that goes.
-- Pete
^ permalink raw reply
* Re: [PATCH] bash-completion: Make use of git status
From: Ron Panduwana @ 2011-08-20 12:46 UTC (permalink / raw)
To: Thomas Rast; +Cc: git, Shawn O. Pearce, Junio C Hamano, Lee Marlow
In-Reply-To: <201108191210.59538.trast@student.ethz.ch>
On Fri, Aug 19, 2011 at 5:10 PM, Thomas Rast <trast@student.ethz.ch> wrote:
> Some thoughts:
>
> * running git-status for . has some issues: it doesn't work in the
> case of
>
> cd subdir
> git add ../some/file[TAB]
>
> It's also inefficient if you are at the top level and
>
> git add path/to/file/a/few/levels/down[TAB]
>
> since it wouldn't actually have to look for untracked files in the
> entire repo.
OK I'll change it to git-status $cur*
> * -uall is not required unless you are looking for untracked files.
> For the other commands you could speed up completion by passing
> -uno instead.
Will add second param to __git_files_having_status
Ron Panduwana
--
panduwana@gmail.com
^ permalink raw reply
* Re: What's cooking in git.git (Aug 2011, #05; Fri, 19)
From: Nguyen Thai Ngoc Duy @ 2011-08-20 12:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeff King
In-Reply-To: <7vsjox11tr.fsf@alter.siamese.dyndns.org>
On Sat, Aug 20, 2011 at 5:19 AM, Junio C Hamano <gitster@pobox.com> wrote:
> * jk/add-i-hunk-filter (2011-07-27) 5 commits
> (merged to 'next' on 2011-08-11 at 8ff9a56)
> + add--interactive: add option to autosplit hunks
> + add--interactive: allow negatation of hunk filters
> + add--interactive: allow hunk filtering on command line
> + add--interactive: factor out regex error handling
> + add--interactive: refactor patch mode argument processing
>
> Needs documentation updates.
And tests, and its user "git add". Yeah, I'm getting to that.
Sorry it takes long time because I wanted to bring this feature to
diff machinery so that I could filter hunks with "git diff
--pickaxe-hunks -Gregex" and reuse option names for "git add -p". It
turns out very intrusive changes to split/merge hunks, so I dropped it
and will go back to simple "git add -p" changes.
--
Duy
^ permalink raw reply
* Re: [PATCH v4 1/4] commit: remove global variable head_sha1[]
From: Nguyen Thai Ngoc Duy @ 2011-08-20 12:03 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vfwkx2pqw.fsf@alter.siamese.dyndns.org>
2011/8/20 Junio C Hamano <gitster@pobox.com>:
> Also wouldn't these three be equivalents?
>
> head_commit == NULL
> is_null_sha1(head_sha1)
> initial_commit
Right.
> Perhaps like this instead?
Yup. Looks good.
--
Duy
^ permalink raw reply
* Re: How to use .gitattributes to tell git that .ini is text file?
From: Chris Packham @ 2011-08-20 9:59 UTC (permalink / raw)
To: jelly
Cc: "Git讨论组(无须订阅)"
In-Reply-To: <bfe44ee.e467.131e4f0b471.Coremail.sinojelly@163.com>
On 20/08/11 14:06, jelly wrote:
> When I use like this, it takes no effect.
> C:\Users\jelly\Documents\My Knowledge\Plugins>cat .gitattributes*.ini text
> C:\Users\jelly\Documents\My Knowledge\Plugins>git diffdiff --git a/Misc/plugin.ini b/Misc/plugin.iniindex 078c8a9..f73153c 100755Binary files a/Misc/plugin.ini and b/Misc/plugin.ini differ
I think you might be running into some problems with the end of line
detection. What git version are you running? I think what you want in
.gitattributes is '*.ini diff' (see below)
I did some quick tests with 1.7.5 and I seem to get sane behaviour
without needing .gitattributes
printf '\r\nfoo\r\n' > foo.ini
git commit -m"" foo.ini
# change content
printf '\r\nbar\r\n' > foo.ini
git diff
diff --git a/foo.ini b/foo.ini
index 35c61a9..b2b98e6 100644
--- a/foo.ini
+++ b/foo.ini
@@ -1,2 +1,2 @@
-foo
+bar
# insert non-printable character
printf '\nbar\0\n' > foo.ini
git diff
diff --git a/foo.ini b/foo.ini
index 35c61a9..aff5caf 100644
Binary files a/foo.ini and b/foo.ini differ
Even with the binary content I can tell git diff to treat the files as
text (not that you can see it in the email but with less as my pager I
actually get an indication that there is a funny character there)
git diff -a
diff --git a/foo.ini b/foo.ini
index 35c61a9..aff5caf 100644
--- a/foo.ini
+++ b/foo.ini
@@ -1,2 +1,2 @@
-
-foo
+
+bar
If you want to tell git diff to always use the -a option for .ini files
try this
echo '*.ini diff' > .gitattributes
git diff
diff --git a/foo.ini b/foo.ini
index 35c61a9..aff5caf 100644
--- a/foo.ini
+++ b/foo.ini
@@ -1,2 +1,2 @@
-
-foo
+
+bar
^ permalink raw reply
* Re: [PATCH v3] fast-import: do not write bad delta for replaced subtrees
From: Andreas Schwab @ 2011-08-20 9:08 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: Dmitry Ivankov, git, Shawn O. Pearce, David Barr
In-Reply-To: <20110820010901.GA2512@elie.sbx02827.chicail.wayport.net>
Jonathan Nieder <jrnieder@gmail.com> writes:
> Thanks again for this. Abusing (S_ISDIR | S_ISUID) still leaves a bad
> taste in my mouth, but after your description I'm convinced that
> behavior-wise it's the right thing to do.
$ git grep S_ISUID
compat/mingw.h:#define S_ISUID 0
Andreas.
--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox