* [PATCH 1/2] pretty: Add failing tests: user format ignores i18n.logOutputEncoding setting
From: Alexey Shumkin @ 2011-09-09 8:43 UTC (permalink / raw)
To: git; +Cc: Alexey Shumkin
In-Reply-To: <1315557806-5555-1-git-send-email-zapped@mail.ru>
The following two ought to give the same output to a terminal:
$ git log --oneline --no-color
$ git log --pretty=format:'%h %s'
However, the former pays attention to i18n.logOutputEncoding
configuration, while the latter does not when it format "%s". A log
messages written in an encoding i18n.commitEncoding that differs
from terminal encoding are shown corrupted with the latter even
when i18n.logOutputEncoding and terminal encoding are the same.
The same corruption is true for
$ git diff --submodule=log
and
$ git rev-list --pretty=format:%s HEAD
Signed-off-by: Alexey Shumkin <zapped@mail.ru>
---
t/t4041-diff-submodule-option.sh | 44 +++++----
t/t4205-log-pretty-formats.sh | 22 ++++-
t/t6006-rev-list-format.sh | 192 +++++++++++++++++++++++---------------
3 files changed, 161 insertions(+), 97 deletions(-)
diff --git a/t/t4041-diff-submodule-option.sh b/t/t4041-diff-submodule-option.sh
index bf9a752..4460cff 100755
--- a/t/t4041-diff-submodule-option.sh
+++ b/t/t4041-diff-submodule-option.sh
@@ -1,6 +1,7 @@
#!/bin/sh
#
# Copyright (c) 2009 Jens Lehmann, based on t7401 by Ping Yin
+# Copyright (c) 2011 Alexey Shumkin (+ non-UTF-8 commit encoding tests)
#
test_description='Support for verbose submodule differences in git diff
@@ -10,6 +11,7 @@ This test tries to verify the sanity of the --submodule option of git diff.
. ./test-lib.sh
+added=$(printf "\320\264\320\276\320\261\320\260\320\262\320\273\320\265\320\275")
add_file () {
sm=$1
shift
@@ -19,8 +21,12 @@ add_file () {
echo "$name" > "$name" &&
git add "$name" &&
test_tick &&
- git commit -m "Add $name"
+ git config i18n.commitEncoding cp1251 &&
+ echo "Add $name ($added $name)" \
+ | iconv -f utf-8 -t cp1251 \
+ | xargs -I{} git commit -m "{}"
done >/dev/null
+ git config --unset i18n.commitEncoding
git rev-parse --verify HEAD | cut -c1-7
cd "$owd"
}
@@ -37,7 +43,7 @@ head1=$(add_file sm1 foo1 foo2)
test_expect_success 'added submodule' "
git add sm1 &&
git diff-index -p --submodule=log HEAD >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 0000000...$head1 (new submodule)
EOF
test_cmp expected actual
@@ -48,27 +54,27 @@ head2=$(add_file sm1 foo3)
test_expect_success 'modified submodule(forward)' "
git diff-index -p --submodule=log HEAD >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 $head1..$head2:
- > Add foo3
+ > Add foo3 ($added foo3)
EOF
test_cmp expected actual
"
test_expect_success 'modified submodule(forward)' "
git diff --submodule=log >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 $head1..$head2:
- > Add foo3
+ > Add foo3 ($added foo3)
EOF
test_cmp expected actual
"
test_expect_success 'modified submodule(forward) --submodule' "
git diff --submodule >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 $head1..$head2:
- > Add foo3
+ > Add foo3 ($added foo3)
EOF
test_cmp expected actual
"
@@ -98,10 +104,10 @@ head3=$(
test_expect_success 'modified submodule(backward)' "
git diff-index -p --submodule=log HEAD >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 $head2..$head3 (rewind):
- < Add foo3
- < Add foo2
+ < Add foo3 ($added foo3)
+ < Add foo2 ($added foo2)
EOF
test_cmp expected actual
"
@@ -110,12 +116,12 @@ head4=$(add_file sm1 foo4 foo5) &&
head4_full=$(GIT_DIR=sm1/.git git rev-parse --verify HEAD)
test_expect_success 'modified submodule(backward and forward)' "
git diff-index -p --submodule=log HEAD >actual &&
- cat >expected <<-EOF &&
+ cat >expected <<EOF
Submodule sm1 $head2...$head4:
- > Add foo5
- > Add foo4
- < Add foo3
- < Add foo2
+ > Add foo5 ($added foo5)
+ > Add foo4 ($added foo4)
+ < Add foo3 ($added foo3)
+ < Add foo2 ($added foo2)
EOF
test_cmp expected actual
"
@@ -131,10 +137,10 @@ mv sm1-bak sm1
test_expect_success 'typechanged submodule(submodule->blob), --cached' "
git diff --submodule=log --cached >actual &&
cat >expected <<-EOF &&
-Submodule sm1 41fbea9...0000000 (submodule deleted)
+Submodule sm1 $head4...0000000 (submodule deleted)
diff --git a/sm1 b/sm1
new file mode 100644
-index 0000000..9da5fb8
+index 0000000..$head5
--- /dev/null
+++ b/sm1
@@ -0,0 +1 @@
@@ -148,7 +154,7 @@ test_expect_success 'typechanged submodule(submodule->blob)' "
cat >expected <<-EOF &&
diff --git a/sm1 b/sm1
deleted file mode 100644
-index 9da5fb8..0000000
+index $head5..0000000
--- a/sm1
+++ /dev/null
@@ -1 +0,0 @@
diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh
index 2ae9faa..96e0229 100755
--- a/t/t4205-log-pretty-formats.sh
+++ b/t/t4205-log-pretty-formats.sh
@@ -1,6 +1,7 @@
#!/bin/sh
#
# Copyright (c) 2010, Will Palmer
+# Copyright (c) 2011, Alexey Shumkin (+ non-UTF-8 commit encoding tests)
#
test_description='Test pretty formats'
@@ -11,10 +12,13 @@ test_expect_success 'set up basic repos' '
>bar &&
git add foo &&
test_tick &&
- git commit -m initial &&
+ git config i18n.commitEncoding cp1251 &&
+ printf "initial \320\272\320\276\320\274\320\274\320\270\321\202" \
+ | iconv -f utf-8 -t cp1251 | xargs -I{} git commit -m "{}" &&
git add bar &&
test_tick &&
- git commit -m "add bar"
+ git commit -m "add bar" &&
+ git config --unset i18n.commitEncoding
'
test_expect_success 'alias builtin format' '
@@ -38,6 +42,20 @@ test_expect_success 'alias user-defined format' '
test_cmp expected actual
'
+test_expect_success 'alias user-defined tformat with %s (cp1251 encoding)' '
+ git config i18n.logOutputEncoding cp1251 &&
+ git log --oneline >expected-s &&
+ git log --pretty="tformat:%h %s" >actual-s &&
+ git config --unset i18n.logOutputEncoding &&
+ test_cmp expected-s actual-s
+'
+
+test_expect_success 'alias user-defined tformat with %s (utf-8 encoding)' '
+ git log --oneline >expected-s &&
+ git log --pretty="tformat:%h %s" >actual-s &&
+ test_cmp expected-s actual-s
+'
+
test_expect_success 'alias user-defined tformat' '
git log --pretty="tformat:%h" >expected &&
git config pretty.test-alias "tformat:%h" &&
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index d918cc0..8d99635 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -1,13 +1,34 @@
#!/bin/sh
+# Copyright (c) 2009 Jens Lehmann
+# Copyright (c) 2011 Alexey Shumkin (+ non-UTF-8 commit encoding tests)
+
test_description='git rev-list --pretty=format test'
. ./test-lib.sh
test_tick
+added=$(printf "added (\320\264\320\276\320\261\320\260\320\262\320\273\320\265\320\275) foo")
+added_cp1251=$(echo "$added" | iconv -f utf-8 -t cp1251)
+changed=$(printf "changed (\320\264\320\276\320\261\320\260\320\262\320\273\320\265\320\275) foo")
+changed_cp1251=$(echo "$changed" | iconv -f utf-8 -t cp1251)
+
test_expect_success 'setup' '
-touch foo && git add foo && git commit -m "added foo" &&
- echo changed >foo && git commit -a -m "changed foo"
+ touch foo &&
+ git add foo &&
+ git config i18n.commitEncoding cp1251 &&
+ git commit -m "$added_cp1251" &&
+ head1=$(git rev-parse --verify HEAD) &&
+ head1_7=$(echo $head1 | cut -c1-7) &&
+ echo "$changed" > foo &&
+ git commit -a -m "$changed_cp1251" &&
+ head2=$(git rev-parse --verify HEAD) &&
+ head2_7=$(echo $head2 | cut -c1-7) &&
+ tree2=$(git cat-file -p HEAD | grep tree | cut -f 2 -d" ") &&
+ tree2_7=$(echo $tree2 | cut -c1-7) &&
+ head2_parent=$(git cat-file -p HEAD | grep parent | cut -f 2 -d" ") &&
+ head2_parent_7=$(echo $head2_parent | cut -c1-7) &&
+ git config --unset i18n.commitEncoding
'
# usage: test_format name format_string <expected_output
@@ -19,49 +40,49 @@ test_cmp expect.$1 output.$1
"
}
-test_format percent %%h <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
+test_format percent %%h <<EOF
+commit $head2
%h
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head1
%h
EOF
-test_format hash %H%n%h <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-131a310eb913d107dd3c09a65d1651175898735d
-131a310
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-86c75cf
+test_format hash %H%n%h <<EOF
+commit $head2
+$head2
+$head2_7
+commit $head1
+$head1
+$head1_7
EOF
-test_format tree %T%n%t <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-fe722612f26da5064c32ca3843aa154bdb0b08a0
-fe72261
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+test_format tree %T%n%t <<EOF
+commit $head2
+$tree2
+$tree2_7
+commit $head1
4d5fcadc293a348e88f777dc0920f11e7d71441c
4d5fcad
EOF
-test_format parents %P%n%p <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-86c75cf
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+test_format parents %P%n%p <<EOF
+commit $head2
+$head1
+$head2_parent_7
+commit $head1
EOF
# we don't test relative here
-test_format author %an%n%ae%n%ad%n%aD%n%at <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
+test_format author %an%n%ae%n%ad%n%aD%n%at <<EOF
+commit $head2
A U Thor
author@example.com
Thu Apr 7 15:13:13 2005 -0700
Thu, 7 Apr 2005 15:13:13 -0700
1112911993
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head1
A U Thor
author@example.com
Thu Apr 7 15:13:13 2005 -0700
@@ -69,14 +90,14 @@ Thu, 7 Apr 2005 15:13:13 -0700
1112911993
EOF
-test_format committer %cn%n%ce%n%cd%n%cD%n%ct <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
+test_format committer %cn%n%ce%n%cd%n%cD%n%ct <<EOF
+commit $head2
C O Mitter
committer@example.com
Thu Apr 7 15:13:13 2005 -0700
Thu, 7 Apr 2005 15:13:13 -0700
1112911993
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head1
C O Mitter
committer@example.com
Thu Apr 7 15:13:13 2005 -0700
@@ -84,86 +105,105 @@ Thu, 7 Apr 2005 15:13:13 -0700
1112911993
EOF
-test_format encoding %e <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+test_format encoding %e <<EOF
+commit $head2
+commit $head1
EOF
-test_format subject %s <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-changed foo
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-added foo
+expected=$(printf "commit $head2\n\
+$changed\n\
+commit $head1\n\
+$added
+")
+
+test_format subject %s <<EOF
+$expected
EOF
-test_format body %b <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+test_format body %b <<EOF
+commit $head2
+commit $head1
EOF
-test_format raw-body %B <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
-changed foo
+expected=$(printf "commit $head2\n\
+$changed\n\
+\n\
+commit $head1\n\
+$added
+")
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-added foo
+test_format raw-body %B <<EOF
+$expected
EOF
-test_format colors %Credfoo%Cgreenbar%Cbluebaz%Cresetxyzzy <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
+test_format colors %Credfoo%Cgreenbar%Cbluebaz%Cresetxyzzy <<EOF
+commit $head2
^[[31mfoo^[[32mbar^[[34mbaz^[[mxyzzy
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head1
^[[31mfoo^[[32mbar^[[34mbaz^[[mxyzzy
EOF
-test_format advanced-colors '%C(red yellow bold)foo%C(reset)' <<'EOF'
-commit 131a310eb913d107dd3c09a65d1651175898735d
+test_format advanced-colors '%C(red yellow bold)foo%C(reset)' <<EOF
+commit $head2
^[[1;31;43mfoo^[[m
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head1
^[[1;31;43mfoo^[[m
EOF
-cat >commit-msg <<'EOF'
+iconv -f utf-8 -t cp1251 > commit-msg <<EOF
Test printing of complex bodies
This commit message is much longer than the others,
-and it will be encoded in iso8859-1. We should therefore
-include an iso8859 character: ¡bueno!
+and it will be encoded in cp1251. We should therefore
+include an cp1251 character: так вот!
EOF
+
test_expect_success 'setup complex body' '
-git config i18n.commitencoding iso8859-1 &&
- echo change2 >foo && git commit -a -F commit-msg
+ git config i18n.commitencoding cp1251 &&
+ echo change2 >foo && git commit -a -F commit-msg &&
+ head3=$(git rev-parse --verify HEAD) &&
+ head3_7=$(echo $head3 | cut -c1-7)
'
-test_format complex-encoding %e <<'EOF'
-commit f58db70b055c5718631e5c61528b28b12090cdea
-iso8859-1
-commit 131a310eb913d107dd3c09a65d1651175898735d
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+test_format complex-encoding %e <<EOF
+commit $head3
+cp1251
+commit $head2
+cp1251
+commit $head1
+cp1251
EOF
-test_format complex-subject %s <<'EOF'
-commit f58db70b055c5718631e5c61528b28b12090cdea
-Test printing of complex bodies
-commit 131a310eb913d107dd3c09a65d1651175898735d
-changed foo
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
-added foo
+# unset commit encoding config
+# otherwise %e does not print encoding value
+# and following test fails
+git config --unset i18n.commitencoding
+
+expected=$(printf "commit $head3\n\
+Test printing of complex bodies\n\
+commit $head2\n\
+$changed\n\
+commit $head1\n\
+$added
+")
+
+test_format complex-subject %s <<EOF
+$expected
EOF
-test_format complex-body %b <<'EOF'
-commit f58db70b055c5718631e5c61528b28b12090cdea
+test_format complex-body %b <<EOF
+commit $head3
This commit message is much longer than the others,
-and it will be encoded in iso8859-1. We should therefore
-include an iso8859 character: ¡bueno!
+and it will be encoded in cp1251. We should therefore
+include an cp1251 character: так вот!
-commit 131a310eb913d107dd3c09a65d1651175898735d
-commit 86c75cfd708a0e5868dc876ed5b8bb66c80b4873
+commit $head2
+commit $head1
EOF
test_expect_success '%x00 shows NUL' '
- echo >expect commit f58db70b055c5718631e5c61528b28b12090cdea &&
+ echo >expect commit $head3 &&
echo >>expect fooQbar &&
git rev-list -1 --format=foo%x00bar HEAD >actual.nul &&
nul_to_q <actual.nul >actual &&
@@ -210,12 +250,12 @@ test_expect_success 'add LF before non-empty (2)' '
test_expect_success 'add SP before non-empty (1)' '
git show -s --pretty=format:"%s% bThanks" HEAD^^ >actual &&
- test $(wc -w <actual) = 2
+ test $(wc -w <actual) = 3
'
test_expect_success 'add SP before non-empty (2)' '
git show -s --pretty=format:"%s% sThanks" HEAD^^ >actual &&
- test $(wc -w <actual) = 4
+ test $(wc -w <actual) = 6
'
test_expect_success '--abbrev' '
--
1.7.6.1.g8f21c
^ permalink raw reply related
* [PATCH v2] pretty: user format ignores i18n.logOutputEncoding setting
From: Alexey Shumkin @ 2011-09-09 8:43 UTC (permalink / raw)
To: git
In-Reply-To: <7vwrf6qh49.fsf@alter.siamese.dyndns.org>
I discovered two more commands affected with this bug.
So I reroll this patch and tests for it.
>>(question) Does this change affect other commands, most notably
format-patch, and if so how?
No, it does not affect format-patc
---
builtin/rev-list.c | 1 +
builtin/shortlog.c | 1 +
log-tree.c | 1 +
submodule.c | 3 +
t/t4041-diff-submodule-option.sh | 44 +++++----
t/t4205-log-pretty-formats.sh | 22 ++++-
t/t6006-rev-list-format.sh | 192 +++++++++++++++++++++++---------------
7 files changed, 167 insertions(+), 97 deletions(-)
^ permalink raw reply
* Re: git repository size / compression
From: Carlos Martín Nieto @ 2011-09-09 8:23 UTC (permalink / raw)
To: neubyr; +Cc: git
In-Reply-To: <CALFxCvzVjC+u=RDkDCQp0QqPETsv8ROE8tY=37tmMWxmQoJOEw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]
On Thu, 2011-09-08 at 21:37 -0500, neubyr wrote:
> I have a test git repository with just two files in it. One of the
> file in it has a set of two lines that is repeated n times.
> e.g.:
> {{{
> $ for i in {1..5}; do cat ./lexico.txt >> lexico1.txt && cat
> ./lexico.txt >> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
> }}}
>
So you've just created some data that can be compressed quite
efficiently.
> I ran above command few times and performed commit after each run. Now
> disk usage of this repository directory is mentioned below. The 419M
> is working directory size and 2.7M is git repository/database size.
>
> {{{
> $ du -h -d 1 .
> 2.7M ./.git
> 419M .
>
> }}}
>
> Is it because of the compression performed by git before storing data
> (or before sending commit)??
>
Yes. Git stores its objects (the commit, the snapshot of the files,
etc.) compressed. When these objects are stored in a pack, the size can
be further reduced by storing some objects as deltas which describe the
difference between itself and some other object in the object-db.
> Following were results with subversion:
>
> Subversion client (redundant(?) copy exists in .svn/text-base/
> directory, hence double size in client):
> {{{
> $ du -h -d 1
> 416M ./.svn
> 832M .
> }}}
Subversion stores the "pristines" (which is the status of the files in
the latest revision) inside the .svn directory. I wouldn't call this
copy redundant, though, as it allows you to run diff locally. The
pristines are stored uncompressed, which is why you half of the space is
taken up by the .svn directory.
>
> Subversion repo/server:
> {{{
> $ du -h -d 1
> 12K ./conf
> 1.2M ./db
> 36K ./hooks
> 8.0K ./locks
> 1.2M .
> }}}
I don't know how the repository is stored in Subversion, but it may also
be compressed. You may be able to reduced your git repository size by
(re)generating packs with 'git repack' and doing some cleanups with 'git
gc', but the repository size is not often a concern.
cmn
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply
* Re: The imporantance of including http credential caching in 1.7.7
From: Miles Bader @ 2011-09-09 8:12 UTC (permalink / raw)
To: Michael J Gruber; +Cc: Jeff King, John Szakmeister, Kyle Neath, git
In-Reply-To: <4E69C8DC.7060008@drmicha.warpmail.net>
Michael J Gruber <git@drmicha.warpmail.net> writes:
>> Agreed. Anything harder than ssh keys is right out the window,
>> because they're always the alternative these people could be using
>> (but can't or don't want to).
>
> Sue, the question was: What is easy enough? I hoped that people
> would be using gpg to check signed tags, and that there might be a
> simple, convenient gnupg installer for Win and Mac which ties into
> the respective wallet systems or provides one they use already.
I wouldn't be surprised if many people just don't check signed tags at
all -- if the repositories they're using even have them in the first
place -- particularly amongst the audience in question.
-miles
--
What the fuck do white people have to be blue about!? Banana Republic ran
out of Khakis? The Espresso Machine is jammed? Hootie and The Blowfish
are breaking up??! Shit, white people oughtta understand, their job is to
GIVE people the blues, not to get them! -- George Carlin
^ permalink raw reply
* Re: The imporantance of including http credential caching in 1.7.7
From: Michael J Gruber @ 2011-09-09 8:06 UTC (permalink / raw)
To: Jeff King; +Cc: Kyle Neath, git
In-Reply-To: <20110908191053.GA16064@sigill.intra.peff.net>
Jeff King venit, vidit, dixit 08.09.2011 21:10:
> On Wed, Sep 07, 2011 at 02:56:03PM +0200, Michael J Gruber wrote:
>
>> So, it's been a year or more that you've been aware of the importance
>> of this issue (from your/github's perspective), and we hear about it
>> now, at the end of the rc phase. I don't know whether
>> jk/http-auth-keyring has been done on github payroll or during spare
>> time.
>
> To be absolutely clear here, this feature was 100% paid for by GitHub
> (which isn't to say that I don't think it's a good idea. On the
> contrary, I think it's awesome; but GitHub money is what provided the
> time for me to work on it).
>
> When I started at GitHub in January, I was given a giant list of things
> that GitHub felt would make core git better, but that they hadn't the
> personnel to improve. And I was told to use my own judgement in adding
> or removing items from the list based on what I thought git needed, and
> to prioritize as I saw fit. The fact that it took six months for me to
> come up with credential patches is because that's how long it took me to
> figure out what I wanted to write, and to clear my backlog of other git
> tasks.
>
> So I think the wheels have been turning on this for quite a while from
> GitHub's perspective.
Thanks for clarifying. While it should make no difference for the
acceptance of patches, it's great to see GitHub invest into scratching
their Git itches, and thus contribute back. That's how open source works
as a business model :)
...
> In the meantime, the best thing we can do to push it forward is to write
> helpers. I implemented some basic ones that should work anywhere, but
> aren't as nice as integration with existing keychains. Some people are
> working on Linux ones. The single best thing GitHub can do to push this
> forward right now is to provide a well-written OS X Keychain helper, and
> to provide feedback on whether git's end of the API is good enough.
... and one for Git on Windows? It seems we're lacking both Win and OS X
developers here.
... continuing in the other subthread...
^ permalink raw reply
* Re: The imporantance of including http credential caching in 1.7.7
From: Michael J Gruber @ 2011-09-09 8:05 UTC (permalink / raw)
To: Jeff King; +Cc: John Szakmeister, Kyle Neath, git
In-Reply-To: <20110908191842.GB16064@sigill.intra.peff.net>
Jeff King venit, vidit, dixit 08.09.2011 21:18:
> On Thu, Sep 08, 2011 at 11:02:11AM -0400, John Szakmeister wrote:
>
>> On Thu, Sep 8, 2011 at 9:17 AM, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>> [snip]
>>> It would be interesting to know what we can rely on in the user group
>>> you're thinking about (which I called ssh-challenged). Setting up ssh
>>> keys is too complicated. Can we require a working gpg setup? They do
>>> want to check sigs, don't they?
>>
>> I don't think you can require a working gpg setup (at least for not
>> addressing the ssh-challenged group).
>
> Agreed. Anything harder than ssh keys is right out the window, because
> they're always the alternative these people could be using (but can't or
> don't want to).
Sue, the question was: What is easy enough? I hoped that people would be
using gpg to check signed tags, and that there might be a simple,
convenient gnupg installer for Win and Mac which ties into the
respective wallet systems or provides one they use already.
> We could make our own gpg-based password wallet system, but I think it's
> a really bad idea, for two reasons:
>
> 1. It's reinventing the wheel. Which is bad enough as it is, but is
> doubly bad with security-related code, because it's very easy to
> screw something up when you're writing a lot of new code.
So please let's not deploy credential-store...
> 2. It's inconvenient for users. Nobody wants a separate wallet system
> with its own master password. They want to integrate with the
> wallet system they're already using. Which is generally going to be
> way nicer _anyway_, because it's going to be part of the OS and do
> helpful things like unlock the secret store using their login
> credentials.
On 1.+2.: The idea/hope was to use an existing wallet system which
people use for gnupg already to store their passphrase. If that is not
used then my suggestion does not help much (the issue of widespread
deployment), though it still is a secure version of credential-store for
those who want a desktop-independent secure credential store.
>>> So: What credential store/password wallet/etc. can we rely on for this
>>> group? Is gpg fair game?
>>
>> I think there probably need to be providers for using Keychain under
>> the Mac, gnome-keyring and kwallet under Linux, and probably something
>> using the wincrypt API under Windows. I don't think there's a
>> one-store-fits-all solution here, unfortunately. :-(
>
> Exactly. That's why the helpers communicate via pipes. They don't have
> to be included with core git at all; you should be able to just drop a
> third-party git-credential-foo into your PATH.
>
>> I'm actually tempted try and work on a couple of those myself.
>
> Please do! I mentioned a few people working on helpers elsewhere in this
> thread, so you may want to see what they've done and/or coordinate to
> avoid duplicate effort. Let me know if you have trouble finding the
> appropriate threads in the list archive.
It seemed appropriate to leverage GitHub for this:
https://github.com/gitigit/git/wiki/Git-Credentials-Hub
Feel free to add!
Cheers,
Michael
^ permalink raw reply
* Re: [PATCHv2 4/5] branch: introduce --list option
From: Michael J Gruber @ 2011-09-09 6:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git
In-Reply-To: <7vmxeewx7f.fsf@alter.siamese.dyndns.org>
Junio C Hamano venit, vidit, dixit 08.09.2011 23:17:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>
>>> Jeff King venit, vidit, dixit 07.09.2011 21:56:
>>> ...
>>>> It does make me a little nervous about the "'git branch -v'
>>>> automatically means 'git branch --list -v'" patch, though. It closes the
>>>> door in the future to us being more or less verbose about branch
>>>> creation details (and while helpful, it creates a slight inconsistency
>>>> in the interface).
>>
>> Hasn't 'git branch -v' meant listing in verbose mode for a long enough
>> time that changing it now would mean a moderately major regression?
>>
>> At least my copy of v1.7.0 seems to list with "git branch -v".
>
> Ah, nevermind.
>
> As the series is already in 'next', here is what I came up with.
I thought you'll rebuild next anyways after 1.7.7, but either way it's
fine. Thanks for holding this series in next long enough to really cook
it (and Jeff for revisiting it), it's much better now, keeping the
(undocumented, but expected) behavior of "git branch -v foo".
>
> -- >8 --
> From: Michael J Gruber <git@drmicha.warpmail.net>
> Date: Thu, 8 Sep 2011 14:09:50 -0700
> Subject: [PATCH] branch: -v does not automatically imply --list
>
> "branch -v" without other options or parameters still works in the list
> mode, but that is not because there is "-v" but because there is no
> parameter nor option.
>
> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> Documentation/git-branch.txt | 6 +++---
> builtin/branch.c | 3 +--
> t/t3203-branch-output.sh | 8 ++++++--
> 3 files changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/git-branch.txt b/Documentation/git-branch.txt
> index 2b8bc84..f46013c 100644
> --- a/Documentation/git-branch.txt
> +++ b/Documentation/git-branch.txt
> @@ -21,7 +21,7 @@ DESCRIPTION
> With no arguments, existing branches are listed and the current branch will
> be highlighted with an asterisk. Option `-r` causes the remote-tracking
> branches to be listed, and option `-a` shows both. This list mode is also
> -activated by the `--list` and `-v` options (see below).
> +activated by the `--list` option (see below).
> <pattern> restricts the output to matching branches, the pattern is a shell
> wildcard (i.e., matched using fnmatch(3))
> Multiple patterns may be given; if any of them matches, the tag is shown.
> @@ -120,10 +120,10 @@ OPTIONS
>
> -v::
> --verbose::
> - Show sha1 and commit subject line for each head, along with
> + When in list mode,
> + show sha1 and commit subject line for each head, along with
> relationship to upstream branch (if any). If given twice, print
> the name of the upstream branch, as well.
> - `--list` is implied by all verbosity options.
>
> --abbrev=<length>::
> Alter the sha1's minimum display length in the output listing.
> diff --git a/builtin/branch.c b/builtin/branch.c
> index 98a420f..099c75c 100644
> --- a/builtin/branch.c
> +++ b/builtin/branch.c
> @@ -712,8 +712,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
> argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
> 0);
>
> - if (!delete && !rename && !force_create &&
> - (argc == 0 || (verbose && argc)))
> + if (!delete && !rename && !force_create && argc == 0)
> list = 1;
>
> if (!!delete + !!rename + !!force_create + !!list > 1)
> diff --git a/t/t3203-branch-output.sh b/t/t3203-branch-output.sh
> index f2b294b..76fe7e0 100755
> --- a/t/t3203-branch-output.sh
> +++ b/t/t3203-branch-output.sh
> @@ -84,12 +84,16 @@ cat >expect <<'EOF'
> two
> one
> EOF
> -test_expect_success 'git branch -v pattern shows branch summaries' '
> - git branch -v branch* >tmp &&
> +test_expect_success 'git branch --list -v pattern shows branch summaries' '
> + git branch --list -v branch* >tmp &&
> awk "{print \$NF}" <tmp >actual &&
> test_cmp expect actual
> '
>
> +test_expect_success 'git branch -v pattern does not show branch summaries' '
> + test_must_fail git branch -v branch*
> +'
> +
> cat >expect <<'EOF'
> * (no branch)
> branch-one
^ permalink raw reply
* Re: git-rebase skips automatically no more needed commits
From: Francis Moreau @ 2011-09-09 6:43 UTC (permalink / raw)
To: Martin von Zweigbergk; +Cc: Junio C Hamano, Michael J Gruber, git
In-Reply-To: <alpine.DEB.2.00.1109082123300.12564@debian>
On Fri, Sep 9, 2011 at 4:23 AM, Martin von Zweigbergk
<martin.von.zweigbergk@gmail.com> wrote:
> On Thu, 8 Sep 2011, Francis Moreau wrote:
>
>> On Thu, Sep 8, 2011 at 3:14 PM, Martin von Zweigbergk
>> <martin.von.zweigbergk@gmail.com> wrote:
>> >
>> > Patches that are in both sides of v2.6.39...foo will be filtered, but
>>
>> what do you mean by "both sides" ?
>
> See the section called "SPECIFYING RANGES" in "git --help
> rev-parse". It doesn't define "both sides", but what I mean by that is
> "both in v2.6.39..foo and in foo..v2.6.39". I believe that section
> will answer most of the other questions too.
>
>> Yes my use of "git rebase --onto master foo~10 foo" is equivalent to
>> "git rebase master foo, the only difference is that the --onto variant
>> allow me to limit the range of commits that I want to rebase. So I
>> still want git rebase to do its the filtering process.
>
> Ok, that is different. I think you used v2.6.39 instead of foo~10
> previously. Assuming that v2.6.39 is the merge base of foo and master
> and that foo~10 is a later commit than v2.6.39,
That's what I meant.
> you are right that
> "git rebase --onto master foo~10 foo" could potentially filter out
> patches already in foo..master, without calculating patch-ids for all
> commits in master..foo for that matter. I think that would make sense
you meant master..foo~10, didn't you ?
> and as I said, it has been on my todo list for a long time. If
please let me know when you submitting your work, I'm interested to see it.
> necessary, we could have a flag to disable the filtering e.g. when the
> user knows that master is part of a completely separate history from
> foo.
Can't git figure this out itself ? (I'm not saying the switch is useless)
Thanks
--
Francis
^ permalink raw reply
* Re: Git Bug - diff in commit message.
From: Johannes Sixt @ 2011-09-09 6:14 UTC (permalink / raw)
To: Thomas Rast; +Cc: Michael Witten, anikey, git
In-Reply-To: <201109090256.36306.trast@student.ethz.ch>
Am 9/9/2011 2:56, schrieb Thomas Rast:
> Michael Witten wrote:
>> Perhaps `git rebase' should be reimplemented to use `git cherry-pick',
>> or does that suffer from the same problem?
>
> It doesn't, since it uses a three-way merge. But there's no real
> reason to reimplement anything; just use an interactive rebase, it
> uses cherry-pick in a loop.
But you can just as well use 'git rebase -m', which uses cherry-pick (or
something equivalent, dunno).
-- Hannes
^ permalink raw reply
* Re: git rebase fails with: Patch does not have a valid e-mail address.
From: Kyle Moffett @ 2011-09-09 4:45 UTC (permalink / raw)
To: James Blackburn; +Cc: git
In-Reply-To: <CACyv8dckmRxgb9_FDTW+=1Y9bS27b3XZQCHnhjpfHiJig5p6wg@mail.gmail.com>
On Thu, Sep 8, 2011 at 07:47, James Blackburn <jamesblackburn@gmail.com> wrote:
> I'm trying to rewrite some history and git's telling me:
>
> -bash:jamesb:lc-cam-025:33079> git rebase
> 7f58969b933745d4cb9bb128bbd3fa8d441cdb92
> First, rewinding head to replay your work on top of it...
> Patch does not have a valid e-mail address.
>
> Now it's true there isn't an email address for the author - the author
> no longer works for the company, and the email address was removed
> during the conversion. Therefore the repo contains "Author <>".
I can't really speak directly on the "rebase" issue, but...
You probably don't want to remove the email address from the repository
during a rewrite. When I was converting some old CVS repositories for
my company I very intentionally looked up all of our old user emails to
be able to convert them reliably (even though most of the addresses at
that point did not work). Even for the users where I could not find a
functional address I would just pick something reasonable based on the
username convention at the time.
In cases where you can't accurately attribute the commit (IE: username
of "root" or "cvs" or something), you probably want to rewrite it using an
internal mailing list address. For example, if the kernel had been in a
CVS repository with commits by "root", I would probably rewrite those
to be created by:
Linux Kernel Developers <lkml@vger.kernel.org>
Cheers,
Kyle Moffett
^ permalink raw reply
* git repository size / compression
From: neubyr @ 2011-09-09 2:37 UTC (permalink / raw)
To: git
I have a test git repository with just two files in it. One of the
file in it has a set of two lines that is repeated n times.
e.g.:
{{{
$ for i in {1..5}; do cat ./lexico.txt >> lexico1.txt && cat
./lexico.txt >> lexico1.txt && mv ./lexico1.txt ./lexico.txt; done
}}}
I ran above command few times and performed commit after each run. Now
disk usage of this repository directory is mentioned below. The 419M
is working directory size and 2.7M is git repository/database size.
{{{
$ du -h -d 1 .
2.7M ./.git
419M .
}}}
Is it because of the compression performed by git before storing data
(or before sending commit)??
Following were results with subversion:
Subversion client (redundant(?) copy exists in .svn/text-base/
directory, hence double size in client):
{{{
$ du -h -d 1
416M ./.svn
832M .
}}}
Subversion repo/server:
{{{
$ du -h -d 1
12K ./conf
1.2M ./db
36K ./hooks
8.0K ./locks
1.2M .
}}}
--
neuby.r
^ permalink raw reply
* Re: git-rebase skips automatically no more needed commits
From: Martin von Zweigbergk @ 2011-09-09 2:23 UTC (permalink / raw)
To: Francis Moreau
Cc: Martin von Zweigbergk, Junio C Hamano, Michael J Gruber, git
In-Reply-To: <CAC9WiBiMYUfaPtrXyW=W7qaZnJqLeFO-B3od7X4u8wUrb8hfUA@mail.gmail.com>
On Thu, 8 Sep 2011, Francis Moreau wrote:
> On Thu, Sep 8, 2011 at 3:14 PM, Martin von Zweigbergk
> <martin.von.zweigbergk@gmail.com> wrote:
> >
> > Patches that are in both sides of v2.6.39...foo will be filtered, but
>
> what do you mean by "both sides" ?
See the section called "SPECIFYING RANGES" in "git --help
rev-parse". It doesn't define "both sides", but what I mean by that is
"both in v2.6.39..foo and in foo..v2.6.39". I believe that section
will answer most of the other questions too.
> Yes my use of "git rebase --onto master foo~10 foo" is equivalent to
> "git rebase master foo, the only difference is that the --onto variant
> allow me to limit the range of commits that I want to rebase. So I
> still want git rebase to do its the filtering process.
Ok, that is different. I think you used v2.6.39 instead of foo~10
previously. Assuming that v2.6.39 is the merge base of foo and master
and that foo~10 is a later commit than v2.6.39, you are right that
"git rebase --onto master foo~10 foo" could potentially filter out
patches already in foo..master, without calculating patch-ids for all
commits in master..foo for that matter. I think that would make sense
and as I said, it has been on my todo list for a long time. If
necessary, we could have a flag to disable the filtering e.g. when the
user knows that master is part of a completely separate history from
foo.
Martin
^ permalink raw reply
* Re: [PATCH] send-mail: Add option to sleep between sending each email.
From: mfwitten @ 2011-09-09 2:12 UTC (permalink / raw)
To: Junio C Hamano
Cc: Matthieu Moy, Ramkumar Ramachandra, Georgi Chorbadzhiyski, git
In-Reply-To: <7vbouvx8j5.fsf@alter.siamese.dyndns.org>
On Thu, 08 Sep 2011 10:12:46 -0700, Junio C Hamano wrote:
> Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> writes:
>
>> There have been discussion (and IIRC a patch) proposing this already in
>> the past. One advantage of sleeping a bit between each email is that it
>> increase the chances for the receiver to receive the emails in the right
>> order.
>
> Huh? Even in the presense of MTAs in the middle that are free to reorder
> messages?
>
> IIRC, "git send-email" does its best to force ordering by assigning
> monotonically increasing timestamps on the Date: field, so that the
> recipients can sort the messages based on it, in addition to the
> In-Reply-To field to help threading. I personally do not think there is
> anything more than that that should done in the program.
The previous, rather lengthy discussion involved my patch and took place
over 2 years ago. The thread starts here:
Message-ID: <1239139522-24118-1-git-send-email-mfwitten@gmail.com?
http://thread.gmane.org/gmane.comp.version-control.git/115988
continues here (because of my email header mistake):
Message-ID: <49dcb464.06d7720a.66ca.ffffbd30@mx.google.com>
http://thread.gmane.org/gmane.comp.version-control.git/116083
and ultimately, the final patch review was proferred here:
Message-Id: <1239647037-15381-11-git-send-email-mfwitten@gmail.com>
http://article.gmane.org/gmane.comp.version-control.git/116471
>From a quick glance, my patch would appear to have become more advanced,
as per your own request, Junio:
Message-ID: <7vskkh1va5.fsf@gitster.siamese.dyndns.org>
http://thread.gmane.org/gmane.comp.version-control.git/116083
Here's the documentation I wrote for it:
diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index 5f7d640..236e578 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -178,6 +178,36 @@ Automating
cc list. Default is the value of 'sendemail.signedoffbycc' configuration
value; if that is unspecified, default to --signed-off-by-cc.
+--sleep=<seconds>[,<burst>]::
+ This option specfies that send-email should sleep for <seconds>
+ after sending <burst> messages as quickly as possible; <seconds>
+ should be an integer >= 0 and <burst> should be an integer >= 1.
+ This mode of operation attacks 2 problems: email throttling and
+ arrival disorder. Default is the value of the 'sendemail.sleep'
+ configuration variable, or '0' if that does not exist.
++
+By default, send-email tries to send one patch per email as quickly as
+possible. Unfortunately, some email services restrict a user by refusing
+to send more than some maximum number of email messages, M, in a given
+period of seconds, S. This can be troublesome if the patch series has
+more than M patches, because the server will ultimately refuse to send
+some of them. In this case, simply pass '--sleep=S,M' or '--sleep S,M'
+or set sendemail.sleep to 'S,M'.
++
+Moreover, the emails often arrive at the final destination out of order;
+though send-email manipulates the date fields and usually chains subsequent
+emails via the In-Reply-To headers, some mail viewers nevertheless insist
+on presenting them by order of arrival. This may be mitigated by using
+something like '--sleep 60' (the equivalent of '--sleep 60,1'), so that
+there is a 60 second delay between sending any two messages.
++
+*Note*: Because of varying routes and batching schemes, there is no delay
+that can guarantee the correct arrival order. Obviously, one solution is to
+choose an obscenely large number, so be prepared to run send-email in the
+background. Of course, spreading emails across time makes it more likely
+that unrelated email messages arrive between patches. Therefore, send-email
+warns you if both --sleep and --no-chain-reply-to are used.
+
--suppress-cc=<category>::
Specify an additional category of recipients to suppress the
auto-cc of:
Sincerely,
Michael Witten
^ permalink raw reply
* [PATCH 2/2] tree-walk: micro-optimization in tree_entry_interesting
From: Dan McGee @ 2011-09-09 2:02 UTC (permalink / raw)
To: git
In-Reply-To: <1315533766-25901-1-git-send-email-dpmcgee@gmail.com>
In the case of a wide breadth top-level tree (~2400 entries, all trees
in this case), we can see a noticeable cost in the profiler calling
strncmp() here. Most of the time we are at the base level of the
repository, so base is "" and baselen == 0, which means we will always
test true. Break out this one tiny case so we can short circuit the
strncmp() call.
Test cases are as follows. packages.git is the Arch Linux git-svn clone
of the packages repository which has the characteristics above.
Commands:
[1] packages.git, /usr/bin/time git log >/dev/null
[2] packages.git, /usr/bin/time git log -- autogen/trunk pacman/trunk wget/trunk >/dev/null
[3] linux.git, /usr/bin/time git log >/dev/null
[4] linux.git, /usr/bin/time git log -- drivers/ata drivers/uio tools >/dev/null
Results:
before after %faster
[1] 2.56 2.55 0.4%
[2] 51.82 48.66 6.5%
[3] 5.58 5.61 -0.5%
[4] 1.55 1.51 0.2%
The takeaway here is this doesn't matter in many operations, but it does
for a certain style of repository and operation where it nets a 6.5%
measured improvement. The other changes are likely not significant by
reasonable statistics methods.
Note: the measured improvement when originally submitted was ~11% (43 to
38 secs) for operation [2]. At the time, the repository had 117220
commits; it now has 137537 commits.
Signed-off-by: Dan McGee <dpmcgee@gmail.com>
---
tree-walk.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tree-walk.c b/tree-walk.c
index dbcd94a..e401f07 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -591,8 +591,8 @@ int tree_entry_interesting(const struct name_entry *entry,
ps->max_depth);
}
- /* Does the base match? */
- if (!strncmp(base_str, match, baselen)) {
+ /* Either there must be no base, or the base must match. */
+ if (baselen == 0 || !strncmp(base_str, match, baselen)) {
if (match_entry(entry, pathlen,
match + baselen, matchlen - baselen,
&never_interesting))
--
1.7.6.1
^ permalink raw reply related
* [PATCH 1/2] tree-walk: drop unused parameter from match_dir_prefix
From: Dan McGee @ 2011-09-09 2:02 UTC (permalink / raw)
To: git
In-Reply-To: <7v62le1vlx.fsf@alter.siamese.dyndns.org>
Signed-off-by: Dan McGee <dpmcgee@gmail.com>
---
tree-walk.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tree-walk.c b/tree-walk.c
index 33f749e..dbcd94a 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -522,7 +522,7 @@ static int match_entry(const struct name_entry *entry, int pathlen,
return 0;
}
-static int match_dir_prefix(const char *base, int baselen,
+static int match_dir_prefix(const char *base,
const char *match, int matchlen)
{
if (strncmp(base, match, matchlen))
@@ -579,7 +579,7 @@ int tree_entry_interesting(const struct name_entry *entry,
if (baselen >= matchlen) {
/* If it doesn't match, move along... */
- if (!match_dir_prefix(base_str, baselen, match, matchlen))
+ if (!match_dir_prefix(base_str, match, matchlen))
goto match_wildcards;
if (!ps->recursive || ps->max_depth == -1)
--
1.7.6.1
^ permalink raw reply related
* [PATCH] Disable useless check_for_new_submodule_commits() by default
From: Junio C Hamano @ 2011-09-09 1:56 UTC (permalink / raw)
To: git; +Cc: Jens Lehmann, Heiko Voigt
Imagine a project with 20k paths with a history 250k commits deep, without
any submodule.
Imagine another project with the same depth of history but with 1k paths,
among which there are 200 submodules.
Further, imagine fetching from one of the above repositories into a
repository that is very behind, and updating many remote tracking
branches. Now, think about what check_for_new_submodule_commits() in
submodule.c that is called from update_local_ref() in builtin/fetch.c
would do.
For each updated remote tracking branch (or anything that is not a tag),
the problem function will run the equivalent of:
git log --raw $new --not --all
which would mean 250k rounds of diff-tree to enumerate the submodules that
may have been updated, and it does it for each and every refs outside the
refs/tags hierarchy.
Presumably, this is so that it only has to actually fetch in the submodule
"on demand", but even if in a project _with_ submodules (i.e. the latter
example above), this is simply not acceptable. You could just enumerate
those 200 submodules at the tip that _might_ matter and that would be
million times cheaper. To add insult to injury, this "on demand" behaviour
is on by default, which hurts projects without any submodules (i.e. the
former example above) a lot.
In short, if "on demand" check is million times more expensive than
actually doing it, the check does not have any value.
In the longer term, people who want to have the on-demand behaviour need
to come up with a cheaper way to determine if it is necessary to recurse
into submodules by fixing check_for_new_submodule_commits(), but until
that happens, we should disable submodule recursion by default unless the
user explicitly asks for it from the command line or from the configuration.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* This is meant for 1.7.6.X maintenance track to fix the huge regression
caused by 88a2197 (fetch/pull: recurse into submodules when necessary,
2011-03-06). I would very much appreciate a simpler and trivially
correct patch as a counter-proposal that makes sure that the expensive
check_for_new_submodule_commits() and fetch_populated_submodules()
functions are never called when the user does not have any command line
option or configuration variable to tell otherwise. I suspect that this
patch may be disabling more things than absolutely necessary.
I also suspect that it might be just a matter of finding any and all
comparison of the form:
if (recurse != ON && recurse != OFF)
that was present before the on-demand stuff and replace that with
if (recurse == ON_DEMAND)
but I didn't look very closely. At least this patch is much less error
prone for the purpose of not triggering the expensive and pointless
check ;-)
builtin/fetch.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 93c9938..71232c1 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -29,7 +29,7 @@ enum {
};
static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;
-static int progress, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
+static int progress, recurse_submodules = RECURSE_SUBMODULES_OFF;
static int tags = TAGS_DEFAULT;
static const char *depth;
static const char *upload_pack;
^ permalink raw reply related
* Re: [PATCH 2/2] push -s: skeleton
From: Robin H. Johnson @ 2011-09-09 1:30 UTC (permalink / raw)
To: Jeff King, Git Mailing List; +Cc: Robin H. Johnson, Shawn O. Pearce
In-Reply-To: <20110908200343.GD16064@sigill.intra.peff.net>
On Thu, Sep 08, 2011 at 04:03:43PM -0400, Jeff King wrote:
> On Wed, Sep 07, 2011 at 11:55:44PM +0000, Robin H. Johnson wrote:
> > There's a couple of related things we've been considering on the Gentoo
> > side:
> > - detached signatures of blobs (either the SHA1 of the blob or the blob
> > itself)
> There's not much point in signing the blob itself and not the sha1; the
> first thing any signature algorithm will do is make a fixed-size digest
> of the content anyway. So it is only useful if you don't trust sha1 as
> your digest algorithm (which maybe is a reasonable concern these
> days...).
To clarify here, there's two things gained by this over signing the
SHA1:
1. Ability to use a different digest algorithm right now, without
having to trust SHA1.
2. Ability to use HMAC.
I agree that both of these are bandaids to the security of SHA*.
> > - The signature covering the message+blob details, but NOT the chain of
> > history: this opens up the ability to cherry-pick and rebase iff there
> > are no conflicts and the blobs are identical, all while preserving the
> > signature.
> The problem is that many of the blobs won't be identical, because
> they'll have new content from the new commits you rebased on top of. So
> _some_ blobs will be the same, but you'll end up with a half-signed
> commit. I think you're better to just re-sign the new history.
Possibly, the rebase/cherry-pick isn't critical.
> But I'd have to see a longer description of your scheme to really
> critique it. I'm not 100% sure what your security goals are here.
The primary goal is that a developer can certify their commits when they
are ready to push them, regardless of the means of how they are going to
push them (I've had requests for some git-bundle usage help in pushes).
> > - concerns about a pre-image attack against Git. tl;dr version:
> > 1. Attacker prepares decoy file in advance, that hashes to the same as
> > the malicious file.
> > 2. Attacker sends decoy in as an innocuous real commit.
> > 3. Months later, the attacker breaks into the system and alters the
> > packfile to include the new malicious file.
> > 4. All new clones from that point forward get the malicious version.
> Nit: I think you mean "collision attack" here. Pre-image attacks are
> matching a malicious file to what is already in the tree, but are much
> harder to execute.
My bad, it was really late when I was writing the above.
> But yeah, it is a potential problem. I don't keep up very well with that
> sort of news anymore, but AFAIK, we still don't have any actual
> collisions in sha1. Wikipedia seems to seem to think the best attacks
> are in the 2^50-ish range, but nobody has successfully found one. So we
> may still be a few years away from a realistic attack. If the attacks
> are anything like the MD5 attacks, the decoy and malicious files will
> need to have a bunch of random garbage in them. Which may be hard to
> disguise, depending on your repo contents.
Joey Hess discussed this two years ago, and again last week:
http://kitenet.net/~joey/blog/entry/size_of_the_git_sha1_collision_attack_surface/
http://kitenet.net/~joey/blog/entry/sha-1/
This is easy in the kernel tree, it's got lots of eyeballs and only few
binary files. This isn't true for lots of other Git trees, a tree with a
JPEG image or a gzip file would be a great target.
It's 2^53 last I saw, which is only ~250 days of cputime on my i7
desktop. Also notably that is under the complexity of the 2^56 EFF DES
cracker was built do to (and the more recent COPACOBANA unit).
> I think, though, that the sane fix at that point is not to start trying
> to make per-blob signatures or anything like that, but to consider "git
> version 2" with SHA-256, or whatever ends up becoming SHA-3 next year.
> It would involve rewriting all of your history and dropping support for
> older git clients, of course, but it may be worth it at the point that
> sha1 is completely broken.
I think this is needed sooner rather than later.
> > Re your comment on always needing to resign commits above, we'd been
> > considering post-signing commits, not when they are initially made.
> > After your commit is clean and ready to ship, you can fire the commit
> > ids into the signature tool, which can generate a detached signature
> > note for each commit.
> Agreed. This is just an interface problem, not a cryptographic or
> technical one. However, I do think there's a subtle difference between
> the two ideas. Signing each commit individually just indicates some
> approval of particular commits. But signing a push certificate is about
> associating particular commits with particular refs (e.g., saying "move
> 'master' from commit X to commit Y). I think there may be uses for both
> forms.
Yes, there is use for both forms. The additional one is that you can
separate who pushes from who commits.
--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
^ permalink raw reply
* Re: Git is not scalable with too many refs/*
From: Thomas Rast @ 2011-09-09 1:13 UTC (permalink / raw)
To: Martin Fick, Jens Lehmann; +Cc: git
In-Reply-To: <201109090305.15896.trast@student.ethz.ch>
Thomas Rast wrote:
> + const char *argv[] = {NULL, NULL, "--not", "--all", NULL};
> + int argc = ARRAY_SIZE(argv) - 1;
> +
> + init_revisions(&rev, NULL);
>
> which means that the --all needs to walk all commits reachable from
> all refs and flag them as uninteresting.
Scratch that, it "only" needs to mark every tip commit and then walk
them back to about where the interesting commits end.
In any case, since the uninteresting set only gets larger, it should
be possible to reuse the same revision walker.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Chances of getting gitk series merged
From: Martin von Zweigbergk @ 2011-09-09 1:08 UTC (permalink / raw)
To: Paul Mackerras; +Cc: git
Hi Paul,
I was happy to see in another thread that you are still here. If I
understand correctly, you are the maintainer of gitk, meaning that all
gitk patches go through you and then Junio pulls from you. I have this
series [1] that I sent in out in April that I would like to get
merged. Could you have a look at it?
Thanks,
Martin
[1] http://thread.gmane.org/gmane.comp.version-control.git/170853
^ permalink raw reply
* Re: [PATCHv2 4/5] branch: introduce --list option
From: Jeff King @ 2011-09-09 1:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Michael J Gruber, git
In-Reply-To: <7vmxeewx7f.fsf@alter.siamese.dyndns.org>
On Thu, Sep 08, 2011 at 02:17:24PM -0700, Junio C Hamano wrote:
> "branch -v" without other options or parameters still works in the list
> mode, but that is not because there is "-v" but because there is no
> parameter nor option.
> [...]
> +test_expect_success 'git branch -v pattern does not show branch summaries' '
> + test_must_fail git branch -v branch*
> +'
You might also want to affirm that it does not just fail, but still
creates a branch, like:
test_expect_success 'git branch -v <name> creates a branch' '
git rev-parse HEAD >expect &&
git branch -v foo &&
git rev-parse foo >actual &&
test_cmp expect actual
'
-Peff
^ permalink raw reply
* Re: Git is not scalable with too many refs/*
From: Thomas Rast @ 2011-09-09 1:05 UTC (permalink / raw)
To: Martin Fick, Jens Lehmann; +Cc: git
In-Reply-To: <1315529522448-6774328.post@n2.nabble.com>
Martin Fick wrote:
> An update, I bisected it down to this commit:
>
> 88a21979c5717e3f37b9691e90b6dbf2b94c751a
>
> fetch/pull: recurse into submodules when necessary
>
> Since this can be disabled with the --no-recurse-submodules switch, I tried
> that and indeed, even with the latest 1.7.7rc it becomes fast (~8mins)
> again. The strange part about this is that the repository does not have any
> submodules. Anyway, I hope that this can be useful to others since it is a
> workaround which speed things up enormously. Let me know if you have any
> other tests that you want me to perform,
Jens should know about this, so let's Cc him.
I took a quick look and I'm guessing that there's at least one
quadratic behaviour: in check_for_new_submodule_commits(), I see
+ const char *argv[] = {NULL, NULL, "--not", "--all", NULL};
+ int argc = ARRAY_SIZE(argv) - 1;
+
+ init_revisions(&rev, NULL);
which means that the --all needs to walk all commits reachable from
all refs and flag them as uninteresting. But that function is called
for every ref update, so IIUC the time spent is on the order of
#ref updates*#commits.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: Git Bug - diff in commit message.
From: Thomas Rast @ 2011-09-09 0:56 UTC (permalink / raw)
To: Michael Witten; +Cc: anikey, git
In-Reply-To: <CAMOZ1BtbpbG+19G6Hfau_2_F5L3Ad+x-Payd9aKajJxU_V_tyA@mail.gmail.com>
Michael Witten wrote:
> On Thu, Sep 8, 2011 at 14:49, anikey <arty.anikey@gmail.com> wrote:
> > Hi, peops. I'm pretty much sure that's a bug.
> >
> > What I did was putting git diff (i needed to tell people that for my changes
> > to start working they needed to aplly message-inline patch to some code
> > which was not under git) in commit message. Like adding:
>
> It would appear that `git rebase' is in fact producing patches with
> `git format-patch' and then applying the resulting patches with `git
> am', which gets confused by your inline diff; this can be clearly seen
> in the `git-rebase--am[.sh]' file.
>
> Perhaps `git rebase' should be reimplemented to use `git cherry-pick',
> or does that suffer from the same problem?
It doesn't, since it uses a three-way merge. But there's no real
reason to reimplement anything; just use an interactive rebase, it
uses cherry-pick in a loop.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* git-p4.skipSubmitEdit
From: L. A. Linden Levy @ 2011-09-08 20:40 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]
Hi All,
I have been using git-p4 for a while and it has allowed me to completely
change the way I develop and still be able to use perforce which my
company has for its main VCS. One thing that was driving me nuts was
that "git p4 submit" cycles through all of my individual commits and
asks me if I want to change them. The way I develop I often am checking
in 20 to 50 different small commits each with a descriptive git comment.
I felt like I was doing double duty by having emacs open on every commit
into perforce. So I modified git-p4 to have an option to skip the
editor. This option coupled with git-p4.skipSubmitEditCheck will make
the submission non-interactive for "git p4 submit".
Below are the patch and environment results:
$ git config -l
...
user.name=Loren A. Linden Levy
git-p4.skipsubmitedit=true
git-p4.skipsubmiteditcheck=true
...
$ git format-patch origin/master --stdout
From 16c4344de0047cbaf3381eca590a3e59b0d0a25c Mon Sep 17 00:00:00 2001
From: "Loren A. Linden Levy" <lindenle@gmail.com>
Date: Thu, 8 Sep 2011 13:37:22 -0700
Subject: [PATCH] changed git-p4
---
contrib/fast-import/git-p4 | 15 ++++++++++-----
1 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 2f7b270..a438b3e 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -935,18 +935,23 @@ class P4Submit(Command, P4UserMap):
tmpFile.write(submitTemplate + separatorLine + diff +
newdiff)
tmpFile.close()
mtime = os.stat(fileName).st_mtime
- if os.environ.has_key("P4EDITOR"):
- editor = os.environ.get("P4EDITOR")
+ if gitConfig("git-p4.skipSubmitEdit") == "true":
+ pass
else:
- editor = read_pipe("git var GIT_EDITOR").strip()
- system(editor + " " + fileName)
-
+ if os.environ.has_key("P4EDITOR"):
+ editor = os.environ.get("P4EDITOR")
+ else:
+ editor = read_pipe("git var GIT_EDITOR").strip()
+
+ system(editor + " " + fileName)
+
if gitConfig("git-p4.skipSubmitEditCheck") == "true":
checkModTime = False
else:
checkModTime = True
response = "y"
+
if checkModTime and (os.stat(fileName).st_mtime <= mtime):
response = "x"
while response != "y" and response != "n":
--
1.7.6.347.g4db0d
--
Alex Linden Levy
Senior Software Engineer
MobiTV, Inc.
6425 Christie Avenue, 5th Floor, Emeryville, CA 94608
phone 510.450.5190 mobile 720.352.8394
email alevy@mobitv.com web www.mobitv.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply related
* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-09 0:52 UTC (permalink / raw)
To: git
In-Reply-To: <1315511619144-6773496.post@n2.nabble.com>
An update, I bisected it down to this commit:
88a21979c5717e3f37b9691e90b6dbf2b94c751a
fetch/pull: recurse into submodules when necessary
Since this can be disabled with the --no-recurse-submodules switch, I tried
that and indeed, even with the latest 1.7.7rc it becomes fast (~8mins)
again. The strange part about this is that the repository does not have any
submodules. Anyway, I hope that this can be useful to others since it is a
workaround which speed things up enormously. Let me know if you have any
other tests that you want me to perform,
-Martin
--
View this message in context: http://git.661346.n2.nabble.com/Git-is-not-scalable-with-too-many-refs-tp6456443p6774328.html
Sent from the git mailing list archive at Nabble.com.
^ permalink raw reply
* Re: Git is not scalable with too many refs/*
From: Martin Fick @ 2011-09-08 19:53 UTC (permalink / raw)
To: git
In-Reply-To: <BANLkTi=GZDLu-ey1=h8LLDbWssoSpsM_jd7R-oFr+b+82Otb8g@mail.gmail.com>
Just thought that I should add some numbers to this thread as it seems that
the later versions of git are worse off by several orders of magnitude on
this one.
We have a Gerrit repo with just under 100K refs in refs/changes/*. When I
fetch them all with git 1.7.6 it does not seem to complete. Even after 5
days, it is just under half way through the ref #s! It appears, but I am
not sure, that it is getting slower with time also, so it may not even
complete after 10 days, I couldn't wait any longer. However, the same
command works in under 8 mins with git 1.7.3.3 on the same machine!
Syncing 100K refs:
git 1.7.6 > 8 days?
git 1.7.3.3 ~8mins
That is quite a difference! Have there been any obvious changes to git that
should cause this? If needed, I can bisect git to find out where things go
sour, but I thought that perhaps there would be someone who already
understands the problem and why older versions aren't nearly as bas as
recent ones.
Some more things that I have tried: after syncing the repo locally with all
100K refs under refs/changes, I cloned it locally again and tried fetching
locally with both git 1.7.6 and 1.7.3.3. I got the same results as
remotely, so it does not appear to be related to round trips.
The original git remote syncing takes just a bit of time, and then it
outputs lines like these:
...
* [new branch] refs/changes/13/66713/2 -> refs/changes/13/66713/2
* [new branch] refs/changes/13/66713/3 -> refs/changes/13/66713/3
* [new branch] refs/changes/13/66713/4 -> refs/changes/13/66713/4
* [new branch] refs/changes/13/66713/5 -> refs/changes/13/66713/5
...
This is the part that takes forever. The lines seem to scroll by slower and
slower (with git 1.7.6). In the beginning, the lines might be a screens
worth a minute, after 5 days, about 1 a minute. My CPU is pegged at 100%
during this time (one core). Since I have some good test data for this, let
me know if I should test anything specific.
Thanks,
-Martin
Employee of Qualcomm Innovation Center, Inc. which is a member of Code
Aurora Forum
--
View this message in context: http://git.661346.n2.nabble.com/Git-is-not-scalable-with-too-many-refs-tp6456443p6773496.html
Sent from the git mailing list archive at Nabble.com.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox