* Re: Resumable clone/Gittorrent (again)
From: Luke Kenneth Casson Leighton @ 2011-01-06 17:50 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: Thomas Rast, Git Mailing List, Nicolas Pitre
In-Reply-To: <AANLkTi=-gomYOpX6+RSboBXBytPry1Qhf31ohmv1dC5d@mail.gmail.com>
On Thu, Jan 6, 2011 at 1:47 AM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> On Thu, Jan 6, 2011 at 1:07 AM, Luke Kenneth Casson Leighton
> <luke.leighton@gmail.com> wrote:
>> the plan is to turn that variation in the git pack-objects responses,
>> across multiple peers, into an *advantage* not a liability. how?
>> like this:
>>
>> * a client requiring objects from commit abcd0123 up to commit
>> efga3456 sends out a DHT broadcast query to all and sundry who have
>> commits abcd0123 and everything in between up to efga3456.
>>
>> * those clients that can be bothered to respond, do so [refinements below]
>>
>> * the requestor selects a few of them, and asks them to create git
>> pack-objects. this takes time, but that's ok. once created, the size
>> of the git pack-object is sent as part of the acknowledgement.
>>
>> * the requestor, on receipt of all the sizes, selects the *smallest*
>> one to begin the p2p (.torrent) from (by asking the remote client to
>> create a .torrent specifically for that purpose, with the filename
>> abcd0123-ebga3456).
>
> That defeats the purpose of distributing. You are putting pressure on
> certain peers.
that's unavoidable, but it's not actually as bad as it seems. think
about it. normally, "pressure" is put onto a git server, by forcing
that server to perform multiple "git pack-object" calculations,
repeatedly, for each and every "git pull".
so, the principle behind this RFC (is it an RFC? yes, kinda...) is
that a) you cache those git pack-objects, thus avoiding heavy CPU
usage b) you make the requests to _many_ peers that you'll likely find
already are in the process of distributing that particular
commit-range _anyway_ so will _definitely_ have it ... etc. etc.
so there's a ton of reasons why it's quite a big improvement over the
present star-network arrangement.
>
>> now, an immediately obvious refinement of this is that those .torrent
>> (pack-objects) "stick around", in a cache (with a hard limit defined
>> on the cache size of course). and so, when the client that requires a
>> pack-object makes the request, of course, those remote clients that
>> *already* have that cached pack-object for that specific commit-range
>> should be given first priority, to avoid other clients from having to
>> make massive amounts of git pack-objects.
>
> Cache have its limits too. Suppose I half-fetch a pack then stop and
> go wild for a month. The next month I restart the fetch, the pack may
> no longer in cache. A new pack may or may not be identical to the old
> pack.
correct. that's not in the slightest bit a problem. the peer which
has that new pack will be asked to make a new .torrent for _that_
pack. with a new name that uniquely identifies it (the md5sum of the
pack would do as the .torrent filename)
> Also if you go with packs, you are tied to the peer that generates
> that pack. Two different peers can, in theory, generate different
> packs (in encoding) for the same input.
yes. correct. i _did_ say that you pick the one that is the
smallest of the two (or three. or 10). in this way you actually do
much better than you would otherwise in a "star network" such as a
standard HTTP git server, because you've asked 2, 3 or 10 (whatever)
peers, and you'll end up with _the_ most efficient representation of
that commit-range. statistically speaking, of course :)
> Another thing with packs (ok, not exactly with packs) is how you
> verify that's you have got what you asked.
ok - how do you verify that you've got what you asked, when you ask
from a git server using HTTP?
> Bittorrent can verify every
> piece a peer receives because sha-1 sum of those pieces are recorded
> in .torrent file.
yes. this is simply a part of the bittorrent protocol, to ensure
that the file being transferred is correctly transferred.
these verifications steps should be _trusted_ and should _not_ be
confused with anything else (i've deleted the rest of the paragraph
you wrote, in order to reduce any opportunity for confusion).
if you mentally keep git separate from bittorrent it helps. imagine
that bittorrent is merely a drop-in replacement for git over HTTP
(nicolas kindly explained the plugin system for git which would add
another protocol for downloading of git repos, and yes this can all be
implemented as a plugin)
>> so, can you see that a) this is a far cry from the "simplistic
>> transfer of blobs and trees" b) it's *not* going to overload peoples'
>> systems by splattering (eek!) millions of md5 sums across the internet
>> as bittorrent files c) it _does_ fit neatly into the bittorrent
>> protocol d) it combines the best of git with the best of p2p
>> distributed networking principles...
>
> How can you advertise what you have to another peer?
you don't. it's done "on-demand".
the concept of "git push" becomes virtually a null-op, updating the
bittorrent tracker and that's... about it.
it's where "git pull" that all the work is done, starting with that
DHT query [no i know "bittorrent the protocol" doesn't have DHT, but
many bittorrent clients _do_ have DHT, and Tribler has an extremely
good one].
l.
^ permalink raw reply
* Re: git checkout branch behaves differently from what specified in man page
From: Jonathan Nieder @ 2011-01-06 17:35 UTC (permalink / raw)
To: r.ductor; +Cc: git
In-Reply-To: <20110106154418.3348.29438.reportbug@localhost>
Hi,
r. ductor wrote[1]:
> git checkout [<branch>], git checkout -b <new branch> [<start point>]
> This form switches branches by updating the index, working tree,
> and HEAD to reflect the specified branch. "
>
> The key is the meaning of "to reflect": a normal (?) human being
> like me understands that git checkout branch will reset the
> contents of working directory, and index exactly equal to that of
> the head of the branch. As you might know this is not true (see
> example below).
[example with both unregistered and staged local modifications]
Yes, I agree that this could be a lot clearer. For reference, here's
the current description section:
Updates files in the working tree to match the version in the index
or the specified tree. If no paths are given, git checkout will also
update HEAD to set the specified branch as the current branch.
git checkout [<branch>], git checkout -b|-B <new_branch> [<start point>]
This form switches branches by updating the index, working tree,
and HEAD to reflect the specified branch.
If -b is given,
[... description of -b and -B ...]
git checkout [--patch] [<tree-ish>] [--] <pathspec>...
When <paths> or --patch are given, git checkout does not switch
branches. It updates the named paths in the working tree from
the index file or from a named <tree-ish> (most often a commit).
In this case, the -b and --track options are meaningless and
giving either of them results in an error. The <tree-ish>
argument can be used to specify a specific tree-ish (i.e.
commit, tag or tree) to update the index for the given paths
before updating the working tree.
The index may contain unmerged entries because of a previous
failed merge.
[... more details about the unmerged entries case ...]
Proposed clearer text would be welcome, especially if in the form of
a patch to Documentation/git-checkout.txt (see Documentation/SubmittingPatches).
Currently I think the examples section explains it better. The
details are something like this:
- git checkout <branch> --
tries to change the current branch to <branch>, dragging other
state kicking and screaming along with it.
- if a file does _not_ differ between the current commit and
<branch>, it does not need to be touched. Any local changes
in that file will be preserved.
- if a file that _does_ differ between the current commit and
<branch> has local changes, git gives up --- the operation is
cancelled and all state is preserved.
This behavior can be tweaked with the --merge option.
- git checkout <tree> -- <path specifiers>
keeping the current branch the same, grabs some files from <tree>
to replace the current versions in the index and work tree
- no matter what, the local state of the specified paths is
forgotten and both the index and worktree made to match <tree>
- paths not specified are left alone.
If you just want to reset the worktree and index to match some
branch, then you can use
git reset --hard <branch>
or
git checkout <tree> -- .
. The former even forgets what branch you were on (well, it is
remembered in the reflog). The latter adopts the content of <tree>
as content registered for the next commit on the current branch.
Hope that helps,
Jonathan
[1] http://bugs.debian.org/609128
^ permalink raw reply
* Re: Concurrent pushes updating the same ref
From: Ilari Liusvaara @ 2011-01-06 17:28 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Jeff King, Marc Branchaud, Git Mailing List
In-Reply-To: <AANLkTinGYjExrnHCqcTPu-APzk9WynxwWoKVOLaSkZ2y@mail.gmail.com>
On Thu, Jan 06, 2011 at 08:48:11AM -0800, Shawn Pearce wrote:
> On Thu, Jan 6, 2011 at 08:30, Jeff King <peff@peff.net> wrote:
> >
> > Right now you are getting contention on the lock itself. But may
> > you not also run afoul of step (6) above? That is, one push updates
> > the ref from A to B, then the other one in attempting to go from A
> > to B sees that it has already changed to B under our feet and
> > complains?
>
> Not if its a force push. :-)
IIRC, there are no wire protocol bits to denote a forced push, the
force option only overrides client-side checks. Thus, even forced pushes
can fail due to race conditions...
-Ilari
^ permalink raw reply
* Re: Repeatable test t9010-svn-fe.sh failure w/ master (685e9d9, 1.7.4.rc1)
From: A Large Angry SCM @ 2011-01-06 17:20 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20110106165958.GA11190@burratino>
[-- Attachment #1: Type: text/plain, Size: 569 bytes --]
On 01/06/2011 11:59 AM, Jonathan Nieder wrote:
> A Large Angry SCM wrote:
>
>> *** t9010-svn-fe.sh ***
>> ok 1 - empty dump
>> ok 2 - v3 dumps not supported
>> not ok - 3 t9135/svn.dump
>
> Thanks for reporting. Unfortunately I haven't been able to reproduce
> this on the machines I have access to (amd64 with svn 1.5.0 and i386
> with svn 1.6.15).
>
> Does running
>
> sh t9010-svn-fe.sh -v
>
> from the t/ directory give an error message? If not, how about
>
> GIT_TRACE=1 sh -x t9010-svn-fe.sh -v
>
> ?
>
See the attached typescript files from the above runs.
[-- Attachment #2: typescript1.txt --]
[-- Type: text/plain, Size: 3525 bytes --]
Script started on Thu 06 Jan 2011 12:11:03 PM EST
XXXXXXXX@debian:~/GIT/git/t$ sh t9010-svn-fe.sh -v
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
^[[33mexpecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
^[(B^[[m
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects: 5000
Total objects: 0 ( 0 duplicates )
blobs : 0 ( 0 duplicates 0 deltas)
trees : 0 ( 0 duplicates 0 deltas)
commits: 0 ( 0 duplicates 0 deltas)
tags : 0 ( 0 duplicates 0 deltas)
Total branches: 0 ( 0 loads )
marks: 1024 ( 0 unique )
atoms: 0
Memory total: 2282 KiB
pools: 2048 KiB
objects: 234 KiB
---------------------------------------------------------------------
pack_report: getpagesize() = 4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit = 8589934592
pack_report: pack_used_ctr = 0
pack_report: pack_mmap_calls = 0
pack_report: pack_open_windows = 0 / 0
pack_report: pack_mapped = 0 / 0
---------------------------------------------------------------------
ok 1 - empty dump^[(B^[[m
^[[33mexpecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
^[(B^[[m
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
fatal: expected svn dump format version <= 2, found 3
ok 2 - v3 dumps not supported^[(B^[[m
^[[33mexpecting success:
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
^[(B^[[m
<<< Started new transaction, based on original revision 1
* adding path : branches ... done.
* adding path : trunk ... done.
svnadmin: Couldn't perform atomic initialization
svnadmin: Couldn't perform atomic initialization
svnadmin: SQLite compiled for 3.7.4, but running with 3.7.3
^[[1m^[[31mnot ok - 3 t9135/svn.dump^[(B^[[m
#
# svnadmin create simple-svn &&
# svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
# svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
# git init simple-git &&
# test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
# (
# cd simple-git &&
# git fast-import <../simple.fe
# ) &&
# (
# cd simple-svnco &&
# git init &&
# git add . &&
# git fetch ../simple-git master &&
# git diff --exit-code FETCH_HEAD
# )
#
^[[1m^[[31m# failed 1 among 3 test(s)^[(B^[[m
^[[33m1..3^[(B^[[m
XXXXXXXX@debian:~/GIT/git/t$ exit
Script done on Thu 06 Jan 2011 12:11:16 PM EST
[-- Attachment #3: typescript2.txt --]
[-- Type: text/plain, Size: 18184 bytes --]
Script started on Thu 06 Jan 2011 12:11:40 PM EST
XXXXXXXX@debian:~/GIT/git/t$ GIT_TRACE=1 sh -x t9010-svn-fe.sh -v
+ test_description=check svn dumpfile importer
+ . ./test-lib.sh
+ ORIGINAL_TERM=xterm
+ LANG=C
+ LC_ALL=C
+ PAGER=cat
+ TZ=UTC
+ TERM=dumb
+ export LANG LC_ALL PAGER TERM TZ
+ EDITOR=:
+ unset VISUAL
+ unset GIT_EDITOR
+ unset AUTHOR_DATE
+ unset AUTHOR_EMAIL
+ unset AUTHOR_NAME
+ unset COMMIT_AUTHOR_EMAIL
+ unset COMMIT_AUTHOR_NAME
+ unset EMAIL
+ unset GIT_ALTERNATE_OBJECT_DIRECTORIES
+ unset GIT_AUTHOR_DATE
+ GIT_AUTHOR_EMAIL=author@example.com
+ GIT_AUTHOR_NAME=A U Thor
+ unset GIT_COMMITTER_DATE
+ GIT_COMMITTER_EMAIL=committer@example.com
+ GIT_COMMITTER_NAME=C O Mitter
+ unset GIT_DIFF_OPTS
+ unset GIT_DIR
+ unset GIT_WORK_TREE
+ unset GIT_EXTERNAL_DIFF
+ unset GIT_INDEX_FILE
+ unset GIT_OBJECT_DIRECTORY
+ unset GIT_CEILING_DIRECTORIES
+ unset SHA1_FILE_DIRECTORIES
+ unset SHA1_FILE_DIRECTORY
+ unset GIT_NOTES_REF
+ unset GIT_NOTES_DISPLAY_REF
+ unset GIT_NOTES_REWRITE_REF
+ unset GIT_NOTES_REWRITE_MODE
+ GIT_MERGE_VERBOSITY=5
+ export GIT_MERGE_VERBOSITY
+ export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
+ export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
+ export EDITOR
+ unset CDPATH
+ unset GREP_OPTIONS
+ echo 1
+ tr [A-Z] [a-z]
+ echo * warning: Some tests will not work if GIT_TRACE is set as to trace on STDERR ! *
* warning: Some tests will not work if GIT_TRACE is set as to trace on STDERR ! *
+ echo * warning: Please set GIT_TRACE to something other than 1, 2 or true ! *
* warning: Please set GIT_TRACE to something other than 1, 2 or true ! *
+ _x05=[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]
+ _x40=[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]
+ [ xxterm != xdumb ]
+ TERM=xterm
+ export TERM
+ [ -t 1 ]
+ tput bold
+ tput setaf 1
+ tput sgr0
+ color=t
+ test 1 -ne 0
+ verbose=t
+ shift
+ test 0 -ne 0
+ test -n t
+ test check svn dumpfile importer !=
+ test = t
+ exec
+ test t = t
+ exec
+ test_failure=0
+ test_count=0
+ test_fixed=0
+ test_broken=0
+ test_success=0
+ test_external_has_tap=0
+ GIT_EXIT_OK=
+ trap die EXIT
+ satisfied=
+ test -z
+ pwd
+ TEST_DIRECTORY=/home/XXXXXXXX/GIT/git/t
+ GIT_BUILD_DIR=/home/XXXXXXXX/GIT/git/t/..
+ test -n
+ test -n
+ git_bin_dir=/home/XXXXXXXX/GIT/git/t/../bin-wrappers
+ test -x /home/XXXXXXXX/GIT/git/t/../bin-wrappers/git
+ PATH=/home/XXXXXXXX/GIT/git/t/../bin-wrappers:/home/XXXXXXXX/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
+ GIT_EXEC_PATH=/home/XXXXXXXX/GIT/git/t/..
+ test -n
+ GIT_TEMPLATE_DIR=/home/XXXXXXXX/GIT/git/t/../templates/blt
+ unset GIT_CONFIG
+ GIT_CONFIG_NOSYSTEM=1
+ GIT_CONFIG_NOGLOBAL=1
+ export PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR GIT_CONFIG_NOSYSTEM GIT_CONFIG_NOGLOBAL
+ . /home/XXXXXXXX/GIT/git/t/../GIT-BUILD-OPTIONS
+ SHELL_PATH=/bin/sh
+ PERL_PATH=/usr/bin/perl
+ DIFF=diff
+ PYTHON_PATH=/usr/bin/python
+ TAR=tar
+ NO_CURL=
+ NO_PERL=
+ NO_PYTHON=
+ test -z
+ test -n
+ GIT_TEST_CMP=diff -u
+ GITPERLLIB=/home/XXXXXXXX/GIT/git/t/../perl/blib/lib:/home/XXXXXXXX/GIT/git/t/../perl/blib/arch/auto/Git
+ export GITPERLLIB
+ test -d /home/XXXXXXXX/GIT/git/t/../templates/blt
+ test -z
+ test -z
+ GITPYTHONLIB=/home/XXXXXXXX/GIT/git/t/../git_remote_helpers/build/lib
+ export GITPYTHONLIB
+ test -d /home/XXXXXXXX/GIT/git/t/../git_remote_helpers/build
+ test -x /home/XXXXXXXX/GIT/git/t/../test-chmtime
+ basename t9010-svn-fe.sh .sh
+ test=trash directory.t9010-svn-fe
+ test -n
+ TRASH_DIRECTORY=/home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe
+ test ! -z
+ remove_trash=/home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe
+ rm -fr trash directory.t9010-svn-fe
+ test_create_repo trash directory.t9010-svn-fe
+ test 1 = 1
+ repo=trash directory.t9010-svn-fe
+ mkdir -p trash directory.t9010-svn-fe
+ cd trash directory.t9010-svn-fe
+ /home/XXXXXXXX/GIT/git/t/../git-init --template=/home/XXXXXXXX/GIT/git/t/../templates/blt/
trace: built-in: git 'init' '--template=/home/XXXXXXXX/GIT/git/t/../templates/blt/'
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
+ mv .git/hooks .git/hooks-disabled
+ cd -P trash directory.t9010-svn-fe
+ pwd
+ HOME=/home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe
+ export HOME
+ this_test=t9010-svn-fe.sh
+ this_test=t9010
+ uname -s
+ test_set_prereq POSIXPERM
+ satisfied= POSIXPERM
+ test_set_prereq BSLASHPSPEC
+ satisfied= POSIXPERM BSLASHPSPEC
+ test_set_prereq EXECKEEPSPID
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID
+ test_set_prereq NOT_MINGW
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID NOT_MINGW
+ test -z
+ test_set_prereq PERL
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID NOT_MINGW PERL
+ test -z
+ test_set_prereq PYTHON
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID NOT_MINGW PERL PYTHON
+ ln -s x y
+ test -h y
+ test_set_prereq SYMLINKS
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID NOT_MINGW PERL PYTHON SYMLINKS
+ rm -f y
+ test -w /
+ test_set_prereq SANITY
+ satisfied= POSIXPERM BSLASHPSPEC EXECKEEPSPID NOT_MINGW PERL PYTHON SYMLINKS SANITY
+ svnadmin -h
+ svnconf=/home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/svnconf
+ export svnconf
+
+ test_expect_success empty dump
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ test 2 = 3
+ test_prereq=
+ test 2 = 2
+ export test_prereq
+ test_skip empty dump
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ test_count=1
+ to_skip=
+ test -z
+ test -n
+ false
+ say expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ say_color info expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ TERM=xterm
+ export TERM
+ tput setaf 3
^[[33m+ shift
+ printf %s expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ tput sgr0
^[(B^[[m+ echo
+ test_run_
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ test_cleanup=:
+ eval
reinit_git &&
echo "SVN-fs-dump-format-version: 2" >input &&
test-svn-fe input >stream &&
git fast-import <stream
+ reinit_git
+ rm -fr .git
+ git init
trace: built-in: git 'init'
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
+ echo SVN-fs-dump-format-version: 2
+ test-svn-fe input
+ git fast-import
trace: exec: 'git-fast-import'
trace: run_command: 'git-fast-import'
git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects: 5000
Total objects: 0 ( 0 duplicates )
blobs : 0 ( 0 duplicates 0 deltas)
trees : 0 ( 0 duplicates 0 deltas)
commits: 0 ( 0 duplicates 0 deltas)
tags : 0 ( 0 duplicates 0 deltas)
Total branches: 0 ( 0 loads )
marks: 1024 ( 0 unique )
atoms: 0
Memory total: 2282 KiB
pools: 2048 KiB
objects: 234 KiB
---------------------------------------------------------------------
pack_report: getpagesize() = 4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit = 8589934592
pack_report: pack_used_ctr = 0
pack_report: pack_mmap_calls = 0
pack_report: pack_open_windows = 0 / 0
pack_report: pack_mapped = 0 / 0
---------------------------------------------------------------------
+ eval_ret=0
+ eval :
+ :
+ test t = t
+ test -n
+ return 0
+ [ 0 = 0 -a 0 = 0 ]
+ test_ok_ empty dump
+ test_success=1
+ say_color ok 1 - empty dump
+ TERM=xterm
+ export TERM
+ test -n
+ shift
+ printf %s ok 1 - empty dump
ok 1 - empty dump+ tput sgr0
^[(B^[[m+ echo
+ echo
+ test_expect_success v3 dumps not supported
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ test 2 = 3
+ test_prereq=
+ test 2 = 2
+ export test_prereq
+ test_skip v3 dumps not supported
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ test_count=2
+ to_skip=
+ test -z
+ test -n
+ false
+ say expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ say_color info expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ TERM=xterm
+ export TERM
+ tput setaf 3
^[[33m+ shift
+ printf %s expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
expecting success:
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ tput sgr0
^[(B^[[m+ echo
+ test_run_
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ test_cleanup=:
+ eval
reinit_git &&
echo "SVN-fs-dump-format-version: 3" >input &&
test_must_fail test-svn-fe input >stream &&
test_cmp empty stream
+ reinit_git
+ rm -fr .git
+ git init
trace: built-in: git 'init'
Initialized empty Git repository in /home/XXXXXXXX/GIT/git/t/trash directory.t9010-svn-fe/.git/
+ echo SVN-fs-dump-format-version: 3
+ test_must_fail test-svn-fe input
+ test-svn-fe input
fatal: expected svn dump format version <= 2, found 3
+ exit_code=128
+ test 128 = 0
+ test 128 -gt 129 -a 128 -le 192
+ test 128 = 127
+ return 0
+ test_cmp empty stream
+ diff -u empty stream
+ eval_ret=0
+ eval :
+ :
+ test t = t
+ test -n
+ return 0
+ [ 0 = 0 -a 0 = 0 ]
+ test_ok_ v3 dumps not supported
+ test_success=2
+ say_color ok 2 - v3 dumps not supported
+ TERM=xterm
+ export TERM
+ test -n
+ shift
+ printf %s ok 2 - v3 dumps not supported
ok 2 - v3 dumps not supported+ tput sgr0
^[(B^[[m+ echo
+ echo
+ test_expect_success t9135/svn.dump
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ test 2 = 3
+ test_prereq=
+ test 2 = 2
+ export test_prereq
+ test_skip t9135/svn.dump
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ test_count=3
+ to_skip=
+ test -z
+ test -n
+ false
+ say expecting success:
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ say_color info expecting success:
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ TERM=xterm
+ export TERM
+ tput setaf 3
^[[33m+ shift
+ printf %s expecting success:
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
expecting success:
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ tput sgr0
^[(B^[[m+ echo
+ test_run_
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ test_cleanup=:
+ eval
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ svnadmin create simple-svn
+ svnadmin load simple-svn
<<< Started new transaction, based on original revision 1
* adding path : branches ... done.
* adding path : trunk ... done.
svnadmin: Couldn't perform atomic initialization
svnadmin: Couldn't perform atomic initialization
svnadmin: SQLite compiled for 3.7.4, but running with 3.7.3
+ eval_ret=1
+ eval :
+ :
+ test t = t
+ test -n
+ return 0
+ [ 0 = 0 -a 1 = 0 ]
+ test_failure_ t9135/svn.dump
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ test_failure=1
+ say_color error not ok - 3 t9135/svn.dump
+ TERM=xterm
+ export TERM
+ tput bold
^[[1m+ tput setaf 1
^[[31m+ shift
+ printf %s not ok - 3 t9135/svn.dump
not ok - 3 t9135/svn.dump+ tput sgr0
^[(B^[[m+ echo
+ shift
+ echo
svnadmin create simple-svn &&
svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
git init simple-git &&
test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
(
cd simple-git &&
git fast-import <../simple.fe
) &&
(
cd simple-svnco &&
git init &&
git add . &&
git fetch ../simple-git master &&
git diff --exit-code FETCH_HEAD
)
+ sed -e s/^/# /
#
# svnadmin create simple-svn &&
# svnadmin load simple-svn <"$TEST_DIRECTORY/t9135/svn.dump" &&
# svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
# git init simple-git &&
# test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
# (
# cd simple-git &&
# git fast-import <../simple.fe
# ) &&
# (
# cd simple-svnco &&
# git init &&
# git add . &&
# git fetch ../simple-git master &&
# git diff --exit-code FETCH_HEAD
# )
#
+ test =
+ echo
+ test_done
+ GIT_EXIT_OK=t
+ test -z
+ test_results_dir=/home/XXXXXXXX/GIT/git/t/test-results
+ mkdir -p /home/XXXXXXXX/GIT/git/t/test-results
+ test_results_path=/home/XXXXXXXX/GIT/git/t/test-results/t9010-svn-fe-14042.counts
+ echo total 3
+ echo success 2
+ echo fixed 0
+ echo broken 0
+ echo failed 1
+ echo
+ test 0 != 0
+ test 0 != 0
+ msg=3 test(s)
+ test 0 -eq 0
+ say_color error # failed 1 among 3 test(s)
+ TERM=xterm
+ export TERM
+ tput bold
^[[1m+ tput setaf 1
^[[31m+ shift
+ printf %s # failed 1 among 3 test(s)
# failed 1 among 3 test(s)+ tput sgr0
^[(B^[[m+ echo
+ say 1..3
+ say_color info 1..3
+ TERM=xterm
+ export TERM
+ tput setaf 3
^[[33m+ shift
+ printf %s 1..3
1..3+ tput sgr0
^[(B^[[m+ echo
+ exit 1
+ die
+ code=1
+ test -n t
+ exit 1
XXXXXXXX@debian:~/GIT/git/t$ exit
Script done on Thu 06 Jan 2011 12:12:07 PM EST
^ permalink raw reply
* Re: Concurrent pushes updating the same ref
From: Marc Branchaud @ 2011-01-06 17:12 UTC (permalink / raw)
To: Jeff King; +Cc: Git Mailing List
In-Reply-To: <20110106163035.GA7812@sigill.intra.peff.net>
On 11-01-06 11:30 AM, Jeff King wrote:
> On Thu, Jan 06, 2011 at 10:46:38AM -0500, Marc Branchaud wrote:
>
>> fatal: Unable to create
>> '/usr/xiplink/git/public/Main.git/refs/builds/3.3.0-3.lock': File exists.
>> If no other git process is currently running, this probably means a
>> git process crashed in this repository earlier. Make sure no other git
>> process is running and remove the file manually to continue.
>> fatal: The remote end hung up unexpectedly
>>
>> I think the cause is pretty obvious, and in a normal interactive situation
>> the solution would be to simply try again. But in a script trying again
>> isn't so straightforward.
>>
>> So I'm wondering if there's any sense or desire to make git a little more
>> flexible here. Maybe teach it to wait and try again once or twice when it
>> sees a lock file. I presume that normally a ref lock file should disappear
>> pretty quickly, so there shouldn't be a need to wait very long.
>
> Yeah, we probably should try again. The simplest possible (and untested)
> patch is below. However, a few caveats:
>
> 1. This patch unconditionally retries for all lock files. Do all
> callers want that? I wonder if there are any exploratory lock
> acquisitions that would rather return immediately than have some
> delay.
>
> 2. The number of tries and sleep time are pulled out of a hat.
>
> 3. Even with retries, I don't know if you will get the behavior you
> want. The lock procedure for refs is:
>
> 1. get the lock
> 2. check and remember the sha1
> 3. release the lock
> 4. do some long-running work (like the actual push)
> 5. get the lock
> 6. check that the sha1 is the same as the remembered one
> 7. update the sha1
> 8. release the lock
>
> Right now you are getting contention on the lock itself. But may
> you not also run afoul of step (6) above? That is, one push updates
> the ref from A to B, then the other one in attempting to go from A
> to B sees that it has already changed to B under our feet and
> complains?
Could not anything run afoul of step (6)? Who knows what might happen in
step (4)...
However, in my particular case I'm using a "force" refspec:
git push origin +HEAD:refs/builds/${TAG}
so (as Shawn says) step (6) shouldn't matter, right? Plus, all the
concurrent pushes are setting the ref to the same value anyway.
This is fairly degenerate behaviour though.
> I can certainly think of a rule around that special case (if we are
> going to B, and it already changed to B, silently leave it alone
> and pretend we wrote it), but I don't know how often that would be
> useful in the real world.
Yes -- useful in my case, but otherwise... Still, I think it would be
more-correct to do that.
M.
^ permalink raw reply
* Re: Resumable clone/Gittorrent (again) - stable packs?
From: Shawn Pearce @ 2011-01-06 17:05 UTC (permalink / raw)
To: Zenaan Harkness; +Cc: git
In-Reply-To: <AANLkTikv+L5Da7A5VM7BAgnue=m0O_-nHmHchJzfGxJa@mail.gmail.com>
On Wed, Jan 5, 2011 at 18:29, Zenaan Harkness <zen@freedbms.net> wrote:
> Bittorrent requires some stability around torrent files.
>
> Can packs be generated deterministically?
No. We have been trying to avoid doing that, because it ties us into
one particular compression scheme. We can't tune the algorithm and
get better compression later, because it would generate a different
pack. We also rely on the system's libz to generate the compressed
data. A version change to libz may generate a different encoding for
the same uncompressed data, simply because they made a tweak to how
the compression was performed. Likewise our own delta compression
code can be tweaked to produce a different (but logically identical)
delta between the same two objects.
Right now packs aren't deterministic because they use multiple threads
to generate the deltas, the thread scheduling impacts which base
objects deltas are tried against because threads can steal work from
each other if one finishes before the other one. Disabling threading
entirely slows down delta compression considerably on multi-core
machines, but does remove this work-stealing, making the pack
deterministic... but only for this exact Git binary, with this same
shared libz. If the system libz or Git changes, all bets are off.
We've been down this road before; we don't want to box ourselves into
a tight corner by setting for all time these tunable portions of the
compression algorithms.
--
Shawn.
^ permalink raw reply
* log -p hides changes in merge commit
From: Phillip Susi @ 2011-01-06 17:07 UTC (permalink / raw)
To: git
git log -p never shows a diff for merge commits. It is nice that it
does not show a giant diff that is the sum of all of the changes being
merged, but any manual changes made on top of the merge are also lost
from view, and this is not good. Here is a reproduction recipe:
git init
echo foo > a
git add a
git commit -m "added a"
git branch other
git checkout other
echo bar > b
git add b
git commit -m "added b"
git checkout master
git merge other
git log -p
At this point there is no diff shown in the log output. This is fine
since there were no changes needed to complete the merge, and the
addition of b is already documented in the merged commit. The problem
is that if you add --no-merge to the git merge, and then:
echo bart > a
git add a
git commit
Now in addition to merging b, you have modified a, but git log -p still
shows no diff, effectively hiding the fact that you snuck in a
modification to a during the merge.
It seems that adding -c or --cc to the log correctly shows the change to
a, but why is this not shown by default?
^ permalink raw reply
* Re: Repeatable test t9010-svn-fe.sh failure w/ master (685e9d9, 1.7.4.rc1)
From: Jonathan Nieder @ 2011-01-06 16:59 UTC (permalink / raw)
To: A Large Angry SCM; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <4D25E818.5050909@gmail.com>
A Large Angry SCM wrote:
> *** t9010-svn-fe.sh ***
> ok 1 - empty dump
> ok 2 - v3 dumps not supported
> not ok - 3 t9135/svn.dump
Thanks for reporting. Unfortunately I haven't been able to reproduce
this on the machines I have access to (amd64 with svn 1.5.0 and i386
with svn 1.6.15).
Does running
sh t9010-svn-fe.sh -v
from the t/ directory give an error message? If not, how about
GIT_TRACE=1 sh -x t9010-svn-fe.sh -v
?
^ permalink raw reply
* Re: --find-copies-harder finds fewer copies/renames than -C does
From: Stefan Haller @ 2011-01-06 16:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7voc7vch35.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
> lists@haller-berlin.de (Stefan Haller) writes:
>
> > I had expected --find-copies-harder to still do inexact rename detection
> > among the changed files in the commit in this case, and turn it off only
> > for the unmodified files; I'm not familiar enough with the code to tell
> > whether that would be easy to implement though.
> >
> > Any thoughts?
>
> Two. When you can spend unlimited amount of resources, it would feel more
> intuitive if -C -C lifted rename-limit altogether. On the other hand, in
> a project where the difference does matter (i.e. you have far too many
> candidate sources), it is likely that -C -C without rename limit would run
> out of memory and not produce any result, so automatic lifting of rename
> limit is unacceptable as the default.
But what about the suggestion of falling back to -C if
--find-copies-harder exceeds the rename limit, but -C does not?
Wouldn't that be the desired behaviour?
--
Stefan Haller
Berlin, Germany
http://www.haller-berlin.de/
^ permalink raw reply
* Re: Concurrent pushes updating the same ref
From: Shawn Pearce @ 2011-01-06 16:48 UTC (permalink / raw)
To: Jeff King; +Cc: Marc Branchaud, Git Mailing List
In-Reply-To: <20110106163035.GA7812@sigill.intra.peff.net>
On Thu, Jan 6, 2011 at 08:30, Jeff King <peff@peff.net> wrote:
>
> Yeah, we probably should try again. The simplest possible (and untested)
> patch is below. However, a few caveats:
>
> 1. This patch unconditionally retries for all lock files. Do all
> callers want that? I wonder if there are any exploratory lock
> acquisitions that would rather return immediately than have some
> delay.
I don't see why not. We shouldn't be exploring to see if a lock is
possible anywhere.
> 2. The number of tries and sleep time are pulled out of a hat.
FWIW, JGit has started to do some of this stuff for Windows. We're
using 10 retries, with a delay of 100 milliseconds between each. This
was also pulled out of a hat, but it seems to have resolved the bug
reports that came in on Windows. We unfortunately have to do retries
on directory and file deletion.
> 3. Even with retries, I don't know if you will get the behavior you
> want. The lock procedure for refs is:
>
> 1. get the lock
> 2. check and remember the sha1
> 3. release the lock
Why are we locking the ref to read it? You can read a ref atomically
without locking.
> 4. do some long-running work (like the actual push)
> 5. get the lock
> 6. check that the sha1 is the same as the remembered one
> 7. update the sha1
> 8. release the lock
>
> Right now you are getting contention on the lock itself. But may
> you not also run afoul of step (6) above? That is, one push updates
> the ref from A to B, then the other one in attempting to go from A
> to B sees that it has already changed to B under our feet and
> complains?
Not if its a force push. :-)
--
Shawn.
^ permalink raw reply
* Re: Concurrent pushes updating the same ref
From: Jeff King @ 2011-01-06 16:30 UTC (permalink / raw)
To: Marc Branchaud; +Cc: Git Mailing List
In-Reply-To: <4D25E3DE.7050801@xiplink.com>
On Thu, Jan 06, 2011 at 10:46:38AM -0500, Marc Branchaud wrote:
> fatal: Unable to create
> '/usr/xiplink/git/public/Main.git/refs/builds/3.3.0-3.lock': File exists.
> If no other git process is currently running, this probably means a
> git process crashed in this repository earlier. Make sure no other git
> process is running and remove the file manually to continue.
> fatal: The remote end hung up unexpectedly
>
> I think the cause is pretty obvious, and in a normal interactive situation
> the solution would be to simply try again. But in a script trying again
> isn't so straightforward.
>
> So I'm wondering if there's any sense or desire to make git a little more
> flexible here. Maybe teach it to wait and try again once or twice when it
> sees a lock file. I presume that normally a ref lock file should disappear
> pretty quickly, so there shouldn't be a need to wait very long.
Yeah, we probably should try again. The simplest possible (and untested)
patch is below. However, a few caveats:
1. This patch unconditionally retries for all lock files. Do all
callers want that? I wonder if there are any exploratory lock
acquisitions that would rather return immediately than have some
delay.
2. The number of tries and sleep time are pulled out of a hat.
3. Even with retries, I don't know if you will get the behavior you
want. The lock procedure for refs is:
1. get the lock
2. check and remember the sha1
3. release the lock
4. do some long-running work (like the actual push)
5. get the lock
6. check that the sha1 is the same as the remembered one
7. update the sha1
8. release the lock
Right now you are getting contention on the lock itself. But may
you not also run afoul of step (6) above? That is, one push updates
the ref from A to B, then the other one in attempting to go from A
to B sees that it has already changed to B under our feet and
complains?
I can certainly think of a rule around that special case (if we are
going to B, and it already changed to B, silently leave it alone
and pretend we wrote it), but I don't know how often that would be
useful in the real world.
Anyway, patch (for discussion, not inclusion) is below.
diff --git a/lockfile.c b/lockfile.c
index b0d74cd..3329719 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -122,7 +122,7 @@ static char *resolve_symlink(char *p, size_t s)
}
-static int lock_file(struct lock_file *lk, const char *path, int flags)
+static int lock_file_single(struct lock_file *lk, const char *path, int flags)
{
if (strlen(path) >= sizeof(lk->filename))
return -1;
@@ -155,6 +155,21 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
return lk->fd;
}
+static int lock_file(struct lock_file *lk, const char *path, int flags)
+{
+ int tries;
+ int fd;
+ for (tries = 0; tries < 3; tries++) {
+ fd = lock_file_single(lk, path, flags);
+ if (fd >= 0)
+ return fd;
+ if (errno != EEXIST)
+ return fd;
+ sleep(1);
+ }
+ return fd;
+}
+
static char *unable_to_lock_message(const char *path, int err)
{
struct strbuf buf = STRBUF_INIT;
^ permalink raw reply related
* Repeatable test t9010-svn-fe.sh failure w/ master (685e9d9, 1.7.4.rc1)
From: A Large Angry SCM @ 2011-01-06 16:04 UTC (permalink / raw)
To: Jonathan Nieder, Junio C Hamano; +Cc: Git Mailing List
System:
Linux debian 2.6.32-5-amd64 #1 SMP Fri Dec 10 15:35:08 UTC 2010 x86_64
GNU/Linux
All installed packages up-to-date as of 2011-01-06 15:55 UTC
Git:
Latest master (685e9d9)
$ ./git --version
git version 1.7.4.rc1
Svn:
libsvn-perl 1.6.12dfsg-3
libsvn1 1.6.12dfsg-3
subversion 1.6.12dfsg-3
Invocation:
(make NO_OPENSSL=1 'gitexecdir=$(bindir)' all test strip)
*** t9010-svn-fe.sh ***
ok 1 - empty dump
ok 2 - v3 dumps not supported
not ok - 3 t9135/svn.dump
#
# svnadmin create simple-svn &&
# svnadmin load simple-svn
<"$TEST_DIRECTORY/t9135/svn.dump" &&
# svn_cmd export "file://$PWD/simple-svn" simple-svnco &&
# git init simple-git &&
# test-svn-fe "$TEST_DIRECTORY/t9135/svn.dump" >simple.fe &&
# (
# cd simple-git &&
# git fast-import <../simple.fe
# ) &&
# (
# cd simple-svnco &&
# git init &&
# git add . &&
# git fetch ../simple-git master &&
# git diff --exit-code FETCH_HEAD
# )
#
# failed 1 among 3 test(s)
1..3
^ permalink raw reply
* Concurrent pushes updating the same ref
From: Marc Branchaud @ 2011-01-06 15:46 UTC (permalink / raw)
To: Git Mailing List
Hi all,
[ BACKGROUND: I've modified our build system to push a custom ref at the
start of each build. The aim is to identify in the repo which revision got
built. For us, an overall "build" consists of creating about a dozen
products, all from the same source tree. The build system (Hudson) launches
each product's build concurrently on one or more build slaves. Each of those
individual product builds clones the repo, checks out the appropriate
revision, and pushes up the custom ref. (I would have liked to make the
Hudson master job push up the ref, instead of all the slave jobs, but I
couldn't find a way to do that.) ]
Usually this works: Each slave is setting the ref to the same value, so the
order of the updates doesn't matter. But every once in a while, the push
fails with:
fatal: Unable to create
'/usr/xiplink/git/public/Main.git/refs/builds/3.3.0-3.lock': File exists.
If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.
fatal: The remote end hung up unexpectedly
I think the cause is pretty obvious, and in a normal interactive situation
the solution would be to simply try again. But in a script trying again
isn't so straightforward.
So I'm wondering if there's any sense or desire to make git a little more
flexible here. Maybe teach it to wait and try again once or twice when it
sees a lock file. I presume that normally a ref lock file should disappear
pretty quickly, so there shouldn't be a need to wait very long.
Thoughts?
M.
^ permalink raw reply
* [PATCH 4/4] t9129: use "$PERL_PATH" instead of "perl"
From: Yann Droneaud @ 2011-01-06 14:22 UTC (permalink / raw)
To: git; +Cc: Yann Droneaud
In-Reply-To: <cover.1294312018.git.yann@droneaud.fr>
Signed-off-by: Yann Droneaud <yann@droneaud.fr>
---
t/t9129-git-svn-i18n-commitencoding.sh | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh
index f3bbde4..6dda569 100755
--- a/t/t9129-git-svn-i18n-commitencoding.sh
+++ b/t/t9129-git-svn-i18n-commitencoding.sh
@@ -22,7 +22,7 @@ compare_svn_head_with () {
# extract just the log message and strip out committer info.
# don't use --limit here since svn 1.1.x doesn't have it,
LC_ALL=$GIT_LC_UTF8 svn log `git svn info --url` | \
- LC_ALL=$GIT_LC_UTF8 perl -w -e '
+ LC_ALL=$GIT_LC_UTF8 $PERL_PATH -w -e '
use bytes;
$/ = ("-"x72) . "\n";
my @x = <STDIN>;
--
1.7.3.4
^ permalink raw reply related
* [PATCH 3/4] test: use test_utf8 and GIT_LC_UTF8 where an en_US.UTF-8 locale is required
From: Yann Droneaud @ 2011-01-06 14:22 UTC (permalink / raw)
To: git; +Cc: Yann Droneaud
In-Reply-To: <cover.1294312018.git.yann@droneaud.fr>
Signed-off-by: Yann Droneaud <yann@droneaud.fr>
---
t/t9100-git-svn-basic.sh | 25 ++++++++-----------------
t/t9129-git-svn-i18n-commitencoding.sh | 13 +++----------
2 files changed, 11 insertions(+), 27 deletions(-)
diff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh
index b041516..fcdffef 100755
--- a/t/t9100-git-svn-basic.sh
+++ b/t/t9100-git-svn-basic.sh
@@ -4,20 +4,14 @@
#
test_description='git svn basic tests'
-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}
. ./lib-git-svn.sh
say 'define NO_SVN_TESTS to skip git svn tests'
-case "$GIT_SVN_LC_ALL" in
-*.UTF-8)
- test_set_prereq UTF8
- ;;
-*)
- say "# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)"
- ;;
-esac
+if ! test_utf8 ; then
+ say "# UTF-8 locale not set, some tests skipped"
+fi
test_expect_success \
'initialize git svn' '
@@ -172,15 +166,12 @@ test_expect_success "$name" '
test ! -h "$SVN_TREE"/exec-2.sh &&
test_cmp help "$SVN_TREE"/exec-2.sh'
-name="commit with UTF-8 message: locale: $GIT_SVN_LC_ALL"
-LC_ALL="$GIT_SVN_LC_ALL"
-export LC_ALL
+name="commit with UTF-8 message: locale: $GIT_LC_UTF8"
test_expect_success UTF8 "$name" "
- echo '# hello' >> exec-2.sh &&
- git update-index exec-2.sh &&
- git commit -m 'éï∏' &&
- git svn set-tree HEAD"
-unset LC_ALL
+ LC_ALL=$GIT_LC_UTF8 echo '# hello' >> exec-2.sh &&
+ LC_ALL=$GIT_LC_UTF8 git update-index exec-2.sh &&
+ LC_ALL=$GIT_LC_UTF8 git commit -m 'éï∏' &&
+ LC_ALL=$GIT_LC_UTF8 git svn set-tree HEAD"
name='test fetch functionality (svn => git) with alternate GIT_SVN_ID'
GIT_SVN_ID=alt
diff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh
index 8cfdfe7..f3bbde4 100755
--- a/t/t9129-git-svn-i18n-commitencoding.sh
+++ b/t/t9129-git-svn-i18n-commitencoding.sh
@@ -14,22 +14,15 @@ compare_git_head_with () {
test_cmp current "$1"
}
-a_utf8_locale=$(locale -a | sed -n '/\.[uU][tT][fF]-*8$/{
- p
- q
-}')
-
-if test -n "$a_utf8_locale"
-then
- test_set_prereq UTF8
-else
+if ! test_utf8 ; then
say "# UTF-8 locale not available, some tests are skipped"
fi
compare_svn_head_with () {
# extract just the log message and strip out committer info.
# don't use --limit here since svn 1.1.x doesn't have it,
- LC_ALL="$a_utf8_locale" svn log `git svn info --url` | perl -w -e '
+ LC_ALL=$GIT_LC_UTF8 svn log `git svn info --url` | \
+ LC_ALL=$GIT_LC_UTF8 perl -w -e '
use bytes;
$/ = ("-"x72) . "\n";
my @x = <STDIN>;
--
1.7.3.4
^ permalink raw reply related
* [PATCH 2/4] test-lib.sh: add test_utf8() function
From: Yann Droneaud @ 2011-01-06 14:22 UTC (permalink / raw)
To: git; +Cc: Yann Droneaud
In-Reply-To: <cover.1294312018.git.yann@droneaud.fr>
test_utf8() checks for en_US.UTF-8 locale availability using lib-locale.pl.
The function returns 1 if a locale was not found, otherwise it returns 0,
set prereq UTF8 and export GIT_LC_UTF8 with the locale name.
If a test needs to use an en_US.UTF-8 locale, it has to call test_utf8() first.
Then it can do tests based on prereq UTF8 availability and use LC_ALL=$GIT_LC_UTF8.
Signed-off-by: Yann Droneaud <yann@droneaud.fr>
---
t/test-lib.sh | 14 ++++++++++++++
1 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh
index cb1ca97..3e92360 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1066,6 +1066,20 @@ case $(uname -s) in
;;
esac
+# check UTF-8 locale availability
+test_utf8 () {
+ if test_have_prereq PERL ; then
+ # output an en_US.UTF-8 locale compatible name
+ GIT_LC_UTF8=`$PERL_PATH $GIT_BUILD_DIR/t/lib-locale.pl`
+ fi
+ if test -z "$GIT_LC_UTF8" ; then
+ return 1
+ else
+ test_set_prereq UTF8
+ return 0
+ fi
+}
+
test -z "$NO_PERL" && test_set_prereq PERL
test -z "$NO_PYTHON" && test_set_prereq PYTHON
--
1.7.3.4
^ permalink raw reply related
* [PATCH/RFC 0/4] en_US.UTF-8 locale detection
From: Yann Droneaud @ 2011-01-06 14:22 UTC (permalink / raw)
To: git; +Cc: Yann Droneaud
In-Reply-To: <1274720888.4838.13.camel@localhost>
Following discussions[1] about test t9129 UTF-8 locale detection,
here's a generic rewrite of the en_US.UTF-8 locale detection
to be used with all other tests.
[1] http://thread.gmane.org/gmane.comp.version-control.git/147283
The proposed mechanism could work for system without "locale" command,
and would detect en_US.UTF-8 locale when named en_US.utf8 or some other
variations.
It must be tested on a wider range of systems (especially non-Linux).
Yann Droneaud (4):
test: added a library to detect an en_US.UTF-8 locale
test-lib.sh: added test_utf8() function
test: use test_utf8 and GIT_LC_UTF8 where an en_US.UTF-8 locale is
required
t9129: use "$PERL_PATH" instead of "perl"
t/lib-locale.pl | 167 ++++++++++++++++++++++++++++++++
t/t9100-git-svn-basic.sh | 25 ++---
t/t9129-git-svn-i18n-commitencoding.sh | 13 +--
t/test-lib.sh | 14 +++
4 files changed, 192 insertions(+), 27 deletions(-)
create mode 100755 t/lib-locale.pl
--
1.7.3.4
^ permalink raw reply
* [PATCH 1/4] test: add a library to detect an en_US.UTF-8 locale
From: Yann Droneaud @ 2011-01-06 14:22 UTC (permalink / raw)
To: git; +Cc: Yann Droneaud
In-Reply-To: <cover.1294312018.git.yann@droneaud.fr>
Since one can't rely on "locale" command availability nor its -a output,
perl script t/lib-locale.pl first use setlocale() and langinfo(CODESET)
to search for a working en_US.UTF-8 locale among many name variants.
If this fail, the script fallback to "locale" usage with two steps:
- try the "charmap" keyword, for example LC_ALL=en_US locale charmap
- then try "-a" option and match a pattern looking to the
locale names
Signed-off-by: Yann Droneaud <yann@droneaud.fr>
---
t/lib-locale.pl | 167 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 167 insertions(+), 0 deletions(-)
create mode 100755 t/lib-locale.pl
diff --git a/t/lib-locale.pl b/t/lib-locale.pl
new file mode 100755
index 0000000..33d47a5
--- /dev/null
+++ b/t/lib-locale.pl
@@ -0,0 +1,167 @@
+#!/usr/bin/env perl
+
+#
+# Copyright (c) 2011 Yann Droneaud <yann@droneaud.fr>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/ .
+#
+
+use strict;
+if ($] > 5.004) {
+ use warnings; # unless $] < 5.0004;
+}
+use POSIX;
+
+# according to manpage: setlocale can be used starting from perl 5.004
+# starting from May 1997.
+require 5.004;
+
+# it can also be tested with perl -V:d_setlocale
+# which must return something like "d_setlocale='define';"
+
+my $default_locale = setlocale(LC_ALL);
+
+# try a locale and return its codeset if available
+sub locale_get_codeset
+{
+ my ($locale) = @_;
+ my $codeset;
+ my $ret;
+
+ $ret = setlocale(LC_ALL, $locale);
+ if (!defined($ret)) {
+ # print "can't set locale $locale\n";
+ return undef;
+ }
+
+ #
+ # I18N::Langinfo is not available everywhere
+ # use the trick from perldoc I18N::Langinfo
+ #
+ $codeset = eval {
+ require I18N::Langinfo;
+ I18N::Langinfo->import(qw(langinfo CODESET));
+ langinfo(CODESET()); # note the ()
+ };
+
+ if (defined($@) && $@) {
+ print STDERR "error while testing codeset:\n $@";
+ return undef;
+ }
+
+ # restore locale
+ setlocale(LC_ALL, $default_locale);
+
+ return $codeset;
+}
+
+# check that given codeset match UTF-8
+sub codeset_check
+{
+ my ($codeset) = @_;
+
+ if (defined($codeset) && $codeset =~ /^UTF[_-]?8$/i) {
+ return 1;
+ }
+
+ return 0;
+}
+
+# try all those locales
+# require en_US with UTF-8
+my @locales = ( "en_US.UTF-8",
+ "en_US.utf8",
+ "en_US.UTF8",
+ "en_US.utf-8",
+ "en_US.UTF_8",
+ "en_US.utf_8",
+ "en_US" );
+
+my $locale;
+
+my $codeset;
+
+#
+# Check locale from within perl
+#
+foreach $locale (@locales) {
+
+ $codeset = locale_get_codeset($locale);
+
+ if (codeset_check($codeset)) {
+ print $locale, "\n";
+ exit 0;
+ }
+}
+
+#
+# if 'locale' command is available, test 'locale charmap'
+#
+foreach $locale (@locales) {
+
+ $codeset = `LC_ALL=$locale locale charmap 2>/dev/null`;
+
+ # if any error, skip
+ if ($!) {
+ last;
+ }
+
+ if (codeset_check($codeset)) {
+ print $locale, "\n";
+ exit 0;
+ }
+}
+
+#
+# try to execute "locale -a" command
+# this command is not always available and
+# output format is not normalized
+#
+use IPC::Open3;
+use File::Spec;
+
+open(NULLR, "<", File::Spec->devnull) || die "Can't open devnull: $!";
+open(NULLW, ">", File::Spec->devnull) || die "Can't open devnull: $!";
+
+my $pid = open3("<&NULLR", \*LOCALES, ">&NULLW" , "locale", "-a") || die "Can't launch locale -a: $!";
+
+while(<LOCALES>) {
+ chomp;
+ if (/(en_US\.([\w-]+))/i) {
+ if (codeset_check($2)) {
+ $locale = $1;
+ last;
+ }
+ }
+}
+
+waitpid($pid, 0);
+
+my $retcode = $?;
+
+close(LOCALES);
+
+if ($retcode == 0) {
+ if (defined $locale) {
+ print "$locale\n";
+ exit 0;
+ }
+}
+
+#
+# Nothing available,
+# at last, return an error
+#
+
+exit 1;
--
1.7.3.4
^ permalink raw reply related
* Re: git mergetool broken when rerere active
From: Martin von Zweigbergk @ 2011-01-06 12:47 UTC (permalink / raw)
To: git
Cc: Magnus Baeck, Avery Pennarun, Jay Soffian, David Aguilar,
Junio C Hamano
In-Reply-To: <alpine.DEB.1.10.1101052119530.26654@debian>
On Wed, 5 Jan 2011, Martin von Zweigbergk wrote:
> (2) When running mergetool again after resolving some (or all)
> conflicts, so that some of the files have already been added to
> the index, mergetool will now print something like
>
> file1: file does not need merging
> Continue merging other unresolved paths (y/n) ?
>
The same problem should arise even the first time mergetool if run if
rerere autoupdate is active and has added some files to the index.
> I would like to have both the original properties in (1) and (2) back,
> i.e. being ready for commit once 'git mergetool' has been successfully
> completed, and having it ignore any files that have already been added
> to the index.
Well, if rerere.autoupdate is disabled, any files resolved by rerere
should not be touched by 'git mergetool' (this was the original
request that lead to the change), but they would not be added to the
index, so we would not be ready for commit. I take that part of my
request back. I guess it would be ok (or even necessary) if, after
resolving all the conflicts reported by 'git mergetool', the user
would still have to explicitly add any files resolved by rerere and
thus ignored by 'git mergetool'.
So I guess what I'm saying here, is that I think only the problem with
'delete/modify' conflicts should be fixed regarding issue (1).
^ permalink raw reply
* Re: False positives in git diff-index
From: Alexander Gladysh @ 2011-01-06 12:12 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20110105080802.GA15997@sigill.intra.peff.net>
Hi, Jeff,
Apologies for delay. Apparently, after all, the problem does not
reproduce each time (like 33% of the time, or there is some another
factor that I did not realized yet). And the build process is quite
lengthy, so I was not able to gather stats fast enough. But here they
are, below:
On Wed, Jan 5, 2011 at 08:08, Jeff King <peff@peff.net> wrote:
> On Wed, Jan 05, 2011 at 07:46:19AM +0000, Alexander Gladysh wrote:
>> ----> Rebuilding manifest...
>> Making manifest for .
>> Generating index.html for .
>> :100644 100644 483a7292436daecc9bea0ab265ee19d587b14298
>> 0000000000000000000000000000000000000000
>> M cluster/localhost-ag/rocks/index.html
>> :100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25
>> 0000000000000000000000000000000000000000
>> M cluster/localhost-ag/rocks/manifest
>> So, does that mean that HGFS violates consistency guarantees?
>
> Hmm. That could be the problem. It may not violate traditional
> consistency guarantees, but I wonder if it is returning slightly
> different stat information between the program runs. That would mean
> "git status" does an index refresh and puts some stat information in the
> index, but the followup "git diff-index" might see different stat
> information.
>
> That's just a theory, though. You might try the patch below:
<...>
----> Rebuilding manifest...
Making manifest for .
Generating index.html for .
changed (3): cluster/localhost-ag/rocks/index.html
changed (3): cluster/localhost-ag/rocks/manifest
changed (3): cluster/localhost-ag/rocks/index.html
changed (3): cluster/localhost-ag/rocks/manifest
changed (115): cluster/localhost-ag/versions/versions-current.lua
changed (115): cluster/localhost-ag/versions/versions-current.lua
changed (3): cluster/localhost-ag/rocks/index.html
changed (3): cluster/localhost-ag/rocks/manifest
:100644 100644 483a7292436daecc9bea0ab265ee19d587b14298
0000000000000000000000000000000000000000
M cluster/localhost-ag/rocks/index.html
:100644 100644 fcb9ff896fd1a1bd15663fa9be19b250789d4a25
0000000000000000000000000000000000000000
M cluster/localhost-ag/rocks/manifest
----> Comitting changed manifest...
changed (3): cluster/localhost-ag/rocks/index.html
changed (3): cluster/localhost-ag/rocks/index.html
changed (3): cluster/localhost-ag/rocks/manifest
changed (3): cluster/localhost-ag/rocks/manifest
changed (115): cluster/localhost-ag/versions/versions-current.lua
changed (115): cluster/localhost-ag/versions/versions-current.lua
9760438c65f7b0293459e622153f235434436ad6
changed (115): cluster/localhost-ag/versions/versions-current.lua
# On branch ag/git-debugging
# Changed but not updated:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# typechange: cluster/localhost-ag/versions/versions-current.lua
#
no changes added to commit (use "git add" and/or "git commit -a")
Hope this makes some sense,
Alexander.
^ permalink raw reply
* Re: git color in commit editor
From: Ævar Arnfjörð Bjarmason @ 2011-01-06 10:58 UTC (permalink / raw)
To: Pavan Savoy; +Cc: git
In-Reply-To: <AANLkTimMZaSW4ZUS60Vo38vyuq-PPqPhGdCVZjFzF2Gs@mail.gmail.com>
On Thu, Jan 6, 2011 at 07:42, Pavan Savoy <pavan_savoy@sify.com> wrote:
> I have git version git version 1.5.6.3, on 1 system and get the colors
> right during a git commit (commit editor being vim..)
FWIW here's how to get those colors under Emacs:
https://github.com/rafl/git-commit-mode
^ permalink raw reply
* Re: git color in commit editor
From: Matthijs Kooijman @ 2011-01-06 10:15 UTC (permalink / raw)
To: Pavan Savoy; +Cc: git
In-Reply-To: <AANLkTimMZaSW4ZUS60Vo38vyuq-PPqPhGdCVZjFzF2Gs@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 507 bytes --]
Hey Pavan,
> I have git version git version 1.5.6.3, on 1 system and get the colors
> right during a git commit (commit editor being vim..)
> On another machine with same vim version and same gitconfig and pretty
> much all the same settings except that the
> git version is git version 1.6.3.3, I don't have color in the commit editor...
Try the ":syntax on" command in your vim (without the quotes, with the
colon).
You can put it in your ~/.vimrc file as well (without the colon there).
Gr.
Matthijs
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: git color in commit editor
From: Thomas Rast @ 2011-01-06 10:06 UTC (permalink / raw)
To: Pavan Savoy; +Cc: git
In-Reply-To: <AANLkTimMZaSW4ZUS60Vo38vyuq-PPqPhGdCVZjFzF2Gs@mail.gmail.com>
Pavan Savoy wrote:
> On another machine with same vim version and same gitconfig and pretty
> much all the same settings except that the
> git version is git version 1.6.3.3, I don't have color in the commit editor...
>
> Moving to newer GIT versions didn't help either...
> What am I missing?
That's squarely in vim's domain, git does not (and cannot) impose any
colors on the editor. So you need to configure vim on the second
machine for the highlighting.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* git color in commit editor
From: Pavan Savoy @ 2011-01-06 6:42 UTC (permalink / raw)
To: git
I am not sure, If this is a right list, Is there a git users mailing
list too somewhere out there ?
Anyway the question is simple,
I have git version git version 1.5.6.3, on 1 system and get the colors
right during a git commit (commit editor being vim..)
On another machine with same vim version and same gitconfig and pretty
much all the same settings except that the
git version is git version 1.6.3.3, I don't have color in the commit editor...
Moving to newer GIT versions didn't help either...
What am I missing?
regards,
Pavan
^ permalink raw reply
* Re: Resumable clone/Gittorrent (again)
From: Nguyen Thai Ngoc Duy @ 2011-01-06 6:36 UTC (permalink / raw)
To: Maaartin-1; +Cc: git
In-Reply-To: <4D25385B.3010103@seznam.cz>
On Thu, Jan 6, 2011 at 10:34 AM, Maaartin-1 <grajcar1@seznam.cz> wrote:
> In theory, I could create many commits per seconds. I could create many
> unique paths per seconds, too. But I don't think it really happens. I do
> know no larger repository than git.git and I don't want to download it
> just to see how many commits, paths, and object it contains, but I'd
> suppose it's less than one million commits, which should be manageable,
> especially when commits get grouped together as I described below.
In pratice, commits are created every day in an active project. Paths
on the other hand are added less often (perhaps except webkit).
I've got some numbers:
- wine.git has 72k commits, 260k trees, 200k blobs, 12k paths
- git.git has 24k commits, 39k trees, 24k blobs, 2.7k paths
- linux-2.6.git has 160k commits, 760k trees, 442k blobs, 46k paths
Large repos are more interesting because small ones can be cloned with
git-clone.
Listing all those commits in linux-2.6.git takes 160k*20=3M (I suppose
compressing is useless because SHA-1 is random). A compressed listing
of those 46k paths takes 200k.
>> And commits depend on other commits so
>> you can't verify a commit until you have got all of its parents. That
>> does apply to file, but then this file chain does not interfere other
>> file chains.
>
> That's true, but the verification is something done locally on the
> client, it consumes no network traffic and no server resources, so I
> consider it to be cheap. I need less than half a minute (using only a
> single core) for verifying of the whole git.git repository (36 MB). This
> is no problem, even when it had to wait until the download finishes. I'm
> sure, the OP of [1] would be happy if he could wait for this.
The point is you need to fetch its parent commits first in order to
verify a commit. Fetching a whole commit is more expensive than a
file. So while you can fetch a few commit bases and request for packs
from those bases in parallel, the cost of initial commit bases will be
high.
> I see I didn't explain it clear enough (or am missing something
> completely). I know why the packs normally used by git can't be used for
> this purpose. Let me retry: Let's assume there's a commit chain
> A-B-C-D-E-F-..., the client has already commit B and requests commit F.
> It may send requests to up to 4 servers, asking for C, D, E, and F,
> respectively. The server being asked for E _creates_ a pack containing
> all the information needed to create E given _all of_ A, B, C, D. As
> base for any blob/whatever in E it may choose any blob contained in any
> of these commits. Of course, it may also choose a blob already packed in
> this pack. It may not choose any other blob, so any client having all
> ancestors of E can use the pack. Different server and/or program
> versions may create different packs for E, but all of them are
> _interchangeable_. Because of this, it makes sense to _store_ it for
> future reuse.
They are interchangeable as a whole, yes. But you cannot fetch half
the pack from server A and the other half from server B. You can try
to recover as many deltas as possible in a broken pack, but how do you
request a server to send the rest of the pack to you?
--
Duy
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox