* local clone performance @ 2006-07-26 23:38 Jon Smirl 2006-07-27 18:15 ` Petr Baudis 0 siblings, 1 reply; 3+ messages in thread From: Jon Smirl @ 2006-07-26 23:38 UTC (permalink / raw) To: git Comparing git-clone to cg-clone [jonsmirl@jonsmirl apps]$ time git-clone git foo ...... real 0m5.755s user 0m4.548s sys 0m0.512s [jonsmirl@jonsmirl apps]$ time cg-clone git foo .... real 0m18.970s user 0m10.737s sys 0m1.392s Why does it take cg clone so long to set up the missing tags? git-clone is over 3x faster. -- Jon Smirl jonsmirl@gmail.com ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: local clone performance 2006-07-26 23:38 local clone performance Jon Smirl @ 2006-07-27 18:15 ` Petr Baudis 2006-07-27 18:36 ` Jon Smirl 0 siblings, 1 reply; 3+ messages in thread From: Petr Baudis @ 2006-07-27 18:15 UTC (permalink / raw) To: Jon Smirl; +Cc: git Dear diary, on Thu, Jul 27, 2006 at 01:38:00AM CEST, I got a letter where Jon Smirl <jonsmirl@gmail.com> said that... > Comparing git-clone to cg-clone > > [jonsmirl@jonsmirl apps]$ time git-clone git foo > ...... > real 0m5.755s > user 0m4.548s > sys 0m0.512s > > [jonsmirl@jonsmirl apps]$ time cg-clone git foo > .... > real 0m18.970s > user 0m10.737s > sys 0m1.392s > > Why does it take cg clone so long to set up the missing tags? > git-clone is over 3x faster. How many tags do you have? It's true that cg-clone does not scale very well with big amounts of tags, because it won't fetch tagged objects you wouldn't get otherwise (so if you tagged some random huge blob or a history line which is not part of any branch you have, cg-clone won't grab it all). This unfortunately costs us one special fetch invocation per tag; I will look into hacking git-local-fetch to accept multiple commit ids at once, that should speed it up considerably. That said, with cg-clone -a I will be able to cut that difference entirely and blindly take everything from the other repository. Could you please try the patch below and check if it has at least any measurable impact on the performance at all? Thanks. diff --git a/cg-fetch b/cg-fetch index a6e6959..23e0e77 100755 --- a/cg-fetch +++ b/cg-fetch @@ -54,7 +54,6 @@ _git_wc_unneeded=1 fetch_progress() { - [ $verbose -ge 2 ] && exec cat if [ -t 1 ]; then exec "${COGITO_LIB}"cg-Xfetchprogress "$_git_objects" else @@ -166,7 +165,11 @@ fetch_http() { whead= [ "$3" ] && whead="-w $3" - (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress + if [ $verbose -ge 2 ]; then + git-http-fetch -a -v $whead $recovery "$1" "$2/" + else + (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress + fi return ${PIPESTATUS[0]} } @@ -197,7 +200,11 @@ fetch_local() { whead= [ "$3" ] && whead="-w $3" - (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress + if [ $verbose -ge 2 ]; then + git-local-fetch -a -l -v $whead $recovery "$1" "$2" + else + (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress + fi return ${PIPESTATUS[0]} } @@ -230,6 +237,7 @@ fetch_tags() # if so, fetch the tag -- which should be # a cheap operation -- to complete the chain. echo -n "Missing tag ${tagname#tags/}... " + local verbose=2 if $fetch "$tagname" "$uri" "$tagname" 2>/dev/null >&2; then echo "retrieved" else -- Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ Snow falling on Perl. White noise covering line noise. Hides all the bugs too. -- J. Putnam ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: local clone performance 2006-07-27 18:15 ` Petr Baudis @ 2006-07-27 18:36 ` Jon Smirl 0 siblings, 0 replies; 3+ messages in thread From: Jon Smirl @ 2006-07-27 18:36 UTC (permalink / raw) To: Petr Baudis; +Cc: git On 7/27/06, Petr Baudis <pasky@suse.cz> wrote: > Dear diary, on Thu, Jul 27, 2006 at 01:38:00AM CEST, I got a letter > where Jon Smirl <jonsmirl@gmail.com> said that... > > Comparing git-clone to cg-clone > > > > [jonsmirl@jonsmirl apps]$ time git-clone git foo > > ...... > > real 0m5.755s > > user 0m4.548s > > sys 0m0.512s > > > > [jonsmirl@jonsmirl apps]$ time cg-clone git foo > > .... > > real 0m18.970s > > user 0m10.737s > > sys 0m1.392s > > > > Why does it take cg clone so long to set up the missing tags? > > git-clone is over 3x faster. > > How many tags do you have? I am cloning the git source tree. It looks to have about 40 tags. > It's true that cg-clone does not scale very well with big amounts of > tags, because it won't fetch tagged objects you wouldn't get otherwise > (so if you tagged some random huge blob or a history line which is not > part of any branch you have, cg-clone won't grab it all). This > unfortunately costs us one special fetch invocation per tag; I will look > into hacking git-local-fetch to accept multiple commit ids at once, that > should speed it up considerably. > > That said, with cg-clone -a I will be able to cut that difference > entirely and blindly take everything from the other repository. > > Could you please try the patch below and check if it has at least > any measurable impact on the performance at all? Thanks. New timings real 0m11.259s user 0m9.977s sys 0m1.280s That's a 40% improvement. > diff --git a/cg-fetch b/cg-fetch > index a6e6959..23e0e77 100755 > --- a/cg-fetch > +++ b/cg-fetch > @@ -54,7 +54,6 @@ _git_wc_unneeded=1 > > fetch_progress() > { > - [ $verbose -ge 2 ] && exec cat > if [ -t 1 ]; then > exec "${COGITO_LIB}"cg-Xfetchprogress "$_git_objects" > else > @@ -166,7 +165,11 @@ fetch_http() > { > whead= > [ "$3" ] && whead="-w $3" > - (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress > + if [ $verbose -ge 2 ]; then > + git-http-fetch -a -v $whead $recovery "$1" "$2/" > + else > + (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress > + fi > return ${PIPESTATUS[0]} > } > > @@ -197,7 +200,11 @@ fetch_local() > { > whead= > [ "$3" ] && whead="-w $3" > - (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress > + if [ $verbose -ge 2 ]; then > + git-local-fetch -a -l -v $whead $recovery "$1" "$2" > + else > + (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress > + fi > return ${PIPESTATUS[0]} > } > > @@ -230,6 +237,7 @@ fetch_tags() > # if so, fetch the tag -- which should be > # a cheap operation -- to complete the chain. > echo -n "Missing tag ${tagname#tags/}... " > + local verbose=2 > if $fetch "$tagname" "$uri" "$tagname" 2>/dev/null >&2; then > echo "retrieved" > else > > > -- > Petr "Pasky" Baudis > Stuff: http://pasky.or.cz/ > Snow falling on Perl. White noise covering line noise. > Hides all the bugs too. -- J. Putnam > -- Jon Smirl jonsmirl@gmail.com ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2006-07-27 18:36 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2006-07-26 23:38 local clone performance Jon Smirl 2006-07-27 18:15 ` Petr Baudis 2006-07-27 18:36 ` Jon Smirl
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).