* Re: Git benchmarks at OpenOffice.org wiki
From: Jan Holesovsky @ 2007-05-02 14:41 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Jakub Narebski, git, releases
In-Reply-To: <Pine.LNX.4.64.0705020143460.4010@racer.site>
Hi Johannes,
On Wednesday 02 May 2007 12:24, Johannes Schindelin wrote:
> On Tue, 1 May 2007, Jakub Narebski wrote:
> > 'Checkout time' (which should be renamed to 'Initial checkout time'),
> > in which git also loses with 130 minutes (Linux, 2MBit DSL) [from
> > go-oo.org], 100min (Linux, 2MBit DSL, Wireless, no proxy) [from
> > go-oo.org] versus 117 minutes (Linux, 2MBit DSL), 26 minutes (Linux,
> > 2MBit DSL, with compression (-z 6)) for CVS, and 60 Minutes (Windows,
> > 34Mbit Line) for Subversion, would also be helped by the above.
>
> FWIW I can confirm the number "100min".
>
> Something I realized with pain is that the refs/ directory is 24MB big.
> Yep. Really. They have 3464 heads and 2639 tags. I suspect that this is
> the reason why.
I should probably produce even a tree where would be the merged branches
deleted, right...
Regards,
Jan
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Jan Holesovsky @ 2007-05-02 14:37 UTC (permalink / raw)
To: Andy Parkins; +Cc: git, Julian Phillips, Jakub Narebski
In-Reply-To: <200705021158.04481.andyparkins@gmail.com>
Hi Andy,
On Wednesday 02 May 2007 12:58, Andy Parkins wrote:
> On Wednesday 2007 May 02, Julian Phillips wrote:
> > A fully packed clone of the OOo git repo was indeed 1.3G, and the entrire
> > checkout + repo was indeed 8.5G (using git 1.5.1.2).
>
> I'm more confused now then. I assumed the figures were accurate, but they
> cannot be:
>
> CVS git SVN
> Size of data on the server 8.5G 1.3G n/a
> Size of checkout 1.4G 2.8G 1.5G
>
> I don't doubt the 1.3G on the server - and assume that is fully packed.
> The checkout sizes are suspicious though. Is that 2.8G packed?
> - If it is, then we can deduce that this is a repo+source size, since the
> server is packed size+0 therefore the size of the source tree is
> 2.8G - 1.3G = 1.5G
> In which case the other figures are wrong:
> - CVS checkout is 1.4G - impossible, the source tree is 1.5G. And where
> is the overhead of the CVS directories which would make it more than 1.5G?
Unfortunately I don't have the _exact_ numbers here any more so I cannot prove
it ;-) - but this is a rounding problem [CVS checkout is slightly more than
1.4G]. Similarly, overhead of of CVS directories is 0 when we count in
gigabytes.
> - SVN checkout overhead is always _at least_ the size of the source tree
> because it keeps a pristine copy of HEAD. If the source tree is 1.5G, then
> this figure should be at least 3G.
Yes, this surprises me as well. I've heard about some improvements in the
recent SVN, but 0.1M sounds very small.
> - If it is not, then we're back to "I don't believe that git was packed"
It was, IIRC with 'git-repack -a -d -f'.
> Something smells fishy here - either the source tree size is included in
> some, but not in others or the git repository wasn't packed.
As I wrote, I am looking forward to seeing the SVN tree myself for further
testing.
Regards,
Jan
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Johannes Schindelin @ 2007-05-02 14:35 UTC (permalink / raw)
To: Jan Holesovsky; +Cc: Jakub Narebski, git, releases
In-Reply-To: <200705021624.25560.kendy@suse.cz>
Hi,
On Wed, 2 May 2007, Jan Holesovsky wrote:
> On Tuesday 01 May 2007 23:46, Jakub Narebski wrote:
>
> > What I am concerned about is some of git benchmark results at Git page
> > on OpenOffice.org wiki:
> > http://wiki.services.openoffice.org/wiki/Git#Comparison
> > Actually it is comparison with CVS and Subversion, although most
> > benchmarks are done only for git.
>
> I did the git numbers, so if they are wrong - blame me :-)
Good to have you here!
> > 'Checkout time' (which should be renamed to 'Initial checkout time'),
> > in which git also loses with 130 minutes (Linux, 2MBit DSL) [from
> > go-oo.org], 100min (Linux, 2MBit DSL, Wireless, no proxy) [from
> > go-oo.org] versus 117 minutes (Linux, 2MBit DSL), 26 minutes (Linux,
> > 2MBit DSL, with compression (-z 6)) for CVS, and 60 Minutes (Windows,
> > 34Mbit Line) for Subversion, would also be helped by the above.
>
> Good point, and I already changed the page in the morning. I also added the
> checkout time that I got over a fast line [it was 44min].
It took me longer here, but the reason might be that my "local" repository
is on NFS, due to quota on the machine.
> > What I'm really concerned about is branch switch and merging branches,
> > when one of the branches is an old one (e.g. unxsplash branch), which
> > takes 3min (!) according to the benchmark. 13-25sec for commit is also
> > bit long, but BRANCH SWITCHING which takes 3 MINUTES!? There is no
> > comparison benchmark for CVS or Subversion, though...
I imagine that might be related to the vast amount of remote branches.
IIRC we do not pack them with git-gc, and ext3 is not that good with big
directories (remember: 3464 branches!).
Maybe oprofile knows a bit more where the hotspots are.
Ciao,
Dscho
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Julian Phillips @ 2007-05-02 14:28 UTC (permalink / raw)
To: Andy Parkins; +Cc: git, Jakub Narebski, Jan Holesovsky
In-Reply-To: <200705021158.04481.andyparkins@gmail.com>
On Wed, 2 May 2007, Andy Parkins wrote:
> On Wednesday 2007 May 02, Julian Phillips wrote:
>
>> A fully packed clone of the OOo git repo was indeed 1.3G, and the entrire
>> checkout + repo was indeed 8.5G (using git 1.5.1.2).
oops, meant 2.7G not 8.5G there ... sorry, was working from memory.
jp3@electron: ooo(unxsplash)>du -sh .git
1.3G .git
jp3@electron: ooo(unxsplash)>du -sh .
2.7G .
jp3@electron: ooo(unxsplash)>ls .git/objects/
info pack
> I'm more confused now then. I assumed the figures were accurate, but they
> cannot be:
>
> CVS git SVN
> Size of data on the server 8.5G 1.3G n/a
> Size of checkout 1.4G 2.8G 1.5G
>
> I don't doubt the 1.3G on the server - and assume that is fully packed. The
> checkout sizes are suspicious though. Is that 2.8G packed?
> - If it is, then we can deduce that this is a repo+source size, since the
> server is packed size+0 therefore the size of the source tree is
> 2.8G - 1.3G = 1.5G
the difference between 2.7G and 2.8G may be due to filesystem difference?
> In which case the other figures are wrong:
> - CVS checkout is 1.4G - impossible, the source tree is 1.5G. And where is
> the overhead of the CVS directories which would make it more than 1.5G?
> - SVN checkout overhead is always _at least_ the size of the source tree
> because it keeps a pristine copy of HEAD. If the source tree is 1.5G,
> then this figure should be at least 3G.
I was wondering about the subversion figures too ...
> - If it is not, then we're back to "I don't believe that git was packed"
>
> Something smells fishy here - either the source tree size is included in some,
> but not in others or the git repository wasn't packed.
1.3G is the packed size ...
jp3@electron: ooo(unxsplash)>ls -sh .git/objects/pack/
total 1.3G
37M pack-87efcac9bcb117328e8a1b0c1b42c88c3603c5b7.idx
1.2G pack-87efcac9bcb117328e8a1b0c1b42c88c3603c5b7.pack
--
Julian
---
To err is humor.
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Jan Holesovsky @ 2007-05-02 14:24 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, releases
In-Reply-To: <200705012346.14997.jnareb@gmail.com>
Hi Jakub,
On Tuesday 01 May 2007 23:46, Jakub Narebski wrote:
> OpenOffice.org is looking for a new SCM (Software Configuration
> Management) tool, or at least was on Friday, 19 Jan 2007;
> see: http://blogs.sun.com/GullFOSS/entry/openoffice_org_scm
>
> One of the SCMs considered is Git. One of others is Subversion.
> There is a functional git tree with the entire OOo history for testing
> purposes that can be found at: http://go-oo.org/git.
>
> What I am concerned about is some of git benchmark results at Git page
> on OpenOffice.org wiki:
> http://wiki.services.openoffice.org/wiki/Git#Comparison
> Actually it is comparison with CVS and Subversion, although most
> benchmarks are done only for git.
I did the git numbers, so if they are wrong - blame me :-) I am also curious
about the SVN numbers, because the SVN conversion [from my point of view]
cheats a lot. From what I know, it does not contain the historical branches
(yes, the >3000 of them that are in the git tree), and if I understood that
correctly, instead of history in the branches, they commit just
'integration commits' [one commit for all the changes in the branch] which
breaks 'svn blame' completely.
Unfortunately, I did not have a chance to try the SVN tree yet to see it
myself to prove this true or false :-(
> In 'Size of data on the server' git has CVS beat hands down: 1.3G vs
> 8.5G for sources, 591M vs 1.1G for third party. I think it is similar
> for Subversion. I hope that repository is fully packed: IIRC the Mozilla
> CVS repository import was about 0.6GB pack file, not 1.3GB.
>
> The problem is with 'Size of checkout': to start working in repository
> one needs 1.4G (sources) and 98M (third party) for CVS checkout (it is
> 1.5G for sources for Subversion checkout). Ordinary for distributed SCM
> you would need size of repository + size of sources (working area),
> which is 2.8G for sources and 688M for third party stuff files you can
> hack on + the history]. This makes some prefer to go centralized SCM
> route, i.e. Subversion as replacement for CVS (+ CWS, ChildWorkSpace).
Considering the size OOo needs for build (>8G without languages),
the ~1.4G overhead for history is very well bearable. I am surprised about
the 100M overhead for SVN as well - from my experience it is usually about
the size of the project itself; but maybe they improved something in SVN
in the meantime.
> What might help here is splitting repository into current (e.g. from
> OOo 2.0) and historical part,
No, I don't want this ;-)
> and / or using shallow clone. Implementing
> partial checkouts, i.e. checking out only part of working area (and
> using 'theirs' strategy for merging not-checked-out part for merges)
> would help. Splitting repository into submodules, and submodule
> support -- it depends on organization of OOo sources, would certainly
> help for third party stuff repository.
We should better split the OOo sources; it's a process that already started
[UNO runtime environment vs. OOo without URE], and I proposed some more
changes already.
> 'Checkout time' (which should be renamed to 'Initial checkout time'),
> in which git also loses with 130 minutes (Linux, 2MBit DSL) [from
> go-oo.org], 100min (Linux, 2MBit DSL, Wireless, no proxy) [from
> go-oo.org] versus 117 minutes (Linux, 2MBit DSL), 26 minutes (Linux,
> 2MBit DSL, with compression (-z 6)) for CVS, and 60 Minutes (Windows,
> 34Mbit Line) for Subversion, would also be helped by the above.
Good point, and I already changed the page in the morning. I also added the
checkout time that I got over a fast line [it was 44min].
> What I'm really concerned about is branch switch and merging branches,
> when one of the branches is an old one (e.g. unxsplash branch), which
> takes 3min (!) according to the benchmark. 13-25sec for commit is also
> bit long, but BRANCH SWITCHING which takes 3 MINUTES!? There is no
> comparison benchmark for CVS or Subversion, though...
I am really curious about the SVN tree. As I said, I did not see it yet.
There is just some info about it here:
http://wiki.services.openoffice.org/wiki/SVNMigration, but I cannot check it
now, the Wiki is down :-(
> Comparison / benchmark lacks some crucial info, like what computer was
> used (CPU, RAM, HDD), what filesystem was used, git version etc. It
> does have commands used for tests (benchmarks).
For the git tests, it was:
CPU: AMD Athlon(tm) 64 Processor 3200+
RAM: 1G RAM
Disk (info from bonnie):
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
one 1*2000 37819 77.6 44296 16.8 16982 5.1 35203 63.9 45915 6.6 152.4 0.4
Filesystem: ext3
> Could you confirm (or deny) those results? go-oo.org uses git 1.4.3.4;
> was there some improvement or bugfix related to the speed of checkout?
Regards,
Jan
^ permalink raw reply
* Re: [PATCH] Makefile: use --unsafe option under Cygwin with asciidoc
From: Eric Blake @ 2007-05-02 13:06 UTC (permalink / raw)
To: Jari Aalto; +Cc: git
In-Reply-To: <ejlze97d.fsf@cante.net>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
According to Jari Aalto on 5/2/2007 5:49 AM:
> New: variable ASCIIDOC_FLAGS was introduced. It was also added to two
> new targets. The old ASCIIDOC_EXTRA is set to --unsafe under Cygwin to
> ignore asciidoc error about unsafe include.
This isn't just for cygwin, since it benefits any platform where the
installed asciidoc is 8.1 or better (asciidoc is currently at 8.2.1 if you
build the from tarballs).
- --
Don't work too hard, make some time for fun as well!
Eric Blake ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGOIy+84KuGfSFAYARAo/XAKCuKOXb7cYzPL0AU4oiz5VbS54mtwCfVzX/
bTMHegP6uvmaFZB5pKL5Dxo=
=QYTe
-----END PGP SIGNATURE-----
^ permalink raw reply
* Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>
From: Karl Hasselström @ 2007-05-02 13:10 UTC (permalink / raw)
To: Marco Costalba; +Cc: Catalin Marinas, git
In-Reply-To: <e5bfff550705020417h14f6297fu957eb64e58119770@mail.gmail.com>
On 2007-05-02 13:17:01 +0200, Marco Costalba wrote:
> On 5/2/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > It's not near the top of my kill list by any stretch of the
> > imagination, so no need to worry. And even if it were, Catalin
> > would certainly stand as a wall of sanity between qgit and my
> > chain saw. :-)
>
> Currently I check for the existence of <git dir>/patches directory
> as a quick exit in case a repository does NOT have a StGIT repo on
> it (the common case).
>
> This avoids a costly and 99% of cases not needed 'stg <something>'
> call.
>
> I ask if it will be still a safe check in the long period or it is
> better to change the check to something else (as existence of
> <git dir>/refs/patches) instead ?
I personally have no plan to attempt to remove all of .git/patches --
a lot of data in there is redundant, but certainly not all of it, and
there's no compelling reason to move it. But Catalin has the final
word, of course.
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
* [PATCH] http-fetch: Disable use of curl multi support for libcurl < 7.16.
From: Alexandre Julliard @ 2007-05-02 12:53 UTC (permalink / raw)
To: git
curl_multi_remove_handle() is broken in libcurl < 7.16, in that it
doesn't correctly update the active handles count when a request is
aborted. This causes the transfer to hang forever waiting for the
handle count to become less than the number of active requests.
Signed-off-by: Alexandre Julliard <julliard@winehq.org>
---
http.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/http.h b/http.h
index 324fcf4..69b6b66 100644
--- a/http.h
+++ b/http.h
@@ -6,7 +6,7 @@
#include <curl/curl.h>
#include <curl/easy.h>
-#if LIBCURL_VERSION_NUM >= 0x070908
+#if LIBCURL_VERSION_NUM >= 0x071000
#define USE_CURL_MULTI
#define DEFAULT_MAX_REQUESTS 5
#endif
--
1.5.2.rc1.4.g8c87-dirty
--
Alexandre Julliard
julliard@winehq.org
^ permalink raw reply related
* Re: problem using git on cygwin
From: Jari Aalto @ 2007-05-02 11:55 UTC (permalink / raw)
To: git
In-Reply-To: <ace3f33d0705012104r3cf0b99ayb2ec2a69833e6ea1@mail.gmail.com>
"srinivas naga vutukuri" <srinivas.vutukuri@gmail.com> writes:
> Hi,
>
> On 4/30/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>> Hi,
>>
>> On Mon, 30 Apr 2007, srinivas naga vutukuri wrote:
>>
>> > $ git add project/test.c
>> > error: bad index file sha1 signature
>> > fatal: index file corrupt
>>
>> I could imagine that you do not have "binary mode" default for opening
>> files.
>
> i do't think it is in binary mode when opening the files. I am
> used vim to create a new file and editing through that. And also am
> used file format (set ff=unix).
I think Johannes meant Cygwn filesystem mount attributes. Please run
$ mount
And you should see "binmode". Here are examples:
D:\cygwin on / type system (binmode)
D:\cygwin\bin on /usr/bin type system (binmode)
D:\cygwin\lib on /usr/lib type system (binmode)
E:\data\src on /usr/src type system (binmode)
If not, you need to remount directories with "-b" option. See man
mount(1)
Jari
^ permalink raw reply
* [PATCH] Makefile: use --unsafe option under Cygwin with asciidoc
From: Jari Aalto @ 2007-05-02 11:49 UTC (permalink / raw)
To: git
New: variable ASCIIDOC_FLAGS was introduced. It was also added to two
new targets. The old ASCIIDOC_EXTRA is set to --unsafe under Cygwin to
ignore asciidoc error about unsafe include.
Signed-off-by: Jari Aalto <jari.aalto@cante.net>
---
Documentation/Makefile | 18 +++++++++++++-----
1 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 3f92783..399ff12 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -37,6 +37,14 @@ man7dir=$(mandir)/man7
ASCIIDOC=asciidoc
ASCIIDOC_EXTRA =
+
+ifneq (,$(CYGWIN))
+# Ignore 'include' call errors
+ASCIIDOC_EXTRA = --unsafe
+endif
+
+ASCIIDOC_FLAGS += $(ASCIIDOC_EXTRA)
+
INSTALL?=install
DOC_REF = origin/man
@@ -112,7 +120,7 @@ clean:
%.html : %.txt
rm -f $@+ $@
$(ASCIIDOC) -b xhtml11 -d manpage -f asciidoc.conf \
- $(ASCIIDOC_EXTRA) -o - $< | \
+ $(ASCIIDOC_FLAGS) -o - $< | \
sed -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' >$@+
mv $@+ $@
@@ -122,12 +130,12 @@ clean:
%.xml : %.txt
rm -f $@+ $@
$(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
- $(ASCIIDOC_EXTRA) -o - $< | \
+ $(ASCIIDOC_FLAGS) -o - $< | \
sed -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' >$@+
mv $@+ $@
user-manual.xml: user-manual.txt user-manual.conf
- $(ASCIIDOC) -b docbook -d book $<
+ $(ASCIIDOC) -b docbook -d book $(ASCIIDOC_FLAGS) $<
XSLT = http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl
XSLTOPTS = --xinclude --stringparam html.stylesheet docbook-xsl.css
@@ -141,13 +149,13 @@ howto-index.txt: howto-index.sh $(wildcard howto/*.txt)
mv $@+ $@
$(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt
- $(ASCIIDOC) -b xhtml11 $*.txt
+ $(ASCIIDOC) -b xhtml11 $(ASCIIDOC_FLAGS) $*.txt
WEBDOC_DEST = /pub/software/scm/git/docs
$(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
rm -f $@+ $@
- sed -e '1,/^$$/d' $< | $(ASCIIDOC) -b xhtml11 - >$@+
+ sed -e '1,/^$$/d' $< | $(ASCIIDOC) -b xhtml11 $(ASCIIDOC_FLAGS) - >$@+
mv $@+ $@
install-webdoc : html
--
1.5.1.3
^ permalink raw reply related
* Re: problem using git on cygwin
From: Alex Riesen @ 2007-05-02 12:37 UTC (permalink / raw)
To: srinivas naga vutukuri; +Cc: Johannes Schindelin, git
In-Reply-To: <ace3f33d0705012104r3cf0b99ayb2ec2a69833e6ea1@mail.gmail.com>
On 5/2/07, srinivas naga vutukuri <srinivas.vutukuri@gmail.com> wrote:
> > On Mon, 30 Apr 2007, srinivas naga vutukuri wrote:
> >
> > > $ git add project/test.c
> > > error: bad index file sha1 signature
> > > fatal: index file corrupt
> >
> > I could imagine that you do not have "binary mode" default for opening
> > files.
> >
> i do't think it is in binary mode when opening the files. I am
> used vim to create a new file and editing through that. And also am
> used file format (set ff=unix).
Johannes was referring to binary mode for writing files for _cygwin_
itself, not for files the git has to manage. See "Default Text File Type"
in cygwins setup.exe (maybe even follow the link "Read more...").
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Jakub Narebski @ 2007-05-02 11:33 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, dev, Jan Holesovsky
In-Reply-To: <Pine.LNX.4.64.0705020143460.4010@racer.site>
Hi!
On 5/2/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Tue, 1 May 2007, Jakub Narebski wrote:
>
> > 'Checkout time' (which should be renamed to 'Initial checkout time'),
> > in which git also loses with 130 minutes (Linux, 2MBit DSL) [from
> > go-oo.org], 100min (Linux, 2MBit DSL, Wireless, no proxy) [from
> > go-oo.org] versus 117 minutes (Linux, 2MBit DSL), 26 minutes (Linux,
> > 2MBit DSL, with compression (-z 6)) for CVS, and 60 Minutes (Windows,
> > 34Mbit Line) for Subversion, would also be helped by the above.
>
> FWIW I can confirm the number "100min".
>
> Something I realized with pain is that the refs/ directory is 24MB big.
> Yep. Really. They have 3464 heads and 2639 tags. I suspect that this is
> the reason why.
Then packed refs would certainly help with speed and a bit with size.
--
Jakub Narebski
^ permalink raw reply
* Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>
From: Marco Costalba @ 2007-05-02 11:17 UTC (permalink / raw)
To: Karl Hasselström; +Cc: Catalin Marinas, git
In-Reply-To: <20070502065054.GA9919@diana.vm.bytemark.co.uk>
On 5/2/07, Karl Hasselström <kha@treskal.com> wrote:
>
> It's not near the top of my kill list by any stretch of the
> imagination, so no need to worry. And even if it were, Catalin would
> certainly stand as a wall of sanity between qgit and my chain saw. :-)
>
Currently I check for the existence of <git dir>/patches directory as
a quick exit in case a repository does NOT have a StGIT repo on it
(the common case).
This avoids a costly and 99% of cases not needed 'stg <something>' call.
I ask if it will be still a safe check in the long period or it is
better to change the check to something else (as existence of <git
dir>/refs/patches) instead ?
Thanks
Marco
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Andy Parkins @ 2007-05-02 10:58 UTC (permalink / raw)
To: git; +Cc: Julian Phillips, Jakub Narebski, Jan Holesovsky
In-Reply-To: <Pine.LNX.4.64.0705021046230.2425@reaper.quantumfyre.co.uk>
On Wednesday 2007 May 02, Julian Phillips wrote:
> A fully packed clone of the OOo git repo was indeed 1.3G, and the entrire
> checkout + repo was indeed 8.5G (using git 1.5.1.2).
I'm more confused now then. I assumed the figures were accurate, but they
cannot be:
CVS git SVN
Size of data on the server 8.5G 1.3G n/a
Size of checkout 1.4G 2.8G 1.5G
I don't doubt the 1.3G on the server - and assume that is fully packed. The
checkout sizes are suspicious though. Is that 2.8G packed?
- If it is, then we can deduce that this is a repo+source size, since the
server is packed size+0 therefore the size of the source tree is
2.8G - 1.3G = 1.5G
In which case the other figures are wrong:
- CVS checkout is 1.4G - impossible, the source tree is 1.5G. And where is
the overhead of the CVS directories which would make it more than 1.5G?
- SVN checkout overhead is always _at least_ the size of the source tree
because it keeps a pristine copy of HEAD. If the source tree is 1.5G,
then this figure should be at least 3G.
- If it is not, then we're back to "I don't believe that git was packed"
Something smells fishy here - either the source tree size is included in some,
but not in others or the git repository wasn't packed.
Andy
--
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Johannes Schindelin @ 2007-05-02 10:24 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git, releases, Jan Holesovsky
In-Reply-To: <200705012346.14997.jnareb@gmail.com>
Hi,
On Tue, 1 May 2007, Jakub Narebski wrote:
> 'Checkout time' (which should be renamed to 'Initial checkout time'),
> in which git also loses with 130 minutes (Linux, 2MBit DSL) [from
> go-oo.org], 100min (Linux, 2MBit DSL, Wireless, no proxy) [from
> go-oo.org] versus 117 minutes (Linux, 2MBit DSL), 26 minutes (Linux,
> 2MBit DSL, with compression (-z 6)) for CVS, and 60 Minutes (Windows,
> 34Mbit Line) for Subversion, would also be helped by the above.
FWIW I can confirm the number "100min".
Something I realized with pain is that the refs/ directory is 24MB big.
Yep. Really. They have 3464 heads and 2639 tags. I suspect that this is
the reason why.
Will play with it.
Ciao,
Dscho
^ permalink raw reply
* Re: FAQ Q: another version control system?
From: Johannes Schindelin @ 2007-05-02 10:08 UTC (permalink / raw)
To: Peter; +Cc: git
In-Reply-To: <9e7886190705020013k5721480aj767c1c8643fc8d23@mail.gmail.com>
On Wed, 2 May 2007, Peter wrote:
> Do we really need another version control system? As I understand it,
> this is like a distributed, central-server-less, CVS, sorry, SVN server.
It is obvious that you are a fresh subscriber, since the answer to your
question is given multiple times a day. Not like in "This is why you
cannot enhance SVN to behave as Git does", but more like in "This feature
of Git allows you to do [...]".
I can only invite you to sift through the mail archive. I _could_ point
you to a really recent mail from Linus, which explains very much why SVN
just cannot do -- however hard you try -- what Git can. But then, you
would miss out on so many other interesting emails which you haven't read
yet.
So, now it is up to you to actually invest some time and learn something.
Ciao,
Dscho
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Julian Phillips @ 2007-05-02 9:51 UTC (permalink / raw)
To: Andy Parkins; +Cc: git, Jakub Narebski, releases, Jan Holesovsky
In-Reply-To: <200705020955.04582.andyparkins@gmail.com>
On Wed, 2 May 2007, Andy Parkins wrote:
> On Tuesday 2007 May 01, Jakub Narebski wrote:
>
>> In 'Size of data on the server' git has CVS beat hands down: 1.3G vs
>> 8.5G for sources, 591M vs 1.1G for third party. I think it is similar
>> for Subversion. I hope that repository is fully packed: IIRC the Mozilla
>> CVS repository import was about 0.6GB pack file, not 1.3GB.
>
> I'm fairly sure it's not. If so that would also affect the speed of
> operations wouldn't it?
A fully packed clone of the OOo git repo was indeed 1.3G, and the entrire
checkout + repo was indeed 8.5G (using git 1.5.1.2).
Took about 46m to clone on a server with decent bandwith, ~5.5m user time,
~1.5m system.
>
> I also doubt the subversion checkout size - subversion keeps a pristine copy
> of the HEAD file - so a subversion checkout is usually over twice the size of
> the source tree.
>
>> takes 3min (!) according to the benchmark. 13-25sec for commit is also
>> bit long
>
> I wonder if they are measuring the time for the generation of the commit
> message or something? Or perhaps by using "git-commit -a" is causing a check
> of the whole tree for changed files?
>
>> Comparison / benchmark lacks some crucial info, like what computer was
>> used (CPU, RAM, HDD), what filesystem was used, git version etc. It
>> does have commands used for tests (benchmarks).
>
> I'd also like to see some of the numbers for the other systems, I tried to use
> subversion with the linux kernel once and got fed up waiting for it to do
> anything. I suspect the reason numbers aren't shown for the others is that
> they haven't finished yet :-)
>
>> Could you confirm (or deny) those results? go-oo.org uses git 1.4.3.4;
>> was there some improvement or bugfix related to the speed of checkout?
>
> Wasn't there a recent change that made repacking after a clone unnecessary?
> That would certainly reduce the checkout size.
Not from the numbers that are quoted it won't, they are fully packed
sizes.
--
Julian
---
"Do you have blacks, too?"
George W. Bush
To Brazilian president Fernando Cardoso
November 8, 2001
Washington, D.C.
^ permalink raw reply
* FFmpeg considering GIT
From: Panagiotis Issaris @ 2007-05-02 9:29 UTC (permalink / raw)
To: git
Hi,
Some of the people of the FFmpeg project are looking at both GIT and Mercurial
as possible replacements for the current Subversion repository. They have some
questions regarding the possibility of doing certain things, which I prefer not
to answer as I am not sure my answer would be correct :) Which is why I am
posting here...
The questions are stated in this e-mail [1]. One of the things that are being
discussed is the following action on a publicly mirrored repository:
git branch -m master dead_end
git branch -m last_good master
I'd think this would fail as people could have pulled from the repository while
the "dead_end" commit was already available, right?
There are some other things the FFmpeg maintainer mentions, namely:
* He wants to be able to revert a commit in some way without "wiping" history.
That is without committing a patch which reverses the broken commit, as this
would pollute "git blame". The maintainer sees this as critical feature for
switching to git as it apparently can be doing using Subversion:
"in svn we can do this with svn cp from a specific
revission git and mercurial lack proper copy support"
* And finally, he noticed that when copying files, history is sometimes lost
(mentioned at the bottom of [1]).
Any answers are greatly appreciated, as I'd really like to see FFmpeg switch to
GIT.
With friendly regards,
Takis
[1]
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/49673
[2]
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/49656
^ permalink raw reply
* Re: FAQ Q: another version control system?
From: Andy Parkins @ 2007-05-02 9:25 UTC (permalink / raw)
To: git; +Cc: Peter
In-Reply-To: <9e7886190705020013k5721480aj767c1c8643fc8d23@mail.gmail.com>
On Wednesday 2007 May 02, Peter wrote:
I suspect you are a troll as your questions have such obvious answers that
they hardly seem worth giving. Despite that, I'll answer them for the sake
of the googlers who stumble on your message.
> Do we really need another version control system? As I understand it,
> this is like a distributed, central-server-less, CVS, sorry, SVN
> server.
Why didn't the subversion guys just add their features to CVS? The subversion
guys are not stupid people - why haven't they added these features to
subversion? The answer is that they fundamentally don't fit. Distributed
version control is not just something you patch in later as a feature - it is
designed in from the beginning. Have a look at the monstrosity that is svk
(distributed version control with a subversion backend).
> But why isn't such a feature set added into SVN? Why do I have to
> download another version control system just so that I can "git"
> libxcb and X11 so that I can compile gtk so that I can compile xchat
> so that I can go onto the IRC server for madwifi so that I can ask
> someone there why their wifi drivers don't create a /dev/wifi0 on my
> system so that I can get my wireless working????
Given that git is converting a lot of subversion/CVS users - including the
projects you list above, do you not think that they might have had a reason
for doing so? Don't ask git developers why they wrote git - go and ask those
projects why they are using it.
Practically, I think you're overstating the difficulty of "downloading another
version control system". On debian/ubuntu you would do:
$ sudo apt-get install git-core
If you really did want to compile these things, then you would do:
$ cd src
$ git clone git://anongit.freedesktop.org/git/xorg/xserver.git
$ git clone git://anongit.freedesktop.org/git/xcb.git
... etc ...
But are you seriously suggesting that you need to compile the latest and
greatest X11/xcb/gtk/xchat just to chat on IRC? Why not just do
$ sudo apt-get install xchat
If your distribution doesn't supply an apt-get equivalent then you must have
picked one where you compile everything from source by hand - in which case -
what are you moaning about - you did this to yourself?
Andy
--
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
^ permalink raw reply
* Re: Git benchmarks at OpenOffice.org wiki
From: Andy Parkins @ 2007-05-02 8:55 UTC (permalink / raw)
To: git; +Cc: Jakub Narebski, releases, Jan Holesovsky
In-Reply-To: <200705012346.14997.jnareb@gmail.com>
On Tuesday 2007 May 01, Jakub Narebski wrote:
> In 'Size of data on the server' git has CVS beat hands down: 1.3G vs
> 8.5G for sources, 591M vs 1.1G for third party. I think it is similar
> for Subversion. I hope that repository is fully packed: IIRC the Mozilla
> CVS repository import was about 0.6GB pack file, not 1.3GB.
I'm fairly sure it's not. If so that would also affect the speed of
operations wouldn't it?
I also doubt the subversion checkout size - subversion keeps a pristine copy
of the HEAD file - so a subversion checkout is usually over twice the size of
the source tree.
> takes 3min (!) according to the benchmark. 13-25sec for commit is also
> bit long
I wonder if they are measuring the time for the generation of the commit
message or something? Or perhaps by using "git-commit -a" is causing a check
of the whole tree for changed files?
> Comparison / benchmark lacks some crucial info, like what computer was
> used (CPU, RAM, HDD), what filesystem was used, git version etc. It
> does have commands used for tests (benchmarks).
I'd also like to see some of the numbers for the other systems, I tried to use
subversion with the linux kernel once and got fed up waiting for it to do
anything. I suspect the reason numbers aren't shown for the others is that
they haven't finished yet :-)
> Could you confirm (or deny) those results? go-oo.org uses git 1.4.3.4;
> was there some improvement or bugfix related to the speed of checkout?
Wasn't there a recent change that made repacking after a clone unnecessary?
That would certainly reduce the checkout size.
Andy
--
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com
^ permalink raw reply
* Re: FAQ Q: another version control system?
From: Jakub Narebski @ 2007-05-02 8:35 UTC (permalink / raw)
To: git
In-Reply-To: <9e7886190705020013k5721480aj767c1c8643fc8d23@mail.gmail.com>
[Cc: Peter <petervanbohning@gmail.com>, git@vger.kernel.org]
Peter wrote:
> Do we really need another version control system? As I understand it,
> this is like a distributed, central-server-less, CVS, sorry, SVN
> server.
There is large difference between centralized and distributed SCMs.
Besides, thats like asking why we have so many Linux distributions, or why
we have KDE, GNOME, XFCE, etc.
> But why isn't such a feature set added into SVN?
There is SVK which is distributed SCM built on top of Subversion.
But there are some things that cannot be done because it uses SVN
as engine. Performance is one of them.
> Why do I have to
> download another version control system just so that I can "git"
> libxcb and X11 so that I can compile gtk so that I can compile xchat
> so that I can go onto the IRC server for madwifi so that I can ask
> someone there why their wifi drivers don't create a /dev/wifi0 on my
> system so that I can get my wireless working????
You don't need to install git to download and install programs; there are
tarballs and packages. You don't even need git to work on program and send
corrections, although it is recommended.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* FAQ Q: another version control system?
From: Peter @ 2007-05-02 7:13 UTC (permalink / raw)
To: git
In-Reply-To: <9e7886190705020012n72070874nae5a74b6d1387e3@mail.gmail.com>
Do we really need another version control system? As I understand it,
this is like a distributed, central-server-less, CVS, sorry, SVN
server.
But why isn't such a feature set added into SVN? Why do I have to
download another version control system just so that I can "git"
libxcb and X11 so that I can compile gtk so that I can compile xchat
so that I can go onto the IRC server for madwifi so that I can ask
someone there why their wifi drivers don't create a /dev/wifi0 on my
system so that I can get my wireless working????
Aaaaah.
^ permalink raw reply
* Re: [StGIT RFC PATCH] Don't use refs/bases/<branchname>
From: Karl Hasselström @ 2007-05-02 6:50 UTC (permalink / raw)
To: Marco Costalba; +Cc: Catalin Marinas, git
In-Reply-To: <e5bfff550705011259w567a98dj6d0a68b0fbe90994@mail.gmail.com>
On 2007-05-01 21:59:34 +0200, Marco Costalba wrote:
> On 5/1/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > (But if I were you, I'd look for the patches under patches/refs
> > anyway; they _have_ to be there to survive garbage collection, so
> > no amount of stgit refactoring will break qgit.)
>
> Well, I did. ;-)
>
> Actually I pushed a patch few hours ago to read patches sha under
> refs/patches.
Good. This was precisely what I was hoping to scare you into doing!
:-)
> The problem is that the patch (for now) is pushed only for the new
> development version of qgit, not the stable one and, worse, all the
> currently released versions will break if you remove
> <git dir>/patches/ directory.
>
> So please, if possible deprecate <git dir>/patches/ directory but do
> not remove for a while, so to let users to upgrade gracefully.
It's not near the top of my kill list by any stretch of the
imagination, so no need to worry. And even if it were, Catalin would
certainly stand as a wall of sanity between qgit and my chain saw. :-)
Thanks.
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
* Re: problem using git on cygwin
From: srinivas naga vutukuri @ 2007-05-02 4:04 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0704301343020.29859@racer.site>
Hi,
On 4/30/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Mon, 30 Apr 2007, srinivas naga vutukuri wrote:
>
> > $ git add project/test.c
> > error: bad index file sha1 signature
> > fatal: index file corrupt
>
> I could imagine that you do not have "binary mode" default for opening
> files.
>
i do't think it is in binary mode when opening the files. I am
used vim to create a new file and editing through that. And also am
used file format (set ff=unix).
> However, if you do, maybe your .git/index went corrupt?
>
> As a workaround, you could try "git read-tree HEAD" and then "git add"
> again, but make sure you have a backup of .git/index, since it could be a
> bug in Git(cygwin) and if so, I'd like to fix it.
>
I have taken the initial index (once repository created, after
initial commit) to backup.
and followed the above steps.
1. cp index index.orig
2. git read-tree HEAD
3. git add test.c (my new file)
But end up with the same errors. Would you think there is a bug in the
git working on cygwin ?. I hope would the sequence help you in
catching the bug.
> Ciao,
> Dscho
>
>
best regards,
srinivas.
^ permalink raw reply
* [RFD/PATCH] Implement pack.compression and pack-objects --compression=N
From: Dana How @ 2007-05-02 3:18 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Git Mailing List, danahow
Git's object store and packing are optimized for
* Expensive repo to repo bandwidth; and
* Small-ish files
This justifies extensive use of compression.
In a multi-developer *office* with inter-repository
transfers occurring over a 100Mb+ LAN, there is less
reason to compress files and slow down response times.
Response times suffer even more when large files are involved.
However, *off-line* pack compression may still be
desirable to reduce storage space.
Consequently, for such a usage pattern it is useful
to specify different compression levels for loose
objects and packs. This patch implements a config
variable pack.compression in addition to the existing
core.compression, meant to be used for repacking.
It also adds --compression=N to pack-objects,
meant for push/pull/fetch, if different, or if different
on a per-repository basis.
** THIS PATCH IS UNTESTED AND MEANT FOR DISCUSSION. **
git-repack.sh might also need to be modified,
and how to pass --compression=N during push/pull/fetch
has not been investigated.
This applies on top of the git-repack --max-pack-size patchset.
Signed-off-by: Dana L. How <danahow@gmail.com>
---
builtin-pack-objects.c | 56 ++++++++++++++++++++++++++++++++++++------------
1 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 69fec34..b663c15 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -70,6 +70,7 @@ static uint32_t pack_size_limit;
static int pack_to_stdout;
static int num_preferred_base;
static struct progress progress_state;
+static int pack_compression_level, pack_compression_seen;
/*
* The object names in objects array are hashed with this hashtable,
@@ -414,6 +415,16 @@ static unsigned long write_object(struct sha1file *f,
/* write limit if limited packsize and not first object */
unsigned long limit = pack_size_limit && nr_written ?
pack_size_limit - write_offset : 0;
+ /* no if no delta */
+ int usable_delta = !entry->delta ? 0 :
+ /* yes if unlimited packfile */
+ !pack_size_limit ? 1 :
+ /* no if base written to previous pack */
+ entry->delta->offset == (off_t)-1 ? 0 :
+ /* otherwise double-check written to this
+ * pack, like we do below
+ */
+ entry->delta->offset ? 1 : 0;
if (!pack_to_stdout)
crc32_begin(f);
@@ -423,8 +434,7 @@ static unsigned long write_object(struct sha1file *f,
to_reuse = 0; /* can't reuse what we don't have */
else if (obj_type == OBJ_REF_DELTA || obj_type == OBJ_OFS_DELTA)
/* check_object() decided it for us ... */
- to_reuse = !pack_size_limit ||
- (entry->delta->offset && entry->delta->offset != (off_t)-1);
+ to_reuse = usable_delta;
/* ... but pack split may override that */
else if (obj_type != entry->in_pack_type)
to_reuse = 0; /* pack has delta which is unusable */
@@ -435,6 +445,10 @@ static unsigned long write_object(struct sha1file *f,
* and we do not need to deltify it.
*/
+ /* differing core & pack compression when loose object -> must recompress */
+ if (!entry->in_pack && pack_compression_level != zlib_compression_level)
+ to_reuse = 0;
+ else
if (!entry->in_pack && !entry->delta) {
unsigned char *map;
unsigned long mapsize;
@@ -462,16 +476,6 @@ static unsigned long write_object(struct sha1file *f,
z_stream stream;
unsigned long maxsize;
void *out;
- /* no if no delta */
- int usable_delta = !entry->delta ? 0 :
- /* yes if unlimited packfile */
- !pack_size_limit ? 1 :
- /* no if base written to previous pack */
- entry->delta->offset == (off_t)-1 ? 0 :
- /* otherwise double-check written to this
- * pack, like we do below
- */
- entry->delta->offset ? 1 : 0;
buf = read_sha1_file(entry->sha1, &type, &size);
if (!buf)
die("unable to read %s", sha1_to_hex(entry->sha1));
@@ -493,7 +497,7 @@ static unsigned long write_object(struct sha1file *f,
}
/* compress the data to store and put compressed length in datalen */
memset(&stream, 0, sizeof(stream));
- deflateInit(&stream, zlib_compression_level);
+ deflateInit(&stream, pack_compression_level);
maxsize = deflateBound(&stream, size);
out = xmalloc(maxsize);
/* Compress it */
@@ -606,7 +610,7 @@ static unsigned long write_object(struct sha1file *f,
unuse_pack(&w_curs);
reused++;
}
- if (entry->delta)
+ if (usable_delta)
written_delta++;
written++;
if (!pack_to_stdout)
@@ -1622,6 +1626,16 @@ static int git_pack_config(const char *k, const char *v)
window = git_config_int(k, v);
return 0;
}
+ if (!strcmp(k, "pack.compression")) {
+ int level = git_config_int(k, v);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die("bad pack compression level %d", level);
+ pack_compression_level = level;
+ pack_compression_seen = 1;
+ return 0;
+ }
return git_default_config(k, v);
}
@@ -1732,6 +1746,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
rp_ac = 2;
git_config(git_pack_config);
+ if (!pack_compression_seen)
+ pack_compression_level = zlib_compression_level;
progress = isatty(2);
for (i = 1; i < argc; i++) {
@@ -1759,6 +1775,18 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
usage(pack_usage);
continue;
}
+ if (!prefixcmp(arg, "--compression=")) {
+ char *end;
+ int level = strtoul(arg+14, &end, 0);
+ if (!arg[14] || *end)
+ usage(pack_usage);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die("bad pack compression level %d", level);
+ pack_compression_level = level;
+ continue;
+ }
if (!prefixcmp(arg, "--window=")) {
char *end;
window = strtoul(arg+9, &end, 0);
--
1.5.2.rc0.787.g0014
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox