* Re: Merge with git-pasky II.
From: Christopher Li @ 2005-04-14 2:16 UTC (permalink / raw)
To: Paul Jackson; +Cc: torvalds, pasky, git
In-Reply-To: <20050413220341.13e5ce0f.pj@engr.sgi.com>
On Wed, Apr 13, 2005 at 10:03:41PM -0700, Paul Jackson wrote:
>
> If you have a thin skin or tend to annoy others with a bit too much
> attitude or can't pass up a good language war (which is my failing, and
> why I am responding to a discussion that I've not been involved in for
> days) then the resulting flamage could be distracting.
Oh, my bad. I am not trying to start a language war here.
That is why I am hesitated about Python.
Just try to find out the acceptability. No pushing.
Chris
^ permalink raw reply
* Re: Merge with git-pasky II.
From: Paul Jackson @ 2005-04-14 5:03 UTC (permalink / raw)
To: Christopher Li; +Cc: torvalds, pasky, git
In-Reply-To: <20050414012352.GA17700@64m.dyndns.org>
> Do you have preference about what language of script we used?
Do you have a thick skin? <grin>
Can you easily ignore language wars with an amused wave of the hand
and a happy chuckle at the oh so predictable weaknesses of humans?
Then I'd wager it will be fine.
If you have a thin skin or tend to annoy others with a bit too much
attitude or can't pass up a good language war (which is my failing, and
why I am responding to a discussion that I've not been involved in for
days) then the resulting flamage could be distracting.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Christopher Li @ 2005-04-14 1:23 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, git
In-Reply-To: <Pine.LNX.4.58.0504132020550.7211@ppc970.osdl.org>
On Wed, Apr 13, 2005 at 08:51:50PM -0700, Linus Torvalds wrote:
>
>
> On Thu, 14 Apr 2005, Petr Baudis wrote:
>
> Thick skin is the name of the game. I'd not get any work done otherwise.
>
> On that note - I've been avoiding doing the merge-tree thing, in the hope
> that somebody else does what I've described. I really do suck at scripting
> things, yet this is clearly something where using C to do a lot of the
> stuff is pointless.
>
> Almost all the parts do seem to be there, ie Daniel did the "common
> parent" part, and the rest really does seem to be more about scripting
> than writing more C plumbing stuff..
Do you have preference about what language of script we used? I actually
hesitated to introduce my Python script to git.
I can build some script extension for git just like the one I did for
sparse, is that some thing you want to see?
Chris
^ permalink raw reply
* Re: Yet another base64 patch
From: H. Peter Anvin @ 2005-04-14 4:25 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: git
In-Reply-To: <425DEF64.60108@zytor.com>
H. Peter Anvin wrote:
>
> It seems that the flat format, at least on ext3 with dircache, is
> actually a major performance win, and that the second level loses quite
> a bit.
>
s/dircache/dir_index/
-hpa
^ permalink raw reply
* Yet another base64 patch
From: H. Peter Anvin @ 2005-04-14 4:19 UTC (permalink / raw)
To: git
I am assuming this will be the last one one way or another...
I decided that filenames/tags beginning with - was a really bad thing,
so I decided that, ugly though it might be, the best was to do a hybrid
between regular base64 (+ /) and filesystem-safe base64 (- _) and use
+ _ as the nonalpha characters needed. I have updated the base64
patches as well as gitcvt, and also put out a flat version of gitcvt.
gitcvt also now converts the HEAD file over. This requires pointing it
at the .dircache/.git directory instead of the objects directory inside.
I have tested it on both the git and the kernel-test repositories.
Checking out the total kernel tree (time checkout-cache -a into an empty
directory):
Cache cold Cache hot
stock 3:46.95 19.95
base64 5:56.20 23.74
flat 2:44.13 15.68
It seems that the flat format, at least on ext3 with dircache, is
actually a major performance win, and that the second level loses quite
a bit.
-hpa
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Linus Torvalds @ 2005-04-14 3:51 UTC (permalink / raw)
To: Petr Baudis; +Cc: Christopher Li, git
In-Reply-To: <20050414004504.GW25711@pasky.ji.cz>
On Thu, 14 Apr 2005, Petr Baudis wrote:
>
> http://www.theregister.co.uk/2005/04/11/torvalds_attack/ ... I'm nothing
> like a regular reader of (R), but I thought the guys have at least a bit
> of sense. Duh. :/ Or is April 11 now yet another joke day after April 1?
I actually _am_ a fairly regular reader, and hey, being opinionated and a
bit over the top is what makes the site worthwhile. It's obviously what
motivates the people.
And then, occasionally, when they bite you, hey, that's the price of
having a high profile. I worry more about sometimes not listening to
critics than I do about the critics themselves.
Thick skin is the name of the game. I'd not get any work done otherwise.
On that note - I've been avoiding doing the merge-tree thing, in the hope
that somebody else does what I've described. I really do suck at scripting
things, yet this is clearly something where using C to do a lot of the
stuff is pointless.
Almost all the parts do seem to be there, ie Daniel did the "common
parent" part, and the rest really does seem to be more about scripting
than writing more C plumbing stuff..
Linus
^ permalink raw reply
* Re: [PATCH] merge-base
From: Linus Torvalds @ 2005-04-14 3:49 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504131447480.30848-100000@iabervon.org>
On Wed, 13 Apr 2005, Daniel Barkalow wrote:
>
> I wrote a pretty dumb program that will find some common ancestor for a
> pair of commits. It finds the one which is closest to one of the commits
> by number of generations. This is, at least, not strictly worse than any
> other common ancestor, at least (like picking the original release would
> be).
>
> It wouldn't be too hard to do this much with rev-tree, but should be
> easier to extend.
I agree. But I did the silly "common revision tracking" part slightly
differently and in particular I already made fsck and rev-tree use the
same exact code.
Also, I don't see why you did the "common parent" thing as part of the
"library", since that really does seem to be a very specific to this
problem, and neither fsck nor rev-tree really wants it.
Also, the date parsing really is a separate issue from the revision
tracking (fsck does not want date parsing, but rev-tool does), so I think
you might want to do for date parsing what I just did for the revision.h
thing? No point in tying them together.
So could I ask you to re-factor it and base it on my current tree? Make
the "merge-base" program have that common parent thing in it, and factor
out the common date parsing into "parse-date.c" or something?
Linus
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Christopher Li @ 2005-04-13 22:00 UTC (permalink / raw)
To: Petr Baudis; +Cc: torvalds, git
In-Reply-To: <20050414004504.GW25711@pasky.ji.cz>
On Thu, Apr 14, 2005 at 02:45:04AM +0200, Petr Baudis wrote:
> Dear diary, on Wed, Apr 13, 2005 at 11:25:46PM CEST, I got a letter
> where Christopher Li <git@chrisli.org> told me that...
> Well, unless Linus will want me otherwise, I'd like to postpone this
> until I'm finally done with the damn merge - enough things already got
> into my way today, so I would really like to focus on this tomorrow. So
> I'll be probably merging only (or mostly) bugfixes until I have that
> finished.
Sure, whenever you are ready.
> P.S.: Just staring at
> http://www.theregister.co.uk/2005/04/11/torvalds_attack/ ... I'm nothing
> like a regular reader of (R), but I thought the guys have at least a bit
> of sense. Duh. :/ Or is April 11 now yet another joke day after April 1?
Whatever, is the news site. They never mention git though.
Chris
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Petr Baudis @ 2005-04-14 0:45 UTC (permalink / raw)
To: Christopher Li; +Cc: torvalds, git
In-Reply-To: <20050413212546.GA17236@64m.dyndns.org>
Dear diary, on Wed, Apr 13, 2005 at 11:25:46PM CEST, I got a letter
where Christopher Li <git@chrisli.org> told me that...
> While you are there, do you mind to move the shell script
> to a sub directory? Let's try how rename works.
Well, unless Linus will want me otherwise, I'd like to postpone this
until I'm finally done with the damn merge - enough things already got
into my way today, so I would really like to focus on this tomorrow. So
I'll be probably merging only (or mostly) bugfixes until I have that
finished.
P.S.: Just staring at
http://www.theregister.co.uk/2005/04/11/torvalds_attack/ ... I'm nothing
like a regular reader of (R), but I thought the guys have at least a bit
of sense. Duh. :/ Or is April 11 now yet another joke day after April 1?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Merge with git-pasky II.
From: Christopher Li @ 2005-04-13 21:25 UTC (permalink / raw)
To: Petr Baudis; +Cc: torvalds, git
In-Reply-To: <20050414002902.GU25711@pasky.ji.cz>
While you are there, do you mind to move the shell script
to a sub directory? Let's try how rename works.
Chris
On Thu, Apr 14, 2005 at 02:29:02AM +0200, Petr Baudis wrote:
> Hello Linus,
>
> I think my tree should be ready for merging with you. It is the final
> tree and I've already switched my main branch for it, so it's what
> people doing git pull are getting for some time already.
>
> Its main contents are all of my shell scripts. Apart of that, some
> tiny fixes scattered all around can be found there, as well as some
> patches which went through the mailing list. My last merge with you
> concerned your commit 39021759c903a943a33a28cfbd5070d36d851581.
>
> It's again
>
> rsync://pasky.or.cz/git/
>
> this time my HEAD is fba83970090ef54c6eb86dcc2c2d5087af5ac637.
>
> Note that my rsync tree still contains even my old branch; I thought
> I'd leave it around in the public objects database for some time, shall
> anyone want to have a look at the history of some of the scripts. But if
> you want it gone, tell me and I will prune it (and perhaps offer it in
> /git-old/ or whatever). I'm using the following:
>
> fsck-cache --unreachable $(commit-id) | grep unreachable \
> | cut -d ' ' -f 2 | sed 's/^\(..\)/.git\/objects\/\1\//' \
> | xargs rm
>
> Thanks,
>
> --
> Petr "Pasky" Baudis
> Stuff: http://pasky.or.cz/
> C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Merge with git-pasky II.
From: Petr Baudis @ 2005-04-14 0:30 UTC (permalink / raw)
To: torvalds; +Cc: git
In-Reply-To: <20050414002902.GU25711@pasky.ji.cz>
Dear diary, on Thu, Apr 14, 2005 at 02:29:02AM CEST, I got a letter
where Petr Baudis <pasky@ucw.cz> told me that...
> Its main contents are all of my shell scripts. Apart of that, some
> tiny fixes scattered all around can be found there, as well as some
> patches which went through the mailing list. My last merge with you
> concerned your commit 39021759c903a943a33a28cfbd5070d36d851581.
>
> It's again
>
> rsync://pasky.or.cz/git/
>
> this time my HEAD is fba83970090ef54c6eb86dcc2c2d5087af5ac637.
I forgot to add that after merging, you will probably want to change the
VERSION file (to contain whatever you want).
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Merge with git-pasky II.
From: Petr Baudis @ 2005-04-14 0:29 UTC (permalink / raw)
To: torvalds; +Cc: git
Hello Linus,
I think my tree should be ready for merging with you. It is the final
tree and I've already switched my main branch for it, so it's what
people doing git pull are getting for some time already.
Its main contents are all of my shell scripts. Apart of that, some
tiny fixes scattered all around can be found there, as well as some
patches which went through the mailing list. My last merge with you
concerned your commit 39021759c903a943a33a28cfbd5070d36d851581.
It's again
rsync://pasky.or.cz/git/
this time my HEAD is fba83970090ef54c6eb86dcc2c2d5087af5ac637.
Note that my rsync tree still contains even my old branch; I thought
I'd leave it around in the public objects database for some time, shall
anyone want to have a look at the history of some of the scripts. But if
you want it gone, tell me and I will prune it (and perhaps offer it in
/git-old/ or whatever). I'm using the following:
fsck-cache --unreachable $(commit-id) | grep unreachable \
| cut -d ' ' -f 2 | sed 's/^\(..\)/.git\/objects\/\1\//' \
| xargs rm
Thanks,
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Re: teach ls-tree how to handle names with embedded LF
From: Petr Baudis @ 2005-04-14 0:20 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, git
In-Reply-To: <7v7jj6tdq9.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Thu, Apr 14, 2005 at 02:15:58AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:
>
> PB> Actually, what about just making it consistent with diff-tree and making
> PB> it always terminate the records with '\0'?
>
> Sounds good to me. I need '-z' for a script I am working on to
> generate data for incremental updates of object database.
I actually use ls-tree only in various one-shot hacks, so it's not an
issue for me neither.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* [ANNOUNCE] git-pasky-0.4
From: Petr Baudis @ 2005-04-14 0:19 UTC (permalink / raw)
To: git
Hello,
I'm happy to announce git-pasky-0.4, my set of scripts upon Linus
Torvald's git, which aims to provide a humanly usable interface, to a
degree similar to a SCM tool. You can get it at
http://pasky.or.cz/~pasky/dev/git/
See the READMEs etc for some introduction.
It is difficult to sum up the changes, since so much has changed,
including almost the complete tree history, which was cleaned up and
sanitized. Things should be faster, better, less buggy and generally
smoother.
My immediate plans are to support several working trees connected to a
single object database. The scenes are set, prepared, and it should be
easy. Then, supporting merges in a separate temporary tree will be a
breeze. ;-)
Have fun,
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: teach ls-tree how to handle names with embedded LF
From: Junio C Hamano @ 2005-04-14 0:15 UTC (permalink / raw)
To: Petr Baudis; +Cc: Linus Torvalds, git
In-Reply-To: <20050413235018.GQ25711@pasky.ji.cz>
>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:
PB> Actually, what about just making it consistent with diff-tree and making
PB> it always terminate the records with '\0'?
Sounds good to me. I need '-z' for a script I am working on to
generate data for incremental updates of object database.
^ permalink raw reply
* Re: teach ls-tree how to handle names with embedded LF
From: Petr Baudis @ 2005-04-13 23:50 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, git
In-Reply-To: <7vd5sytf94.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Thu, Apr 14, 2005 at 01:43:03AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> GIT dircache can represent filenames with embedded newline.
> Give ls-tree an option to terminate its output with '\0', so that
> its output can be further processed with tools like "sort -z"
> (and custom perl/python scripts).
Actually, what about just making it consistent with diff-tree and making
it always terminate the records with '\0'?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* [PATCH] teach ls-tree how to handle names with embedded LF
From: Junio C Hamano @ 2005-04-13 23:43 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, git
GIT dircache can represent filenames with embedded newline.
Give ls-tree an option to terminate its output with '\0', so that
its output can be further processed with tools like "sort -z"
(and custom perl/python scripts).
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
ls-tree.c | 9 ++++++++-
1 files changed, 8 insertions(+), 1 deletion(-)
ls-tree.c: 3e2a6c7d183a42e41f1073dfec6794e8f8a5e75c
--- ls-tree.c
+++ ls-tree.c 2005-04-13 16:17:29.000000000 -0700
@@ -5,6 +5,8 @@
*/
#include "cache.h"
+int line_termination = '\n';
+
static int list(unsigned char *sha1)
{
void *buffer;
@@ -31,7 +33,8 @@
* It seems not worth it to read each file just to get this
* and the file size. -- pasky@ucw.cz */
type = S_ISDIR(mode) ? "tree" : "blob";
- printf("%03o\t%s\t%s\t%s\n", mode, type, sha1_to_hex(sha1), path);
+ printf("%03o\t%s\t%s\t%s%c", mode, type, sha1_to_hex(sha1),
+ path, line_termination);
}
return 0;
}
@@ -40,6 +43,10 @@
{
unsigned char sha1[20];
+ if (argc == 3 && !strcmp(argv[1], "-z")) {
+ line_termination = 0;
+ argc--; argv++;
+ }
if (argc != 2)
usage("ls-tree <key>");
if (get_sha1_hex(argv[1], sha1) < 0)
---
head before this patch: f6803ddd4e97a8b0cc090c546d1ff085ed0a4316
tree after this patch: 0aa0af6254226f0811c72282c46e7e4d9438371c
head after this patch: 63582e5e8c40790d6255206d6bde65f4ce3820bf
^ permalink raw reply
* Re: Git usage and resource usage
From: Darren Williams @ 2005-04-13 23:18 UTC (permalink / raw)
To: Erik van Konijnenburg; +Cc: Git Mailing List, Linus Torvalds, Petr Baudis
In-Reply-To: <20050413152948.D2442@banaan.localdomain>
Hi Erik
On Wed, 13 Apr 2005, Erik van Konijnenburg wrote:
> On Wed, Apr 13, 2005 at 11:06:22PM +1000, Darren Williams wrote:
> > Git(pasky) requires an excessive quantity of file
> > descriptors, more than 1024 in a simple case and
> > more than 16384 in a high workload case.
>
> Does this help? Posted earlier today to the list,
> but since there are no archives yet its hard to find.
>
Yes that fixes it.
% ulimit -n
1024
% quilt pop -a
% time git commit
...
Enter commit message, terminated by ctrl-D on a separate line:
real 0m7.762s
user 0m7.282s
sys 0m0.459s
> Regards,
> Erik
>
> tree 70189060273990e945015845a1d97c03eac2346a
> parent 97c9a63e76bf667c21f24a5cfa8172aff0dd1294
> author Erik van Konijnenburg,,, <konijn@framboos> Wed Apr 13 13:32:13 2005
> committer Erik van Konijnenburg,,, <konijn@framboos> Wed Apr 13 13:32:13 2005
>
> avoid fd leak.
> .
> Index: read-cache.c
> ===================================================================
> --- 833bb44b8e49daf0424f44cf300748da17945859/read-cache.c (mode:100644 sha1:7cdd0f82992789e64f6ea272e43ee4af8cdf7f2a)
> +++ 70189060273990e945015845a1d97c03eac2346a/read-cache.c (mode:100644 sha1:7957377f2ddd107aa686f3ca4da51b74e6679baf)
> @@ -251,11 +251,15 @@
> if (fd < 0)
> return -1;
> map = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
> - if (map == MAP_FAILED)
> + if (map == MAP_FAILED) {
> + close(fd);
> return -1;
> + }
> if (memcmp(buf, map, size))
> return error("SHA1 collision detected!"
> " This is bad, bad, BAD!\a\n");
> + munmap(map, size);
> + close(fd);
> #endif
> return 0;
> }
--------------------------------------------------
Darren Williams <dsw AT gelato.unsw.edu.au>
Gelato@UNSW <www.gelato.unsw.edu.au>
--------------------------------------------------
^ permalink raw reply
* Re: Slight enhancement of GIT wrapper
From: Johannes Schindelin @ 2005-04-13 23:10 UTC (permalink / raw)
To: Petr Baudis; +Cc: Pavel Pisa, git
In-Reply-To: <20050413225509.GL25711@pasky.ji.cz>
Hi,
please ignore the mail I just sent.
Ciao,
Dscho
^ permalink raw reply
* Re: Slight enhancement of GIT wrapper
From: Johannes Schindelin @ 2005-04-13 23:07 UTC (permalink / raw)
To: Petr Baudis; +Cc: Pavel Pisa, git
In-Reply-To: <20050413225509.GL25711@pasky.ji.cz>
Hi,
On Thu, 14 Apr 2005, Petr Baudis wrote:
> Dear diary, on Wed, Apr 13, 2005 at 10:49:16PM CEST, I got a letter
> where Pavel Pisa <pisa@cmp.felk.cvut.cz> told me that...
>
> > +
> > +if [ -h "$0" ]; then
> > + GIT_DIR=`ls -l "$0" | sed 's/^.*-> *\(.*\) *$/\1/'`
>
> Well, I don't like this much, but I'm not sure how to do better.
How about
GIT_DIR=$(ls -L "$0")
Ciao,
Dscho
^ permalink raw reply
* Re: Slight enhancement of GIT wrapper
From: Petr Baudis @ 2005-04-13 22:55 UTC (permalink / raw)
To: Pavel Pisa; +Cc: git
In-Reply-To: <200504132249.16848.pisa@cmp.felk.cvut.cz>
Dear diary, on Wed, Apr 13, 2005 at 10:49:16PM CEST, I got a letter
where Pavel Pisa <pisa@cmp.felk.cvut.cz> told me that...
> Ahoj Petre,
Hi,
cc'ing git@vger.kernel.org so that we hopefully get less of duplicated
work (and maybe even some more insights).
> Index: commit-id
> ===================================================================
> Index: git
> ===================================================================
> --- 6c355da7a316f50742eeffa49405a15e75526012/git (mode:100755 sha1:f766a87adff6e17d24cd97417d7303f2cdab6f3c)
> +++ 85f2d458ffebf52569e777af222accc1268afdf9/git (mode:100755 sha1:c179a65c3955e39bda18dc8a314b061bc0fb5de7)
> @@ -9,6 +9,7 @@
> # This command mostly only multiplexes to the individual script based
> # on the first argument.
>
> +
> error () {
> echo git: $@ >&2
> }
> @@ -17,6 +18,25 @@
> exit 1
> }
>
> +cwd="`pwd 2> /dev/null`"
Please use $() instead of ``.
Why do you silence stderr?
> +
> +if [ -h "$0" ]; then
> + GIT_DIR=`ls -l "$0" | sed 's/^.*-> *\(.*\) *$/\1/'`
Well, I don't like this much, but I'm not sure how to do better.
> + GIT_DIR=`dirname "$GIT_DIR"`
> +else
> + GIT_DIR=`dirname "$0"`
> +fi
We use tabs for indentation.
> +cd "$GIT_DIR"
> +GIT_DIR="`pwd 2> /dev/null`"
Why do you silence stderr?
> +cd "$cwd" 2> /dev/null
> +if [ $? -gt 0 ] ; then
> + die "Cannot return back to the CWD $cwd"
> +fi
I'd personally just cd "$cwd" || exit 1
> +
> +export PATH=$GIT_DIR:$PATH
You should add $GIT_DIR to $PATH only when it is not there already. This
way you can still override some of the git tools by stuffing your
overrides to $PATH.
> +
> +#echo GIT_DIR=$GIT_DIR
> +#echo PATH=$PATH
?
>
> help () {
> cat <<__END__
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* git base64 repo conversion tool
From: H. Peter Anvin @ 2005-04-13 22:48 UTC (permalink / raw)
To: git
I realized writing a git repository conversion tool is actually quite
trivial in Perl, so I wrote one up and tried it on the git repo included
in the git-0.04 tarball. It fsck-cache's cleanly on both the nonflat
and flat base64 patches, at least after I had fixed a minor bug in
fsck-cache in the nonflat patch (now uploaded.)
It takes an source and a target object directory as arguments. By
default it is set up for non-flat representation; change $flat = 0; to
$flat = 1; near the top to use flat representation.
ftp://ftp.kernel.org/pub/linux/kernel/people/hpa/gitcvt
Comments appreciated, of course.
-hpa
^ permalink raw reply
* Re: Index/hash order
From: Florian Weimer @ 2005-04-13 22:48 UTC (permalink / raw)
To: Linus Torvalds; +Cc: H. Peter Anvin, Ingo Molnar, git
In-Reply-To: <Pine.LNX.4.58.0504131503180.4501@ppc970.osdl.org>
* Linus Torvalds:
> I will bet you that a git checkout is _faster_ than a kernel source tree
> copy. The time will be dominated by the IO costs (in particular the read
> costs), and the IO costs are lower thanks to compression. So I think that
> the cold-cache case will beat your 40 seconds by a clear margin. It
> generally compresses to half the size, so 20 seconds is not impossible
> (although seek costs would tend to stay constant, so I'd expect it to be
> somewhere in between the two).
It's indeed slightly faster (34 seconds). The hot-cache case is about
6 seconds. Still okay.
However, I should redo these tests with a real git. The numbers could
be quite different because seek overhead is a bit hard to predict.
Which version should I try?
> That's actually pretty encouraging. Your 1.1GB number implies to me that a
> compressed file setup should be about half that, which in turn says that
> the cost of full-file is not at all outrageous.
I usually try to avoid the typical O(f(n)) fallacy because constant
factors do matter in practice. But the way you put it -- maybe delta
compression isn't worth the complexity after all. At least I'm
beginning to have doubts.
Especially since the same Subversion repository, stored by the
Berkeley DB backend, requires a whopping 1.3 GB of disk space.
> Or maybe I misunderstood what you were comparing?
My estimates only cover file data, not metadata. Based on the
Subversion dumps, it might be possible to get some rough estimates for
the cost of storing directory information. What is the average size
of a directory blob? Is it true that for each tree revision, you need
to store a new directory blob for each directory which indirectly
contains a modified file?
Does your 50% estimate include wasted space due to the file system
block size?
^ permalink raw reply
* Re: Re: Re: Remove need to untrack before tracking new branch
From: Petr Baudis @ 2005-04-13 22:19 UTC (permalink / raw)
To: Martin Schlemmer; +Cc: git
In-Reply-To: <1113394537.23299.51.camel@nosferatu.lan>
I didn't weed out the replies since this was not cc'd to the mailing
list and I believe it could contain some useful information for google
or whatever to pick. :-)
Dear diary, on Wed, Apr 13, 2005 at 02:15:37PM CEST, I got a letter
where Martin Schlemmer <azarah@nosferatu.za.org> told me that...
> On Wed, 2005-04-13 at 11:26 +0200, Petr Baudis wrote:
> > BTW, why aren't we cc'ing the list?
> >
>
> Hmm, no reason really. If you want to, will do next time.
>
> > Dear diary, on Wed, Apr 13, 2005 at 10:41:12AM CEST, I got a letter
> > where Martin Schlemmer <azarah@nosferatu.za.org> told me that...
> > > On Wed, 2005-04-13 at 09:54 +0200, Petr Baudis wrote:
> > > PS: not having looked deeper yet, why does fsck-cache always find
> > > unreferenced blobs/commits (no matter what tree is tracked, they stay
> > > the same) ? And trying to remove them leads to more, which leads to an
> > > empty .git/opjects/ =) Also, leading to this, will adding an option to
> > > remove disconnected commits/blobs from local commits (that was
> > > disconnected with a pull) be a viable option to add?
> >
> > fsck-cache is concerned only by the objects database, so all the HEADs
> > are unreferenced commits too. This is a right thing, the HEAD tracking
> > should stay purely in the scripts - if we want to make fsck-cache
> > smarter about that, we should implement git fsck or something.
> >
> > Killing unreferenced blobs should be safe, I think.
> >
> > > First, about the 'git diff' thing I asked yesterday .. what I meant, was
> > > should it actually output this:
> > >
> > > ----
> > > COPYING: fe2a4177a760fd110e78788734f167bd633be8de 33
> > > Makefile: 929aa49a3dbe683ad52094099797bc636a7949a6 33
> > > README: 46c6a9ea48ddd1dda45ca585f49975a6869ffe51 33
> > > ...
> > > ----
> > >
> > > Shouldn't it just show actual changes?
> >
> > This is an actual change. It's just that it's a change to metadata
> > (somewhat esotherically described by the "33"), not the file contents.
> >
> > BTW, git diff does actually something completely different from git diff
> > with any arguments. It diffs to the directory cache, not to any tree! It
> > just wraps show-diff, which has also a different output format (not
> > outputting "git diffs"). The worst thing is that it requires a different
> > -p option to apply. Someone should purge this wart, I think.
> >
>
> Check applied patch (also in the new output).
Please send patches inline and properly signed off.
> > > Also on the same note .. should 'git ci' without listed files to be
> > > committed, really add a reference to all files as it currently do in the
> > > commit/blob/whatever info, instead of just the changed/added files (see
> > > the git-seperate-dir.patch you have not yet commented on for reference)?
> >
> > ...
> >
>
> Patch will also resolve this.
I'm sorry - the ellipsis was there so that I'll write up a reply for
that later but I forgot.
Your patch is bad - it removes the pure metadata changes, but you
definitively do not want to do that! If you are annoyed by meaningless
time changes etc, do update-cache --refresh. Ignoring mode changes is a
pure disaster.
> > > Secondly, how about style/error issues? Basically on the style front,
> > > currently some scripts use bash - how about pure bash for all:
> > >
> > > ----
> > > [[ -n ${tracking} ]] && \
> > > ----
> > >
> > > rather than:
> > >
> > > ----
> > > [ "$tracking" ] && \
> > > ----
> > >
> > > as you do not need the quotes, and for checking files, etc, it just do
> > > better handling than single quotes. I am prepared to do patches if you
> > > like.
> >
> > Well. I'm just not used to it and not quite sure about how it behaves
> > etc. So far I tried to make it as sh-like as possible, using bash
> > features only when absolutely necessary. I'm not saying this is the best
> > approach, but I wouldn't change it *now* (especially since it would slow
> > me down which I really don't want now) if someone doesn't give me some
> > really compelling arguments.
> >
>
> No problem.
>
..snip..
> > > 2) Did not add the gitapply.sh that was new in your tree
> >
> > This very bug should be fixed by introduction of gitapply.sh. ;-)
> >
> > Actually I forgot to implement this. Fixed now.
> >
> > > I guess I might have done things wrong (wrong way around)? But the
> > > actual question is this .. when I tried to switch, it failed on the
> > > gitapply.sh hunk, but went on anyway ... how about doing a patch
> > > --dry-run first, and showing the user what would happen if that fails,
> > > asking for confirmation?
> >
> > Just s/merge -b/diff/ on the command line and it will show precisely
> > what it is going to apply. When we get the automatic base detection
> > something to just print the base could be indeed useful.
> >
> > If that fails, all what will happen is that the user gets rejects. No
> > big deal. He can anytime just back out by
> >
> > git diff | patch -p0 -R
> >
> > So, what did it "went on" with? The merge (nor the tracked pull) never
> > commits anything. That's up to you, after you resolve the conflicts and
> > make sure everything is right.
> >
>
> Ah, right.
>
> > > I know its in its infancy, but I am not sure on what scm you are basing
> > > it, so not sure how things should behave.
> >
> > I'm trying to base it on common sense and principle of least surprise.
> > :-)
> >
>
> Ok, I'll just bug you then if I am not sure on how you want something ;p
Or do it somehow and I'll bug you back if I don't like it. ;-)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Index/hash order
From: Linus Torvalds @ 2005-04-13 22:11 UTC (permalink / raw)
To: Florian Weimer; +Cc: H. Peter Anvin, Ingo Molnar, git
In-Reply-To: <87aco2gxu2.fsf@deneb.enyo.de>
On Wed, 13 Apr 2005, Florian Weimer wrote:
>
> I've run a few tests, just to get a few numbers of the overhead
> involved. I used the last ~8,000 changesets from the BKCVS kernel
> repository. With cold cache, a checkout from cold cache takes about
> 250 seconds on my laptop. I don't have git numbers, but a mere copy
> of the kernel tree needs 40 seconds.
I will bet you that a git checkout is _faster_ than a kernel source tree
copy. The time will be dominated by the IO costs (in particular the read
costs), and the IO costs are lower thanks to compression. So I think that
the cold-cache case will beat your 40 seconds by a clear margin. It
generally compresses to half the size, so 20 seconds is not impossible
(although seek costs would tend to stay constant, so I'd expect it to be
somewhere in between the two).
> For the hot-cache case, the difference is 140 seconds vs. 2.5 seconds
> (or 6 seconds with decompression).
>
> Uh-oh. I wouldn't have imaged the difference would be *that*
> dramatic. The file system layer is *fast*.
Did I mention that I designed git for speed?
Yes. The whole damn design is really about performance, distribution, and
built-in integrity checking.
> On the storage front, we have 220 MB for the skip deltas vs. 106 MB
> for pure deltas-to-previous vs. 1.1 GB for uncompressed files
> (directories are always delta-compressed, so to speak[1]).
That's actually pretty encouraging. Your 1.1GB number implies to me that a
compressed file setup should be about half that, which in turn says that
the cost of full-file is not at all outrageous. Sure, it's 2-3 times
larger than your skip deltas, but considering that the performance is
about fifty times faster (and I can do distributed stuff without any
locking synchronization and you can't), that's a tradeoff I'm more than
happy with.
Or maybe I misunderstood what you were comparing?
Of course, the numbers will all depend on how the history looks etc, so
this is all pretty much just guidelines.
Linus
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox