* [PATCH] compress output of `cd' when installing templates
From: Gang Chen @ 2006-05-21 15:41 UTC (permalink / raw)
To: GIT
When CDPATH is set in Bash, `cd' writes new working directory to
stdout, which corrupts output of `tar cf - .'.
Signed-off-by: Gang Chen <goncha.ml@gmail.com>
---
Makefile | 2 +-
1 files changed, 1 insertion(+), 1 deletion(-)
diff --git a/templates/Makefile b/templates/Makefile
index 8f7f4fe..cb67489 100644
--- a/templates/Makefile
+++ b/templates/Makefile
@@ -44,5 +44,5 @@ clean:
install: all
$(INSTALL) -d -m755 $(call shellquote,$(DESTDIR)$(template_dir))
- (cd blt && $(TAR) cf - .) | \
+ (cd blt > /dev/null && $(TAR) cf - .) | \
(cd $(call shellquote,$(DESTDIR)$(template_dir)) && $(TAR) xf -)
--
If it makes you happy, it can't be that bad.
^ permalink raw reply related
* First tarball release of qgit with tabs
From: Marco Costalba @ 2006-05-21 14:29 UTC (permalink / raw)
To: git
This is qgit1.3rc1
Rewritten a big chunk of UI code and added support for tabbed views
and powerful key bindings. Now should be far easier and quick to
browse through patches.
1.3 release will be mainly about user friendliness. Any suggestion,
comment and of course bug is, as usual, greatly appreciated.
Detailed changelog in shipped file or directly through public git repo.
Note: Sourceforge seems to have some download problem today!
Marco
Download
--------------
Git archive is at
http://digilander.libero.it/mcostalba/scm/qgit.git
See http://digilander.libero.it/mcostalba/ for detailed download information.
Install
--------
git 1.3.0 is required.
To install from tarball:
./configure
make
make install-strip
To install from git archive:
autoreconf -i
./configure
make
make install-strip
Or check the shipped README for detailed information.
^ permalink raw reply
* Re: synchronizing incremental git changes to cvs
From: Jim Meyering @ 2006-05-21 13:40 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0605210104470.3148@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Sun, 21 May 2006, Jim Meyering wrote:
>
>> Why am I interested? I want to switch the development of GNU coreutils
>> from cvs to git. I would also like to continue making the repository
>> available via cvs, for the sake of continuity. At worst, I can always
>> cut the CVS cord, but that's a last resort.
>
> If you only want to make a cvs repository available for tracking the
> project, git-cvsserver is what you want. It is even faster than the
> original cvs...
That might work if I had sufficient access to the system hosting the
public CVS repository. But there are restrictions (like no ssh access).
Currently I rsync the master repo to an intermediate site, from which
it is periodically pulled by savannah. Paranoia on both sides.
If I end up leaving savannah, can someone propose a good site,
i.e., secure, yet with git and rsync access?
I haven't made the leap to git yet, but git-cvsimport (from git-1.3.2)
seems to do a very good job of converting the cvs module (89MB).
FYI, here are some stats on the resulting git repository:
Size (nothing repacked):
1051MB (du -sh, actual, on reiserfs 3)
708MB (du --apparent-size)
Size repacked, (via git-repack -a -d && git-prune-packed)
65MB (du --si -s)
20k+ patchsets (counted by cvsps)
40k+ revisions (counted by cvs ... rlog cu|grep -c '^revision')
While repacking, git said something about more than 100K objects.
There were 120K files under .git/ before repacking.
^ permalink raw reply
* Re: [idea] Converting sha1 evaluator into parser/interpreter
From: Jakub Narebski @ 2006-05-21 10:13 UTC (permalink / raw)
To: git
In-Reply-To: <e4p71u$t0s$1@sea.gmane.org>
Jakub Narebski wrote:
> Shawn Pearce wrote:
>
>> There was just a short conversation on #git about converting
>> the sha1 expression evaluator into a split parser/interpreter
>> model. The idea here would be to convert an expression such as
>>
>> HEAD@{yesterday}~3^{tree}
>>
>> into a an expression tree such as (in LISP style):
>>
>> (peel-onion (walk-back 3 (date-spec yesterday (ref HEAD))))
>>
>> with such a tree it is relatively easy to evaluate the expression,
>> but its also easy to determine if a ref name is valid. Just pass
>> it through the parser and see if you get back anything more complex
>> then '(ref <input>)'.
>
> Didn't you meant to see if we get correct tree (not a forest),
> and if the root of said tree is '(ref <commit-ish>)' [1]?
I'm sorry. Of course branch names (ref names) should be limited to the ones
that doesn't make problems for the parser, i.e. they should parse to single
element parse tree:
<input> ---parses to---> (ref <input>)
and doesn't give problems for other parsers (e.g.commit list parser
including '^A B' and 'A..B'; remotes branch mapping parser 'srcref:destref'
and '+srcref:destref') and shell (e.g. globbing).
P.S. Hmmm... is '--' branch name (heads/--) possible?
P.P.S. Would branch name which include '!' and/or '$' cause trouble?
--
Jakub Narebski
Warsaw, Poland
^ permalink raw reply
* Re: irc usage..
From: Thomas Glanzmann @ 2006-05-21 9:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Donnie Berkholz, Yann Dirson, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0605201543260.3649@g5.osdl.org>
Hello Linus,
> and I'm a humanitarian - rescuing people from CVS is
> to me not just a good idea, it's a moral imperative.
you're a very brave man.
Thomas
^ permalink raw reply
* Re: [PATCH 5/5] Enable ref log creation in git checkout -b.
From: Junio C Hamano @ 2006-05-21 9:42 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20060519091716.GM22257@spearce.org>
I've swallowed all 10 and pushed them out in "pu", but could you
add tests to check the Porcelainish commands you touched with
this series to make sure they all log correctly?
BTW, I noticed that a patch earlier in the series depended on
something not in "master" (it's rfc2822_date from js/fmt-patch
series). Generally I do not want to make a branch hostage of
another branch by introducing a dependency, but for now I'll
pull in early part of js/fmt-patch branch into sp/reflog branch
and see what happens.
If sp/reflog branch graduates to the "master" first, it will
pull early parts of js/fmt-patch along with it, but the built-in
will be called "git fmt-patch" in the result, so it wouldn't
affect the use of "git format-patch".
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Junio C Hamano @ 2006-05-21 9:40 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20060521084313.GA12825@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> What do you suggest for this performance issue, then?
> A manual no-delta trigger, or just going to get a cup of coffee while
> pushing (my tests showed 5-6x slowdown from deltifying)?
In the short term, give --depth to send-pack, would be better
than nothing.
I agree we should be able to do better. But let me grab some
sleep now first ;-).
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Jakub Narebski @ 2006-05-21 9:24 UTC (permalink / raw)
To: git
In-Reply-To: <20060521084403.GB12825@coredump.intra.peff.net>
Jeff King wrote:
> On Sun, May 21, 2006 at 10:24:37AM +0200, Jakub Narebski wrote:
>
>> Hmmm... isn't the patch slightly against git coding style?
>
> Oops, yes (though the point is moot since the patch is conceptually
> wrong). Is there a git coding style document somewhere?
Not that I know of. It's better to follow the coding style used
in the rest of files.
Besides, differentiating between
if (condition)
and
function(argument);
makes IMVHO sense.
--
Jakub Narebski
Warsaw, Poland
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Junio C Hamano @ 2006-05-21 9:14 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20060521084403.GB12825@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> On Sun, May 21, 2006 at 10:24:37AM +0200, Jakub Narebski wrote:
>
>> Hmmm... isn't the patch slightly against git coding style?
>
> Oops, yes (though the point is moot since the patch is conceptually
> wrong). Is there a git coding style document somewhere?
When in doubt, please follow the kernel coding style ;-).
^ permalink raw reply
* Re: [PATCH] Reference git-check-ref-format in git-branch.
From: Junio C Hamano @ 2006-05-21 9:07 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <20060521015446.GA7605@spearce.org>
Shawn Pearce <spearce@spearce.org> writes:
> Sorry about this patch being built on pu. It clearly has no
> relationship to current pu, but the new -l appears in the hunk
> below...
"am -3 -s -u" is how I handle patches anyway, and it did not
have much trouble backporting it to the tip of "master".
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Jeff King @ 2006-05-21 8:44 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <e4p83e$uqt$1@sea.gmane.org>
On Sun, May 21, 2006 at 10:24:37AM +0200, Jakub Narebski wrote:
> Hmmm... isn't the patch slightly against git coding style?
Oops, yes (though the point is moot since the patch is conceptually
wrong). Is there a git coding style document somewhere?
-Peff
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Jeff King @ 2006-05-21 8:43 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vlksvzsmd.fsf@assigned-by-dhcp.cox.net>
On Sun, May 21, 2006 at 01:31:22AM -0700, Junio C Hamano wrote:
> We do not delta an otherwise perfectly deltifiable object if its
> delta base happens to be at the depth edge in the original pack.
> So no, we do _NOT_ know if it is not worth it merely from the
> fact that it is not deltified in the existing pack. And the
Yes, you're right. What do you suggest for this performance issue, then?
A manual no-delta trigger, or just going to get a cup of coffee while
pushing (my tests showed 5-6x slowdown from deltifying)?
-Peff
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Junio C Hamano @ 2006-05-21 8:31 UTC (permalink / raw)
To: git
In-Reply-To: <20060521081435.GA4526@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> On Sat, May 20, 2006 at 11:17:42PM -0700, Junio C Hamano wrote:
>
>> base delta for that object to skip computation". What you want
>> here is "if the object we are going to send is not a delta in
>> the source, and there are sufficient number of other objects the
>> object could have been deltified against, then it is very likely
>> that it was not worth deltifying when it was packed; so it is
>> probably not worth deltifying it now".
>
> I think we can make a stronger statement in many cases: "if the object
> we are going to send is not a delta in the source, and there are no
> other objects it could be deltified against, then it is not worth
> deltifying." That is, in the case that we just packed we KNOW that it's
> not worth it, since we're not sending anything that isn't already
> packed.
Careful.
We do not delta an otherwise perfectly deltifiable object if its
delta base happens to be at the depth edge in the original pack.
So no, we do _NOT_ know if it is not worth it merely from the
fact that it is not deltified in the existing pack. And the
latter part of your test "there are no other objects it could be
deltified against" is either expensive (you have to try first to
see if that is the case to really see it) or stupid (you just
assume there is no suitable delta base without looking at other
objects like we currently do).
^ permalink raw reply
* Re: [RFD] Git glossary: 'branch' and 'head' description
From: Jakub Narebski @ 2006-05-21 8:30 UTC (permalink / raw)
To: git
In-Reply-To: <87y7wyv72m.fsf@morpheus.hq.vtech>
David Kågedal wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
[...]
>> So from the user's point of view, 'branch' is simply _named line of
>> development_. Refer to topic and tracking branches.
>
> But the definition of 'branch' in git is quite different from the
> definition in CVS or many other systems. It CVS, each revision
> (commit) belongs to a branch, and the branch is a linear sequence of
> revisions, not a full DAG. In git, a commit doesn't really "belong"
> in any specific branch.
>
> So, while it makes sense to describe branches as "lines of
> development" in general terms, it is also important to note the
> specific meaning of 'branch' in the context of git; i.e. as the
> history of a single head commit.
We can always say that branch is 1-st parent linear history of head branch,
up to unmarked but computable branching/fork/creation point, i.e.
{i = 0..N: branch-head~i}
where N is the length of the branch.
BTW. HEAD~0 == HEAD, isn't it?
--
Jakub Narebski
Warsaw, Poland
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Jakub Narebski @ 2006-05-21 8:24 UTC (permalink / raw)
To: git
In-Reply-To: <20060521081435.GA4526@coredump.intra.peff.net>
Jeff King wrote:
Hmmm... isn't the patch slightly against git coding style?
> + if(unpacked)
[...]
>+ for(i = 0; i < len; i++)
compare to:
> for (ref = refs; ref; ref = ref->next) {
--
Jakub Narebski
Warsaw, Poland
^ permalink raw reply
* Re: [PATCH 2/2] Add builtin "git rm" command
From: Junio C Hamano @ 2006-05-21 8:20 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0605191607580.10823@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> I think the new behaviour is strictly an improvement, but perhaps more
> importantly, it is _different_. As a special case, the semantics are
> identical for the single-file case (which is the only one our test-suite
> seems to test).
>
> The other question is what to do with leading directories. The old "git
> rm" script didn't do anything, which is somewhat inconsistent. This one
> will actually clean up directories that have become empty as a result of
> removing the last file, but maybe we want to have a flag to decide the
> behaviour?
I too think these are improvements. Thanks for the patch.
BTW, this needed another "evil merge" into "next", so this time
I made a separate evil merge branch that I speculated as a
possibly better alternative approach in an earlier message.
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Jeff King @ 2006-05-21 8:14 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vy7wvx5o9.fsf@assigned-by-dhcp.cox.net>
On Sat, May 20, 2006 at 11:17:42PM -0700, Junio C Hamano wrote:
> base delta for that object to skip computation". What you want
> here is "if the object we are going to send is not a delta in
> the source, and there are sufficient number of other objects the
> object could have been deltified against, then it is very likely
> that it was not worth deltifying when it was packed; so it is
> probably not worth deltifying it now".
I think we can make a stronger statement in many cases: "if the object
we are going to send is not a delta in the source, and there are no
other objects it could be deltified against, then it is not worth
deltifying." That is, in the case that we just packed we KNOW that it's
not worth it, since we're not sending anything that isn't already
packed.
Following this logic, I believe we could always turn off deltification
if there are no loose objects. It seems a bit special case, but it
optimizes the "repack -d then push" workflow which I suspect may be
relatively common.
The patch below runs git-rev-list with --unpacked; if there are no
objects returned, it tells pack-objects to set depth=0 (is that really
what we want? It's OK to use existing deltas; we just don't want to
compute any new ones. I'm not sure how --depth behaves in that respect).
-Peff
---
f3e7fb4a5025cc8157557f3da6f9dc7d0a89395f
send-pack.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++-----------
1 files changed, 58 insertions(+), 14 deletions(-)
f3e7fb4a5025cc8157557f3da6f9dc7d0a89395f
diff --git a/send-pack.c b/send-pack.c
index 409f188..0bddc0a 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -25,18 +25,7 @@ static int is_zero_sha1(const unsigned c
return 1;
}
-static void exec_pack_objects(void)
-{
- static const char *args[] = {
- "pack-objects",
- "--stdout",
- NULL
- };
- execv_git_cmd(args);
- die("git-pack-objects exec failed (%s)", strerror(errno));
-}
-
-static void exec_rev_list(struct ref *refs)
+static void exec_rev_list(struct ref *refs, int unpacked)
{
struct ref *ref;
static const char *args[1000];
@@ -47,6 +36,8 @@ static void exec_rev_list(struct ref *re
args[i++] = "--objects-edge";
else
args[i++] = "--objects";
+ if(unpacked)
+ args[i++] = "--unpacked";
/* First send the ones we care about most */
for (ref = refs; ref; ref = ref->next) {
@@ -85,6 +76,59 @@ static void exec_rev_list(struct ref *re
die("git-rev-list exec failed (%s)", strerror(errno));
}
+static int rev_list_loose_objects(struct ref *refs)
+{
+ int fd[2];
+ pid_t pid;
+
+ if(pipe(fd) < 0)
+ die("rev-list setup: pipe failed");
+ pid = fork();
+ if(pid < 0)
+ die("rev-list setup: fork failed");
+ if(!pid) {
+ dup2(fd[1], 1);
+ close(fd[0]);
+ close(fd[1]);
+ exec_rev_list(refs, 1);
+ die("rev-list failed");
+ }
+ close(fd[1]);
+ return fd[0];
+}
+
+static int count_loose_objects(struct ref *refs)
+{
+ int fd = rev_list_loose_objects(refs);
+ unsigned count = 0;
+ char buf[4096];
+ int len;
+
+ while((len = read(fd, buf, sizeof buf)) > 0) {
+ int i;
+ for(i = 0; i < len; i++)
+ if(buf[i] == '\n')
+ count++;
+ }
+ if(len < 0)
+ die("rev-list read failed");
+ return count;
+}
+
+static void exec_pack_objects(struct ref *refs)
+{
+ static const char *args[] = {
+ "pack-objects",
+ "--stdout",
+ NULL,
+ NULL
+ };
+ if(count_loose_objects(refs) == 0)
+ args[2] = "--depth=0";
+ execv_git_cmd(args);
+ die("git-pack-objects exec failed (%s)", strerror(errno));
+}
+
static void rev_list(int fd, struct ref *refs)
{
int pipe_fd[2];
@@ -99,7 +143,7 @@ static void rev_list(int fd, struct ref
close(pipe_fd[0]);
close(pipe_fd[1]);
close(fd);
- exec_pack_objects();
+ exec_pack_objects(refs);
die("pack-objects setup failed");
}
if (pack_objects_pid < 0)
@@ -108,7 +152,7 @@ static void rev_list(int fd, struct ref
close(pipe_fd[0]);
close(pipe_fd[1]);
close(fd);
- exec_rev_list(refs);
+ exec_rev_list(refs, 0);
}
static int pack_objects(int fd, struct ref *refs)
--
1.3.3.gf1cf-dirty
^ permalink raw reply related
* Re: [idea] Converting sha1 evaluator into parser/interpreter
From: Jakub Narebski @ 2006-05-21 8:06 UTC (permalink / raw)
To: git
Shawn Pearce wrote:
> There was just a short conversation on #git about converting
> the sha1 expression evaluator into a split parser/interpreter
> model. The idea here would be to convert an expression such as
>
> HEAD@{yesterday}~3^{tree}
>
> into a an expression tree such as (in LISP style):
>
> (peel-onion (walk-back 3 (date-spec yesterday (ref HEAD))))
Rather
(peel-onion 'tree (walk-back 3 (date-spec yesterday (ref HEAD))))
> with such a tree it is relatively easy to evaluate the expression,
> but its also easy to determine if a ref name is valid. Just pass
> it through the parser and see if you get back anything more complex
> then '(ref <input>)'.
Didn't you meant to see if we get correct tree (not a forest),
and if the root of said tree is '(ref <commit-ish>)' [1]?
Interpreting said parse tree anch checking if it folds to correct object
reference is the task of interpreter, not parser...
[*1*] if I understand currectly that <commit-ish> mean direct sha1,
shortened sha1, or ref (head or tag)? commit-ish is not in
git glossary...
--
Jakub Narebski
Warsaw, Poland
^ permalink raw reply
* Re: [PATCH] Remove possible segfault in http-fetch.
From: Junio C Hamano @ 2006-05-21 7:49 UTC (permalink / raw)
To: Sean; +Cc: git, Nick Hengeveld
In-Reply-To: <BAYC1-PASMTP082397700A9527CC2F3786AEA40@CEZ.ICE>
Sean <seanlkml@sympatico.ca> writes:
> Free the curl string lists after running http_cleanup to
> avoid an occasional segfault in the curl library. Seems
> to only occur if the website returns a 405 error.
>...
> It comes with a big disclaimer because I don't really know the
> code in here all that well. However gdb reports the segfault
> happens in a strncasecmp call, and seeing as we've released a
> bunch of strings prior to the call....
>
> http-fetch.c | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/http-fetch.c b/http-fetch.c
> index 861644b..178f1ee 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -1269,10 +1269,10 @@ int main(int argc, char **argv)
> if (pull(commit_id))
> rc = 1;
>
> - curl_slist_free_all(no_pragma_header);
> -
> http_cleanup();
>
> + curl_slist_free_all(no_pragma_header);
> +
> if (corrupt_object_found) {
> fprintf(stderr,
> "Some loose object were found to be corrupt, but they might be just\n"
curl_easy_cleanup() which is called from http_cleanup() says it
is safe to remove the strings _after_ you call that function, so
I think the change makes sense -- it was apparently unsafe to
free them before calling cleanup.
Knowing nothing about quirks in curl libraries, one thing that
is mystery to me is that we slist_append() to other two lists
(pragma_header and range_header) but we do not seem to ever free
them. Another slist dav_headers is allocated and then freed
inside a function, so that call-pattern seems well-formed.
Nick, care to help us out?
^ permalink raw reply
* Re: [RFC] send-pack: allow skipping delta when sending pack
From: Junio C Hamano @ 2006-05-21 6:17 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20060521054827.GA18530@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> The result is much better performance in my case. However, the method
> seems quite hack-ish, so I wanted to get comments on how this should be
> done. Possibilities I considered:
> 1. A command line option to git-send-pack. The problem with this is
> that support is required from git-push and cg-push to pass the
> option through.
When you pull from such a repository you would also need to be
able to control this. The repository owner knows what's in the
repository a lot more than the downloader, so some repository
configuration that tells upload-pack to use such-and-such delta
window is also needed. But as you say below:
> 3. Ideally, we could do some heuristic to see if deltification will
> yield helpful results. In particular, we may already have a pack
> with these commits in it (especially if we just repack before a
> push). If we can re-use this information, it at least saves
> deltifying twice (once to pack, once to push). In theory, I would
> think the fact that we don't pass --no-reuse-delta to pack-objects
> means that this would happen automatically, but it clearly doesn't.
The lack of --no-reuse-delta just means "if the object we are
going to send is a delta in the source, and its delta base is
also something we are going to send, then pretend that it is the
base delta for that object to skip computation". What you want
here is "if the object we are going to send is not a delta in
the source, and there are sufficient number of other objects the
object could have been deltified against, then it is very likely
that it was not worth deltifying when it was packed; so it is
probably not worth deltifying it now".
^ permalink raw reply
* [RFC] send-pack: allow skipping delta when sending pack
From: Jeff King @ 2006-05-21 5:48 UTC (permalink / raw)
To: git
I have a git repo where I keep relatively large files (digital photos).
I have a local repo and a "master" repo on a server which I access over
ssh. Deltifying a bunch of large images takes a relatively long time. I
can live with this while packing (though it is slightly annoying to have
to pack separately on both repos, I understand why it might be hard to
reuse the deltification).
However, it is extremely annoying to add a large set of images and then
push them to the server. The server is on a 100Mbit LAN, so the
deltification part of the process takes up most of the time (and
typically ends up making no deltas, since the files are unrelated
images). The patch below causes the GIT_NODELTA environment variable to
set the window depth to 0 when sending a pack, preventing deltification.
The result is much better performance in my case. However, the method
seems quite hack-ish, so I wanted to get comments on how this should be
done. Possibilities I considered:
1. A command line option to git-send-pack. The problem with this is
that support is required from git-push and cg-push to pass the
option through.
2. A repo config variable that says not to deltify on sending (or
potentially, not to deltify at all, which makes packing in general
much nicer -- however, I don't think this is a good idea, as I do
still want deltification rarely, it's just that it mostly will
fail). This should probably be per-remote for the obvious reason
that one might push to local and remote repos. One drawback is that
sometimes deltification may be a win; it's just that I sometimes
know that it won't be (because I added a bunch of unrelated large
files). It's nice to selectively turn this option on for a given
push.
3. Ideally, we could do some heuristic to see if deltification will
yield helpful results. In particular, we may already have a pack
with these commits in it (especially if we just repack before a
push). If we can re-use this information, it at least saves
deltifying twice (once to pack, once to push). In theory, I would
think the fact that we don't pass --no-reuse-delta to pack-objects
means that this would happen automatically, but it clearly doesn't.
Comments?
---
f1cf653120dd492d1c86ee2a92a9c8221023cef1
send-pack.c | 6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)
f1cf653120dd492d1c86ee2a92a9c8221023cef1
diff --git a/send-pack.c b/send-pack.c
index 409f188..4ad6489 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -30,8 +30,14 @@ static void exec_pack_objects(void)
static const char *args[] = {
"pack-objects",
"--stdout",
+ NULL,
NULL
};
+ const char *nodelta;
+
+ nodelta = getenv("GIT_NODELTA");
+ if(nodelta && !strcmp(nodelta, "1"))
+ args[2] = "--depth=0";
execv_git_cmd(args);
die("git-pack-objects exec failed (%s)", strerror(errno));
}
--
1.3.3.g288c-dirty
^ permalink raw reply related
* Re: [PATCH 0/5] More ref logging
From: Sean @ 2006-05-21 5:09 UTC (permalink / raw)
To: Shawn Pearce; +Cc: junkio, git
In-Reply-To: <20060521045146.GA8269@spearce.org>
On Sun, 21 May 2006 00:51:46 -0400
Shawn Pearce <spearce@spearce.org> wrote:
> Any chance we could get some details on why so many tags were useful?
> You have a good argument here but I'm not sure how a better tag
> store could be structured. :-)
It is a conversion from another SCM. So every one of the ~16K commits
was tagged with the reference number taken from the original SCM. This
has some very nice benefits in that you can refer to every commit
in git by the original changeset #. For example, if someone reports a
bug mentioning the original scm's reference id, you can say something
like: "git show p4/1234" without having to go back to the old scm.
Also, qgit, gitk and gitweb display them nicely which can be helpful
during the conversion. And if/when they're not needed any longer,
you just delete them without having to rewrite the history etc..
> Yea - despite being the author of ref log I'm still slightly unhappy
> with the fact that it doesn't make reuse of existing GIT plumbing.
> But I'm sort of OK with that right now as you can't map two indexes
> into memory at once currently, nor is there a way to easily update
> multiple refs at once if the ref log must serialize access to create
> a string of trees and commits.
Well it's not the end of the world either way, and sometimes it's just
better to implement a workable solution rather than wait for one that's
theoretically cleaner. It just seemed like it was worth mentioning in
case you saw a way to make it happen without a lot of grief.
Sean
^ permalink raw reply
* Re: [PATCH 0/5] More ref logging
From: Shawn Pearce @ 2006-05-21 4:51 UTC (permalink / raw)
To: Sean; +Cc: junkio, git
In-Reply-To: <20060520224344.7ebca48b.seanlkml@sympatico.ca>
Sean <seanlkml@sympatico.ca> wrote:
> On Sat, 20 May 2006 20:50:09 -0400
> Shawn Pearce <spearce@spearce.org> wrote:
>
> > It sort of is a new way of tagging commits with extra data. But its
> > also sort of a way of versioning your ref `database'. Using tags
> > to save the points in time might be useful but it would generate
> > a lot of temporary files. A commit every 5 minutes for a typical
> > working week would generate 480 tags per week. That's just too much.
>
> But isn't that just an implementation detail? I've actually run
> into another situation where tags would be perfect if only they weren't
> so expensive (ie. entire repo was in a 50Mb pack including tag objects,
> but the .git/refs/tags directory was over 100Mb).
Any chance we could get some details on why so many tags were useful?
You have a good argument here but I'm not sure how a better tag
store could be structured. :-)
> So, if we found a way to store tags more efficiently your 480 tags per
> week shouldn't be a problem at all. The main point being to extend
> and optimize the existing infrastructure rather than bolting on a new
> class of objects (ie. ref log) which only serves a narrow (albeit
> important) purpose.
Yea - despite being the author of ref log I'm still slightly unhappy
with the fact that it doesn't make reuse of existing GIT plumbing.
But I'm sort of OK with that right now as you can't map two indexes
into memory at once currently, nor is there a way to easily update
multiple refs at once if the ref log must serialize access to create
a string of trees and commits.
--
Shawn.
^ permalink raw reply
* [PATCH] git-svn: ignore expansion of svn:keywords
From: Eric Wong @ 2006-05-21 3:03 UTC (permalink / raw)
To: Tommi Virtanen; +Cc: git
In-Reply-To: <446A0CCF.2060903@inoi.fi>
Can you try this patch?
This weakens an integrity in order to work on certain
repositories (see comments). It's probably safe to use,
though... More testing and feedback would be nice.
I've split out the test for this feature to make things easier
to manage and test, too.
Also changed assert_svn_wc_clean() to only work on svn, and
require a separate call to assert_tree() to check wc integrity
against git in preparation for another change I'm planning.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
@Junio: please don't apply this to git.git just yet, thanks.
contrib/git-svn/Makefile | 1
contrib/git-svn/git-svn.perl | 79 +++++++++++++++++-----
contrib/git-svn/t/lib-git-svn.sh | 39 +++++++++++
contrib/git-svn/t/t0000-contrib-git-svn.sh | 43 +-----------
contrib/git-svn/t/t0001-contrib-git-svn-props.sh | 51 ++++++++++++++
5 files changed, 153 insertions(+), 60 deletions(-)
create mode 100644 contrib/git-svn/t/lib-git-svn.sh
create mode 100644 contrib/git-svn/t/t0001-contrib-git-svn-props.sh
eb8f17257c7d15ec6815baf18208af06f72b3cdb
diff --git a/contrib/git-svn/Makefile b/contrib/git-svn/Makefile
index acedf73..48f60b3 100644
--- a/contrib/git-svn/Makefile
+++ b/contrib/git-svn/Makefile
@@ -30,6 +30,7 @@ git-svn.html : git-svn.txt
-f ../../Documentation/asciidoc.conf $<
test: git-svn
cd t && $(SHELL) ./t0000-contrib-git-svn.sh
+ cd t && $(SHELL) ./t0001-contrib-git-svn-props.sh
clean:
rm -f git-svn *.xml *.html *.1
diff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl
index de13a96..86b687a 100755
--- a/contrib/git-svn/git-svn.perl
+++ b/contrib/git-svn/git-svn.perl
@@ -26,6 +26,7 @@ use Carp qw/croak/;
use IO::File qw//;
use File::Basename qw/dirname basename/;
use File::Path qw/mkpath/;
+use File::Copy qw/cp/;
use Getopt::Long qw/:config gnu_getopt no_ignore_case auto_abbrev/;
use File::Spec qw//;
use POSIX qw/strftime/;
@@ -207,7 +208,7 @@ sub rebuild {
push @svn_up, '--ignore-externals' unless $_no_ignore_ext;
sys(@svn_up,"-r$newest_rev");
$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;
- git_addremove();
+ index_changes();
exec('git-write-tree');
}
waitpid $pid, 0;
@@ -249,7 +250,7 @@ sub fetch {
chdir $SVN_WC or croak $!;
read_uuid();
$last_commit = git_commit($base, @parents);
- assert_svn_wc_clean($base->{revision}, $last_commit);
+ assert_tree($last_commit);
} else {
chdir $SVN_WC or croak $!;
read_uuid();
@@ -259,7 +260,11 @@ sub fetch {
push @svn_up, '--ignore-externals' unless $_no_ignore_ext;
my $last = $base;
while (my $log_msg = next_log_entry($svn_log)) {
- assert_svn_wc_clean($last->{revision}, $last_commit);
+ # this assertion is commented out because it breaks keywords
+ # on https://svn.musicpd.org/Jamming/trunk (r166:167), but
+ # I can't seem to reproduce something like that on a test...
+ # assert_svn_wc_clean($last->{revision});
+ assert_tree($last_commit);
if ($last->{revision} >= $log_msg->{revision}) {
croak "Out of order: last >= current: ",
"$last->{revision} >= $log_msg->{revision}\n";
@@ -268,7 +273,8 @@ sub fetch {
$last_commit = git_commit($log_msg, $last_commit, @parents);
$last = $log_msg;
}
- assert_svn_wc_clean($last->{revision}, $last_commit);
+ assert_svn_wc_clean($last->{revision});
+ assert_tree($last_commit);
unless (-e "$GIT_DIR/refs/heads/master") {
sys(qw(git-update-ref refs/heads/master),$last_commit);
}
@@ -314,7 +320,6 @@ sub commit {
$svn_current_rev = svn_commit_tree($svn_current_rev, $c);
}
print "Done committing ",scalar @revs," revisions to SVN\n";
-
}
sub show_ignore {
@@ -367,13 +372,11 @@ sub setup_git_svn {
}
sub assert_svn_wc_clean {
- my ($svn_rev, $treeish) = @_;
+ my ($svn_rev) = @_;
croak "$svn_rev is not an integer!\n" unless ($svn_rev =~ /^\d+$/);
- croak "$treeish is not a sha1!\n" unless ($treeish =~ /^$sha1$/o);
my $lcr = svn_info('.')->{'Last Changed Rev'};
if ($svn_rev != $lcr) {
print STDERR "Checking for copy-tree ... ";
- # use
my @diff = grep(/^Index: /,(safe_qx(qw(svn diff),
"-r$lcr:$svn_rev")));
if (@diff) {
@@ -389,7 +392,6 @@ sub assert_svn_wc_clean {
print STDERR $_ foreach @status;
croak;
}
- assert_tree($treeish);
}
sub assert_tree {
@@ -416,7 +418,7 @@ sub assert_tree {
unlink $tmpindex or croak $!;
}
$ENV{GIT_INDEX_FILE} = $tmpindex;
- git_addremove();
+ index_changes(1);
chomp(my $tree = `git-write-tree`);
if ($old_index) {
$ENV{GIT_INDEX_FILE} = $old_index;
@@ -426,6 +428,7 @@ sub assert_tree {
if ($tree ne $expected) {
croak "Tree mismatch, Got: $tree, Expected: $expected\n";
}
+ unlink $tmpindex;
}
sub parse_diff_tree {
@@ -562,7 +565,8 @@ sub precommit_check {
sub svn_checkout_tree {
my ($svn_rev, $treeish) = @_;
my $from = file_to_s("$REV_DIR/$svn_rev");
- assert_svn_wc_clean($svn_rev,$from);
+ assert_svn_wc_clean($svn_rev);
+ assert_tree($from);
print "diff-tree $from $treeish\n";
my $pid = open my $diff_fh, '-|';
defined $pid or croak $!;
@@ -852,13 +856,50 @@ sub svn_info {
sub sys { system(@_) == 0 or croak $? }
-sub git_addremove {
- system( "git-diff-files --name-only -z ".
- " | git-update-index --remove -z --stdin && ".
- "git-ls-files -z --others ".
- "'--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude'".
- " | git-update-index --add -z --stdin"
- ) == 0 or croak $?
+sub do_update_index {
+ my ($z_cmd, $cmd, $no_text_base) = @_;
+
+ my $z = open my $p, '-|';
+ defined $z or croak $!;
+ unless ($z) { exec @$z_cmd or croak $! }
+
+ my $pid = open my $ui, '|-';
+ defined $pid or croak $!;
+ unless ($pid) {
+ exec('git-update-index',"--$cmd",'-z','--stdin') or croak $!;
+ }
+ local $/ = "\0";
+ while (my $x = <$p>) {
+ chomp $x;
+ if (!$no_text_base && lstat $x && ! -l _) {
+ my $mode = -x _ ? 0755 : 0644;
+ my ($v,$d,$f) = File::Spec->splitpath($x);
+ my $tb = File::Spec->catfile($d, '.svn', 'tmp',
+ 'text-base',"$f.svn-base");
+ $tb =~ s#^/##;
+ unless (-f $tb) {
+ $tb = File::Spec->catfile($d, '.svn',
+ 'text-base',"$f.svn-base");
+ $tb =~ s#^/##;
+ }
+ unlink $x or croak $!;
+ cp($tb, $x) or croak $!;
+ chmod(($mode &~ umask), $x) or croak $!;
+ }
+ print $ui $x,"\0";
+ }
+ close $ui or croak $!;
+}
+
+sub index_changes {
+ my $no_text_base = shift;
+ do_update_index([qw/git-diff-files --name-only -z/],
+ 'remove',
+ $no_text_base);
+ do_update_index([qw/git-ls-files -z --others/,
+ "--exclude-from=$GIT_DIR/$GIT_SVN/info/exclude"],
+ 'add',
+ $no_text_base);
}
sub s_to_file {
@@ -936,7 +977,7 @@ sub git_commit {
defined $pid or croak $!;
if ($pid == 0) {
$ENV{GIT_INDEX_FILE} = $GIT_SVN_INDEX;
- git_addremove();
+ index_changes();
chomp(my $tree = `git-write-tree`);
croak if $?;
if (exists $tree_map{$tree}) {
diff --git a/contrib/git-svn/t/lib-git-svn.sh b/contrib/git-svn/t/lib-git-svn.sh
new file mode 100644
index 0000000..a98e9d1
--- /dev/null
+++ b/contrib/git-svn/t/lib-git-svn.sh
@@ -0,0 +1,39 @@
+PATH=$PWD/../:$PATH
+if test -d ../../../t
+then
+ cd ../../../t
+else
+ echo "Must be run in contrib/git-svn/t" >&2
+ exit 1
+fi
+
+. ./test-lib.sh
+
+GIT_DIR=$PWD/.git
+GIT_SVN_DIR=$GIT_DIR/git-svn
+SVN_TREE=$GIT_SVN_DIR/tree
+
+svnadmin >/dev/null 2>&1
+if test $? != 1
+then
+ test_expect_success 'skipping contrib/git-svn test' :
+ test_done
+ exit
+fi
+
+svn >/dev/null 2>&1
+if test $? != 1
+then
+ test_expect_success 'skipping contrib/git-svn test' :
+ test_done
+ exit
+fi
+
+svnrepo=$PWD/svnrepo
+
+set -e
+
+svnadmin create $svnrepo
+svnrepo="file://$svnrepo/test-git-svn"
+
+
diff --git a/contrib/git-svn/t/t0000-contrib-git-svn.sh b/contrib/git-svn/t/t0000-contrib-git-svn.sh
index 80ad357..8b3a0d9 100644
--- a/contrib/git-svn/t/t0000-contrib-git-svn.sh
+++ b/contrib/git-svn/t/t0000-contrib-git-svn.sh
@@ -3,48 +3,10 @@ #
# Copyright (c) 2006 Eric Wong
#
-
-PATH=$PWD/../:$PATH
test_description='git-svn tests'
-if test -d ../../../t
-then
- cd ../../../t
-else
- echo "Must be run in contrib/git-svn/t" >&2
- exit 1
-fi
-
-. ./test-lib.sh
-
-GIT_DIR=$PWD/.git
-GIT_SVN_DIR=$GIT_DIR/git-svn
-SVN_TREE=$GIT_SVN_DIR/tree
-
-svnadmin >/dev/null 2>&1
-if test $? != 1
-then
- test_expect_success 'skipping contrib/git-svn test' :
- test_done
- exit
-fi
-
-svn >/dev/null 2>&1
-if test $? != 1
-then
- test_expect_success 'skipping contrib/git-svn test' :
- test_done
- exit
-fi
-
-svnrepo=$PWD/svnrepo
-
-set -e
-
-svnadmin create $svnrepo
-svnrepo="file://$svnrepo/test-git-svn"
+. ./lib-git-svn.sh
mkdir import
-
cd import
echo foo > foo
@@ -55,10 +17,9 @@ mkdir -p bar
echo 'zzz' > bar/zzz
echo '#!/bin/sh' > exec.sh
chmod +x exec.sh
-svn import -m 'import for git-svn' . $svnrepo >/dev/null
+svn import -m 'import for git-svn' . "$svnrepo" >/dev/null
cd ..
-
rm -rf import
test_expect_success \
diff --git a/contrib/git-svn/t/t0001-contrib-git-svn-props.sh b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh
new file mode 100644
index 0000000..20c5c4e
--- /dev/null
+++ b/contrib/git-svn/t/t0001-contrib-git-svn-props.sh
@@ -0,0 +1,51 @@
+#!/bin/sh
+#
+# Copyright (c) 2006 Eric Wong
+#
+
+test_description='git-svn property tests'
+. ./lib-git-svn.sh
+
+mkdir import
+
+cd import
+ cat >> kw.c <<''
+/* Make it look like somebody copied a file from CVS into SVN: */
+/* $Id: kw.c,v 1.1.1.1 1994/03/06 00:00:00 eric Exp $ */
+
+ svn import -m 'import for git-svn' . "$svnrepo" >/dev/null
+cd ..
+
+rm -rf import
+svn co "$svnrepo" test_wc
+
+cd test_wc
+ echo 'Greetings' >> kw.c
+ svn commit -m 'Not yet an $Id$'
+ svn up
+
+ echo 'Hello world' >> kw.c
+ svn commit -m 'Modified file, but still not yet an $Id$'
+ svn up
+
+ svn propset svn:keywords Id kw.c
+ svn commit -m 'Propset $Id$'
+ svn up
+cd ..
+
+git-svn init "$svnrepo"
+git-svn fetch
+
+git checkout -b mybranch remotes/git-svn
+echo 'Hi again' >> kw.c
+name='test svn:keywords ignoring'
+
+git commit -a -m "$name"
+git-svn commit remotes/git-svn..mybranch
+git pull . remotes/git-svn
+
+expect='/* $Id$ */'
+got="`sed -ne 2p kw.c`"
+test_expect_success 'raw $Id$ found in kw.c' "test '$expect' = '$got'"
+
+test_done
--
1.3.2.g7d11
^ permalink raw reply related
* Re: [PATCH 0/5] More ref logging
From: Sean @ 2006-05-21 2:43 UTC (permalink / raw)
To: Shawn Pearce; +Cc: junkio, git
In-Reply-To: <20060521005009.GA7179@spearce.org>
On Sat, 20 May 2006 20:50:09 -0400
Shawn Pearce <spearce@spearce.org> wrote:
> It sort of is a new way of tagging commits with extra data. But its
> also sort of a way of versioning your ref `database'. Using tags
> to save the points in time might be useful but it would generate
> a lot of temporary files. A commit every 5 minutes for a typical
> working week would generate 480 tags per week. That's just too much.
But isn't that just an implementation detail? I've actually run
into another situation where tags would be perfect if only they weren't
so expensive (ie. entire repo was in a 50Mb pack including tag objects,
but the .git/refs/tags directory was over 100Mb).
So, if we found a way to store tags more efficiently your 480 tags per
week shouldn't be a problem at all. The main point being to extend
and optimize the existing infrastructure rather than bolting on a new
class of objects (ie. ref log) which only serves a narrow (albeit
important) purpose.
> I was actually thinking this morning that another way to do this
> is to keep a metadata branch within the repository which records
> all of the refs in tree objects, then save the root commit under
> the special ref `LOG` in GIT_DIR. Every update to a logged ref
> would cause the tree to be updated and a new commit to be built.
> The branch would be a relatively simple string of pearls as its
> doubtful you would branch it.
>
> There are a number of downsides to this, not the least of which is
> I'd like to put a commit or tag SHA1 into the tree object rather than
> writing each ref as a blob (saves space). Currently commits and tags
> aren't permitted in a tree object so that would require some effort.
> But on the other hand you could pull (and track!) someone elses
> ref log through the standard GIT protocol.
Yes, Linus proposed something similar earlier to hold meta data.
But i've come to see tags as a place to store any arbitrary meta
data associated with a commit. If their implementation was more
efficient you could use them for your project and they could be used
for any number of other purposes as well.
> But this is starting to head down into the `bind commit` discussion;
> how do we record a number of commits as being related and tie them
> up into a single super commit?
Well, a tag that allowed the listing of multiple heads....
Sean
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox