* Re: [PATCH] git-p4: Avoid modules deprecated in Python 2.6.
From: Reilly Grant @ 2009-08-26 18:39 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vhbvucuj5.fsf@alter.siamese.dyndns.org>
Junio C Hamano wrote:
> Reilly Grant <reillyeon@qotw.net> writes:
>
>
>> The popen2, sha and sets modules are deprecated in Python 2.6 (sha in
>> Python 2.5). Both popen2 and sha are not actually used in git-p4.
>> Replace usage of sets.Set with the builtin set object.
>>
>
> Does the code already rely on a feature not found in Python older than 2.4
> before your patch? Otherwise I would have liked to see the last sentence
> like this:
>
> Replace usage of sets.Set with the builtin set object, which has
> been available since Python 2.4.
>
> This change makes the script unusable with Python older than 2.4,
> which was released in 2004. Hopefully nobody uses ancient 2.3.
>
> So that I did not have to check myself to get a feel of how safe this
> change is.
>
Thank you for the advice, this is my first patch. The existing code
uses the built-in set module in other places so the 2.4 requirement
already exists. I should have mentioned this in the original description.
--
Reilly Grant
reillyeon@qotw.net http://www.qotw.net/~reillyeon
GPG Key Signature: 2A41 A3E5 F3CA D3A0 F5CF 02DF B1EB CDEC 7850 E278
^ permalink raw reply
* Re: [PATCH] Add option -b/--branch to clone for select a new HEAD
From: Jeff King @ 2009-08-26 19:05 UTC (permalink / raw)
To: Björn Steinbrink; +Cc: Kirill A. Korinskiy, Junio C Hamano, git
In-Reply-To: <20090826174823.GA1202@coredump.intra.peff.net>
On Wed, Aug 26, 2009 at 01:48:23PM -0400, Jeff King wrote:
> Later in the code, we look at head_points_at->peer_ref, which means it
> _must_ come from mapped_refs. And which means the code I posted is
> bogus, as the ref we look up in "refs" will not have that member filled
> in. So I think we do need:
>
> head_points_at = find_ref_by_name(mapped_refs, branch_head.buf);
Actually, it is much more complicated than that. We want to do several
things with the remote HEAD:
1. set up our HEAD; this uses head_points_at now
2. set up a pointer in refs/remotes/$origin/HEAD; this uses
head_points_at now
3. check out the actual contents; this uses remote_head (and it can't
just blindly use head_points_at, because it may be a detached HEAD).
So you can see by (1) and (2) that we actually need to distinguish
between the remote's HEAD and where we want our HEAD to be. And we need
to checkout _our_ HEAD, if available, falling back to the remote's head.
I really wish clone was implemented simply as "init && remote add &&
fetch && checkout", which would have made this a lot easier. We would
simply be munging the HEAD file in the middle.
Anyway, here is a patch which I think does the right thing, and tests
each of the desired behaviors in the test script. If there are further
"should it do X or Y" questions, please phrase them in the form of a
patch to the test script. ;)
I also tried to roll in comments on the documentation to make it a bit
clearer.
-- >8 --
Subject: [PATCH] clone: add --branch option to select a different HEAD
We currently point the HEAD of a newly cloned repo to the
same ref as the parent repo's HEAD. While a user can then
"git checkout -b foo origin/foo" whichever branch they
choose, it is more convenient and more efficient to tell
clone which branch you want in the first place.
Based on a patch by Kirill A. Korinskiy.
Signed-off-by: Jeff King <peff@peff.net>
---
Documentation/git-clone.txt | 7 ++++
builtin-clone.c | 75 +++++++++++++++++++++++++++----------------
t/t5706-clone-branch.sh | 68 +++++++++++++++++++++++++++++++++++++++
3 files changed, 122 insertions(+), 28 deletions(-)
create mode 100755 t/t5706-clone-branch.sh
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 2c63a0f..1cd1ecc 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -127,6 +127,13 @@ objects from the source repository into a pack in the cloned repository.
Instead of using the remote name 'origin' to keep track
of the upstream repository, use <name>.
+--branch <name>::
+-b <name>::
+ Instead of pointing the newly created HEAD to the branch pointed
+ to by the cloned repositoroy's HEAD, point to <name> branch
+ instead. In a non-bare repository, this is the branch that will
+ be checked out.
+
--upload-pack <upload-pack>::
-u <upload-pack>::
When given, and the repository to clone from is accessed
diff --git a/builtin-clone.c b/builtin-clone.c
index 32dea74..9d79301 100644
--- a/builtin-clone.c
+++ b/builtin-clone.c
@@ -41,6 +41,7 @@ static int option_quiet, option_no_checkout, option_bare, option_mirror;
static int option_local, option_no_hardlinks, option_shared;
static char *option_template, *option_reference, *option_depth;
static char *option_origin = NULL;
+static char *option_branch = NULL;
static char *option_upload_pack = "git-upload-pack";
static int option_verbose;
@@ -65,6 +66,8 @@ static struct option builtin_clone_options[] = {
"reference repository"),
OPT_STRING('o', "origin", &option_origin, "branch",
"use <branch> instead of 'origin' to track upstream"),
+ OPT_STRING('b', "branch", &option_branch, "branch",
+ "checkout <branch> instead of the remote's HEAD"),
OPT_STRING('u', "upload-pack", &option_upload_pack, "path",
"path to git-upload-pack on the remote"),
OPT_STRING(0, "depth", &option_depth, "depth",
@@ -347,7 +350,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
const char *repo_name, *repo, *work_tree, *git_dir;
char *path, *dir;
int dest_exists;
- const struct ref *refs, *head_points_at, *remote_head, *mapped_refs;
+ const struct ref *refs, *remote_head, *mapped_refs;
+ const struct ref *remote_head_points_at;
+ const struct ref *our_head_points_at;
struct strbuf key = STRBUF_INIT, value = STRBUF_INIT;
struct strbuf branch_top = STRBUF_INIT, reflog_msg = STRBUF_INIT;
struct transport *transport = NULL;
@@ -519,11 +524,31 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
mapped_refs = write_remote_refs(refs, refspec, reflog_msg.buf);
remote_head = find_ref_by_name(refs, "HEAD");
- head_points_at = guess_remote_head(remote_head, mapped_refs, 0);
+ remote_head_points_at =
+ guess_remote_head(remote_head, mapped_refs, 0);
+
+ if (option_branch) {
+ struct strbuf head = STRBUF_INIT;
+ strbuf_addstr(&head, src_ref_prefix);
+ strbuf_addstr(&head, option_branch);
+ our_head_points_at =
+ find_ref_by_name(mapped_refs, head.buf);
+ strbuf_release(&head);
+
+ if (!our_head_points_at) {
+ warning("Remote branch %s not found in "
+ "upstream %s, using HEAD instead",
+ option_branch, option_origin);
+ our_head_points_at = remote_head_points_at;
+ }
+ }
+ else
+ our_head_points_at = remote_head_points_at;
}
else {
warning("You appear to have cloned an empty repository.");
- head_points_at = NULL;
+ our_head_points_at = NULL;
+ remote_head_points_at = NULL;
remote_head = NULL;
option_no_checkout = 1;
if (!option_bare)
@@ -531,41 +556,35 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
"refs/heads/master");
}
- if (head_points_at) {
- /* Local default branch link */
- create_symref("HEAD", head_points_at->name, NULL);
+ if (remote_head_points_at && !option_bare) {
+ struct strbuf head_ref = STRBUF_INIT;
+ strbuf_addstr(&head_ref, branch_top.buf);
+ strbuf_addstr(&head_ref, "HEAD");
+ create_symref(head_ref.buf,
+ remote_head_points_at->peer_ref->name,
+ reflog_msg.buf);
+ }
+ if (our_head_points_at) {
+ /* Local default branch link */
+ create_symref("HEAD", our_head_points_at->name, NULL);
if (!option_bare) {
- struct strbuf head_ref = STRBUF_INIT;
- const char *head = head_points_at->name;
-
- if (!prefixcmp(head, "refs/heads/"))
- head += 11;
-
- /* Set up the initial local branch */
-
- /* Local branch initial value */
+ const char *head = skip_prefix(our_head_points_at->name,
+ "refs/heads/");
update_ref(reflog_msg.buf, "HEAD",
- head_points_at->old_sha1,
+ our_head_points_at->old_sha1,
NULL, 0, DIE_ON_ERR);
-
- strbuf_addstr(&head_ref, branch_top.buf);
- strbuf_addstr(&head_ref, "HEAD");
-
- /* Remote branch link */
- create_symref(head_ref.buf,
- head_points_at->peer_ref->name,
- reflog_msg.buf);
-
install_branch_config(0, head, option_origin,
- head_points_at->name);
+ our_head_points_at->name);
}
} else if (remote_head) {
/* Source had detached HEAD pointing somewhere. */
- if (!option_bare)
+ if (!option_bare) {
update_ref(reflog_msg.buf, "HEAD",
remote_head->old_sha1,
NULL, REF_NODEREF, DIE_ON_ERR);
+ our_head_points_at = remote_head;
+ }
} else {
/* Nothing to checkout out */
if (!option_no_checkout)
@@ -597,7 +616,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
opts.src_index = &the_index;
opts.dst_index = &the_index;
- tree = parse_tree_indirect(remote_head->old_sha1);
+ tree = parse_tree_indirect(our_head_points_at->old_sha1);
parse_tree(tree);
init_tree_desc(&t, tree->buffer, tree->size);
unpack_trees(1, &t, &opts);
diff --git a/t/t5706-clone-branch.sh b/t/t5706-clone-branch.sh
new file mode 100755
index 0000000..f3f9a76
--- /dev/null
+++ b/t/t5706-clone-branch.sh
@@ -0,0 +1,68 @@
+#!/bin/sh
+
+test_description='clone --branch option'
+. ./test-lib.sh
+
+check_HEAD() {
+ echo refs/heads/"$1" >expect &&
+ git symbolic-ref HEAD >actual &&
+ test_cmp expect actual
+}
+
+check_file() {
+ echo "$1" >expect &&
+ test_cmp expect file
+}
+
+test_expect_success 'setup' '
+ mkdir parent &&
+ (cd parent && git init &&
+ echo one >file && git add file && git commit -m one &&
+ git checkout -b two &&
+ echo two >file && git add file && git commit -m two &&
+ git checkout master)
+'
+
+test_expect_success 'vanilla clone chooses HEAD' '
+ git clone parent clone &&
+ (cd clone &&
+ check_HEAD master &&
+ check_file one
+ )
+'
+
+test_expect_success 'clone -b chooses specified branch' '
+ git clone -b two parent clone-two &&
+ (cd clone-two &&
+ check_HEAD two &&
+ check_file two
+ )
+'
+
+test_expect_success 'clone -b sets up tracking' '
+ (cd clone-two &&
+ echo origin >expect &&
+ git config branch.two.remote >actual &&
+ echo refs/heads/two >>expect &&
+ git config branch.two.merge >>actual &&
+ test_cmp expect actual
+ )
+'
+
+test_expect_success 'clone -b does not munge remotes/origin/HEAD' '
+ (cd clone-two &&
+ echo refs/remotes/origin/master >expect &&
+ git symbolic-ref refs/remotes/origin/HEAD >actual &&
+ test_cmp expect actual
+ )
+'
+
+test_expect_success 'clone -b with bogus branch chooses HEAD' '
+ git clone -b bogus parent clone-bogus &&
+ (cd clone-bogus &&
+ check_HEAD master &&
+ check_file one
+ )
+'
+
+test_done
--
1.6.4.1.340.ge9f66.dirty
^ permalink raw reply related
* Re: [git-svn] [FEATURE-REQ] track merges from git
From: Bryan Donlan @ 2009-08-26 19:06 UTC (permalink / raw)
To: Ximin Luo; +Cc: git
In-Reply-To: <4A9565ED.4010608@cam.ac.uk>
On Wed, Aug 26, 2009 at 12:42 PM, Ximin Luo<xl269@cam.ac.uk> wrote:
> Hi,
>
> I'm have 2 separate svn projects from googlecode imported into a single git
> repo. One is a semi-fork of the other, so I thought I'd be able to use git's
> merge feature to repeatedly merge from the mother project (and possibly vice
> versa too).
>
> However, this doesn't happen. I "git pull" and this works fine, but when I "git
> svn dcommit" back into svn, this rewrites my git history and it loses track of
> the merge (and next time I try to pull, the same conflicts appear).
>
> For now I just have a .git/info/grafts, but this doesn't get exported anywhere,
> so if other people "git svn clone" from svn, or "git clone" from my git repo,
> they don't get the merge information.
>
> It would be nice if git-svn saved the merge info somewhere instead of getting
> rid of it. #git tells me this is impossible at the moment, hence the mail.
> Relevant parts of the convo are pasted below.
>
> I understand if this is a low priority, but I don't think it would be a major
> PITA to implement (some suggestions are listed in the convo log). And it'd be
> useful for people converting from svn to git.
>
> Thanks for your time.
>
> X
>
> P.S. please don't troll me.
For the particular use case you have, I suspect svk would be a better
tool for merging between those two projects...
git-svn is rather specifically designed to only deal with a single
upstream repository, you see, and it isn't very easy to change this to
accept multiple repos.
^ permalink raw reply
* [PATCH v3] import-tars: Allow per-tar author and commit message.
From: Peter Krefting @ 2009-08-26 19:26 UTC (permalink / raw)
To: git
If the "--metainfo=<ext>" option is given on the command line, a file
called "<filename.tar>.<ext>" will be used to create the commit message
for "<filename.tar>", instead of using "Imported from filename.tar".
The author and committer of the tar ball can also be overridden by
embedding an "Author:" or "Committer:" header in the metainfo file.
Signed-off-by: Peter Krefting <peter@softwolves.pp.se>
---
This version adds a command line option --metainfo that is used to
enable the new behaviour, as suggested by Junio C Hamano.
contrib/fast-import/import-tars.perl | 54 +++++++++++++++++++++++++++++++--
1 files changed, 50 insertions(+), 4 deletions(-)
diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl
index 78e40d2..15835cb 100755
--- a/contrib/fast-import/import-tars.perl
+++ b/contrib/fast-import/import-tars.perl
@@ -8,9 +8,20 @@
## perl import-tars.perl *.tar.bz2
## git whatchanged import-tars
##
+## Use --metainfo to specify the extension for a meta data file, where
+## import-tars can read the commit message and optionally author and
+## committer information.
+##
+## echo 'This is the commit message' > myfile.tar.bz2.msg
+## perl import-tars.perl --metainfo=msg myfile.tar.bz2
use strict;
-die "usage: import-tars *.tar.{gz,bz2,Z}\n" unless @ARGV;
+use Getopt::Long;
+
+my $metaext = '';
+
+die "usage: import-tars [--metainfo=extension] *.tar.{gz,bz2,Z}\n"
+ unless GetOptions('metainfo=s' => \$metaext) && @ARGV;
my $branch_name = 'import-tars';
my $branch_ref = "refs/heads/$branch_name";
@@ -109,12 +120,47 @@ foreach my $tar_file (@ARGV)
$have_top_dir = 0 if $top_dir ne $1;
}
+ my $commit_msg = "Imported from $tar_file.";
+ my $this_committer_name = $committer_name;
+ my $this_committer_email = $committer_email;
+ my $this_author_name = $author_name;
+ my $this_author_email = $author_email;
+ if ($metaext ne '')
+ {
+ # Optionally read a commit message from <filename.tar>.msg
+ # Add a line on the form "Committer: name <e-mail>" to override
+ # the committer and "Author: name <e-mail>" to override the
+ # author for this tar ball.
+ if (open MSG, '<', "${tar_file}.${metaext}")
+ {
+ $commit_msg = '';
+ while (<MSG>)
+ {
+ if (/^Committer:\s+([^<>]*)\s+<(.*)>\s*$/i)
+ {
+ $this_committer_name = $1;
+ $this_committer_email = $2;
+ }
+ elsif (/^Author:\s+([^<>]*)\s+<(.*)>\s*$/i)
+ {
+ $this_author_name = $1;
+ $this_author_email = $2;
+ }
+ else
+ {
+ $commit_msg .= $_;
+ }
+ }
+ close MSG;
+ }
+ }
+
print FI <<EOF;
commit $branch_ref
-author $author_name <$author_email> $author_time +0000
-committer $committer_name <$committer_email> $commit_time +0000
+author $this_author_name <$this_author_email> $author_time +0000
+committer $this_committer_name <$this_committer_email> $commit_time +0000
data <<END_OF_COMMIT_MESSAGE
-Imported from $tar_file.
+$commit_msg
END_OF_COMMIT_MESSAGE
deleteall
--
1.6.3.3
^ permalink raw reply related
* [PATCH] git-gui: fix diff for partially staged submodule changes
From: Jens Lehmann @ 2009-08-26 20:25 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Junio C Hamano, git
When a submodule commit had already been staged and another commit had
been checked out inside the submodule, the diff always displayed the
submodule commit log messages between the last supermodule commit and
the working tree, totally ignoring the commit in the index.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
This fixes a regression introduced by this recent patch of mine:
"git-gui: display summary when showing diff of a submodule"
Sorry for the inconvenience.
git-gui/lib/diff.tcl | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/git-gui/lib/diff.tcl b/git-gui/lib/diff.tcl
index ae1ea3a..d593323 100644
--- a/git-gui/lib/diff.tcl
+++ b/git-gui/lib/diff.tcl
@@ -298,7 +298,12 @@ proc start_show_diff {cont_info {add_opts {}}} {
if {[string match {160000 *} [lindex $s 2]]
|| [string match {160000 *} [lindex $s 3]]} {
- set cmd {submodule summary -- $current_diff_path}
+ set is_submodule_diff 1
+ if {$w eq $ui_index} {
+ set cmd {submodule summary --cached -- $current_diff_path}
+ } else {
+ set cmd {submodule summary --files -- $current_diff_path}
+ }
}
if {[catch {set fd [eval git_read --nice $cmd]} err]} {
@@ -343,9 +348,6 @@ proc read_diff {fd cont_info} {
}
set ::current_diff_inheader 0
- if {[regexp {^\* } $line]} {
- set is_submodule_diff 1
- }
# -- Automatically detect if this is a 3 way diff.
#
if {[string match {@@@ *} $line]} {set is_3way_diff 1}
--
1.6.4.184.ge7b6.dirty
^ permalink raw reply related
* Re: [git-svn] [BUG] merge-tracking inconsistencies; Was: [FEATURE-REQ] track merges from git
From: Ximin Luo @ 2009-08-26 20:55 UTC (permalink / raw)
To: Bryan Donlan; +Cc: git
In-Reply-To: <4A95A032.3000801@cam.ac.uk>
Ximin Luo wrote:
> the tip of trunk is NOT a tip of test1
er, that should read "the tip of trunk is NOT a parent of the tip of test1"
X
^ permalink raw reply
* Re: [git-svn] [BUG] merge-tracking inconsistencies; Was: [FEATURE-REQ] track merges from git
From: Ximin Luo @ 2009-08-26 21:13 UTC (permalink / raw)
To: Bryan Donlan; +Cc: git
In-Reply-To: <4A95A032.3000801@cam.ac.uk>
Ximin Luo wrote:
> The only difference between the two runs is that in the unfucked version, we
> run "git svn dcommit" after every git commit.
Hmm, now that I think about it, the "bug" would be quite hard to "fix"...
Basically, it happens if you try to dcommit a commit A which has two parents, B
and X, where X is in a different branch, and hasn't already been dcommited. It
would seem that there isn't (in general) a way to detect whether X would become
(ie. in the future) a dcommitted svn commit - and actually this might not even
be the case, if eg. someone "svn commited" before we could get that dcommit in.
However about having git-svn outputting a warning when it detects merge
commits, one of whose parents is *not* a dcommitted commit, but does belongs to
a branch that is also being tracked by git-svn?
Something like "Warning: commit aaaa has parent bbbb; however, parent bbbb has
not been dcommited to the remote svn yet. If you proceed with this dcommit, the
merge history will be lost; to preserve the history, dcommit the branch
containing bbbb instead and then continue to dcommit this branch"?
X
^ permalink raw reply
* Re: What's cooking in git.git (Aug 2009, #04; Sun, 23)
From: Brandon Casey @ 2009-08-26 23:07 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <YE4QMh4rA1r2X3ZG5TvGJZspm0UdCWyP-r6KFthp8PuFewAhHPJ3GQ@cipher.nrlssc.navy.mil>
Brandon Casey wrote:
> Nicolas Pitre wrote:
>> On Sun, 23 Aug 2009, Junio C Hamano wrote:
>>
>>> * lt/block-sha1 (2009-08-17) 4 commits
>>> (merged to 'next' on 2009-08-18 at 67a1ce8)
>>> + remove ARM and Mozilla SHA1 implementations
>>> + block-sha1: guard gcc extensions with __GNUC__
>>> + make sure byte swapping is optimal for git
>>> + block-sha1: make the size member first in the context struct
>>>
>>> Finishing touches ;-) There were a few Solaris portability patches
>>> floated around that I didn't pick up, waiting for them to finalize.
>> Those would be described better as Solaris _optimization_ patches. The
>> code is already fully portable as it is, except not necessarily optimal
>> in some cases.
>
> Nicolas is right, the code compiles and executes correctly on Solaris as-is.
>
> Here is the state of the two unsubmitted optimization patches:
>
> 1) Change things like __i386__ to __i386 since GCC defines both, but
> SUNWspro only defines __i386.
>
> This works correctly in my testing. I'm assuming that a test for
> __amd64 is not necessary and expect that __x86_64 is set whenever
> __amd64 is set.
>
> 2) Set __GNUC__ on SUNWspro v5.10 and up.
>
> This compiles correctly and passes the test suite, but produces
> warnings for __attribute__'s that sun's compiler has not implemented.
> This produces a very noisy compile.
>
> I've wanted to do some performance testing to see whether this actually
> produces an _improvement_. I'll try today.
Ok, I've done some testing.
I've compiled on two Solaris 5.10 x86 boxes. One has Sun compiler 5.10,
the other has version 5.8. The 5.10 version supports GCC inline assembler,
statement expressions, and __builtin_x functions. I timed how long
'git fsck --full HEAD' took on the git repository (best of three runs at
each stage).
It seems that #1 provides almost 1% improvement when using Sun compiler
v5.10, but a 2.5% regression on compiler v5.8. #2 (implemented using
Junio's suggestion, not by setting __GNUC__), which additionally enables
the fast htonl/ntohl and the rol/ror assembly in block-sha1 when using
the v5.10 compiler, produces a performance regression. I tried compiling
with '-fast -native', and also with just '-O', and both were slower with
the addition of part #2.
If this is the only data point for non-GNU compilers on x86, and since
there was only <1% improvement with the v5.10 compiler, I'm inclined
to say that we leave the series as it is and don't apply either change.
-brandon
^ permalink raw reply
* Re: [PATCH] upload-pack: add a trigger for post-upload-pack hook
From: Junio C Hamano @ 2009-08-26 23:39 UTC (permalink / raw)
To: Jeff King, Tom Werner; +Cc: Junio C Hamano, Tom Preston-Werner, git
In-Reply-To: <7vprajmp16.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
>> Is there any other information that might be useful to other non-GitHub
>> users of the hook? The only thing I can think of is the list of refs
>> that were fetched.
>
> I do not think that information is available. "want" will tell you what
> object they want, but that does not necessarily uniquey translate to a
> ref.
>
> If we are allowed to talk about asking for the moon, and if one of the
> primary reason for this new hook is statistics, it would be useful to see
> the number of bytes given, where the fetch-pack came from, and if we are
> using git-daemon virtual hosting which of our domain served the request.
I briefly looked at upload-pack.c.
As I said, the names of the refs asked is not available but in that file,
create_pack_file() should be able to gather transfer statistics (timeing
and bytes transferred). It also has access to the want_obj[]/have_obj[]
array, so it should be able to keep the rev-list parameters to be fed to
the hook if it wanted to (and with that information you could recreate the
exact pack data later if necessary).
And want_obj[]/have_obj[] information is much more useful than a crude
"fetch/clone" distinction. You can not just tell if the clients have
nothing, but how stale they are.
So at a minimum, if this is primarily meant as a statistics hook, I would
suggest the hook not to take _any_ argument, but is fed the information
from its command line in the following form, one piece of information per
line, from the very beginning:
want 40-byte SHA-1 - what were in want_obj[] array
have 40-byte SHA-1 - what were in have_obj[] array
As long as the hook scripts are written to ignore lines they do not
understand, in later rounds, we should also be able to feed two more
pieces of information with minimum modification to create_pack_file():
time float - number of seconds create_pack_file spent,
size decimal - number of bytes transferred
Here is an illustration patch.
upload-pack.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 66 insertions(+), 2 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 4d8be83..69a6f46 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -141,8 +141,60 @@ static int do_rev_list(int fd, void *create_full_pack)
return 0;
}
+static int feed_obj_to_hook(const char *label, struct object_array *oa, int i, int fd)
+{
+ int cnt;
+ char buf[512];
+
+ cnt = sprintf(buf, "%s %s\n", label,
+ sha1_to_hex(oa->objects[i].item->sha1));
+ return write_in_full(fd, buf, cnt) != cnt;
+}
+
+static int run_post_upload_pack_hook(size_t total, struct timeval *tv)
+{
+ const char *argv[2];
+ struct child_process proc;
+ int err, i;
+ int cnt;
+ char buf[512];
+
+ argv[0] = "hooks/post-upload-pack";
+ argv[1] = NULL;
+
+ if (access(argv[0], X_OK) < 0)
+ return 0;
+
+ memset(&proc, 0, sizeof(proc));
+ proc.argv = argv;
+ proc.in = -1;
+ proc.stdout_to_stderr = 1;
+ err = start_command(&proc);
+ if (err)
+ return err;
+ for (i = 0; !err && i < want_obj.nr; i++)
+ err |= feed_obj_to_hook("want", &want_obj, i, proc.in);
+ for (i = 0; !err && i < have_obj.nr; i++)
+ err |= feed_obj_to_hook("have", &have_obj, i, proc.in);
+ if (!err) {
+ cnt = sprintf(buf, "time %ld.%06ld\n",
+ (long)tv->tv_sec, (long)tv->tv_usec);
+ err |= (write_in_full(proc.in, buf, cnt) != cnt);
+ }
+ if (!err) {
+ cnt = sprintf(buf, "size %ld\n", (long)total);
+ err |= (write_in_full(proc.in, buf, cnt) != cnt);
+ }
+ if (close(proc.in))
+ err = 1;
+ if (finish_command(&proc))
+ err = 1;
+ return err;
+}
+
static void create_pack_file(void)
{
+ struct timeval start_tv, tv;
struct async rev_list;
struct child_process pack_objects;
int create_full_pack = (nr_our_refs == want_obj.nr && !have_obj.nr);
@@ -150,10 +202,12 @@ static void create_pack_file(void)
char abort_msg[] = "aborting due to possible repository "
"corruption on the remote side.";
int buffered = -1;
- ssize_t sz;
+ ssize_t sz, total_sz;
const char *argv[10];
int arg = 0;
+ gettimeofday(&start_tv, NULL);
+ total_sz = 0;
if (shallow_nr) {
rev_list.proc = do_rev_list;
rev_list.data = 0;
@@ -262,7 +316,7 @@ static void create_pack_file(void)
sz = xread(pack_objects.out, cp,
sizeof(data) - outsz);
if (0 < sz)
- ;
+ total_sz += sz;
else if (sz == 0) {
close(pack_objects.out);
pack_objects.out = -1;
@@ -314,6 +368,16 @@ static void create_pack_file(void)
}
if (use_sideband)
packet_flush(1);
+
+ gettimeofday(&tv, NULL);
+ tv.tv_sec -= start_tv.tv_sec;
+ if (tv.tv_usec < start_tv.tv_usec) {
+ tv.tv_sec--;
+ tv.tv_usec += 1000000;
+ }
+ tv.tv_usec -= start_tv.tv_usec;
+ if (run_post_upload_pack_hook(total_sz, &tv))
+ warning("post-upload-hook failed");
return;
fail:
^ permalink raw reply related
* What's cooking in git.git (Aug 2009, #05; Wed, 26)
From: Junio C Hamano @ 2009-08-26 23:45 UTC (permalink / raw)
To: git
Here are the topics that have been cooking. Commits prefixed with '-' are
only in 'pu' while commits prefixed with '+' are in 'next'. The ones
marked with '.' do not appear in any of the integration branches, but I am
still holding onto them.
After the 1.6.5 cycle, the next release will be 1.7.0, and we will push
out the planned "push safety" change. 1.7.0 would be a good time to
introduce "justifiable" changes that are not strictly backward compatible.
During 1.6.5 cycle, 'next' will hold topics meant for 1.6.5 and 1.7.0.
--------------------------------------------------
[Graduated to "master"]
* aj/fix-read-tree-from-scratch (2009-08-17) 1 commit
(merged to 'next' on 2009-08-20 at 7a04133)
+ read-tree: Fix regression with creation of a new index file.
* jc/maint-checkout-index-to-prefix (2009-08-16) 1 commit
(merged to 'next' on 2009-08-20 at 2f6aea2)
+ check_path(): allow symlinked directories to checkout-index --prefix
* jl/submodule-summary-diff-files (2009-08-15) 2 commits
(merged to 'next' on 2009-08-15 at 165bd8e)
+ Documentaqtion/git-submodule.txt: Typofix
(merged to 'next' on 2009-08-14 at a702e78)
+ git submodule summary: add --files option
* lh/short-decorate (2009-08-15) 1 commit
(merged to 'next' on 2009-08-18 at b8c1d96)
+ git-log: allow --decorate[=short|full]
* oa/stash-na (2009-08-11) 1 commit
(merged to 'next' on 2009-08-14 at 12c2e2b)
+ git stash: Give friendlier errors when there is nothing to apply
--------------------------------------------------
[New Topics]
* np/maint-1.6.3-deepen (2009-08-24) 1 commit.
(merged to 'next' on 2009-08-25 at 8e383d4)
+ fix simple deepening of a repo
* jk/maint-1.6.3-checkout-unborn (2009-08-24) 1 commit.
(merged to 'next' on 2009-08-25 at 5f29625)
+ checkout: do not imply "-f" on unborn branches
* mr/gitweb-snapshot (2009-08-25) 3 commits
- gitweb: add t9501 tests for checking HTTP status codes
- gitweb: split test suite into library and tests
- gitweb: improve snapshot error handling
--------------------------------------------------
[Stalled]
* js/stash-dwim (2009-07-27) 1 commit.
(merged to 'next' on 2009-08-16 at 67896c4)
+ Make 'git stash -k' a short form for 'git stash save --keep-index'
(this branch is used by tr/reset-checkout-patch.)
* tr/reset-checkout-patch (2009-08-18) 8 commits.
(merged to 'next' on 2009-08-18 at e465bb3)
+ tests: disable interactive hunk selection tests if perl is not available
(merged to 'next' on 2009-08-16 at 67896c4)
+ DWIM 'git stash save -p' for 'git stash -p'
+ Implement 'git stash save --patch'
+ Implement 'git checkout --patch'
+ Implement 'git reset --patch'
+ builtin-add: refactor the meat of interactive_add()
+ Add a small patch-mode testing library
+ git-apply--interactive: Refactor patch mode code
(this branch uses js/stash-dwim.)
There was a discussion on better DWIMmery for the above two topics to (1)
forbid "git stash save --anything-with-dash" and (2) redirect with any
option "git stash --opt" to "git stash save --opt", to keep it flexible
and safe at the same time. I think it is a sane thing to do, but nothing
has happened lately.
* jn/gitweb-blame (2009-08-06) 3 commits
- gitweb: Create links leading to 'blame_incremental' using JavaScript
- gitweb: Incremental blame (WIP)
- gitweb: Add optional "time to generate page" info in footer
Ajax-y blame WIP
* db/vcs-helper (2009-08-09) 17 commits
- Allow helpers to request marks for fast-import
- Allow helpers to report in "list" command that the ref is unchanged
- Add support for "import" helper command
- transport-helper_init(): fix a memory leak in error path
- Add a config option for remotes to specify a foreign vcs
- Allow programs to not depend on remotes having urls
- Allow fetch to modify refs
- Use a function to determine whether a remote is valid
- Use a clearer style to issue commands to remote helpers
(merged to 'next' on 2009-08-07 at f3533ba)
+ Makefile: install hardlinks for git-remote-<scheme> supported by libcurl if possible
+ Makefile: do not link three copies of git-remote-* programs
+ Makefile: git-http-fetch does not need expat
(merged to 'next' on 2009-08-06 at 15da79d)
+ http-fetch: Fix Makefile dependancies
+ Add transport native helper executables to .gitignore
(merged to 'next' on 2009-08-05 at 33d491e)
+ git-http-fetch: not a builtin
+ Use an external program to implement fetching with curl
+ Add support for external programs for handling native fetches
(this branch is used by jh/cvs-helper.)
There was a discussion that suggests that the use of colon ':' before vcs
helper name needs to be corrected. Nothing happened since.
* je/send-email-no-subject (2009-08-05) 1 commit
- send-email: confirm on empty mail subjects
This seems to break t9001. Near the tip of 'pu' I have a iffy
workaround.
--------------------------------------------------
[Cooking]
* mm/reset-report (2009-08-21) 2 commits
(merged to 'next' on 2009-08-25 at f2a4424)
+ reset: make the reminder output consistent with "checkout"
+ Rename REFRESH_SAY_CHANGED to REFRESH_IN_PORCELAIN.
* wl/insta-mongoose (2009-08-21) 1 commit
(merged to 'next' on 2009-08-25 at da1d566)
+ Add support for the Mongoose web server.
* lt/approxidate (2009-08-22) 2 commits
(merged to 'next' on 2009-08-26 at 62853f9)
+ Further 'approxidate' improvements
+ Improve on 'approxidate'
Fixes a few "reasonably formatted but thus-far misparsed" date strings.
As Nico suggested, we would need a test to prevent regression to existing
support for date strings that are "reasonably formatted".
* jc/mailinfo-scissors (2009-08-25) 2 commits
- Documentation: describe the scissors mark support of "git am"
- Teach mailinfo to ignore everything before -- >8 -- mark
* tf/diff-whitespace-incomplete-line (2009-08-23) 2 commits.
(merged to 'next' on 2009-08-26 at 4fc7784)
+ xutils: Fix xdl_recmatch() on incomplete lines
+ xutils: Fix hashing an incomplete line with whitespaces at the end
* cc/sequencer-rebase-i (2009-08-21) 17 commits.
- rebase -i: use "git sequencer--helper --cherry-pick"
- sequencer: add "--cherry-pick" option to "git sequencer--helper"
- sequencer: add "do_commit()" and related functions
- pick: libify "pick_help_msg()"
- revert: libify pick
- rebase -i: use "git sequencer--helper --fast-forward"
- sequencer: let "git sequencer--helper" callers set "allow_dirty"
- sequencer: add "--fast-forward" option to "git sequencer--helper"
- sequencer: add "do_fast_forward()" to perform a fast forward
- rebase -i: use "git sequencer--helper --reset-hard"
- sequencer: add "--reset-hard" option to "git sequencer--helper"
- sequencer: add comments about reset_almost_hard()
- sequencer: add "reset_almost_hard()" and related functions
- rebase -i: use "git sequencer--helper --make-patch"
- sequencer: free memory used in "make_patch" function
- sequencer: add "make_patch" function to save a patch
- sequencer: add "builtin-sequencer--helper.c"
Migrating "rebase -i" bit by bit to C. I am inclined to agree with Dscho
that maybe this approach forces the migration to follow the structure of
the shell script too much, and could force a suboptimal end result, but
we'll see.
* as/maint-graph-interesting-fix (2009-08-21) 2 commits.
(merged to 'next' on 2009-08-25 at 9d5e215)
+ Add tests for rev-list --graph with options that simplify history
+ graph API: fix bug in graph_is_interesting()
Looked sane.
* jc/shortstatus (2009-08-15) 11 commits
(merged to 'next' on 2009-08-15 at 7e40766)
+ git commit --dry-run -v: show diff in color when asked
+ Documentation/git-commit.txt: describe --dry-run
(merged to 'next' on 2009-08-12 at 53bda17)
+ wt-status: collect untracked files in a separate "collect" phase
+ Make git_status_config() file scope static to builtin-commit.c
+ wt-status: move wt_status_colors[] into wt_status structure
+ wt-status: move many global settings to wt_status structure
+ commit: --dry-run
(merged to 'next' on 2009-08-06 at fe8cb94)
+ status: show worktree status of conflicted paths separately
+ wt-status.c: rework the way changes to the index and work tree are summarized
+ diff-index: keep the original index intact
+ diff-index: report unmerged new entries
(this branch is used by jc/1.7.0-status.)
Will cook for a bit more and then merge.
* jc/maint-unpack-objects-strict (2009-08-13) 1 commit.
(merged to 'next' on 2009-08-23 at 38eb750)
+ Fix "unpack-objects --strict"
Will merge.
* jh/submodule-foreach (2009-08-20) 9 commits
(merged to 'next' on 2009-08-20 at 671bea4)
+ git clone: Add --recursive to automatically checkout (nested) submodules
+ t7407: Use 'rev-parse --short' rather than bash's substring expansion notation
(merged to 'next' on 2009-08-18 at f4a881d)
+ git submodule status: Add --recursive to recurse into nested submodules
+ git submodule update: Introduce --recursive to update nested submodules
+ git submodule foreach: Add --recursive to recurse into nested submodules
+ git submodule foreach: test access to submodule name as '$name'
+ Add selftest for 'git submodule foreach'
+ git submodule: Cleanup usage string and add option parsing to cmd_foreach()
+ git submodule foreach: Provide access to submodule name, as '$name'
Will merge.
* jh/notes (2009-07-29) 8 commits.
- t3302-notes-index-expensive: Speed up create_repo()
- fast-import: Add support for importing commit notes
- First draft of notes tree parser with support for fanout subtrees
- Teach "-m <msg>" and "-F <file>" to "git notes edit"
- Add an expensive test for git-notes
- Speed up git notes lookup
- Add a script to edit/inspect notes
- Introduce commit notes
The cvs-helper series depends on this one. I have a recollection
that some people were not happy about the fan-out of the notes tree
layout, but has the issue been resolved to a concensus?
* ne/rev-cache (2009-08-21) 6 commits
. support for path name caching in rev-cache
. full integration of rev-cache into git, completed test suite
. administrative functions for rev-cache, start of integration into git
. support for non-commit object caching in rev-cache
. basic revision cache system, no integration or features
. man page and technical discussion for rev-cache
Updated but seems to break upload-pack tests when merged to 'pu'; given
what this series touches, breakages in that area are expected.
May discard if a working reroll comes, to give it a fresh start.
* jh/cvs-helper (2009-08-18) 7 commits
- More fixes to the git-remote-cvs installation procedure
- Fix the Makefile-generated path to the git_remote_cvs package in git-remote-cvs
- Add simple selftests of git-remote-cvs functionality
- git-remote-cvs: Remote helper program for CVS repositories
- 2/2: Add Python support library for CVS remote helper
- 1/2: Add Python support library for CVS remote helper
- Basic build infrastructure for Python scripts
(this branch uses db/vcs-helper.)
Builds on db/vcs-helper (which is stalled, so this cannot move).
The testing of Python part seemed to be still fragile even with the latest
fix on one of my boches with an earlier round already installed, but I
didn't look very deeply before removing the older installation.
* sr/gfi-options (2009-08-24) 4 commits
- fast-import: test the new option command
- fast-import: add option command
- fast-import: put marks reading in it's own function
- fast-import: put option parsing code in seperate functions
Will merge to 'next' shortly.
* lt/block-sha1 (2009-08-17) 4 commits
(merged to 'next' on 2009-08-18 at 67a1ce8)
+ remove ARM and Mozilla SHA1 implementations
+ block-sha1: guard gcc extensions with __GNUC__
+ make sure byte swapping is optimal for git
+ block-sha1: make the size member first in the context struct
May merge soon; Solaris performance patches that was discussed
earlier can happen on 'master', as the series is usable as-is.
* nd/sparse (2009-08-20) 20 commits
- sparse checkout: inhibit empty worktree
- Add tests for sparse checkout
- read-tree: add --no-sparse-checkout to disable sparse checkout support
- unpack-trees(): ignore worktree check outside checkout area
- unpack_trees(): apply $GIT_DIR/info/sparse-checkout to the final index
- unpack-trees(): "enable" sparse checkout and load $GIT_DIR/info/sparse-checkout
- unpack-trees.c: generalize verify_* functions
- unpack-trees(): add CE_WT_REMOVE to remove on worktree alone
- Introduce "sparse checkout"
- dir.c: export excluded_1() and add_excludes_from_file_1()
- excluded_1(): support exclude files in index
- unpack-trees(): carry skip-worktree bit over in merged_entry()
- Read .gitignore from index if it is skip-worktree
- Avoid writing to buffer in add_excludes_from_file_1()
- Teach Git to respect skip-worktree bit (writing part)
- Teach Git to respect skip-worktree bit (reading part)
- Introduce "skip-worktree" bit in index, teach Git to get/set this bit
- Add test-index-version
- update-index: refactor mark_valid() in preparation for new options
(merged to 'next' on 2009-08-20 at ea167d7)
+ Prevent diff machinery from examining assume-unchanged entries on worktree
The first one was an independent fix; the rest has been replaced with the
"return of no-checkout" series.
--------------------------------------------------
[For 1.7.0]
* jc/1.7.0-status (2009-08-15) 3 commits
(merged to 'next' on 2009-08-22 at b3507bb)
+ git status: not "commit --dry-run" anymore
+ git stat -s: short status output
+ git stat: the beginning of "status that is not a dry-run of commit"
(this branch uses jc/shortstatus.)
With this, "git status" is no longer "git commit --preview".
* jc/1.7.0-send-email-no-thread-default (2009-08-22) 1 commit
(merged to 'next' on 2009-08-22 at 5106de8)
+ send-email: make --no-chain-reply-to the default
* jc/1.7.0-diff-whitespace-only-status (2009-05-23) 2 commits.
(merged to 'next' on 2009-08-02 at 9c08420)
+ diff: Rename QUIET internal option to QUICK
+ diff: change semantics of "ignore whitespace" options
This changes exit code from "git diff --ignore-whitespace" and friends
when there is no actual output. It is a backward incompatible change, but
we could argue that it is a bugfix.
* jc/1.7.0-push-safety (2009-02-09) 2 commits
(merged to 'next' on 2009-08-02 at 38b82fe)
+ Refuse deleting the current branch via push
+ Refuse updating the current branch in a non-bare repository via push
--------------------------------------------------
[I have been too busy to purge these]
* jc/log-tz (2009-03-03) 1 commit.
- Allow --date=local --date=other-format to work as expected
Maybe some people care about this. I dunno.
* jc/mailinfo-remove-brackets (2009-07-15) 1 commit.
- mailinfo: -b option keeps [bracketed] strings that is not a [PATCH] marker
Maybe some people care about this. I dunno.
* ar/maint-1.6.2-merge-recursive-d-f (2009-05-11) 2 commits.
. Fix for a merge where a branch has an F->D transition
. Add a reminder test case for a merge with F/D transition
* jc/merge-convert (2009-01-26) 1 commit.
. git-merge-file: allow converting the results for the work tree
* lt/read-directory (2009-05-15) 3 commits.
. Add initial support for pathname conversion to UTF-8
. read_directory(): infrastructure for pathname character set conversion
. Add 'fill_directory()' helper function for directory traversal
* ps/blame (2009-03-12) 1 commit.
. blame.c: start libifying the blame infrastructure
* pb/tracking (2009-07-16) 7 commits.
. branch.c: if remote is not config'd for branch, don't try delete push config
. branch, checkout: introduce autosetuppush
. move deletion of merge configuration to branch.c
. remote: add per-remote autosetupmerge and autosetuprebase configuration
. introduce a struct tracking_config
. branch: install_branch_config and struct tracking refactoring
. config: allow false and true values for branch.autosetuprebase
Has been ejected from 'pu' for some time, expecting a reroll.
^ permalink raw reply
* Re: What's cooking in git.git (Aug 2009, #05; Wed, 26)
From: Shawn O. Pearce @ 2009-08-26 23:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, sverre
In-Reply-To: <7vfxbeb0mt.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
> * sr/gfi-options (2009-08-24) 4 commits
> - fast-import: test the new option command
> - fast-import: add option command
> - fast-import: put marks reading in it's own function
> - fast-import: put option parsing code in seperate functions
>
> Will merge to 'next' shortly.
Please don't.
There is an off-git ML thread running between the various DVCS
tool developers who work on the fast-import/fast-export tools for
the respective DVCSes. In that thread we have decided to slightly
change this grammar and this series needs to be respun.
If you are itching to do something, eject it from pu and wait for
the respin.
--
Shawn.
^ permalink raw reply
* Re: What's cooking in git.git (Aug 2009, #05; Wed, 26)
From: Sverre Rabbelier @ 2009-08-27 0:10 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Junio C Hamano, git
In-Reply-To: <20090826234903.GR1033@spearce.org>
Heya,
On Wed, Aug 26, 2009 at 16:49, Shawn O. Pearce<spearce@spearce.org> wrote:
> In that thread we have decided to slightly
> change this grammar and this series needs to be respun.
Speaking of which, do you want me to add the feature command to the
grammar and rebase this on top of that? I got the impression we were
far enough discussing that for at least an RPC patch?
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: What's cooking in git.git (Aug 2009, #05; Wed, 26)
From: Shawn O. Pearce @ 2009-08-27 0:12 UTC (permalink / raw)
To: Sverre Rabbelier; +Cc: Junio C Hamano, git
In-Reply-To: <fabb9a1e0908261710l2a957basff5eb5d7225ce099@mail.gmail.com>
Sverre Rabbelier <srabbelier@gmail.com> wrote:
> On Wed, Aug 26, 2009 at 16:49, Shawn O. Pearce<spearce@spearce.org> wrote:
> > In that thread we have decided to slightly
> > change this grammar and this series needs to be respun.
>
> Speaking of which, do you want me to add the feature command to the
> grammar and rebase this on top of that? I got the impression we were
> far enough discussing that for at least an RPC patch?
RFC patch. And yes, because I think we already agreed in that
thread that the date-format option is actually a feature command,
and not an option command. The other feature commands being kicked
around can be held for another series.
--
Shawn.
^ permalink raw reply
* [git-svn] always prompted for passphrase with subversion 1.6
From: Tim Potter @ 2009-08-27 0:38 UTC (permalink / raw)
To: git
Hi everyone. I am using git-svn with the Subversion 1.6 client compiled
with GNOME Keyring support. This neat features allows a SSL client
certificate password to be cached inside GNOME Keyring instead of being
prompted to enter it every time. However the git-svn script doesn't
appear to know about this and always prompts for a password.
Obviously there's some tweak required in the _auth_providers()
subroutine but I don't know enough about the Subversion Perl client to
figure out a fix.
Has anyone else run in to this problem? I did a quick search on the
list but didn't find anything relevant.
Regards,
Tim.
^ permalink raw reply
* [PATCH] upload-pack: add a trigger for post-upload-pack hook
From: Junio C Hamano @ 2009-08-27 0:47 UTC (permalink / raw)
To: Jeff King, Tom Werner; +Cc: Tom Preston-Werner, git
In-Reply-To: <7v1vmycfhd.fsf@alter.siamese.dyndns.org>
After upload-pack successfully finishes its operation, post-upload-pack
hook can be called for logging purposes.
The hook is passed various pieces of information, one per line, from its
standard input. Currently the following items can be fed to the hook, but
more types of information may be added in the future:
want SHA-1::
40-byte hexadecimal object name the client asked to include in the
resulting pack. Can occur one or more times in the input.
have SHA-1::
40-byte hexadecimal object name the client asked to exclude from
the resulting pack, claiming to have them already. Can occur zero
or more times in the input.
time float::
Number of seconds spent for creating the packfile.
size decimal::
Size of the resulting packfile in bytes.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
> Here is an illustration patch.
And here is a bit more polished one with necessary supporting material.
Documentation/git-upload-pack.txt | 2 +
Documentation/githooks.txt | 25 +++++++++++++
t/t5501-post-upload-pack.sh | 49 ++++++++++++++++++++++++++
upload-pack.c | 68 +++++++++++++++++++++++++++++++++++-
4 files changed, 142 insertions(+), 2 deletions(-)
create mode 100755 t/t5501-post-upload-pack.sh
diff --git a/Documentation/git-upload-pack.txt b/Documentation/git-upload-pack.txt
index b8e49dc..63f3b5c 100644
--- a/Documentation/git-upload-pack.txt
+++ b/Documentation/git-upload-pack.txt
@@ -20,6 +20,8 @@ The UI for the protocol is on the 'git-fetch-pack' side, and the
program pair is meant to be used to pull updates from a remote
repository. For push operations, see 'git-send-pack'.
+After finishing the operation successfully, `post-upload-pack`
+hook is called (see linkgit:githooks[5]).
OPTIONS
-------
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 1c73673..036f6c7 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -307,6 +307,31 @@ Both standard output and standard error output are forwarded to
'git-send-pack' on the other end, so you can simply `echo` messages
for the user.
+post-upload-pack
+----------------
+
+After upload-pack successfully finishes its operation, this hook is called
+for logging purposes.
+
+The hook is passed various pieces of information, one per line, from its
+standard input. Currently the following items can be fed to the hook, but
+more types of information may be added in the future:
+
+want SHA-1::
+ 40-byte hexadecimal object name the client asked to include in the
+ resulting pack. Can occur one or more times in the input.
+
+have SHA-1::
+ 40-byte hexadecimal object name the client asked to exclude from
+ the resulting pack, claiming to have them already. Can occur zero
+ or more times in the input.
+
+time float::
+ Number of seconds spent for creating the packfile.
+
+size decimal::
+ Size of the resulting packfile in bytes.
+
pre-auto-gc
-----------
diff --git a/t/t5501-post-upload-pack.sh b/t/t5501-post-upload-pack.sh
new file mode 100755
index 0000000..2cb63f8
--- /dev/null
+++ b/t/t5501-post-upload-pack.sh
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='post upload-hook'
+
+. ./test-lib.sh
+
+LOGFILE=".git/post-upload-pack-log"
+
+test_expect_success setup '
+ test_commit A &&
+ test_commit B &&
+ git reset --hard A &&
+ test_commit C &&
+ git branch prev B &&
+ mkdir -p .git/hooks &&
+ {
+ echo "#!$SHELL_PATH" &&
+ echo "cat >post-upload-pack-log"
+ } >".git/hooks/post-upload-pack" &&
+ chmod +x .git/hooks/post-upload-pack
+'
+
+: test_expect_success initial '
+ rm -fr sub &&
+ git init sub &&
+ (
+ cd sub &&
+ git fetch --no-tags .. prev
+ ) &&
+ want=$(sed -n "s/^want //p" "$LOGFILE") &&
+ test "$want" = "$(git rev-parse --verify B)" &&
+ ! grep "^have " "$LOGFILE"
+'
+
+test_expect_success second '
+ rm -fr sub &&
+ git init sub &&
+ (
+ cd sub &&
+ git fetch --no-tags .. prev:refs/remotes/prev &&
+ git fetch --no-tags .. master
+ ) &&
+ want=$(sed -n "s/^want //p" "$LOGFILE") &&
+ test "$want" = "$(git rev-parse --verify C)" &&
+ have=$(sed -n "s/^have //p" "$LOGFILE") &&
+ test "$have" = "$(git rev-parse --verify B)"
+'
+
+test_done
diff --git a/upload-pack.c b/upload-pack.c
index 4d8be83..69a6f46 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -141,8 +141,60 @@ static int do_rev_list(int fd, void *create_full_pack)
return 0;
}
+static int feed_obj_to_hook(const char *label, struct object_array *oa, int i, int fd)
+{
+ int cnt;
+ char buf[512];
+
+ cnt = sprintf(buf, "%s %s\n", label,
+ sha1_to_hex(oa->objects[i].item->sha1));
+ return write_in_full(fd, buf, cnt) != cnt;
+}
+
+static int run_post_upload_pack_hook(size_t total, struct timeval *tv)
+{
+ const char *argv[2];
+ struct child_process proc;
+ int err, i;
+ int cnt;
+ char buf[512];
+
+ argv[0] = "hooks/post-upload-pack";
+ argv[1] = NULL;
+
+ if (access(argv[0], X_OK) < 0)
+ return 0;
+
+ memset(&proc, 0, sizeof(proc));
+ proc.argv = argv;
+ proc.in = -1;
+ proc.stdout_to_stderr = 1;
+ err = start_command(&proc);
+ if (err)
+ return err;
+ for (i = 0; !err && i < want_obj.nr; i++)
+ err |= feed_obj_to_hook("want", &want_obj, i, proc.in);
+ for (i = 0; !err && i < have_obj.nr; i++)
+ err |= feed_obj_to_hook("have", &have_obj, i, proc.in);
+ if (!err) {
+ cnt = sprintf(buf, "time %ld.%06ld\n",
+ (long)tv->tv_sec, (long)tv->tv_usec);
+ err |= (write_in_full(proc.in, buf, cnt) != cnt);
+ }
+ if (!err) {
+ cnt = sprintf(buf, "size %ld\n", (long)total);
+ err |= (write_in_full(proc.in, buf, cnt) != cnt);
+ }
+ if (close(proc.in))
+ err = 1;
+ if (finish_command(&proc))
+ err = 1;
+ return err;
+}
+
static void create_pack_file(void)
{
+ struct timeval start_tv, tv;
struct async rev_list;
struct child_process pack_objects;
int create_full_pack = (nr_our_refs == want_obj.nr && !have_obj.nr);
@@ -150,10 +202,12 @@ static void create_pack_file(void)
char abort_msg[] = "aborting due to possible repository "
"corruption on the remote side.";
int buffered = -1;
- ssize_t sz;
+ ssize_t sz, total_sz;
const char *argv[10];
int arg = 0;
+ gettimeofday(&start_tv, NULL);
+ total_sz = 0;
if (shallow_nr) {
rev_list.proc = do_rev_list;
rev_list.data = 0;
@@ -262,7 +316,7 @@ static void create_pack_file(void)
sz = xread(pack_objects.out, cp,
sizeof(data) - outsz);
if (0 < sz)
- ;
+ total_sz += sz;
else if (sz == 0) {
close(pack_objects.out);
pack_objects.out = -1;
@@ -314,6 +368,16 @@ static void create_pack_file(void)
}
if (use_sideband)
packet_flush(1);
+
+ gettimeofday(&tv, NULL);
+ tv.tv_sec -= start_tv.tv_sec;
+ if (tv.tv_usec < start_tv.tv_usec) {
+ tv.tv_sec--;
+ tv.tv_usec += 1000000;
+ }
+ tv.tv_usec -= start_tv.tv_usec;
+ if (run_post_upload_pack_hook(total_sz, &tv))
+ warning("post-upload-hook failed");
return;
fail:
--
1.6.4.1.288.g10d22
^ permalink raw reply related
* [PATCHv4 00/12] git notes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
Another iteration of the 'git notes' feature. Rebased on top of 'next':
- Patches 1-7 are unchanged from (patches 1-5 + 7-8 of) the last iteration.
- Patch 8 introduces the new notes lookup code that offers both handling of
fanout subtrees, and other performance improvements.
- Patch 9 adds a selftest that verifies correct parsing of notes trees with
various fanouts.
- Patch 10 adds simple memory pooling to parts of the data structure from
patch 8. This improves performance slightly.
- Patches 11-12 adds the '%N' format specifier for pretty-printing commit
notes (as suggested by Dscho in the previous notes thread).
Some performance numbers from the notes lookup code:
(these numbers are from my Core 2 Quad, 4 GB RAM)
Test scenario I: Running t3302-notes-index-expensive
Timing numbers from the 'time_notes 100' following 'create_repo 10000'
no-notes notes
before 16.22s 23.74s
after 16.31s 22.16s
after+mempool 16.24s 22.03s
Comments: This is a worst case scenario for pretty much any notes lookup
algorithm: Looking up all 10000 notes with no fanout (fanout level 0).
The new implementation does marginally better than the old.
Test scenario II: Repo with 100,000 commits, 1 note per commit.
Timing 100 repetitions of 'git log -n 10 refs/heads/master >/dev/null'
without notes fanout level 0 fanout level 1 fanout level 2
before 0.20s 32.44s N/A N/A
after 0.19s 16.66s 0.85s 0.61s
after+mempool 0.19s 16.20s 0.83s 0.57s
Comments: This hopefully gives a better simulation of a common use case
(displaying only a handful of commits, and their notes). In the (relative)
worst case (fanout level 0), the new code almost twice as fast as the old one.
As we add fanout, the runtime plummets (since we only need to unpack a handful
of subtrees).
In practice, this means that with even a modest 2/38 or 2/2/36 fanout in the
notes tree (fanout level 1 and 2, respectively), the 'git log' user experience
goes from unbearable to barely noticeable in a repo with hundreds of thousands
of notes.
Have fun! :)
...Johan
Johan Herland (7):
Teach "-m <msg>" and "-F <file>" to "git notes edit"
fast-import: Add support for importing commit notes
t3302-notes-index-expensive: Speed up create_repo()
Teach the notes lookup code to parse notes trees with various fanout schemes
Selftests verifying semantics when loading notes trees with various fanouts
notes.c: Implement simple memory pooling of leaf nodes
Add flags to get_commit_notes() to control the format of the note string
Johannes Schindelin (5):
Introduce commit notes
Add a script to edit/inspect notes
Speed up git notes lookup
Add an expensive test for git-notes
Add '%N'-format for pretty-printing commit notes
.gitignore | 1 +
Documentation/config.txt | 13 ++
Documentation/git-fast-import.txt | 45 +++++-
Documentation/git-notes.txt | 60 +++++++
Documentation/pretty-formats.txt | 1 +
Makefile | 3 +
cache.h | 4 +
command-list.txt | 1 +
commit.c | 1 +
config.c | 5 +
environment.c | 1 +
fast-import.c | 88 +++++++++-
git-notes.sh | 121 +++++++++++++
notes.c | 338 +++++++++++++++++++++++++++++++++++++
notes.h | 10 +
pretty.c | 10 +
t/t3301-notes.sh | 150 ++++++++++++++++
t/t3302-notes-index-expensive.sh | 118 +++++++++++++
t/t3303-notes-subtrees.sh | 206 ++++++++++++++++++++++
t/t9300-fast-import.sh | 166 ++++++++++++++++++
20 files changed, 1332 insertions(+), 10 deletions(-)
create mode 100644 Documentation/git-notes.txt
create mode 100755 git-notes.sh
create mode 100644 notes.c
create mode 100644 notes.h
create mode 100755 t/t3301-notes.sh
create mode 100755 t/t3302-notes-index-expensive.sh
create mode 100755 t/t3303-notes-subtrees.sh
^ permalink raw reply
* [PATCHv4 01/12] Introduce commit notes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Commit notes are blobs which are shown together with the commit
message. These blobs are taken from the notes ref, which you can
configure by the config variable core.notesRef, which in turn can
be overridden by the environment variable GIT_NOTES_REF.
The notes ref is a branch which contains "files" whose names are
the names of the corresponding commits (i.e. the SHA-1).
The rationale for putting this information into a ref is this: we
want to be able to fetch and possibly union-merge the notes,
maybe even look at the date when a note was introduced, and we
want to store them efficiently together with the other objects.
This patch has been improved by the following contributions:
- Thomas Rast: fix core.notesRef documentation
- Tor Arne Vestbø: fix printing of multi-line notes
- Alex Riesen: Using char array instead of char pointer costs less BSS
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Tor Arne Vestbø <tavestbo@trolltech.com>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Documentation/config.txt | 13 +++++++++
Makefile | 2 +
cache.h | 4 +++
commit.c | 1 +
config.c | 5 +++
environment.c | 1 +
notes.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++
notes.h | 7 +++++
pretty.c | 5 +++
9 files changed, 106 insertions(+), 0 deletions(-)
create mode 100644 notes.c
create mode 100644 notes.h
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 5256c7f..645775f 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -439,6 +439,19 @@ On some file system/operating system combinations, this is unreliable.
Set this config setting to 'rename' there; However, This will remove the
check that makes sure that existing object files will not get overwritten.
+core.notesRef::
+ When showing commit messages, also show notes which are stored in
+ the given ref. This ref is expected to contain files named
+ after the full SHA-1 of the commit they annotate.
++
+If such a file exists in the given ref, the referenced blob is read, and
+appended to the commit message, separated by a "Notes:" line. If the
+given ref itself does not exist, it is not an error, but means that no
+notes should be printed.
++
+This setting defaults to "refs/notes/commits", and can be overridden by
+the `GIT_NOTES_REF` environment variable.
+
add.ignore-errors::
Tells 'git-add' to continue adding files when some files cannot be
added due to indexing errors. Equivalent to the '--ignore-errors'
diff --git a/Makefile b/Makefile
index 4190a5d..95571db 100644
--- a/Makefile
+++ b/Makefile
@@ -426,6 +426,7 @@ LIB_H += ll-merge.h
LIB_H += log-tree.h
LIB_H += mailmap.h
LIB_H += merge-recursive.h
+LIB_H += notes.h
LIB_H += object.h
LIB_H += pack.h
LIB_H += pack-refs.h
@@ -509,6 +510,7 @@ LIB_OBJS += match-trees.o
LIB_OBJS += merge-file.o
LIB_OBJS += merge-recursive.o
LIB_OBJS += name-hash.o
+LIB_OBJS += notes.o
LIB_OBJS += object.o
LIB_OBJS += pack-check.o
LIB_OBJS += pack-refs.o
diff --git a/cache.h b/cache.h
index 808daba..739b42c 100644
--- a/cache.h
+++ b/cache.h
@@ -371,6 +371,8 @@ static inline enum object_type object_type(unsigned int mode)
#define GITATTRIBUTES_FILE ".gitattributes"
#define INFOATTRIBUTES_FILE "info/attributes"
#define ATTRIBUTE_MACRO_PREFIX "[attr]"
+#define GIT_NOTES_REF_ENVIRONMENT "GIT_NOTES_REF"
+#define GIT_NOTES_DEFAULT_REF "refs/notes/commits"
extern int is_bare_repository_cfg;
extern int is_bare_repository(void);
@@ -565,6 +567,8 @@ enum object_creation_mode {
extern enum object_creation_mode object_creation_mode;
+extern char *notes_ref_name;
+
extern int grafts_replace_parents;
#define GIT_REPO_VERSION 0
diff --git a/commit.c b/commit.c
index e2bcbe8..ae6bff4 100644
--- a/commit.c
+++ b/commit.c
@@ -5,6 +5,7 @@
#include "utf8.h"
#include "diff.h"
#include "revision.h"
+#include "notes.h"
int save_commit_buffer = 1;
diff --git a/config.c b/config.c
index e87edea..70a7d34 100644
--- a/config.c
+++ b/config.c
@@ -467,6 +467,11 @@ static int git_default_core_config(const char *var, const char *value)
return 0;
}
+ if (!strcmp(var, "core.notesref")) {
+ notes_ref_name = xstrdup(value);
+ return 0;
+ }
+
if (!strcmp(var, "core.pager"))
return git_config_string(&pager_program, var, value);
diff --git a/environment.c b/environment.c
index 5de6837..571ab56 100644
--- a/environment.c
+++ b/environment.c
@@ -49,6 +49,7 @@ enum push_default_type push_default = PUSH_DEFAULT_MATCHING;
#define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS
#endif
enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
+char *notes_ref_name;
int grafts_replace_parents = 1;
/* Parallel index stat data preload? */
diff --git a/notes.c b/notes.c
new file mode 100644
index 0000000..401966d
--- /dev/null
+++ b/notes.c
@@ -0,0 +1,68 @@
+#include "cache.h"
+#include "commit.h"
+#include "notes.h"
+#include "refs.h"
+#include "utf8.h"
+#include "strbuf.h"
+
+static int initialized;
+
+void get_commit_notes(const struct commit *commit, struct strbuf *sb,
+ const char *output_encoding)
+{
+ static const char utf8[] = "utf-8";
+ struct strbuf name = STRBUF_INIT;
+ unsigned char sha1[20];
+ char *msg, *msg_p;
+ unsigned long linelen, msglen;
+ enum object_type type;
+
+ if (!initialized) {
+ const char *env = getenv(GIT_NOTES_REF_ENVIRONMENT);
+ if (env)
+ notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
+ else if (!notes_ref_name)
+ notes_ref_name = GIT_NOTES_DEFAULT_REF;
+ if (notes_ref_name && read_ref(notes_ref_name, sha1))
+ notes_ref_name = NULL;
+ initialized = 1;
+ }
+
+ if (!notes_ref_name)
+ return;
+
+ strbuf_addf(&name, "%s:%s", notes_ref_name,
+ sha1_to_hex(commit->object.sha1));
+ if (get_sha1(name.buf, sha1))
+ return;
+
+ if (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||
+ type != OBJ_BLOB)
+ return;
+
+ if (output_encoding && *output_encoding &&
+ strcmp(utf8, output_encoding)) {
+ char *reencoded = reencode_string(msg, output_encoding, utf8);
+ if (reencoded) {
+ free(msg);
+ msg = reencoded;
+ msglen = strlen(msg);
+ }
+ }
+
+ /* we will end the annotation by a newline anyway */
+ if (msglen && msg[msglen - 1] == '\n')
+ msglen--;
+
+ strbuf_addstr(sb, "\nNotes:\n");
+
+ for (msg_p = msg; msg_p < msg + msglen; msg_p += linelen + 1) {
+ linelen = strchrnul(msg_p, '\n') - msg_p;
+
+ strbuf_addstr(sb, " ");
+ strbuf_add(sb, msg_p, linelen);
+ strbuf_addch(sb, '\n');
+ }
+
+ free(msg);
+}
diff --git a/notes.h b/notes.h
new file mode 100644
index 0000000..79d21b6
--- /dev/null
+++ b/notes.h
@@ -0,0 +1,7 @@
+#ifndef NOTES_H
+#define NOTES_H
+
+void get_commit_notes(const struct commit *commit, struct strbuf *sb,
+ const char *output_encoding);
+
+#endif
diff --git a/pretty.c b/pretty.c
index f5983f8..e25db81 100644
--- a/pretty.c
+++ b/pretty.c
@@ -6,6 +6,7 @@
#include "string-list.h"
#include "mailmap.h"
#include "log-tree.h"
+#include "notes.h"
#include "color.h"
static char *user_format;
@@ -975,5 +976,9 @@ void pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit,
*/
if (fmt == CMIT_FMT_EMAIL && sb->len <= beginning_of_body)
strbuf_addch(sb, '\n');
+
+ if (fmt != CMIT_FMT_ONELINE)
+ get_commit_notes(commit, sb, encoding);
+
free(reencoded);
}
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 02/12] Add a script to edit/inspect notes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
The script 'git notes' allows you to edit and show commit notes, by
calling either
git notes show <commit>
or
git notes edit <commit>
This patch has been improved by the following contributions:
- Tor Arne Vestbø: fix printing of multi-line notes
- Michael J Gruber: test and handle empty notes gracefully
- Thomas Rast:
- only clean up message file when editing
- use GIT_EDITOR and core.editor over VISUAL/EDITOR
- t3301: fix confusing quoting in test for valid notes ref
- t3301: use test_must_fail instead of !
- refuse to edit notes outside refs/notes/
- Junio C Hamano: tests: fix "export var=val"
- Christian Couder: documentation: fix 'linkgit' macro in "git-notes.txt"
- Johan Herland: minor cleanup and bugfixing in git-notes.sh (v2)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Tor Arne Vestbø <tavestbo@trolltech.com>
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
.gitignore | 1 +
Documentation/git-notes.txt | 46 +++++++++++++++++
Makefile | 1 +
command-list.txt | 1 +
git-notes.sh | 73 +++++++++++++++++++++++++++
t/t3301-notes.sh | 114 +++++++++++++++++++++++++++++++++++++++++++
6 files changed, 236 insertions(+), 0 deletions(-)
create mode 100644 Documentation/git-notes.txt
create mode 100755 git-notes.sh
create mode 100755 t/t3301-notes.sh
diff --git a/.gitignore b/.gitignore
index c446290..703241b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -86,6 +86,7 @@ git-mktag
git-mktree
git-name-rev
git-mv
+git-notes
git-pack-redundant
git-pack-objects
git-pack-refs
diff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt
new file mode 100644
index 0000000..7136016
--- /dev/null
+++ b/Documentation/git-notes.txt
@@ -0,0 +1,46 @@
+git-notes(1)
+============
+
+NAME
+----
+git-notes - Add/inspect commit notes
+
+SYNOPSIS
+--------
+[verse]
+'git-notes' (edit | show) [commit]
+
+DESCRIPTION
+-----------
+This command allows you to add notes to commit messages, without
+changing the commit. To discern these notes from the message stored
+in the commit object, the notes are indented like the message, after
+an unindented line saying "Notes:".
+
+To disable commit notes, you have to set the config variable
+core.notesRef to the empty string. Alternatively, you can set it
+to a different ref, something like "refs/notes/bugzilla". This setting
+can be overridden by the environment variable "GIT_NOTES_REF".
+
+
+SUBCOMMANDS
+-----------
+
+edit::
+ Edit the notes for a given commit (defaults to HEAD).
+
+show::
+ Show the notes for a given commit (defaults to HEAD).
+
+
+Author
+------
+Written by Johannes Schindelin <johannes.schindelin@gmx.de>
+
+Documentation
+-------------
+Documentation by Johannes Schindelin
+
+GIT
+---
+Part of the linkgit:git[7] suite
diff --git a/Makefile b/Makefile
index 95571db..a2ec7b0 100644
--- a/Makefile
+++ b/Makefile
@@ -315,6 +315,7 @@ SCRIPT_SH += git-merge-one-file.sh
SCRIPT_SH += git-merge-resolve.sh
SCRIPT_SH += git-mergetool.sh
SCRIPT_SH += git-mergetool--lib.sh
+SCRIPT_SH += git-notes.sh
SCRIPT_SH += git-parse-remote.sh
SCRIPT_SH += git-pull.sh
SCRIPT_SH += git-quiltimport.sh
diff --git a/command-list.txt b/command-list.txt
index fb03a2e..4296941 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -74,6 +74,7 @@ git-mktag plumbingmanipulators
git-mktree plumbingmanipulators
git-mv mainporcelain common
git-name-rev plumbinginterrogators
+git-notes mainporcelain
git-pack-objects plumbingmanipulators
git-pack-redundant plumbinginterrogators
git-pack-refs ancillarymanipulators
diff --git a/git-notes.sh b/git-notes.sh
new file mode 100755
index 0000000..f06c254
--- /dev/null
+++ b/git-notes.sh
@@ -0,0 +1,73 @@
+#!/bin/sh
+
+USAGE="(edit | show) [commit]"
+. git-sh-setup
+
+test -n "$3" && usage
+
+test -z "$1" && usage
+ACTION="$1"; shift
+
+test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="$(git config core.notesref)"
+test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="refs/notes/commits"
+
+COMMIT=$(git rev-parse --verify --default HEAD "$@") ||
+die "Invalid commit: $@"
+
+case "$ACTION" in
+edit)
+ if [ "${GIT_NOTES_REF#refs/notes/}" = "$GIT_NOTES_REF" ]; then
+ die "Refusing to edit notes in $GIT_NOTES_REF (outside of refs/notes/)"
+ fi
+
+ MSG_FILE="$GIT_DIR/new-notes-$COMMIT"
+ GIT_INDEX_FILE="$MSG_FILE.idx"
+ export GIT_INDEX_FILE
+
+ trap '
+ test -f "$MSG_FILE" && rm "$MSG_FILE"
+ test -f "$GIT_INDEX_FILE" && rm "$GIT_INDEX_FILE"
+ ' 0
+
+ GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
+
+ CURRENT_HEAD=$(git show-ref "$GIT_NOTES_REF" | cut -f 1 -d ' ')
+ if [ -z "$CURRENT_HEAD" ]; then
+ PARENT=
+ else
+ PARENT="-p $CURRENT_HEAD"
+ git read-tree "$GIT_NOTES_REF" || die "Could not read index"
+ git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
+ fi
+
+ core_editor="$(git config core.editor)"
+ ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+
+ grep -v ^# < "$MSG_FILE" | git stripspace > "$MSG_FILE".processed
+ mv "$MSG_FILE".processed "$MSG_FILE"
+ if [ -s "$MSG_FILE" ]; then
+ BLOB=$(git hash-object -w "$MSG_FILE") ||
+ die "Could not write into object database"
+ git update-index --add --cacheinfo 0644 $BLOB $COMMIT ||
+ die "Could not write index"
+ else
+ test -z "$CURRENT_HEAD" &&
+ die "Will not initialise with empty tree"
+ git update-index --force-remove $COMMIT ||
+ die "Could not update index"
+ fi
+
+ TREE=$(git write-tree) || die "Could not write tree"
+ NEW_HEAD=$(echo Annotate $COMMIT | git commit-tree $TREE $PARENT) ||
+ die "Could not annotate"
+ git update-ref -m "Annotate $COMMIT" \
+ "$GIT_NOTES_REF" $NEW_HEAD $CURRENT_HEAD
+;;
+show)
+ git rev-parse -q --verify "$GIT_NOTES_REF":$COMMIT > /dev/null ||
+ die "No note for commit $COMMIT."
+ git show "$GIT_NOTES_REF":$COMMIT
+;;
+*)
+ usage
+esac
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
new file mode 100755
index 0000000..73e53be
--- /dev/null
+++ b/t/t3301-notes.sh
@@ -0,0 +1,114 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Johannes E. Schindelin
+#
+
+test_description='Test commit notes'
+
+. ./test-lib.sh
+
+cat > fake_editor.sh << \EOF
+echo "$MSG" > "$1"
+echo "$MSG" >& 2
+EOF
+chmod a+x fake_editor.sh
+VISUAL=./fake_editor.sh
+export VISUAL
+
+test_expect_success 'cannot annotate non-existing HEAD' '
+ (MSG=3 && export MSG && test_must_fail git notes edit)
+'
+
+test_expect_success setup '
+ : > a1 &&
+ git add a1 &&
+ test_tick &&
+ git commit -m 1st &&
+ : > a2 &&
+ git add a2 &&
+ test_tick &&
+ git commit -m 2nd
+'
+
+test_expect_success 'need valid notes ref' '
+ (MSG=1 GIT_NOTES_REF=/ && export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit) &&
+ (MSG=2 GIT_NOTES_REF=/ && export MSG GIT_NOTES_REF &&
+ test_must_fail git notes show)
+'
+
+test_expect_success 'refusing to edit in refs/heads/' '
+ (MSG=1 GIT_NOTES_REF=refs/heads/bogus &&
+ export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit)
+'
+
+test_expect_success 'refusing to edit in refs/remotes/' '
+ (MSG=1 GIT_NOTES_REF=refs/remotes/bogus &&
+ export MSG GIT_NOTES_REF &&
+ test_must_fail git notes edit)
+'
+
+# 1 indicates caught gracefully by die, 128 means git-show barked
+test_expect_success 'handle empty notes gracefully' '
+ git notes show ; test 1 = $?
+'
+
+test_expect_success 'create notes' '
+ git config core.notesRef refs/notes/commits &&
+ MSG=b1 git notes edit &&
+ test ! -f .git/new-notes &&
+ test 1 = $(git ls-tree refs/notes/commits | wc -l) &&
+ test b1 = $(git notes show) &&
+ git show HEAD^ &&
+ test_must_fail git notes show HEAD^
+'
+
+cat > expect << EOF
+commit 268048bfb8a1fb38e703baceb8ab235421bf80c5
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:14:13 2005 -0700
+
+ 2nd
+
+Notes:
+ b1
+EOF
+
+test_expect_success 'show notes' '
+ ! (git cat-file commit HEAD | grep b1) &&
+ git log -1 > output &&
+ test_cmp expect output
+'
+test_expect_success 'create multi-line notes (setup)' '
+ : > a3 &&
+ git add a3 &&
+ test_tick &&
+ git commit -m 3rd &&
+ MSG="b3
+c3c3c3c3
+d3d3d3" git notes edit
+'
+
+cat > expect-multiline << EOF
+commit 1584215f1d29c65e99c6c6848626553fdd07fd75
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:15:13 2005 -0700
+
+ 3rd
+
+Notes:
+ b3
+ c3c3c3c3
+ d3d3d3
+EOF
+
+printf "\n" >> expect-multiline
+cat expect >> expect-multiline
+
+test_expect_success 'show multi-line notes' '
+ git log -2 > output &&
+ test_cmp expect-multiline output
+'
+
+test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 03/12] Speed up git notes lookup
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To avoid looking up each and every commit in the notes ref's tree
object, which is very expensive, speed things up by slurping the tree
object's contents into a hash_map.
The idea for the hashmap singleton is from David Reiss, initial
benchmarking by Jeff King.
Note: the implementation allows for arbitrary entries in the notes
tree object, ignoring those that do not reference a valid object. This
allows you to annotate arbitrary branches, or objects.
This patch has been improved by the following contributions:
- Junio C Hamano: fixed an obvious error in initialize_hash_map()
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
notes.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
1 files changed, 102 insertions(+), 10 deletions(-)
diff --git a/notes.c b/notes.c
index 401966d..9172154 100644
--- a/notes.c
+++ b/notes.c
@@ -4,15 +4,112 @@
#include "refs.h"
#include "utf8.h"
#include "strbuf.h"
+#include "tree-walk.h"
+
+struct entry {
+ unsigned char commit_sha1[20];
+ unsigned char notes_sha1[20];
+};
+
+struct hash_map {
+ struct entry *entries;
+ off_t count, size;
+};
static int initialized;
+static struct hash_map hash_map;
+
+static int hash_index(struct hash_map *map, const unsigned char *sha1)
+{
+ int i = ((*(unsigned int *)sha1) % map->size);
+
+ for (;;) {
+ unsigned char *current = map->entries[i].commit_sha1;
+
+ if (!hashcmp(sha1, current))
+ return i;
+
+ if (is_null_sha1(current))
+ return -1 - i;
+
+ if (++i == map->size)
+ i = 0;
+ }
+}
+
+static void add_entry(const unsigned char *commit_sha1,
+ const unsigned char *notes_sha1)
+{
+ int index;
+
+ if (hash_map.count + 1 > hash_map.size >> 1) {
+ int i, old_size = hash_map.size;
+ struct entry *old = hash_map.entries;
+
+ hash_map.size = old_size ? old_size << 1 : 64;
+ hash_map.entries = (struct entry *)
+ xcalloc(sizeof(struct entry), hash_map.size);
+
+ for (i = 0; i < old_size; i++)
+ if (!is_null_sha1(old[i].commit_sha1)) {
+ index = -1 - hash_index(&hash_map,
+ old[i].commit_sha1);
+ memcpy(hash_map.entries + index, old + i,
+ sizeof(struct entry));
+ }
+ free(old);
+ }
+
+ index = hash_index(&hash_map, commit_sha1);
+ if (index < 0) {
+ index = -1 - index;
+ hash_map.count++;
+ }
+
+ hashcpy(hash_map.entries[index].commit_sha1, commit_sha1);
+ hashcpy(hash_map.entries[index].notes_sha1, notes_sha1);
+}
+
+static void initialize_hash_map(const char *notes_ref_name)
+{
+ unsigned char sha1[20], commit_sha1[20];
+ unsigned mode;
+ struct tree_desc desc;
+ struct name_entry entry;
+ void *buf;
+
+ if (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||
+ get_tree_entry(commit_sha1, "", sha1, &mode))
+ return;
+
+ buf = fill_tree_descriptor(&desc, sha1);
+ if (!buf)
+ die("Could not read %s for notes-index", sha1_to_hex(sha1));
+
+ while (tree_entry(&desc, &entry))
+ if (!get_sha1(entry.path, commit_sha1))
+ add_entry(commit_sha1, entry.sha1);
+ free(buf);
+}
+
+static unsigned char *lookup_notes(const unsigned char *commit_sha1)
+{
+ int index;
+
+ if (!hash_map.size)
+ return NULL;
+
+ index = hash_index(&hash_map, commit_sha1);
+ if (index < 0)
+ return NULL;
+ return hash_map.entries[index].notes_sha1;
+}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
const char *output_encoding)
{
static const char utf8[] = "utf-8";
- struct strbuf name = STRBUF_INIT;
- unsigned char sha1[20];
+ unsigned char *sha1;
char *msg, *msg_p;
unsigned long linelen, msglen;
enum object_type type;
@@ -23,17 +120,12 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
else if (!notes_ref_name)
notes_ref_name = GIT_NOTES_DEFAULT_REF;
- if (notes_ref_name && read_ref(notes_ref_name, sha1))
- notes_ref_name = NULL;
+ initialize_hash_map(notes_ref_name);
initialized = 1;
}
- if (!notes_ref_name)
- return;
-
- strbuf_addf(&name, "%s:%s", notes_ref_name,
- sha1_to_hex(commit->object.sha1));
- if (get_sha1(name.buf, sha1))
+ sha1 = lookup_notes(commit->object.sha1);
+ if (!sha1)
return;
if (!(msg = read_sha1_file(sha1, &type, &msglen)) || !msglen ||
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 04/12] Add an expensive test for git-notes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
git-notes have the potential of being pretty expensive, so test with
a lot of commits. A lot. So to make things cheaper, you have to
opt-in explicitely, by setting the environment variable
GIT_NOTES_TIMING_TESTS.
This patch has been improved by the following contributions:
- Junio C Hamano: tests: fix "export var=val"
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
t/t3302-notes-index-expensive.sh | 98 ++++++++++++++++++++++++++++++++++++++
1 files changed, 98 insertions(+), 0 deletions(-)
create mode 100755 t/t3302-notes-index-expensive.sh
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
new file mode 100755
index 0000000..0ef3e95
--- /dev/null
+++ b/t/t3302-notes-index-expensive.sh
@@ -0,0 +1,98 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Johannes E. Schindelin
+#
+
+test_description='Test commit notes index (expensive!)'
+
+. ./test-lib.sh
+
+test -z "$GIT_NOTES_TIMING_TESTS" && {
+ say Skipping timing tests
+ test_done
+ exit
+}
+
+create_repo () {
+ number_of_commits=$1
+ nr=0
+ parent=
+ test -d .git || {
+ git init &&
+ tree=$(git write-tree) &&
+ while [ $nr -lt $number_of_commits ]; do
+ test_tick &&
+ commit=$(echo $nr | git commit-tree $tree $parent) ||
+ return
+ parent="-p $commit"
+ nr=$(($nr+1))
+ done &&
+ git update-ref refs/heads/master $commit &&
+ {
+ GIT_INDEX_FILE=.git/temp; export GIT_INDEX_FILE;
+ git rev-list HEAD | cat -n | sed "s/^[ ][ ]*/ /g" |
+ while read nr sha1; do
+ blob=$(echo note $nr | git hash-object -w --stdin) &&
+ echo $sha1 | sed "s/^/0644 $blob 0 /"
+ done | git update-index --index-info &&
+ tree=$(git write-tree) &&
+ test_tick &&
+ commit=$(echo notes | git commit-tree $tree) &&
+ git update-ref refs/notes/commits $commit
+ } &&
+ git config core.notesRef refs/notes/commits
+ }
+}
+
+test_notes () {
+ count=$1 &&
+ git config core.notesRef refs/notes/commits &&
+ git log | grep "^ " > output &&
+ i=1 &&
+ while [ $i -le $count ]; do
+ echo " $(($count-$i))" &&
+ echo " note $i" &&
+ i=$(($i+1));
+ done > expect &&
+ git diff expect output
+}
+
+cat > time_notes << \EOF
+ mode=$1
+ i=1
+ while [ $i -lt $2 ]; do
+ case $1 in
+ no-notes)
+ GIT_NOTES_REF=non-existing; export GIT_NOTES_REF
+ ;;
+ notes)
+ unset GIT_NOTES_REF
+ ;;
+ esac
+ git log >/dev/null
+ i=$(($i+1))
+ done
+EOF
+
+time_notes () {
+ for mode in no-notes notes
+ do
+ echo $mode
+ /usr/bin/time sh ../time_notes $mode $1
+ done
+}
+
+for count in 10 100 1000 10000; do
+
+ mkdir $count
+ (cd $count;
+
+ test_expect_success "setup $count" "create_repo $count"
+
+ test_expect_success 'notes work' "test_notes $count"
+
+ test_expect_success 'notes timing' "time_notes 100"
+ )
+done
+
+test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 05/12] Teach "-m <msg>" and "-F <file>" to "git notes edit"
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
The "-m" and "-F" options are already the established method
(in both git-commit and git-tag) to specify a commit/tag message
without invoking the editor. This patch teaches "git notes edit"
to respect the same options for specifying a notes message without
invoking the editor.
Multiple "-m" and/or "-F" options are concatenated as separate
paragraphs.
The patch also updates the "git notes" documentation and adds
selftests for the new functionality. Unfortunately, the added
selftests include a couple of lines with trailing whitespace
(without these the test will fail). This may cause git to warn
about "whitespace errors".
This patch has been improved by the following contributions:
- Thomas Rast: fix trailing whitespace in t3301
Signed-off-by: Johan Herland <johan@herland.net>
---
Documentation/git-notes.txt | 16 ++++++++++-
git-notes.sh | 64 +++++++++++++++++++++++++++++++++++++-----
t/t3301-notes.sh | 36 ++++++++++++++++++++++++
3 files changed, 107 insertions(+), 9 deletions(-)
diff --git a/Documentation/git-notes.txt b/Documentation/git-notes.txt
index 7136016..94cceb1 100644
--- a/Documentation/git-notes.txt
+++ b/Documentation/git-notes.txt
@@ -8,7 +8,7 @@ git-notes - Add/inspect commit notes
SYNOPSIS
--------
[verse]
-'git-notes' (edit | show) [commit]
+'git-notes' (edit [-F <file> | -m <msg>] | show) [commit]
DESCRIPTION
-----------
@@ -33,6 +33,20 @@ show::
Show the notes for a given commit (defaults to HEAD).
+OPTIONS
+-------
+-m <msg>::
+ Use the given note message (instead of prompting).
+ If multiple `-m` (or `-F`) options are given, their
+ values are concatenated as separate paragraphs.
+
+-F <file>::
+ Take the note message from the given file. Use '-' to
+ read the note message from the standard input.
+ If multiple `-F` (or `-m`) options are given, their
+ values are concatenated as separate paragraphs.
+
+
Author
------
Written by Johannes Schindelin <johannes.schindelin@gmx.de>
diff --git a/git-notes.sh b/git-notes.sh
index f06c254..e642e47 100755
--- a/git-notes.sh
+++ b/git-notes.sh
@@ -1,16 +1,59 @@
#!/bin/sh
-USAGE="(edit | show) [commit]"
+USAGE="(edit [-F <file> | -m <msg>] | show) [commit]"
. git-sh-setup
-test -n "$3" && usage
-
test -z "$1" && usage
ACTION="$1"; shift
test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="$(git config core.notesref)"
test -z "$GIT_NOTES_REF" && GIT_NOTES_REF="refs/notes/commits"
+MESSAGE=
+while test $# != 0
+do
+ case "$1" in
+ -m)
+ test "$ACTION" = "edit" || usage
+ shift
+ if test "$#" = "0"; then
+ die "error: option -m needs an argument"
+ else
+ if [ -z "$MESSAGE" ]; then
+ MESSAGE="$1"
+ else
+ MESSAGE="$MESSAGE
+
+$1"
+ fi
+ shift
+ fi
+ ;;
+ -F)
+ test "$ACTION" = "edit" || usage
+ shift
+ if test "$#" = "0"; then
+ die "error: option -F needs an argument"
+ else
+ if [ -z "$MESSAGE" ]; then
+ MESSAGE="$(cat "$1")"
+ else
+ MESSAGE="$MESSAGE
+
+$(cat "$1")"
+ fi
+ shift
+ fi
+ ;;
+ -*)
+ usage
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
COMMIT=$(git rev-parse --verify --default HEAD "$@") ||
die "Invalid commit: $@"
@@ -29,19 +72,24 @@ edit)
test -f "$GIT_INDEX_FILE" && rm "$GIT_INDEX_FILE"
' 0
- GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
-
CURRENT_HEAD=$(git show-ref "$GIT_NOTES_REF" | cut -f 1 -d ' ')
if [ -z "$CURRENT_HEAD" ]; then
PARENT=
else
PARENT="-p $CURRENT_HEAD"
git read-tree "$GIT_NOTES_REF" || die "Could not read index"
- git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
fi
- core_editor="$(git config core.editor)"
- ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+ if [ -z "$MESSAGE" ]; then
+ GIT_NOTES_REF= git log -1 $COMMIT | sed "s/^/#/" > "$MSG_FILE"
+ if [ ! -z "$CURRENT_HEAD" ]; then
+ git cat-file blob :$COMMIT >> "$MSG_FILE" 2> /dev/null
+ fi
+ core_editor="$(git config core.editor)"
+ ${GIT_EDITOR:-${core_editor:-${VISUAL:-${EDITOR:-vi}}}} "$MSG_FILE"
+ else
+ echo "$MESSAGE" > "$MSG_FILE"
+ fi
grep -v ^# < "$MSG_FILE" | git stripspace > "$MSG_FILE".processed
mv "$MSG_FILE".processed "$MSG_FILE"
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
index 73e53be..1e34f48 100755
--- a/t/t3301-notes.sh
+++ b/t/t3301-notes.sh
@@ -110,5 +110,41 @@ test_expect_success 'show multi-line notes' '
git log -2 > output &&
test_cmp expect-multiline output
'
+test_expect_success 'create -m and -F notes (setup)' '
+ : > a4 &&
+ git add a4 &&
+ test_tick &&
+ git commit -m 4th &&
+ echo "xyzzy" > note5 &&
+ git notes edit -m spam -F note5 -m "foo
+bar
+baz"
+'
+
+whitespace=" "
+cat > expect-m-and-F << EOF
+commit 15023535574ded8b1a89052b32673f84cf9582b8
+Author: A U Thor <author@example.com>
+Date: Thu Apr 7 15:16:13 2005 -0700
+
+ 4th
+
+Notes:
+ spam
+$whitespace
+ xyzzy
+$whitespace
+ foo
+ bar
+ baz
+EOF
+
+printf "\n" >> expect-m-and-F
+cat expect-multiline >> expect-m-and-F
+
+test_expect_success 'show -m and -F notes' '
+ git log -3 > output &&
+ test_cmp expect-m-and-F output
+'
test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 07/12] t3302-notes-index-expensive: Speed up create_repo()
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
Creating repos with 10/100/1000/10000 commits and notes takes a lot of time.
However, using git-fast-import to do the job is a lot more efficient than
using plumbing commands to do the same.
This patch decreases the overall run-time of this test on my machine from
~3 to ~1 minutes.
Signed-off-by: Johan Herland <johan@herland.net>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/t3302-notes-index-expensive.sh | 74 ++++++++++++++++++++++++--------------
1 files changed, 47 insertions(+), 27 deletions(-)
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 0ef3e95..ee84fc4 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -16,30 +16,50 @@ test -z "$GIT_NOTES_TIMING_TESTS" && {
create_repo () {
number_of_commits=$1
nr=0
- parent=
test -d .git || {
git init &&
- tree=$(git write-tree) &&
- while [ $nr -lt $number_of_commits ]; do
- test_tick &&
- commit=$(echo $nr | git commit-tree $tree $parent) ||
- return
- parent="-p $commit"
- nr=$(($nr+1))
- done &&
- git update-ref refs/heads/master $commit &&
- {
- GIT_INDEX_FILE=.git/temp; export GIT_INDEX_FILE;
- git rev-list HEAD | cat -n | sed "s/^[ ][ ]*/ /g" |
- while read nr sha1; do
- blob=$(echo note $nr | git hash-object -w --stdin) &&
- echo $sha1 | sed "s/^/0644 $blob 0 /"
- done | git update-index --index-info &&
- tree=$(git write-tree) &&
+ (
+ while [ $nr -lt $number_of_commits ]; do
+ nr=$(($nr+1))
+ mark=$(($nr+$nr))
+ notemark=$(($mark+1))
+ test_tick &&
+ cat <<INPUT_END &&
+commit refs/heads/master
+mark :$mark
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+commit #$nr
+COMMIT
+
+M 644 inline file
+data <<EOF
+file in commit #$nr
+EOF
+
+blob
+mark :$notemark
+data <<EOF
+note for commit #$nr
+EOF
+
+INPUT_END
+
+ echo "N :$notemark :$mark" >> note_commit
+ done &&
test_tick &&
- commit=$(echo notes | git commit-tree $tree) &&
- git update-ref refs/notes/commits $commit
- } &&
+ cat <<INPUT_END &&
+commit refs/notes/commits
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes
+COMMIT
+
+INPUT_END
+
+ cat note_commit
+ ) |
+ git fast-import --quiet &&
git config core.notesRef refs/notes/commits
}
}
@@ -48,13 +68,13 @@ test_notes () {
count=$1 &&
git config core.notesRef refs/notes/commits &&
git log | grep "^ " > output &&
- i=1 &&
- while [ $i -le $count ]; do
- echo " $(($count-$i))" &&
- echo " note $i" &&
- i=$(($i+1));
+ i=$count &&
+ while [ $i -gt 0 ]; do
+ echo " commit #$i" &&
+ echo " note for commit #$i" &&
+ i=$(($i-1));
done > expect &&
- git diff expect output
+ test_cmp expect output
}
cat > time_notes << \EOF
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 06/12] fast-import: Add support for importing commit notes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
Introduce a 'notemodify' subcommand of the 'commit' command. This subcommand
is similar to 'filemodify', except that no mode is supplied (all notes have
mode 0644), and the path is set to the hex SHA1 of the given "comittish".
This enables fast import of note objects along with their associated commits,
since the notes can now be named using the mark references of their
corresponding commits.
The patch also includes a test case of the added functionality.
Signed-off-by: Johan Herland <johan@herland.net>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
---
Documentation/git-fast-import.txt | 45 +++++++++--
fast-import.c | 88 +++++++++++++++++++-
t/t9300-fast-import.sh | 166 +++++++++++++++++++++++++++++++++++++
3 files changed, 289 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index c2f483a..288032c 100644
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -316,7 +316,7 @@ change to the project.
data
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
- (filemodify | filedelete | filecopy | filerename | filedeleteall)*
+ (filemodify | filedelete | filecopy | filerename | filedeleteall | notemodify)*
LF?
....
@@ -339,14 +339,13 @@ commit message use a 0 length data. Commit messages are free-form
and are not interpreted by Git. Currently they must be encoded in
UTF-8, as fast-import does not permit other encodings to be specified.
-Zero or more `filemodify`, `filedelete`, `filecopy`, `filerename`
-and `filedeleteall` commands
+Zero or more `filemodify`, `filedelete`, `filecopy`, `filerename`,
+`filedeleteall` and `notemodify` commands
may be included to update the contents of the branch prior to
creating the commit. These commands may be supplied in any order.
However it is recommended that a `filedeleteall` command precede
-all `filemodify`, `filecopy` and `filerename` commands in the same
-commit, as `filedeleteall`
-wipes the branch clean (see below).
+all `filemodify`, `filecopy`, `filerename` and `notemodify` commands in
+the same commit, as `filedeleteall` wipes the branch clean (see below).
The `LF` after the command is optional (it used to be required).
@@ -595,6 +594,40 @@ more memory per active branch (less than 1 MiB for even most large
projects); so frontends that can easily obtain only the affected
paths for a commit are encouraged to do so.
+`notemodify`
+^^^^^^^^^^^^
+Included in a `commit` command to add a new note (annotating a given
+commit) or change the content of an existing note. This command has
+two different means of specifying the content of the note.
+
+External data format::
+ The data content for the note was already supplied by a prior
+ `blob` command. The frontend just needs to connect it to the
+ commit that is to be annotated.
++
+....
+ 'N' SP <dataref> SP <committish> LF
+....
++
+Here `<dataref>` can be either a mark reference (`:<idnum>`)
+set by a prior `blob` command, or a full 40-byte SHA-1 of an
+existing Git blob object.
+
+Inline data format::
+ The data content for the note has not been supplied yet.
+ The frontend wants to supply it as part of this modify
+ command.
++
+....
+ 'N' SP 'inline' SP <committish> LF
+ data
+....
++
+See below for a detailed description of the `data` command.
+
+In both formats `<committish>` is any of the commit specification
+expressions also accepted by `from` (see above).
+
`mark`
~~~~~~
Arranges for fast-import to save a reference to the current object, allowing
diff --git a/fast-import.c b/fast-import.c
index 7ef9865..97e6e9e 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -22,8 +22,8 @@ Format of STDIN stream:
('author' sp name sp '<' email '>' sp when lf)?
'committer' sp name sp '<' email '>' sp when lf
commit_msg
- ('from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)?
- ('merge' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)*
+ ('from' sp committish lf)?
+ ('merge' sp committish lf)*
file_change*
lf?;
commit_msg ::= data;
@@ -41,15 +41,18 @@ Format of STDIN stream:
file_obm ::= 'M' sp mode sp (hexsha1 | idnum) sp path_str lf;
file_inm ::= 'M' sp mode sp 'inline' sp path_str lf
data;
+ note_obm ::= 'N' sp (hexsha1 | idnum) sp committish lf;
+ note_inm ::= 'N' sp 'inline' sp committish lf
+ data;
new_tag ::= 'tag' sp tag_str lf
- 'from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf
+ 'from' sp committish lf
('tagger' sp name sp '<' email '>' sp when lf)?
tag_msg;
tag_msg ::= data;
reset_branch ::= 'reset' sp ref_str lf
- ('from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)?
+ ('from' sp committish lf)?
lf?;
checkpoint ::= 'checkpoint' lf
@@ -88,6 +91,7 @@ Format of STDIN stream:
# stream formatting is: \, " and LF. Otherwise these values
# are UTF8.
#
+ committish ::= (ref_str | hexsha1 | sha1exp_str | idnum);
ref_str ::= ref;
sha1exp_str ::= sha1exp;
tag_str ::= tag;
@@ -2003,6 +2007,80 @@ static void file_change_cr(struct branch *b, int rename)
leaf.tree);
}
+static void note_change_n(struct branch *b)
+{
+ const char *p = command_buf.buf + 2;
+ static struct strbuf uq = STRBUF_INIT;
+ struct object_entry *oe = oe;
+ struct branch *s;
+ unsigned char sha1[20], commit_sha1[20];
+ uint16_t inline_data = 0;
+
+ /* <dataref> or 'inline' */
+ if (*p == ':') {
+ char *x;
+ oe = find_mark(strtoumax(p + 1, &x, 10));
+ hashcpy(sha1, oe->sha1);
+ p = x;
+ } else if (!prefixcmp(p, "inline")) {
+ inline_data = 1;
+ p += 6;
+ } else {
+ if (get_sha1_hex(p, sha1))
+ die("Invalid SHA1: %s", command_buf.buf);
+ oe = find_object(sha1);
+ p += 40;
+ }
+ if (*p++ != ' ')
+ die("Missing space after SHA1: %s", command_buf.buf);
+
+ /* <committish> */
+ s = lookup_branch(p);
+ if (s) {
+ hashcpy(commit_sha1, s->sha1);
+ } else if (*p == ':') {
+ uintmax_t commit_mark = strtoumax(p + 1, NULL, 10);
+ struct object_entry *commit_oe = find_mark(commit_mark);
+ if (commit_oe->type != OBJ_COMMIT)
+ die("Mark :%" PRIuMAX " not a commit", commit_mark);
+ hashcpy(commit_sha1, commit_oe->sha1);
+ } else if (!get_sha1(p, commit_sha1)) {
+ unsigned long size;
+ char *buf = read_object_with_reference(commit_sha1,
+ commit_type, &size, commit_sha1);
+ if (!buf || size < 46)
+ die("Not a valid commit: %s", p);
+ free(buf);
+ } else
+ die("Invalid ref name or SHA1 expression: %s", p);
+
+ if (inline_data) {
+ static struct strbuf buf = STRBUF_INIT;
+
+ if (p != uq.buf) {
+ strbuf_addstr(&uq, p);
+ p = uq.buf;
+ }
+ read_next_command();
+ parse_data(&buf);
+ store_object(OBJ_BLOB, &buf, &last_blob, sha1, 0);
+ } else if (oe) {
+ if (oe->type != OBJ_BLOB)
+ die("Not a blob (actually a %s): %s",
+ typename(oe->type), command_buf.buf);
+ } else {
+ enum object_type type = sha1_object_info(sha1, NULL);
+ if (type < 0)
+ die("Blob not found: %s", command_buf.buf);
+ if (type != OBJ_BLOB)
+ die("Not a blob (actually a %s): %s",
+ typename(type), command_buf.buf);
+ }
+
+ tree_content_set(&b->branch_tree, sha1_to_hex(commit_sha1), sha1,
+ S_IFREG | 0644, NULL);
+}
+
static void file_change_deleteall(struct branch *b)
{
release_tree_content_recursive(b->branch_tree.tree);
@@ -2172,6 +2250,8 @@ static void parse_new_commit(void)
file_change_cr(b, 1);
else if (!prefixcmp(command_buf.buf, "C "))
file_change_cr(b, 0);
+ else if (!prefixcmp(command_buf.buf, "N "))
+ note_change_n(b);
else if (!strcmp("deleteall", command_buf.buf))
file_change_deleteall(b);
else {
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 821be7c..b49815d 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -1088,4 +1088,170 @@ INPUT_END
test_expect_success 'P: fail on blob mark in gitlink' '
test_must_fail git fast-import <input'
+###
+### series Q (notes)
+###
+
+note1_data="Note for the first commit"
+note2_data="Note for the second commit"
+note3_data="Note for the third commit"
+
+test_tick
+cat >input <<INPUT_END
+blob
+mark :2
+data <<EOF
+$file2_data
+EOF
+
+commit refs/heads/notes-test
+mark :3
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+first (:3)
+COMMIT
+
+M 644 :2 file2
+
+blob
+mark :4
+data $file4_len
+$file4_data
+commit refs/heads/notes-test
+mark :5
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+second (:5)
+COMMIT
+
+M 644 :4 file4
+
+commit refs/heads/notes-test
+mark :6
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+third (:6)
+COMMIT
+
+M 644 inline file5
+data <<EOF
+$file5_data
+EOF
+
+M 755 inline file6
+data <<EOF
+$file6_data
+EOF
+
+blob
+mark :7
+data <<EOF
+$note1_data
+EOF
+
+blob
+mark :8
+data <<EOF
+$note2_data
+EOF
+
+commit refs/notes/foobar
+mark :9
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes (:9)
+COMMIT
+
+N :7 :3
+N :8 :5
+N inline :6
+data <<EOF
+$note3_data
+EOF
+
+INPUT_END
+test_expect_success \
+ 'Q: commit notes' \
+ 'git fast-import <input &&
+ git whatchanged notes-test'
+test_expect_success \
+ 'Q: verify pack' \
+ 'for p in .git/objects/pack/*.pack;do git verify-pack $p||exit;done'
+
+commit1=$(git rev-parse notes-test~2)
+commit2=$(git rev-parse notes-test^)
+commit3=$(git rev-parse notes-test)
+
+cat >expect <<EOF
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+first (:3)
+EOF
+test_expect_success \
+ 'Q: verify first commit' \
+ 'git cat-file commit notes-test~2 | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+parent $commit1
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+second (:5)
+EOF
+test_expect_success \
+ 'Q: verify second commit' \
+ 'git cat-file commit notes-test^ | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+parent $commit2
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+third (:6)
+EOF
+test_expect_success \
+ 'Q: verify third commit' \
+ 'git cat-file commit notes-test | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect <<EOF
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+notes (:9)
+EOF
+test_expect_success \
+ 'Q: verify notes commit' \
+ 'git cat-file commit refs/notes/foobar | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect.unsorted <<EOF
+100644 blob $commit1
+100644 blob $commit2
+100644 blob $commit3
+EOF
+cat expect.unsorted | sort >expect
+test_expect_success \
+ 'Q: verify notes tree' \
+ 'git cat-file -p refs/notes/foobar^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ test_cmp expect actual'
+
+echo "$note1_data" >expect
+test_expect_success \
+ 'Q: verify note for first commit' \
+ 'git cat-file blob refs/notes/foobar:$commit1 >actual && test_cmp expect actual'
+
+echo "$note2_data" >expect
+test_expect_success \
+ 'Q: verify note for second commit' \
+ 'git cat-file blob refs/notes/foobar:$commit2 >actual && test_cmp expect actual'
+
+echo "$note3_data" >expect
+test_expect_success \
+ 'Q: verify note for third commit' \
+ 'git cat-file blob refs/notes/foobar:$commit3 >actual && test_cmp expect actual'
+
test_done
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 08/12] Teach the notes lookup code to parse notes trees with various fanout schemes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce, Johannes Schindelin
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
The semantics used when parsing notes trees (with regards to fanout subtrees)
follow Dscho's proposal fairly closely:
- No concatenation/merging of notes is performed. If there are several notes
objects referencing a given commit, only one of those objects are used.
- If a notes object for a given commit is present in the "root" notes tree,
no subtrees are consulted; the object in the root tree is used directly.
- If there are more than one subtree that prefix-matches the given commit,
only the subtree with the longest matching prefix is consulted. This
means that if the given commit is e.g. "deadbeef", and the notes tree have
subtrees "de" and "dead", then the following paths in the notes tree are
searched: "deadbeef", "dead/beef". Note that "de/adbeef" is NOT searched.
- Fanout directories (subtrees) must references a whole number of bytes
from the SHA1 sum they subdivide. E.g. subtrees "dead" and "de" are
acceptable; "d" and "dea" are not.
- Multiple levels of fanout are allowed. All the above rules apply
recursively. E.g. "de/adbeef" is preferred over "de/adbe/ef", etc.
This patch changes the in-memory datastructure for holding parsed notes:
Instead of holding all note (and subtree) entries in a hash table, a
simple 16-tree structure is used instead. The tree structure consists of
16-arrays as internal nodes, and note/subtree entries as leaf nodes. The
tree is traversed by indexing subsequent nibbles of the search key until
a leaf node is encountered. If a subtree entry is encountered while
searching for a note, the subtree is unpacked into the 16-tree structure,
and the search continues into that subtree.
The new algorithm performs significantly better in the cases where only
a fraction of the notes need to be looked up (this is assumed to be the
common case for notes lookup). The new code even performs marginally
better in the worst case (where _all_ the notes are looked up).
In addition to this, comes the massive performance win associated with
organizing the notes tree according to some fanout scheme. Even a simple
2/38 fanout scheme is dramatically quicker to traverse (going from tens of
seconds to sub-second runtimes).
As for memory usage, the new code is marginally better than the old code in
the worst case, but in the case of looking up only some notes from a notes
tree with proper fanout, the new code uses only a small fraction of the
memory needed to hold the entire notes tree.
However, there is one casualty of this patch. The old notes lookup code was
able to parse notes that were associated with non-SHA1s (e.g. refs). The new
code requires the referenced object to be named by a SHA1 sum. Still, this
is not considered a major setback, since the notes infrastructure was not
originally intended to annotate objects outside the Git object database.
Cc: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johan Herland <johan@herland.net>
---
notes.c | 294 +++++++++++++++++++++++++++++++++++++++++++++++++--------------
1 files changed, 228 insertions(+), 66 deletions(-)
diff --git a/notes.c b/notes.c
index 9172154..a637056 100644
--- a/notes.c
+++ b/notes.c
@@ -6,103 +6,265 @@
#include "strbuf.h"
#include "tree-walk.h"
-struct entry {
- unsigned char commit_sha1[20];
- unsigned char notes_sha1[20];
+/*
+ * Use a non-balancing simple 16-tree structure with struct int_node as
+ * internal nodes, and struct leaf_node as leaf nodes. Each int_node has a
+ * 16-array of pointers to its children.
+ * The bottom 2 bits of each pointer is used to identify the pointer type
+ * - ptr & 3 == 0 - NULL pointer, assert(ptr == NULL)
+ * - ptr & 3 == 1 - pointer to next internal node - cast to struct int_node *
+ * - ptr & 3 == 2 - pointer to note entry - cast to struct leaf_node *
+ * - ptr & 3 == 3 - pointer to subtree entry - cast to struct leaf_node *
+ *
+ * The root node is a statically allocated struct int_node.
+ */
+struct int_node {
+ void *a[16];
};
-struct hash_map {
- struct entry *entries;
- off_t count, size;
+/*
+ * Leaf nodes come in two variants, note entries and subtree entries,
+ * distinguished by the LSb of the leaf node pointer (see above).
+ * As a note entry, the key is the SHA1 of the referenced commit, and the
+ * value is the SHA1 of the note object.
+ * As a subtree entry, the key is the prefix SHA1 (w/trailing NULs) of the
+ * referenced commit, using the last byte of the key to store the length of
+ * the prefix. The value is the SHA1 of the tree object containing the notes
+ * subtree.
+ */
+struct leaf_node {
+ unsigned char key_sha1[20];
+ unsigned char val_sha1[20];
};
+#define PTR_TYPE_NULL 0
+#define PTR_TYPE_INTERNAL 1
+#define PTR_TYPE_NOTE 2
+#define PTR_TYPE_SUBTREE 3
+
+#define GET_PTR_TYPE(ptr) ((uintptr_t) (ptr) & 3)
+#define CLR_PTR_TYPE(ptr) ((void *) ((uintptr_t) (ptr) & ~3))
+#define SET_PTR_TYPE(ptr, type) ((void *) ((uintptr_t) (ptr) | (type)))
+
+#define GET_NIBBLE(n, sha1) (((sha1[n >> 1]) >> ((n & 0x01) << 2)) & 0x0f)
+
+#define MATCHING_SUBTREE(key_sha1, subtree_sha1) \
+ (!memcmp(key_sha1, subtree_sha1, subtree_sha1[19]))
+
static int initialized;
-static struct hash_map hash_map;
-static int hash_index(struct hash_map *map, const unsigned char *sha1)
-{
- int i = ((*(unsigned int *)sha1) % map->size);
+static struct int_node root_node;
- for (;;) {
- unsigned char *current = map->entries[i].commit_sha1;
+static void load_subtree(struct leaf_node *subtree, struct int_node *node,
+ unsigned int n);
- if (!hashcmp(sha1, current))
- return i;
+/*
+ * To find a leaf_node:
+ * 1. Start at the root node, with n = 0
+ * 2. Use the nth nibble of the key as an index into a:
+ * - If a[n] is an int_node, recurse into that node and increment n
+ * - If a leaf_node with matching key, return leaf_node (assert note entry)
+ * - If a matching subtree entry, unpack that subtree entry (and remove it);
+ * restart search at the current level.
+ * - Otherwise, we end up at a NULL pointer, or a non-matching leaf_node.
+ * Backtrack out of the recursion, one level at a time and check a[0]:
+ * - If a[0] at the current level is a matching subtree entry, unpack that
+ * subtree entry (and remove it); restart search at the current level.
+ */
+static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
+ const unsigned char *key_sha1)
+{
+ struct leaf_node *l;
+ unsigned char i = GET_NIBBLE(n, key_sha1);
+ void *p = tree->a[i];
- if (is_null_sha1(current))
- return -1 - i;
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_INTERNAL:
+ l = note_tree_find(CLR_PTR_TYPE(p), n + 1, key_sha1);
+ if (l)
+ return l;
+ break;
+ case PTR_TYPE_NOTE:
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!hashcmp(key_sha1, l->key_sha1))
+ return l; /* return note object matching given key */
+ break;
+ case PTR_TYPE_SUBTREE:
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (MATCHING_SUBTREE(key_sha1, l->key_sha1)) {
+ /* unpack tree and resume search */
+ tree->a[i] = NULL;
+ load_subtree(l, tree, n);
+ free(l);
+ return note_tree_find(tree, n, key_sha1);
+ }
+ break;
+ case PTR_TYPE_NULL:
+ default:
+ assert(!p);
+ break;
+ }
- if (++i == map->size)
- i = 0;
+ /*
+ * Did not find key at this (or any lower) level.
+ * Check if there's a matching subtree entry in tree->a[0].
+ * If so, unpack tree and resume search.
+ */
+ p = tree->a[0];
+ if (GET_PTR_TYPE(p) != PTR_TYPE_SUBTREE)
+ return NULL;
+ l = (struct leaf_node *) CLR_PTR_TYPE(p);
+ if (MATCHING_SUBTREE(key_sha1, l->key_sha1)) {
+ /* unpack tree and resume search */
+ tree->a[0] = NULL;
+ load_subtree(l, tree, n);
+ free(l);
+ return note_tree_find(tree, n, key_sha1);
}
+ return NULL;
}
-static void add_entry(const unsigned char *commit_sha1,
- const unsigned char *notes_sha1)
+/*
+ * To insert a leaf_node:
+ * 1. Start at the root node, with n = 0
+ * 2. Use the nth nibble of the key as an index into a:
+ * - If a[n] is NULL, store the tweaked pointer directly into a[n]
+ * - If a[n] is an int_node, recurse into that node and increment n
+ * - If a[n] is a leaf_node:
+ * 1. Check if they're equal, and handle that (abort? overwrite?)
+ * 2. Create a new int_node, and store both leaf_nodes there
+ * 3. Store the new int_node into a[n].
+ */
+static int note_tree_insert(struct int_node *tree, unsigned char n,
+ const struct leaf_node *entry, unsigned char type)
{
- int index;
-
- if (hash_map.count + 1 > hash_map.size >> 1) {
- int i, old_size = hash_map.size;
- struct entry *old = hash_map.entries;
-
- hash_map.size = old_size ? old_size << 1 : 64;
- hash_map.entries = (struct entry *)
- xcalloc(sizeof(struct entry), hash_map.size);
-
- for (i = 0; i < old_size; i++)
- if (!is_null_sha1(old[i].commit_sha1)) {
- index = -1 - hash_index(&hash_map,
- old[i].commit_sha1);
- memcpy(hash_map.entries + index, old + i,
- sizeof(struct entry));
- }
- free(old);
+ struct int_node *new_node;
+ const struct leaf_node *l;
+ int ret;
+ unsigned char i = GET_NIBBLE(n, entry->key_sha1);
+ void *p = tree->a[i];
+ assert(GET_PTR_TYPE(entry) == PTR_TYPE_NULL);
+ switch(GET_PTR_TYPE(p)) {
+ case PTR_TYPE_NULL:
+ assert(!p);
+ tree->a[i] = SET_PTR_TYPE(entry, type);
+ return 0;
+ case PTR_TYPE_INTERNAL:
+ return note_tree_insert(CLR_PTR_TYPE(p), n + 1, entry, type);
+ default:
+ assert(GET_PTR_TYPE(p) == PTR_TYPE_NOTE ||
+ GET_PTR_TYPE(p) == PTR_TYPE_SUBTREE);
+ l = (const struct leaf_node *) CLR_PTR_TYPE(p);
+ if (!hashcmp(entry->key_sha1, l->key_sha1))
+ return -1; /* abort insert on matching key */
+ new_node = (struct int_node *)
+ xcalloc(sizeof(struct int_node), 1);
+ ret = note_tree_insert(new_node, n + 1,
+ CLR_PTR_TYPE(p), GET_PTR_TYPE(p));
+ if (ret) {
+ free(new_node);
+ return -1;
+ }
+ tree->a[i] = SET_PTR_TYPE(new_node, PTR_TYPE_INTERNAL);
+ return note_tree_insert(new_node, n + 1, entry, type);
}
+}
- index = hash_index(&hash_map, commit_sha1);
- if (index < 0) {
- index = -1 - index;
- hash_map.count++;
+/*
+ * Convert a partial SHA1 hex string to the corresponding partial SHA1 value.
+ * - hex - Partial SHA1 segment in ASCII hex format
+ * - hex_len - Length of above segment. Must be multiple of 2 between 0 and 40
+ * - sha1 - Partial SHA1 value is written here
+ * - sha1_len - Max #bytes to store in sha1, Must be >= hex_len / 2, and < 20
+ * Returns -1 on error (invalid arguments or invalid SHA1 (not in hex format).
+ * Otherwise, returns number of bytes written to sha1 (i.e. hex_len / 2).
+ * Pads sha1 with NULs up to sha1_len (not included in returned length).
+ */
+static int get_sha1_hex_segment(const char *hex, unsigned int hex_len,
+ unsigned char *sha1, unsigned int sha1_len)
+{
+ unsigned int i, len = hex_len >> 1;
+ if (hex_len % 2 != 0 || len > sha1_len)
+ return -1;
+ for (i = 0; i < len; i++) {
+ unsigned int val = (hexval(hex[0]) << 4) | hexval(hex[1]);
+ if (val & ~0xff)
+ return -1;
+ *sha1++ = val;
+ hex += 2;
}
+ for (; i < sha1_len; i++)
+ *sha1++ = 0;
+ return len;
+}
+
+static void load_subtree(struct leaf_node *subtree, struct int_node *node,
+ unsigned int n)
+{
+ unsigned char commit_sha1[20];
+ unsigned int prefix_len;
+ int status;
+ void *buf;
+ struct tree_desc desc;
+ struct name_entry entry;
+
+ buf = fill_tree_descriptor(&desc, subtree->val_sha1);
+ if (!buf)
+ die("Could not read %s for notes-index",
+ sha1_to_hex(subtree->val_sha1));
+
+ prefix_len = subtree->key_sha1[19];
+ assert(prefix_len * 2 >= n);
+ memcpy(commit_sha1, subtree->key_sha1, prefix_len);
+ while (tree_entry(&desc, &entry)) {
+ int len = get_sha1_hex_segment(entry.path, strlen(entry.path),
+ commit_sha1 + prefix_len, 20 - prefix_len);
+ if (len < 0)
+ continue; /* entry.path is not a SHA1 sum. Skip */
+ len += prefix_len;
- hashcpy(hash_map.entries[index].commit_sha1, commit_sha1);
- hashcpy(hash_map.entries[index].notes_sha1, notes_sha1);
+ /*
+ * If commit SHA1 is complete (len == 20), assume note object
+ * If commit SHA1 is incomplete (len < 20), assume note subtree
+ */
+ if (len <= 20) {
+ unsigned char type = PTR_TYPE_NOTE;
+ struct leaf_node *l = (struct leaf_node *)
+ xcalloc(sizeof(struct leaf_node), 1);
+ hashcpy(l->key_sha1, commit_sha1);
+ hashcpy(l->val_sha1, entry.sha1);
+ if (len < 20) {
+ l->key_sha1[19] = (unsigned char) len;
+ type = PTR_TYPE_SUBTREE;
+ }
+ status = note_tree_insert(node, n, l, type);
+ assert(!status);
+ }
+ }
+ free(buf);
}
-static void initialize_hash_map(const char *notes_ref_name)
+static void initialize_notes(const char *notes_ref_name)
{
unsigned char sha1[20], commit_sha1[20];
unsigned mode;
- struct tree_desc desc;
- struct name_entry entry;
- void *buf;
+ struct leaf_node root_tree;
if (!notes_ref_name || read_ref(notes_ref_name, commit_sha1) ||
get_tree_entry(commit_sha1, "", sha1, &mode))
return;
- buf = fill_tree_descriptor(&desc, sha1);
- if (!buf)
- die("Could not read %s for notes-index", sha1_to_hex(sha1));
-
- while (tree_entry(&desc, &entry))
- if (!get_sha1(entry.path, commit_sha1))
- add_entry(commit_sha1, entry.sha1);
- free(buf);
+ hashclr(root_tree.key_sha1);
+ hashcpy(root_tree.val_sha1, sha1);
+ load_subtree(&root_tree, &root_node, 0);
}
static unsigned char *lookup_notes(const unsigned char *commit_sha1)
{
- int index;
-
- if (!hash_map.size)
- return NULL;
-
- index = hash_index(&hash_map, commit_sha1);
- if (index < 0)
- return NULL;
- return hash_map.entries[index].notes_sha1;
+ struct leaf_node *found = note_tree_find(&root_node, 0, commit_sha1);
+ if (found)
+ return found->val_sha1;
+ return NULL;
}
void get_commit_notes(const struct commit *commit, struct strbuf *sb,
@@ -120,7 +282,7 @@ void get_commit_notes(const struct commit *commit, struct strbuf *sb,
notes_ref_name = getenv(GIT_NOTES_REF_ENVIRONMENT);
else if (!notes_ref_name)
notes_ref_name = GIT_NOTES_DEFAULT_REF;
- initialize_hash_map(notes_ref_name);
+ initialize_notes(notes_ref_name);
initialized = 1;
}
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
* [PATCHv4 10/12] notes.c: Implement simple memory pooling of leaf nodes
From: Johan Herland @ 2009-08-27 1:43 UTC (permalink / raw)
To: gitster
Cc: git, johan, Johannes.Schindelin, trast, tavestbo, git, chriscool,
spearce
In-Reply-To: <1251337437-16947-1-git-send-email-johan@herland.net>
Allocate page-sized chunks for holding struct leaf_node objects.
This slightly (but consistently) improves runtime performance of notes
lookup, at a very slight increase (~2K on average) in memory usage.
When allocating a new memory pool, the older pool is leaked, but this is
no worse than the current situation, where (pretty much) all leaf_nodes
are leaked anyway.
Signed-off-by: Johan Herland <johan@herland.net>
---
notes.c | 22 ++++++++++++++++++----
1 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/notes.c b/notes.c
index a637056..5d1ee17 100644
--- a/notes.c
+++ b/notes.c
@@ -55,6 +55,23 @@ static int initialized;
static struct int_node root_node;
+/* leaf nodes are allocated in simple memory pools */
+#define LEAF_NODE_POOL_SIZE 100
+static struct leaf_node *leaf_node_pool;
+static unsigned int leaf_node_pool_used;
+
+
+static struct leaf_node *new_leaf_node()
+{
+ if (!leaf_node_pool || leaf_node_pool_used >= LEAF_NODE_POOL_SIZE) {
+ /* MEMORY LEAK: */
+ leaf_node_pool = (struct leaf_node *)
+ xcalloc(sizeof(struct leaf_node), LEAF_NODE_POOL_SIZE);
+ leaf_node_pool_used = 0;
+ }
+ return leaf_node_pool + leaf_node_pool_used++;
+}
+
static void load_subtree(struct leaf_node *subtree, struct int_node *node,
unsigned int n);
@@ -95,7 +112,6 @@ static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
/* unpack tree and resume search */
tree->a[i] = NULL;
load_subtree(l, tree, n);
- free(l);
return note_tree_find(tree, n, key_sha1);
}
break;
@@ -118,7 +134,6 @@ static struct leaf_node *note_tree_find(struct int_node *tree, unsigned char n,
/* unpack tree and resume search */
tree->a[0] = NULL;
load_subtree(l, tree, n);
- free(l);
return note_tree_find(tree, n, key_sha1);
}
return NULL;
@@ -229,8 +244,7 @@ static void load_subtree(struct leaf_node *subtree, struct int_node *node,
*/
if (len <= 20) {
unsigned char type = PTR_TYPE_NOTE;
- struct leaf_node *l = (struct leaf_node *)
- xcalloc(sizeof(struct leaf_node), 1);
+ struct leaf_node *l = new_leaf_node();
hashcpy(l->key_sha1, commit_sha1);
hashcpy(l->val_sha1, entry.sha1);
if (len < 20) {
--
1.6.4.304.g1365c.dirty
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox