* [PATCH 0/5] Memory-limited pack-object window support
From: Brian Downing @ 2007-07-12 3:14 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Brian Downing
This patch series implements a memory limit on the window size for
pack-objects and repack. Basically, the window size will temporarily
grow smaller than the --window option specifies if the total memory
usage of the window is over the specified limit.
With this series I can run even with --window=1000 on my troublesome
repository (described in an email a couple of days ago; basically, it
has a 20MB file with around 200 revisions, plus a bunch of normal sized
files) with memory usage limited to 512MB and it will happily scale down
the window to accommodate the file without blowing out my machine.
The --window option still specifies the size of the window and is still
required; the window will not grow to reach the memory limit, it will
only shrink to fit. I think this may be a feature, as running with a
very large window depth and a memory limit basically means that you will
pack at an approximately constant slow speed, rather than rushing ahead
as it does now for very small objects.
I took the easy way out and expire objects out of the window after
allocation has occurred, rather than figuring out how much needed to
be cleared before allocating. This made the logic much more feasible,
though.
I chose --window-bytes=N and pack.windowBytes=N as my option and
configuration names. I'm not in love with them, though; no other
configuration in Git mentions bytes in the name as far as I can see,
but I wanted to clearly differentiate it from the window "size" that
--window gets you.
The first patch in this series is optional, but I recommend it. It makes
it so that files that are much smaller than a potential delta source
don't even try to delta with it. This is handy for running in a mixed
file size repository with a large window, as it means that when you get
to small files again you start moving fast without having to wait for
the large objects to trickle out of the window. The cut-off is currently
if it is 1/32 the size, but that number was completely arbitrary.
[PATCH 1/5] Don't try to delta if target is much smaller than source
[PATCH 2/5] Support fetching the memory usage of a delta index
[PATCH 3/5] Add pack-objects window memory usage limit
[PATCH 4/5] Add --window-bytes option to git-repack
[PATCH 5/5] Add documentation for --window-bytes, pack.windowBytes
Documentation/config.txt | 5 +++
Documentation/git-pack-objects.txt | 8 +++++
Documentation/git-repack.txt | 8 +++++
builtin-pack-objects.c | 58 +++++++++++++++++++++++++++++++----
delta.h | 7 ++++
diff-delta.c | 10 ++++++
git-repack.sh | 3 +-
7 files changed, 91 insertions(+), 8 deletions(-)
-bcd
^ permalink raw reply
* [PATCH 2/5] Support fetching the memory usage of a delta index
From: Brian Downing @ 2007-07-12 3:14 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Brian Downing
In-Reply-To: <11842100581060-git-send-email-bdowning@lavos.net>
Delta indexes, at least on 64-bit platforms, tend to be larger than
the actual uncompressed data. As such, keeping track of this storage
is important if you want to successfully limit the memory size of your
pack window.
Squirrel away the total allocation size inside the delta_index struct,
and add an accessor "sizeof_delta_index" to access it.
Signed-off-by: Brian Downing <bdowning@lavos.net>
---
delta.h | 7 +++++++
diff-delta.c | 10 ++++++++++
2 files changed, 17 insertions(+), 0 deletions(-)
diff --git a/delta.h b/delta.h
index 7b3f86d..40ccf5a 100644
--- a/delta.h
+++ b/delta.h
@@ -24,6 +24,13 @@ create_delta_index(const void *buf, unsigned long bufsize);
extern void free_delta_index(struct delta_index *index);
/*
+ * sizeof_delta_index: returns memory usage of delta index
+ *
+ * Given pointer must be what create_delta_index() returned, or NULL.
+ */
+extern unsigned long sizeof_delta_index(struct delta_index *index);
+
+/*
* create_delta: create a delta from given index for the given buffer
*
* This function may be called multiple times with different buffers using
diff --git a/diff-delta.c b/diff-delta.c
index faf96e4..3af5835 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -119,6 +119,7 @@ struct index_entry {
};
struct delta_index {
+ unsigned long memsize;
const void *src_buf;
unsigned long src_size;
unsigned int hash_mask;
@@ -159,6 +160,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
mem = hash + hsize;
entry = mem;
+ index->memsize = memsize;
index->src_buf = buf;
index->src_size = bufsize;
index->hash_mask = hmask;
@@ -228,6 +230,14 @@ void free_delta_index(struct delta_index *index)
free(index);
}
+unsigned long sizeof_delta_index(struct delta_index *index)
+{
+ if (index)
+ return index->memsize;
+ else
+ return 0;
+}
+
/*
* The maximum size for any opcode sequence, including the initial header
* plus Rabin window plus biggest copy.
--
1.5.2.GIT
^ permalink raw reply related
* [PATCH 1/5] Don't try to delta if target is much smaller than source
From: Brian Downing @ 2007-07-12 3:14 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Brian Downing
In-Reply-To: <11842100581060-git-send-email-bdowning@lavos.net>
Add a new try_delta heuristic: Don't bother trying to make a delta if
the target object size is much smaller (currently 1/32) than the source,
as it's very likely not going to get a match. Even if it does, you will
have to read at least 32x the size of the new file to reassemble it,
which isn't such a good deal. This leads to a considerable performance
improvement when deltifying a mix of small and large files with a very
large window, because you don't have to wait for the large files to
percolate out of the window before things start going fast again.
Signed-off-by: Brian Downing <bdowning@lavos.net>
---
builtin-pack-objects.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 54b9d26..132ce96 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1342,6 +1342,8 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
sizediff = src_size < trg_size ? trg_size - src_size : 0;
if (sizediff >= max_size)
return 0;
+ if (trg_size < src_size / 32)
+ return 0;
/* Load data if not already done */
if (!trg->data) {
--
1.5.2.GIT
^ permalink raw reply related
* [PATCH] Pack information tool
From: Brian Downing @ 2007-07-12 3:02 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Brian Downing, Nicolas Pitre, git
In-Reply-To: <7vwsx61u7l.fsf@assigned-by-dhcp.cox.net>
This tool will print vaguely pretty information about a pack. It
expects the output of "git-verify-pack -v" as input on stdin.
$ git-verify-pack -v | packinfo.pl
See the documentation in the script (contrib/stats/packinfo.pl)
for more information.
Signed-off-by: Brian Downing <bdowning@lavos.net>
---
I've been fighting with the mail configuraton on my laptop so that
"git send-email" can send email that vger will accept. My apologies
if anybody got dupes or mangled messages.
contrib/stats/packinfo.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 212 insertions(+), 0 deletions(-)
create mode 100755 contrib/stats/packinfo.pl
diff --git a/contrib/stats/packinfo.pl b/contrib/stats/packinfo.pl
new file mode 100755
index 0000000..792673a
--- /dev/null
+++ b/contrib/stats/packinfo.pl
@@ -0,0 +1,212 @@
+#!/bin/perl
+#
+# This tool will print vaguely pretty information about a pack. It
+# expects the output of "git-verify-pack -v" as input on stdin.
+#
+# $ git-verify-pack -v | packinfo.pl
+#
+# This prints some full-pack statistics; currently "all sizes", "all
+# path sizes", "tree sizes", "tree path sizes", and "depths".
+#
+# * "all sizes" stats are across every object size in the file;
+# full sizes for base objects, and delta size for deltas.
+# * "all path sizes" stats are across all object's "path sizes".
+# A path size is the sum of the size of the delta chain, including the
+# base object. In other words, it's how many bytes need be read to
+# reassemble the file from deltas.
+# * "tree sizes" are object sizes grouped into delta trees.
+# * "tree path sizes" are path sizes grouped into delta trees.
+# * "depths" should be obvious.
+#
+# When run as:
+#
+# $ git-verify-pack -v | packinfo.pl -tree
+#
+# the trees of objects are output along with the stats. This looks
+# like:
+#
+# 0 commit 031321c6... 803 803
+#
+# 0 blob 03156f21... 1767 1767
+# 1 blob f52a9d7f... 10 1777
+# 2 blob a8cc5739... 51 1828
+# 3 blob 660e90b1... 15 1843
+# 4 blob 0cb8e3bb... 33 1876
+# 2 blob e48607f0... 311 2088
+# size: count 6 total 2187 min 10 max 1767 mean 364.50 median 51 std_dev 635.85
+# path size: count 6 total 11179 min 1767 max 2088 mean 1863.17 median 1843 std_dev 107.26
+#
+# The first number after the sha1 is the object size, the second
+# number is the path size. The statistics are across all objects in
+# the previous delta tree. Obviously they are omitted for trees of
+# one object.
+#
+# When run as:
+#
+# $ git-verify-pack -v | packinfo.pl -tree -filenames
+#
+# it adds filenames to the tree. Getting this information is slow:
+#
+# 0 blob 03156f21... 1767 1767 Documentation/git-lost-found.txt @ tags/v1.2.0~142
+# 1 blob f52a9d7f... 10 1777 Documentation/git-lost-found.txt @ tags/v1.5.0-rc1~74
+# 2 blob a8cc5739... 51 1828 Documentation/git-lost+found.txt @ tags/v0.99.9h^0
+# 3 blob 660e90b1... 15 1843 Documentation/git-lost+found.txt @ master~3222^2~2
+# 4 blob 0cb8e3bb... 33 1876 Documentation/git-lost+found.txt @ master~3222^2~3
+# 2 blob e48607f0... 311 2088 Documentation/git-lost-found.txt @ tags/v1.5.2-rc3~4
+# size: count 6 total 2187 min 10 max 1767 mean 364.50 median 51 std_dev 635.85
+# path size: count 6 total 11179 min 1767 max 2088 mean 1863.17 median 1843 std_dev 107.26
+#
+# When run as:
+#
+# $ git-verify-pack -v | packinfo.pl -dump
+#
+# it prints out "sha1 size pathsize depth" for each sha1 in lexical
+# order.
+#
+# 000079a2eaef17b7eae70e1f0f635557ea67b644 30 472 7
+# 00013cafe6980411aa6fdd940784917b5ff50f0a 44 1542 4
+# 000182eacf99cde27d5916aa415921924b82972c 499 499 0
+# ...
+#
+# This is handy for comparing two packs. Adding "-filenames" will add
+# filenames, as per "-tree -filenames" above.
+
+use strict;
+use Getopt::Long;
+
+my $filenames = 0;
+my $tree = 0;
+my $dump = 0;
+GetOptions("tree" => \$tree,
+ "filenames" => \$filenames,
+ "dump" => \$dump);
+
+my %parents;
+my %children;
+my %sizes;
+my @roots;
+my %paths;
+my %types;
+my @commits;
+my %names;
+my %depths;
+my @depths;
+
+while (<STDIN>) {
+ my ($sha1, $type, $size, $offset, $depth, $parent) = split(/\s+/, $_);
+ next unless ($sha1 =~ /^[0-9a-f]{40}$/);
+ $depths{$sha1} = $depth || 0;
+ push(@depths, $depth || 0);
+ push(@commits, $sha1) if ($type eq 'commit');
+ push(@roots, $sha1) unless $parent;
+ $parents{$sha1} = $parent;
+ $types{$sha1} = $type;
+ push(@{$children{$parent}}, $sha1);
+ $sizes{$sha1} = $size;
+}
+
+if ($filenames && ($tree || $dump)) {
+ open(NAMES, "git-name-rev --all|");
+ while (<NAMES>) {
+ if (/^(\S+)\s+(.*)$/) {
+ my ($sha1, $name) = ($1, $2);
+ $names{$sha1} = $name;
+ }
+ }
+ close NAMES;
+
+ for my $commit (@commits) {
+ my $name = $names{$commit};
+ open(TREE, "git-ls-tree -t -r $commit|");
+ print STDERR "Plumbing tree $name\n";
+ while (<TREE>) {
+ if (/^(\S+)\s+(\S+)\s+(\S+)\s+(.*)$/) {
+ my ($mode, $type, $sha1, $path) = ($1, $2, $3, $4);
+ $paths{$sha1} = "$path @ $name";
+ }
+ }
+ close TREE;
+ }
+}
+
+sub stats {
+ my @data = sort {$a <=> $b} @_;
+ my $min = $data[0];
+ my $max = $data[$#data];
+ my $total = 0;
+ my $count = scalar @data;
+ for my $datum (@data) {
+ $total += $datum;
+ }
+ my $mean = $total / $count;
+ my $median = $data[int(@data / 2)];
+ my $diff_sum = 0;
+ for my $datum (@data) {
+ $diff_sum += ($datum - $mean)**2;
+ }
+ my $std_dev = sqrt($diff_sum / $count);
+ return ($count, $total, $min, $max, $mean, $median, $std_dev);
+}
+
+sub print_stats {
+ my $name = shift;
+ my ($count, $total, $min, $max, $mean, $median, $std_dev) = stats(@_);
+ printf("%s: count %s total %s min %s max %s mean %.2f median %s std_dev %.2f\n",
+ $name, $count, $total, $min, $max, $mean, $median, $std_dev);
+}
+
+my @sizes;
+my @path_sizes;
+my @all_sizes;
+my @all_path_sizes;
+my %path_sizes;
+
+sub dig {
+ my ($sha1, $depth, $path_size) = @_;
+ $path_size += $sizes{$sha1};
+ push(@sizes, $sizes{$sha1});
+ push(@all_sizes, $sizes{$sha1});
+ push(@path_sizes, $path_size);
+ push(@all_path_sizes, $path_size);
+ $path_sizes{$sha1} = $path_size;
+ if ($tree) {
+ printf("%3d%s %6s %s %8d %8d %s\n",
+ $depth, (" " x $depth), $types{$sha1},
+ $sha1, $sizes{$sha1}, $path_size, $paths{$sha1});
+ }
+ for my $child (@{$children{$sha1}}) {
+ dig($child, $depth + 1, $path_size);
+ }
+}
+
+my @tree_sizes;
+my @tree_path_sizes;
+
+for my $root (@roots) {
+ undef @sizes;
+ undef @path_sizes;
+ dig($root, 0, 0);
+ my ($aa, $sz_total) = stats(@sizes);
+ my ($bb, $psz_total) = stats(@path_sizes);
+ push(@tree_sizes, $sz_total);
+ push(@tree_path_sizes, $psz_total);
+ if ($tree) {
+ if (@sizes > 1) {
+ print_stats(" size", @sizes);
+ print_stats("path size", @path_sizes);
+ }
+ print "\n";
+ }
+}
+
+if ($dump) {
+ for my $sha1 (sort keys %sizes) {
+ print "$sha1 $sizes{$sha1} $path_sizes{$sha1} $depths{$sha1} $paths{$sha1}\n";
+ }
+} else {
+ print_stats(" all sizes", @all_sizes);
+ print_stats(" all path sizes", @all_path_sizes);
+ print_stats(" tree sizes", @tree_sizes);
+ print_stats("tree path sizes", @tree_path_sizes);
+ print_stats(" depths", @depths);
+}
--
1.5.2.GIT
^ permalink raw reply related
* Re: [PATCH 0/4] The rest of builtin-fetch
From: Daniel Barkalow @ 2007-07-12 1:49 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0707120244200.4516@racer.site>
On Thu, 12 Jul 2007, Johannes Schindelin wrote:
> Hi,
>
> On Wed, 11 Jul 2007, Daniel Barkalow wrote:
>
> > This series is the rest of builtin-fetch. It should probably not be used
> > without a "[PATCH 3.5/4] Support bundles in transport fetch" which
> > Johannes was going to write.
>
> 8-)
>
> It is 3am, and I am already in the middle of git hacking, and should go to
> bed, so that'll have to wait for tomorrow...
We're presumably waiting for 1.5.3 before this actually goes anywhere, so
there's no big hurry. Just don't want it to get forgotten and have
mainline versions where the tests don't pass due to bundles not working
(or, worse, an actual release where we forget about them in this context).
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: [PATCH 0/4] The rest of builtin-fetch
From: Johannes Schindelin @ 2007-07-12 1:44 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0707112113460.6977@iabervon.org>
Hi,
On Wed, 11 Jul 2007, Daniel Barkalow wrote:
> This series is the rest of builtin-fetch. It should probably not be used
> without a "[PATCH 3.5/4] Support bundles in transport fetch" which
> Johannes was going to write.
8-)
It is 3am, and I am already in the middle of git hacking, and should go to
bed, so that'll have to wait for tomorrow...
Ciao,
Dscho
^ permalink raw reply
* [PATCH 4/4] Make fetch a builtin
From: Daniel Barkalow @ 2007-07-12 1:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
As an implementation of fetch, this changes a few small bits of behavior:
branch.<name>.merge is parsed as if it were the lhs of a fetch
refspec, and does not have to exactly match the actual lhs of a
refspec, so long as it is a valid abbreviation for the same ref.
branch.<name>.merge is no longer ignored if the remote is configured
with a branches/* file. Neither behavior is useful, because there can
only be one ref that gets fetched, but this is more consistant.
Changes to the tests and documentation are included in this patch. Without
a fetch-from-bundle implementation, one of the tests will fail with this
patch applied.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
Documentation/config.txt | 9 +-
Makefile | 2 +-
builtin-fetch.c | 552 ++++++++++++++++++++
builtin.h | 1 +
cache.h | 1 +
git-fetch.sh | 377 -------------
git.c | 1 +
t/t5515/fetch.br-branches-default-merge | 2 +-
...etch.br-branches-default-merge_branches-default | 2 +-
t/t5515/fetch.br-branches-default-octopus | 2 +-
...ch.br-branches-default-octopus_branches-default | 2 +-
t/t5515/fetch.br-branches-one-merge | 2 +-
t/t5515/fetch.br-branches-one-merge_branches-one | 2 +-
t/t5515/fetch.br-config-glob-octopus | 2 +-
t/t5515/fetch.br-config-glob-octopus_config-glob | 2 +-
t/t5515/fetch.br-remote-glob-octopus | 2 +-
t/t5515/fetch.br-remote-glob-octopus_remote-glob | 2 +-
17 files changed, 571 insertions(+), 392 deletions(-)
create mode 100644 builtin-fetch.c
delete mode 100755 git-fetch.sh
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 50503e8..389b21d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -312,10 +312,11 @@ branch.<name>.remote::
If this option is not given, `git fetch` defaults to remote "origin".
branch.<name>.merge::
- When in branch <name>, it tells `git fetch` the default refspec to
- be marked for merging in FETCH_HEAD. The value has exactly to match
- a remote part of one of the refspecs which are fetched from the remote
- given by "branch.<name>.remote".
+ When in branch <name>, it tells `git fetch` the default
+ refspec to be marked for merging in FETCH_HEAD. The value is
+ handled like the remote part of a refspec, and must match a
+ ref which is fetched from the remote given by
+ "branch.<name>.remote".
The merge information is used by `git pull` (which at first calls
`git fetch`) to lookup the default branch for merging. Without
this option, `git pull` defaults to merge the first refspec fetched.
diff --git a/Makefile b/Makefile
index 83e699f..899e8c1 100644
--- a/Makefile
+++ b/Makefile
@@ -201,7 +201,6 @@ BASIC_LDFLAGS =
SCRIPT_SH = \
git-bisect.sh git-checkout.sh \
git-clean.sh git-clone.sh git-commit.sh \
- git-fetch.sh \
git-ls-remote.sh \
git-merge-one-file.sh git-mergetool.sh git-parse-remote.sh \
git-pull.sh git-rebase.sh git-rebase--interactive.sh \
@@ -340,6 +339,7 @@ BUILTIN_OBJS = \
builtin-diff-files.o \
builtin-diff-index.o \
builtin-diff-tree.o \
+ builtin-fetch.o \
builtin-fetch-pack.o \
builtin-fetch--tool.o \
builtin-fmt-merge-msg.o \
diff --git a/builtin-fetch.c b/builtin-fetch.c
new file mode 100644
index 0000000..2f24c22
--- /dev/null
+++ b/builtin-fetch.c
@@ -0,0 +1,552 @@
+/*
+ * "git fetch"
+ */
+#include "cache.h"
+#include "refs.h"
+#include "commit.h"
+#include "builtin.h"
+#include "path-list.h"
+#include "remote.h"
+#include "transport.h"
+
+static const char fetch_usage[] = "git-fetch [-a | --append] [--upload-pack <upload-pack>] [-f | --force] [--no-tags] [-t | --tags] [-k | --keep] [-u | --update-head-ok] [--depth <depth>] [-v | --verbose] [<repository> <refspec>...]";
+
+static int append, force, tags, no_tags, update_head_ok, verbose, quiet;
+
+static int unpacklimit;
+
+static char *default_rla = NULL;
+
+static void find_merge_config(struct ref *ref_map, struct remote *remote)
+{
+ struct ref *rm = ref_map;
+ struct branch *branch = branch_get(NULL);
+
+ for (rm = ref_map; rm; rm = rm->next) {
+ if (!branch_has_merge_config(branch)) {
+ if (remote && remote->fetch &&
+ !strcmp(remote->fetch[0].src, rm->name))
+ rm->merge = 1;
+ } else {
+ if (branch_merges(branch, rm->name))
+ rm->merge = 1;
+ }
+ }
+}
+
+static struct ref *get_ref_map(struct transport *transport,
+ struct refspec *refs, int ref_count, int tags,
+ int *autotags)
+{
+ int i;
+ struct ref *rm;
+ struct ref *ref_map = NULL;
+ struct ref **tail = &ref_map;
+
+ struct ref *remote_refs = transport_get_remote_refs(transport);
+
+ if (ref_count || tags) {
+ for (i = 0; i < ref_count; i++) {
+ get_fetch_map(remote_refs, &refs[i], &tail);
+ if (refs[i].dst && refs[i].dst[0])
+ *autotags = 1;
+ }
+ /* Merge everything on the command line, but not --tags */
+ for (rm = ref_map; rm; rm = rm->next)
+ rm->merge = 1;
+ if (tags) {
+ struct refspec refspec;
+ refspec.src = "refs/tags/";
+ refspec.dst = "refs/tags/";
+ refspec.pattern = 1;
+ refspec.force = 0;
+ get_fetch_map(remote_refs, &refspec, &tail);
+ }
+ } else {
+ /* Use the defaults */
+ struct remote *remote = transport->remote;
+ if (remote->fetch_refspec_nr) {
+ for (i = 0; i < remote->fetch_refspec_nr; i++) {
+ get_fetch_map(remote_refs, &remote->fetch[i], &tail);
+ if (remote->fetch[i].dst &&
+ remote->fetch[i].dst[0])
+ *autotags = 1;
+ }
+ find_merge_config(ref_map, remote);
+ } else {
+ ref_map = get_remote_ref(remote_refs, "HEAD");
+
+ ref_map->merge = 1;
+ }
+ }
+
+ return ref_map;
+}
+
+static void show_new(enum object_type type, unsigned char *sha1_new)
+{
+ fprintf(stderr, " %s: %s\n", typename(type),
+ find_unique_abbrev(sha1_new, DEFAULT_ABBREV));
+}
+
+static int update_ref(const char *action,
+ struct ref *ref,
+ int check_old)
+{
+ char msg[1024];
+ char *rla = getenv("GIT_REFLOG_ACTION");
+ static struct ref_lock *lock;
+
+ if (!rla)
+ rla = default_rla;
+ snprintf(msg, sizeof(msg), "%s: %s", rla, action);
+ lock = lock_any_ref_for_update(ref->name,
+ check_old ? ref->old_sha1 : NULL, 0);
+ if (!lock)
+ return 1;
+ if (write_ref_sha1(lock, ref->new_sha1, msg) < 0)
+ return 1;
+ return 0;
+}
+
+static int update_local_ref(struct ref *ref,
+ const char *note,
+ int verbose)
+{
+ char oldh[41], newh[41];
+ struct commit *current = NULL, *updated;
+ enum object_type type;
+ struct branch *current_branch = branch_get(NULL);
+
+ type = sha1_object_info(ref->new_sha1, NULL);
+ if (type < 0)
+ die("object %s not found", sha1_to_hex(ref->new_sha1));
+
+ if (!*ref->name) {
+ /* Not storing */
+ if (verbose) {
+ fprintf(stderr, "* fetched %s\n", note);
+ show_new(type, ref->new_sha1);
+ }
+ return 0;
+ }
+
+ if (!hashcmp(ref->old_sha1, ref->new_sha1)) {
+ if (verbose) {
+ fprintf(stderr, "* %s: same as %s\n",
+ ref->name, note);
+ show_new(type, ref->new_sha1);
+ }
+ return 0;
+ }
+
+ if (!strcmp(ref->name, current_branch->name) &&
+ !(update_head_ok || is_bare_repository()) &&
+ !is_null_sha1(ref->old_sha1)) {
+ /*
+ * If this is the head, and it's not okay to update
+ * the head, and the old value of the head isn't empty...
+ */
+ fprintf(stderr,
+ " * %s: Cannot fetch into the current branch.\n",
+ ref->name);
+ return 1;
+ }
+
+ if (!is_null_sha1(ref->old_sha1) &&
+ !prefixcmp(ref->name, "refs/tags/")) {
+ fprintf(stderr, "* %s: updating with %s\n",
+ ref->name, note);
+ show_new(type, ref->new_sha1);
+ return update_ref("updating tag", ref, 0);
+ }
+
+ current = lookup_commit_reference(ref->old_sha1);
+ updated = lookup_commit_reference(ref->new_sha1);
+ if (!current || !updated) {
+ char *msg;
+ if (!strncmp(ref->name, "refs/tags/", 10))
+ msg = "storing tag";
+ else
+ msg = "storing head";
+ fprintf(stderr, "* %s: storing %s\n",
+ ref->name, note);
+ show_new(type, ref->new_sha1);
+ return update_ref(msg, ref, 0);
+ }
+
+ strcpy(oldh, find_unique_abbrev(current->object.sha1, DEFAULT_ABBREV));
+ strcpy(newh, find_unique_abbrev(ref->new_sha1, DEFAULT_ABBREV));
+
+ if (in_merge_bases(current, &updated, 1)) {
+ fprintf(stderr, "* %s: fast forward to %s\n",
+ ref->name, note);
+ fprintf(stderr, " old..new: %s..%s\n", oldh, newh);
+ return update_ref("fast forward", ref, 1);
+ }
+ if (!force && !ref->force) {
+ fprintf(stderr,
+ "* %s: not updating to non-fast forward %s\n",
+ ref->name, note);
+ fprintf(stderr,
+ " old...new: %s...%s\n", oldh, newh);
+ return 1;
+ }
+ fprintf(stderr,
+ "* %s: forcing update to non-fast forward %s\n",
+ ref->name, note);
+ fprintf(stderr, " old...new: %s...%s\n", oldh, newh);
+ return update_ref("forced-update", ref, 1);
+}
+
+static void store_updated_refs(const char *url, struct ref *ref_map)
+{
+ FILE *fp;
+ struct commit *commit;
+ int url_len, i, note_len;
+ char note[1024];
+ const char *what, *kind;
+ struct ref *rm;
+
+ fp = fopen(git_path("FETCH_HEAD"), "a");
+ for (rm = ref_map; rm; rm = rm->next) {
+ struct ref *ref = NULL;
+
+ if (rm->peer_ref) {
+ ref = xcalloc(1, sizeof(*ref) + strlen(rm->peer_ref->name) + 1);
+ strcpy(ref->name, rm->peer_ref->name);
+ hashcpy(ref->old_sha1, rm->peer_ref->old_sha1);
+ hashcpy(ref->new_sha1, rm->old_sha1);
+ ref->force = rm->force;
+ }
+
+ commit = lookup_commit_reference(rm->old_sha1);
+ if (!commit)
+ rm->merge = 0;
+
+ if (!strcmp(rm->name, "HEAD")) {
+ kind = "";
+ what = "";
+ }
+ else if (!prefixcmp(rm->name, "refs/heads/")) {
+ kind = "branch";
+ what = rm->name + 11;
+ }
+ else if (!prefixcmp(rm->name, "refs/tags/")) {
+ kind = "tag";
+ what = rm->name + 10;
+ }
+ else if (!prefixcmp(rm->name, "refs/remotes/")) {
+ kind = "remote branch";
+ what = rm->name + 13;
+ }
+ else {
+ kind = "";
+ what = rm->name;
+ }
+
+ url_len = strlen(url);
+ for (i = url_len - 1; url[i] == '/' && 0 <= i; i--)
+ ;
+ url_len = i + 1;
+ if (4 < i && !strncmp(".git", url + i - 3, 4))
+ url_len = i - 3;
+
+ note_len = 0;
+ if (*what) {
+ if (*kind)
+ note_len += sprintf(note + note_len, "%s ",
+ kind);
+ note_len += sprintf(note + note_len, "'%s' of ", what);
+ }
+ note_len += sprintf(note + note_len, "%.*s", url_len, url);
+ fprintf(fp, "%s\t%s\t%s\n",
+ sha1_to_hex(commit ? commit->object.sha1 :
+ rm->old_sha1),
+ rm->merge ? "" : "not-for-merge",
+ note);
+
+ if (ref)
+ update_local_ref(ref, note, verbose);
+ }
+ fclose(fp);
+}
+
+static int fetch_refs(struct transport *transport, struct ref *ref_map)
+{
+ int ret = transport_fetch_refs(transport, ref_map);
+ if (!ret)
+ store_updated_refs(transport->url, ref_map);
+ return ret;
+}
+
+static int add_existing(const char *refname, const unsigned char *sha1,
+ int flag, void *cbdata)
+{
+ struct path_list *list = (struct path_list *)cbdata;
+ path_list_insert(refname, list);
+ return 0;
+}
+
+static struct ref *find_non_local_tags(struct transport *transport,
+ struct ref *fetch_map)
+{
+ static struct path_list existing_refs = { NULL, 0, 0, 0 };
+ struct path_list new_refs = { NULL, 0, 0, 1 };
+ char *ref_name;
+ int ref_name_len;
+ unsigned char *ref_sha1;
+ struct ref *tag_ref;
+ struct ref *rm = NULL;
+ struct ref *ref_map = NULL;
+ struct ref **tail = &ref_map;
+ struct ref *ref;
+
+ for_each_ref(add_existing, &existing_refs);
+ for (ref = transport_get_remote_refs(transport); ref; ref = ref->next) {
+ if (prefixcmp(ref->name, "refs/tags"))
+ continue;
+
+ ref_name = xstrdup(ref->name);
+ ref_name_len = strlen(ref_name);
+ ref_sha1 = ref->old_sha1;
+
+ if (!strcmp(ref_name + ref_name_len - 3, "^{}")) {
+ ref_name[ref_name_len - 3] = 0;
+ tag_ref = transport_get_remote_refs(transport);
+ while (tag_ref) {
+ if (!strcmp(tag_ref->name, ref_name)) {
+ ref_sha1 = tag_ref->old_sha1;
+ break;
+ }
+ tag_ref = tag_ref->next;
+ }
+ }
+
+ if (!path_list_has_path(&existing_refs, ref_name) &&
+ !path_list_has_path(&new_refs, ref_name) &&
+ lookup_object(ref->old_sha1)) {
+ fprintf(stderr, "Auto-following %s\n",
+ ref_name);
+
+ path_list_insert(ref_name, &new_refs);
+
+ rm = alloc_ref(strlen(ref_name) + 1);
+ strcpy(rm->name, ref_name);
+ rm->peer_ref = alloc_ref(strlen(ref_name) + 1);
+ strcpy(rm->peer_ref->name, ref_name);
+ hashcpy(rm->old_sha1, ref_sha1);
+
+ *tail = rm;
+ tail = &rm->next;
+ }
+ free(ref_name);
+ }
+
+ return ref_map;
+}
+
+static int do_fetch(struct transport *transport,
+ struct refspec *refs, int ref_count)
+{
+ struct ref *ref_map, *fetch_map;
+ struct ref *rm;
+ int autotags = (transport->remote->fetch_tags == 1);
+ if (transport->remote->fetch_tags == 2 && !no_tags)
+ tags = 1;
+ if (transport->remote->fetch_tags == -1)
+ no_tags = 1;
+
+ if (!transport->ops || !transport->ops->get_refs_list ||
+ !(transport->ops->fetch_refs || transport->ops->fetch_objs))
+ die("Don't know how to fetch from %s", transport->url);
+
+ /* if not appending, truncate FETCH_HEAD */
+ if (!append)
+ fclose(fopen(git_path("FETCH_HEAD"), "w"));
+
+ ref_map = get_ref_map(transport, refs, ref_count, tags, &autotags);
+
+ for (rm = ref_map; rm; rm = rm->next) {
+ if (rm->peer_ref)
+ read_ref(rm->peer_ref->name, rm->peer_ref->old_sha1);
+
+ printf("%s : %s\n", rm->name, rm->peer_ref ? rm->peer_ref->name : NULL);
+ printf(" < %s\n", sha1_to_hex(rm->old_sha1));
+ if (rm->peer_ref)
+ printf(" > %s\n", sha1_to_hex(rm->peer_ref->old_sha1));
+ if (!rm->peer_ref ||
+ hashcmp(rm->old_sha1, rm->peer_ref->old_sha1)) {
+ printf("%s needs update.\n", rm->name);
+ }
+ }
+
+ fetch_refs(transport, ref_map);
+
+ fetch_map = ref_map;
+
+ /* if neither --no-tags nor --tags was specified, do automated tag
+ * following ... */
+ if (!(tags || no_tags) && autotags) {
+ ref_map = find_non_local_tags(transport, fetch_map);
+ if (ref_map) {
+ transport_set_option(transport, TRANS_OPT_DEPTH, "0");
+ fetch_refs(transport, ref_map);
+ }
+ free_refs(ref_map);
+ }
+
+ free_refs(fetch_map);
+
+ return 0;
+}
+
+static int fetch_config(const char *var, const char *value)
+{
+ if (strcmp(var, "fetch.unpacklimit") == 0) {
+ unpacklimit = git_config_int(var, value);
+ return 0;
+ }
+
+ if (strcmp(var, "transfer.unpacklimit") == 0) {
+ unpacklimit = git_config_int(var, value);
+ return 0;
+ }
+
+ return git_default_config(var, value);
+}
+
+int cmd_fetch(int argc, const char **argv, const char *prefix)
+{
+ struct remote *remote;
+ struct transport *transport;
+ int i, j, rla_offset;
+ static const char **refs = NULL;
+ int ref_nr = 0;
+ int cmd_len = 0;
+ const char *depth = NULL, *upload_pack = NULL;
+ int keep = 0;
+
+ git_config(fetch_config);
+
+ for (i = 1; i < argc; i++) {
+ const char *arg = argv[i];
+ cmd_len += strlen(arg);
+
+ if (arg[0] != '-') {
+ break;
+ }
+ if (!strcmp(arg, "--append") || !strcmp(arg, "-a")) {
+ append = 1;
+ continue;
+ }
+ if (!prefixcmp(arg, "--upload-pack=")) {
+ upload_pack = arg + 14;
+ continue;
+ }
+ if (!strcmp(arg, "--upload-pack")) {
+ i++;
+ if (i == argc)
+ usage(fetch_usage);
+ upload_pack = argv[i];
+ continue;
+ }
+ if (!strcmp(arg, "--force") || !strcmp(arg, "-f")) {
+ force = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--no-tags")) {
+ no_tags = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--tags") || !strcmp(arg, "-t")) {
+ tags = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--keep") || !strcmp(arg, "-k")) {
+ keep = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--update-head-ok") || !strcmp(arg, "-u")) {
+ update_head_ok = 1;
+ continue;
+ }
+ if (!prefixcmp(arg, "--depth=")) {
+ depth = arg + 8;
+ continue;
+ }
+ if (!strcmp(arg, "--depth")) {
+ i++;
+ if (i == argc)
+ usage(fetch_usage);
+ depth = argv[i];
+ continue;
+ }
+ if (!strcmp(arg, "--quiet")) {
+ quiet = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--verbose") || !strcmp(arg, "-v")) {
+ verbose++;
+ continue;
+ }
+ usage(fetch_usage);
+ }
+
+ for (j = i; j < argc; j++) {
+ cmd_len += strlen(argv[j]);
+ }
+
+ default_rla = xmalloc(cmd_len + 5 + argc + 1);
+ sprintf(default_rla, "fetch");
+ rla_offset = strlen(default_rla);
+ for (j = 1; j < argc; j++) {
+ sprintf(default_rla + rla_offset, " %s", argv[j]);
+ rla_offset += strlen(argv[j]);
+ }
+
+ if (i == argc)
+ remote = remote_get(NULL);
+ else
+ remote = remote_get(argv[i++]);
+
+ transport = transport_get(remote, remote->uri[0], 1);
+ if (verbose >= 2)
+ transport->verbose = 1;
+ if (quiet)
+ transport->verbose = 0;
+ if (upload_pack)
+ transport_set_option(transport, TRANS_OPT_UPLOADPACK, upload_pack);
+ if (keep)
+ transport_set_option(transport, TRANS_OPT_KEEP, "yes");
+ transport_set_option(transport, TRANS_OPT_DEPTH, depth);
+
+ if (!transport->url)
+ die("Where do you want to fetch from today?");
+
+ if (i < argc) {
+ int j = 0;
+ refs = xcalloc(argc - i + 1, sizeof(const char *));
+ while (i < argc) {
+ if (!strcmp(argv[i], "tag")) {
+ char *ref;
+ i++;
+ ref = xmalloc(strlen(argv[i]) * 2 + 22);
+ strcpy(ref, "refs/tags/");
+ strcat(ref, argv[i]);
+ strcat(ref, ":refs/tags/");
+ strcat(ref, argv[i]);
+ refs[j++] = ref;
+ } else
+ refs[j++] = argv[i];
+ i++;
+ }
+ refs[j] = NULL;
+ ref_nr = j;
+ for (j = 0; refs[j]; j++) {
+ printf("ref: %s\n", refs[j]);
+ }
+ }
+
+ return do_fetch(transport, parse_ref_spec(ref_nr, refs), ref_nr);
+}
diff --git a/builtin.h b/builtin.h
index 39564d3..b0c538b 100644
--- a/builtin.h
+++ b/builtin.h
@@ -31,6 +31,7 @@ extern int cmd_diff_files(int argc, const char **argv, const char *prefix);
extern int cmd_diff_index(int argc, const char **argv, const char *prefix);
extern int cmd_diff(int argc, const char **argv, const char *prefix);
extern int cmd_diff_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_fetch(int argc, const char **argv, const char *prefix);
extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
extern int cmd_fetch__tool(int argc, const char **argv, const char *prefix);
extern int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);
diff --git a/cache.h b/cache.h
index 4818c8c..b93b605 100644
--- a/cache.h
+++ b/cache.h
@@ -475,6 +475,7 @@ struct ref {
unsigned char old_sha1[20];
unsigned char new_sha1[20];
unsigned char force;
+ unsigned char merge;
struct ref *peer_ref; /* when renaming */
char name[FLEX_ARRAY]; /* more */
};
diff --git a/git-fetch.sh b/git-fetch.sh
deleted file mode 100755
index 6d3a346..0000000
--- a/git-fetch.sh
+++ /dev/null
@@ -1,377 +0,0 @@
-#!/bin/sh
-#
-
-USAGE='<fetch-options> <repository> <refspec>...'
-SUBDIRECTORY_OK=Yes
-. git-sh-setup
-set_reflog_action "fetch $*"
-cd_to_toplevel ;# probably unnecessary...
-
-. git-parse-remote
-_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
-
-LF='
-'
-IFS="$LF"
-
-no_tags=
-tags=
-append=
-force=
-verbose=
-update_head_ok=
-exec=
-keep=
-shallow_depth=
-no_progress=
-test -t 1 || no_progress=--no-progress
-quiet=
-while case "$#" in 0) break ;; esac
-do
- case "$1" in
- -a|--a|--ap|--app|--appe|--appen|--append)
- append=t
- ;;
- --upl|--uplo|--uploa|--upload|--upload-|--upload-p|\
- --upload-pa|--upload-pac|--upload-pack)
- shift
- exec="--upload-pack=$1"
- ;;
- --upl=*|--uplo=*|--uploa=*|--upload=*|\
- --upload-=*|--upload-p=*|--upload-pa=*|--upload-pac=*|--upload-pack=*)
- exec=--upload-pack=$(expr "z$1" : 'z-[^=]*=\(.*\)')
- shift
- ;;
- -f|--f|--fo|--for|--forc|--force)
- force=t
- ;;
- -t|--t|--ta|--tag|--tags)
- tags=t
- ;;
- -n|--n|--no|--no-|--no-t|--no-ta|--no-tag|--no-tags)
- no_tags=t
- ;;
- -u|--u|--up|--upd|--upda|--updat|--update|--update-|--update-h|\
- --update-he|--update-hea|--update-head|--update-head-|\
- --update-head-o|--update-head-ok)
- update_head_ok=t
- ;;
- -q|--q|--qu|--qui|--quie|--quiet)
- quiet=--quiet
- ;;
- -v|--verbose)
- verbose="$verbose"Yes
- ;;
- -k|--k|--ke|--kee|--keep)
- keep='-k -k'
- ;;
- --depth=*)
- shallow_depth="--depth=`expr "z$1" : 'z-[^=]*=\(.*\)'`"
- ;;
- --depth)
- shift
- shallow_depth="--depth=$1"
- ;;
- -*)
- usage
- ;;
- *)
- break
- ;;
- esac
- shift
-done
-
-case "$#" in
-0)
- origin=$(get_default_remote)
- test -n "$(get_remote_url ${origin})" ||
- die "Where do you want to fetch from today?"
- set x $origin ; shift ;;
-esac
-
-if test -z "$exec"
-then
- # No command line override and we have configuration for the remote.
- exec="--upload-pack=$(get_uploadpack $1)"
-fi
-
-remote_nick="$1"
-remote=$(get_remote_url "$@")
-refs=
-rref=
-rsync_slurped_objects=
-
-if test "" = "$append"
-then
- : >"$GIT_DIR/FETCH_HEAD"
-fi
-
-# Global that is reused later
-ls_remote_result=$(git ls-remote $exec "$remote") ||
- die "Cannot get the repository state from $remote"
-
-append_fetch_head () {
- flags=
- test -n "$verbose" && flags="$flags$LF-v"
- test -n "$force$single_force" && flags="$flags$LF-f"
- GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION" \
- git-fetch--tool $flags append-fetch-head "$@"
-}
-
-# updating the current HEAD with git-fetch in a bare
-# repository is always fine.
-if test -z "$update_head_ok" && test $(is_bare_repository) = false
-then
- orig_head=$(git-rev-parse --verify HEAD 2>/dev/null)
-fi
-
-# Allow --notags from remote.$1.tagopt
-case "$tags$no_tags" in
-'')
- case "$(git-config --get "remote.$1.tagopt")" in
- --no-tags)
- no_tags=t ;;
- esac
-esac
-
-# If --tags (and later --heads or --all) is specified, then we are
-# not talking about defaults stored in Pull: line of remotes or
-# branches file, and just fetch those and refspecs explicitly given.
-# Otherwise we do what we always did.
-
-reflist=$(get_remote_refs_for_fetch "$@")
-if test "$tags"
-then
- taglist=`IFS=' ' &&
- echo "$ls_remote_result" |
- git-show-ref --exclude-existing=refs/tags/ |
- while read sha1 name
- do
- echo ".${name}:${name}"
- done` || exit
- if test "$#" -gt 1
- then
- # remote URL plus explicit refspecs; we need to merge them.
- reflist="$reflist$LF$taglist"
- else
- # No explicit refspecs; fetch tags only.
- reflist=$taglist
- fi
-fi
-
-fetch_all_at_once () {
-
- eval=$(echo "$1" | git-fetch--tool parse-reflist "-")
- eval "$eval"
-
- ( : subshell because we muck with IFS
- IFS=" $LF"
- (
- if test "$remote" = . ; then
- git-show-ref $rref || echo failed "$remote"
- elif test -f "$remote" ; then
- test -n "$shallow_depth" &&
- die "shallow clone with bundle is not supported"
- git-bundle unbundle "$remote" $rref ||
- echo failed "$remote"
- else
- if test -d "$remote" &&
-
- # The remote might be our alternate. With
- # this optimization we will bypass fetch-pack
- # altogether, which means we cannot be doing
- # the shallow stuff at all.
- test ! -f "$GIT_DIR/shallow" &&
- test -z "$shallow_depth" &&
-
- # See if all of what we are going to fetch are
- # connected to our repository's tips, in which
- # case we do not have to do any fetch.
- theirs=$(echo "$ls_remote_result" | \
- git-fetch--tool -s pick-rref "$rref" "-") &&
-
- # This will barf when $theirs reach an object that
- # we do not have in our repository. Otherwise,
- # we already have everything the fetch would bring in.
- git-rev-list --objects $theirs --not --all \
- >/dev/null 2>/dev/null
- then
- echo "$ls_remote_result" | \
- git-fetch--tool pick-rref "$rref" "-"
- else
- flags=
- case $verbose in
- YesYes*)
- flags="-v"
- ;;
- esac
- git-fetch-pack --thin $exec $keep $shallow_depth \
- $quiet $no_progress $flags "$remote" $rref ||
- echo failed "$remote"
- fi
- fi
- ) |
- (
- flags=
- test -n "$verbose" && flags="$flags -v"
- test -n "$force" && flags="$flags -f"
- GIT_REFLOG_ACTION="$GIT_REFLOG_ACTION" \
- git-fetch--tool $flags native-store \
- "$remote" "$remote_nick" "$refs"
- )
- ) || exit
-
-}
-
-fetch_per_ref () {
- reflist="$1"
- refs=
- rref=
-
- for ref in $reflist
- do
- refs="$refs$LF$ref"
-
- # These are relative path from $GIT_DIR, typically starting at refs/
- # but may be HEAD
- if expr "z$ref" : 'z\.' >/dev/null
- then
- not_for_merge=t
- ref=$(expr "z$ref" : 'z\.\(.*\)')
- else
- not_for_merge=
- fi
- if expr "z$ref" : 'z+' >/dev/null
- then
- single_force=t
- ref=$(expr "z$ref" : 'z+\(.*\)')
- else
- single_force=
- fi
- remote_name=$(expr "z$ref" : 'z\([^:]*\):')
- local_name=$(expr "z$ref" : 'z[^:]*:\(.*\)')
-
- rref="$rref$LF$remote_name"
-
- # There are transports that can fetch only one head at a time...
- case "$remote" in
- http://* | https://* | ftp://*)
- test -n "$shallow_depth" &&
- die "shallow clone with http not supported"
- proto=`expr "$remote" : '\([^:]*\):'`
- if [ -n "$GIT_SSL_NO_VERIFY" ]; then
- curl_extra_args="-k"
- fi
- if [ -n "$GIT_CURL_FTP_NO_EPSV" -o \
- "`git-config --bool http.noEPSV`" = true ]; then
- noepsv_opt="--disable-epsv"
- fi
-
- # Find $remote_name from ls-remote output.
- head=$(echo "$ls_remote_result" | \
- git-fetch--tool -s pick-rref "$remote_name" "-")
- expr "z$head" : "z$_x40\$" >/dev/null ||
- die "No such ref $remote_name at $remote"
- echo >&2 "Fetching $remote_name from $remote using $proto"
- case "$quiet" in '') v=-v ;; *) v= ;; esac
- git-http-fetch $v -a "$head" "$remote" || exit
- ;;
- rsync://*)
- test -n "$shallow_depth" &&
- die "shallow clone with rsync not supported"
- TMP_HEAD="$GIT_DIR/TMP_HEAD"
- rsync -L -q "$remote/$remote_name" "$TMP_HEAD" || exit 1
- head=$(git-rev-parse --verify TMP_HEAD)
- rm -f "$TMP_HEAD"
- case "$quiet" in '') v=-v ;; *) v= ;; esac
- test "$rsync_slurped_objects" || {
- rsync -a $v --ignore-existing --exclude info \
- "$remote/objects/" "$GIT_OBJECT_DIRECTORY/" || exit
-
- # Look at objects/info/alternates for rsync -- http will
- # support it natively and git native ones will do it on
- # the remote end. Not having that file is not a crime.
- rsync -q "$remote/objects/info/alternates" \
- "$GIT_DIR/TMP_ALT" 2>/dev/null ||
- rm -f "$GIT_DIR/TMP_ALT"
- if test -f "$GIT_DIR/TMP_ALT"
- then
- resolve_alternates "$remote" <"$GIT_DIR/TMP_ALT" |
- while read alt
- do
- case "$alt" in 'bad alternate: '*) die "$alt";; esac
- echo >&2 "Getting alternate: $alt"
- rsync -av --ignore-existing --exclude info \
- "$alt" "$GIT_OBJECT_DIRECTORY/" || exit
- done
- rm -f "$GIT_DIR/TMP_ALT"
- fi
- rsync_slurped_objects=t
- }
- ;;
- esac
-
- append_fetch_head "$head" "$remote" \
- "$remote_name" "$remote_nick" "$local_name" "$not_for_merge" || exit
-
- done
-
-}
-
-fetch_main () {
- case "$remote" in
- http://* | https://* | ftp://* | rsync://* )
- fetch_per_ref "$@"
- ;;
- *)
- fetch_all_at_once "$@"
- ;;
- esac
-}
-
-fetch_main "$reflist" || exit
-
-# automated tag following
-case "$no_tags$tags" in
-'')
- case "$reflist" in
- *:refs/*)
- # effective only when we are following remote branch
- # using local tracking branch.
- taglist=$(IFS=' ' &&
- echo "$ls_remote_result" |
- git-show-ref --exclude-existing=refs/tags/ |
- while read sha1 name
- do
- git-cat-file -t "$sha1" >/dev/null 2>&1 || continue
- echo >&2 "Auto-following $name"
- echo ".${name}:${name}"
- done)
- esac
- case "$taglist" in
- '') ;;
- ?*)
- # do not deepen a shallow tree when following tags
- shallow_depth=
- fetch_main "$taglist" || exit ;;
- esac
-esac
-
-# If the original head was empty (i.e. no "master" yet), or
-# if we were told not to worry, we do not have to check.
-case "$orig_head" in
-'')
- ;;
-?*)
- curr_head=$(git-rev-parse --verify HEAD 2>/dev/null)
- if test "$curr_head" != "$orig_head"
- then
- git-update-ref \
- -m "$GIT_REFLOG_ACTION: Undoing incorrectly fetched HEAD." \
- HEAD "$orig_head"
- die "Cannot fetch into the current branch."
- fi
- ;;
-esac
diff --git a/git.c b/git.c
index 71fa9c7..7ae9230 100644
--- a/git.c
+++ b/git.c
@@ -307,6 +307,7 @@ static void handle_internal_command(int argc, const char **argv)
{ "diff-files", cmd_diff_files },
{ "diff-index", cmd_diff_index, RUN_SETUP },
{ "diff-tree", cmd_diff_tree, RUN_SETUP },
+ { "fetch", cmd_fetch, RUN_SETUP },
{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
{ "fetch--tool", cmd_fetch__tool, RUN_SETUP },
{ "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP },
diff --git a/t/t5515/fetch.br-branches-default-merge b/t/t5515/fetch.br-branches-default-merge
index ea65f31..828bfd8 100644
--- a/t/t5515/fetch.br-branches-default-merge
+++ b/t/t5515/fetch.br-branches-default-merge
@@ -1,5 +1,5 @@
# br-branches-default-merge
-754b754407bf032e9a2f9d5a9ad05ca79a6b228f branch 'master' of ../
+754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-branches-default-merge_branches-default b/t/t5515/fetch.br-branches-default-merge_branches-default
index 7b5fa94..f148673 100644
--- a/t/t5515/fetch.br-branches-default-merge_branches-default
+++ b/t/t5515/fetch.br-branches-default-merge_branches-default
@@ -1,5 +1,5 @@
# br-branches-default-merge branches-default
-754b754407bf032e9a2f9d5a9ad05ca79a6b228f branch 'master' of ../
+754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-branches-default-octopus b/t/t5515/fetch.br-branches-default-octopus
index 128397d..bb1a191 100644
--- a/t/t5515/fetch.br-branches-default-octopus
+++ b/t/t5515/fetch.br-branches-default-octopus
@@ -1,5 +1,5 @@
# br-branches-default-octopus
-754b754407bf032e9a2f9d5a9ad05ca79a6b228f branch 'master' of ../
+754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-branches-default-octopus_branches-default b/t/t5515/fetch.br-branches-default-octopus_branches-default
index 4b37cd4..970fc93 100644
--- a/t/t5515/fetch.br-branches-default-octopus_branches-default
+++ b/t/t5515/fetch.br-branches-default-octopus_branches-default
@@ -1,5 +1,5 @@
# br-branches-default-octopus branches-default
-754b754407bf032e9a2f9d5a9ad05ca79a6b228f branch 'master' of ../
+754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-branches-one-merge b/t/t5515/fetch.br-branches-one-merge
index 3a4e77e..24099fd 100644
--- a/t/t5515/fetch.br-branches-one-merge
+++ b/t/t5515/fetch.br-branches-one-merge
@@ -1,5 +1,5 @@
# br-branches-one-merge
-8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
+8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge branch 'one' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-branches-one-merge_branches-one b/t/t5515/fetch.br-branches-one-merge_branches-one
index 00e04b4..e4b4fde 100644
--- a/t/t5515/fetch.br-branches-one-merge_branches-one
+++ b/t/t5515/fetch.br-branches-one-merge_branches-one
@@ -1,5 +1,5 @@
# br-branches-one-merge branches-one
-8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
+8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge branch 'one' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-config-glob-octopus b/t/t5515/fetch.br-config-glob-octopus
index 9ee213e..938e532 100644
--- a/t/t5515/fetch.br-config-glob-octopus
+++ b/t/t5515/fetch.br-config-glob-octopus
@@ -2,7 +2,7 @@
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
0567da4d5edd2ff4bb292a465ba9e64dcad9536b not-for-merge branch 'three' of ../
-6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 not-for-merge branch 'two' of ../
+6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 branch 'two' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-config-glob-octopus_config-glob b/t/t5515/fetch.br-config-glob-octopus_config-glob
index 44bd0ec..c9225bf 100644
--- a/t/t5515/fetch.br-config-glob-octopus_config-glob
+++ b/t/t5515/fetch.br-config-glob-octopus_config-glob
@@ -2,7 +2,7 @@
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
0567da4d5edd2ff4bb292a465ba9e64dcad9536b not-for-merge branch 'three' of ../
-6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 not-for-merge branch 'two' of ../
+6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 branch 'two' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-remote-glob-octopus b/t/t5515/fetch.br-remote-glob-octopus
index c1554f8..b08e046 100644
--- a/t/t5515/fetch.br-remote-glob-octopus
+++ b/t/t5515/fetch.br-remote-glob-octopus
@@ -2,7 +2,7 @@
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
0567da4d5edd2ff4bb292a465ba9e64dcad9536b not-for-merge branch 'three' of ../
-6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 not-for-merge branch 'two' of ../
+6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 branch 'two' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
diff --git a/t/t5515/fetch.br-remote-glob-octopus_remote-glob b/t/t5515/fetch.br-remote-glob-octopus_remote-glob
index e613434..d4d547c 100644
--- a/t/t5515/fetch.br-remote-glob-octopus_remote-glob
+++ b/t/t5515/fetch.br-remote-glob-octopus_remote-glob
@@ -2,7 +2,7 @@
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge branch 'master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 branch 'one' of ../
0567da4d5edd2ff4bb292a465ba9e64dcad9536b not-for-merge branch 'three' of ../
-6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 not-for-merge branch 'two' of ../
+6134ee8f857693b96ff1cc98d3e2fd62b199e5a8 branch 'two' of ../
754b754407bf032e9a2f9d5a9ad05ca79a6b228f not-for-merge tag 'tag-master' of ../
8e32a6d901327a23ef831511badce7bf3bf46689 not-for-merge tag 'tag-one' of ../
22feea448b023a2d864ef94b013735af34d238ba not-for-merge tag 'tag-one-tree' of ../
--
1.5.2.2.1600.ga4e5
^ permalink raw reply related
* [PATCH 3/4] Add fetch methods to transport library.
From: Daniel Barkalow @ 2007-07-12 1:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Johannes Schindelin
These don't include bundle or rsync. It would be great if people would
write at least bundle; rsync may well be deprecatable at this point (since
people are more likely to have http than anonymous rsync and more likely
to have ssh+git than non-anonymous rsync).
In any case, it identifies such URLs and fails cleanly on them.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
transport.c | 286 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
transport.h | 23 +++++-
2 files changed, 305 insertions(+), 4 deletions(-)
diff --git a/transport.c b/transport.c
index ce03133..487d315 100644
--- a/transport.c
+++ b/transport.c
@@ -1,6 +1,39 @@
#include "cache.h"
#include "transport.h"
#include "run-command.h"
+#include "http.h"
+#include "pkt-line.h"
+#include "fetch-pack.h"
+#include "walker.h"
+
+/* Generic functions for using commit walkers */
+
+static int fetch_objs_via_walker(const struct transport *transport,
+ int nr_objs, char **objs)
+{
+ char *dest = xstrdup(transport->url);
+ struct walker *walker = transport->data;
+
+ walker->get_all = 1;
+ walker->get_tree = 1;
+ walker->get_history = 1;
+ walker->get_verbosely = transport->verbose;
+ walker->get_recover = 0;
+
+ if (walker_fetch(walker, nr_objs, objs, NULL, dest))
+ die("Fetch failed.");
+
+ free(dest);
+ return 0;
+}
+
+static int disconnect_walker(struct transport *transport)
+{
+ struct walker *walker = transport->data;
+ if (walker)
+ walker_free(walker);
+ return 0;
+}
static const struct transport_ops rsync_transport = {
};
@@ -37,9 +70,119 @@ static int curl_transport_push(struct transport *transport, int refspec_nr, cons
return !!err;
}
+#ifndef NO_CURL
+static int missing__target(int code, int result)
+{
+ return /* file:// URL -- do we ever use one??? */
+ (result == CURLE_FILE_COULDNT_READ_FILE) ||
+ /* http:// and https:// URL */
+ (code == 404 && result == CURLE_HTTP_RETURNED_ERROR) ||
+ /* ftp:// URL */
+ (code == 550 && result == CURLE_FTP_COULDNT_RETR_FILE)
+ ;
+}
+
+#define missing_target(a) missing__target((a)->http_code, (a)->curl_result)
+
+static struct ref *get_refs_via_curl(const struct transport *transport)
+{
+ struct buffer buffer;
+ char *data, *start, *mid;
+ char *ref_name;
+ char *refs_url;
+ int i = 0;
+
+ struct active_request_slot *slot;
+ struct slot_results results;
+
+ struct ref *refs = NULL;
+ struct ref *ref = NULL;
+ struct ref *last_ref = NULL;
+
+ data = xmalloc(4096);
+ buffer.size = 4096;
+ buffer.posn = 0;
+ buffer.buffer = data;
+
+ refs_url = xmalloc(strlen(transport->url) + 11);
+ sprintf(refs_url, "%s/info/refs", transport->url);
+
+ http_init();
+
+ slot = get_active_slot();
+ slot->results = &results;
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, refs_url);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (results.curl_result != CURLE_OK) {
+ if (missing_target(&results)) {
+ free(buffer.buffer);
+ return NULL;
+ } else {
+ free(buffer.buffer);
+ error("%s", curl_errorstr);
+ return NULL;
+ }
+ }
+ } else {
+ free(buffer.buffer);
+ error("Unable to start request");
+ return NULL;
+ }
+
+ http_cleanup();
+
+ data = buffer.buffer;
+ start = NULL;
+ mid = data;
+ while (i < buffer.posn) {
+ if (!start)
+ start = &data[i];
+ if (data[i] == '\t')
+ mid = &data[i];
+ if (data[i] == '\n') {
+ data[i] = 0;
+ ref_name = mid + 1;
+ ref = xmalloc(sizeof(struct ref) +
+ strlen(ref_name) + 1);
+ memset(ref, 0, sizeof(struct ref));
+ strcpy(ref->name, ref_name);
+ get_sha1_hex(start, ref->old_sha1);
+ if (!refs)
+ refs = ref;
+ if (last_ref)
+ last_ref->next = ref;
+ last_ref = ref;
+ start = NULL;
+ }
+ i++;
+ }
+
+ free(buffer.buffer);
+
+ return refs;
+}
+
+#else
+
+static struct ref *get_refs_via_curl(const struct transport *transport)
+{
+ die("Cannot fetch from '%s' without curl ...", transport->url);
+ return NULL;
+}
+
+#endif
+
static const struct transport_ops curl_transport = {
/* set_option */ NULL,
- /* push */ curl_transport_push
+ /* get_refs_list */ get_refs_via_curl,
+ /* fetch_refs */ NULL,
+ /* fetch_objs */ fetch_objs_via_walker,
+ /* push */ curl_transport_push,
+ /* disconnect */ disconnect_walker
};
static const struct transport_ops bundle_transport = {
@@ -47,7 +190,13 @@ static const struct transport_ops bundle_transport = {
struct git_transport_data {
unsigned thin : 1;
+ unsigned keep : 1;
+ int unpacklimit;
+
+ int depth;
+
+ const char *uploadpack;
const char *receivepack;
};
@@ -55,16 +204,86 @@ static int set_git_option(struct transport *connection,
const char *name, const char *value)
{
struct git_transport_data *data = connection->data;
- if (!strcmp(name, TRANS_OPT_RECEIVEPACK)) {
+ if (!strcmp(name, TRANS_OPT_UPLOADPACK)) {
+ data->uploadpack = value;
+ return 0;
+ } else if (!strcmp(name, TRANS_OPT_RECEIVEPACK)) {
data->receivepack = value;
return 0;
} else if (!strcmp(name, TRANS_OPT_THIN)) {
data->thin = !!value;
return 0;
+ } else if (!strcmp(name, TRANS_OPT_KEEP)) {
+ data->keep = !!value;
+ return 0;
+ } else if (!strcmp(name, TRANS_OPT_UNPACKLIMIT)) {
+ data->unpacklimit = atoi(value);
+ return 0;
+ } else if (!strcmp(name, TRANS_OPT_DEPTH)) {
+ if (!value)
+ data->depth = 0;
+ else
+ data->depth = atoi(value);
+ return 0;
}
return 1;
}
+static struct ref *get_refs_via_connect(const struct transport *transport)
+{
+ struct git_transport_data *data = transport->data;
+ struct ref *refs;
+ int fd[2];
+ pid_t pid;
+ char *dest = xstrdup(transport->url);
+
+ pid = git_connect(fd, dest, data->uploadpack, 0);
+
+ if (pid < 0)
+ die("Failed to connect to \"%s\"", transport->url);
+
+ get_remote_heads(fd[0], &refs, 0, NULL, 0);
+ packet_flush(fd[1]);
+
+ finish_connect(pid);
+
+ free(dest);
+
+ return refs;
+}
+
+static int fetch_refs_via_pack(const struct transport *transport,
+ int nr_heads, char **heads)
+{
+ struct git_transport_data *data = transport->data;
+ struct ref *refs;
+ char *dest = xstrdup(transport->url);
+ struct fetch_pack_args args;
+
+ args.uploadpack = data->uploadpack;
+ args.quiet = 0;
+ args.keep_pack = data->keep;
+ args.unpacklimit = data->unpacklimit;
+ args.use_thin_pack = data->thin;
+ args.fetch_all = 0;
+ args.verbose = transport->verbose;
+ args.depth = data->depth;
+ args.no_progress = 0;
+
+ setup_fetch_pack(&args);
+
+ refs = fetch_pack(dest, nr_heads, heads);
+
+ // ???? check that refs got everything?
+
+ /* free the memory used for the refs list ... */
+
+ free_refs(refs);
+
+ free(dest);
+ return 0;
+}
+
static int git_transport_push(struct transport *transport, int refspec_nr, const char **refspec, int flags) {
struct git_transport_data *data = transport->data;
const char **argv;
@@ -111,6 +330,9 @@ static int git_transport_push(struct transport *transport, int refspec_nr, const
static const struct transport_ops git_transport = {
/* set_option */ set_git_option,
+ /* get_refs_list */ get_refs_via_connect,
+ /* fetch_refs */ fetch_refs_via_pack,
+ /* fetch_objs */ NULL,
/* push */ git_transport_push
};
@@ -141,7 +363,10 @@ struct transport *transport_get(struct remote *remote, const char *url,
!prefixcmp(url, "ftp://")) {
ret = xmalloc(sizeof(*ret));
ret->ops = &curl_transport;
- ret->data = NULL;
+ if (fetch)
+ ret->data = get_http_walker(url);
+ else
+ ret->data = NULL;
} else if (is_local(url) && is_file(url)) {
ret = xmalloc(sizeof(*ret));
ret->data = NULL;
@@ -151,14 +376,19 @@ struct transport *transport_get(struct remote *remote, const char *url,
ret = xcalloc(1, sizeof(*ret));
ret->data = data;
data->thin = 1;
+ data->uploadpack = "git-upload-pack";
+ if (remote->uploadpack)
+ data->uploadpack = remote->uploadpack;
data->receivepack = "git-receive-pack";
if (remote->receivepack)
data->receivepack = remote->receivepack;
+ data->unpacklimit = -1;
ret->ops = &git_transport;
}
if (ret) {
ret->remote = remote;
ret->url = url;
+ ret->remote_refs = NULL;
ret->fetch = !!fetch;
}
return ret;
@@ -187,6 +417,54 @@ int transport_push(struct transport *transport,
return transport->ops->push(transport, refspec_nr, refspec, flags);
}
+struct ref *transport_get_remote_refs(struct transport *transport)
+{
+ if (!transport->remote_refs)
+ transport->remote_refs =
+ transport->ops->get_refs_list(transport);
+ return transport->remote_refs;
+}
+
+#define PACK_HEADS_CHUNK_COUNT 256
+
+int transport_fetch_refs(struct transport *transport, struct ref *refs)
+{
+ int i;
+ int nr_heads = 0;
+ char **heads = xmalloc(PACK_HEADS_CHUNK_COUNT * sizeof(char *));
+ struct ref *rm;
+ int use_objs = !transport->ops->fetch_refs;
+
+ for (rm = refs; rm; rm = rm->next) {
+ if (rm->peer_ref &&
+ !hashcmp(rm->peer_ref->old_sha1, rm->old_sha1))
+ continue;
+ if (use_objs) {
+ heads[nr_heads++] = xstrdup(sha1_to_hex(rm->old_sha1));
+ } else {
+ heads[nr_heads++] = xstrdup(rm->name);
+ }
+ if (nr_heads % PACK_HEADS_CHUNK_COUNT == 0)
+ heads = xrealloc(heads,
+ (nr_heads + PACK_HEADS_CHUNK_COUNT) *
+ sizeof(char *));
+ }
+
+ if (use_objs) {
+ if (transport->ops->fetch_objs(transport, nr_heads, heads))
+ return -1;
+ } else {
+ if (transport->ops->fetch_refs(transport, nr_heads, heads))
+ return -1;
+ }
+
+ /* free the memory used for the heads list ... */
+ for (i = 0; i < nr_heads; i++)
+ free(heads[i]);
+ free(heads);
+ return 0;
+}
+
int transport_disconnect(struct transport *transport)
{
int ret = 0;
@@ -195,3 +473,5 @@ int transport_disconnect(struct transport *transport)
free(transport);
return ret;
}
+
+
diff --git a/transport.h b/transport.h
index 5c2eb95..947db66 100644
--- a/transport.h
+++ b/transport.h
@@ -29,6 +29,11 @@ struct transport_ops {
int (*set_option)(struct transport *connection, const char *name,
const char *value);
+ struct ref *(*get_refs_list)(const struct transport *transport);
+ int (*fetch_refs)(const struct transport *transport,
+ int nr_refs, char **refs);
+ int (*fetch_objs)(const struct transport *transport,
+ int nr_objs, char **objs);
int (*push)(struct transport *connection, int refspec_nr, const char **refspec, int flags);
int (*disconnect)(struct transport *connection);
@@ -40,12 +45,24 @@ struct transport *transport_get(struct remote *remote, const char *url,
/* Transport options which apply to git:// and scp-style URLs */
+/* The program to use on the remote side to send a pack */
+#define TRANS_OPT_UPLOADPACK "uploadpack"
+
/* The program to use on the remote side to receive a pack */
#define TRANS_OPT_RECEIVEPACK "receivepack"
/* Transfer the data as a thin pack if not null */
#define TRANS_OPT_THIN "thin"
+/* Keep the pack that was transferred if not null */
+#define TRANS_OPT_KEEP "keep"
+
+/* Unpack the objects if fewer than this number of objects are fetched */
+#define TRANS_OPT_UNPACKLIMIT "unpacklimit"
+
+/* Limit the depth of the fetch if not null */
+#define TRANS_OPT_DEPTH "depth"
+
/**
* Returns 0 if the option was used, non-zero otherwise. Prints a
* message to stderr if the option is not used.
@@ -53,9 +70,13 @@ struct transport *transport_get(struct remote *remote, const char *url,
int transport_set_option(struct transport *transport, const char *name,
const char *value);
-int transport_push(struct transport *connection,
+int transport_push(struct transport *connection,
int refspec_nr, const char **refspec, int flags);
+struct ref *transport_get_remote_refs(struct transport *transport);
+
+int transport_fetch_refs(struct transport *transport, struct ref *refs);
+
int transport_disconnect(struct transport *transport);
#endif
--
1.5.2.2.1600.ga4e5
^ permalink raw reply related
* [PATCH 2/4] Add matching and parsing for fetch-side refspec rules
From: Daniel Barkalow @ 2007-07-12 1:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Also exports parse_ref_spec() and reads the "tagopt" config option.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
remote.c | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
remote.h | 24 ++++++++++++
2 files changed, 148 insertions(+), 1 deletions(-)
diff --git a/remote.c b/remote.c
index fa4af05..bba44c3 100644
--- a/remote.c
+++ b/remote.c
@@ -208,6 +208,7 @@ static void read_branches_file(struct remote *remote)
}
add_uri(remote, p);
add_fetch_refspec(remote, branch);
+ remote->fetch_tags = 1; /* always auto-follow */
}
static int handle_config(const char *key, const char *value)
@@ -274,6 +275,9 @@ static int handle_config(const char *key, const char *value)
remote->uploadpack = xstrdup(value);
else
error("more than one uploadpack given, using the first");
+ } else if (!strcmp(subkey, ".tagopt")) {
+ if (!strcmp(value, "--no-tags"))
+ remote->fetch_tags = -1;
}
return 0;
}
@@ -296,7 +300,7 @@ static void read_config(void)
git_config(handle_config);
}
-static struct refspec *parse_ref_spec(int nr_refspec, const char **refspec)
+struct refspec *parse_ref_spec(int nr_refspec, const char **refspec)
{
int i;
struct refspec *rs = xcalloc(sizeof(*rs), nr_refspec);
@@ -352,6 +356,10 @@ struct remote *remote_get(const char *name)
add_uri(ret, name);
if (!ret->uri)
return NULL;
+ if (!strcmp(name, ".")) {
+ // we always fetch "refs/*:refs/*", which is trivial
+ add_fetch_refspec(ret, "refs/*:refs/*");
+ }
ret->fetch = parse_ref_spec(ret->fetch_refspec_nr, ret->fetch_refspec);
ret->push = parse_ref_spec(ret->push_refspec_nr, ret->push_refspec);
return ret;
@@ -424,6 +432,14 @@ struct ref *alloc_ref(unsigned namelen)
return ret;
}
+static struct ref *copy_ref(struct ref *ref)
+{
+ struct ref *ret = xmalloc(sizeof(struct ref) + strlen(ref->name) + 1);
+ memcpy(ret, ref, sizeof(struct ref) + strlen(ref->name) + 1);
+ ret->next = NULL;
+ return ret;
+}
+
void free_refs(struct ref *ref)
{
struct ref *next;
@@ -734,3 +750,110 @@ int branch_merges(struct branch *branch, const char *refname)
}
return 0;
}
+
+static struct ref *get_expanded_map(struct ref *remote_refs,
+ const struct refspec *refspec)
+{
+ struct ref *ref;
+ struct ref *ret = NULL;
+ struct ref **tail = &ret;
+
+ int remote_prefix_len = strlen(refspec->src);
+ int local_prefix_len = strlen(refspec->dst);
+
+ for (ref = remote_refs; ref; ref = ref->next) {
+ if (strchr(ref->name, '^'))
+ continue; /* a dereference item */
+ if (!prefixcmp(ref->name, refspec->src)) {
+ char *match;
+ struct ref *cpy = copy_ref(ref);
+ match = ref->name + remote_prefix_len;
+
+ cpy->peer_ref = alloc_ref(local_prefix_len +
+ strlen(match) + 1);
+ sprintf(cpy->peer_ref->name, "%s%s",
+ refspec->dst, match);
+ if (refspec->force)
+ cpy->peer_ref->force = 1;
+ *tail = cpy;
+ tail = &cpy->next;
+ }
+ }
+
+ return ret;
+}
+
+static struct ref *find_ref_by_name_abbrev(struct ref *refs, const char *name)
+{
+ struct ref *ref;
+ for (ref = refs; ref; ref = ref->next) {
+ if (ref_matches_abbrev(name, ref->name))
+ return ref;
+ }
+ return NULL;
+}
+
+struct ref *get_remote_ref(struct ref *remote_refs, const char *name)
+{
+ struct ref *ref = find_ref_by_name_abbrev(remote_refs, name);
+
+ if (!ref)
+ die("Couldn't find remote ref %s\n", name);
+
+ return copy_ref(ref);
+}
+
+static struct ref *get_local_ref(const char *name)
+{
+ struct ref *ret;
+ if (!name)
+ return NULL;
+
+ if (!prefixcmp(name, "refs/")) {
+ ret = alloc_ref(strlen(name) + 1);
+ strcpy(ret->name, name);
+ return ret;
+ }
+
+ if (!prefixcmp(name, "heads/") ||
+ !prefixcmp(name, "tags/") ||
+ !prefixcmp(name, "remotes/")) {
+ ret = alloc_ref(strlen(name) + 6);
+ sprintf(ret->name, "refs/%s", name);
+ return ret;
+ }
+
+ ret = alloc_ref(strlen(name) + 12);
+ sprintf(ret->name, "refs/heads/%s", name);
+ return ret;
+}
+
+int get_fetch_map(struct ref *remote_refs,
+ const struct refspec *refspec,
+ struct ref ***tail)
+{
+ struct ref *ref_map, *rm;
+
+ if (refspec->pattern) {
+ ref_map = get_expanded_map(remote_refs, refspec);
+ } else {
+ ref_map = get_remote_ref(remote_refs,
+ refspec->src[0] ?
+ refspec->src : "HEAD");
+
+ ref_map->peer_ref = get_local_ref(refspec->dst);
+
+ if (refspec->force)
+ ref_map->peer_ref->force = 1;
+ }
+
+ for (rm = ref_map; rm; rm = rm->next) {
+ if (rm->peer_ref && check_ref_format(rm->peer_ref->name + 5))
+ die("* refusing to create funny ref '%s' locally",
+ rm->peer_ref->name);
+ }
+
+ tail_link_ref(ref_map, tail);
+
+ return 0;
+}
diff --git a/remote.h b/remote.h
index 56e0ba6..2a5e87e 100644
--- a/remote.h
+++ b/remote.h
@@ -15,6 +15,14 @@ struct remote {
struct refspec *fetch;
int fetch_refspec_nr;
+ /*
+ * -1 to never fetch tags
+ * 0 to auto-follow tags on heuristic (default)
+ * 1 to always auto-follow tags
+ * 2 to always fetch tags
+ */
+ int fetch_tags;
+
const char *receivepack;
const char *uploadpack;
};
@@ -38,10 +46,26 @@ struct ref *alloc_ref(unsigned namelen);
*/
void free_refs(struct ref *ref);
+struct refspec *parse_ref_spec(int nr_refspec, const char **refspec);
+
int match_refs(struct ref *src, struct ref *dst, struct ref ***dst_tail,
int nr_refspec, char **refspec, int all);
/*
+ * Given a list of the remote refs and the specification of things to
+ * fetch, makes a (separate) list of the refs to fetch and the local
+ * refs to store into.
+ *
+ * *tail is the pointer to the tail pointer of the list of results
+ * beforehand, and will be set to the tail pointer of the list of
+ * results afterward.
+ */
+int get_fetch_map(struct ref *remote_refs, const struct refspec *refspec,
+ struct ref ***tail);
+
+struct ref *get_remote_ref(struct ref *remote_refs, const char *name);
+
+/*
* For the given remote, reads the refspec's src and sets the other fields.
*/
int remote_find_tracking(struct remote *remote, struct refspec *refspec);
--
1.5.2.2.1600.ga4e5
^ permalink raw reply related
* [PATCH 1/4] Add uploadpack configuration info to remote.
From: Daniel Barkalow @ 2007-07-12 1:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Read remote.<name>.uploadpack, just like receivepack.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
remote.c | 5 +++++
remote.h | 1 +
2 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/remote.c b/remote.c
index 500ca4d..793681a 100644
--- a/remote.c
+++ b/remote.c
@@ -196,6 +196,11 @@ static int handle_config(const char *key, const char *value)
remote->receivepack = xstrdup(value);
else
error("more than one receivepack given, using the first");
+ } else if (!strcmp(subkey, ".uploadpack")) {
+ if (!remote->uploadpack)
+ remote->uploadpack = xstrdup(value);
+ else
+ error("more than one uploadpack given, using the first");
}
return 0;
}
diff --git a/remote.h b/remote.h
index 01dbcef..f50105c 100644
--- a/remote.h
+++ b/remote.h
@@ -16,6 +16,7 @@ struct remote {
int fetch_refspec_nr;
const char *receivepack;
+ const char *uploadpack;
};
struct remote *remote_get(const char *name);
--
1.5.2.2.1600.ga4e5
^ permalink raw reply related
* [PATCH 0/4] The rest of builtin-fetch
From: Daniel Barkalow @ 2007-07-12 1:36 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Johannes Schindelin
This series is the rest of builtin-fetch. It should probably not be used
without a "[PATCH 3.5/4] Support bundles in transport fetch" which
Johannes was going to write.
The first patch of this is independant of anything else, but the rest
depend on all of the other series I have sent recently. One thing I
already plan to fix while doing revisions for comments is that the
"remote.<name>.tagopt" config variable parsing is added by the second
patch, when it would be more logically added by the first.
When the development cycle is at the right point for getting all this in,
I'll be sending out the complete set of patches (aside from the pair that
were already accepted). I'm also going to send out another revision of
builtin-fetch-pack.c that adds a bit of memory freeing and depends on the
first of the accepted pair. And I've got a builtin-send-pack (with a
function for transport.c to call directly), although that's not in the
critical path.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: [PATCH] gitweb: snapshot cleanups & support for offering multiple formats
From: Matt McCutchen @ 2007-07-12 1:15 UTC (permalink / raw)
To: Junio C Hamano, Jakub Narebski; +Cc: git, Petr Baudis, Luben Tuikov
In-Reply-To: <7vir8q4opc.fsf@assigned-by-dhcp.cox.net>
On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > I'm not sure if we want to store whole 'application/x-gzip' or only
> > 'x-gzip' part of mime type, and if we want to store compressor as
> > '| gzip' or simply as 'gzip'.
Storing only 'x-gzip' assumes that all archive formats have MIME types
beginning with 'application/'. Even if this assumption is justified
by the MIME specification, I felt it was inappropriate to code it into
gitweb. The advantage of '| gzip' is that the lack of a compressor is
not a special case. This is why I wrote %known_snapshot_formats the
way I did, but of course you all are welcome to overrule me.
> > This would break not only existing _gitweb_ configuration (when
> > gitweb admin installs new gitweb it isn't that hard to correct
> > gitweb config), but also git _repositories_ config: gitweb.snapshot
> > no longer work as it worked before, for example neither 'gzip'
> > nor 'bzip2' values work anymore ('zip' doesn't stop working).
>
> I realized after seeing your other message on this patch that
> this can be done while retaining backward compatibility, as you
> suggested. Matt, does Jakub's suggestion make sense to you?
It's not clear to me what the suggestion is: offer format names 'gzip'
and 'bzip2' instead of 'tgz' and 'tbz2', or in addition to them, or
what? I prefer 'tgz' and 'tbz2' because they carry more information
and are properly analogous to 'zip', so I don't want to offer 'gzip'
and 'bzip2' instead of them. Furthermore, I would like the user to
see 'snapshot (tgz tbz2)' even if the repository owner wrote 'gzip
bzip2', so just adding two rows to %known_snapshot_formats is
insufficient. Either an additional column could be added to
%known_snapshot_formats for the display name, or 'gzip' and 'bzip2'
could be specified as aliases in %known_snapshot_formats and
feature_snapshot could be taught to resolve them. I would prefer the
second option; shall I implement it?
It would be possible to make the gitweb site configuration
backward-compatible too; here's one possible approach. On startup,
gitweb would check whether $feature{'snapshot'}{'default'} is a
three-element list that appears to be in the old format. If so, it
would save the settings in $known_snapshot_formats{'default'} and then
set $feature{'snapshot'}{'default'} = 'default' . This is a hack; is
it justified by the compatibility benefit?
Matt
^ permalink raw reply
* Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
From: Carlos Rica @ 2007-07-12 0:14 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Johannes Schindelin, Kristian Høgsberg
In-Reply-To: <7vd4yy1svw.fsf@assigned-by-dhcp.cox.net>
2007/7/12, Junio C Hamano <gitster@pobox.com>:
> Carlos Rica <jasampler@gmail.com> writes:
> > @@ -28,52 +26,67 @@ static int cleanup(char *line, int len)
> > * Remove empty lines from the beginning and end
> > * and also trailing spaces from every line.
> > *
> > + * Note that the buffer will not be null-terminated.
> > + *
>
> The name of the sentinel character '\0' is NUL, not null (which
> is a different word, used to call a pointer that points
> nowhere). The buffer will not be "NUL-terminated".
Thank you Junio, I will use it on the future.
> > int cmd_stripspace(int argc, const char **argv, const char *prefix)
> > {
> > - stripspace(stdin, stdout, 0);
> > + char *buffer;
> > + unsigned long size;
> > +
> > + size = 1024;
> > + buffer = xmalloc(size);
> > + if (read_pipe(0, &buffer, &size))
> > + die("could not read the input");
>
> The command used to be capable of streaming and filtering a few
> hundred gigabytes of text on a machine with small address space,
> as it operated one line at a time, but now it cannot as it has
> to hold everything in core before starting.
>
> I do not think we miss that loss of capability too much, but I
> wonder if we can be a bit more clever about it, perhaps feeding
> a chunk at a time. Not a very strong request, but just
> wondering if it is an easy change.
I did those changes because I was needing those tests that
I had written before in order to develop the function. After that,
we now can restore the previous function with file descriptors to
make it capable of filter a few hundred gigabytes of text, provided
that the text does not have long long lines on it.
Indeed, the implementation for composing a tag (header, cleaned
message and optional signature) in "builtin-tag.c", now pass it to
the function write_sha1_file as a buffer on memory, so it won't support
sizes bigger than memory available on the system. Messages should
not be so big, but I don't know how to limit those.
^ permalink raw reply
* Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
From: Junio C Hamano @ 2007-07-12 0:03 UTC (permalink / raw)
To: Carlos Rica
Cc: Bill Lear, Junio C Hamano, git, Johannes Schindelin,
Kristian Høgsberg
In-Reply-To: <1b46aba20707111641y615ff82dpf2da4afab2b17d58@mail.gmail.com>
"Carlos Rica" <jasampler@gmail.com> writes:
> Now the function cleanup is not removing spaces at all, but only
> counting where the line of text ends.
>
> Before, in the previous version, the function cleanup was taking a
> string as argument, and then it needed to modify that string. Now, it
> just returns the new length, "not counting" the spaces and newline at
> the end of the buffer passed. Its name and comments then could be
> different, but I didn't know which ones.
Ah, you are right. I misread the patch --- cleanup() does not
even touch the buffer.
^ permalink raw reply
* Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
From: Carlos Rica @ 2007-07-11 23:41 UTC (permalink / raw)
To: Bill Lear
Cc: Junio C Hamano, git, Johannes Schindelin, Kristian Høgsberg
In-Reply-To: <18069.26029.224024.66576@lisa.zopyra.com>
Now the function cleanup is not removing spaces at all, but only
counting where the line of text ends.
Before, in the previous version, the function cleanup was taking a
string as argument, and then it needed to modify that string. Now, it
just returns the new length, "not counting" the spaces and newline at
the end of the buffer passed. Its name and comments then could be
different, but I didn't know which ones.
^ permalink raw reply
* Re: [PATCH] git-merge: run commit hooks when making merge commits
From: Sam Vilain @ 2007-07-11 23:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vd4yy4opa.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> Sam Vilain <sam.vilain@catalyst.net.nz> writes:
>
>> git-merge.sh was not running the commit hooks, so run them in the two
>> places where we go to commit.
>>
>> Signed-off-by: Sam Vilain <sam.vilain@catalyst.net.nz>
>> ---
>> Not sure if it should call these or some specialist hooks, like
>> git-am does.
>
> I suspect some people have pre-commit scripts that have been
> meant to catch style errors for their own commits, and invoking
> that on merge would wreak havoc --- there is not much you can do
> if you want to get the work done by somebody else at that point.
> Introducing a new pre-merge-commit hook would probably be safer;
> if one wants to use the same check as one's pre-commit does, the
> new hook in the repository can exec $GIT_DIR/hooks/pre-commit.
>
> The commit-msg hook I have no clue what people usually use it
> for in the real world, but a merge commit message tends to be
> quite different from the message you would give to your own
> straight line commits, so custom reformatting rules people have
> in commit-msg hook may not apply to merge commit messages.
True. OTOH, if you commit with `git commit` after a merge which failed
or was called with --no-commit, then it will call the commit hook. So
those scripts would have to deal with that case anyway.
So, should `git commit` detect it is committing a merge and call the
merge-hooks, should we use the same hooks, or, should this be something
like hooks/*-automerge ?
Sam.
^ permalink raw reply
* Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
From: Bill Lear @ 2007-07-11 23:20 UTC (permalink / raw)
To: Junio C Hamano
Cc: Carlos Rica, git, Johannes Schindelin, Kristian Høgsberg
In-Reply-To: <7vd4yy1svw.fsf@assigned-by-dhcp.cox.net>
On Wednesday, July 11, 2007 at 15:24:03 (-0700) Junio C Hamano writes:
>Carlos Rica <jasampler@gmail.com> writes:
>...
>> + * Returns the length of a line removing trailing spaces.
>
>This did not parse well for me; perhaps a comma before
>"removing" would make it easier to read?
Or:
Returns the length of a line, after removing trailing spaces.
Bill
^ permalink raw reply
* Re: [PATCH] Support wholesale directory renames in fast-import
From: David Frech @ 2007-07-11 23:11 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20070711075744.GO4436@spearce.org>
On 7/11/07, Shawn O. Pearce <spearce@spearce.org> wrote:
> I'm tired. I just worked an 18 hour day. I need to go do it all
> over again in about 4 hours. So I'm going to head off to bed. But
> I did manage to implement this (I think). Its totally untested.
> But feel free to poke at it:
>
> git://repo.or.cz/git/fastimport.git copy-wip
>
> I'll write documentation and unit tests tomorrow. And fix any bugs,
> if any get identified.
Shawn, don't knock yourself out. ;-)
I think getting the semantics right (of the command set in
fast-import) is important, but it doesn't have to be right
*yesterday*. I won't need to point my command stream at fast-import
for at least another day or two. ;-) I have some subtle code to write
first that'll take me a bit to get right...
Thanks for your enthusiasm though!
Cheers,
- David
> --
> Shawn.
>
--
If I have not seen farther, it is because I have stood in the
footsteps of giants.
^ permalink raw reply
* [PATCH] gitweb: new cgi parameter: option
From: Miklos Vajna @ 2007-07-11 23:00 UTC (permalink / raw)
To: Jakub Narebski, gitster; +Cc: git
In-Reply-To: <f73hhc$uo1$1@sea.gmane.org>
Currently the only supported value is '--no-merges' for the 'rss', 'atom',
'log', 'shortlog' and 'history' actions, but it can be easily extended to allow
other parameters for other actions.
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
---
Na Wed, Jul 11, 2007 at 11:19:41PM +0200, Jakub Narebski <jnareb@gmail.com> pisal(a):
> Miklos Vajna wrote:
>
> > +((defined $filter and $filter == "nomerges") ? ("--no-merges") : ()),
>
> Shouldn't it be '$filter eq "nomerges"' instead?
Yes, that works too (I'm not a perl addict :) )
> Besides, I'd rather have generalized way to provide additional options
> to git commands, like '--no-merges' for RSS and Atom feeds, log, shortlog
> and history views, '-C' for commitdiff view, '--remove-empty' for history
> view for a file, perhaps even '-c' or '--cc' for commitdiff for merges
> instead of abusing 'hp' argument for that.
>
> But that doesn't mean that this patch should be not applied... it doesn't
> mean it should be applied neither ;-)
What about this one?
gitweb/gitweb.perl | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index dc609f4..f3530ba 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -383,6 +383,20 @@ if (defined $hash_base) {
}
}
+my %options = (
+ "--no-merges" => [('rss', 'atom', 'log', 'shortlog', 'history')],
+);
+
+our $option = $cgi->param('option');
+if (defined $option) {
+ if (not grep(/^$option$/, keys %options)) {
+ die_error(undef, "Invalid option parameter");
+ }
+ if (not grep(/^$action$/, @{$options{$option}})) {
+ die_error(undef, "Invalid option parameter for this action");
+ }
+}
+
our $hash_parent_base = $cgi->param('hpb');
if (defined $hash_parent_base) {
if (!validate_refname($hash_parent_base)) {
@@ -534,6 +548,7 @@ sub href(%) {
action => "a",
file_name => "f",
file_parent => "fp",
+ option => "option",
hash => "h",
hash_parent => "hp",
hash_base => "hb",
@@ -1770,6 +1785,7 @@ sub parse_commits {
($arg ? ($arg) : ()),
("--max-count=" . $maxcount),
("--skip=" . $skip),
+ ((defined $option) ? ($option) : ()),
$commit_id,
"--",
($filename ? ($filename) : ())
--
1.5.3.rc0.39.g46f7-dirty
^ permalink raw reply related
* Re: [PATCH 1/2] Function stripspace now gets a buffer instead file descriptors.
From: Junio C Hamano @ 2007-07-11 22:24 UTC (permalink / raw)
To: Carlos Rica; +Cc: git, Johannes Schindelin, Kristian Høgsberg
In-Reply-To: <4695267A.7080202@gmail.com>
Carlos Rica <jasampler@gmail.com> writes:
> diff --git a/builtin-stripspace.c b/builtin-stripspace.c
> index d8358e2..949b640 100644
> --- a/builtin-stripspace.c
> +++ b/builtin-stripspace.c
> @@ -2,12 +2,11 @@
> #include "cache.h"
>
> /*
> - * Remove trailing spaces from a line.
> + * Returns the length of a line removing trailing spaces.
This did not parse well for me; perhaps a comma before
"removing" would make it easier to read?
> @@ -28,52 +26,67 @@ static int cleanup(char *line, int len)
> * Remove empty lines from the beginning and end
> * and also trailing spaces from every line.
> *
> + * Note that the buffer will not be null-terminated.
> + *
The name of the sentinel character '\0' is NUL, not null (which
is a different word, used to call a pointer that points
nowhere). The buffer will not be "NUL-terminated".
> -void stripspace(FILE *in, FILE *out, int skip_comments)
> +size_t stripspace(char *buffer, size_t length, int skip_comments)
> {
> + newlen = cleanup(buffer + i, len);
>
> /* Not just an empty line? */
> + if (newlen) {
> + if (empties != -1)
> + buffer[j++] = '\n';
> if (empties > 0)
> + buffer[j++] = '\n';
> empties = 0;
> + memmove(buffer + j, buffer + i, newlen);
> continue;
> }
It somehow strikes me odd that, given this:
buffer j i
**********************texttext \n.......
you would first do this with cleanup():
buffer j i
**********************texttext\n.......
and then do this with this memmove():
buffer j i
*************texttext\n.......
Would it become simpler if cleanup() knew where the final text
goes (i.e. buffer+j)?
> int cmd_stripspace(int argc, const char **argv, const char *prefix)
> {
> - stripspace(stdin, stdout, 0);
> + char *buffer;
> + unsigned long size;
> +
> + size = 1024;
> + buffer = xmalloc(size);
> + if (read_pipe(0, &buffer, &size))
> + die("could not read the input");
The command used to be capable of streaming and filtering a few
hundred gigabytes of text on a machine with small address space,
as it operated one line at a time, but now it cannot as it has
to hold everything in core before starting.
I do not think we miss that loss of capability too much, but I
wonder if we can be a bit more clever about it, perhaps feeding
a chunk at a time. Not a very strong request, but just
wondering if it is an easy change.
^ permalink raw reply
* Re: [PATCH] Fix --cherry-pick with given paths
From: Junio C Hamano @ 2007-07-11 22:00 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Sam Vilain, git
In-Reply-To: <Pine.LNX.4.64.0707101449220.4047@racer.site>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > Sam Vilain wrote:
> > > I'm interested in extending the cherry analysis to allow
> > > specification of the diff options used when calculating diff
> > > IDs, presenting the calculated diff IDs for analysis by
> > > external tools, and even calculating and dealing with per-hunk
> > > diff IDs.
> >
> > On reflection I think so long as --cherry-pick works together
> > with a file selection, it would probably be enough.
> >
> > ie, if I have these commits:
> >
> > A---B
> > \
> > \
> > C
> >
> > B changes a file foo.c, adding a line of text. C changes foo.c
> > as well as bar.c, but the change in foo.c was identical to
> > change B.
> >
> > I want this to show me nothing:
> >
> > git-rev-list --left-right --cherry-pick B...C -- foo.c
Thanks, both.
^ permalink raw reply
* Re: test suite fails if sh != bash || tar != GNU tar
From: David Frech @ 2007-07-11 21:57 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, Linus Torvalds, git
In-Reply-To: <7v7ip639cb.fsf@assigned-by-dhcp.cox.net>
On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> "David Frech" <david@nimblemachines.com> writes:
>
> > One issue is that my server is on dynamic IP, and my lame ISP (the
> > local telco) doesn't give me a proper SMTP relay - they want us to
> > send our mail via HTTP to MSN! Completely lame.
> >
> > So sending mail can be an issue, if the receiver blocks mail from dynamic IPs.
>
> I think I heard gmail has incoming SMTP for its subscribers, and
> I would not be surprised if other free e-mail providers have the
> same. Perhaps you can use one of them for this purpose?
Yes, I have managed to relay thru gmail. It was a bit hard to set up -
build Postfix with SASL, sniff the traffic to figure out who signed
gmail's SSL cert (Thawte), download CA cert - but I know how to do it.
;-)
The downside is, the gmail relay rewrites the headers in annoying
ways, so I'm not using it in general. If necessary I could relay just
the "build breakage" thru them.
Actually my incoming nimblemachines.com mail gets sent from my dynamic
IP thru normal (non-AUTH, non-SSL) SMTP to gmail, and they accept it
fine. So if I can send the build results to a gmail account, there are
no problems. Hmm - and that account could forward to the git list...
- David
--
If I have not seen farther, it is because I have stood in the
footsteps of giants.
^ permalink raw reply
* Re: test suite fails if sh != bash || tar != GNU tar
From: Johannes Schindelin @ 2007-07-11 21:56 UTC (permalink / raw)
To: David Frech; +Cc: Junio C Hamano, Linus Torvalds, git
In-Reply-To: <7154c5c60707111433r64ae5109o314778655cbc017e@mail.gmail.com>
Hi,
On Wed, 11 Jul 2007, David Frech wrote:
> On 7/11/07, Junio C Hamano <gitster@pobox.com> wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > >> I'll see what I can do. As I'm planning on running git on both
> > >> FreeBSD and DragonFly for the forseeable future, and plan to track
> > >> git's evolution (running stable releases if not more bleeding-edge
> > >> code), I can run the test suite every time I build a new git.
> > >
> > > If you want to, I can help you setting up a nightly cron job to
> > > fetch what is the current "next", run the tests, and report failures
> > > by email.
> >
> > Wow. Nightly builds of 'next' on various platforms would actually be
> > quite useful, especially from non Linux and non bash world.
>
> I found and fixed the shell issues. Once I've got a "fix" for tar I'll
> send a patch. I think the BSD sh has a bug wrt to negating the return
> code from a pipeline.
Cool! Please be sure to give Documentation/SubmittingPatches a quick
glance before sending, to avoid hassles for the reviewers.
> I'd be happy to do a nightly build on my DragonFly box, and that should
> catch anything that also doesn't work for FreeBSD. The failure modes
> were exactly the same - though the DFly box has an additional
> iconv-related problem (with git-mailinfo) that I still haven't tracked
> down...
>
> Is there a canned script to get me started?
Well, I would have started from scratch, as in
5 0 * * * (cd /xx/git && sh test-it.sh)
And in test-it.sh there could be something like
(git pull origin next && make test > test-it.out 2>&1 ) ||
some-script-that-sends-the-email.sh
> One issue is that my server is on dynamic IP, and my lame ISP (the local
> telco) doesn't give me a proper SMTP relay - they want us to send our
> mail via HTTP to MSN! Completely lame.
It is lame.
> So sending mail can be an issue, if the receiver blocks mail from
> dynamic IPs.
But maybe you can just upload the status somewhere public? Or ssh into a
machine which allows you to send an email, with public key authentication?
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Shoddy pack information tool
From: Junio C Hamano @ 2007-07-11 21:55 UTC (permalink / raw)
To: Brian Downing; +Cc: Nicolas Pitre, git
In-Reply-To: <20070709193017.GN4087@lavos.net>
I do not think this is particularlly Shoddy. Care to move it
somewhere in contrib/ (say, contrib/stats/packinfo.pl) and sign
off the patch?
^ permalink raw reply
* Re: [PATCH] git-cvsserver: detect/diagnose write failure, etc.
From: Junio C Hamano @ 2007-07-11 21:52 UTC (permalink / raw)
To: Jim Meyering; +Cc: git, Frank Lichtenheld
In-Reply-To: <87vecx4tel.fsf@rho.meyering.net>
Jim Meyering <jim@meyering.net> writes:
> Beware: in some contexts (when running as server), one must
> not "die", but rather return an "error" indicator to the client.
> I did that in the second hunk. However, I haven't carefully
> audited the others, and so, some of the "die" calls I added may
> end up killing the server, when a gentler failure is required.
>
> I've just looked, and can confirm that my change to req_Modified
> (first hunk) is wrong. It should not die. Rather, it should probably
> do something like the "print "E ... in hunk#2. Ideally, someone
> would write a test to exercise code like this, to make sure it works.
>
> Jim
> diff --git a/git-cvsserver.perl b/git-cvsserver.perl
> index 5cbf27e..8d8d6f5 100755
> --- a/git-cvsserver.perl
> +++ b/git-cvsserver.perl
> @@ -644,7 +644,7 @@ sub req_Modified
> $bytesleft -= $blocksize;
> }
>
> - close $fh;
> + close $fh or die "Failed to write temporary, $filename: $!";
True; dying is not appropriate here.
> @@ -901,8 +901,13 @@ sub req_update
> # projects (heads in this case) to checkout.
> #
> if ($state->{module} eq '') {
> + my $heads_dir = $state->{CVSROOT} . '/refs/heads';
> + if (!opendir HEADS, $heads_dir) {
> + print "E [server aborted]: Failed to open directory, $heads_dir: $!\n"
> + . "error\n";
> + return 0;
> + }
> print "E cvs update: Updating .\n";
> - opendir HEADS, $state->{CVSROOT} . '/refs/heads';
> while (my $head = readdir(HEADS)) {
> if (-f $state->{CVSROOT} . '/refs/heads/' . $head) {
> print "E cvs update: New directory `$head'\n";
Hmph. The clients get no entries in the listing either way, but
I would say this is an improvement.
> @@ -1754,7 +1759,9 @@ sub req_annotate
> # git-jsannotate telling us about commits we are hiding
> # from the client.
>
> - open(ANNOTATEHINTS, ">$tmpdir/.annotate_hints") or die "Error opening > $tmpdir/.annotate_hints $!";
> + my $a_hints = "$tmpdir/.annotate_hints";
> + open(ANNOTATEHINTS, '>', $a_hints)
> + or die "Failed to open '$a_hints' for writing: $!";
> for (my $i=0; $i < @$revisions; $i++)
> {
> print ANNOTATEHINTS $revisions->[$i][2];
Hmph, we seem to have:
$log->warn("Error in annotate output! LINE: $_");
print "E Annotate error \n";
which suggest me that throwing "E error" at the client and
returning might be better?
> @@ -1765,11 +1772,11 @@ sub req_annotate
> }
>
> print ANNOTATEHINTS "\n";
> - close ANNOTATEHINTS;
> + close ANNOTATEHINTS or die "Failed to write $a_hints: $!";
>
> - my $annotatecmd = 'git-annotate';
> - open(ANNOTATE, "-|", $annotatecmd, '-l', '-S', "$tmpdir/.annotate_hints", $filename)
> - or die "Error invoking $annotatecmd -l -S $tmpdir/.annotate_hints $filename : $!";
> + my @cmd = (qw(git-annotate -l -S), $a_hints, $filename);
> + open(ANNOTATE, "-|", @cmd)
> + or die "Error invoking ". join(' ',@cmd) .": $!";
> my $metadata = {};
> print "E Annotations for $filename\n";
> print "E ***************\n";
Likewise...
> @@ -1996,12 +2003,12 @@ sub transmitfile
> {
> open NEWFILE, ">", $targetfile or die("Couldn't open '$targetfile' for writing : $!");
> print NEWFILE $_ while ( <$fh> );
> - close NEWFILE;
> + close NEWFILE or die("Failed to write '$targetfile': $!");
> } else {
> print "$size\n";
> print while ( <$fh> );
> }
> - close $fh or die ("Couldn't close filehandle for transmitfile()");
> + close $fh or die ("Couldn't close filehandle for transmitfile(): $!");
> } else {
> die("Couldn't execute git-cat-file");
> }
Ok.
> @@ -2501,17 +2508,14 @@ sub update
> if ($parent eq $lastpicked) {
> next;
> }
> - open my $p, 'git-merge-base '. $lastpicked . ' '
> - . $parent . '|';
> - my @output = (<$p>);
> - close $p;
> - my $base = join('', @output);
> + my $base = safe_pipe_capture('git-merge-base',
> + $lastpicked, $parent);
> chomp $base;
> if ($base) {
> my @merged;
> # print "want to log between $base $parent \n";
> open(GITLOG, '-|', 'git-log', "$base..$parent")
> - or die "Cannot call git-log: $!";
> + or die "Cannot call git-log: $!";
> my $mergedhash;
> while (<GITLOG>) {
> chomp;
Ok.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox