* Re: WARNING: THIS PATCH CAN BREAK YOUR REPO, was Re: [PATCH 2/3] Only repack active packs by skipping over kept packs.
From: Jan Harkes @ 2006-10-30 20:52 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Nicolas Pitre, Junio C Hamano, git
In-Reply-To: <20061030202611.GA5775@spearce.org>
On Mon, Oct 30, 2006 at 03:26:11PM -0500, Shawn Pearce wrote:
> Actually the breakage is easier to reproduce without trashing
> a repository.
>
> Do the above so you have everything in one pack. Now use rev-list
> to simulate the object list construction in pack-objects as though
> we were doing a 'git repack -a -d':
>
> git-rev-list --objects --all \
> --unpacked=.git/objects/pack/pack-*.pack \
> | wc -l
>
> gives me 102 (WRONG WRONG WRONG WRONG!!!!!!)
The problem seems to be that as soon as we hit something that is found
in a pack that is not on the ignore list, that object and all it's
parents are marked as uninteresting. So if the kept pack contains a
slice of commits (v1.4.3..v1.4.3.3) the revision walker will only return
the recent stuff (v1.4.3.3..) and drop the older data (..v1.4.3).
The following patch does fix the problem Nicolas reported, but for some
reason I'm still getting only 102 objects (only tags and the commits
they refer to?) with your test.
Jan
----
diff --git a/revision.c b/revision.c
index 280e92b..a69c873 100644
--- a/revision.c
+++ b/revision.c
@@ -418,9 +418,6 @@ static void limit_list(struct rev_info *
if (revs->max_age != -1 && (commit->date < revs->max_age))
obj->flags |= UNINTERESTING;
- if (revs->unpacked &&
- has_sha1_pack(obj->sha1, revs->ignore_packed))
- obj->flags |= UNINTERESTING;
add_parents_to_list(revs, commit, &list);
if (obj->flags & UNINTERESTING) {
mark_parents_uninteresting(commit);
@@ -1149,17 +1146,18 @@ struct commit *get_revision(struct rev_i
* that we'd otherwise have done in limit_list().
*/
if (!revs->limited) {
- if ((revs->unpacked &&
- has_sha1_pack(commit->object.sha1,
- revs->ignore_packed)) ||
- (revs->max_age != -1 &&
- (commit->date < revs->max_age)))
+ if (revs->max_age != -1 &&
+ (commit->date < revs->max_age))
continue;
add_parents_to_list(revs, commit, &revs->commits);
}
if (commit->object.flags & SHOWN)
continue;
+ if (revs->unpacked && has_sha1_pack(commit->object.sha1,
+ revs->ignore_packed))
+ continue;
+
/* We want to show boundary commits only when their
* children are shown. When path-limiter is in effect,
^ permalink raw reply related
* [PATCH] gitweb: esc_html() author in blame
From: Luben Tuikov @ 2006-10-30 20:37 UTC (permalink / raw)
To: git
Blame fails for example on
block/ll_rw_blk.c at v2.6.19-rc3.
Signed-off-by: Luben Tuikov <ltuikov@yahoo.com>
---
gitweb/gitweb.perl | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c52e19d..177efd3 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -2751,7 +2751,7 @@ HTML
print "<tr class=\"$rev_color[$current_color]\">\n";
if ($group_size) {
print "<td class=\"sha1\"";
- print " title=\"$author, $date\"";
+ print " title=\"". esc_html($author) . ", $date\"";
print " rowspan=\"$group_size\"" if ($group_size > 1);
print ">";
print $cgi->a({-href => href(action=>"commit",
--
1.4.3.3.g1e64f-dirty
^ permalink raw reply related
* Re: how to ignore merge conflicts?
From: Linus Torvalds @ 2006-10-30 20:29 UTC (permalink / raw)
To: Len Brown; +Cc: git
In-Reply-To: <200610301448.38222.len.brown@intel.com>
On Mon, 30 Oct 2006, Len Brown wrote:
>
> Sometimes when a multiple-file merge give conflicts, I don't want to edit
> one of the resulting <<<<<=====>>>>> files.
> Instead, I just want to choose the version of that particular file that
> existed in one of the two merged branches and commit that along with
> the rest of the merge.
>
> How to do this?
Well, if you promise not to do what has happened several times before in
people who maintained their own CVS trees, for example (which is to just
ignore all merge problems, and force _their_ version, even though the
reason for the merge problem was that somebody else had fixed a bug, that
was now unfixed by the "merge"), here's the trivial way to do it:
git checkout HEAD the/file/you/wanted.c
(or, if you want to take it from the source you are merging _from_, just
use MERGE_HEAD instead of HEAD).
And you're done.
^ permalink raw reply
* Re: WARNING: THIS PATCH CAN BREAK YOUR REPO, was Re: [PATCH 2/3] Only repack active packs by skipping over kept packs.
From: Shawn Pearce @ 2006-10-30 20:26 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610301332440.11384@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Sun, 29 Oct 2006, Shawn Pearce wrote:
>
> > During `git repack -a -d` only repack objects which are loose or
> > which reside in an active (a non-kept) pack. This allows the user
> > to keep large packs as-is without continuous repacking and can be
> > very helpful on large repositories.
>
> Something is really broken here.
>
> Here's how to destroy your GIT's git repository.
>
> WARNING: MAKE A COPY BEFORE TRYING THIS! I'm serious.
>
> First, let's make a single pack just to make things simpler and
> reproducible:
>
> $ git-repack -a -f -d
> $ git-prune
> $ git-fsck-objects --full
Actually the breakage is easier to reproduce without trashing
a repository.
Do the above so you have everything in one pack. Now use rev-list
to simulate the object list construction in pack-objects as though
we were doing a 'git repack -a -d':
git-rev-list --objects --all \
--unpacked=.git/objects/pack/pack-*.pack \
| wc -l
gives me 102 (WRONG WRONG WRONG WRONG!!!!!!)
and
git-rev-list --objects --all | wc -l
gives me 31912 (correct). The --unpacked flag is horribly broken.
--
^ permalink raw reply
* Re: how to ignore merge conflicts?
From: Jakub Narebski @ 2006-10-30 20:06 UTC (permalink / raw)
To: git
In-Reply-To: <200610301448.38222.len.brown@intel.com>
Len Brown wrote:
> Sometimes when a multiple-file merge give conflicts, I don't want to edit
> one of the resulting <<<<<=====>>>>> files.
> Instead, I just want to choose the version of that particular file that
> existed in one of the two merged branches and commit that along with
> the rest of the merge.
>
> How to do this?
$ git cat-file -p :<n>:<filename> > <filename>
$ git update-index <filename>
where <n> is one of stages, 1 or 2 to choose one branch version if I
remember correctly. Check out documentation.
Or just use 'ours' as merge strategy...
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: git.kernel.org disconnects when git-1.3.3 tries to pull changes
From: Jon Loeliger @ 2006-10-30 19:53 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Simon Arlott, Git List
In-Reply-To: <7vbqo2wlpl.fsf@assigned-by-dhcp.cox.net>
On Mon, 2006-10-23 at 20:23, Junio C Hamano wrote:
> Yes, this is a recent breakage. Thanks for bringing it up.
>
> We need at least a fix like this in 'maint'.
Oooo... Sorry. That looks like my bad.
Thanks.
jdl
^ permalink raw reply
* Re: how to ignore merge conflicts?
From: Shawn Pearce @ 2006-10-30 19:53 UTC (permalink / raw)
To: Len Brown; +Cc: git
In-Reply-To: <200610301448.38222.len.brown@intel.com>
Len Brown <len.brown@intel.com> wrote:
> Sometimes when a multiple-file merge give conflicts, I don't want to edit
> one of the resulting <<<<<=====>>>>> files.
> Instead, I just want to choose the version of that particular file that
> existed in one of the two merged branches and commit that along with
> the rest of the merge.
>
> How to do this?
Your branch is in stage 2 and the branch you are pulling is in
stage 3. So if I ran:
git pull . other
then my HEAD before the pull is stage 2 and other is stage 3.
You can use checkout-index to get your version (stage 2) or the
other version (stage 3):
git checkout-index -f --stage=2 .../path; # mine
git checkout-index -f --stage=3 .../path; # other
then just make sure you update-index before you commit (git commit
-a would do the trick too).
--
^ permalink raw reply
* how to ignore merge conflicts?
From: Len Brown @ 2006-10-30 19:48 UTC (permalink / raw)
To: git
Sometimes when a multiple-file merge give conflicts, I don't want to edit
one of the resulting <<<<<=====>>>>> files.
Instead, I just want to choose the version of that particular file that
existed in one of the two merged branches and commit that along with
the rest of the merge.
How to do this?
thanks,
^ permalink raw reply
* Re: WARNING: THIS PATCH CAN BREAK YOUR REPO, was Re: [PATCH 2/3] Only repack active packs by skipping over kept packs.
From: Shawn Pearce @ 2006-10-30 19:23 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610301332440.11384@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Sun, 29 Oct 2006, Shawn Pearce wrote:
>
> > During `git repack -a -d` only repack objects which are loose or
> > which reside in an active (a non-kept) pack. This allows the user
> > to keep large packs as-is without continuous repacking and can be
> > very helpful on large repositories.
>
> Something is really broken here.
Holy cow. Since this is now in 'next', 'next' is now seriously
broken if you have a .keep file.
> So... what is the --unpacked=<pack>.pack switch supposed to mean? It is
> not documented anywhere and it certainly doesn't produce the expected
> result with a repack.
Junio introduced --unpacked=<pack>.pack a while ago for this
application. What it does is skip an object unless its a loose
object file or it is in the named pack. The idea being that
pack-objects would only consider object files which are loose or
ready to be repacked.
In your example above we should have copied all objects from your
first pack into the new pack during the final destructive repack,
but we didn't. I don't know why.
--
^ permalink raw reply
* [PATCH 5/5] add tests for shallow stuff
From: Johannes Schindelin @ 2006-10-30 19:10 UTC (permalink / raw)
To: git, junkio
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/t5500-fetch-pack.sh | 45 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 45 insertions(+), 0 deletions(-)
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index f7625a6..fa10840 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -128,4 +128,49 @@ pull_to_client 2nd "B" $((64*3))
pull_to_client 3rd "A" $((1*3)) # old fails
+test_expect_success "clone shallow" "git-clone --depth 2 . shallow"
+
+(cd shallow; git-count-objects -v) > count.shallow
+
+test_expect_success "clone shallow object count" \
+ "test \"in-pack: 18\" = \"$(grep in-pack count.shallow)\""
+
+test_expect_success "clone shallow object count (part 2)" \
+ "test -z \"$(grep -v in-pack count.shallow | sed "s/^.*: 0//")\""
+
+test_expect_success "fsck in shallow repo" \
+ "(cd shallow; git-fsck-objects --full)"
+
+#test_done; exit
+
+add B66 $B65
+add B67 $B66
+
+test_expect_success "pull in shallow repo" \
+ "(cd shallow; git pull .. B)"
+
+(cd shallow; git-count-objects -v) > count.shallow
+test_expect_success "clone shallow object count" \
+ "test \"count: 6\" = \"$(grep count count.shallow)\""
+
+add B68 $B67
+add B69 $B68
+
+test_expect_success "deepening pull in shallow repo" \
+ "(cd shallow; git pull --depth 4 .. B)"
+
+(cd shallow; git-count-objects -v) > count.shallow
+test_expect_success "clone shallow object count" \
+ "test \"count: 12\" = \"$(grep count count.shallow)\""
+
+test_expect_success "deepening fetch in shallow repo" \
+ "(cd shallow; git fetch --depth 4 .. A:A)"
+
+(cd shallow; git-count-objects -v) > count.shallow
+test_expect_success "clone shallow object count" \
+ "test \"count: 18\" = \"$(grep count count.shallow)\""
+
+test_expect_failure "pull in shallow repo with missing merge base" \
+ "(cd shallow; git pull --depth 4 .. A)"
+
test_done
--
1.4.3.3.gca42
^ permalink raw reply related
* [PATCH 3/5] allow cloning a repository "shallowly"
From: Johannes Schindelin @ 2006-10-30 19:09 UTC (permalink / raw)
To: git, junkio
By specifying a depth, you can now clone a repository such that
all fetched ancestor-chains' length is at most "depth". For example,
if the upstream repository has only 2 branches ("A" and "B"), which
are linear, and you specify depth 3, you will get A, A~1, A~2, A~3,
B, B~1, B~2, and B~3. The ends are automatically made shallow
commits.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
fetch-pack.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
git-clone.sh | 19 +++++++++++++++--
upload-pack.c | 21 ++++++++++++++++++-
3 files changed, 96 insertions(+), 5 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 488adc9..9619d6e 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -8,8 +8,9 @@ static int keep_pack;
static int quiet;
static int verbose;
static int fetch_all;
+static int depth;
static const char fetch_pack_usage[] =
-"git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack] [host:]directory <refs>...";
+"git-fetch-pack [--all] [-q] [-v] [-k] [--thin] [--exec=upload-pack] [--depth=<n>] [host:]directory <refs>...";
static const char *exec = "git-upload-pack";
#define COMPLETE (1U << 0)
@@ -179,10 +180,29 @@ static int find_common(int fd[2], unsign
}
if (is_repository_shallow())
write_shallow_commits(fd[1], 1);
+ if (depth > 0)
+ packet_write(fd[1], "deepen %d", depth);
packet_flush(fd[1]);
if (!fetching)
return 1;
+ if (depth > 0) {
+ char line[1024];
+ unsigned char sha1[20];
+ int len;
+
+ while ((len = packet_read_line(fd[0], line, sizeof(line)))) {
+ if (!strncmp("shallow ", line, 8)) {
+ if (get_sha1_hex(line + 8, sha1))
+ die("invalid shallow line: %s", line);
+ /* no need making it shallow if we have it already */
+ if (lookup_object(sha1))
+ continue;
+ register_shallow(sha1);
+ }
+ }
+ }
+
flushes = 0;
retval = -1;
while ((sha1 = get_rev())) {
@@ -480,6 +500,8 @@ int main(int argc, char **argv)
char *dest = NULL, **heads;
int fd[2];
pid_t pid;
+ struct stat st;
+ struct lock_file lock;
setup_git_directory();
@@ -513,6 +535,12 @@ int main(int argc, char **argv)
verbose = 1;
continue;
}
+ if (!strncmp("--depth=", arg, 8)) {
+ depth = strtol(arg + 8, NULL, 0);
+ if (stat(git_path("shallow"), &st))
+ st.st_mtime = 0;
+ continue;
+ }
usage(fetch_pack_usage);
}
dest = arg;
@@ -522,6 +550,8 @@ int main(int argc, char **argv)
}
if (!dest)
usage(fetch_pack_usage);
+ if (is_repository_shallow() && depth > 0)
+ die("Deepening of a shallow repository not yet supported!");
pid = git_connect(fd, dest, exec);
if (pid < 0)
return 1;
@@ -543,5 +573,34 @@ int main(int argc, char **argv)
}
}
+ if (!ret && depth > 0) {
+ struct cache_time mtime;
+ char *shallow = git_path("shallow");
+ int fd;
+
+ mtime.sec = st.st_mtime;
+#ifdef USE_NSEC
+ mtime.usec = st.st_mtim.usec;
+#endif
+ if (stat(shallow, &st)) {
+ if (mtime.sec)
+ die("shallow file was removed during fetch");
+ } else if (st.st_mtime != mtime.sec
+#ifdef USE_NSEC
+ || st.st_mtim.usec != mtime.usec
+#endif
+ )
+ die("shallow file was changed during fetch");
+
+ fd = hold_lock_file_for_update(&lock, shallow, 1);
+ if (!write_shallow_commits(fd, 0)) {
+ unlink(lock.filename);
+ rollback_lock_file(&lock);
+ } else {
+ close(fd);
+ commit_lock_file(&lock);
+ }
+ }
+
return !!ret;
}
diff --git a/git-clone.sh b/git-clone.sh
index 3f006d1..595c070 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -14,7 +14,7 @@ die() {
}
usage() {
- die "Usage: $0 [--template=<template_directory>] [--use-separate-remote] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [-n] <repo> [<dir>]"
+ die "Usage: $0 [--template=<template_directory>] [--use-separate-remote] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] <repo> [<dir>]"
}
get_repo_base() {
@@ -116,6 +116,7 @@ reference=
origin=
origin_override=
use_separate_remote=
+depth=
while
case "$#,$1" in
0,*) break ;;
@@ -158,6 +159,10 @@ while
*,-u|*,--upload-pack)
shift
upload_pack="--exec=$1" ;;
+ 1,--depth) usage;;
+ *,--depth)
+ shift
+ depth="--depth=$1";;
*,-*) usage ;;
*) break ;;
esac
@@ -265,6 +270,10 @@ yes,yes)
*)
case "$repo" in
rsync://*)
+ case "$depth" in
+ "") ;;
+ *) die "shallow over rsync not supported" ;;
+ esac
rsync $quiet -av --ignore-existing \
--exclude info "$repo/objects/" "$GIT_DIR/objects/" ||
exit
@@ -293,6 +302,10 @@ yes,yes)
git-ls-remote "$repo" >"$GIT_DIR/CLONE_HEAD" || exit 1
;;
https://*|http://*|ftp://*)
+ case "$depth" in
+ "") ;;
+ *) die "shallow over http or ftp not supported" ;;
+ esac
if test -z "@@NO_CURL@@"
then
clone_dumb_http "$repo" "$D"
@@ -302,8 +315,8 @@ yes,yes)
;;
*)
case "$upload_pack" in
- '') git-fetch-pack --all -k $quiet "$repo" ;;
- *) git-fetch-pack --all -k $quiet "$upload_pack" "$repo" ;;
+ '') git-fetch-pack --all -k $quiet $depth "$repo" ;;
+ *) git-fetch-pack --all -k $quiet "$upload_pack" $depth "$repo" ;;
esac >"$GIT_DIR/CLONE_HEAD" ||
die "fetch-pack from '$repo' failed."
;;
diff --git a/upload-pack.c b/upload-pack.c
index 8dd6121..ebe1e5a 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -488,7 +488,7 @@ static void receive_needs(void)
{
struct object_array shallows = {0, 0, NULL};
static char line[1000];
- int len;
+ int len, depth = 0;
for (;;) {
struct object *o;
@@ -509,6 +509,13 @@ static void receive_needs(void)
add_object_array(object, NULL, &shallows);
continue;
}
+ if (!strncmp("deepen ", line, 7)) {
+ char *end;
+ depth = strtol(line + 7, &end, 0);
+ if (end == line + 7 || depth <= 0)
+ die("Invalid deepen: %s", line);
+ continue;
+ }
if (strncmp("want ", line, 5) ||
get_sha1_hex(line+5, sha1_buf))
die("git-upload-pack: protocol error, "
@@ -540,6 +547,18 @@ static void receive_needs(void)
add_object_array(o, NULL, &want_obj);
}
}
+ if (depth > 0) {
+ struct commit_list *result, *backup;
+ if (shallows.nr > 0)
+ die("Deepening a shallow repository not yet supported");
+ backup = result = get_shallow_commits(&want_obj, depth);
+ while (result) {
+ packet_write(1, "shallow %s",
+ sha1_to_hex(result->item->object.sha1));
+ result = result->next;
+ }
+ free_commit_list(backup);
+ }
if (shallows.nr > 0) {
int i;
for (i = 0; i < shallows.nr; i++)
--
1.4.3.3.gca42
^ permalink raw reply related
* [PATCH 2/5] support fetching into a shallow repository
From: Johannes Schindelin @ 2006-10-30 19:09 UTC (permalink / raw)
To: git, junkio
A shallow commit is a commit which has parents, which in turn are
"grafted away", i.e. the commit appears as if it were a root.
Since these shallow commits should not be edited by the user, but
only by core git, they are recorded in the file $GIT_DIR/shallow.
A repository containing shallow commits is called shallow.
The advantage of a shallow repository is that even if the upstream
contains lots of history, your local (shallow) repository needs not
occupy much disk space.
The disadvantage is that you might miss a merge base when pulling
some remote branch.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Makefile | 2 +-
commit.c | 21 ++++++++++++
commit.h | 8 ++++-
fetch-pack.c | 4 ++
shallow.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
upload-pack.c | 22 ++++++++++++-
6 files changed, 150 insertions(+), 4 deletions(-)
diff --git a/Makefile b/Makefile
index 28a5d91..5f57aad 100644
--- a/Makefile
+++ b/Makefile
@@ -263,7 +263,7 @@ LIB_OBJS = \
fetch-clone.o revision.o pager.o tree-walk.o xdiff-interface.o \
write_or_die.o trace.o list-objects.o grep.o \
alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
- color.o wt-status.o archive-zip.o archive-tar.o
+ color.o wt-status.o archive-zip.o archive-tar.o shallow.o
BUILTIN_OBJS = \
builtin-add.o \
diff --git a/commit.c b/commit.c
index a6d543e..bffa278 100644
--- a/commit.c
+++ b/commit.c
@@ -1,6 +1,7 @@
#include "cache.h"
#include "tag.h"
#include "commit.h"
+#include "pkt-line.h"
int save_commit_buffer = 1;
@@ -221,6 +222,8 @@ static void prepare_commit_graft(void)
return;
graft_file = get_graft_file();
read_graft_file(graft_file);
+ /* make sure shallows are read */
+ is_repository_shallow();
commit_graft_prepared = 1;
}
@@ -234,6 +237,24 @@ static struct commit_graft *lookup_commi
return commit_graft[pos];
}
+int write_shallow_commits(int fd, int use_pack_protocol)
+{
+ int i, count = 0;
+ for (i = 0; i < commit_graft_nr; i++)
+ if (commit_graft[i]->nr_parent < 0) {
+ const char *hex =
+ sha1_to_hex(commit_graft[i]->sha1);
+ count++;
+ if (use_pack_protocol)
+ packet_write(fd, "shallow %s", hex);
+ else {
+ write(fd, hex, 40);
+ write(fd, "\n", 1);
+ }
+ }
+ return count;
+}
+
int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)
{
char *tail = buffer;
diff --git a/commit.h b/commit.h
index fc13de9..c559510 100644
--- a/commit.h
+++ b/commit.h
@@ -97,7 +97,7 @@ void sort_in_topological_order_fn(struct
struct commit_graft {
unsigned char sha1[20];
- int nr_parent;
+ int nr_parent; /* < 0 if shallow commit */
unsigned char parent[FLEX_ARRAY][20]; /* more */
};
@@ -107,4 +107,10 @@ int read_graft_file(const char *graft_fi
extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2, int cleanup);
+extern int register_shallow(const unsigned char *sha1);
+extern int write_shallow_commits(int fd, int use_pack_protocol);
+extern int is_repository_shallow();
+extern struct commit_list *get_shallow_commits(struct object_array *heads,
+ int depth);
+
#endif /* COMMIT_H */
diff --git a/fetch-pack.c b/fetch-pack.c
index 36ea092..488adc9 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -177,6 +177,8 @@ static int find_common(int fd[2], unsign
packet_write(fd[1], "want %s\n", sha1_to_hex(remote));
fetching++;
}
+ if (is_repository_shallow())
+ write_shallow_commits(fd[1], 1);
packet_flush(fd[1]);
if (!fetching)
return 1;
@@ -423,6 +425,8 @@ static int fetch_pack(int fd[2], int nr_
int status;
get_remote_heads(fd[0], &ref, 0, NULL, 0);
+ if (is_repository_shallow() && !server_supports("shallow"))
+ die("Server does not support shallow clients");
if (server_supports("multi_ack")) {
if (verbose)
fprintf(stderr, "Server supports multi_ack\n");
diff --git a/shallow.c b/shallow.c
new file mode 100644
index 0000000..3cf2127
--- /dev/null
+++ b/shallow.c
@@ -0,0 +1,97 @@
+#include "cache.h"
+#include "commit.h"
+
+static int is_shallow = -1;
+
+int register_shallow(const unsigned char *sha1)
+{
+ struct commit_graft *graft =
+ xmalloc(sizeof(struct commit_graft));
+ struct commit *commit = lookup_commit(sha1);
+
+ hashcpy(graft->sha1, sha1);
+ graft->nr_parent = -1;
+ if (commit && commit->object.parsed)
+ commit->parents = NULL;
+ return register_commit_graft(graft, 0);
+}
+
+int is_repository_shallow()
+{
+ FILE *fp;
+ char buf[1024];
+
+ if (is_shallow >= 0)
+ return is_shallow;
+
+ fp = fopen(git_path("shallow"), "r");
+ if (!fp) {
+ is_shallow = 0;
+ return is_shallow;
+ }
+ is_shallow = 1;
+
+ while (fgets(buf, sizeof(buf), fp)) {
+ unsigned char sha1[20];
+ if (get_sha1_hex(buf, sha1))
+ die("bad shallow line: %s", buf);
+ register_shallow(sha1);
+ }
+ fclose(fp);
+ return is_shallow;
+}
+
+struct commit_list *get_shallow_commits(struct object_array *heads, int depth)
+{
+ int i = 0, cur_depth = 0;
+ struct commit_list *result = NULL;
+ struct object_array stack = {0, 0, NULL};
+ struct commit *commit = NULL;
+
+ while (commit || i < heads->nr || stack.nr) {
+ struct commit_list *p;
+ if (!commit) {
+ if (i < heads->nr) {
+ commit = (struct commit *)
+ heads->objects[i++].item;
+ if (commit->object.type != OBJ_COMMIT) {
+ commit = NULL;
+ continue;
+ }
+ commit->util = xcalloc(1, sizeof(int));
+ cur_depth = 0;
+ } else {
+ commit = (struct commit *)
+ stack.objects[--stack.nr].item;
+ cur_depth = *(int *)commit->util;
+ }
+ }
+ parse_commit(commit);
+ cur_depth++;
+ for (p = commit->parents, commit = NULL; p; p = p->next) {
+ if (!p->item->util) {
+ int *pointer = xmalloc(sizeof(int));
+ p->item->util = pointer;
+ *pointer = cur_depth;
+ } else {
+ int *pointer = p->item->util;
+ if (cur_depth >= *pointer)
+ continue;
+ *pointer = cur_depth;
+ }
+ if (cur_depth < depth) {
+ if (p->next)
+ add_object_array(&p->item->object,
+ NULL, &stack);
+ else {
+ commit = p->item;
+ cur_depth = *(int *)commit->util;
+ }
+ } else
+ commit_list_insert(p->item, &result);
+ }
+ }
+
+ return result;
+}
+
diff --git a/upload-pack.c b/upload-pack.c
index 7f7df2a..8dd6121 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -486,6 +486,7 @@ static int get_common_commits(void)
static void receive_needs(void)
{
+ struct object_array shallows = {0, 0, NULL};
static char line[1000];
int len;
@@ -495,8 +496,19 @@ static void receive_needs(void)
len = packet_read_line(0, line, sizeof(line));
reset_timeout();
if (!len)
- return;
+ break;
+ if (!strncmp("shallow ", line, 8)) {
+ unsigned char sha1[20];
+ struct object *object;
+ if (get_sha1(line + 8, sha1))
+ die("invalid shallow line: %s", line);
+ object = parse_object(sha1);
+ if (!object)
+ die("did not find object for %s", line);
+ add_object_array(object, NULL, &shallows);
+ continue;
+ }
if (strncmp("want ", line, 5) ||
get_sha1_hex(line+5, sha1_buf))
die("git-upload-pack: protocol error, "
@@ -528,11 +540,17 @@ static void receive_needs(void)
add_object_array(o, NULL, &want_obj);
}
}
+ if (shallows.nr > 0) {
+ int i;
+ for (i = 0; i < shallows.nr; i++)
+ register_shallow(shallows.objects[i].item->sha1);
+ }
}
static int send_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
{
- static const char *capabilities = "multi_ack thin-pack side-band side-band-64k ofs-delta";
+ static const char *capabilities = "multi_ack thin-pack side-band"
+ " side-band-64k ofs-delta shallow";
struct object *o = parse_object(sha1);
if (!o)
--
1.4.3.3.gca42
^ permalink raw reply related
* [PATCH 4/5] allow deepening of a shallow repository
From: Johannes Schindelin @ 2006-10-30 19:09 UTC (permalink / raw)
To: git, junkio
Now, by saying "git fetch -depth <n> <repo>" you can deepen
a shallow repository.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
commit.c | 13 +++++++++++++
commit.h | 3 ++-
fetch-pack.c | 22 ++++++++++++++++------
git-fetch.sh | 12 +++++++++++-
shallow.c | 8 ++++++--
upload-pack.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++-----------
6 files changed, 94 insertions(+), 21 deletions(-)
diff --git a/commit.c b/commit.c
index bffa278..d5103cd 100644
--- a/commit.c
+++ b/commit.c
@@ -255,6 +255,19 @@ int write_shallow_commits(int fd, int us
return count;
}
+int unregister_shallow(const unsigned char *sha1)
+{
+ int pos = commit_graft_pos(sha1);
+ if (pos < 0)
+ return -1;
+ if (pos + 1 < commit_graft_nr)
+ memcpy(commit_graft + pos, commit_graft + pos + 1,
+ sizeof(struct commit_graft *)
+ * (commit_graft_nr - pos - 1));
+ commit_graft_nr--;
+ return 0;
+}
+
int parse_commit_buffer(struct commit *item, void *buffer, unsigned long size)
{
char *tail = buffer;
diff --git a/commit.h b/commit.h
index c559510..e9e158f 100644
--- a/commit.h
+++ b/commit.h
@@ -108,9 +108,10 @@ int read_graft_file(const char *graft_fi
extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2, int cleanup);
extern int register_shallow(const unsigned char *sha1);
+extern int unregister_shallow(const unsigned char *sha1);
extern int write_shallow_commits(int fd, int use_pack_protocol);
extern int is_repository_shallow();
extern struct commit_list *get_shallow_commits(struct object_array *heads,
- int depth);
+ int depth, int shallow_flag, int not_shallow_flag);
#endif /* COMMIT_H */
diff --git a/fetch-pack.c b/fetch-pack.c
index 9619d6e..a82a5ba 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -199,7 +199,17 @@ static int find_common(int fd[2], unsign
if (lookup_object(sha1))
continue;
register_shallow(sha1);
- }
+ } else if (!strncmp("unshallow ", line, 10)) {
+ if (get_sha1_hex(line + 10, sha1))
+ die("invalid unshallow line: %s", line);
+ if (!lookup_object(sha1))
+ die("object not found: %s", line);
+ /* make sure that it is parsed as shallow */
+ parse_object(sha1);
+ if (unregister_shallow(sha1))
+ die("no shallow found: %s", line);
+ } else
+ die("expected shallow/unshallow, got %s", line);
}
}
@@ -388,9 +398,11 @@ static int everything_local(struct ref *
}
}
- for_each_ref(mark_complete, NULL);
- if (cutoff)
- mark_recent_complete_commits(cutoff);
+ if (!depth) {
+ for_each_ref(mark_complete, NULL);
+ if (cutoff)
+ mark_recent_complete_commits(cutoff);
+ }
/*
* Mark all complete remote refs as common refs.
@@ -550,8 +562,6 @@ int main(int argc, char **argv)
}
if (!dest)
usage(fetch_pack_usage);
- if (is_repository_shallow() && depth > 0)
- die("Deepening of a shallow repository not yet supported!");
pid = git_connect(fd, dest, exec);
if (pid < 0)
return 1;
diff --git a/git-fetch.sh b/git-fetch.sh
index 539dff6..2a4b7a0 100755
--- a/git-fetch.sh
+++ b/git-fetch.sh
@@ -21,6 +21,7 @@ update_head_ok=
exec=
upload_pack=
keep=--thin
+depth=
while case "$#" in 0) break ;; esac
do
case "$1" in
@@ -56,6 +57,13 @@ do
--reflog-action=*)
rloga=`expr "z$1" : 'z-[^=]*=\(.*\)'`
;;
+ --depth=*)
+ depth="--depth=`expr "z$1" : 'z-[^=]*=\(.*\)'`"
+ ;;
+ --depth)
+ shift
+ depth="--depth=$1"
+ ;;
-*)
usage
;;
@@ -296,6 +304,7 @@ fetch_main () {
# There are transports that can fetch only one head at a time...
case "$remote" in
http://* | https://* | ftp://*)
+ test -n "$depth" && die "shallow clone with http not supported"
proto=`expr "$remote" : '\([^:]*\):'`
if [ -n "$GIT_SSL_NO_VERIFY" ]; then
curl_extra_args="-k"
@@ -324,6 +333,7 @@ fetch_main () {
git-http-fetch -v -a "$head" "$remote/" || exit
;;
rsync://*)
+ test -n "$depth" && die "shallow clone with rsync not supported"
TMP_HEAD="$GIT_DIR/TMP_HEAD"
rsync -L -q "$remote/$remote_name" "$TMP_HEAD" || exit 1
head=$(git-rev-parse --verify TMP_HEAD)
@@ -370,7 +380,7 @@ fetch_main () {
( : subshell because we muck with IFS
IFS=" $LF"
(
- git-fetch-pack $exec $keep "$remote" $rref || echo failed "$remote"
+ git-fetch-pack $exec $keep $depth "$remote" $rref || echo failed "$remote"
) |
while read sha1 remote_name
do
diff --git a/shallow.c b/shallow.c
index 3cf2127..58a7b20 100644
--- a/shallow.c
+++ b/shallow.c
@@ -41,7 +41,8 @@ int is_repository_shallow()
return is_shallow;
}
-struct commit_list *get_shallow_commits(struct object_array *heads, int depth)
+struct commit_list *get_shallow_commits(struct object_array *heads, int depth,
+ int shallow_flag, int not_shallow_flag)
{
int i = 0, cur_depth = 0;
struct commit_list *result = NULL;
@@ -67,6 +68,7 @@ struct commit_list *get_shallow_commits(
}
}
parse_commit(commit);
+ commit->object.flags |= not_shallow_flag;
cur_depth++;
for (p = commit->parents, commit = NULL; p; p = p->next) {
if (!p->item->util) {
@@ -87,8 +89,10 @@ struct commit_list *get_shallow_commits(
commit = p->item;
cur_depth = *(int *)commit->util;
}
- } else
+ } else {
commit_list_insert(p->item, &result);
+ p->item->object.flags |= shallow_flag;
+ }
}
}
diff --git a/upload-pack.c b/upload-pack.c
index ebe1e5a..162dd34 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -134,6 +134,7 @@ static void create_pack_file(void)
} else {
for (i = 0; i < want_obj.nr; i++) {
struct object *o = want_obj.objects[i].item;
+ o->flags &= ~UNINTERESTING;
add_pending_object(&revs, o, NULL);
}
for (i = 0; i < have_obj.nr; i++) {
@@ -490,6 +491,9 @@ static void receive_needs(void)
static char line[1000];
int len, depth = 0;
+#define SHALLOW (1u<<8)
+#define NOT_SHALLOW (1u<<9)
+#define CLIENT_SHALLOW (1u<<10)
for (;;) {
struct object *o;
unsigned char sha1_buf[20];
@@ -501,16 +505,19 @@ static void receive_needs(void)
if (!strncmp("shallow ", line, 8)) {
unsigned char sha1[20];
struct object *object;
+ use_thin_pack = 0;
if (get_sha1(line + 8, sha1))
die("invalid shallow line: %s", line);
object = parse_object(sha1);
if (!object)
die("did not find object for %s", line);
+ object->flags |= CLIENT_SHALLOW;
add_object_array(object, NULL, &shallows);
continue;
}
if (!strncmp("deepen ", line, 7)) {
char *end;
+ use_thin_pack = 0;
depth = strtol(line + 7, &end, 0);
if (end == line + 7 || depth <= 0)
die("Invalid deepen: %s", line);
@@ -547,23 +554,51 @@ static void receive_needs(void)
add_object_array(o, NULL, &want_obj);
}
}
+ if (depth == 0 && shallows.nr == 0)
+ return;
if (depth > 0) {
struct commit_list *result, *backup;
- if (shallows.nr > 0)
- die("Deepening a shallow repository not yet supported");
- backup = result = get_shallow_commits(&want_obj, depth);
+ int i;
+ backup = result = get_shallow_commits(&want_obj, depth,
+ SHALLOW, NOT_SHALLOW);
while (result) {
- packet_write(1, "shallow %s",
- sha1_to_hex(result->item->object.sha1));
+ struct object *object = &result->item->object;
+ if (!(object->flags & CLIENT_SHALLOW)) {
+ packet_write(1, "shallow %s",
+ sha1_to_hex(object->sha1));
+ register_shallow(object->sha1);
+ }
result = result->next;
}
free_commit_list(backup);
- }
- if (shallows.nr > 0) {
- int i;
- for (i = 0; i < shallows.nr; i++)
- register_shallow(shallows.objects[i].item->sha1);
- }
+ for (i = 0; i < shallows.nr; i++) {
+ struct object *object = shallows.objects[i].item;
+ if (object->flags & NOT_SHALLOW) {
+ struct commit_list *parents;
+ packet_write(1, "unshallow %s",
+ sha1_to_hex(object->sha1));
+ object->flags &= ~CLIENT_SHALLOW;
+ /* make sure the real parents are parsed */
+ unregister_shallow(object->sha1);
+ parse_commit((struct commit *)object);
+ parents = ((struct commit *)object)->parents;
+ while (parents) {
+ add_object_array(&parents->item->object,
+ NULL, &want_obj);
+ parents = parents->next;
+ }
+ }
+ /* make sure commit traversal conforms to client */
+ register_shallow(object->sha1);
+ }
+ packet_flush(1);
+ } else
+ if (shallows.nr > 0) {
+ int i;
+ for (i = 0; i < shallows.nr; i++)
+ register_shallow(shallows.objects[i].item->sha1);
+ }
+ free(shallows.objects);
}
static int send_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
--
1.4.3.3.gca42
^ permalink raw reply related
* [PATCH 1/5] upload-pack: no longer call rev-list
From: Johannes Schindelin @ 2006-10-30 19:08 UTC (permalink / raw)
To: git, junkio
It is trivial to do now, and it is needed for the upcoming shallow
clone stuff.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
upload-pack.c | 93 ++++++++++++++++++++++++++++++++++++--------------------
1 files changed, 60 insertions(+), 33 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index 4572fff..7f7df2a 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -9,6 +9,9 @@
#include "object.h"
#include "commit.h"
#include "exec_cmd.h"
+#include "diff.h"
+#include "revision.h"
+#include "list-objects.h"
static const char upload_pack_usage[] = "git-upload-pack [--strict] [--timeout=nn] <dir>";
@@ -57,6 +60,40 @@ static ssize_t send_client_data(int fd,
return safe_write(fd, data, sz);
}
+FILE *pack_pipe = NULL;
+static void show_commit(struct commit *commit)
+{
+ if (commit->object.flags & BOUNDARY)
+ fputc('-', pack_pipe);
+ if (fputs(sha1_to_hex(commit->object.sha1), pack_pipe) < 0)
+ die("broken output pipe");
+ fputc('\n', pack_pipe);
+ fflush(pack_pipe);
+ free(commit->buffer);
+ commit->buffer = NULL;
+}
+
+static void show_object(struct object_array_entry *p)
+{
+ /* An object with name "foo\n0000000..." can be used to
+ * confuse downstream git-pack-objects very badly.
+ */
+ const char *ep = strchr(p->name, '\n');
+ if (ep) {
+ fprintf(pack_pipe, "%s %.*s\n", sha1_to_hex(p->item->sha1),
+ (int) (ep - p->name),
+ p->name);
+ }
+ else
+ fprintf(pack_pipe, "%s %s\n",
+ sha1_to_hex(p->item->sha1), p->name);
+}
+
+static void show_edge(struct commit *commit)
+{
+ fprintf(pack_pipe, "-%s\n", sha1_to_hex(commit->object.sha1));
+}
+
static void create_pack_file(void)
{
/* Pipes between rev-list to pack-objects, pack-objects to us
@@ -78,48 +115,38 @@ static void create_pack_file(void)
if (!pid_rev_list) {
int i;
- int args;
- const char **argv;
- const char **p;
- char *buf;
+ struct rev_info revs;
- if (create_full_pack) {
- args = 10;
- use_thin_pack = 0; /* no point doing it */
- }
- else
- args = have_obj.nr + want_obj.nr + 5;
- p = xmalloc(args * sizeof(char *));
- argv = (const char **) p;
- buf = xmalloc(args * 45);
+ pack_pipe = fdopen(lp_pipe[1], "w");
- dup2(lp_pipe[1], 1);
- close(0);
- close(lp_pipe[0]);
- close(lp_pipe[1]);
- *p++ = "rev-list";
- *p++ = use_thin_pack ? "--objects-edge" : "--objects";
if (create_full_pack)
- *p++ = "--all";
- else {
+ use_thin_pack = 0; /* no point doing it */
+ init_revisions(&revs, NULL);
+ revs.tag_objects = 1;
+ revs.tree_objects = 1;
+ revs.blob_objects = 1;
+ if (use_thin_pack)
+ revs.edge_hint = 1;
+
+ if (create_full_pack) {
+ const char *args[] = {"rev-list", "--all", NULL};
+ setup_revisions(2, args, &revs, NULL);
+ } else {
for (i = 0; i < want_obj.nr; i++) {
struct object *o = want_obj.objects[i].item;
- *p++ = buf;
- memcpy(buf, sha1_to_hex(o->sha1), 41);
- buf += 41;
+ add_pending_object(&revs, o, NULL);
}
- }
- if (!create_full_pack)
for (i = 0; i < have_obj.nr; i++) {
struct object *o = have_obj.objects[i].item;
- *p++ = buf;
- *buf++ = '^';
- memcpy(buf, sha1_to_hex(o->sha1), 41);
- buf += 41;
+ o->flags |= UNINTERESTING;
+ add_pending_object(&revs, o, NULL);
}
- *p++ = NULL;
- execv_git_cmd(argv);
- die("git-upload-pack: unable to exec git-rev-list");
+ setup_revisions(0, NULL, &revs, NULL);
+ }
+ prepare_revision_walk(&revs);
+ mark_edges_uninteresting(revs.commits, &revs, show_edge);
+ traverse_commit_list(&revs, show_commit, show_object);
+ exit(0);
}
if (pipe(pu_pipe) < 0)
--
1.4.3.3.gca42
^ permalink raw reply related
* [PATCH (WISP) 0/5] Support shallow repositories
From: Johannes Schindelin @ 2006-10-30 19:08 UTC (permalink / raw)
To: git, junkio
Hi,
Def.: Shallow commits do have parents, but not in the shallow
repo, and therefore grafts are introduced pretending that
these commits have no parents.
The basic idea is to write the SHA1s of shallow commits into
$GIT_DIR/shallow, and handle its contents like the contents
of $GIT_DIR/info/grafts (with the difference that shallow
cannot contain parent information).
This information is stored in a new file instead of grafts, or
even the config, since the user should not touch that file
at all (even I did not once edit it myself during development!).
Each line contains exactly one SHA1. When read, a commit_graft
will be constructed, which has nr_parent < 0 to make it easier
to discern from user provided grafts.
Since fsck-objects relies on the library to read the objects,
it honours shallow commits automatically.
This stuff is definitely "pu" material:
- I am not quite sure if we have to force non-thin packs when
fetching into a shallow repo (ATM they are forced non-thin).
Comments?
- A special handling of a shallow upstream is needed. At some
stage, upload-pack has to check if it sends a shallow commit,
and it should send that information early (or fail, if the
client does not support shallow repositories). There is no
support at all for this in this patch series.
- Instead of locking $GIT_DIR/shallow at the start, just
the timestamp of it is noted, and when it comes to writing it,
a check is performed if the mtime is still the same, dying if
it is not. Is this enough?
- I have not touched the "push into/from a shallow repo" issue.
Don't even try it yet.
- If you deepen a history, you'd want to get the tags of the
newly stored (but older!) commits. I don't think this works
right now.
This series does the following:
- gets rid of the call to rev-list in upload-pack
It is just a tiny change. It is needed so that the client can
tell the server which commits to ignore for the moment. Note that
this is bad for porting to Windows: it should be relatively easy
to spawn other programs in Win32, but is probably lot more work
to just fork _and_ maintain the memory and fd's.
- support fetching into shallow repositories
Up until now (including this patch), we do not have tool
support to make shallow repos. This patch is separated from
the rest for your reviewing pleasure only.
- support making a shallow repository by cloning with a depth
"git-clone --depth 20 repo" will lead to a shallow repository,
which contains only commit chains with a length of at most 20. It
also writes an appropriate $GIT_DIR/shallow.
- support deepening a shallow repository
"git-fetch --depth 20 repo branch" will fetch branch from repo,
but stop at depth 20, updating $GIT_DIR/shallow.
- add tests to t5500
Since t5500 conveniently provides all we need to test shallow
clones, no new test script was added, but t5500 extended.
Unfortunately, my time resources are scarce in the next few days,
but I did not want to hold this stuff back any longer.
Ciao,
Dscho
^ permalink raw reply
* WARNING: THIS PATCH CAN BREAK YOUR REPO, was Re: [PATCH 2/3] Only repack active packs by skipping over kept packs.
From: Nicolas Pitre @ 2006-10-30 19:07 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Junio C Hamano, git
In-Reply-To: <20061029093754.GD3847@spearce.org>
On Sun, 29 Oct 2006, Shawn Pearce wrote:
> During `git repack -a -d` only repack objects which are loose or
> which reside in an active (a non-kept) pack. This allows the user
> to keep large packs as-is without continuous repacking and can be
> very helpful on large repositories.
Something is really broken here.
Here's how to destroy your GIT's git repository.
WARNING: MAKE A COPY BEFORE TRYING THIS! I'm serious.
First, let's make a single pack just to make things simpler and
reproducible:
$ git-repack -a -f -d
$ git-prune
$ git-fsck-objects --full
So far everything should be fine. It's still time to make a backup copy
of your .git directory if you've not done so.
Now let's create a second pack containing a subset of the existing one.
$ git-rev-list --objects v1.4.3..v1.4.3.3 | \
git-pack-objects --stdout | \
git-index-pack --stdin -v --keep
$ git-fsck-objects --full
At this point the repository is still fine, but the --keep to
git-index-pack above will have created a file called
.git/objects/pack/pack-aceb4c6394c586abaf65d76dd6cf088f50a5b806.keep and
that is the source of all the trouble to come. You still can remove
that file if you don't have a backup yet.
If you still want to give it the coup de grace, just do:
$ git-repack -a -d
And now you've just lost a large amount of objects. To see the extent
of the dammage, just do:
$ git-fsck-objects
So... what is the --unpacked=<pack>.pack switch supposed to mean? It is
not documented anywhere and it certainly doesn't produce the expected
result with a repack.
^ permalink raw reply
* [PATCH 0/n] gitweb: Better quoting and New improved patchset view
From: Jakub Narebski @ 2006-10-30 18:53 UTC (permalink / raw)
To: git
This series of patches is meant to introduce
"New improved patchset view" in part-by-part
case.
Would change only gitweb/gitweb.perl and gitweb/gitweb/css
--
Jakub Narebski
^ permalink raw reply
* [PATCH/RFC 1/n] gitweb: Better git-unquoting and gitweb-quoting of pathnames
From: Jakub Narebski @ 2006-10-30 18:58 UTC (permalink / raw)
To: git
In-Reply-To: <200610301953.01875.jnareb@gmail.com>
Extend unquote subroutine, which unquotes quoted and escaped filenames
which git may return, to deal not only with octal char sequence
quoting, but also quoting ordinary characters including '\"' and '\\'
which are respectively quoted '"' and '\', and to deal also with
C escape sequences including '\t' for TAB and '\n' for LF.
Add esc_path subroutine for gitweb quoting and HTML escaping filenames
(currently it does equivalent of ls' --hide-control-chars, which means
showing undisplayable characters (including '\n' and '\t') as '?'
(question mark) character. Convert gitweb to use esc_path instead of
simply esc_html to print pathnames.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
By the way, testing this patch uncovered some errors in gitweb, some
related to files with strange name, some unrelated. I'll address them
in further patches. They are:
1. Using m/..\t(.+)$/; to catch filename instead of m/..\t(.+)$/s;
2. Lack of '--' after $hash/$hash_base parameter which gives error
if there exist branch (ref) and file (or directory) with the same
name
The current implementation of esc_path is meant as preliminary: if you
have better idea for quoting names in gitweb, please tell us, or better
send code/patches.
gitweb/gitweb.perl | 68 +++++++++++++++++++++++++++++++++++++--------------
1 files changed, 49 insertions(+), 19 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index ec46b80..a15e916 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -563,12 +563,42 @@ sub esc_html {
return $str;
}
+# quote unsafe characters and escape filename to HTML
+sub esc_path {
+ my $str = shift;
+ $str = esc_html($str);
+ $str =~ s/[[:cntrl:]\a\b\e\f\n\r\t\011]/?/g; # like --hide-control-chars in ls
+ return $str;
+}
+
# git may return quoted and escaped filenames
sub unquote {
my $str = shift;
+
+ sub unq {
+ my $seq = shift;
+ my %es = (
+ 't' => "\t", # tab (HT, TAB)
+ 'n' => "\n", # newline (NL)
+ 'r' => "\r", # return (CR)
+ 'f' => "\f", # form feed (FF)
+ 'b' => "\b", # backspace (BS)
+ 'a' => "\a", # alarm (bell) (BEL)
+ #'e' => "\e", # escape (ESC)
+ 'v' => "\011", # vertical tab (VT)
+ );
+
+ # octal char sequence
+ return chr(oct($seq)) if ($seq =~ m/^[0-7]{1,3}$/);
+ # C escape sequence (this includes '\n' (LF) and '\t' (TAB))
+ return $es{$seq} if ($seq =~ m/^[abefnrtv]$/);
+ # quted ordinary character (this includes '\\' and '\"')
+ return $seq;
+ }
+
if ($str =~ m/^"(.*)"$/) {
$str = $1;
- $str =~ s/\\([0-7]{1,3})/chr(oct($1))/eg;
+ $str =~ s/\\([^0-7]|[0-7]{1,3})/unq($1)/eg;
}
return $str;
}
@@ -1435,7 +1465,7 @@ sub git_header_html {
if (defined $action) {
$title .= "/$action";
if (defined $file_name) {
- $title .= " - " . esc_html($file_name);
+ $title .= " - " . esc_path($file_name);
if ($action eq "tree" && $file_name !~ m|/$|) {
$title .= "/";
}
@@ -1706,20 +1736,20 @@ sub git_print_page_path {
$fullname .= ($fullname ? '/' : '') . $dir;
print $cgi->a({-href => href(action=>"tree", file_name=>$fullname,
hash_base=>$hb),
- -title => $fullname}, esc_html($dir));
+ -title => $fullname}, esc_path($dir));
print " / ";
}
if (defined $type && $type eq 'blob') {
print $cgi->a({-href => href(action=>"blob_plain", file_name=>$file_name,
hash_base=>$hb),
- -title => $name}, esc_html($basename));
+ -title => $name}, esc_path($basename));
} elsif (defined $type && $type eq 'tree') {
print $cgi->a({-href => href(action=>"tree", file_name=>$file_name,
hash_base=>$hb),
- -title => $name}, esc_html($basename));
+ -title => $name}, esc_path($basename));
print " / ";
} else {
- print esc_html($basename);
+ print esc_path($basename);
}
}
print "<br/></div>\n";
@@ -1791,7 +1821,7 @@ sub git_print_tree_entry {
print "<td class=\"list\">" .
$cgi->a({-href => href(action=>"blob", hash=>$t->{'hash'},
file_name=>"$basedir$t->{'name'}", %base_key),
- -class => "list"}, esc_html($t->{'name'})) . "</td>\n";
+ -class => "list"}, esc_path($t->{'name'})) . "</td>\n";
print "<td class=\"link\">";
print $cgi->a({-href => href(action=>"blob", hash=>$t->{'hash'},
file_name=>"$basedir$t->{'name'}", %base_key)},
@@ -1818,7 +1848,7 @@ sub git_print_tree_entry {
print "<td class=\"list\">";
print $cgi->a({-href => href(action=>"tree", hash=>$t->{'hash'},
file_name=>"$basedir$t->{'name'}", %base_key)},
- esc_html($t->{'name'}));
+ esc_path($t->{'name'}));
print "</td>\n";
print "<td class=\"link\">";
print $cgi->a({-href => href(action=>"tree", hash=>$t->{'hash'},
@@ -1883,7 +1913,7 @@ sub git_difftree_body {
print "<td>";
print $cgi->a({-href => href(action=>"blob", hash=>$diff{'to_id'},
hash_base=>$hash, file_name=>$diff{'file'}),
- -class => "list"}, esc_html($diff{'file'}));
+ -class => "list"}, esc_path($diff{'file'}));
print "</td>\n";
print "<td>$mode_chng</td>\n";
print "<td class=\"link\">";
@@ -1899,7 +1929,7 @@ sub git_difftree_body {
print "<td>";
print $cgi->a({-href => href(action=>"blob", hash=>$diff{'from_id'},
hash_base=>$parent, file_name=>$diff{'file'}),
- -class => "list"}, esc_html($diff{'file'}));
+ -class => "list"}, esc_path($diff{'file'}));
print "</td>\n";
print "<td>$mode_chng</td>\n";
print "<td class=\"link\">";
@@ -1939,7 +1969,7 @@ sub git_difftree_body {
print "<td>";
print $cgi->a({-href => href(action=>"blob", hash=>$diff{'to_id'},
hash_base=>$hash, file_name=>$diff{'file'}),
- -class => "list"}, esc_html($diff{'file'}));
+ -class => "list"}, esc_path($diff{'file'}));
print "</td>\n";
print "<td>$mode_chnge</td>\n";
print "<td class=\"link\">";
@@ -1979,11 +2009,11 @@ sub git_difftree_body {
print "<td>" .
$cgi->a({-href => href(action=>"blob", hash_base=>$hash,
hash=>$diff{'to_id'}, file_name=>$diff{'to_file'}),
- -class => "list"}, esc_html($diff{'to_file'})) . "</td>\n" .
+ -class => "list"}, esc_path($diff{'to_file'})) . "</td>\n" .
"<td><span class=\"file_status $nstatus\">[$nstatus from " .
$cgi->a({-href => href(action=>"blob", hash_base=>$parent,
hash=>$diff{'from_id'}, file_name=>$diff{'from_file'}),
- -class => "list"}, esc_html($diff{'from_file'})) .
+ -class => "list"}, esc_path($diff{'from_file'})) .
" with " . (int $diff{'similarity'}) . "% similarity$mode_chng]</span></td>\n" .
"<td class=\"link\">";
if ($diff{'to_id'} ne $diff{'from_id'}) {
@@ -2113,7 +2143,7 @@ sub git_patchset_body {
$file ||= $diffinfo->{'file'};
$file = $cgi->a({-href => href(action=>"blob", hash_base=>$hash_parent,
hash=>$diffinfo->{'from_id'}, file_name=>$file),
- -class => "list"}, esc_html($file));
+ -class => "list"}, esc_path($file));
$patch_line =~ s|a/.*$|a/$file|g;
print "<div class=\"diff from_file\">$patch_line</div>\n";
@@ -2125,7 +2155,7 @@ sub git_patchset_body {
$file ||= $diffinfo->{'file'};
$file = $cgi->a({-href => href(action=>"blob", hash_base=>$hash,
hash=>$diffinfo->{'to_id'}, file_name=>$file),
- -class => "list"}, esc_html($file));
+ -class => "list"}, esc_path($file));
$patch_line =~ s|b/.*|b/$file|g;
print "<div class=\"diff to_file\">$patch_line</div>\n";
@@ -3373,8 +3403,8 @@ sub git_blobdiff {
} else {
while (my $line = <$fd>) {
- $line =~ s!a/($hash|$hash_parent)!'a/'.esc_html($diffinfo{'from_file'})!eg;
- $line =~ s!b/($hash|$hash_parent)!'b/'.esc_html($diffinfo{'to_file'})!eg;
+ $line =~ s!a/($hash|$hash_parent)!'a/'.esc_path($diffinfo{'from_file'})!eg;
+ $line =~ s!b/($hash|$hash_parent)!'b/'.esc_path($diffinfo{'to_file'})!eg;
print $line;
@@ -3729,7 +3759,7 @@ sub git_search {
print $cgi->a({-href => href(action=>"blob", hash_base=>$co{'id'},
hash=>$set{'id'}, file_name=>$set{'file'}),
-class => "list"},
- "<span class=\"match\">" . esc_html($set{'file'}) . "</span>") .
+ "<span class=\"match\">" . esc_path($set{'file'}) . "</span>") .
"<br/>\n";
}
print "</td>\n" .
@@ -3863,7 +3893,7 @@ XML
if (!($line =~ m/^:([0-7]{6}) ([0-7]{6}) ([0-9a-fA-F]{40}) ([0-9a-fA-F]{40}) (.)([0-9]{0,3})\t(.*)$/)) {
next;
}
- my $file = esc_html(unquote($7));
+ my $file = esc_path(unquote($7));
$file = to_utf8($file);
print "$file<br/>\n";
}
--
1.4.3.3
^ permalink raw reply related
* [PATCH 2/n] gitweb: Use '&iquot;' instead of '?' in esc_path
From: Jakub Narebski @ 2006-10-30 18:59 UTC (permalink / raw)
To: git
In-Reply-To: <200610301953.01875.jnareb@gmail.com>
Use "&iquot;" Latin 1 entity ("¿" -- inverted question mark =
turned question mark, U+00BF ISOnum) instead '?' as replacements for
control characters and other undisplayable characters.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
gitweb/gitweb.perl | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index a15e916..edca27d 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -567,7 +567,7 @@ sub esc_html {
sub esc_path {
my $str = shift;
$str = esc_html($str);
- $str =~ s/[[:cntrl:]\a\b\e\f\n\r\t\011]/?/g; # like --hide-control-chars in ls
+ $str =~ s/[[:cntrl:]\a\b\e\f\n\r\t\011]/¿/g; # like --hide-control-chars in ls
return $str;
}
--
1.4.3.3
^ permalink raw reply related
* [PATCH 2/2] git push: add verbose flag and allow overriding of default target repository
From: Linus Torvalds @ 2006-10-30 16:28 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0610300823250.25218@g5.osdl.org>
This adds a command line flag "-v" to enable a more verbose mode, and
"--repo=" to override the default target repository for "git push" (which
otherwise always defaults to "origin").
This, together with the patch to allow dashes in config variable names,
allows me to do
[alias]
push-all = push -v --repo=all
in my user-global config file, and then I can (for any project I maintain)
add to the project-local config file
[remote "all"]
url=one.target.repo:/directory
url=another.target:/pub/somewhere/else
and now "git push-all" just updates all the target repositories, and shows
me what it does - regardless of which repo I am in.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
Maybe this is just useful to me? I dunno. I have long had a per-repository
"push-all" script that I have in the root directory of the repos I
maintain, but this is much more practical. With this, I can do things like
git push-all --tags
and it will push all the new tags to all the public repositories I
maintain for that particular repository.
Special Linus-only behaviour? Maybe. On the other hand, I think this is a
pretty clean patch regardless.
diff --git a/builtin-push.c b/builtin-push.c
index f5150ed..3151fb7 100644
--- a/builtin-push.c
+++ b/builtin-push.c
@@ -10,7 +10,7 @@
static const char push_usage[] = "git-push [--all] [--tags] [-f | --force] <repository> [<refspec>...]";
-static int all, tags, force, thin = 1;
+static int all, tags, force, thin = 1, verbose;
static const char *execute;
#define BUF_SIZE (2084)
@@ -248,6 +248,8 @@ static int do_push(const char *repo)
while (dest_refspec_nr--)
argv[dest_argc++] = *dest_refspec++;
argv[dest_argc] = NULL;
+ if (verbose)
+ fprintf(stderr, "Pushing to %s\n", dest);
err = run_command_v(argc, argv);
if (!err)
continue;
@@ -281,6 +283,14 @@ int cmd_push(int argc, const char **argv
i++;
break;
}
+ if (!strcmp(arg, "-v")) {
+ verbose=1;
+ continue;
+ }
+ if (!strncmp(arg, "--repo=", 7)) {
+ repo = arg+7;
+ continue;
+ }
if (!strcmp(arg, "--all")) {
all = 1;
^ permalink raw reply related
* [PATCH 1/2] Allow '-' in config variable names
From: Linus Torvalds @ 2006-10-30 16:25 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
I need this in order to allow aliases of the same form as "ls-tree",
"rev-parse" etc, so that I can use
[alias]
my-cat=--paginate cat-file -p
to add a "git my-cat" command.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
This will be followed by another patch to "builtin-push.c" to allow me to
make a useful (to me) "push-all" alias.
diff --git a/config.c b/config.c
index e8f0caf..3cae390 100644
--- a/config.c
+++ b/config.c
@@ -103,6 +103,11 @@ static char *parse_value(void)
}
}
+static inline int iskeychar(int c)
+{
+ return isalnum(c) || c == '-';
+}
+
static int get_value(config_fn_t fn, char *name, unsigned int len)
{
int c;
@@ -113,7 +118,7 @@ static int get_value(config_fn_t fn, cha
c = get_next_char();
if (c == EOF)
break;
- if (!isalnum(c))
+ if (!iskeychar(c))
break;
name[len++] = tolower(c);
if (len >= MAXNAME)
@@ -181,7 +186,7 @@ static int get_base_var(char *name)
return baselen;
if (isspace(c))
return get_extended_base_var(name, baselen, c);
- if (!isalnum(c) && c != '.')
+ if (!iskeychar(c) && c != '.')
return -1;
if (baselen > MAXNAME / 2)
return -1;
@@ -573,7 +578,7 @@ int git_config_set_multivar(const char*
dot = 1;
/* Leave the extended basename untouched.. */
if (!dot || i > store.baselen) {
- if (!isalnum(c) || (i == store.baselen+1 && !isalpha(c))) {
+ if (!iskeychar(c) || (i == store.baselen+1 && !isalpha(c))) {
fprintf(stderr, "invalid key: %s\n", key);
free(store.key);
^ permalink raw reply related
* Re: Progress reporting (was: VCS comparison table)
From: Nicolas Pitre @ 2006-10-30 15:21 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <ei4jia$vj0$1@sea.gmane.org>
On Mon, 30 Oct 2006, Jakub Narebski wrote:
> I was bitten lately by git lack of progress reporting for git-push.
> While it nicely reports local progress (generating data) it unfortunately
> lacks wget like, "curl -o" like or scp like pack upload progress
> reporting. And while usually push is fast, initial push of whole
> project to empty repository can be quite slow on low-bandwidth link
> (or busy network).
What about this patch?
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 41e1e74..7f87ae8 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1524,6 +1524,10 @@ int cmd_pack_objects(int argc, const cha
progress = 1;
continue;
}
+ if (!strcmp("--all-progress", arg)) {
+ progress = 2;
+ continue;
+ }
if (!strcmp("--incremental", arg)) {
incremental = 1;
continue;
@@ -1641,7 +1645,7 @@ int cmd_pack_objects(int argc, const cha
else {
if (nr_result)
prepare_pack(window, depth);
- if (progress && pack_to_stdout) {
+ if (progress == pack_to_stdout) {
/* the other end usually displays progress itself */
struct itimerval v = {{0,},};
setitimer(ITIMER_REAL, &v, NULL);
diff --git a/send-pack.c b/send-pack.c
index 0e90548..9280481 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -30,6 +30,7 @@ static void exec_pack_objects(void)
{
static const char *args[] = {
"pack-objects",
+ "--all-progress",
"--stdout",
NULL
^ permalink raw reply related
* Re: [PATCH] Bash snippet to show branch and patch in bash prompt
From: Eran Tromer @ 2006-10-30 15:03 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <20061030105926.14328.55180.stgit@lathund.dewire.com>
On 2006-10-30 12:59, Robin Rosenberg wrote:
> From: Robin Rosenberg <robin.rosenberg@dewire.com>
> + function stgtag
> + {
> + git_dir=$(git-rev-parse --git-dir 2> /dev/null)
> + ref=$(git-symbolic-ref HEAD 2> /dev/null)
Abort early if we're not in a git repo:
git_dir=$(git-rev-parse --git-dir 2> /dev/null) || return
ref=$(git-symbolic-ref HEAD 2> /dev/null) || return
> + br=${ref/refs\/heads\//}
You want to strip a prefix only, so this is safer:
br=${ref#refs/heads/}
> + top=$(cat $git_dir/patches/$br/current 2>/dev/null)
All variables should be declared local to avoid polluting the bash
variable namespace. Likewise, the function name deserves a couple of
underscores.
> + if [[ -n "$br$top" ]];then
> + echo "[$top@$br]"
It seems better to put the StGIT top after the tag, so that stg push/pop
shifts less of the prompt, making it easier to see the change visually.
Corresponding modified version:
-------------------------------------------
if [ "$PS1" ]; then
function __prompt_git()
{
local git_dir ref br top;
git_dir=$(git-rev-parse --git-dir 2> /dev/null) || return
ref=$(git-symbolic-ref HEAD 2> /dev/null) || return
br=${ref#refs/heads/}
top=$(cat $git_dir/patches/$br/current 2>/dev/null) \
&& top="#$top"
echo "[$br$top]"
}
PS1='\u@\h$(__prompt_git)\w\$ '
fi
-------------------------------------------
Conditionally prepending the "#" to $top can be done more concisely via
${top:+#$top} but I used the more readable version.
^ permalink raw reply
* Re: [PATCH/RFC (take 2)] gitweb: New improved patchset view
From: Jakub Narebski @ 2006-10-30 13:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Luben Tuikov, git
In-Reply-To: <200610301043.22033.jnareb@gmail.com>
Jakub Narebski wrote:
> Junio C Hamano wrote:
>>
>> I think the UI layer like gitweb should have freedom to choose
>> its own pathname handling, and should read from -z output.
>
> That's a very good idea. I'll send separate patch (if noone else will
> do this, that is) which would convert gitweb to always use -z output,
> both git-ls-tree and git-diff-tree... oh, well, there is no -z patch
> output, so in patch part we would have to replace git quoted part by
> gitweb quoted part.
I have realized that it is not as easy as it sounds, at least for the
git-diff-tree output. For the LF-terminated output (without '-z') you
know that LF separates records, and you can split on LF ('\n'). It is
not the case for '-z' '\0' delimited output: NUL ('\0') might also mean
end of one of the filenames in the rename/copy case, and is used to
separate filename(s) from the score (although here TAB would be
enough). And that is probably the case that gitweb uses default
git-diff-tree output, and _tries_ to unescape(...) filename.
The solution would be perhaps to add '--zz' option to use '-z' output
but to separate records by double NUL, i.e. '\0\0'...
By the way, why diff-tree "raw" format for merge gives only one, final,
filename?
--
Jakub Narebski
^ permalink raw reply
* [PATCH] Bash snippet to show branch and patch in bash prompt
From: Robin Rosenberg @ 2006-10-30 10:59 UTC (permalink / raw)
To: Catalin Marinas; +Cc: Eran Tromer, git
In-Reply-To: <4545CC6F.90001@tromer.org>
From: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
---
contrib/stgbashprompt.sh | 18 ++++++++++++++++++
1 files changed, 18 insertions(+), 0 deletions(-)
diff --git a/contrib/stgbashprompt.sh b/contrib/stgbashprompt.sh
new file mode 100755
index 0000000..a79561e
--- /dev/null
+++ b/contrib/stgbashprompt.sh
@@ -0,0 +1,18 @@
+# include this in your bashrc or copy to /etc/bash_completions.d
+
+if [ "$PS1" ]; then
+ # trap 'PS1="\u@\h [$(stg top)] \w]\$ "' DEBUG
+ function stgtag
+ {
+ git_dir=$(git-rev-parse --git-dir 2> /dev/null)
+ ref=$(git-symbolic-ref HEAD 2> /dev/null)
+ br=${ref/refs\/heads\//}
+ top=$(cat $git_dir/patches/$br/current 2>/dev/null)
+ if [[ -n "$br$top" ]];then
+ echo "[$top@$br]"
+ return
+ fi
+ }
+ PS1='\u@\h$(stgtag)\w\$ '
+
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox