* Re: How do I clear the directory cache
From: eschvoca @ 2005-10-21 20:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vmzl34ikl.fsf@assigned-by-dhcp.cox.net>
On 10/21/05, Junio C Hamano <junkio@cox.net> wrote:
> eschvoca <eschvoca@gmail.com> writes:
>
> > Yes, ... is what I want (how do I do this with pure git?).
>
> How do you do *what* with pure git is something not clear to me,
> so let me try to rephrase you and see if I understood what you
> want correctly, in pure git terms:
>
> 0. Your last commit (.git/HEAD), your index file (.git/index)
> and the files in your working tree are in sync to begin
> with.
>
> 1. Then you do random changes in the working tree (add, modify,
> remove) and do git-update-index on them without making a
> commit. If your index file is fully in sync with your
> working tree, git-diff-files would say nothing after this.
> If you did git-update-index on some but not all, then
> git-diff-files would report unrecorded changes.
>
> 2. But you want to revert these changes to the index file. By
> "clearing the directory cache", you mean you want the index
> file to have what it had in step 0, but you want to keep
> what is on the filesystem intact.
>
> If that is what you want, then:
>
> $ git-read-tree HEAD
> $ git-update-index --refresh >/dev/null
Yes, this does exaclty what I want. Thankyou.
> would reset the index to the tree recorded in your last commit.
> You can see that
>
> $ git diff --name-status --cached HEAD
>
> does not report any difference between HEAD and your index, while
>
> $ git diff --name-status
>
> shows the differences between your index and your working tree.
>
> HOWEVER, the index file does not record "intent to add", so you
> would not see files you added in step 1. mentioned in the output
> of the last command.
>
>
>
>
^ permalink raw reply
* Re: [PATCH] Do not call git-rev-list from git-fetch-pack
From: Daniel Barkalow @ 2005-10-21 20:25 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.63.0510211111440.4950@wbgn013.biozentrum.uni-wuerzburg.de>
On Fri, 21 Oct 2005, Johannes Schindelin wrote:
> Hi,
>
> On Fri, 21 Oct 2005, Junio C Hamano wrote:
>
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > > + Branch1 has 40 commits since it last merged with branch2,
> > > + and branch2 has 2 more commits since then.
> > > +
> > > + On the other end, branch1 is up-to-date, but branch2 is not.
> > > + When fetching branch1, the 40 commits get sent, the first
> > > + is acknowledged as common, and the local head of branch2
> > > + never gets sent.
> > > +
> > > + As a consequence, the whole history of branch2
> > > + (git-rev-list branch2 ^branch1) gets packed and transmitted.
> >
> > Arrrgggggggh. The description above captures the essense of the
> > problem very well, but faking dates and still popping by recency
> > looks like a really ugly hack to me.
>
> It is ugly. But it is a good heuristic. And the code is simple, because I
> could use the existing functions for inserting into a commit_list.
The existing functions aren't really very complicated. You could easily
separate out the pop and the adding of parents, and go first through a
list of tips and then through the date-sorted list of ancestors. And I
think that separation would be useful for some changes Linus was making to
rev-list recently.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: How do I clear the directory cache
From: eschvoca @ 2005-10-21 20:40 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20051021105235.GF30889@pasky.or.cz>
On 10/21/05, Petr Baudis <pasky@suse.cz> wrote:
> Dear diary, on Fri, Oct 21, 2005 at 05:23:28AM CEST, I got a letter
> where eschvoca <eschvoca@gmail.com> told me that...
> > Yes, "cg-reset --adds-removals" is what I want (how do I do this with
> > pure git?).
>
> git-read-tree HEAD
> git-update-cache --refresh
Thanks, it solves my problem.
> > I would like to clear/reset the index because I've screwed it all up.
> > I don't think I can do as you suggested because of the way I got into
> > this mess.
> >
> > I'm using git/cogito to version control my hard drive and I've been
> > gradually adding more entries into the .gitignore file because some
> > files change too frequently or I don't want them backed up. The OS
> > modified a bunch of files, I cg-rm'd 1/4 of them, then I changed my
> > mind and added them back, also did some genuine cg-adds, etc. and now
> > I'm all confused (it's a whole hard drive).
>
> Well, cg-status should show you what you effectively did, and then you
> could just do something like:
>
> cg-status -w | grep ^D | tr '\n' '\0' | xargs -0 cg-add
> cg-status -w | grep ^A | tr '\n' '\0' | xargs -0 cg-rm
This is the right idea (and what I like to do) but you would have to
remove the status flag (1st 2 chars) otherwise it gets included as part
of the filename. I think most shells have some pretty small command
length limits so i prefer to pipe into a loop rather than use xargs.
> > If other people are interested in doing this I can pass on the lessons
> > I learned.
> >
> > What I found it git is amazingly fast! cg-status only takes a few
> > seconds. I think there are some problems if you try to do:
> >
> > cd /
> > cg-add -r usr
> > cg-commit -m "take a long break"
> >
> > It seems that cg-add-ing and cg-commit-ing smaller chunks is faster
> > than one big chunk.
>
> Interesting. I cannot spot anything which would bog it down in Cogito.
> Is both cg-add and cg-commit significantly slower? (That is, if it takes
> longer than sum of the smaller chunks.) Perhaps it's a cache issue, not
> everything from the chunk fits into your cache during cg-add, so
> cg-commit has to reread it from the disk.
I'll do some tests when I have time; currently this is just a gut feeling.
> > I think commands for the following should be added to cogito:
>
> I'd prefer:
>
> > cg-status -<status_flag> # list files with given status flag (without
> > status flag in column 1)
> > git-ls-files [--others|--deleted|etc] --exclude-per-directory=/.gitignore
>
> All right, this might be useful. Implemented as cg-status -s '?' and such,
> thanks for the idea.
This is great but it would be easier to work with if there was another
switch to turn off printing
out the status flag. Otherwise you have to 'sed' or 'awk' out the
status flag which is a pain, especially when files have spaces in
them.
> > cg-add [-r] -<status_flag> # add files with a given status flag
> > git-ls-files [--others|--deleted|etc]
> > --exclude-per-directory=/.gitignore | while read i; do cg-add "$i;
> > done
> >
> > cg-rm [-r] -<status_flag> # you get the idea.
> >
> > cg-addremove # recursively add new files, remove deleted files
>
> I implemented the cg-reset --adds-removes option, but I don't feel
> comfortable with the cg-add change - just -r would be enough to add new
> files, and if you are in mess big enough, you can just cg-reset. It
> would be useful to make cg-rm symmetric to cg-add, though. Then you
> could do just
>
> cg-add -r . && cg-rm -r .
>
> and it would be equivalent to cg-addremove.
Yes, that would be better -- more modular without being too much extra.
Also, the "cg-add -r" exits when a file is bad. It would have saved
me a few hours if it would keep not exit on a failure (and add the bad
files to .gitignore ... but that is probably asking for too much).
> > I use "while read i" so it will work with spaces in filenames.
>
> And break with leading spaces unless you modified $IFS properly. Note
> that those people having filenames starting by spaces are either
> seriously sick or script kiddies who just rooted you (or warez kiddies
> on your FTP server with anonymously-writable incoming).
Yeah, spaces are a nuisance and quite useless at the beginning of a
file...CrossOver Office had some spaces in directory names.
^ permalink raw reply
* Re: Scribblings for a cogito/git tutorial
From: Petr Baudis @ 2005-10-21 20:51 UTC (permalink / raw)
To: Horst von Brand; +Cc: git, Martin Langhoff (CatalystIT)
In-Reply-To: <200510171504.j9HF4stb006164@laptop11.inf.utfsm.cl>
Dear diary, on Mon, Oct 17, 2005 at 05:04:54PM CEST, I got a letter
where Horst von Brand <vonbrand@inf.utfsm.cl> told me that...
> I've also been asked around here for a cogito+git tutorial, to that end
> I've made up a script that simulates several developers interacting.
> Hacking around is simulated by patching, ed(1) scripts (merges don't turn
> out the same diff every time), and plain copying new files in. I've set up
> a GPG key with an empty passphrase (comment is "Experimental") to have
> signed tags, etc. in a convenient manner. The idea is to create interesting
> histories (for browsing) and show off the commands in a compact way. If
> only there was a convenient way to run a strech of the (bash) script, look
> at the results, and then resume...
>
> Comments, suggestions, patches are welcome!
>
> Repository of the script and supporting files is at
> <http://pincoya.inf.utfsm.cl/Script.git>
Thanks, it's very nice! If you don't mind (actually, is it / can it be GPL?),
I added it to Cogito as Documentation/tutorial-script/ .
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: How do I clear the directory cache
From: Linus Torvalds @ 2005-10-21 21:00 UTC (permalink / raw)
To: eschvoca; +Cc: Junio C Hamano, git
In-Reply-To: <2b05065b0510211316j632c461apbb6f2f162e5fc7ef@mail.gmail.com>
On Fri, 21 Oct 2005, eschvoca wrote:
> >
> > If that is what you want, then:
> >
> > $ git-read-tree HEAD
> > $ git-update-index --refresh >/dev/null
>
>
> Yes, this does exaclty what I want. Thankyou.
Btw, if this ends up being something you do often (or more than once),
it's best to use the "--reset" flag to git-read-tree.
That will cause git-read-tree to still try to use any old usable entries
in the old index, which most of the time means that 99% of the index will
be up-to-date, and that in turn makes the index refresh _much_ faster.
For small projects it doesn't matter, of course.
In fact, those two lines can also be written simply as
git reset
which will do this, and more (in particular, it will also remove any sign
of a partial merge-head).
Linus
^ permalink raw reply
* Re: [PATCH] Do not call git-rev-list from git-fetch-pack
From: Johannes Schindelin @ 2005-10-21 21:08 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0510211618420.25300@iabervon.org>
Hi,
here comes a WIP on that patch (needs to be tested lots more):
fetch-pack.c | 154 ++++++++++++++++++++++++++++++++++++++++++++--------------
1 files changed, 117 insertions(+), 37 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 8566ab1..33ce435 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -13,18 +13,109 @@ static const char fetch_pack_usage[] =
static const char *exec = "git-upload-pack";
#define COMPLETE (1U << 0)
+#define COMMON (1U << 1)
+#define COMMON_REF (1U << 2 | COMMON)
+#define SEEN (1U << 3)
+#define POPPED (1U << 4)
+#define MAX_HAS (16)
+
+static struct commit_list *rev_list = NULL;
+static struct commit_list *rev_list_end = NULL;
+static unsigned long non_common_revs = 0;
+
+static void rev_list_append(struct commit *commit, int mark)
+{
+ if (!(commit->object.flags & mark)) {
+ commit->object.flags |= mark;
+
+ if (rev_list == NULL) {
+ commit_list_insert(commit, &rev_list);
+ rev_list_end = rev_list;
+ } else {
+ commit_list_insert(commit, &(rev_list_end->next));
+ rev_list_end = rev_list_end->next;
+ }
+
+ if (!(commit->object.flags & COMMON))
+ non_common_revs++;
+ }
+}
+
+static int rev_list_append_sha1(const char *path, const unsigned char *sha1)
+{
+ struct object *o = deref_tag(parse_object(sha1));
+
+ if (o->type == commit_type)
+ rev_list_append((struct commit *)o, SEEN);
+
+ return 0;
+}
+
+static void mark_common(struct object *o)
+{
+ if (o != NULL && !(o->flags & COMMON)) {
+ o->flags |= COMMON;
+ if (!(o->flags & SEEN))
+ rev_list_append((struct commit *)o, SEEN);
+ else if (!(o->flags & POPPED))
+ non_common_revs--;
+ }
+}
+
+/*
+ Get the next rev to send, ignoring the common.
+*/
+
+static const unsigned char* get_rev()
+{
+ struct commit *commit = NULL;
+
+ while (commit == NULL) {
+ unsigned int mark;
+ struct commit_list* parents;
+
+ if (rev_list == NULL || non_common_revs == 0)
+ return NULL;
+
+ commit = rev_list->item;
+ commit->object.flags |= POPPED;
+ if (!(commit->object.flags & COMMON))
+ non_common_revs--;
+
+ parents = commit->parents;
+
+ if (commit->object.flags & COMMON) {
+ /* do not send "have", and ignore ancestors */
+ commit = NULL;
+ mark = COMMON | SEEN;
+ } else if (commit->object.flags & COMMON_REF)
+ /* send "have", and ignore ancestors */
+ mark = COMMON | SEEN;
+ else
+ /* send "have", also for its ancestors */
+ mark = SEEN;
+
+ while (parents) {
+ if (mark & COMMON)
+ mark_common((struct object *)parents->item);
+ else
+ rev_list_append(parents->item, mark);
+ parents = parents->next;
+ }
+ }
+
+ return commit->object.sha1;
+}
static int find_common(int fd[2], unsigned char *result_sha1,
struct ref *refs)
{
int fetching;
- static char line[1000];
- static char rev_command[1024];
- int count = 0, flushes = 0, retval, rev_command_len;
- FILE *revs;
+ int count = 0, flushes = 0, ack_count = 0, retval;
+ const unsigned char *sha1;
+
+ for_each_ref(rev_list_append_sha1);
- strcpy(rev_command, "git-rev-list $(git-rev-parse --all)");
- rev_command_len = strlen(rev_command);
fetching = 0;
for ( ; refs ; refs = refs->next) {
unsigned char *remote = refs->old_sha1;
@@ -42,25 +133,15 @@ static int find_common(int fd[2], unsign
*/
if (((o = lookup_object(remote)) != NULL) &&
(o->flags & COMPLETE)) {
- struct commit_list *p;
- struct commit *commit =
- (struct commit *) (o = deref_tag(o));
- if (!o)
- goto repair;
- if (o->type != commit_type)
- continue;
- p = commit->parents;
- while (p &&
- rev_command_len + 44 < sizeof(rev_command)) {
- snprintf(rev_command + rev_command_len, 44,
- " ^%s",
- sha1_to_hex(p->item->object.sha1));
- rev_command_len += 43;
- p = p->next;
- }
+ o = deref_tag(o);
+
+ if (o->type == commit_type)
+ rev_list_append((struct commit *)o,
+ COMMON_REF | SEEN);
+
continue;
}
- repair:
+
packet_write(fd[1], "want %s\n", sha1_to_hex(remote));
fetching++;
}
@@ -68,21 +149,16 @@ static int find_common(int fd[2], unsign
if (!fetching)
return 1;
- revs = popen(rev_command, "r");
- if (!revs)
- die("unable to run 'git-rev-list'");
-
flushes = 1;
retval = -1;
- while (fgets(line, sizeof(line), revs) != NULL) {
- unsigned char sha1[20];
- if (get_sha1_hex(line, sha1))
- die("git-fetch-pack: expected object name, got crud");
+ while ((sha1 = get_rev())) {
packet_write(fd[1], "have %s\n", sha1_to_hex(sha1));
if (verbose)
fprintf(stderr, "have %s\n", sha1_to_hex(sha1));
if (!(31 & ++count)) {
packet_flush(fd[1]);
+ if (verbose)
+ fprintf(stderr, "flush\n");
flushes++;
/*
@@ -92,16 +168,19 @@ static int find_common(int fd[2], unsign
if (count == 32)
continue;
if (get_ack(fd[0], result_sha1)) {
- flushes = 0;
- retval = 0;
+ mark_common(lookup_object(result_sha1));
if (verbose)
- fprintf(stderr, "got ack\n");
- break;
+ fprintf(stderr, "got ack %s\n",
+ sha1_to_hex(result_sha1));
+ if (++ack_count > MAX_HAS) {
+ flushes = 0;
+ retval = 0;
+ break;
+ }
}
flushes--;
}
}
- pclose(revs);
packet_write(fd[1], "done\n");
if (verbose)
fprintf(stderr, "done\n");
@@ -109,7 +188,8 @@ static int find_common(int fd[2], unsign
flushes--;
if (get_ack(fd[0], result_sha1)) {
if (verbose)
- fprintf(stderr, "got ack\n");
+ fprintf(stderr, "got ack %s\n",
+ sha1_to_hex(result_sha1));
return 0;
}
}
^ permalink raw reply related
* Re: [PATCH] Do not call git-rev-list from git-fetch-pack
From: Junio C Hamano @ 2005-10-21 21:32 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0510212134570.5542@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Yes. This makes it more simple, and is very efficient in the common case.
> How about increasing MAX_HAS to 64?
The other day I saw HPA tried to sneak in a change to increase
it to 64, mingled with other changes. I think increasing it
makes sense.
Now the question is, if we count MAX_HAS on the fetch-pack side,
how would we coordinate that value with the real limit the other
end uses. I guess that would not matter too much. It would not
affect the correctness anyway.
^ permalink raw reply
* Re: How do I clear the directory cache
From: Petr Baudis @ 2005-10-21 21:43 UTC (permalink / raw)
To: eschvoca; +Cc: git
In-Reply-To: <2b05065b0510211340y9551767i53fe53c0dc14460a@mail.gmail.com>
Dear diary, on Fri, Oct 21, 2005 at 10:40:07PM CEST, I got a letter
where eschvoca <eschvoca@gmail.com> told me that...
> On 10/21/05, Petr Baudis <pasky@suse.cz> wrote:
> > Dear diary, on Fri, Oct 21, 2005 at 05:23:28AM CEST, I got a letter
> > where eschvoca <eschvoca@gmail.com> told me that...
> > > cg-status -<status_flag> # list files with given status flag (without
> > > status flag in column 1)
> > > git-ls-files [--others|--deleted|etc] --exclude-per-directory=/.gitignore
> >
> > All right, this might be useful. Implemented as cg-status -s '?' and such,
> > thanks for the idea.
>
> This is great but it would be easier to work with if there was another
> switch to turn off printing
> out the status flag. Otherwise you have to 'sed' or 'awk' out the
> status flag which is a pain, especially when files have spaces in
> them.
Not such a huge pain, but if we already have -s... I added -n which does
what you want.
> Also, the "cg-add -r" exits when a file is bad. It would have saved
> me a few hours if it would keep not exit on a failure (and add the bad
> files to .gitignore ... but that is probably asking for too much).
I don't like the idea of adding the files automagically to .gitignore,
but it won't abort the whole operation because of them anymore.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: Cogito RFE: cg-commit -q
From: Petr Baudis @ 2005-10-21 22:06 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: Git Mailing List
In-Reply-To: <434E9D50.2040807@zytor.com>
Dear diary, on Thu, Oct 13, 2005 at 07:45:52PM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> I would find it very useful if cg-commit had a "-q" option, meaning
> "silently skip this commit if there is nothing to commit." There are
> some automatic release scripts that I have which enforces consistency
> before release, but if the repository is already correctly set up for
> release, there is nothing to do.
>
> This is the opposite of -f, which would create a commit object pointing
> to the same tree.
Thanks for the idea, implemented.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* git-rev-list: add "--dense" flag
From: Linus Torvalds @ 2005-10-21 23:40 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
This is what the recent git-rev-list changes have all been gearing up for.
When we use a path filter to git-rev-list, the new "--dense" flag asks
git-rev-list to compress the history so that it _only_ contains commits
that change files in the path filter. It also rewrites the parent
information so that tools like "gitk" will see the result as a dense
history tree.
For example, on the current kernel archive:
[torvalds@g5 linux]$ git-rev-list HEAD | wc -l
9904
[torvalds@g5 linux]$ git-rev-list HEAD -- kernel | wc -l
5442
[torvalds@g5 linux]$ git-rev-list --dense HEAD -- kernel | wc -l
356
which shows that while we have almost ten thousand commits, we can prune
down the work to slightly more than half by only following the merges
that are interesting. But further, we can then compress the history to
just 356 entries that actually make changes to the kernel subdirectory.
To see this in action, try something like
gitk --dense -- gitk
to see just the history that affects gitk. Or, to show that true
parallell development still remains parallell, do
gitk --dense -- daemon.c
which shows some parallell commits in the current git tree.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---
I'm really happy with how this turned out. It's a bit expensive to run on
big archives, but I _really_ think it's quite spectacular. And likely very
useful indeed.
For example, say you love gitk, but only care about networking changes.
Easy enough, just do
gitk --dense -- net/ include/net/
and off you go. It's not free (we do a _lot_ of tree comparisons), but
dammit, it's fast enough that it's very very useful. The tree comparisons
are done very efficiently.
This is _way_ more powerful than annotate. Interested in just a single
file? Just do
gitk --dense -- kernel/exit.c
and it will show the 17 or so commits that change kernel/exit.c with the
right history (it turns out that there is no parallell development at all
in that file, so in this case it will linearize history entirely).
Damn, I'm good.
diff --git a/rev-list.c b/rev-list.c
index b5dbb9f..5f125fd 100644
--- a/rev-list.c
+++ b/rev-list.c
@@ -11,6 +11,7 @@
#define INTERESTING (1u << 1)
#define COUNTED (1u << 2)
#define SHOWN (1u << 3)
+#define TREECHANGE (1u << 4)
static const char rev_list_usage[] =
"git-rev-list [OPTION] commit-id <commit-id>\n"
@@ -27,6 +28,7 @@ static const char rev_list_usage[] =
" --merge-order [ --show-breaks ]\n"
" --topo-order";
+static int dense = 0;
static int unpacked = 0;
static int bisect_list = 0;
static int tag_objects = 0;
@@ -79,6 +81,26 @@ static void show_commit(struct commit *c
fflush(stdout);
}
+static void rewrite_one(struct commit **pp)
+{
+ for (;;) {
+ struct commit *p = *pp;
+ if (p->object.flags & (TREECHANGE | UNINTERESTING))
+ return;
+ /* Only single-parent commits don't have TREECHANGE */
+ *pp = p->parents->item;
+ }
+}
+
+static void rewrite_parents(struct commit *commit)
+{
+ struct commit_list *parent = commit->parents;
+ while (parent) {
+ rewrite_one(&parent->item);
+ parent = parent->next;
+ }
+}
+
static int filter_commit(struct commit * commit)
{
if (stop_traversal && (commit->object.flags & BOUNDARY))
@@ -95,6 +117,11 @@ static int filter_commit(struct commit *
return STOP;
if (no_merges && (commit->parents && commit->parents->next))
return CONTINUE;
+ if (paths && dense) {
+ if (!(commit->object.flags & TREECHANGE))
+ return CONTINUE;
+ rewrite_parents(commit);
+ }
return DO;
}
@@ -404,6 +431,14 @@ static struct diff_options diff_opt = {
.change = file_change,
};
+static int same_tree(struct tree *t1, struct tree *t2)
+{
+ is_different = 0;
+ if (diff_tree_sha1(t1->object.sha1, t2->object.sha1, "", &diff_opt) < 0)
+ return 0;
+ return !is_different;
+}
+
static struct commit *try_to_simplify_merge(struct commit *commit, struct commit_list *parent)
{
if (!commit->tree)
@@ -415,11 +450,7 @@ static struct commit *try_to_simplify_me
parse_commit(p);
if (!p->tree)
continue;
- is_different = 0;
- if (diff_tree_sha1(commit->tree->object.sha1,
- p->tree->object.sha1, "", &diff_opt) < 0)
- continue;
- if (!is_different)
+ if (same_tree(commit->tree, p->tree))
return p;
}
return NULL;
@@ -485,6 +516,27 @@ static void add_parents_to_list(struct c
}
}
+static void compress_list(struct commit_list *list)
+{
+ while (list) {
+ struct commit *commit = list->item;
+ struct commit_list *parent = commit->parents;
+ list = list->next;
+
+ /*
+ * Exactly one parent? Check if it leaves the tree
+ * unchanged
+ */
+ if (parent && !parent->next) {
+ struct tree *t1 = commit->tree;
+ struct tree *t2 = parent->item->tree;
+ if (!t1 || !t2 || same_tree(t1, t2))
+ continue;
+ }
+ commit->object.flags |= TREECHANGE;
+ }
+}
+
static struct commit_list *limit_list(struct commit_list *list)
{
struct commit_list *newlist = NULL;
@@ -514,6 +566,8 @@ static struct commit_list *limit_list(st
}
if (tree_objects)
mark_edges_uninteresting(newlist);
+ if (paths && dense)
+ compress_list(newlist);
if (bisect_list)
newlist = find_bisection(newlist);
return newlist;
@@ -700,6 +754,10 @@ int main(int argc, const char **argv)
limited = 1;
continue;
}
+ if (!strcmp(arg, "--dense")) {
+ dense = 1;
+ continue;
+ }
if (!strcmp(arg, "--")) {
paths = get_pathspec(prefix, argv + i + 1);
if (paths) {
^ permalink raw reply related
* Re: git-rev-list: add "--dense" flag
From: Linus Torvalds @ 2005-10-21 23:47 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211631400.10477@g5.osdl.org>
On Fri, 21 Oct 2005, Linus Torvalds wrote:
>
> parallell
Oh, please also fix my spelling. That's one 'l' too many at the end.
Damn, I think I'm a reasonably good speller most of the time, but this
time it shows that Swedish is my first language (where it really _is_
written that way, with four 'l's total).
Linus
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Linus Torvalds @ 2005-10-21 23:55 UTC (permalink / raw)
To: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211631400.10477@g5.osdl.org>
On Fri, 21 Oct 2005, Linus Torvalds wrote:
>
> [torvalds@g5 linux]$ git-rev-list --dense HEAD -- kernel | wc -l
> 356
Btw, one additional point.
This took 0.91 seconds to complete on the current kernel history on my
machine with a pretty much fully packed archive, and the memory footprint
was a total of about 12MB.
And it scales pretty well too. On the historical linux archive, which is
three years of history, the same thing takes me just over 12 seconds and
52MB, and that's for the _whole_ history. And it's not just following one
file: it's following that subdirectory.
So it really is pretty damn cool.
Of course, I might have a bug somewhere, but it all _seems_ to work very
well indeed.
Linus
^ permalink raw reply
* Re: [PATCH] Do not call git-rev-list from git-fetch-pack
From: Johannes Schindelin @ 2005-10-22 0:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vbr1iv9nz.fsf@assigned-by-dhcp.cox.net>
Hi,
On Fri, 21 Oct 2005, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > Yes. This makes it more simple, and is very efficient in the common case.
> > How about increasing MAX_HAS to 64?
>
> The other day I saw HPA tried to sneak in a change to increase
> it to 64, mingled with other changes. I think increasing it
> makes sense.
Agree.
However, I do not see a way for upload-pack to force fetch-pack to send
"have" lines. Maybe this would be an important change to the protocol? I
imagine that I could craft a custom fetch-pack which DoS'es any
current upload-pack.
> Now the question is, if we count MAX_HAS on the fetch-pack side,
> how would we coordinate that value with the real limit the other
> end uses. I guess that would not matter too much. It would not
> affect the correctness anyway.
Really, there should be a way for upload-pack to say that enough is
enough, and it wants to send the pack now.
Ciao,
Dscho
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Petr Baudis @ 2005-10-22 0:37 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211631400.10477@g5.osdl.org>
Dear diary, on Sat, Oct 22, 2005 at 01:40:54AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> When we use a path filter to git-rev-list, the new "--dense" flag asks
> git-rev-list to compress the history so that it _only_ contains commits
> that change files in the path filter. It also rewrites the parent
> information so that tools like "gitk" will see the result as a dense
> history tree.
There is no documentation for the --dense flag and it is even missing
from the usage string.
But my main concern is - will it be possible to do the rename detection
here as well? Using --dense instead of explicit diff-tree calls in
cg-log would be nice optimization, but I was about to add support for
optional following of renames for cg-log <filename>. That's really
pretty useful to have, every time I hit the Junio's big scripts rename
I keep repeating that to myself. ;-) Now when core GIT got the comfort
of per-file history, I only hope that it will start to annoy you as well.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Handling renames.
From: Petr Baudis @ 2005-10-22 0:47 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20051022003733.GA8351@pasky.or.cz>
(Having Apr 14 flashbacks? Good memory!)
Dear diary, on Sat, Oct 22, 2005 at 02:37:33AM CEST, I got a letter
where Petr Baudis <pasky@suse.cz> told me that...
> But my main concern is - will it be possible to do the rename detection
> here as well? Using --dense instead of explicit diff-tree calls in
> cg-log would be nice optimization, but I was about to add support for
> optional following of renames for cg-log <filename>. That's really
> pretty useful to have, every time I hit the Junio's big scripts rename
> I keep repeating that to myself. ;-) Now when core GIT got the comfort
> of per-file history, I only hope that it will start to annoy you as well.
After all, this might be as good time to bring this up as any. Heavy
post-1.0 material follows:
How to track renames? I believe the situation has changed in the last
half a year. GIT really is a full-fledged SCM by now (at least its
major part code-wise), and I think it's hopefully becoming obvious that
we need to track renames. I actually decided to skip the whole
discussion why so, because we already _do_ concern ourselves with
renames - that's what the cool diff -M gadget does. And people start to
want to use it for all kind of history-digging stuff (not just for nice
diffs between trees).
So the problem is whether we should make this explicit. diff -M is
only a heuristic and it can go wrong, while it was empirically found
out in other SCMs that people actually don't mind telling their SCM
about renames explicitly - no more than telling it about adds and
removals explicitly. So the user is willing to tell us what precisely
happened and it would be foolish to throw that away and insist on
guessing. Besides, guessing (and even doing that everytime we go
through the history) is fundamentally slow, orders of magnitude more
than just a tree diff.
If I convince you that it is worth tracking the renames explicitly,
"how" is already a minor question. One idea of mine was to add an "edge"
object describing the edge between two trees (that's optimized for
flexible use - some people use GIT for really weird things and perhaps
do not use commits at all; edge between two commits would be optimized
for flexibility in case we will later think of some other cool stuff to
track at the edge):
trees c53f757133bb84a2d87e901c49207e9b7c48e1a6 6bc7aa4f652d0ef49108d9e30a7ea7fbf8e44639
rename git-pull-script\0git-pull.sh\0
copy somefile1\0somefile2\0
rewrite anotherfile1\0anotherfile2\0
(The "rewrite" line might be controversial. And you might want to
merge this with the delta objects during packing, or do something
similarly clever.)
Then you could e.g. pass the edge object ID as the second ID on the
parent line:
parent 99977bd5fdeabbd0608a70e9411c243007ec4ea2 edgeobjectid
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Linus Torvalds @ 2005-10-22 1:26 UTC (permalink / raw)
To: Petr Baudis; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20051022003733.GA8351@pasky.or.cz>
On Sat, 22 Oct 2005, Petr Baudis wrote:
>
> There is no documentation for the --dense flag and it is even missing
> from the usage string.
I'm not much for docs ;)
The whole path argument also isn't even there (and without paths, --dense
doesn't matter)
> But my main concern is - will it be possible to do the rename detection
> here as well?
Yes. Note that git-rev-list doesn't actually _do_ the diff, it only checks
whether the tree is changed. You still just get a list of commits out of
it, and it is up to you to decide what to do with it.
If you're just going to feed them to diff-tree _anyway_, then you might as
well not even do the dense thing, because quite frankly, you're just doing
extra work.
But let's say that you want to follow a certain filename, what you can do
is basically (fake shell syntax with "goto restart")
rev=HEAD
restart:
git-rev-list $rev --dense --parents -- "$filename" |
while read commit parent1 restofparents
do
if [ "$restofparents" ]; then
.. it's a merge, do whatever it is you do
with merges ..
else
shaold=$(git-ls-tree $parent -- "$filename")
shanew=$(git-ls-tree $commit -- "$filename")
# Did it disappear?
# Maybe it got renamed from something
if [ -z "$shaold" ]; then
old=$(git-diff-tree -M -r $commit |
grep " R .* $filename")
echo "rename? $old"
filename=figure-it-out-from-$old
rev=$parent
goto restart
fi
git-diff-tree -p $commit -- "$filename"
fi
while
or something similar.
In other words: you'll basically have to figure out the renames on your
own, and follow the renaming, but something like the above should do it.
Linus
^ permalink raw reply
* Re: Handling renames.
From: Linus Torvalds @ 2005-10-22 1:28 UTC (permalink / raw)
To: Petr Baudis; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20051022004743.GN30889@pasky.or.cz>
On Sat, 22 Oct 2005, Petr Baudis wrote:
>
> How to track renames? I believe the situation has changed in the last
> half a year.
I disagree.
Every single thing that said that renames were a bad idea to track when
git started is still equally true.
> If I convince you that it is worth tracking the renames explicitly,
> "how" is already a minor question.
Never. I'm 100% convinced that tracking renames is WRONG WRONG WRONG.
You can follow renames _afterwards_.
Git tracks contents. And I think we've proven that figuring out renames
after-the-fact from those contents is not only doable, but very well
supported already.
I'm convinced that git handles renames better than any other SCM ever.
Exactly because we figure it out when it matters.
Linus
^ permalink raw reply
* Re: Handling renames.
From: Petr Baudis @ 2005-10-22 1:51 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211826350.10477@g5.osdl.org>
> Every single thing that said that renames were a bad idea to track
> when git started is still equally true.
It'd be good to clarify whether we discuss whether the idea of tracking
renames is good or bad, or whether having the user explicitly specify
renames is better than figuring that out automagically. Your first
comment would indicate the former, but the rest of your reply the
latter.
> You can follow renames _afterwards_.
I can - crudely, but what's the point, if the user is dying to give me
the information.
> Git tracks contents. And I think we've proven that figuring out renames
> after-the-fact from those contents is not only doable, but very well
> supported already.
It's unreliable and it's slow (well, perhaps I should get some numbers
to back that out, but given how it is done I take it for granted). Does
not sound too "very well" to me.
> I'm convinced that git handles renames better than any other SCM ever.
> Exactly because we figure it out when it matters.
It matters at least every time you show per-file history and every time
you merge cross the rename. I think that can be both pretty common if
you ever do the rename. That means you can do an expensive guess
every time you hit that, and the guess can get it wrong, in which case
there is no way around that and you lose.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: Handling renames.
From: Junio C Hamano @ 2005-10-22 2:10 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20051022015109.GO30889@pasky.or.cz>
Petr Baudis <pasky@suse.cz> writes:
>> I'm convinced that git handles renames better than any other SCM ever.
>> Exactly because we figure it out when it matters.
>
> It matters at least every time you show per-file history and every time
> you merge cross the rename. I think that can be both pretty common if
> you ever do the rename. That means you can do an expensive guess
> every time you hit that, and the guess can get it wrong, in which case
> there is no way around that and you lose.
I think it is OK for the higher level layer (like your
single-file-history follower) to use what you outlined with
"edges", as either a hint/request from the user and/or a cache
of what the expensive and unreliable thing figured out.
I however do not necessarily think adding the "edges"
information as an optional second item on "parent " line in a
commit is a good idea. Rather, treating the set of "edges" just
like we treat the grafts feel more appropriate to me. IOW,
create .git/info/edges and keep your edge information there.
^ permalink raw reply
* Re: Handling renames.
From: Petr Baudis @ 2005-10-22 2:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vbr1is3m4.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Sat, Oct 22, 2005 at 04:10:59AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> I think it is OK for the higher level layer (like your
> single-file-history follower) to use what you outlined with
> "edges", as either a hint/request from the user and/or a cache
> of what the expensive and unreliable thing figured out.
>
> I however do not necessarily think adding the "edges"
> information as an optional second item on "parent " line in a
> commit is a good idea.
Well, it's obviously unflying idea when it doesn't become core part of
GIT. ;-) That won't stop me with Cogito, though...
> Rather, treating the set of "edges" just
> like we treat the grafts feel more appropriate to me. IOW,
> create .git/info/edges and keep your edge information there.
The thing is, you aren't supposed to have a lot of grafts, they are
expected to be rare. OTOH, renames aren't really all that rare, and as
the history goes, you'll accumulate a lot of them. Besides, it's not
just a cache - in case the user manually recorded the rename, it's an
important historical information, as important as e.g. the commit
message. It should be also distributed with the rest of the history
data, not be a per-repository thing. (Actually, this does not mix with
the cache at all - that should be a separate thing, quite possibly in
info/.)
But if GIT won't do this, I will have to do this in some Cogito-specific
way. My preliminary plan is appending something like this to the commit
message:
---
!%$rename foo\x20bar baz\nquux
!%$ being some arbitrary magic literal, and filter this out from cg-log
and such. Also, all Cogito commits adding/removing files but not
renaming any would have a '!%$norenames' line in the commit message, so
that I know that I don't have to do the expensive heuristics because no
rename/copy happened.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Petr Baudis @ 2005-10-22 2:56 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211814050.10477@g5.osdl.org>
Dear diary, on Sat, Oct 22, 2005 at 03:26:27AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> On Sat, 22 Oct 2005, Petr Baudis wrote:
> > There is no documentation for the --dense flag and it is even missing
> > from the usage string.
>
> I'm not much for docs ;)
>
> The whole path argument also isn't even there (and without paths, --dense
> doesn't matter)
Then it should be added to the docs as well. ;-)
Oh well, I guess I can always send a patch when yours will get applied. :-)
> > But my main concern is - will it be possible to do the rename detection
> > here as well?
>
> Yes. Note that git-rev-list doesn't actually _do_ the diff, it only checks
> whether the tree is changed. You still just get a list of commits out of
> it, and it is up to you to decide what to do with it.
Ok, fair enough.
> If you're just going to feed them to diff-tree _anyway_, then you might as
> well not even do the dense thing, because quite frankly, you're just doing
> extra work.
fork() is awfully expensive for me. fork()ing something (supposedly)
trivial (it was stat) slowed my (non-trivial) loop about 4 times. So I
think it will still pay off hugely if I will be able do the diff-tree
only on the interesting commits.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: Handling renames.
From: Linus Torvalds @ 2005-10-22 3:23 UTC (permalink / raw)
To: Petr Baudis; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20051022015109.GO30889@pasky.or.cz>
On Sat, 22 Oct 2005, Petr Baudis wrote:
>
> > You can follow renames _afterwards_.
>
> I can - crudely, but what's the point, if the user is dying to give me
> the information.
No the user is NOT.
The fact is, users have not a frigging clue when a rename happens.
I told you before, I'll tell you again: if you depend on users telling you
about renames, you'll get it wrong. You'll get it wrong quite often, in
fact.
This is not something I'm going to discuss again. Go back to all the same
arguments from 6 months ago. I was right then, I'm right now.
Linus
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Linus Torvalds @ 2005-10-22 3:30 UTC (permalink / raw)
To: Petr Baudis; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <20051022025638.GQ30889@pasky.or.cz>
On Sat, 22 Oct 2005, Petr Baudis wrote:
>
> > If you're just going to feed them to diff-tree _anyway_, then you might as
> > well not even do the dense thing, because quite frankly, you're just doing
> > extra work.
>
> fork() is awfully expensive for me. fork()ing something (supposedly)
> trivial (it was stat) slowed my (non-trivial) loop about 4 times.
Not fork. More like
git-rev-list .. | git-diff-tree --stdin
If you do it that way (which is quite common - git-whatchanged, git-log
etc), there's no reason to ask for a dense output, because git-diff-tree
can handle the non-dense one as fast as git-rev-list can.
But the --dense option is wonderful for "gitk", and for "slow stuff".
Where "slow" can indeed be a fork, or just an interpreter loop like a
shell/perl script, which would fork/exec git-diff-tree for each commit.
>From a commit-list standpoint
git-rev-list --dense ... -- paths
is the same as
git-rev-list -- paths | git-diff-tree -s -r --stdin -- paths
but it's somewhat more efficient, and more importantly, it rewrites the
parents. So quite often, you'd want to use "--dense" together with
"--parent" so that you get the "rewritten" history. That's how come gitk
didn't need any changes, and it "just worked". It's also how my example
script did the rename detection without having to traverse unnecessary
commits.
Linus
^ permalink raw reply
* Re: git-rev-list: add "--dense" flag
From: Junio C Hamano @ 2005-10-22 3:55 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0510211814050.10477@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> Yes. Note that git-rev-list doesn't actually _do_ the diff, it only checks
> whether the tree is changed. You still just get a list of commits out of
> it, and it is up to you to decide what to do with it.
>
> If you're just going to feed them to diff-tree _anyway_, then you might as
> well not even do the dense thing, because quite frankly, you're just doing
> extra work.
>
> But let's say that you want to follow a certain filename, what you can do
> is basically (fake shell syntax with "goto restart")
>...
> or something similar.
Isn't that only true because you are not doing more than "have
these paths change" in the new rev-list that already has part of
diff-tree?
If rev-list can optionally be told to detect renames internally
(it has necessary bits after all), it could adjust the set of
paths to follow when it sees something got renamed, either by
replacing the original path given from the command line with its
previous name, or adding its previous name to the set of path
limitters (to cover the copy case as well).
^ permalink raw reply
* [PATCH] Allow caching of generated pack for full cloning.
From: Junio C Hamano @ 2005-10-22 9:00 UTC (permalink / raw)
To: git; +Cc: H Peter Anvin
git-pack-objects can reuse pack files stored in $GIT_DIR/pack-cache
directory, when a necessary pack is found. This is hopefully useful
when upload-pack (called from git-daemon) is expected to receive
requests for the same set of objects many times (e.g full cloning
request of any project, or updates from the set of heads previous day
to the latest for a slow moving project).
Currently git-pack-objects does *not* keep pack files it creates for
reusing. It might be useful to implement its --update-cache option,
which would let it store pack files it created in the pack-cache
directory, and prune rarely used ones from it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
Right now, this is not very useful except perhaps preparing
for many clone requests by statically creating and storing a
full pack in pack-cache directory. I am expecting that
enabling the unimplemented --update-cache option of
git-pack-objects would let the server keep recently generated
packs, hoping that fetch requests close together would be for
the same "master" head, relative to the same previous heads
(multiple people making a habit of pulling every day, or every
week, or whatever). These cached packs need to be purged from
the pack-cache directory quite often. They would become
useless once you update a popular ref in the repository.
Even if this caching would help git-daemon by reusing
generated packs, I see one potential problem; --update-cache
option would require the process to be able to write into the
pack-cache directory, but I expect git-daemon would run as a
user that does not have any write privilege to the filesystem.
Makefile | 2 +
cache.h | 1 +
copy.c | 37 +++++++++++++++++++++++
pack-objects.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++++-------
upload-pack.c | 12 ++++++-
5 files changed, 126 insertions(+), 16 deletions(-)
create mode 100644 copy.c
applies-to: a0d57ba5b9245eb3a4cc15fb029af51a40eb8136
dd42e422104f43b369929c4f900362d401d2e962
diff --git a/Makefile b/Makefile
index 903c57c..3d8503d 100644
--- a/Makefile
+++ b/Makefile
@@ -159,7 +159,7 @@ LIB_OBJS = \
object.o pack-check.o patch-delta.o path.o pkt-line.o \
quote.o read-cache.o refs.o run-command.o \
server-info.o setup.o sha1_file.o sha1_name.o strbuf.o \
- tag.o tree.o usage.o config.o environment.o ctype.o \
+ tag.o tree.o usage.o config.o environment.o ctype.o copy.o \
$(DIFF_OBJS)
LIBS = $(LIB_FILE)
diff --git a/cache.h b/cache.h
index d776016..2e36cc5 100644
--- a/cache.h
+++ b/cache.h
@@ -413,4 +413,5 @@ static inline int sane_case(int x, int h
return x;
}
+extern int copy_fd(int ifd, int ofd);
#endif /* CACHE_H */
diff --git a/copy.c b/copy.c
new file mode 100644
index 0000000..2009275
--- /dev/null
+++ b/copy.c
@@ -0,0 +1,37 @@
+#include "cache.h"
+
+int copy_fd(int ifd, int ofd)
+{
+ while (1) {
+ int len;
+ char buffer[8192];
+ char *buf = buffer;
+ len = read(ifd, buffer, sizeof(buffer));
+ if (!len)
+ break;
+ if (len < 0) {
+ if (errno == EAGAIN)
+ continue;
+ return error("copy-fd: read returned %s",
+ strerror(errno));
+ }
+ while (1) {
+ int written = write(ofd, buf, len);
+ if (written > 0) {
+ buf += written;
+ len -= written;
+ if (!len)
+ break;
+ }
+ if (!written)
+ return error("copy-fd: write returned 0");
+ if (errno == EAGAIN || errno == EINTR)
+ continue;
+ return error("copy-fd: write returned %s",
+ strerror(errno));
+ }
+ }
+ close(ifd);
+ return 0;
+}
+
diff --git a/pack-objects.c b/pack-objects.c
index b3e6152..915469e 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -4,7 +4,7 @@
#include "pack.h"
#include "csum-file.h"
-static const char pack_usage[] = "git-pack-objects [--local] [--incremental] [--window=N] [--depth=N] {--stdout | base-name} < object-list";
+static const char pack_usage[] = "git-pack-objects [--local] [--incremental] [--window=N] [--depth=N] {--stdout | base-name} [--update-cache] < object-list";
struct object_entry {
unsigned char sha1[20];
@@ -400,6 +400,71 @@ static void find_deltas(struct object_en
free(array);
}
+static void prepare_pack(int window, int depth)
+{
+ get_object_details();
+
+ fprintf(stderr, "Packing %d objects\n", nr_objects);
+
+ sorted_by_type = create_sorted_list(type_size_sort);
+ if (window && depth)
+ find_deltas(sorted_by_type, window+1, depth);
+ write_pack_file();
+}
+
+static int reuse_cached_pack(unsigned char *sha1, int pack_to_stdout)
+{
+ static const char cache[] = "pack-cache/pack-%s.%s";
+ char *cached_pack, *cached_idx;
+ int ifd, ofd, ifd_ix = -1;
+
+ cached_pack = git_path(cache, sha1_to_hex(sha1), "pack");
+ ifd = open(cached_pack, O_RDONLY);
+ if (ifd < 0)
+ return 0;
+
+ if (!pack_to_stdout) {
+ cached_idx = git_path(cache, sha1_to_hex(sha1), "idx");
+ ifd_ix = open(cached_idx, O_RDONLY);
+ if (ifd_ix < 0) {
+ close(ifd);
+ return 0;
+ }
+ }
+
+ fprintf(stderr, "Reusing %d objects pack %s\n", nr_objects,
+ sha1_to_hex(sha1));
+
+ if (pack_to_stdout) {
+ if (copy_fd(ifd, 1))
+ exit(1);
+ close(ifd);
+ }
+ else {
+ char name[PATH_MAX];
+ snprintf(name, sizeof(name),
+ "%s-%s.%s", base_name, sha1_to_hex(sha1), "pack");
+ ofd = open(name, O_CREAT | O_EXCL | O_WRONLY, 0666);
+ if (ofd < 0)
+ die("unable to open %s (%s)", name, strerror(errno));
+ if (copy_fd(ifd, ofd))
+ exit(1);
+ close(ifd);
+
+ snprintf(name, sizeof(name),
+ "%s-%s.%s", base_name, sha1_to_hex(sha1), "idx");
+ ofd = open(name, O_CREAT | O_EXCL | O_WRONLY, 0666);
+ if (ofd < 0)
+ die("unable to open %s (%s)", name, strerror(errno));
+ if (copy_fd(ifd_ix, ofd))
+ exit(1);
+ close(ifd_ix);
+ puts(sha1_to_hex(sha1));
+ }
+
+ return 1;
+}
+
int main(int argc, char **argv)
{
SHA_CTX ctx;
@@ -424,6 +489,10 @@ int main(int argc, char **argv)
incremental = 1;
continue;
}
+ if (!strcmp("--update-cache", arg)) {
+ /* Not implemented */
+ continue;
+ }
if (!strncmp("--window=", arg, 9)) {
char *end;
window = strtoul(arg+9, &end, 0);
@@ -472,9 +541,6 @@ int main(int argc, char **argv)
}
if (non_empty && !nr_objects)
return 0;
- get_object_details();
-
- fprintf(stderr, "Packing %d objects\n", nr_objects);
sorted_by_sha = create_sorted_list(sha1_sort);
SHA1_Init(&ctx);
@@ -485,14 +551,14 @@ int main(int argc, char **argv)
}
SHA1_Final(object_list_sha1, &ctx);
- sorted_by_type = create_sorted_list(type_size_sort);
- if (window && depth)
- find_deltas(sorted_by_type, window+1, depth);
-
- write_pack_file();
- if (!pack_to_stdout) {
- write_index_file();
- puts(sha1_to_hex(object_list_sha1));
+ if (reuse_cached_pack(object_list_sha1, pack_to_stdout))
+ ;
+ else {
+ prepare_pack(window, depth);
+ if (!pack_to_stdout) {
+ write_index_file();
+ puts(sha1_to_hex(object_list_sha1));
+ }
}
return 0;
}
diff --git a/upload-pack.c b/upload-pack.c
index 8a41caf..6fb8eb7 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -29,6 +29,7 @@ static void create_pack_file(void)
{
int fd[2];
pid_t pid;
+ int create_full_pack = (MAX_NEEDS <= nr_needs);
if (pipe(fd) < 0)
die("git-upload-pack: unable to create pipe");
@@ -43,7 +44,7 @@ static void create_pack_file(void)
char *buf;
char **p;
- if (MAX_NEEDS <= nr_needs)
+ if (create_full_pack)
args = nr_has + 10;
else
args = nr_has + nr_needs + 5;
@@ -57,7 +58,7 @@ static void create_pack_file(void)
close(fd[1]);
*p++ = "git-rev-list";
*p++ = "--objects";
- if (MAX_NEEDS <= nr_needs)
+ if (create_full_pack)
*p++ = "--all";
else {
for (i = 0; i < nr_needs; i++) {
@@ -79,7 +80,12 @@ static void create_pack_file(void)
dup2(fd[0], 0);
close(fd[0]);
close(fd[1]);
- execlp("git-pack-objects", "git-pack-objects", "--stdout", NULL);
+ if (create_full_pack)
+ execlp("git-pack-objects", "git-pack-objects",
+ "--stdout", NULL);
+ else
+ execlp("git-pack-objects", "git-pack-objects",
+ "--stdout", "--update-cache", NULL);
die("git-upload-pack: unable to exec git-pack-objects");
}
---
0.99.8.GIT
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox