* Re: [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Junio C Hamano @ 2009-08-20 23:03 UTC (permalink / raw)
To: Thomas Rast; +Cc: Junio C Hamano, git, Björn Steinbrink, Abhijit Menon-Sen
In-Reply-To: <200908210042.05802.trast@student.ethz.ch>
Thomas Rast <trast@student.ethz.ch> writes:
>> If that is indeed the case and if it is a common thing to ask, isn't it
>> more productive to teach "show" a way to do so in a simpler way than
>> doing, say,
>>
>> $ git show --pretty=format: HEAD
>
> That still doesn't get rid of the stray newline.
And the reason you don't want that newline, especially when you know there
always is one so if you really wanted to you could easily sed it out,
is...?
^ permalink raw reply
* Re: [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Junio C Hamano @ 2009-08-20 23:01 UTC (permalink / raw)
To: Jeff King
Cc: Junio C Hamano, Thomas Rast, git, Björn Steinbrink,
Abhijit Menon-Sen
In-Reply-To: <20090820223424.GA5843@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> On Thu, Aug 20, 2009 at 03:25:16PM -0700, Junio C Hamano wrote:
>
>> Thomas Rast <trast@student.ethz.ch> writes:
>>
>> > Ideally, we'd like 'git diff C^!' to show the same diff that 'git show C'
>> > does (with log.showroot enabled). This gives easy access to a readable
>> > diff for the commit, irrespective of how many parents it has and without
>> > any trickery to remove the commit message from the git-show output.
>>
>> Not interested yet, as the "git show" discussion is not convincing at all.
>>
>> Is the message annoying enough to warrant this change?
>
> I thought the same thing when I saw his message, but reading further,
> the current output is nonsensical. So if not this patch, we should
> probably at least complain about bogus input and die (though if it is
> easy to make it work, why not...).
Ok.
^ permalink raw reply
* Re: Continue git clone after interruption
From: Sam Vilain @ 2009-08-20 22:57 UTC (permalink / raw)
To: Jakub Narebski
Cc: Nicolas Pitre, Tomasz Kontusz, git, Johannes Schindelin,
nick edelen
In-Reply-To: <200908200937.05412.jnareb@gmail.com>
On Thu, 2009-08-20 at 09:37 +0200, Jakub Narebski wrote:
> You would have the same (or at least quite similar) problems with
> deepening part (the 'incrementals' transfer part) as you found with my
> first proposal of server bisection / division of rev-list, and serving
> 1/Nth of revisions (where N is selected so packfile is reasonable) to
> client as incrementals. Yours is top-down, mine was bottom-up approach
> to sending series of smaller packs. The problem is how to select size
> of incrementals, and that incrementals are all-or-nothing (but see also
> comment below).
I've defined a way to do this which doesn't have the complexity of
bisect in GitTorrent, making the compromise that you can't guarantee
each chunk is exactly the same size... I'll have a crack at doing it
based on the rev-cache code in C instead of the horrendously slow
Perl/Berkeley solution I have at the moment to see how well it fares.
> Another solution would be to try to come up with some sort of stable
> sorting of objects so that packfile generated for the same parameters
> (endpoints) would be always byte-for-byte the same. But that might be
> difficult, or even impossible.
delta compression is not repeatable enough for this.
This was an assumption made by the first version of GitTorrent, that
this would be an appropriate solution.
So, first you have to sort the objects - that's fine, --date-order is a
good starting point, then I reasoned that interleaving new objects for
each commit with commit objects would be a useful sort order. You also
need to tie-break for commits with the same commit date; I just used the
SHA-1 of the commit for that. Finally, when making packs to avoid
excessive transfer you have to try to make sure that they are "thin"
packs.
Currently, thin packs can only work starting at the beginning of history
and working forward, which is opposite to what happens most of the time
in packs. I think this is the source of much of the inefficiency caused
by chopping up the object lists mentioned in my other e-mail. It might
be possible, if you could also know which earlier objects were using
this object as a delta base, to try delta'ing against all those objects
and see which one results in the smallest delta.
> Well, we could send the list of objects in pack in order used later by
> pack creation to client (non-resumable but small part), and if packfile
> transport was interrupted in the middle client would compare list of
> complete objects in part of packfile against this manifest, and sent
> request to server with *sorted* list of object it doesn't have yet.
> Server would probably have to check validity of objects list first (the
> object list might be needed to be more than just object list; it might
> need to specify topology of deltas, i.e. which objects are base for which
> ones). Then it would generate rest of packfile.
Mmm. It's a bit chatty, that. Object lists add another 10-20% on,
which I think should be avoidable if the thin pack problem, plus the
problem of some objects ending up in more than one of the thin packs
that are created, should be reduced to very little.
Sam
^ permalink raw reply
* Re: [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Thomas Rast @ 2009-08-20 22:42 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Björn Steinbrink, Abhijit Menon-Sen
In-Reply-To: <7vab1up1gz.fsf@alter.siamese.dyndns.org>
Junio C Hamano wrote:
> Thomas Rast <trast@student.ethz.ch> writes:
>
> > Ideally, we'd like 'git diff C^!' to show the same diff that 'git show C'
> > does (with log.showroot enabled). This gives easy access to a readable
> > diff for the commit, irrespective of how many parents it has and without
> > any trickery to remove the commit message from the git-show output.
>
> Not interested yet, as the "git show" discussion is not convincing at all.
Well, it is currently half-supported, diff doesn't complain about it,
yet it does something wildly different from what people expect in the
zero- and two-parent cases.
And yes, people expect this to work; searching the #git logs (I won't
link here as it takes a fair while even for one request) shows that
people such as Dscho, Thiago, Jakub and Björn have recommended this
syntax in the context of diff.
> Is the message annoying enough to warrant this change?
>
> If that is indeed the case and if it is a common thing to ask, isn't it
> more productive to teach "show" a way to do so in a simpler way than
> doing, say,
>
> $ git show --pretty=format: HEAD
That still doesn't get rid of the stray newline.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: [PATCH JGIT] Ensure created test repositories use canonical paths
From: Robin Rosenberg @ 2009-08-20 22:35 UTC (permalink / raw)
To: Jonas Fonseca; +Cc: Shawn O. Pearce, git
In-Reply-To: <1250687891-17916-1-git-send-email-fonseca@diku.dk>
onsdag 19 augusti 2009 15:18:11 skrev Jonas Fonseca <fonseca@diku.dk>:
> Fixes breakage in the RepositoryCacheTest when running tests using:
>
> mvn -f ./jgit-maven/jgit/pom.xml test
>
> which in turn will lead to test repositories using paths, such as:
>
> /path/to/jgit/./jgit-maven/jgit/trash/trash1250647279819.186/.git
>
> Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
> ---
> .../org/spearce/jgit/lib/RepositoryTestCase.java | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryTestCase.java b/org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryTestCase.java
> index b1adde9..d1aef78 100644
> --- a/org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryTestCase.java
> +++ b/org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryTestCase.java
> @@ -217,7 +217,7 @@ public void setUp() throws Exception {
> final String name = getClass().getName() + "." + getName();
> recursiveDelete(trashParent, true, name, false); // Cleanup old failed stuff
> trash = new File(trashParent,"trash"+System.currentTimeMillis()+"."+(testcount++));
> - trash_git = new File(trash, ".git");
> + trash_git = new File(trash, ".git").getCanonicalFile();
> if (shutdownhook == null) {
> shutdownhook = new Thread() {
> @Override
> @@ -307,7 +307,7 @@ protected Repository createNewEmptyRepo() throws IOException {
> protected Repository createNewEmptyRepo(boolean bare) throws IOException {
> final File newTestRepo = new File(trashParent, "new"
> + System.currentTimeMillis() + "." + (testcount++)
> - + (bare ? "" : "/") + ".git");
> + + (bare ? "" : "/") + ".git").getCanonicalFile();
> assertFalse(newTestRepo.exists());
> final Repository newRepo = new Repository(newTestRepo);
> newRepo.create();
We use getCanonicalFile here and Repository.gitDir is initialized with getAbsoluteDir. Does this work on all platforms?
Seems linux normalized things when you do things like, but I'm not sure that happens everywhere.
ln -s JGIT x;cd x;pwd;jev 'System.out.println(System.getProperty("user.dir"));'
/home/me/SW/x
/home/me/SW/JGIT
-- robin
^ permalink raw reply
* Re: [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Jeff King @ 2009-08-20 22:34 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Thomas Rast, git, Björn Steinbrink, Abhijit Menon-Sen
In-Reply-To: <7vab1up1gz.fsf@alter.siamese.dyndns.org>
On Thu, Aug 20, 2009 at 03:25:16PM -0700, Junio C Hamano wrote:
> Thomas Rast <trast@student.ethz.ch> writes:
>
> > Ideally, we'd like 'git diff C^!' to show the same diff that 'git show C'
> > does (with log.showroot enabled). This gives easy access to a readable
> > diff for the commit, irrespective of how many parents it has and without
> > any trickery to remove the commit message from the git-show output.
>
> Not interested yet, as the "git show" discussion is not convincing at all.
>
> Is the message annoying enough to warrant this change?
I thought the same thing when I saw his message, but reading further,
the current output is nonsensical. So if not this patch, we should
probably at least complain about bogus input and die (though if it is
easy to make it work, why not...).
-Peff
^ permalink raw reply
* Re: [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Junio C Hamano @ 2009-08-20 22:25 UTC (permalink / raw)
To: Thomas Rast; +Cc: git, Björn Steinbrink, Abhijit Menon-Sen
In-Reply-To: <86d1201d8adf53c1f48c0f3526d8e81475b18244.1250806019.git.trast@student.ethz.ch>
Thomas Rast <trast@student.ethz.ch> writes:
> Ideally, we'd like 'git diff C^!' to show the same diff that 'git show C'
> does (with log.showroot enabled). This gives easy access to a readable
> diff for the commit, irrespective of how many parents it has and without
> any trickery to remove the commit message from the git-show output.
Not interested yet, as the "git show" discussion is not convincing at all.
Is the message annoying enough to warrant this change?
If that is indeed the case and if it is a common thing to ask, isn't it
more productive to teach "show" a way to do so in a simpler way than
doing, say,
$ git show --pretty=format: HEAD
^ permalink raw reply
* [PATCH] Make 'diff C^!' show the same diff as 'show C'
From: Thomas Rast @ 2009-08-20 22:10 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Björn Steinbrink, Abhijit Menon-Sen
Ideally, we'd like 'git diff C^!' to show the same diff that 'git show C'
does (with log.showroot enabled). This gives easy access to a readable
diff for the commit, irrespective of how many parents it has and without
any trickery to remove the commit message from the git-show output.
cmd_diff relied on telling the various diff invocations apart from
only the number of revisions parsed by setup_revision() (with a twist
for A...B). In the case of C^! this failed on two counts:
* If C has no parents, setup_revision() turns C^! into simply C. This
meant that 'git diff C^!' compared the current worktree to C, which
is certainly not what the user asked for.
* Otherwise setup_revision puts C itself last, i.e., the rev.pending
are ^C^1 ... ^C^N C. So the first revision is uninteresting and in
the case of exactly two parents, the symmetric difference revspec
(diff A...B) case fired, and compared C only to C^1 (instead of
showing a combined diff).
Detect the presence of A...B or C^! style arguments before running
setup_revisions(), so that we know in which case we are in. We can
then dispatch to the right case without peeking at UNINTERESTING
flags.
There's still some complication in builtin_diff_combined() because
0fe7c1d (built-in diff: assorted updates., 2006-04-29) advertises that
'git diff T0 T1 ... Tn' does a combined diff of arbitrary trees where T0
is the merge result, so we have to handle both this case and C^!.
Note that UNINTERESTING is not a good criterion at all, as it is
tacked onto *trees*; if any of the involved revisions share the same
trees, the flags will overwrite each other.
Thanks to Abhijit "crab" Menon-Sen for noticing that 'diff C^!' didn't
work as expected on root commits, and Björn "doener" Steinbrink for
helpful discussions.
---
Error checking is still iffy, but I'm not sure that can be fixed
without throwing out the whole "argument parsing through
setup_revisions" code and handrolling it.
Documentation/git-diff.txt | 10 +++++++-
builtin-diff.c | 54 +++++++++++++++++++++++++++++++++++-------
t/t4013-diff-various.sh | 3 ++
t/t4013/diff.diff_initial^! | 28 ++++++++++++++++++++++
t/t4013/diff.diff_master^! | 29 +++++++++++++++++++++++
t/t4013/diff.diff_side^! | 32 +++++++++++++++++++++++++
6 files changed, 146 insertions(+), 10 deletions(-)
create mode 100644 t/t4013/diff.diff_initial^!
create mode 100644 t/t4013/diff.diff_master^!
create mode 100644 t/t4013/diff.diff_side^!
diff --git a/Documentation/git-diff.txt b/Documentation/git-diff.txt
index 0ac7112..5d0f2a6 100644
--- a/Documentation/git-diff.txt
+++ b/Documentation/git-diff.txt
@@ -62,9 +62,17 @@ forced by --no-index.
"git diff $(git-merge-base A B) B". You can omit any one
of <commit>, which has the same effect as using HEAD instead.
+'git diff' [--options] <commit>^{caret}! [--] [<path>...]::
+
+ This shows the changes that <commit> made relative to its
+ parents. For an ordinary commit it is the same as `git diff
+ <commit>{caret} <commit>`. For a root commit it shows a
+ creation patch and for a merge commit it shows a combined
+ diff.
+
Just in case if you are doing something exotic, it should be
noted that all of the <commit> in the above description, except
-for the last two forms that use ".." notations, can be any
+for the two forms that use ".." notations, can be any
<tree-ish>.
For a more complete list of ways to spell <commit>, see
diff --git a/builtin-diff.c b/builtin-diff.c
index ffcdd05..285bf29 100644
--- a/builtin-diff.c
+++ b/builtin-diff.c
@@ -163,10 +163,17 @@ static int builtin_diff_tree(struct rev_info *revs,
return 0;
}
+enum diff_mode {
+ DIFF_MODE_NORMAL,
+ DIFF_MODE_SYMMETRIC,
+ DIFF_MODE_SHOW
+};
+
static int builtin_diff_combined(struct rev_info *revs,
int argc, const char **argv,
struct object_array_entry *ent,
- int ents)
+ int ents,
+ enum diff_mode mode)
{
const unsigned char (*parent)[20];
int i;
@@ -177,8 +184,18 @@ static int builtin_diff_combined(struct rev_info *revs,
if (!revs->dense_combined_merges && !revs->combine_merges)
revs->dense_combined_merges = revs->combine_merges = 1;
parent = xmalloc(ents * sizeof(*parent));
- for (i = 0; i < ents; i++)
- hashcpy((unsigned char *)(parent + i), ent[i].item->sha1);
+
+ if (mode == DIFF_MODE_SHOW) {
+ /* diff C^!, we exploit knowledge that C is last */
+ for (i = 1; i < ents; i++)
+ hashcpy((unsigned char *)(parent + i),
+ ent[i-1].item->sha1);
+ hashcpy((unsigned char *)(parent),
+ ent[ents-1].item->sha1);
+ } else {
+ for (i = 0; i < ents; i++)
+ hashcpy((unsigned char *)(parent + i), ent[i].item->sha1);
+ }
diff_tree_combined(parent[0], parent + 1, ents - 1,
revs->dense_combined_merges, revs);
return 0;
@@ -254,6 +271,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
struct blobinfo blob[2];
int nongit;
int result = 0;
+ enum diff_mode mode = DIFF_MODE_NORMAL;
/*
* We could get N tree-ish in the rev.pending_objects list.
@@ -292,6 +310,17 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
/* Otherwise, we are doing the usual "git" diff */
rev.diffopt.skip_stat_unmatch = !!diff_auto_refresh_index;
+ for (i = 1; i < argc; i++) {
+ if (prefixcmp(argv[i], "--")) {
+ if (strstr(argv[i], "..."))
+ mode = DIFF_MODE_SYMMETRIC;
+ else if (strstr(argv[i], "^!"))
+ mode = DIFF_MODE_SHOW;
+ } else if (!strcmp(argv[i], "--")) {
+ break;
+ }
+ }
+
/* Default to let external and textconv be used */
DIFF_OPT_SET(&rev.diffopt, ALLOW_EXTERNAL);
DIFF_OPT_SET(&rev.diffopt, ALLOW_TEXTCONV);
@@ -403,11 +432,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
}
else if (blobs)
usage(builtin_diff_usage);
- else if (ents == 1)
- result = builtin_diff_index(&rev, argc, argv);
- else if (ents == 2)
- result = builtin_diff_tree(&rev, argc, argv, ent);
- else if ((ents == 3) && (ent[0].item->flags & UNINTERESTING)) {
+ else if (mode == DIFF_MODE_SYMMETRIC) {
/* diff A...B where there is one sane merge base between
* A and B. We have ent[0] == merge-base, ent[1] == A,
* and ent[2] == B. Show diff between the base and B.
@@ -415,9 +440,20 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
ent[1] = ent[2];
result = builtin_diff_tree(&rev, argc, argv, ent);
}
+ else if (ents == 1 && mode == DIFF_MODE_SHOW) {
+ /* diff R^! where R is a root commit: creation patch */
+ diff_tree_sha1((const unsigned char *) EMPTY_TREE_SHA1_BIN,
+ ent[0].item->sha1, "", &rev.diffopt);
+ log_tree_diff_flush(&rev);
+ result = 0;
+ }
+ else if (ents == 1)
+ result = builtin_diff_index(&rev, argc, argv);
+ else if (ents == 2)
+ result = builtin_diff_tree(&rev, argc, argv, ent);
else
result = builtin_diff_combined(&rev, argc, argv,
- ent, ents);
+ ent, ents, mode);
result = diff_result_code(&rev.diffopt, result);
if (1 < rev.diffopt.skip_stat_unmatch)
refresh_index_quietly();
diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
index 8b33321..2ce7204 100755
--- a/t/t4013-diff-various.sh
+++ b/t/t4013-diff-various.sh
@@ -273,6 +273,9 @@ diff --no-index --name-status -- dir2 dir
diff --no-index dir dir3
diff master master^ side
diff --dirstat master~1 master~2
+diff initial^!
+diff side^!
+diff master^!
EOF
test_done
diff --git a/t/t4013/diff.diff_initial^! b/t/t4013/diff.diff_initial^!
new file mode 100644
index 0000000..22f6bb7
--- /dev/null
+++ b/t/t4013/diff.diff_initial^!
@@ -0,0 +1,28 @@
+$ git diff initial^!
+diff --git a/dir/sub b/dir/sub
+new file mode 100644
+index 0000000..35d242b
+--- /dev/null
++++ b/dir/sub
+@@ -0,0 +1,2 @@
++A
++B
+diff --git a/file0 b/file0
+new file mode 100644
+index 0000000..01e79c3
+--- /dev/null
++++ b/file0
+@@ -0,0 +1,3 @@
++1
++2
++3
+diff --git a/file2 b/file2
+new file mode 100644
+index 0000000..01e79c3
+--- /dev/null
++++ b/file2
+@@ -0,0 +1,3 @@
++1
++2
++3
+$
diff --git a/t/t4013/diff.diff_master^! b/t/t4013/diff.diff_master^!
new file mode 100644
index 0000000..ca2eaa1
--- /dev/null
+++ b/t/t4013/diff.diff_master^!
@@ -0,0 +1,29 @@
+$ git diff master^!
+diff --cc dir/sub
+index cead32e,7289e35..992913c
+--- a/dir/sub
++++ b/dir/sub
+@@@ -1,6 -1,4 +1,8 @@@
+ A
+ B
+ +C
+ +D
+ +E
+ +F
++ 1
++ 2
+diff --cc file0
+index b414108,f4615da..10a8a9f
+--- a/file0
++++ b/file0
+@@@ -1,6 -1,6 +1,9 @@@
+ 1
+ 2
+ 3
+ +4
+ +5
+ +6
++ A
++ B
++ C
+$
diff --git a/t/t4013/diff.diff_side^! b/t/t4013/diff.diff_side^!
new file mode 100644
index 0000000..6d4378a
--- /dev/null
+++ b/t/t4013/diff.diff_side^!
@@ -0,0 +1,32 @@
+$ git diff side^!
+diff --git a/dir/sub b/dir/sub
+index 35d242b..7289e35 100644
+--- a/dir/sub
++++ b/dir/sub
+@@ -1,2 +1,4 @@
+ A
+ B
++1
++2
+diff --git a/file0 b/file0
+index 01e79c3..f4615da 100644
+--- a/file0
++++ b/file0
+@@ -1,3 +1,6 @@
+ 1
+ 2
+ 3
++A
++B
++C
+diff --git a/file3 b/file3
+new file mode 100644
+index 0000000..7289e35
+--- /dev/null
++++ b/file3
+@@ -0,0 +1,4 @@
++A
++B
++1
++2
+$
--
1.6.4.363.g2183a
^ permalink raw reply related
* Re: Continue git clone after interruption
From: Nicolas Pitre @ 2009-08-20 18:41 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Tomasz Kontusz, git, Johannes Schindelin
In-Reply-To: <200908200937.05412.jnareb@gmail.com>
On Thu, 20 Aug 2009, Jakub Narebski wrote:
> On Wed, 19 Aug 2009, Nicolas Pitre wrote:
> > You'll get the very latest revision for HEAD, and only that. The size
> > of the transfer will be roughly the size of a daily snapshot, except it
> > is fully up to date. It is however non resumable in the event of a
> > network outage. My proposal is to replace this with a "git archive"
> > call. It won't get all branches, but for the purpose of initialising
> > one's repository that should be good enough. And the "git archive" can
> > be fully resumable as I explained.
>
> It is however only 2.5 MB out of 37 MB that are resumable, which is 7%
> (well, that of course depends on repository). Not that much that is
> resumable.
Take the Linux kernel then. It is more like 75 MB.
> > Now to deepen that history. Let's say you want 10 more revisions going
> > back then you simply perform the fetch again with a --depth=10. Right
> > now it doesn't seem to work optimally, but the pack that is then being
> > sent could be made of deltas against objects found in the commits we
> > already have. Currently it seems that a pack that also includes those
> > objects we already have in addition to those we want is created, which
> > is IMHO a flaw in the shallow support that shouldn't be too hard to fix.
> > Each level of deepening should then be as small as standard fetches
> > going forward when updating the repository with new revisions.
>
> You would have the same (or at least quite similar) problems with
> deepening part (the 'incrementals' transfer part) as you found with my
> first proposal of server bisection / division of rev-list, and serving
> 1/Nth of revisions (where N is selected so packfile is reasonable) to
> client as incrementals. Yours is top-down, mine was bottom-up approach
> to sending series of smaller packs. The problem is how to select size
> of incrementals, and that incrementals are all-or-nothing (but see also
> comment below).
Yes and no. Combined with a slight reordering of commit objects, it
could be possible to receive a partial pack and still be able to extract
a bunch of full revisions. The biggest issue is to be able to transfer
revision x (75 MB for Linux), but revision x-1 usually requires only a
few kilobytes, revision x-2 a few other kilobytes, etc. Remember that
you are likely to have only a few deltas from one revision to another,
which is not the case for the very first revision you get. A special
mode to pack-object could place commit objects only after all the
objects needed to create that revision. So once you get a commit object
on the receiving end, you could assume that all objects reachable from
that commit are already received, or you had them locally already.
> In proposal using git-archive and shallow clone deepening as incrementals
> you have this small seed (how small it depends on repository: 50% - 5%)
> which is resumable. And presumably with deepening you can somehow make
> some use from incomplete packfile, only part of which was transferred
> before network error / disconnect. And even tell server about objects
> which you managed to extract from *.pack.part.
yes. And at that point resuming the transfer is just another case of
shallow repository deepening.
> *NEW IDEA*
>
> Another solution would be to try to come up with some sort of stable
> sorting of objects so that packfile generated for the same parameters
> (endpoints) would be always byte-for-byte the same. But that might be
> difficult, or even impossible.
And I don't want to commit to that either. Having some flexibility in
object ordering makes it possible to improve on the packing heuristics.
We certainly should avoid imposing strong restrictions like that for
little gain. Even the deltas are likely to be different from one
request to another when using threads as one thread might be getting
more CPU time than another slightly modifying the outcome.
> Well, we could send the list of objects in pack in order used later by
> pack creation to client (non-resumable but small part), and if packfile
> transport was interrupted in the middle client would compare list of
> complete objects in part of packfile against this manifest, and sent
> request to server with *sorted* list of object it doesn't have yet.
Well... actually that's one of the item for pack V4. Lots of SHA1s are
duplicated in tree and commit objects, in addition to the pack index
file. With pack v4 all those SHA1s would be stored only once in a table
and objects would index that table instead.
Still, that is not _that_ small though. Just look at the size of the
pack index file for the Linux repository to give you an idea.
> Server would probably have to check validity of objects list first (the
> object list might be needed to be more than just object list; it might
> need to specify topology of deltas, i.e. which objects are base for which
> ones). Then it would generate rest of packfile.
I'm afraid that has the looks of something adding lots of complexity to
a piece of git that is already quite complex already, namely
pack-objects. And there is already only a few individuals with their
brain around it.
> > > It would be useful if it was possible to generate part of this rock-solid
> > > file for partial (range, resume) request, without need to generate
> > > (calculate) parts that client already downloaded. Otherwise server has
> > > to either waste disk space and IO for caching, or waste CPU (and IO)
> > > on generating part which is not needed and dropping it to /dev/null.
> > > git-archive you say has this feature.
> >
> > "Could easily have" is more appropriate.
>
> O.K. And I can see how this can be easy done.
>
> > > Next you need to tell server that you have those objects got using
> > > resumable download part ("git archive HEAD" in your proposal), and
> > > that it can use them and do not include them in prepared file/pack.
> > > "have" is limited to commits, and "have <sha1>" tells server that
> > > you have <sha1> and all its prerequisites (dependences). You can't
> > > use "have <sha1>" with git-archive solution. I don't know enough
> > > about 'shallow' capability (and what it enables) to know whether
> > > it can be used for that. Can you elaborate?
> >
> > See above, or Documentation/technical/shallow.txt.
>
> Documentation/technical/shallow.txt doesn't cover "shallow", "unshallow"
> and "deepen" commands from 'shallow' capability extension to git pack
> protocol (http://git-scm.com/gitserver.txt).
404 Not Found
Maybe that should be committed to git in Documentation/technical/ as
well?
> > > Then you have to finish clone / fetch. All solutions so far include
> > > some kind of incremental improvements. My first proposal of bisect
> > > fetching 1/nth or predefined size pack is buttom-up solution, where
> > > we build full clone from root commits up. You propose, from what
> > > I understand build full clone from top commit down, using deepening
> > > from shallow clone. In this step you either get full incremental
> > > or not; downloading incremental (from what I understand) is not
> > > resumable / they do not support partial fetch.
> >
> > Right. However, like I said, the incremental part should be much
> > smaller and therefore less susceptible to network troubles.
>
> If you have 7% total pack size of git-archive resumable part, how small
> do you plan to have those incremental deepening? Besides in my 1/Nth
> proposal those bottom-up packs werealso meant to be sufficiently small
> to avoid network troubles.
Two issues here: 1) people with slow links might not be interested in a
deep history as it costs them time. 2) Extra revisions should typically
require only a few KB each, therefore we might manage to ask for the
full history after the initial revision is downloaded and salvage as
much as we can if a network outage is encountered. There is no need for
arbitrary size, unless the user decides arbitrarily to get only 10 more
revisions, or 100 more, etc.
> P.S. As you can see implementing resumable clone isn't easy...
I've been saying that all along for quite a while now. ;-)
Nicolas
^ permalink raw reply
* Re: question concerning branches
From: Theodore Tso @ 2009-08-20 17:37 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Ingo Brueckl, git
In-Reply-To: <200908192257.23347.jnareb@gmail.com>
On Wed, Aug 19, 2009 at 10:57:21PM +0200, Jakub Narebski wrote:
> Errr... you are aware that you can use "git stash save <message>" (i.e.
> specify commit message for stash; well, the subject), don't you?
I wasn't aware, but usually I like leaving more notes to myself than
just a single lines' worth of state. I should probably take another
look at "git stash" and see if it's a handy tool for me to use; but so
far I've been happy enough with "git checkout -b topic-name; git
commit".
- Ted
^ permalink raw reply
* Re: [PATCH next] t7407: use 'cut' utility rather than bash's substring expansion notation
From: Brandon Casey @ 2009-08-20 16:59 UTC (permalink / raw)
To: Johan Herland; +Cc: Junio C Hamano, git, Brandon Casey
In-Reply-To: <200908201124.54193.johan@herland.net>
Johan Herland wrote:
> On Thursday 20 August 2009, Junio C Hamano wrote:
>> Johan Herland <johan@herland.net> writes:
>>>> - $sub1sha1 sub1 (${sub1sha1:0:7})
>>>> - $sub2sha1 sub2 (${sub1sha1:0:7})
>>>> + $sub1sha1 sub1 ($(echo $sub1sha1 | cut -c 1-7))
>>>> + $sub2sha1 sub2 ($(echo $sub1sha1 | cut -c 1-7))
>>> Typo (both in the original, and the patch), should be:
>>> $sub2sha1 sub2 ($(echo $sub2sha1 | cut -c 1-7))
>>>
>>>> $sub3sha1 sub3 (heads/master)
>>>> EOF
>>> Otherwise:
>>>
>>> Acked-by: Johan Herland <johan@herland.net>
>> Hmm, what does the command use to shorten these object names? It may be
>> safer and more correct to use "rev-parse --short" in case these object
>> names were ambigous in their first 7 hexdigits.
>
> git submodule status (which is what we're testing here) uses
> 'git describe' to generate the short object name (see
> set_name_rev in git-submodule.sh). In this case, it falls back
> to 'git describe --all --always', which calls find_unique_abbrev()
> on the SHA1. 'git rev-parse --short' ends up calling the same
> find_unique_abbrev(), so I guess it is better to use it here.
>
> Try this instead:
>
> From: Johan Herland <johan@herland.net>
>
> Subject: [PATCH] t7407: Use 'rev-parse --short' rather than bash's substring expansion notation
>
> The substring expansion notation is a bashism that we have not so far
> adopted. Use 'git rev-parse --short' instead, as this also handles
> the case where the unique abbreviation is longer than 7 characters.
Works for me.
Maybe it should be mentioned that a typo was fixed too.
-brandon
^ permalink raw reply
* Re: [RFC PATCH v4 00/19] Sparse checkout
From: Matthieu Moy @ 2009-08-20 15:31 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Nguyễn Thái Ngọc Duy, git
In-Reply-To: <m3ocqamrz6.fsf@localhost.localdomain>
Jakub Narebski <jnareb@gmail.com> writes:
> Content-Type: text/plain; charset=utf-16be
^^^^^^^^
I don't know what it should have been, but not UTF-16 ;-)
(readable message below)
> Nguyen Thai
>> Welcome to the fourth round of sparse checkout this year, dubbed "The
>> mummy^W^W^W^W^Wno-checkout returns", although the bit now comes under
>> a new name "skip-worktree" [1]. This series has two parts: sparse
>> worktree and sparse checkout. Details will be given below.
>>
>> For brave users of this series: I have pushed it to my sparse-checkout
>> branch [2] so you can just clone it and test away. Visible changes:
>>
>> - the spec is now .git/info/sparse-checkout
>> - the spec is positive matching, that is it matches what _is_ in
>> worktree, not what is out of worktree like the last series
>> - you need to set core.sparsecheckout no additional command argument
>> is needed.
> [...]
>
>> [1] There are a few reasons behind this rename:
>> - there is nothing about "checkout" in the first part, it's about
>> "sparse worktree"
>> - the double negation issue with "no-" part of "no-checkout"
>> - new name makes cache.h diff align beautifully
>
> Well, on one hand side it is more clear what is checked out,
> and perhaps more common case of checking out only selected files
> is simpler.
>
> On the other hand if it was .git/info/no-checkout or checkout-exclude,
> or worktree-exclude, empty file or lack of such file would mean full
> checkout. Then core.sparsecheckout (or core.sparsefile) would be
> needed only for performance reasons (no need to examine
> checkout-exclude file) even if we decide to ship example file with
> comments describing its syntax, in templates/info--checkout-exclude
>
> But that is just bikeshedding, don't let it distract you from the real
> issue of this series...
>
> --
> Jakub Narebski
>
> In related news: there is really no question that bike sheds should be
> painted red. Really.
> (Johannes Schindelin, on git mailing list)
--
Matthieu
^ permalink raw reply
* Re: [RFC PATCH v4 00/19] Sparse checkout
From: Jakub Narebski @ 2009-08-20 15:21 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git
In-Reply-To: <1250776033-12395-1-git-send-email-pclouds@gmail.com>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-16be, Size: 1805 bytes --]
Nguyþÿ\x1eÅn Thþÿ
> Welcome to the fourth round of sparse checkout this year, dubbed "The
> mummy^W^W^W^W^Wno-checkout returns", although the bit now comes under
> a new name "skip-worktree" [1]. This series has two parts: sparse
> worktree and sparse checkout. Details will be given below.
>
> For brave users of this series: I have pushed it to my sparse-checkout
> branch [2] so you can just clone it and test away. Visible changes:
>
> - the spec is now .git/info/sparse-checkout
> - the spec is positive matching, that is it matches what _is_ in
> worktree, not what is out of worktree like the last series
> - you need to set core.sparsecheckout no additional command argument
> is needed.
[...]
> [1] There are a few reasons behind this rename:
> - there is nothing about "checkout" in the first part, it's about
> "sparse worktree"
> - the double negation issue with "no-" part of "no-checkout"
> - new name makes cache.h diff align beautifully
Well, on one hand side it is more clear what is checked out,
and perhaps more common case of checking out only selected files
is simpler.
On the other hand if it was .git/info/no-checkout or checkout-exclude,
or worktree-exclude, empty file or lack of such file would mean full
checkout. Then core.sparsecheckout (or core.sparsefile) would be
needed only for performance reasons (no need to examine
checkout-exclude file) even if we decide to ship example file with
comments describing its syntax, in templates/info--checkout-exclude
But that is just bikeshedding, don't let it distract you from the real
issue of this series...
--
Jakub Narþÿ\x01\x19bski
In related news: there is really no question that bike sheds should be
painted red. Really.
(Johannes Schindelin, on git mailing list)
^ permalink raw reply
* Re: gitweb and symbolic links
From: Jakub Narebski @ 2009-08-20 15:03 UTC (permalink / raw)
To: Marco Gualtieri; +Cc: git
In-Reply-To: <2367E9DF-6FB1-48E6-AC9A-73E84C9D54E1@math.utoronto.ca>
Marco Gualtieri <mgualt@math.utoronto.ca> writes:
> Hello,
>
> I am wondering about a possibly easy bug which you may be aware of.
>
> For some reason if I place a symbolic link in a directory under the
> project root, gitweb is no longer able to find the project. I was
> looking through the .cgi file but I don't know enough perl to fix the
> problem myself.
>
> Let me know if this makes sense.
WORKSFORME. I have gitweb setup so some of repositories are in
$projectroot directory via symlink, either to repository itself, or
with symlinked earlier part of path, and it works correctly and shows
all those repositories.
I suspect that your web server is configured to not follow symbolic
links, or your user account used by web server is not allowed (doesn't
have permissions) to follow symbolic links.
P.S. You should look into source file gitweb/gitweb.perl, not
installed script gitweb.cgi
--
Jakub Narebski
Git User's Survey 2009: http://tinyurl.com/GitSurvey2009
^ permalink raw reply
* Re: question concerning branches
From: Jakub Narebski @ 2009-08-20 14:59 UTC (permalink / raw)
To: Johannes Sixt; +Cc: Ingo Brueckl, Linus Torvalds, git
In-Reply-To: <4A8D53F3.3050500@viscovery.net>
Johannes Sixt <j.sixt@viscovery.net> writes:
> Ingo Brueckl schrieb:
> > In a branch, I learned, I have to commit or stash before I return to master
> > for push/pull to follow the project. If I forget, I'm screwed, because files
> > have changed due to the rewrite (in that branch), I won't get a warning until
> > my first commit (in that branch) and commits (in master) will conflict.
Errr... if having unknown files in status info when comitting doesn't
clue you in that you have spurious uncomitted changes,
# On branch master
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: somefile
and neither commit diff summary
n files changed, kk insertions(+), ll deletions(-)
doesn't clue you in, then you have more serous problems!
Second, you can use git-aware prompt to tell you if you have
uncomitted changes, so you will know when switching branches that you
have some changes that don't belong to branch you switch from.
>
> You are obviously of a CVS or SVN mindset, where making a commit is such
> an important operation that you don't dare to make it until your work is
> *completed*.
>
> With a git mindset, it won't happen that you "forget" whether you have
> anything uncommitted; you simply never have because committing half-baked
> stuff is the rule, not the exception. That is, before you get a cup of
> coffee, you commit; before you answer a phone call, you commit; before you
> turn your attention away, you commit. (That may be exaggerated, perhaps it
> even isn't, but you get the point.)
>
> When you have completed your work, you go back to make your commit history
> look nice, comprehensible, and bisectable.
...with "git rebase --interactive" or patch management interface
(StGit, Guilt), or topic branch management interface (TopGit).
>
> And only then comes the heavy operation: You publish your work for
> consumption by interested parties. This may be even only you yourself:
> "Consumption" would be to merge the work into your release branch. This is
> the right time to care about upstream again.
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply
* [PATCH v2] Fix symlink __stdcall problem at MSVC
From: Frank Li @ 2009-08-20 14:21 UTC (permalink / raw)
To: git, msysgit; +Cc: Frank Li
MSVC requires that __stdcall be between return value and function name.
Further, all Win32 API definitions look like this:
TYPE WINAPI function_name...
Signed-off-by: Frank Li <lznuaa@gmail.com>
---
compat/mingw.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/compat/mingw.c b/compat/mingw.c
index fd642e4..4256243 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1421,7 +1421,7 @@ int link(const char *oldpath, const char *newpath)
int symlink(const char *oldpath, const char *newpath)
{
- typedef BOOL WINAPI (*symlink_fn)(const char*, const char*, DWORD);
+ typedef BOOL (WINAPI *symlink_fn)(const char*, const char*, DWORD);
static symlink_fn create_symbolic_link = NULL;
if (!create_symbolic_link) {
create_symbolic_link = (symlink_fn) GetProcAddress(
--
1.6.4.msysgit.0
^ permalink raw reply related
* Re: [PATCH] Fix symlink __stdcall problem at MSVC
From: Frank Li @ 2009-08-20 14:16 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, msysgit
In-Reply-To: <alpine.DEB.1.00.0908201546500.4719@intel-tinevez-2-302>
>
> Like what? Seems you did not paste my complete commit message.
>
Yes, Sorry about this
^ permalink raw reply
* [RFC PATCH v4 19/19] sparse checkout: inhibit empty worktree
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-19-git-send-email-pclouds@gmail.com>
The way sparse checkout works, users may empty their worktree
completely, because of non-matching sparse-checkout spec, or empty
spec. I believe this is not desired. This patch makes Git refuse to
produce such worktree.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
t/t1009-read-tree-sparse-checkout.sh | 10 +++-------
unpack-trees.c | 7 +++++++
2 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/t/t1009-read-tree-sparse-checkout.sh b/t/t1009-read-tree-sparse-checkout.sh
index 2192f5a..62246db 100755
--- a/t/t1009-read-tree-sparse-checkout.sh
+++ b/t/t1009-read-tree-sparse-checkout.sh
@@ -55,20 +55,16 @@ test_expect_success 'read-tree --no-sparse-checkout with empty .git/info/sparse-
test -f sub/added
'
-cat >expected.swt <<EOF
-S init.t
-S sub/added
-EOF
test_expect_success 'read-tree with empty .git/info/sparse-checkout' '
git config core.sparsecheckout true &&
echo > .git/info/sparse-checkout &&
- git read-tree -m -u HEAD &&
+ test_must_fail git read-tree -m -u HEAD &&
git ls-files --stage > result &&
test_cmp expected result &&
git ls-files -t > result &&
test_cmp expected.swt result &&
- test ! -f init.t &&
- test ! -f sub/added
+ test -f init.t &&
+ test -f sub/added
'
cat >expected.swt <<EOF
diff --git a/unpack-trees.c b/unpack-trees.c
index 72743b3..80ae2a0 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -498,6 +498,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
}
if (!o->skip_sparse_checkout) {
+ int empty_worktree = 1;
for (i = 0;i < o->result.cache_nr;i++) {
struct cache_entry *ce = o->result.cache[i];
@@ -512,8 +513,14 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
*/
if (ce_skip_worktree(ce))
ce->ce_flags &= ~(CE_UPDATE | CE_REMOVE);
+ else
+ empty_worktree = 0;
}
+ if (o->result.cache_nr && empty_worktree) {
+ ret = unpack_failed(o, "Sparse checkout leaves no entry on working directory");
+ goto done;
+ }
}
o->src_index = NULL;
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 18/19] Add tests for sparse checkout
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-18-git-send-email-pclouds@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
t/t1009-read-tree-sparse-checkout.sh | 154 ++++++++++++++++++++++++++++++++++
1 files changed, 154 insertions(+), 0 deletions(-)
create mode 100755 t/t1009-read-tree-sparse-checkout.sh
diff --git a/t/t1009-read-tree-sparse-checkout.sh b/t/t1009-read-tree-sparse-checkout.sh
new file mode 100755
index 0000000..2192f5a
--- /dev/null
+++ b/t/t1009-read-tree-sparse-checkout.sh
@@ -0,0 +1,154 @@
+#!/bin/sh
+
+test_description='sparse checkout tests'
+
+. ./test-lib.sh
+
+cat >expected <<EOF
+100644 77f0ba1734ed79d12881f81b36ee134de6a3327b 0 init.t
+100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 sub/added
+EOF
+test_expect_success 'setup' '
+ test_commit init &&
+ echo modified >> init.t &&
+ mkdir sub &&
+ touch sub/added &&
+ git add init.t sub/added &&
+ git commit -m "modified and added" &&
+ git tag top &&
+ git rm sub/added &&
+ git commit -m removed &&
+ git tag removed &&
+ git checkout top &&
+ git ls-files --stage > result &&
+ test_cmp expected result
+'
+
+cat >expected.swt <<EOF
+H init.t
+H sub/added
+EOF
+test_expect_success 'read-tree without .git/info/sparse-checkout' '
+ git read-tree -m -u HEAD &&
+ git ls-files --stage > result &&
+ test_cmp expected result &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result
+'
+
+test_expect_success 'read-tree with .git/info/sparse-checkout but disabled' '
+ echo > .git/info/sparse-checkout
+ git read-tree -m -u HEAD &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test -f init.t &&
+ test -f sub/added
+'
+
+test_expect_success 'read-tree --no-sparse-checkout with empty .git/info/sparse-checkout and enabled' '
+ git config core.sparsecheckout true &&
+ echo > .git/info/sparse-checkout &&
+ git read-tree --no-sparse-checkout -m -u HEAD &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test -f init.t &&
+ test -f sub/added
+'
+
+cat >expected.swt <<EOF
+S init.t
+S sub/added
+EOF
+test_expect_success 'read-tree with empty .git/info/sparse-checkout' '
+ git config core.sparsecheckout true &&
+ echo > .git/info/sparse-checkout &&
+ git read-tree -m -u HEAD &&
+ git ls-files --stage > result &&
+ test_cmp expected result &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test ! -f init.t &&
+ test ! -f sub/added
+'
+
+cat >expected.swt <<EOF
+S init.t
+H sub/added
+EOF
+test_expect_success 'match directories with trailing slash' '
+ echo sub/ > .git/info/sparse-checkout &&
+ git read-tree -m -u HEAD &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test ! -f init.t &&
+ test -f sub/added
+'
+
+cat >expected.swt <<EOF
+H init.t
+H sub/added
+EOF
+test_expect_failure 'match directories without trailing slash' '
+ echo init.t > .git/info/sparse-checkout &&
+ echo sub >> .git/info/sparse-checkout &&
+ git read-tree -m -u HEAD &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test ! -f init.t &&
+ test -f sub/added
+'
+
+cat >expected.swt <<EOF
+H init.t
+S sub/added
+EOF
+test_expect_success 'checkout area changes' '
+ echo init.t > .git/info/sparse-checkout &&
+ git read-tree -m -u HEAD &&
+ git ls-files -t > result &&
+ test_cmp expected.swt result &&
+ test -f init.t &&
+ test ! -f sub/added
+'
+
+test_expect_success 'read-tree updates worktree, absent case' '
+ echo sub/added > .git/info/sparse-checkout &&
+ git checkout -f top &&
+ git read-tree -m -u HEAD^ &&
+ test ! -f init.t
+'
+
+test_expect_success 'read-tree updates worktree, dirty case' '
+ echo sub/added > .git/info/sparse-checkout &&
+ git checkout -f top &&
+ echo dirty > init.t &&
+ git read-tree -m -u HEAD^ &&
+ grep -q dirty init.t &&
+ rm init.t
+'
+
+test_expect_success 'read-tree removes worktree, dirty case' '
+ echo init.t > .git/info/sparse-checkout &&
+ git checkout -f top &&
+ echo dirty > added &&
+ git read-tree -m -u HEAD^ &&
+ grep -q dirty added
+'
+
+test_expect_success 'read-tree adds to worktree, absent case' '
+ echo init.t > .git/info/sparse-checkout &&
+ git checkout -f removed &&
+ git read-tree -u -m HEAD^ &&
+ test ! -f sub/added
+'
+
+test_expect_success 'read-tree adds to worktree, dirty case' '
+ echo init.t > .git/info/sparse-checkout &&
+ git checkout -f removed &&
+ mkdir sub &&
+ echo dirty > sub/added &&
+ git read-tree -u -m HEAD^ &&
+ grep -q dirty sub/added
+'
+
+test_done
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 17/19] read-tree: add --no-sparse-checkout to disable sparse checkout support
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-17-git-send-email-pclouds@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
Documentation/git-read-tree.txt | 6 +++++-
builtin-read-tree.c | 4 +++-
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index fc3f08b..ea7b0b2 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -10,7 +10,7 @@ SYNOPSIS
--------
'git read-tree' [[-m [--trivial] [--aggressive] | --reset | --prefix=<prefix>]
[-u [--exclude-per-directory=<gitignore>] | -i]]
- [--index-output=<file>]
+ [--index-output=<file>] [--no-sparse-checkout]
<tree-ish1> [<tree-ish2> [<tree-ish3>]]
@@ -110,6 +110,10 @@ OPTIONS
directories the index file and index output file are
located in.
+--no-sparse-checkout::
+ Disable sparse checkout support even if `core.sparseCheckout`
+ is true.
+
<tree-ish#>::
The id of the tree object(s) to be read/merged.
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 9c2d634..f5acb1a 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -31,7 +31,7 @@ static int list_tree(unsigned char *sha1)
}
static const char * const read_tree_usage[] = {
- "git read-tree [[-m [--trivial] [--aggressive] | --reset | --prefix=<prefix>] [-u [--exclude-per-directory=<gitignore>] | -i]] [--index-output=<file>] <tree-ish1> [<tree-ish2> [<tree-ish3>]]",
+ "git read-tree [[-m [--trivial] [--aggressive] | --reset | --prefix=<prefix>] [-u [--exclude-per-directory=<gitignore>] | -i]] [--no-sparse-checkout] [--index-output=<file>] <tree-ish1> [<tree-ish2> [<tree-ish3>]]",
NULL
};
@@ -98,6 +98,8 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
PARSE_OPT_NONEG, exclude_per_directory_cb },
OPT_SET_INT('i', NULL, &opts.index_only,
"don't check the working tree after merging", 1),
+ OPT_SET_INT(0, "no-sparse-checkout", &opts.skip_sparse_checkout,
+ "skip applying sparse checkout filter", 1),
OPT_END()
};
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 16/19] unpack-trees(): ignore worktree check outside checkout area
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-16-git-send-email-pclouds@gmail.com>
verify_absent() and verify_uptodate() are used to ensure worktree
is safe to be updated, then CE_REMOVE or CE_UPDATE will be set.
Finally check_updates() bases on CE_REMOVE, CE_UPDATE and the
recently added CE_WT_REMOVE to update working directory accordingly.
The entries that are checked may eventually be left out of checkout
area (done later in apply_sparse_checkout()). We don't want to update
outside checkout area. This patch teaches Git to assume "good",
skip these checks when it's sure those entries will be outside checkout
area, and clear CE_REMOVE|CE_UPDATE that could be set due to this
assumption.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
unpack-trees.c | 12 ++++++++++++
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/unpack-trees.c b/unpack-trees.c
index 2d8ecb7..72743b3 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -505,6 +505,14 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
ret = -1;
goto done;
}
+ /*
+ * Merge strategies may set CE_UPDATE|CE_REMOVE outside checkout
+ * area as a result of ce_skip_worktree() shortcuts in
+ * verify_absent() and verify_uptodate(). Clear them.
+ */
+ if (ce_skip_worktree(ce))
+ ce->ce_flags &= ~(CE_UPDATE | CE_REMOVE);
+
}
}
@@ -577,6 +585,8 @@ static int verify_uptodate_1(struct cache_entry *ce,
static int verify_uptodate(struct cache_entry *ce,
struct unpack_trees_options *o)
{
+ if (!o->skip_sparse_checkout && will_have_skip_worktree(ce, o))
+ return 0;
return verify_uptodate_1(ce, o, ERRORMSG(o, not_uptodate_file));
}
@@ -776,6 +786,8 @@ static int verify_absent_1(struct cache_entry *ce, const char *action,
static int verify_absent(struct cache_entry *ce, const char *action,
struct unpack_trees_options *o)
{
+ if (!o->skip_sparse_checkout && will_have_skip_worktree(ce, o))
+ return 0;
return verify_absent_1(ce, action, o, ERRORMSG(o, would_lose_untracked));
}
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 14/19] unpack-trees(): "enable" sparse checkout and load $GIT_DIR/info/sparse-checkout
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-14-git-send-email-pclouds@gmail.com>
This patch introduces core.sparseCheckout, which will control whether
sparse checkout support is enabled in unpack_trees()
It also loads sparse-checkout file that will be used in the next patch.
I split it out so the next patch will be shorter, easier to read.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
Documentation/config.txt | 4 ++++
Documentation/git-read-tree.txt | 4 +++-
cache.h | 1 +
config.c | 5 +++++
environment.c | 1 +
unpack-trees.c | 36 ++++++++++++++++++++++++++++++------
unpack-trees.h | 4 ++++
7 files changed, 48 insertions(+), 7 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 5256c7f..bfaa78d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -439,6 +439,10 @@ On some file system/operating system combinations, this is unreliable.
Set this config setting to 'rename' there; However, This will remove the
check that makes sure that existing object files will not get overwritten.
+core.sparseCheckout::
+ Enable "sparse checkout" feature. See section "Sparse checkout" in
+ linkgit:git-read-tree[1] for more information.
+
add.ignore-errors::
Tells 'git-add' to continue adding files when some files cannot be
added due to indexing errors. Equivalent to the '--ignore-errors'
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index 8b39716..fc3f08b 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -401,7 +401,9 @@ follows:
----------------
Then you can disable sparse checkout. Sparse checkout support in "git
-read-tree" and similar commands is disabled by default.
+read-tree" and similar commands is disabled by default. You need to
+turn `core.sparseCheckout` on in order to have sparse checkout
+support.
SEE ALSO
diff --git a/cache.h b/cache.h
index 0e912a9..a401daf 100644
--- a/cache.h
+++ b/cache.h
@@ -528,6 +528,7 @@ extern int auto_crlf;
extern int read_replace_refs;
extern int fsync_object_files;
extern int core_preload_index;
+extern int core_apply_sparse_checkout;
enum safe_crlf {
SAFE_CRLF_FALSE = 0,
diff --git a/config.c b/config.c
index e87edea..abd762e 100644
--- a/config.c
+++ b/config.c
@@ -503,6 +503,11 @@ static int git_default_core_config(const char *var, const char *value)
return 0;
}
+ if (!strcmp(var, "core.sparsecheckout")) {
+ core_apply_sparse_checkout = git_config_bool(var, value);
+ return 0;
+ }
+
/* Add other config variables here and to Documentation/config.txt. */
return 0;
}
diff --git a/environment.c b/environment.c
index 5de6837..f9ea207 100644
--- a/environment.c
+++ b/environment.c
@@ -50,6 +50,7 @@ enum push_default_type push_default = PUSH_DEFAULT_MATCHING;
#endif
enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
int grafts_replace_parents = 1;
+int core_apply_sparse_checkout;
/* Parallel index stat data preload? */
int core_preload_index = 0;
diff --git a/unpack-trees.c b/unpack-trees.c
index 8eb4b70..44f8fdf 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -378,6 +378,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
{
int ret;
static struct cache_entry *dfc;
+ struct exclude_list el;
if (len > MAX_UNPACK_TREES)
die("unpack_trees takes at most %d trees", MAX_UNPACK_TREES);
@@ -387,6 +388,16 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
state.quiet = 1;
state.refresh_cache = 1;
+ memset(&el, 0, sizeof(el));
+ if (!core_apply_sparse_checkout || !o->update)
+ o->skip_sparse_checkout = 1;
+ if (!o->skip_sparse_checkout) {
+ if (add_excludes_from_file_to_list(git_path("info/sparse-checkout"), "", 0, NULL, &el, 0) < 0)
+ o->skip_sparse_checkout = 1;
+ else
+ o->el = ⪙
+ }
+
memset(&o->result, 0, sizeof(o->result));
o->result.initialized = 1;
if (o->src_index) {
@@ -407,26 +418,39 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
info.fn = unpack_callback;
info.data = o;
- if (traverse_trees(len, t, &info) < 0)
- return unpack_failed(o, NULL);
+ if (traverse_trees(len, t, &info) < 0) {
+ ret = unpack_failed(o, NULL);
+ goto done;
+ }
}
/* Any left-over entries in the index? */
if (o->merge) {
while (o->pos < o->src_index->cache_nr) {
struct cache_entry *ce = o->src_index->cache[o->pos];
- if (unpack_index_entry(ce, o) < 0)
- return unpack_failed(o, NULL);
+ if (unpack_index_entry(ce, o) < 0) {
+ ret = unpack_failed(o, NULL);
+ goto done;
+ }
}
}
- if (o->trivial_merges_only && o->nontrivial_merge)
- return unpack_failed(o, "Merge requires file-level merging");
+ if (o->trivial_merges_only && o->nontrivial_merge) {
+ ret = unpack_failed(o, "Merge requires file-level merging");
+ goto done;
+ }
o->src_index = NULL;
ret = check_updates(o) ? (-2) : 0;
if (o->dst_index)
*o->dst_index = o->result;
+
+done:
+ for (i = 0;i < el.nr;i++)
+ free(el.excludes[i]);
+ if (el.excludes)
+ free(el.excludes);
+
return ret;
}
diff --git a/unpack-trees.h b/unpack-trees.h
index d19df44..5c9e98a 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -4,6 +4,7 @@
#define MAX_UNPACK_TREES 8
struct unpack_trees_options;
+struct exclude_list;
typedef int (*merge_fn_t)(struct cache_entry **src,
struct unpack_trees_options *options);
@@ -28,6 +29,7 @@ struct unpack_trees_options {
skip_unmerged,
initial_checkout,
diff_index_cached,
+ skip_sparse_checkout,
gently;
const char *prefix;
int pos;
@@ -44,6 +46,8 @@ struct unpack_trees_options {
struct index_state *dst_index;
struct index_state *src_index;
struct index_state result;
+
+ struct exclude_list *el; /* for internal use */
};
extern int unpack_trees(unsigned n, struct tree_desc *t,
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 15/19] unpack_trees(): apply $GIT_DIR/info/sparse-checkout to the final index
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-15-git-send-email-pclouds@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
unpack-trees.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
unpack-trees.h | 2 +
2 files changed, 83 insertions(+), 1 deletions(-)
diff --git a/unpack-trees.c b/unpack-trees.c
index 44f8fdf..2d8ecb7 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -32,6 +32,12 @@ static struct unpack_trees_error_msgs unpack_plumbing_errors = {
/* bind_overlap */
"Entry '%s' overlaps with '%s'. Cannot bind.",
+
+ /* sparse_not_uptodate_file */
+ "Entry '%s' not uptodate. Cannot update sparse checkout.",
+
+ /* would_lose_orphaned */
+ "Working tree file '%s' would be %s by sparse checkout update.",
};
#define ERRORMSG(o,fld) \
@@ -125,6 +131,57 @@ static int check_updates(struct unpack_trees_options *o)
return errs != 0;
}
+static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);
+static int verify_absent_sparse(struct cache_entry *ce, const char *action, struct unpack_trees_options *o);
+
+static int will_have_skip_worktree(const struct cache_entry *ce, struct unpack_trees_options *o)
+{
+ const char *basename;
+
+ if (ce_stage(ce))
+ return 0;
+
+ basename = strrchr(ce->name, '/');
+ basename = basename ? basename+1 : ce->name;
+ return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, o->el) <= 0;
+}
+
+static int apply_sparse_checkout(struct cache_entry *ce, struct unpack_trees_options *o)
+{
+ int was_skip_worktree = ce_skip_worktree(ce);
+
+ if (will_have_skip_worktree(ce, o))
+ ce->ce_flags |= CE_SKIP_WORKTREE;
+ else
+ ce->ce_flags &= ~CE_SKIP_WORKTREE;
+
+ /*
+ * We only care about files getting into the checkout area
+ * If merge strategies want to remove some, go ahead, this
+ * flag will be removed eventually in unpack_trees() if it's
+ * outside checkout area.
+ */
+ if (ce->ce_flags & CE_REMOVE)
+ return 0;
+
+ if (!was_skip_worktree && ce_skip_worktree(ce)) {
+ /*
+ * If CE_UPDATE is set, verify_uptodate() must be called already
+ * also stat info may have lost after merged_entry() so calling
+ * verify_uptodate() again may fail
+ */
+ if (!(ce->ce_flags & CE_UPDATE) && verify_uptodate_sparse(ce, o))
+ return -1;
+ ce->ce_flags |= CE_WT_REMOVE;
+ }
+ if (was_skip_worktree && !ce_skip_worktree(ce)) {
+ if (verify_absent_sparse(ce, "overwritten", o))
+ return -1;
+ ce->ce_flags |= CE_UPDATE;
+ }
+ return 0;
+}
+
static inline int call_unpack_fn(struct cache_entry **src, struct unpack_trees_options *o)
{
int ret = o->fn(src, o);
@@ -376,7 +433,7 @@ static int unpack_failed(struct unpack_trees_options *o, const char *message)
*/
int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options *o)
{
- int ret;
+ int i, ret;
static struct cache_entry *dfc;
struct exclude_list el;
@@ -440,6 +497,17 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
goto done;
}
+ if (!o->skip_sparse_checkout) {
+ for (i = 0;i < o->result.cache_nr;i++) {
+ struct cache_entry *ce = o->result.cache[i];
+
+ if (apply_sparse_checkout(ce, o)) {
+ ret = -1;
+ goto done;
+ }
+ }
+ }
+
o->src_index = NULL;
ret = check_updates(o) ? (-2) : 0;
if (o->dst_index)
@@ -512,6 +580,12 @@ static int verify_uptodate(struct cache_entry *ce,
return verify_uptodate_1(ce, o, ERRORMSG(o, not_uptodate_file));
}
+static int verify_uptodate_sparse(struct cache_entry *ce,
+ struct unpack_trees_options *o)
+{
+ return verify_uptodate_1(ce, o, ERRORMSG(o, sparse_not_uptodate_file));
+}
+
static void invalidate_ce_path(struct cache_entry *ce, struct unpack_trees_options *o)
{
if (ce)
@@ -705,6 +779,12 @@ static int verify_absent(struct cache_entry *ce, const char *action,
return verify_absent_1(ce, action, o, ERRORMSG(o, would_lose_untracked));
}
+static int verify_absent_sparse(struct cache_entry *ce, const char *action,
+ struct unpack_trees_options *o)
+{
+ return verify_absent_1(ce, action, o, ERRORMSG(o, would_lose_orphaned));
+}
+
static int merged_entry(struct cache_entry *merge, struct cache_entry *old,
struct unpack_trees_options *o)
{
diff --git a/unpack-trees.h b/unpack-trees.h
index 5c9e98a..95ff36c 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -15,6 +15,8 @@ struct unpack_trees_error_msgs {
const char *not_uptodate_dir;
const char *would_lose_untracked;
const char *bind_overlap;
+ const char *sparse_not_uptodate_file;
+ const char *would_lose_orphaned;
};
struct unpack_trees_options {
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 13/19] unpack-trees.c: generalize verify_* functions
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-13-git-send-email-pclouds@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
unpack-trees.c | 23 ++++++++++++++++++-----
1 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/unpack-trees.c b/unpack-trees.c
index 6a51a69..8eb4b70 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -452,8 +452,9 @@ static int same(struct cache_entry *a, struct cache_entry *b)
* When a CE gets turned into an unmerged entry, we
* want it to be up-to-date
*/
-static int verify_uptodate(struct cache_entry *ce,
- struct unpack_trees_options *o)
+static int verify_uptodate_1(struct cache_entry *ce,
+ struct unpack_trees_options *o,
+ const char *error_msg)
{
struct stat st;
@@ -478,7 +479,13 @@ static int verify_uptodate(struct cache_entry *ce,
if (errno == ENOENT)
return 0;
return o->gently ? -1 :
- error(ERRORMSG(o, not_uptodate_file), ce->name);
+ error(error_msg, ce->name);
+}
+
+static int verify_uptodate(struct cache_entry *ce,
+ struct unpack_trees_options *o)
+{
+ return verify_uptodate_1(ce, o, ERRORMSG(o, not_uptodate_file));
}
static void invalidate_ce_path(struct cache_entry *ce, struct unpack_trees_options *o)
@@ -586,8 +593,9 @@ static int icase_exists(struct unpack_trees_options *o, struct cache_entry *dst,
* We do not want to remove or overwrite a working tree file that
* is not tracked, unless it is ignored.
*/
-static int verify_absent(struct cache_entry *ce, const char *action,
- struct unpack_trees_options *o)
+static int verify_absent_1(struct cache_entry *ce, const char *action,
+ struct unpack_trees_options *o,
+ const char *error_msg)
{
struct stat st;
@@ -667,6 +675,11 @@ static int verify_absent(struct cache_entry *ce, const char *action,
}
return 0;
}
+static int verify_absent(struct cache_entry *ce, const char *action,
+ struct unpack_trees_options *o)
+{
+ return verify_absent_1(ce, action, o, ERRORMSG(o, would_lose_untracked));
+}
static int merged_entry(struct cache_entry *merge, struct cache_entry *old,
struct unpack_trees_options *o)
--
1.6.3.GIT
^ permalink raw reply related
* [RFC PATCH v4 12/19] unpack-trees(): add CE_WT_REMOVE to remove on worktree alone
From: Nguyễn Thái Ngọc Duy @ 2009-08-20 13:47 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
In-Reply-To: <1250776033-12395-12-git-send-email-pclouds@gmail.com>
CE_REMOVE now removes both worktree and index versions. Sparse
checkout must be able to remove worktree version while keep the
index intact when checkout area is narrowed.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
cache.h | 3 +++
unpack-trees.c | 9 ++++++++-
2 files changed, 11 insertions(+), 1 deletions(-)
diff --git a/cache.h b/cache.h
index 3b0e5fc..0e912a9 100644
--- a/cache.h
+++ b/cache.h
@@ -177,6 +177,9 @@ struct cache_entry {
#define CE_HASHED (0x100000)
#define CE_UNHASHED (0x200000)
+/* Only remove in work directory, not index */
+#define CE_WT_REMOVE (0x400000)
+
/*
* Extended on-disk flags
*/
diff --git a/unpack-trees.c b/unpack-trees.c
index dc6d74a..6a51a69 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -78,7 +78,7 @@ static int check_updates(struct unpack_trees_options *o)
if (o->update && o->verbose_update) {
for (total = cnt = 0; cnt < index->cache_nr; cnt++) {
struct cache_entry *ce = index->cache[cnt];
- if (ce->ce_flags & (CE_UPDATE | CE_REMOVE))
+ if (ce->ce_flags & (CE_UPDATE | CE_REMOVE | CE_WT_REMOVE))
total++;
}
@@ -92,6 +92,13 @@ static int check_updates(struct unpack_trees_options *o)
for (i = 0; i < index->cache_nr; i++) {
struct cache_entry *ce = index->cache[i];
+ if (ce->ce_flags & CE_WT_REMOVE) {
+ display_progress(progress, ++cnt);
+ if (o->update)
+ unlink_entry(ce);
+ continue;
+ }
+
if (ce->ce_flags & CE_REMOVE) {
display_progress(progress, ++cnt);
if (o->update)
--
1.6.3.GIT
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox