* Re: [RFC] Add --create-cache to repack
From: Nicolas Pitre @ 2011-01-31 21:48 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Johannes Sixt, git, Junio C Hamano, John Hawley
In-Reply-To: <AANLkTimW=fuKrhw6+ZDipEtSGX_oR4LbTZzyAxZ8Pry1@mail.gmail.com>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 3348 bytes --]
On Mon, 31 Jan 2011, Shawn Pearce wrote:
> On Fri, Jan 28, 2011 at 17:32, Shawn Pearce <spearce@spearce.org> wrote:
> >>> >
> >>> >> This started because I was looking for a way to speed up clones coming
> >>> >> from a JGit server. Cloning the linux-2.6 repository is painful,
> >
> > Well, scratch the idea in this thread. I think.
>
> Nope, I'm back in favor with this after fixing JGit's thin pack
> generation. Here's why.
>
> Take linux-2.6.git as of Jan 12th, with the cache root as of Dec 28th:
>
> $ git update-ref HEAD f878133bf022717b880d0e0995b8f91436fd605c
> $ git-repack.sh --create-cache \
> --cache-root=b52e2a6d6d05421dea6b6a94582126af8cd5cca2 \
> --cache-include=v2.6.11-tree
> $ git repack -a -d
>
> $ ls -lh objects/pack/
> total 456M
> 1.4M pack-74af5edca80797736fe4de7279b2a81af98470a5.idx
> 38M pack-74af5edca80797736fe4de7279b2a81af98470a5.pack
>
> 49M pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.idx
> 89 pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.keep
> 368M pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.pack
>
> Our "recent history" is 38M, and our "cached pack" is 368M. Its a bit
> more disk than is strictly necessary, this should be ~380M. Call it
> ~26M of wasted disk.
This is fine. When doing an incremental fetch, the thin pack does
minimize the transfer size, but it does increase the stored pack size by
appending a bunch of non delta objects to make the pack complete.
What happens though, is that when gc kicks in, the wasted space is
collected back. Here with a single pack we wouldn't claim that space
back as our current euristics is to reuse delta (non) pairing by
default. Maybe in that case we could simply not reuse deltas if they're
of the REF_DELTA type.
> The "cached object list" I proposed elsewhere in
> this thread would cost about 41M of disk and is utterly useless except
> for initial clones. Here we are wasting about 26M of disk to have
> slightly shorter delta chains in the cached pack (otherwise known as
> our ancient history). So its a slightly smaller waste, and we get
> some (minor) benefit.
Well, of course the ancient history you're willing to keep stable for a
while could be repacked even more aggressively than usual.
> Using the cached pack increased our total data transfer by 2.39 MiB,
That's more than acceptable IMHO. That's less than 1% of the total
transfer.
> I think this is worthwhile. If we are afraid of the extra 2.39 MiB
> data transfer this forces on the client when the repository owner
> enables the feature, we should go back and improve our thin-pack code.
> Transferring 11 MiB to catch up a kernel from Dec 28th to Jan 12th
> sounds like a lot of data,
Well, your timing for this test corresponds with the 2.6.38 merge window
which is a high activity peak for this repository. Still, that would
probably fit the usage scenario in practice pretty well where the cache
pack would be produced on a tagged release which happens right before
the merge window.
> and any improvements in the general
> thin-pack code would shrink the leading thin-pack, possibly getting us
> that 2.39 MiB back.
Any improvement to the thin pack would require more CPU cycles, possibly
lot more. So given this transfer overhead is less than 1% already I
don't think we need to bother.
Nicolas
^ permalink raw reply
* [1.8.0] make two-argument fetch update remote branches
From: Thomas Rast @ 2011-01-31 21:44 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwrll57ha.fsf@alter.siamese.dyndns.org>
Proposal:
Running "git fetch origin master" only updates FETCH_HEAD, not
origin/master, which turns out to be quite confusing for newcomers
especially after running 'git pull origin master'.
Since the remote branches in some sense reflect the "last known state"
of the remote, it would make sense to also update them to whatever a
two-argument fetch got.
Risks:
Scripts might rely on the current behaviour. The most likely case I
can think of would be to go along the lines of
git fetch origin master
git rev-list origin/master...FETCH_HEAD | do_something
to avoid relying on reflogs to get the same result. Seems a bit
arcane to me though. Such usage would see the updated state, i.e.,
process an empty range.
Migration plan:
Add a fetch.updateRemoteNamespace (or so) configuration variable that
defaults to false. When enabled, it turns on the auto-updating
behaviour.
In 1.8.0, flip the default.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: [RFC] fast-import: notemodify (N) command
From: Sverre Rabbelier @ 2011-01-31 21:19 UTC (permalink / raw)
To: Jonathan Nieder
Cc: vcs-fast-import-devs, git, chriscool, sam, Johannes.Schindelin,
trast, Johan Herland, gitster, git, Augie Fackler
In-Reply-To: <20110131190128.GC32374@burratino>
Heya,
On Mon, Jan 31, 2011 at 20:01, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Thanks, good to know. I suppose this definitely needs a feature name,
> then (I'll send a patch to make it "feature notes").
SGTM.
> [Aside: I suspect part of the reason "git notes" adoption is not so
> great is the lack of git notes fetch/git notes push. Sample size
> of 1: I use notes heavily as a consumer, to dig up mailing list
> threads[1], but have put off sharing my own annotations until I can
> figure out how to make it convenient for others to use.]
That's another thing Augie mentioned that he (and I guess the hg
community at large) dislikes, the fact that they're not propagated.
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Nicolas Pitre @ 2011-01-31 21:33 UTC (permalink / raw)
To: Matthieu Moy; +Cc: Junio C Hamano, git
In-Reply-To: <vpqwrlkg4r8.fsf@bauges.imag.fr>
On Mon, 31 Jan 2011, Matthieu Moy wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > other than adding merge burden on me and rebase burden on others, that
> > is.
>
> That can be seen as a test of how good Git is at bulk rename
> management ;-).
>
> All that said, I cannot really say whether the benefit is higher than
> the cost.
There is a huge value in inflicting on ourselves such a test case for
the tool we produce. That helps avoiding the ivory tower syndrome.
Nicolas
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Nicolas Pitre @ 2011-01-31 21:28 UTC (permalink / raw)
To: Jeff King; +Cc: Junio C Hamano, git
In-Reply-To: <20110131210045.GB14419@sigill.intra.peff.net>
On Mon, 31 Jan 2011, Jeff King wrote:
> On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
>
> > We do have subdirectories for documentation, tests, contributions, etc.
> > But a sizeable part of the tree is just a big splat of source files
> > dumped right in the root of the tree.
> >
> > So I'd suggest doing the following:
> >
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
> > the builtin directory from the root directory to it.
>
> Wouldn't this just be the same giant splat of source files, but in a
> different tree? I don't really see the advantage, and it seems like an
> extra annoyance.
Like I said to Junio, if you don't see the advantage, there's nothing I
can do for you. To me this is simple good source code hygiene.
> Besides being just one more directory to go up and down, it does make
> history browsing more annoying. As much as I love git's "don't record
> renames" philosophy, our handling of renames on the viewing side is
> often annoying. I already get annoyed sometimes following stuff across
> the s!builtin-!builtin/! change. This would be like that but more so.
So... we do suck at something? So why not take this opportunity to
shake yourself out of this easy comfort and improve Git as a result on
both front? :-)
> Or maybe it is a good thing for that reason, as we will eat our own
> rename dogfood. :)
Exactly! And maybe we'll make Git even more useful in the process.
> > 5) Rename t/ to testsuite/ so this doesn't look like some garbage
> > leftover.
>
> Ugh, more typing. :P
Come on! You sound like an old fart now! ;-)
Nicolas
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Nicolas Pitre @ 2011-01-31 21:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vvd144wrl.fsf@alter.siamese.dyndns.org>
On Mon, 31 Jan 2011, Junio C Hamano wrote:
> Nicolas Pitre <nico@fluxnic.net> writes:
>
> > 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
> > the builtin directory from the root directory to it.
> >
> > 2) Create a build/ directory, or bin/ if prefered, to hold the result of
> > the build.
> >...
> > 6) And fix up all the Makefiles to cope with the above movements.
> >
> > What do you think?
>
> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc. Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,
> other than adding merge burden on me and rebase burden on others, that
> is.
I really think that the top directory is not the proper place for source
files to live, especially considering how big a project Git is now.
The top directory should be like a table of content and not the content
itself.
But if you the maintainer doesn't see a long-term value in this to be
greater than the one-time burden, then I'm afraid there's nothing I can
do to help it.
> But that is just a knee-jerk reaction, just to fill the "Risks:" section
> you didn't fill. Your missing "Migration Plans" section might outline a
> clever approach to lessen the interim hurt while merging in-flight topics.
Well, there is no such plan. Given that 1.8 is meant to be an inflexion
point for users, it could as well be for developers the best time to
clean up this mess too.
Nicolas
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Matthieu Moy @ 2011-01-31 21:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nicolas Pitre, git
In-Reply-To: <7vvd144wrl.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
> Knee-jerk reaction: not very motivated to make the top-level directory
> just a skeleton that holds various directories with a handful of
> administrative files like Makefile, README, etc. Under your proposal, the
> bulk of the current content at the top would simply move to another single
> directory anyway, so I don't immediately see much point of such a move,
There would be at least one obvious benefit: currently, we have this
git$ ls | wc -l
623
(that's after a build)
It's a bit hard to find the interesting bits (README, Documentation/,
contrib/ for example) in the output of "ls".
> other than adding merge burden on me and rebase burden on others, that
> is.
That can be seen as a test of how good Git is at bulk rename
management ;-).
All that said, I cannot really say whether the benefit is higher than
the cost.
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Jeff King @ 2011-01-31 21:00 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <alpine.LFD.2.00.1101311459000.8580@xanadu.home>
On Mon, Jan 31, 2011 at 03:28:37PM -0500, Nicolas Pitre wrote:
> We do have subdirectories for documentation, tests, contributions, etc.
> But a sizeable part of the tree is just a big splat of source files
> dumped right in the root of the tree.
>
> So I'd suggest doing the following:
>
> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
> the builtin directory from the root directory to it.
Wouldn't this just be the same giant splat of source files, but in a
different tree? I don't really see the advantage, and it seems like an
extra annoyance. Besides being just one more directory to go up and
down, it does make history browsing more annoying. As much as I love
git's "don't record renames" philosophy, our handling of renames on the
viewing side is often annoying. I already get annoyed sometimes
following stuff across the s!builtin-!builtin/! change. This would be
like that but more so.
Or maybe it is a good thing for that reason, as we will eat our own
rename dogfood. :)
> 5) Rename t/ to testsuite/ so this doesn't look like some garbage
> leftover.
Ugh, more typing. :P
-Peff
^ permalink raw reply
* Re: [1.8.0] reorganize the mess that the source tree has become
From: Junio C Hamano @ 2011-01-31 20:57 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <alpine.LFD.2.00.1101311459000.8580@xanadu.home>
Nicolas Pitre <nico@fluxnic.net> writes:
> 1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
> the builtin directory from the root directory to it.
>
> 2) Create a build/ directory, or bin/ if prefered, to hold the result of
> the build.
>...
> 6) And fix up all the Makefiles to cope with the above movements.
>
> What do you think?
Knee-jerk reaction: not very motivated to make the top-level directory
just a skeleton that holds various directories with a handful of
administrative files like Makefile, README, etc. Under your proposal, the
bulk of the current content at the top would simply move to another single
directory anyway, so I don't immediately see much point of such a move,
other than adding merge burden on me and rebase burden on others, that
is.
But that is just a knee-jerk reaction, just to fill the "Risks:" section
you didn't fill. Your missing "Migration Plans" section might outline a
clever approach to lessen the interim hurt while merging in-flight topics.
^ permalink raw reply
* [1.8.0] (v2) default "git merge" without argument to "git merge @{u}"
From: Junio C Hamano @ 2011-01-31 20:50 UTC (permalink / raw)
To: Jeff King; +Cc: git, Miles Bader
In-Reply-To: <7vsjw957fq.fsf_-_@alter.siamese.dyndns.org>
Ok, here is an update.
I wonder how we should keep track of the "proposal" series, though.
Sending the full-text replacement like I am doing here feels eh not
gittish.
Perhaps I should start a new directory in todo branch (say, 1.8.0), accept
patches from people? I'd grudgingly admit that using Wiki on k.org may be
less burdensome (I hate editing inside the browser myself), but I'd want
to keep the mailing list the center of discussion and am afraid that
forcing people to go to Wiki would fragment the discussion.
I dunno.
--
Proposal:
Currently "git merge" without saying what to merge fails loudly. As many
downstream developers (that are by definition more numerous than people
who do not have "upstream") run merge from the configured upstream of the
current branch most of the time, it would make it more convenient for them
to change the default to merge from the upstream of the current branch.
Running "git merge" without argument, when there is no upstream defined
for the current branch, will be an error.
Risks:
Existing scripts may prepare what to merge in an array (e.g. in Bourne,
accumulating them in "$@" by repeatedly doing 'set "$@" "$newbranch"') and
call 'git merge "$@"', relying on the current behaviour that zero argument
will flag an error condition. Such scripts will be broken by this change.
Driving "git merge" with xargs without --no-run-if-empty (not POSIX),
feeding the branches to merge in an Octopus, will be broken the same way.
Migration plan:
Add merge.defaultToUpstream configuration variable, which defaults to
false when unconfigured. Change "git merge" so that when this variable is
set and the command is run without the commit to merge to use the
configured upstream of the current branch (or error out if there isn't one).
Merge this change in the next 1.7.x series.
Update the error message issued when when "git merge" is run without the
commit to merge and this configuration variable is not explicitly set
either way to notify the user of upcoming incompatibility.
In 1.8.0, flip the default for merge.defaultUpstream to true.
Helped-by: Jeff King <peff@peff.net>
^ permalink raw reply
* Re: [1.8.0] default "git merge" without argument to "git merge @{u}"
From: Felipe Contreras @ 2011-01-31 20:32 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git, Miles Bader
In-Reply-To: <7vaaig6d64.fsf@alter.siamese.dyndns.org>
On Mon, Jan 31, 2011 at 10:17 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jeff King <peff@peff.net> writes:
>
>>> In 1.8.0, flip the default for merge.defaultUpstream to true.
>>
>> Other than that, I think the proposal and migration plan are fine.
>
> Heh, thanks.
>
> It's not my proposal though ;-). It is just an illustration of how I see
> an item from people's wishlist should look like to make it easier to
> discuss them.
Ok, I share the same opinion as Jeff; we should not care about scripts
doing something so wrong. But if the migration path is the only way to
get the change in, I guess I can do that.
--
Felipe Contreras
^ permalink raw reply
* [1.8.0] reorganize the mess that the source tree has become
From: Nicolas Pitre @ 2011-01-31 20:28 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwrll57ha.fsf@alter.siamese.dyndns.org>
To me the source tree looks highly untidy to me.
We do have subdirectories for documentation, tests, contributions, etc.
But a sizeable part of the tree is just a big splat of source files
dumped right in the root of the tree.
So I'd suggest doing the following:
1) Create a src/ directory and move *.c, *.h, *.sh, *.perl, *.py and
the builtin directory from the root directory to it.
2) Create a build/ directory, or bin/ if prefered, to hold the result of
the build.
3) Consider dropping the ppc/ directory. Unless someone really cares
deeply, it would be nice to simply always use the block-sha1 code and
move it straight into src/.
4) Consider moving some more directories into src/ such as xdiff/.
I'd leave compat/ outside src/ to make it more explicit that this is
not about Git proper.
5) Rename t/ to testsuite/ so this doesn't look like some garbage
leftover.
6) And fix up all the Makefiles to cope with the above movements.
What do you think?
Nicolas
^ permalink raw reply
* [PATCH] tree_entry_interesting(): with no pathspecs, everything will match
From: Junio C Hamano @ 2011-01-31 20:21 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: git
In-Reply-To: <AANLkTik_7FUuocoXa7Rk5DWSR4ie7FBfYNa0MSQzwnsZ@mail.gmail.com>
Hence the function should return "2" in such a case as advertised.
Note that finding a directory that busts the depth limit does not mean non
directories at the same level would also bust the limit (they don't), so
we still need to return 1, not 2 in the within_depth() codepath.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:
>> When depth check is in effect, the result depends on the mode of the
>> entry, so we cannot short-circuit by returning 2, but at least we should
>> do so when (max_depth == -1), no?
>
> Yes, should be 2.
Thanks. I'll queue this.
tree-walk.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tree-walk.c b/tree-walk.c
index 9b43ad5..ece8cc0 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -560,7 +560,7 @@ int tree_entry_interesting(const struct name_entry *entry,
if (!ps->nr) {
if (!ps->recursive || ps->max_depth == -1)
- return 1;
+ return 2;
return !!within_depth(base->buf + base_offset, baselen,
!!S_ISDIR(entry->mode),
ps->max_depth);
--
1.7.4.261.g705f2
^ permalink raw reply related
* Re: [1.8.0] default "git merge" without argument to "git merge @{u}"
From: Junio C Hamano @ 2011-01-31 20:17 UTC (permalink / raw)
To: Jeff King; +Cc: Junio C Hamano, git, Miles Bader
In-Reply-To: <20110131201419.GA9070@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
>> In 1.8.0, flip the default for merge.defaultUpstream to true.
>
> Other than that, I think the proposal and migration plan are fine.
Heh, thanks.
It's not my proposal though ;-). It is just an illustration of how I see
an item from people's wishlist should look like to make it easier to
discuss them.
^ permalink raw reply
* Re: [1.8.0] default "git merge" without argument to "git merge @{u}"
From: Jeff King @ 2011-01-31 20:14 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Miles Bader
In-Reply-To: <7vsjw957fq.fsf_-_@alter.siamese.dyndns.org>
On Mon, Jan 31, 2011 at 09:06:33AM -0800, Junio C Hamano wrote:
> Existing scripts may prepare what to merge in an array (e.g. in Bourne,
> accumulating them in "$@" by repeatedly doing 'set "$@" "$newbranch"') and
> call 'git merge "$@"', relying on the current behaviour that zero argument
> means no-op. Such scripts will be broken by this change. Driving "git
> merge" with xargs without --no-run-if-empty (not POSIX), feeding the
> branches to merge in an Octopus, will be broken the same way.
I am not sure these things are not already broken. "git merge" without
arguments right now is not a no-op. It is an error that spews usage to
stderr and exits 129. Yes, there can be scripts which stupidly do not
bother to see if the merge succeeded, but I'm not sure how much we
should care about such poorly written junk.
> Migration plan:
>
> Add merge.defaultUpstream configuration variable, which defaults to false
> when unconfigured. Change "git merge" so that when this configuration is
> set and the command is run without the commit to merge to use the
> configured upstream of the current branch (or error out if there isn't
> one). Merge this change in the next 1.7.x series.
One nit: upon reading the name of the variable, I assumed it would be
"the default upstream to merge". Perhaps "merge.defaultToUpstream" is a
more descriptive name?
> One release before 1.8.0, issue a warning when "git merge" is run without
> the commit to merge and this configuration variable is not explicitly set
> either way, and notify the user of upcoming incompatibility.
Don't we already issue a giant warning when "git merge" is run without a
commit, namely:
usage: git merge [options] <remote>...
... etc ...
? If people are already not paying attention to that (either because
they are throwing away stderr and exit code indiscriminately, or because
the no-arguments case is a simply an obscure corner that their script
doesn't usually exercise), why would they pay attention to a new
warning?
> In 1.8.0, flip the default for merge.defaultUpstream to true.
Other than that, I think the proposal and migration plan are fine.
-Peff
^ permalink raw reply
* [PATCH] t6004: add pathspec globbing test for log family
From: Junio C Hamano @ 2011-01-31 20:09 UTC (permalink / raw)
To: git; +Cc: Nguyen Thai Ngoc Duy
Earlier e10cb0f (tree_entry_interesting(): support wildcard matching,
2010-12-15) and b3d4b34 (tree_entry_interesting(): optimize wildcard
matching when base is matched, 2010-12-15) added tests for globbing
support for diff-tree plumbing. This is a follow-up to update the test
for revision traversal and path pruning machinery for the same topic.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
t/t6004-rev-list-path-optim.sh | 69 +++++++++++++++++++++++++++++++++-------
1 files changed, 57 insertions(+), 12 deletions(-)
diff --git a/t/t6004-rev-list-path-optim.sh b/t/t6004-rev-list-path-optim.sh
index 5dabf1c..3e8c42e 100755
--- a/t/t6004-rev-list-path-optim.sh
+++ b/t/t6004-rev-list-path-optim.sh
@@ -1,51 +1,96 @@
#!/bin/sh
-test_description='git rev-list trivial path optimization test'
+test_description='git rev-list trivial path optimization test
+
+ d/z1
+ b0 b1
+ o------------------------*----o master
+ / /
+ o---------o----o----o----o side
+ a0 c0 c1 a1 c2
+ d/f0 d/f1
+ d/z0
+
+'
. ./test-lib.sh
test_expect_success setup '
-echo Hello > a &&
-git add a &&
-git commit -m "Initial commit" a &&
-initial=$(git rev-parse --verify HEAD)
+ echo Hello >a &&
+ mkdir d &&
+ echo World >d/f &&
+ echo World >d/z &&
+ git add a d &&
+ test_tick &&
+ git commit -m "Initial commit" &&
+ git rev-parse --verify HEAD &&
+ git tag initial
'
test_expect_success path-optimization '
- commit=$(echo "Unchanged tree" | git commit-tree "HEAD^{tree}" -p HEAD) &&
- test $(git rev-list $commit | wc -l) = 2 &&
- test $(git rev-list $commit -- . | wc -l) = 1
+ test_tick &&
+ commit=$(echo "Unchanged tree" | git commit-tree "HEAD^{tree}" -p HEAD) &&
+ test $(git rev-list $commit | wc -l) = 2 &&
+ test $(git rev-list $commit -- . | wc -l) = 1
'
test_expect_success 'further setup' '
git checkout -b side &&
echo Irrelevant >c &&
- git add c &&
+ echo Irrelevant >d/f &&
+ git add c d/f &&
+ test_tick &&
git commit -m "Side makes an irrelevant commit" &&
+ git tag side_c0 &&
echo "More Irrelevancy" >c &&
git add c &&
+ test_tick &&
git commit -m "Side makes another irrelevant commit" &&
echo Bye >a &&
git add a &&
+ test_tick &&
git commit -m "Side touches a" &&
- side=$(git rev-parse --verify HEAD) &&
+ git tag side_a1 &&
echo "Yet more Irrelevancy" >c &&
git add c &&
+ test_tick &&
git commit -m "Side makes yet another irrelevant commit" &&
git checkout master &&
echo Another >b &&
- git add b &&
+ echo Munged >d/z &&
+ git add b d/z &&
+ test_tick &&
git commit -m "Master touches b" &&
+ git tag master_b0 &&
git merge side &&
echo Touched >b &&
git add b &&
+ test_tick &&
git commit -m "Master touches b again"
'
test_expect_success 'path optimization 2' '
- ( echo "$side"; echo "$initial" ) >expected &&
+ git rev-parse side_a1 initial >expected &&
git rev-list HEAD -- a >actual &&
test_cmp expected actual
'
+test_expect_success 'pathspec with leading path' '
+ git rev-parse master^ master_b0 side_c0 initial >expected &&
+ git rev-list HEAD -- d >actual &&
+ test_cmp expected actual
+'
+
+test_expect_success 'pathspec with glob (1)' '
+ git rev-parse master^ master_b0 side_c0 initial >expected &&
+ git rev-list HEAD -- "d/*" >actual &&
+ test_cmp expected actual
+'
+
+test_expect_success 'pathspec with glob (2)' '
+ git rev-parse side_c0 initial >expected &&
+ git rev-list HEAD -- "d/[a-m]*" >actual &&
+ test_cmp expected actual
+'
+
test_done
--
1.7.4.261.g705f2
^ permalink raw reply related
* Re: gitk "find commit adding/removing string"/possible pickaxe bug?
From: Jeff King @ 2011-01-31 20:00 UTC (permalink / raw)
To: Thomas Rast; +Cc: git, Sebastian Hahn
In-Reply-To: <20110118205040.GA20970@sigill.intra.peff.net>
On Tue, Jan 18, 2011 at 03:50:40PM -0500, Jeff King wrote:
> > doesn't; it only gives a diff for the commit that introduced 'bar'. I
> > guess this makes sense: -S notices that the number of 'bar's is
> > actually the same as in *one* merge parent, hence the merge cannot be
> > all that interesting. OTOH it still shows the merge commit in the
> > history, which is a bit strange. --pickaxe-all does not make a
> > difference either;
>
> Hrm. What I expected[1] to happen would be for the diff machinery to
> look at each filepair individually, one of them to trigger -S, which
> shows the commit, and then to fail to produce a combined diff because we
> threw away the other uninteresting filepair. But in that case,
> --pickaxe-all _should_ show something, as its point is to keep all of
> the filepairs. And that's clearly not happening.
>
> So now I don't know what's going on. I'll try to trace through the diff
> machinery and see if that gives a clue.
>
> -Peff
>
> [1] That's what I expect, but not necessarily what I want. I think what
> I would want is for it to do a token count of the merge commit, and if
> it fails to match _every_ parent, then it it interesting. Otherwise, the
> content presumably came from that parent.
I looked into this, and sadly the "wanted" behavior I described above is
not easy to do. It turns out that we never actually see the whole 3-way
diff as a single unit in diffcore-pickaxe. Instead, log-tree calls into
diff_tree_combined, which diffs each parent _individually_, including
running diffcore magic on it. And then if one of those appears
interesting, we show the merge.
So diffcore-pickaxe never even knows that we are doing a combined diff.
It just sees the difference between M and M^, and then separately the
difference between M and M^2. This works OK in my example:
commit() {
echo $1 >file && git add file && git commit -m $1
}
commit base
commit master
git checkout -b other HEAD^
commit other
git merge master
commit resolved
as doing "git log -Sother -c" will show both the commit "other" _and_
the merge commit (since it removed "other" in favor of "resolved"). But
you could also construct a case where it isn't true. For example,
consider a case where two sides add the same token, and the resolution
is to keep both. E.g.:
echo base >file && git add file && git commit -m base
echo foo bar >file && git commit -a -m master
git checkout -b other HEAD^
echo foo baz >file && git commit -a -m other
git merge master
(echo foo bar; echo foo baz) >file && git commit -a -m resolved
That shows the merge commit, even though it didn't actually introduce or
delete that token at all. OTOH, it is part of a conflict region, so it
is really difficult to say whether it is interesting or not. I dunno
what the right semantics are (and note that the definition I gave in the
above email would also trigger on this case).
I have the nagging feeling there is another less ambiguous corner case
that is wrong, but I'm having trouble constructing one.
Anyway, the real point is that we can't do anything special to pickaxe
merge commits at the diffcore level without some pretty major diff
surgery. So where does that leave us? You can still get pretty
reasonable results from turning on "-c". I was curious what the CPU cost
was of turning "-c" on by default, and was very surprised by the
results (in git.git):
$ time git log -Sfoo >/dev/null
real 0m11.532s
user 0m11.273s
sys 0m0.116s
$ time git log -c -Sfoo >/dev/null
real 3m7.530s
user 3m3.991s
sys 0m2.948s
A 1700% slowdown? Wow. There are ~20000 non-merge commits in git.git and
~4500 merge commits. Each merge commit has two parents (since we don't
tend to octopus merge), each of which is diffed individually. So I'd
expect it to add about 9000 diffs, or roughly 50% on top of the
11-second case.
My guess is that the subtree merges from gitk and git-gui are very
expensive to look at, since from one parent's perspective we will have
created the entire git project from scratch. On every merge. Yikes.
-Peff
^ permalink raw reply
* [PATCH] git-cvsimport.perl: Bail out right away when reading from the server fails
From: Fabian Keil @ 2011-01-31 19:29 UTC (permalink / raw)
To: git
If the CVS server is down, this reduced the git-cvsimport output from:
ssh: connect to host ijbswa.cvs.sourceforge.net port 22: Connection refused
Use of uninitialized value $rep in scalar chomp at /usr/local/libexec/git-core/git-cvsimport line 369.
Use of uninitialized value $rep in substitution (s///) at /usr/local/libexec/git-core/git-cvsimport line 370.
Expected Valid-requests from server, but got: <unknown>
to the less noisy:
ssh: connect to host ijbswa.cvs.sourceforge.net port 22: Connection refused
Failed to read from server at /usr/local/libexec/git-core/git-cvsimport line 370.
In this case a silent exit() instead of the die() would probably do,
but I assume that there could be cases where the connection attempt
succeeds, but reading from the server fails for other reasons.
Signed-off-by: Fabian Keil <fk@fabiankeil.de>
---
git-cvsimport.perl | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
1d8b7c2be2b7dd281a0799735d00c8111404260f
diff --git a/git-cvsimport.perl b/git-cvsimport.perl
index 8e683e5..2a76d9e 100755
--- a/git-cvsimport.perl
+++ b/git-cvsimport.perl
@@ -366,7 +366,9 @@ sub conn {
$self->{'socketo'}->write("valid-requests\n");
$self->{'socketo'}->flush();
- chomp(my $rep=$self->readline());
+ my $rep=$self->readline();
+ die "Failed to read from server" unless defined $rep;
+ chomp($rep);
if ($rep !~ s/^Valid-requests\s*//) {
$rep="<unknown>" unless $rep;
die "Expected Valid-requests from server, but got: $rep\n";
--
1.7.3.5
^ permalink raw reply related
* Re: [RFC] fast-import: notemodify (N) command
From: Jonathan Nieder @ 2011-01-31 19:01 UTC (permalink / raw)
To: Sverre Rabbelier
Cc: vcs-fast-import-devs, git, chriscool, sam, Johannes.Schindelin,
trast, Johan Herland, gitster, git, Augie Fackler
In-Reply-To: <AANLkTi=3P0xc3mJj5Tsg_P26SAXmgVfCk1VeYQ=k2UtT@mail.gmail.com>
Sverre Rabbelier wrote:
> I talked with Augie Fackler (from hg) about this on IM and he says:
>> We don't support anything like that at present (no demand, when we check
>> nobody really seems to use git notes for anything)
>> so it doesn't seem relevant in fast-export
>
> So at least HG doesn't (currently) have any functionality that they
> could use to implement the importing of such a stream.
Thanks, good to know. I suppose this definitely needs a feature name,
then (I'll send a patch to make it "feature notes").
Jonathan
[Aside: I suspect part of the reason "git notes" adoption is not so
great is the lack of git notes fetch/git notes push. Sample size
of 1: I use notes heavily as a consumer, to dig up mailing list
threads[1], but have put off sharing my own annotations until I can
figure out how to make it convenient for others to use.]
[1] http://thread.gmane.org/gmane.comp.version-control.git/109074/focus=109542
^ permalink raw reply
* Re: [Vcs-fast-import-devs] [RFC] fast-import: notemodify (N) command
From: Sverre Rabbelier @ 2011-01-31 18:48 UTC (permalink / raw)
To: Jonathan Nieder
Cc: vcs-fast-import-devs, git, chriscool, sam, Johannes.Schindelin,
trast, Johan Herland, tavestbo, gitster, git, Augie Fackler
In-Reply-To: <20110131183350.GB31826@burratino>
Heya,
On Mon, Jan 31, 2011 at 19:33, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Most DVCSes do not allow one to non-disruptively change the log
> message for a commit. But sometimes people want to attach information to a
> commit after the fact:
>
> - whether it was tested and worked correctly
> - who liked or disliked the commit (Acked-by, Reviewed-by)
> - corresponding revision number after export to another version
> control system
> - bug number
> - corresponding compiled binary
I talked with Augie Fackler (from hg) about this on IM and he says:
> We don't support anything like that at present (no demand, when we check
> nobody really seems to use git notes for anything)
> so it doesn't seem relevant in fast-export
So at least HG doesn't (currently) have any functionality that they
could use to implement the importing of such a stream.
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: [RFC] Add --create-cache to repack
From: Shawn Pearce @ 2011-01-31 18:47 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Johannes Sixt, git, Junio C Hamano, John Hawley
In-Reply-To: <AANLkTi=U7qRRij=BQXC1Goqa9toDFfaVKT=+-8zYxCcc@mail.gmail.com>
On Fri, Jan 28, 2011 at 17:32, Shawn Pearce <spearce@spearce.org> wrote:
>>> >
>>> >> This started because I was looking for a way to speed up clones coming
>>> >> from a JGit server. Cloning the linux-2.6 repository is painful,
>
> Well, scratch the idea in this thread. I think.
Nope, I'm back in favor with this after fixing JGit's thin pack
generation. Here's why.
Take linux-2.6.git as of Jan 12th, with the cache root as of Dec 28th:
$ git update-ref HEAD f878133bf022717b880d0e0995b8f91436fd605c
$ git-repack.sh --create-cache \
--cache-root=b52e2a6d6d05421dea6b6a94582126af8cd5cca2 \
--cache-include=v2.6.11-tree
$ git repack -a -d
$ ls -lh objects/pack/
total 456M
1.4M pack-74af5edca80797736fe4de7279b2a81af98470a5.idx
38M pack-74af5edca80797736fe4de7279b2a81af98470a5.pack
49M pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.idx
89 pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.keep
368M pack-d3e77c8b3045c7f54fa2fb6bbfd4dceca1e2b9fa.pack
Our "recent history" is 38M, and our "cached pack" is 368M. Its a bit
more disk than is strictly necessary, this should be ~380M. Call it
~26M of wasted disk. The "cached object list" I proposed elsewhere in
this thread would cost about 41M of disk and is utterly useless except
for initial clones. Here we are wasting about 26M of disk to have
slightly shorter delta chains in the cached pack (otherwise known as
our ancient history). So its a slightly smaller waste, and we get
some (minor) benefit.
Clone without pack caching:
$ time git clone --bare git://localhost/tmp_linux26_withTag tmp_in.git
Cloning into bare repository tmp_in.git...
remote: Counting objects: 1861830, done
remote: Finding sources: 100% (1861830/1861830)
remote: Getting sizes: 100% (88243/88243)
remote: Compressing objects: 100% (88184/88184)
Receiving objects: 100% (1861830/1861830), 376.01 MiB | 19.01 MiB/s, done.
remote: Total 1861830 (delta 4706), reused 1851053 (delta 1553844)
Resolving deltas: 100% (1564621/1564621), done.
real 3m19.005s
user 1m36.250s
sys 0m10.290s
Clone with pack caching:
$ time git clone --bare git://localhost/tmp_linux26_withTag tmp_in.git
Cloning into bare repository tmp_in.git...
remote: Counting objects: 1601, done
remote: Counting objects: 1828460, done
remote: Finding sources: 100% (50475/50475)
remote: Getting sizes: 100% (18843/18843)
remote: Compressing objects: 100% (7585/7585)
remote: Total 1861830 (delta 2407), reused 1856197 (delta 37510)
Receiving objects: 100% (1861830/1861830), 378.40 MiB | 31.31 MiB/s, done.
Resolving deltas: 100% (1559477/1559477), done.
real 2m2.938s
user 1m35.890s
sys 0m9.830s
Using the cached pack increased our total data transfer by 2.39 MiB,
but saved 1m17s on server computation time. If we go back and look at
our cached pack size (368M), the leading thin-pack should be about
10.4 MiB (378.40M - 368M = 10.4M). If I modify the tmp_in.git client
to have only the cached pack's tip and fetch using CGit, we see the
thin pack to bring ourselves current is 11.07 MiB (JGit does this in
10.96 MiB):
$ cd tmp_in.git
$ git update-ref HEAD b52e2a6d6d05421dea6b6a94582126af8cd5cca2
$ git repack -a -d ; # yay we are at ~1 month ago
$ time git fetch ../tmp_linux26_withTag
remote: Counting objects: 60570, done.
remote: Compressing objects: 100% (11924/11924), done.
remote: Total 49804 (delta 42196), reused 44837 (delta 37231)
Receiving objects: 100% (49804/49804), 11.07 MiB | 7.37 MiB/s, done.
Resolving deltas: 100% (42196/42196), completed with 4956 local objects.
From ../tmp_linux26_withTag
* branch HEAD -> FETCH_HEAD
real 0m35.083s
user 0m25.710s
sys 0m1.190s
The pack caching feature is *no worse* in transfer size than if the
client copied the pack from 1 month ago, and then did an incremental
fetch to bring themselves current. Compared to the naive clone, it
saves an incredible amount of working set space and CPU time. The
server only needs to keep track of the incremental thin pack, and can
completely ignore the ancient history objects. Its a great
alternative for projects that want users to rsync/http dumb transport
down a large stable repository, then incremental fetch themselves
current. Or busy mirror sites that are willing to trade some small
bandwidth for server CPU and memory.
In this particular example, there is ~11 MiB of data that cannot be
safely resumed, or the first 2.9%. At 56 KiB/s, a client needs to get
through the first 3 minutes of transfer before they can reach the
resumable checkpoint (where the thin pack ends, and the cached pack
starts). It would be better if we could resume anywhere in the
stream, but being able to resume the last 97% is infinitely better
than being able to resume nothing. If someone wants to really go
crazy, this is where a "gittorrent" client could start up and handle
the remaining 97% of the transfer. :-)
I think this is worthwhile. If we are afraid of the extra 2.39 MiB
data transfer this forces on the client when the repository owner
enables the feature, we should go back and improve our thin-pack code.
Transferring 11 MiB to catch up a kernel from Dec 28th to Jan 12th
sounds like a lot of data, and any improvements in the general
thin-pack code would shrink the leading thin-pack, possibly getting us
that 2.39 MiB back.
--
Shawn.
^ permalink raw reply
* [RFC] fast-import: notemodify (N) command
From: Jonathan Nieder @ 2011-01-31 18:33 UTC (permalink / raw)
To: vcs-fast-import-devs
Cc: Johan Herland, git, gitster, Johannes.Schindelin, trast, tavestbo,
git, chriscool, spearce, sam
In-Reply-To: <1255083738-23263-8-git-send-email-johan@herland.net>
Dear fast importers,
Another week, another fast-import protocol extension.
Most DVCSes do not allow one to non-disruptively change the log
message for a commit. But sometimes people want to attach information to a
commit after the fact:
- whether it was tested and worked correctly
- who liked or disliked the commit (Acked-by, Reviewed-by)
- corresponding revision number after export to another version
control system
- bug number
- corresponding compiled binary
The N command allows such notes to be attached to commits, like so:
1. first the commit is imported as usual (let's say it's ":1").
2. commit annotations are added separately, like so:
commit refs/notes/commits
committer A. U. Thor <author@example.com> Mon, 31 Jan 2011 12:15:59 -0600
data <<END
Notes after review.
END
N inline :1
data <<END
Acked-by: me
END
Details:
- there can be multiple categories of notes: "refs/notes/commits"
contains ordinary addenda to the commit message, but one might also
see refs/notes/bugzilla, refs/notes/svn-commit, and so on.
- each commit gets at most one blob of notes in each category. Later
notemodify (N) commands overwrite the effect from earlier ones.
- the syntax of a notemodify command is as follows:
'N' sp <dataref> sp <committish> lf
The <dataref> represents a blob with the annotations to be used
("inline" is allowed, too, just like with filemodify). The
<committish> can be any expression allowed in a 'from' command
(branch name, mark reference :<idnum>, other commit name) and
represents the commit that is to be annotated.
- this has been supported in git since v1.6.6. There is no
"feature" for it --- I don't think the feature declaration
facility existed yet.
Do other DVCSes support something like this? Should it get a
feature name?
Jonathan
^ permalink raw reply
* [1.8.0] Unify "pathspec" semantics
From: Junio C Hamano @ 2011-01-31 17:07 UTC (permalink / raw)
To: git
In-Reply-To: <7vwrll57ha.fsf@alter.siamese.dyndns.org>
Proposal:
Traditionally, Git had two distinct semantics for pathspecs. Anything
based on tree-diff (i.e. "log" family of commands when limiting the
history by paths or "diff" family of commands limiting the output) used
"leading paths match" without globbing support. All others (e.g. "grep",
"ls-files") supported globbing. This resulted in subtly inconsistent
behaviour when one part of the program collected paths from the index and
the working tree while another part of the program used differences
between the index and the HEAD, e.g. "git add". Unify "pathspec"
semantics to make all of them learn the globbing.
Risks:
If coded poorly, performance bugs can be introduced to the tree-diff
codepath, making it inefficient.
Some projects may track a file whose name is asterisk (e.g. "foo/*") and
output from "git log 'foo/*'" would look different. Before the change,
only commits that touch that exact path would be shown, but after the
change, any commit that touch a path underneath "foo/" directory will be
shown. This is a backward incompatible change.
Migration plan:
We could conditionally enable globbing support when implementing unified
pathspec API, default to the traditional and inconsisntent behaviour
during the 1.7.x series, and flip the default to accept globs everwhere in
the 1.8.0 release. Practically, however, nobody sane would track paths
that have shell metacharacters in them, so we may not need to do the usual
"introduce as an opt-in, warn about incompatibility, and flip the default"
migration.
^ permalink raw reply
* [1.8.0] default "git merge" without argument to "git merge @{u}"
From: Junio C Hamano @ 2011-01-31 17:06 UTC (permalink / raw)
To: git; +Cc: Miles Bader
In-Reply-To: <7vwrll57ha.fsf@alter.siamese.dyndns.org>
Proposal:
Currently "git merge" without saying what to merge is a no-op, and
succeeds silently. As many downstream developers (that are by definition
more numerous than people who do not have "upstream") run merge from the
configured upstream of the current branch most of the time, it would make
it more convenient for them to change the default to merge from the
upstream of the current branch. Running "git merge" without argument,
when there is no upstream defined for the current branch, will be an
error.
Risks:
Existing scripts may prepare what to merge in an array (e.g. in Bourne,
accumulating them in "$@" by repeatedly doing 'set "$@" "$newbranch"') and
call 'git merge "$@"', relying on the current behaviour that zero argument
means no-op. Such scripts will be broken by this change. Driving "git
merge" with xargs without --no-run-if-empty (not POSIX), feeding the
branches to merge in an Octopus, will be broken the same way.
Migration plan:
Add merge.defaultUpstream configuration variable, which defaults to false
when unconfigured. Change "git merge" so that when this configuration is
set and the command is run without the commit to merge to use the
configured upstream of the current branch (or error out if there isn't
one). Merge this change in the next 1.7.x series.
One release before 1.8.0, issue a warning when "git merge" is run without
the commit to merge and this configuration variable is not explicitly set
either way, and notify the user of upcoming incompatibility.
In 1.8.0, flip the default for merge.defaultUpstream to true.
^ permalink raw reply
* Planning for 1.7.5 and 1.8.0
From: Junio C Hamano @ 2011-01-31 17:05 UTC (permalink / raw)
To: git
In-Reply-To: <7vzkqh8vqw.fsf@alter.siamese.dyndns.org>
Now the 1.7.4 release is out, I'd like people to help thinking about the
next cycle(s).
As a discussion-starter, here are my random wishes. Even though this does
not attempt to be exhaustive, keeping the number of goals manageably small
may help us focus.
* The i18n effort Ævar Arnfjörð Bjarmason started two cycles ago has
stalled. If enough people feel i18n's Porcelain UI is worth having, I
think we would need a brief calming period in the entire tree in order
for us to get the minimum support (definition of _() macro that is
empty is a good start) and _() mark-up of existing strings in, and then
ask everybody to rebase their ongoing work on top of it.
* There was a discussion on documentation updates to reduce "here we tell
you only the basics; see elsewhere for details", and consolidating the
description of configuration options in one place.
* Nguyễn has been scratching my longstanding itch by attempting to unify
two pathspec semantics (the ones based on tree-diff matches only
leading path while others know globs) to reduce inconsistencies. I
would really want to see this polished and in the main release.
* Elijah's fix to "rev-list --objects", together with the updated
pathspec semantics will allow us to cleanly implement narrow cloning
(possibly deprecating and replacing the narrow checkout in the
future). I am hoping that we can lay groundwork on this inside one
cycle and the initial end-to-end implementation in another.
* Shawn Pearce says that the diff implementation JGit uses (histogram
diff) performs way better than the xdiff implementation we use by
default. It would be great if somebody can spend time taking a look at
it and possibly port it back to C-git.
Over the time we have discussed minor glitches and inconsistencies that we
all (or at least most of us anyway) agreed we would have done differently
if we were writing Git from scratch, yet we cannot retroactively introduce
differences not to harm existing users. We may also want to revisit these
discussions during this round--if there are reasonable number of them that
we can agree the benefit of tweaked semantics/behaviour outweighs the risk
of breaking and having to update ancient scripts that exploited obscure
corner case behaviour of Git, we would want warn the users loudly, bite
the bullet and break them so that we can move forward. We would however
need to roll such potentially disruptive changes into a big single cycle,
like we did in 1.6.0.
I'll follow-up this message with a couple of example proposals. Please
send your own, imitating the format of the message, as a reply to this
message. Do not forget to retitle your message when you do so (iow, I
don't want to see "Re: Planning for 1.7.5 and 1.8.0").
Thanks.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox