* Re: Overriding ~/.gitconfig using GIT_CONFIG
From: Junio C Hamano @ 2011-08-12 19:16 UTC (permalink / raw)
To: Richard Purdie
Cc: Daniel Barkalow, Nguyễn Thái Ngọc Duy,
GIT Mailing-list
In-Reply-To: <1313163498.14274.505.camel@rex>
Richard Purdie <rpurdie@rpsys.net> writes:
> Looking through the manuals/code, it suggests I should be able to do:
>
> GIT_CONFIG=/dev/null git XXX
>
> and all should work happily. It doesn't though. As an example, with a
> ~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
> accessing the file in ~ and then acting upon it.
If the manual says the above is expected for any value of XXX, then that
is a bug in the manual since mid 2008, I think.
See dc87183 (Only use GIT_CONFIG in "git config", not other programs,
2008-06-30).
I _think_ these days a workaround to force a known config is to set HOME
to a value that has a known .gitconfig (or no such file), and decline
usage of /etc/git.config by exporting GIT_CONFIG_NOSYSTEM.
^ permalink raw reply
* Re: git ls-files --error-unmatch weirdness
From: Junio C Hamano @ 2011-08-12 17:56 UTC (permalink / raw)
To: SZEDER Gábor; +Cc: git
In-Reply-To: <20110812132436.GA12147@goldbirke>
SZEDER Gábor <szeder@ira.uka.de> writes:
> repo (master)$ git ls-files --others --error-unmatch
The "--error-unmatch" is about reporting errors in pathspecs you gave from
the command line. The behaviour is undefined if you do not give any, like
the above command line.
Having said that, I wouldn't be surprised if it triggered when run from an
empty subdirectory.
^ permalink raw reply
* Re: [BUG] Git won't commit
From: Michael Witten @ 2011-08-12 17:30 UTC (permalink / raw)
To: Fabricio Nascimento; +Cc: git
In-Reply-To: <1313169225059-6681082.post@n2.nabble.com>
On Fri, Aug 12, 2011 at 17:13, Fabricio Nascimento <fabriciosn@gmail.com> wrote:
> Hello Everybody,
>
> Hope this is the right place to send this bug report.
>
> It just appeared today, last time I've made a successful commit was 3 days
> ago. Now every commit, in no matter which repository I have (I've tried old
> ones, cloned today, github ones etc) fails with the following message.
>
> [master 09-mapas] git commit
> fatal: could not read 'template': No such file or directory
>
> It happens both with the stock git binary that came with Xcode 4.1 on OSX
> Lion (1.7.4.4) [/usr/local/bin/git], and the homebrew version (1.7.6)
> [/usr/bin/git].
>
> I haven't changed much of my system during those days besides installing
> macports and tinycdb (with ports). Don't see any close relation, both
> tinycdb and macports were removed.
>
> Curiously, it works well with the git binary that comes with Github.app for
> mac, whose version is 1.7.4.
What does the following output for you?
git config --get commit.template
^ permalink raw reply
* Re: open files limit
From: Pete Wyckoff @ 2011-08-12 17:26 UTC (permalink / raw)
To: Shawn Pearce; +Cc: git
In-Reply-To: <CAJo=hJsj8_VjD5wN9Gge_Me-eXKK-P7nLAxERiiLp0+ayiEBbg@mail.gmail.com>
spearce@spearce.org wrote on Fri, 12 Aug 2011 09:09 -0700:
> On Fri, Aug 12, 2011 at 08:15, Pete Wyckoff <pw@padd.com> wrote:
> > Somebody at $work found this problem:
> >
> > $ git ls-files -s | wc
> > error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
> > [..lots more similar errors..]
> >
> > Turns out his shell's open file descriptor limit was 500. And
> > there are 1600 pack files in the repo.
> >
> > Increasing the descriptor limit to 1024 fixed it. I could
> > probably get him to repack, which may also fix it.
> >
> > Does it seem feasible to look for EMFILE errors and close
> > some packs? Or at least spit out a more intuitive error?
>
> What version of Git? I remember fixing this already.... :-)
Initially 1.7.5.4. Same problem on 1.7.6 and master.
I have your "Limit file descriptors used by packs" (c793430, 28
feb 2011).
It fails here:
if (!is_pack_valid(p)) {
error("packfile %s cannot be accessed", p->pack_name);
goto next;
}
because p->pack_fd is -1, because an earlier git_open_noatime()
got EMFILE. The function unuse_one_window() is never able to
find anything to close.
I'll do some more debugging this weekend. Thanks for pointing
out that it _should_ be fixed.
-- Pete
^ permalink raw reply
* Re: [BUG] Git won't commit
From: Fabricio Nascimento @ 2011-08-12 17:20 UTC (permalink / raw)
To: git
In-Reply-To: <1313169225059-6681082.post@n2.nabble.com>
I have installed an stock version of git 1.7.6 from here, and I've got the
same problem.
http://code.google.com/p/git-osx-installer/downloads/detail?name=git-1.7.6-x86_64-snow-leopard.dmg&can=3&q=
I've forgot to mention but I've updated all homebrew formulas during this
past days too, so it might be that previously I had a homebrew version of
git whose version where below or equal 1.4, and that something changed
causing this error now.
--
View this message in context: http://git.661346.n2.nabble.com/BUG-Git-won-t-commit-tp6681082p6681098.html
Sent from the git mailing list archive at Nabble.com.
^ permalink raw reply
* [BUG] Git won't commit
From: Fabricio Nascimento @ 2011-08-12 17:13 UTC (permalink / raw)
To: git
Hello Everybody,
Hope this is the right place to send this bug report.
It just appeared today, last time I've made a successful commit was 3 days
ago. Now every commit, in no matter which repository I have (I've tried old
ones, cloned today, github ones etc) fails with the following message.
[master 09-mapas] git commit
fatal: could not read 'template': No such file or directory
It happens both with the stock git binary that came with Xcode 4.1 on OSX
Lion (1.7.4.4) [/usr/local/bin/git], and the homebrew version (1.7.6)
[/usr/bin/git].
I haven't changed much of my system during those days besides installing
macports and tinycdb (with ports). Don't see any close relation, both
tinycdb and macports were removed.
Curiously, it works well with the git binary that comes with Github.app for
mac, whose version is 1.7.4.
--
View this message in context: http://git.661346.n2.nabble.com/BUG-Git-won-t-commit-tp6681082p6681082.html
Sent from the git mailing list archive at Nabble.com.
^ permalink raw reply
* Overriding ~/.gitconfig using GIT_CONFIG
From: Richard Purdie @ 2011-08-12 15:38 UTC (permalink / raw)
To: GIT Mailing-list
I work on build systems which need to check code out of source control
systems like git. I need to do this in a way in which there are not
outside influences and a user's $HOME/.gitconfig file can contain things
we don't want. I therefore need a way to disable it.
Looking through the manuals/code, it suggests I should be able to do:
GIT_CONFIG=/dev/null git XXX
and all should work happily. It doesn't though. As an example, with a
~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
accessing the file in ~ and then acting upon it.
I've searched through the code and whilst config_exclusive_filename
appears to be the magic variable that should get set when I set
GIT_CONFIG in the environment, nothing ever sets it apart from within a
git-config command.
The following patch sets the variable from the environment initially and
should do what the code intends if I read understand it correctly. Its
not ideal for my use case as I actually want the repo_config and only
the repo config to be used but I can live with setting GIT_CONFIG to
point at it.
Alternative ideas welcome, I've considered changing $HOME but that seems
a little too ugly and likely to cause other problems.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
diff --git a/config.c b/config.c
index d06fb19..19e7565 100644
--- a/config.c
+++ b/config.c
@@ -860,6 +860,8 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
int ret = 0, found = 0;
const char *home = NULL;
+ config_exclusive_filename = getenv(CONFIG_ENVIRONMENT);
+
/* Setting $GIT_CONFIG makes git read _only_ the given config file. */
if (config_exclusive_filename)
return git_config_from_file(fn, config_exclusive_filename, data);
--
Linux Foundation
http://www.yoctoproject.org/
^ permalink raw reply related
* Re: open files limit
From: Shawn Pearce @ 2011-08-12 16:09 UTC (permalink / raw)
To: Pete Wyckoff; +Cc: git
In-Reply-To: <20110812151548.GA14385@padd.com>
On Fri, Aug 12, 2011 at 08:15, Pete Wyckoff <pw@padd.com> wrote:
> Somebody at $work found this problem:
>
> $ git ls-files -s | wc
> error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
> [..lots more similar errors..]
>
> Turns out his shell's open file descriptor limit was 500. And
> there are 1600 pack files in the repo.
>
> Increasing the descriptor limit to 1024 fixed it. I could
> probably get him to repack, which may also fix it.
>
> Does it seem feasible to look for EMFILE errors and close
> some packs? Or at least spit out a more intuitive error?
What version of Git? I remember fixing this already.... :-)
--
Shawn.
^ permalink raw reply
* git_checkattr() is inefficient when repeated [Re: [PATCH 00/11] Micro-optimizing lookup_object()]
From: Thomas Rast @ 2011-08-12 15:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <1313085196-13249-1-git-send-email-gitster@pobox.com>
Junio C Hamano wrote:
> 4-way cuckoo
Cool stuff!
While looking at the performance of it, I noticed something odd about
packing: stracing the command you gave for your timings
strace -o pack.trace \
./git-pack-objects --count-only --keep-true-parents --honor-pack-keep \
--non-empty --all --reflog --no-reuse-delta --delta-base-offset \
--stdout </dev/null >/dev/null
yields the fairly crazy
$ grep -c 'open.*attrib' pack.trace
4398
including runs such as (with a line of context for clarity)
munmap(0x7f9cd39f7000, 4096) = 0
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
open("t/.gitattributes", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
So calling git_checkattr in a loop is quite inefficient. Indeed
there's a good optimization opportunity: compiled for 4-way hashing I
have (best of 3)
6.76user 0.23system 0:07.02elapsed 99%CPU (0avgtext+0avgdata 479792maxresident)
but making no_try_delta() in pack-objects.c a dummy 'return 0' gives
6.45user 0.13system 0:06.61elapsed 99%CPU (0avgtext+0avgdata 478256maxresident)
Which would be a 4.5% speedup. Obviously that won't quite be
attainable since we want the attributes mechanism to work, but we
still shouldn't have to open 4398 .gitattributes files when there are
only 8 .gitattributes plus one .git/info/attributes.
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Bug report: git log --[num|short]stat sometimes counts lines wrong
From: Alexander Pepper @ 2011-08-12 15:21 UTC (permalink / raw)
To: git
Hi there.
This is my first contribution to git (if you count a bug report as a contribution) and I'm not really familiar where to report bugs. In the irc channel #git at freenode somebody pointed me to this mailing list.
First of: I'm running git version 1.7.6 on OS X 10.6.8.
Let me describe what I observed.
repository: https://github.com/voldemort/voldemort.git
The command "git log --numstat c21ad764ea1bae7f7bd83b5e2cb015dcbc44d586" shows for the commit c21ad764 and file '.../readonly/mr/HadoopStoreBuilderReducer.java' 25 lines added and 22 lines removed. But the patch of HadoopStoreBuilderReducer.java that I get with "git show c21ad764ea1bae7f7bd83b5e2cb015dcbc44d586 -- contrib/hadoop-store-builder/src/java/voldemort/store/readonly/mr/HadoopStoreBuilderReducer.java" adds 30 lines and removes 27.
Why does "git log --numstat" drops 5 added lines and 5 removed lines? This also holds true for "git log --stat" and "git log --shortstat".
Is this a bug or am I missing an option to git log or git show?
More commits where I observed this problem on the same repository:
7e00fb6d2cf131dfed59c180f2171952808cc336 src/java/voldemort/client/rebalance/MigratePartitions.java
78ad6f2a6ea327dbae2110f4530a5bd07e5deaac src/java/voldemort/client/rebalance/MigratePartitions.java (same commit on another branch)
7871933f0f0f056e2eeac03a01db1e9cf81f8bda src/java/voldemort/client/protocol/admin/AdminClient.java
2d6f68b09c3bdc23dcf3ae1f91c9285fbd668820 src/java/voldemort/store/readonly/ExternalSorter.java
6fcacee866307ec34eb32b268e2c2b885a949319 build.xml
Greetings from Berlin
Alex
^ permalink raw reply
* open files limit
From: Pete Wyckoff @ 2011-08-12 15:15 UTC (permalink / raw)
To: git
Somebody at $work found this problem:
$ git ls-files -s | wc
error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
[..lots more similar errors..]
Turns out his shell's open file descriptor limit was 500. And
there are 1600 pack files in the repo.
Increasing the descriptor limit to 1024 fixed it. I could
probably get him to repack, which may also fix it.
Does it seem feasible to look for EMFILE errors and close
some packs? Or at least spit out a more intuitive error?
-- Pete
^ permalink raw reply
* Re: git ls-files --error-unmatch weirdness
From: Nguyen Thai Ngoc Duy @ 2011-08-12 14:17 UTC (permalink / raw)
To: SZEDER Gábor; +Cc: git
In-Reply-To: <20110812132436.GA12147@goldbirke>
2011/8/12 SZEDER Gábor <szeder@ira.uka.de>:
> So, the same 'git ls-files' command is OK when run at the top of the
> worktree, but errors out in a subdirectory. What's going on? Is this
> a bug? If not, why not?
report_path_error() in ls-files.c is meant to check pathspecs given by
user. However when in a subdir, get_pathspec() automatically adds a
pathspec (that is prefix). This new pathspec confuses
report_path_error(). The following patch may help (possibly corrupted
because I paste in gmail, but you get the idea)
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 0e98bff..e0611ac 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -545,7 +545,8 @@ int cmd_ls_files(int argc, const char **argv,
const char *cmd_prefix)
max_prefix_len = max_prefix ? strlen(max_prefix) : 0;
/* Treat unmatching pathspec elements as errors */
- if (pathspec && error_unmatch) {
+ if (pathspec && error_unmatch &&
+ (!prefix || pathspec[1])) {
int num;
for (num = 0; pathspec[num]; num++)
;
--
Duy
^ permalink raw reply related
* [PATCH] update-index: add --swap to swap index and worktree content
From: Nguyễn Thái Ngọc Duy @ 2011-08-12 14:07 UTC (permalink / raw)
To: git; +Cc: Nguyễn Thái Ngọc Duy
Sometimes "git add -p" with "e" to edit the patch does not satisfy me.
What I want is a quick way to modify index content without changing
worktree, then I can continue adding more hunks to the index.
With this option, I can swap index out for a quick edit, then swap it in
again.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
Not sure if anybody has the same needs, enough to polish it and make
it to master.
builtin/update-index.c | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/builtin/update-index.c b/builtin/update-index.c
index a6a23fa..b96065a 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -22,6 +22,7 @@
static int allow_add;
static int allow_remove;
static int allow_replace;
+static int swap_entry;
static int info_only;
static int force_remove;
static int verbose;
@@ -104,6 +105,19 @@ static int add_one_path(struct cache_entry *old, const char *path, int len, stru
free(ce);
return -1;
}
+ if (swap_entry) {
+ struct checkout state;
+ if (allow_add || allow_remove)
+ die("--add, --replace and --swap do not play together");
+ memset(&state, 0, sizeof(state));
+ state.force = 1;
+ state.not_new = 1;
+ if (add_cache_entry(ce, 0))
+ return error("%s: cannot add to the index", path);
+ if (checkout_entry(old, &state, NULL))
+ return error("%s: cannot swap", path);
+ return 0;
+ }
option = allow_add ? ADD_CACHE_OK_TO_ADD : 0;
option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0;
if (add_cache_entry(ce, option))
@@ -727,6 +741,8 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
"let files replace directories and vice-versa", 1),
OPT_SET_INT(0, "remove", &allow_remove,
"notice files missing from worktree", 1),
+ OPT_SET_INT(0, "swap", &swap_entry,
+ "swap the content of index and worktree", 1),
OPT_BIT(0, "unmerged", &refresh_args.flags,
"refresh even if index contains unmerged entries",
REFRESH_UNMERGED),
--
1.7.4.74.g639db
^ permalink raw reply related
* git ls-files --error-unmatch weirdness
From: SZEDER Gábor @ 2011-08-12 13:24 UTC (permalink / raw)
To: git
Hi,
I was looking for untracked files when noticed this:
repo$ git init
Initialized empty Git repository in /home/szeder/tmp/git/repo/.git/
repo (master)$ mkdir dir
repo (master)$ touch dir/file
repo (master)$ git add dir/file
repo (master)$ git commit -m 'first'
[master (root-commit) 2c0990c] first
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 dir/file
repo (master)$ git ls-files --others --error-unmatch
repo/dir (master)$ echo $?
0
repo (master)$ cd dir
repo/dir (master)$ git ls-files --others --error-unmatch
error: pathspec '' did not match any file(s) known to git.
Did you forget to 'git add'?
repo/dir (master)$ echo $?
1
So, the same 'git ls-files' command is OK when run at the top of the
worktree, but errors out in a subdirectory. What's going on? Is this
a bug? If not, why not?
Thanks,
Gábor
^ permalink raw reply
* Linux Kernel + GIT SCM
From: Vecu BOSSEUR @ 2011-08-12 13:27 UTC (permalink / raw)
To: git
Hello,
I need GIT SCM to allow me to suspend an operation and to resume it
later : initial cloning, synchronizing repositories, ...
I believed this was the purpose of the "GIT Sequencer" feature
developped at 2011 Google Summmer Of Code ; please, tell me if I
missed something.
And I would like to know whether "kernel.org" servers will support
this feature soon or not.
Answers much appreciated.
Sincerely,
Valentin QUEQUET
( vecu.bosseur@gmail.com )
^ permalink raw reply
* [PATCH 3/3] fast-import: prevent producing bad delta
From: Dmitry Ivankov @ 2011-08-12 10:32 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
In-Reply-To: <1313145170-24471-1-git-send-email-divanorama@gmail.com>
To produce deltas for tree objects fast-import tracks two versions
of tree's entries - base and current one. Base version stands both
for a delta base of this tree, and for a entry inside a delta base
of a parent tree. So care should be taken to keep it in sync.
tree_content_set cuts away a whole subtree and replaces it with a
new one (or NULL for lazy load of a tree with known sha1). It
keeps a base sha1 for this subtree (needed for parent tree). And
here is the problem, 'subtree' tree root doesn't have the implied
base version entries.
Adjusting the subtree to include them would mean a deep rewrite of
subtree. Invalidating the subtree base version would mean recursive
invalidation of parents' base versions. So just mark this tree as
do-not-delta me. Abuse setuid bit for this purpose.
tree_content_replace is the same as tree_content_set except that is
is used to replace the root, so just clearing base sha1 here (instead
of setting the bit) is fine.
[di: log message]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 33 +++++++++++++++++++++++++++++----
t/t9300-fast-import.sh | 2 +-
2 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 8196d1b..d9049af 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -170,6 +170,11 @@ Format of STDIN stream:
#define DEPTH_BITS 13
#define MAX_DEPTH ((1<<DEPTH_BITS)-1)
+/*
+ * We abuse the setuid bit on directories to mean "do not delta".
+ */
+#define NO_DELTA S_ISUID
+
struct object_entry {
struct pack_idx_entry idx;
struct object_entry *next;
@@ -1434,8 +1439,9 @@ static void mktree(struct tree_content *t, int v, struct strbuf *b)
struct tree_entry *e = t->entries[i];
if (!e->versions[v].mode)
continue;
- strbuf_addf(b, "%o %s%c", (unsigned int)e->versions[v].mode,
- e->name->str_dat, '\0');
+ strbuf_addf(b, "%o %s%c",
+ (unsigned int)(e->versions[v].mode & ~NO_DELTA),
+ e->name->str_dat, '\0');
strbuf_add(b, e->versions[v].sha1, 20);
}
}
@@ -1445,7 +1451,7 @@ static void store_tree(struct tree_entry *root)
struct tree_content *t = root->tree;
unsigned int i, j, del;
struct last_object lo = { STRBUF_INIT, 0, 0, /* no_swap */ 1 };
- struct object_entry *le;
+ struct object_entry *le = NULL;
if (!is_null_sha1(root->versions[1].sha1))
return;
@@ -1456,6 +1462,7 @@ static void store_tree(struct tree_entry *root)
}
if (!is_null_sha1(root->versions[0].sha1)
+ && !(root->versions[0].mode & NO_DELTA)
&& S_ISDIR(root->versions[0].mode)) {
unsigned char old_tree_sha1[20];
mktree(t, 0, &old_tree);
@@ -1499,6 +1506,7 @@ static void tree_content_replace(
{
if (!S_ISDIR(mode))
die("Root cannot be a non-directory");
+ hashclr(root->versions[0].sha1);
hashcpy(root->versions[1].sha1, sha1);
if (root->tree)
release_tree_content_recursive(root->tree);
@@ -1543,6 +1551,23 @@ static int tree_content_set(
if (e->tree)
release_tree_content_recursive(e->tree);
e->tree = subtree;
+
+ /*
+ * We need to leave e->versions[0].sha1 alone
+ * to avoid modifying the preimage tree used
+ * when writing out the parent directory.
+ * But after replacing the subdir with a
+ * completely different one, it's not a good
+ * delta base any more, and besides, we've
+ * thrown away the tree entries needed to
+ * make a delta against it.
+ *
+ * So let's just explicitly disable deltas
+ * for the subtree.
+ */
+ if (S_ISDIR(e->versions[0].mode))
+ e->versions[0].mode |= NO_DELTA;
+
hashclr(root->versions[1].sha1);
return 1;
}
@@ -2957,7 +2982,7 @@ static void print_ls(int mode, const unsigned char *sha1, const char *path)
/* mode SP type SP object_name TAB path LF */
strbuf_reset(&line);
strbuf_addf(&line, "%06o %s %s\t",
- mode, type, sha1_to_hex(sha1));
+ mode & ~NO_DELTA, type, sha1_to_hex(sha1));
quote_c_style(path, &line, NULL, 0);
strbuf_addch(&line, '\n');
}
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index c70e489..50b22f0 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -764,7 +764,7 @@ M 040000 @E g/b
M 040000 @E g/b/h
INPUT_END
-test_expect_failure \
+test_expect_success \
'L: verify internal tree delta base' \
'git fast-import <input &&
A=$(git ls-tree L2 a | tr " " "\t" | cut -f 3) &&
--
1.7.3.4
^ permalink raw reply related
* [PATCH 2/3] fast-import: add a check for tree delta base sha1
From: Dmitry Ivankov @ 2011-08-12 10:32 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
In-Reply-To: <1313145170-24471-1-git-send-email-divanorama@gmail.com>
fast-import is able to write imported tree objects in delta format.
It holds a tree structure in memory where each tree entry may have
a delta base sha1 assigned. When delta base data is needed it is
reconstructed from this in-memory structure. Though sometimes the
delta base data doesn't match the delta base sha1 so wrong or even
corrupt pack is produced.
To create a small easily reproducible test, add an excessive check
for delta base sha1. It's not likely that computing sha1 for each
tree delta base costs us much.
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 20 +++++++++++++++-----
t/t9300-fast-import.sh | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 53 insertions(+), 5 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index d0f8580..8196d1b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1455,12 +1455,22 @@ static void store_tree(struct tree_entry *root)
store_tree(t->entries[i]);
}
- le = find_object(root->versions[0].sha1);
- if (S_ISDIR(root->versions[0].mode) && le && le->pack_id == pack_id) {
+ if (!is_null_sha1(root->versions[0].sha1)
+ && S_ISDIR(root->versions[0].mode)) {
+ unsigned char old_tree_sha1[20];
mktree(t, 0, &old_tree);
- lo.data = old_tree;
- lo.offset = le->idx.offset;
- lo.depth = t->delta_depth;
+ prepare_object_hash(OBJ_TREE, &old_tree,
+ NULL, NULL, old_tree_sha1);
+
+ if (hashcmp(old_tree_sha1, root->versions[0].sha1))
+ die("internal tree delta base sha1 mismatch");
+
+ le = find_object(root->versions[0].sha1);
+ if (le && le->pack_id == pack_id) {
+ lo.data = old_tree;
+ lo.offset = le->idx.offset;
+ lo.depth = t->delta_depth;
+ }
}
mktree(t, 1, &new_tree);
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f256475..c70e489 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -734,6 +734,44 @@ test_expect_success \
git diff-tree --abbrev --raw L^ L >output &&
test_cmp expect output'
+cat >input <<INPUT_END
+blob
+mark :1
+data <<EOF
+the data
+EOF
+
+commit refs/heads/L2
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+init L2
+COMMIT
+
+M 644 :1 a/b/c
+M 644 :1 a/b/d
+M 644 :1 a/e/f
+INPUT_END
+
+cat >input2 <<INPUT_END
+commit refs/heads/L2
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+update L2
+COMMIT
+from refs/heads/L2^0
+M 040000 @A g
+M 040000 @E g/b
+M 040000 @E g/b/h
+INPUT_END
+
+test_expect_failure \
+ 'L: verify internal tree delta base' \
+ 'git fast-import <input &&
+ A=$(git ls-tree L2 a | tr " " "\t" | cut -f 3) &&
+ E=$(git ls-tree L2 a/e | tr " " "\t" | cut -f 3) &&
+ cat input2 | sed -e "s/@A/$A/" -e "s/@E/$E/" >input &&
+ git fast-import <input'
+
###
### series M
###
--
1.7.3.4
^ permalink raw reply related
* [PATCH 1/3] fast-import: extract object preparation function
From: Dmitry Ivankov @ 2011-08-12 10:32 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
In-Reply-To: <1313145170-24471-1-git-send-email-divanorama@gmail.com>
We're constructing raw objects and compute their sha1's in fast-import
just before saving them.
Extract header and sha1 computations so that we can get sha1 without
actually saving the object.
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
---
fast-import.c | 32 +++++++++++++++++++++++++-------
1 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 7cc2262..d0f8580 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1006,6 +1006,30 @@ static void cycle_packfile(void)
start_packfile();
}
+static void prepare_object_hash(
+ enum object_type type,
+ struct strbuf *dat,
+ unsigned char *hdr_out,
+ unsigned long *hdrlen_out,
+ unsigned char *sha1_out
+)
+{
+ unsigned char hdr_[96];
+ unsigned char *hdr = hdr_out ? hdr_out : hdr_;
+ unsigned long hdrlen;
+ git_SHA_CTX c;
+
+ hdrlen = sprintf((char *)hdr,"%s %lu", typename(type),
+ (unsigned long)dat->len) + 1;
+ git_SHA1_Init(&c);
+ git_SHA1_Update(&c, hdr, hdrlen);
+ git_SHA1_Update(&c, dat->buf, dat->len);
+ git_SHA1_Final(sha1_out, &c);
+
+ if (hdrlen_out)
+ *hdrlen_out = hdrlen;
+}
+
static int store_object(
enum object_type type,
struct strbuf *dat,
@@ -1018,15 +1042,9 @@ static int store_object(
unsigned char hdr[96];
unsigned char sha1[20];
unsigned long hdrlen, deltalen;
- git_SHA_CTX c;
git_zstream s;
- hdrlen = sprintf((char *)hdr,"%s %lu", typename(type),
- (unsigned long)dat->len) + 1;
- git_SHA1_Init(&c);
- git_SHA1_Update(&c, hdr, hdrlen);
- git_SHA1_Update(&c, dat->buf, dat->len);
- git_SHA1_Final(sha1, &c);
+ prepare_object_hash(type, dat, hdr, &hdrlen, sha1);
if (sha1out)
hashcpy(sha1out, sha1);
--
1.7.3.4
^ permalink raw reply related
* [PATCH 0/3] fix data corruption in fast-import
From: Dmitry Ivankov @ 2011-08-12 10:32 UTC (permalink / raw)
To: git; +Cc: Jonathan Nieder, Shawn O. Pearce, David Barr, Dmitry Ivankov
Finally the bug reported first in [1] is solved and has a small testcase.
Preliminary attempts can be found in [2] for curious.
And the actual "3/3: fix" comes from [3].
Brief introduction. While testing huge imports produced by svn-fe I've found
a "failed to unpack delta" error in fast-import, which is actually caused by
"sha1 mismatch" error in a packfile, and this one is caused by a bug of producing
wrong deltas for tree objects in fast-import.
Looks like that only 'M 040000 sha1_or_mark path' commands could trigger it.
They were introduced in tags/v1.7.3-rc0~75^2
(30 Jun 2010 334fba65.. Teach fast-import to import subtrees named by tree id)
This series should resolve the bug for any copy/rename/set/delete trees scenario
anyway.
I've tested it on a gcc svn repository import - went fine, trees match the gcc
git mirror on github. One more test is ~700k commits from kde repository - fine
too.
1/3 just extracts a sha1 calculation function for 2/3
2/3 adds a die() for "corrupted" delta data and a testcase that triggers it
3/3 is the fix
[1] http://thread.gmane.org/gmane.comp.version-control.git/176753
[2] http://thread.gmane.org/gmane.comp.version-control.git/178007
[3] http://thread.gmane.org/gmane.comp.version-control.git/176753/focus=178053
Dmitry Ivankov (3):
fast-import: extract object preparation function
fast-import: add a check for tree delta base sha1
fast-import: prevent producing bad delta
fast-import.c | 85 +++++++++++++++++++++++++++++++++++++++---------
t/t9300-fast-import.sh | 38 +++++++++++++++++++++
2 files changed, 107 insertions(+), 16 deletions(-)
--
1.7.3.4
^ permalink raw reply
* Re: [PATCHv2 27/56] string-list: Add API to remove an item from an unsorted list
From: Alex Riesen @ 2011-08-12 9:27 UTC (permalink / raw)
To: Johannes Sixt; +Cc: Elijah Newren, gitster, git, Jim Foucar, Johannes Sixt
In-Reply-To: <4E44CF9D.8030501@viscovery.net>
On Fri, Aug 12, 2011 at 09:00, Johannes Sixt <j.sixt@viscovery.net> wrote:
> BTW, the code does this:
>
>> + list->items[i] = list->items[list->nr-1];
>
> i.e., it assigns an entire struct. Is this perhaps problematic with older
> C compilers? (I don't know, I'm more used to C++, where this is well-defined.)
Unlikely. As long as I can remember it always was a memcpy-like operation.
^ permalink raw reply
* Re: [PATCHv2 27/56] string-list: Add API to remove an item from an unsorted list
From: Johannes Sixt @ 2011-08-12 7:00 UTC (permalink / raw)
To: Elijah Newren; +Cc: gitster, git, Jim Foucar, Johannes Sixt
In-Reply-To: <1313126429-17368-28-git-send-email-newren@gmail.com>
Am 8/12/2011 7:20, schrieb Elijah Newren:
> Here's an attempt for a delete_item API (note: only compile-tested).
Seriously? You haven't even tested this patch, and still don't mark it
with RFC?
> Bike-shed painters welcome: delete_item, remove_item, free_item?
You should know that a sentence like this shouldn't appear in the commit
message.
Yeah, I know, you just copy-pasted my email text. But that was not a
commit message. Perhaps like this:
This implements removal of an entry in O(1) runtime by moving
the last entry to the vacated spot. As such, the routine works
only for unsorted lists.
BTW, the code does this:
> + list->items[i] = list->items[list->nr-1];
i.e., it assigns an entire struct. Is this perhaps problematic with older
C compilers? (I don't know, I'm more used to C++, where this is well-defined.)
-- Hannes
^ permalink raw reply
* Re: [PATCHv2 00/57] Re-roll of en/merge-recursive from pu
From: Junio C Hamano @ 2011-08-12 5:48 UTC (permalink / raw)
To: Elijah Newren; +Cc: git, Jim Foucar
In-Reply-To: <1313126429-17368-1-git-send-email-newren@gmail.com>
Elijah Newren <newren@gmail.com> writes:
> Because it's so hard to rule out regressions with so many changes to a
> complicated portion of the code (though hopefully it is less complicated
> now), and because we've had multiple problems in the past with the
> changes I've been making to merge-recursive, I came up with an idea to
> test this series more thoroughly. So, I wrote a script to take every
> single merge commit in git.git that had exactly two parents (no octopus
> merges) and redid them both with /usr/bin/git and the version of git
> from this series. I checked to ensure that the two different versions
> of git:
> (a) EITHER both failed to merge cleanly OR both merged cleanly
> AND
> (b) the output of 'git ls-tree -r HEAD' matched between the two
>
> I ran this process with the original version of the series and indeed
> found that my original series mis-merged half a dozen or so merges (out
> of about 5000).
Thanks for doing this; note that the previous "simple one-side renamed the
other just updated in-place are merged incorrectly" was caught by such a
test.
One thing we may need to be careful about is to compare the conflicted
state a failed merge leaves behind. The latter half of (a) together with
(b) will ensure that you did not introduce silent mismerges.
Avoiding silent mismerges is of course one of the most important criteria,
but we also need to make sure that a conflicted state left in the index
and the working tree files must not be harder to reconcile than what we
have been giving our users---otherwise the change will be seen as a
regression by them.
> *** What is still missing ***
>
> Two things:
> * Junio had a great suggestion about alternate handling of the index in the
> case of rename/rename(2to1) and directory/file conflicts (just rename the
> entry in the index to match how we are renaming in the working copy to
> some new unique name, in order to allow 'git diff' to provide more useful
> information to the user). Just didn't get to it.
> * Support for running break detection in diffs, in order to fix the testcase
> corrected by Peff in this series. Simply didn't get around to it either.
I would say it is sensible to leave these two out. They can be done as a
follow-up series after dust settles.
Thanks.
^ permalink raw reply
* [PATCHv2 11/56] t6036: criss-cross w/ rename/rename(1to2)/modify+rename/rename(2to1)/modify
From: Elijah Newren @ 2011-08-12 5:19 UTC (permalink / raw)
To: gitster; +Cc: git, Jim Foucar, Elijah Newren
In-Reply-To: <1313126429-17368-1-git-send-email-newren@gmail.com>
This test is mostly just designed for testing optimality of the virtual
merge base in the event of a rename/rename(1to2) conflict. The current
choice for resolving this in git seems somewhat confusing and suboptimal.
Signed-off-by: Elijah Newren <newren@gmail.com>
---
Changes since v1: More thorough testing of results, usage of some of
Junio's wording in a comment.
t/t6036-recursive-corner-cases.sh | 88 +++++++++++++++++++++++++++++++++++++
1 files changed, 88 insertions(+), 0 deletions(-)
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index acff84d..38cace6 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -556,4 +556,92 @@ test_expect_failure 'merge of E2 & D fails but has appropriate contents' '
test -f a~D^0
'
+#
+# criss-cross with rename/rename(1to2)/modify followed by
+# rename/rename(2to1)/modify:
+#
+# B D
+# o---o
+# / \ / \
+# A o X ? F
+# \ / \ /
+# o---o
+# C E
+#
+# Commit A: new file: a
+# Commit B: rename a->b, modifying by adding a line
+# Commit C: rename a->c
+# Commit D: merge B&C, resolving conflict by keeping contents in newname
+# Commit E: merge B&C, resolving conflict similar to D but adding another line
+#
+# There is a conflict merging B & C, but one of filename not of file
+# content. Whoever created D and E chose specific resolutions for that
+# conflict resolution. Now, since: (1) there is no content conflict
+# merging B & C, (2) D does not modify that merged content further, and (3)
+# both D & E resolve the name conflict in the same way, the modification to
+# newname in E should not cause any conflicts when it is merged with D.
+# (Note that this can be accomplished by having the virtual merge base have
+# the merged contents of b and c stored in a file named a, which seems like
+# the most logical choice anyway.)
+#
+# Comment from Junio: I do not necessarily agree with the choice "a", but
+# it feels sound to say "B and C do not agree what the final pathname
+# should be, but we know this content was derived from the common A:a so we
+# use one path whose name is arbitrary in the virtual merge base X between
+# D and E" and then further let the rename detection to notice that that
+# arbitrary path gets renamed between X-D to "newname" and X-E also to
+# "newname" to resolve it as both sides renaming it to the same new
+# name. It is akin to what we do at the content level, i.e. "B and C do not
+# agree what the final contents should be, so we leave the conflict marker
+# but that may cancel out at the final merge stage".
+
+test_expect_success 'setup rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
+ git reset --hard &&
+ git rm -rf . &&
+ git clean -fdqx &&
+ rm -rf .git &&
+ git init &&
+
+ printf "1\n2\n3\n4\n5\n6\n" >a &&
+ git add a &&
+ git commit -m A &&
+ git tag A &&
+
+ git checkout -b B A &&
+ git mv a b &&
+ echo 7 >>b &&
+ git add -u &&
+ git commit -m B &&
+
+ git checkout -b C A &&
+ git mv a c &&
+ git commit -m C &&
+
+ git checkout -q B^0 &&
+ git merge --no-commit -s ours C^0 &&
+ git mv b newname &&
+ git commit -m "Merge commit C^0 into HEAD" &&
+ git tag D &&
+
+ git checkout -q C^0 &&
+ git merge --no-commit -s ours B^0 &&
+ git mv c newname &&
+ printf "7\n8\n" >>newname &&
+ git add -u &&
+ git commit -m "Merge commit B^0 into HEAD" &&
+ git tag E
+'
+
+test_expect_failure 'handle rename/rename(1to2)/modify followed by what looks like rename/rename(2to1)/modify' '
+ git checkout D^0 &&
+
+ git merge -s recursive E^0 &&
+
+ test 1 -eq $(git ls-files -s | wc -l) &&
+ test 0 -eq $(git ls-files -u | wc -l) &&
+ test 0 -eq $(git ls-files -o | wc -l) &&
+
+ test $(git rev-parse HEAD:newname) = $(git rev-parse E:newname)
+'
+
test_done
--
1.7.6.100.gac5c1
^ permalink raw reply related
* [PATCHv2 17/56] merge-recursive: Correct a comment
From: Elijah Newren @ 2011-08-12 5:19 UTC (permalink / raw)
To: gitster; +Cc: git, Jim Foucar, Elijah Newren
In-Reply-To: <1313126429-17368-1-git-send-email-newren@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
No changes since v1.
merge-recursive.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/merge-recursive.c b/merge-recursive.c
index 3fcd0a5..3d464d9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1439,7 +1439,7 @@ static int process_df_entry(struct merge_options *o,
handle_delete_modify(o, path, new_path,
a_sha, a_mode, b_sha, b_mode);
} else if (!o_sha && !!a_sha != !!b_sha) {
- /* directory -> (directory, file) */
+ /* directory -> (directory, file) or <nothing> -> (directory, file) */
const char *add_branch;
const char *other_branch;
unsigned mode;
--
1.7.6.100.gac5c1
^ permalink raw reply related
* [PATCHv2 09/56] t6036: criss-cross with weird content can fool git into clean merge
From: Elijah Newren @ 2011-08-12 5:19 UTC (permalink / raw)
To: gitster; +Cc: git, Jim Foucar, Elijah Newren
In-Reply-To: <1313126429-17368-1-git-send-email-newren@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
No changes since v1.
t/t6036-recursive-corner-cases.sh | 83 +++++++++++++++++++++++++++++++++++++
1 files changed, 83 insertions(+), 0 deletions(-)
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index 90b50bb..991c56d 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -314,4 +314,87 @@ test_expect_failure 'git detects conflict merging criss-cross+modify/delete, rev
test $(git rev-parse :3:file) = $(git rev-parse B:file)
'
+#
+# criss-cross + modify/modify with very contrived file contents:
+#
+# B D
+# o---o
+# / \ / \
+# A o X ? F
+# \ / \ /
+# o---o
+# C E
+#
+# Commit A: file with contents 'A\n'
+# Commit B: file with contents 'B\n'
+# Commit C: file with contents 'C\n'
+# Commit D: file with contents 'D\n'
+# Commit E: file with contents:
+# <<<<<<< Temporary merge branch 1
+# C
+# =======
+# B
+# >>>>>>> Temporary merge branch 2
+#
+# Now, when we merge commits D & E, does git detect the conflict?
+
+test_expect_success 'setup differently handled merges of content conflict' '
+ git clean -fdqx &&
+ rm -rf .git &&
+ git init &&
+
+ echo A >file &&
+ git add file &&
+ test_tick &&
+ git commit -m A &&
+
+ git branch B &&
+ git checkout -b C &&
+ echo C >file &&
+ git add file &&
+ test_tick &&
+ git commit -m C &&
+
+ git checkout B &&
+ echo B >file &&
+ git add file &&
+ test_tick &&
+ git commit -m B &&
+
+ git checkout B^0 &&
+ test_must_fail git merge C &&
+ echo D >file &&
+ git add file &&
+ test_tick &&
+ git commit -m D &&
+ git tag D &&
+
+ git checkout C^0 &&
+ test_must_fail git merge B &&
+ cat <<EOF >file &&
+<<<<<<< Temporary merge branch 1
+C
+=======
+B
+>>>>>>> Temporary merge branch 2
+EOF
+ git add file &&
+ test_tick &&
+ git commit -m E &&
+ git tag E
+'
+
+test_expect_failure 'git detects conflict w/ criss-cross+contrived resolution' '
+ git checkout D^0 &&
+
+ test_must_fail git merge -s recursive E^0 &&
+
+ test 3 -eq $(git ls-files -s | wc -l) &&
+ test 3 -eq $(git ls-files -u | wc -l) &&
+ test 0 -eq $(git ls-files -o | wc -l) &&
+
+ test $(git rev-parse :2:file) = $(git rev-parse D:file) &&
+ test $(git rev-parse :3:file) = $(git rev-parse E:file)
+'
+
test_done
--
1.7.6.100.gac5c1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox