* general protection faults with "git grep" version 1.7.7.1
From: Markus Trippelsdorf @ 2011-10-24 20:11 UTC (permalink / raw)
To: git
Suddenly I'm getting strange protection faults when I run "git grep" on
the gcc tree:
git[4245] general protection ip:7f291f01461f sp:7fff5618a8b0 error:0 in libc-2.14.90.so[7f291ef9a000+15d000]
% gdb git
GNU gdb (Gentoo 7.3.1 p1) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/bin/git...done.
(gdb) run grep composite_pointer_type
Starting program: /usr/bin/git grep composite_pointer_type
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff7859700 (LWP 18367)]
[New Thread 0x7ffff7058700 (LWP 18368)]
[New Thread 0x7ffff6857700 (LWP 18369)]
[New Thread 0x7ffff6056700 (LWP 18370)]
[New Thread 0x7ffff5855700 (LWP 18371)]
[New Thread 0x7ffff5054700 (LWP 18372)]
[New Thread 0x7ffff4853700 (LWP 18373)]
[New Thread 0x7ffff4052700 (LWP 18374)]
Program received signal SIGSEGV, Segmentation fault.
_int_malloc (av=0x7ffff7bbb600, bytes=21) at malloc.c:3463
3463 while ((pp = catomic_compare_and_exchange_val_acq (fb, victim->fd, victim))
(gdb) bt
#0 _int_malloc (av=0x7ffff7bbb600, bytes=21) at malloc.c:3463
#1 0x00007ffff78d7300 in __GI___libc_malloc (bytes=21) at malloc.c:2924
#2 0x00007ffff78dc692 in __GI___strdup (s=0x7ffff2665760 "gcc/ada/i-cexten.ads") at strdup.c:43
#3 0x00000000004d5069 in xstrdup (str=0x7ffff2665760 "gcc/ada/i-cexten.ads") at wrapper.c:23
#4 0x000000000042c448 in grep_file_async (filename=0x7ffff2665760 "gcc/ada/i-cexten.ads", name=0x59fee0 "gcc/ada/i-cexten.ads",
opt=<optimized out>) at builtin/grep.c:148
#5 grep_file (opt=0x7fffffffbfc0, filename=0x7ffff2665760 "gcc/ada/i-cexten.ads") at builtin/grep.c:459
#6 0x000000000042ddb0 in grep_cache (cached=0, pathspec=0x7fffffffbf70, opt=0x7fffffffbfc0) at builtin/grep.c:528
#7 cmd_grep (argc=<optimized out>, argv=0x7ffff2665760, prefix=0x0) at builtin/grep.c:1062
#8 0x00000000004045b0 in run_builtin (argv=0x7fffffffe110, argc=2, p=0x536ba0) at git.c:308
#9 handle_internal_command (argc=2, argv=0x7fffffffe110) at git.c:466
#10 0x00000000004047ac in run_argv (argv=0x7fffffffdfa0, argcp=0x7fffffffdfac) at git.c:512
#11 main (argc=2, argv=0x7fffffffe110) at git.c:585
(gdb)
or:
% git grep composite_pointer_type
*** glibc detected *** git: double free or corruption (fasttop): 0x0000000001919800 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x792b6)[0x7f6ad1d392b6]
git[0x42bebb]
/lib64/libpthread.so.0(+0x7c9e)[0x7f6ad202ec9e]
/lib64/libc.so.6(clone+0x6d)[0x7f6ad1d99b8d]
======= Memory map: ========
00400000-00536000 r-xp 00000000 08:12 2310166 /usr/bin/git
00536000-0053d000 rw-p 00136000 08:12 2310166 /usr/bin/git
0053d000-0058b000 rw-p 00000000 00:00 0
01906000-01927000 rw-p 00000000 00:00 0 [heap]
...
And strange output:
...
gcc/cp/typeck.c: result_type = composite_pointer_type (type0, type1, op0, op1,
gcc/cp/typeck.c: result_type = composite_pointer_type (type0, type1, op0, op1,
error: 'gcc/testsuite/ada/acats/tests/c5/c54a24a.ada': short read No such file or directory
error: 'gcc/testsuite/ada/acats/tests/c5/c54a13a.ada': short read No such file or directory
error: 'gcc/testsuite/ada/acats/tests/c6/c64104m.ada': short read No such file or directory
error: 'gcc/testsuite/ada/acats/tests/cd/cd7101f.dep': short read No such file or directory
error: 'gcc/testsuite/ada/acats/tests/ce/ce3904a.ada': short read No such file or directory
error: 'gcc/testsuite/g++.dg/abi/pr39188-3b.C': short read No such file or directory
error: '<90>Dz^A': short read Is a directory
Note that all the above files actually exist.
All of this started with version v1.7.7.1, which I installed today. I
never had any problems with git before.
Any ideas what might be going on?
--
Markus
^ permalink raw reply
* Re: [IGNORETHIS/PATCH] Choosing the sha1 prefix of your commits
From: Jeff King @ 2011-10-24 20:47 UTC (permalink / raw)
To: Ævar Arnfjörð Bjarmason; +Cc: Git Mailing List
In-Reply-To: <20111020025149.GA31549@sigill.intra.peff.net>
On Wed, Oct 19, 2011 at 10:51:49PM -0400, Jeff King wrote:
> I couldn't resist:
>
> $ git commit -q -m foo --collide=deadbeef &&
> git rev-list -1 HEAD
> deadbeefdbd6e62a2185606a4fad653e22509b56
Here's a followup with a few cleanups:
1. It will actually finish and report an error if we fail to find a
commit after ULONG_MAX tries. On 64-bit machines this is unlikely,
but it might be possible on a 32-bit machine if you have a lot of
patience.
2. It uses git_SHA* now, so it will build properly against BLK_SHA1 if
NO_OPENSSL is defined.
3. It will split the the collision-finding work across online_cpus()
threads.
Somebody else suggested farming the work out to a GPU. I'll leave that
as an exercise to the reader.
-Peff
---
builtin/commit.c | 56 ++++++++++++++++++-
commit.c | 165 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
commit.h | 4 +
3 files changed, 223 insertions(+), 2 deletions(-)
diff --git a/builtin/commit.c b/builtin/commit.c
index c46f2d1..734a7ab 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -105,6 +105,10 @@
static const char *only_include_assumed;
static struct strbuf message;
+static int collide;
+static unsigned char collide_sha1[20];
+static unsigned char collide_mask[20];
+
static int null_termination;
static enum {
STATUS_FORMAT_LONG,
@@ -125,6 +129,52 @@ static int opt_parse_m(const struct option *opt, const char *arg, int unset)
return 0;
}
+static int parse_partial_sha1(const char *s, unsigned char sha1[20])
+{
+ unsigned int i;
+
+ hashclr(sha1);
+
+ for (i = 0; i < 40 && s[i]; i++) {
+ unsigned int v = hexval(s[i]);
+ if (v & ~0xf)
+ break;
+ if (!(i & 1))
+ v <<= 4;
+ sha1[i/2] |= v;
+ }
+ return i;
+}
+
+static void fill_sha1_mask(int n, unsigned char mask[20]) {
+ int i;
+
+ hashclr(mask);
+ for (i = 0; i < n/2; i++)
+ mask[i] = 0xff;
+ if (n & 1)
+ mask[i] = 0xf0;
+}
+
+static int opt_parse_collide(const struct option *opt, const char *arg,
+ int unset)
+{
+ if (unset)
+ collide = 0;
+ else {
+ int n = parse_partial_sha1(arg, collide_sha1);
+ if (!arg[n])
+ fill_sha1_mask(n, collide_mask);
+ else if (arg[n] == '/')
+ parse_partial_sha1(arg + n + 1, collide_mask);
+ else
+ die("invalid --collide sha1: %s", arg);
+ collide = 1;
+ }
+ return 0;
+}
+
+
static struct option builtin_commit_options[] = {
OPT__QUIET(&quiet, "suppress summary after successful commit"),
OPT__VERBOSE(&verbose, "show diff in commit message template"),
@@ -144,6 +194,7 @@ static int opt_parse_m(const struct option *opt, const char *arg, int unset)
OPT_BOOLEAN('e', "edit", &edit_flag, "force edit of commit"),
OPT_STRING(0, "cleanup", &cleanup_arg, "default", "how to strip spaces and #comments from message"),
OPT_BOOLEAN(0, "status", &include_status, "include status in commit message template"),
+ OPT_CALLBACK(0, "collide", NULL, "sha1[/mask]", "choose commit sha1 like <sha1>", opt_parse_collide),
/* end commit message options */
OPT_GROUP("Commit contents options"),
@@ -1483,8 +1534,9 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
exit(1);
}
- if (commit_tree(sb.buf, active_cache_tree->sha1, parents, sha1,
- author_ident.buf)) {
+ if (commit_tree_collide(sb.buf, active_cache_tree->sha1, parents, sha1,
+ author_ident.buf,
+ collide ? collide_sha1 : NULL, collide_mask)) {
rollback_index_files();
die(_("failed to write commit object"));
}
diff --git a/commit.c b/commit.c
index 73b7e00..7beea9b 100644
--- a/commit.c
+++ b/commit.c
@@ -6,6 +6,7 @@
#include "diff.h"
#include "revision.h"
#include "notes.h"
+#include "thread-utils.h"
int save_commit_buffer = 1;
@@ -840,6 +841,158 @@ struct commit_list *reduce_heads(struct commit_list *heads)
return result;
}
+static inline int sha1_match_mask(const unsigned char *sha1,
+ const unsigned char *want,
+ const unsigned char *mask)
+{
+ int i;
+ for (i = 0; i < 20; i++)
+ if ((want[i] & mask[i]) != (sha1[i] & mask[i]))
+ return 0;
+ return 1;
+}
+
+static unsigned long find_collision(const unsigned char *want,
+ const unsigned char *mask,
+ const git_SHA_CTX *base,
+ unsigned long start,
+ unsigned long end)
+{
+ unsigned long lulz;
+
+ for (lulz = start; lulz < end; lulz++) {
+ git_SHA_CTX guess;
+ unsigned char sha1[20];
+
+ memcpy(&guess, base, sizeof(guess));
+ git_SHA1_Update(&guess, &lulz, sizeof(lulz));
+ git_SHA1_Final(sha1, &guess);
+
+ if (sha1_match_mask(sha1, want, mask))
+ return lulz;
+ }
+ return end;
+}
+
+#ifndef NO_PTHREADS
+struct collision_thread_data {
+ const unsigned char *want;
+ const unsigned char *mask;
+ const git_SHA_CTX *base;
+ unsigned long start;
+ unsigned long end;
+
+ pthread_mutex_t *mutex;
+ pthread_cond_t *cond;
+ int *threads_alive;
+ unsigned long *answer;
+};
+
+static void *collision_thread(void *vdata)
+{
+ struct collision_thread_data *d = vdata;
+ unsigned long r;
+
+ pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
+
+ r = find_collision(d->want, d->mask, d->base,
+ d->start, d->end);
+
+ pthread_mutex_lock(d->mutex);
+
+ (*d->threads_alive)--;
+ if (r != d->end) {
+ *d->answer = r;
+ pthread_cond_signal(d->cond);
+ }
+ else {
+ /* If we failed to find it, and we're the last thread,
+ * wake up the parent so it can report failure. */
+ if (!*d->threads_alive)
+ pthread_cond_signal(d->cond);
+ }
+
+ pthread_mutex_unlock(d->mutex);
+ return NULL;
+}
+
+static unsigned long find_collision_threaded(int nthreads,
+ const unsigned char *want,
+ const unsigned char *mask,
+ const git_SHA_CTX *base)
+{
+ int i;
+ pthread_t *threads;
+ struct collision_thread_data *data;
+ pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
+ pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
+ int threads_alive = nthreads;
+ unsigned long ret = ULONG_MAX;
+
+ threads = xmalloc(nthreads * sizeof(*threads));
+ data = xmalloc(nthreads * sizeof(*data));
+
+ pthread_mutex_lock(&mutex);
+
+ for (i = 0; i < nthreads; i++) {
+ data[i].want = want;
+ data[i].mask = mask;
+ data[i].base = base;
+ data[i].start = i * (ULONG_MAX / nthreads);
+ data[i].end = (i+1) * (ULONG_MAX / nthreads);
+ data[i].mutex = &mutex;
+ data[i].cond = &cond;
+ data[i].answer = &ret;
+ data[i].threads_alive = &threads_alive;
+ pthread_create(&threads[i], NULL, collision_thread, &data[i]);
+ }
+
+ pthread_cond_wait(&cond, &mutex);
+
+ for (i = 0; i < nthreads; i++) {
+ pthread_cancel(threads[i]);
+ pthread_join(threads[i], NULL);
+ }
+
+ free(threads);
+ free(data);
+ return ret;
+}
+#endif /* NO_PTHREADS */
+
+
+static void collide_commit(struct strbuf *data,
+ const unsigned char *want,
+ const unsigned char *mask)
+{
+ static const char terminator[] = { 0 };
+ char header[32];
+ int header_len;
+ unsigned long lulz;
+ git_SHA_CTX base;
+
+ header_len = snprintf(header, sizeof(header),
+ "commit %lu",
+ data->len + 1 + sizeof(lulz)) + 1;
+ git_SHA1_Init(&base);
+ git_SHA1_Update(&base, header, header_len);
+ git_SHA1_Update(&base, data->buf, data->len);
+ git_SHA1_Update(&base, terminator, sizeof(terminator));
+
+#ifdef NO_PTHREADS
+ lulz = find_collision(want, mask, &base, 0, ULONG_MAX);
+#else
+ lulz = find_collision_threaded(online_cpus(), want, mask, &base);
+#endif /* NO_PTHREADS */
+
+ if (lulz != ULONG_MAX) {
+ strbuf_add(data, terminator, sizeof(terminator));
+ strbuf_add(data, &lulz, sizeof(lulz));
+ }
+ else
+ warning("sorry, I couldn't find a collision!");
+}
+
static const char commit_utf8_warn[] =
"Warning: commit message does not conform to UTF-8.\n"
"You may want to amend it after fixing the message, or set the config\n"
@@ -849,6 +1002,15 @@ int commit_tree(const char *msg, unsigned char *tree,
struct commit_list *parents, unsigned char *ret,
const char *author)
{
+ return commit_tree_collide(msg, tree, parents, ret, author,
+ NULL, NULL);
+}
+
+int commit_tree_collide(const char *msg, unsigned char *tree,
+ struct commit_list *parents, unsigned char *ret,
+ const char *author, const unsigned char *want,
+ const unsigned char *mask)
+{
int result;
int encoding_is_utf8;
struct strbuf buffer;
@@ -890,6 +1052,9 @@ int commit_tree(const char *msg, unsigned char *tree,
if (encoding_is_utf8 && !is_utf8(buffer.buf))
fprintf(stderr, commit_utf8_warn);
+ if (want && mask)
+ collide_commit(&buffer, want, mask);
+
result = write_sha1_file(buffer.buf, buffer.len, commit_type, ret);
strbuf_release(&buffer);
return result;
diff --git a/commit.h b/commit.h
index 009b113..337dcbd 100644
--- a/commit.h
+++ b/commit.h
@@ -184,5 +184,9 @@ static inline int single_parent(struct commit *commit)
extern int commit_tree(const char *msg, unsigned char *tree,
struct commit_list *parents, unsigned char *ret,
const char *author);
+extern int commit_tree_collide(const char *msg, unsigned char *tree,
+ struct commit_list *parents, unsigned char *ret,
+ const char *author, const unsigned char *sha1,
+ const unsigned char *mask);
#endif /* COMMIT_H */
--
1.7.7.40.g1a72f
^ permalink raw reply related
* Re: general protection faults with "git grep" version 1.7.7.1
From: Richard W.M. Jones @ 2011-10-24 21:49 UTC (permalink / raw)
To: Markus Trippelsdorf, meyering; +Cc: git
In-Reply-To: <20111024201153.GA1647@x4.trippels.de>
On Mon, Oct 24, 2011 at 10:11:53PM +0200, Markus Trippelsdorf wrote:
> Suddenly I'm getting strange protection faults when I run "git grep" on
> the gcc tree:
Jim Meyering and I are trying to chase what looks like a similar or
identical bug in git-grep. We've not got much further than gdb and
valgrind so far, but see:
https://bugzilla.redhat.com/show_bug.cgi?id=747377
It's slightly suspicious that this bug only started to happen with the
latest glibc, but that could be coincidence, or could be just that
glibc exposes a latent bug in git-grep.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
^ permalink raw reply
* Re: [PATCH] read-cache.c: fix index memory allocation
From: René Scharfe @ 2011-10-24 21:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, John Hsing, Matthieu Moy, git
In-Reply-To: <7vipne50lz.fsf@alter.siamese.dyndns.org>
Am 24.10.2011 09:07, schrieb Junio C Hamano:
> Thanks.
>
> This approach may be the most appropriate for the maintenance track, but
> for the purpose of going forward, I wonder if we really want to keep the
> "estimate and allocate a large pool, and carve out individual pieces".
>
> This bulk-allocate dates back to the days when we didn't have ondisk vs
> incore representation differences, IIRC, and as the result we deliberately
> leak cache entries whenever an entry in the index is replaced with a new
> one. Does the overhead to allocate individually really kill us that much
> for say a tree with 30k files in it?
Something like this (applies to master)? Very basic testing didn't show
any slowdown of git status in the Linux repo.
-- >8 --
Subject: read-cache.c: allocate index entries individually
The code to estimate the in-memory size of the index based on its on-disk
representation is subtly wrong for certain architecture-dependent struct
layouts. Instead of fixing it, replace the code to keep the index entries
in a single large block of memory and allocate each entry separately
instead. This is both simpler and more flexible, as individual entries
can now be freed. Actually using that added flexibility is left for a
later patch.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
---
Note: This patch was brought to you by the option --histogram, which
produced nicer output than the default diff method.
cache.h | 1 -
read-cache.c | 86 +++++++++++++++++++++++----------------------------------
2 files changed, 35 insertions(+), 52 deletions(-)
diff --git a/cache.h b/cache.h
index be07ec7..ec0e571 100644
--- a/cache.h
+++ b/cache.h
@@ -316,7 +316,6 @@ struct index_state {
struct string_list *resolve_undo;
struct cache_tree *cache_tree;
struct cache_time timestamp;
- void *alloc;
unsigned name_hash_initialized : 1,
initialized : 1;
struct hash_table name_hash;
diff --git a/read-cache.c b/read-cache.c
index 01a0e25..7f75bfa 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1202,10 +1202,35 @@ int read_index(struct index_state *istate)
return read_index_from(istate, get_index_file());
}
-static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_entry *ce)
+static struct cache_entry *create_from_disk(struct ondisk_cache_entry *ondisk)
{
+ struct cache_entry *ce;
size_t len;
const char *name;
+ unsigned int flags;
+
+ /* On-disk flags are just 16 bits */
+ flags = ntohs(ondisk->flags);
+ len = flags & CE_NAMEMASK;
+
+ if (flags & CE_EXTENDED) {
+ struct ondisk_cache_entry_extended *ondisk2;
+ int extended_flags;
+ ondisk2 = (struct ondisk_cache_entry_extended *)ondisk;
+ extended_flags = ntohs(ondisk2->flags2) << 16;
+ /* We do not yet understand any bit out of CE_EXTENDED_FLAGS */
+ if (extended_flags & ~CE_EXTENDED_FLAGS)
+ die("Unknown index entry format %08x", extended_flags);
+ flags |= extended_flags;
+ name = ondisk2->name;
+ }
+ else
+ name = ondisk->name;
+
+ if (len == CE_NAMEMASK)
+ len = strlen(name);
+
+ ce = xmalloc(cache_entry_size(len));
ce->ce_ctime.sec = ntohl(ondisk->ctime.sec);
ce->ce_mtime.sec = ntohl(ondisk->mtime.sec);
@@ -1217,48 +1242,16 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en
ce->ce_uid = ntohl(ondisk->uid);
ce->ce_gid = ntohl(ondisk->gid);
ce->ce_size = ntohl(ondisk->size);
- /* On-disk flags are just 16 bits */
- ce->ce_flags = ntohs(ondisk->flags);
+ ce->ce_flags = flags;
hashcpy(ce->sha1, ondisk->sha1);
- len = ce->ce_flags & CE_NAMEMASK;
-
- if (ce->ce_flags & CE_EXTENDED) {
- struct ondisk_cache_entry_extended *ondisk2;
- int extended_flags;
- ondisk2 = (struct ondisk_cache_entry_extended *)ondisk;
- extended_flags = ntohs(ondisk2->flags2) << 16;
- /* We do not yet understand any bit out of CE_EXTENDED_FLAGS */
- if (extended_flags & ~CE_EXTENDED_FLAGS)
- die("Unknown index entry format %08x", extended_flags);
- ce->ce_flags |= extended_flags;
- name = ondisk2->name;
- }
- else
- name = ondisk->name;
-
- if (len == CE_NAMEMASK)
- len = strlen(name);
/*
* NEEDSWORK: If the original index is crafted, this copy could
* go unchecked.
*/
memcpy(ce->name, name, len + 1);
-}
-
-static inline size_t estimate_cache_size(size_t ondisk_size, unsigned int entries)
-{
- long per_entry;
-
- per_entry = sizeof(struct cache_entry) - sizeof(struct ondisk_cache_entry);
-
- /*
- * Alignment can cause differences. This should be "alignof", but
- * since that's a gcc'ism, just use the size of a pointer.
- */
- per_entry += sizeof(void *);
- return ondisk_size + entries*per_entry;
+ return ce;
}
/* remember to discard_cache() before reading a different cache! */
@@ -1266,7 +1259,7 @@ int read_index_from(struct index_state *istate, const char *path)
{
int fd, i;
struct stat st;
- unsigned long src_offset, dst_offset;
+ unsigned long src_offset;
struct cache_header *hdr;
void *mmap;
size_t mmap_size;
@@ -1305,29 +1298,18 @@ int read_index_from(struct index_state *istate, const char *path)
istate->cache_nr = ntohl(hdr->hdr_entries);
istate->cache_alloc = alloc_nr(istate->cache_nr);
istate->cache = xcalloc(istate->cache_alloc, sizeof(struct cache_entry *));
-
- /*
- * The disk format is actually larger than the in-memory format,
- * due to space for nsec etc, so even though the in-memory one
- * has room for a few more flags, we can allocate using the same
- * index size
- */
- istate->alloc = xmalloc(estimate_cache_size(mmap_size, istate->cache_nr));
istate->initialized = 1;
src_offset = sizeof(*hdr);
- dst_offset = 0;
for (i = 0; i < istate->cache_nr; i++) {
struct ondisk_cache_entry *disk_ce;
struct cache_entry *ce;
disk_ce = (struct ondisk_cache_entry *)((char *)mmap + src_offset);
- ce = (struct cache_entry *)((char *)istate->alloc + dst_offset);
- convert_from_disk(disk_ce, ce);
+ ce = create_from_disk(disk_ce);
set_index_entry(istate, i, ce);
src_offset += ondisk_ce_size(ce);
- dst_offset += ce_size(ce);
}
istate->timestamp.sec = st.st_mtime;
istate->timestamp.nsec = ST_MTIME_NSEC(st);
@@ -1361,11 +1343,15 @@ unmap:
int is_index_unborn(struct index_state *istate)
{
- return (!istate->cache_nr && !istate->alloc && !istate->timestamp.sec);
+ return (!istate->cache_nr && !istate->timestamp.sec);
}
int discard_index(struct index_state *istate)
{
+ int i;
+
+ for (i = 0; i < istate->cache_nr; i++)
+ free(istate->cache[i]);
resolve_undo_clear_index(istate);
istate->cache_nr = 0;
istate->cache_changed = 0;
@@ -1374,8 +1360,6 @@ int discard_index(struct index_state *istate)
istate->name_hash_initialized = 0;
free_hash(&istate->name_hash);
cache_tree_free(&(istate->cache_tree));
- free(istate->alloc);
- istate->alloc = NULL;
istate->initialized = 0;
/* no need to throw away allocated active_cache */
--
1.7.7
^ permalink raw reply related
* Re: [PATCH v4 3/3] upload-archive: use start_command instead of fork
From: Jeff King @ 2011-10-24 22:39 UTC (permalink / raw)
To: Erik Faye-Lund; +Cc: git, gitster, j6t, rene.scharfe
In-Reply-To: <1319472131-3968-4-git-send-email-kusmabite@gmail.com>
On Mon, Oct 24, 2011 at 06:02:11PM +0200, Erik Faye-Lund wrote:
> The POSIX-function fork is not supported on Windows. Use our
> start_command API instead.
>
> As this is the last call-site that depends on the fork-stub in
> compat/mingw.h, remove that as well.
>
> Add an undocumented flag to git-archive that tells it that the
> action originated from a remote, so features can be disabled.
> Thanks to Jeff King for work on this part.
>
> Remove the NOT_MINGW-prereq for t5000, as git-archive --remote
> now works.
>
> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
> Helped-by: Jeff King <peff@peff.net>
> ---
As with other versions of this series, I have no comment on the mingw
bits. The archive.c parts still look sane to me. :)
-Peff
^ permalink raw reply
* Re: [PATCH 00/22] Refactor to accept NUL in commit messages
From: Jeff King @ 2011-10-24 22:45 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Nguyen Thai Ngoc Duy, Ævar Arnfjörð
In-Reply-To: <7vy5wb3sto.fsf@alter.siamese.dyndns.org>
On Sun, Oct 23, 2011 at 09:40:51PM -0700, Junio C Hamano wrote:
> >> But as Duy mentions, we have an encoding header. Shouldn't we treat it
> >> like binary goo until we do reencode_log_message, and _then_ we can
> >> break it into lines?
> >
> > That's sensible. If we go that route, I think the "one allocation of
> > separate struct commit_buffer pointed from a pointer field in struct
> > commit to replace the current member 'buffer'" is a reasonable thing
> > to do.
>
> Having given that "sensible" comment, I am not convinced if this is worth
> it. We are talking about what is left in the ephemeral COMMIT_EDITMSG by
> the chosen editor, but are there really editors that can _only_ write in
> UTF-16 and not in UTF-8, and is it worth bending backwards to add support
> such an editor?
Couldn't you make the same argument about iso8859-1, or any other
encoding? The user has some encoding that they want to use, for whatever
reason[1]. We have a slot for an encoding header; is there a reason that
git would allow some encodings and not others?
I mean, besides the obvious that UTF-16 is annoying and contains
embedded NULs and newlines.
-Peff
[1] English is my first language, so it's rare for me to even step
outside of ASCII, let alone latin1. But aren't there some languages in
which utf-16 is more efficient than utf-8?
^ permalink raw reply
* Re: general protection faults with "git grep" version 1.7.7.1
From: Markus Trippelsdorf @ 2011-10-24 22:58 UTC (permalink / raw)
To: Richard W.M. Jones; +Cc: meyering, git
In-Reply-To: <20111024214949.GA5237@amd.home.annexia.org>
On 2011.10.24 at 22:49 +0100, Richard W.M. Jones wrote:
> On Mon, Oct 24, 2011 at 10:11:53PM +0200, Markus Trippelsdorf wrote:
> > Suddenly I'm getting strange protection faults when I run "git grep" on
> > the gcc tree:
>
> Jim Meyering and I are trying to chase what looks like a similar or
> identical bug in git-grep. We've not got much further than gdb and
> valgrind so far, but see:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=747377
>
> It's slightly suspicious that this bug only started to happen with the
> latest glibc, but that could be coincidence, or could be just that
> glibc exposes a latent bug in git-grep.
Thanks for the pointer.
Compiling git with -O1 "solves" the problem for me.
This issue is independent of the exact git version being used (I tried
three different ones and always hit the problem).
It happens always on the _second_ run of "git grep" on my machine. The
first run always succeeds. So this might be a cache related issue.
--
Markus
^ permalink raw reply
* Re: [PATCH] Makefile: do not set setgid bit on directories on GNU/kFreeBSD
From: Greg Troxel @ 2011-10-24 23:07 UTC (permalink / raw)
To: Jonathan Nieder
Cc: Junio C Hamano, git, Alex Riesen, Christopher M. Fuhrman,
Stefan Sperling
In-Reply-To: <20111022111107.GA12130@elie.domain.sunraytvi.com>
[-- Attachment #1: Type: text/plain, Size: 1895 bytes --]
* On some BSD systems, adding +s bit on directories is detrimental
(it is not necessary on BSD to begin with). The installation
procedure has been updated to take this into account.
I looked at the NetBSD 5 sources, and as expected files are created
(unconditionally) with the gid of the parent directory.
Setting the setgid flag is only allowed if the inode's gid is in the
process gid set. This is really about files that might be executed,
but the check is independent of regular file/directory.
"git init --shared" creates a repository, mode 2775, and that normally
seems fine. It seems good to have the sgid bit on, in case the
repository is transferred to another machine with different semantics,
and it's a clue to humans about the intended behavior, even if it's
non-optional on BSD.
I created a directory, mode 755, owned by me, and with group that I *do
not* belong to. Then, "git init --shared" produced:
fatal: Could not make /home/gdt/FOO/.git/refs writable by group
but really the issue was setting the sgid bit:
# all with git version 1.7.6.3
13 $ l -d .git/refs/
drwxr-xr-x 2 gdt kmem 512 Oct 24 18:53 .git/refs/
14 $ chmod g+w .git/refs/
15 $ l -d .git/refs/
drwxrwxr-x 2 gdt kmem 512 Oct 24 18:53 .git/refs/
16 $ chmod g+s .git/refs/
chmod: .git/refs/: Operation not permitted
However, this is a pathological situation, because I've created a shared
repository that I can write to because I own it, and group kmem people
can write to because they're in the group, but I couldn't write to other
group kmem resources.
Is this not-allowed-to-set-setgid issue the problem the patch is trying
to avoid? Or something else?
I did run the regression tests at one point and don't remember this
failing.
So all in all I am agnostic as to whether DIR_HAS_BSD_GROUP_SEMANTICS
should be defined on NetBSD; personally I prefer to see the setgid
bits.
[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]
^ permalink raw reply
* Re: Git Bug - diff in commit message.
From: Martin von Zweigbergk @ 2011-10-24 23:24 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Michael Witten, anikey
In-Reply-To: <7v62jvdthi.fsf@alter.siamese.dyndns.org>
Hi,
On Tue, Oct 11, 2011 at 4:00 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> In the longer term, I think "git-rebase--am.sh" should be rewritten to
>> have format-patch avoid the cost of actually generating the patch text,
I'm (slowly) working on this.
> And here is a quick hack to do that using "log --cherry-pick --right-only".
Thanks. I had done something similar. I have now adopted most of your
changes into my patches, but I have a few questions.
> + if test -f "$dotest/rebasing"
> + then
> commit=$(sed -e 's/^From \([0-9a-f]*\) .*/\1/' \
> -e q "$dotest/$msgnum") &&
> + test "$(git cat-file -t "$commit")" = commit || {
> + stop_here $this
> + }
Are these braces needed?
> + git diff-tree -p --root "$commit" >"$dotest/patch"
In your previous mail in this thread, this was
+ git diff-tree --root --binary -m
--first-parent "$commit" >"$dotest/patch"
I see why you dropped "-m --first-parent"; we should simply never
receive such patches to "git-am --rebasing". But why isn't --binary
necessary?
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
>
> [...]
Why was this part left out? Did you get it to work using 'git log
--cherry-pick --right-only'? I'm asking because I did not get it to
work with except for the case where $onto=upstream (basically when
--onto is not given).
Martin
^ permalink raw reply
* Re: [PATCH] read-cache.c: fix index memory allocation
From: Nguyen Thai Ngoc Duy @ 2011-10-24 23:34 UTC (permalink / raw)
To: René Scharfe
Cc: Junio C Hamano, Jeff King, John Hsing, Matthieu Moy, git
In-Reply-To: <4EA5DFB2.3050406@lsrfire.ath.cx>
On Mon, Oct 24, 2011 at 11:59:14PM +0200, René Scharfe wrote:
> Am 24.10.2011 09:07, schrieb Junio C Hamano:
> > Thanks.
> >
> > This approach may be the most appropriate for the maintenance track, but
> > for the purpose of going forward, I wonder if we really want to keep the
> > "estimate and allocate a large pool, and carve out individual pieces".
> >
> > This bulk-allocate dates back to the days when we didn't have ondisk vs
> > incore representation differences, IIRC, and as the result we deliberately
> > leak cache entries whenever an entry in the index is replaced with a new
> > one. Does the overhead to allocate individually really kill us that much
> > for say a tree with 30k files in it?
>
> Something like this (applies to master)? Very basic testing didn't show
> any slowdown of git status in the Linux repo.
"git status" is slow. If your changes causes slowdown, it won't likely
stand out while other fast commands may show (read_cache() is used in
nearly all commands). So I tested using the following patch.
The result on linux-2.6 shows about 10-20 us slowdown per each
read_cache() call (30-40 us on webkit, ~50k files) I think your patch
is good enough :-)
-- 8< --
diff --git a/Makefile b/Makefile
index 3139c19..c8b47bc 100644
--- a/Makefile
+++ b/Makefile
@@ -444,6 +444,7 @@ TEST_PROGRAMS_NEED_X += test-string-pool
TEST_PROGRAMS_NEED_X += test-subprocess
TEST_PROGRAMS_NEED_X += test-svn-fe
TEST_PROGRAMS_NEED_X += test-treap
+TEST_PROGRAMS_NEED_X += test-read-cache
TEST_PROGRAMS = $(patsubst %,%$X,$(TEST_PROGRAMS_NEED_X))
diff --git a/test-read-cache.c b/test-read-cache.c
index e69de29..1ad80a0 100644
--- a/test-read-cache.c
+++ b/test-read-cache.c
@@ -0,0 +1,14 @@
+#include "cache.h"
+
+int main(int argc, char **argv)
+{
+ struct timeval tv1, tv2;
+ int i;
+
+ gettimeofday(&tv1, NULL);
+ for (i = 0; i < 100; i++)
+ read_cache();
+ gettimeofday(&tv2, NULL);
+ printf("%lu\n", (unsigned long)(tv2.tv_usec - tv1.tv_usec));
+ return 0;
+}
-- 8< --
^ permalink raw reply related
* Re: general protection faults with "git grep" version 1.7.7.1
From: Bernt Hansen @ 2011-10-25 0:00 UTC (permalink / raw)
To: Markus Trippelsdorf, Jeff King; +Cc: Richard W.M. Jones, meyering, git
In-Reply-To: <20111024225836.GA1678@x4.trippels.de>
Markus Trippelsdorf <markus@trippelsdorf.de> writes:
> On 2011.10.24 at 22:49 +0100, Richard W.M. Jones wrote:
>> On Mon, Oct 24, 2011 at 10:11:53PM +0200, Markus Trippelsdorf wrote:
>> > Suddenly I'm getting strange protection faults when I run "git grep" on
>> > the gcc tree:
>>
>> Jim Meyering and I are trying to chase what looks like a similar or
>> identical bug in git-grep. We've not got much further than gdb and
>> valgrind so far, but see:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=747377
>>
>> It's slightly suspicious that this bug only started to happen with the
>> latest glibc, but that could be coincidence, or could be just that
>> glibc exposes a latent bug in git-grep.
>
> Thanks for the pointer.
>
> Compiling git with -O1 "solves" the problem for me.
> This issue is independent of the exact git version being used (I tried
> three different ones and always hit the problem).
> It happens always on the _second_ run of "git grep" on my machine. The
> first run always succeeds. So this might be a cache related issue.
Hi,
I updated from an old commit 2883969 (Sync with maint, 2011-10-15)
to origin/master 10b2a48 (Merge branch 'maint', 2011-10-23) today and
promptly got segfaults on git status in my org-mode repository.
Going back to the old commit makes it work again.
Git bisect identifies the following commit as the problem:
[2548183badb98d62079beea62f9d2e1f47e99902] fix phantom untracked files when core.ignorecase is set
I'm doing make && make install to my local home directory.
On the above commit I get this:
--8<---------------cut here---------------start------------->8---
bernt@gollum:~/git/org$ git status
*** glibc detected *** git: free(): invalid next size (normal): 0x08ce1a88 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6(+0x6b281)[0xb749d281]
/lib/i686/cmov/libc.so.6(+0x6cad8)[0xb749ead8]
/lib/i686/cmov/libc.so.6(cfree+0x6d)[0xb74a1bbd]
/lib/i686/cmov/libc.so.6(+0x5c0a0)[0xb748e0a0]
/lib/i686/cmov/libc.so.6(fopen64+0x2c)[0xb749067c]
git[0x80ba49c]
git[0x810d0ab]
git[0x80627af]
git[0x804b867]
git[0x804ba73]
/lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7448c76]
git[0x804b141]
======= Memory map: ========
08048000-0814f000 r-xp 00000000 08:01 6555228 /home/bernt/git/bin/git
0814f000-08154000 rw-p 00106000 08:01 6555228 /home/bernt/git/bin/git
08154000-0819c000 rw-p 00000000 00:00 0
08cd7000-08cf8000 rw-p 00000000 00:00 0 [heap]
b7300000-b7321000 rw-p 00000000 00:00 0
b7321000-b7400000 ---p 00000000 00:00 0
b740f000-b742c000 r-xp 00000000 08:01 4603917 /lib/libgcc_s.so.1
b742c000-b742d000 rw-p 0001c000 08:01 4603917 /lib/libgcc_s.so.1
b742d000-b742e000 rw-p 00000000 00:00 0
b742e000-b7430000 r-xp 00000000 08:01 4620339 /lib/i686/cmov/libdl-2.11.2.so
b7430000-b7431000 r--p 00001000 08:01 4620339 /lib/i686/cmov/libdl-2.11.2.so
b7431000-b7432000 rw-p 00002000 08:01 4620339 /lib/i686/cmov/libdl-2.11.2.so
b7432000-b7572000 r-xp 00000000 08:01 4622760 /lib/i686/cmov/libc-2.11.2.so
b7572000-b7574000 r--p 0013f000 08:01 4622760 /lib/i686/cmov/libc-2.11.2.so
b7574000-b7575000 rw-p 00141000 08:01 4622760 /lib/i686/cmov/libc-2.11.2.so
b7575000-b7578000 rw-p 00000000 00:00 0
b7578000-b758d000 r-xp 00000000 08:01 4620333 /lib/i686/cmov/libpthread-2.11.2.so
b758d000-b758e000 r--p 00014000 08:01 4620333 /lib/i686/cmov/libpthread-2.11.2.so
b758e000-b758f000 rw-p 00015000 08:01 4620333 /lib/i686/cmov/libpthread-2.11.2.so
b758f000-b7592000 rw-p 00000000 00:00 0
b7592000-b76cf000 r-xp 00000000 08:01 794957 /usr/lib/i686/cmov/libcrypto.so.0.9.8
b76cf000-b76e7000 rw-p 0013c000 08:01 794957 /usr/lib/i686/cmov/libcrypto.so.0.9.8
b76e7000-b76ea000 rw-p 00000000 00:00 0
b76ea000-b76fd000 r-xp 00000000 08:01 286811 /usr/lib/libz.so.1.2.3.4
b76fd000-b76fe000 rw-p 00013000 08:01 286811 /usr/lib/libz.so.1.2.3.4
b771b000-b771d000 rw-p 00000000 00:00 0
b771d000-b771e000 r-xp 00000000 00:00 0 [vdso]
b771e000-b7739000 r-xp 00000000 08:01 4604271 /lib/ld-2.11.2.so
b7739000-b773a000 r--p 0001a000 08:01 4604271 /lib/ld-2.11.2.so
b773a000-b773b000 rw-p 0001b000 08:01 4604271 /lib/ld-2.11.2.so
bfa3d000-bfa52000 rw-p 00000000 00:00 0 [stack]
Aborted (core dumped)
--8<---------------cut here---------------end--------------->8---
Let me know if I can provide any more information.
Regards
Bernt
^ permalink raw reply
* Re: [PATCH] read-cache.c: fix index memory allocation
From: Nguyen Thai Ngoc Duy @ 2011-10-25 0:01 UTC (permalink / raw)
To: René Scharfe
Cc: Junio C Hamano, Jeff King, John Hsing, Matthieu Moy, git
In-Reply-To: <20111024233427.GA24956@duynguyen-vnpc.dektech.internal>
On Tue, Oct 25, 2011 at 10:34 AM, Nguyen Thai Ngoc Duy
<pclouds@gmail.com> wrote:
> "git status" is slow. If your changes causes slowdown, it won't likely
> stand out while other fast commands may show (read_cache() is used in
> nearly all commands). So I tested using the following patch.
>
> The result on linux-2.6 shows about 10-20 us slowdown per each
> read_cache() call (30-40 us on webkit, ~50k files) I think your patch
> is good enough :-)
That was with -O0 by the way. valgrind/massif shows about 200kb memory
more with your patch on webkit repository (7.497 MB vs 7.285 MB),
using the same test, so memory overhead is ok too.
--
Duy
^ permalink raw reply
* Re: What's cooking in git.git (Oct 2011, #09; Sun, 23)
From: Jakub Narebski @ 2011-10-25 0:07 UTC (permalink / raw)
To: Kato Kazuyoshi; +Cc: git, Junio C Hamano
In-Reply-To: <7v39ei4d9o.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
[...]
> By the way, I am planning to stop pushing the generated documentation
> branches to the above repositories in the near term, as they are not
> sources. The only reason the source repository at k.org has hosted these
> branches was because it was the only repository over there that was
> writable by me; it was an ugly historical and administrative workaround
> and not a demonstration of the best practice.
>
> They are pushed to their own separate repositories instead:
>
> git://git.kernel.org/pub/scm/git/git-{htmldocs,manpages}.git/
> git://repo.or.cz/git-{htmldocs,manpages}.git/
> https://code.google.com/p/git-{htmldocs,manpages}.git/
> https://github.com/gitster/git-{htmldocs,manpages}.git/
Junio, don't forget to update MaintNotes ("A note from maintainer")
when this happens, though.
> * kk/gitweb-side-by-side-diff (2011-10-17) 2 commits
> - gitweb: add a feature to show side-by-side diff
> - gitweb: change format_diff_line() to remove leading SP from $diff_class
Nb. I have originally intended to wait for Kato for re-roll with
cleaned up whitespace to be able to apply it and run it before doing
detailed review.
But then I got sidetracked with noticing inefficiency in diff body
line classification and Junio's proposal to fix it. Thic change as
written makes first patch in above series unnecessary. But because of
a bit unnecessary change it was not trivial to resolve Kato's changes
on top of it. This is 'gitweb/side-by-side-diff' branch
http://repo.or.cz/w/git/jnareb-git.git/patches/refs/heads/origin..refs/heads/gitweb/side-by-side-diff
Then I have separated adding navigation from other changes, and
basically rewrote layout for side-by-side diff, with adding "filler"
lines instead of box within box as in original submission.
While at it I have refactored HTML-formatting of chunk header lines
into separate subroutines, making format_diff_line much easier tor
read.
This can be found in 'gitweb/side-by-side-diff-v2' branch
http://repo.or.cz/w/git/jnareb-git.git/patches/refs/heads/origin..refs/heads/gitweb/side-by-side-diff-v2
Next I went back to Kato's box in box layout idea... kind of, as
context lines are grouped together, and not output line by line.
Also I have added here subtle backround color: separate for pure add,
pure remove, and [supposed] change. I think this makes side-by-side
diff more readable.
http://repo.or.cz/w/git/jnareb-git.git/patches/refs/heads/origin..refs/heads/gitweb/side-by-side-diff-v3
> Fun.
> Will keep in 'pu' until the planned re-roll comes.
I am now working on layout using HTML and CSS which would allow to
have a bit of separation between left and right panes in side-by-side
diff, but use pure-CSS "slantalicious"[1] to join background color
markers, e.g. a la http://meld.sourceforge.net/meld_file1.png
More fun!
[1] http://meyerweb.com/eric/css/edge/slantastic/demo.html
--
Jakub Narebski
^ permalink raw reply
* Re: general protection faults with "git grep" version 1.7.7.1
From: Jeff King @ 2011-10-25 5:53 UTC (permalink / raw)
To: Bernt Hansen; +Cc: Markus Trippelsdorf, Richard W.M. Jones, meyering, git
In-Reply-To: <878voaym7k.fsf@norang.ca>
On Mon, Oct 24, 2011 at 08:00:15PM -0400, Bernt Hansen wrote:
> I updated from an old commit 2883969 (Sync with maint, 2011-10-15)
> to origin/master 10b2a48 (Merge branch 'maint', 2011-10-23) today and
> promptly got segfaults on git status in my org-mode repository.
>
> Going back to the old commit makes it work again.
>
> Git bisect identifies the following commit as the problem:
>
> [2548183badb98d62079beea62f9d2e1f47e99902] fix phantom untracked files when core.ignorecase is set
I think this is a separate problem. See this thread and patch:
http://thread.gmane.org/gmane.comp.version-control.git/184094/focus=184148
-Peff
^ permalink raw reply
* Re: [PATCH v0] fast-import: Add drop command
From: Vitor Antunes @ 2011-10-25 9:56 UTC (permalink / raw)
To: Sverre Rabbelier; +Cc: Dmitry Ivankov, Jonathan Nieder, git, David Barr
In-Reply-To: <CAGdFq_heamPfKpK2sQ1RUvceaeGRVAwkv=KAn-ByPyPkNtoZBA@mail.gmail.com>
On Mon, Oct 24, 2011 at 7:01 PM, Sverre Rabbelier <srabbelier@gmail.com> wrote:
> I for one welcome our new branch deleting overlords :).
>
> You mention that checkpointing solves some of the concerns raised by
> others in this thread, would automatic checkpointing be way to make
> sure everything is as it should be?
Apparently I did not explain myself correctly. Let me try again :)
This is what I am doing:
1) import topic
2) checkpoint
3) diff-tree and processing
4) exit if processing returns ok
5) reset topic to another HEAD
6) goto 1)
In this scenario it is the checkpoint that "breaks" everything because
it will write the original tree to disk. When fast-import exits it will
find the old tree on disk but not within "topic" tree.
So, no, I don't think that automatic checkpointing would make anything
easier. Quite the opposite!
--
Vitor Antunes
^ permalink raw reply
* Re: [PATCH 00/22] Refactor to accept NUL in commit messages
From: Štěpán Němec @ 2011-10-25 10:16 UTC (permalink / raw)
To: Jeff King
Cc: Junio C Hamano, git, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð
In-Reply-To: <20111024224558.GB10481@sigill.intra.peff.net>
On Tue, 25 Oct 2011 00:45:58 +0200
Jeff King wrote:
> [1] English is my first language, so it's rare for me to even step
> outside of ASCII, let alone latin1. But aren't there some languages in
> which utf-16 is more efficient than utf-8?
You sometimes hear something along the lines of the second
"disadvantage" listed in the article below, i.e. "Characters U+0800
through U+FFFF use three bytes in UTF-8, but only two in UTF-16.":
https://en.wikipedia.org/wiki/UTF-8#Disadvantages_4
--
Štěpán
^ permalink raw reply
* Behavior of "git push --mirror repo"
From: Sebastian Schuberth @ 2011-10-25 10:25 UTC (permalink / raw)
To: git
Hi,
I cloned a repository from "origin" to my local disk. I only have a local branch for "master", although there are more remote branches. Now I want to initialize a new empty bare repository with an exact copy of the repository, including all branches, also those remote branches that I have no local branch for. So I did:
$ git remote add sf <url>
$ git push --mirror sf
Which prints:
* [new branch] master -> master
* [new branch] refs/notes/commits -> refs/notes/commits
* [new branch] refs/original/refs/heads/master -> refs/original/refs/heads/master
* [new branch] origin/HEAD -> origin/HEAD
* [new branch] origin/bourke -> origin/bourke
* [new branch] origin/colorscheme -> origin/colorscheme
* [new branch] origin/cpuinfo -> origin/cpuinfo
* [new branch] origin/demo-ssao -> origin/demo-ssao
* [new branch] origin/master -> origin/master
* [new branch] origin/mesh-improvements -> origin/mesh-improvements
* [new branch] origin/mesh-iterators-subdiv -> origin/mesh-iterators-subdiv
* [new tag] gale2-static-dummy-window -> gale2-static-dummy-window
In the target repository "sf" I now alony have the "master" branch and the "gale2-static-dummy-window" tag, but none of the remote branches from "origin". I was reading the --mirror option as if this should happen:
* [new branch] origin/bourke -> sf/bourke
etc.
Is this behavior expected? If yes, what the correct way to mirror a repository then (without creating a local branch for each remote branch first)?
Thanks.
--
Sebastian Schuberth
^ permalink raw reply
* Re: general protection faults with "git grep" version 1.7.7.1
From: Bernt Hansen @ 2011-10-25 11:11 UTC (permalink / raw)
To: Jeff King; +Cc: Markus Trippelsdorf, Richard W.M. Jones, meyering, git
In-Reply-To: <20111025055310.GB1902@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> On Mon, Oct 24, 2011 at 08:00:15PM -0400, Bernt Hansen wrote:
>
>> I updated from an old commit 2883969 (Sync with maint, 2011-10-15)
>> to origin/master 10b2a48 (Merge branch 'maint', 2011-10-23) today and
>> promptly got segfaults on git status in my org-mode repository.
>>
>> Going back to the old commit makes it work again.
>>
>> Git bisect identifies the following commit as the problem:
>>
>> [2548183badb98d62079beea62f9d2e1f47e99902] fix phantom untracked files when core.ignorecase is set
>
> I think this is a separate problem. See this thread and patch:
>
> http://thread.gmane.org/gmane.comp.version-control.git/184094/focus=184148
Thanks,
I'll look at that thread.
-Bernt
^ permalink raw reply
* Re: Behavior of "git push --mirror repo"
From: Sebastian Schuberth @ 2011-10-25 12:46 UTC (permalink / raw)
Cc: git
In-Reply-To: <j862ts$d75$1@dough.gmane.org>
On 25.10.2011 12:25, Sebastian Schuberth wrote:
> I cloned a repository from "origin" to my local disk. I only have a
Got it: I was missing "--bare" when doing my clone.
--
Sebastian Schuberth
^ permalink raw reply
* git stash show doesn't show stashed untracked files
From: Kirill Likhodedov @ 2011-10-25 13:21 UTC (permalink / raw)
To: git
Git 1.7.7 introduced a very useful feature - stashing untracked files via "-u" option.
However, there is a problems with it:
'git stash show' doesn't show stashed untracked files.
# git version
git version 1.7.7
# touch untracked.txt
# git stash save -u
# git stash show // no output
If changes in tracked files are stashed along with untracked files, then only tracked changes are shown in "git stash show" output.
Moreover, if I have the same file in the working tree, I wouldn't be able to 'git stash pop':
# git stash pop -v
untracked.txt already exists, no checkout
Could not restore untracked files from stash
In this case the only way to access the stashed content is to remove the tracked file and pop the stash again.
----------------------------------
Kirill Likhodedov
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"
^ permalink raw reply
* Re: general protection faults with "git grep" version 1.7.7.1
From: Thomas Rast @ 2011-10-25 13:50 UTC (permalink / raw)
To: Richard W.M. Jones
Cc: Markus Trippelsdorf, meyering, git, Shawn O. Pearce, Jeff King,
Nicolas Pitre
In-Reply-To: <20111024214949.GA5237@amd.home.annexia.org>
[Shawn, Peff, Nicolas: maybe you can say something on the
(non)raciness of xmalloc() in parallel with read_sha1_file(). See the
last paragraph below.]
Richard W.M. Jones wrote:
> On Mon, Oct 24, 2011 at 10:11:53PM +0200, Markus Trippelsdorf wrote:
> > Suddenly I'm getting strange protection faults when I run "git grep" on
> > the gcc tree:
>
> Jim Meyering and I are trying to chase what looks like a similar or
> identical bug in git-grep. We've not got much further than gdb and
> valgrind so far, but see:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=747377
>
> It's slightly suspicious that this bug only started to happen with the
> latest glibc, but that could be coincidence, or could be just that
> glibc exposes a latent bug in git-grep.
I'm tempted to write this off as a GCC bug. If that's ok for you,
I'll leave further investigation and communication with the GCC folks
to you.
My findings are as follows:
It's easy to reproduce the behavior described in the above bug report,
using an F16 beta install in a VM. I gave the VM two cores, but
didn't test what happens with only one. By "easy" I mean I didn't
have to do any fiddling and it crashes at least one out of two times.
I looked at how git builds grep.o by saying
rm builtin/grep.o; make V=1
I then modified this to give me the assembly output from the compiler
gcc -S -s builtin/grep.o -c -MF builtin/.depend/grep.o.d -MMD -MP -g -O2 -Wall -I. -DHAVE_PATHS_H -DSHA1_HEADER='<openssl/sha.h>' -DNO_STRLCPY -DNO_MKSTEMPS builtin/grep.c
and looked at the result. To interpret the output, I would like to
remind you of the following snippets:
#define grep_lock() pthread_mutex_lock(&grep_mutex)
#define grep_unlock() pthread_mutex_unlock(&grep_mutex)
...
static struct work_item *get_work(void)
{
struct work_item *ret;
grep_lock();
while (todo_start == todo_end && !all_work_added) {
pthread_cond_wait(&cond_add, &grep_mutex);
}
...
}
...
static void *run(void *arg)
{
int hit = 0;
struct grep_opt *opt = arg;
while (1) {
struct work_item *w = get_work();
...
}
...
}
Getting back to assembly, near the beginning of run() I see (labels
and .p2align snipped):
.loc 1 162 0
movl todo_end(%rip), %ebx
.loc 1 125 0
movl $grep_mutex, %edi
call pthread_mutex_lock
.loc 1 126 0
movl todo_start(%rip), %eax
cmpl %ebx, %eax
I should say that I don't really know much about assembly, in
particular not enough to write two correct lines of it. But I can't
help noticing that it moved the load of todo_end *out of* the section
where grep_mutex is locked. And the comment near the top of the file
does say that the whole todo_* family is supposed to be protected by
that mutex. What's extra odd is that the .loc seems to indicate that
the moved load comes from work_done() instead of get_work(), which is
an entirely separate locked section!
Un-inlining the get_work helper using __attribute__((noinline)) makes
the assembly
movl $grep_mutex, %edi
call pthread_mutex_lock
.loc 1 127 0
movl todo_start(%rip), %eax
cmpl todo_end(%rip), %eax
je .L15
instead; i.e., the load is now after the lock. (Note that line
numbers were wiggled by inserting an __attribute__ line.) The
beginning of run() turns into exactly the same code if I instead
prohibit inlining of work_done().
So AFAICS, we're just unlucky to hit a GCC optimizer bug that voids
all guarantees given on locks.
That being said, I'm not entirely convinced that the code in
builtin/grep.c works in the face of memory pressure. It guards
against concurrent access to read_sha1_file() with the
read_sha1_mutex, but any call to xmalloc() outside of that mutex can
still potentially invoke the try_to_free_routine. Maybe one of the
pack experts can say whether this is safe. (However, I implemented
locking around try_to_free_routine as a quick hack and it did not fix
the issue discussed in the bug report.)
--
Thomas Rast
trast@{inf,student}.ethz.ch
^ permalink raw reply
* Re: [PATCH 00/22] Refactor to accept NUL in commit messages
From: Junio C Hamano @ 2011-10-25 14:07 UTC (permalink / raw)
To: Jeff King; +Cc: git, Nguyen Thai Ngoc Duy, Ævar Arnfjörð
In-Reply-To: <20111024224558.GB10481@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> I mean, besides the obvious that UTF-16 is ...
Yes, you could, besides the obvious. But that obvious reason makes it
sufficiently different that it may not be so outrageous to draw the line
between it and all the others.
^ permalink raw reply
* Re: Git Bug - diff in commit message.
From: Junio C Hamano @ 2011-10-25 14:11 UTC (permalink / raw)
To: Martin von Zweigbergk; +Cc: git, Michael Witten, anikey
In-Reply-To: <CAOeW2eG_muSdbWUaPG36=1-Ay6h6-4qHgWPNdjqY5Zpb52XisQ@mail.gmail.com>
Martin von Zweigbergk <martin.von.zweigbergk@gmail.com> writes:
>> And here is a quick hack to do that using "log --cherry-pick --right-only".
>
> Thanks. I had done something similar. I have now adopted most of your
> changes into my patches, but I have a few questions.
It was a quick hack and not necessarily well thought out.
> Are these braces needed?
Probably not.
> In your previous mail in this thread, this was
> ...
> I see why you dropped "-m --first-parent"; we should simply never
> receive such patches to "git-am --rebasing". But why isn't --binary
> necessary?
It may be; I somehow thought we made it a no-op but didn't check.
> Why was this part left out?
No particular reason other than I was not looking at the earlier "how
about this" patch closely. It may be needed, it may not be, I dunno, and I
didn't check when I wrote this second quick hack.
Thanks.
^ permalink raw reply
* [PATCH/RFC] mingw: implement PTHREAD_MUTEX_INITIALIZER
From: Erik Faye-Lund @ 2011-10-25 14:55 UTC (permalink / raw)
To: msysgit; +Cc: git, johannes.schindelin, j.sixt
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---
Every now and then, someone suggest using PTHREAD_MUTEX_INITIALIZER,
but some times gets show down by our lack of support on Windows.
I'm working on something myself that could benefit from this, so I
gave it a stab. The result looks promising to me, but I haven't
really debugged it yet.
But is there a fundamental reason why we haven't done something like
this before? :)
compat/win32/pthread.c | 28 +++++++++++++++++++++++++---
compat/win32/pthread.h | 18 ++++++++++++------
2 files changed, 37 insertions(+), 9 deletions(-)
diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c
index 010e875..14f91d5 100644
--- a/compat/win32/pthread.c
+++ b/compat/win32/pthread.c
@@ -57,6 +57,28 @@ pthread_t pthread_self(void)
return t;
}
+int pthread_mutex_init(pthread_mutex_t *mutex, const pthread_mutexattr_t *attr)
+{
+ InitializeCriticalSection(&mutex->cs);
+ mutex->autoinit = 0;
+ return 0;
+}
+
+int pthread_mutex_lock(pthread_mutex_t *mutex)
+{
+ if (mutex->autoinit) {
+ if (InterlockedCompareExchange(&mutex->autoinit, -1, 1) != -1) {
+ pthread_mutex_init(mutex, NULL);
+ mutex->autoinit = 0;
+ } else
+ while (mutex->autoinit != 0)
+ ; /* wait for other thread */
+ }
+
+ EnterCriticalSection(&mutex->cs);
+ return 0;
+}
+
int pthread_cond_init(pthread_cond_t *cond, const void *unused)
{
cond->waiters = 0;
@@ -85,7 +107,7 @@ int pthread_cond_destroy(pthread_cond_t *cond)
return 0;
}
-int pthread_cond_wait(pthread_cond_t *cond, CRITICAL_SECTION *mutex)
+int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex)
{
int last_waiter;
@@ -99,7 +121,7 @@ int pthread_cond_wait(pthread_cond_t *cond, CRITICAL_SECTION *mutex)
* waiters count above, so there's no problem with
* leaving mutex unlocked before we wait on semaphore.
*/
- LeaveCriticalSection(mutex);
+ LeaveCriticalSection(&mutex->cs);
/* let's wait - ignore return value */
WaitForSingleObject(cond->sema, INFINITE);
@@ -133,7 +155,7 @@ int pthread_cond_wait(pthread_cond_t *cond, CRITICAL_SECTION *mutex)
*/
}
/* lock external mutex again */
- EnterCriticalSection(mutex);
+ EnterCriticalSection(&mutex->cs);
return 0;
}
diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h
index 2e20548..647e6d4 100644
--- a/compat/win32/pthread.h
+++ b/compat/win32/pthread.h
@@ -16,14 +16,20 @@
/*
* Defines that adapt Windows API threads to pthreads API
*/
-#define pthread_mutex_t CRITICAL_SECTION
+typedef struct {
+ CRITICAL_SECTION cs;
+ LONG volatile autoinit;
+} pthread_mutex_t;
-#define pthread_mutex_init(a,b) (InitializeCriticalSection((a)), 0)
-#define pthread_mutex_destroy(a) DeleteCriticalSection((a))
-#define pthread_mutex_lock EnterCriticalSection
-#define pthread_mutex_unlock LeaveCriticalSection
+#define PTHREAD_MUTEX_INITIALIZER { { 0 }, 1 }
typedef int pthread_mutexattr_t;
+
+int pthread_mutex_init(pthread_mutex_t *, const pthread_mutexattr_t *);
+#define pthread_mutex_destroy(a) DeleteCriticalSection(&(a)->cs)
+int pthread_mutex_lock(pthread_mutex_t *);
+#define pthread_mutex_unlock(a) LeaveCriticalSection(&(a)->cs)
+
#define pthread_mutexattr_init(a) (*(a) = 0)
#define pthread_mutexattr_destroy(a) do {} while (0)
#define pthread_mutexattr_settype(a, t) 0
@@ -47,7 +53,7 @@ typedef struct {
extern int pthread_cond_init(pthread_cond_t *cond, const void *unused);
extern int pthread_cond_destroy(pthread_cond_t *cond);
-extern int pthread_cond_wait(pthread_cond_t *cond, CRITICAL_SECTION *mutex);
+extern int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex);
extern int pthread_cond_signal(pthread_cond_t *cond);
extern int pthread_cond_broadcast(pthread_cond_t *cond);
--
1.7.7.msysgit.1.1.g7b316
^ permalink raw reply related
* Re: general protection faults with "git grep" version 1.7.7.1
From: Jim Meyering @ 2011-10-25 15:17 UTC (permalink / raw)
To: Thomas Rast
Cc: Richard W.M. Jones, Markus Trippelsdorf, git, Shawn O. Pearce,
Jeff King, Nicolas Pitre
In-Reply-To: <201110251550.22248.trast@student.ethz.ch>
Thomas Rast wrote:
> [Shawn, Peff, Nicolas: maybe you can say something on the
> (non)raciness of xmalloc() in parallel with read_sha1_file(). See the
> last paragraph below.]
>
> Richard W.M. Jones wrote:
>> On Mon, Oct 24, 2011 at 10:11:53PM +0200, Markus Trippelsdorf wrote:
>> > Suddenly I'm getting strange protection faults when I run "git grep" on
>> > the gcc tree:
>>
>> Jim Meyering and I are trying to chase what looks like a similar or
>> identical bug in git-grep. We've not got much further than gdb and
>> valgrind so far, but see:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=747377
>>
>> It's slightly suspicious that this bug only started to happen with the
>> latest glibc, but that could be coincidence, or could be just that
>> glibc exposes a latent bug in git-grep.
>
> I'm tempted to write this off as a GCC bug. If that's ok for you,
> I'll leave further investigation and communication with the GCC folks
> to you.
>
> My findings are as follows:
>
> It's easy to reproduce the behavior described in the above bug report,
> using an F16 beta install in a VM. I gave the VM two cores, but
> didn't test what happens with only one. By "easy" I mean I didn't
> have to do any fiddling and it crashes at least one out of two times.
>
> I looked at how git builds grep.o by saying
>
> rm builtin/grep.o; make V=1
>
> I then modified this to give me the assembly output from the compiler
>
> gcc -S -s builtin/grep.o -c -MF builtin/.depend/grep.o.d -MMD -MP -g -O2 -Wall -I. -DHAVE_PATHS_H -DSHA1_HEADER='<openssl/sha.h>' -DNO_STRLCPY -DNO_MKSTEMPS builtin/grep.c
...
> So AFAICS, we're just unlucky to hit a GCC optimizer bug that voids
> all guarantees given on locks.
Thanks for the investigation.
Actually, isn't gcc -O2's code-motion justified?
While we *know* that those globals may be modified asynchronously,
builtin/grep.c forgot to tell gcc about that.
Once you do that (via "volatile"), gcc knows not to move things.
This patch solved the problem for me:
>From 8521b8033b8ecbff2e459f9e0070beb712b9b73d Mon Sep 17 00:00:00 2001
From: Jim Meyering <meyering@redhat.com>
Date: Tue, 25 Oct 2011 17:07:05 +0200
Subject: [PATCH] declare grep's thread-related global scalars to be
"volatile"
This avoids heap corruption problems that would otherwise
arise when gcc -O2 moves code out of critical sections.
For details, see http://bugzilla.redhat.com/747377 and
http://thread.gmane.org/gmane.comp.version-control.git/184184/focus=184205
Signed-off-by: Jim Meyering <meyering@redhat.com>
---
builtin/grep.c | 12 ++++++------
1 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 7d0779f..38f92de 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -64,14 +64,14 @@ struct work_item {
*/
#define TODO_SIZE 128
static struct work_item todo[TODO_SIZE];
-static int todo_start;
-static int todo_end;
-static int todo_done;
+static volatile int todo_start;
+static volatile int todo_end;
+static volatile int todo_done;
-/* Has all work items been added? */
-static int all_work_added;
+/* Have all work items been added? */
+static volatile int all_work_added;
-/* This lock protects all the variables above. */
+/* This lock protects all of the above variables. */
static pthread_mutex_t grep_mutex;
/* Used to serialize calls to read_sha1_file. */
--
1.7.7.419.g87009
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox