* Re: [ANNOUNCE] git/gitweb.git repository
From: Petr Baudis @ 2007-09-04 9:38 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Junio C Hamano, git, jnareb, ltuikov
In-Reply-To: <46DCF41D.2070905@op5.se>
On Tue, Sep 04, 2007 at 07:58:53AM CEST, Andreas Ericsson wrote:
> Junio C Hamano wrote:
>> * Incremental blame
>> It does not seem to break the blame, but at least from where I
>> sit accessing repo.or.cz this does not look incremental to me.
>> The entire browser session freezes until the blame page
>> displays in full. My local installation behaves the same way.
>
> This is most likely due to what browser you're using. Some don't
> start rendering until they've read the entire output of an URL.
The incremental blame uses AJAX, so this shouldn't be an issue. But for
some browsers it seems to take long time to actually render the new ids
or something; I was reluctant about moving this particular one from next
to master because I have also seen problems with it and wanted to do
more tests. The benefit is mostly on files with long history buried deep
in the past.
--
Petr "Pasky" Baudis
Early to rise and early to bed makes a male healthy and wealthy and dead.
-- James Thurber
^ permalink raw reply
* Re: [PATCH] Add a new lstat and fstat implementation based on Win32 API
From: David Kastrup @ 2007-09-04 8:53 UTC (permalink / raw)
To: git
In-Reply-To: <46DD0C16.70101@eudaptics.com>
Johannes Sixt <j.sixt@eudaptics.com> writes:
> Marius Storm-Olsen schrieb:
>> Johannes Schindelin wrote:
>>> To make it easier on others, I just uploaded it into the "teststat"
>>> branch on 4msysgit.git (subject to removal in a few days).
>>
>> Ok, I've updated the patch in the 4msysgit.git repo, 'teststat' branch.
>> RFC, and please test.
>
> Thanks a lot! I've pushed it out in mingw.git's master.
>
> The reason that t4200-rerere.sh fails is that we now store UTC in
> st_mtime. However, for the garbage-collection we compare this entry
> to a local time stamp. Therefore, I've pushed out a fixup patch at
> the top of mingw.git's devel branch that converts mtime to local
> time
> (http://repo.or.cz/w/git/mingw.git?a=commitdiff;h=1b62ecb31068af06c2fa7664f06c6c36316aac2c). Would
> you kindly conduct the performance test with this patch? I'm afraid
> that this makes us substantially slower.
Wouldn't it make more sense to convert local time to mtime? That's
one conversion per second at most rather than one conversion per file.
--
David Kastrup
^ permalink raw reply
* [PATCH] Rework strbuf API and semantics.
From: Pierre Habouzit @ 2007-09-04 8:47 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <buobqcjrycl.fsf@dhapc248.dev.necel.com>
A strbuf can be used to store byte arrays, or as an extended string
library. The `buf' member can be passed to any C legacy string function,
because strbuf operations always ensure there is a terminating \0 at the end
of the buffer, not accounted in the `len' field of the structure.
A strbuf can be used to generate a string/buffer whose final size is not
really known, and then "strbuf_detach" can be used to get the built buffer,
and keep the wrapping "strbuf" structure usable for further work again.
Other interesting feature: buffer_ensure(sb, size) ensure that there is
enough allocated space in `sb' to put `size' new octets of data in the
buffer. It helps avoiding reallocating data for nothing when the problem the
strbuf helps to solve has a known typical size.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
archive-tar.c | 65 +++++++++++++---------------------------------------
fast-import.c | 24 +++++++++---------
mktree.c | 4 +--
strbuf.c | 71 +++++++++++++++++++++++++++++++++++++++++++++-----------
strbuf.h | 34 +++++++++++++++++++++++++++
5 files changed, 120 insertions(+), 78 deletions(-)
diff --git a/archive-tar.c b/archive-tar.c
index 66fe3e3..ccdd7d5 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -78,19 +78,6 @@ static void write_trailer(void)
}
}
-static void strbuf_append_string(struct strbuf *sb, const char *s)
-{
- int slen = strlen(s);
- int total = sb->len + slen;
- if (total + 1 > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total + 1);
- sb->alloc = total + 1;
- }
- memcpy(sb->buf + sb->len, s, slen);
- sb->len = total;
- sb->buf[total] = '\0';
-}
-
/*
* pax extended header records have the format "%u %s=%s\n". %u contains
* the size of the whole string (including the %u), the first %s is the
@@ -100,26 +87,17 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
static void strbuf_append_ext_header(struct strbuf *sb, const char *keyword,
const char *value, unsigned int valuelen)
{
- char *p;
- int len, total, tmp;
+ int len, tmp;
/* "%u %s=%s\n" */
len = 1 + 1 + strlen(keyword) + 1 + valuelen + 1;
for (tmp = len; tmp > 9; tmp /= 10)
len++;
- total = sb->len + len;
- if (total > sb->alloc) {
- sb->buf = xrealloc(sb->buf, total);
- sb->alloc = total;
- }
-
- p = sb->buf;
- p += sprintf(p, "%u %s=", len, keyword);
- memcpy(p, value, valuelen);
- p += valuelen;
- *p = '\n';
- sb->len = total;
+ strbuf_ensure(sb, len);
+ strbuf_addf(sb, "%u %s=", len, keyword);
+ strbuf_add(sb, value, valuelen);
+ strbuf_addch(sb, '\n');
}
static unsigned int ustar_header_chksum(const struct ustar_header *header)
@@ -153,8 +131,7 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
struct strbuf ext_header;
memset(&header, 0, sizeof(header));
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+ strbuf_init(&ext_header);
if (!sha1) {
*header.typeflag = TYPEFLAG_GLOBAL_HEADER;
@@ -225,8 +202,8 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
if (ext_header.len > 0) {
write_entry(sha1, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
}
+ strbuf_wipe(&ext_header);
write_blocked(&header, sizeof(header));
if (S_ISREG(mode) && buffer && size > 0)
write_blocked(buffer, size);
@@ -235,11 +212,11 @@ static void write_entry(const unsigned char *sha1, struct strbuf *path,
static void write_global_extended_header(const unsigned char *sha1)
{
struct strbuf ext_header;
- ext_header.buf = NULL;
- ext_header.len = ext_header.alloc = 0;
+
+ strbuf_init(&ext_header);
strbuf_append_ext_header(&ext_header, "comment", sha1_to_hex(sha1), 40);
write_entry(NULL, NULL, 0, ext_header.buf, ext_header.len);
- free(ext_header.buf);
+ strbuf_wipe(&ext_header);
}
static int git_tar_config(const char *var, const char *value)
@@ -260,28 +237,18 @@ static int write_tar_entry(const unsigned char *sha1,
const char *base, int baselen,
const char *filename, unsigned mode, int stage)
{
- static struct strbuf path;
+ static struct strbuf path = STRBUF_INIT;
int filenamelen = strlen(filename);
void *buffer;
enum object_type type;
unsigned long size;
- if (!path.alloc) {
- path.buf = xmalloc(PATH_MAX);
- path.alloc = PATH_MAX;
- path.len = path.eof = 0;
- }
- if (path.alloc < baselen + filenamelen + 1) {
- free(path.buf);
- path.buf = xmalloc(baselen + filenamelen + 1);
- path.alloc = baselen + filenamelen + 1;
- }
- memcpy(path.buf, base, baselen);
- memcpy(path.buf + baselen, filename, filenamelen);
- path.len = baselen + filenamelen;
- path.buf[path.len] = '\0';
+ strbuf_ensure(&path, MAX(PATH_MAX, baselen + filenamelen + 1));
+ strbuf_reset(&path);
+ strbuf_add(&path, base, baselen);
+ strbuf_add(&path, filename, filenamelen);
if (S_ISDIR(mode) || S_ISGITLINK(mode)) {
- strbuf_append_string(&path, "/");
+ strbuf_addch(&path, '/');
buffer = NULL;
size = 0;
} else {
diff --git a/fast-import.c b/fast-import.c
index 078079d..ab3927b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -340,7 +340,7 @@ static struct tag *last_tag;
/* Input stream parsing */
static whenspec_type whenspec = WHENSPEC_RAW;
-static struct strbuf command_buf;
+static struct strbuf command_buf = STRBUF_INIT;
static int unread_command_buf;
static struct recent_command cmd_hist = {&cmd_hist, &cmd_hist, NULL};
static struct recent_command *cmd_tail = &cmd_hist;
@@ -1595,7 +1595,7 @@ static void read_next_command(void)
} else {
struct recent_command *rc;
- command_buf.buf = NULL;
+ strbuf_detach(&command_buf);
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
return;
@@ -1610,7 +1610,7 @@ static void read_next_command(void)
free(rc->buf);
}
- rc->buf = command_buf.buf;
+ rc->buf = command_buf.buf;
rc->prev = cmd_tail;
rc->next = cmd_hist.prev;
rc->prev->next = rc;
@@ -1649,7 +1649,9 @@ static void *cmd_data (size_t *size)
size_t sz = 8192, term_len = command_buf.len - 5 - 2;
length = 0;
buffer = xmalloc(sz);
- command_buf.buf = NULL;
+
+ /* XXX possible memory leak ? */
+ strbuf_detach(&command_buf);
for (;;) {
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
@@ -1657,11 +1659,9 @@ static void *cmd_data (size_t *size)
if (term_len == command_buf.len
&& !strcmp(term, command_buf.buf))
break;
- ALLOC_GROW(buffer, length + command_buf.len, sz);
- memcpy(buffer + length,
- command_buf.buf,
- command_buf.len - 1);
- length += command_buf.len - 1;
+ ALLOC_GROW(buffer, length + command_buf.len + 1, sz);
+ memcpy(buffer + length, command_buf.buf, command_buf.len);
+ length += command_buf.len;
buffer[length++] = '\n';
}
free(term);
@@ -2101,7 +2101,7 @@ static void cmd_new_commit(void)
}
/* file_change* */
- while (!command_buf.eof && command_buf.len > 1) {
+ while (!command_buf.eof && command_buf.len > 0) {
if (!prefixcmp(command_buf.buf, "M "))
file_change_m(b);
else if (!prefixcmp(command_buf.buf, "D "))
@@ -2256,7 +2256,7 @@ static void cmd_reset_branch(void)
else
b = new_branch(sp);
read_next_command();
- if (!cmd_from(b) && command_buf.len > 1)
+ if (!cmd_from(b) && command_buf.len > 0)
unread_command_buf = 1;
}
@@ -2273,7 +2273,7 @@ static void cmd_checkpoint(void)
static void cmd_progress(void)
{
- fwrite(command_buf.buf, 1, command_buf.len - 1, stdout);
+ fwrite(command_buf.buf, 1, command_buf.len, stdout);
fputc('\n', stdout);
fflush(stdout);
skip_optional_lf();
diff --git a/mktree.c b/mktree.c
index d86dde8..86de5eb 100644
--- a/mktree.c
+++ b/mktree.c
@@ -92,7 +92,6 @@ int main(int ac, char **av)
strbuf_init(&sb);
while (1) {
- int len;
char *ptr, *ntr;
unsigned mode;
enum object_type type;
@@ -101,7 +100,6 @@ int main(int ac, char **av)
read_line(&sb, stdin, line_termination);
if (sb.eof)
break;
- len = sb.len;
ptr = sb.buf;
/* Input is non-recursive ls-tree output format
* mode SP type SP sha1 TAB name
@@ -111,7 +109,7 @@ int main(int ac, char **av)
die("input format error: %s", sb.buf);
ptr = ntr + 1; /* type */
ntr = strchr(ptr, ' ');
- if (!ntr || sb.buf + len <= ntr + 41 ||
+ if (!ntr || sb.buf + sb.len <= ntr + 40 ||
ntr[41] != '\t' ||
get_sha1_hex(ntr + 1, sha1))
die("input format error: %s", sb.buf);
diff --git a/strbuf.c b/strbuf.c
index e33d06b..86d5965 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -2,40 +2,83 @@
#include "strbuf.h"
void strbuf_init(struct strbuf *sb) {
- sb->buf = NULL;
- sb->eof = sb->alloc = sb->len = 0;
+ memset(sb, 0, sizeof(*sb));
}
-static void strbuf_begin(struct strbuf *sb) {
+void strbuf_wipe(struct strbuf *sb) {
free(sb->buf);
- strbuf_init(sb);
+ memset(sb, 0, sizeof(*sb));
}
-static void inline strbuf_add(struct strbuf *sb, int ch) {
- if (sb->alloc <= sb->len) {
- sb->alloc = sb->alloc * 3 / 2 + 16;
+void strbuf_reset(struct strbuf *sb) {
+ if (sb->len)
+ sb->buf[sb->len = 0] = '\0';
+}
+
+char *strbuf_detach(struct strbuf *sb) {
+ char *res = sb->buf;
+ strbuf_init(sb);
+ return res;
+}
+
+void strbuf_ensure(struct strbuf *sb, int extra) {
+ if (sb->len + extra >= sb->alloc) {
+ while (sb->len + extra >= sb->alloc)
+ sb->alloc = sb->alloc * 3 / 2 + 16;
sb->buf = xrealloc(sb->buf, sb->alloc);
}
- sb->buf[sb->len++] = ch;
}
-static void strbuf_end(struct strbuf *sb) {
- strbuf_add(sb, 0);
+void strbuf_add(struct strbuf *sb, const void *data, int len) {
+ strbuf_ensure(sb, len);
+ memcpy(sb->buf + sb->len, data, len);
+ sb->len += len;
+ sb->buf[sb->len] = '\0';
+}
+
+void strbuf_addvf(struct strbuf *sb, const char *fmt, va_list ap)
+{
+ int len;
+ va_list ap2;
+
+ va_copy(ap2, ap);
+
+ len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap);
+ if (len < 0) {
+ len = 0;
+ }
+ if (len >= sb->alloc - sb->len) {
+ strbuf_ensure(sb, len);
+ len = vsnprintf(sb->buf + sb->len, sb->alloc - sb->len, fmt, ap2);
+ if (len >= sb->alloc - sb->len) {
+ len = sb->alloc - sb->len - 1;
+ }
+ }
+ sb->len = sb->len + len;
+ sb->buf[sb->len] = '\0';
}
void read_line(struct strbuf *sb, FILE *fp, int term) {
int ch;
- strbuf_begin(sb);
if (feof(fp)) {
+ strbuf_wipe(sb);
sb->eof = 1;
return;
}
+
+ strbuf_reset(sb);
while ((ch = fgetc(fp)) != EOF) {
if (ch == term)
break;
- strbuf_add(sb, ch);
+ strbuf_ensure(sb, 1);
+ sb->buf[sb->len++] = ch;
}
- if (ch == EOF && sb->len == 0)
+ if (ch == EOF && sb->len == 0) {
+ strbuf_wipe(sb);
sb->eof = 1;
- strbuf_end(sb);
+ }
+
+ strbuf_ensure(sb, 1);
+ sb->buf[sb->len] = '\0';
}
+
diff --git a/strbuf.h b/strbuf.h
index 74cc012..ee72419 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -7,7 +7,41 @@ struct strbuf {
char *buf;
};
+#define STRBUF_INIT { 0, 0, 0, NULL }
+
+/* strbuf life cycle */
extern void strbuf_init(struct strbuf *);
+extern void strbuf_wipe(struct strbuf *);
+extern void strbuf_reset(struct strbuf *);
+extern char *strbuf_detach(struct strbuf *);
+
+
+extern void strbuf_ensure(struct strbuf *, int);
+extern void strbuf_add(struct strbuf *, const void *data, int len);
+
+static inline void strbuf_addstr(struct strbuf *sb, const char *s) {
+ strbuf_add(sb, s, strlen(s));
+}
+static inline void strbuf_addbuf(struct strbuf *sb, struct strbuf *sb2) {
+ strbuf_add(sb, sb2->buf, sb2->len);
+}
+static inline void strbuf_addch(struct strbuf *sb, int c) {
+ strbuf_ensure(sb, 1);
+ sb->buf[sb->len++] = c;
+ sb->buf[sb->len] = '\0';
+}
+
+__attribute__((format(printf,2,0)))
+extern void strbuf_addvf(struct strbuf *, const char *, va_list);
+
+static inline __attribute__((format(printf,2,3)))
+void strbuf_addf(struct strbuf *sb, const char *fmt, ...) {
+ va_list ap;
+ va_start(ap, fmt);
+ strbuf_addvf(sb, fmt, ap);
+ va_end(ap);
+}
+
extern void read_line(struct strbuf *, FILE *, int);
#endif /* STRBUF_H */
--
1.5.3
^ permalink raw reply related
* [PATCH] Add strbuf_fread, use it in fast-import.c.
From: Pierre Habouzit @ 2007-09-04 8:48 UTC (permalink / raw)
To: git; +Cc: Pierre Habouzit
In-Reply-To: <buobqcjrycl.fsf@dhapc248.dev.necel.com>
This is a good proof of concept that combined with strbuf_detach it often
helps improving readability a lot, while remaining as efficient as before.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
fast-import.c | 26 ++++++++++----------------
strbuf.c | 12 ++++++++++++
strbuf.h | 3 +++
3 files changed, 25 insertions(+), 16 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index ab3927b..2d8daeb 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1638,20 +1638,15 @@ static void cmd_mark(void)
static void *cmd_data (size_t *size)
{
- size_t length;
- char *buffer;
+ struct strbuf buffer = STRBUF_INIT;
if (prefixcmp(command_buf.buf, "data "))
die("Expected 'data n' command, found: %s", command_buf.buf);
if (!prefixcmp(command_buf.buf + 5, "<<")) {
char *term = xstrdup(command_buf.buf + 5 + 2);
- size_t sz = 8192, term_len = command_buf.len - 5 - 2;
- length = 0;
- buffer = xmalloc(sz);
+ size_t term_len = command_buf.len - 5 - 2;
- /* XXX possible memory leak ? */
- strbuf_detach(&command_buf);
for (;;) {
read_line(&command_buf, stdin, '\n');
if (command_buf.eof)
@@ -1659,19 +1654,18 @@ static void *cmd_data (size_t *size)
if (term_len == command_buf.len
&& !strcmp(term, command_buf.buf))
break;
- ALLOC_GROW(buffer, length + command_buf.len + 1, sz);
- memcpy(buffer + length, command_buf.buf, command_buf.len);
- length += command_buf.len;
- buffer[length++] = '\n';
+ strbuf_addbuf(&buffer, &command_buf);
+ strbuf_addch(&buffer, '\n');
}
free(term);
}
else {
- size_t n = 0;
+ size_t n = 0, length;
+
length = strtoul(command_buf.buf + 5, NULL, 10);
- buffer = xmalloc(length);
+
while (n < length) {
- size_t s = fread(buffer + n, 1, length - n, stdin);
+ size_t s = strbuf_fread(&buffer, length - n, stdin);
if (!s && feof(stdin))
die("EOF in data (%lu bytes remaining)",
(unsigned long)(length - n));
@@ -1680,8 +1674,8 @@ static void *cmd_data (size_t *size)
}
skip_optional_lf();
- *size = length;
- return buffer;
+ *size = buffer.len;
+ return strbuf_detach(&buffer);
}
static int validate_raw_date(const char *src, char *result, int maxlen)
diff --git a/strbuf.c b/strbuf.c
index 86d5965..693fbcf 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -58,6 +58,18 @@ void strbuf_addvf(struct strbuf *sb, const char *fmt, va_list ap)
sb->buf[sb->len] = '\0';
}
+size_t strbuf_fread(struct strbuf *sb, size_t size, FILE *f) {
+ size_t res;
+
+ strbuf_ensure(sb, size);
+ res = fread(sb->buf + sb->len, 1, size, f);
+ if (res > 0) {
+ sb->len += res;
+ sb->buf[sb->len] = '\0';
+ }
+ return res;
+}
+
void read_line(struct strbuf *sb, FILE *fp, int term) {
int ch;
if (feof(fp)) {
diff --git a/strbuf.h b/strbuf.h
index ee72419..20c5255 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -42,6 +42,9 @@ void strbuf_addf(struct strbuf *sb, const char *fmt, ...) {
va_end(ap);
}
+
+extern size_t strbuf_fread(struct strbuf *, size_t, FILE *);
+
extern void read_line(struct strbuf *, FILE *, int);
#endif /* STRBUF_H */
--
1.5.3
^ permalink raw reply related
* strbuf new semantics, let's give it a try
From: Pierre Habouzit @ 2007-09-04 8:47 UTC (permalink / raw)
To: git
In-Reply-To: <buobqcjrycl.fsf@dhapc248.dev.necel.com>
Allright, here is a proposal for a new strbuf semantics like the one I
proposed before. Like said, with this proposal, strbuf's always have a
'\0' at the end of the buffer, not accounted in its length, so that any
C str* function can still be used.
The second patch demonstrate that strbufs makes the code smaller,
simpler to read, while keeping the exact same efficiency (as may
mallocs, memcpys, and fread's in that case).
I've tried to keep the amount of inlines low. Though, I wonder if
strbuf_add shouldn't be inlined (so that memcpys could be even more
inlined/optmized by gcc). And strbuf_addstr is inlined so that the
strlen call is optimized when the argument is an immediate string.
Meaning that calling strbuf_addstr(sb, "foo") is not sloppy at all.
I don't really like the buffer growth scheme right now, I just stick
with the one that was used in git before. Though I really belive it's
inelegant as is, and should be simplified (strbuf_extend).
Cheers,
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Junio C Hamano @ 2007-09-04 8:19 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: Petr Baudis, git, jnareb, ltuikov
In-Reply-To: <20070904055048.GW18160@spearce.org>
"Shawn O. Pearce" <spearce@spearce.org> writes:
> Junio C Hamano <gitster@pobox.com> wrote:
>> * Incremental blame
>>
>> It does not seem to break the blame, but at least from where I
>> sit accessing repo.or.cz this does not look incremental to me.
>> The entire browser session freezes until the blame page
>> displays in full. My local installation behaves the same way.
>
> What kind of repo are you trying that on?
linux-2.6.git (block/ll_rw_blk.c).
But I think I figured it out. With firebug disabled, I can see
it is trying incremental.
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Luben Tuikov @ 2007-09-04 7:56 UTC (permalink / raw)
To: Junio C Hamano, Petr Baudis; +Cc: git, jnareb, ltuikov
In-Reply-To: <7v8x7n7zqn.fsf@gitster.siamese.dyndns.org>
--- Junio C Hamano <gitster@pobox.com> wrote:
> Here is my impressions on patches in:
>
> git://repo.or.cz/git/gitweb.git/ next
>
> * Removal of git_blame
>
> I think this makes sense; we do not need two not-so-different
> implementations, and the other one has been the default for
> quite some time.
My impression of the running gitweb installation at said site
is that it doesn't implement 244a70e608204a515c214a11c43f3ecf7642533a.
It is very easy to verify that it is not supported.
This is a data mining feature that I use regularly.
Luben
>
> * Extra columns in blame
>
> I can see why some people especially with wide screen would
> want this, and toggling the display for extra columns is also
> a good idea. But this makes me wonder if the single button at
> the very top is easy to locate (maybe explain what [+] does?)
> and easy to access (maybe have it at the top and at least at
> the bottom as well?).
>
> * Incremental blame
>
> It does not seem to break the blame, but at least from where I
> sit accessing repo.or.cz this does not look incremental to me.
> The entire browser session freezes until the blame page
> displays in full. My local installation behaves the same way.
>
>
>
>
>
^ permalink raw reply
* Re: [PATCH] Add a new lstat and fstat implementation based on Win32 API
From: Johannes Sixt @ 2007-09-04 7:41 UTC (permalink / raw)
To: Marius Storm-Olsen; +Cc: Johannes Schindelin, Johannes Sixt, Git Mailing List
In-Reply-To: <46DC5ED4.8050202@trolltech.com>
Marius Storm-Olsen schrieb:
> Johannes Schindelin wrote:
>> To make it easier on others, I just uploaded it into the "teststat"
>> branch on 4msysgit.git (subject to removal in a few days).
>
> Ok, I've updated the patch in the 4msysgit.git repo, 'teststat' branch.
> RFC, and please test.
Thanks a lot! I've pushed it out in mingw.git's master.
The reason that t4200-rerere.sh fails is that we now store UTC in st_mtime.
However, for the garbage-collection we compare this entry to a local time
stamp. Therefore, I've pushed out a fixup patch at the top of mingw.git's
devel branch that converts mtime to local time
(http://repo.or.cz/w/git/mingw.git?a=commitdiff;h=1b62ecb31068af06c2fa7664f06c6c36316aac2c).
Would you kindly conduct the performance test with this patch? I'm afraid
that this makes us substantially slower.
-- Hannes
^ permalink raw reply
* Re: [PATCH] Improve bash prompt to detect merge / rebase in progress
From: Shawn O. Pearce @ 2007-09-04 7:13 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: Johannes Sixt, git, Junio C Hamano
In-Reply-To: <200709021240.36807.robin.rosenberg@dewire.com>
Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> This patch makes the git prompt (when enabled) show if a merge or a
> rebase is unfinished. It also detects if a bisect is being done as
> well as detached checkouts.
What about this instead?
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index cad842a..41a558c 100755
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -64,12 +64,41 @@ __gitdir ()
__git_ps1 ()
{
- local b="$(git symbolic-ref HEAD 2>/dev/null)"
- if [ -n "$b" ]; then
+ local g="$(git rev-parse --git-dir 2>/dev/null)"
+ if [ -n "$g" ]; then
+ local r
+ local b
+ if [ -d "$g/../.dotest" ]
+ then
+ r="|AM/REBASE"
+ b="$(git symbolic-ref HEAD 2>/dev/null)"
+ elif [ -f "$g/.dotest-merge/interactive" ]
+ then
+ r="|REBASE-i"
+ b="$(cat $g/.dotest-merge/head-name)"
+ elif [ -d "$g/.dotest-merge" ]
+ then
+ r="|REBASE-m"
+ b="$(cat $g/.dotest-merge/head-name)"
+ elif [ -f "$g/MERGE_HEAD" ]
+ then
+ r="|MERGING"
+ b="$(git symbolic-ref HEAD 2>/dev/null)"
+ else
+ if [ -f $g/BISECT_LOG ]
+ then
+ r="|BISECTING"
+ fi
+ if ! b="$(git symbolic-ref HEAD 2>/dev/null)"
+ then
+ b="$(cut -c1-7 $g/HEAD)..."
+ fi
+ fi
+
if [ -n "$1" ]; then
- printf "$1" "${b##refs/heads/}"
+ printf "$1" "${b##refs/heads/}$r"
else
- printf " (%s)" "${b##refs/heads/}"
+ printf " (%s)" "${b##refs/heads/}$r"
fi
fi
}
--
Shawn.
^ permalink raw reply related
* [PATCH] fix memleak from tree pathspec in git-blame
From: Junio C Hamano @ 2007-09-04 6:54 UTC (permalink / raw)
To: git
We repeatedly run diff-tree while finding code movements. The
pathspec data is changed into a list of paths and their lengths
for quicker access by diff_tree_setup_paths(), and you need to
release the extra memory when you are done using that pathspec
data.
Not releasing it is not a problem for "git log -- path" (or its
plumbing equivalent "git rev-list -- paths"), because the
pathspec never changes during the traversal, but it was
noticeable for blame especially with -C -C.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
builtin-blame.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/builtin-blame.c b/builtin-blame.c
index dc88a95..aace08c 100644
--- a/builtin-blame.c
+++ b/builtin-blame.c
@@ -380,6 +380,7 @@ static struct origin *find_origin(struct scoreboard *sb,
}
}
diff_flush(&diff_opts);
+ diff_tree_release_paths(&diff_opts);
if (porigin) {
/*
* Create a freestanding copy that is not part of
@@ -436,6 +437,7 @@ static struct origin *find_rename(struct scoreboard *sb,
}
}
diff_flush(&diff_opts);
+ diff_tree_release_paths(&diff_opts);
return porigin;
}
@@ -1157,6 +1159,7 @@ static int find_copy_in_parent(struct scoreboard *sb,
}
}
diff_flush(&diff_opts);
+ diff_tree_release_paths(&diff_opts);
return retval;
}
^ permalink raw reply related
* Re: [PATCH 0/5] gitweb: Support for arbitrary diffs
From: Martin Koegler @ 2007-09-04 6:31 UTC (permalink / raw)
To: Jakub Narebski; +Cc: Petr Baudis, git
In-Reply-To: <200709031023.42366.jnareb@gmail.com>
On Mon, Sep 03, 2007 at 10:23:41AM +0200, Jakub Narebski wrote:
> On Mon, 3 September 2007, Petr "Pasky" Baudis wrote:
>
> > To hijack this post a bit, another patch in the queue (the incremental
> > blame thingie) introduces blame.js. Do you think that we should keep the
> > .js files separate, or instead have one big gitweb.js with everything?
> > I'm inclined to the second possibility in order to reduce the number of
> > requests, but it comes at a price of slightly worse maintainability.
Why is the maintainablility reduced?
gitweb.perl is also a collection of different function, which are kept
in one file. Where is the difference to a javascript file?
Keeping everything into one file will make it more likely, that the
code is not developed totally different (in the terms of code style,
variable/function names, ...).
In the moment, there is the problem, which patch should introduce the
js file. We need a common base patch, which introduces an empty
gitweb.js.
Keeping the two functions separate has problems. Every patch need to
hook into the onload event. To avoid merge conflicts, my current patch
saves the old handler and overwrite it with its on version, which
calls back into the saved handler.
If we would have on js file, we can add in the base patch an empty
JavaScript function for this.
Then Hooking in the onload event would mean, to only add some new
lines to the function. The resulting merge conflicts are
easier to resolve (as they affect independet lines) compared to
the current '<body onload="hook1(); hook2();">'
> On the other hand if we have blame.js separate, we could load it
> (require it) only for the 'blame' view, it means only when needed.
>
> gitweb.js would contain JavaScript code used by all (or almost all)
> views, then...
>
> I don't think gitweb.js would be as large as gitweb.perl, if we are
> talking about maintability ;-)
The size of gitweb.js should not matter. On a modern browser, the
first request will fetch the whole Javascript file. For subseqent
request, the webserver returns "not modified". Having two javascript
file means two checks.
mfg Martin Kögler
^ permalink raw reply
* Re: Calculating tree nodes
From: Shawn O. Pearce @ 2007-09-04 6:26 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910709032026s7f94eed9h25d5165840cc38d2@mail.gmail.com>
Jon Smirl <jonsmirl@gmail.com> wrote:
> Index is the key here, we may want other kinds of indexes in the
> future. It was the mail about auto-generating the Maintainers list
> that caused me to think about this. If file objects are a table with
> triggers, building a hierarchical index for the Maintainers field
> doesn't make sense.
There's nothing stopping us from creating additional indexes.
For example we have been kicking around this idea of a "note"
object that can be attached to commits. Lightweight enough that one
could be attached to every commit, such as one way to implement the
"Signed-off-by" lines. Notes can be looked up by commit SHA-1 using
a hash, giving near O(1) time to locate the note for any commit.
But we can also store the notes alongside the commits in the
packfile, so that if the data for the commit has been paged in
by the kernel then the note data is also most likely in memory,
and if not, is in the read-ahead queue. Clustering the notes
alongside the commits makes access to them even faster, as we
don't need to consult an external hash to locate the position.
So I guess where I'm going is additional indexes can be implemented,
efficiently, without changing any of the core storage model in Git.
We just haven't made it easily user pluggable yet, because nobody
has really thought about the applications for such a function.
And code hasn't been posted for it (with the exception of the
notes prototypes).
--
Shawn.
^ permalink raw reply
* Re: Calculating tree nodes
From: David Tweed @ 2007-09-04 6:16 UTC (permalink / raw)
To: Jon Smirl; +Cc: Shawn O. Pearce, Git Mailing List
In-Reply-To: <9e4733910709032252x1fe6f436wdd13bcb1a6f76636@mail.gmail.com>
On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> On 9/4/07, David Tweed <david.tweed@gmail.com> wrote:
> > On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > Git has picked up the hierarchical storage scheme since it was built
> > > on a hierarchical file system.
> >
> > FWIW my memory is that initial git used path-to-blob lists (as you're
> > describing but without delta-ing) and tree nodes were added after a
> > couple of weeks, the motivation _at the time_ being they were a
> > natural way to dramatically reduce the size of repos.
> >
> > One of the nice things about tree nodes is that for doing a diff
> > between versions you can, to overwhelming probability, decide
> > equality/inequality of two arbitrarily deep and complicated subtrees
> > by comparing 40 characters, regardless of how remote and convoluted
> > their common ancestry. With delta chains don't you end up having to
> > trace back to a common "entry" in the history? (Of course, I don't
> > know how packs affect this - presumably there's some delta chasing to
> > get to the bare objects as well.)
>
> While it is a 40 character compare, how many disk accesses were needed
> to get those two SHAs into memory?
That's a difficult question. Clearly if you're starting on a machine
without anything cached, you're probably a bit worse off. If not and
you've got a "wide, shallow tree where changes occur clustered within
directories rather than spread uniformly throughout the tree" then
it's likely already there.
But it's all I suspect it should be looked at in relative terms: with
the respect to the delta-d list of SHA's, do you need to do more or
less disk reads to be able to compare stuff? Dunno: I'd imagine this
depends on precisely what's delta'd against what and how many of those
you need to read in order to put two entries into a comparable form.
The key point I was making was not so much the 40 characters as that
(at least with loose objects) if you can are given the top-level
SHA's, you can efficiently decide equality (a key to efficient
diffing) _without having to care how those two are related in the
history or read any extra history_.
--
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"we had no idea that when we added templates we were adding a Turing-
complete compile-time language." -- C++ standardisation committee
^ permalink raw reply
* Re: Calculating tree nodes
From: Shawn O. Pearce @ 2007-09-04 6:16 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Jon Smirl, David Tweed, Git Mailing List
In-Reply-To: <46DCF361.2090402@op5.se>
Andreas Ericsson <ae@op5.se> wrote:
> Jon Smirl wrote:
> >On 9/4/07, David Tweed <david.tweed@gmail.com> wrote:
> >>On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> >>>Git has picked up the hierarchical storage scheme since it was built
> >>>on a hierarchical file system.
...
> >>One of the nice things about tree nodes is that for doing a diff
> >>between versions you can, to overwhelming probability, decide
> >>equality/inequality of two arbitrarily deep and complicated subtrees
> >>by comparing 40 characters, regardless of how remote and convoluted
> >>their common ancestry. With delta chains don't you end up having to
> >>trace back to a common "entry" in the history? (Of course, I don't
> >>know how packs affect this - presumably there's some delta chasing to
> >>get to the bare objects as well.)
> >
> >While it is a 40 character compare, how many disk accesses were needed
> >to get those two SHAs into memory?
>
> One more than there would have been to read only the commit, and one more
> per level of recursion, assuming you never ever pack your repository.
>
> If you *do* pack it, the tree(s) needed to compare are likely already
> inside the sliding packfile window. In that case, there are no extra
> disk accesses.
Even better, lets do some back of the napkin math on the Linux
kernel tree. My local (out of date but close enough) copy has
22,730 files in the tip revision. Values shown are uncompressed
and compressed (gzip -9 | wc -c), but are excluding deltification.
Current Scheme Jon's Flat Scheme
----------------- -----------------
commit raw 932 932 + 22,730*20 = 455,532
(compressed) 521 456,338
root tree raw 876 0
(compressed) 805 0
I'm not even bothering with the individual subtrees. The numbers
will fall off quickly when you start to do subtree elimination and
only load the levels you need.
You are talking about doing disk IO for less than 4KiB with
the current scheme, and almost 456 KiB for the flat scheme.
That's before deltification. So if you also assume deltification
its going to be higher as you need to read back to a base object
that is roughly the final size and then unpack the smaller deltas
to reach the real commit.
Remember, SHA-1s can be stored as 20 bytes of binary data but they
are also generally uncompressible. That's why the root tree does
not compress very well, the SHA-1 data inside the tree cannot be
compressed and only the filenames have any shot at being compressed.
--
Shawn.
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Andreas Ericsson @ 2007-09-04 5:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Petr Baudis, git, jnareb, ltuikov
In-Reply-To: <7v8x7n7zqn.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
> * Incremental blame
>
> It does not seem to break the blame, but at least from where I
> sit accessing repo.or.cz this does not look incremental to me.
> The entire browser session freezes until the blame page
> displays in full. My local installation behaves the same way.
>
This is most likely due to what browser you're using. Some don't
start rendering until they've read the entire output of an URL.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Calculating tree nodes
From: Andreas Ericsson @ 2007-09-04 5:55 UTC (permalink / raw)
To: Jon Smirl; +Cc: David Tweed, Shawn O. Pearce, Git Mailing List
In-Reply-To: <9e4733910709032252x1fe6f436wdd13bcb1a6f76636@mail.gmail.com>
Jon Smirl wrote:
> On 9/4/07, David Tweed <david.tweed@gmail.com> wrote:
>> On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
>>> Git has picked up the hierarchical storage scheme since it was built
>>> on a hierarchical file system.
>> FWIW my memory is that initial git used path-to-blob lists (as you're
>> describing but without delta-ing) and tree nodes were added after a
>> couple of weeks, the motivation _at the time_ being they were a
>> natural way to dramatically reduce the size of repos.
>>
>> One of the nice things about tree nodes is that for doing a diff
>> between versions you can, to overwhelming probability, decide
>> equality/inequality of two arbitrarily deep and complicated subtrees
>> by comparing 40 characters, regardless of how remote and convoluted
>> their common ancestry. With delta chains don't you end up having to
>> trace back to a common "entry" in the history? (Of course, I don't
>> know how packs affect this - presumably there's some delta chasing to
>> get to the bare objects as well.)
>
> While it is a 40 character compare, how many disk accesses were needed
> to get those two SHAs into memory?
>
One more than there would have been to read only the commit, and one more
per level of recursion, assuming you never ever pack your repository.
If you *do* pack it, the tree(s) needed to compare are likely already
inside the sliding packfile window. In that case, there are no extra
disk accesses.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Calculating tree nodes
From: Jon Smirl @ 2007-09-04 5:52 UTC (permalink / raw)
To: David Tweed; +Cc: Shawn O. Pearce, Git Mailing List
In-Reply-To: <e1dab3980709032119r381f7a91ia84ba09039c21be1@mail.gmail.com>
On 9/4/07, David Tweed <david.tweed@gmail.com> wrote:
> On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> > Git has picked up the hierarchical storage scheme since it was built
> > on a hierarchical file system.
>
> FWIW my memory is that initial git used path-to-blob lists (as you're
> describing but without delta-ing) and tree nodes were added after a
> couple of weeks, the motivation _at the time_ being they were a
> natural way to dramatically reduce the size of repos.
>
> One of the nice things about tree nodes is that for doing a diff
> between versions you can, to overwhelming probability, decide
> equality/inequality of two arbitrarily deep and complicated subtrees
> by comparing 40 characters, regardless of how remote and convoluted
> their common ancestry. With delta chains don't you end up having to
> trace back to a common "entry" in the history? (Of course, I don't
> know how packs affect this - presumably there's some delta chasing to
> get to the bare objects as well.)
While it is a 40 character compare, how many disk accesses were needed
to get those two SHAs into memory?
>
> --
> cheers, dave tweed__________________________
> david.tweed@gmail.com
> Rm 124, School of Systems Engineering, University of Reading.
> "we had no idea that when we added templates we were adding a Turing-
> complete compile-time language." -- C++ standardisation committee
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Calculating tree nodes
From: Andreas Ericsson @ 2007-09-04 5:51 UTC (permalink / raw)
To: Jon Smirl
Cc: Martin Langhoff, Johannes Schindelin, Shawn O. Pearce,
Git Mailing List
In-Reply-To: <9e4733910709032237y65ccafdai4889078533908fb0@mail.gmail.com>
Jon Smirl wrote:
> On 9/4/07, Martin Langhoff <martin.langhoff@gmail.com> wrote:
>> On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
>>>> Yes. For performance reasons, since a simple commit would kill you in any
>>>> reasonably sized repo.
>>> That's not an obvious conclusion. A new commit is just a series of
>> Hi Jon!
>>
>> If you search the archives you'll find Linus explaining that the
>> initial git had all the directory structure in one single "tree"
>> object that held all the paths, not matter how deep. The problem with
>> that was taht every commit generated a huge new tree object, so he
>> switched to the current "nested trees" structure, which also has the
>> nice feature of speeding up diffs/merges if whole subtrees haven't
>> changed.
>
> In my scheme the commit is only a list of SHA's. The paths are stored
> as attributes of the file objects. Commits are just edits to the list
> of SHA's in the commit objects. If these lists are kept sorted, then
> the delta should be tiny. Just the info on the adds/deletes to the
> list.
>
It will stop being fast when you need to apply (revisions*avg_num_files_changed)
patches before you can start diffing things properly.
> This is very different that a single tree blob that contains all of the paths.
>
> Diffing two trees in the scheme is quite fast. Just get their commit
> objects into RAM and compare the lists of SHAs.
>
That's not a very useful diff though. I'd run, screaming, from an SCM that didn't
tell me *how* things have changed in addition to *what*.
>>> edits to the previous commit. Start with the previous commit, edit it,
>>> delta it and store it. Storing of the file objects is the same. Why
>>> isn't this scheme fast than the current one?
>> I think you're a bit confused between 2 different things:
>>
>> - git is _snapshot_ based, so every commit-tree-blob set is
>> completely independent. The "canonical" storage is each of those
>> gzipped in .git/objects
>> - however, for performance and on-disk-footprint, we delta them (very
>> efficiently I hear)
>
> The systems are essential the same with a little reorganization. In
> the current system the paths and SHA for a commit are spread over the
> tree nodes.
>
> In my scheme the path info is moved into the file object nodes and the
> SHA list is in the commit node.
>
> git still works exactly as it has before. I just moved things around
> in the storage system. The only thing that should be impacted is
> performance.
>
Perhaps, but negatively so. Git is fast when applying patches, primarily
because it can exclude entire subtrees. It knows it can exclude those
subtrees because their SHA1 hashes are identical. It wouldn't know that
if there weren't the tree objects (well, it could, but walking all the
commits, counting changes and considering '0' to be "no changes" doesn't
scale, so that's a moot point).
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Shawn O. Pearce @ 2007-09-04 5:50 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Petr Baudis, git, jnareb, ltuikov
In-Reply-To: <7v8x7n7zqn.fsf@gitster.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
> * Incremental blame
>
> It does not seem to break the blame, but at least from where I
> sit accessing repo.or.cz this does not look incremental to me.
> The entire browser session freezes until the blame page
> displays in full. My local installation behaves the same way.
What kind of repo are you trying that on?
I haven't myself tried the gitweb incremental blame but I do know
that one of my day-job repositories can take *ages* for git-gui's
`git blame --incremental -C -C -w` pass. Most of the revisions
for some of the blobs are far back in history, and there's a lot of
files to look at for copy/movement. Plus some of that history is
downright *messy* (I posted gitk screenshots of it not too long ago).
I'll try to give the gitweb incremental blame a try on that repo
tomorrow.
--
Shawn.
^ permalink raw reply
* Re: Calculating tree nodes
From: Jon Smirl @ 2007-09-04 5:50 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, Shawn O. Pearce, Git Mailing List
In-Reply-To: <7vmyw3835y.fsf@gitster.siamese.dyndns.org>
On 9/4/07, Junio C Hamano <gitster@pobox.com> wrote:
> "Jon Smirl" <jonsmirl@gmail.com> writes:
>
> >> Yes. For performance reasons, since a simple commit would kill you in any
> >> reasonably sized repo.
> >
> > That's not an obvious conclusion. A new commit is just a series of
> > edits to the previous commit. Start with the previous commit, edit it,
> > delta it and store it. Storing of the file objects is the same. Why
> > isn't this scheme fast than the current one?
>
> I think you seem to be forgetting about tree comparison.
>
> With a large project that has a reasonable directory structure
> (i.e. not insanely narrow), a commit touches isolated subparts
> of the whole tree. Think of an architecture specific patch to
> the Linux kernel touching only include/asm-i386 and arch/i386
> directories.
>
> Being able to cull an entire subdirectory (e.g. drivers/ which
> has 5700 files underneath) by only looking at the tree SHA-1 of
> the containing tree is a _HUGE_ win.
In my scheme you have all of the SHAs for the commit in RAM because
the are contained in the commit and you have the commit in RAM. It
take microseconds to compare these two lists in RAM.
The current scheme is doing disk accesses to get those tree nodes so
of course it is a win to cull the 5700 files.
>
> And this is not just about two tree comparison. When you say:
>
> git log v2.6.20 -- arch/i386/
>
> what you are seeing is a simplified history that consists of
> commits that touch only these paths. How would we determine if
> a commit touch these paths efficiently? By comparing the "i386"
> entry in tree objects for $commit^:arch and $commit:arch. You
> do not have to look inside arch/i386/ trees to see if any of the
> 330 files in it is different. You just check a single SHA-1
> pair.
It's more than just comparing a SHA, you have to do disk accesses to
retrieve the SHA.
I'm proposing that we only really need commit and file objects. I also
mentioned that if you think of the file objects as a table you could
use triggers to build cached indexes. To get performance back to the
current level we may want to construct some of these indexes. We need
to explore the scheme more before we can figure out the best cached
indexes to build.
Right now we only have a single index type, the tree nodes. And it's a
permanent part of the storage not cached. A hierarchical index is not
very useful of indexing non file name attributes.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Adam Roben @ 2007-09-04 5:47 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Petr Baudis, git, jnareb, ltuikov
In-Reply-To: <7v8x7n7zqn.fsf@gitster.siamese.dyndns.org>
On Sep 3, 2007, at 10:42 PM, Junio C Hamano wrote:
> * Incremental blame
>
> It does not seem to break the blame, but at least from where I
> sit accessing repo.or.cz this does not look incremental to me.
> The entire browser session freezes until the blame page
> displays in full. My local installation behaves the same way.
For me, [1] loads incrementally as expected in Safari 3.0.3 on OS X.
-Adam
[1] http://repo.or.cz/w/alt-git.git?a=blame_incremental;f=git-svn.perl;h=d3c8cd0b8e3cfb97c6deb2453097c6f2e16e3bcf;hb=1e61b7640d09015213dbcae3564fa27ac6a8c151
^ permalink raw reply
* Re: [PATCH 2/3] archive: specfile support (--pretty=format: in archive files)
From: Andreas Ericsson @ 2007-09-04 5:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: René Scharfe, Git Mailing List, Michael Gernoth,
Thomas Glanzmann
In-Reply-To: <7vtzqb8fw2.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
>
>> The attribute is useful for creating auto-updating specfiles. It is
>> limited by the underlying function format_commit_message(), though.
>> E.g. currently there is no placeholder for git-describe like output,
>> and expanded specfiles can't contain NUL bytes. That can be fixed
>> in format_commit_message() later and will then benefit users of
>> git-log, too.
>
> Interesting. I however wonder if "specfile" is a good name for
> this attribute, although I admit I do not think of anything
> better offhand.
>
"releasefile", perhaps?
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [ANNOUNCE] git/gitweb.git repository
From: Junio C Hamano @ 2007-09-04 5:42 UTC (permalink / raw)
To: Petr Baudis; +Cc: git, jnareb, ltuikov
In-Reply-To: <20070831000149.GK1219@pasky.or.cz>
Here is my impressions on patches in:
git://repo.or.cz/git/gitweb.git/ next
* Removal of git_blame
I think this makes sense; we do not need two not-so-different
implementations, and the other one has been the default for
quite some time.
* Extra columns in blame
I can see why some people especially with wide screen would
want this, and toggling the display for extra columns is also
a good idea. But this makes me wonder if the single button at
the very top is easy to locate (maybe explain what [+] does?)
and easy to access (maybe have it at the top and at least at
the bottom as well?).
* Incremental blame
It does not seem to break the blame, but at least from where I
sit accessing repo.or.cz this does not look incremental to me.
The entire browser session freezes until the blame page
displays in full. My local installation behaves the same way.
^ permalink raw reply
* Re: Calculating tree nodes
From: Jon Smirl @ 2007-09-04 5:37 UTC (permalink / raw)
To: Martin Langhoff; +Cc: Johannes Schindelin, Shawn O. Pearce, Git Mailing List
In-Reply-To: <46a038f90709032121v54454c6fi500ee15497eec85c@mail.gmail.com>
On 9/4/07, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On 9/4/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > Yes. For performance reasons, since a simple commit would kill you in any
> > > reasonably sized repo.
> >
> > That's not an obvious conclusion. A new commit is just a series of
>
> Hi Jon!
>
> If you search the archives you'll find Linus explaining that the
> initial git had all the directory structure in one single "tree"
> object that held all the paths, not matter how deep. The problem with
> that was taht every commit generated a huge new tree object, so he
> switched to the current "nested trees" structure, which also has the
> nice feature of speeding up diffs/merges if whole subtrees haven't
> changed.
In my scheme the commit is only a list of SHA's. The paths are stored
as attributes of the file objects. Commits are just edits to the list
of SHA's in the commit objects. If these lists are kept sorted, then
the delta should be tiny. Just the info on the adds/deletes to the
list.
This is very different that a single tree blob that contains all of the paths.
Diffing two trees in the scheme is quite fast. Just get their commit
objects into RAM and compare the lists of SHAs.
> > edits to the previous commit. Start with the previous commit, edit it,
> > delta it and store it. Storing of the file objects is the same. Why
> > isn't this scheme fast than the current one?
>
> I think you're a bit confused between 2 different things:
>
> - git is _snapshot_ based, so every commit-tree-blob set is
> completely independent. The "canonical" storage is each of those
> gzipped in .git/objects
> - however, for performance and on-disk-footprint, we delta them (very
> efficiently I hear)
The systems are essential the same with a little reorganization. In
the current system the paths and SHA for a commit are spread over the
tree nodes.
In my scheme the path info is moved into the file object nodes and the
SHA list is in the commit node.
git still works exactly as it has before. I just moved things around
in the storage system. The only thing that should be impacted is
performance.
>
> So if you ask the GIT APIs about a tree, you end up dealing with the
> nested trees I describe. Similarly, if you ask for a blob, you get the
> blob. But internally git _is_ delta-compressing them.
>
> It's not compressing them immediately -- only when you run git gc. But
> from an API perspective, you don't have to worry about that.
>
> HTH
>
>
> martin
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Calculating tree nodes
From: Junio C Hamano @ 2007-09-04 4:28 UTC (permalink / raw)
To: Jon Smirl; +Cc: Johannes Schindelin, Shawn O. Pearce, Git Mailing List
In-Reply-To: <9e4733910709032054y4407ce62o6b21935502bfacdb@mail.gmail.com>
"Jon Smirl" <jonsmirl@gmail.com> writes:
>> Yes. For performance reasons, since a simple commit would kill you in any
>> reasonably sized repo.
>
> That's not an obvious conclusion. A new commit is just a series of
> edits to the previous commit. Start with the previous commit, edit it,
> delta it and store it. Storing of the file objects is the same. Why
> isn't this scheme fast than the current one?
I think you seem to be forgetting about tree comparison.
With a large project that has a reasonable directory structure
(i.e. not insanely narrow), a commit touches isolated subparts
of the whole tree. Think of an architecture specific patch to
the Linux kernel touching only include/asm-i386 and arch/i386
directories.
Being able to cull an entire subdirectory (e.g. drivers/ which
has 5700 files underneath) by only looking at the tree SHA-1 of
the containing tree is a _HUGE_ win.
And this is not just about two tree comparison. When you say:
git log v2.6.20 -- arch/i386/
what you are seeing is a simplified history that consists of
commits that touch only these paths. How would we determine if
a commit touch these paths efficiently? By comparing the "i386"
entry in tree objects for $commit^:arch and $commit:arch. You
do not have to look inside arch/i386/ trees to see if any of the
330 files in it is different. You just check a single SHA-1
pair.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox