* Re: git diff - no newline after @@ ?
From: Petr Sebor @ 2006-08-18 13:19 UTC (permalink / raw)
To: Jeff King; +Cc: Alex Riesen, git
In-Reply-To: <20060818124428.GA22328@sigio.intra.peff.net>
Jeff King wrote:
> Try diff -p to get a hunk comment.
>
> -Peff
>
Ouch.... got it. Is there a way to turn the comment generation off btw?
Sorry for the noise...
Petr
^ permalink raw reply
* Re: [PATCH] gitweb: use action dispatcher for non-project actions, too.
From: Petr Baudis @ 2006-08-18 13:16 UTC (permalink / raw)
To: Carl Worth; +Cc: Junio C Hamano, Martin Waitz, git
In-Reply-To: <87y7tnxv9g.wl%cworth@cworth.org>
Dear diary, on Thu, Aug 17, 2006 at 05:00:11PM CEST, I got a letter
where Carl Worth <cworth@cworth.org> said that...
> PS. A secondary problem is the difficulty of publishing a new
> branch. But if I can get the 3 URLs above down to 1, then I could
> probably live with the user having to figure out how to combine the
> repository URL with the branch name, (in different ways depending on
> what the user is doing).
Could you be more specific here, please? Is it just that Git won't
automagically pick up new branches published on the server?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: Huge win, compressing a window of delta runs as a unit
From: Jon Smirl @ 2006-08-18 13:15 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Shawn Pearce, git
In-Reply-To: <Pine.LNX.4.64.0608172323520.11359@localhost.localdomain>
On 8/18/06, Nicolas Pitre <nico@cam.org> wrote:
> A better way to get such a size saving is to increase the window and
> depth parameters. For example, a window of 20 and depth of 20 can
> usually provide a pack size saving greater than 11% with none of the
> disadvantages mentioned above.
Our window size is effectively infinite. I am handing him all of the
revisions from a single file in optimal order. This includes branches.
He takes these revisions, runs xdiff on them, and then puts the entire
result into a single zlib blob.
I suspect the size reduction is directly proportional to the age of
the repository. The kernel repository only has three years worth of
data in it. Linus has the full history in another repository that is
not in general distribution. We can get it from him when he gets back
from vacation.
If the repository doesn't contain long delta chains the optimization
doesn't help that much. On the other hand it doesn't hurt either since
the chains weren't long. My repository is four times as old as the
kernel one and I am getting 4x the benefit.
This is a good format for archival data that is infrequency accessed.
That is why I proposed a two pack system. One pack would contain the
archival data and be highly optimized for size and the second pack
would contain recent changes and be optimized for speed. The size
optimization is important for controlling bandwidth costs.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: Huge win, compressing a window of delta runs as a unit
From: Jon Smirl @ 2006-08-18 12:53 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Shawn Pearce, git
In-Reply-To: <Pine.LNX.4.64.0608172323520.11359@localhost.localdomain>
[-- Attachment #1: Type: text/plain, Size: 370 bytes --]
I attached Shawn's code. He is gone until Monday and can't defend it.
Do note that I am running this on the Mozilla CVS which is over 10
years old. Some of the files have over 2,000 deltas. I average 10
deltas per file but the distribution is not at all even. Many files
get checked-in and never changed, for example 1000's of images.
--
Jon Smirl
jonsmirl@gmail.com
[-- Attachment #2: fast-import3.c --]
[-- Type: text/x-csrc, Size: 31535 bytes --]
/*
Format of STDIN stream:
stream ::= cmd*;
cmd ::= new_blob
| new_branch
| new_commit
| new_tag
;
new_blob ::= 'blob' lf
mark?
file_content;
file_content ::= data;
new_branch ::= 'branch' sp ref_str lf
('from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf)?
lf;
new_commit ::= 'commit' sp ref_str lf
mark?
('author' sp name '<' email '>' ts tz lf)?
'committer' sp name '<' email '>' ts tz lf
commit_msg
file_change*
lf;
commit_msg ::= data;
file_change ::= 'M' sp mode sp (hexsha1 | idnum) sp path_str lf
| 'D' sp path_str lf
;
mode ::= '644' | '755';
new_tag ::= 'tag' sp tag_str lf
'from' sp (ref_str | hexsha1 | sha1exp_str | idnum) lf
'tagger' sp name '<' email '>' ts tz lf
tag_msg;
tag_msg ::= data;
# note: the first idnum in a stream should be 1 and subsequent
# idnums should not have gaps between values as this will cause
# the stream parser to reserve space for the gapped values. An
# idnum can be updated in the future to a new object by issuing
# a new mark directive with the old idnum.
#
mark ::= 'mark' sp idnum lf;
# note: declen indicates the length of binary_data in bytes.
# declen does not include the lf preceeding or trailing the
# binary data.
#
data ::= 'data' sp declen lf
binary_data
lf;
# note: quoted strings are C-style quoting supporting \c for
# common escapes of 'c' (e..g \n, \t, \\, \") or \nnn where nnn
# is the signed byte value in octal. Note that the only
# characters which must actually be escaped to protect the
# stream formatting is: \, " and LF. Otherwise these values
# are UTF8.
#
ref_str ::= ref | '"' quoted(ref) '"' ;
sha1exp_str ::= sha1exp | '"' quoted(sha1exp) '"' ;
tag_str ::= tag | '"' quoted(tag) '"' ;
path_str ::= path | '"' quoted(path) '"' ;
declen ::= # unsigned 32 bit value, ascii base10 notation;
binary_data ::= # file content, not interpreted;
sp ::= # ASCII space character;
lf ::= # ASCII newline (LF) character;
# note: a colon (':') must precede the numerical value assigned to
# an idnum. This is to distinguish it from a ref or tag name as
# GIT does not permit ':' in ref or tag strings.
#
idnum ::= ':' declen;
path ::= # GIT style file path, e.g. "a/b/c";
ref ::= # GIT ref name, e.g. "refs/heads/MOZ_GECKO_EXPERIMENT";
tag ::= # GIT tag name, e.g. "FIREFOX_1_5";
sha1exp ::= # Any valid GIT SHA1 expression;
hexsha1 ::= # SHA1 in hexadecimal format;
# note: name and email are UTF8 strings, however name must not
# contain '<' or lf and email must not contain any of the
# following: '<', '>', lf.
#
name ::= # valid GIT author/committer name;
email ::= # valid GIT author/committer email;
ts ::= # time since the epoch in seconds, ascii base10 notation;
tz ::= # GIT style timezone;
*/
#include "builtin.h"
#include "cache.h"
#include "object.h"
#include "blob.h"
#include "tree.h"
#include "delta.h"
#include "pack.h"
#include "refs.h"
#include "csum-file.h"
#include "strbuf.h"
#include "quote.h"
struct object_entry
{
struct object_entry *next;
enum object_type type;
unsigned long offset;
unsigned char sha1[20];
};
struct object_entry_pool
{
struct object_entry_pool *next_pool;
struct object_entry *next_free;
struct object_entry *end;
struct object_entry entries[FLEX_ARRAY]; /* more */
};
struct last_object
{
void *data;
unsigned int len;
unsigned int depth;
unsigned char sha1[20];
};
struct mem_pool
{
struct mem_pool *next_pool;
char *next_free;
char *end;
char space[FLEX_ARRAY]; /* more */
};
struct atom_str
{
struct atom_str *next_atom;
int str_len;
char str_dat[FLEX_ARRAY]; /* more */
};
struct tree_content;
struct tree_entry
{
struct tree_content *tree;
struct atom_str* name;
unsigned int mode;
unsigned char sha1[20];
};
struct tree_content
{
unsigned int entry_capacity; /* must match avail_tree_content */
unsigned int entry_count;
struct tree_entry *entries[FLEX_ARRAY]; /* more */
};
struct avail_tree_content
{
unsigned int entry_capacity; /* must match tree_content */
struct avail_tree_content *next_avail;
};
struct branch
{
struct branch *table_next_branch;
struct branch *active_next_branch;
const char *name;
unsigned long last_commit;
struct tree_entry branch_tree;
unsigned char sha1[20];
};
/* Stats and misc. counters */
static int max_depth = 10;
static unsigned long alloc_count;
static unsigned long branch_count;
static unsigned long object_count;
static unsigned long duplicate_count;
static unsigned long object_count_by_type[9];
static unsigned long duplicate_count_by_type[9];
static unsigned long stream_cnt;
/* Memory pools */
static size_t mem_pool_alloc = 2*1024*1024 - sizeof(struct mem_pool);
static size_t total_allocd;
static struct mem_pool *mem_pool;
/* Atom management */
static unsigned int atom_table_sz = 4451;
static unsigned int atom_cnt;
static struct atom_str **atom_table;
/* The .pack file being generated */
static int pack_fd;
static unsigned long pack_offset;
static unsigned char pack_sha1[20];
static z_stream zs;
static int zs_live;
static void* zs_out;
static unsigned long zs_outlen;
/* Table of objects we've written. */
static unsigned int object_entry_alloc = 1000;
static struct object_entry_pool *blocks;
static struct object_entry *object_table[1 << 16];
/* Our last blob */
static struct last_object last_blob;
/* Tree management */
static unsigned int tree_entry_alloc = 1000;
static void *avail_tree_entry;
static unsigned int avail_tree_table_sz = 100;
static struct avail_tree_content **avail_tree_table;
/* Branch data */
static unsigned int max_active_branches = 5;
static unsigned int cur_active_branches;
static unsigned int branch_table_sz = 1039;
static struct branch **branch_table;
static struct branch *active_branches;
/* Input stream parsing */
static struct strbuf command_buf;
static unsigned long command_mark;
static void alloc_objects(int cnt)
{
struct object_entry_pool *b;
b = xmalloc(sizeof(struct object_entry_pool)
+ cnt * sizeof(struct object_entry));
b->next_pool = blocks;
b->next_free = b->entries;
b->end = b->entries + cnt;
blocks = b;
alloc_count += cnt;
}
static struct object_entry* new_object(unsigned char *sha1)
{
struct object_entry *e;
if (blocks->next_free == blocks->end)
alloc_objects(object_entry_alloc);
e = blocks->next_free++;
memcpy(e->sha1, sha1, sizeof(e->sha1));
return e;
}
static struct object_entry* find_object(unsigned char *sha1)
{
unsigned int h = sha1[0] << 8 | sha1[1];
struct object_entry *e;
for (e = object_table[h]; e; e = e->next)
if (!memcmp(sha1, e->sha1, sizeof(e->sha1)))
return e;
return NULL;
}
static struct object_entry* insert_object(unsigned char *sha1)
{
unsigned int h = sha1[0] << 8 | sha1[1];
struct object_entry *e = object_table[h];
struct object_entry *p = NULL;
while (e) {
if (!memcmp(sha1, e->sha1, sizeof(e->sha1)))
return e;
p = e;
e = e->next;
}
e = new_object(sha1);
e->next = NULL;
e->offset = 0;
if (p)
p->next = e;
else
object_table[h] = e;
return e;
}
static unsigned int hc_str(const char *s, size_t len)
{
unsigned int r = 0;
while (len-- > 0)
r = r * 31 + *s++;
return r;
}
static void* pool_alloc(size_t len)
{
struct mem_pool *p;
void *r;
for (p = mem_pool; p; p = p->next_pool)
if ((p->end - p->next_free >= len))
break;
if (!p) {
if (len >= (mem_pool_alloc/2)) {
total_allocd += len;
return xmalloc(len);
}
total_allocd += sizeof(struct mem_pool) + mem_pool_alloc;
p = xmalloc(sizeof(struct mem_pool) + mem_pool_alloc);
p->next_pool = mem_pool;
p->next_free = p->space;
p->end = p->next_free + mem_pool_alloc;
mem_pool = p;
}
r = p->next_free;
p->next_free += len;
return r;
}
static void* pool_calloc(size_t count, size_t size)
{
size_t len = count * size;
void *r = pool_alloc(len);
memset(r, 0, len);
return r;
}
static char* pool_strdup(const char *s)
{
char *r = pool_alloc(strlen(s) + 1);
strcpy(r, s);
return r;
}
static struct atom_str* to_atom(const char *s, size_t len)
{
unsigned int hc = hc_str(s, len) % atom_table_sz;
struct atom_str *c;
for (c = atom_table[hc]; c; c = c->next_atom)
if (c->str_len == len && !strncmp(s, c->str_dat, len))
return c;
c = pool_alloc(sizeof(struct atom_str) + len + 1);
c->str_len = len;
strncpy(c->str_dat, s, len);
c->str_dat[len] = 0;
c->next_atom = atom_table[hc];
atom_table[hc] = c;
atom_cnt++;
return c;
}
static struct branch* lookup_branch(const char *name)
{
unsigned int hc = hc_str(name, strlen(name)) % branch_table_sz;
struct branch *b;
for (b = branch_table[hc]; b; b = b->table_next_branch)
if (!strcmp(name, b->name))
return b;
return NULL;
}
static struct branch* new_branch(const char *name)
{
unsigned int hc = hc_str(name, strlen(name)) % branch_table_sz;
struct branch* b = lookup_branch(name);
if (b)
die("Invalid attempt to create duplicate branch: %s", name);
if (check_ref_format(name))
die("Branch name doesn't conform to GIT standards: %s", name);
b = pool_calloc(1, sizeof(struct branch));
b->name = pool_strdup(name);
b->table_next_branch = branch_table[hc];
branch_table[hc] = b;
branch_count++;
return b;
}
static unsigned int hc_entries(unsigned int cnt)
{
cnt = cnt & 7 ? (cnt / 8) + 1 : cnt / 8;
return cnt < avail_tree_table_sz ? cnt : avail_tree_table_sz - 1;
}
static struct tree_content* new_tree_content(unsigned int cnt)
{
struct avail_tree_content *f, *l = NULL;
struct tree_content *t;
unsigned int hc = hc_entries(cnt);
for (f = avail_tree_table[hc]; f; l = f, f = f->next_avail)
if (f->entry_capacity >= cnt)
break;
if (f) {
if (l)
l->next_avail = f->next_avail;
else
avail_tree_table[hc] = f->next_avail;
} else {
cnt = cnt & 7 ? ((cnt / 8) + 1) * 8 : cnt;
f = pool_alloc(sizeof(*t) + sizeof(t->entries[0]) * cnt);
f->entry_capacity = cnt;
}
t = (struct tree_content*)f;
t->entry_count = 0;
return t;
}
static void release_tree_entry(struct tree_entry *e);
static void release_tree_content(struct tree_content *t)
{
struct avail_tree_content *f = (struct avail_tree_content*)t;
unsigned int hc = hc_entries(f->entry_capacity);
unsigned int i;
for (i = 0; i < t->entry_count; i++)
release_tree_entry(t->entries[i]);
f->next_avail = avail_tree_table[hc];
avail_tree_table[hc] = f;
}
static struct tree_content* grow_tree_content(
struct tree_content *t,
int amt)
{
struct tree_content *r = new_tree_content(t->entry_count + amt);
r->entry_count = t->entry_count;
memcpy(r->entries,t->entries,t->entry_count*sizeof(t->entries[0]));
release_tree_content(t);
return r;
}
static struct tree_entry* new_tree_entry()
{
struct tree_entry *e;
if (!avail_tree_entry) {
unsigned int n = tree_entry_alloc;
avail_tree_entry = e = xmalloc(n * sizeof(struct tree_entry));
while (n--) {
*((void**)e) = e + 1;
e++;
}
}
e = avail_tree_entry;
avail_tree_entry = *((void**)e);
return e;
}
static void release_tree_entry(struct tree_entry *e)
{
if (e->tree)
release_tree_content(e->tree);
*((void**)e) = avail_tree_entry;
avail_tree_entry = e;
}
static void yread(int fd, void *buffer, size_t length)
{
ssize_t ret = 0;
while (ret < length) {
ssize_t size = xread(fd, (char *) buffer + ret, length - ret);
if (!size)
die("Read from descriptor %i: end of stream", fd);
if (size < 0)
die("Read from descriptor %i: %s", fd, strerror(errno));
ret += size;
}
}
static void ywrite(int fd, void *buffer, size_t length)
{
ssize_t ret = 0;
while (ret < length) {
ssize_t size = xwrite(fd, (char *) buffer + ret, length - ret);
if (!size)
die("Write to descriptor %i: end of file", fd);
if (size < 0)
die("Write to descriptor %i: %s", fd, strerror(errno));
ret += size;
}
}
static size_t encode_header(
enum object_type type,
size_t size,
unsigned char *hdr)
{
int n = 1;
unsigned char c;
if (type < OBJ_COMMIT || type > OBJ_DELTA)
die("bad type %d", type);
c = (type << 4) | (size & 15);
size >>= 4;
while (size) {
*hdr++ = c | 0x80;
c = size & 0x7f;
size >>= 7;
n++;
}
*hdr = c;
return n;
}
static void zs_finish()
{
if (zs_live) {
while (deflate(&zs, Z_FINISH) == Z_OK) {
if (zs.total_out) {
ywrite(pack_fd, zs_out, zs.total_out);
pack_offset += zs.total_out;
zs.next_out = zs_out;
zs.avail_out = zs_outlen;
zs.total_out = 0;
}
}
deflateEnd(&zs);
free(zs_out);
stream_cnt++;
}
}
static int store_object(
enum object_type type,
void *dat,
size_t datlen,
struct last_object *last,
unsigned char *sha1out)
{
void *delta;
struct object_entry *e;
unsigned char hdr[96];
unsigned char sha1[20];
unsigned long hdrlen, deltalen;
SHA_CTX c;
hdrlen = sprintf((char*)hdr,"%s %lu",type_names[type],datlen) + 1;
SHA1_Init(&c);
SHA1_Update(&c, hdr, hdrlen);
SHA1_Update(&c, dat, datlen);
SHA1_Final(sha1, &c);
if (sha1out)
memcpy(sha1out, sha1, sizeof(sha1));
e = insert_object(sha1);
if (e->offset) {
duplicate_count++;
duplicate_count_by_type[type]++;
return 1;
}
e->type = type;
e->offset = pack_offset;
object_count++;
object_count_by_type[type]++;
if (last && last->data && last->depth < max_depth)
delta = diff_delta(last->data, last->len,
dat, datlen,
&deltalen, 0);
else {
zs_finish();
delta = 0;
memset(&zs, 0, sizeof(zs));
deflateInit(&zs, zlib_compression_level);
zs_live = 1;
zs_outlen = 1024*1024;
zs_out = xmalloc(zs_outlen);
zs.next_out = zs_out;
zs.avail_out = zs_outlen;
}
if (delta) {
last->depth++;
zs.next_in = delta;
zs.avail_in = deltalen;
hdrlen = encode_header(OBJ_DELTA, deltalen, hdr);
ywrite(pack_fd, hdr, hdrlen);
ywrite(pack_fd, last->sha1, sizeof(sha1));
pack_offset += hdrlen + sizeof(sha1);
} else {
if (last)
last->depth = 0;
zs.next_in = dat;
zs.avail_in = datlen;
hdrlen = encode_header(type, datlen, hdr);
ywrite(pack_fd, hdr, hdrlen);
pack_offset += hdrlen;
}
while (deflate(&zs, Z_NO_FLUSH) == Z_OK) {
if (zs.total_out) {
ywrite(pack_fd, zs_out, zs.total_out);
pack_offset += zs.total_out;
zs.next_out = zs_out;
zs.avail_out = zs_outlen;
zs.total_out = 0;
}
}
if (delta)
free(delta);
if (last) {
if (last->data)
free(last->data);
last->data = dat;
last->len = datlen;
memcpy(last->sha1, sha1, sizeof(sha1));
}
return 0;
}
static const char *get_mode(const char *str, unsigned int *modep)
{
unsigned char c;
unsigned int mode = 0;
while ((c = *str++) != ' ') {
if (c < '0' || c > '7')
return NULL;
mode = (mode << 3) + (c - '0');
}
*modep = mode;
return str;
}
static void load_tree(struct tree_entry *root)
{
struct object_entry *myoe;
struct tree_content *t;
unsigned long size;
char *buf;
const char *c;
char type[20];
root->tree = t = new_tree_content(8);
if (!memcmp(root->sha1, null_sha1, 20))
return;
myoe = find_object(root->sha1);
if (myoe) {
die("FIXME");
} else {
buf = read_sha1_file(root->sha1, type, &size);
if (!buf || strcmp(type, tree_type))
die("Can't load existing tree %s", sha1_to_hex(root->sha1));
}
c = buf;
while (c != (buf + size)) {
struct tree_entry *e = new_tree_entry();
if (t->entry_count == t->entry_capacity)
root->tree = t = grow_tree_content(t, 8);
t->entries[t->entry_count++] = e;
e->tree = NULL;
c = get_mode(c, &e->mode);
if (!c)
die("Corrupt mode in %s", sha1_to_hex(root->sha1));
e->name = to_atom(c, strlen(c));
c += e->name->str_len + 1;
memcpy(e->sha1, c, sizeof(e->sha1));
c += 20;
}
free(buf);
}
static int tecmp (const void *_a, const void *_b)
{
struct tree_entry *a = *((struct tree_entry**)_a);
struct tree_entry *b = *((struct tree_entry**)_b);
return base_name_compare(
a->name->str_dat, a->name->str_len, a->mode,
b->name->str_dat, b->name->str_len, b->mode);
}
static void store_tree(struct tree_entry *root)
{
struct tree_content *t = root->tree;
unsigned int i;
size_t maxlen;
char *buf, *c;
if (memcmp(root->sha1, null_sha1, 20))
return;
maxlen = 0;
for (i = 0; i < t->entry_count; i++) {
maxlen += t->entries[i]->name->str_len + 34;
if (t->entries[i]->tree)
store_tree(t->entries[i]);
}
qsort(t->entries, t->entry_count, sizeof(t->entries[0]), tecmp);
buf = c = xmalloc(maxlen);
for (i = 0; i < t->entry_count; i++) {
struct tree_entry *e = t->entries[i];
c += sprintf(c, "%o", e->mode);
*c++ = ' ';
strcpy(c, e->name->str_dat);
c += e->name->str_len + 1;
memcpy(c, e->sha1, 20);
c += 20;
}
store_object(OBJ_TREE, buf, c - buf, NULL, root->sha1);
free(buf);
}
static int tree_content_set(
struct tree_entry *root,
const char *p,
const unsigned char *sha1,
const unsigned int mode)
{
struct tree_content *t = root->tree;
const char *slash1;
unsigned int i, n;
struct tree_entry *e;
slash1 = strchr(p, '/');
if (slash1)
n = slash1 - p;
else
n = strlen(p);
for (i = 0; i < t->entry_count; i++) {
e = t->entries[i];
if (e->name->str_len == n && !strncmp(p, e->name->str_dat, n)) {
if (!slash1) {
if (e->mode == mode && !memcmp(e->sha1, sha1, 20))
return 0;
e->mode = mode;
memcpy(e->sha1, sha1, 20);
if (e->tree) {
release_tree_content(e->tree);
e->tree = NULL;
}
memcpy(root->sha1, null_sha1, 20);
return 1;
}
if (!S_ISDIR(e->mode)) {
e->tree = new_tree_content(8);
e->mode = S_IFDIR;
}
if (!e->tree)
load_tree(e);
if (tree_content_set(e, slash1 + 1, sha1, mode)) {
memcpy(root->sha1, null_sha1, 20);
return 1;
}
return 0;
}
}
if (t->entry_count == t->entry_capacity)
root->tree = t = grow_tree_content(t, 8);
e = new_tree_entry();
e->name = to_atom(p, n);
t->entries[t->entry_count++] = e;
if (slash1) {
e->tree = new_tree_content(8);
e->mode = S_IFDIR;
tree_content_set(e, slash1 + 1, sha1, mode);
} else {
e->tree = NULL;
e->mode = mode;
memcpy(e->sha1, sha1, 20);
}
memcpy(root->sha1, null_sha1, 20);
return 1;
}
static int tree_content_remove(struct tree_entry *root, const char *p)
{
struct tree_content *t = root->tree;
const char *slash1;
unsigned int i, n;
struct tree_entry *e;
slash1 = strchr(p, '/');
if (slash1)
n = slash1 - p;
else
n = strlen(p);
for (i = 0; i < t->entry_count; i++) {
e = t->entries[i];
if (e->name->str_len == n && !strncmp(p, e->name->str_dat, n)) {
if (!slash1 || !S_ISDIR(e->mode))
goto del_entry;
if (!e->tree)
load_tree(e);
if (tree_content_remove(e, slash1 + 1)) {
if (!e->tree->entry_count)
goto del_entry;
memcpy(root->sha1, null_sha1, 20);
return 1;
}
return 0;
}
}
return 0;
del_entry:
for (i++; i < t->entry_count; i++)
t->entries[i-1] = t->entries[i];
t->entry_count--;
release_tree_entry(e);
memcpy(root->sha1, null_sha1, 20);
return 1;
}
static void init_pack_header()
{
const char* magic = "PACK";
unsigned long version = 3;
unsigned long zero = 0;
version = htonl(version);
ywrite(pack_fd, (char*)magic, 4);
ywrite(pack_fd, &version, 4);
ywrite(pack_fd, &zero, 4);
pack_offset = 4 * 3;
}
static void fixup_header_footer()
{
SHA_CTX c;
char hdr[8];
unsigned long cnt;
char *buf;
size_t n;
zs_finish();
if (lseek(pack_fd, 0, SEEK_SET) != 0)
die("Failed seeking to start: %s", strerror(errno));
SHA1_Init(&c);
yread(pack_fd, hdr, 8);
SHA1_Update(&c, hdr, 8);
cnt = htonl(object_count);
SHA1_Update(&c, &cnt, 4);
ywrite(pack_fd, &cnt, 4);
buf = xmalloc(128 * 1024);
for (;;) {
n = xread(pack_fd, buf, 128 * 1024);
if (n <= 0)
break;
SHA1_Update(&c, buf, n);
}
free(buf);
SHA1_Final(pack_sha1, &c);
ywrite(pack_fd, pack_sha1, sizeof(pack_sha1));
}
static int oecmp (const void *_a, const void *_b)
{
struct object_entry *a = *((struct object_entry**)_a);
struct object_entry *b = *((struct object_entry**)_b);
return memcmp(a->sha1, b->sha1, sizeof(a->sha1));
}
static void write_index(const char *idx_name)
{
struct sha1file *f;
struct object_entry **idx, **c, **last;
struct object_entry *e;
struct object_entry_pool *o;
unsigned int array[256];
int i;
/* Build the sorted table of object IDs. */
idx = xmalloc(object_count * sizeof(struct object_entry*));
c = idx;
for (o = blocks; o; o = o->next_pool)
for (e = o->entries; e != o->next_free; e++)
*c++ = e;
last = idx + object_count;
qsort(idx, object_count, sizeof(struct object_entry*), oecmp);
/* Generate the fan-out array. */
c = idx;
for (i = 0; i < 256; i++) {
struct object_entry **next = c;;
while (next < last) {
if ((*next)->sha1[0] != i)
break;
next++;
}
array[i] = htonl(next - idx);
c = next;
}
f = sha1create("%s", idx_name);
sha1write(f, array, 256 * sizeof(int));
for (c = idx; c != last; c++) {
unsigned int offset = htonl((*c)->offset);
sha1write(f, &offset, 4);
sha1write(f, (*c)->sha1, sizeof((*c)->sha1));
}
sha1write(f, pack_sha1, sizeof(pack_sha1));
sha1close(f, NULL, 1);
free(idx);
}
static void dump_branches()
{
static const char *msg = "fast-import";
unsigned int i;
struct branch *b;
struct ref_lock *lock;
for (i = 0; i < branch_table_sz; i++) {
for (b = branch_table[i]; b; b = b->table_next_branch) {
lock = lock_any_ref_for_update(b->name, NULL, 0);
if (!lock || write_ref_sha1(lock, b->sha1, msg) < 0)
die("Can't write %s", b->name);
}
}
}
static void read_next_command()
{
read_line(&command_buf, stdin, '\n');
}
static void cmd_mark()
{
if (!strncmp("mark :", command_buf.buf, 6)) {
command_mark = strtoul(command_buf.buf + 6, NULL, 10);
read_next_command();
}
else
command_mark = 0;
}
static void* cmd_data (size_t *size)
{
size_t n = 0;
void *buffer;
size_t length;
if (strncmp("data ", command_buf.buf, 5))
die("Expected 'data n' command, found: %s", command_buf.buf);
length = strtoul(command_buf.buf + 5, NULL, 10);
buffer = xmalloc(length);
while (n < length) {
size_t s = fread((char*)buffer + n, 1, length - n, stdin);
if (!s && feof(stdin))
die("EOF in data (%lu bytes remaining)", length - n);
n += s;
}
if (fgetc(stdin) != '\n')
die("An lf did not trail the binary data as expected.");
*size = length;
return buffer;
}
static void cmd_new_blob()
{
size_t datlen;
void *dat;
unsigned char sha1[20];
read_next_command();
cmd_mark();
dat = cmd_data(&datlen);
if (store_object(OBJ_BLOB, dat, datlen, &last_blob, sha1))
free(dat);
}
static void unload_one_branch()
{
while (cur_active_branches >= max_active_branches) {
unsigned long min_commit = ULONG_MAX;
struct branch *e, *l = NULL, *p = NULL;
for (e = active_branches; e; e = e->active_next_branch) {
if (e->last_commit < min_commit) {
p = l;
min_commit = e->last_commit;
}
l = e;
}
if (p) {
e = p->active_next_branch;
p->active_next_branch = e->active_next_branch;
} else {
e = active_branches;
active_branches = e->active_next_branch;
}
e->active_next_branch = NULL;
if (e->branch_tree.tree) {
release_tree_content(e->branch_tree.tree);
e->branch_tree.tree = NULL;
}
cur_active_branches--;
}
}
static void load_branch(struct branch *b)
{
load_tree(&b->branch_tree);
b->active_next_branch = active_branches;
active_branches = b;
cur_active_branches++;
}
static void file_change_m(struct branch *b)
{
const char *p = command_buf.buf + 2;
char *p_uq;
const char *endp;
struct object_entry *oe;
unsigned char sha1[20];
unsigned int mode;
char type[20];
p = get_mode(p, &mode);
if (!p)
die("Corrupt mode: %s", command_buf.buf);
switch (mode) {
case S_IFREG | 0644:
case S_IFREG | 0755:
case 0644:
case 0755:
/* ok */
break;
default:
die("Corrupt mode: %s", command_buf.buf);
}
if (get_sha1_hex(p, sha1))
die("Invalid SHA1: %s", command_buf.buf);
p += 40;
if (*p++ != ' ')
die("Missing space after SHA1: %s", command_buf.buf);
p_uq = unquote_c_style(p, &endp);
if (p_uq) {
if (*endp)
die("Garbage after path in: %s", command_buf.buf);
p = p_uq;
}
oe = find_object(sha1);
if (oe) {
if (oe->type != OBJ_BLOB)
die("Not a blob (actually a %s): %s",
command_buf.buf, type_names[oe->type]);
} else {
if (sha1_object_info(sha1, type, NULL))
die("Blob not found: %s", command_buf.buf);
if (strcmp(blob_type, type))
die("Not a blob (actually a %s): %s",
command_buf.buf, type);
}
tree_content_set(&b->branch_tree, p, sha1, S_IFREG | mode);
if (p_uq)
free(p_uq);
}
static void file_change_d(struct branch *b)
{
const char *p = command_buf.buf + 2;
char *p_uq;
const char *endp;
p_uq = unquote_c_style(p, &endp);
if (p_uq) {
if (*endp)
die("Garbage after path in: %s", command_buf.buf);
p = p_uq;
}
tree_content_remove(&b->branch_tree, p);
if (p_uq)
free(p_uq);
}
static void cmd_new_commit()
{
struct branch *b;
void *msg;
size_t msglen;
char *str_uq;
const char *endp;
char *sp;
char *author = NULL;
char *committer = NULL;
char *body;
/* Obtain the branch name from the rest of our command */
sp = strchr(command_buf.buf, ' ') + 1;
str_uq = unquote_c_style(sp, &endp);
if (str_uq) {
if (*endp)
die("Garbage after ref in: %s", command_buf.buf);
sp = str_uq;
}
b = lookup_branch(sp);
if (!b)
die("Branch not declared: %s", sp);
if (str_uq)
free(str_uq);
read_next_command();
cmd_mark();
if (!strncmp("author ", command_buf.buf, 7)) {
author = strdup(command_buf.buf);
read_next_command();
}
if (!strncmp("committer ", command_buf.buf, 10)) {
committer = strdup(command_buf.buf);
read_next_command();
}
if (!committer)
die("Expected committer but didn't get one");
msg = cmd_data(&msglen);
/* ensure the branch is active/loaded */
if (!b->branch_tree.tree) {
unload_one_branch();
load_branch(b);
}
/* file_change* */
for (;;) {
read_next_command();
if (1 == command_buf.len)
break;
else if (!strncmp("M ", command_buf.buf, 2))
file_change_m(b);
else if (!strncmp("D ", command_buf.buf, 2))
file_change_d(b);
else
die("Unsupported file_change: %s", command_buf.buf);
}
/* build the tree and the commit */
store_tree(&b->branch_tree);
body = xmalloc(97 + msglen
+ (author
? strlen(author) + strlen(committer)
: 2 * strlen(committer)));
sp = body;
sp += sprintf(sp, "tree %s\n", sha1_to_hex(b->branch_tree.sha1));
if (memcmp(b->sha1, null_sha1, 20))
sp += sprintf(sp, "parent %s\n", sha1_to_hex(b->sha1));
if (author)
sp += sprintf(sp, "%s\n", author);
else
sp += sprintf(sp, "author %s\n", committer + 10);
sp += sprintf(sp, "%s\n\n", committer);
memcpy(sp, msg, msglen);
sp += msglen;
if (author)
free(author);
free(committer);
free(msg);
store_object(OBJ_COMMIT, body, sp - body, NULL, b->sha1);
free(body);
b->last_commit = object_count_by_type[OBJ_COMMIT];
}
static void cmd_new_branch()
{
struct branch *b;
char *str_uq;
const char *endp;
char *sp;
/* Obtain the new branch name from the rest of our command */
sp = strchr(command_buf.buf, ' ') + 1;
str_uq = unquote_c_style(sp, &endp);
if (str_uq) {
if (*endp)
die("Garbage after ref in: %s", command_buf.buf);
sp = str_uq;
}
b = new_branch(sp);
if (str_uq)
free(str_uq);
read_next_command();
/* from ... */
if (!strncmp("from ", command_buf.buf, 5)) {
const char *from;
struct branch *s;
from = strchr(command_buf.buf, ' ') + 1;
str_uq = unquote_c_style(from, &endp);
if (str_uq) {
if (*endp)
die("Garbage after string in: %s", command_buf.buf);
from = str_uq;
}
s = lookup_branch(from);
if (b == s)
die("Can't create a branch from itself: %s", b->name);
else if (s) {
memcpy(b->sha1, s->sha1, 20);
memcpy(b->branch_tree.sha1, s->branch_tree.sha1, 20);
} else if (!get_sha1(from, b->sha1)) {
if (!memcmp(b->sha1, null_sha1, 20))
memcpy(b->branch_tree.sha1, null_sha1, 20);
else {
unsigned long size;
char *buf;
buf = read_object_with_reference(b->sha1,
type_names[OBJ_COMMIT], &size, b->sha1);
if (!buf || size < 46)
die("Not a valid commit: %s", from);
if (memcmp("tree ", buf, 5)
|| get_sha1_hex(buf + 5, b->branch_tree.sha1))
die("The commit %s is corrupt", sha1_to_hex(b->sha1));
free(buf);
}
} else
die("Invalid ref name or SHA1 expression: %s", from);
if (str_uq)
free(str_uq);
read_next_command();
} else {
memcpy(b->sha1, null_sha1, 20);
memcpy(b->branch_tree.sha1, null_sha1, 20);
}
if (command_buf.eof || command_buf.len > 1)
die("An lf did not terminate the branch command as expected.");
}
int main(int argc, const char **argv)
{
const char *base_name = argv[1];
int est_obj_cnt = atoi(argv[2]);
char *pack_name;
char *idx_name;
struct stat sb;
setup_ident();
git_config(git_default_config);
pack_name = xmalloc(strlen(base_name) + 6);
sprintf(pack_name, "%s.pack", base_name);
idx_name = xmalloc(strlen(base_name) + 5);
sprintf(idx_name, "%s.idx", base_name);
pack_fd = open(pack_name, O_RDWR|O_CREAT|O_EXCL, 0666);
if (pack_fd < 0)
die("Can't create %s: %s", pack_name, strerror(errno));
init_pack_header();
alloc_objects(est_obj_cnt);
strbuf_init(&command_buf);
atom_table = xcalloc(atom_table_sz, sizeof(struct atom_str*));
branch_table = xcalloc(branch_table_sz, sizeof(struct branch*));
avail_tree_table = xcalloc(avail_tree_table_sz, sizeof(struct avail_tree_content*));
for (;;) {
read_next_command();
if (command_buf.eof)
break;
else if (!strcmp("blob", command_buf.buf))
cmd_new_blob();
else if (!strncmp("branch ", command_buf.buf, 7))
cmd_new_branch();
else if (!strncmp("commit ", command_buf.buf, 7))
cmd_new_commit();
else
die("Unsupported command: %s", command_buf.buf);
}
fixup_header_footer();
close(pack_fd);
write_index(idx_name);
dump_branches();
fprintf(stderr, "%s statistics:\n", argv[0]);
fprintf(stderr, "---------------------------------------------------\n");
fprintf(stderr, "Alloc'd objects: %10lu (%10lu overflow )\n", alloc_count, alloc_count - est_obj_cnt);
fprintf(stderr, "Total objects: %10lu (%10lu duplicates)\n", object_count, duplicate_count);
fprintf(stderr, " blobs : %10lu (%10lu duplicates)\n", object_count_by_type[OBJ_BLOB], duplicate_count_by_type[OBJ_BLOB]);
fprintf(stderr, " trees : %10lu (%10lu duplicates)\n", object_count_by_type[OBJ_TREE], duplicate_count_by_type[OBJ_TREE]);
fprintf(stderr, " commits: %10lu (%10lu duplicates)\n", object_count_by_type[OBJ_COMMIT], duplicate_count_by_type[OBJ_COMMIT]);
fprintf(stderr, " tags : %10lu (%10lu duplicates)\n", object_count_by_type[OBJ_TAG], duplicate_count_by_type[OBJ_TAG]);
fprintf(stderr, "Total streams: %10u\n", stream_cnt);
fprintf(stderr, "Total branches: %10lu\n", branch_count);
fprintf(stderr, "Total atoms: %10u\n", atom_cnt);
fprintf(stderr, "Memory total: %10lu KiB\n", (total_allocd + alloc_count*sizeof(struct object_entry))/1024);
fprintf(stderr, " pools: %10lu KiB\n", total_allocd/1024);
fprintf(stderr, " objects: %10lu KiB\n", (alloc_count*sizeof(struct object_entry))/1024);
fprintf(stderr, "---------------------------------------------------\n");
stat(pack_name, &sb);
fprintf(stderr, "Pack size: %10lu KiB\n", (unsigned long)(sb.st_size/1024));
stat(idx_name, &sb);
fprintf(stderr, "Index size: %10lu KiB\n", (unsigned long)(sb.st_size/1024));
fprintf(stderr, "\n");
return 0;
}
^ permalink raw reply
* Re: git diff - no newline after @@ ?
From: Jeff King @ 2006-08-18 12:44 UTC (permalink / raw)
To: Petr Sebor; +Cc: Alex Riesen, git
In-Reply-To: <44E5B496.3070901@scssoft.com>
On Fri, Aug 18, 2006 at 02:37:42PM +0200, Petr Sebor wrote:
> >>has anyone noticed that the 'git diff' output somehow lacks the newline
> >>after the @@ tag?
> >Why should it? It's hunk comment.
> Hunk comment? Maybe we don't understand each other...
The intent of a hunk comment is to give the user some context. Note how
it's usually the C function name, even if the declaration of that
function isn't the line right before the context lines?
> check another example:
>
> --- a/include/linux/kernel.h
> +++ b/include/linux/kernel.h
> @@ -13,6 +13,7 @@ #include <linux/stddef.h>
> #include <linux/types.h>
> #include <linux/compiler.h>
> #include <linux/bitops.h>
> +#include <linux/memleak.h>
> #include <asm/byteorder.h>
> #include <asm/bug.h>
The function detection is very simple (I believe it just looks for the
closest preceding line without any prior whitespace).
> whereas gnu diff would output
Try diff -p to get a hunk comment.
-Peff
^ permalink raw reply
* Re: git diff - no newline after @@ ?
From: Jakub Narebski @ 2006-08-18 12:40 UTC (permalink / raw)
To: git
In-Reply-To: <44E5B496.3070901@scssoft.com>
Petr Sebor wrote:
> Alex Riesen wrote:
>> On 8/18/06, Petr Sebor <petr@scssoft.com> wrote:
>>> Hello,
>>>
>>> has anyone noticed that the 'git diff' output somehow lacks the newline
>>> after the @@ tag?
>>>
>>
>> Why should it? It's hunk comment.
> Hunk comment? Maybe we don't understand each other...
Hunk comment as in "diff -p":
-p Show which C function each change is in.
(which is simpliefied to: show first line preceding hunk which begins with
word character in first column).
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* Re: git diff - no newline after @@ ?
From: Petr Sebor @ 2006-08-18 12:37 UTC (permalink / raw)
To: Alex Riesen; +Cc: git
In-Reply-To: <81b0412b0608180525t284d3124j867a9461bf14ed91@mail.gmail.com>
Alex Riesen wrote:
> On 8/18/06, Petr Sebor <petr@scssoft.com> wrote:
>> Hello,
>>
>> has anyone noticed that the 'git diff' output somehow lacks the newline
>> after the @@ tag?
>>
>
> Why should it? It's hunk comment.
Hunk comment? Maybe we don't understand each other...
check another example:
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -13,6 +13,7 @@ #include <linux/stddef.h>
#include <linux/types.h>
#include <linux/compiler.h>
#include <linux/bitops.h>
+#include <linux/memleak.h>
#include <asm/byteorder.h>
#include <asm/bug.h>
whereas gnu diff would output
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -13,6 +13,7 @@
#include <linux/stddef.h>
#include <linux/types.h>
#include <linux/compiler.h>
#include <linux/bitops.h>
+#include <linux/memleak.h>
#include <asm/byteorder.h>
#include <asm/bug.h>
Petr
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Petr Baudis @ 2006-08-18 12:31 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: David Rientjes, git
In-Reply-To: <Pine.LNX.4.63.0608181209210.28360@wbgn013.biozentrum.uni-wuerzburg.de>
Hi,
Dear diary, on Fri, Aug 18, 2006 at 12:21:11PM CEST, I got a letter
where Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...
> However, you should know that there is _no way_ to use both hashes on the
> same project. Yes, you could rewrite the history, trying to convert also
> the hashes in the commit objects, but people actually started relying on
> naming commits with the short-SHA1.
I don't really like having IDs ambiguous in this sense - having the same
type of IDs in all git-tracked projects has some cute benefits which are
of the kind that you don't know ahead that you will need them: joining
history of two distinct projects in a merge and theoretical possibility
of having subprojects where the main project references an exact
tree/commit of the sub project.
If we are ever going to implement support for multiple hashes, the hash
type should at least be part of the object id, in textual representation
as e.g. the first letter. This can still lead to convergence issues and
duplicate objects, but it enables smooth transition without rewriting
the history and it is much less confusing than just switching to a
different function.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam
^ permalink raw reply
* Re: git diff - no newline after @@ ?
From: Alex Riesen @ 2006-08-18 12:25 UTC (permalink / raw)
To: Petr Sebor; +Cc: git
In-Reply-To: <44E5AE42.6090506@scssoft.com>
On 8/18/06, Petr Sebor <petr@scssoft.com> wrote:
> Hello,
>
> has anyone noticed that the 'git diff' output somehow lacks the newline
> after the @@ tag?
>
Why should it? It's hunk comment.
^ permalink raw reply
* git diff - no newline after @@ ?
From: Petr Sebor @ 2006-08-18 12:10 UTC (permalink / raw)
To: git
Hello,
has anyone noticed that the 'git diff' output somehow lacks the newline
after the @@ tag?
The linux-kernel list is full of diffs whose hunks begin like:
@@ -1475,6 +1475,7 @@ int do_coredump(long signr, int exit_cod
@@ -1516,22 +1517,34 @@ int do_coredump(long signr, int exit_cod
@@ -1428,7 +1442,7 @@ static int elf_core_dump(long signr, str
instead of:
@@ -1475,6 +1475,7 @@
int do_coredump(long signr, int exit_cod
@@ -1516,22 +1517,34 @@
int do_coredump(long signr, int exit_cod
@@ -1428,7 +1442,7 @@
static int elf_core_dump(long signr, str
It is not only a problem of the linux-kernel, my local git installation
suffers from this as well...
Regards
Petr
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Johannes Schindelin @ 2006-08-18 11:37 UTC (permalink / raw)
To: Trekie; +Cc: git
In-Reply-To: <44E5A416.9040709@sinister.cz>
Hi,
On Fri, 18 Aug 2006, Trekie wrote:
> Johannes Schindelin wrote:
> > SHA1 has been broken (collisions have been found):
> >
> > http://www.schneier.com/blog/archives/2005/02/sha1_broken.html
>
> I don't think you're right. That blog just says, that Wang can find
>
> "collisions in the the full SHA-1 in 2**69 hash operations, much less
> than the brute-force attack of 2**80 operations based on the hash length."
True. I have not heard of a collision either.
> The point is why use MD5 if anyone can compute a collision?
It does not suffice to generate collisions to make a hash unusable for our
purposes: you would have to find a way to produce another text for a
_given_ hash. Plus, this text would not only have to look meaningful, but
compile. And preferrably introduce a back door.
Granted, once people find out how to generate another text, they can try
to "optimize" some block between "/*" and "*/", so that the hash stays the
same. But AFAICT none of the breaks of SHA1 or MD5 point into such a
direction. Yet.
But _even if_ somebody succeeds in all that, that somebody has to convince
_you_ to pull. And if you already have that object (the "good" version),
it will not get overwritten.
Ciao,
Dscho
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Trekie @ 2006-08-18 11:27 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0608181255060.28360@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin wrote:
> SHA1 has been broken (collisions have been found):
>
> http://www.schneier.com/blog/archives/2005/02/sha1_broken.html
I don't think you're right. That blog just says, that Wang can find
"collisions in the the full SHA-1 in 2**69 hash operations, much less
than the brute-force attack of 2**80 operations based on the hash length."
That doesn't mean any collision has been found. In academic
cryptography, any attack that has less computational complexity than the
expected time needed for brute force is considered a break.
In a document (http://www.rsasecurity.com/rsalabs/node.asp?id=2927) that
has been released 6 months after that blog post is said a collision can
be found in 2^63 operations.
Well, if someone use the fastest computer today
(http://www.top500.org/system/7747) to get a collision it would take a
day to found one.
The point is why use MD5 if anyone can compute a collision?
David Brodsky
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Johannes Schindelin @ 2006-08-18 10:56 UTC (permalink / raw)
To: Trekie; +Cc: David Rientjes, git
In-Reply-To: <44E59BF6.2070909@sinister.cz>
Hi,
On Fri, 18 Aug 2006, Trekie wrote:
> I'd like to point out that while finding collision for SHA1 according to
> the Wikipedia needs 2^63 operations and AFAIK no collision has been
> found yet, finding collision for MD5 can be achieved in a minute and
> less (see http://cryptography.hyperlink.cz/2006/tunnels.pdf for details).
SHA1 has been broken (collisions have been found):
http://www.schneier.com/blog/archives/2005/02/sha1_broken.html
Ciao,
Dscho
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Trekie @ 2006-08-18 10:52 UTC (permalink / raw)
To: David Rientjes; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0608172259280.25827@chino.corp.google.com>
Hi,
I'd like to point out that while finding collision for SHA1 according to
the Wikipedia needs 2^63 operations and AFAIK no collision has been
found yet, finding collision for MD5 can be achieved in a minute and
less (see http://cryptography.hyperlink.cz/2006/tunnels.pdf for details).
David Brodsky
^ permalink raw reply
* [PATCH] git-mv: special case destination "."
From: Johannes Schindelin @ 2006-08-18 10:42 UTC (permalink / raw)
To: Fredrik Kuivinen; +Cc: git
In-Reply-To: <20060818072328.GA5255@c165.ib.student.liu.se>
Since the normalized basename of "." is "", the check for directory
failed erroneously.
Noticed by Fredrik Kuivinen.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
> Subject: Re: Another git-mv bug
... and another fix.
On Fri, 18 Aug 2006, Fredrik Kuivinen wrote:
> In a Git repository:
>
> $ git mv templates/info--exclude .
> fatal: renaming templates/info--exclude failed: No such file or directory
builtin-mv.c | 5 ++++-
t/t7001-mv.sh | 4 ++++
2 files changed, 8 insertions(+), 1 deletions(-)
diff --git a/builtin-mv.c b/builtin-mv.c
index c0c8764..b2ecc26 100644
--- a/builtin-mv.c
+++ b/builtin-mv.c
@@ -114,7 +114,10 @@ int cmd_mv(int argc, const char **argv,
modes = xcalloc(count, sizeof(enum update_mode));
dest_path = copy_pathspec(prefix, argv + argc - 1, 1, 0);
- if (!lstat(dest_path[0], &st) &&
+ if (dest_path[0][0] == '\0')
+ /* special case: "." was normalized to "" */
+ destination = copy_pathspec(dest_path[0], argv + i, count, 1);
+ else if (!lstat(dest_path[0], &st) &&
S_ISDIR(st.st_mode)) {
dest_path[0] = add_slash(dest_path[0]);
destination = copy_pathspec(dest_path[0], argv + i, count, 1);
diff --git a/t/t7001-mv.sh b/t/t7001-mv.sh
index e5e0bb9..b7fcdb3 100755
--- a/t/t7001-mv.sh
+++ b/t/t7001-mv.sh
@@ -82,4 +82,8 @@ test_expect_failure \
'do not move directory over existing directory' \
'mkdir path0 && mkdir path0/path2 && git-mv path2 path0'
+test_expect_success \
+ 'move into "."' \
+ 'git-mv path1/path2/ .'
+
test_done
--
1.4.2.ge0502-dirty
^ permalink raw reply related
* Why git-ls-tree has no --stdin option?
From: Jakub Narebski @ 2006-08-18 10:40 UTC (permalink / raw)
To: git
Why git-ls-tree has no --stdin option to give <tree-ish> (feed it from
git-ref-list for example), like git-diff-tree has? The output format could
be similar: treeish, then git-ls-tree output, treeish, tree,...
Also, is there a combination of options which would display only exact
matches to path limit, i.e. tree entry only if path is directory, and file
(blob) entry only if path is ordinary file?
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
* (unknown),
From: Wolfgang Denk @ 2006-08-18 10:35 UTC (permalink / raw)
To: git
Hi,
"gitk" procudes output like this:
Author: Stefan Roese <sr@denx.de> 2006-01-26 13:51:42
Committer: Stefan Roese <sr@denx.de> 2006-01-26 13:51:42
Parent: f565c2d4c8b945b5bfbafb8dda50b8784167b4e3 (Merge with /home/git/linux-2.6)
Child: d216d89489cce4021a75dea19ef1b19ad93958ec (Git rid of debug output.) Branches: origin, master
Follows: DENX-v2.6.15
Precedes: DENX-2006-02-01-2215
Is there a CLI tool or a hidden option to "git log" or "cg-log" or
some other clever command to get similar "Follows:" and "Precedes:"
tag information?
Best regards,
Wolfgang Denk
--
Software Engineering: Embedded and Realtime Systems, Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
The IQ of the group is the lowest IQ of a member of the group divided
by the number of people in the group.
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Johannes Schindelin @ 2006-08-18 10:21 UTC (permalink / raw)
To: David Rientjes; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0608172259280.25827@chino.corp.google.com>
Hi,
On Thu, 17 Aug 2006, David Rientjes wrote:
> I'd like to solicit some comments about implementing support for md5 as a
> hash function that could be determined at runtime by the user during a
> project init-db.
Make it a config variable, too, right?
It would be interesting to have other hashes, for three reasons:
1. they could be faster to calculate,
2. they could reduce clashes, and related to that,
3. it is possible that some day SHA1 is broken, i.e. that there is an
algorithm to generate a different text for a given hash.
As for 2 and 3, it seems MD5 is equivalent, since another sort of attacks
was already successful on both SHA1 and MD5: generating two different
texts with the same hash.
So, 1 could be a good reason to have another hash. IIRC SHA1 is about 25%
slower than MD5, so it could be worth it.
However, you should know that there is _no way_ to use both hashes on the
same project. Yes, you could rewrite the history, trying to convert also
the hashes in the commit objects, but people actually started relying on
naming commits with the short-SHA1.
I think it would be a nice thing to play through (for example, to find
out how much impact the hash calculation has on the overall performance
of git), but I doubt it will ever come to real use.
Ciao,
Dscho
^ permalink raw reply
* Re: recur status on linux-2.6
From: Johannes Schindelin @ 2006-08-18 10:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwt96d6s1.fsf@assigned-by-dhcp.cox.net>
Hi,
On Thu, 17 Aug 2006, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Sun, 13 Aug 2006, Junio C Hamano wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >>
> >> > I fail to see how this is worse than -recursive...
> >>
> >> These are what I got. ls-files -u output followed by git diff.
> >
> > I am a little confused here: I thought it would be enough to compare the
> > outputs of "git-ls-files --stage". But that seems wrong.
> >
> > What are the stages for, again?
>
> I do not offhand remember what git-merge-recursive and
> git-merge-recur store in stage #1 when they recurse to create a
> virtual common ancestor. I expected it would contain the blob
> used as the base for the final file-level three-way merge
> (i.e. the blob in the virtual common ancestor), and if that is
> the case, i.e. if the blob matches the second argument for
> "merge" (from RCS), it should be enough to check that the stages
> match to verify two implementations do the same thing.
>
> But in practice, stage #1 is not very interesting nor useful
> after a conflicted merge (git diff --ours and git diff --theirs
> are more useful, so is git log -p --merge), so it is possible
> that merge-recursive is leaving the blob from one of the true
> common ancestors there while using the blob from the virtual
> common ancestor to produce the final result in the working tree
> and nobody has noticed. I dunno.
Makes sense.
I finally got around to do updated tests, with both git-ls-files --stage
and "git diff". It seems like the only issue in the git repository _is_
10a6653c8. I have not yet had the time to analyze this merge (it is
huge!), but I guess recur becomes confused by the timestamps.
Ciao,
Dscho
^ permalink raw reply
* Re: [RFC] adding support for md5
From: Nguyễn Thái Ngọc Duy @ 2006-08-18 9:59 UTC (permalink / raw)
To: David Rientjes; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0608172259280.25827@chino.corp.google.com>
On 8/18/06, David Rientjes <rientjes@google.com> wrote:
> I'd like to solicit some comments about implementing support for md5 as a
> hash function that could be determined at runtime by the user during a
> project init-db. md5, which I implemented as a configurable option in my
> own tree, is a 128-bit hash that is slightly more recognized than sha1.
> Likewise, it is also available in openssl/md5.h just as sha1 is available
> through a library in openssl/sha1.h. My patch to move the hash name
> comparison was a step in this direction in isolating many of the
> particulars of hash-specific dependencies.
Just curious, but why md5? Is there any benefit using md5 over sha1?
^ permalink raw reply
* Re: Unresolved issues #3
From: Catalin Marinas @ 2006-08-18 9:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vbqqics3g.fsf@assigned-by-dhcp.cox.net>
On 18/08/06, Junio C Hamano <junkio@cox.net> wrote:
> Catalin Marinas <catalin.marinas@arm.com> writes:
> > In StGIT I can configure the merge command and I currently use the
> > attached script (I need to add it to the StGIT repository). It tries
> > diff3 first and, if that fails, invokes emacs' merge (can use xxdiff
> > as well). It also checks whether the file was modified in case I want
> > to exit and solve the conflict later (maybe after getting conflict
> > information for the other files).
>
> Thanks. That is interesting. However, the workflow Martin
> wants to use this is a bit different from this. He wants
> regular diff3/merge to leave conflicted result and have "xxdiff
> -U" to sort out the resulting mess (I do that manually myself
> but without any wrapper).
The problem with this approach is that you lose the ancestor
information. You want "diff3 -E" command to not report conflicts if
the changes are the same but the output no longer has the ancestor
information. The -A option would keep the ancestor (and use "xxdiff
--unmerge3" instead) but it always reports a conflict even if the
changes are the same. Are there any diff3 options I missed?
That's why I prefer to explicitely pass the 3 files to either emacs or xxdiff.
> > # default merger
> > if os.system('diff3 -L current -L ancestor -L patched -m -E '
> > '"%s" "%s" "%s" > "%s"'
> > % (branch1, ancestor, branch2, output)):
> > # interactive merge
> > if os.path.exists(output):
> > mtime = os.path.getmtime(output)
> > else:
> > mtime = 0
>
> I wonder why you do not error out when "if os.path.exists(output)"
> does not hold true; you are redirecting into it, so the possibilities
> are:
You are right. I was thinking about the file removed in the current
branch but modified in the other. However, this would never invoke
diff3.
Thanks.
--
Catalin
^ permalink raw reply
* Re: [PATCH] cleans up builtin-mv
From: Johannes Schindelin @ 2006-08-18 9:51 UTC (permalink / raw)
To: David Rientjes; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0608172301520.25827@chino.corp.google.com>
Hi,
On Thu, 17 Aug 2006, David Rientjes wrote:
> On Thu, 17 Aug 2006, David Rientjes wrote:
>
> > Cleans up builtin-mv by removing a needless check of source's length,
> > redefinition of source's length, and misuse of strlen call that was
> > already assigned.
> >
>
> I'm not sure when this command had been added to the tree
Tip of the day: "git log builtin-mv.c"
> because it definitely was not included six months ago in a git tree I
> use everyday. It seems to me like this would more appropriately be
> handled by a simple shell script
Nooooooo!
First, it _was_ a perl script, which you probably could find out by
checking your old git.
Second, it was rewritten to use Git.pm, and because _that_ did not work,
git-mv was rewritten as a builtin.
> that would be much simpler to implement and could not possibly be slower
> than this implementation.
Not slower? I beg to differ, admitting it is only a few percent. But your
statement is obviously uncorrect.
As for the "simpler to implement": "harder to port" comes into mind. And
do not try to argue that everybody and his dog could switch to Linux.
> This patch is a small fraction of what could be changed in this
> implementation and I don't doubt it will undergo a complete rewrite in
> the future.
Well, the patch has an improvement factor of almost none. I actually read
the patch, and asked myself: why would anybody fix a non-problem?
> I think the problems with it have compounded on top of itself over time
> which doesn't make a lot of sense since it appears to be a relatively
> new addition.
>
> For example:
> (length = strlen(source[i])) >= 0
Yes. Taken out of context, this sure sounds silly.
What you cleverly did not mention: It was inside a
if (!bad &&
(length = strlen(source[i])) >= 0 &&
!strncmp(destination[i], source[i], length) &&
(destination[i][length] == 0 || destination[i][length] == '/'))
construct. So, we assign the "length" variable only if we have to. And the
">= 0" trick is a common one. I could have done
!strncmp(destination[i], source[i], (length = strlen(source[i])))
but even I find that ugly.
> was _completely_ unnecessary since the previous instruction was a call to
> lstat(source[i], ...) which would return ENOENT if source[i] was empty.
Clarified enough?
> strlen(source[i]) was assigned to a variable later in the function, this
> time called "len" instead.
Only if source[i] is a directory. So again, we only do it when we need to.
> This code is _utterly_ unsatisfactory.
I disagree.
What is unsatisfactory to me is that I expected some performance
improvements from one of your earlier mails, and I see patches which
rearrange working code. This _can_ result in performance improvements, but
in these cases, no, it doesn't.
Having said that, I do not have anything against the patch being applied,
but if I see more of these i-would-like-the-cupboard-here-not-there
patches, I will just not review them any more.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] git-apply: document remaining options in the man page
From: Junio C Hamano @ 2006-08-18 9:43 UTC (permalink / raw)
To: Jonas Fonseca; +Cc: git
In-Reply-To: <20060818084545.GB4717@diku.dk>
Jonas Fonseca <fonseca@diku.dk> writes:
> Signed-off-by: Jonas Fonseca <fonseca@diku.dk>
> ---
>
> [ On top of next if that matters. ]
I'd appreciate if you can hold this off for a while; I'll be
updating git-apply a bit further, to add --verbose ("patching
foo...", "rejected hunk #4", etc.) and perhaps --clobber (apply
to working tree files that have local modifications), together
with --reject (apply hunks that could, while leaving
unapplicable ones out). I've been reluctant to spray .rej files
all over the tree and was planning to spit all into a single
stream (stdout), which is what is on "pu" now, but I changed my
mind and decided to mimic what other people do (that format is
more useful to work with because other tools like wiggle expect
.rej files next to the target files).
> I'm not sure if I got the --exclude note right, which says that running
> apply from a subdirectory automatically excludes files outside the
> subdirectory.
I was thinking about dropping the exclude altogether, actually.
Also I've never thought about the possibility of anybody running
git-apply from a subdirectory. git diffs by design always come
with full paths from the project toplevel, so I simply had a
preconception that everybody understood that applying would also
be always from the toplevel.
Don't get me wrong -- I do not mean to say that running apply
from a subdirectory is wrong. I just haven't thought about
doing so, and need to think the implications through.
> Lastly, `log -S<option>` is so great when doing these man pages updates
> and you, like me, don't know the code very well, since you can "just"
> paraphrase the commit message that will sometimes also describe the
> scenario where the option is applicable. :)
Praise pickaxe, and say no to annotate ;-).
^ permalink raw reply
* Re: Unresolved issues #3
From: Junio C Hamano @ 2006-08-18 9:26 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
In-Reply-To: <tnx1wre4e6s.fsf@arm.com>
Catalin Marinas <catalin.marinas@arm.com> writes:
> Junio C Hamano <junkio@cox.net> wrote:
>> * Martin Langhoff proposed git-xxdiff as a helper after a failed
>> merge.
>>
>> Message-ID: <11546492331601-git-send-email-martin@catalyst.net.nz>
>>
>> I like the general idea of this a lot, but am having a bit of
>> trouble envisioning how we can integrate this while making
>> sure mergers other than xxdiff can be added easily without
>> disrupting the end user experience.
>
> In StGIT I can configure the merge command and I currently use the
> attached script (I need to add it to the StGIT repository). It tries
> diff3 first and, if that fails, invokes emacs' merge (can use xxdiff
> as well). It also checks whether the file was modified in case I want
> to exit and solve the conflict later (maybe after getting conflict
> information for the other files).
Thanks. That is interesting. However, the workflow Martin
wants to use this is a bit different from this. He wants
regular diff3/merge to leave conflicted result and have "xxdiff
-U" to sort out the resulting mess (I do that manually myself
but without any wrapper).
Having said that...
> # default merger
> if os.system('diff3 -L current -L ancestor -L patched -m -E '
> '"%s" "%s" "%s" > "%s"'
> % (branch1, ancestor, branch2, output)):
> # interactive merge
> if os.path.exists(output):
> mtime = os.path.getmtime(output)
> else:
> mtime = 0
This reminds me that I should make the git barebone
porcelain-ish to use diff3 directly without using merge
wrapper.
I wonder why you do not error out when "if os.path.exists(output)"
does not hold true; you are redirecting into it, so the possibilities
are:
- the filesystem is full;
- your quota ran out;
- you do not have write permission to the directory "output"
wants to be in;
- the user gave 'fooled you"; rm -fr . "bye' as the value of
sys.argv[4];
etc. No matter what the reason, the fall-back interactive merge
would not be able to write into output either anyway, no?
Also I wonder what would happen upon a user error where an
existing file that is unwritable by the user is specified as
output.
^ permalink raw reply
* Re: Unresolved issues #3
From: Jakub Narebski @ 2006-08-18 8:56 UTC (permalink / raw)
To: git
In-Reply-To: <7vpseyelcw.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> * Jeff Garzik reports that the summary page of gitweb does not
> look at anything other than "master" which is not appropriate
> for his tree.
>
> Message-ID: <44D874F0.6000907@garzik.org>
>
> I probably should bug gitweb gang (Jakub, Luben, Martin Waitz,
> Aneesh) about this.
For performance reasons this should probably wait for functioning
git-show-refs command. BTW. summary page doesn't show "Last Updated" based
on "master", but based on "HEAD" (current branch)... which usually is
"master" I guess in typical usage.
Another thing that I think is/might be important to gitweb, is to rewrite
commitdiff and blobdiff to not use external diff, but builtin
git-diff-tree... this probably would lead to changes in the output.
I'm thinking about "-B" without "-C" nor "-M" for plain output (is this
no-op for this combination?), and "-C", "-M" for HTML output.
--
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox