* Re: CAREFUL! No more delta object support!
From: Daniel Barkalow @ 2005-06-28 20:01 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.58.0506281111480.19755@ppc970.osdl.org>
On Tue, 28 Jun 2005, Linus Torvalds wrote:
> On Tue, 28 Jun 2005, Linus Torvalds wrote:
> >
> > I can certainly add an option to git-pack-file that disables writing of
> > the index file, and just writes the pack-file to stdout.
>
> Done.
What I actually meant was that it would be useful for git-ssh-push to be
able to pack stuff as a function call rather than execing an external
program, because just sticking git-ssh-push at the end of a pipeline
doesn't work if you don't remember what the remote side has.
> > I'm not sure I
> > want to write the "parse incoming pack-file" thing, but git-unpack-objects
> > comes _reasonably_ close (but right now it seeks around using the index
> > file to resolve deltas, instead of keeping them in memory and resolving
> > them when possible).
>
> I'm still thinking about this one. I think I'll just do it.
One possibility would be to put a special type tag (like '\0') before the
hash, so that the format is more deterministic.
> One problem here is that since we don't know how big the incoming
> pack-file will be, in a streaming input environment the receiver needs to
> either make the pack-file reception be the last thing it sees, or it will
> have to live with the fact that "git-unpack-objects" will read some more
> than it needs before it notices that it got it all...
In a completely streaming environment, yes; but the receiving side is the
one sending commands, so you don't run into the next thing unless you're
overlapping requests. Failing that, we can just keep a 4k buffer of stuff
we've already read around; we don't have to worry about reading into
something we won't want to read at all.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: kernel.org and GIT tree rebuilding
From: Nicolas Pitre @ 2005-06-28 21:08 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Chris Mason
In-Reply-To: <Pine.LNX.4.58.0506281201510.19755@ppc970.osdl.org>
On Tue, 28 Jun 2005, Linus Torvalds wrote:
>
>
> On Tue, 28 Jun 2005, Nicolas Pitre wrote:
> >
> > Here's one improvement to the pack format, breaking it early so it won't
> > affect anyone at this point: compressed object header. Instead of the
> > fixed 5 byte header, this patch convert it to a variable size granted
> > most object are small enough to save on the storage of the significant
> > size bytes which will be zero and packing the non-zero byte position
> > with the object type.
>
> Ok, this is against an older version and doesn't have that "read_sha1"
> thing, but yes, something like this would work.
>
> I'd prefer the encoding to be a bit different, though: make the size be
> encoded in seven bits per byte, with the high bit meaning "more to come".
> We can use four bits from the "type" byte for the initial value, making
> lengths 0-15 be free.
>
> unsigned long size;
> unsigned char c;
>
> c = *pack++;
> type =
> size = c & 15;
> type = (c >> 4) & 7;
> while (c & 0x80) {
> c = *pack++;
> size = (size << 7) + (pack & 0x7f);
> }
>
> or something. That's even denser.
OK. New patch below.
> However, I also end up wanting to add a "global header" to the pack-file,
> that contains at least the number of objects packed. We may not know how
> big the pack-file will be, but we'll at least know how many objects it
> has before we start writing it.
Probably a version signature would be a good thing too.
=====
This patch adds compressed object header. Instead of the fixed 5 byte
header, this patch convert it to a variable size granted most object are
small enough to save on the storage of the size's most significant bytes
which are zero, even ffolding bits with the object type. Objects up to
2047 bytes in size will have a header of only 2 bytes.
Signed-off-by: Nicolas Pitre <nico@cam.org>
diff --git a/pack-objects.c b/pack-objects.c
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -3,24 +3,17 @@
#include "object.h"
#include "delta.h"
#include "csum-file.h"
+#include "pack.h"
static const char pack_usage[] = "git-pack-objects [--window=N] [--depth=N] {--stdout | base-name} < object-list";
-/*
- * The object type is a single-character shorthand:
- * - 'C' for "Commit"
- * - 'T' for "Tree"
- * - 'B' for "Blob"
- * - 'G' for "taG"
- * - 'D' for "Delta"
- */
struct object_entry {
unsigned char sha1[20];
unsigned long size;
unsigned long offset;
unsigned int depth;
unsigned int hash;
- unsigned char type;
+ packed_obj_type type;
unsigned long delta_size;
struct object_entry *delta;
};
@@ -63,21 +56,30 @@ static unsigned long write_object(struct
die("object %s size inconsistency (%lu vs %lu)", sha1_to_hex(entry->sha1), size, entry->size);
/*
- * The object header is a byte of 'type' followed by four bytes of
- * length, except for deltas that has the 20 bytes of delta sha
- * instead.
+ * The object header first byte has its low 4 bits representing the
+ * object type, the 4 upper bits indicating which of the following
+ * bytes are used to build the object size. For delta objects the
+ * sha1 of the reference object is also appended.
*/
header[0] = entry->type;
- hdrlen = 5;
if (entry->delta) {
- header[0] = 'D';
- memcpy(header+5, entry->delta, 20);
+ header[0] = PACKED_DELTA;
buf = delta_against(buf, size, entry);
size = entry->delta_size;
- hdrlen = 25;
}
- datalen = htonl(size);
- memcpy(header+1, &datalen, 4);
+ header[0] |= size << 3;
+ hdrlen = 1;
+ datalen = size >> 4;
+ while (datalen) {
+ header[hdrlen - 1] |= 0x80;
+ header[hdrlen++] = datalen;
+ datalen >>= 7;
+ }
+ if (entry->delta) {
+ memcpy(header+hdrlen, entry->delta, 20);
+ hdrlen += 20;
+ }
+
sha1write(f, header, hdrlen);
datalen = sha1write_compressed(f, buf, size);
free(buf);
@@ -168,13 +170,13 @@ static void check_object(struct object_e
if (!sha1_object_info(entry->sha1, type, &entry->size)) {
if (!strcmp(type, "commit")) {
- entry->type = 'C';
+ entry->type = PACKED_COMMIT;
} else if (!strcmp(type, "tree")) {
- entry->type = 'T';
+ entry->type = PACKED_TREE;
} else if (!strcmp(type, "blob")) {
- entry->type = 'B';
+ entry->type = PACKED_BLOB;
} else if (!strcmp(type, "tag")) {
- entry->type = 'G';
+ entry->type = PACKED_TAG;
} else
die("unable to pack object %s of type %s",
sha1_to_hex(entry->sha1), type);
diff --git a/pack.h b/pack.h
new file mode 100644
--- /dev/null
+++ b/pack.h
@@ -0,0 +1,19 @@
+#ifndef PACK_H
+#define PACK_H
+
+/*
+ * The packed object type is stored in the low 3 bits of a byte.
+ * The type value 0 is a reserved prefix if ever there is more than 7
+ * object types, or any future format extensions.
+ */
+
+typedef enum {
+ PACKED_RESERVED = 0,
+ PACKED_COMMIT = 1,
+ PACKED_TREE = 2,
+ PACKED_BLOB = 3,
+ PACKED_TAG = 4,
+ PACKED_DELTA = 7
+} packed_obj_type;
+
+#endif
diff --git a/unpack-objects.c b/unpack-objects.c
--- a/unpack-objects.c
+++ b/unpack-objects.c
@@ -1,6 +1,7 @@
#include "cache.h"
#include "object.h"
#include "delta.h"
+#include "pack.h"
static int dry_run;
static int nr_entries;
@@ -12,6 +13,14 @@ struct pack_entry {
unsigned char sha1[20];
};
+static char *type_s[] = {
+ [PACKED_COMMIT] = "commit",
+ [PACKED_TREE] = "tree",
+ [PACKED_BLOB] = "blob",
+ [PACKED_TAG] = "tag",
+ [PACKED_DELTA] "delta"
+};
+
static void *pack_base;
static unsigned long pack_size;
static void *index_base;
@@ -92,7 +101,7 @@ static int check_index(void)
}
static int unpack_non_delta_entry(struct pack_entry *entry,
- int kind,
+ char *type,
unsigned char *data,
unsigned long size,
unsigned long left)
@@ -101,9 +110,8 @@ static int unpack_non_delta_entry(struct
z_stream stream;
char *buffer;
unsigned char sha1[20];
- char *type_s;
- printf("%s %c %lu\n", sha1_to_hex(entry->sha1), kind, size);
+ printf("%s %s %lu\n", sha1_to_hex(entry->sha1), type, size);
if (dry_run)
return 0;
@@ -120,22 +128,15 @@ static int unpack_non_delta_entry(struct
inflateEnd(&stream);
if ((st != Z_STREAM_END) || stream.total_out != size)
goto err_finish;
- switch (kind) {
- case 'C': type_s = "commit"; break;
- case 'T': type_s = "tree"; break;
- case 'B': type_s = "blob"; break;
- case 'G': type_s = "tag"; break;
- default: goto err_finish;
- }
- if (write_sha1_file(buffer, size, type_s, sha1) < 0)
+ if (write_sha1_file(buffer, size, type, sha1) < 0)
die("failed to write %s (%s)",
- sha1_to_hex(entry->sha1), type_s);
- printf("%s %s\n", sha1_to_hex(sha1), type_s);
+ sha1_to_hex(entry->sha1), type);
+ printf("%s %s\n", sha1_to_hex(sha1), type);
if (memcmp(sha1, entry->sha1, 20))
- die("resulting %s have wrong SHA1", type_s);
+ die("resulting %s have wrong SHA1", type);
- finish:
st = 0;
+ finish:
free(buffer);
return st;
err_finish:
@@ -184,15 +185,13 @@ static int unpack_delta_entry(struct pac
die("truncated pack file");
data = base_sha1 + 20;
data_size = left - 20;
- printf("%s D %lu", sha1_to_hex(entry->sha1), delta_size);
+ printf("%s delta %lu", sha1_to_hex(entry->sha1), delta_size);
printf(" %s\n", sha1_to_hex(base_sha1));
if (dry_run)
return 0;
- /* pack+5 is the base sha1, unless we have it, we need to
- * unpack it first.
- */
+ /* unless we have the base sha1, we need to unpack it first. */
if (!has_sha1_file(base_sha1)) {
struct pack_entry *base;
if (!find_pack_entry(base_sha1, &base))
@@ -237,7 +236,9 @@ static int unpack_delta_entry(struct pac
static void unpack_entry(struct pack_entry *entry)
{
unsigned long offset, size, left;
- unsigned char *pack;
+ unsigned char *pack, sizebits;
+ packed_obj_type type;
+ int i;
/* Have we done this one already due to deltas based on it? */
if (lookup_object(entry->sha1))
@@ -247,17 +248,28 @@ static void unpack_entry(struct pack_ent
if (offset > pack_size - 5)
die("object offset outside of pack file");
pack = pack_base + offset;
- size = (pack[1] << 24) + (pack[2] << 16) + (pack[3] << 8) + pack[4];
- left = pack_size - offset - 5;
- switch (*pack) {
- case 'C': case 'T': case 'B': case 'G':
- unpack_non_delta_entry(entry, *pack, pack+5, size, left);
+ sizebits = *pack++;
+ type = sizebits & 0x07;
+ size = (sizebits & ~0x80) >> 3;
+ i = 4;
+ while (sizebits & 0x80) {
+ sizebits = *pack++;
+ size |= (sizebits & ~0x80) << i;
+ i += 7;
+ }
+ left = pack_size - ((void *)pack - pack_base);
+ switch (type) {
+ case PACKED_COMMIT:
+ case PACKED_TREE:
+ case PACKED_BLOB:
+ case PACKED_TAG:
+ unpack_non_delta_entry(entry, type_s[type], pack, size, left);
break;
- case 'D':
- unpack_delta_entry(entry, pack+5, size, left);
+ case PACKED_DELTA:
+ unpack_delta_entry(entry, pack, size, left);
break;
default:
- die("corrupted pack file");
+ die("corrupted pack file(unknown object type %d)", type);
}
}
^ permalink raw reply
* Stacked GIT 0.3 (now more Quilt-like)
From: Catalin Marinas @ 2005-06-28 21:26 UTC (permalink / raw)
To: GIT
A new StGIT release is available from http://procode.org/stgit/
What's new in version 0.3:
* closer to the Quilt functionality
* there is only one commit object per patch which can be
indefinitely modified using the 'refresh' command. The commit
objects are stacked on top of the base and can also be accessed
via standard GIT commands
* no 'commit' command. Use 'refresh' instead
StGIT is a Python application providing similar functionality to Quilt
(i.e. pushing/popping patches to/from a stack) on top of GIT. These
operations are performed using GIT commands and the patches are stored
as GIT commit objects, allowing easy merging of the StGIT patches into
other repositories using standard GIT functionality.
Note that StGIT is not an SCM interface on top of GIT and it expects a
previously initialised GIT repository. For standard SCM operations,
either use plain GIT commands or the Cogito tool.
For more information, see the README file in the archive.
--
Catalin
^ permalink raw reply
* Re: kernel.org and GIT tree rebuilding
From: Linus Torvalds @ 2005-06-28 21:27 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Git Mailing List, Chris Mason
In-Reply-To: <Pine.LNX.4.63.0506281655140.1667@localhost.localdomain>
On Tue, 28 Jun 2005, Nicolas Pitre wrote:
>
> OK. New patch below.
Dammit, I wasted all that time doing it myself.
I just committed and pushed out my version. But mine also does sha1_file.c
right, so that you can use a packed archive in .git/objects/pack. Yours
has some other cleanups, so..
Can you double-check my version (it hasn't mirrored out yet, it seems, but
it should be there soon).
> Probably a version signature would be a good thing too.
I did that too, same format as for the index file. Nothing checks it
though.
Linus
^ permalink raw reply
* [PATCH] Bugfix: initialize pack_base to NULL.
From: Junio C Hamano @ 2005-06-28 21:55 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.58.0506281424420.19755@ppc970.osdl.org>
This was causing random segfaults, because use_packed_git() got
confused by random garbage there.
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
sha1_file.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
facb119577a28bbb3f2ac1e5f8db37fd2f6d31d8
diff --git a/sha1_file.c b/sha1_file.c
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -395,6 +395,7 @@ static struct packed_git *add_packed_git
p->pack_size = st.st_size;
p->index_base = idx_map;
p->next = NULL;
+ p->pack_base = NULL;
p->pack_last_used = 0;
return p;
}
------------
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Horst von Brand @ 2005-06-28 21:54 UTC (permalink / raw)
To: Andrew Thompson
Cc: Petr Baudis, Christopher Li, mercurial, Jeff Garzik, Linux Kernel,
Git Mailing List
In-Reply-To: <42C16877.6000909@aktzero.com>
Andrew Thompson <andrewkt@aktzero.com> wrote:
> Petr Baudis wrote:
> >>Mercurial's undo is taking a snapshot of all the changed file's repo
> >>file length at every commit or pull. It just truncate the file to
> >>original size and undo is done.
> > "Trunactes"? That sounds very wrong... you mean replace with old
> > version? Anyway, what if the file has same length? It just doesn't make
> > much sense to me.
> I believe this works because the files stored in a binary format that
> appends new changesets onto the end. Thus, truncating the new stuff
> from the end effectively removes the commit.
And is exactly the wrong way around. Even RCS stored the _last_ version and
differences to earlier ones (you'll normally want the last one (or
something near), and so occasionally having to reconstruct earlier ones by
going back isn't a big deal; having to build up the current version by
starting from /dev/null and applying each and every patch that ever touched
the file each time is expensive given enough history, besides that any
error in the file is guaranteed to destroy the current version, not
(hopefully) just making old versions unavailable). It also means that
losing old history (what you'll want to do once in a while, e.g. forget
everything before 2.8) is simple: Chop off at the right point.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Christopher Li @ 2005-06-28 18:47 UTC (permalink / raw)
To: Horst von Brand
Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <200506282154.j5SLsETL010486@laptop11.inf.utfsm.cl>
On Tue, Jun 28, 2005 at 05:54:14PM -0400, Horst von Brand wrote:
>
> And is exactly the wrong way around. Even RCS stored the _last_ version and
> differences to earlier ones (you'll normally want the last one (or
> something near), and so occasionally having to reconstruct earlier ones by
> going back isn't a big deal; having to build up the current version by
> starting from /dev/null and applying each and every patch that ever touched
> the file each time is expensive given enough history, besides that any
Mercurial store a full text node when it detect the delta gets too long
to reach certain point. So what you describe here will not happen on
mercurial. Having it append only is a very nice feature.
> error in the file is guaranteed to destroy the current version, not
> (hopefully) just making old versions unavailable). It also means that
> losing old history (what you'll want to do once in a while, e.g. forget
> everything before 2.8) is simple: Chop off at the right point.
You can still chop of the history before the full node, but rebuilding the
repositories. Mercurial save some much space that you would wonder why do you
what to chop the history if you can keep it.
Chris
^ permalink raw reply
* [PATCH 3/3] Update fsck-cache (take 2)
From: Junio C Hamano @ 2005-06-28 21:58 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <7vslz2x3vg.fsf@assigned-by-dhcp.cox.net>
The fsck-cache complains if objects referred to by files in
.git/refs/ or objects stored in files under .git/objects/??/ are
not found as stand-alone SHA1 files (i.e. found in alternate
object pools GIT_ALTERNATE_OBJECT_DIRECTORIES or packed archives
stored under .git/objects/pack).
Although this is a good semantics to maintain consistency of a
single .git/objects directory as a self contained set of
objects, it sometimes is useful to consider it is OK as long as
these "outside" objects are available.
This commit introduces a new flag, --standalone, to
git-fsck-cache. When it is not specified, connectivity checks
and .git/refs pointer checks are taught that it is OK when
expected objects do not exist under .git/objects/?? hierarchy
but are available from an packed archive or in an alternate
object pool.
Another new flag, --full, makes git-fsck-cache to check not only
the current GIT_OBJECT_DIRECTORY but also objects found in
alternate object pools and packed GIT archives.a
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
*** This completes "the other half" the fsck updates I did last
*** night was missing. Please discard that one and use this
*** instead.
Documentation/git-fsck-cache.txt | 18 +++++++++-
fsck-cache.c | 71 ++++++++++++++++++++++++++++++++------
2 files changed, 76 insertions(+), 13 deletions(-)
5cae1fa43bfeae6722d916aa764fa75d9ce1839a
diff --git a/Documentation/git-fsck-cache.txt b/Documentation/git-fsck-cache.txt
--- a/Documentation/git-fsck-cache.txt
+++ b/Documentation/git-fsck-cache.txt
@@ -9,7 +9,7 @@ git-fsck-cache - Verifies the connectivi
SYNOPSIS
--------
-'git-fsck-cache' [--tags] [--root] [--unreachable] [--cache] [<object>*]
+'git-fsck-cache' [--tags] [--root] [--unreachable] [--cache] [--standalone | --full] [<object>*]
DESCRIPTION
-----------
@@ -37,6 +37,22 @@ OPTIONS
Consider any object recorded in the cache also as a head node for
an unreachability trace.
+--standalone::
+ Limit checks to the contents of GIT_OBJECT_DIRECTORY
+ (.git/objects), making sure that it is consistent and
+ complete without referring to objects found in alternate
+ object pools listed in GIT_ALTERNATE_OBJECT_DIRECTORIES,
+ nor packed GIT archives found in .git/objects/pack;
+ cannot be used with --full.
+
+--full::
+ Check not just objects in GIT_OBJECT_DIRECTORY
+ (.git/objects), but also the ones found in alternate
+ object pools listed in GIT_ALTERNATE_OBJECT_DIRECTORIES,
+ and in packed GIT archives found in .git/objects/pack
+ and corresponding pack subdirectories in alternate
+ object pools; cannot be used with --standalone.
+
It tests SHA1 and general object sanity, and it does full tracking of
the resulting reachability and everything else. It prints out any
corruption it finds (missing or bad objects), and if you use the
diff --git a/fsck-cache.c b/fsck-cache.c
--- a/fsck-cache.c
+++ b/fsck-cache.c
@@ -12,6 +12,8 @@
static int show_root = 0;
static int show_tags = 0;
static int show_unreachable = 0;
+static int standalone = 0;
+static int check_full = 0;
static int keep_cache_objects = 0;
static unsigned char head_sha1[20];
@@ -25,13 +27,17 @@ static void check_connectivity(void)
struct object_list *refs;
if (!obj->parsed) {
- printf("missing %s %s\n",
- obj->type, sha1_to_hex(obj->sha1));
+ if (!standalone && has_sha1_file(obj->sha1))
+ ; /* it is in pack */
+ else
+ printf("missing %s %s\n",
+ obj->type, sha1_to_hex(obj->sha1));
continue;
}
for (refs = obj->refs; refs; refs = refs->next) {
- if (refs->item->parsed)
+ if (refs->item->parsed ||
+ (!standalone && has_sha1_file(refs->item->sha1)))
continue;
printf("broken link from %7s %s\n",
obj->type, sha1_to_hex(obj->sha1));
@@ -315,8 +321,11 @@ static int read_sha1_reference(const cha
return -1;
obj = lookup_object(sha1);
- if (!obj)
+ if (!obj) {
+ if (!standalone && has_sha1_file(sha1))
+ return 0; /* it is in pack */
return error("%s: invalid sha1 pointer %.40s", path, hexname);
+ }
obj->used = 1;
mark_reachable(obj, REACHABLE);
@@ -366,10 +375,20 @@ static void get_default_heads(void)
die("No default references");
}
+static void fsck_object_dir(const char *path)
+{
+ int i;
+ for (i = 0; i < 256; i++) {
+ static char dir[4096];
+ sprintf(dir, "%s/%02x", path, i);
+ fsck_dir(i, dir);
+ }
+ fsck_sha1_list();
+}
+
int main(int argc, char **argv)
{
int i, heads;
- char *sha1_dir;
for (i = 1; i < argc; i++) {
const char *arg = argv[i];
@@ -390,17 +409,45 @@ int main(int argc, char **argv)
keep_cache_objects = 1;
continue;
}
+ if (!strcmp(arg, "--standalone")) {
+ standalone = 1;
+ continue;
+ }
+ if (!strcmp(arg, "--full")) {
+ check_full = 1;
+ continue;
+ }
if (*arg == '-')
- usage("git-fsck-cache [--tags] [[--unreachable] [--cache] <head-sha1>*]");
+ usage("git-fsck-cache [--tags] [[--unreachable] [--cache] [--standalone | --full] <head-sha1>*]");
}
- sha1_dir = get_object_directory();
- for (i = 0; i < 256; i++) {
- static char dir[4096];
- sprintf(dir, "%s/%02x", sha1_dir, i);
- fsck_dir(i, dir);
+ if (standalone && check_full)
+ die("Only one of --standalone or --full can be used.");
+ if (standalone)
+ unsetenv("GIT_ALTERNATE_OBJECT_DIRECTORIES");
+
+ fsck_object_dir(get_object_directory());
+ if (check_full) {
+ int j;
+ struct packed_git *p;
+ prepare_alt_odb();
+ for (j = 0; alt_odb[j].base; j++) {
+ alt_odb[j].name[-1] = 0; /* was slash */
+ fsck_object_dir(alt_odb[j].base);
+ alt_odb[j].name[-1] = '/';
+ }
+ prepare_packed_git();
+ for (p = packed_git; p; p = p->next) {
+ int num = num_packed_objects(p);
+ for (i = 0; i < num; i++) {
+ unsigned char sha1[20];
+ nth_packed_object_sha1(p, i, sha1);
+ if (fsck_sha1(sha1) < 0)
+ fprintf(stderr, "bad sha1 entry '%s'\n", sha1_to_hex(sha1));
+
+ }
+ }
}
- fsck_sha1_list();
heads = 0;
for (i = 1; i < argc; i++) {
------------
^ permalink raw reply
* [PATCH] Expose packed_git and alt_odb.
From: Junio C Hamano @ 2005-06-28 21:56 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <7vslz2x3vg.fsf@assigned-by-dhcp.cox.net>
The commands git-fsck-cache and probably git-*-pull needs to
have a way to enumerate objects contained in packed GIT archives
and alternate object pools. This commit exposes the data
structure used to keep track of them from sha1_file.c, and adds
a couple of accessor interface functions for use by the enhanced
git-fsck-cache command.
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
cache.h | 19 +++++++++++++++++++
sha1_file.c | 43 ++++++++++++++++++++++++-------------------
2 files changed, 43 insertions(+), 19 deletions(-)
da37711700d11f8c7f44fcb6819c724978c840b7
diff --git a/cache.h b/cache.h
--- a/cache.h
+++ b/cache.h
@@ -233,4 +233,23 @@ struct checkout {
extern int checkout_entry(struct cache_entry *ce, struct checkout *state);
+extern struct alternate_object_database {
+ char *base;
+ char *name;
+} *alt_odb;
+extern void prepare_alt_odb(void);
+
+extern struct packed_git {
+ struct packed_git *next;
+ unsigned long index_size;
+ unsigned long pack_size;
+ unsigned int *index_base;
+ void *pack_base;
+ unsigned int pack_last_used;
+ char pack_name[0]; /* something like ".git/objects/pack/xxxxx.pack" */
+} *packed_git;
+extern void prepare_packed_git(void);
+extern int num_packed_objects(const struct packed_git *p);
+extern int nth_packed_object_sha1(const struct packed_git *, int, unsigned char*);
+
#endif /* CACHE_H */
diff --git a/sha1_file.c b/sha1_file.c
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -184,10 +184,7 @@ char *sha1_file_name(const unsigned char
return base;
}
-static struct alternate_object_database {
- char *base;
- char *name;
-} *alt_odb;
+struct alternate_object_database *alt_odb;
/*
* Prepare alternate object database registry.
@@ -205,13 +202,15 @@ static struct alternate_object_database
* pointed by base fields of the array elements with one xmalloc();
* the string pool immediately follows the array.
*/
-static void prepare_alt_odb(void)
+void prepare_alt_odb(void)
{
int pass, totlen, i;
const char *cp, *last;
char *op = NULL;
const char *alt = gitenv(ALTERNATE_DB_ENVIRONMENT) ? : "";
+ if (alt_odb)
+ return;
/* The first pass counts how large an area to allocate to
* hold the entire alt_odb structure, including array of
* structs and path buffers for them. The second pass fills
@@ -258,8 +257,7 @@ static char *find_sha1_file(const unsign
if (!stat(name, st))
return name;
- if (!alt_odb)
- prepare_alt_odb();
+ prepare_alt_odb();
for (i = 0; (name = alt_odb[i].name) != NULL; i++) {
fill_sha1_path(name, sha1);
if (!stat(alt_odb[i].base, st))
@@ -271,15 +269,7 @@ static char *find_sha1_file(const unsign
#define PACK_MAX_SZ (1<<26)
static int pack_used_ctr;
static unsigned long pack_mapped;
-static struct packed_git {
- struct packed_git *next;
- unsigned long index_size;
- unsigned long pack_size;
- unsigned int *index_base;
- void *pack_base;
- unsigned int pack_last_used;
- char pack_name[0]; /* something like ".git/objects/pack/xxxxx.pack" */
-} *packed_git;
+struct packed_git *packed_git;
struct pack_entry {
unsigned int offset;
@@ -430,7 +420,7 @@ static void prepare_packed_git_one(char
}
}
-static void prepare_packed_git(void)
+void prepare_packed_git(void)
{
int i;
static int run_once = 0;
@@ -439,8 +429,7 @@ static void prepare_packed_git(void)
return;
prepare_packed_git_one(get_object_directory());
- if (!alt_odb)
- prepare_alt_odb();
+ prepare_alt_odb();
for (i = 0; alt_odb[i].base != NULL; i++) {
alt_odb[i].name[0] = 0;
prepare_packed_git_one(alt_odb[i].base);
@@ -750,6 +739,22 @@ static void *unpack_entry(struct pack_en
return unpack_non_delta_entry(pack+5, size, left);
}
+int num_packed_objects(const struct packed_git *p)
+{
+ /* See check_packed_git_idx and pack-objects.c */
+ return (p->index_size - 20 - 20 - 4*256) / 24;
+}
+
+int nth_packed_object_sha1(const struct packed_git *p, int n,
+ unsigned char* sha1)
+{
+ void *index = p->index_base + 256;
+ if (n < 0 || num_packed_objects(p) <= n)
+ return -1;
+ memcpy(sha1, (index + 24 * n + 4), 20);
+ return 0;
+}
+
static int find_pack_entry_1(const unsigned char *sha1,
struct pack_entry *e, struct packed_git *p)
{
------------
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Matt Mackall @ 2005-06-28 22:14 UTC (permalink / raw)
To: Sean
Cc: mercurial, Petr Baudis, Linux Kernel, Kyle Moffett, Jeff Garzik,
Git Mailing List
In-Reply-To: <3886.10.10.10.24.1119991512.squirrel@linux1>
On Tue, Jun 28, 2005 at 04:45:12PM -0400, Sean wrote:
> On Tue, June 28, 2005 4:27 pm, Kyle Moffett said:
> > On Jun 28, 2005, at 14:01:57, Matt Mackall wrote:
> >> Everything in Mercurial is an append-only log. A transaction journal
> >> records the original length of each log so that it can be restored on
> >> failure.
> >
> > Does this mean that (excepting the "undo" feature) one could set the
> > ext3 "append-only" attribute on the repository files to avoid losing
> > data due to user account compromise?
> >
>
> Probably. In Git, which is a bit more flexible than Mecurial you can
> chmod your objects to read-only or use the ext3 immutable setting to
> protect your existing objects.
You can do this in Mercurial as well.
> You can even have a setup where objects
> are archived onto write-once media like DVD and still participate in a
> live repository, where new objects are written to hard disk, but older
> object are (automatically) sourced from the DVD.
Have fun with that. It's an excellent way to destroy your DVD drive.
Git's completely structureless filename hashing pretty much guarantees
that disk layout will degrade to worst-case random access behavior
over time. Just walking through the 2000 commit blobs in the current
tree can take minutes cold cache on a fast hard disk. Walking the 1700
tree blobs in a given version takes quite a while too.
Put that on a DVD and that could easily turn into hours of continuous
seeking for a simple operation like checking out tip of tree.
And as far as I know, ISO9660 and UDF don't really handle huge
directories well. So if you try and put the whole kernel history (200k
files, some huge number of directory blobs, and 30k-60k commit blobs)
on a DVD, you'll be really hurting.
Meanwhile the whole history (>30k changesets) with Mercurial fits on a
regular CD, with reasonable directory sizes and I/O patterns.
Not that it's really worth the trouble. It takes longer for me to burn
an ISO image to disc than to download a complete kernel repo from
kernel.org.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Sean @ 2005-06-28 22:23 UTC (permalink / raw)
To: Matt Mackall
Cc: mercurial, Petr Baudis, Linux Kernel, Kyle Moffett, Jeff Garzik,
Git Mailing List
In-Reply-To: <20050628221422.GT12006@waste.org>
On Tue, June 28, 2005 6:14 pm, Matt Mackall said:
>> You can even have a setup where objects
>> are archived onto write-once media like DVD and still participate in a
>> live repository, where new objects are written to hard disk, but older
>> object are (automatically) sourced from the DVD.
>
> Have fun with that. It's an excellent way to destroy your DVD drive.
Oh come on, stop the FUD. You pack all the objects up into a single pack
file (see new feature in Git) and you burn it _once_ to dvd or cdrom.
>
> Git's completely structureless filename hashing pretty much guarantees
> that disk layout will degrade to worst-case random access behavior
> over time. Just walking through the 2000 commit blobs in the current
> tree can take minutes cold cache on a fast hard disk. Walking the 1700
> tree blobs in a given version takes quite a while too.
>
> Put that on a DVD and that could easily turn into hours of continuous
> seeking for a simple operation like checking out tip of tree.
>
> And as far as I know, ISO9660 and UDF don't really handle huge
> directories well. So if you try and put the whole kernel history (200k
> files, some huge number of directory blobs, and 30k-60k commit blobs)
> on a DVD, you'll be really hurting.
>
> Meanwhile the whole history (>30k changesets) with Mercurial fits on a
> regular CD, with reasonable directory sizes and I/O patterns.
>
> Not that it's really worth the trouble. It takes longer for me to burn
> an ISO image to disc than to download a complete kernel repo from
> kernel.org.
>
Git is still developing, there will be new ways to seek and cache things
etc eventually that remove any performance issue. Git gets this right, it
concentrates on what is important, stays flexible and trusts that down the
road as things mature any performance problems can be dealt with.
It already has some tools that are better than BK ever had (gitk, gitweb,
etc..)
Cheers,
Sean
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Kyle Moffett @ 2005-06-28 22:47 UTC (permalink / raw)
To: Sean; +Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <3993.10.10.10.24.1119997389.squirrel@linux1>
On Jun 28, 2005, at 18:23:09, Sean wrote:
> Git is still developing, there will be new ways to seek and cache
> things
> etc eventually that remove any performance issue. Git gets this
> right, it
> concentrates on what is important, stays flexible and trusts that
> down the
> road as things mature any performance problems can be dealt with.
Have you tried (or even looked at) Mercurial? I'm now using it for four
different projects that used to be in CVS and I'm loving it.
> It already has some tools that are better than BK ever had (gitk,
> gitweb,
> etc..)
Likewise for Mercurial, except that IMHO, a from-scratch Mercurial
pull via
HTTP + Mercurial checkout is faster than a BK or GIT checkout alone.
And
then there's the fact that it stores the whole mess in a fraction of the
space used by git.
Please, just _try_ it first. You'll like it, I promise. (It's also a
much smaller codebase too)
Cheers,
Kyle Moffett
--
I lost interest in "blade servers" when I found they didn't throw
knives at people who weren't supposed to be in your machine room.
-- Anthony de Boer
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Matt Mackall @ 2005-06-28 22:49 UTC (permalink / raw)
To: Sean
Cc: mercurial, Petr Baudis, Linux Kernel, Kyle Moffett, Jeff Garzik,
Git Mailing List
In-Reply-To: <3993.10.10.10.24.1119997389.squirrel@linux1>
On Tue, Jun 28, 2005 at 06:23:09PM -0400, Sean wrote:
> On Tue, June 28, 2005 6:14 pm, Matt Mackall said:
> >> You can even have a setup where objects
> >> are archived onto write-once media like DVD and still participate in a
> >> live repository, where new objects are written to hard disk, but older
> >> object are (automatically) sourced from the DVD.
> >
> > Have fun with that. It's an excellent way to destroy your DVD drive.
>
> Oh come on, stop the FUD. You pack all the objects up into a single pack
> file (see new feature in Git) and you burn it _once_ to dvd or cdrom.
And even as one big file, it will _still_ be layed out on disk in
pessimal order.
> > Git's completely structureless filename hashing pretty much guarantees
> > that disk layout will degrade to worst-case random access behavior
> > over time. Just walking through the 2000 commit blobs in the current
> > tree can take minutes cold cache on a fast hard disk. Walking the 1700
> > tree blobs in a given version takes quite a while too.
> >
> > Put that on a DVD and that could easily turn into hours of continuous
> > seeking for a simple operation like checking out tip of tree.
> >
> > And as far as I know, ISO9660 and UDF don't really handle huge
> > directories well. So if you try and put the whole kernel history (200k
> > files, some huge number of directory blobs, and 30k-60k commit blobs)
> > on a DVD, you'll be really hurting.
> >
> > Meanwhile the whole history (>30k changesets) with Mercurial fits on a
> > regular CD, with reasonable directory sizes and I/O patterns.
> >
> > Not that it's really worth the trouble. It takes longer for me to burn
> > an ISO image to disc than to download a complete kernel repo from
> > kernel.org.
> >
>
> Git is still developing, there will be new ways to seek and cache things
> etc eventually that remove any performance issue.
Again, have fun with that. Mercurial already went down this path a
month ago, discovered it couldn't reasonably be fixed without
abandoning the hashes as file name scheme, and changed repo layout.
Git's going to have a much harder time as it's pretty solidly tied to
lookup by contents hash. If you throw that out, you might as well use
Mercurial.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Sean @ 2005-06-28 22:59 UTC (permalink / raw)
To: Matt Mackall
Cc: mercurial, Petr Baudis, Linux Kernel, Kyle Moffett, Jeff Garzik,
Git Mailing List
In-Reply-To: <20050628224946.GU12006@waste.org>
On Tue, June 28, 2005 6:49 pm, Matt Mackall said:
> Again, have fun with that. Mercurial already went down this path a
> month ago, discovered it couldn't reasonably be fixed without
> abandoning the hashes as file name scheme, and changed repo layout.
>
> Git's going to have a much harder time as it's pretty solidly tied to
> lookup by contents hash. If you throw that out, you might as well use
> Mercurial.
>
By the sounds of it, git could just use Mecurial or some variation thereof
as a back end. Git is not tied to it's back end. Afterall, Mecurial
just took the basic ideas from Linus' and adapted them to a different back
end. But there are very few situation where Git performance is a
practical problem, and where it is things are being addressed. Git is
already so much better for the things I do than BK ever was, I'll stick
with it.
Sean.
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Kyle Moffett @ 2005-06-28 23:25 UTC (permalink / raw)
To: Sean; +Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <4846.10.10.10.24.1119999568.squirrel@linux1>
On Jun 28, 2005, at 18:59:28, Sean wrote:
> By the sounds of it, git could just use Mecurial or some variation
> thereof
> as a back end.
Umm, you seem to miss the point, sir. If you use Mercurial, there is no
reason you should layer any part of Git on top of it. It already does
everything that git does anyways.
> Git is already so much better for the things I do than BK ever was,
> I'll
> stick with it.
This is like saying "Windows 3.1 is already so much better for the
things
I do than DOS ever was, I'll stick with it." :-D
Cheers,
Kyle Moffett
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$
r !y?(-)
------END GEEK CODE BLOCK------
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Matt Mackall @ 2005-06-28 23:29 UTC (permalink / raw)
To: Sean
Cc: mercurial, Petr Baudis, Linux Kernel, Kyle Moffett, Jeff Garzik,
Git Mailing List
In-Reply-To: <4846.10.10.10.24.1119999568.squirrel@linux1>
On Tue, Jun 28, 2005 at 06:59:28PM -0400, Sean wrote:
> Afterall, Mecurial just took the basic ideas from Linus' and adapted
> them to a different back end.
??!
Just for the record, Mercurial had working distributed merge before
Git had any sort of merge at all. So it's hardly a case of me copying
git.
Linus and I both freely borrowed ideas from Monotone and others and
thus there is some rough similarity between the two.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Sean @ 2005-06-28 23:37 UTC (permalink / raw)
To: Kyle Moffett
Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <40A9C7C2-1AFE-45BC-90A5-571628304479@mac.com>
On Tue, June 28, 2005 7:25 pm, Kyle Moffett said:
> On Jun 28, 2005, at 18:59:28, Sean wrote:
>> By the sounds of it, git could just use Mecurial or some variation
>> thereof
>> as a back end.
>
> Umm, you seem to miss the point, sir. If you use Mercurial, there is no
> reason you should layer any part of Git on top of it. It already does
> everything that git does anyways.
No, you seem to miss the point. Git already does everything Mercurial
does, and does it pretty well too. The _point_ was that if the big
"feature" of Mercurial is it's on disk format, Git is perfectly capable of
copying it at any point. The on disk format just ISN'T CLOSE TO BEING
THE MOST IMPORTANT THING AT THE MOMENT.
>
>> Git is already so much better for the things I do than BK ever was,
>> I'll
>> stick with it.
>
> This is like saying "Windows 3.1 is already so much better for the
> things
> I do than DOS ever was, I'll stick with it." :-D
Yes, so what's your point? Mercurial is trying to solve a problem that is
already perfectly well handled for me by Git. Therefore I have _zero_
motivation to direct my efforts elsewhere.
Cheers,
Sean
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Kyle Moffett @ 2005-06-29 0:08 UTC (permalink / raw)
To: Sean; +Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <1765.10.10.10.24.1120001856.squirrel@linux1>
On Jun 28, 2005, at 19:37:36, Sean wrote:
> No, you seem to miss the point. Git already does everything Mercurial
> does, and does it pretty well too. The _point_ was that if the big
> "feature" of Mercurial is it's on disk format, Git is perfectly
> capable of
> copying it at any point. The on disk format just ISN'T CLOSE TO
> BEING
> THE MOST IMPORTANT THING AT THE MOMENT.
Firstly, no need to shout, we can all hear you :-D.
Git and Mercurial have all of the same core functionality. The only
significant remaining difference is that Mercurial uses 1/20th the
network bandwidth and disk space. If you happen to be interested in
that advantage (as I am, due to my aging equipment and poor internet
connection), then there are two options: (1) fix git, or (2) just use
Mercurial. From my point of view, option 2 is much more productive.
You may (and probably do) have different priorities and requirements
than I do, but in my view, Mercurial is an excellent tool.
> Yes, so what's your point? Mercurial is trying to solve a problem
> that is
> already perfectly well handled for me by Git. Therefore I have
> _zero_
> motivation to direct my efforts elsewhere.
Actually, Mercurial solved some of the problems first, before git did;
distributed merge is one example that comes to mind. In any case, I'm
not trying to tell you what to use, I'm just pointing out alternatives
that are available and explaining why I like them, in case you haven't
seen them or tried them before.
Cheers,
Kyle Moffett
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$
r !y?(-)
------END GEEK CODE BLOCK------
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Kyle Moffett @ 2005-06-29 0:12 UTC (permalink / raw)
To: Horst von Brand
Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <200506282154.j5SLsETL010486@laptop11.inf.utfsm.cl>
On Jun 28, 2005, at 17:54:14, Horst von Brand wrote:
> Andrew Thompson <andrewkt@aktzero.com> wrote:
>> I believe this works because the files stored in a binary format that
>> appends new changesets onto the end. Thus, truncating the new stuff
>> from the end effectively removes the commit.
>
> And is exactly the wrong way around. Even RCS stored the _last_
> version and
> differences to earlier ones (you'll normally want the last one (or
> something near), and so occasionally having to reconstruct earlier
> ones by
> going back isn't a big deal; having to build up the current version by
> starting from /dev/null and applying each and every patch that ever
> touched
> the file each time is expensive given enough history, besides that any
> error in the file is guaranteed to destroy the current version, not
> (hopefully) just making old versions unavailable). It also means that
> losing old history (what you'll want to do once in a while, e.g.
> forget
> everything before 2.8) is simple: Chop off at the right point.
If we have versions A through A+N, Mercurial will create a new revlog
file and
store a new full version when the total size of the changes between A
and A+N
is greater than a certain amount, effectively ensuring that
retrieving the
latest version of a file is O(size-of-file) instead of O(size-of-
file*revisions).
This is the same speed as RCS for the tip, and significantly faster
than RCS
for non-tip, which is crucial for merges.
Cheers,
Kyle Moffett
--
There are two ways of constructing a software design. One way is to
make it so simple that there are obviously no deficiencies. And the
other way is to make it so complicated that there are no obvious
deficiencies.
-- C.A.R. Hoare
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Sean @ 2005-06-29 0:25 UTC (permalink / raw)
To: Kyle Moffett
Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <40A4071C-ED45-4280-928F-BCFC8761F47E@mac.com>
On Tue, June 28, 2005 8:08 pm, Kyle Moffett said:
> Firstly, no need to shout, we can all hear you :-D.
ok
>
> Git and Mercurial have all of the same core functionality. The only
> significant remaining difference is that Mercurial uses 1/20th the
> network bandwidth and disk space. If you happen to be interested in
> that advantage (as I am, due to my aging equipment and poor internet
> connection), then there are two options: (1) fix git, or (2) just use
> Mercurial. From my point of view, option 2 is much more productive.
> You may (and probably do) have different priorities and requirements
> than I do, but in my view, Mercurial is an excellent tool.
well the feature set for both are changing rapidly. i like the emphasis
placed on functionality over performance shown by the git developers (not
that git is slow, it's _way_ faster than bk ever was). also the web
interface that i looked at for mecurial (admittedly four or fivve weeks
back) didn't come close to gitweb. and the work done by jon seymour and
others on the history lineralization is just great. it's something bk
lacked and was always a thorn in my side.
> Actually, Mercurial solved some of the problems first, before git did;
> distributed merge is one example that comes to mind. In any case, I'm
> not trying to tell you what to use, I'm just pointing out alternatives
> that are available and explaining why I like them, in case you haven't
> seen them or tried them before.
there will be a price to pay if the linux community fragments over choice
of scm. the good news is that we're no longer locked into the whims of
some proprietary system. so it should be straight forward for those who
choose any tool to work with those who've chosen another. this is already
evidenced by the fact that the git repository is pulled and re-exeported
with mecurial.
anyway, all the best, just wish you guys would spend less time trying to
convert git users and more time advancing your own tool.
sean
^ permalink raw reply
* [PATCH] Use enhanced diff_delta() in the similarity estimator.
From: Junio C Hamano @ 2005-06-28 23:58 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
The diff_delta() interface has been extended to reject
generating too big a delta while we were working on the packed
GIT archive format. Take advantage of that when generating
delta in the similarity estimator used in diffcore-rename.c
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
diffcore-rename.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
e4495245cf1ffc1c443df9177c95ce9d1b5052b0
diff --git a/diffcore-rename.c b/diffcore-rename.c
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -136,6 +136,7 @@ static int estimate_similarity(struct di
*/
void *delta;
unsigned long delta_size, base_size, src_copied, literal_added;
+ unsigned long delta_limit;
int score;
/* We deal only with regular files. Symlink renames are handled
@@ -163,9 +164,13 @@ static int estimate_similarity(struct di
if (diff_populate_filespec(src, 0) || diff_populate_filespec(dst, 0))
return 0; /* error but caught downstream */
+ delta_limit = base_size * (MAX_SCORE-minimum_score) / MAX_SCORE;
delta = diff_delta(src->data, src->size,
dst->data, dst->size,
- &delta_size, ~0UL);
+ &delta_size, delta_limit);
+ if (!delta)
+ /* If delta_limit is exceeded, we have too much differences */
+ return 0;
/* A delta that has a lot of literal additions would have
* big delta_size no matter what else it does.
------------
^ permalink raw reply
* [PATCH] Emit base objects of a delta chain when the delta is output.
From: Junio C Hamano @ 2005-06-29 0:49 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.58.0506280921480.19755@ppc970.osdl.org>
>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:
LT> While adding a new object to a pack file is _possible_ (you add it to the
LT> end of the pack-file, and re-generate the index file), I would strongly
LT> suggest against it for several reasons:
OK, people have convinced me not to dream on ;-).
LT> Btw, I'm not claiming that my current pack format is "optimal" of course.
LT> For example, while I write all objects in recency order, right now that
LT> means that if a recent object has been written as a delta that depends on
LT> an older one, I actually write the delta first (correct) but I won't write
LT> the older object until its recency ordering (wrong).
I agree.
How does this one look? Lightly tested by packing, unpacking
without -n and fsck'ing, not unpacking but placing it under
.git/objects/pack and running fsck with --full, all using the
current GIT repo.
------------
Deltas are useless by themselves and when you use them you need
to get to their base objects. A base object should inherit
recency from the most recent deltified object that is based on
it and that is what this patch teaches git-pack-objects.
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
cd /opt/packrat/playpen/public/in-place/git/git.junio/
jit-diff
# - master: Use enhanced diff_delta() in the similarity estimator.
# + (working tree)
diff --git a/pack-objects.c b/pack-objects.c
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -118,6 +118,23 @@ static unsigned long write_object(struct
return hdrlen + datalen;
}
+static unsigned long write_one(struct sha1file *f,
+ struct object_entry *e,
+ unsigned long offset)
+{
+ if (e->offset)
+ /* offset starts from header size and cannot be zero
+ * if it is written already.
+ */
+ return offset;
+ e->offset = offset;
+ offset += write_object(f, e);
+ /* if we are delitified, write out its base object. */
+ if (e->delta)
+ offset = write_one(f, e->delta, offset);
+ return offset;
+}
+
static void write_pack_file(void)
{
int i;
@@ -135,11 +152,9 @@ static void write_pack_file(void)
hdr.hdr_entries = htonl(nr_objects);
sha1write(f, &hdr, sizeof(hdr));
offset = sizeof(hdr);
- for (i = 0; i < nr_objects; i++) {
- struct object_entry *entry = objects + i;
- entry->offset = offset;
- offset += write_object(f, entry);
- }
+ for (i = 0; i < nr_objects; i++)
+ offset = write_one(f, objects + i, offset);
+
sha1close(f, pack_file_sha1, 1);
mb = offset >> 20;
offset &= 0xfffff;
Compilation finished at Tue Jun 28 17:43:31
^ permalink raw reply
* Re: new features in gitk
From: Paul Mackerras @ 2005-06-28 23:41 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.58.0506271819180.19755@ppc970.osdl.org>
Linus Torvalds writes:
> Hey, cool, the automatic merge actually worked.
Yes, very cool. It'll get more interesting if you make changes to
gitk in your repository, of course. :)
Paul.
^ permalink raw reply
* Re: CAREFUL! No more delta object support!
From: Linus Torvalds @ 2005-06-29 3:53 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.58.0506281111480.19755@ppc970.osdl.org>
On Tue, 28 Jun 2005, Linus Torvalds wrote:
>
> > I'm not sure I
> > want to write the "parse incoming pack-file" thing, but git-unpack-objects
> > comes _reasonably_ close (but right now it seeks around using the index
> > file to resolve deltas, instead of keeping them in memory and resolving
> > them when possible).
>
> I'm still thinking about this one. I think I'll just do it.
Ok, done. I had to basically rewrite that unpacking logic, but the end
result is actually slightly smaller and cleaner, and it can now unpack
from a stream. That stream reading logic that uncompresses directly from
the stream buffer might be considered a bit too subtle (and somebody
should really double-check it), but hey, it works for me.
In fact, I just did this:
#
# Create empty git archive "~/unpack"
#
mkdir ~/unpack
cd ~/unpack
git-init-db
#
# Copy the git archive there over a pipe
#
cd ~/git
git-rev-list --objects HEAD | git-pack-objects --depth=50 --window=50 --stdout | (cd ~/unpack ; git-unpack-objects)
#
# Go to new archive, set up the head, and fsck to verify
#
cd ~/unpack
cat ~/git/.git/HEAD > .git/HEAD
git-fsck-cache --unreachable
Now, the above is a silly example, since I _could_ just have moved the
pack file into .git/objects/pack, but that was not the point of this whole
thing. The point was to do what a "git-ssh-push" would basically boil down
to.
I'd like somebody who knows zlib intimately to take a look at how I do the
streaming input thing (in particular, the "use(len - stream.avail_in);"
part in the inflate loop in the "get_data()" function).
Linus
^ permalink raw reply
* Re: Mercurial vs Updated git HOWTO for kernel hackers
From: Kyle Moffett @ 2005-06-29 3:53 UTC (permalink / raw)
To: Sean; +Cc: mercurial, Petr Baudis, Linux Kernel, Jeff Garzik,
Git Mailing List
In-Reply-To: <2661.10.10.10.24.1120004702.squirrel@linux1>
On Jun 28, 2005, at 20:25:02, Sean wrote:
> there will be a price to pay if the linux community fragments over
> choice
> of scm.
I don't agree. With the current set of SCMs, I don't think it will
be long
before somebody invents a gitweb/Mercurial/whatever gateway, such
that I can
"hg serve" from my Mercurial repository and have Linus "git pull" from a
multiprotocol bridge.
> the good news is that we're no longer locked into the whims of
> some proprietary system. so it should be straight forward for
> those who
> choose any tool to work with those who've chosen another. this is
> already
> evidenced by the fact that the git repository is pulled and re-
> exeported
> with mecurial.
I agree completely! Cheers to the end of proprietary revision storage!
> anyway, all the best, just wish you guys would spend less time
> trying to
> convert git users and more time advancing your own tool.
A project with no users isn't much of a project, now is it? In any
case,
this thread has long since passed its usefulness, so let's let it
die, ok?
Cheers,
Kyle Moffett
--
I lost interest in "blade servers" when I found they didn't throw
knives at people who weren't supposed to be in your machine room.
-- Anthony de Boer
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox