* Re: [PATCH] Add '--create-index' to git-unpack-objects
From: Linus Torvalds @ 2005-10-12 15:20 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: Johannes Schindelin, git, junkio
In-Reply-To: <20051012145548.GA2539@master.mivlgu.local>
On Wed, 12 Oct 2005, Sergey Vlasov wrote:
>
> Hmm, pack-objects.c:write_one() does exactly the opposite - it writes
> the base object _after_ writing out the delta (but it does not ensure
> that ordering completely, so references to base objects can be
> pointing in both directions). Why?
pack-objects.c is actually going to some trouble to make sure that the
resulting pack is "optimal" in layout for the most recent case.
Not that I have actually verified optimality, but it was _meant_ to be
that way. And my limited tests seemed to agree.
So it writes out all objects in "recency order", which is the order it
gets them from git-rev-list: it's the same order as the objects are
discovered when we traverse the history in time (except all commits come
first, since most operations will traverse the commit history more than
they will traverse the rest of the object links).
So the objects that are reachable in the most recent tree are all supposed
to be at the beginning of the pack-file, just after the commits.
Now, think about what happens if such an object is a delta against
something else...
In other words, if the most recent tree contains a delta against a much
older object, we want not only the _delta_ to be early in the pack-file,
we want the object that it is a delta _against_ to be there too (just
_after_ the delta, to be exact: we obviously read the delta first, so it
should come first in the pack).
The point being, that if you unpack the latest tree (ie "git checkout" or
any of the normal "git diff" behaviour), the pack-file will basically be
walked in a dense manner, and linearly starting roughly from the
beginning. Which is the optimal IO pattern. Dense and ascending reads.
Now, if the object is reachable through some recent branch, but the delta
is not, then that is not true. In that case, you want to write the recent
base object early in the pack-file, but you do _not_ want to write the
delta together with it, because that would be the wrong thing for the
"recent head" case: it would add stuff to the beginning of the pack-file
that isn't needed for recent objects.
So that's why it's an assymmetric thing. The preferred ordering of time
breaks the symmetry.
Linus
^ permalink raw reply
* Re: [PATCH] Add '--create-index' to git-unpack-objects
From: Johannes Schindelin @ 2005-10-12 15:08 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: git, junkio
In-Reply-To: <20051012145548.GA2539@master.mivlgu.local>
Hi,
On Wed, 12 Oct 2005, Sergey Vlasov wrote:
> Hmm, pack-objects.c:write_one() does exactly the opposite - it writes
> the base object _after_ writing out the delta (but it does not ensure
> that ordering completely, so references to base objects can be
> pointing in both directions). Why?
Okay, I did not read that far. However, having quite a few packs out there
in this format, there is no option to change the format now.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Add git-index-pack utility
From: Sergey Vlasov @ 2005-10-12 15:01 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.63.0510121632040.6307@wbgn013.biozentrum.uni-wuerzburg.de>
[-- Attachment #1: Type: text/plain, Size: 288 bytes --]
On Wed, Oct 12, 2005 at 04:33:17PM +0200, Johannes Schindelin wrote:
> you cheated! You use mmap(), not lseek()! Note that mmap() is more
> efficient only if the platform provides mmap()...
Yes, I need to steal some code for handling the deflated streams from
unpack-objects.c...
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] Try URI quoting for embedded TAB and LF in pathnames
From: Linus Torvalds @ 2005-10-12 14:59 UTC (permalink / raw)
To: Paul Eggert
Cc: Junio C Hamano, Robert Fitzsimons, Alex Riesen, git, Kai Ruemmler
In-Reply-To: <877jcjmdmq.fsf@penguin.cs.ucla.edu>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1949 bytes --]
On Tue, 11 Oct 2005, Paul Eggert wrote:
>
> > - any _raw_byte_ is in the 0x80-0x9f range (it might not be UTF-8)
>
> Why quote the raw bytes? Is this for terminal escapes on older xterm
> (or xterm-like) implementations that don't understand UTF-8?
It's not about "understanding" UTF-8.
Even a perfectly modern xterm may simply not be in UTF-8 mode: if it
wasn't in an UTF-8 locale, then it won't do UTF-8 decoding.
> If so, I'm not sure I'd bother, as it would introduce a lot of annoying
> quoting with perfectly reasonable UTF-8, and (if we assume the world
> is moving to UTF-8) it addresses a problem that is going away.
UTF-8 is only _now_ getting really widespread, and I think it's because
RedHat bit the bullet and made UTF-8 the default locale a few years ago.
These things take _decades_.
I don't know if you realize it, but it's only within the last couple of
years that the old 7-bit "finnish ASCII" went away. Finnish and Swedish
have three extra characters: åäö (latin1) and åäö (utf-8). But only
within the last few years has the really _old_ ASCII representation really
gone away so much that I don't see it at all (the characters '{' '}' and
'|' were taken over, so that if you had a Finnish ASCII font, programming
in C was really funky - but it was common enough that I could do it
without thinking much about it ;)
So lots of people still use the byte-wide encodings. Whether really old
ASCII only or some special locale-dependent one (of which latin1 and the
"win-latin1" thing are obviously the most common by far). And in that
locale, it's not the UTF-8 control characters that matter, it's the _byte_
control characters that do.
So if you want to support any other locale than UTF-8, you need to escape
them. Assuming you want to escape control characters at all, of course (I
still think it's perfectly fine to just let the raw mess through and
depend on escaping at higher levels)
Linus
^ permalink raw reply
* Re: [PATCH] Add '--create-index' to git-unpack-objects
From: Sergey Vlasov @ 2005-10-12 14:55 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, junkio
In-Reply-To: <Pine.LNX.4.63.0510121612340.6307@wbgn013.biozentrum.uni-wuerzburg.de>
[-- Attachment #1: Type: text/plain, Size: 1242 bytes --]
On Wed, Oct 12, 2005 at 04:25:22PM +0200, Johannes Schindelin wrote:
> On Wed, 12 Oct 2005, Sergey Vlasov wrote:
> > Saving unpacked objects in memory would obviously be unacceptable.
>
> Actually, this is what git-unpack-objects does. All unresolved deltas are
> stored in a linked list, and handled later.
Yes, this may be a problem if the pack is large and contains many
deltas. But these stored deltas are thrown away immediately when the
base object is found; if you want to implement a streaming reindex,
you will need to store them until you reach the end of pack (or write
those objects to some temporary files).
> Of course, it would be nicer to use a seekable file if you have one. But
> then, I am not at all sure that base objects should be allowed to come
> later in the file: since the delta chains must not be cyclic, the objects
> can be sorted. Thus, it could be guaranteed that the base objects are
> already unpacked when unpacking the derived object.
Hmm, pack-objects.c:write_one() does exactly the opposite - it writes
the base object _after_ writing out the delta (but it does not ensure
that ordering completely, so references to base objects can be
pointing in both directions). Why?
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] gitk: Add "Refs" menu
From: Pavel Roskin @ 2005-10-12 14:34 UTC (permalink / raw)
To: skimo; +Cc: Paul Mackerras, git
In-Reply-To: <20051012073139.GV8383MdfPADPa@greensroom.kotnet.org>
On Wed, 2005-10-12 at 09:31 +0200, Sven Verdoolaege wrote:
> On Tue, Oct 11, 2005 at 09:26:20PM -0400, Pavel Roskin wrote:
> > Browsing trees and opening files for a given commit would be great and
> > helpful from developers migrating from CVS.
>
> Something like this ?
>
> From: Ingo Bormuth <ibormuth@efil.de>
> To: git@vger.kernel.org
> Cc: paulus@samba.org
> Subject: [PATCH] Gitk tree view (correction)
> Message-ID: <20050824223550.GA23693@kruemel>
That's pretty good, thank you. Although I think "Tree" and "Commit"
should be links on top of the view pane rather than a button. "Commit"
button is especially confusing - one could think that it would commit
something.
Also, it would be really great to put line numbers in a separate widget
so that they are never selected with the text. When done correctly,
most users will never want to turn the line numbers off.
--
Regards,
Pavel Roskin
^ permalink raw reply
* Re: [PATCH] Add git-index-pack utility
From: Johannes Schindelin @ 2005-10-12 14:33 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: Junio C Hamano, git
In-Reply-To: <20051012135405.CDE55E005E3@center4.mivlgu.local>
Hi,
you cheated! You use mmap(), not lseek()! Note that mmap() is more
efficient only if the platform provides mmap()...
Ciao,
Dscho
^ permalink raw reply
* Re: cg-mv]
From: Zack Brown @ 2005-10-12 14:28 UTC (permalink / raw)
To: Petr Baudis; +Cc: Git Mailing List
In-Reply-To: <20051012100757.GM30889@pasky.or.cz>
On Wed, Oct 12, 2005 at 12:07:57PM +0200, Petr Baudis wrote:
> Dear diary, on Fri, Oct 07, 2005 at 04:33:33PM CEST, I got a letter
> where Zack Brown <zbrown@tumblerings.org> told me that...
> > Hi,
>
> Hello,
>
> > IIRC, file renaming is something we only care about at read time, we don't
> > actually need to track it while making the change, because git allows us to
> > track data from file to file without having to tell it that the data is moving.
> >
> > So, just to keep certain people happy, why not have the cg-mv command defined to
> > something like this:
> >
> > #!/bin/bash
> > cp $1 $2
> > cg-rm $1
> > cg-add $2
>
> so it should keep the file under the original name as well, but
> untraced? That's weird.
Yes, I mistyped. cp is wrong.
> What about
>
> #!/usr/bin/env bash
>
> if [ -e $2 ]; then
> ! got_parameter -f && die "dest exists"
> [ -e $1 ] || die "no source nor destination"
> fi
> ( [ -e $1 ] && mv $1 $2 ) && cg-add $2 && cg-rm $1
>
> plus quoting and stuff?
Yes, that's nicer. Maybe the last line should be:
( [ -e $1 ] && mv $1 $2 ) && cg-add $2 && cg-rm $1 || die "unable to rename file"
But you've already done "-e $1" earlier, right? So maybe you don't need it at
the end. Just have:
mv $1 $2 && cg-add $2 && cg-rm $1 || die "unable to rename file"
No?
Be well,
Zack
>
> --
> Petr "Pasky" Baudis
> Stuff: http://pasky.or.cz/
> VI has two modes: the one in which it beeps and the one in which
> it doesn't.
--
Zack Brown
^ permalink raw reply
* Re: [PATCH] Add '--create-index' to git-unpack-objects
From: Johannes Schindelin @ 2005-10-12 14:25 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: git, junkio
In-Reply-To: <20051012173426.56fd5c1c.vsu@altlinux.ru>
Hi,
On Wed, 12 Oct 2005, Sergey Vlasov wrote:
> On Wed, 12 Oct 2005 13:02:36 +0200 (CEST) Johannes Schindelin wrote:
>
> > static void write_object(void *buf, unsigned long size, const char *type)
> > {
> > unsigned char sha1[20];
> > - if (write_sha1_file(buf, size, type, sha1) < 0)
> > + if (create_index) {
> > + char header[100];
> > + SHA_CTX c;
> > +
> > + SHA1_Init(&c);
> > + SHA1_Update(&c, header, 1+sprintf(header, "%s %lu", type, size));
> > + SHA1_Update(&c, buf, size);
> > + SHA1_Final(current_sha1, &c);
> > + } else if (write_sha1_file(buf, size, type, sha1) < 0)
> > die("failed to write object");
>
> Sorry, but this cannot work. git-unpack-objects does a streaming
> unpack, and it needs to be able to read back the objects it has written
> out previously (in case a delta later in the stream references some
> older object).
Even worse, my code did not anticipate that the base objects could have
been handled earlier (and thus the deltas would never be resolved).
> Saving unpacked objects in memory would obviously be unacceptable.
Actually, this is what git-unpack-objects does. All unresolved deltas are
stored in a linked list, and handled later.
Of course, it would be nicer to use a seekable file if you have one. But
then, I am not at all sure that base objects should be allowed to come
later in the file: since the delta chains must not be cyclic, the objects
can be sorted. Thus, it could be guaranteed that the base objects are
already unpacked when unpacking the derived object.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH Cogito] Fix README asciidoc formatting
From: Jonas Fonseca @ 2005-10-12 14:22 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20051011214318.GY22079@pasky.or.cz>
Petr Baudis <pasky@suse.cz> wrote Tue, Oct 11, 2005:
> Dear diary, on Sun, Oct 02, 2005 at 12:56:01PM CEST, I got a letter
> where Jonas Fonseca <fonseca@diku.dk> told me that...
> > BTW, what about adding some notatation info for those boxes? It looks
> > very creative for an introduction document.
>
> What do you mean by "notation info"?
I mean some kind of explanatory caption accompanying the ASCII
illustrations. At least I lack an explanation of what you mean
by '<' in
+--------+
$ branch <
+--------+
If you could please "massage that gently into my frontal cortex" I would
be very happy.
Also you use '<-M-' for merging? Why not use '<-U- for updating instead
of '< < < <'.
--
Jonas Fonseca
^ permalink raw reply
* [report] cvsimport, gitweb
From: Nico -telmich- Schottelius @ 2005-10-12 14:08 UTC (permalink / raw)
To: Git ML
[-- Attachment #1: Type: text/plain, Size: 833 bytes --]
Hey Guys!
Just wanted to tell you that a merge from cvs/pserver to git with ssh and gitweb
works pretty fine here. Converting to git took about 15 minutes, which is pretty
okay for a two years old project.
We even saved some space:
srsyg01:~# du -sh /home/server/git/walderlift.git/ /home/cvs/walderlift/
152M /home/server/git/walderlift.git/
194M /home/cvs/walderlift/
Greetings,
Nico
P.S.: Just a small thing one of our developers found: In gitweb there's a small
redundacy when viewing the tree;
drwxr-xr-x Code tree
"tree" and "Code" link to the same thing, why is 'tree' used additionally?
[just cosmetic thing, nothing really to care about]
--
Latest project: cconfig (http://nico.schotteli.us/papers/linux/cconfig/)
Open Source nutures open minds and free, creative developers.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply
* [PATCH] cogito: use locale date_fmt in obtaining default date format
From: Junichi Uekawa @ 2005-10-12 14:02 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
[-- Attachment #1: Type: text/plain, Size: 350 bytes --]
Obtain the default date format information from the locale database
if it's available. Currently, the default date is in a fixed format that
is mostly US-centric; which results in a very weird-looking Japanese
date.
Signed-off-by: Junichi Uekawa <dancer@debian.org>
---
cg-Xlib | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
[-- Attachment #2: date_in_locale.diff --]
[-- Type: application/octet-stream, Size: 515 bytes --]
diff --git a/cg-Xlib b/cg-Xlib
index 0ed275f..d6d0cc1 100755
--- a/cg-Xlib
+++ b/cg-Xlib
@@ -210,7 +210,9 @@ showdate()
# bash doesn't like leading zeros
[ "${tzhours:1:1}" = 0 ] && tzhours=${2:0:1}${2:2:1}
secs=$(($secs + $tzhours * 3600 + $tzmins * 60))
- [ "$format" ] || format="+%a, %d %b %Y %H:%M:%S $2"
+ [ "$format" ] || \
+ format="+$(locale date_fmt)" || \
+ format="+%a, %d %b %Y %H:%M:%S $2"
if [ "$has_gnudate" ]; then
LANG=C $has_gnudate -ud "1970-01-01 UTC + $secs sec" "$format"
else
^ permalink raw reply related
* [PATCH] Add git-index-pack utility
From: Sergey Vlasov @ 2005-10-12 13:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <20051012173426.56fd5c1c.vsu@altlinux.ru>
git-index-pack builds a pack index file for an existing packed
archive. With this utility a packed archive which was transferred
without the corresponding pack index can be added to objects/pack/
without repacking.
Signed-off-by: Sergey Vlasov <vsu@altlinux.ru>
---
Documentation/git-index-pack.txt | 44 ++++
Documentation/git.txt | 3
Makefile | 2
index-pack.c | 451 ++++++++++++++++++++++++++++++++++++++
t/t5300-pack-object.sh | 18 ++
5 files changed, 517 insertions(+), 1 deletions(-)
create mode 100644 Documentation/git-index-pack.txt
create mode 100644 index-pack.c
230c0068103bd1ed87788a6464dda2d7de58c2e2
diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt
new file mode 100644
--- /dev/null
+++ b/Documentation/git-index-pack.txt
@@ -0,0 +1,44 @@
+git-index-pack(1)
+=================
+
+NAME
+----
+git-index-pack - Build pack index file for an existing packed archive
+
+
+SYNOPSIS
+--------
+'git-index-pack' [-o <index-file>] <pack-file>
+
+
+DESCRIPTION
+-----------
+Reads a packed archive (.pack) from the specified file, and
+builds a pack index file (.idx) for it. The packed archive
+together with the pack index can then be placed in the
+objects/pack/ directory of a git repository.
+
+
+OPTIONS
+-------
+-o <index-file>::
+ Write the generated pack index into the specified
+ file. Without this option the name of pack index
+ file is constructed from the name of packed archive
+ file by replacing .pack with .idx (and the program
+ fails if the name of packed archive does not end
+ with .pack).
+
+
+Author
+------
+Written by Sergey Vlasov <vsu@altlinux.ru>
+
+Documentation
+-------------
+Documentation by Sergey Vlasov
+
+GIT
+---
+Part of the gitlink:git[7] suite
+
diff --git a/Documentation/git.txt b/Documentation/git.txt
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -68,6 +68,9 @@ gitlink:git-commit-tree[1]::
gitlink:git-hash-object[1]::
Computes the object ID from a file.
+gitlink:git-index-pack.html[1]::
+ Build pack index file for an existing packed archive.
+
gitlink:git-init-db[1]::
Creates an empty git object database
diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -110,7 +110,7 @@ PROGRAMS = \
git-convert-objects$X git-diff-files$X \
git-diff-index$X git-diff-stages$X \
git-diff-tree$X git-fetch-pack$X git-fsck-objects$X \
- git-hash-object$X git-init-db$X \
+ git-hash-object$X git-index-pack$X git-init-db$X \
git-local-fetch$X git-ls-files$X git-ls-tree$X git-merge-base$X \
git-merge-index$X git-mktag$X git-pack-objects$X git-patch-id$X \
git-peek-remote$X git-prune-packed$X git-read-tree$X \
diff --git a/index-pack.c b/index-pack.c
new file mode 100644
--- /dev/null
+++ b/index-pack.c
@@ -0,0 +1,451 @@
+#include "cache.h"
+#include "delta.h"
+#include "pack.h"
+#include "csum-file.h"
+
+static const char index_pack_usage[] =
+"git-index-pack [-o index-file] pack-file";
+
+struct object_entry
+{
+ unsigned long offset;
+ enum object_type type;
+ enum object_type real_type;
+ unsigned char sha1[20];
+};
+
+struct delta_entry
+{
+ struct object_entry *obj;
+ unsigned char base_sha1[20];
+};
+
+static const char *pack_name;
+static unsigned char *pack_base;
+static unsigned long pack_size;
+static struct object_entry *objects;
+static struct delta_entry *deltas;
+static int nr_objects;
+static int nr_deltas;
+
+static void open_pack_file(void)
+{
+ int fd;
+ struct stat st;
+
+ fd = open(pack_name, O_RDONLY);
+ if (fd < 0)
+ die("cannot open packfile '%s': %s", pack_name,
+ strerror(errno));
+ if (fstat(fd, &st)) {
+ int err = errno;
+ close(fd);
+ die("cannot fstat packfile '%s': %s", pack_name,
+ strerror(err));
+ }
+ pack_size = st.st_size;
+ pack_base = mmap(NULL, pack_size, PROT_READ, MAP_PRIVATE, fd, 0);
+ if (pack_base == MAP_FAILED) {
+ int err = errno;
+ close(fd);
+ die("cannot mmap packfile '%s': %s", pack_name,
+ strerror(err));
+ }
+ close(fd);
+}
+
+static void parse_pack_header(void)
+{
+ const struct pack_header *hdr;
+ unsigned char sha1[20];
+ SHA_CTX ctx;
+
+ /* Ensure there are enough bytes for the header and final SHA1 */
+ if (pack_size < sizeof(struct pack_header) + 20)
+ die("packfile '%s' is too small", pack_name);
+
+ /* Header consistency check */
+ hdr = (void *)pack_base;
+ if (hdr->hdr_signature != htonl(PACK_SIGNATURE))
+ die("packfile '%s' signature mismatch", pack_name);
+ if (hdr->hdr_version != htonl(PACK_VERSION))
+ die("packfile '%s' version %d different from ours %d",
+ pack_name, ntohl(hdr->hdr_version), PACK_VERSION);
+
+ nr_objects = ntohl(hdr->hdr_entries);
+
+ /* Check packfile integrity */
+ SHA1_Init(&ctx);
+ SHA1_Update(&ctx, pack_base, pack_size - 20);
+ SHA1_Final(sha1, &ctx);
+ if (memcmp(sha1, pack_base + pack_size - 20, 20))
+ die("packfile '%s' SHA1 mismatch", pack_name);
+}
+
+static void bad_object(unsigned long offset, const char *format,
+ ...) NORETURN __attribute__((format (printf, 2, 3)));
+
+static void bad_object(unsigned long offset, const char *format, ...)
+{
+ va_list params;
+ char buf[1024];
+
+ va_start(params, format);
+ vsnprintf(buf, sizeof(buf), format, params);
+ va_end(params);
+ die("packfile '%s': bad object at offset %lu: %s",
+ pack_name, offset, buf);
+}
+
+static void *unpack_entry_data(unsigned long offset,
+ unsigned long *current_pos, unsigned long size)
+{
+ unsigned long pack_limit = pack_size - 20;
+ unsigned long pos = *current_pos;
+ z_stream stream;
+ void *buf = xmalloc(size);
+
+ memset(&stream, 0, sizeof(stream));
+ stream.next_out = buf;
+ stream.avail_out = size;
+ stream.next_in = pack_base + pos;
+ stream.avail_in = pack_limit - pos;
+ inflateInit(&stream);
+
+ for (;;) {
+ int ret = inflate(&stream, 0);
+ if (ret == Z_STREAM_END)
+ break;
+ if (ret != Z_OK)
+ bad_object(offset, "inflate returned %d", ret);
+ }
+ inflateEnd(&stream);
+ if (stream.total_out != size)
+ bad_object(offset, "size mismatch (expected %lu, got %lu)",
+ size, stream.total_out);
+ *current_pos = pack_limit - stream.avail_in;
+ return buf;
+}
+
+static void *unpack_raw_entry(unsigned long offset,
+ enum object_type *obj_type,
+ unsigned long *obj_size,
+ unsigned char *delta_base,
+ unsigned long *next_obj_offset)
+{
+ unsigned long pack_limit = pack_size - 20;
+ unsigned long pos = offset;
+ unsigned char c;
+ unsigned long size;
+ unsigned shift;
+ enum object_type type;
+ void *data;
+
+ c = pack_base[pos++];
+ type = (c >> 4) & 7;
+ size = (c & 15);
+ shift = 4;
+ while (c & 0x80) {
+ if (pos >= pack_limit)
+ bad_object(offset, "object extends past end of pack");
+ c = pack_base[pos++];
+ size += (c & 0x7fUL) << shift;
+ shift += 7;
+ }
+
+ switch (type) {
+ case OBJ_DELTA:
+ if (pos + 20 >= pack_limit)
+ bad_object(offset, "object extends past end of pack");
+ memcpy(delta_base, pack_base + pos, 20);
+ pos += 20;
+ /* fallthru */
+ case OBJ_COMMIT:
+ case OBJ_TREE:
+ case OBJ_BLOB:
+ case OBJ_TAG:
+ data = unpack_entry_data(offset, &pos, size);
+ break;
+ default:
+ bad_object(offset, "bad object type %d", type);
+ }
+
+ *obj_type = type;
+ *obj_size = size;
+ *next_obj_offset = pos;
+ return data;
+}
+
+static int find_delta(const unsigned char *base_sha1)
+{
+ int first = 0, last = nr_deltas;
+
+ while (first < last) {
+ int next = (first + last) / 2;
+ struct delta_entry *delta = &deltas[next];
+ int cmp;
+
+ cmp = memcmp(base_sha1, delta->base_sha1, 20);
+ if (!cmp)
+ return next;
+ if (cmp < 0) {
+ last = next;
+ continue;
+ }
+ first = next+1;
+ }
+ return -first-1;
+}
+
+static int find_deltas_based_on_sha1(const unsigned char *base_sha1,
+ int *first_index, int *last_index)
+{
+ int first = find_delta(base_sha1);
+ int last = first;
+ int end = nr_deltas - 1;
+
+ if (first < 0)
+ return -1;
+ while (first > 0 && !memcmp(deltas[first-1].base_sha1, base_sha1, 20))
+ --first;
+ while (last < end && !memcmp(deltas[last+1].base_sha1, base_sha1, 20))
+ ++last;
+ *first_index = first;
+ *last_index = last;
+ return 0;
+}
+
+static void sha1_object(const void *data, unsigned long size,
+ enum object_type type, unsigned char *sha1)
+{
+ SHA_CTX ctx;
+ char header[50];
+ int header_size;
+ const char *type_str;
+
+ switch (type) {
+ case OBJ_COMMIT: type_str = "commit"; break;
+ case OBJ_TREE: type_str = "tree"; break;
+ case OBJ_BLOB: type_str = "blob"; break;
+ case OBJ_TAG: type_str = "tag"; break;
+ default:
+ die("bad type %d", type);
+ }
+
+ header_size = sprintf(header, "%s %lu", type_str, size) + 1;
+
+ SHA1_Init(&ctx);
+ SHA1_Update(&ctx, header, header_size);
+ SHA1_Update(&ctx, data, size);
+ SHA1_Final(sha1, &ctx);
+}
+
+static void resolve_delta(struct delta_entry *delta, void *base_data,
+ unsigned long base_size, enum object_type type)
+{
+ struct object_entry *obj = delta->obj;
+ void *delta_data;
+ unsigned long delta_size;
+ void *result;
+ unsigned long result_size;
+ enum object_type delta_type;
+ unsigned char base_sha1[20];
+ unsigned long next_obj_offset;
+ int j, first, last;
+
+ obj->real_type = type;
+ delta_data = unpack_raw_entry(obj->offset, &delta_type,
+ &delta_size, base_sha1,
+ &next_obj_offset);
+ result = patch_delta(base_data, base_size, delta_data, delta_size,
+ &result_size);
+ free(delta_data);
+ if (!result)
+ bad_object(obj->offset, "failed to apply delta");
+ sha1_object(result, result_size, type, obj->sha1);
+ if (!find_deltas_based_on_sha1(obj->sha1, &first, &last)) {
+ for (j = first; j <= last; j++)
+ resolve_delta(&deltas[j], result, result_size, type);
+ }
+ free(result);
+}
+
+static int compare_delta_entry(const void *a, const void *b)
+{
+ const struct delta_entry *delta_a = a;
+ const struct delta_entry *delta_b = b;
+ return memcmp(delta_a->base_sha1, delta_b->base_sha1, 20);
+}
+
+static void parse_pack_objects(void)
+{
+ int i;
+ unsigned long offset = sizeof(struct pack_header);
+ unsigned char base_sha1[20];
+ void *data;
+ unsigned long data_size;
+
+ /*
+ * First pass:
+ * - find locations of all objects;
+ * - calculate SHA1 of all non-delta objects;
+ * - remember base SHA1 for all deltas.
+ */
+ for (i = 0; i < nr_objects; i++) {
+ struct object_entry *obj = &objects[i];
+ obj->offset = offset;
+ data = unpack_raw_entry(offset, &obj->type, &data_size,
+ base_sha1, &offset);
+ obj->real_type = obj->type;
+ if (obj->type == OBJ_DELTA) {
+ struct delta_entry *delta = &deltas[nr_deltas++];
+ delta->obj = obj;
+ memcpy(delta->base_sha1, base_sha1, 20);
+ } else
+ sha1_object(data, data_size, obj->type, obj->sha1);
+ free(data);
+ }
+ if (offset != pack_size - 20)
+ die("packfile '%s' has junk at the end", pack_name);
+
+ /* Sort deltas by base SHA1 for fast searching */
+ qsort(deltas, nr_deltas, sizeof(struct delta_entry),
+ compare_delta_entry);
+
+ /*
+ * Second pass:
+ * - for all non-delta objects, look if it is used as a base for
+ * deltas;
+ * - if used as a base, uncompress the object and apply all deltas,
+ * recursively checking if the resulting object is used as a base
+ * for some more deltas.
+ */
+ for (i = 0; i < nr_objects; i++) {
+ struct object_entry *obj = &objects[i];
+ int j, first, last;
+
+ if (obj->type == OBJ_DELTA)
+ continue;
+ if (find_deltas_based_on_sha1(obj->sha1, &first, &last))
+ continue;
+ data = unpack_raw_entry(obj->offset, &obj->type, &data_size,
+ base_sha1, &offset);
+ for (j = first; j <= last; j++)
+ resolve_delta(&deltas[j], data, data_size, obj->type);
+ free(data);
+ }
+
+ /* Check for unresolved deltas */
+ for (i = 0; i < nr_deltas; i++) {
+ if (deltas[i].obj->real_type == OBJ_DELTA)
+ die("packfile '%s' has unresolved deltas", pack_name);
+ }
+}
+
+static int sha1_compare(const void *_a, const void *_b)
+{
+ struct object_entry *a = *(struct object_entry **)_a;
+ struct object_entry *b = *(struct object_entry **)_b;
+ return memcmp(a->sha1, b->sha1, 20);
+}
+
+static void write_index_file(const char *index_name)
+{
+ struct sha1file *f;
+ struct object_entry **sorted_by_sha =
+ xcalloc(nr_objects, sizeof(struct object_entry *));
+ struct object_entry **list = sorted_by_sha;
+ struct object_entry **last = sorted_by_sha + nr_objects;
+ unsigned int array[256];
+ int i;
+
+ for (i = 0; i < nr_objects; ++i)
+ sorted_by_sha[i] = &objects[i];
+ qsort(sorted_by_sha, nr_objects, sizeof(sorted_by_sha[0]),
+ sha1_compare);
+
+ unlink(index_name);
+ f = sha1create("%s", index_name);
+
+ /*
+ * Write the first-level table (the list is sorted,
+ * but we use a 256-entry lookup to be able to avoid
+ * having to do eight extra binary search iterations).
+ */
+ for (i = 0; i < 256; i++) {
+ struct object_entry **next = list;
+ while (next < last) {
+ struct object_entry *obj = *next;
+ if (obj->sha1[0] != i)
+ break;
+ next++;
+ }
+ array[i] = htonl(next - sorted_by_sha);
+ list = next;
+ }
+ sha1write(f, array, 256 * sizeof(int));
+
+ /*
+ * Write the actual SHA1 entries..
+ */
+ list = sorted_by_sha;
+ for (i = 0; i < nr_objects; i++) {
+ struct object_entry *obj = *list++;
+ unsigned int offset = htonl(obj->offset);
+ sha1write(f, &offset, 4);
+ sha1write(f, obj->sha1, 20);
+ }
+ sha1write(f, pack_base + pack_size - 20, 20);
+ sha1close(f, NULL, 1);
+ free(sorted_by_sha);
+}
+
+int main(int argc, char **argv)
+{
+ int i;
+ char *index_name = NULL;
+ char *index_name_buf = NULL;
+
+ for (i = 1; i < argc; i++) {
+ const char *arg = argv[i];
+
+ if (*arg == '-') {
+ if (!strcmp(arg, "-o")) {
+ if (index_name || (i+1) >= argc)
+ usage(index_pack_usage);
+ index_name = argv[++i];
+ } else
+ usage(index_pack_usage);
+ continue;
+ }
+
+ if (pack_name)
+ usage(index_pack_usage);
+ pack_name = arg;
+ }
+
+ if (!pack_name)
+ usage(index_pack_usage);
+ if (!index_name) {
+ int len = strlen(pack_name);
+ if (len < 5 || strcmp(pack_name + len - 5, ".pack"))
+ die("packfile name '%s' does not end with '.pack'",
+ pack_name);
+ index_name_buf = xmalloc(len - 1);
+ memcpy(index_name_buf, pack_name, len - 5);
+ strcpy(index_name_buf + len - 5, ".idx");
+ index_name = index_name_buf;
+ }
+
+ open_pack_file();
+ parse_pack_header();
+ objects = xcalloc(nr_objects, sizeof(struct object_entry));
+ deltas = xcalloc(nr_objects, sizeof(struct delta_entry));
+ parse_pack_objects();
+ free(deltas);
+ write_index_file(index_name);
+ free(objects);
+ free(index_name_buf);
+
+ return 0;
+}
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -165,4 +165,22 @@ test_expect_success \
:'
+test_expect_success \
+ 'build pack index for an existing pack' \
+ 'cp test-1-${packname_1}.pack test-3.pack &&
+ git-index-pack -o tmp.idx test-3.pack &&
+ cmp tmp.idx test-1-${packname_1}.idx &&
+
+ git-index-pack test-3.pack &&
+ cmp test-3.idx test-1-${packname_1}.idx &&
+
+ cp test-2-${packname_2}.pack test-3.pack &&
+ git-index-pack -o tmp.idx test-2-${packname_2}.pack &&
+ cmp tmp.idx test-2-${packname_2}.idx &&
+
+ git-index-pack test-3.pack &&
+ cmp test-3.idx test-2-${packname_2}.idx &&
+
+ :'
+
test_done
^ permalink raw reply
* Re: [PATCH] Add '--create-index' to git-unpack-objects
From: Sergey Vlasov @ 2005-10-12 13:34 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, junkio
In-Reply-To: <Pine.LNX.4.63.0510121301340.30679@wbgn013.biozentrum.uni-wuerzburg.de>
[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]
On Wed, 12 Oct 2005 13:02:36 +0200 (CEST) Johannes Schindelin wrote:
> Add the option '--create-index' to git-unpack-objects, which makes it
> create an index file instead of expanding its contents. While at it,
> document the dry-run option '-n', and optionally take a pack file instead
> of stdin.
> @@ -104,7 +119,15 @@ static void added_object(unsigned char *
> static void write_object(void *buf, unsigned long size, const char *type)
> {
> unsigned char sha1[20];
> - if (write_sha1_file(buf, size, type, sha1) < 0)
> + if (create_index) {
> + char header[100];
> + SHA_CTX c;
> +
> + SHA1_Init(&c);
> + SHA1_Update(&c, header, 1+sprintf(header, "%s %lu", type, size));
> + SHA1_Update(&c, buf, size);
> + SHA1_Final(current_sha1, &c);
> + } else if (write_sha1_file(buf, size, type, sha1) < 0)
> die("failed to write object");
Sorry, but this cannot work. git-unpack-objects does a streaming
unpack, and it needs to be able to read back the objects it has written
out previously (in case a delta later in the stream references some
older object). Saving unpacked objects in memory would obviously be
unacceptable.
However, if you need to create a pack index, you obviously have a pack
file with random access ability, and in this case it is possible to
build the index efficiently (in two passes over the pack file) without
storing unpacked objects in the filesystem. I made a separate utility
to do this; will send a patch in some minutes.
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: cg-mv
From: Josef Weidendorfer @ 2005-10-12 13:14 UTC (permalink / raw)
To: git
In-Reply-To: <20051012100757.GM30889@pasky.or.cz>
On Wednesday 12 October 2005 12:07, Petr Baudis wrote:
> so it should keep the file under the original name as well, but
> untraced? That's weird. What about
>
> #!/usr/bin/env bash
>
> if [ -e $2 ]; then
> ! got_parameter -f && die "dest exists"
> [ -e $1 ] || die "no source nor destination"
> fi
> ( [ -e $1 ] && mv $1 $2 ) && cg-add $2 && cg-rm $1
>
> plus quoting and stuff?
Wishlist...
Please make it similar to mv:
* Rename directories, too
* If last argument is an existing directory
move given files/directories into that directory
Josef
^ permalink raw reply
* Re: [PATCH] gitk: Add "Refs" menu
From: Marco Costalba @ 2005-10-12 11:55 UTC (permalink / raw)
To: Pavel Roskin; +Cc: git
In-Reply-To: <1129080380.2427.21.camel@dv>
> And making gitk cooperate with stgit would be a killer application not
> just for gitk and stgit, but for git itself (i.e. it could be the reason
> why git is chosen for development over e.g. Mercurial for new projects).
>
Not to advertise, but qgit (http://digilander.libero.it/mcostalba/) already offers
stgit integration, among other things.
I plan to release a new version implementing various suggestion from the list this week, there are
also important stgit fixes and upgrades.
To have a look at new features check the git arcihve: cg-clone
http://digilander.libero.it/mcostalba/qgit.git
Marco
__________________________________
Yahoo! Music Unlimited
Access over 1 million songs. Try it free.
http://music.yahoo.com/unlimited/
^ permalink raw reply
* Re: Problems cloning
From: Kay Sievers @ 2005-10-12 11:40 UTC (permalink / raw)
To: Petr Baudis; +Cc: Nico -telmich- Schottelius, Git Mailing List
In-Reply-To: <20051012081908.GK30889@pasky.or.cz>
On Wed, Oct 12, 2005 at 10:19:08AM +0200, Petr Baudis wrote:
> Dear diary, on Sun, Oct 09, 2005 at 04:15:47PM CEST, I got a letter
> where Nico -telmich- Schottelius <nico-linux-git@schottelius.org> told me that...
> > Cloning gitweb from kernel.org fails:
> >
> > ----------------------------------------------------------------------
> > [16:10] hydrogenium:build% cg-clone http://www.kernel.org/pub/scm/git/gitweb.git
> > defaulting to local storage area
> > 16:10:35 URL:http://www.kernel.org/pub/scm/git/gitweb.git/HEAD [41/41] -> "refs/heads/.origin-fetching" [1]
> > progress: 84 objects, 295012 bytes
> > Getting pack list
> > Getting alternates list
> > error: Unable to find d263a6bd453df849c9f9211f1966c830c3cf913a under http://www.kernel.org/pub/scm/git/gitweb.git/
> >
> > Cannot obtain needed commit d263a6bd453df849c9f9211f1966c830c3cf913a
> > while processing commit f5dfb3f6a6655d4d60fdd0aaeef7b5b14226147f.
> > cg-fetch: objects fetch failed
> > cg-clone: fetch failed
> > ----------------------------------------------------------------------
> >
> > Info:
> >
> > ----------------------------------------------------------------------
> > [16:01] hydrogenium:cinit% cg --version
> > cogito-0.15.1 (cfeac5893d97b830ac31b9d41951c30f80967410)
> >
> > [16:13] hydrogenium:cinit% git --version
> > git version 0.99.7d
> > ----------------------------------------------------------------------
>
> It seems that the gitweb repository is broken wrt. fetching over HTTP,
> since it does not have the "dumb server info" containing the list of
> packs. Someone needs to run git-update-server-info over there.
Sure, I use plain rsync.
Never tried any of the tools that maintain this info. :)
Kay
^ permalink raw reply
* Re: [PATCH] Adapt tutorial to cygwin and add test case
From: Johannes Schindelin @ 2005-10-12 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vu0fnsu6f.fsf@assigned-by-dhcp.cox.net>
Hi,
On Tue, 11 Oct 2005, Junio C Hamano wrote:
> The original was both ugly and did not use boxquote. There is
> no excuse for ugliness, but not using boxquote was for a
> reason.
I am no expert in non-ugliness, and neither in asciidoc. My first version
of the patch removed the single "+" on a line, because I thought they were
remnants of a corrupted patch... By using git-whatchanged I found out that
this is not so, and finally I found that it means "continuation" in
asciidoc syntax.
So: Sorry, can't help here.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Adapt tutorial to cygwin and add test case
From: Johannes Schindelin @ 2005-10-12 11:04 UTC (permalink / raw)
To: Matthias Urlichs; +Cc: git
In-Reply-To: <pan.2005.10.12.01.27.25.688169@smurf.noris.de>
Hi,
On Wed, 12 Oct 2005, Matthias Urlichs wrote:
> Hi, Junio C Hamano wrote:
>
> > The Kosher way would be
> >
> > tree=$(git-write-tree)
> > commit=$(echo 'Initial commit' | git-commit-tree $tree)
> > git-update-ref HEAD $(commit)
> >
> > but looks quite intimidating as a tutorial material.
>
> ... which is probably why there's a "git commit" command these days.
Concur. The tutorial may contain technical details for the interested (and
for those who broke their setup...), so at that stage, I'd prefer to see
the convenient version.
Ciao,
Dscho
^ permalink raw reply
* [PATCH] Add '--create-index' to git-unpack-objects
From: Johannes Schindelin @ 2005-10-12 11:02 UTC (permalink / raw)
To: git, junkio
Add the option '--create-index' to git-unpack-objects, which makes it
create an index file instead of expanding its contents. While at it,
document the dry-run option '-n', and optionally take a pack file instead
of stdin.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
---
Is anyone interested in writing the index to stdout? It is easy to
add this feature to sha1create...
Documentation/git-unpack-objects.txt | 13 +++-
t/t5300-pack-object.sh | 5 ++
unpack-objects.c | 108 +++++++++++++++++++++++++++++++---
3 files changed, 114 insertions(+), 12 deletions(-)
applies-to: 6dfbbc44a77ad3300caed532814e7feb02ff794b
1bff150c89d2636c54fa45bf62e7b248ba2a7b8d
diff --git a/Documentation/git-unpack-objects.txt b/Documentation/git-unpack-objects.txt
index b716ba1..58120e4 100644
--- a/Documentation/git-unpack-objects.txt
+++ b/Documentation/git-unpack-objects.txt
@@ -8,14 +8,15 @@ git-unpack-objects - Unpack objects from
SYNOPSIS
--------
-'git-unpack-objects' [-q] <pack-file
+'git-unpack-objects' [-q] [-n] [--create-index index-file] [pack-file]
DESCRIPTION
-----------
-Reads a packed archive (.pack) from the standard input, and
+Reads a packed archive (.pack) from a file or stdin, and
expands the objects contained in the pack into "one-file
-one-object" format in $GIT_OBJECT_DIRECTORY.
+one-object" format in $GIT_OBJECT_DIRECTORY, or alternatively,
+create an index for it.
OPTIONS
-------
@@ -23,6 +24,12 @@ OPTIONS
The command usually shows percentage progress. This
flag suppresses it.
+-n::
+ Perform a dry run, i.e. do not write any files.
+
+--create-index <filename>::
+ Instead of unpacking the files, create an index for this
+ pack.
Author
------
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index bb62336..593bfc2 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -79,6 +79,11 @@ test_expect_success \
git-unpack-objects -n <test-2-${packname_2}.pack &&
git-unpack-objects <test-2-${packname_2}.pack'
+test_expect_success \
+ 'create-index from pack' \
+ 'git-unpack-objects --create-index index <test-2-${packname_2}.pack &&
+ cmp test-2-${packname_2}.idx index'
+
unset GIT_OBJECT_DIRECTORY
cd $TRASH/.git2
test_expect_success \
diff --git a/unpack-objects.c b/unpack-objects.c
index 8ae1a1c..b6dad74 100644
--- a/unpack-objects.c
+++ b/unpack-objects.c
@@ -2,17 +2,29 @@
#include "object.h"
#include "delta.h"
#include "pack.h"
+#include "csum-file.h"
#include <sys/time.h>
-static int dry_run, quiet;
-static const char unpack_usage[] = "git-unpack-objects [-q] < pack-file";
+static int dry_run, quiet, create_index;
+static const char unpack_usage[] = "git-unpack-objects [-n] [-q] [--create-index filename] [pack-file]";
/* We always read in 4kB chunks. */
+static int fd_in = 0;
static unsigned char buffer[4096];
+static off_t buffer_offset;
static unsigned long offset, len, eof;
static SHA_CTX ctx;
+/* To recreate an index */
+static const char* index_filename;
+typedef struct {
+ unsigned int offset;
+ unsigned char sha1[20];
+} object_entry;
+static object_entry* objects;
+static unsigned char *current_sha1;
+
/*
* Make sure at least "min" bytes are available in the buffer, and
* return the pointer to the buffer.
@@ -28,10 +40,11 @@ static void * fill(int min)
if (offset) {
SHA1_Update(&ctx, buffer, offset);
memcpy(buffer, buffer + offset, len);
+ buffer_offset += offset;
offset = 0;
}
do {
- int ret = read(0, buffer + len, sizeof(buffer) - len);
+ int ret = read(fd_in, buffer + len, sizeof(buffer) - len);
if (ret <= 0) {
if (!ret)
die("early EOF");
@@ -83,6 +96,7 @@ struct delta_info {
unsigned char base_sha1[20];
unsigned long size;
void *delta;
+ unsigned char* sha1;
struct delta_info *next;
};
@@ -95,6 +109,7 @@ static void add_delta_to_list(unsigned c
memcpy(info->base_sha1, base_sha1, 20);
info->size = size;
info->delta = delta;
+ info->sha1 = current_sha1;
info->next = delta_list;
delta_list = info;
}
@@ -104,7 +119,15 @@ static void added_object(unsigned char *
static void write_object(void *buf, unsigned long size, const char *type)
{
unsigned char sha1[20];
- if (write_sha1_file(buf, size, type, sha1) < 0)
+ if (create_index) {
+ char header[100];
+ SHA_CTX c;
+
+ SHA1_Init(&c);
+ SHA1_Update(&c, header, 1+sprintf(header, "%s %lu", type, size));
+ SHA1_Update(&c, buf, size);
+ SHA1_Final(current_sha1, &c);
+ } else if (write_sha1_file(buf, size, type, sha1) < 0)
die("failed to write object");
added_object(sha1, type, buf, size);
}
@@ -136,6 +159,7 @@ static void added_object(unsigned char *
if (!memcmp(info->base_sha1, sha1, 20)) {
*p = info->next;
p = &delta_list;
+ current_sha1 = info->sha1;
resolve_delta(type, data, size, info->delta, info->size);
free(info);
continue;
@@ -156,8 +180,10 @@ static int unpack_non_delta_entry(enum o
case OBJ_TAG: type = "tag"; break;
default: die("bad type %d", kind);
}
+
if (!dry_run)
write_object(buf, size, type);
+
free(buf);
return 0;
}
@@ -174,7 +200,7 @@ static int unpack_delta_entry(unsigned l
use(20);
delta_data = get_data(delta_size);
- if (dry_run) {
+ if (dry_run && !create_index) {
free(delta_data);
return 0;
}
@@ -239,6 +265,40 @@ static void unpack_one(unsigned nr, unsi
}
}
+int compare_object_entries(const void* a, const void* b)
+{
+ const object_entry *first = a;
+ const object_entry *second = b;
+
+ return memcmp(first->sha1, second->sha1, 20);
+}
+
+void show_index(unsigned int nr_objects, char* pack_sha1)
+{
+ unsigned int n, i;
+ unsigned int top_index[256];
+ struct sha1file* index_file;
+
+ /* sort by sha1 */
+ qsort(objects, nr_objects, sizeof(object_entry), compare_object_entries);
+
+ for (n = i = 0; i < 256; i++) {
+ while (n < nr_objects && objects[n].sha1[0] == i)
+ n++;
+ top_index[i] = htonl(n);
+ }
+ top_index[255] = nr_objects;
+
+ /* write to file */
+ index_file = sha1create(index_filename);
+ sha1write(index_file, top_index, sizeof(top_index));
+ sha1write(index_file, objects, sizeof(object_entry)*nr_objects);
+ sha1write(index_file, pack_sha1, 20);
+ sha1close(index_file, NULL, 1);
+
+ free(objects);
+}
+
/*
* We unpack from the end, older files first. Now, usually
* there are deltas etc, so we'll not actually write the
@@ -251,17 +311,29 @@ static void unpack_all(void)
unsigned version = ntohl(hdr->hdr_version);
unsigned nr_objects = ntohl(hdr->hdr_entries);
+ if (create_index)
+ objects = xmalloc(sizeof(object_entry)*nr_objects);
+
if (ntohl(hdr->hdr_signature) != PACK_SIGNATURE)
die("bad pack file");
if (version != PACK_VERSION)
die("unable to handle pack file version %d", version);
- fprintf(stderr, "Unpacking %d objects\n", nr_objects);
+ if (!quiet)
+ fprintf(stderr, "Unpacking %d objects\n", nr_objects);
use(sizeof(struct pack_header));
- for (i = 0; i < nr_objects; i++)
+ for (i = 0; i < nr_objects; i++) {
+ if (create_index) {
+ objects[i].offset = htonl(buffer_offset+offset);
+ current_sha1 = (unsigned char*)&objects[i].sha1;
+ }
unpack_one(i+1, nr_objects);
+ }
if (delta_list)
die("unresolved deltas left after unpacking");
+
+ if (create_index)
+ show_index(nr_objects, fill(20));
}
int main(int argc, char **argv)
@@ -281,11 +353,22 @@ int main(int argc, char **argv)
quiet = 1;
continue;
}
+ if (!strcmp(arg, "--create-index")) {
+ create_index = 1;
+ if (i >= argc-1)
+ usage(unpack_usage);
+ index_filename = argv[++i];
+ continue;
+ }
usage(unpack_usage);
}
- /* We don't take any non-flag arguments now.. Maybe some day */
- usage(unpack_usage);
+ if (i != argc-1)
+ usage(unpack_usage);
+
+ fd_in = open(argv[i], O_RDONLY);
+ if (fd_in < 0)
+ die("Could not open %s: %s\n", argv[i], strerror(errno));
}
SHA1_Init(&ctx);
unpack_all();
@@ -295,6 +378,9 @@ int main(int argc, char **argv)
die("final sha1 did not match");
use(20);
+ if (create_index)
+ return 0;
+
/* Write the last part of the buffer to stdout */
while (len) {
int ret = write(1, buffer + offset, len);
@@ -312,5 +398,9 @@ int main(int argc, char **argv)
/* All done */
if (!quiet)
fprintf(stderr, "\n");
+
+ if (fd_in)
+ close(fd_in);
+
return 0;
}
---
0.99.8.GIT
^ permalink raw reply related
* Re: cg-mv
From: Petr Baudis @ 2005-10-12 10:07 UTC (permalink / raw)
To: Zack Brown; +Cc: Git Mailing List
In-Reply-To: <20051007143333.GA18843@tumblerings.org>
Dear diary, on Fri, Oct 07, 2005 at 04:33:33PM CEST, I got a letter
where Zack Brown <zbrown@tumblerings.org> told me that...
> Hi,
Hello,
> IIRC, file renaming is something we only care about at read time, we don't
> actually need to track it while making the change, because git allows us to
> track data from file to file without having to tell it that the data is moving.
>
> So, just to keep certain people happy, why not have the cg-mv command defined to
> something like this:
>
> #!/bin/bash
> cp $1 $2
> cg-rm $1
> cg-add $2
so it should keep the file under the original name as well, but
untraced? That's weird. What about
#!/usr/bin/env bash
if [ -e $2 ]; then
! got_parameter -f && die "dest exists"
[ -e $1 ] || die "no source nor destination"
fi
( [ -e $1 ] && mv $1 $2 ) && cg-add $2 && cg-rm $1
plus quoting and stuff?
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: [PATCH] git-fetch --tags: deal with tags with spaces in them.
From: Petr Baudis @ 2005-10-12 8:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Martin Langhoff (CatalystIT)
In-Reply-To: <7vk6gjl2uu.fsf@assigned-by-dhcp.cox.net>
Dear diary, on Wed, Oct 12, 2005 at 07:29:45AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> I do not personally think it is too much of a restriction if we
> said we only allow tags using letters from [-a-zA-Z0-9.] (yes I
> am trying to be controversial by not allowing even latin-1
> names).
I think this is perfectly reasonable, as long as you also throw _ to the
set. ;-) Actually, cg-tag now already does at least
(echo $name | egrep -qv '[^a-zA-Z0-9_.@!:-]') || \
die "name contains invalid characters"
but I'm in no way emotionally attached to the @!: characters.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* Re: Problems cloning
From: Petr Baudis @ 2005-10-12 8:19 UTC (permalink / raw)
To: Nico -telmich- Schottelius; +Cc: kay.sievers, Git Mailing List
In-Reply-To: <20051009141547.GA8609@schottelius.org>
Dear diary, on Sun, Oct 09, 2005 at 04:15:47PM CEST, I got a letter
where Nico -telmich- Schottelius <nico-linux-git@schottelius.org> told me that...
> Cloning gitweb from kernel.org fails:
>
> ----------------------------------------------------------------------
> [16:10] hydrogenium:build% cg-clone http://www.kernel.org/pub/scm/git/gitweb.git
> defaulting to local storage area
> 16:10:35 URL:http://www.kernel.org/pub/scm/git/gitweb.git/HEAD [41/41] -> "refs/heads/.origin-fetching" [1]
> progress: 84 objects, 295012 bytes
> Getting pack list
> Getting alternates list
> error: Unable to find d263a6bd453df849c9f9211f1966c830c3cf913a under http://www.kernel.org/pub/scm/git/gitweb.git/
>
> Cannot obtain needed commit d263a6bd453df849c9f9211f1966c830c3cf913a
> while processing commit f5dfb3f6a6655d4d60fdd0aaeef7b5b14226147f.
> cg-fetch: objects fetch failed
> cg-clone: fetch failed
> ----------------------------------------------------------------------
>
> Info:
>
> ----------------------------------------------------------------------
> [16:01] hydrogenium:cinit% cg --version
> cogito-0.15.1 (cfeac5893d97b830ac31b9d41951c30f80967410)
>
> [16:13] hydrogenium:cinit% git --version
> git version 0.99.7d
> ----------------------------------------------------------------------
It seems that the gitweb repository is broken wrt. fetching over HTTP,
since it does not have the "dumb server info" containing the list of
packs. Someone needs to run git-update-server-info over there.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
^ permalink raw reply
* [PATCH] Use core.filemode.
From: Junio C Hamano @ 2005-10-12 7:43 UTC (permalink / raw)
To: git
In-Reply-To: <7vvf03r36y.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> writes:
> This is a WIP to implement update-index that does not care the
> filesystem mode change...
I finished this one and have it in the master branch. I'd
appreciate feedback from folks on affected platforms, most
notably Cygwin.
It might not be a bad idea to populate a new repository with a
default .git/config file that sets "core.filemode = false" on
Cygwin platform, using the templates mechanism. People on NTFS
might not like that -- I do not know.
Anyway, that is a separate topic.
-- >8 -- cut here -- >8 --
With "[core] filemode = false", you can tell git to ignore
differences in the working tree file only in executable bit.
* "git-update-index --refresh" does not say "needs update" if index
entry and working tree file differs only in executable bit.
* "git-update-index" on an existing path takes executable bit
from the existing index entry, if the path and index entry are
both regular files.
* "git-diff-files" and "git-diff-index" without --cached flag
pretend the path on the filesystem has the same executable
bit as the existing index entry, if the path and index entry
are both regular files.
If you are on a filesystem with unreliable mode bits, you may need to
force the executable bit after registering the path in the index.
* "git-update-index --chmod=+x foo" flips the executable bit of the
index file entry for path "foo" on. Use "--chmod=-x" to flip it
off.
Note that --chmod only works in index file and does not look at nor
update the working tree.
So if you are on a filesystem and do not have working executable bit,
you would do:
1. set the appropriate .git/config option;
2. "git-update-index --add new-file.c"
3. "git-ls-files --stage new-file.c" to see if it has the desired
mode bits. If not, e.g. to drop executable bit picked up from the
filesystem, say "git-update-index --chmod=-x new-file.c".
Signed-off-by: Junio C Hamano <junkio@cox.net>
---
diff-files.c | 10 ++++++++--
diff-index.c | 11 +++++++++--
read-cache.c | 12 ++++++++----
update-index.c | 46 +++++++++++++++++++++++++++++++++++++++++++++-
4 files changed, 70 insertions(+), 9 deletions(-)
applies-to: 8c8a7c6987bcb0eeab559a58170fccd767ce0218
3e09cdfd114651fc61656dbd45d5ec3d9352cb2b
diff --git a/diff-files.c b/diff-files.c
index 96d2c7f..8a8f9b6 100644
--- a/diff-files.c
+++ b/diff-files.c
@@ -81,7 +81,7 @@ int main(int argc, const char **argv)
for (i = 0; i < entries; i++) {
struct stat st;
- unsigned int oldmode;
+ unsigned int oldmode, newmode;
struct cache_entry *ce = active_cache[i];
int changed;
@@ -111,7 +111,13 @@ int main(int argc, const char **argv)
if (!changed && !diff_options.find_copies_harder)
continue;
oldmode = ntohl(ce->ce_mode);
- show_modified(oldmode, DIFF_FILE_CANON_MODE(st.st_mode),
+
+ newmode = DIFF_FILE_CANON_MODE(st.st_mode);
+ if (!trust_executable_bit &&
+ S_ISREG(newmode) && S_ISREG(oldmode) &&
+ ((newmode ^ oldmode) == 0111))
+ newmode = oldmode;
+ show_modified(oldmode, newmode,
ce->sha1, (changed ? null_sha1 : ce->sha1),
ce->name);
}
diff --git a/diff-index.c b/diff-index.c
index 62b36cc..c9a9f4c 100644
--- a/diff-index.c
+++ b/diff-index.c
@@ -15,7 +15,7 @@ static void show_file(const char *prefix
}
static int get_stat_data(struct cache_entry *ce,
- unsigned char **sha1p, unsigned int *modep)
+ unsigned char ** sha1p, unsigned int *modep)
{
unsigned char *sha1 = ce->sha1;
unsigned int mode = ce->ce_mode;
@@ -35,6 +35,10 @@ static int get_stat_data(struct cache_en
changed = ce_match_stat(ce, &st);
if (changed) {
mode = create_ce_mode(st.st_mode);
+ if (!trust_executable_bit &&
+ S_ISREG(mode) && S_ISREG(ce->ce_mode) &&
+ ((mode ^ ce->ce_mode) == 0111))
+ mode = ce->ce_mode;
sha1 = no_sha1;
}
}
@@ -49,7 +53,9 @@ static void show_new_file(struct cache_e
unsigned char *sha1;
unsigned int mode;
- /* New file in the index: it might actually be different in the working copy */
+ /* New file in the index: it might actually be different in
+ * the working copy.
+ */
if (get_stat_data(new, &sha1, &mode) < 0)
return;
@@ -174,6 +180,7 @@ int main(int argc, const char **argv)
int allow_options = 1;
int i;
+ git_config(git_default_config);
diff_setup(&diff_options);
for (i = 1; i < argc; i++) {
const char *arg = argv[i];
diff --git a/read-cache.c b/read-cache.c
index c7f3b26..4ed369a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -36,8 +36,11 @@ int ce_match_stat(struct cache_entry *ce
switch (ntohl(ce->ce_mode) & S_IFMT) {
case S_IFREG:
changed |= !S_ISREG(st->st_mode) ? TYPE_CHANGED : 0;
- /* We consider only the owner x bit to be relevant for "mode changes" */
- if (0100 & (ntohl(ce->ce_mode) ^ st->st_mode))
+ /* We consider only the owner x bit to be relevant for
+ * "mode changes"
+ */
+ if (trust_executable_bit &&
+ (0100 & (ntohl(ce->ce_mode) ^ st->st_mode)))
changed |= MODE_CHANGED;
break;
case S_IFLNK:
@@ -393,7 +396,7 @@ int add_cache_entry(struct cache_entry *
int skip_df_check = option & ADD_CACHE_SKIP_DFCHECK;
pos = cache_name_pos(ce->name, ntohs(ce->ce_flags));
- /* existing match? Just replace it */
+ /* existing match? Just replace it. */
if (pos >= 0) {
active_cache_changed = 1;
active_cache[pos] = ce;
@@ -416,7 +419,8 @@ int add_cache_entry(struct cache_entry *
if (!ok_to_add)
return -1;
- if (!skip_df_check && check_file_directory_conflict(ce, pos, ok_to_replace)) {
+ if (!skip_df_check &&
+ check_file_directory_conflict(ce, pos, ok_to_replace)) {
if (!ok_to_replace)
return -1;
pos = cache_name_pos(ce->name, ntohs(ce->ce_flags));
diff --git a/update-index.c b/update-index.c
index 01b4088..1eeb45d 100644
--- a/update-index.c
+++ b/update-index.c
@@ -67,13 +67,23 @@ static int add_file_to_cache(const char
return error("lstat(\"%s\"): %s", path,
strerror(errno));
}
+
namelen = strlen(path);
size = cache_entry_size(namelen);
ce = xmalloc(size);
memset(ce, 0, size);
memcpy(ce->name, path, namelen);
fill_stat_cache_info(ce, &st);
+
ce->ce_mode = create_ce_mode(st.st_mode);
+ if (!trust_executable_bit) {
+ /* If there is an existing entry, pick the mode bits
+ * from it.
+ */
+ int pos = cache_name_pos(path, namelen);
+ if (0 <= pos)
+ ce->ce_mode = active_cache[pos]->ce_mode;
+ }
ce->ce_flags = htons(namelen);
if (index_path(ce->sha1, path, &st, !info_only))
@@ -253,8 +263,32 @@ static int add_cacheinfo(const char *arg
return add_cache_entry(ce, option);
}
-static struct cache_file cache_file;
+static int chmod_path(int flip, const char *path)
+{
+ int pos;
+ struct cache_entry *ce;
+ unsigned int mode;
+
+ pos = cache_name_pos(path, strlen(path));
+ if (pos < 0)
+ return -1;
+ ce = active_cache[pos];
+ mode = ntohl(ce->ce_mode);
+ if (!S_ISREG(mode))
+ return -1;
+ switch (flip) {
+ case '+':
+ ce->ce_mode |= htonl(0111); break;
+ case '-':
+ ce->ce_mode &= htonl(~0111); break;
+ default:
+ return -1;
+ }
+ active_cache_changed = 1;
+ return 0;
+}
+static struct cache_file cache_file;
static void update_one(const char *path, const char *prefix, int prefix_length)
{
@@ -328,6 +362,8 @@ int main(int argc, const char **argv)
const char *prefix = setup_git_directory();
int prefix_length = prefix ? strlen(prefix) : 0;
+ git_config(git_default_config);
+
newfd = hold_index_file_for_update(&cache_file, get_index_file());
if (newfd < 0)
die("unable to create new cachefile");
@@ -376,6 +412,14 @@ int main(int argc, const char **argv)
i += 3;
continue;
}
+ if (!strcmp(path, "--chmod=-x") ||
+ !strcmp(path, "--chmod=+x")) {
+ if (argc <= i+1)
+ die("git-update-index: %s <path>", path);
+ if (chmod_path(path[8], argv[++i]))
+ die("git-update-index: %s cannot chmod %s", path, argv[i]);
+ continue;
+ }
if (!strcmp(path, "--info-only")) {
info_only = 1;
continue;
---
0.99.8.GIT
^ permalink raw reply related
* Re: [PATCH] gitk: Add "Refs" menu
From: Sven Verdoolaege @ 2005-10-12 7:31 UTC (permalink / raw)
To: Pavel Roskin; +Cc: Paul Mackerras, git
In-Reply-To: <1129080380.2427.21.camel@dv>
On Tue, Oct 11, 2005 at 09:26:20PM -0400, Pavel Roskin wrote:
> Browsing trees and opening files for a given commit would be great and
> helpful from developers migrating from CVS.
Something like this ?
From: Ingo Bormuth <ibormuth@efil.de>
To: git@vger.kernel.org
Cc: paulus@samba.org
Subject: [PATCH] Gitk tree view (correction)
Message-ID: <20050824223550.GA23693@kruemel>
skimo
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox