Git development
 help / color / mirror / Atom feed
* Re: regression in  92392b4
From: Johannes Schindelin @ 2008-07-23 11:37 UTC (permalink / raw)
  To: Pierre Habouzit; +Cc: Björn Steinbrink, spearce, Git ML, Junio C Hamano
In-Reply-To: <20080723111931.GF15243@artemis.madism.org>

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3148 bytes --]

Hi,

On Wed, 23 Jul 2008, Pierre Habouzit wrote:

> On Wed, Jul 23, 2008 at 10:49:04AM +0000, Johannes Schindelin wrote:
> 
> > On Wed, 23 Jul 2008, Björn Steinbrink wrote:
> > 
> > > On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
> > > >   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> > > > did, since git in next cannot fetch on a regular basis for me. The
> > > > culprit seems to be commit  92392b4:
> > > > 
> > > >     ┌─(1:11)──<~/dev/scm/git 92392b4....>──
> > > >     └[artemis] git fetch
> > > >     remote: Counting objects: 461, done.
> > > >     remote: Compressing objects: 100% (141/141), done.
> > > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > > >     fatal: Out of memory, malloc failed
> > > >     fatal: index-pack failed
> > > >     [2]    16674 abort (core dumped)  git fetch
> > > > 
> > > >     ┌─(1:12)──<~/dev/scm/git 92392b4....>──
> > > >     └[artemis] git checkout -m HEAD~1; make git-index-pack
> > > >     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
> > > >     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
> > > >     GIT_VERSION = 1.5.6.3.3.g03993
> > > > 	CC index-pack.o
> > > > 	LINK git-index-pack
> > > > 
> > > >     ┌─(1:12)──<~/dev/scm/git 03993e1....>──
> > > >     └[artemis] git fetch
> > > >     remote: Counting objects: 461, done.
> > > >     remote: Compressing objects: 100% (141/141), done.
> > > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > > >     Resolving deltas: 100% (227/227), completed with 153 local objects.
> > > >     From git://git.kernel.org/pub/scm/git/git
> > > >        5ba2c22..0868a30  html       -> origin/html
> > > >        2857e17..abeeabe  man        -> origin/man
> > > >        93310a4..95f8ebb  master     -> origin/master
> > > >        559998f..e8bf351  next       -> origin/next
> > > > 
> > > > You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> > > > broken, I've absolutely no clue about what happens.
> > > > 
> > > > All I can say is that at some point in get_data_from_pack, obj[1].idx
> > > > points to something that is *not* a sha so it's probably corrupted.
> > > > (from index-pack.c).
> > > 
> > > Here's how to reproduce:
> > 
> > Funny.  That does not reproduce the bug here at all.
> > 
> > But then, it is unsurprising, since both Pierre and me did something 
> > similar yesterday, fetching _just_ the pre-fetch refs into a freshly 
> > initted Git repository, and then fetching from kernel.org.
> > 
> > Tested on x86_64.
> 
> I can reproduce on x86_64 here.

Well, I cannot.  However, I get some pread issue on i686.  To be nice to 
kernel.org, I downloaded the pack in question:

	http://pacific.mpi-cbg.de/git/thin-pack.pack

You should be able to reproduce the behavior by piping this into

git-index-pack --stdin -v --fix-thin --keep=fetch-pack --pack_header=2,263

Hth,
Dscho

^ permalink raw reply

* Re: regression in  92392b4
From: Pierre Habouzit @ 2008-07-23 11:19 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Björn Steinbrink, spearce, Git ML, Junio C Hamano
In-Reply-To: <alpine.DEB.1.00.0807231246560.2830@eeepc-johanness>

[-- Attachment #1: Type: text/plain, Size: 4057 bytes --]

On Wed, Jul 23, 2008 at 10:49:04AM +0000, Johannes Schindelin wrote:
> Hi,
> 
> On Wed, 23 Jul 2008, Björn Steinbrink wrote:
> 
> > On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
> > >   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> > > did, since git in next cannot fetch on a regular basis for me. The
> > > culprit seems to be commit  92392b4:
> > > 
> > >     ┌─(1:11)──<~/dev/scm/git 92392b4....>──
> > >     └[artemis] git fetch
> > >     remote: Counting objects: 461, done.
> > >     remote: Compressing objects: 100% (141/141), done.
> > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > >     fatal: Out of memory, malloc failed
> > >     fatal: index-pack failed
> > >     [2]    16674 abort (core dumped)  git fetch
> > > 
> > >     ┌─(1:12)──<~/dev/scm/git 92392b4....>──
> > >     └[artemis] git checkout -m HEAD~1; make git-index-pack
> > >     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
> > >     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
> > >     GIT_VERSION = 1.5.6.3.3.g03993
> > > 	CC index-pack.o
> > > 	LINK git-index-pack
> > > 
> > >     ┌─(1:12)──<~/dev/scm/git 03993e1....>──
> > >     └[artemis] git fetch
> > >     remote: Counting objects: 461, done.
> > >     remote: Compressing objects: 100% (141/141), done.
> > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > >     Resolving deltas: 100% (227/227), completed with 153 local objects.
> > >     From git://git.kernel.org/pub/scm/git/git
> > >        5ba2c22..0868a30  html       -> origin/html
> > >        2857e17..abeeabe  man        -> origin/man
> > >        93310a4..95f8ebb  master     -> origin/master
> > >        559998f..e8bf351  next       -> origin/next
> > > 
> > > You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> > > broken, I've absolutely no clue about what happens.
> > > 
> > > All I can say is that at some point in get_data_from_pack, obj[1].idx
> > > points to something that is *not* a sha so it's probably corrupted.
> > > (from index-pack.c).
> > 
> > Here's how to reproduce:
> 
> Funny.  That does not reproduce the bug here at all.
> 
> But then, it is unsurprising, since both Pierre and me did something 
> similar yesterday, fetching _just_ the pre-fetch refs into a freshly 
> initted Git repository, and then fetching from kernel.org.
> 
> Tested on x86_64.

I can reproduce on x86_64 here. And I think I get the problem, and IMHO
the pruning thing is flawed. We need more than *one* base to be kept at
a time, pruning is too aggressive, and we still keep pointers to
actually pruned data.

This patch makes the issue non reproducible for me. The rationale is
that get_base_data will already prune and is called as often, and in
safer places.

=================================
diff --git a/index-pack.c b/index-pack.c
index ac20a46..5440e43 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -245,7 +245,6 @@ static void link_base_data(struct base_data *base, struct base_data *c)
 	c->base = base;
 	c->child = NULL;
 	base_cache_used += c->size;
-	prune_base_data(c);
 }
 
 static void unlink_base_data(struct base_data *c)
=================================

*But* I'm absolutely not sure it's enough. This should be written using
reference counting instead of the "retain" hack, and prune should not
free() anything that isn't with a 0 reference counter. The current code
is brittle, it makes my head explode when I try to understand if the
get_base_data() we temporarily keep pointers too may be harmed or not.


-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply related

* Re: [RFC] Git User's Survey 2008
From: Jakub Narebski @ 2008-07-23 11:06 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git
In-Reply-To: <7vsku1gqny.fsf@gitster.siamese.dyndns.org>

On Wed, 23 Jul 2008, Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> >    15. What operating system do you use Git on?
> >        (one or more: multiple choice, as one can use more than one OS)
> >      - Linux, *BSD (FreeBSD, OpenBSD, etc.), MS Windows/Cygwin,
> >        MS Windows/msysGit, MacOS X, other UNIX, other
> 
> Shouldn't we at least name the ones we have specific support in our
> Makefile instead of blanketting them into one "other Unices"?  We may not
> necessarily want to list all of them, but at least major ones like SunOS,
> HP-UX and AIX deserve to be listed, methinks.

There will be a place to specify what "other Unices" one uses; in last
survey (see http://git.or.cz/gitwiki/GitSurvey2007) there were 5 SunOS
users, and only 1 for HP-UX and 1 for AIX (unless I have miscounted:
it was free form question in 2007 survey).

But perhaps it would be good idea to add Solaris, SunOS, AIX, HP-UX,
and perhaps also "MS Windows (unknown)" to the list.

> >      + "What hardware platforms do you use GIT on?" question was
> >        removed; should it stay?
> 
> I think the removal of "hardware platform" question is a good idea.

I was thinking about adding "Number of cores, and number of CPUs"
question instead; it might be also interesting how many people use
32-bit machine (e.g. i386), and how much 64-bit (e.g. x86_64).

But this is just curiosity; I don't think it matters to Git code what
machine it is run on.  Operating system info is more important, as
some of OS are either not POSIX, or have default filesystems with
strange features.

And I'd rather limit number of questions in the survey...

> >    24. If you want to see Git more widely used, what do you
> >        think we could do to make this happen?
> >      + Is this question necessary/useful?  Do we need wider adoption?
> 
> My stance on this has always been that wider adoption, even though it
> might eventually come as a consequence of being the best in the field, is
> never a goal.

O.K.  I'll remove this question, then.  Which is nice, as I'd rather
have this survey be shorter (it is easier both on survey takers, and
also later in survey analysis).

> >    27. Which of the following features do you use?
> >        (zero or more: multiple choice)
> >      - git-gui or other commit tool, gitk or other history viewer, patch
> >        management interface (e.g. StGIT), bundle, eol conversion,
> >        gitattributes, submodules, separate worktree, reflog, stash,
> >        shallow clone, detaching HEAD, mergetool, interactive rebase,
> >        add --interactive or other partial commit helper, commit
> >        templates, bisect, other (not mentioned here)
> >      + should probably be sorted in some resemblance of order
> >      + are there any new features which should be listed here?

I forgot about "keeping uncommitted changes in working tree", aka
"working with dirty tree" (I don't know if it is popular enough
to be included, but it _is_ one of distinguishing features).
 
> The above is a valid and interesting question, but "Which features do you
> find unique and useful ones, compared to other systems?" would be another
> interesting question to ask to people with experience with other systems.

Good idea.  I think I add it to the survey.

> >    28. If you use some important Git features not mentioned above,
> >        what are it?
> >        (free form)
> 
> "rerere"?

As it is not visible feature, one might use it without knowledge of it.
That is why I am reluctant to include it in the above list.  One can
always add it in free-form.

> >    40. Do you read the mailing list?
> >     -  yes/no
> 
> Which mailing list?  Do we want to ask about alternative lists?
> 
> I am not sure how and where, but I think j/egit should also be
> mentioned and/or asked about.

Good idea. So it would be... err, I realized that I don't know if
j/egit has separate mailing list, and what it is.  egit is listed
in "porcelains" one can use.

Nevertheless it would be good, I think, to expand list of possible
choices for this question:

    40. Do you read the mailing list?
        (multiple choice: zero or more; "none" is just in case)
     -  none/git@vger.kernel.org/Git For Human Beings/msysGit

-- 
Jakub Narebski
Poland

^ permalink raw reply

* Re: Not so happy about build system
From: Jan Engelhardt @ 2008-07-23 11:04 UTC (permalink / raw)
  To: Miklos Vajna; +Cc: git
In-Reply-To: <20080723103312.GE32057@genesis.frugalware.org>


On Wednesday 2008-07-23 12:33, Miklos Vajna wrote:

>On Wed, Jul 23, 2008 at 11:33:56AM +0200, Jan Engelhardt <jengelh@medozas.de> wrote:
>> 	./configure --prefix=$HOME/rt --with-openssl=/opt/csw 
>> 	--with-curl=/opt/csw CFLAGS="-O2 -I/opt/csw/include -L/opt/csw/lib 
>> 	-R/opt/csw/lib"
>
>Why do you put -L/opt/csw/lib to CFLAGS instead of LDFLAGS?
>
Oversight I guess. Had --with-{openssl,curl} any effect, I would not even
need the -I and -L.

^ permalink raw reply

* Re: regression in  92392b4
From: Björn Steinbrink @ 2008-07-23 10:56 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Pierre Habouzit, spearce, Git ML, Junio C Hamano
In-Reply-To: <alpine.DEB.1.00.0807231246560.2830@eeepc-johanness>

On 2008.07.23 12:49:04 +0200, Johannes Schindelin wrote:
> Hi,
> 
> On Wed, 23 Jul 2008, Björn Steinbrink wrote:
> 
> > On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
> > >   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> > > did, since git in next cannot fetch on a regular basis for me. The
> > > culprit seems to be commit  92392b4:
> > > 
> > >     ┌─(1:11)──<~/dev/scm/git 92392b4...>──
> > >     └[artemis] git fetch
> > >     remote: Counting objects: 461, done.
> > >     remote: Compressing objects: 100% (141/141), done.
> > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > >     fatal: Out of memory, malloc failed
> > >     fatal: index-pack failed
> > >     [2]    16674 abort (core dumped)  git fetch
> > > 
> > >     ┌─(1:12)──<~/dev/scm/git 92392b4...>──
> > >     └[artemis] git checkout -m HEAD~1; make git-index-pack
> > >     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
> > >     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
> > >     GIT_VERSION = 1.5.6.3.3.g03993
> > > 	CC index-pack.o
> > > 	LINK git-index-pack
> > > 
> > >     ┌─(1:12)──<~/dev/scm/git 03993e1...>──
> > >     └[artemis] git fetch
> > >     remote: Counting objects: 461, done.
> > >     remote: Compressing objects: 100% (141/141), done.
> > >     remote: Total 263 (delta 227), reused 155 (delta 121)
> > >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> > >     Resolving deltas: 100% (227/227), completed with 153 local objects.
> > >     From git://git.kernel.org/pub/scm/git/git
> > >        5ba2c22..0868a30  html       -> origin/html
> > >        2857e17..abeeabe  man        -> origin/man
> > >        93310a4..95f8ebb  master     -> origin/master
> > >        559998f..e8bf351  next       -> origin/next
> > > 
> > > You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> > > broken, I've absolutely no clue about what happens.
> > > 
> > > All I can say is that at some point in get_data_from_pack, obj[1].idx
> > > points to something that is *not* a sha so it's probably corrupted.
> > > (from index-pack.c).
> > 
> > Here's how to reproduce:
> 
> Funny.  That does not reproduce the bug here at all.
> 
> But then, it is unsurprising, since both Pierre and me did something 
> similar yesterday, fetching _just_ the pre-fetch refs into a freshly 
> initted Git repository, and then fetching from kernel.org.
> 
> Tested on x86_64.

Weird. Same arch here. And it's 100% reproducable so far (around 25
times now, I guess).

Björn

^ permalink raw reply

* Re: regression in  92392b4
From: Johannes Schindelin @ 2008-07-23 10:49 UTC (permalink / raw)
  To: Björn Steinbrink; +Cc: Pierre Habouzit, spearce, Git ML, Junio C Hamano
In-Reply-To: <20080723101415.GA23769@atjola.homenet>

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2463 bytes --]

Hi,

On Wed, 23 Jul 2008, Björn Steinbrink wrote:

> On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
> >   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> > did, since git in next cannot fetch on a regular basis for me. The
> > culprit seems to be commit  92392b4:
> > 
> >     ┌─(1:11)──<~/dev/scm/git 92392b4...>──
> >     └[artemis] git fetch
> >     remote: Counting objects: 461, done.
> >     remote: Compressing objects: 100% (141/141), done.
> >     remote: Total 263 (delta 227), reused 155 (delta 121)
> >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> >     fatal: Out of memory, malloc failed
> >     fatal: index-pack failed
> >     [2]    16674 abort (core dumped)  git fetch
> > 
> >     ┌─(1:12)──<~/dev/scm/git 92392b4...>──
> >     └[artemis] git checkout -m HEAD~1; make git-index-pack
> >     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
> >     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
> >     GIT_VERSION = 1.5.6.3.3.g03993
> > 	CC index-pack.o
> > 	LINK git-index-pack
> > 
> >     ┌─(1:12)──<~/dev/scm/git 03993e1...>──
> >     └[artemis] git fetch
> >     remote: Counting objects: 461, done.
> >     remote: Compressing objects: 100% (141/141), done.
> >     remote: Total 263 (delta 227), reused 155 (delta 121)
> >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> >     Resolving deltas: 100% (227/227), completed with 153 local objects.
> >     From git://git.kernel.org/pub/scm/git/git
> >        5ba2c22..0868a30  html       -> origin/html
> >        2857e17..abeeabe  man        -> origin/man
> >        93310a4..95f8ebb  master     -> origin/master
> >        559998f..e8bf351  next       -> origin/next
> > 
> > You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> > broken, I've absolutely no clue about what happens.
> > 
> > All I can say is that at some point in get_data_from_pack, obj[1].idx
> > points to something that is *not* a sha so it's probably corrupted.
> > (from index-pack.c).
> 
> Here's how to reproduce:

Funny.  That does not reproduce the bug here at all.

But then, it is unsurprising, since both Pierre and me did something 
similar yesterday, fetching _just_ the pre-fetch refs into a freshly 
initted Git repository, and then fetching from kernel.org.

Tested on x86_64.

Ciao,
Dscho

^ permalink raw reply

* Re: regression in  92392b4
From: Pierre Habouzit @ 2008-07-23 10:38 UTC (permalink / raw)
  To: Björn Steinbrink; +Cc: spearce, Git ML, Junio C Hamano
In-Reply-To: <20080723101415.GA23769@atjola.homenet>


[-- Attachment #1.1: Type: text/plain, Size: 1598 bytes --]

On Wed, Jul 23, 2008 at 10:14:15AM +0000, Björn Steinbrink wrote:
> Here's how to reproduce:
> 
> #!/bin/bash
> 
> [ -d git-bug ] || \
> 	git clone git://git.kernel.org/pub/scm/git/git git-bug
> cd git-bug
> 
> git update-ref refs/remotes/origin/html 5ba2c22
> git update-ref refs/remotes/origin/man 2857e17
> git update-ref refs/remotes/origin/maint 2d9c572
> git update-ref refs/remotes/origin/master 93310a4
> git update-ref refs/remotes/origin/next 559998f
> git update-ref refs/remotes/origin/pu 010581c8
> 
> git reset --hard origin/master
> 
> sleep 1
> 
> git reflog expire --expire=0 --all
> 
> git repack -A -d -f --depth=10 --window=10
> git prune
> 
> git config core.deltaBaseCacheLimit 100
> 
> git fetch

  interestingly, replacing git-index-pack with:

#!/bin/sh
exec valgrind --log-file=trace.$$ $(dirname $0)/git-index-pack2 "$@"

  it yields a different error in the console:
    remote: Counting objects: 461, done.
    remote: Compressing objects: 100% (141/141), done.
    remote: Total 263 (delta 227), reused 155 (delta 121)
    Receiving objects: 100% (263/263), 95.55 KiB | 71 KiB/s, done.
    fatal: serious inflate inconsistency
    fatal: index-pack failed
    git fetch  2,63s user 0,08s system 51% cpu 5,275 total

the valgrind log is attached, and my index-pack.c as well since I did
the c->data = NULL change which may mess up line counts.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #1.2: index-pack.c --]
[-- Type: text/plain, Size: 26656 bytes --]

#include "cache.h"
#include "delta.h"
#include "pack.h"
#include "csum-file.h"
#include "blob.h"
#include "commit.h"
#include "tag.h"
#include "tree.h"
#include "progress.h"
#include "fsck.h"

static const char index_pack_usage[] =
"git index-pack [-v] [-o <index-file>] [{ ---keep | --keep=<msg> }] [--strict] { <pack-file> | --stdin [--fix-thin] [<pack-file>] }";

struct object_entry
{
	struct pack_idx_entry idx;
	unsigned long size;
	unsigned int hdr_size;
	enum object_type type;
	enum object_type real_type;
};

union delta_base {
	unsigned char sha1[20];
	off_t offset;
};

struct base_data {
	struct base_data *base;
	struct base_data *child;
	struct object_entry *obj;
	void *data;
	unsigned long size;
};

/*
 * Even if sizeof(union delta_base) == 24 on 64-bit archs, we really want
 * to memcmp() only the first 20 bytes.
 */
#define UNION_BASE_SZ	20

#define FLAG_LINK (1u<<20)
#define FLAG_CHECKED (1u<<21)

struct delta_entry
{
	union delta_base base;
	int obj_no;
};

static struct object_entry *objects;
static struct delta_entry *deltas;
static struct base_data *base_cache;
static size_t base_cache_used;
static int nr_objects;
static int nr_deltas;
static int nr_resolved_deltas;

static int from_stdin;
static int strict;
static int verbose;

static struct progress *progress;

/* We always read in 4kB chunks. */
static unsigned char input_buffer[4096];
static unsigned int input_offset, input_len;
static off_t consumed_bytes;
static SHA_CTX input_ctx;
static uint32_t input_crc32;
static int input_fd, output_fd, pack_fd;

static int mark_link(struct object *obj, int type, void *data)
{
	if (!obj)
		return -1;

	if (type != OBJ_ANY && obj->type != type)
		die("object type mismatch at %s", sha1_to_hex(obj->sha1));

	obj->flags |= FLAG_LINK;
	return 0;
}

/* The content of each linked object must have been checked
   or it must be already present in the object database */
static void check_object(struct object *obj)
{
	if (!obj)
		return;

	if (!(obj->flags & FLAG_LINK))
		return;

	if (!(obj->flags & FLAG_CHECKED)) {
		unsigned long size;
		int type = sha1_object_info(obj->sha1, &size);
		if (type != obj->type || type <= 0)
			die("object of unexpected type");
		obj->flags |= FLAG_CHECKED;
		return;
	}
}

static void check_objects(void)
{
	unsigned i, max;

	max = get_max_object_index();
	for (i = 0; i < max; i++)
		check_object(get_indexed_object(i));
}


/* Discard current buffer used content. */
static void flush(void)
{
	if (input_offset) {
		if (output_fd >= 0)
			write_or_die(output_fd, input_buffer, input_offset);
		SHA1_Update(&input_ctx, input_buffer, input_offset);
		memmove(input_buffer, input_buffer + input_offset, input_len);
		input_offset = 0;
	}
}

/*
 * Make sure at least "min" bytes are available in the buffer, and
 * return the pointer to the buffer.
 */
static void *fill(int min)
{
	if (min <= input_len)
		return input_buffer + input_offset;
	if (min > sizeof(input_buffer))
		die("cannot fill %d bytes", min);
	flush();
	do {
		ssize_t ret = xread(input_fd, input_buffer + input_len,
				sizeof(input_buffer) - input_len);
		if (ret <= 0) {
			if (!ret)
				die("early EOF");
			die("read error on input: %s", strerror(errno));
		}
		input_len += ret;
		if (from_stdin)
			display_throughput(progress, consumed_bytes + input_len);
	} while (input_len < min);
	return input_buffer;
}

static void use(int bytes)
{
	if (bytes > input_len)
		die("used more bytes than were available");
	input_crc32 = crc32(input_crc32, input_buffer + input_offset, bytes);
	input_len -= bytes;
	input_offset += bytes;

	/* make sure off_t is sufficiently large not to wrap */
	if (consumed_bytes > consumed_bytes + bytes)
		die("pack too large for current definition of off_t");
	consumed_bytes += bytes;
}

static char *open_pack_file(char *pack_name)
{
	if (from_stdin) {
		input_fd = 0;
		if (!pack_name) {
			static char tmpfile[PATH_MAX];
			snprintf(tmpfile, sizeof(tmpfile),
				 "%s/tmp_pack_XXXXXX", get_object_directory());
			output_fd = xmkstemp(tmpfile);
			pack_name = xstrdup(tmpfile);
		} else
			output_fd = open(pack_name, O_CREAT|O_EXCL|O_RDWR, 0600);
		if (output_fd < 0)
			die("unable to create %s: %s\n", pack_name, strerror(errno));
		pack_fd = output_fd;
	} else {
		input_fd = open(pack_name, O_RDONLY);
		if (input_fd < 0)
			die("cannot open packfile '%s': %s",
			    pack_name, strerror(errno));
		output_fd = -1;
		pack_fd = input_fd;
	}
	SHA1_Init(&input_ctx);
	return pack_name;
}

static void parse_pack_header(void)
{
	struct pack_header *hdr = fill(sizeof(struct pack_header));

	/* Header consistency check */
	if (hdr->hdr_signature != htonl(PACK_SIGNATURE))
		die("pack signature mismatch");
	if (!pack_version_ok(hdr->hdr_version))
		die("pack version %"PRIu32" unsupported",
			ntohl(hdr->hdr_version));

	nr_objects = ntohl(hdr->hdr_entries);
	use(sizeof(struct pack_header));
}

static void bad_object(unsigned long offset, const char *format,
		       ...) NORETURN __attribute__((format (printf, 2, 3)));

static void bad_object(unsigned long offset, const char *format, ...)
{
	va_list params;
	char buf[1024];

	va_start(params, format);
	vsnprintf(buf, sizeof(buf), format, params);
	va_end(params);
	die("pack has bad object at offset %lu: %s", offset, buf);
}

static void prune_base_data(struct base_data *retain)
{
	struct base_data *b = base_cache;
	for (b = base_cache;
	     base_cache_used > delta_base_cache_limit && b;
	     b = b->child) {
		if (b->data && b != retain) {
			free(b->data);
			b->data = NULL;
			base_cache_used -= b->size;
		}
	}
}

static void link_base_data(struct base_data *base, struct base_data *c)
{
	if (base)
		base->child = c;
	else
		base_cache = c;

	c->base = base;
	c->child = NULL;
	base_cache_used += c->size;
	prune_base_data(c);
}

static void unlink_base_data(struct base_data *c)
{
	struct base_data *base = c->base;
	if (base)
		base->child = NULL;
	else
		base_cache = NULL;
	if (c->data) {
		free(c->data);
		c->data = NULL;
		base_cache_used -= c->size;
	}
}

static void *unpack_entry_data(unsigned long offset, unsigned long size)
{
	z_stream stream;
	void *buf = xmalloc(size);

	memset(&stream, 0, sizeof(stream));
	stream.next_out = buf;
	stream.avail_out = size;
	stream.next_in = fill(1);
	stream.avail_in = input_len;
	inflateInit(&stream);

	for (;;) {
		int ret = inflate(&stream, 0);
		use(input_len - stream.avail_in);
		if (stream.total_out == size && ret == Z_STREAM_END)
			break;
		if (ret != Z_OK)
			bad_object(offset, "inflate returned %d", ret);
		stream.next_in = fill(1);
		stream.avail_in = input_len;
	}
	inflateEnd(&stream);
	return buf;
}

static void *unpack_raw_entry(struct object_entry *obj, union delta_base *delta_base)
{
	unsigned char *p, c;
	unsigned long size;
	off_t base_offset;
	unsigned shift;
	void *data;

	obj->idx.offset = consumed_bytes;
	input_crc32 = crc32(0, Z_NULL, 0);

	p = fill(1);
	c = *p;
	use(1);
	obj->type = (c >> 4) & 7;
	size = (c & 15);
	shift = 4;
	while (c & 0x80) {
		p = fill(1);
		c = *p;
		use(1);
		size += (c & 0x7fUL) << shift;
		shift += 7;
	}
	obj->size = size;

	switch (obj->type) {
	case OBJ_REF_DELTA:
		hashcpy(delta_base->sha1, fill(20));
		use(20);
		break;
	case OBJ_OFS_DELTA:
		memset(delta_base, 0, sizeof(*delta_base));
		p = fill(1);
		c = *p;
		use(1);
		base_offset = c & 127;
		while (c & 128) {
			base_offset += 1;
			if (!base_offset || MSB(base_offset, 7))
				bad_object(obj->idx.offset, "offset value overflow for delta base object");
			p = fill(1);
			c = *p;
			use(1);
			base_offset = (base_offset << 7) + (c & 127);
		}
		delta_base->offset = obj->idx.offset - base_offset;
		if (delta_base->offset >= obj->idx.offset)
			bad_object(obj->idx.offset, "delta base offset is out of bound");
		break;
	case OBJ_COMMIT:
	case OBJ_TREE:
	case OBJ_BLOB:
	case OBJ_TAG:
		break;
	default:
		bad_object(obj->idx.offset, "unknown object type %d", obj->type);
	}
	obj->hdr_size = consumed_bytes - obj->idx.offset;

	data = unpack_entry_data(obj->idx.offset, obj->size);
	obj->idx.crc32 = input_crc32;
	return data;
}

static void *get_data_from_pack(struct object_entry *obj)
{
	off_t from = obj[0].idx.offset + obj[0].hdr_size;
	unsigned long len = obj[1].idx.offset - from;
	unsigned long rdy = 0;
	unsigned char *src, *data;
	z_stream stream;
	int st;

	src = xmalloc(len);
	data = src;
	do {
		ssize_t n = pread(pack_fd, data + rdy, len - rdy, from + rdy);
		if (n <= 0)
			die("cannot pread pack file: %s", strerror(errno));
		rdy += n;
	} while (rdy < len);
	data = xmalloc(obj->size);
	memset(&stream, 0, sizeof(stream));
	stream.next_out = data;
	stream.avail_out = obj->size;
	stream.next_in = src;
	stream.avail_in = len;
	inflateInit(&stream);
	while ((st = inflate(&stream, Z_FINISH)) == Z_OK);
	inflateEnd(&stream);
	if (st != Z_STREAM_END || stream.total_out != obj->size)
		die("serious inflate inconsistency");
	free(src);
	return data;
}

static int find_delta(const union delta_base *base)
{
	int first = 0, last = nr_deltas;

        while (first < last) {
                int next = (first + last) / 2;
                struct delta_entry *delta = &deltas[next];
                int cmp;

                cmp = memcmp(base, &delta->base, UNION_BASE_SZ);
                if (!cmp)
                        return next;
                if (cmp < 0) {
                        last = next;
                        continue;
                }
                first = next+1;
        }
        return -first-1;
}

static int find_delta_children(const union delta_base *base,
			       int *first_index, int *last_index)
{
	int first = find_delta(base);
	int last = first;
	int end = nr_deltas - 1;

	if (first < 0)
		return -1;
	while (first > 0 && !memcmp(&deltas[first - 1].base, base, UNION_BASE_SZ))
		--first;
	while (last < end && !memcmp(&deltas[last + 1].base, base, UNION_BASE_SZ))
		++last;
	*first_index = first;
	*last_index = last;
	return 0;
}

static void sha1_object(const void *data, unsigned long size,
			enum object_type type, unsigned char *sha1)
{
	hash_sha1_file(data, size, typename(type), sha1);
	if (has_sha1_file(sha1)) {
		void *has_data;
		enum object_type has_type;
		unsigned long has_size;
		has_data = read_sha1_file(sha1, &has_type, &has_size);
		if (!has_data)
			die("cannot read existing object %s", sha1_to_hex(sha1));
		if (size != has_size || type != has_type ||
		    memcmp(data, has_data, size) != 0)
			die("SHA1 COLLISION FOUND WITH %s !", sha1_to_hex(sha1));
		free(has_data);
	}
	if (strict) {
		if (type == OBJ_BLOB) {
			struct blob *blob = lookup_blob(sha1);
			if (blob)
				blob->object.flags |= FLAG_CHECKED;
			else
				die("invalid blob object %s", sha1_to_hex(sha1));
		} else {
			struct object *obj;
			int eaten;
			void *buf = (void *) data;

			/*
			 * we do not need to free the memory here, as the
			 * buf is deleted by the caller.
			 */
			obj = parse_object_buffer(sha1, type, size, buf, &eaten);
			if (!obj)
				die("invalid %s", typename(type));
			if (fsck_object(obj, 1, fsck_error_function))
				die("Error in object");
			if (fsck_walk(obj, mark_link, 0))
				die("Not all child objects of %s are reachable", sha1_to_hex(obj->sha1));

			if (obj->type == OBJ_TREE) {
				struct tree *item = (struct tree *) obj;
				item->buffer = NULL;
			}
			if (obj->type == OBJ_COMMIT) {
				struct commit *commit = (struct commit *) obj;
				commit->buffer = NULL;
			}
			obj->flags |= FLAG_CHECKED;
		}
	}
}

static void *get_base_data(struct base_data *c)
{
	if (!c->data) {
		struct object_entry *obj = c->obj;

		if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA) {
			void *base = get_base_data(c->base);
			void *raw = get_data_from_pack(obj);
			c->data = patch_delta(
				base, c->base->size,
				raw, obj->size,
				&c->size);
			free(raw);
			if (!c->data)
				bad_object(obj->idx.offset, "failed to apply delta");
		} else
			c->data = get_data_from_pack(obj);

		base_cache_used += c->size;
		prune_base_data(c);
	}
	return c->data;
}

static void resolve_delta(struct object_entry *delta_obj,
			  struct base_data *base_obj, enum object_type type)
{
	void *delta_data;
	unsigned long delta_size;
	union delta_base delta_base;
	int j, first, last;
	struct base_data result;

	delta_obj->real_type = type;
	delta_data = get_data_from_pack(delta_obj);
	delta_size = delta_obj->size;
	result.data = patch_delta(get_base_data(base_obj), base_obj->size,
			     delta_data, delta_size,
			     &result.size);
	free(delta_data);
	if (!result.data)
		bad_object(delta_obj->idx.offset, "failed to apply delta");
	sha1_object(result.data, result.size, type, delta_obj->idx.sha1);
	nr_resolved_deltas++;

	result.obj = delta_obj;
	link_base_data(base_obj, &result);

	hashcpy(delta_base.sha1, delta_obj->idx.sha1);
	if (!find_delta_children(&delta_base, &first, &last)) {
		for (j = first; j <= last; j++) {
			struct object_entry *child = objects + deltas[j].obj_no;
			if (child->real_type == OBJ_REF_DELTA)
				resolve_delta(child, &result, type);
		}
	}

	memset(&delta_base, 0, sizeof(delta_base));
	delta_base.offset = delta_obj->idx.offset;
	if (!find_delta_children(&delta_base, &first, &last)) {
		for (j = first; j <= last; j++) {
			struct object_entry *child = objects + deltas[j].obj_no;
			if (child->real_type == OBJ_OFS_DELTA)
				resolve_delta(child, &result, type);
		}
	}

	unlink_base_data(&result);
}

static int compare_delta_entry(const void *a, const void *b)
{
	const struct delta_entry *delta_a = a;
	const struct delta_entry *delta_b = b;
	return memcmp(&delta_a->base, &delta_b->base, UNION_BASE_SZ);
}

/* Parse all objects and return the pack content SHA1 hash */
static void parse_pack_objects(unsigned char *sha1)
{
	int i;
	struct delta_entry *delta = deltas;
	struct stat st;

	/*
	 * First pass:
	 * - find locations of all objects;
	 * - calculate SHA1 of all non-delta objects;
	 * - remember base (SHA1 or offset) for all deltas.
	 */
	if (verbose)
		progress = start_progress(
				from_stdin ? "Receiving objects" : "Indexing objects",
				nr_objects);
	for (i = 0; i < nr_objects; i++) {
		struct object_entry *obj = &objects[i];
		void *data = unpack_raw_entry(obj, &delta->base);
		obj->real_type = obj->type;
		if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA) {
			nr_deltas++;
			delta->obj_no = i;
			delta++;
		} else
			sha1_object(data, obj->size, obj->type, obj->idx.sha1);
		free(data);
		display_progress(progress, i+1);
	}
	objects[i].idx.offset = consumed_bytes;
	stop_progress(&progress);

	/* Check pack integrity */
	flush();
	SHA1_Final(sha1, &input_ctx);
	if (hashcmp(fill(20), sha1))
		die("pack is corrupted (SHA1 mismatch)");
	use(20);

	/* If input_fd is a file, we should have reached its end now. */
	if (fstat(input_fd, &st))
		die("cannot fstat packfile: %s", strerror(errno));
	if (S_ISREG(st.st_mode) &&
			lseek(input_fd, 0, SEEK_CUR) - input_len != st.st_size)
		die("pack has junk at the end");

	if (!nr_deltas)
		return;

	/* Sort deltas by base SHA1/offset for fast searching */
	qsort(deltas, nr_deltas, sizeof(struct delta_entry),
	      compare_delta_entry);

	/*
	 * Second pass:
	 * - for all non-delta objects, look if it is used as a base for
	 *   deltas;
	 * - if used as a base, uncompress the object and apply all deltas,
	 *   recursively checking if the resulting object is used as a base
	 *   for some more deltas.
	 */
	if (verbose)
		progress = start_progress("Resolving deltas", nr_deltas);
	for (i = 0; i < nr_objects; i++) {
		struct object_entry *obj = &objects[i];
		union delta_base base;
		int j, ref, ref_first, ref_last, ofs, ofs_first, ofs_last;
		struct base_data base_obj;

		if (obj->type == OBJ_REF_DELTA || obj->type == OBJ_OFS_DELTA)
			continue;
		hashcpy(base.sha1, obj->idx.sha1);
		ref = !find_delta_children(&base, &ref_first, &ref_last);
		memset(&base, 0, sizeof(base));
		base.offset = obj->idx.offset;
		ofs = !find_delta_children(&base, &ofs_first, &ofs_last);
		if (!ref && !ofs)
			continue;
		base_obj.data = get_data_from_pack(obj);
		base_obj.size = obj->size;
		base_obj.obj = obj;
		link_base_data(NULL, &base_obj);

		if (ref)
			for (j = ref_first; j <= ref_last; j++) {
				struct object_entry *child = objects + deltas[j].obj_no;
				if (child->real_type == OBJ_REF_DELTA)
					resolve_delta(child, &base_obj, obj->type);
			}
		if (ofs)
			for (j = ofs_first; j <= ofs_last; j++) {
				struct object_entry *child = objects + deltas[j].obj_no;
				if (child->real_type == OBJ_OFS_DELTA)
					resolve_delta(child, &base_obj, obj->type);
			}
		unlink_base_data(&base_obj);
		display_progress(progress, nr_resolved_deltas);
	}
}

static int write_compressed(int fd, void *in, unsigned int size, uint32_t *obj_crc)
{
	z_stream stream;
	unsigned long maxsize;
	void *out;

	memset(&stream, 0, sizeof(stream));
	deflateInit(&stream, zlib_compression_level);
	maxsize = deflateBound(&stream, size);
	out = xmalloc(maxsize);

	/* Compress it */
	stream.next_in = in;
	stream.avail_in = size;
	stream.next_out = out;
	stream.avail_out = maxsize;
	while (deflate(&stream, Z_FINISH) == Z_OK);
	deflateEnd(&stream);

	size = stream.total_out;
	write_or_die(fd, out, size);
	*obj_crc = crc32(*obj_crc, out, size);
	free(out);
	return size;
}

static struct object_entry *append_obj_to_pack(
			       const unsigned char *sha1, void *buf,
			       unsigned long size, enum object_type type)
{
	struct object_entry *obj = &objects[nr_objects++];
	unsigned char header[10];
	unsigned long s = size;
	int n = 0;
	unsigned char c = (type << 4) | (s & 15);
	s >>= 4;
	while (s) {
		header[n++] = c | 0x80;
		c = s & 0x7f;
		s >>= 7;
	}
	header[n++] = c;
	write_or_die(output_fd, header, n);
	obj[0].idx.crc32 = crc32(0, Z_NULL, 0);
	obj[0].idx.crc32 = crc32(obj[0].idx.crc32, header, n);
	obj[1].idx.offset = obj[0].idx.offset + n;
	obj[1].idx.offset += write_compressed(output_fd, buf, size, &obj[0].idx.crc32);
	hashcpy(obj->idx.sha1, sha1);
	return obj;
}

static int delta_pos_compare(const void *_a, const void *_b)
{
	struct delta_entry *a = *(struct delta_entry **)_a;
	struct delta_entry *b = *(struct delta_entry **)_b;
	return a->obj_no - b->obj_no;
}

static void fix_unresolved_deltas(int nr_unresolved)
{
	struct delta_entry **sorted_by_pos;
	int i, n = 0;

	/*
	 * Since many unresolved deltas may well be themselves base objects
	 * for more unresolved deltas, we really want to include the
	 * smallest number of base objects that would cover as much delta
	 * as possible by picking the
	 * trunc deltas first, allowing for other deltas to resolve without
	 * additional base objects.  Since most base objects are to be found
	 * before deltas depending on them, a good heuristic is to start
	 * resolving deltas in the same order as their position in the pack.
	 */
	sorted_by_pos = xmalloc(nr_unresolved * sizeof(*sorted_by_pos));
	for (i = 0; i < nr_deltas; i++) {
		if (objects[deltas[i].obj_no].real_type != OBJ_REF_DELTA)
			continue;
		sorted_by_pos[n++] = &deltas[i];
	}
	qsort(sorted_by_pos, n, sizeof(*sorted_by_pos), delta_pos_compare);

	for (i = 0; i < n; i++) {
		struct delta_entry *d = sorted_by_pos[i];
		enum object_type type;
		int j, first, last;
		struct base_data base_obj;

		if (objects[d->obj_no].real_type != OBJ_REF_DELTA)
			continue;
		base_obj.data = read_sha1_file(d->base.sha1, &type, &base_obj.size);
		if (!base_obj.data)
			continue;

		if (check_sha1_signature(d->base.sha1, base_obj.data,
				base_obj.size, typename(type)))
			die("local object %s is corrupt", sha1_to_hex(d->base.sha1));
		base_obj.obj = append_obj_to_pack(d->base.sha1, base_obj.data,
			base_obj.size, type);
		link_base_data(NULL, &base_obj);

		find_delta_children(&d->base, &first, &last);
		for (j = first; j <= last; j++) {
			struct object_entry *child = objects + deltas[j].obj_no;
			if (child->real_type == OBJ_REF_DELTA)
				resolve_delta(child, &base_obj, type);
		}

		unlink_base_data(&base_obj);
		display_progress(progress, nr_resolved_deltas);
	}
	free(sorted_by_pos);
}

static void final(const char *final_pack_name, const char *curr_pack_name,
		  const char *final_index_name, const char *curr_index_name,
		  const char *keep_name, const char *keep_msg,
		  unsigned char *sha1)
{
	const char *report = "pack";
	char name[PATH_MAX];
	int err;

	if (!from_stdin) {
		close(input_fd);
	} else {
		fsync_or_die(output_fd, curr_pack_name);
		err = close(output_fd);
		if (err)
			die("error while closing pack file: %s", strerror(errno));
		chmod(curr_pack_name, 0444);
	}

	if (keep_msg) {
		int keep_fd, keep_msg_len = strlen(keep_msg);
		if (!keep_name) {
			snprintf(name, sizeof(name), "%s/pack/pack-%s.keep",
				 get_object_directory(), sha1_to_hex(sha1));
			keep_name = name;
		}
		keep_fd = open(keep_name, O_RDWR|O_CREAT|O_EXCL, 0600);
		if (keep_fd < 0) {
			if (errno != EEXIST)
				die("cannot write keep file");
		} else {
			if (keep_msg_len > 0) {
				write_or_die(keep_fd, keep_msg, keep_msg_len);
				write_or_die(keep_fd, "\n", 1);
			}
			if (close(keep_fd) != 0)
				die("cannot write keep file");
			report = "keep";
		}
	}

	if (final_pack_name != curr_pack_name) {
		if (!final_pack_name) {
			snprintf(name, sizeof(name), "%s/pack/pack-%s.pack",
				 get_object_directory(), sha1_to_hex(sha1));
			final_pack_name = name;
		}
		if (move_temp_to_file(curr_pack_name, final_pack_name))
			die("cannot store pack file");
	}

	chmod(curr_index_name, 0444);
	if (final_index_name != curr_index_name) {
		if (!final_index_name) {
			snprintf(name, sizeof(name), "%s/pack/pack-%s.idx",
				 get_object_directory(), sha1_to_hex(sha1));
			final_index_name = name;
		}
		if (move_temp_to_file(curr_index_name, final_index_name))
			die("cannot store index file");
	}

	if (!from_stdin) {
		printf("%s\n", sha1_to_hex(sha1));
	} else {
		char buf[48];
		int len = snprintf(buf, sizeof(buf), "%s\t%s\n",
				   report, sha1_to_hex(sha1));
		write_or_die(1, buf, len);

		/*
		 * Let's just mimic git-unpack-objects here and write
		 * the last part of the input buffer to stdout.
		 */
		while (input_len) {
			err = xwrite(1, input_buffer + input_offset, input_len);
			if (err <= 0)
				break;
			input_len -= err;
			input_offset += err;
		}
	}
}

static int git_index_pack_config(const char *k, const char *v, void *cb)
{
	if (!strcmp(k, "pack.indexversion")) {
		pack_idx_default_version = git_config_int(k, v);
		if (pack_idx_default_version > 2)
			die("bad pack.indexversion=%"PRIu32,
				pack_idx_default_version);
		return 0;
	}
	return git_default_config(k, v, cb);
}

int main(int argc, char **argv)
{
	int i, fix_thin_pack = 0;
	char *curr_pack, *pack_name = NULL;
	char *curr_index, *index_name = NULL;
	const char *keep_name = NULL, *keep_msg = NULL;
	char *index_name_buf = NULL, *keep_name_buf = NULL;
	struct pack_idx_entry **idx_objects;
	unsigned char sha1[20];

	git_config(git_index_pack_config, NULL);

	for (i = 1; i < argc; i++) {
		char *arg = argv[i];

		if (*arg == '-') {
			if (!strcmp(arg, "--stdin")) {
				from_stdin = 1;
			} else if (!strcmp(arg, "--fix-thin")) {
				fix_thin_pack = 1;
			} else if (!strcmp(arg, "--strict")) {
				strict = 1;
			} else if (!strcmp(arg, "--keep")) {
				keep_msg = "";
			} else if (!prefixcmp(arg, "--keep=")) {
				keep_msg = arg + 7;
			} else if (!prefixcmp(arg, "--pack_header=")) {
				struct pack_header *hdr;
				char *c;

				hdr = (struct pack_header *)input_buffer;
				hdr->hdr_signature = htonl(PACK_SIGNATURE);
				hdr->hdr_version = htonl(strtoul(arg + 14, &c, 10));
				if (*c != ',')
					die("bad %s", arg);
				hdr->hdr_entries = htonl(strtoul(c + 1, &c, 10));
				if (*c)
					die("bad %s", arg);
				input_len = sizeof(*hdr);
			} else if (!strcmp(arg, "-v")) {
				verbose = 1;
			} else if (!strcmp(arg, "-o")) {
				if (index_name || (i+1) >= argc)
					usage(index_pack_usage);
				index_name = argv[++i];
			} else if (!prefixcmp(arg, "--index-version=")) {
				char *c;
				pack_idx_default_version = strtoul(arg + 16, &c, 10);
				if (pack_idx_default_version > 2)
					die("bad %s", arg);
				if (*c == ',')
					pack_idx_off32_limit = strtoul(c+1, &c, 0);
				if (*c || pack_idx_off32_limit & 0x80000000)
					die("bad %s", arg);
			} else
				usage(index_pack_usage);
			continue;
		}

		if (pack_name)
			usage(index_pack_usage);
		pack_name = arg;
	}

	if (!pack_name && !from_stdin)
		usage(index_pack_usage);
	if (fix_thin_pack && !from_stdin)
		die("--fix-thin cannot be used without --stdin");
	if (!index_name && pack_name) {
		int len = strlen(pack_name);
		if (!has_extension(pack_name, ".pack"))
			die("packfile name '%s' does not end with '.pack'",
			    pack_name);
		index_name_buf = xmalloc(len);
		memcpy(index_name_buf, pack_name, len - 5);
		strcpy(index_name_buf + len - 5, ".idx");
		index_name = index_name_buf;
	}
	if (keep_msg && !keep_name && pack_name) {
		int len = strlen(pack_name);
		if (!has_extension(pack_name, ".pack"))
			die("packfile name '%s' does not end with '.pack'",
			    pack_name);
		keep_name_buf = xmalloc(len);
		memcpy(keep_name_buf, pack_name, len - 5);
		strcpy(keep_name_buf + len - 5, ".keep");
		keep_name = keep_name_buf;
	}

	curr_pack = open_pack_file(pack_name);
	parse_pack_header();
	objects = xmalloc((nr_objects + 1) * sizeof(struct object_entry));
	deltas = xmalloc(nr_objects * sizeof(struct delta_entry));
	parse_pack_objects(sha1);
	if (nr_deltas == nr_resolved_deltas) {
		stop_progress(&progress);
		/* Flush remaining pack final 20-byte SHA1. */
		flush();
	} else {
		if (fix_thin_pack) {
			char msg[48];
			int nr_unresolved = nr_deltas - nr_resolved_deltas;
			int nr_objects_initial = nr_objects;
			if (nr_unresolved <= 0)
				die("confusion beyond insanity");
			objects = xrealloc(objects,
					   (nr_objects + nr_unresolved + 1)
					   * sizeof(*objects));
			fix_unresolved_deltas(nr_unresolved);
			sprintf(msg, "completed with %d local objects",
				nr_objects - nr_objects_initial);
			stop_progress_msg(&progress, msg);
			fixup_pack_header_footer(output_fd, sha1,
						 curr_pack, nr_objects);
		}
		if (nr_deltas != nr_resolved_deltas)
			die("pack has %d unresolved deltas",
			    nr_deltas - nr_resolved_deltas);
	}
	free(deltas);
	if (strict)
		check_objects();

	idx_objects = xmalloc((nr_objects) * sizeof(struct pack_idx_entry *));
	for (i = 0; i < nr_objects; i++)
		idx_objects[i] = &objects[i].idx;
	curr_index = write_idx_file(index_name, idx_objects, nr_objects, sha1);
	free(idx_objects);

	final(pack_name, curr_pack,
		index_name, curr_index,
		keep_name, keep_msg,
		sha1);
	free(objects);
	free(index_name_buf);
	free(keep_name_buf);
	if (pack_name == NULL)
		free(curr_pack);
	if (index_name == NULL)
		free(curr_index);

	return 0;
}

[-- Attachment #1.3: trace.3429 --]
[-- Type: text/plain, Size: 8992 bytes --]

==3429== Memcheck, a memory error detector.
==3429== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==3429== Using LibVEX rev 1854, a library for dynamic binary translation.
==3429== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==3429== Using valgrind-3.3.1-Debian, a dynamic binary instrumentation framework.
==3429== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==3429== For more details, rerun with: -v
==3429== 
==3429== My PID = 3429, parent PID = 3421.  Prog and args are:
==3429==    /home/madcoder/dev/scm/git/git-index-pack2
==3429==    --stdin
==3429==    -v
==3429==    --fix-thin
==3429==    --keep=fetch-pack 3421 on artemis
==3429==    --pack_header=2,263
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068AF9: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068B26: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068B86: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068B97: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068BD9: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Use of uninitialised value of size 8
==3429==    at 0x5068C2A: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068C8F: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068C98: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Use of uninitialised value of size 8
==3429==    at 0x5068E21: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Use of uninitialised value of size 8
==3429==    at 0x5068E31: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x50689AC: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5066933: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x5068E57: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068E64: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x5068E76: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x506B361: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x5068EAC: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Use of uninitialised value of size 8
==3429==    at 0x5069CDB: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506B7B0: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x5068EAC: (within /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x506728C: deflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x404BD9: main (index-pack.c:674)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x4035EE: get_base_data (index-pack.c:485)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Syscall param pread64(count) contains uninitialised byte(s)
==3429==    at 0x5621353: __pread_nocancel (in /usr/lib/debug/libpthread-2.7.so)
==3429==    by 0x4034C9: get_data_from_pack (index-pack.c:368)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Syscall param pread64(offset) contains uninitialised byte(s)
==3429==    at 0x5621353: __pread_nocancel (in /usr/lib/debug/libpthread-2.7.so)
==3429==    by 0x4034C9: get_data_from_pack (index-pack.c:368)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x4034D9: get_data_from_pack (index-pack.c:372)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x506D058: inflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x40352C: get_data_from_pack (index-pack.c:380)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x506C597: inflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x40352C: get_data_from_pack (index-pack.c:380)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== Conditional jump or move depends on uninitialised value(s)
==3429==    at 0x506C5E9: inflate (in /usr/lib/libz.so.1.2.3.3)
==3429==    by 0x40352C: get_data_from_pack (index-pack.c:380)
==3429==    by 0x4035FB: get_base_data (index-pack.c:496)
==3429==    by 0x403687: get_base_data (index-pack.c:486)
==3429==    by 0x403711: resolve_delta (index-pack.c:516)
==3429==    by 0x4038A8: resolve_delta (index-pack.c:543)
==3429==    by 0x404CF5: main (index-pack.c:762)
==3429== 
==3429== ERROR SUMMARY: 146 errors from 23 contexts (suppressed: 8 from 1)
==3429== malloc/free: in use at exit: 50,109 bytes in 17 blocks.
==3429== malloc/free: 1,223 allocs, 1,206 frees, 18,574,785 bytes allocated.
==3429== For counts of detected errors, rerun with: -v
==3429== searching for pointers to 17 not-freed blocks.
==3429== checked 618,664 bytes.
==3429== 
==3429== LEAK SUMMARY:
==3429==    definitely lost: 0 bytes in 0 blocks.
==3429==      possibly lost: 8,100 bytes in 1 blocks.
==3429==    still reachable: 42,009 bytes in 16 blocks.
==3429==         suppressed: 0 bytes in 0 blocks.
==3429== Rerun with --leak-check=full to see details of leaked memory.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply

* Re: Not so happy about build system
From: Miklos Vajna @ 2008-07-23 10:33 UTC (permalink / raw)
  To: Jan Engelhardt; +Cc: git
In-Reply-To: <alpine.LNX.1.10.0807222036230.23410@fbirervta.pbzchgretzou.qr>

[-- Attachment #1: Type: text/plain, Size: 300 bytes --]

On Wed, Jul 23, 2008 at 11:33:56AM +0200, Jan Engelhardt <jengelh@medozas.de> wrote:
> 	./configure --prefix=$HOME/rt --with-openssl=/opt/csw 
> 	--with-curl=/opt/csw CFLAGS="-O2 -I/opt/csw/include -L/opt/csw/lib 
> 	-R/opt/csw/lib"

Why do you put -L/opt/csw/lib to CFLAGS instead of LDFLAGS?

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply

* Re: regression in  92392b4
From: Pierre Habouzit @ 2008-07-23 10:22 UTC (permalink / raw)
  To: Björn Steinbrink; +Cc: spearce, Git ML, Junio C Hamano
In-Reply-To: <20080723101415.GA23769@atjola.homenet>

[-- Attachment #1: Type: text/plain, Size: 3258 bytes --]

On Wed, Jul 23, 2008 at 10:14:15AM +0000, Björn Steinbrink wrote:
> On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
> >   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> > did, since git in next cannot fetch on a regular basis for me. The
> > culprit seems to be commit  92392b4:
> > 
> >     ┌─(1:11)──<~/dev/scm/git 92392b4...>──
> >     └[artemis] git fetch
> >     remote: Counting objects: 461, done.
> >     remote: Compressing objects: 100% (141/141), done.
> >     remote: Total 263 (delta 227), reused 155 (delta 121)
> >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> >     fatal: Out of memory, malloc failed
> >     fatal: index-pack failed
> >     [2]    16674 abort (core dumped)  git fetch
> > 
> >     ┌─(1:12)──<~/dev/scm/git 92392b4...>──
> >     └[artemis] git checkout -m HEAD~1; make git-index-pack
> >     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
> >     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
> >     GIT_VERSION = 1.5.6.3.3.g03993
> > 	CC index-pack.o
> > 	LINK git-index-pack
> > 
> >     ┌─(1:12)──<~/dev/scm/git 03993e1...>──
> >     └[artemis] git fetch
> >     remote: Counting objects: 461, done.
> >     remote: Compressing objects: 100% (141/141), done.
> >     remote: Total 263 (delta 227), reused 155 (delta 121)
> >     Receiving objects: 100% (263/263), 95.55 KiB, done.
> >     Resolving deltas: 100% (227/227), completed with 153 local objects.
> >     From git://git.kernel.org/pub/scm/git/git
> >        5ba2c22..0868a30  html       -> origin/html
> >        2857e17..abeeabe  man        -> origin/man
> >        93310a4..95f8ebb  master     -> origin/master
> >        559998f..e8bf351  next       -> origin/next
> > 
> > You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> > broken, I've absolutely no clue about what happens.
> > 
> > All I can say is that at some point in get_data_from_pack, obj[1].idx
> > points to something that is *not* a sha so it's probably corrupted.
> > (from index-pack.c).
> 
> Here's how to reproduce:
> 
> #!/bin/bash
> 
> [ -d git-bug ] || \
> 	git clone git://git.kernel.org/pub/scm/git/git git-bug
> cd git-bug
> 
> git update-ref refs/remotes/origin/html 5ba2c22
> git update-ref refs/remotes/origin/man 2857e17
> git update-ref refs/remotes/origin/maint 2d9c572
> git update-ref refs/remotes/origin/master 93310a4
> git update-ref refs/remotes/origin/next 559998f
> git update-ref refs/remotes/origin/pu 010581c8
> 
> git reset --hard origin/master
> 
> sleep 1
> 
> git reflog expire --expire=0 --all
> 
> git repack -A -d -f --depth=10 --window=10
> git prune
> 
> git config core.deltaBaseCacheLimit 100
> 
> git fetch

  *THANKS* I was missing the "git config core.deltaBaseCacheLimit". I
did the same as you but it wasn't failing here. FWIW I don't have
deltaBaseCacheLimit set in my config, but oh well.
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply

* [PATCH 2/2] git-checkout: improve error messages, detect ambiguities.
From: Pierre Habouzit @ 2008-07-23 10:15 UTC (permalink / raw)
  To: git; +Cc: gitster, Pierre Habouzit
In-Reply-To: <1216808133-31919-2-git-send-email-madcoder@debian.org>

The patch is twofold: it moves the option consistency checks just under
the parse_options call so that it doesn't get in the way of the tree
reference vs. pathspecs desambiguation.

The other part rewrites the way to understand arguments so that when
git-checkout fails it does with an understandable message. Compared to the
previous behavior we now have:

  - a better error message when doing:
        git checkout <blob reference> --
    now complains about the reference not pointing to a tree, instead of
    things like:
        error: pathspec <blob reference> did not match any file(s) known to git.
        error: pathspec '--' did not match any file(s) known to git.

    you what it spitted before.

  - a better error message when doing:
        git checkout <path> --
    It now complains about <path> not being a reference instead of the
    completely obscure:
        error: pathspec '--' did not match any file(s) known to git.

  - an error when -- wasn't used and the first argument is ambiguous.

Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
 builtin-checkout.c            |   71 +++++++++++++++++++++++++++++++----------
 t/t2010-checkout-ambiguous.sh |   39 ++++++++++++++++++++++
 2 files changed, 93 insertions(+), 17 deletions(-)
 create mode 100755 t/t2010-checkout-ambiguous.sh

diff --git a/builtin-checkout.c b/builtin-checkout.c
index 9cadf9c..e9ae834 100644
--- a/builtin-checkout.c
+++ b/builtin-checkout.c
@@ -430,6 +430,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		OPT_BOOLEAN('m', NULL, &opts.merge, "merge"),
 		OPT_END(),
 	};
+	int has_dash_dash;
 
 	memset(&opts, 0, sizeof(opts));
 	memset(&new, 0, sizeof(new));
@@ -440,11 +441,52 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, options, checkout_usage,
 			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (!opts.new_branch && (opts.track != git_branch_track))
+		die("git checkout: --track and --no-track require -b");
+
+	if (opts.force && opts.merge)
+		die("git checkout: -f and -m are incompatible");
+
+	/*
+	 * case 1: git checkout <ref> -- [<paths>]
+	 *
+	 *   <ref> must be a valid tree, everything after the '--' must be
+	 *   a path.
+	 *
+	 * case 2: git checkout -- [<paths>]
+	 *
+	 *   everything after the '--' must be paths.
+	 *
+	 * case 3: git checkout <something> [<paths>]
+	 *
+	 *   <something> shall not be ambiguous.
+	 *   - If it's *only* a reference, treat it like case (1).
+	 *   - If it's only a path, treat it like case (2).
+	 *   - else: fail.
+	 *
+	 */
 	if (argc) {
+		if (!strcmp(argv[0], "--")) {       /* case (2) */
+			argv++;
+			argc--;
+			goto no_reference;
+		}
+		
 		arg = argv[0];
-		if (get_sha1(arg, rev))
-			;
-		else if ((new.commit = lookup_commit_reference_gently(rev, 1))) {
+		has_dash_dash = argc > 1 && !strcmp(argv[1], "--");
+
+		if (get_sha1(arg, rev)) {
+			if (has_dash_dash)          /* case (1) */
+				die("invalid reference: %s", arg);
+			goto no_reference;          /* case (3 -> 2) */
+		}
+
+		/* we can't end up being in (2) anymore, eat the argument */
+		argv++;
+		argc--;
+
+		if ((new.commit = lookup_commit_reference_gently(rev, 1))) {
 			new.name = arg;
 			setup_branch_path(&new);
 			if (resolve_ref(new.path, rev, 1, NULL))
@@ -453,25 +495,20 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 				new.path = NULL;
 			parse_commit(new.commit);
 			source_tree = new.commit->tree;
-			argv++;
-			argc--;
-		} else if ((source_tree = parse_tree_indirect(rev))) {
+		} else
+			source_tree = parse_tree_indirect(rev);
+
+		if (!source_tree)                   /* case (1): want a tree */
+			die("reference is not a tree: %s", arg);
+		if (!has_dash_dash)                 /* case (3 -> 1) */
+			verify_non_filename(NULL, arg);
+		else {
 			argv++;
 			argc--;
 		}
 	}
 
-	if (argc && !strcmp(argv[0], "--")) {
-		argv++;
-		argc--;
-	}
-
-	if (!opts.new_branch && (opts.track != git_branch_track))
-		die("git checkout: --track and --no-track require -b");
-
-	if (opts.force && opts.merge)
-		die("git checkout: -f and -m are incompatible");
-
+no_reference:
 	if (argc) {
 		const char **pathspec = get_pathspec(prefix, argv);
 
diff --git a/t/t2010-checkout-ambiguous.sh b/t/t2010-checkout-ambiguous.sh
new file mode 100755
index 0000000..50d1f43
--- /dev/null
+++ b/t/t2010-checkout-ambiguous.sh
@@ -0,0 +1,39 @@
+#!/bin/sh
+
+test_description='checkout and pathspecs/refspecs ambiguities'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	echo hello >world &&
+	echo hello >all &&
+	git add all world &&
+	git commit -m initial &&
+	git branch world
+'
+
+test_expect_success 'reference must be a tree' '
+	test_must_fail git checkout $(git hash-object ./all) --
+'
+
+test_expect_success 'branch switching' '
+	test "refs/heads/master" = "$(git symbolic-ref HEAD)" &&
+	git checkout world -- &&
+	test "refs/heads/world" = "$(git symbolic-ref HEAD)"
+'
+
+test_expect_success 'checkout world from the index' '
+	echo bye > world &&
+	git checkout -- world &&
+	git diff --exit-code --quiet
+'
+
+test_expect_success 'non ambiguous call' '
+	git checkout all
+'
+
+test_expect_success 'ambiguous call' '
+	test_must_fail git checkout world
+'
+
+test_done
-- 
1.6.0.rc0.155.g8e50b

^ permalink raw reply related

* [PATCH 1/2] git-checkout: fix command line parsing.
From: Pierre Habouzit @ 2008-07-23 10:15 UTC (permalink / raw)
  To: git; +Cc: gitster, Pierre Habouzit
In-Reply-To: <1216808133-31919-1-git-send-email-madcoder@debian.org>

This fixes an issue when you use:

    $ git checkout -- <path1> [<paths>...]

and that <path1> can also be understood as a reference. git-checkout
mistakenly understands this as the same as:

    $ git checkout <path1> -- [<paths>...]

because parse-options was eating the '--' and the argument parser thought
he was parsing:

    $ git checkout <path1> [<paths>...]

Where there indeed is an ambiguity

Signed-off-by: Pierre Habouzit <madcoder@debian.org>
---
 builtin-checkout.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/builtin-checkout.c b/builtin-checkout.c
index fbd5105..9cadf9c 100644
--- a/builtin-checkout.c
+++ b/builtin-checkout.c
@@ -438,7 +438,8 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 
 	opts.track = git_branch_track;
 
-	argc = parse_options(argc, argv, options, checkout_usage, 0);
+	argc = parse_options(argc, argv, options, checkout_usage,
+			     PARSE_OPT_KEEP_DASHDASH);
 	if (argc) {
 		arg = argv[0];
 		if (get_sha1(arg, rev))
-- 
1.6.0.rc0.155.g8e50b

^ permalink raw reply related

* Resubmit after a night of sleep
From: Pierre Habouzit @ 2008-07-23 10:15 UTC (permalink / raw)
  To: git; +Cc: gitster
In-Reply-To: <1216774940-4955-1-git-send-email-madcoder@debian.org>

Okay, so after a (too short) night sleep, here is a way better series.
The first patch corrects a real bug, and should be applied to maint.
It's a one liner, the comment is self explanatory.

The second one is a twofold UI improvement wrt error messages that
git-checkout (if we abstract the issues from patch 1) already caught as
errors but with dreadful errors. And with respect to the case where
there is no '--' and that there is an ambiguity. We used to always favor
the interpretation where the first argument is understood as a
path (which is really horrible because it's the destructive choice: at
least if we favored the reference, user would never loose local
modifications), it now barfs.

I reckon I've not checked what git-checkout did when it wasn't a
builtin, so I don't know if the second part of this UI improvement comes
as fixing a regression (in which case the patch could be considered for
maint) or if it's an ambiguity that was here like forever, in which case
it's less urgent, but should IMHO be addressed because git-checkout is
porcelain: see http://gist.github.com/1402 for the user issue that made
me discover that, you'll see we want a desambiguation one way or the
other.

^ permalink raw reply

* Re: regression in  92392b4
From: Björn Steinbrink @ 2008-07-23 10:14 UTC (permalink / raw)
  To: Pierre Habouzit, spearce, Git ML, Junio C Hamano
In-Reply-To: <20080722231745.GD11831@artemis.madism.org>

On 2008.07.23 01:17:45 +0200, Pierre Habouzit wrote:
>   Hi, here is a manual painful down-secting (opposed to a bisect ;P) I
> did, since git in next cannot fetch on a regular basis for me. The
> culprit seems to be commit  92392b4:
> 
>     ┌─(1:11)──<~/dev/scm/git 92392b4...>──
>     └[artemis] git fetch
>     remote: Counting objects: 461, done.
>     remote: Compressing objects: 100% (141/141), done.
>     remote: Total 263 (delta 227), reused 155 (delta 121)
>     Receiving objects: 100% (263/263), 95.55 KiB, done.
>     fatal: Out of memory, malloc failed
>     fatal: index-pack failed
>     [2]    16674 abort (core dumped)  git fetch
> 
>     ┌─(1:12)──<~/dev/scm/git 92392b4...>──
>     └[artemis] git checkout -m HEAD~1; make git-index-pack
>     Previous HEAD position was 92392b4... index-pack: Honor core.deltaBaseCacheLimit when resolving deltas
>     HEAD is now at 03993e1... index-pack: Track the object_entry that creates each base_data
>     GIT_VERSION = 1.5.6.3.3.g03993
> 	CC index-pack.o
> 	LINK git-index-pack
> 
>     ┌─(1:12)──<~/dev/scm/git 03993e1...>──
>     └[artemis] git fetch
>     remote: Counting objects: 461, done.
>     remote: Compressing objects: 100% (141/141), done.
>     remote: Total 263 (delta 227), reused 155 (delta 121)
>     Receiving objects: 100% (263/263), 95.55 KiB, done.
>     Resolving deltas: 100% (227/227), completed with 153 local objects.
>     From git://git.kernel.org/pub/scm/git/git
>        5ba2c22..0868a30  html       -> origin/html
>        2857e17..abeeabe  man        -> origin/man
>        93310a4..95f8ebb  master     -> origin/master
>        559998f..e8bf351  next       -> origin/next
> 
> You can see the commit sha's in the prompt. 03993e1 is fine, 92392b4 is
> broken, I've absolutely no clue about what happens.
> 
> All I can say is that at some point in get_data_from_pack, obj[1].idx
> points to something that is *not* a sha so it's probably corrupted.
> (from index-pack.c).

Here's how to reproduce:

#!/bin/bash

[ -d git-bug ] || \
	git clone git://git.kernel.org/pub/scm/git/git git-bug
cd git-bug

git update-ref refs/remotes/origin/html 5ba2c22
git update-ref refs/remotes/origin/man 2857e17
git update-ref refs/remotes/origin/maint 2d9c572
git update-ref refs/remotes/origin/master 93310a4
git update-ref refs/remotes/origin/next 559998f
git update-ref refs/remotes/origin/pu 010581c8

git reset --hard origin/master

sleep 1

git reflog expire --expire=0 --all

git repack -A -d -f --depth=10 --window=10
git prune

git config core.deltaBaseCacheLimit 100

git fetch



The values for html, man, master and next are taken from Pierre's
output, those for maint and pu are from a repo that works for
reproducing the bug, just in case that future output of those branches
manage to break the reproducability.

Might be, that any fetch will fail if deltaBaseCacheLimit is low enough,
but I'm too lazy to test that as well, now that I realized it. My head
seems not to be working correctly anyway, can't even manage to get a
core dump... Stupid flu...

HTH
Björn

^ permalink raw reply

* Re: [RESEND PATCH] git-checkout: fix argument parsing to detect ambiguous arguments.
From: Johannes Schindelin @ 2008-07-23 10:10 UTC (permalink / raw)
  To: Pierre Habouzit; +Cc: git, gitster
In-Reply-To: <20080723012751.GI11831@artemis.madism.org>

Hi,

On Wed, 23 Jul 2008, Pierre Habouzit wrote:

> Note that it also fix a bug, git checkout -- <path> would be badly

s/fix/fixes/

> understood as git checkout <branch> if <path> is ambiguous with a branch.
> 
> Testcases included.
> 
> Signed-off-by: Pierre Habouzit <madcoder@debian.org>

Instead of the "Testcases included", I would have expected something like

	When calling "git checkout <name>" and <name> is both a path and a 
	branch, Git would silently assume that you meant the branch.  
	Worse, "git checkout -- <name>" would behave the same.

probably even as first paragraph.

>   This is a resend with proper patches that made me realize that my

s/patches/tests/

> previous patch had silly mistakes and wasn't testing thigns properly.
> 
>  builtin-checkout.c            |   23 +++++++++++++++++------
>  t/t2010-checkout-ambiguous.sh |   35 +++++++++++++++++++++++++++++++++++
>  2 files changed, 52 insertions(+), 6 deletions(-)
>  create mode 100755 t/t2010-checkout-ambiguous.sh
> 
> diff --git a/builtin-checkout.c b/builtin-checkout.c
> index fbd5105..97321e6 100644
> --- a/builtin-checkout.c
> +++ b/builtin-checkout.c
> @@ -438,12 +438,17 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>  
>  	opts.track = git_branch_track;
>  
> -	argc = parse_options(argc, argv, options, checkout_usage, 0);
> -	if (argc) {
> +	argc = parse_options(argc, argv, options, checkout_usage,
> +			     PARSE_OPT_KEEP_DASHDASH);
> +
> +	if (argc && strcmp(argv[0], "--")) {
> +		int may_be_ambiguous = argc == 1 || strcmp(argv[1], "--");
> +
>  		arg = argv[0];
> -		if (get_sha1(arg, rev))
> -			;
> -		else if ((new.commit = lookup_commit_reference_gently(rev, 1))) {
> +		if (get_sha1(arg, rev)) {
> +			if (may_be_ambiguous)
> +				verify_filename(NULL, arg);

Still you allow "git checkout <path> --" to succeed, right?  It should 
not.

IMHO you need "must_be_branch" and "must_not_be_path" instead of 
"may_be_ambiguous".

I.e. something like

		int must_be_branch = argc > 1 && !strcmp(argv[1], "--");
		int must_not_be_path = argc > 1 && strcmp(argv[1], "--");

		arg = argv[0];
		if (get_sha1(arg, rev)) {
			if (must_be_branch)
				die ("Not a branch: '%s'", arg);
		}
		else {
			if (must_not_be_path)
				verify_non_filename(NULL, arg);
			if ((new.commit = lookup_commit_reference_gently(rev, 1))) {
				[...]

And maybe you want to add a test for "git checkout <path> --" to fail, 
too.

Ciao,
Dscho

^ permalink raw reply

* Re: [RFC] Git User's Survey 2008
From: Johannes Schindelin @ 2008-07-23  9:53 UTC (permalink / raw)
  To: Jakub Narebski; +Cc: git, Stephan Beyer
In-Reply-To: <200807230325.04184.jnareb@gmail.com>

Hi,

On Wed, 23 Jul 2008, Jakub Narebski wrote:

> First there is a question about the form of survey. Should we use web
> based survey, as the survey before (http://www.survey.net.nz), sending
> emails with link to this survey, or perhaps do email based survey,
> with email Reply-To: address put for this survey alone?

Some people prefer to stay anonymous, so I think email is out.

>    04. Which programming languages you are proficient with?
>        (The choices include programming languages used by git)
>        (zero or more: multiple choice)
>      - C, shell, Perl, Python, Tcl/Tk
>      + (should we include other languages, like C++, Java, PHP,
>         Ruby,...?)

Yes, I think this should be a long list.

>    07. What helped you most in learning to use it?
>        (free form question)

Is it possible to have multiple choice, with "other" (free-form)?  Then 
I'd suggest:

	Colleague/Instructor, User Manual, Manpages, Tutorials, Tutorials 
	(elsewhere; not in git.git), Mailing list, IRC, Git Wiki, Other.

>    08. What did you find hardest in learning Git?
>        What did you find harderst in using Git?

s/harderst/hardest.

>        (free form question)

Again, I'd suggest a multiple choice + Other:

	The amount of commands, the amount of options, the index (AKA 
	staging), branching, user interface, bugs, Other.

> Other SCMs (shortened compared with 2007 survey)
> 
>    10. What other SCM did or do you use?
>        (zero or more: multiple choice)
>      - CVS, Subversion, GNU Arch or arch clone (ArX, tla, ...),
>        Bazaar-NG, Darcs, Mercurial, Monotone, SVK, AccuRev, Perforce,
>        BitKeeper, ClearCase, MS Visual Source Safe, MS Visual Studio
>        Team System, custom, other(*)

PVCS seems to be pretty popular, too.

>    11. Why did you choose Git? (if you use Git)
>        What do you like about using Git?
>        (free form, not to be tabulated)

Again, to avoid hassles with free-form:

	Mandatory: work, mandatory: open source project I am participating 
	in, speed, scalability, It's What Linus Uses, Other.

>    12. Why did you choose other SCMs? (if you use other SCMs)
>        What do you like about using other SCMs?
>        Note: please write name of SCMs you are talking about.
>        (free form, not to be tabulated).

Again:

	ease-of use, simplicity, existing project uses it, I Do Not Like 
	Linus, Other

>    15. What operating system do you use Git on?
>        (one or more: multiple choice, as one can use more than one OS)
>      - Linux, *BSD (FreeBSD, OpenBSD, etc.), MS Windows/Cygwin,
>        MS Windows/msysGit, MacOS X, other UNIX, other

You should include "Dunno", which gets automatically mapped to "MS 
Windows/msysGit" ;-)

>    19. How do you publish/propagate your changes?
>        (zero or more: multiple choice)
>      - push, pull request, format-patch + email, bundle, other

git svn

You might laugh, but it is a sad fact that some guy promotes "Using Git 
with Google Code" by using git-svn to drive their crappy Subversion.

>    22. How does Git compare to other SCM tools you have used?
>      - worse/equal (or comparable)/better
>    23. What would you most like to see improved about Git?
>        (features, bugs, plug-ins, documentation, ...)

Maybe here should be another question "What are the most useful features 
of Git?" but maybe that is covered by earlier questions.

>    24. If you want to see Git more widely used, what do you
>        think we could do to make this happen?
>      + Is this question necessary/useful?  Do we need wider adoption?

I agree with Junio: this is not so interesting for us; we are no company, 
and we have no sales department who could wank of on these answers.

>    27. Which of the following features do you use?
>        (zero or more: multiple choice)
>      - git-gui or other commit tool, gitk or other history viewer, patch
>        management interface (e.g. StGIT), bundle, eol conversion,

For our Windows friends, we should add " (crlf)" to the last item.

>    42. Do you find traffic levels on Git mailing list OK.
>     -  yes/no? (optional)

/too low?  *ducksandrunsforcover*

>    44. If yes, do you find IRC channel useful?
>     -  yes/no (optional)

/somewhat.  Even if I would be the only one choosing that option.

>    45. Did you have problems getting GIT help on mailing list or
>        on IRC channel? What were it? What could be improved?
>        (free form)

Yeah, I know who will answer to that, and what... "yaddayadda very 
unfriendly yaddayadda especially that Johannes guy yaddayadda" (you know 
who you are)... *lol*

Thanks Jakub, I think that your effort is very useful.

Ciao,
Dscho

^ permalink raw reply

* Not so happy about build system
From: Jan Engelhardt @ 2008-07-23  9:33 UTC (permalink / raw)
  To: git

Hi,


I just tried git 1.5.6.4 on a Solaris 10 box. Building mostly went 
smooth, but a few unhappy edges turned up:

~/git-1.5.6.4>gmake MAKE=gmake
    LINK git-daemon
ld: fatal: library -lcurl: not found
ld: fatal: library -lcrypto: not found
ld: fatal: File processing errors. No output written to git-daemon
collect2: ld returned 1 exit status
gmake: *** [git-daemon] Error 1

Of course curl is available, it's just somewhere else. And that is where 
the fun begins that comes with plain Makefiles.

	"What's the variable I ought to set?"

Yes, usually it is LDFLAGS, and this is also true for git. But not 
everyone necessarily adheres to that, and looking through the Makefile 
is not fun. That also reminds me of CFLAGS, which about every Makefile 
write happily sets to

	CFLAGS = -g -O2 -Wall -Wmy-fancy-flags

which would give no way to keep -Wmy-fancy-flags while also overriding 
it. Autotools solved this whereby a developer puts -Wmy-fancy-flags that 
shall always be present into AM_CFLAGS instead and only puts the really 
freely choosable flags into CFLAGS. Of course I could list all the 
developers flags that are needed/wanted (-Wmy-fancy-flags) like

	make CFLAGS="-Wdevelopers-fancy-flags -O3 -g0"

but one would have to look them up first and and etc.
Ok, enough Makefile ramblings, as there seems to be a configure script 
lurking. What's the Makefile good for then if configure will create it 
anyway?

Trying my luck with configure, I call

	./configure --prefix=$HOME/rt --with-openssl=/opt/csw 
	--with-curl=/opt/csw

but what I get is that it obviously did not find neither openssl nor 
curl, though they do exist in /opt/csw/include and /opt/csw/lib:

	checking for SHA1_Init in -lcrypto... no
	checking for SHA1_Init in -lssl... no
	checking for curl_global_init in -lcurl... no
	checking for XML_ParserCreate in -lexpat... no

(someone please fix that)

	./configure --prefix=$HOME/rt --with-openssl=/opt/csw 
	--with-curl=/opt/csw CFLAGS="-O2 -I/opt/csw/include -L/opt/csw/lib 
	-R/opt/csw/lib"
# using /usr/ccs/bin/ld, not GNU ld

That finally made it succeed in finding SHA1_Init. Then however,

	~/git-1.5.6.4>make
	make: Fatal error in reader: Makefile, line 158: Unexpected end 
	of line seen

So configure did not create a POSIX-compatible Makefile. Ok, 
I had gmake, so the story is successfully done here :)

^ permalink raw reply

* Re: git-svn does not seems to work with crlf convertion enabled.
From: Johannes Schindelin @ 2008-07-23  9:18 UTC (permalink / raw)
  To: Alexander Litvinov; +Cc: git
In-Reply-To: <200807231544.23472.litvinov2004@gmail.com>

Hi,

On Wed, 23 Jul 2008, Alexander Litvinov wrote:

> In short: I can't clone svn repo into git when crlf convertion is 
> activated.

This is a known issue, but since nobody with that itch seems to care 
enough to fix it, I doubt it will ever be fixed.

Ciao,
Dscho

^ permalink raw reply

* Re: [PATCH v2] git daemon: avoid waking up too often
From: Johannes Schindelin @ 2008-07-23  9:17 UTC (permalink / raw)
  To: Johannes Sixt; +Cc: git, Junio C Hamano
In-Reply-To: <4886D503.7030106@viscovery.net>

Hi,

On Wed, 23 Jul 2008, Johannes Sixt wrote:

> Johannes Schindelin schrieb:
> > To avoid waking up unnecessarily, a pipe is set up that is only ever
> > written to by child_handler(), when a child disconnects, as suggested
> > per Junio.
> > 
> > This avoids waking up the main process every second to see if a child
> > was disconnected.
> 
> This makes porting this beast to Windows practically impossible because we
> cannot have a poll() implementation that waits both on a listening socket
> and a pipe. :-(

Umm.  We also do not have signal handlers, do we?  AFAICT you wanted to 
simulate _this_ program's fork() with a start_async(), right?  All you 
have to do is to refactor the code with the pipe to do the update 
(properly guarded by a mutex, which is called "CriticalSection" in Windows 
speak, since it is impossible for Microsoft to accept common conventions).

The biggest part to get this beast running on Windows is to move it to the 
start_async() method, not above-mentioned refactoring.

Ciao,
Dscho

^ permalink raw reply

* Re: [PATCH] rebase -i: When an 'edit' stops, mention the commit
From: Johannes Schindelin @ 2008-07-23  9:10 UTC (permalink / raw)
  To: Johannes Sixt; +Cc: Junio C Hamano, Git Mailing List
In-Reply-To: <4886E1DB.7020308@viscovery.net>

Hi,

On Wed, 23 Jul 2008, Johannes Sixt wrote:

> In a rebase session where more than one commit is to be 'edit'ed, and 
> the user spends considerable time to 'edit' a commit, it is easy to 
> forget what one wanted to 'edit' at the individual commits. It would be 
> helpful to see at which commit the rebase stopped.

"... So mention it."

ACK.

Ciao,
Dscho

^ permalink raw reply

* Re: RFC: git rebase -i and root commits
From: Johannes Schindelin @ 2008-07-23  9:05 UTC (permalink / raw)
  To: Stephan Beyer; +Cc: Eric Raible, git
In-Reply-To: <20080723020232.GF5904@leksak.fem-net>

Hi,

On Wed, 23 Jul 2008, Stephan Beyer wrote:

> Eric Raible wrote:
> > My normal workflow is to create a .gitignore in my initial commit.
> > 
> > When I later realize that I've forgotten something from that file
> > I could of course just commit the changes, but I'd rather use "git rebase -i"
> > in the normal way to make myself appear smarter than I am.
> > Especially since this realization usually comes early on
> > (and certainly before publishing).
> > 
> > But rebase can't go all the way to a root ("fatal: Needed a single revision").
> 
> I think this has been fixed recently.

No, it has not.

What has been fixed is that you can cherry-pick a root commit now, not 
rebase onto nothing.

While I am somewhat sympathetic with Eric's use case, I am not sure that 
we should support it with rebase -i.

Ciao,
Dscho

^ permalink raw reply

* Re: [PATCH] Respect crlf attribute in "git add" even if core.autocrlf has not been set
From: Johannes Schindelin @ 2008-07-23  9:02 UTC (permalink / raw)
  To: Steffen Prohaska; +Cc: Junio C Hamano, git, Dmitry Potapov
In-Reply-To: <719E03C0-E8C3-4C35-AE9C-9BD5A7BCDF03@zib.de>

Hi,

On Wed, 23 Jul 2008, Steffen Prohaska wrote:

> On Jul 23, 2008, at 3:31 AM, Johannes Schindelin wrote:
> 
> >When a file's crlf attribute is explicitely set, it does not make sense 
> >to ignore it when adding the file, just because the config variable 
> >core.autocrlf has not been set.
> 
> Your patch is not about core.autocrlf unset, but about 
> core.autocrlf=false.

No, no, no!

It is about autocrlf _unset_, i.e. when the user was _not_ explicit!

> >The alternative would be risking to get CR/LF files into the repository 
> >just because one user forgot to set core.autocrlf.
> 
> Git could help the user *and* the maintainer of the repository if we
> chose core.autocrlf=input as the default on Unix.

NO!

The machinery behind core.autocrlf _does_ slow down Git, even if only by a 
few percent.  But with patch queues as maintainers of busy projects 
encounter frequently, this makes a difference.

So no, I am _totally_ opposed to punishing Unix users for Windows' faults.  
Totally.

> >diff --git a/convert.c b/convert.c
> >index 78efed8..d038d2f 100644
> >--- a/convert.c
> >+++ b/convert.c
> >@@ -126,7 +126,7 @@ static int crlf_to_git(const char *path, const char *src,
> >size_t len,
> > struct text_stat stats;
> > char *dst;
> >
> >-	if ((action == CRLF_BINARY) || !auto_crlf || !len)
> >+	if ((action == CRLF_BINARY) || (!auto_crlf && action < 0) || !len)
> 
> I think we should strictly follow the documentation, so this should read
> 
> +       if ((action == CRLF_BINARY) || (!auto_crlf && action != CRLF_INPUT) ||
> !len)

NO!

Your version would break action == CRLF_TEXT.

> >diff --git a/t/t0020-crlf.sh b/t/t0020-crlf.sh
> >index 1be7446..0bb3e6f 100755
> >--- a/t/t0020-crlf.sh
> >+++ b/t/t0020-crlf.sh
> >@@ -436,4 +436,14 @@ test_expect_success 'invalid .gitattributes (must not
> >crash)' '
> >
> >'
> >
> >+test_expect_success 'attribute crlf is heeded even without core.autocrlf' '
> >+
> >+	echo "allcrlf crlf=input" > .gitattributes &&
> >+	git config --unset core.autocrlf &&
> 
> You should set core.autocrlf explicitly to false:
> 
>   git config core.autocrlf false

But this is _not_ what I want!  I want it to be _unset_.  Yes, at the 
moment, auto_crlf is initialized to 0.  But I do not _care_ what it is 
set.  I want this test to succeed when the user did _not_ set autocrlf.

> Otherwise the test would pick up the user's global default.

It will not.  We went to great pains not to pick up global defaults in the 
test suite.

> ... and we should verify that we only treat crlf=input specially, but not
> "crlf".

Umm... Why?  Why should a file marked "crlf=true" be allowed to be checked 
in _with_ CRs?

Ciao,
Dscho

^ permalink raw reply

* git-svn does not seems to work with crlf convertion enabled.
From: Alexander Litvinov @ 2008-07-23  8:44 UTC (permalink / raw)
  To: git

Hello list.

In short: I can't clone svn repo into git when crlf convertion is activated.

Long story.
I use latest git: 
$ git version
git version 1.5.6.4

For a long period of time I use git at work. Main repo is svn-powered and I 
use git-svn for linking git and svn. The project itself is a windows cpp 
project. I use git under Linux machine (Debian etch with manually backported 
git from sid) and work with linux-hosted project thru samba. From the begin I 
did not enable crlf convertion and broke crlf notation in files one by one 
during my commits. My co-workers does not like this and finally I decide to 
try to use autocrlf feature of git. So I take a copy of my git repo and 
convert all text files to unix LF line endings:

git filter-branch --tree-filter "find -type f \( -iname '*.h' -or \
-iname '*.cpp' -or -iname '*.vcproj' -or -iname '*.sln' -or \
-iname '*.h.tmpl' -or -iname '*.bat' -or -iname '*.mp' -or \
-iname '*.txt' -or -iname '*.nsi' -or -iname '*.def' -or \
-iname '*.rc' -or -iname '*.ini' -or -iname '*.inf' -or \
-iname '*.skin' -or -iname '*.c' -or -iname '*.dsp' \
-or -iname '*.dsw' \) -print0 | xargs -r0 dos2unix" \
`git branch -a | sed 's/^..//'`

It finished succefully. After fish I have added  .git/info/attributes like 
this:
*               -crlf
*.h             crlf
*.c             crlf
*.cpp           crlf
and so on...
and add set core.autocrlf to true and safecrlf to false. Also I cleared all 
git-svn's caches:
rm -rf .git/svn

As I understand I got pure repo that is capable to work with crlf convertion. 
Lets update it (on branch forked from trunk): git svn rebase
<.. some long list of revs during migration to new git-svn layout..>
Done rebuilding .git/svn/trunk/.rev_map.f1f59411-8b2e-0410-9ee3-aa470c928bf2
        M       FindHistory.cpp
Incomplete data: Delta source ended unexpectedly at /tmp/g/bin/git-svn line 
3856

Oops ! Whats this ? I am not able to update. I can update other branches but 
not trunk.

So I have to try my old original repo without crlf convertion enabled. It was 
updated succeffuly, I cant show log it was lost and I was not able to 
reproduce it.



Is there any way to fix this problem ?

P.S. I can't even clone that svn repo from scratch with crlf convertion 
enabled.

^ permalink raw reply

* Re: [PATCH/RFC] git add: do not add files from a submodule
From: Pierre Habouzit @ 2008-07-23  8:13 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git
In-Reply-To: <7vhcahgl2j.fsf@gitster.siamese.dyndns.org>

[-- Attachment #1: Type: text/plain, Size: 2366 bytes --]

On Wed, Jul 23, 2008 at 06:40:20AM +0000, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > Do you plan to apply the split-up builtin-add enhancments you did a few 
> > nights ago,...
> 
> I have a few updates to that one, I'll be sending them out shortly.
> 
> Switching branches between revs that have and do not have submodule at a
> given path has always been broken.  It is not even a "known breakage",
> which is a word used for something that has a sensible design already is
> worked out but the implementation does not do so.
> 
> If we started the process of diagnosing and fixing these issues earlier,
> and had plausible code to address the issue already in 'next' before the
> current -rc cycle started, the topic would have been an obvious candidate
> for the coming release and I'd further say it would be even worth delaying
> the release for a few weeks if it takes more time.  But I have to say it
> is too late for 1.6.0 now if we are just noticing and starting the
> discussion.

  Well given that we now use submodules at work, and that git is
nowadays somewhere in the top 5 of my most consciously (as opposed to
the compiler that I rarely call by hand) used software suites (among my
editor, my MUA, my shell and my tiling WM), I'm very much interested in
tackling some things about what is (not) done with submodules yet.

  I sent a mail some time ago about those issues, in short words, those
concern git-checkout, git-reset, git-fetch and git-push for starters.
I'm not sure I can help with the code a lot, but I can for sure provide
a comprehensive list of things I would like the porcelain to do with
submodules if that helps.

> This comment goes to the issue Pierre raised last night as well.

  You mean the git checkout issue ? Because the issue is twofold, one
there is the missing warnings that users can shoot themselves in the
foot unnoticed, and there is the fact that "git checkout -- $foo" does
not what it should when "$foo" is a valid revision (and *that* is a
really really bad bug, that if I'm correct changing parse_options flags
to KEEP_DASHDASH fixes alright).

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply

* HP-UX issues (WAS: Re: [RFC] Git User's Survey 2008)
From: Miklos Vajna @ 2008-07-23  7:47 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jakub Narebski, git, Stephan Beyer
In-Reply-To: <7vsku1gqny.fsf@gitster.siamese.dyndns.org>

[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]

On Tue, Jul 22, 2008 at 09:39:29PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> >    15. What operating system do you use Git on?
> >        (one or more: multiple choice, as one can use more than one OS)
> >      - Linux, *BSD (FreeBSD, OpenBSD, etc.), MS Windows/Cygwin,
> >        MS Windows/msysGit, MacOS X, other UNIX, other
> 
> Shouldn't we at least name the ones we have specific support in our
> Makefile instead of blanketting them into one "other Unices"?  We may not
> necessarily want to list all of them, but at least major ones like SunOS,
> HP-UX and AIX deserve to be listed, methinks.

Last time I checked git did not build on HP-UX (11.11) out of the box
because its 'install' did not support -d nor -m. There are a few more
issues installed here:

http://hpux.connect.org.uk/hppd/cgi-bin/wwwtar?/hpux/Development/Tools/git-1.5.6.2/git-1.5.6.2-src-11.11.tar.gz+git-1.5.6.2/HPUX.Install+text

So I think the word "we support" is a bit too strong. ;-)

Being more constructive, what a user using HP-UX is supported to do?

1) Use the patched git from HP.

2) Have coreutils installed. (But then I think it would be good to list
this dependency in INSTALL.)

3) Patch git to use automake's install-sh. (Would such a patch be ever
accepted?)

Thanks.

PS: No, I rarely use git on HP-UX, but I would be happy to bring the
equivalent of HP's most changes to git.git if possible.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply

* [PATCH] rebase -i: When an 'edit' stops, mention the commit
From: Johannes Sixt @ 2008-07-23  7:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, Git Mailing List

From: Johannes Sixt <johannes.sixt@telecom.at>

In a rebase session where more than one commit is to be 'edit'ed, and the
user spends considerable time to 'edit' a commit, it is easy to forget what
one wanted to 'edit' at the individual commits. It would be helpful to see
at which commit the rebase stopped.

Incidentally, if the rebase stopped due to merge conflicts or other errors,
the commit was already reported ("Could not apply $sha1..."), but when
rebase stopped after successfully applying an "edit" commit, it would not
mention it. With this change the commit is reported.

Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
---
 git-rebase--interactive.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index e63a864..4e334ba 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -277,7 +277,7 @@ do_next () {
 			die_with_patch $sha1 "Could not apply $sha1... $rest"
 		make_patch $sha1
 		: > "$DOTEST"/amend
-		warn
+		warn "Stopped at $sha1... $rest"
 		warn "You can amend the commit now, with"
 		warn
 		warn "	git commit --amend"
-- 
1.6.0.rc0.956.g7bc0

	

^ permalink raw reply related


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox