Darcs-git: a few notes for Git hackers

Darcs-git: a few notes for Git hackers

[Juliusz Chroboczek]

Hi,

Here are a few notes about Git that should probably be taken into
account by people working on Git itself or on Git wrappers.  The notes
apply to Linus' Git-0.6, which is the code I'm using in Darcs-git;
some of them might no longer be applicable to Darcs.


1. Darcs-git uses the fact that Git updates are atomic when reading
from a Git repository.  Darcs-git almost writes to Git repositories
atomically, with one exception: it performs a non-atomic
read/update/write cycle on .git/HEAD.

For that reason, I'm taking a high-level lock on .git repositories
whenever I write them.  The lockfile is ``.git/lock''.  I haven't
thought about whether Darcs can be easily coerced into accessing Git
repos atomically; have people writing Git wrappers found the need for
a global lock?


2. The files git.h and git.c in Darcs-git are a simple ``libgit'' that
contains just enough functionality for Darcs-git; they use the
functionality of sha1_file.c and read_cache.c from Git-0.6.

I've found a few problems with the interfaces in these files:

 - the global variables sha1_file_directory, active_cache, active_nr
   and active_alloc are not marked ``extern'' in cache.h.  This breaks
   linkers that don't grok common symbols, such as the one in GHCi
   (silly GHCi).

 - the function write_sha1_file takes the metadata and the data in a
   contiguous buffer, which is a problem when the data has been
   allocated by a higher layer.  I'm currently working around the
   problem by memcpy-ing everything into a temp buffer, but that's
   obviously not a good thing.  I don't care whether write_sha1_file
   is changed to use a writev-like interface, or to take the metadata
   explicitly (as in char *type, unsigned long length).

 - there is no (usable) function to write a tree; there's the code in
   write_tree.c, but it's not generally useful.  See the function
   ``git_write_tree_done'' in git.c for the type of interface I'm
   thinking of.

 - there's no way to have multiple simultaneous caches, short of
   hacking at the values of Git's global variables by hand.

As I'd rather not maintain my own version of Git, I'd be mighty
grateful if some friendly Git hacker could fix the above.

                                        Juliusz

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Mon, May 09, 2005 at 08:01:25PM CEST, I got a letter
where Juliusz Chroboczek <Juliusz.Chroboczek@pps.jussieu.fr> told me that...
> 1. Darcs-git uses the fact that Git updates are atomic when reading
> from a Git repository.  Darcs-git almost writes to Git repositories
> atomically, with one exception: it performs a non-atomic
> read/update/write cycle on .git/HEAD.
> 
> For that reason, I'm taking a high-level lock on .git repositories
> whenever I write them.  The lockfile is ``.git/lock''.  I haven't
> thought about whether Darcs can be easily coerced into accessing Git
> repos atomically; have people writing Git wrappers found the need for
> a global lock?

FWIW, Cogito does not lock at all yet - this is one of the things which
should be fixed soon.

>  - there's no way to have multiple simultaneous caches, short of
>    hacking at the values of Git's global variables by hand.

See the Brad Robert's patches of Apr 21. I've decided not to apply them
since it appears a lot has changed since then and it would be some pain;
but they may be a worthy starting point for a more up-to-date patch.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Juliusz Chroboczek]

Ahoj,

> FWIW, Cogito does not lock at all yet - this is one of the things which
> should be fixed soon.

I see.  Let me know if you decide to use a different name for the lock
file so I can switch to using the same one as yours.

                                        Julek

Re: Darcs-git: a few notes for Git hackers

[H. Peter Anvin]

Juliusz Chroboczek wrote:
> Ahoj,
> 
> 
>>FWIW, Cogito does not lock at all yet - this is one of the things which
>>should be fixed soon.
> 
> 
> I see.  Let me know if you decide to use a different name for the lock
> file so I can switch to using the same one as yours.
> 

Are you using flock(), or some other contraption that breaks if a 
process dies unexpectedly?

	-hpa

Re: Darcs-git: a few notes for Git hackers

[Juliusz Chroboczek]

>> I see.  Let me know if you decide to use a different name for the
>> lock file so I can switch to using the same one as yours.

> Are you using flock(), or some other contraption that breaks if a
> process dies unexpectedly?

No, I'm using a file that is created by the NFS-safe equivalent of
open(O_CREAT | O_EXCL).  This is what Darcs has been doing basically
forever.

Darcs usually doesn't die unexpectedly -- it's a Haskell program, so
bugs usually manifest themselves with an exception being thrown
allowing Darcs to clean-up after itself.

The one exception is when Darcs gets killed by the OOM killer (which,
as you doubtless know, doesn't give any advance warning to a process,
thus making it impossible for a process to deal with it gracefully).
In such cases, manual intervention is necessary anyway -- a file could
have been written half-way.

                                        Juliusz

Re: Darcs-git: a few notes for Git hackers

[H. Peter Anvin]

Juliusz Chroboczek wrote:
>>>I see.  Let me know if you decide to use a different name for the
>>>lock file so I can switch to using the same one as yours.
> 
> 
>>Are you using flock(), or some other contraption that breaks if a
>>process dies unexpectedly?
> 
> No, I'm using a file that is created by the NFS-safe equivalent of
> open(O_CREAT | O_EXCL).  This is what Darcs has been doing basically
> forever.
> 
> Darcs usually doesn't die unexpectedly -- it's a Haskell program, so
> bugs usually manifest themselves with an exception being thrown
> allowing Darcs to clean-up after itself.
> 
> The one exception is when Darcs gets killed by the OOM killer (which,
> as you doubtless know, doesn't give any advance warning to a process,
> thus making it impossible for a process to deal with it gracefully).
> In such cases, manual intervention is necessary anyway -- a file could
> have been written half-way.
> 

In the case of git, it should not be necessary even then; there might be 
a broken file in the repository but nothing would reference it so it 
shouldn't have any effect.  Functionally speaking, operations on the git 
repository are in themselves atomic.

	-hpa

Re: Darcs-git: a few notes for Git hackers

[Juliusz Chroboczek]

>> In such cases, manual intervention is necessary anyway -- a file could
>> have been written half-way.

> In the case of git, it should not be necessary even then; there might
> be a broken file in the repository but nothing would reference it so
> it shouldn't have any effect.  Functionally speaking, operations on
> the git repository are in themselves atomic.

Yes, you're right.

I still prefer using a lockfile to flock -- NFS-safety is important
for us.  And experience with the Darcs user base (who are probably
less Unix-savvy then the Git userbase) shows that they have no problem
doing

  $ ps
  $ rm _darcs/lock
  $ darcs check

when Darcs complains about a stray lockfile.

                                        Juliusz

Re: Darcs-git: a few notes for Git hackers

[Brad Roberts]

> >  - there's no way to have multiple simultaneous caches, short of
> >    hacking at the values of Git's global variables by hand.
>
> See the Brad Robert's patches of Apr 21. I've decided not to apply them
> since it appears a lot has changed since then and it would be some pain;
> but they may be a worthy starting point for a more up-to-date patch.
>
> --
> 				Petr "Pasky" Baudis

Since there's interest, I'll pull tip of your tree and re-do them.  I
haven't bothered todate since no one seemed interested.  Do you want them
piece meal like I did last time or just one big diff?

Later,
Brad

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Tue, May 10, 2005 at 12:50:33AM CEST, I got a letter
where Brad Roberts <braddr@puremagic.com> told me that...
> > >  - there's no way to have multiple simultaneous caches, short of
> > >    hacking at the values of Git's global variables by hand.
> >
> > See the Brad Robert's patches of Apr 21. I've decided not to apply them
> > since it appears a lot has changed since then and it would be some pain;
> > but they may be a worthy starting point for a more up-to-date patch.
> >
> > --
> > 				Petr "Pasky" Baudis
> 
> Since there's interest, I'll pull tip of your tree and re-do them.  I
> haven't bothered todate since no one seemed interested.  Do you want them
> piece meal like I did last time or just one big diff?

Piece meal would be excellent.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Junio C Hamano]

I'd rather see it done against the tip of the Linus tree.

Re: Darcs-git: a few notes for Git hackers

[Brad Roberts]

On Mon, 9 May 2005, Brad Roberts wrote:

> Date: Mon, 9 May 2005 15:50:33 -0700 (PDT)
> From: Brad Roberts <braddr@puremagic.com>
> To: Petr Baudis <pasky@ucw.cz>
> Cc: Juliusz Chroboczek <Juliusz.Chroboczek@pps.jussieu.fr>,
>      Git Mailing List <git@vger.kernel.org>, darcs-devel@abridgegame.org
> Subject: Re: Darcs-git: a few notes for Git hackers
>
> > >  - there's no way to have multiple simultaneous caches, short of
> > >    hacking at the values of Git's global variables by hand.
> >
> > See the Brad Robert's patches of Apr 21. I've decided not to apply them
> > since it appears a lot has changed since then and it would be some pain;
> > but they may be a worthy starting point for a more up-to-date patch.
> >
> > --
> > 				Petr "Pasky" Baudis
>
> Since there's interest, I'll pull tip of your tree and re-do them.  I
> haven't bothered todate since no one seemed interested.  Do you want them
> piece meal like I did last time or just one big diff?
>
> Later,
> Brad

I wasn't able to finish redoing these against linus tip, but I got most of
it done (patches 1-14 of the original 19):

  http://gameboy2.puremagic.com:8090/
  rsync://gameboy2.puremagic.com/git/

The second, third, and forth to last changes need a careful review,
they're direct applications of the original patches which were lightly
tested during the first round and nothing other than compile tested in
this round.

I suspect the remaining parts of the original patch series will go in
fairly smoothly.  If no one gets to them before tonight I'll finish
it up after work.

Later,
Brad


The commit comments:

Signed-off-by: Brad Roberts <braddr@puremagic.com>

!-------------------------------------------------------------flip-

- remove the no-longer-true comment about the cache being in native byte order
- move the cache_header struct into read-cache.c since it's in internal detail
  of the cache, not a publicly accessed element

!-------------------------------------------------------------flip-

Drop the active_cache and active_nr parameters to write_cache

!-------------------------------------------------------------flip-

- Introduce set_cache_entry(ce, pos)
- Migrate update-cache.c, the only place that does a active_cache[pos] = ce to use it
- Migrate all the same style code in read-cache.c to use it also, except for
  read_cache itself which is setting up the initial active_cache entries

TODO: rewrite the code that deal with pointers into the active_cache array such as
read-tree.c's merging code.

!-------------------------------------------------------------flip-

Introduce get_cache_entry(int pos) and use it for all trivial calls like:
  ce = active_cache[pos]

TODO: rework the non-trivial active_cache manipulations

!-------------------------------------------------------------flip-

remove active_cache_changed from cache.h

!-------------------------------------------------------------flip-

Introduce get_num_cache_entries() and migrate all the trivial callers to it

!-------------------------------------------------------------flip-

Remove active_alloc from cache.h

!-------------------------------------------------------------flip-

Restructure the diff algorythm to use indexes rather than pointer math.
The resulting code is probably a little less efficient but abstracts
the data structure.

!-------------------------------------------------------------flip-

Restructure the write tree algorythm to use indexes rather than moving
the base pointer and reducing the num entries and start using the
cache abstraction apis.

!-------------------------------------------------------------flip-

Move from pointer math to indexes and use the abstractions

!-------------------------------------------------------------flip-

- convert the last caller from touching active_cache directly
- drop active_cache and active_nr from cache.h

!-------------------------------------------------------------flip-

Re: Darcs-git: a few notes for Git hackers

[Brad Roberts]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2652 bytes --]

Resending, I left off the mailing list on the to list last time.

> > I wasn't able to finish redoing these against linus tip, but I got most of
> > it done (patches 1-14 of the original 19):
> >
> >   http://gameboy2.puremagic.com:8090/
> >   rsync://gameboy2.puremagic.com/git/
> >
> > The second, third, and forth to last changes need a careful review,
> > they're direct applications of the original patches which were lightly
> > tested during the first round and nothing other than compile tested in
> > this round.
> >
> > I suspect the remaining parts of the original patch series will go in
> > fairly smoothly.  If no one gets to them before tonight I'll finish
> > it up after work.
> >
> > Later,
> > Brad
>
> I've completed the re-merge, and moved to tip of git-pb.git rather than
> tip of git.git.  Unfortunatly that merge was also somewhat intrusive and
> my individual diffs along the way are somewhat useless now.  The entire
> history is available about the above locations still.  Attached is the
> full diff vs git-pb @ 902b92e00e491a60d55c4b2bce122903b8347f34.
>
> The unit tests that are being added are a wonderful thing, thanks so much
> for doing them Junio.
>
> These changes feel rough to me still.  Some areas to discuss / think
> about:
>
> 1) The hunks that change the merge routines from pointer based to index
> based could have been left much less intruded upon by adding a
> get_cache_entry_array(cache) api.
>
> 2) Should the index changing areas be constructing a new index instead of
> shuffling bits within the current index?
>
> 3) The vocabulary and code is inconsistent between cache and index.
>
> 4) read-cache.c does much more than reading.
>
> 5) Like before, cleaning up memory for the cache is a rarity, preferring
> to let the end of the process garbage collect everything.
>
>  cache.h          |   52 ++++++-------
>  check-files.c    |   12 +--
>  checkout-cache.c |   26 +++---
>  diff-cache.c     |   60 ++++++++-------
>  diff-files.c     |   31 ++++---
>  diff-helper.c    |    6 -
>  diff-tree.c      |   50 ++++++------
>  diff.c           |   36 +++++----
>  diff.h           |   11 +-
>  fsck-cache.c     |    6 -
>  local-pull.c     |    2
>  ls-files.c       |   47 ++++++-----
>  merge-cache.c    |   29 +++----
>  read-cache.c     |  217 ++++++++++++++++++++++++++++++++++---------------------
>  read-tree.c      |   65 +++++++++-------
>  sha1_file.c      |    4 -
>  tree.c           |   15 ++-
>  update-cache.c   |   45 +++++------
>  write-tree.c     |   29 ++++---
>  19 files changed, 416 insertions(+), 327 deletions(-)
>
> Signed-off-by: Brad Roberts <braddr@puremagic.com>
>
>


[-- Attachment #2: diff-vs-git-pb --]
[-- Type: TEXT/PLAIN, Size: 63306 bytes --]

Index: cache.h
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/cache.h  (mode:100644)
+++ uncommitted/cache.h  (mode:100644)
@@ -40,19 +40,8 @@
 
 /*
  * Basic data structures for the directory cache
- *
- * NOTE NOTE NOTE! This is all in the native CPU byte format. It's
- * not even trying to be portable. It's trying to be efficient. It's
- * just a cache, after all.
  */
 
-#define CACHE_SIGNATURE 0x44495243	/* "DIRC" */
-struct cache_header {
-	unsigned int hdr_signature;
-	unsigned int hdr_version;
-	unsigned int hdr_entries;
-};
-
 /*
  * The "cache_time" is just the low 32 bits of the
  * time. It doesn't matter if it overflows - we only
@@ -89,6 +78,9 @@
 #define CE_STAGEMASK (0x3000)
 #define CE_STAGESHIFT 12
 
+extern int ce_match_stat(struct cache_entry *ce, struct stat *st);
+extern int ce_same_name(struct cache_entry *a, struct cache_entry *b);
+
 #define create_ce_flags(len, stage) htons((len) | ((stage) << CE_STAGESHIFT))
 #define ce_namelen(ce) (CE_NAMEMASK & ntohs((ce)->ce_flags))
 #define ce_size(ce) cache_entry_size(ce_namelen(ce))
@@ -104,9 +96,6 @@
 
 #define cache_entry_size(len) ((offsetof(struct cache_entry,name) + (len) + 8) & ~7)
 
-extern struct cache_entry **active_cache;
-extern unsigned int active_nr, active_alloc, active_cache_changed;
-
 #define GIT_DIR_ENVIRONMENT "GIT_DIR"
 #define DEFAULT_GIT_DIR_ENVIRONMENT ".git"
 #define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"
@@ -120,17 +109,19 @@
 #define alloc_nr(x) (((x)+16)*3/2)
 
 /* Initialize and use the cache information */
-extern int read_cache(void);
-extern int write_cache(int newfd, struct cache_entry **cache, int entries);
-extern int cache_name_pos(const char *name, int namelen);
+extern struct cache *new_cache(void);
+extern struct cache *read_cache(void);
+extern int write_cache(struct cache *cache, int newfd);
+extern void free_cache(struct cache *cache);
+extern int cache_name_pos(struct cache *cache, const char *name, int namelen);
 #define ADD_CACHE_OK_TO_ADD 1		/* Ok to add */
 #define ADD_CACHE_OK_TO_REPLACE 2	/* Ok to replace file/directory */
-extern int add_cache_entry(struct cache_entry *ce, int option);
-extern int remove_entry_at(int pos);
-extern int remove_file_from_cache(char *path);
-extern int same_name(struct cache_entry *a, struct cache_entry *b);
-extern int cache_match_stat(struct cache_entry *ce, struct stat *st);
-extern int index_fd(unsigned char *sha1, int fd, struct stat *st);
+extern int add_cache_entry(struct cache *cache, struct cache_entry *ce, int option);
+extern int remove_file_from_cache(struct cache *cache, char *path);
+extern int get_num_cache_entries(struct cache *cache);
+extern struct cache_entry *get_cache_entry(struct cache *cache, int pos);
+extern void set_cache_entry(struct cache *cache, struct cache_entry *ce, int pos);
+extern int remove_cache_entry_at(struct cache *cache, int pos);
 
 #define MTIME_CHANGED	0x0001
 #define CTIME_CHANGED	0x0002
@@ -148,11 +139,12 @@
 extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);
 extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);
 extern int write_sha1_file(char *buf, unsigned long len, const char *type, unsigned char *return_sha1);
+extern int index_fd(unsigned char *sha1, int fd, struct stat *st);
 
 extern int check_sha1_signature(unsigned char *sha1, void *buf, unsigned long size, const char *type);
 
 /* Read a tree into the cache */
-extern int read_tree(void *buffer, unsigned long size, int stage);
+extern int read_tree(struct cache *cache, void *buffer, unsigned long size, int stage);
 
 extern int write_sha1_from_fd(const unsigned char *sha1, int fd);
 
@@ -179,7 +171,7 @@
 void parse_date(char *date, char *buf, int bufsize);
 void datestamp(char *buf, int bufsize);
 
-static inline void *xmalloc(int size)
+static inline void *xmalloc(size_t size)
 {
 	void *ret = malloc(size);
 	if (!ret)
@@ -187,7 +179,7 @@
 	return ret;
 }
 
-static inline void *xrealloc(void *ptr, int size)
+static inline void *xrealloc(void *ptr, size_t size)
 {
 	void *ret = realloc(ptr, size);
 	if (!ret)
@@ -195,4 +187,12 @@
 	return ret;
 }
 
+static inline void *xcalloc(size_t nmemb, size_t size)
+{
+	void *ret = calloc(nmemb, size);
+	if (!ret)
+		die("Out of memory, calloc failed");
+	return ret;
+}
+
 #endif /* CACHE_H */
Index: check-files.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/check-files.c  (mode:100644)
+++ uncommitted/check-files.c  (mode:100644)
@@ -8,7 +8,7 @@
  */
 #include "cache.h"
 
-static void check_file(const char *path)
+static void check_file(struct cache *cache, const char *path)
 {
 	int fd = open(path, O_RDONLY);
 	struct cache_entry *ce;
@@ -23,15 +23,15 @@
 	}
 
 	/* Exists but is not in the cache is not fine */
-	pos = cache_name_pos(path, strlen(path));
+	pos = cache_name_pos(cache, path, strlen(path));
 	if (pos < 0)
 		die("preparing to update existing file '%s' not in cache", path);
-	ce = active_cache[pos];
+	ce = get_cache_entry(cache, pos);
 
 	if (lstat(path, &st) < 0)
 		die("lstat(%s): %s", path, strerror(errno));
 
-	changed = cache_match_stat(ce, &st);
+	changed = ce_match_stat(ce, &st);
 	if (changed)
 		die("preparing to update file '%s' not uptodate in cache", path);
 }
@@ -39,9 +39,9 @@
 int main(int argc, char **argv)
 {
 	int i;
+	struct cache *cache = read_cache();
 
-	read_cache();
 	for (i = 1; i < argc ; i++)
-		check_file(argv[i]);
+		check_file(cache, argv[i]);
 	return 0;
 }
Index: checkout-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/checkout-cache.c  (mode:100644)
+++ uncommitted/checkout-cache.c  (mode:100644)
@@ -167,7 +167,7 @@
 	strcpy(path + len, ce->name);
 
 	if (!lstat(path, &st)) {
-		unsigned changed = cache_match_stat(ce, &st);
+		unsigned changed = ce_match_stat(ce, &st);
 		if (!changed)
 			return 0;
 		if (!force) {
@@ -188,30 +188,30 @@
 	return write_entry(ce, path);
 }
 
-static int checkout_file(const char *name, const char *base_dir)
+static int checkout_file(struct cache *cache, const char *name, const char *base_dir)
 {
-	int pos = cache_name_pos(name, strlen(name));
+	int pos = cache_name_pos(cache, name, strlen(name));
 	if (pos < 0) {
 		if (!quiet) {
 			pos = -pos - 1;
 			fprintf(stderr,
 				"checkout-cache: %s is %s.\n",
 				name,
-				(pos < active_nr &&
-				 !strcmp(active_cache[pos]->name, name)) ?
+				(pos < get_num_cache_entries(cache) &&
+				 !strcmp(get_cache_entry(cache, pos)->name, name)) ?
 				"unmerged" : "not in the cache");
 		}
 		return -1;
 	}
-	return checkout_entry(active_cache[pos], base_dir);
+	return checkout_entry(get_cache_entry(cache, pos), base_dir);
 }
 
-static int checkout_all(const char *base_dir)
+static int checkout_all(struct cache *cache, const char *base_dir)
 {
 	int i;
 
-	for (i = 0; i < active_nr ; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < get_num_cache_entries(cache) ; i++) {
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		if (ce_stage(ce))
 			continue;
 		if (checkout_entry(ce, base_dir) < 0)
@@ -225,15 +225,15 @@
 	int i, force_filename = 0;
 	const char *base_dir = "";
 
-	if (read_cache() < 0) {
+	struct cache *cache = read_cache();
+	if (!cache) 
 		die("invalid cache");
-	}
 
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!force_filename) {
 			if (!strcmp(arg, "-a")) {
-				checkout_all(base_dir);
+				checkout_all(cache, base_dir);
 				continue;
 			}
 			if (!strcmp(arg, "--")) {
@@ -257,7 +257,7 @@
 				continue;
 			}
 		}
-		checkout_file(arg, base_dir);
+		checkout_file(cache, arg, base_dir);
 	}
 	return 0;
 }
Index: diff-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff-cache.c  (mode:100644)
+++ uncommitted/diff-cache.c  (mode:100644)
@@ -7,10 +7,10 @@
 static int line_termination = '\n';
 
 /* A file entry went away or appeared */
-static void show_file(const char *prefix, struct cache_entry *ce, unsigned char *sha1, unsigned int mode)
+static void show_file(struct cache *cache, const char *prefix, struct cache_entry *ce, unsigned char *sha1, unsigned int mode)
 {
 	if (generate_patch)
-		diff_addremove(prefix[0], ntohl(mode), sha1, ce->name, NULL);
+		diff_addremove(cache, prefix[0], ntohl(mode), sha1, ce->name, NULL);
 	else
 		printf("%s%06o\tblob\t%s\t%s%c", prefix, ntohl(mode),
 		       sha1_to_hex(sha1), ce->name, line_termination);
@@ -33,7 +33,7 @@
 			}
 			return -1;
 		}
-		changed = cache_match_stat(ce, &st);
+		changed = ce_match_stat(ce, &st);
 		if (changed) {
 			mode = create_ce_mode(st.st_mode);
 			sha1 = no_sha1;
@@ -45,7 +45,7 @@
 	return 0;
 }
 
-static void show_new_file(struct cache_entry *new)
+static void show_new_file(struct cache *cache, struct cache_entry *new)
 {
 	unsigned char *sha1;
 	unsigned int mode;
@@ -54,10 +54,11 @@
 	if (get_stat_data(new, &sha1, &mode) < 0)
 		return;
 
-	show_file("+", new, sha1, mode);
+	show_file(cache, "+", new, sha1, mode);
 }
 
-static int show_modified(struct cache_entry *old,
+static int show_modified(struct cache *cache,
+			 struct cache_entry *old,
 			 struct cache_entry *new,
 			 int report_missing)
 {
@@ -67,7 +68,7 @@
 
 	if (get_stat_data(new, &sha1, &mode) < 0) {
 		if (report_missing)
-			show_file("-", old, old->sha1, old->ce_mode);
+			show_file(cache, "-", old, old->sha1, old->ce_mode);
 		return -1;
 	}
 
@@ -79,7 +80,7 @@
 	oldmode = ntohl(oldmode);
 
 	if (generate_patch)
-		diff_change(oldmode, mode,
+		diff_change(cache, oldmode, mode,
 			    old->sha1, sha1, old->name, NULL);
 	else {
 		strcpy(old_sha1_hex, sha1_to_hex(old->sha1));
@@ -90,30 +91,34 @@
 	return 0;
 }
 
-static int diff_cache(struct cache_entry **ac, int entries)
+static int diff_cache(struct cache *cache)
 {
-	while (entries) {
-		struct cache_entry *ce = *ac;
-		int same = (entries > 1) && same_name(ce, ac[1]);
+	int pos = 0, num = get_num_cache_entries(cache);
+
+	while (pos < num) {
+		struct cache_entry *ce = get_cache_entry(cache, pos);
+		struct cache_entry *ce_next = ((pos+1) < num) ?
+			get_cache_entry(cache, pos+1) : NULL;
+		int same = ce_next && ce_same_name(ce, ce_next);
 
 		switch (ce_stage(ce)) {
 		case 0:
 			/* No stage 1 entry? That means it's a new file */
 			if (!same) {
-				show_new_file(ce);
+				show_new_file(cache, ce);
 				break;
 			}
 			/* Show difference between old and new */
-			show_modified(ac[1], ce, 1);
+			show_modified(cache, ce_next, ce, 1);
 			break;
 		case 1:
 			/* No stage 3 (merge) entry? That means it's been deleted */
 			if (!same) {
-				show_file("-", ce, ce->sha1, ce->ce_mode);
+				show_file(cache, "-", ce, ce->sha1, ce->ce_mode);
 				break;
 			}
 			/* We come here with ce pointing at stage 1
-			 * (original tree) and ac[1] pointing at stage
+			 * (original tree) and ce_next pointing at stage
 			 * 3 (unmerged).  show-modified with
 			 * report-mising set to false does not say the
 			 * file is deleted but reports true if work
@@ -122,12 +127,12 @@
 			 * Otherwise, we show the differences between
 			 * the original tree and the work tree.
 			 */
-			if (!cached_only && !show_modified(ce, ac[1], 0))
+			if (!cached_only && !show_modified(cache, ce, ce_next, 0))
 				break;
 			/* fallthru */
 		case 3:
 			if (generate_patch)
-				diff_unmerge(ce->name);
+				diff_unmerge(cache, ce->name);
 			else
 				printf("U %s%c", ce->name, line_termination);
 			break;
@@ -141,9 +146,8 @@
 		 * we've handled the relevant cases now.
 		 */
 		do {
-			ac++;
-			entries--;
-		} while (entries && same_name(ce, ac[0]));
+			pos++;
+		} while (pos < num && ce_same_name(ce, get_cache_entry(cache, pos)));
 	}
 	return 0;
 }
@@ -153,11 +157,11 @@
  * when we read in the new tree (into "stage 1"), we won't lose sight
  * of the fact that we had unmerged entries.
  */
-static void mark_merge_entries(void)
+static void mark_merge_entries(struct cache *cache)
 {
 	int i;
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < get_num_cache_entries(cache); i++) {
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		if (!ce_stage(ce))
 			continue;
 		ce->ce_flags |= htons(CE_STAGEMASK);
@@ -172,8 +176,8 @@
 	unsigned char tree_sha1[20];
 	void *tree;
 	unsigned long size;
+	struct cache * cache = read_cache();
 
-	read_cache();
 	while (argc > 2) {
 		char *arg = argv[1];
 		argv++;
@@ -204,13 +208,13 @@
 	if (argc != 2 || get_sha1(argv[1], tree_sha1))
 		usage(diff_cache_usage);
 
-	mark_merge_entries();
+	mark_merge_entries(cache);
 
 	tree = read_object_with_reference(tree_sha1, "tree", &size, 0);
 	if (!tree)
 		die("bad tree object %s", argv[1]);
-	if (read_tree(tree, size, 1))
+	if (read_tree(cache, tree, size, 1))
 		die("unable to read tree object %s", argv[1]);
 
-	return diff_cache(active_cache, active_nr);
+	return diff_cache(cache);
 }
Index: diff-files.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff-files.c  (mode:100644)
+++ uncommitted/diff-files.c  (mode:100644)
@@ -28,18 +28,18 @@
 	return 0;
 }
 
-static void show_unmerge(const char *path)
+static void show_unmerge(struct cache *cache, const char *path)
 {
 	if (generate_patch)
-		diff_unmerge(path);
+		diff_unmerge(cache, path);
 	else
 		printf("U %s%c", path, line_termination);
 }
 
-static void show_file(int pfx, struct cache_entry *ce)
+static void show_file(struct cache *cache, int pfx, struct cache_entry *ce)
 {
 	if (generate_patch)
-		diff_addremove(pfx, ntohl(ce->ce_mode), ce->sha1,
+		diff_addremove(cache, pfx, ntohl(ce->ce_mode), ce->sha1,
 			       ce->name, NULL);
 	else
 		printf("%c%06o\t%s\t%s\t%s%c",
@@ -47,7 +47,7 @@
 		       sha1_to_hex(ce->sha1), ce->name, line_termination);
 }
 
-static void show_modified(int oldmode, int mode,
+static void show_modified(struct cache *cache, int oldmode, int mode,
 			  const char *old_sha1, const char *sha1,
 			  char *path)
 {
@@ -55,7 +55,7 @@
 	strcpy(old_sha1_hex, sha1_to_hex(old_sha1));
 
 	if (generate_patch)
-		diff_change(oldmode, mode, old_sha1, sha1, path, NULL);
+		diff_change(cache, oldmode, mode, old_sha1, sha1, path, NULL);
 	else
 		printf("*%06o->%06o\tblob\t%s->%s\t%s%c",
 		       oldmode, mode, old_sha1_hex, sha1_to_hex(sha1), path,
@@ -65,7 +65,8 @@
 int main(int argc, char **argv)
 {
 	static const char null_sha1[20] = { 0, };
-	int entries = read_cache();
+	struct cache *cache = read_cache();
+	int entries;
 	int i;
 
 	while (1 < argc && argv[1][0] == '-') {
@@ -87,15 +88,17 @@
 	/* At this point, if argc == 1, then we are doing everything.
 	 * Otherwise argv[1] .. argv[argc-1] have the explicit paths.
 	 */
-	if (entries < 0) {
+	if (!cache) {
 		perror("read_cache");
 		exit(1);
 	}
 
+	entries = get_num_cache_entries(cache);
+
 	for (i = 0; i < entries; i++) {
 		struct stat st;
 		unsigned int oldmode, mode;
-		struct cache_entry *ce = active_cache[i];
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		int changed;
 
 		if (1 < argc &&
@@ -103,9 +106,9 @@
 			continue;
 
 		if (ce_stage(ce)) {
-			show_unmerge(ce->name);
+			show_unmerge(cache, ce->name);
 			while (i < entries &&
-			       !strcmp(ce->name, active_cache[i]->name))
+			       !strcmp(ce->name, get_cache_entry(cache, i)->name))
 				i++;
 			i--; /* compensate for loop control increments */
 			continue;
@@ -118,10 +121,10 @@
 			}	
 			if (silent)
 				continue;
-			show_file('-', ce);
+			show_file(cache, '-', ce);
 			continue;
 		}
-		changed = cache_match_stat(ce, &st);
+		changed = ce_match_stat(ce, &st);
 		if (!changed)
 			continue;
 
@@ -129,7 +132,7 @@
 		mode = (S_ISLNK(st.st_mode) ? S_IFLNK :
 			S_IFREG | ce_permissions(st.st_mode));
 
-		show_modified(oldmode, mode, ce->sha1, null_sha1,
+		show_modified(cache, oldmode, mode, ce->sha1, null_sha1,
 			      ce->name);
 	}
 	return 0;
Index: diff-helper.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff-helper.c  (mode:100644)
+++ uncommitted/diff-helper.c  (mode:100644)
@@ -56,7 +56,7 @@
 	switch (*cp++) {
 	case 'U':
 		if (!cnt || matches_pathspec(cp + 1, spec, cnt))
-			diff_unmerge(cp + 1);
+			diff_unmerge(NULL, cp + 1);
 		return 0;
 	case '+':
 		old.file_valid = 0;
@@ -102,9 +102,9 @@
 	}
 	if (!cnt || matches_pathspec(path, spec, cnt)) {
 		if (reverse)
-			run_external_diff(path, &new, &old);
+			run_external_diff(NULL, path, &new, &old);
 		else
-			run_external_diff(path, &old, &new);
+			run_external_diff(NULL, path, &old, &new);
 	}
 	return 0;
 }
Index: diff-tree.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff-tree.c  (mode:100644)
+++ uncommitted/diff-tree.c  (mode:100644)
@@ -17,7 +17,7 @@
 static char **paths = NULL;
 static int *pathlens = NULL;
 
-static int diff_tree_sha1(const unsigned char *old, const unsigned char *new, const char *base);
+static int diff_tree_sha1(struct cache *cache, const unsigned char *old, const unsigned char *new, const char *base);
 
 static void update_tree_entry(void **bufp, unsigned long *sizep)
 {
@@ -53,19 +53,19 @@
 	return newbase;
 }
 
-static void show_file(const char *prefix, void *tree, unsigned long size, const char *base);
+static void show_file(struct cache *cache, const char *prefix, void *tree, unsigned long size, const char *base);
 
 /* A whole sub-tree went away or appeared */
-static void show_tree(const char *prefix, void *tree, unsigned long size, const char *base)
+static void show_tree(struct cache *cache, const char *prefix, void *tree, unsigned long size, const char *base)
 {
 	while (size) {
-		show_file(prefix, tree, size, base);
+		show_file(cache, prefix, tree, size, base);
 		update_tree_entry(&tree, &size);
 	}
 }
 
 /* A file entry went away or appeared */
-static void show_file(const char *prefix, void *tree, unsigned long size, const char *base)
+static void show_file(struct cache *cache, const char *prefix, void *tree, unsigned long size, const char *base)
 {
 	unsigned mode;
 	const char *path;
@@ -89,7 +89,7 @@
 		if (!tree || strcmp(type, "tree"))
 			die("corrupt tree sha %s", sha1_to_hex(sha1));
 
-		show_tree(prefix, tree, size, newbase);
+		show_tree(cache, prefix, tree, size, newbase);
 		
 		free(tree);
 		free(newbase);
@@ -98,7 +98,7 @@
 
 	if (generate_patch) {
 		if (!S_ISDIR(mode))
-			diff_addremove(prefix[0], mode, sha1, base, path);
+			diff_addremove(cache, prefix[0], mode, sha1, base, path);
 	}
 	else
 		printf("%s%06o\t%s\t%s\t%s%s%c", prefix, mode,
@@ -107,7 +107,7 @@
 		       line_termination);
 }
 
-static int compare_tree_entry(void *tree1, unsigned long size1, void *tree2, unsigned long size2, const char *base)
+static int compare_tree_entry(struct cache *cache, void *tree1, unsigned long size1, void *tree2, unsigned long size2, const char *base)
 {
 	unsigned mode1, mode2;
 	const char *path1, *path2;
@@ -122,11 +122,11 @@
 	pathlen2 = strlen(path2);
 	cmp = cache_name_compare(path1, pathlen1, path2, pathlen2);
 	if (cmp < 0) {
-		show_file("-", tree1, size1, base);
+		show_file(cache, "-", tree1, size1, base);
 		return -1;
 	}
 	if (cmp > 0) {
-		show_file("+", tree2, size2, base);
+		show_file(cache, "+", tree2, size2, base);
 		return 1;
 	}
 	if (!memcmp(sha1, sha2, 20) && mode1 == mode2)
@@ -137,15 +137,15 @@
 	 * file, we need to consider it a remove and an add.
 	 */
 	if (S_ISDIR(mode1) != S_ISDIR(mode2)) {
-		show_file("-", tree1, size1, base);
-		show_file("+", tree2, size2, base);
+		show_file(cache, "-", tree1, size1, base);
+		show_file(cache, "+", tree2, size2, base);
 		return 0;
 	}
 
 	if (recursive && S_ISDIR(mode1)) {
 		int retval;
 		char *newbase = malloc_base(base, path1, pathlen1);
-		retval = diff_tree_sha1(sha1, sha2, newbase);
+		retval = diff_tree_sha1(cache, sha1, sha2, newbase);
 		free(newbase);
 		return retval;
 	}
@@ -159,7 +159,7 @@
 
 	if (generate_patch) {
 		if (!S_ISDIR(mode1))
-			diff_change(mode1, mode2, sha1, sha2, base, path1);
+			diff_change(cache, mode1, mode2, sha1, sha2, base, path1);
 	}
 	else {
 		strcpy(old_sha1_hex, sha1_to_hex(sha1));
@@ -217,7 +217,7 @@
 	return 0; /* No matches */
 }
 
-static int diff_tree(void *tree1, unsigned long size1, void *tree2, unsigned long size2, const char *base)
+static int diff_tree(struct cache *cache, void *tree1, unsigned long size1, void *tree2, unsigned long size2, const char *base)
 {
 	while (size1 | size2) {
 		if (nr_paths && size1 && !interesting(tree1, size1, base)) {
@@ -229,16 +229,16 @@
 			continue;
 		}
 		if (!size1) {
-			show_file("+", tree2, size2, base);
+			show_file(cache, "+", tree2, size2, base);
 			update_tree_entry(&tree2, &size2);
 			continue;
 		}
 		if (!size2) {
-			show_file("-", tree1, size1, base);
+			show_file(cache, "-", tree1, size1, base);
 			update_tree_entry(&tree1, &size1);
 			continue;
 		}
-		switch (compare_tree_entry(tree1, size1, tree2, size2, base)) {
+		switch (compare_tree_entry(cache, tree1, size1, tree2, size2, base)) {
 		case -1:
 			update_tree_entry(&tree1, &size1);
 			continue;
@@ -254,7 +254,7 @@
 	return 0;
 }
 
-static int diff_tree_sha1(const unsigned char *old, const unsigned char *new, const char *base)
+static int diff_tree_sha1(struct cache *cache, const unsigned char *old, const unsigned char *new, const char *base)
 {
 	void *tree1, *tree2;
 	unsigned long size1, size2;
@@ -266,7 +266,7 @@
 	tree2 = read_object_with_reference(new, "tree", &size2, 0);
 	if (!tree2)
 		die("unable to read destination tree (%s)", sha1_to_hex(new));
-	retval = diff_tree(tree1, size1, tree2, size2, base);
+	retval = diff_tree(cache, tree1, size1, tree2, size2, base);
 	free(tree1);
 	free(tree2);
 	return retval;
@@ -342,7 +342,7 @@
 	return this_header;
 }
 
-static int diff_tree_stdin(char *line)
+static int diff_tree_stdin(struct cache *cache, char *line)
 {
 	int len = strlen(line);
 	unsigned char commit[20], parent[20];
@@ -360,7 +360,7 @@
 		line[81] = 0;
 		sprintf(this_header, "%s (from %s)\n", line, line+41);
 		header = this_header;
-		return diff_tree_sha1(parent, commit, "");
+		return diff_tree_sha1(cache, parent, commit, "");
 	}
 	buf = read_object_with_reference(commit, "commit", &size, NULL);
 	if (!buf)
@@ -378,7 +378,7 @@
 		if (get_sha1_hex(buf + offset + 7, parent))
 			return -1;
 		header = generate_header(line, sha1_to_hex(parent), buf, size);
-		diff_tree_sha1(parent, commit, "");
+		diff_tree_sha1(cache, parent, commit, "");
 		if (!header && verbose_header)
 			header_prefix = "\ndiff-tree ";
 		offset += 48;
@@ -458,10 +458,10 @@
 	}
 
 	if (!read_stdin)
-		return diff_tree_sha1(old, new, "");
+		return diff_tree_sha1(NULL, old, new, "");
 
 	while (fgets(line, sizeof(line), stdin))
-		diff_tree_stdin(line);
+		diff_tree_stdin(NULL, line);
 
 	return 0;
 }
Index: diff.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff.c  (mode:100644)
+++ uncommitted/diff.c  (mode:100644)
@@ -147,7 +147,7 @@
  * the work tree has that object contents, return true, so that
  * prepare_temp_file() does not have to inflate and extract.
  */
-static int work_tree_matches(const char *name, const unsigned char *sha1)
+static int work_tree_matches(struct cache *cache, const char *name, const unsigned char *sha1)
 {
 	struct cache_entry *ce;
 	struct stat st;
@@ -165,17 +165,17 @@
 	 * by diff-cache --cached, which does read the cache before
 	 * calling us.
 	 */ 
-	if (!active_cache)
+	if (!get_num_cache_entries(cache))
 		return 0;
 
 	len = strlen(name);
-	pos = cache_name_pos(name, len);
+	pos = cache_name_pos(cache, name, len);
 	if (pos < 0)
 		return 0;
-	ce = active_cache[pos];
+	ce = get_cache_entry(cache, pos);
 	if ((lstat(name, &st) < 0) ||
 	    !S_ISREG(st.st_mode) ||
-	    cache_match_stat(ce, &st) ||
+	    ce_match_stat(ce, &st) ||
 	    memcmp(sha1, ce->sha1, 20))
 		return 0;
 	return 1;
@@ -202,7 +202,8 @@
 	sprintf(temp->mode, "%06o", mode);
 }
 
-static void prepare_temp_file(const char *name,
+static void prepare_temp_file(struct cache *cache,
+			      const char *name,
 			      struct diff_tempfile *temp,
 			      struct diff_spec *one)
 {
@@ -222,7 +223,7 @@
 
 	if (one->sha1_valid &&
 	    (!memcmp(one->blob_sha1, null_sha1, sizeof(null_sha1)) ||
-	     work_tree_matches(name, one->blob_sha1)))
+	     work_tree_matches(cache, name, one->blob_sha1)))
 		use_work_tree = 1;
 
 	if (!one->sha1_valid || use_work_tree) {
@@ -293,7 +294,8 @@
  *               infile2 infile2-sha1 infile2-mode.
  *
  */
-void run_external_diff(const char *name,
+void run_external_diff(struct cache *cache,
+		       const char *name,
 		       struct diff_spec *one,
 		       struct diff_spec *two)
 {
@@ -303,8 +305,8 @@
 	static int atexit_asked = 0;
 
 	if (one && two) {
-		prepare_temp_file(name, &temp[0], one);
-		prepare_temp_file(name, &temp[1], two);
+		prepare_temp_file(cache, name, &temp[0], one);
+		prepare_temp_file(cache, name, &temp[1], two);
 		if (! atexit_asked &&
 		    (temp[0].name == temp[0].tmp_path ||
 		     temp[1].name == temp[1].tmp_path)) {
@@ -357,7 +359,8 @@
 	remove_tempfile();
 }
 
-void diff_addremove(int addremove, unsigned mode,
+void diff_addremove(struct cache *cache,
+		    int addremove, unsigned mode,
 		    const unsigned char *sha1,
 		    const char *base, const char *path)
 {
@@ -379,10 +382,11 @@
 		strcpy(concatpath, base);
 		strcat(concatpath, path);
 	}
-	run_external_diff(path ? concatpath : base, one, two);
+	run_external_diff(cache, path ? concatpath : base, one, two);
 }
 
-void diff_change(unsigned old_mode, unsigned new_mode,
+void diff_change(struct cache *cache,
+		 unsigned old_mode, unsigned new_mode,
 		 const unsigned char *old_sha1,
 		 const unsigned char *new_sha1,
 		 const char *base, const char *path) {
@@ -400,10 +404,10 @@
 		strcpy(concatpath, base);
 		strcat(concatpath, path);
 	}
-	run_external_diff(path ? concatpath : base, &spec[0], &spec[1]);
+	run_external_diff(cache, path ? concatpath : base, &spec[0], &spec[1]);
 }
 
-void diff_unmerge(const char *path)
+void diff_unmerge(struct cache *cache, const char *path)
 {
-	run_external_diff(path, NULL, NULL);
+	run_external_diff(cache, path, NULL, NULL);
 }
Index: diff.h
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/diff.h  (mode:100644)
+++ uncommitted/diff.h  (mode:100644)
@@ -4,18 +4,20 @@
 #ifndef DIFF_H
 #define DIFF_H
 
-extern void diff_addremove(int addremove,
+extern void diff_addremove(struct cache *cache,
+			   int addremove,
 			   unsigned mode,
 			   const unsigned char *sha1,
 			   const char *base,
 			   const char *path);
 
-extern void diff_change(unsigned mode1, unsigned mode2,
+extern void diff_change(struct cache *cache,
+			     unsigned mode1, unsigned mode2,
 			     const unsigned char *sha1,
 			     const unsigned char *sha2,
 			     const char *base, const char *path);
 
-extern void diff_unmerge(const char *path);
+extern void diff_unmerge(struct cache *cache, const char *path);
 
 /* These are for diff-helper */
 
@@ -31,7 +33,8 @@
 	unsigned file_valid : 1; /* if false the file does not even exist */
 };
 
-extern void run_external_diff(const char *name,
+extern void run_external_diff(struct cache *cache,
+			      const char *name,
 			      struct diff_spec *, struct diff_spec *);
 
 #endif /* DIFF_H */
Index: fsck-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/fsck-cache.c  (mode:100644)
+++ uncommitted/fsck-cache.c  (mode:100644)
@@ -356,9 +356,9 @@
 
 	if (keep_cache_objects) {
 		int i;
-		read_cache();
-		for (i = 0; i < active_nr; i++) {
-			struct blob *blob = lookup_blob(active_cache[i]->sha1);
+		struct cache *cache = read_cache();
+		for (i = 0; i < get_num_cache_entries(cache); i++) {
+			struct blob *blob = lookup_blob(get_cache_entry(cache, i)->sha1);
 			struct object *obj;
 			if (!blob)
 				continue;
Index: local-pull.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/local-pull.c  (mode:100644)
+++ uncommitted/local-pull.c  (mode:100644)
@@ -61,7 +61,7 @@
 		}
 		map = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, ifd, 0);
 		close(ifd);
-		if (-1 == (int)(long)map) {
+		if (MAP_FAILED == map) {
 			fprintf(stderr, "cannot mmap %s\n", filename);
 			return -1;
 		}
Index: ls-files.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/ls-files.c  (mode:100644)
+++ uncommitted/ls-files.c  (mode:100644)
@@ -98,11 +98,11 @@
 static int nr_dir;
 static int dir_alloc;
 
-static void add_name(const char *pathname, int len)
+static void add_name(struct cache *cache, const char *pathname, int len)
 {
 	struct nond_on_fs *ent;
 
-	if (cache_name_pos(pathname, len) >= 0)
+	if (cache_name_pos(cache, pathname, len) >= 0)
 		return;
 
 	if (nr_dir == dir_alloc) {
@@ -124,7 +124,7 @@
  * Also, we currently ignore all names starting with a dot.
  * That likely will not change.
  */
-static void read_directory(const char *path, const char *base, int baselen)
+static void read_directory(struct cache *cache, const char *path, const char *base, int baselen)
 {
 	DIR *dir = opendir(path);
 
@@ -157,14 +157,14 @@
 				/* fallthrough */
 			case DT_DIR:
 				memcpy(fullname + baselen + len, "/", 2);
-				read_directory(fullname, fullname,
+				read_directory(cache, fullname, fullname,
 					       baselen + len + 1);
 				continue;
 			case DT_REG:
 			case DT_LNK:
 				break;
 			}
-			add_name(fullname, baselen + len);
+			add_name(cache, fullname, baselen + len);
 		}
 		closedir(dir);
 	}
@@ -179,7 +179,7 @@
 				  e2->name, e2->len);
 }
 
-static void show_killed_files()
+static void show_killed_files(struct cache *cache)
 {
 	int i;
 	for (i = 0; i < nr_dir; i++) {
@@ -193,28 +193,28 @@
 				/* If ent->name is prefix of an entry in the
 				 * cache, it will be killed.
 				 */
-				pos = cache_name_pos(ent->name, ent->len);
+				pos = cache_name_pos(cache, ent->name, ent->len);
 				if (0 <= pos)
 					die("bug in show-killed-files");
 				pos = -pos - 1;
-				while (pos < active_nr &&
-				       ce_stage(active_cache[pos]))
+				while (pos < get_num_cache_entries(cache) &&
+				       ce_stage(get_cache_entry(cache, pos)))
 					pos++; /* skip unmerged */
-				if (active_nr <= pos)
+				if (get_num_cache_entries(cache) <= pos)
 					break;
 				/* pos points at a name immediately after
 				 * ent->name in the cache.  Does it expect
 				 * ent->name to be a directory?
 				 */
-				len = ce_namelen(active_cache[pos]);
+				len = ce_namelen(get_cache_entry(cache, pos));
 				if ((ent->len < len) &&
-				    !strncmp(active_cache[pos]->name,
+				    !strncmp(get_cache_entry(cache, pos)->name,
 					     ent->name, ent->len) &&
-				    active_cache[pos]->name[ent->len] == '/')
+				    get_cache_entry(cache, pos)->name[ent->len] == '/')
 					killed = 1;
 				break;
 			}
-			if (0 <= cache_name_pos(ent->name, sp - ent->name)) {
+			if (0 <= cache_name_pos(cache, ent->name, sp - ent->name)) {
 				/* If any of the leading directories in
 				 * ent->name is registered in the cache,
 				 * ent->name will be killed.
@@ -230,13 +230,13 @@
 	}
 }
 
-static void show_files(void)
+static void show_files(struct cache *cache)
 {
 	int i;
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
-		read_directory(".", "", 0);
+		read_directory(cache, ".", "", 0);
 		qsort(dir, nr_dir, sizeof(struct nond_on_fs *), cmp_name);
 		if (show_others)
 			for (i = 0; i < nr_dir; i++)
@@ -244,11 +244,11 @@
 				       dir[i]->len, dir[i]->name,
 				       line_terminator);
 		if (show_killed)
-			show_killed_files();
+			show_killed_files(cache);
 	}
 	if (show_cached | show_stage) {
-		for (i = 0; i < active_nr; i++) {
-			struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < get_num_cache_entries(cache); i++) {
+			struct cache_entry *ce = get_cache_entry(cache, i);
 			if (excluded(ce->name) != show_ignored)
 				continue;
 			if (show_unmerged && !ce_stage(ce))
@@ -269,8 +269,8 @@
 		}
 	}
 	if (show_deleted) {
-		for (i = 0; i < active_nr; i++) {
-			struct cache_entry *ce = active_cache[i];
+		for (i = 0; i < get_num_cache_entries(cache); i++) {
+			struct cache_entry *ce = get_cache_entry(cache, i);
 			struct stat st;
 			if (excluded(ce->name) != show_ignored)
 				continue;
@@ -289,6 +289,7 @@
 int main(int argc, char **argv)
 {
 	int i;
+	struct cache *cache;
 
 	for (i = 1; i < argc; i++) {
 		char *arg = argv[i];
@@ -341,7 +342,7 @@
 	if (!(show_stage | show_deleted | show_others | show_unmerged | show_killed))
 		show_cached = 1;
 
-	read_cache();
-	show_files();
+	cache = read_cache();
+	show_files(cache);
 	return 0;
 }
Index: merge-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/merge-cache.c  (mode:100644)
+++ uncommitted/merge-cache.c  (mode:100644)
@@ -34,11 +34,11 @@
 	}
 }
 
-static int merge_entry(int pos, const char *path)
+static int merge_entry(struct cache *cache, int pos, const char *path)
 {
 	int found;
 	
-	if (pos >= active_nr)
+	if (pos >= get_num_cache_entries(cache))
 		die("merge-cache: %s not in the cache", path);
 	arguments[0] = pgm;
 	arguments[1] = "";
@@ -52,7 +52,7 @@
 	do {
 		static char hexbuf[4][60];
 		static char ownbuf[4][60];
-		struct cache_entry *ce = active_cache[pos];
+		struct cache_entry *ce = get_cache_entry(cache, pos);
 		int stage = ce_stage(ce);
 
 		if (strcmp(ce->name, path))
@@ -62,44 +62,45 @@
 		sprintf(ownbuf[stage], "%o", ntohl(ce->ce_mode) & (~S_IFMT));
 		arguments[stage] = hexbuf[stage];
 		arguments[stage + 4] = ownbuf[stage];
-	} while (++pos < active_nr);
+	} while (++pos < get_num_cache_entries(cache));
 	if (!found)
 		die("merge-cache: %s not in the cache", path);
 	run_program();
 	return found;
 }
 
-static void merge_file(const char *path)
+static void merge_file(struct cache *cache, const char *path)
 {
-	int pos = cache_name_pos(path, strlen(path));
+	int pos = cache_name_pos(cache, path, strlen(path));
 
 	/*
 	 * If it already exists in the cache as stage0, it's
 	 * already merged and there is nothing to do.
 	 */
 	if (pos < 0)
-		merge_entry(-pos-1, path);
+		merge_entry(cache, -pos-1, path);
 }
 
-static void merge_all(void)
+static void merge_all(struct cache *cache)
 {
 	int i;
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
+	for (i = 0; i < get_num_cache_entries(cache); i++) {
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		if (!ce_stage(ce))
 			continue;
-		i += merge_entry(i, ce->name)-1;
+		i += merge_entry(cache, i, ce->name)-1;
 	}
 }
 
 int main(int argc, char **argv)
 {
 	int i, force_file = 0;
+	struct cache *cache;
 
 	if (argc < 3)
 		usage("merge-cache [-o] <merge-program> (-a | <filename>*)");
 
-	read_cache();
+	cache = read_cache();
 
 	i = 1;
 	if (!strcmp(argv[1], "-o")) {
@@ -115,12 +116,12 @@
 				continue;
 			}
 			if (!strcmp(arg, "-a")) {
-				merge_all();
+				merge_all(cache);
 				continue;
 			}
 			die("merge-cache: unknown option %s", arg);
 		}
-		merge_file(arg);
+		merge_file(cache, arg);
 	}
 	if (err)
 		die("merge program failed");
Index: read-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/read-cache.c  (mode:100644)
+++ uncommitted/read-cache.c  (mode:100644)
@@ -6,10 +6,49 @@
 #include <stdarg.h>
 #include "cache.h"
 
-struct cache_entry **active_cache = NULL;
-unsigned int active_nr = 0, active_alloc = 0, active_cache_changed = 0;
+/*
+ * Basic data structures for the directory cache
+ */
+
+#define CACHE_SIGNATURE 0x44495243	/* "DIRC" */
+struct cache_header {
+	unsigned int hdr_signature;
+	unsigned int hdr_version;
+	unsigned int hdr_entries;
+};
+
+struct mmap_holder {
+	void * ptr;
+	size_t size;
+};
+
+struct cache {
+	struct mmap_holder   map;
+	struct cache_header *header;
+	struct cache_entry **entries;
+	unsigned int num_entries;
+	unsigned int allocated_entries;
+	unsigned int active_cache_changed;
+};
+
+struct cache * new_cache()
+{
+	struct cache *cache = xcalloc(1, sizeof(struct cache));
+
+	cache->map.ptr = MAP_FAILED;
+
+	return cache;
+}
+
+void free_cache(struct cache *cache)
+{
+	if (cache->map.ptr != MAP_FAILED)
+		munmap(cache->map.ptr, cache->map.size);
+
+	free(cache);
+}
 
-int cache_match_stat(struct cache_entry *ce, struct stat *st)
+int ce_match_stat(struct cache_entry *ce, struct stat *st)
 {
 	unsigned int changed = 0;
 
@@ -75,15 +114,15 @@
 	return 0;
 }
 
-int cache_name_pos(const char *name, int namelen)
+int cache_name_pos(struct cache *cache, const char *name, int namelen)
 {
 	int first, last;
 
 	first = 0;
-	last = active_nr;
+	last = cache->num_entries;
 	while (last > first) {
 		int next = (last + first) >> 1;
-		struct cache_entry *ce = active_cache[next];
+		struct cache_entry *ce = get_cache_entry(cache, next);
 		int cmp = cache_name_compare(name, namelen, ce->name, htons(ce->ce_flags));
 		if (!cmp)
 			return next;
@@ -97,27 +136,27 @@
 }
 
 /* Remove entry, return true if there are more entries to go.. */
-int remove_entry_at(int pos)
+int remove_cache_entry_at(struct cache *cache, int pos)
 {
-	active_cache_changed = 1;
-	active_nr--;
-	if (pos >= active_nr)
+	cache->active_cache_changed = 1;
+	cache->num_entries--;
+	if (pos >= cache->num_entries)
 		return 0;
-	memmove(active_cache + pos, active_cache + pos + 1, (active_nr - pos) * sizeof(struct cache_entry *));
+	memmove(cache->entries + pos, cache->entries + pos + 1, (cache->num_entries - pos) * sizeof(struct cache_entry *));
 	return 1;
 }
 
-int remove_file_from_cache(char *path)
+int remove_file_from_cache(struct cache *cache, char *path)
 {
-	int pos = cache_name_pos(path, strlen(path));
+	int pos = cache_name_pos(cache, path, strlen(path));
 	if (pos < 0)
 		pos = -pos-1;
-	while (pos < active_nr && !strcmp(active_cache[pos]->name, path))
-		remove_entry_at(pos);
+	while (pos < get_num_cache_entries(cache) && !strcmp(get_cache_entry(cache, pos)->name, path))
+		remove_cache_entry_at(cache, pos);
 	return 0;
 }
 
-int same_name(struct cache_entry *a, struct cache_entry *b)
+int ce_same_name(struct cache_entry *a, struct cache_entry *b)
 {
 	int len = ce_namelen(a);
 	return ce_namelen(b) == len && !memcmp(a->name, b->name, len);
@@ -132,7 +171,8 @@
  * from the cache so the caller should recompute the insert position.
  * When this happens, we return non-zero.
  */
-static int check_file_directory_conflict(const struct cache_entry *ce,
+static int check_file_directory_conflict(struct cache *cache,
+					 const struct cache_entry *ce,
 					 int ok_to_replace)
 {
 	int pos, replaced = 0;
@@ -155,7 +195,7 @@
 		if (!ep)
 			break;
 		*ep = 0;    /* first cut it at slash */
-		pos = cache_name_pos(pathbuf,
+		pos = cache_name_pos(cache, pathbuf,
 				     htons(create_ce_flags(ep-cp, stage)));
 		if (0 <= pos) {
 			/* Our leading path component is registered as a file,
@@ -167,7 +207,7 @@
 				return -1;
 			}
 			fprintf(stderr, "removing file '%s' to replace it with a directory to create '%s'.\n", pathbuf, path);
-			remove_entry_at(pos);
+			remove_cache_entry_at(cache, pos);
 			replaced = 1;
 		}
 		*ep = '/';  /* then restore it and go downwards */
@@ -179,7 +219,7 @@
 	 * of it?  That is, are we creating a file where they already expect
 	 * a directory there?
 	 */
-	pos = cache_name_pos(path,
+	pos = cache_name_pos(cache, path,
 			     htons(create_ce_flags(namelen, stage)));
 
 	/* (0 <= pos) cannot happen because add_cache_entry()
@@ -206,8 +246,8 @@
 	 * path of an existing entry anymore.
 	 */
 
-	while (pos < active_nr) {
-		struct cache_entry *other = active_cache[pos];
+	while (pos < get_num_cache_entries(cache)) {
+		struct cache_entry *other = get_cache_entry(cache, pos);
 		if (strncmp(other->name, path, namelen))
 			break; /* it is not our "subdirectory" anymore */
 		if ((ce_stage(other) == stage) &&
@@ -215,7 +255,7 @@
 			if (!ok_to_replace)
 				return -1;
 			fprintf(stderr, "removing file '%s' under '%s' to be replaced with a file\n", other->name, path);
-			remove_entry_at(pos);
+			remove_cache_entry_at(cache, pos);
 			replaced = 1;
 			continue; /* cycle without updating pos */
 		}
@@ -224,17 +264,35 @@
 	return replaced;
 }
 
-int add_cache_entry(struct cache_entry *ce, int option)
+void set_cache_entry(struct cache *cache, struct cache_entry *ce, int pos)
+{
+	cache->active_cache_changed = 1;
+	cache->entries[pos] = ce;
+}
+
+int get_num_cache_entries(struct cache *cache)
+{
+	return cache->num_entries;
+}
+
+struct cache_entry *get_cache_entry(struct cache *cache, int pos)
+{
+ 	/* You can NOT just free cache->entries[pos] here, since it
+  	 * might not be necessarily malloc()ed but can also come
+  	 * from mmap(). */
+	return cache->entries[pos];
+}
+
+int add_cache_entry(struct cache *cache, struct cache_entry *ce, int option)
 {
 	int pos;
 	int ok_to_add = option & ADD_CACHE_OK_TO_ADD;
 	int ok_to_replace = option & ADD_CACHE_OK_TO_REPLACE;
-	pos = cache_name_pos(ce->name, htons(ce->ce_flags));
+	pos = cache_name_pos(cache, ce->name, htons(ce->ce_flags));
 
 	/* existing match? Just replace it */
 	if (pos >= 0) {
-		active_cache_changed = 1;
-		active_cache[pos] = ce;
+		set_cache_entry(cache, ce, pos);
 		return 0;
 	}
 	pos = -pos-1;
@@ -243,10 +301,10 @@
 	 * Inserting a merged entry ("stage 0") into the index
 	 * will always replace all non-merged entries..
 	 */
-	if (pos < active_nr && ce_stage(ce) == 0) {
-		while (same_name(active_cache[pos], ce)) {
+	if (pos < get_num_cache_entries(cache) && ce_stage(ce) == 0) {
+		while (ce_same_name(get_cache_entry(cache, pos), ce)) {
 			ok_to_add = 1;
-			if (!remove_entry_at(pos))
+			if (!remove_cache_entry_at(cache, pos))
 				break;
 		}
 	}
@@ -254,25 +312,24 @@
 	if (!ok_to_add)
 		return -1;
 
-	if (check_file_directory_conflict(ce, ok_to_replace)) {
+	if (check_file_directory_conflict(cache, ce, ok_to_replace)) {
 		if (!ok_to_replace)
 			return -1;
-		pos = cache_name_pos(ce->name, htons(ce->ce_flags));
+		pos = cache_name_pos(cache, ce->name, htons(ce->ce_flags));
 		pos = -pos-1;
 	}
 
 	/* Make sure the array is big enough .. */
-	if (active_nr == active_alloc) {
-		active_alloc = alloc_nr(active_alloc);
-		active_cache = xrealloc(active_cache, active_alloc * sizeof(struct cache_entry *));
+	if (cache->num_entries == cache->allocated_entries) {
+		cache->allocated_entries = alloc_nr(cache->allocated_entries);
+		cache->entries = xrealloc(cache->entries, cache->allocated_entries * sizeof(struct cache_entry *));
 	}
 
 	/* Add it in.. */
-	active_nr++;
-	if (active_nr > pos)
-		memmove(active_cache + pos + 1, active_cache + pos, (active_nr - pos - 1) * sizeof(ce));
-	active_cache[pos] = ce;
-	active_cache_changed = 1;
+	cache->num_entries++;
+	if (cache->num_entries > pos)
+		memmove(cache->entries + pos + 1, cache->entries + pos, (cache->num_entries - pos - 1) * sizeof(ce));
+	set_cache_entry(cache, ce, pos);
 	return 0;
 }
 
@@ -293,54 +350,56 @@
 	return 0;
 }
 
-int read_cache(void)
+struct cache *read_cache(void)
 {
 	int fd, i;
 	struct stat st;
-	unsigned long size, offset;
-	void *map;
-	struct cache_header *hdr;
-
-	errno = EBUSY;
-	if (active_cache)
-		return error("more than one cachefile");
+	unsigned long offset;
+	struct cache *cache = new_cache();
+
 	errno = ENOENT;
 	fd = open(get_index_file(), O_RDONLY);
-	if (fd < 0)
-		return (errno == ENOENT) ? 0 : error("open failed");
+	if (fd < 0) {
+		if (errno == ENOENT)
+			return cache;
+		else {
+			free_cache(cache);
+			return NULL;
+		}
+	}
 
-	size = 0; // avoid gcc warning
-	map = (void *)-1;
 	if (!fstat(fd, &st)) {
-		size = st.st_size;
+		cache->map.size = st.st_size;
 		errno = EINVAL;
-		if (size >= sizeof(struct cache_header) + 20)
-			map = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
+		if (cache->map.size >= sizeof(struct cache_header) + 20)
+			cache->map.ptr = mmap(NULL, cache->map.size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
 	}
 	close(fd);
-	if (-1 == (int)(long)map)
-		return error("mmap failed");
+	if (MAP_FAILED == cache->map.ptr) {
+		error("mmap failed");
+		free_cache(cache);
+		return NULL;
+	}
 
-	hdr = map;
-	if (verify_hdr(hdr, size) < 0)
-		goto unmap;
-
-	active_nr = ntohl(hdr->hdr_entries);
-	active_alloc = alloc_nr(active_nr);
-	active_cache = calloc(active_alloc, sizeof(struct cache_entry *));
-
-	offset = sizeof(*hdr);
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = map + offset;
-		offset = offset + ce_size(ce);
-		active_cache[i] = ce;
+	cache->header = cache->map.ptr;
+	if (verify_hdr(cache->header, cache->map.size) < 0) {
+		free_cache(cache);
+		errno = EINVAL;
+		error("verify header failed");
+		return NULL;
 	}
-	return active_nr;
 
-unmap:
-	munmap(map, size);
-	errno = EINVAL;
-	return error("verify header failed");
+	cache->num_entries = ntohl(cache->header->hdr_entries);
+	cache->allocated_entries = alloc_nr(cache->num_entries);
+	cache->entries = xcalloc(cache->allocated_entries, sizeof(struct cache_entry *));
+
+	offset = sizeof(*cache->header);
+	for (i = 0; i < cache->num_entries; i++) {
+		struct cache_entry *ce = cache->map.ptr + offset;
+		offset = offset + ce_size(ce);
+		cache->entries[i] = ce;
+	}
+	return cache;
 }
 
 #define WRITE_BUFFER_SIZE 8192
@@ -386,7 +445,7 @@
 	return 0;
 }
 
-int write_cache(int newfd, struct cache_entry **cache, int entries)
+int write_cache(struct cache *cache, int newfd)
 {
 	SHA_CTX c;
 	struct cache_header hdr;
@@ -394,14 +453,14 @@
 
 	hdr.hdr_signature = htonl(CACHE_SIGNATURE);
 	hdr.hdr_version = htonl(2);
-	hdr.hdr_entries = htonl(entries);
+	hdr.hdr_entries = htonl(get_num_cache_entries(cache));
 
 	SHA1_Init(&c);
 	if (ce_write(&c, newfd, &hdr, sizeof(hdr)) < 0)
 		return -1;
 
-	for (i = 0; i < entries; i++) {
-		struct cache_entry *ce = cache[i];
+	for (i = 0; i < get_num_cache_entries(cache); i++) {
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		if (ce_write(&c, newfd, ce, ce_size(ce)) < 0)
 			return -1;
 	}
Index: read-tree.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/read-tree.c  (mode:100644)
+++ uncommitted/read-tree.c  (mode:100644)
@@ -7,7 +7,7 @@
 
 static int stage = 0;
 
-static int unpack_tree(unsigned char *sha1)
+static int unpack_tree(struct cache *cache, unsigned char *sha1)
 {
 	void *buffer;
 	unsigned long size;
@@ -16,7 +16,7 @@
 	buffer = read_object_with_reference(sha1, "tree", &size, 0);
 	if (!buffer)
 		return -1;
-	ret = read_tree(buffer, size, stage);
+	ret = read_tree(cache, buffer, size, stage);
 	free(buffer);
 	return ret;
 }
@@ -93,26 +93,30 @@
 	return NULL;
 }
 
-static void trivially_merge_cache(struct cache_entry **src, int nr)
+/* rather than doing the 'right' thing of deleting entries as we merge,
+ * walk dst through the cache, overwriting entries as we go and at the
+ * end truncate the size of the cache */
+static void trivially_merge_cache(struct cache *cache)
 {
 	static struct cache_entry null_entry;
-	struct cache_entry **dst = src;
 	struct cache_entry *old = &null_entry;
+	int src = 0, dst = 0, nr = get_num_cache_entries(cache);
 
-	while (nr) {
+	while (src < nr) {
 		struct cache_entry *ce, *result;
 
-		ce = src[0];
+		ce = get_cache_entry(cache, src);
 
 		/* We throw away original cache entries except for the stat information */
 		if (!ce_stage(ce)) {
 			old = ce;
 			src++;
-			nr--;
-			active_nr--;
 			continue;
 		}
-		if (nr > 2 && (result = merge_entries(ce, src[1], src[2])) != NULL) {
+		if ((src < (nr - 2)) &&
+		    (result = merge_entries(ce,
+					    get_cache_entry(cache, src + 1),
+					    get_cache_entry(cache, src + 2))) != NULL) {
 			/*
 			 * See if we can re-use the old CE directly?
 			 * That way we get the uptodate stat info.
@@ -122,40 +126,46 @@
 			ce = result;
 			ce->ce_flags &= ~htons(CE_STAGEMASK);
 			src += 2;
-			nr -= 2;
-			active_nr -= 2;
 		}
-		*dst++ = ce;
+		set_cache_entry(cache, ce, dst);
+		dst++;
 		src++;
+	}
+	/* this could be replaced by a truncate api */
+	while (nr > dst) {
 		nr--;
+		remove_cache_entry_at(cache, nr);
 	}
 }
 
-static void merge_stat_info(struct cache_entry **src, int nr)
+static void merge_stat_info(struct cache *cache)
 {
 	static struct cache_entry null_entry;
-	struct cache_entry **dst = src;
 	struct cache_entry *old = &null_entry;
+	int src = 0, dst = 0, nr = get_num_cache_entries(cache);
 
-	while (nr) {
+	while (src < nr) {
 		struct cache_entry *ce;
 
-		ce = src[0];
+		ce = get_cache_entry(cache, src);
 
 		/* We throw away original cache entries except for the stat information */
 		if (!ce_stage(ce)) {
 			old = ce;
 			src++;
-			nr--;
-			active_nr--;
 			continue;
 		}
 		if (path_matches(ce, old) && same(ce, old))
 			*ce = *old;
 		ce->ce_flags &= ~htons(CE_STAGEMASK);
-		*dst++ = ce;
+		set_cache_entry(cache, ce, dst);
+		dst++;
 		src++;
+	}
+	/* this could be replaced by a truncate api */
+	while (nr > dst) {
 		nr--;
+		remove_cache_entry_at(cache, nr);
 	}
 }
 
@@ -167,6 +177,7 @@
 	unsigned char sha1[20];
 	static char lockfile[MAXPATHLEN+1];
 	const char *indexfile = get_index_file();
+	struct cache *cache = NULL;
 
 	snprintf(lockfile, sizeof(lockfile), "%s.lock", indexfile);
 
@@ -185,9 +196,9 @@
 			int i;
 			if (stage)
 				die("-m needs to come first");
-			read_cache();
-			for (i = 0; i < active_nr; i++) {
-				if (ce_stage(active_cache[i]))
+			cache = read_cache();
+			for (i = 0; i < get_num_cache_entries(cache); i++) {
+				if (ce_stage(get_cache_entry(cache, i)))
 					die("you need to resolve your current index first");
 			}
 			stage = 1;
@@ -198,23 +209,25 @@
 			usage(read_tree_usage);
 		if (stage > 3)
 			usage(read_tree_usage);
-		if (unpack_tree(sha1) < 0)
+		if (!cache)
+			cache = new_cache();
+		if (unpack_tree(cache, sha1) < 0)
 			die("failed to unpack tree object %s", arg);
 		stage++;
 	}
 	if (merge) {
 		switch (stage) {
 		case 4:	/* Three-way merge */
-			trivially_merge_cache(active_cache, active_nr);
+			trivially_merge_cache(cache);
 			break;
 		case 2:	/* Just read a tree, merge with old cache contents */
-			merge_stat_info(active_cache, active_nr);
+			merge_stat_info(cache);
 			break;
 		default:
 			die("just how do you expect me to merge %d trees?", stage-1);
 		}
 	}
-	if (write_cache(newfd, active_cache, active_nr) || rename(lockfile, indexfile))
+	if (write_cache(cache, newfd) || rename(lockfile, indexfile))
 		die("unable to write new index file");
 	lockfile_name = NULL;
 	return 0;
Index: sha1_file.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/sha1_file.c  (mode:100644)
+++ uncommitted/sha1_file.c  (mode:100644)
@@ -302,7 +302,7 @@
 	}
 	map = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
 	close(fd);
-	if (-1 == (int)(long)map)
+	if (MAP_FAILED == map)
 		return NULL;
 	*size = st.st_size;
 	return map;
@@ -587,7 +587,7 @@
 	if (size)
 		buf = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
 	close(fd);
-	if ((int)(long)buf == -1)
+	if (buf == MAP_FAILED)
 		return -1;
 
 	ret = write_sha1_file(buf, size, "blob", sha1);
Index: tree.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/tree.c  (mode:100644)
+++ uncommitted/tree.c  (mode:100644)
@@ -5,7 +5,8 @@
 
 const char *tree_type = "tree";
 
-static int read_one_entry(unsigned char *sha1, const char *base, int baselen, const char *pathname, unsigned mode, int stage)
+static int read_one_entry(struct cache *cache, unsigned char *sha1, const char *base,
+		          int baselen, const char *pathname, unsigned mode, int stage)
 {
 	int len = strlen(pathname);
 	unsigned int size = cache_entry_size(baselen + len);
@@ -18,10 +19,10 @@
 	memcpy(ce->name, base, baselen);
 	memcpy(ce->name + baselen, pathname, len+1);
 	memcpy(ce->sha1, sha1, 20);
-	return add_cache_entry(ce, ADD_CACHE_OK_TO_ADD);
+	return add_cache_entry(cache, ce, ADD_CACHE_OK_TO_ADD);
 }
 
-static int read_tree_recursive(void *buffer, unsigned long size,
+static int read_tree_recursive(struct cache* cache, void *buffer, unsigned long size,
 			       const char *base, int baselen, int stage)
 {
 	while (size) {
@@ -53,7 +54,7 @@
 			memcpy(newbase, base, baselen);
 			memcpy(newbase + baselen, path, pathlen);
 			newbase[baselen + pathlen] = '/';
-			retval = read_tree_recursive(eltbuf, eltsize,
+			retval = read_tree_recursive(cache, eltbuf, eltsize,
 						     newbase,
 						     baselen + pathlen + 1, stage);
 			free(eltbuf);
@@ -62,15 +63,15 @@
 				return -1;
 			continue;
 		}
-		if (read_one_entry(sha1, base, baselen, path, mode, stage) < 0)
+		if (read_one_entry(cache, sha1, base, baselen, path, mode, stage) < 0)
 			return -1;
 	}
 	return 0;
 }
 
-int read_tree(void *buffer, unsigned long size, int stage)
+int read_tree(struct cache *cache, void *buffer, unsigned long size, int stage)
 {
-	return read_tree_recursive(buffer, size, "", 0, stage);
+	return read_tree_recursive(cache, buffer, size, "", 0, stage);
 }
 
 struct tree *lookup_tree(unsigned char *sha1)
Index: update-cache.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/update-cache.c  (mode:100644)
+++ uncommitted/update-cache.c  (mode:100644)
@@ -51,7 +51,7 @@
 	ce->ce_size = htonl(st->st_size);
 }
 
-static int add_file_to_cache(char *path)
+static int add_file_to_cache(struct cache *cache, char *path)
 {
 	int size, namelen, option, status;
 	struct cache_entry *ce;
@@ -71,7 +71,7 @@
 		 */
 		if (status == 0 || (errno == ENOENT || errno == ENOTDIR)) {
 			if (allow_remove)
-				return remove_file_from_cache(path);
+				return remove_file_from_cache(cache, path);
 		}
 		return error("open(\"%s\"): %s", path, strerror(errno));
 	}
@@ -106,7 +106,7 @@
 	}
 	option = allow_add ? ADD_CACHE_OK_TO_ADD : 0;
 	option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0;
-	return add_cache_entry(ce, option);
+	return add_cache_entry(cache, ce, option);
 }
 
 static int match_data(int fd, void *buffer, unsigned long size)
@@ -191,7 +191,7 @@
 	if (lstat(ce->name, &st) < 0)
 		return ERR_PTR(-errno);
 
-	changed = cache_match_stat(ce, &st);
+	changed = ce_match_stat(ce, &st);
 	if (!changed)
 		return ce;
 
@@ -222,19 +222,19 @@
 	return updated;
 }
 
-static int refresh_cache(void)
+static int refresh_cache(struct cache *cache)
 {
 	int i;
 	int has_errors = 0;
 
-	for (i = 0; i < active_nr; i++) {
+	for (i = 0; i < get_num_cache_entries(cache); i++) {
 		struct cache_entry *ce, *new;
-		ce = active_cache[i];
+		ce = get_cache_entry(cache, i);
 		if (ce_stage(ce)) {
 			printf("%s: needs merge\n", ce->name);
 			has_errors = 1;
-			while ((i < active_nr) &&
-			       ! strcmp(active_cache[i]->name, ce->name))
+			while ((i < get_num_cache_entries(cache)) &&
+			       ! strcmp(get_cache_entry(cache, i)->name, ce->name))
 				i++;
 			i--;
 			continue;
@@ -248,11 +248,7 @@
 			}
 			continue;
 		}
-		active_cache_changed = 1;
-		/* You can NOT just free active_cache[i] here, since it
-		 * might not be necessarily malloc()ed but can also come
-		 * from mmap(). */
-		active_cache[i] = new;
+		set_cache_entry(cache, new, i);
 	}
 	return has_errors;
 }
@@ -285,7 +281,7 @@
 	}
 }
 
-static int add_cacheinfo(char *arg1, char *arg2, char *arg3)
+static int add_cacheinfo(struct cache *cache, char *arg1, char *arg2, char *arg3)
 {
 	int size, len, option;
 	unsigned int mode;
@@ -310,7 +306,7 @@
 	ce->ce_mode = create_ce_mode(mode);
 	option = allow_add ? ADD_CACHE_OK_TO_ADD : 0;
 	option |= allow_replace ? ADD_CACHE_OK_TO_REPLACE : 0;
-	return add_cache_entry(ce, option);
+	return add_cache_entry(cache, ce, option);
 }
 
 static const char *lockfile_name = NULL;
@@ -328,10 +324,11 @@
 
 int main(int argc, char **argv)
 {
-	int i, newfd, entries, has_errors = 0;
+	int i, newfd, has_errors = 0;
 	int allow_options = 1;
 	static char lockfile[MAXPATHLEN+1];
 	const char *indexfile = get_index_file();
+	struct cache *cache = NULL;
 
 	snprintf(lockfile, sizeof(lockfile), "%s.lock", indexfile);
 
@@ -343,8 +340,8 @@
 	atexit(remove_lock_file);
 	lockfile_name = lockfile;
 
-	entries = read_cache();
-	if (entries < 0)
+	cache = read_cache();
+	if (!cache)
 		die("cache corrupted");
 
 	for (i = 1 ; i < argc; i++) {
@@ -368,13 +365,13 @@
 				continue;
 			}
 			if (!strcmp(path, "--refresh")) {
-				has_errors |= refresh_cache();
+				has_errors |= refresh_cache(cache);
 				continue;
 			}
 			if (!strcmp(path, "--cacheinfo")) {
 				if (i+3 >= argc)
 					die("update-cache: --cacheinfo <mode> <sha1> <path>");
-				if (add_cacheinfo(argv[i+1], argv[i+2], argv[i+3]))
+				if (add_cacheinfo(cache, argv[i+1], argv[i+2], argv[i+3]))
 					die("update-cache: --cacheinfo cannot add %s", argv[i+3]);
 				i += 3;
 				continue;
@@ -382,7 +379,7 @@
 			if (!strcmp(path, "--force-remove")) {
 				if (argc <= i + 1)
 					die("update-cache: --force-remove <path>");
-				if (remove_file_from_cache(argv[i+1]))
+				if (remove_file_from_cache(cache, argv[i+1]))
 					die("update-cache: --force-remove cannot remove %s", argv[i+1]);
 				i++;
 				continue;
@@ -398,10 +395,10 @@
 			fprintf(stderr, "Ignoring path %s\n", argv[i]);
 			continue;
 		}
-		if (add_file_to_cache(path))
+		if (add_file_to_cache(cache, path))
 			die("Unable to add %s to database", path);
 	}
-	if (write_cache(newfd, active_cache, active_nr) || rename(lockfile, indexfile))
+	if (write_cache(cache, newfd) || rename(lockfile, indexfile))
 		die("Unable to write new cachefile");
 
 	lockfile_name = NULL;
Index: write-tree.c
===================================================================
--- c8c77f67ea85ee65a0f055f2cb76e6059eaa0776/write-tree.c  (mode:100644)
+++ uncommitted/write-tree.c  (mode:100644)
@@ -17,7 +17,7 @@
 	return ret;
 }
 
-static int write_tree(struct cache_entry **cachep, int maxentries, const char *base, int baselen, unsigned char *returnsha1)
+static int write_tree(struct cache *cache, int start_pos, const char *base, int baselen, unsigned char *returnsha1)
 {
 	unsigned char subdir_sha1[20];
 	unsigned long size, offset;
@@ -30,8 +30,8 @@
 	offset = 0;
 
 	nr = 0;
-	while (nr < maxentries) {
-		struct cache_entry *ce = cachep[nr];
+	while ((start_pos + nr) < get_num_cache_entries(cache)) {
+		struct cache_entry *ce = get_cache_entry(cache, start_pos + nr);
 		const char *pathname = ce->name, *filename, *dirname;
 		int pathlen = ce_namelen(ce), entrylen;
 		unsigned char *sha1;
@@ -41,16 +41,13 @@
 		if (baselen >= pathlen || memcmp(base, pathname, baselen))
 			break;
 
-		sha1 = ce->sha1;
-		mode = ntohl(ce->ce_mode);
-
 		/* Do we have _further_ subdirectories? */
 		filename = pathname + baselen;
 		dirname = strchr(filename, '/');
 		if (dirname) {
 			int subdir_written;
 
-			subdir_written = write_tree(cachep + nr, maxentries - nr, pathname, dirname-pathname+1, subdir_sha1);
+			subdir_written = write_tree(cache, start_pos + nr, pathname, dirname-pathname+1, subdir_sha1);
 			nr += subdir_written;
 
 			/* Now we need to write out the directory entry into this tree.. */
@@ -60,6 +57,9 @@
 			/* ..but the directory entry doesn't count towards the total count */
 			nr--;
 			sha1 = subdir_sha1;
+		} else {
+			sha1 = ce->sha1;
+			mode = ntohl(ce->ce_mode);
 		}
 
 		if (check_valid_sha1(sha1) < 0)
@@ -85,16 +85,19 @@
 int main(int argc, char **argv)
 {
 	int i, funny;
-	int entries = read_cache();
+	struct cache *cache = read_cache();
+	int entries;
 	unsigned char sha1[20];
 
-	if (entries < 0)
+	if (!cache)
 		die("write-tree: error reading cache");
 
+	entries = get_num_cache_entries(cache);
+
 	/* Verify that the tree is merged */
 	funny = 0;
 	for (i = 0; i < entries; i++) {
-		struct cache_entry *ce = active_cache[i];
+		struct cache_entry *ce = get_cache_entry(cache, i);
 		if (ntohs(ce->ce_flags) & ~CE_NAMEMASK) {
 			if (10 < ++funny) {
 				fprintf(stderr, "...\n");
@@ -116,8 +119,8 @@
 		 * the cache is sorted.  Also path can appear only once,
 		 * which means conflicting one would immediately follow.
 		 */
-		const char *this_name = active_cache[i]->name;
-		const char *next_name = active_cache[i+1]->name;
+		const char *this_name = get_cache_entry(cache, i)->name;
+		const char *next_name = get_cache_entry(cache, i+1)->name;
 		int this_len = strlen(this_name);
 		if (this_len < strlen(next_name) &&
 		    strncmp(this_name, next_name, this_len) == 0 &&
@@ -134,7 +137,7 @@
 		die("write-tree: not able to write tree");
 
 	/* Ok, write it out */
-	if (write_tree(active_cache, entries, "", 0, sha1) != entries)
+	if (write_tree(cache, 0, "", 0, sha1) != entries)
 		die("write-tree: internal error");
 	printf("%s\n", sha1_to_hex(sha1));
 	return 0;

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Sun, May 15, 2005 at 04:04:25AM CEST, I got a letter
where Brad Roberts <braddr@puremagic.com> told me that...
> > I wasn't able to finish redoing these against linus tip, but I got most of
> > it done (patches 1-14 of the original 19):
> >
> >   http://gameboy2.puremagic.com:8090/
> >   rsync://gameboy2.puremagic.com/git/
> >
> > The second, third, and forth to last changes need a careful review,
> > they're direct applications of the original patches which were lightly
> > tested during the first round and nothing other than compile tested in
> > this round.
> >
> > I suspect the remaining parts of the original patch series will go in
> > fairly smoothly.  If no one gets to them before tonight I'll finish
> > it up after work.
> >
> > Later,
> > Brad
> 
> I've completed the re-merge, and moved to tip of git-pb.git rather than
> tip of git.git.  Unfortunatly that merge was also somewhat intrusive and
> my individual diffs along the way are somewhat useless now.  The entire
> history is available about the above locations still.  Attached is the
> full diff vs git-pb @ 902b92e00e491a60d55c4b2bce122903b8347f34.

I've merged some of the minor stuff for now.

> 2) Should the index changing areas be constructing a new index instead of
> shuffling bits within the current index?

When I have a big cache (the only time it matters), I do usually only
relatively small changes to it, so...

> 3) The vocabulary and code is inconsistent between cache and index.

Yes...

> 4) read-cache.c does much more than reading.

and yes. And cache.h is full of crap. Perhaps we could move read-cache.c
to cache.c?


I'd imagine the plan of attack to continue by changing active_cache to
be struct cache, then making it local.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Brad Roberts]

> Date: Sun, 15 May 2005 13:48:47 +0200
> From: Petr Baudis <pasky@ucw.cz>
> To: Brad Roberts <braddr@puremagic.com>
> Cc: Juliusz Chroboczek <Juliusz.Chroboczek@pps.jussieu.fr>,
>      git@vger.kernel.org
> Subject: Re: Darcs-git: a few notes for Git hackers
>
> Dear diary, on Sun, May 15, 2005 at 04:04:25AM CEST, I got a letter
> where Brad Roberts <braddr@puremagic.com> told me that...
> > > I wasn't able to finish redoing these against linus tip, but I got most of
> > > it done (patches 1-14 of the original 19):
> > >
> > >   http://gameboy2.puremagic.com:8090/
> > >   rsync://gameboy2.puremagic.com/git/
> > >
> > > The second, third, and forth to last changes need a careful review,
> > > they're direct applications of the original patches which were lightly
> > > tested during the first round and nothing other than compile tested in
> > > this round.
> > >
> > > I suspect the remaining parts of the original patch series will go in
> > > fairly smoothly.  If no one gets to them before tonight I'll finish
> > > it up after work.
> > >
> > > Later,
> > > Brad
> >
> > I've completed the re-merge, and moved to tip of git-pb.git rather than
> > tip of git.git.  Unfortunatly that merge was also somewhat intrusive and
> > my individual diffs along the way are somewhat useless now.  The entire
> > history is available about the above locations still.  Attached is the
> > full diff vs git-pb @ 902b92e00e491a60d55c4b2bce122903b8347f34.
>
> I've merged some of the minor stuff for now.

Cool, though there appears to have been some objections. :)

> > 2) Should the index changing areas be constructing a new index instead of
> > shuffling bits within the current index?
>
> When I have a big cache (the only time it matters), I do usually only
> relatively small changes to it, so...

The entire index is bit shuffled around even if nothing changed.  At least
today, size and amount changed doesn't matter.

> > 3) The vocabulary and code is inconsistent between cache and index.
>
> Yes...
>
> > 4) read-cache.c does much more than reading.
>
> and yes. And cache.h is full of crap. Perhaps we could move read-cache.c
> to cache.c?

At least parts of it, probably yes.

> I'd imagine the plan of attack to continue by changing active_cache to
> be struct cache, then making it local.

Which is what the rest of that patch does.

Thanks for looking at this.

Later,
Brad

Re: Darcs-git: a few notes for Git hackers

[Junio C Hamano]

>>>>> "BR" == Brad Roberts <braddr@puremagic.com> writes:

>> I've merged some of the minor stuff for now.

BR> Cool, though there appears to have been some objections. :)

I do not have any problem with what the _patch_ does at all.  I
had more trouble in the process of how the patch appeared in
git-pb tree, and I still do.

Please consider the revert request retracted.  Request to
forewarn people in the mailing list still stands.

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Sun, May 15, 2005 at 09:25:06PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "BR" == Brad Roberts <braddr@puremagic.com> writes:
> 
> >> I've merged some of the minor stuff for now.
> 
> BR> Cool, though there appears to have been some objections. :)
> 
> I do not have any problem with what the _patch_ does at all.  I
> had more trouble in the process of how the patch appeared in
> git-pb tree, and I still do.
> 
> Please consider the revert request retracted.  Request to
> forewarn people in the mailing list still stands.

Well, it isn't like this was some huge large-scale change; the diff is
quite small. Do you want me to announce _any_ identifier renames in
advance on the mailing list? Or where should be the threshold? These
were three renames of not-so-frequently used identifiers.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Brad Roberts]

> Date: Sun, 15 May 2005 21:48:03 +0200
> From: Petr Baudis <pasky@ucw.cz>
> To: Junio C Hamano <junkio@cox.net>
> Cc: Brad Roberts <braddr@puremagic.com>,
>      Juliusz Chroboczek <Juliusz.Chroboczek@pps.jussieu.fr>,
>      git@vger.kernel.org
> Subject: Re: Darcs-git: a few notes for Git hackers
>
> Dear diary, on Sun, May 15, 2005 at 09:25:06PM CEST, I got a letter
> where Junio C Hamano <junkio@cox.net> told me that...
> > >>>>> "BR" == Brad Roberts <braddr@puremagic.com> writes:
> >
> > >> I've merged some of the minor stuff for now.
> >
> > BR> Cool, though there appears to have been some objections. :)
> >
> > I do not have any problem with what the _patch_ does at all.  I
> > had more trouble in the process of how the patch appeared in
> > git-pb tree, and I still do.
> >
> > Please consider the revert request retracted.  Request to
> > forewarn people in the mailing list still stands.
>
> Well, it isn't like this was some huge large-scale change; the diff is
> quite small. Do you want me to announce _any_ identifier renames in
> advance on the mailing list? Or where should be the threshold? These
> were three renames of not-so-frequently used identifiers.

Additionally, all of these changes were posted originally on 4/22, this
was just a bring-forward of them as requested.

Later,
Brad

Re: Darcs-git: a few notes for Git hackers

[Junio C Hamano]

>>>>> "BR" == Brad Roberts <braddr@puremagic.com> writes:

BR> Additionally, all of these changes were posted originally on 4/22, this
BR> was just a bring-forward of them as requested.

I admit I overreacted without first seeing the extent of damage.
Sorry I made too big a fuss about this.

The 4/22 one I found very good and expected it to be in good
shape when finished.  I simply did not expect that to be taken
piecemeal like Petr did.  I overreacted, thinking he did so
_without_ considering much about possible conflicts.

The request to _continue_ using good judgements when to forewarn
people still stands ;-).  In this case, Petr used good
judgement.  My apologies to both of you if my knee-jerk reaction
caused you bad feelings.

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Sun, May 15, 2005 at 10:36:54PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> The request to _continue_ using good judgements when to forewarn
> people still stands ;-).  In this case, Petr used good
> judgement.  My apologies to both of you if my knee-jerk reaction
> caused you bad feelings.

No problem here. I'm of course going to consult the mailing list on any
larger-scale code changes.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Petr Baudis]

Dear diary, on Sun, May 15, 2005 at 09:06:27PM CEST, I got a letter
where Brad Roberts <braddr@puremagic.com> told me that...
> > > 2) Should the index changing areas be constructing a new index instead of
> > > shuffling bits within the current index?
> >
> > When I have a big cache (the only time it matters), I do usually only
> > relatively small changes to it, so...
> 
> The entire index is bit shuffled around even if nothing changed.  At least
> today, size and amount changed doesn't matter.

At the very least, it probably shouldn't get shuffled around if there
are no changes whatsoever.

> > I'd imagine the plan of attack to continue by changing active_cache to
> > be struct cache, then making it local.
> 
> Which is what the rest of that patch does.

Well, the important word there was "then". :-) I think the patch is too
big, do you think it would be possible to separate it to those two
stages? (I could do it on my own when I get enough time, but who knows
when that happens... ;-)

Thanks,

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Darcs-git: a few notes for Git hackers

[Daniel Barkalow]

On Mon, 9 May 2005, Juliusz Chroboczek wrote:

> Hi,
> 
> Here are a few notes about Git that should probably be taken into
> account by people working on Git itself or on Git wrappers.  The notes
> apply to Linus' Git-0.6, which is the code I'm using in Darcs-git;
> some of them might no longer be applicable to Darcs.
> 
> 
> 1. Darcs-git uses the fact that Git updates are atomic when reading
> from a Git repository.  Darcs-git almost writes to Git repositories
> atomically, with one exception: it performs a non-atomic
> read/update/write cycle on .git/HEAD.
> 
> For that reason, I'm taking a high-level lock on .git repositories
> whenever I write them.  The lockfile is ``.git/lock''.  I haven't
> thought about whether Darcs can be easily coerced into accessing Git
> repos atomically; have people writing Git wrappers found the need for
> a global lock?

I think most things are using the O_CREAT | O_EXCL write to a file and
then rename or link/unlink to the desired location. I have some code to do
this with refs/*/* as well, and I think people have generally settled on
symlinking HEAD to something in refs/heads/. So it shouldn't be necessary
to lock the whole repository, unless you're doing some operation like
swapping two heads.

> 2. The files git.h and git.c in Darcs-git are a simple ``libgit'' that
> contains just enough functionality for Darcs-git; they use the
> functionality of sha1_file.c and read_cache.c from Git-0.6.
> 
> I've found a few problems with the interfaces in these files:
> 
>  - the global variables sha1_file_directory, active_cache, active_nr
>    and active_alloc are not marked ``extern'' in cache.h.  This breaks
>    linkers that don't grok common symbols, such as the one in GHCi
>    (silly GHCi).

Should be trivial to fix.

>  - the function write_sha1_file takes the metadata and the data in a
>    contiguous buffer, which is a problem when the data has been
>    allocated by a higher layer.  I'm currently working around the
>    problem by memcpy-ing everything into a temp buffer, but that's
>    obviously not a good thing.  I don't care whether write_sha1_file
>    is changed to use a writev-like interface, or to take the metadata
>    explicitly (as in char *type, unsigned long length).

I've got some patches to make new functions of the write_sha1_file sort
easier to write cleanly (for making git-*-pull clean); it wouldn't be too
hard to have an open/write/close set.

>  - there is no (usable) function to write a tree; there's the code in
>    write_tree.c, but it's not generally useful.  See the function
>    ``git_write_tree_done'' in git.c for the type of interface I'm
>    thinking of.

I'm working on making this cleaner. Are you wanting to write a tree from
something other than a cache?

I can post my patches, but Linus is on vacation, so they couldn't go into
the mainline until Friday or so anyway.

	-Daniel
*This .sig left intentionally blank*