[PATCH 0/7 (v5)] rev-cache

[PATCH 0/7 (v5)] rev-cache

[Nick Edelen]

SUGGESTED FOR 'PU':

Traversing objects is currently very costly, as every commit and tree must be
loaded and parsed.  Much time and energy could be saved by caching metadata and
topological info in an efficient, easily accessible manner.  Furthermore, this
could improve git's interfacing potential, by providing a condensed summary of
a repository's commit tree.

This is a series to implement such a revision caching mechanism, aptly named
rev-cache.  The series will provide:
 - a core API to manipulate and traverse caches
 - an integration into the internal revision walker
 - a porcelain front-end providing access to users and (shell) applications
 - a series of tests to verify/demonstrate correctness
 - documentation of the API, porcelain and core concepts

In cold starts rev-cache has sped up packing and walking by a factor of 4, and
over twice that on warm starts.  The mechanism is minimally intrusive: most of
the changes take place in seperate files, and only a handful of git's existing
functions are modified.

Slides from the presentation I gave at GitTogether'09 give a good overview of
the mechanism, usage and performance of rev-cache.  (well, they complement the
documentation well.  they're a bit bare without the dialogue...)  You can find
them at http://www.flickr.com/photos/sirnot/sets/72157623652754819/

The patchset is structured so that each patchfile represents a fully
functional, incremental development of rev-cache.  Unfortunately the addition
of several files makes for some obscenely large patchfiles.  I apologize and
regret that this is somewhat unavoidable; my only consolation is that
patchfiles are less 'patches' and more additions of files.

Hope you find this useful.

 - Nick

---
Hi everyone.  I'm sorry about the (very) belated patchset -- University has a
tendency of being very absorbing.  It's spring break now though, so I've
finally got around to finalizing rev-cache!  The only significant change from
last time (besides the bugfixes) is how structures are read/written --
excessive care is now taken to ensure a standardized file format.

This time round I've also got some swanky slides I made for GitTogether.  Hehe.

 Documentation/git-rev-cache.txt       |  194 +++
 Documentation/technical/rev-cache.txt |  634 +++++++++
 Makefile                              |    2 +
 builtin.h                             |    1 +
 builtin/gc.c                          |    9 +
 builtin/rev-cache.c                   |  339 +++++
 command-list.txt                      |    1 +
 commit.c                              |   36 +-
 git.c                                 |    1 +
 list-objects.c                        |   46 +-
 object.h                              |    3 +-
 rev-cache.c                           | 2468 +++++++++++++++++++++++++++++++++
 rev-cache.h                           |  123 ++
 revision.c                            |   90 +-
 revision.h                            |   44 +-
 t/t6019-rev-cache-list.sh             |  263 ++++
 16 files changed, 4227 insertions(+), 27 deletions(-)
 create mode 100644 Documentation/git-rev-cache.txt
 create mode 100644 Documentation/technical/rev-cache.txt
 create mode 100644 builtin/rev-cache.c
 create mode 100644 rev-cache.c
 create mode 100644 rev-cache.h
 create mode 100644 t/t6019-rev-cache-list.sh

Re: [PATCH 0/7 (v5)] rev-cache

[Björn Gustavsson]

2010/4/6 Junio C Hamano <gitster@pobox.com>:

> So don't take this as a complaint; it is unfair to call this a bug.  But
> it certainly feels like it is in the same area that can be improved.

Yes, I agree. I'll have a look.

-- 
Björn Gustavsson, Erlang/OTP, Ericsson AB

Re: [PATCH 0/7 (v5)] rev-cache

[Junio C Hamano]

In case people are wondering what the message Björn is responding to
said...

  From: Junio C Hamano <gitster@pobox.com>
  Subject: Re: [PATCH 0/7 (v5)] rev-cache
  Date: Mon, 05 Apr 2010 23:49:57 -0700
  References: <4BBA40B7.1060100@gmail.com>
  Message-ID: <7vaathjcru.fsf@alter.siamese.dyndns.org>

  Björn, I thought your "apply --whitespace=fix" patch was supposed to be
  able to cope with this series whose [2/7] introduces a new file with a
  trailing blank line "t/t6019-rev-cache-list.sh" (which is correctly fixed
  not to have that extra blank line at the end), and then a subsequent patch
  in the series [3/7] still tries to apply a patch assuming that the extra
  blank line was still there.

  But that is apparently not the case.  I think your series dealt only for
  the cases where the additional contents came _after_ the fixed-and-gone
  blank lines, and never dealt with the case where such lines appear in the
  context.

The message didn't go to vger because I botched while editing To/Cc lines.