From: "Shawn O. Pearce" <spearce@spearce.org>
To: Robin Rosenberg <robin.rosenberg@dewire.com>
Cc: git@vger.kernel.org
Subject: [EGIT PATCH 00/26] New DirCache API
Date: Mon, 11 Aug 2008 18:07:47 -0700 [thread overview]
Message-ID: <1218503293-14057-1-git-send-email-spearce@spearce.org> (raw)
This is (finally) my long-promised DirCache API. It has been evolving
and growing since before April I think. Its finally cleaned up enough
for public viewing and usage.
I started this change because I didn't see an easy way to plug
GitIndex into TreeWalk, I found GitIndex to be too slow, and it
required direct access to the working directory in order to handle
certain operations. DirCache has none of these issues.
The API is meant to replace GitIndex, as we can now merge-sort a 'DIRC'
(aka .git/index) file against any other tree as part of any active
TreeWalk instance. This makes it really easy to do n-way diffs against
commits, one or more index files, and the working directory.
The implementation is leaner and meaner than GitIndex. It avoids a
lot of unnecessary conversion overheads, and it tries to reduce the
per-entry memory usage as much as possible.
We almost support the 'TREE' cache extension. Updates to the index
through DirCacheBuilder or DirCacheEditor wind up throwing away the
entire 'TREE' extension, instead of invalidating only those trees
we know to be dirty. This is certainly suboptimal and is an area
for future improvement.
>From here I'm going to start building 3 way merge support onto
this API, by taking advantage of TreeWalk to do the parallel
tree unpacks necessary for the merge operation.
Yes, its ~4.5k lines (excluding some test vectors). However it
will allow us to eventually remove about 2k lines from jgit alone:
AbstractIndexTreeVisitor 74
GitIndex 912
IndexDiff 160
IndexTreeWalker 241
IndexTreeVisitor 97
TreeIterator 203
WorkDirCheckout 401
----------------------------
2088
Other classes like GitResourceDecorator simplify I think as a result
of this new API being available. I have only started to port that
over, but found I was getting side-tracked in porting the existing
GitIndex users and I really want to focus on the merge code instead.
So overall we may see about a 2k line increase as a result of this
API being used in the tree rather than GitIndex, but since it fits
into the overall TreeWalk architecture its better suited for reuse,
so we may see more than 2k saved over time.
Shawn O. Pearce (26):
Force all source code to UTF-8 encoding by default
Protect WorkingTreeIterator's name encoding from weird ByteBuffers
Add Constants.encode as a utility for quick encoding in UTF-8
Rely upon Constants.CHARSET over Constants.CHARACTER_ENCODING
Allow AbstractTreeIterators to find out about StopWalkExceptions
Implement a new .git/index (aka dircache) read interface
Export the new DirCache API to Eclipse plugins using jgit
Support locking (and unlocking) a .git/index through DirCache
Support writing a .git/index through DirCache
Support the 'TREE' extension in .git/index through DirCache
Support using a DirCache within a TreeWalk
Support recreating a .git/index through DirCache
Support iterating and building a DirCache at the same time
Support creating a new DirCacheEntry for an arbitrary path
Support a simplified model of editing index entries
Support recursively getting all entries under a subtree path
Support copying meta fields from one DirCacheEntry to another
Add JUnit tests for new DirCache API
Add JUnit tests for DirCache compatibility with C Git
Allow the new DirCacheIterator in command line arguments
Add debugging commands to interact with the new DirCache code
Add a basic command line implementation of rm
Rewrite GitMoveDeleteHook to use DirCacheBuilder
Teach GitMoveDeleteHook how to move a folder recursively
Rewrite UntrackOperation to use DirCacheBuilder
Rewrite AssumeUnchangedOperation to use DirCache
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../org/spearce/egit/core/GitMoveDeleteHook.java | 156 ++-
.../egit/core/op/AssumeUnchangedOperation.java | 133 +-
.../org/spearce/egit/core/op/UntrackOperation.java | 98 +-
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../services/org.spearce.jgit.pgm.TextBuiltin | 6 +
.../src/org/spearce/jgit/pgm/Rm.java | 92 ++
.../org/spearce/jgit/pgm/debug/MakeCacheTree.java | 67 +
.../org/spearce/jgit/pgm/debug/ReadDirCache.java | 53 +
.../org/spearce/jgit/pgm/debug/ShowCacheTree.java | 69 +
.../org/spearce/jgit/pgm/debug/ShowDirCache.java | 72 +
.../org/spearce/jgit/pgm/debug/WriteDirCache.java | 54 +
.../jgit/pgm/opt/AbstractTreeIteratorHandler.java | 13 +
.../.settings/org.eclipse.core.resources.prefs | 3 +
.../spearce/jgit/dircache/DirCacheBasicTest.java | 185 +++
.../jgit/dircache/DirCacheBuilderIteratorTest.java | 91 ++
.../spearce/jgit/dircache/DirCacheBuilderTest.java | 253 ++++
.../dircache/DirCacheCGitCompatabilityTest.java | 204 +++
.../spearce/jgit/dircache/DirCacheFindTest.java | 86 ++
.../jgit/dircache/DirCacheIteratorTest.java | 273 ++++
.../spearce/jgit/dircache/DirCacheTreeTest.java | 150 ++
.../tst/org/spearce/jgit/dircache/gitgit.index | Bin 0 -> 134799 bytes
.../tst/org/spearce/jgit/dircache/gitgit.lsfiles | 1437 ++++++++++++++++++++
.../tst/org/spearce/jgit/dircache/gitgit.lstree | 331 +++++
.../spearce/jgit/lib/ConstantsEncodingTest.java | 89 ++
.../.settings/org.eclipse.core.resources.prefs | 3 +
org.spearce.jgit/META-INF/MANIFEST.MF | 3 +-
.../spearce/jgit/dircache/BaseDirCacheEditor.java | 194 +++
.../src/org/spearce/jgit/dircache/DirCache.java | 681 +++++++++
.../jgit/dircache/DirCacheBuildIterator.java | 124 ++
.../org/spearce/jgit/dircache/DirCacheBuilder.java | 175 +++
.../org/spearce/jgit/dircache/DirCacheEditor.java | 265 ++++
.../org/spearce/jgit/dircache/DirCacheEntry.java | 385 ++++++
.../spearce/jgit/dircache/DirCacheIterator.java | 205 +++
.../org/spearce/jgit/dircache/DirCacheTree.java | 453 ++++++
.../src/org/spearce/jgit/lib/Constants.java | 25 +
.../src/org/spearce/jgit/lib/ObjectWriter.java | 2 +-
.../src/org/spearce/jgit/lib/RefDatabase.java | 9 +-
.../src/org/spearce/jgit/lib/RepositoryConfig.java | 4 +-
.../org/spearce/jgit/transport/PacketLineIn.java | 3 +-
.../jgit/transport/SideBandInputStream.java | 3 +-
.../spearce/jgit/transport/TransportBundle.java | 3 +-
.../spearce/jgit/transport/WalkPushConnection.java | 4 +-
.../jgit/transport/WalkRemoteObjectDatabase.java | 5 +-
.../jgit/treewalk/AbstractTreeIterator.java | 11 +
.../spearce/jgit/treewalk/EmptyTreeIterator.java | 6 +
.../src/org/spearce/jgit/treewalk/TreeWalk.java | 12 +-
.../spearce/jgit/treewalk/WorkingTreeIterator.java | 2 +-
.../spearce/jgit/treewalk/filter/PathFilter.java | 10 +-
54 files changed, 6333 insertions(+), 190 deletions(-)
create mode 100644 org.spearce.egit-feature/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.egit-updatesite/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.egit.core.test/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.egit.core/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.egit.ui/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.egit/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.jgit.pgm/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/Rm.java
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/debug/MakeCacheTree.java
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/debug/ReadDirCache.java
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/debug/ShowCacheTree.java
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/debug/ShowDirCache.java
create mode 100644 org.spearce.jgit.pgm/src/org/spearce/jgit/pgm/debug/WriteDirCache.java
create mode 100644 org.spearce.jgit.test/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheBasicTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheBuilderIteratorTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheBuilderTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheCGitCompatabilityTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheFindTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheIteratorTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/DirCacheTreeTest.java
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/gitgit.index
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/gitgit.lsfiles
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/dircache/gitgit.lstree
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/ConstantsEncodingTest.java
create mode 100644 org.spearce.jgit/.settings/org.eclipse.core.resources.prefs
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/BaseDirCacheEditor.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCache.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheBuildIterator.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheBuilder.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEditor.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheEntry.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheIterator.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheTree.java
next reply other threads:[~2008-08-12 1:09 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-08-12 1:07 Shawn O. Pearce [this message]
2008-08-12 1:07 ` [EGIT PATCH 01/26] Force all source code to UTF-8 encoding by default Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 02/26] Protect WorkingTreeIterator's name encoding from weird ByteBuffers Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 03/26] Add Constants.encode as a utility for quick encoding in UTF-8 Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 04/26] Rely upon Constants.CHARSET over Constants.CHARACTER_ENCODING Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 05/26] Allow AbstractTreeIterators to find out about StopWalkExceptions Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 06/26] Implement a new .git/index (aka dircache) read interface Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 07/26] Export the new DirCache API to Eclipse plugins using jgit Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 08/26] Support locking (and unlocking) a .git/index through DirCache Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 09/26] Support writing " Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 10/26] Support the 'TREE' extension in " Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 11/26] Support using a DirCache within a TreeWalk Shawn O. Pearce
2008-08-12 1:07 ` [EGIT PATCH 12/26] Support recreating a .git/index through DirCache Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 13/26] Support iterating and building a DirCache at the same time Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 14/26] Support creating a new DirCacheEntry for an arbitrary path Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 15/26] Support a simplified model of editing index entries Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 16/26] Support recursively getting all entries under a subtree path Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 17/26] Support copying meta fields from one DirCacheEntry to another Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 18/26] Add JUnit tests for new DirCache API Shawn O. Pearce
[not found] ` <1218503293-14057-20-git-send-email-spearce@spearce.org>
2008-08-12 1:08 ` [EGIT PATCH 20/26] Allow the new DirCacheIterator in command line arguments Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 21/26] Add debugging commands to interact with the new DirCache code Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 22/26] Add a basic command line implementation of rm Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 23/26] Rewrite GitMoveDeleteHook to use DirCacheBuilder Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 24/26] Teach GitMoveDeleteHook how to move a folder recursively Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 25/26] Rewrite UntrackOperation to use DirCacheBuilder Shawn O. Pearce
2008-08-12 1:08 ` [EGIT PATCH 26/26] Rewrite AssumeUnchangedOperation to use DirCache Shawn O. Pearce
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1218503293-14057-1-git-send-email-spearce@spearce.org \
--to=spearce@spearce.org \
--cc=git@vger.kernel.org \
--cc=robin.rosenberg@dewire.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).