[EGIT PATCH 00/11] Misc. cleanups and improvements

[EGIT PATCH 00/11] Misc. cleanups and improvements

[Shawn O. Pearce]

This really is two parts; the first fixes a couple of bugs and
the latter section expands our treewalk API to handle the Eclipse
workbench natively as well as expose APIs needed to integrate the
DIRC format (.git/index) into the treewalk structure.

Shawn O. Pearce (11):
  Fix RawParseUtils.endOfParagraph to work on all corner cases
  Add test case for the RevCommit parsing code
  Notify AbstractTreeIterator implementations of skipped tree entries
  Allow AbstractTreeIterator subclasses to supply their own path array
  Allow WorkingTreeIterators to define their prefix path when created
  Add getTree to TreeWalk for locating the current iterator instance
  Allow WorkingTreeIterator to track last modified time for entries
  Expose the current entry's length, last modified in
    WorkingTreeIterator
  Expose idBuffer,idOffset in AbstractTreeIterator to applications
  Add a TreeWalk iterator implementation for IContainer
  Teach NB how to encode/decode an unsigned 16 bit integer

 .../spearce/egit/core/ContainerTreeIterator.java   |  181 +++++++++++++++++++
 .../spearce/jgit/revwalk/RevCommitParseTest.java   |  190 ++++++++++++++++++++
 .../jgit/treewalk/AbstractTreeIterator.java        |   79 ++++++++-
 .../spearce/jgit/treewalk/CanonicalTreeParser.java |    4 +-
 .../spearce/jgit/treewalk/EmptyTreeIterator.java   |    4 +-
 .../spearce/jgit/treewalk/FileTreeIterator.java    |    9 +
 .../src/org/spearce/jgit/treewalk/TreeWalk.java    |   36 ++++-
 .../spearce/jgit/treewalk/WorkingTreeIterator.java |   56 ++++++-
 org.spearce.jgit/src/org/spearce/jgit/util/NB.java |   35 ++++
 .../src/org/spearce/jgit/util/RawParseUtils.java   |   11 +-
 10 files changed, 591 insertions(+), 14 deletions(-)
 create mode 100644 org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevCommitParseTest.java

[EGIT PATCH 01/11] Fix RawParseUtils.endOfParagraph to work on all corner cases

[Shawn O. Pearce]

We rely on this method to find the end of the "oneline" part of a
commit message, which is then used in displays like the History
view in EGit or in some tooltips.  However a number of weird commit
formats were throwing its parsing off, causing us to split the text
in a way we didn't expect (or want) to split it.

This change resolves the splitting by making sure we don't throw an
IndexOutOfBoundsException from RevCommit.getShortMessage due to the
end of the paragraph coming out before the beginning of it.  This
happened sometimes when the commit message was empty.

We also now properly trim the trailing LFs which end the paragraph,
as these were showing up sometimes as spaces at the end of a short
message string (as we do LF->space conversion prior to returning).

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../src/org/spearce/jgit/util/RawParseUtils.java   |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java b/org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java
index f268ffc..dbc2e83 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java
@@ -422,20 +422,21 @@ public final class RawParseUtils {
 	 * 
 	 * @param b
 	 *            buffer to scan.
-	 * @param ptr
+	 * @param start
 	 *            position in buffer to start the scan at. Most callers will
 	 *            want to pass the first position of the commit message (as
 	 *            found by {@link #commitMessage(byte[], int)}.
 	 * @return position of the LF at the end of the paragraph;
 	 *         <code>b.length</code> if no paragraph end could be located.
 	 */
-	public static final int endOfParagraph(final byte[] b, int ptr) {
+	public static final int endOfParagraph(final byte[] b, final int start) {
+		int ptr = start;
 		final int sz = b.length;
 		while (ptr < sz && b[ptr] != '\n')
 			ptr = next(b, ptr, '\n');
-		if (ptr < sz && b[ptr] == '\n')
-			return ptr - 1;
-		return sz;
+		while (0 < ptr && start < ptr && b[ptr - 1] == '\n')
+			ptr--;
+		return ptr;
 	}
 
 	private RawParseUtils() {
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 02/11] Add test case for the RevCommit parsing code

[Shawn O. Pearce]

Most of these are to address the errors identified in getShortMessage
when handling the different corner cases of a commit body.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../spearce/jgit/revwalk/RevCommitParseTest.java   |  190 ++++++++++++++++++++
 1 files changed, 190 insertions(+), 0 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevCommitParseTest.java

diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevCommitParseTest.java b/org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevCommitParseTest.java
new file mode 100644
index 0000000..2645337
--- /dev/null
+++ b/org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevCommitParseTest.java
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2008, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *   copyright notice, this list of conditions and the following
+ *   disclaimer in the documentation and/or other materials provided
+ *   with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ *   names of its contributors may be used to endorse or promote
+ *   products derived from this software without specific prior
+ *   written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.revwalk;
+
+import org.spearce.jgit.lib.ObjectId;
+import org.spearce.jgit.lib.PersonIdent;
+import org.spearce.jgit.lib.RepositoryTestCase;
+
+public class RevCommitParseTest extends RepositoryTestCase {
+	public void testParse_NoParents() throws Exception {
+		final ObjectId treeId = id("9788669ad918b6fcce64af8882fc9a81cb6aba67");
+		final String authorName = "A U. Thor";
+		final String authorEmail = "a_u_thor@example.com";
+		final int authorTime = 1218123387;
+
+		final String committerName = "C O. Miter";
+		final String committerEmail = "comiter@example.com";
+		final int committerTime = 1218123390;
+		final StringBuilder body = new StringBuilder();
+
+		body.append("tree ");
+		body.append(treeId);
+		body.append("\n");
+
+		body.append("author ");
+		body.append(authorName);
+		body.append(" <");
+		body.append(authorEmail);
+		body.append("> ");
+		body.append(authorTime);
+		body.append(" +0700\n");
+
+		body.append("committer ");
+		body.append(committerName);
+		body.append(" <");
+		body.append(committerEmail);
+		body.append("> ");
+		body.append(committerTime);
+		body.append(" -0500\n");
+
+		body.append("\n");
+
+		final RevWalk rw = new RevWalk(db);
+		final RevCommit c;
+
+		c = new RevCommit(id("9473095c4cb2f12aefe1db8a355fe3fafba42f67"));
+		assertNull(c.getTree());
+		assertNull(c.parents);
+
+		c.parseCanonical(rw, body.toString().getBytes("UTF-8"));
+		assertNotNull(c.getTree());
+		assertEquals(treeId, c.getTree().getId());
+		assertSame(rw.lookupTree(treeId), c.getTree());
+
+		assertNotNull(c.parents);
+		assertEquals(0, c.parents.length);
+		assertEquals("", c.getFullMessage());
+
+		final PersonIdent cAuthor = c.getAuthorIdent();
+		assertNotNull(cAuthor);
+		assertEquals(authorName, cAuthor.getName());
+		assertEquals(authorEmail, cAuthor.getEmailAddress());
+
+		final PersonIdent cCommitter = c.getCommitterIdent();
+		assertNotNull(cCommitter);
+		assertEquals(committerName, cCommitter.getName());
+		assertEquals(committerEmail, cCommitter.getEmailAddress());
+	}
+
+	private RevCommit create(final String msg) throws Exception {
+		final StringBuilder b = new StringBuilder();
+		b.append("tree 9788669ad918b6fcce64af8882fc9a81cb6aba67\n");
+		b.append("author A U. Thor <a_u_thor@example.com> 1218123387 +0700\n");
+		b.append("committer C O. Miter <c@example.com> 1218123390 -0500\n");
+		b.append("\n");
+		b.append(msg);
+
+		final RevCommit c;
+		c = new RevCommit(id("9473095c4cb2f12aefe1db8a355fe3fafba42f67"));
+		c.parseCanonical(new RevWalk(db), b.toString().getBytes("UTF-8"));
+		return c;
+	}
+
+	public void testParse_WeirdHeaderOnlyCommit() throws Exception {
+		final StringBuilder b = new StringBuilder();
+		b.append("tree 9788669ad918b6fcce64af8882fc9a81cb6aba67\n");
+		b.append("author A U. Thor <a_u_thor@example.com> 1218123387 +0700\n");
+		b.append("committer C O. Miter <c@example.com> 1218123390 -0500\n");
+
+		final RevCommit c;
+		c = new RevCommit(id("9473095c4cb2f12aefe1db8a355fe3fafba42f67"));
+		c.parseCanonical(new RevWalk(db), b.toString().getBytes("UTF-8"));
+
+		assertEquals("", c.getFullMessage());
+		assertEquals("", c.getShortMessage());
+	}
+
+	public void testParse_NoMessage() throws Exception {
+		final String msg = "";
+		final RevCommit c = create(msg);
+		assertEquals(msg, c.getFullMessage());
+		assertEquals(msg, c.getShortMessage());
+	}
+
+	public void testParse_OnlyLFMessage() throws Exception {
+		final RevCommit c = create("\n");
+		assertEquals("\n", c.getFullMessage());
+		assertEquals("", c.getShortMessage());
+	}
+
+	public void testParse_ShortLineOnlyNoLF() throws Exception {
+		final String shortMsg = "This is a short message.";
+		final RevCommit c = create(shortMsg);
+		assertEquals(shortMsg, c.getFullMessage());
+		assertEquals(shortMsg, c.getShortMessage());
+	}
+
+	public void testParse_ShortLineOnlyEndLF() throws Exception {
+		final String shortMsg = "This is a short message.";
+		final String fullMsg = shortMsg + "\n";
+		final RevCommit c = create(fullMsg);
+		assertEquals(fullMsg, c.getFullMessage());
+		assertEquals(shortMsg, c.getShortMessage());
+	}
+
+	public void testParse_ShortLineOnlyEmbeddedLF() throws Exception {
+		final String fullMsg = "This is a\nshort message.";
+		final String shortMsg = fullMsg.replace('\n', ' ');
+		final RevCommit c = create(fullMsg);
+		assertEquals(fullMsg, c.getFullMessage());
+		assertEquals(shortMsg, c.getShortMessage());
+	}
+
+	public void testParse_ShortLineOnlyEmbeddedAndEndingLF() throws Exception {
+		final String fullMsg = "This is a\nshort message.\n";
+		final String shortMsg = "This is a short message.";
+		final RevCommit c = create(fullMsg);
+		assertEquals(fullMsg, c.getFullMessage());
+		assertEquals(shortMsg, c.getShortMessage());
+	}
+
+	public void testParse_GitStyleMessage() throws Exception {
+		final String shortMsg = "This fixes a bug.";
+		final String body = "We do it with magic and pixie dust and stuff.\n"
+				+ "\n" + "Signed-off-by: A U. Thor <author@example.com>\n";
+		final String fullMsg = shortMsg + "\n" + "\n" + body;
+		final RevCommit c = create(fullMsg);
+		assertEquals(fullMsg, c.getFullMessage());
+		assertEquals(shortMsg, c.getShortMessage());
+	}
+
+	private static ObjectId id(final String str) {
+		return ObjectId.fromString(str);
+	}
+}
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 03/11] Notify AbstractTreeIterator implementations of skipped tree entries

[Shawn O. Pearce]

Some tree iterators may benefit from knowing when their driving TreeWalk
has chosen to skip past their current entry and not report it to client
applications.  This can be useful for an index update scenario where the
client application has applied a TreeFilter to only see the entries that
it wants to modify in this session.

By default the new skip() method just calls next(), as most types of the
tree iterator do not have this distinction between skipped entry and a
non-skipped entry.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../jgit/treewalk/AbstractTreeIterator.java        |   16 ++++++++++++++++
 .../src/org/spearce/jgit/treewalk/TreeWalk.java    |   13 ++++++++++++-
 2 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
index 448c547..0c0257c 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
@@ -46,6 +46,7 @@ import org.spearce.jgit.lib.Constants;
 import org.spearce.jgit.lib.FileMode;
 import org.spearce.jgit.lib.ObjectId;
 import org.spearce.jgit.lib.Repository;
+import org.spearce.jgit.treewalk.filter.TreeFilter;
 
 /**
  * Walks a Git tree (directory) in Git sort order.
@@ -316,4 +317,19 @@ public abstract class AbstractTreeIterator {
 	 *             the tree is invalid.
 	 */
 	public abstract void next() throws CorruptObjectException;
+
+	/**
+	 * Advance to the next tree entry, populating this iterator with its data.
+	 * <p>
+	 * This method behaves like {@link #next()} but is called by
+	 * {@link TreeWalk} only if a {@link TreeFilter} was used and ruled out the
+	 * current entry from the results. In such cases this tree iterator may
+	 * perform special behavior.
+	 * 
+	 * @throws CorruptObjectException
+	 *             the tree is invalid.
+	 */
+	public void skip() throws CorruptObjectException {
+		next();
+	}
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
index 42f8b25..7ea16b5 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
@@ -395,7 +395,7 @@ public class TreeWalk {
 
 				currentHead = t;
 				if (!filter.include(this)) {
-					popEntriesEqual();
+					skipEntriesEqual();
 					continue;
 				}
 
@@ -635,6 +635,17 @@ public class TreeWalk {
 		}
 	}
 
+	private void skipEntriesEqual() throws CorruptObjectException {
+		final AbstractTreeIterator ch = currentHead;
+		for (int i = 0; i < trees.length; i++) {
+			final AbstractTreeIterator t = trees[i];
+			if (t.matches == ch) {
+				t.skip();
+				t.matches = null;
+			}
+		}
+	}
+
 	private void exitSubtree() throws CorruptObjectException {
 		depth--;
 		for (int i = 0; i < trees.length; i++)
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 04/11] Allow AbstractTreeIterator subclasses to supply their own path array

[Shawn O. Pearce]

Some forms of AbstractTreeIterator may already have the path buffer
they want their child iterator to use in memory when the child is
being created.  Therefore they do not want the existing constructor
as it would modify the parent's path buffer, and force the subclass
to copy the new child's path.

By allowing subclass implementations to define their own path buffer
we must rely on them to do the right thing in terms of setting up a
valid buffer for iteration, but the implementation can avoid copies
and unwanted data changes to the parent array.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../jgit/treewalk/AbstractTreeIterator.java        |   23 ++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
index 0c0257c..64bb5be 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
@@ -146,6 +146,29 @@ public abstract class AbstractTreeIterator {
 	}
 
 	/**
+	 * Create an iterator for a subtree of an existing iterator.
+	 * <p>
+	 * The caller is responsible for setting up the path of the child iterator.
+	 * 
+	 * @param p
+	 *            parent tree iterator.
+	 * @param childPath
+	 *            path array to be used by the child iterator. This path must
+	 *            contain the path from the top of the walk to the first child
+	 *            and must end with a '/'.
+	 * @param childPathOffset
+	 *            position within <code>childPath</code> where the child can
+	 *            insert its data. The value at
+	 *            <code>childPath[childPathOffset-1]</code> must be '/'.
+	 */
+	protected AbstractTreeIterator(final AbstractTreeIterator p,
+			final byte[] childPath, final int childPathOffset) {
+		parent = p;
+		path = childPath;
+		pathOffset = childPathOffset;
+	}
+
+	/**
 	 * Grow the path buffer larger.
 	 * 
 	 * @param len
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 05/11] Allow WorkingTreeIterators to define their prefix path when created

[Shawn O. Pearce]

When iterating a subset of the working directory over the full
repository tree (e.g. a CanonicalTreeParser run in parallel in
the same TreeWalk) we need the working directory to know what
prefix it must apply to all of its own generated paths so they
match up with the paths of the CanonicalTreeParser coming from
the object database.

The prefix is only set on the root level, as we only need to
inject it into the first iterator.  After that the shared path
buffer will ensure the subtree iterators (if any are created)
will have the proper path in them too.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../jgit/treewalk/AbstractTreeIterator.java        |   36 ++++++++++++++++++++
 .../spearce/jgit/treewalk/WorkingTreeIterator.java |   20 +++++++++++
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
index 64bb5be..cf67836 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
@@ -39,6 +39,8 @@
 package org.spearce.jgit.treewalk;
 
 import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.CharBuffer;
 
 import org.spearce.jgit.errors.CorruptObjectException;
 import org.spearce.jgit.errors.IncorrectObjectTypeException;
@@ -128,6 +130,40 @@ public abstract class AbstractTreeIterator {
 	}
 
 	/**
+	 * Create a new iterator with no parent and a prefix.
+	 * <p>
+	 * The prefix path supplied is inserted in front of all paths generated by
+	 * this iterator. It is intended to be used when an iterator is being
+	 * created for a subsection of an overall repository and needs to be
+	 * combined with other iterators that are created to run over the entire
+	 * repository namespace.
+	 * 
+	 * @param prefix
+	 *            position of this iterator in the repository tree. The value
+	 *            may be null or the empty string to indicate the prefix is the
+	 *            root of the repository. A trailing slash ('/') is
+	 *            automatically appended if the prefix does not end in '/'.
+	 */
+	protected AbstractTreeIterator(final String prefix) {
+		parent = null;
+
+		if (prefix != null && prefix.length() > 0) {
+			final ByteBuffer b;
+
+			b = Constants.CHARSET.encode(CharBuffer.wrap(prefix));
+			pathLen = b.limit();
+			path = new byte[Math.max(DEFAULT_PATH_SIZE, pathLen + 1)];
+			b.get(path, 0, pathLen);
+			if (path[pathLen - 1] != '/')
+				path[pathLen++] = '/';
+			pathOffset = pathLen;
+		} else {
+			path = new byte[DEFAULT_PATH_SIZE];
+			pathOffset = 0;
+		}
+	}
+
+	/**
 	 * Create an iterator for a subtree of an existing iterator.
 	 * 
 	 * @param p
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
index 73c4b8b..afac77b 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
@@ -100,6 +100,26 @@ public abstract class WorkingTreeIterator extends AbstractTreeIterator {
 	}
 
 	/**
+	 * Create a new iterator with no parent and a prefix.
+	 * <p>
+	 * The prefix path supplied is inserted in front of all paths generated by
+	 * this iterator. It is intended to be used when an iterator is being
+	 * created for a subsection of an overall repository and needs to be
+	 * combined with other iterators that are created to run over the entire
+	 * repository namespace.
+	 * 
+	 * @param prefix
+	 *            position of this iterator in the repository tree. The value
+	 *            may be null or the empty string to indicate the prefix is the
+	 *            root of the repository. A trailing slash ('/') is
+	 *            automatically appended if the prefix does not end in '/'.
+	 */
+	protected WorkingTreeIterator(final String prefix) {
+		super(prefix);
+		nameEncoder = Constants.CHARSET.newEncoder();
+	}
+
+	/**
 	 * Create an iterator for a subtree of an existing iterator.
 	 * 
 	 * @param p
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 06/11] Add getTree to TreeWalk for locating the current iterator instance

[Shawn O. Pearce]

Since TreeWalk creates new child iterators as it dives into a subtree
(and pops them when it comes out of the subtree) we must allow callers
to locate their current iterator if they need additional APIs from it.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../src/org/spearce/jgit/treewalk/TreeWalk.java    |   23 ++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
index 7ea16b5..9f373e6 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
@@ -413,6 +413,29 @@ public class TreeWalk {
 	}
 
 	/**
+	 * Obtain the tree iterator for the current entry.
+	 * <p>
+	 * Entering into (or exiting out of) a subtree causes the current tree
+	 * iterator instance to be changed for the nth tree. This allows the tree
+	 * iterators to manage only one list of items, with the diving handled by
+	 * recursive trees.
+	 * 
+	 * @param <T>
+	 *            type of the tree iterator expected by the caller.
+	 * @param nth
+	 *            tree to obtain the current iterator of.
+	 * @param clazz
+	 *            type of the tree iterator expected by the caller.
+	 * @return r the current iterator of the requested type; null if the tree
+	 *         has no entry to match the current path.
+	 */
+	public <T extends AbstractTreeIterator> T getTree(final int nth,
+			final Class<T> clazz) {
+		final AbstractTreeIterator t = trees[nth];
+		return t.matches == currentHead ? (T) t : null;
+	}
+
+	/**
 	 * Obtain the raw {@link FileMode} bits for the current entry.
 	 * <p>
 	 * Every added tree supplies mode bits, even if the tree does not contain
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 07/11] Allow WorkingTreeIterator to track last modified time for entries

[Shawn O. Pearce]

We need this last modified time to compare against the index to decide
if a file is dirty or not.  Its computed on demand only for the items
we actually care about, as we don't need the last modified time for a
directory.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../spearce/jgit/treewalk/FileTreeIterator.java    |    9 +++++++++
 .../spearce/jgit/treewalk/WorkingTreeIterator.java |   13 +++++++++++++
 2 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/FileTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/FileTreeIterator.java
index 331f153..25425dd 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/FileTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/FileTreeIterator.java
@@ -106,6 +106,8 @@ public class FileTreeIterator extends WorkingTreeIterator {
 
 		private long length = -1;
 
+		private long lastModified;
+
 		FileEntry(final File f) {
 			file = f;
 
@@ -138,6 +140,13 @@ public class FileTreeIterator extends WorkingTreeIterator {
 		}
 
 		@Override
+		public long getLastModified() {
+			if (lastModified == 0)
+				lastModified = file.lastModified();
+			return lastModified;
+		}
+
+		@Override
 		public InputStream openInputStream() throws IOException {
 			return new FileInputStream(file);
 		}
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
index afac77b..cb4a089 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
@@ -433,6 +433,19 @@ public abstract class WorkingTreeIterator extends AbstractTreeIterator {
 		public abstract long getLength();
 
 		/**
+		 * Get the last modified time of this entry.
+		 * <p>
+		 * <b>Note: Efficient implementation required.</b>
+		 * <p>
+		 * The implementation of this method must be efficient. If a subclass
+		 * needs to compute the value they should cache the reference within an
+		 * instance member instead.
+		 * 
+		 * @return time since the epoch (in ms) of the last change.
+		 */
+		public abstract long getLastModified();
+
+		/**
 		 * Get the name of this entry within its directory.
 		 * <p>
 		 * Efficient implementations are not required. The caller will obtain
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 08/11] Expose the current entry's length, last modified in WorkingTreeIterator

[Shawn O. Pearce]

We need the last modified time available to us in application level
code so we can pull the current WorkingTreeIterator from TreeWalk
and then grab the last modified time off it to compare against the
index file.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../spearce/jgit/treewalk/WorkingTreeIterator.java |   19 +++++++++++++++++++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
index cb4a089..05d9282 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
@@ -259,6 +259,25 @@ public abstract class WorkingTreeIterator extends AbstractTreeIterator {
 		pathLen = pathOffset + nameLen;
 	}
 
+	/**
+	 * Get the byte length of this entry.
+	 * 
+	 * @return size of this file, in bytes.
+	 */
+	public long getEntryLength() {
+		return current().getLength();
+	}
+
+	/**
+	 * Get the last modified time of this entry.
+	 * 
+	 * @return last modified time of this file, in milliseconds since the epoch
+	 *         (Jan 1, 1970 UTC).
+	 */
+	public long getEntryLastModified() {
+		return current().getLastModified();
+	}
+
 	private static final Comparator<Entry> ENTRY_CMP = new Comparator<Entry>() {
 		public int compare(final Entry o1, final Entry o2) {
 			final byte[] a = o1.encodedName;
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 09/11] Expose idBuffer,idOffset in AbstractTreeIterator to applications

[Shawn O. Pearce]

Application code needs to be able to efficiently copy an id from
an iterator's current position to some buffer, such as in the
index file or a tree object, or any other data structure.  Doing
a conversion to ObjectId and then back again to the raw byte form
is less efficient than just copying the bytes between two arrays.

So these methods must be public for performance reasons, even if
it makes the object API slightly more obtuse.  Besides we still
have getEntryObjectId should applications prefer that approach
and they don't need the performance.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../jgit/treewalk/AbstractTreeIterator.java        |    4 ++--
 .../spearce/jgit/treewalk/CanonicalTreeParser.java |    4 ++--
 .../spearce/jgit/treewalk/EmptyTreeIterator.java   |    4 ++--
 .../spearce/jgit/treewalk/WorkingTreeIterator.java |    4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
index cf67836..6d7159c 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/AbstractTreeIterator.java
@@ -321,7 +321,7 @@ public abstract class AbstractTreeIterator {
 	 * @return byte array the implementation stores object IDs within.
 	 * @see #getEntryObjectId()
 	 */
-	protected abstract byte[] idBuffer();
+	public abstract byte[] idBuffer();
 
 	/**
 	 * Get the position within {@link #idBuffer()} of this entry's ObjectId.
@@ -329,7 +329,7 @@ public abstract class AbstractTreeIterator {
 	 * @return offset into the array returned by {@link #idBuffer()} where the
 	 *         ObjectId must be copied out of.
 	 */
-	protected abstract int idOffset();
+	public abstract int idOffset();
 
 	/**
 	 * Create a new iterator for the current entry's subtree.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/CanonicalTreeParser.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/CanonicalTreeParser.java
index 56bcab0..55942ed 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/CanonicalTreeParser.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/CanonicalTreeParser.java
@@ -112,12 +112,12 @@ public class CanonicalTreeParser extends AbstractTreeIterator {
 	}
 
 	@Override
-	protected byte[] idBuffer() {
+	public byte[] idBuffer() {
 		return raw;
 	}
 
 	@Override
-	protected int idOffset() {
+	public int idOffset() {
 		return rawPtr - Constants.OBJECT_ID_LENGTH;
 	}
 
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/EmptyTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/EmptyTreeIterator.java
index 73557bc..09d2bde 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/EmptyTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/EmptyTreeIterator.java
@@ -69,12 +69,12 @@ public class EmptyTreeIterator extends AbstractTreeIterator {
 	}
 
 	@Override
-	protected byte[] idBuffer() {
+	public byte[] idBuffer() {
 		return zeroid;
 	}
 
 	@Override
-	protected int idOffset() {
+	public int idOffset() {
 		return 0;
 	}
 
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
index 05d9282..4ac711b 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/WorkingTreeIterator.java
@@ -131,7 +131,7 @@ public abstract class WorkingTreeIterator extends AbstractTreeIterator {
 	}
 
 	@Override
-	protected byte[] idBuffer() {
+	public byte[] idBuffer() {
 		if (contentIdFromPtr == ptr - 1)
 			return contentId;
 		if (entries == EOF)
@@ -229,7 +229,7 @@ public abstract class WorkingTreeIterator extends AbstractTreeIterator {
 	}
 
 	@Override
-	protected int idOffset() {
+	public int idOffset() {
 		return 0;
 	}
 
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 10/11] Add a TreeWalk iterator implementation for IContainer

[Shawn O. Pearce]

Treating the Eclipse workspace as a filesystem backend for TreeWalk
can give us some nice caching on the file modification times and on
directory contents.  We also get nice APIs to open a file and read
its contents.

This iterator allows combining a walk over an IContainer with any
other sort of directory walk we may do, like against a stored tree
object in the object database or against the index file, or even
any other IContainer in the workspace (e.g. diff two projects).

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 .../spearce/egit/core/ContainerTreeIterator.java   |  181 ++++++++++++++++++++
 1 files changed, 181 insertions(+), 0 deletions(-)
 create mode 100644 org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java

diff --git a/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
new file mode 100644
index 0000000..17b8414
--- /dev/null
+++ b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
@@ -0,0 +1,181 @@
+/*******************************************************************************
+ * Copyright (C) 2008, Google Inc.
+ *
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * See LICENSE for the full license text, also available.
+ *******************************************************************************/
+
+package org.spearce.egit.core;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+
+import org.eclipse.core.resources.IContainer;
+import org.eclipse.core.resources.IFile;
+import org.eclipse.core.resources.IResource;
+import org.eclipse.core.runtime.CoreException;
+import org.spearce.egit.core.project.RepositoryMapping;
+import org.spearce.jgit.errors.IncorrectObjectTypeException;
+import org.spearce.jgit.lib.Constants;
+import org.spearce.jgit.lib.FileMode;
+import org.spearce.jgit.lib.ObjectId;
+import org.spearce.jgit.lib.Repository;
+import org.spearce.jgit.treewalk.AbstractTreeIterator;
+import org.spearce.jgit.treewalk.WorkingTreeIterator;
+import org.spearce.jgit.util.FS;
+
+/**
+ * Adapts an Eclipse {@link IContainer} for use in a <code>TreeWalk</code>.
+ * <p>
+ * This iterator converts an Eclipse IContainer object into something that a
+ * TreeWalk instance can iterate over in parallel with any other Git tree data
+ * structure, such as another working directory tree from outside of the
+ * workspace or a stored tree from a Repository object database.
+ * <p>
+ * Modification times provided by this iterator are obtained from the cache
+ * Eclipse uses to track external resource modification. This can be faster, but
+ * requires the user refresh their workspace when external modifications take
+ * place. This is not really a concern as it is common practice to need to do a
+ * workspace refresh after externally modifying a file.
+ * 
+ * @see org.spearce.jgit.treewalk.TreeWalk
+ */
+public class ContainerTreeIterator extends WorkingTreeIterator {
+	private static String computePrefix(final IContainer base) {
+		final RepositoryMapping rm = RepositoryMapping.getMapping(base);
+		if (rm == null)
+			throw new IllegalArgumentException("Not in a Git project: " + base);
+		return rm.getRepoRelativePath(base);
+	}
+
+	private final IContainer node;
+
+	/**
+	 * Construct a new iterator from the workspace.
+	 * <p>
+	 * The iterator will support traversal over the named container, but only if
+	 * it is contained within a project which has the Git repository provider
+	 * connected and this resource is mapped into a Git repository. During the
+	 * iteration the paths will be automatically generated to match the proper
+	 * repository paths for this container's children.
+	 * 
+	 * @param base
+	 *            the part of the workspace the iterator will walk over.
+	 */
+	public ContainerTreeIterator(final IContainer base) {
+		super(computePrefix(base));
+		node = base;
+	}
+
+	private ContainerTreeIterator(final WorkingTreeIterator p,
+			final IContainer base) {
+		super(p);
+		node = base;
+	}
+
+	@Override
+	public AbstractTreeIterator createSubtreeIterator(final Repository db)
+			throws IncorrectObjectTypeException, IOException {
+		if (FileMode.TREE.equals(mode))
+			return new ContainerTreeIterator(this,
+					(IContainer) ((ResourceEntry) current()).rsrc);
+		else
+			throw new IncorrectObjectTypeException(ObjectId.zeroId(),
+					Constants.TYPE_TREE);
+	}
+
+	@Override
+	protected Entry[] getEntries() throws IOException {
+		final IResource[] all;
+		try {
+			all = node.members(IContainer.INCLUDE_HIDDEN);
+		} catch (CoreException err) {
+			final IOException ioe = new IOException(err.getMessage());
+			ioe.initCause(err);
+			throw ioe;
+		}
+
+		final Entry[] r = new Entry[all.length];
+		for (int i = 0; i < r.length; i++)
+			r[i] = new ResourceEntry(all[i]);
+		return r;
+	}
+
+	static class ResourceEntry extends Entry {
+		final IResource rsrc;
+
+		private final FileMode mode;
+
+		private long length = -1;
+
+		ResourceEntry(final IResource f) {
+			rsrc = f;
+
+			switch (f.getType()) {
+			case IResource.FILE:
+				if (FS.INSTANCE.canExecute(asFile()))
+					mode = FileMode.EXECUTABLE_FILE;
+				else
+					mode = FileMode.REGULAR_FILE;
+				break;
+			case IResource.FOLDER: {
+				final IContainer c = (IContainer) f;
+				if (c.findMember(".git") != null)
+					mode = FileMode.GITLINK;
+				else
+					mode = FileMode.TREE;
+				break;
+			}
+			default:
+				mode = FileMode.MISSING;
+				break;
+			}
+		}
+
+		@Override
+		public FileMode getMode() {
+			return mode;
+		}
+
+		@Override
+		public String getName() {
+			return rsrc.getName();
+		}
+
+		@Override
+		public long getLength() {
+			if (length < 0) {
+				if (rsrc instanceof IFile)
+					length = asFile().length();
+				else
+					length = 0;
+			}
+			return length;
+		}
+
+		@Override
+		public long getLastModified() {
+			return rsrc.getLocalTimeStamp();
+		}
+
+		@Override
+		public InputStream openInputStream() throws IOException {
+			if (rsrc instanceof IFile) {
+				try {
+					return ((IFile) rsrc).getContents(true);
+				} catch (CoreException err) {
+					final IOException ioe = new IOException(err.getMessage());
+					ioe.initCause(err);
+					throw ioe;
+				}
+			}
+			throw new IOException("Not a regular file: " + rsrc);
+		}
+
+		private File asFile() {
+			return ((IFile) rsrc).getLocation().toFile();
+		}
+	}
+}
-- 
1.6.0.rc2.219.g1250ab

[EGIT PATCH 11/11] Teach NB how to encode/decode an unsigned 16 bit integer

[Shawn O. Pearce]

The DIRC (aka index) file format in Git uses a 16 bit unsigned int
field in at least one of the members for a file record.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 org.spearce.jgit/src/org/spearce/jgit/util/NB.java |   35 ++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/util/NB.java b/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
index 3af293a..ef21a4b 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/util/NB.java
@@ -91,6 +91,22 @@ public final class NB {
 	}
 
 	/**
+	 * Convert sequence of 2 bytes (network byte order) into unsigned value.
+	 * 
+	 * @param intbuf
+	 *            buffer to acquire the 2 bytes of data from.
+	 * @param offset
+	 *            position within the buffer to begin reading from. This
+	 *            position and the next byte after it (for a total of 2 bytes)
+	 *            will be read.
+	 * @return unsigned integer value that matches the 16 bits read.
+	 */
+	public static int decodeUInt16(final byte[] intbuf, final int offset) {
+		int r = (intbuf[offset] << 8) & 0xff;
+		return r | (intbuf[offset + 1] & 0xff);
+	}
+
+	/**
 	 * Convert sequence of 4 bytes (network byte order) into signed value.
 	 * 
 	 * @param intbuf
@@ -148,6 +164,25 @@ public final class NB {
 	}
 
 	/**
+	 * Write a 16 bit integer as a sequence of 2 bytes (network byte order).
+	 * 
+	 * @param intbuf
+	 *            buffer to write the 2 bytes of data into.
+	 * @param offset
+	 *            position within the buffer to begin writing to. This position
+	 *            and the next byte after it (for a total of 2 bytes) will be
+	 *            replaced.
+	 * @param v
+	 *            the value to write.
+	 */
+	public static void encodeInt16(final byte[] intbuf, final int offset, int v) {
+		intbuf[offset + 1] = (byte) v;
+		v >>>= 8;
+
+		intbuf[offset] = (byte) v;
+	}
+
+	/**
 	 * Write a 32 bit integer as a sequence of 4 bytes (network byte order).
 	 * 
 	 * @param intbuf
-- 
1.6.0.rc2.219.g1250ab

Re: [EGIT PATCH 10/11] Add a TreeWalk iterator implementation for IContainer

[Robin Rosenberg]

söndagen den 10 augusti 2008 10.46.25 skrev Shawn O. Pearce:
> diff --git a/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
> new file mode 100644
> index 0000000..17b8414
> --- /dev/null
> +++ b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
...
> +	@Override
> +	protected Entry[] getEntries() throws IOException {
> +		final IResource[] all;
> +		try {
> +			all = node.members(IContainer.INCLUDE_HIDDEN);

INCLUDE_HIDDEN is a 3.4 flag.

-- robin

Re: [EGIT PATCH 10/11] Add a TreeWalk iterator implementation for IContainer

[Shawn O. Pearce]

Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> söndagen den 10 augusti 2008 10.46.25 skrev Shawn O. Pearce:
> > diff --git a/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
> > new file mode 100644
> > index 0000000..17b8414
> > --- /dev/null
> > +++ b/org.spearce.egit.core/src/org/spearce/egit/core/ContainerTreeIterator.java
> ...
> > +	@Override
> > +	protected Entry[] getEntries() throws IOException {
> > +		final IResource[] all;
> > +		try {
> > +			all = node.members(IContainer.INCLUDE_HIDDEN);
> 
> INCLUDE_HIDDEN is a 3.4 flag.

Oh.  Can you amend that to be 0?

I'd rather not rebuild my entire branch space just for this one minor
change so I can resend the patch.  I've got far too many large open
branches as it is, with everything depending on everything else.  :-|

-- 
Shawn.

Re: [EGIT PATCH 10/11] Add a TreeWalk iterator implementation for IContainer

[Robin Rosenberg]

torsdagen den 14 augusti 2008 06.19.41 skrev Shawn O. Pearce:
> Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> > söndagen den 10 augusti 2008 10.46.25 skrev Shawn O. Pearce:
> > INCLUDE_HIDDEN is a 3.4 flag.
> 
> Oh.  Can you amend that to be 0?
Sure.
> 
> I'd rather not rebuild my entire branch space just for this one minor
> change so I can resend the patch.  I've got far too many large open
> branches as it is, with everything depending on everything else.  :-|
> 

-- robin