[JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Nigel Magnay]

JGit is used as a library in external projects such as build tools.
Some of the representations of git data structures are useful
in these external tools - but - it is often desirable to be able to
either persist these objects, or serialize them across the wire.

Apply Externalizable to URIish and RefSpec, and Serializable to
ObjectId and RemoteConfig (in order to avoid the undesirable
requirement of a public, no-args constructor needed for Externalizable).

 Signed-off-by: Nigel Magnay <nigel.magnay@gmail.com>

---
 .../src/org/spearce/jgit/lib/ObjectId.java         |   22 +++++-
 .../src/org/spearce/jgit/transport/RefSpec.java    |   77 +++++++++++++------
 .../org/spearce/jgit/transport/RemoteConfig.java   |   79 +++++++++++++++++++-
 .../src/org/spearce/jgit/transport/URIish.java     |   29 +++++++-
 4 files changed, 179 insertions(+), 28 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
index 52ce0d4..7c3b922 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.lib;

+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.Serializable;
 import java.io.UnsupportedEncodingException;

 import org.spearce.jgit.util.NB;
@@ -45,7 +49,7 @@
 /**
  * A SHA-1 abstraction.
  */
-public class ObjectId extends AnyObjectId {
+public class ObjectId extends AnyObjectId implements Serializable {
 	private static final ObjectId ZEROID;

 	private static final String ZEROID_STR;
@@ -269,4 +273,20 @@ protected ObjectId(final AnyObjectId src) {
 	public ObjectId toObjectId() {
 		return this;
 	}
+
+	private void writeObject(ObjectOutputStream os)  throws IOException {
+		os.writeInt(w1);
+		os.writeInt(w2);
+		os.writeInt(w3);
+		os.writeInt(w4);
+		os.writeInt(w5);
+	}
+	
+	private void readObject(ObjectInputStream ois)  throws IOException {
+		w1 = ois.readInt();
+		w2 = ois.readInt();
+		w3 = ois.readInt();
+		w4 = ois.readInt();
+		w5 = ois.readInt();
+	}
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
index 521110b..0ee89b0 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
@@ -37,6 +37,11 @@

 package org.spearce.jgit.transport;

+import java.io.Externalizable;
+import java.io.IOException;
+import java.io.ObjectInput;
+import java.io.ObjectOutput;
+
 import org.spearce.jgit.lib.Constants;
 import org.spearce.jgit.lib.Ref;

@@ -46,7 +51,7 @@
  * A ref specification provides matching support and limited rules to rewrite a
  * reference in one repository to another reference in another repository.
  */
-public class RefSpec {
+public class RefSpec implements Externalizable {
 	/**
 	 * Suffix for wildcard ref spec component, that indicate matching all refs
 	 * with specified prefix.
@@ -109,30 +114,7 @@ public RefSpec() {
 	 *             the specification is invalid.
 	 */
 	public RefSpec(final String spec) {
-		String s = spec;
-		if (s.startsWith("+")) {
-			force = true;
-			s = s.substring(1);
-		}
-
-		final int c = s.indexOf(':');
-		if (c == 0) {
-			s = s.substring(1);
-			if (isWildcard(s))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-			dstName = s;
-		} else if (c > 0) {
-			srcName = s.substring(0, c);
-			dstName = s.substring(c + 1);
-			if (isWildcard(srcName) && isWildcard(dstName))
-				wildcard = true;
-			else if (isWildcard(srcName) || isWildcard(dstName))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-		} else {
-			if (isWildcard(s))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-			srcName = s;
-		}
+		initializeFromString(spec);
 	}

 	/**
@@ -161,6 +143,42 @@ private RefSpec(final RefSpec p) {
 	}

 	/**
+	 * Initialize the ref specification from a string.
+	 *
+	 * @param spec
+	 *            string describing the specification.
+	 * @throws IllegalArgumentException
+	 *             the specification is invalid.
+	 */
+	private void initializeFromString(final String spec) {
+		srcName = null;
+		String s = spec;
+		if (s.startsWith("+")) {
+			force = true;
+			s = s.substring(1);
+		}
+
+		final int c = s.indexOf(':');
+		if (c == 0) {
+			s = s.substring(1);
+			if (isWildcard(s))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+			dstName = s;
+		} else if (c > 0) {
+			srcName = s.substring(0, c);
+			dstName = s.substring(c + 1);
+			if (isWildcard(srcName) && isWildcard(dstName))
+				wildcard = true;
+			else if (isWildcard(srcName) || isWildcard(dstName))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+		} else {
+			if (isWildcard(s))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+			srcName = s;
+		}
+	}
+	
+	/**
 	 * Check if this specification wants to forcefully update the destination.
 	 *
 	 * @return true if this specification asks for updates without merge tests.
@@ -421,4 +439,13 @@ public String toString() {
 		}
 		return r.toString();
 	}
+
+	public void readExternal(ObjectInput in) throws IOException,
+			ClassNotFoundException {
+		initializeFromString(in.readUTF());	
+	}
+
+	public void writeExternal(ObjectOutput out) throws IOException {
+		out.writeUTF(toString());
+	}
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
index 5bbf664..899f73f 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.transport;

+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.Serializable;
 import java.net.URISyntaxException;
 import java.util.ArrayList;
 import java.util.Collections;
@@ -53,7 +57,7 @@
  * describing how refs should be transferred between this repository and the
  * remote repository.
  */
-public class RemoteConfig {
+public class RemoteConfig implements Serializable {
 	private static final String SECTION = "remote";

 	private static final String KEY_URL = "url";
@@ -382,4 +386,77 @@ public TagOpt getTagOpt() {
 	public void setTagOpt(final TagOpt option) {
 		tagopt = option != null ? option : TagOpt.AUTO_FOLLOW;
 	}
+	
+	private void writeObject(ObjectOutputStream os) throws IOException {
+
+	    // Name
+	    os.writeUTF(name);
+	
+	    // Key, Value pairs
+	    for (URIish uri : uris) {
+            os.writeUTF(KEY_URL);
+            os.writeUTF(uri.toPrivateString());
+        }
+
+        for (RefSpec refspec : fetch) {
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
+
+        os.writeUTF(KEY_UPLOADPACK);
+        os.writeUTF(uploadpack);
+
+        os.writeUTF(KEY_RECEIVEPACK);
+        os.writeUTF(receivepack);
+
+        os.writeUTF(KEY_TAGOPT);
+        os.writeUTF(tagopt.option());
+
+        // End marker
+        os.writeUTF("");
+    }
+
+    private void readObject(ObjectInputStream ois) throws IOException {
+		uris = new ArrayList<URIish>();
+        fetch = new ArrayList<RefSpec>();
+        push = new ArrayList<RefSpec>();
+        uploadpack = DEFAULT_UPLOAD_PACK;
+        receivepack = DEFAULT_RECEIVE_PACK;
+
+        name = ois.readUTF();
+
+		for (String key = ois.readUTF(); key.length() > 0; key = ois.readUTF()) {
+			String value = ois.readUTF();
+
+			if (key.equals(KEY_URL)) {
+				try {
+					uris.add(new URIish(value));
+				} catch (URISyntaxException e) {
+					throw new IOException("Invalid URI in RemoteConfig : "
+							+ value);
+				}
+			} else if (key.equals(KEY_FETCH)) {
+				fetch.add(new RefSpec(value));
+
+			} else if (key.equals(KEY_PUSH)) {
+				push.add(new RefSpec(value));
+
+			} else if (key.equals(KEY_UPLOADPACK)) {
+				uploadpack = value;
+
+			} else if (key.equals(KEY_RECEIVEPACK)) {
+				receivepack = value;
+
+			} else if (key.equals(KEY_TAGOPT)) {
+				tagopt = TagOpt.fromOption(value);
+			}
+
+		}
+
+    }
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
index b86e00c..6b85f45 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.transport;

+import java.io.Externalizable;
+import java.io.IOException;
+import java.io.ObjectInput;
+import java.io.ObjectOutput;
 import java.net.URISyntaxException;
 import java.net.URL;
 import java.util.regex.Matcher;
@@ -49,7 +53,7 @@
  * RFC 2396 URI's is that no URI encoding/decoding ever takes place. A space or
  * any special character is written as-is.
  */
-public class URIish {
+public class URIish implements Externalizable {
 	private static final Pattern FULL_URI = Pattern
 			.compile("^(?:([a-z][a-z0-9+-]+)://(?:([^/]+?)(?::([^/]+?))?@)?(?:([^/]+?))?(?::(\\d+))?)?((?:[A-Za-z]:)?/.+)$");

@@ -75,6 +79,16 @@
 	 * @throws URISyntaxException
 	 */
 	public URIish(String s) throws URISyntaxException {
+		initializeFromString(s);
+	}
+	
+	/**
+	 * Set fields from string based URI.
+	 *
+	 * @param s
+	 * @throws URISyntaxException
+	 */
+	private void initializeFromString(String s)  throws URISyntaxException {
 		s = s.replace('\\', '/');
 		Matcher matcher = FULL_URI.matcher(s);
 		if (matcher.matches()) {
@@ -357,4 +371,17 @@ private String format(final boolean includePassword) {

 		return r.toString();
 	}
+
+	public void readExternal(ObjectInput in) throws IOException,
+			ClassNotFoundException {
+	    try {
+			initializeFromString(in.readUTF());
+		} catch (URISyntaxException e) {
+			throw new IOException("Incorrect format URI");
+		}
+	}
+
+	public void writeExternal(ObjectOutput out) throws IOException {
+		out.writeUTF(format(true));
+	}
 }
-- 
1.6.0.2

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Johannes Schindelin]

Hi,

On Mon, 16 Feb 2009, Nigel Magnay wrote:

> JGit is used as a library in external projects such as build tools.
> Some of the representations of git data structures are useful
> in these external tools - but - it is often desirable to be able to
> either persist these objects, or serialize them across the wire.

I am still missing the description of the format.

>  Signed-off-by: Nigel Magnay <nigel.magnay@gmail.com>

And I wonder what that leading space is all about.

Ciao,
Dscho

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Shawn O. Pearce]

Nigel Magnay <nigel.magnay@gmail.com> wrote:
> +public class ObjectId extends AnyObjectId implements Serializable {

We should define our own serialVersionUID:

  private static final long serialVersionUID = 1L;

is good enough to make Java happy.

> @@ -269,4 +273,20 @@ protected ObjectId(final AnyObjectId src) {
>  	public ObjectId toObjectId() {
>  		return this;
>  	}
> +
> +	private void writeObject(ObjectOutputStream os)  throws IOException {
> +	private void readObject(ObjectInputStream ois)  throws IOException {

Minor nit: Only 1 space between ) and throws, please.

> diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
> b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
> index 5bbf664..899f73f 100644
> --- a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
> +++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
> @@ -53,7 +57,7 @@
>   * describing how refs should be transferred between this repository and the
>   * remote repository.
>   */
> -public class RemoteConfig {
> +public class RemoteConfig implements Serializable {

Please set a serialVersionUID.

> @@ -382,4 +386,77 @@ public TagOpt getTagOpt() {
>  	public void setTagOpt(final TagOpt option) {
>  		tagopt = option != null ? option : TagOpt.AUTO_FOLLOW;
>  	}
> +	
> +	private void writeObject(ObjectOutputStream os) throws IOException {
> +
> +	    // Name
> +	    os.writeUTF(name);
> +	
> +	    // Key, Value pairs
> +	    for (URIish uri : uris) {
> +            os.writeUTF(KEY_URL);
> +            os.writeUTF(uri.toPrivateString());
> +        }
> +
> +        for (RefSpec refspec : fetch) {

There is some sort of whitespace damage here, the second for loop
is not lined up at the same starting column as the first for loop.
My guess is, you have tabs in here.  We only indent with spaces.

> +            os.writeUTF(KEY_FETCH);
> +            os.writeUTF(refspec.toString());
> +        }
> +
> +        for (RefSpec refspec : push) {
> +        	os.writeUTF(KEY_PUSH);
> +            os.writeUTF(refspec.toString());
> +        }

Should we maybe allow RefSpec to serialize itself with
os.writeObject() rather than using writeUTF() directly?

FWIW, I did find this new implementation to be much easier to read.
Thanks.

-- 
Shawn.

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Robin Rosenberg]

Shawn wrote:
> > +            os.writeUTF(KEY_FETCH);
> > +            os.writeUTF(refspec.toString());
> > +        }
> > +
> > +        for (RefSpec refspec : push) {
> > +        	os.writeUTF(KEY_PUSH);
> > +            os.writeUTF(refspec.toString());
> > +        }
> 
> Should we maybe allow RefSpec to serialize itself with
> os.writeObject() rather than using writeUTF() directly?

Doesn't the style above make it easy to define and document
a format that is easy for non-java programs to write and read,
while writeObject introduces java-centric stuff (depending on
the full class name etc).

-- robin

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Shawn O. Pearce]

Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> Shawn wrote:
> > > +            os.writeUTF(KEY_FETCH);
> > > +            os.writeUTF(refspec.toString());
> > > +        }
> > > +
> > > +        for (RefSpec refspec : push) {
> > > +        	os.writeUTF(KEY_PUSH);
> > > +            os.writeUTF(refspec.toString());
> > > +        }
> > 
> > Should we maybe allow RefSpec to serialize itself with
> > os.writeObject() rather than using writeUTF() directly?
> 
> Doesn't the style above make it easy to define and document
> a format that is easy for non-java programs to write and read,
> while writeObject introduces java-centric stuff (depending on
> the full class name etc).

Non-Java reading a Java serialization stream?  Seriously?

-- 
Shawn.

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Sverre Rabbelier]

On Wed, Feb 18, 2009 at 22:48, Shawn O. Pearce <spearce@spearce.org> wrote:
> Non-Java reading a Java serialization stream?  Seriously?

That would be a contender for UJSFWIINI :P (with JS standing for Java
Serialization).

-- 
Cheers,

Sverre Rabbelier

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Robin Rosenberg]

onsdag 18 februari 2009 22:48:59 skrev "Shawn O. Pearce" <spearce@spearce.org>:
> Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> > Shawn wrote:
> > > > +            os.writeUTF(KEY_FETCH);
> > > > +            os.writeUTF(refspec.toString());
> > > > +        }
> > > > +
> > > > +        for (RefSpec refspec : push) {
> > > > +        	os.writeUTF(KEY_PUSH);
> > > > +            os.writeUTF(refspec.toString());
> > > > +        }
> > > 
> > > Should we maybe allow RefSpec to serialize itself with
> > > os.writeObject() rather than using writeUTF() directly?
> > 
> > Doesn't the style above make it easy to define and document
> > a format that is easy for non-java programs to write and read,
> > while writeObject introduces java-centric stuff (depending on
> > the full class name etc).
> 
> Non-Java reading a Java serialization stream?  Seriously?

No, that was my objection to using writeObject, as that make
it a Java-only stream, but then it might not be worth doing
it via the serialization mechanism.

-- robin

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Shawn O. Pearce]

Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:
> onsdag 18 februari 2009 22:48:59 skrev "Shawn O. Pearce" <spearce@spearce.org>:
> > 
> > Non-Java reading a Java serialization stream?  Seriously?
> 
> No, that was my objection to using writeObject, as that make
> it a Java-only stream, but then it might not be worth doing
> it via the serialization mechanism.

IMHO, if we are talking about either java.io.Serializable or
java.io.Externalizable, there's no point in considering a non
Java peer.

If you want a non-Java format, we'd need to consider a much
more neutral encoding, like Google's protobuf, or *shudder*
XML/JSON, or cooking up our own format.

That wasn't this thread started with.  The original poster just
wanted an easy way to serialize some basic data types from JGit,
as part of some higher level stream being done in the container
application.  Since that higher level stream is a apparently a
Java object serialization stream, we just need to match that.

-- 
Shawn.

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Nigel Magnay]

On Wed, Feb 18, 2009 at 4:30 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
>
> Nigel Magnay <nigel.magnay@gmail.com> wrote:
> > +public class ObjectId extends AnyObjectId implements Serializable {
>
> We should define our own serialVersionUID:
>
>  private static final long serialVersionUID = 1L;
>
> is good enough to make Java happy.

I'll re-roll. Actually both that and the whitespace damage I'm
partially blaming on eclipse. Eclipse curiously removes the warning to
declare serialVersionUID if you declare both writeObject and
readObject - that's wrong!

Also the formatting came from
.settings/org.eclipse.core.resources.prefs which has a tab policy of
"Tabs only" which has overridden my environment setting of
spaces-only.

I can modify it to spaces only - but the comment in 9268ced9d38 talks
about spaces-per-tab..?

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

[Shawn O. Pearce]

Nigel Magnay <nigel.magnay@gmail.com> wrote:
> On Wed, Feb 18, 2009 at 4:30 PM, Shawn O. Pearce <spearce@spearce.org>wrote:
> 
> > Nigel Magnay <nigel.magnay@gmail.com> wrote:
> > > +public class ObjectId extends AnyObjectId implements Serializable {
> >
> > We should define our own serialVersionUID:
> >
> >  private static final long serialVersionUID = 1L;
> >
> > is good enough to make Java happy.
> >
> 
> I'll re-roll. Actually both that and the whitespace damage I'm partially
> blaming on eclipse. Eclipse curiously removes the warning to declare
> serialVersionUID if you declare both writeObject and readObject - that's
> wrong!

Yikes.  What a nice feature.
 
> Also the formatting came from .settings/org.eclipse.core.resources.prefs
> which has a tab policy of "Tabs only" which has overridden my environment
> setting of spaces-only.
>
> I can modify it to spaces only - but the comment in 9268ced9d38 talks about
> spaces-per-tab..?

Oh, yea, my bad.

We prefer tabs, and only tabs, and we only indent the leading part of
the line, we never try to "line up" columns of variables (for example).

I guess there was a mixture here.  Not sure why.  Sometimes I've
seen Eclipse not immediately honor the per-project settings files.
Closing or deleting and re-importing the project usually fixes it,
but not always.  :-|

-- 
Shawn.