Re: [PATCH JGIT] Allow writeObject() write to OutputStream

[Daniel Cheng <j16sdiz+freenet@gmail.com>]

On Fri, Mar 13, 2009 at 11:08 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> "Daniel Cheng (aka SDiZ)" <j16sdiz+freenet@gmail.com> wrote:
>>
>> This patch make factor out the object writing code in ObjectWriter,
>> allow it to write to any OutputStream.
>> Subclass class may then override
>>   writeObject(final int type, long len, InputStream is, boolean store)
>> to make it write to alternative locations.
>>
>> There are some discussion on devl@freenetproject.org to use raw
>> (uncompressed) object to freenet. This patch allow the testing.
>
> Ok, I understand the code as-is, but I'm not sure I understand the
> reasoning for the change, or where you are trying to go with it.

In freenet (and most content-addressable network), file can be dropped
out when unused.
We need some method to "heal" the lost data.
The easiest way is to do this is re-insert the very same file we used
originally.

Pack files may change on different compression parameter, object order, etc.
It need some tricks to get the original file.
Loose object are immutable, so it is easier to use loose objects.

Nothing have finalized yet, we are just evaluating different approaches.

I know loose object are always larger, and have very large number of them.
So it may backfire ....

> Are you guys talking about making every object a loose object on
> freenet, and avoiding pack files?  Or making a form of JGit that
> access a Repository directly stored on freenode?
>
> I ask because there's some folks starting to talk about putting JGit
> onto a distributed hash table sort of system like Hadoop HBase,
> to allow the underlying storage to scale efficiently for really
> big hosting sites.  I would rather see a pooling of effort here
> than folks going in different directions.
>
> --
> Shawn.
>