Re: http://tech.slashdot.org/comments.pl?sid=1885890&cid=34358134

[Luke Kenneth Casson Leighton <luke.leighton@gmail.com>]

On Sat, Nov 27, 2010 at 6:36 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Sat, Nov 27, 2010 at 18:33, Luke Kenneth Casson Leighton
> <luke.leighton@gmail.com> wrote:
>
>> I believe it is important that git-over-p2p be given a higher
>> priority than it is at present.
>
> We give "priority" to the stuff people submit patches for. There isn't
> a git-over-p2p because nobody is working on it, would you like to work
> on it?

 i was / have been.  unfortunately it's unfunded work, and, thanks to
receiving less money than i require for living and for feeding my
family i now face a court hearing on 16th december (3 weeks time)
which has the aim and ultimate goal of making us homeless.  i
therefore respectfull request that you please don't try to make me
feel like i _should_ be working on this in order to "earn your
respect" ( that goes for you too, johnathon ).

> I'm also not convinced that this is actually needed. It's trivial to
> set up a p2p-like network by just emulating a darknet by manually
> adding remotes & fetching/pushing.

 ah.. but that's the single-use case for a single shared repository.
git "as-is" the fetch/push over http is limited to a single
repository.  i'm thinking more in terms of a global network, with
searches for checksummed objects "accidentally" potentially coming
from wildly different repositories.

 the work i did a few months ago showed that just "dropping" git on
top of e.g. the bittorrent protocol as-is, files/dirs mapped to git
objects, wouldn't cut the mustard, because the bloody bittorrent
protocol "chunks" system treats the entire fileset as an ordered but
contiguous "stream" of data (metadata in the actual .torrent file says
how long each of the files are and in what order they are, and this is
utilised at the receiving end to reconstruct the files after the
datablocks have been received.  it means that if you want just one
single file, even of 2 bytes in length, you might have to pull 2
blocks of e.g. 256k and grab the byte and the end of the 1st and the
byte at the beginning of the 2nd! yukk...)

but i believe that a system where each git object was given its own
.torrent file _would_ work, and you utilise DHT or other mechanism
(perhaps even a torrent containing the metadata!) to get the
structural information.  the success of this model depends on everyone
being connected to everyone (a global network), in order to be able to
do a DHT search for the node(s) that are seeding that git object's
torrent file.

 the "global network" concept has the advantage of being far more
robust in the face of DNS take-overs, as you actually just need to
know _one_ DNS name out of potentially an unlimited number of DNS
names (or even an IP address or set of IP addresses) and you're joined
to other peers.

> That's not viable for the stuff that usually gets distributed on p2p
> networks, but it sure is viable when you have people working on the
> same codebase.

 codebase-in-a-git, wiki-in-a-git, website-in-a-git,
backups-of-filesystem-in-a-git (joey hess, one of the most amazing
debian developers around, has done all these things :)

> Which would be the use case for a git-over-p2p client,
> right?

yes it would... but i'm thinking well beyond that :)

at first glance the concept of git-over-darknet for example looks
great.  however it doesn't satisfy what i believe to be one of the key
strategic requirements to be "many-pushers to many-pullers real-time
resilient".  lose access to the darknet, or even if the git server
dies, what do you do??  if it's a darknet, you've got serious
problems: you can't find out who it was!

git-over-p2p on the other hand _would_ be [m-p-t-m-p-r-t-r], because
once the git-objects have been pulled by one client, they can be
pulled from that client by other clients too.  it's this feature of
p2p distribution that is so distinct from the way in which git is
normally described as "a distributed revision control system", and
makes it a bugger to explain what the fuss is all about over this
git-p2p stuff :)

 l.