Re: Request for detailed documentation of git pack protocol

[Nicolas Pitre <nico@cam.org>]

On Thu, 14 May 2009, Jakub Narebski wrote:

> I was afraid of this: that the people who know pack protocol good
> enough to be able to write it down are otherwise busy. But we get
> detailed / updated packfile and pack index format descriptions some
> time ago (thanks all that contributed to it!). I hope that the same
> would happen with pack _protocol_ description.

If someone with the wish for such a document volunteers to work on it 
then I'm sure people with fuller knowledge will review and comment on 
the result as appropriate.

> I was hoping of document in RFC format; dreaming about having it
> submitted to IETF as (at least) unofficial RFC like Atom Publication
> Protocol (or is it proper RFC these days?), and then accepted like
> HTTP protocol.

I think we'd have to move to a new version of the protocol for that.  
The current protocol, even if it does the job, is not particularly 
elegant.

> > And lets not even start to mention Dulwich not completing a thin
> > pack before storing it on disk.  Those sorts of on disk things
> > matter to other more popular Git implementations (c git, jgit).
> 
> Ugh! Errr... aren't thin packs send only if other side has the
> capability for it? What is then Dulwich doing announcing such 
> capability when not supporting it correctly...

They probably don't bother because in theory you don't need to complete 
a thin pack for the system to still work.  We require that any pack 
never contain a delta which base object is in a different pack because 
that makes for better performances when accessing the pack and when 
repacking.  And not doing so makes pack validation (think verify-pack) 
impossible without the dependent objects, and that makes incremental 
repacking much much harder wrt prevention of delta cycles.

Those validation tools from C git (fsck, verify-pack, etc.) should be 
quite useful for people wishing to implement their own git.


Nicolas