Enumerating all objects in the Git object store

Enumerating all objects in the Git object store

[Paul Richards]

Hi,
I'd like to visualize the objects in a Git repository using Graphviz
and "dot".  I'd like to see commits as circles, trees as triangles,
blobs as rectangles, and have arrows linking these objects showing how
they reference each other in the Git DAG.  For small repositories this
may be a useful way to visualize how Git objects work.

I could not find a pre-existing script to do this so I am considering
writing one.  Is there a way to enumerate all the objects in the Git
object store, and for each one figure out its type (commit, tree, or
blob) and obtain a list of the objects it references?  If not, is
there a way to do this for a single object at a time?  (I can then
simply recurse through the whole structure.)


-- 
Paul Richards

Re: Enumerating all objects in the Git object store

[Johannes Sixt]

Am 3/26/2010 16:31, schrieb Paul Richards:
> Is there a way to enumerate all the objects in the Git
> object store, and for each one figure out its type (commit, tree, or
> blob) and obtain a list of the objects it references?  If not, is
> there a way to do this for a single object at a time?  (I can then
> simply recurse through the whole structure.)

Use 'git rev-list --objects --all' to find all objects connected to all
refs (use something else than --all if you want to). Do *not* look at the
path that the tree and blob objects have attached - it is not authorative
information.

Use 'git cat-file -t' to find the type.

Use 'git cat-file -p' to pretty-print the content of an object. Parse it
depending on the type that you find.

-- Hannes

Re: Enumerating all objects in the Git object store

[Jakub Narebski]

Johannes Sixt <j.sixt@viscovery.net> writes:

> Am 3/26/2010 16:31, schrieb Paul Richards:
> > Is there a way to enumerate all the objects in the Git
> > object store, and for each one figure out its type (commit, tree, or
> > blob) and obtain a list of the objects it references?  If not, is
> > there a way to do this for a single object at a time?  (I can then
> > simply recurse through the whole structure.)
> 
> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.
> 
> Use 'git cat-file -t' to find the type.

For performance it would be better to feed `git rev-list --objects --all`
output to `git cat-file --batch-check` instead.
 
> Use 'git cat-file -p' to pretty-print the content of an object. Parse it
> depending on the type that you find.

For performance it would be better to use `git cat-file --batch`...
but on the other hand side you would have to parse "binary" tree objects
representation.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Enumerating all objects in the Git object store

[Nicolas Pitre]

On Fri, 26 Mar 2010, Johannes Sixt wrote:

> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.

Really??


Nicolas

Re: Enumerating all objects in the Git object store

[Johannes Sixt]

On Freitag, 26. März 2010, Nicolas Pitre wrote:
> On Fri, 26 Mar 2010, Johannes Sixt wrote:
> > Use 'git rev-list --objects --all' to find all objects connected to all
> > refs (use something else than --all if you want to). Do *not* look at the
> > path that the tree and blob objects have attached - it is not authorative
> > information.
>
> Really??

Yes, really. Consider the same content at two differen paths. Now what? 
rev-list will only ever list the object once.

-- Hannes

Re: Enumerating all objects in the Git object store

[Nicolas Pitre]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 589 bytes --]

On Fri, 26 Mar 2010, Johannes Sixt wrote:

> On Freitag, 26. März 2010, Nicolas Pitre wrote:
> > On Fri, 26 Mar 2010, Johannes Sixt wrote:
> > > Use 'git rev-list --objects --all' to find all objects connected to all
> > > refs (use something else than --all if you want to). Do *not* look at the
> > > path that the tree and blob objects have attached - it is not authorative
> > > information.
> >
> > Really??
> 
> Yes, really. Consider the same content at two differen paths. Now what? 
> rev-list will only ever list the object once.

Sure. It is still a valid path though.


Nicolas

Re: Enumerating all objects in the Git object store

[Paul Richards]

On 26 March 2010 15:53, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 3/26/2010 16:31, schrieb Paul Richards:
>> Is there a way to enumerate all the objects in the Git
>> object store, and for each one figure out its type (commit, tree, or
>> blob) and obtain a list of the objects it references?  If not, is
>> there a way to do this for a single object at a time?  (I can then
>> simply recurse through the whole structure.)
>
> Use 'git rev-list --objects --all' to find all objects connected to all
> refs (use something else than --all if you want to). Do *not* look at the
> path that the tree and blob objects have attached - it is not authorative
> information.
>
> Use 'git cat-file -t' to find the type.
>
> Use 'git cat-file -p' to pretty-print the content of an object. Parse it
> depending on the type that you find.
>

Thank you, this looks perfect!

I'll check this out (and perhaps the optimizations suggested by Jakub
if necessary).


-- 
Paul Richards

Re: Enumerating all objects in the Git object store

[Michael J Gruber]

Paul Richards venit, vidit, dixit 26.03.2010 16:31:
> Hi,
> I'd like to visualize the objects in a Git repository using Graphviz
> and "dot".  I'd like to see commits as circles, trees as triangles,
> blobs as rectangles, and have arrows linking these objects showing how
> they reference each other in the Git DAG.  For small repositories this
> may be a useful way to visualize how Git objects work.
> 
> I could not find a pre-existing script to do this so I am considering
> writing one.  Is there a way to enumerate all the objects in the Git
> object store, and for each one figure out its type (commit, tree, or
> blob) and obtain a list of the objects it references?  If not, is
> there a way to do this for a single object at a time?  (I can then
> simply recurse through the whole structure.)
> 

If you don't care about loose (unreferenced) objects then it's enough to
walk through all refs (branches, tags) and, for each ref, look at parent
and tree references (using git cat-file) and recurse.

If you don't care about the sha1 of commits you might even want to
(ab)use git-fast-export for that!

Michael