New Proposal (simple) for Metadata in Git Commits: git-meta

New Proposal (simple) for Metadata in Git Commits: git-meta

[Sam Elliott]

Hi,

I had this idea a day or so again, and have written a simple  
implementation to storing some semi-structured data in a git commit  
message. This means that it's much easier for data to be found and  
scraped from a certain commit, for instance what issue numbers commits  
refer to or close.

The entirety of the idea is expressed here: http://lenary.github.com/hackery/2009/12/git-meta/

I have so far built an implementation of a way to get the data out of  
a commit. I am still investigating how to get this info into the  
commit message before the git-commit command so that it doesn't have  
to be entirely hand-written.

I was just looking for feedback and to see whether the mailing list  
thinks this is a good idea or not? Any help would also be greatly  
appreciated.

Thanks!

Sam
--
Sam Elliott
sam@lenary.co.uk
--

Re: New Proposal (simple) for Metadata in Git Commits: git-meta

[Shawn O. Pearce]

Sam Elliott <sam@lenary.co.uk> wrote:
> I had this idea a day or so again, and have written a simple  
> implementation to storing some semi-structured data in a git commit  
> message. This means that it's much easier for data to be found and  
> scraped from a certain commit, for instance what issue numbers commits  
> refer to or close.

So from your web page, this is basically just YAML shoved into the
footer of the message:

--8<--
	Written half of the functionality. Namely the querying half.

	---git-meta---
	awesome: true
	Github:
	  user: lenary
	---git-meta---
-->8--


Why do we need the --git-meta-- delimiter lines?

JGit and Gerrit Code Review have already been following Git tradition
by using footer messages as metadata.  E.g. we have lines like:

--8<--
    init: Don't abort on empty directory
    
    The following sequence should work:
    
      mkdir testgit
      java -jar gerrit.war init -d testgit
    
    Since testgit is empty, it should be acceptable for us to populate the
    directory with our files.
    
    Bug: issue 358
    Change-Id: Ia85f31802066f8d39b042d3d057d33950a5035fd
    Signed-off-by: Shawn O. Pearce <sop@google.com>
-->8--

The JGit commit message parser has special logic to handle lines
that smell like one of these tag lines.  So long as there is no
blank line in the footer paragraph, each of these tags can be read
and processed.  Continuation lines should start with whitespace.

I don't remember my YAML well enough, but isn't this existing
standard still parseable by a YAML processor?

If you dropped the --git-meta-- tags above, JGit would happily
recognize the awesome: and Github: tags, but it might need a bit
more work to recognize the nested user: tag.  Also, you'd be able
to use git-meta on the git and Linux kernel repositories to pull
out and work with Signed-off-by, Acked-by, etc.

-- 
Shawn.

Re: New Proposal (simple) for Metadata in Git Commits: git-meta

[Shawn O. Pearce]

Sam Elliott <sam@lenary.co.uk> wrote:
> On 15 Dec 2009, at 23:05, Shawn O. Pearce wrote:
>> If you dropped the --git-meta-- tags above, JGit would happily
>> recognize the awesome: and Github: tags, but it might need a bit
>> more work to recognize the nested user: tag.  Also, you'd be able
>> to use git-meta on the git and Linux kernel repositories to pull
>> out and work with Signed-off-by, Acked-by, etc.
>
> I'm not entirely sure about this approach. The current implementation  
> also works with PGP-signed tags, where the information is not  
> necessarily going to be at the bottom of the message when i use `git- 
> cat-file -p`. I think it shouldn't be too hard to also have git-meta  
> read any YAML-like data just before the signing message.

Ah, good point.  But as you point out, it should be simple enough
to detect a PGP signature on the bottom and just clip that off the
end, and then perform the YAML-like data parsing on the footer.

-- 
Shawn.

Re: New Proposal (simple) for Metadata in Git Commits: git-meta

[Johan Herland]

On Wednesday 16 December 2009, Shawn O. Pearce wrote:
> Sam Elliott <sam@lenary.co.uk> wrote:
> > On 15 Dec 2009, at 23:05, Shawn O. Pearce wrote:
> >> If you dropped the --git-meta-- tags above, JGit would happily
> >> recognize the awesome: and Github: tags, but it might need a bit
> >> more work to recognize the nested user: tag.  Also, you'd be able
> >> to use git-meta on the git and Linux kernel repositories to pull
> >> out and work with Signed-off-by, Acked-by, etc.
> >
> > I'm not entirely sure about this approach. The current implementation
> > also works with PGP-signed tags, where the information is not
> > necessarily going to be at the bottom of the message when i use `git-
> > cat-file -p`. I think it shouldn't be too hard to also have git-meta
> > read any YAML-like data just before the signing message.
> 
> Ah, good point.  But as you point out, it should be simple enough
> to detect a PGP signature on the bottom and just clip that off the
> end, and then perform the YAML-like data parsing on the footer.

I agree with Shawn's point that it should be possible to do this without 
embedding it in custom ---tags---.

I would even try to parse the _entire_ commit message, and then discard 
everything that didn't match the "<word>: <free-form value>" format (with 
possible continuation lines). Even though this will generate some false 
positives (probably non-sensical "key: value" pairs), I don't see this as a 
major problem , since most users of this functionality are looking for a 
small set of specific keywords (which are even more unlikely to turn up as 
false positives)

In future versions of Git, you might also want to check for YAML-like data 
in the notes object corresponding to the commit in question (see git-notes 
in v1.6.6 for more details on the new notes feature). This would allow users 
to add/edit such metadata after the commit was made, without having to 
rewrite the commit itself.


Have fun! :)

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net