[RFC/GSoC] Proposal Draft: Unifying git branch -l, git tag -l, and git for-each-ref

[RFC/GSoC] Proposal Draft: Unifying git branch -l, git tag -l, and git for-each-ref

[karthik nayak]

Hello,
I have completed the micro project[1] and have also been working on 
adding a "--literally" option
for cat-file[2]. I have left out the personal information part of the 
proposal here, will fill that in while submitting my final proposal.

Currently, I have been reading about how "branch -l", “tag -l” and 
“for-each-refs” work and how they implement the selection and formatting 
options.

Since this is a draft for my final proposal I would love to hear from 
you all about :

* Suggestions on my take of this idea and how I could improve it or 
modify it.
* Anything more I might have missed out on, in the proposal.

GSOC Proposal : Unifying git branch -l, git tag -l, and git for-each-ref
----------------------------------------------------------------------------------------------------------------------------------------
# Main objectives of the project:

* Build a common library for which can handle both selection and 
formatting of refs.

* Use this library throughout ‘branch -l’, ‘tag -l’ and ‘for-each-ref’.

* Implement options available in some of these commands onto others.

----------------------------------------------------------------------------------------------------------------------------------------

# Amongst  ‘branch -l’, ‘tag -l’ and ‘for-each-ref’ :

* ‘git branch -l’ and ‘git tag -l’ share  the ‘--contains’ option.

* 'git tag' and 'git branch' could use a formatting option (This could 
also be used to implement the verbose options)
	For eg: git branch -v could be implemented using :
	git for-each-ref refs/heads --format='%(refname:short) 
%(objectname:short) %(upstream:track) %(contents:subject)'
	This shows that having a formatting option for these two would mean 
that the verbose options could be implemented using the formatting 
option itself.

  * 'git for-each-refs' could use all the selection options. This would 
enhance the uses of for-each-refs itself. Users can then view only refs 
based on what they may be looking for.

* formatting options for ‘git branch -l’ and ‘git tag -l’. This would 
enable the user to view information as per the users requirements and 
format.

----------------------------------------------------------------------------------------------------------------------------------------
# Approach

All three commands select a subset of the repository’s refs and print 
the result. There has been an attempt to unify these commands by Jeff 
King[3]. I plan on continuing his work[4] and using his approach to 
tackle this project.

As per the common library for ‘branch -l’, ‘tag -l’ and ‘for-each-ref’ I 
plan on creating a file (mostly as ref-filter.c in terms with what Jeff 
has already done) which will provide API’s to add refs to get a list of 
all refs. This will be used along with ‘for_each_*_ref’ for obtaining 
the refs required. This gives us the basic functionality of obtaining 
the refs required by the command.

Here we could have a basic data structure (struct ref_filter_item) which 
would denote a particular ref and have another data structure to hold a 
list of these refs (struct ref_filter). Then after getting the required 
refs, we could print the information.

For extended selection behaviour such as ‘--contains’ or ‘--merged’ we 
could implement these within
the library by providing functions which closely mimic the current 
methods used individually by ‘branch -l’ and ‘tag -l’. For eg to 
implement ‘--merged’ we implement a ‘compute_merge()’ function, which 
with the help of the revision API’s will be able to perform the same 
function as ‘branch -l --merged’.

For formatting functionality provided by ‘for-each-ref’ we replicate the 
‘show_ref’ function in ‘for-each-ref.c’ where the format is given to the 
function and the function uses the format to obtain atom values and 
prints the corresponding atom values to the screen. This feature would 
allow us to provide format functionality which could act as a base for 
the ‘-v’ option also.

As Jeff has already done, we could also add parse options. Although Jeff 
has built a really good base to build upon, I shall use his work as more 
of a reference and work on unification of the three commands from 
scratch. I plan on coding for this project using a test driven 
development, where I will write tests (initially failing) which will be 
based on the objectives of the project and then write code to pass those 
tests.

----------------------------------------------------------------------------------------------------------------------------------------

# Timeline

This is a rough plan of how I will spend the summer working on this project.

Community bonding period:
	Work on understanding how all three commands work in total detail. And 
build up on the design of unification of the three commands. Read 
through Jeff’s attempt at unification and get a grasp of what to do.

Week 1 :
	Write tests and documentation which will the goal of this project. This 
will set the deliverables of the project and what the code will try to 
achieve in the project.

Week 2 :
	Build a basic library which will function to get the required refs.

Week 3 :
	Since ‘tag -l’ has the least options to be satisfied initially, modify 
‘tag -l’ to use the newly created library.

Week 4 - 5 :
	Implement ‘--merged’ and other selection options to the library.

Week 6 :
	Modify ‘branch -l’ to use the library created as this can be done 
without implementing the ‘--format’ options.

Week 7 :
	Implement ‘--format’ options in the library, by using the 
implementation used by ‘for-each-ref’ as a reference.

Week 8 :
	Modify ‘for-each-ref’ to use the library.

Week 9 - 10 :
	Modify all three commands to include most of the selection and 
formatting options.

Week 11 - 12 :
	Polishing of code.
	Write more tests and Documentation as and if required.
	Resolve any issues, if generated.

References :
[1] http://thread.gmane.org/gmane.comp.version-control.git/264911
[2] http://article.gmane.org/gmane.comp.version-control.git/265604
[3] http://thread.gmane.org/gmane.comp.version-control.git/230694/focus=2308
[4] https://github.com/peff/git/commits/jk/for-each-ref-contains-wip

Re: [RFC/GSoC] Proposal Draft: Unifying git branch -l, git tag -l, and git for-each-ref

[Jeff King]

On Mon, Mar 23, 2015 at 06:39:20PM +0530, karthik nayak wrote:

> All three commands select a subset of the repository’s refs and print the
> result. There has been an attempt to unify these commands by Jeff King[3]. I
> plan on continuing his work[4] and using his approach to tackle this
> project.

I would be cautious about the work in my for-each-ref-contains-wip
branch. At one point it was reasonably solid, but it's now a year and a
half old, and I've been rebasing it without paying _too_ much attention
to correctness. I think some subtle bugs have been introduced as it has
been carried forward.

Also, the very first patch (factoring out the contains traversal) is
probably better served by this series:

  http://thread.gmane.org/gmane.comp.version-control.git/252472

I don't remember all of the issues offhand that need to be addressed in
it, but there were plenty of review comments.

> For extended selection behaviour such as ‘--contains’ or ‘--merged’ we could
> implement these within
> the library by providing functions which closely mimic the current methods
> used individually by ‘branch -l’ and ‘tag -l’. For eg to implement
> ‘--merged’ we implement a ‘compute_merge()’ function, which with the help of
> the revision API’s will be able to perform the same function as ‘branch -l
> --merged’.

One trick with making a library-like interface is that some of the
selection routines can work on a "streaming" list of refs (i.e., as we
see each ref we can say "yes" or "no") and some must wait until the end
(e.g., --merged does a single big merge traversal). It's probably not
the end of the world to just always collect all the refs, then filter
them, then sort them, then print them. It may delay the start of output
in some minor cases, but I doubt that's a big deal (and anyway, the
packed-refs code will load them all into an array anyway, so collecting
them in a second array is probably not a big deal).

> For formatting functionality provided by ‘for-each-ref’ we replicate the
> ‘show_ref’ function in ‘for-each-ref.c’ where the format is given to the
> function and the function uses the format to obtain atom values and prints
> the corresponding atom values to the screen. This feature would allow us to
> provide format functionality which could act as a base for the ‘-v’ option
> also.

Yeah, I'd really like to see "--format" for "git branch", and have "-v"
just feed that a hard-coded format string (or even a configurable one).

> Although Jeff has built a really good base to build upon, I shall use
> his work as more of a reference and work on unification of the three
> commands from scratch.

Good. :)

-Peff

Re: [RFC/GSoC] Proposal Draft: Unifying git branch -l, git tag -l, and git for-each-ref

[karthik nayak]

On 03/26/2015 10:07 PM, Jeff King wrote:
> On Mon, Mar 23, 2015 at 06:39:20PM +0530, karthik nayak wrote:
>
> > All three commands select a subset of the repository’s refs and print the
> > result. There has been an attempt to unify these commands by Jeff King[3]. I
> > plan on continuing his work[4] and using his approach to tackle this
> > project.
>
> I would be cautious about the work in my for-each-ref-contains-wip
> branch. At one point it was reasonably solid, but it's now a year and a
> half old, and I've been rebasing it without paying _too_ much attention
> to correctness. I think some subtle bugs have been introduced as it has
> been carried forward.
>
> Also, the very first patch (factoring out the contains traversal) is
> probably better served by this series:
>
>    http://thread.gmane.org/gmane.comp.version-control.git/252472
>
> I don't remember all of the issues offhand that need to be addressed in
> it, but there were plenty of review comments.
Thanks for the link, will go through that.
>
> > For extended selection behaviour such as ‘--contains’ or ‘--merged’ we could
> > implement these within
> > the library by providing functions which closely mimic the current methods
> > used individually by ‘branch -l’ and ‘tag -l’. For eg to implement
> > ‘--merged’ we implement a ‘compute_merge()’ function, which with the help of
> > the revision API’s will be able to perform the same function as ‘branch -l
> > --merged’.
>
> One trick with making a library-like interface is that some of the
> selection routines can work on a "streaming" list of refs (i.e., as we
> see each ref we can say "yes" or "no") and some must wait until the end
> (e.g., --merged does a single big merge traversal). It's probably not
> the end of the world to just always collect all the refs, then filter
> them, then sort them, then print them. It may delay the start of output
> in some minor cases, but I doubt that's a big deal (and anyway, the
> packed-refs code will load them all into an array anyway, so collecting
> them in a second array is probably not a big deal).
I think I noted this down while going through your implementation also.
You even mentioned this on the mailing list if I'm not wrong.
Will have to work out a design around this and think about it more.
>
> > For formatting functionality provided by ‘for-each-ref’ we replicate the
> > ‘show_ref’ function in ‘for-each-ref.c’ where the format is given to the
> > function and the function uses the format to obtain atom values and prints
> > the corresponding atom values to the screen. This feature would allow us to
> > provide format functionality which could act as a base for the ‘-v’ option
> > also.
>
> Yeah, I'd really like to see "--format" for "git branch", and have "-v"
> just feed that a hard-coded format string (or even a configurable one).
>
> > Although Jeff has built a really good base to build upon, I shall use
> > his work as more of a reference and work on unification of the three
> > commands from scratch.
>
> Good. :)
>
> -Peff
>
Thanks for the Review/Tips.

Regards
-Karthik