[GSOC] Git Configuration API improvements

[GSOC] Git Configuration API improvements

[karthik nayak]

Hello to all,

I'm Karthik Nayak, a Computer Science student from Bangalore India. I
will be applying for GSOC 2014. This is my first time applying for
GSOC. I have been using Git for a long time now, so it would be an
ideal organisation for me to contribute to.
As suggested I completed the Micro Project under the guidance of Eric
Sunshine.[1]

I went through the Idea's page[2] and the mails corresponding to the
topic[3]. Currently we have multiple invocation of git_config() in an
individual invocation of git() which is not efficient. Also, it is
hard to implement new features, as mentioned -- such as allowing a
configuration to be unset.

The basic idea is to use a data structure to store the config, the
first time the git_config() is called. And refer to this each and
every time we invoke git_config() further. Jeff suggested [4] to use a
tree or a mapping of keys to values which are stored in a string. I
think that the idea of storing using the config as a tree data
structure would be really advantageous.

This would allow us to easily implement modifications later, and can
help to easily take care of duplicates being created on multiple usage
of config set and unset. As, whenever a node's children have been
deleted the node can also be automatically deleted. The tree can be a
struct with values, header link and link to other configs. This way we
can also create functions to work on the tree easily.

Would be great to hear your thoughts on this topic and also I plan to
start creating a proposal. Nice to have any suggestions or feedback of
any kind.

Thank you,
Karthik

[1] : http://article.gmane.org/gmane.comp.version-control.git/243695/match=karthik+188+gmail+com
[2] : http://git.github.io/SoC-2014-Ideas.html
[3] : http://article.gmane.org/gmane.comp.version-control.git/243500/match=git+configuration+caching
[4] : http://article.gmane.org/gmane.comp.version-control.git/243542

Re: [GSOC] Git Configuration API improvements

[Matthieu Moy]

karthik nayak <karthik.188@gmail.com> writes:

> Currently we have multiple invocation of git_config() in an
> individual invocation of git() which is not efficient. Also, it is
> hard to implement new features,

I think efficiency is not a real concern here. Config files are small
and easy to parse, so parsing them multiple times isn't really
noticeable from the performance point of view.

OTOH, the extensibility is a real concern, and that should be the main
motivation for the project.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [GSOC] Git Configuration API improvements

[karthik nayak]

On Tue, Mar 11, 2014 at 8:21 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> karthik nayak <karthik.188@gmail.com> writes:
>
>> Currently we have multiple invocation of git_config() in an
>> individual invocation of git() which is not efficient. Also, it is
>> hard to implement new features,
>
> I think efficiency is not a real concern here. Config files are small
> and easy to parse, so parsing them multiple times isn't really
> noticeable from the performance point of view.
>
> OTOH, the extensibility is a real concern, and that should be the main
> motivation for the project.
>
> --
> Matthieu Moy
> http://www-verimag.imag.fr/~moy/

Hello Matthieu,

Thanks. I understand what you mean. extensibility is the main motivation of the
project, i think that by implementing the cache system we can fix the
small problems
(reappearing of headers while setting and unsetting configs) and also
implement new features
like to unset a config easily.

Re: [GSOC] Git Configuration API improvements

[Jeff King]

On Tue, Mar 11, 2014 at 09:49:33PM +0530, karthik nayak wrote:

> On Tue, Mar 11, 2014 at 8:21 PM, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
> > karthik nayak <karthik.188@gmail.com> writes:
> >
> >> Currently we have multiple invocation of git_config() in an
> >> individual invocation of git() which is not efficient. Also, it is
> >> hard to implement new features,
> >
> > I think efficiency is not a real concern here. Config files are small
> > and easy to parse, so parsing them multiple times isn't really
> > noticeable from the performance point of view.
> >
> > OTOH, the extensibility is a real concern, and that should be the main
> > motivation for the project.
>
> Thanks. I understand what you mean. extensibility is the main motivation of the
> project, i think that by implementing the cache system we can fix the
> small problems
> (reappearing of headers while setting and unsetting configs) and also
> implement new features
> like to unset a config easily.

I think the most interesting part of the config idea is turning the
fetching of config variables "inside out".

That is, right now we turn control over to the config code, which
invokes our callbacks. So we see all variables in sequence, and pick out
the ones that are interesting.  We implement precedence with a "last one
wins" technique where we keep overwriting a variable with subsequent
config options.

This can lead to difficult ordering situations, such as when a config
option _might_ be respected based on another factor (e.g., the presence
of a command line option, as in the status.branch option).

It also means that it's impossible to tell after the fact whether a
value was set explicitly on the command line, by config, or if we are
using a system default. Most of the time this doesn't matter, but there
are a few cases where we care, and we end up having to manually manage
a separate flag in each case.

By the phrase "inside out" above, I mean that we could read all config,
and then fetch individual values as we need them. We do need to care
about efficiency here, but only insofar as we don't cause any
regressions (that is, the current system is fine speed-wise, we just
need to make sure that we do not create new problems by calling a slow
lookup in a tight loop or anything like that).

-Peff

Re: [GSOC] Git Configuration API improvements

[karthik nayak]

On Fri, Mar 14, 2014 at 10:01 AM, Jeff King <peff@peff.net> wrote:
> On Tue, Mar 11, 2014 at 09:49:33PM +0530, karthik nayak wrote:
>
>> On Tue, Mar 11, 2014 at 8:21 PM, Matthieu Moy
>> <Matthieu.Moy@grenoble-inp.fr> wrote:
>> > karthik nayak <karthik.188@gmail.com> writes:
>> >
>> >> Currently we have multiple invocation of git_config() in an
>> >> individual invocation of git() which is not efficient. Also, it is
>> >> hard to implement new features,
>> >
>> > I think efficiency is not a real concern here. Config files are small
>> > and easy to parse, so parsing them multiple times isn't really
>> > noticeable from the performance point of view.
>> >
>> > OTOH, the extensibility is a real concern, and that should be the main
>> > motivation for the project.
>>
>> Thanks. I understand what you mean. extensibility is the main motivation of the
>> project, i think that by implementing the cache system we can fix the
>> small problems
>> (reappearing of headers while setting and unsetting configs) and also
>> implement new features
>> like to unset a config easily.
>
> I think the most interesting part of the config idea is turning the
> fetching of config variables "inside out".
>
> That is, right now we turn control over to the config code, which
> invokes our callbacks. So we see all variables in sequence, and pick out
> the ones that are interesting.  We implement precedence with a "last one
> wins" technique where we keep overwriting a variable with subsequent
> config options.
>
> This can lead to difficult ordering situations, such as when a config
> option _might_ be respected based on another factor (e.g., the presence
> of a command line option, as in the status.branch option).
>
> It also means that it's impossible to tell after the fact whether a
> value was set explicitly on the command line, by config, or if we are
> using a system default. Most of the time this doesn't matter, but there
> are a few cases where we care, and we end up having to manually manage
> a separate flag in each case.
>
> By the phrase "inside out" above, I mean that we could read all config,
> and then fetch individual values as we need them. We do need to care
> about efficiency here, but only insofar as we don't cause any
> regressions (that is, the current system is fine speed-wise, we just
> need to make sure that we do not create new problems by calling a slow
> lookup in a tight loop or anything like that).
>
> -Peff

Hello Jeff,
Like you said yes, this will be a complete "inside out" change. Thanks
for summing it up, really helpful.
Currently i am going through the code and understanding how it works currently.
Simultaneously working on the proposal[1], would be great to have your
feedback on that.
Thanks,
Karthik

[1] https://gist.github.com/KarthikNayak/98569dd34326f7e6813a