Proposal: sharing .git/config

Proposal: sharing .git/config

[Ramkumar Ramachandra]

Hi,

I have this itch where I want to share my remotes config between
machines.  In my fork, I should be able to specify where my upstream
sources are, so remotes get set up automatically when I clone.  There
are also other things in .git/config that would be nice to share, like
whether to do a --word-diff (why isn't it a configuration variable
yet?) on the repository.  The only problem is that I have no clue how
to implement this: I'm currently thinking a special remote ref?

Ram

Re: Proposal: sharing .git/config

[Thomas Rast]

Ramkumar Ramachandra <artagnon@gmail.com> writes:

> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.

Note that you need to carefully pick only certain bits of the config, as
otherwise there are big security headaches.

> There are also other things in .git/config that would be nice to
> share, like whether to do a --word-diff (why isn't it a configuration
> variable yet?)

Because that would break pretty much every script that uses git-diff?

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Proposal: sharing .git/config

[Ramkumar Ramachandra]

Thomas Rast wrote:
> Ramkumar Ramachandra <artagnon@gmail.com> writes:
>
>> I have this itch where I want to share my remotes config between
>> machines.  In my fork, I should be able to specify where my upstream
>> sources are, so remotes get set up automatically when I clone.
>
> Note that you need to carefully pick only certain bits of the config, as
> otherwise there are big security headaches.

Right.  So, we can just start with remotes for the moment?  Ideally,
there should be a way to specify which configuration options to
publish.

>> There are also other things in .git/config that would be nice to
>> share, like whether to do a --word-diff (why isn't it a configuration
>> variable yet?)
>
> Because that would break pretty much every script that uses git-diff?

diff.c already makes a differentiation between git_diff_ui_config()
and git_diff_basic_config(); there are  configuration options that
should only be applied when the command is called interactively.

Re: Proposal: sharing .git/config

[Thomas Rast]

Ramkumar Ramachandra <artagnon@gmail.com> writes:

> Thomas Rast wrote:
>> Ramkumar Ramachandra <artagnon@gmail.com> writes:
>>> There are also other things in .git/config that would be nice to
>>> share, like whether to do a --word-diff (why isn't it a configuration
>>> variable yet?)
>>
>> Because that would break pretty much every script that uses git-diff?
>
> diff.c already makes a differentiation between git_diff_ui_config()
> and git_diff_basic_config(); there are  configuration options that
> should only be applied when the command is called interactively.

It still breaks every other use of diff unless you make the diff output
depend on whether the user runs directly at the terminal (possibly using
git's own paging).

For example, if you just say something like 'git diff >file' for
inclusion in an email, you expect that to be a git-apply compatible
diff.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: Proposal: sharing .git/config

[Duy Nguyen]

On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra
<artagnon@gmail.com> wrote:
> Hi,
>
> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.  There
> are also other things in .git/config that would be nice to share, like
> whether to do a --word-diff (why isn't it a configuration variable
> yet?) on the repository.  The only problem is that I have no clue how
> to implement this: I'm currently thinking a special remote ref?

If you check out the config file, then include.path should work. You
could add include.ref to point to a ref, but you need to deal with the
attached security implications. This has been proposed before (and
turned down, I think).
-- 
Duy

Re: Proposal: sharing .git/config

[Jeff King]

On Tue, Feb 19, 2013 at 05:34:43PM +0700, Nguyen Thai Ngoc Duy wrote:

> On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra
> <artagnon@gmail.com> wrote:
> > Hi,
> >
> > I have this itch where I want to share my remotes config between
> > machines.  In my fork, I should be able to specify where my upstream
> > sources are, so remotes get set up automatically when I clone.  There
> > are also other things in .git/config that would be nice to share, like
> > whether to do a --word-diff (why isn't it a configuration variable
> > yet?) on the repository.  The only problem is that I have no clue how
> > to implement this: I'm currently thinking a special remote ref?
> 
> If you check out the config file, then include.path should work. You
> could add include.ref to point to a ref, but you need to deal with the
> attached security implications. This has been proposed before (and
> turned down, I think).

Here's the patch:

  http://article.gmane.org/gmane.comp.version-control.git/189144

The basic argument against it is that you would _not_ want to do:

  $ git config include.ref origin/config

because it's unsafe (you immediately start using config fetched from the
remote, before you even get a chance to inspect it). So the recommended
way to use it is:

  $ git config include.ref config
  $ git show origin/config ;# make sure it looks reasonable
  $ git update-ref refs/config origin/config

  [time passes...]

  $ git fetch
  $ git diff config origin/config ;# inspect changes
  $ git update-ref refs/config origin/config

But it was pointed out that you could also just do:

  $ git config include.ref upstream-config
  $ git show origin/config ;# make sure it looks reasonable
  $ git show origin/config >.git/upstream-config

and so forth. There are some ways that a pure ref can be more
convenient (e.g., if you are carrying local changes on top of the
upstream config and want to merge), but ultimately, you can replicate
any include.ref workflow with include.path by adding a "deploy" step
where you copy the file into $GIT_DIR.

-Peff

Re: Proposal: sharing .git/config

[Ramkumar Ramachandra]

Jeff King wrote:
> On Tue, Feb 19, 2013 at 05:34:43PM +0700, Nguyen Thai Ngoc Duy wrote:
>
>> On Tue, Feb 19, 2013 at 4:25 PM, Ramkumar Ramachandra
>> <artagnon@gmail.com> wrote:
>> > Hi,
>> >
>> > I have this itch where I want to share my remotes config between
>> > machines.  In my fork, I should be able to specify where my upstream
>> > sources are, so remotes get set up automatically when I clone.  There
>> > are also other things in .git/config that would be nice to share, like
>> > whether to do a --word-diff (why isn't it a configuration variable
>> > yet?) on the repository.  The only problem is that I have no clue how
>> > to implement this: I'm currently thinking a special remote ref?
>>
>> If you check out the config file, then include.path should work. You
>> could add include.ref to point to a ref, but you need to deal with the
>> attached security implications. This has been proposed before (and
>> turned down, I think).
>
> Here's the patch:
>
>   http://article.gmane.org/gmane.comp.version-control.git/189144
>
> The basic argument against it is that you would _not_ want to do:
>
>   $ git config include.ref origin/config
>
> because it's unsafe (you immediately start using config fetched from the
> remote, before you even get a chance to inspect it). So the recommended
> way to use it is:
>
>   $ git config include.ref config
>   $ git show origin/config ;# make sure it looks reasonable
>   $ git update-ref refs/config origin/config
>
>   [time passes...]
>
>   $ git fetch
>   $ git diff config origin/config ;# inspect changes
>   $ git update-ref refs/config origin/config
>
> But it was pointed out that you could also just do:
>
>   $ git config include.ref upstream-config
>   $ git show origin/config ;# make sure it looks reasonable
>   $ git show origin/config >.git/upstream-config
>
> and so forth. There are some ways that a pure ref can be more
> convenient (e.g., if you are carrying local changes on top of the
> upstream config and want to merge), but ultimately, you can replicate
> any include.ref workflow with include.path by adding a "deploy" step
> where you copy the file into $GIT_DIR.

This seems to be unnecessarily complex and inelegant.  Maybe this
functionality is best managed as a separate git repository: `repo`
from depot_tools uses a manifest repository containing all the project
metadata.  Maybe we can extend it/ write an more general version?

Re: Proposal: sharing .git/config

[Jeff King]

On Tue, Mar 12, 2013 at 01:01:08AM +0530, Ramkumar Ramachandra wrote:

> > But it was pointed out that you could also just do:
> >
> >   $ git config include.ref upstream-config
> >   $ git show origin/config ;# make sure it looks reasonable
> >   $ git show origin/config >.git/upstream-config
> >
> > and so forth. There are some ways that a pure ref can be more
> > convenient (e.g., if you are carrying local changes on top of the
> > upstream config and want to merge), but ultimately, you can replicate
> > any include.ref workflow with include.path by adding a "deploy" step
> > where you copy the file into $GIT_DIR.
> 
> This seems to be unnecessarily complex and inelegant.  Maybe this
> functionality is best managed as a separate git repository: `repo`
> from depot_tools uses a manifest repository containing all the project
> metadata.  Maybe we can extend it/ write an more general version?

I don't think you can avoid the 3-step problem and retain the safety in
the general case.  Forgetting implementation details for a minute, you
have either a 1-step system:

  1. Fetch and start using config from the remote.

which is subject to fetching and executing malicious config, or:

  1. Fetch config from remote.
  2. Inspect it.
  3. Integrate it into the current config.

We can automate the sequence to remove as much friction as possible, but
fundamentally step 2 requires some effort from the user.  Moving the
config to a separate repo does not get rid of those steps.  The user
either does not look at the config before using it, in which case we are
no better than the 1-step scenario, or they do, in which case they are
replicating the 3-step scenario.

The other alternative is to automate step 2. The simplest way would be
to have a whitelist of "ok to share" config, that would not include
things like diff.external that can run arbitrary code. I don't know
whether that would make the system too limited for what people want to
do. Do we have a concrete example of what config people would like to
share in this manner?

-Peff

Re: Proposal: sharing .git/config

[Ramkumar Ramachandra]

Jeff King wrote:
> I don't think you can avoid the 3-step problem and retain the safety in
> the general case.  Forgetting implementation details for a minute, you
> have either a 1-step system:
>
>   1. Fetch and start using config from the remote.
>
> which is subject to fetching and executing malicious config, or:
>
>   1. Fetch config from remote.
>   2. Inspect it.
>   3. Integrate it into the current config.

I don't understand your emphasis on step 2.  Isn't the configuration
written by me?  Why would it be malicious?

I've just started thinking about how to design something that will
allow us to share configuration elegantly [1].  Essentially, the
metadata repository will consist of *.layout files, one for each
repository to clone, containing the .git/config to write after cloning
that repository.  So, a git.layout might look like:

[layout]
	directory = git
[remote "origin"]
	url = git://github.com/git/git
[remote "ram"]
	url = git@github.com:artagnon/git
[remote "junio"]
	url = git://github.com/gitster/git

As you can see the [layout] is a special section which will tell our
fetcher where to place the repository.  Everything else is meant to be
inserted into the repository's .git/config.  However, I can foresee a
problem in scaling: when I ask a specific directory like a/b/c to be
populated (equivalent of repo sync `a/b/c`), it'll have to parse the
layout.directory variable of all the .layout files, and this can be
slow.  So, maybe we should have a special _manifest.layout listing all
the paths?

Further, I see this as a way to work with projects that would
otherwise require nested submodules like the Android project.  What do
you think?

[1]: https://github.com/artagnon/src.layout

Re: Proposal: sharing .git/config

[Jeff King]

On Mon, Mar 18, 2013 at 02:30:23PM +0530, Ramkumar Ramachandra wrote:

> Jeff King wrote:
> > I don't think you can avoid the 3-step problem and retain the safety in
> > the general case.  Forgetting implementation details for a minute, you
> > have either a 1-step system:
> >
> >   1. Fetch and start using config from the remote.
> >
> > which is subject to fetching and executing malicious config, or:
> >
> >   1. Fetch config from remote.
> >   2. Inspect it.
> >   3. Integrate it into the current config.
> 
> I don't understand your emphasis on step 2.  Isn't the configuration
> written by me?  Why would it be malicious?

Maybe I am misunderstanding the use case, but when people talk about
share config, they are often talking about pushing project-wide config
out to developers. So the config is not necessarily written by you, but
by somebody who had write access to the upstream repository.

The obvious counterpoint is that people usually run "make" right after
fetching, so they are trusting what they fetched already. And the
counter-counterpoint is that yes, that's true, but at least with the
"make" case they can use git to inspect the differences before running
them. You may be able to tell that this is not the first time this
discussion has happened. :)

Personally, I do not think it is the end of the world for people to opt
into the "automatically fetch and respect config" method for certain
repositories (and that's why I wrote include.ref support a while ago).
It's a security tradeoff that the user may want to make. But I also
respect the argument that we should not be endorsing risky behavior by
advertising such a feature (especially when the risk is quite subtle, as
many users may not realize that git config can execute arbitrary code).

> I've just started thinking about how to design something that will
> allow us to share configuration elegantly [1].  Essentially, the
> metadata repository will consist of *.layout files, one for each
> repository to clone, containing the .git/config to write after cloning
> that repository.  So, a git.layout might look like:
> 
> [layout]
> 	directory = git
> [remote "origin"]
> 	url = git://github.com/git/git
> [remote "ram"]
> 	url = git@github.com:artagnon/git
> [remote "junio"]
> 	url = git://github.com/gitster/git
> 
> As you can see the [layout] is a special section which will tell our
> fetcher where to place the repository.  Everything else is meant to be
> inserted into the repository's .git/config.  However, I can foresee a
> problem in scaling: when I ask a specific directory like a/b/c to be
> populated (equivalent of repo sync `a/b/c`), it'll have to parse the
> layout.directory variable of all the .layout files, and this can be
> slow.  So, maybe we should have a special _manifest.layout listing all
> the paths?
> 
> Further, I see this as a way to work with projects that would
> otherwise require nested submodules like the Android project.  What do
> you think?

Yeah, reading your layout description, this is less about git config in
particular, and more about managing hierarchies of repos. Which I think
is a fine thing to do, and is a sensible place to put config management
(since you are probably executing arbitrary code as part of the layout
tool anyway). But I don't have a real opinion on the design of such a
tool. I have used repo only once or twice to deal with Android. For my
own menagerie of small repos, I have a hacky custom tool that is mostly
about deciding when there are items to be committed, pushed, or fetched
in each repo; I never found the need to handle git config at all.

-Peff

Re: Proposal: sharing .git/config

[Adam Spiers]

On Tue, Feb 19, 2013 at 9:25 AM, Ramkumar Ramachandra
<artagnon@gmail.com> wrote:
> Hi,
>
> I have this itch where I want to share my remotes config between
> machines.  In my fork, I should be able to specify where my upstream
> sources are, so remotes get set up automatically when I clone.  There
> are also other things in .git/config that would be nice to share, like
> whether to do a --word-diff (why isn't it a configuration variable
> yet?) on the repository.  The only problem is that I have no clue how
> to implement this: I'm currently thinking a special remote ref?

I handle these kinds of configuration tasks out of band using mr,
and it works pretty well:

  https://github.com/aspiers/mr-config/#readme
  https://github.com/aspiers/mr-config/blob/master/sh.d/git-remotes

Food for thought ...