Overriding ~/.gitconfig using GIT_CONFIG

Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

I work on build systems which need to check code out of source control
systems like git. I need to do this in a way in which there are not
outside influences and a user's $HOME/.gitconfig file can contain things
we don't want. I therefore need a way to disable it.

Looking through the manuals/code, it suggests I should be able to do:

GIT_CONFIG=/dev/null git XXX

and all should work happily. It doesn't though. As an example, with a
~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
accessing the file in ~ and then acting upon it.

I've searched through the code and whilst config_exclusive_filename
appears to be the magic variable that should get set when I set
GIT_CONFIG in the environment, nothing ever sets it apart from within a
git-config command.

The following patch sets the variable from the environment initially and
should do what the code intends if I read understand it correctly. Its
not ideal for my use case as I actually want the repo_config and only
the repo config to be used but I can live with setting GIT_CONFIG to
point at it.

Alternative ideas welcome, I've considered changing $HOME but that seems
a little too ugly and likely to cause other problems.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/config.c b/config.c
index d06fb19..19e7565 100644
--- a/config.c
+++ b/config.c
@@ -860,6 +860,8 @@ int git_config_early(config_fn_t fn, void *data, const char *repo_config)
 	int ret = 0, found = 0;
 	const char *home = NULL;
 
+	config_exclusive_filename = getenv(CONFIG_ENVIRONMENT);
+
 	/* Setting $GIT_CONFIG makes git read _only_ the given config file. */
 	if (config_exclusive_filename)
 		return git_config_from_file(fn, config_exclusive_filename, data);


-- 
Linux Foundation
http://www.yoctoproject.org/

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Junio C Hamano]

Richard Purdie <rpurdie@rpsys.net> writes:

> Looking through the manuals/code, it suggests I should be able to do:
>
> GIT_CONFIG=/dev/null git XXX
>
> and all should work happily. It doesn't though. As an example, with a
> ~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
> accessing the file in ~ and then acting upon it.

If the manual says the above is expected for any value of XXX, then that
is a bug in the manual since mid 2008, I think.

See dc87183 (Only use GIT_CONFIG in "git config", not other programs,
2008-06-30).

I _think_ these days a workaround to force a known config is to set HOME
to a value that has a known .gitconfig (or no such file), and decline
usage of /etc/git.config by exporting GIT_CONFIG_NOSYSTEM.

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Richard Purdie <rpurdie@rpsys.net> writes:
>
>> Looking through the manuals/code, it suggests I should be able to do:
>>
>> GIT_CONFIG=/dev/null git XXX
>>
>> and all should work happily. It doesn't though. As an example, with a
>> ~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
>> accessing the file in ~ and then acting upon it.
>
> If the manual says the above is expected for any value of XXX, then that
> is a bug in the manual since mid 2008, I think.
>
> See dc87183 (Only use GIT_CONFIG in "git config", not other programs,
> 2008-06-30).
>
> I _think_ these days a workaround to force a known config is to set HOME
> to a value that has a known .gitconfig (or no such file), and decline
> usage of /etc/git.config by exporting GIT_CONFIG_NOSYSTEM.

Side note. Here is what dc87183 says:

commit dc87183189b54441e315d35d48983d80ab085299
Author: Daniel Barkalow <barkalow@iabervon.org>
Date:   Mon Jun 30 03:37:47 2008 -0400

    Only use GIT_CONFIG in "git config", not other programs
    
    For everything other than using "git config" to read or write a
    git-style config file that isn't the current repo's config file,
    GIT_CONFIG was actively detrimental. Rather than argue over which
    programs are important enough to have work anyway, just fix all of
    them at the root.
    
    Also removes GIT_LOCAL_CONFIG, which would only be useful for programs
    that do want to use global git-specific config, but not the repo's own
    git-specific config, and want to use some other, presumably
    git-specific config. Despite being documented, I can't find any sign that
    it was ever used.
    
    Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

It clearly explains the reason why LOCAL_CONFIG was removed (the reader
does not have to agree with "I can't find any sign that it was ever used",
though), but I cannot read from the first paragraph the reason why it was
felt necessary not to honor GIT_CONFIG in other programs, i.e. "was
actively detrimental" is not backed by any example in the paragraph. I can
sort of sense from "Rather than argue over..." that there may have been a
discussion on the list, and reading the archive from that timeframe may
reveal why many felt it was not a good idea.

Daniel, do you recall the context?

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

On Fri, 2011-08-12 at 12:39 -0700, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Richard Purdie <rpurdie@rpsys.net> writes:
> >
> >> Looking through the manuals/code, it suggests I should be able to do:
> >>
> >> GIT_CONFIG=/dev/null git XXX
> >>
> >> and all should work happily. It doesn't though. As an example, with a
> >> ~/.gitconfig, "GIT_CONFIG=/dev/null git fetch --all" is clearly
> >> accessing the file in ~ and then acting upon it.
> >
> > If the manual says the above is expected for any value of XXX, then that
> > is a bug in the manual since mid 2008, I think.
> >
> > See dc87183 (Only use GIT_CONFIG in "git config", not other programs,
> > 2008-06-30).
> >
> > I _think_ these days a workaround to force a known config is to set HOME
> > to a value that has a known .gitconfig (or no such file), and decline
> > usage of /etc/git.config by exporting GIT_CONFIG_NOSYSTEM.
> 
> Side note. Here is what dc87183 says:
> 
> commit dc87183189b54441e315d35d48983d80ab085299
> Author: Daniel Barkalow <barkalow@iabervon.org>
> Date:   Mon Jun 30 03:37:47 2008 -0400
> 
>     Only use GIT_CONFIG in "git config", not other programs
>     
>     For everything other than using "git config" to read or write a
>     git-style config file that isn't the current repo's config file,
>     GIT_CONFIG was actively detrimental. Rather than argue over which
>     programs are important enough to have work anyway, just fix all of
>     them at the root.
>     
>     Also removes GIT_LOCAL_CONFIG, which would only be useful for programs
>     that do want to use global git-specific config, but not the repo's own
>     git-specific config, and want to use some other, presumably
>     git-specific config. Despite being documented, I can't find any sign that
>     it was ever used.
>     
>     Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
> 
> It clearly explains the reason why LOCAL_CONFIG was removed (the reader
> does not have to agree with "I can't find any sign that it was ever used",
> though), but I cannot read from the first paragraph the reason why it was
> felt necessary not to honor GIT_CONFIG in other programs, i.e. "was
> actively detrimental" is not backed by any example in the paragraph. I can
> sort of sense from "Rather than argue over..." that there may have been a
> discussion on the list, and reading the archive from that timeframe may
> reveal why many felt it was not a good idea.
> 
> Daniel, do you recall the context?

I went digging and this looks like as good a summary as any of the posts
around that time:

http://marc.info/?l=git&m=121476432303314&w=2

It sounds like if you specified GIT_CONFIG when making a clone it would
end up writing the config file specified rather than .git/config.

My problem isn't that I want to specify a specific .gitconfig file, I
just need it to ignore the one in $HOME. I'm happy for the .git/config
file to be used, in fact I need it to be.

I noticed 8f323c00dd3c9b396b01a1aeea74f7dfd061bb7f was committed which
removed GIT_CONFIG_NOGLOBAL support which is the other way to address
the problem. Could we add that back?

I appreciate I can set $HOME to something but that means creating an
empty directory to point at and feels rather like a work around rather
than a solution.

Cheers,

Richard

-- 
Linux Foundation
http://www.yoctoproject.org/

Re: Overriding ~/.gitconfig using GIT_CONFIG

[David Aguilar]

On Fri, Aug 12, 2011 at 09:44:13PM +0100, Richard Purdie wrote:
> My problem isn't that I want to specify a specific .gitconfig file, I
> just need it to ignore the one in $HOME. I'm happy for the .git/config
> file to be used, in fact I need it to be.

If you're writing a tool then it should restrict itself to
git's plumbing commands.  You should be able to do just about
anything without needing to worry about differences in
configuraiton.  Git commands almost always provide a way to
override configuration through the use of flags.

The plumbing commands are listed in the main git manpage.
See "Low-level commands (plumbing)" here:
http://www.kernel.org/pub/software/scm/git/docs/

What is the specific problem solved by overriding the
configuration?  It may be possible to solve it without needing
to get too complicated.

Have fun,
-- 
					David

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

On Sun, 2011-08-28 at 06:05 -0700, David Aguilar wrote:
> On Fri, Aug 12, 2011 at 09:44:13PM +0100, Richard Purdie wrote:
> > My problem isn't that I want to specify a specific .gitconfig file, I
> > just need it to ignore the one in $HOME. I'm happy for the .git/config
> > file to be used, in fact I need it to be.
> 
> If you're writing a tool then it should restrict itself to
> git's plumbing commands.  You should be able to do just about
> anything without needing to worry about differences in
> configuraiton.  Git commands almost always provide a way to
> override configuration through the use of flags.
> 
> The plumbing commands are listed in the main git manpage.
> See "Low-level commands (plumbing)" here:
> http://www.kernel.org/pub/software/scm/git/docs/
> 
> What is the specific problem solved by overriding the
> configuration?  It may be possible to solve it without needing
> to get too complicated.

I'm not sure writing my own porcelain makes sense in this case.

The tool in question is a build system which is primarily interested in
building software. Sometimes the software we want to build is "bleeding
edge" and hence rather than download tarballs, we want to interact
directly with SCMs like git to obtain it.

The commands I'm using are the likes of "git clone" and "git fetch"
although we do use commands listed under the plumbing section too such
as ls-remote and read-tree. We do "cache" checkouts and support
automatically noticing changes and updating/building.

What I do want to be able to say is "ignore whatever the user might have
put in their ~/.gitconfig file" since I've open bug reports about people
putting things in there that break builds.

The fetch/clone commands do what I need, apart from being influenced by
userconfig so reimplementing them myself doesn't seem like a good
approach.

Cheers,

Richard

-- 
Linux Foundation
http://www.yoctoproject.org/

Re: Overriding ~/.gitconfig using GIT_CONFIG

[David Aguilar]

On Mon, Aug 29, 2011 at 01:16:06PM +0100, Richard Purdie wrote:
> On Sun, 2011-08-28 at 06:05 -0700, David Aguilar wrote:
> > On Fri, Aug 12, 2011 at 09:44:13PM +0100, Richard Purdie wrote:
> > > My problem isn't that I want to specify a specific .gitconfig file, I
> > > just need it to ignore the one in $HOME. I'm happy for the .git/config
> > > file to be used, in fact I need it to be.
> > 
> > If you're writing a tool then it should restrict itself to
> > git's plumbing commands.  You should be able to do just about
> > anything without needing to worry about differences in
> > configuraiton.  Git commands almost always provide a way to
> > override configuration through the use of flags.
> > 
> > The plumbing commands are listed in the main git manpage.
> > See "Low-level commands (plumbing)" here:
> > http://www.kernel.org/pub/software/scm/git/docs/
> > 
> > What is the specific problem solved by overriding the
> > configuration?  It may be possible to solve it without needing
> > to get too complicated.
> 
> I'm not sure writing my own porcelain makes sense in this case.

True...  luckily we don't have to go that far.

> The tool in question is a build system which is primarily interested in
> building software. Sometimes the software we want to build is "bleeding
> edge" and hence rather than download tarballs, we want to interact
> directly with SCMs like git to obtain it.
> 
> The commands I'm using are the likes of "git clone" and "git fetch"
> although we do use commands listed under the plumbing section too such
> as ls-remote and read-tree. We do "cache" checkouts and support
> automatically noticing changes and updating/building.
> 
> What I do want to be able to say is "ignore whatever the user might have
> put in their ~/.gitconfig file" since I've open bug reports about people
> putting things in there that break builds.
> 
> The fetch/clone commands do what I need, apart from being influenced by
> userconfig so reimplementing them myself doesn't seem like a good
> approach.

This is what we're interested in.

Do you have the specifics of exactly what in the user
~/.gitconfig file broke the build?

What I'm suggesting is that there's probably a way to avoid
the user-dependent behavior by being explicit on
the command-line.

Specifically regarding fetch -- if you're doing "git fetch"
and relying on the configuration then we can probably come
up with a more explicit fetch command that has an explicit
and predictable behavior independent of the user's
configuraiton.

Using the explicit form of the command can make the build system
more robust.

Let us know what you're cooking, it sounds interesting.

cheers,
-- 
					David

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

On Mon, 2011-08-29 at 20:10 -0700, David Aguilar wrote:
> On Mon, Aug 29, 2011 at 01:16:06PM +0100, Richard Purdie wrote:
> > On Sun, 2011-08-28 at 06:05 -0700, David Aguilar wrote:
> > > On Fri, Aug 12, 2011 at 09:44:13PM +0100, Richard Purdie wrote:
> > > > My problem isn't that I want to specify a specific .gitconfig file, I
> > > > just need it to ignore the one in $HOME. I'm happy for the .git/config
> > > > file to be used, in fact I need it to be.
> > > 
> > > If you're writing a tool then it should restrict itself to
> > > git's plumbing commands.  You should be able to do just about
> > > anything without needing to worry about differences in
> > > configuraiton.  Git commands almost always provide a way to
> > > override configuration through the use of flags.
> > > 
> > > The plumbing commands are listed in the main git manpage.
> > > See "Low-level commands (plumbing)" here:
> > > http://www.kernel.org/pub/software/scm/git/docs/
> > > 
> > > What is the specific problem solved by overriding the
> > > configuration?  It may be possible to solve it without needing
> > > to get too complicated.
> > 
> > I'm not sure writing my own porcelain makes sense in this case.
> 
> True...  luckily we don't have to go that far.
> 
> > The tool in question is a build system which is primarily interested in
> > building software. Sometimes the software we want to build is "bleeding
> > edge" and hence rather than download tarballs, we want to interact
> > directly with SCMs like git to obtain it.
> > 
> > The commands I'm using are the likes of "git clone" and "git fetch"
> > although we do use commands listed under the plumbing section too such
> > as ls-remote and read-tree. We do "cache" checkouts and support
> > automatically noticing changes and updating/building.
> > 
> > What I do want to be able to say is "ignore whatever the user might have
> > put in their ~/.gitconfig file" since I've open bug reports about people
> > putting things in there that break builds.
> > 
> > The fetch/clone commands do what I need, apart from being influenced by
> > userconfig so reimplementing them myself doesn't seem like a good
> > approach.
> 
> This is what we're interested in.
> 
> Do you have the specifics of exactly what in the user
> ~/.gitconfig file broke the build?
> 
> What I'm suggesting is that there's probably a way to avoid
> the user-dependent behavior by being explicit on
> the command-line.
> 
> Specifically regarding fetch -- if you're doing "git fetch"
> and relying on the configuration then we can probably come
> up with a more explicit fetch command that has an explicit
> and predictable behavior independent of the user's
> configuraiton.
> 
> Using the explicit form of the command can make the build system
> more robust.
> 
> Let us know what you're cooking, it sounds interesting.

We've gone through several iterations of this but as things stand now,
to initially clone things we're doing:

git clone --bare --mirror <url> <dir>

but if we already have some existing clone we'd update with:

git remote prune origin
git remote rm origin
git remote add --mirror origin <url>
git fetch --all -t

I have a strong suspicion that the fetch --all doesn't do us any favours
since if the user defines other remotes in their local config it will
also fetch them. We could probably change this to be "git fetch origin
-t" which would be safer however the .gitconfig file that someone
reported causing problems actually defined an origin remote. I don't
want to debate whether that is sane or not, I just want to be 100% sure
we work regardless of what the user may have defined.

For reference, the reasons for the remote commands above is to be 100%
sure that we fetch the right thing if <url> has changed, or if upstream
has deleted any branches in incompatible ways. We've tried not using any
remotes at all but that seems to bring its own set of problems as we
have seen issues with fetch not downloading everything from the upstream
repo (which is what we need since this is a cache/mirror copy). We ended
up needing to be very precise about what we wanted.

When we actually come to build something we make a referenced clone that
contains an actual checkout with the commands:

git clone -s -n <dir> <dir>
git checkout <revision>

which has the nice effect of being kind on disk space and fast :).

Or in some cases where we only want a specific subdirectory:

git read-tree <revision>:<subdir>
git checkout-index -q -f -a

which is fairly evil and leaves the tree in a nasty state but its what
some people need.

In case you're interested, the code in question is part of a tool called
bitbake and is one of several different fetcher modules:

http://git.openembedded.org/cgit.cgi/bitbake/tree/lib/bb/fetch2/git.py

So if a user has an origin remote in their .gitconfig, can we ignore it?

Cheers,

Richard

-- 
Linux Foundation
http://www.yoctoproject.org/

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Jeff King]

On Tue, Aug 30, 2011 at 01:13:01PM +0100, Richard Purdie wrote:

> We've gone through several iterations of this but as things stand now,
> to initially clone things we're doing:
> 
> git clone --bare --mirror <url> <dir>
> 
> but if we already have some existing clone we'd update with:
> 
> git remote prune origin
> git remote rm origin
> git remote add --mirror origin <url>
> git fetch --all -t
>
> [...]
>
> So if a user has an origin remote in their .gitconfig, can we ignore it?

Wouldn't:

  git fetch --prune <url> refs/heads/*:refs/remotes/origin/*

do what you want, and not look at config at all?

-Peff

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

On Tue, 2011-08-30 at 11:56 -0400, Jeff King wrote:
> On Tue, Aug 30, 2011 at 01:13:01PM +0100, Richard Purdie wrote:
> 
> > We've gone through several iterations of this but as things stand now,
> > to initially clone things we're doing:
> > 
> > git clone --bare --mirror <url> <dir>
> > 
> > but if we already have some existing clone we'd update with:
> > 
> > git remote prune origin
> > git remote rm origin
> > git remote add --mirror origin <url>
> > git fetch --all -t
> >
> > [...]
> >
> > So if a user has an origin remote in their .gitconfig, can we ignore it?
> 
> Wouldn't:
> 
>   git fetch --prune <url> refs/heads/*:refs/remotes/origin/*
> 
> do what you want, and not look at config at all?

Since this is a bare/mirror clone, wouldn't that need to be:

 git fetch --prune <url> refs/heads/*:refs/heads/*

?

That also wouldn't fetch tags?

Cheers,

Richard

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Jeff King]

On Tue, Aug 30, 2011 at 07:39:05PM +0100, Richard Purdie wrote:

> On Tue, 2011-08-30 at 11:56 -0400, Jeff King wrote:
> > On Tue, Aug 30, 2011 at 01:13:01PM +0100, Richard Purdie wrote:
> > 
> > > We've gone through several iterations of this but as things stand now,
> > > to initially clone things we're doing:
> > > 
> > > git clone --bare --mirror <url> <dir>
> > > 
> > > but if we already have some existing clone we'd update with:
> > > 
> > > git remote prune origin
> > > git remote rm origin
> > > git remote add --mirror origin <url>
> > > git fetch --all -t
> > >
> > > [...]
> > >
> > > So if a user has an origin remote in their .gitconfig, can we ignore it?
> > 
> > Wouldn't:
> > 
> >   git fetch --prune <url> refs/heads/*:refs/remotes/origin/*
> > 
> > do what you want, and not look at config at all?
> 
> Since this is a bare/mirror clone, wouldn't that need to be:
> 
>  git fetch --prune <url> refs/heads/*:refs/heads/*

Sorry, yes, I forgot about the mirroring bit.

> That also wouldn't fetch tags?

It would only do autofollowing. You could use "-t", but if you really
want a straight mirror, you could do:

  git fetch --prune refs/*:refs/*

to get everything.

-Peff

Re: Overriding ~/.gitconfig using GIT_CONFIG

[Richard Purdie]

On Tue, 2011-08-30 at 14:49 -0400, Jeff King wrote:
> On Tue, Aug 30, 2011 at 07:39:05PM +0100, Richard Purdie wrote:
> 
> > On Tue, 2011-08-30 at 11:56 -0400, Jeff King wrote:
> > > On Tue, Aug 30, 2011 at 01:13:01PM +0100, Richard Purdie wrote:
> > > 
> > > > We've gone through several iterations of this but as things stand now,
> > > > to initially clone things we're doing:
> > > > 
> > > > git clone --bare --mirror <url> <dir>
> > > > 
> > > > but if we already have some existing clone we'd update with:
> > > > 
> > > > git remote prune origin
> > > > git remote rm origin
> > > > git remote add --mirror origin <url>
> > > > git fetch --all -t
> > > >
> > > > [...]
> > > >
> > > > So if a user has an origin remote in their .gitconfig, can we ignore it?
> > > 
> > > Wouldn't:
> > > 
> > >   git fetch --prune <url> refs/heads/*:refs/remotes/origin/*
> > > 
> > > do what you want, and not look at config at all?
> > 
> > Since this is a bare/mirror clone, wouldn't that need to be:
> > 
> >  git fetch --prune <url> refs/heads/*:refs/heads/*
> 
> Sorry, yes, I forgot about the mirroring bit.
> 
> > That also wouldn't fetch tags?
> 
> It would only do autofollowing. You could use "-t", but if you really
> want a straight mirror, you could do:
> 
>   git fetch --prune refs/*:refs/*
> 
> to get everything.

Thanks. Just to updated, I've changed the code to do this so we'll see
how it goes...

Cheers,

Richard