Git credential helper store flushes randomly

Git credential helper store flushes randomly

[Райцин Антон]

Hello.

I have very strange git credentials store behavior on one of my servers.
I Use Ubuntu 14.04 LTS and git  2.10.2. The server have multiple users 
with multiple projects, so they have got many different git credentials 
to different repositories.
I set git config --global credential.helper store, but the credential 
record strings from file ~/.git-credentials for one specific user 
disappears randomly. Especially for one specific repository.

I do not see any TTL for git credentials in credential.helper store on 
git documentation, so I can't even imagine, what could cause such 
strange behavior.

Is this a known problem and is there any solution to fix this problem?


Best regards,
Anton.

Re: Git credential helper store flushes randomly

[Jeff King]

On Fri, May 12, 2017 at 11:05:19AM +0300, Райцин Антон wrote:

> I have very strange git credentials store behavior on one of my servers.
> I Use Ubuntu 14.04 LTS and git  2.10.2. The server have multiple users with
> multiple projects, so they have got many different git credentials to
> different repositories.
> I set git config --global credential.helper store, but the credential record
> strings from file ~/.git-credentials for one specific user disappears
> randomly. Especially for one specific repository.
> 
> I do not see any TTL for git credentials in credential.helper store on git
> documentation, so I can't even imagine, what could cause such strange
> behavior.
> 
> Is this a known problem and is there any solution to fix this problem?

I've never heard of a bug like this, and the credential-store code
hasn't really changed in the 5+ years since it was introduced.

The only time it should remove an entry is when Git asks it to. And the
only time that happens is when Git sees the credential rejected by the
server (e.g., an HTTP 401 even after we fed the stored credential). I
don't know why that would happen unless there's some non-determinism on
the server.

Running with GIT_TRACE=/path/to/logfile in the environment would let you
know when Git invokes the helper with the "erase" argument. But the
credential data itself is sent over stdin, so it won't be logged. And if
this is something that happens occasionally and randomly, you might have
to log for quite a while.

Likewise, running with GIT_CURL_VERBOSE=1 might show what's going on,
but you'd have to actually catch the offending git command (in more
recent versions of curl there is GIT_TRACE_CURL, which you can point at
a specific logfile rather than just stderr). Do note if you try logging
with GIT_CURL_VERBOSE that it includes your password in the output, so
treat it appropriately.

-Peff

Re: Git credential helper store flushes randomly

[Matthieu Moy]

Jeff King <peff@peff.net> writes:

> The only time it should remove an entry is when Git asks it to. And the
> only time that happens is when Git sees the credential rejected by the
> server (e.g., an HTTP 401 even after we fed the stored credential). I
> don't know why that would happen unless there's some non-determinism on
> the server.

I did see a case like this where the server was broken temporarily and
rejected one login attempt. In this case the credential store deletes
the entry for that user, and when the server is repaired, the store
still has the entry deleted.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Git credential helper store flushes randomly

[Jeff King]

On Fri, May 12, 2017 at 10:24:50AM +0200, Matthieu Moy wrote:

> Jeff King <peff@peff.net> writes:
> 
> > The only time it should remove an entry is when Git asks it to. And the
> > only time that happens is when Git sees the credential rejected by the
> > server (e.g., an HTTP 401 even after we fed the stored credential). I
> > don't know why that would happen unless there's some non-determinism on
> > the server.
> 
> I did see a case like this where the server was broken temporarily and
> rejected one login attempt. In this case the credential store deletes
> the entry for that user, and when the server is repaired, the store
> still has the entry deleted.

Right, that's inconvenient if your server is flaky, but is the expected
behavior. Git has to throw away the credential so it can stop trying it
and actually prompt you on the next try (and save the result then if it
works).

-Peff

Re: Git credential helper store flushes randomly

[Райцин Антон]

12.05.2017 11:45, Jeff King пишет:
> On Fri, May 12, 2017 at 10:24:50AM +0200, Matthieu Moy wrote:
>
>> Jeff King <peff@peff.net> writes:
>>
>>> The only time it should remove an entry is when Git asks it to. And the
>>> only time that happens is when Git sees the credential rejected by the
>>> server (e.g., an HTTP 401 even after we fed the stored credential). I
>>> don't know why that would happen unless there's some non-determinism on
>>> the server.
>> I did see a case like this where the server was broken temporarily and
>> rejected one login attempt. In this case the credential store deletes
>> the entry for that user, and when the server is repaired, the store
>> still has the entry deleted.
> Right, that's inconvenient if your server is flaky, but is the expected
> behavior. Git has to throw away the credential so it can stop trying it
> and actually prompt you on the next try (and save the result then if it
> works).
>
> -Peff
>

Thank you all!
I've found the reason and successfully fixed the problem. Here is a 
description.

Most of repositories of this user were on one server EXAMPLE.COM. Some 
origins were configured like https://USER@EXAMPLE.COM. And they worked 
fine. Git just searched in ~/.git-credentials the 
"USER:password@EXAMPLE.COM" entry and everything was good. But some 
repositories' origins were configured without username, just 
https://EXAMPLE.COM. So git tried to find just a EXAMPLE.COM in the 
credentials store. Then git took the first entry and tried to connect to 
remote origin, got 401 unauthorized error and deleted first entry from 
helper store. The randomness was manifested due to the fact that git 
rotates the records in .git-credentials every time it connects to remote 
origin, it moves the right account string to the top of the file.

I think it will be good idea to add this information to 
https://git-scm.com/docs/git-credential-store, especially the info about 
deletion behavior, when git gets 401 error.

Best regards,
Anton.