git with https and client cert asks for password repeatedly

git with https and client cert asks for password repeatedly

[Mark Lodato]

First off, I am fairly new to git, so let me apologize in advance if I
suggest anything stupid.

When fetching or pushing over https:// with a client certificate
(http.sslCert / http.sslKey), git asks for a password for every single
requested file.  For example, here I push three commits with a couple
changed files each:

> git push origin master
Enter PEM pass phrase:
Enter PEM pass phrase:
Fetching remote heads...
  refs/
  refs/tags/
  refs/heads/
updating 'refs/heads/master'
  from 1df865db590b4a7d4991c13053437ac90b2780e4
  to   05e856a6a5ce9b05a5a7d10cb5d10010467eea72
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
    sending 12 objects
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
    done
Updating remote server info

To make matters worse, when you try to CTRL-C from the "Enter PEM pass
phrase" prompt, it just re-prompts you!  If you want to see this in
action, set up a webdav server on https://localhost with a copy of
git.git and try cloning it with a password-protected client
certificate.

This problem makes client-side certificates unusable with git.  A
possible workaround is to leave the key unencrypted, but this is
usually unacceptable for security reasons.  Ideally, I would just type
my password once per invocation and git would remember it.  (This is
how svn works.)

I think the root problem is that git creates a completely new http(s)
connection for every request, rather than using one persistent
connection.  Using a persistent connection would theoretically speed
up the transfers, in addition to fixing the password prompt issue.
I'm pretty sure that calling `curl_easy_cleanup()' after every request
is causing this behavior; I don't think this is necessary.

I tried fixing this myself, but the http/curl code is pretty
confusing.  Just wondering - why is HTTP_MULTI required for http-push?
 I saw a thread from Jan '08 about this, but it never said *why*
HTTP_MULTI is required, only that the push doesn't work without it.
It doesn't appear to me that git uses concurrent connections in any
useful way, so I don't know why having a single connection would not
work.


Finally, is there interest in refactoring the http code to make it a
little cleaner?  That is, make a wrapper library around curl so that
you can just call GET or POST or whatever and not worry about how to
invoke curl?


--
Mark Lodato

Re: git with https and client cert asks for password repeatedly

[Daniel Stenberg]

On Tue, 24 Feb 2009, Mark Lodato wrote:

> I think the root problem is that git creates a completely new http(s) 
> connection for every request, rather than using one persistent connection. 
> Using a persistent connection would theoretically speed up the transfers, in 
> addition to fixing the password prompt issue. I'm pretty sure that calling 
> `curl_easy_cleanup()' after every request is causing this behavior; I don't 
> think this is necessary.

This may or may not be true depending on several circumstances. You didn't 
mention what libcurl version nor did you say if you build git to use the multi 
interface. But assuming that you don't use multi or that the libcurl is fairly 
old, your conclusion is correct.

-- 

  / daniel.haxx.se

Re: git with https and client cert asks for password repeatedly

[Josef Wolf]

On Tue, Feb 24, 2009 at 10:11:40PM -0500, Mark Lodato wrote:

> When fetching or pushing over https:// with a client certificate
> (http.sslCert / http.sslKey), git asks for a password for every single
> requested file.  For example, here I push three commits with a couple
> changed files each:
[ ... ]
> This problem makes client-side certificates unusable with git.  A
> possible workaround is to leave the key unencrypted, but this is
> usually unacceptable for security reasons.

For the (pretty much standard) Basic authentication over https, it is
even worse: you have to put your password in plain text into .netrc :-(
Check out http://marc.info/?l=git&m=122426078301793&w=2 if you have a
couple of minutes left.

> Ideally, I would just type
> my password once per invocation and git would remember it.  (This is
> how svn works.)

Yeah, that would be great!

> I think the root problem is that git creates a completely new http(s)
> connection for every request, rather than using one persistent
> connection.  Using a persistent connection would theoretically speed
> up the transfers, in addition to fixing the password prompt issue.
> I'm pretty sure that calling `curl_easy_cleanup()' after every request
> is causing this behavior; I don't think this is necessary.
> 
> I tried fixing this myself, but the http/curl code is pretty
> confusing.  Just wondering - why is HTTP_MULTI required for http-push?

The integration of curl into git looks somewhat strange to me, also.
It seems to implement some sort of request-queue on top of curl-multi
and falls back to re-implement some curl-multi functionality in the
case when only curl-easy is available.  Or something like that.  I have
not looked too much into the details, and have already forgotten what I
saw there.

Instead, I looked into curl-lib to implement some sort of credential
caching. The idea was to implement something like get_basic_credentials()
in the LWP::UserAgent perl module.

Once such a retrieval callback/cache would be implemented in curl, it
would be pretty easy to make use of it from git without messing/breaking
too much with the existing code.

So I went to the curl-library list.  Please feel free to read the
discussion on http://curl.haxx.se/mail/lib-2008-10/index.html#286

Check out http://curl.haxx.se/mail/lib-2008-11/index.html#38 for a
first draft of the change I suggested.

Unfortunately, I got stuck implementing the regression tests for this
change.  Then I got short on time to proceed the work on this topic.
Check out http://curl.haxx.se/mail/lib-2008-11/index.html#51 for the
last state of affairs I posted.  Once the regression tests are
implemented, implementing the remaining functionality would be pretty
much straight forward.  But currently, I have no clue when I will get
around to continue working on that.

> Finally, is there interest in refactoring the http code to make it a
> little cleaner?  That is, make a wrapper library around curl so that
> you can just call GET or POST or whatever and not worry about how to
> invoke curl?

As I mentioned above, I figured that implementing the callback method
in curl-lib would probably be easier than understanding the usage of
curl within git.  So I'd say refactoring would be an improvement.  But
I bet the git gurus will flame me for saying that =:-S

Re: git with https and client cert asks for password repeatedly

[Johannes Schindelin]

Hi,

[Josef, it is extremely annoying that you culled the Cc: list; I do not 
know if Mark will get this mail, and my time is too scarce to undo your 
mistake.]

On Wed, 25 Feb 2009, Josef Wolf wrote:

> On Tue, Feb 24, 2009 at 10:11:40PM -0500, Mark Lodato wrote:
> 
> > Finally, is there interest in refactoring the http code to make it a 
> > little cleaner?  That is, make a wrapper library around curl so that 
> > you can just call GET or POST or whatever and not worry about how to 
> > invoke curl?
> 
> As I mentioned above, I figured that implementing the callback method in 
> curl-lib would probably be easier than understanding the usage of curl 
> within git.  So I'd say refactoring would be an improvement.  But I bet 
> the git gurus will flame me for saying that =:-S

To the contrary.  I have been waiting for over one year for Mike Hommey to 
submit his refactoring patches.

A very quick list search would find you his patch series, I am sure.

Status quo: the series needs polishing, and we need http-fetch tests (if I 
saw it correctly, Peff started something like this).

Ciao,
Dscho