Re: Is there a debug mode for git-clone https traffic?

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Wed, May 18, 2011 at 01:08, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> I''m trying to debug a problem where over a https proxy I have
> "warning: remote HEAD refers to nonexistent ref, unable to checkout."
> when doing git-clone.
>
> I suspect that it might be an overzealous security scanner sitting in
> the middle.
>
> Is there some debugging mode for git-clone where it'll dump the
> traffic being sent over the wire that I could use to confirm this?

I've debugged this, and the results are strange. Here it is on a
machine that doesn't work (proxy hostname manually munged):

    Cloning into magit...
    * Couldn't find host github.com in the .netrc file; using defaults
    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)
    *   Trying 10.146.207.60... * Connected to
proxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)
    * Establish HTTP proxy tunnel to github.com:443
    > CONNECT github.com:443 HTTP/1.1
    Host: github.com:443
    User-Agent: git/1.7.4.4
    Proxy-Connection: Keep-Alive
    Pragma: no-cache

    < HTTP/1.0 200 Connection established
    <
    * Proxy replied OK to CONNECT request
    * found 158 certificates in /etc/ssl/certs/ca-certificates.crt
    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1
    User-Agent: git/1.7.4.4
    Host: github.com
    Accept: */*
    Pragma: no-cache

    * Connection #0 to host proxy-BROKEN.example.net left intact
    * Couldn't find host github.com in the .netrc file; using defaults
    * Connection #0 seems to be dead!
    * Closing connection #0
    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)
    *   Trying 10.146.207.60... * Connected to
proxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)
    * Establish HTTP proxy tunnel to github.com:443
    > CONNECT github.com:443 HTTP/1.1
    Host: github.com:443
    User-Agent: git/1.7.4.4
    Proxy-Connection: Keep-Alive
    Pragma: no-cache

    < HTTP/1.0 200 Connection established
    <
    * Proxy replied OK to CONNECT request
    * found 158 certificates in /etc/ssl/certs/ca-certificates.crt
    > GET /magit/magit.git/HEAD HTTP/1.1
    User-Agent: git/1.7.4.4
    Host: github.com
    Accept: */*
    Pragma: no-cache

    * Connection #0 to host proxy-BROKEN.example.net left intact
    warning: remote HEAD refers to nonexistent ref, unable to checkout.

And here on a box with a different proxy where this does work:

    Cloning into magit...
    * Couldn't find host github.com in the .netrc file, using defaults
    * About to connect() to proxy proxy-OK.example.net port 3128
    *   Trying 10.147.82.1... * connected
    * Connected to proxy-OK.example.net (10.147.82.1) port 3128
    * Establish HTTP proxy tunnel to github.com:443
    > CONNECT github.com:443 HTTP/1.0
    Host: github.com:443
    User-Agent: git/1.7.2.1
    Proxy-Connection: Keep-Alive
    Pragma: no-cache

    < HTTP/1.0 200 Connection established
    <
    * Proxy replied OK to CONNECT request
    * successfully set certificate verify locations:
    *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
      CApath: none
    * SSL connection using AES256-SHA
    * Server certificate:
    *        subject: /O=*.github.com/OU=Domain Control
Validated/CN=*.github.com
    *        start date: 2009-12-11 05:02:36 GMT
    *        expire date: 2014-12-11 05:02:36 GMT
    *        subjectAltName: github.com matched
    *        issuer: /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com,
Inc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure
Certification Authority/serialNumber=07969287
    * SSL certificate verify ok.
    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1
    User-Agent: git/1.7.2.1
    Host: github.com
    Accept: */*
    Pragma: no-cache

    < HTTP/1.1 200 OK
    < Server: nginx/0.7.67
    < Date: Mon, 23 May 2011 08:14:22 GMT
    < Content-Type: application/x-git-upload-pack-advertisement
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Expires: Fri, 01 Jan 1980 00:00:00 GMT
    < Pragma: no-cache
    < Cache-Control: no-cache, max-age=0, must-revalidate
    * Connection #0 to host proxy-OK.example.net left intact
    * Couldn't find host github.com in the .netrc file, using defaults
    * Re-using existing connection! (#0) with host proxy-OK.example.net
    * Connected to proxy-OK.example.net (10.147.82.1) port 3128
    > POST /magit/magit.git/git-upload-pack HTTP/1.1
    User-Agent: git/1.7.2.1
    Host: github.com
    Accept: */*
    Accept-Encoding: deflate, gzip
    Content-Type: application/x-git-upload-pack-request
    Accept: application/x-git-upload-pack-result
    Content-Length: 828

    0073want 761d6ad09bfce0d354c8fe958f5fc1b0fdde0a9a
multi_ack_detailed side-band-64k thin-pack no-progress ofs-delta
    0032want 292c83d70df3735ba3809dd946d99cf7aa49f6bd
    0032want 1984ce646f25ec073afe77f9d90c11f422659c60
    0032want fd6f6d5ea7b209f8968dfcd999a20f84e6e63edc
    0032want b60cb2bdb31d78670690e22a7fa9eb1a5fb6a40c
    0032want eb0da0582e6635e8b89636170a0de847a71d975e
    0032want 387969a8734377e4baabaf2533dfad8f427dce5f
    0032want cff8d04ef1fc0cce10a274e3737fbbe3ae9be43c
    0032want 8be54f67577478ce05cf68a44a377cab893e9ea0
    0032want 7b3473865309c8a6bc41d0e674ee648e124bf106
    0032want 01aa8d5874b4ad19ffd0423e2c0e0123c5393051
    0032want 00579d3e79504ce0be173b9cd0aec4100cbe8a76
    0032want be85320fabef7af16fb1a27fa74908f54a1f2403
    0032want b4b0a8bffc393137d3a52d62bc92c762a77067e9
    0032want e4d766306adbda64a974b5ea1daa9b7fffdc29d6
    00000009done
    < HTTP/1.1 200 OK
    < Server: nginx/0.7.67
    < Date: Mon, 23 May 2011 08:14:22 GMT
    < Content-Type: application/x-git-upload-pack-result
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Expires: Fri, 01 Jan 1980 00:00:00 GMT
    < Pragma: no-cache
    < Cache-Control: no-cache, max-age=0, must-revalidate
    * Connection #0 to host proxy-OK.example.net left intact

Which is curious, because I can get that file manually with curl on
both of those boxes, i.e. the BROKEN and OK one, respectively:

    $ curl -v https://github.com/magit/magit.git/info/refs?service=git-upload-pack
| sha1sum
    * About to connect() to proxy proxy-BROKEN.example.net port 3128 (#0)
    *   Trying 10.146.207.60...   % Total    % Received % Xferd
Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent
Left  Speed
      0     0    0     0    0     0      0      0 --:--:-- --:--:--
--:--:--     0connected
    * Connected to proxy-BROKEN.example.net (10.146.207.60) port 3128 (#0)
    * Establish HTTP proxy tunnel to github.com:443
    > CONNECT github.com:443 HTTP/1.1
    > Host: github.com:443
    > User-Agent: curl/7.21.6 (i486-pc-linux-gnu) libcurl/7.21.6
OpenSSL/1.0.0d zlib/1.2.3.4 libidn/1.20 libssh2/1.2.8 librtmp/2.3
    > Proxy-Connection: Keep-Alive
    >
    < HTTP/1.0 200 Connection established
    <
    * Proxy replied OK to CONNECT request
    * successfully set certificate verify locations:
    *   CAfile: none
      CApath: /etc/ssl/certs
    * SSLv3, TLS handshake, Client hello (1):
    } [data not shown]
    * SSLv3, TLS handshake, Server hello (2):
    { [data not shown]
    * SSLv3, TLS handshake, CERT (11):
    { [data not shown]
    * SSLv3, TLS handshake, Server finished (14):
    { [data not shown]
    * SSLv3, TLS handshake, Client key exchange (16):
    } [data not shown]
    * SSLv3, TLS change cipher, Client hello (1):
    } [data not shown]
    * SSLv3, TLS handshake, Finished (20):
    } [data not shown]
    * SSLv3, TLS change cipher, Client hello (1):
    { [data not shown]
    * SSLv3, TLS handshake, Finished (20):
    { [data not shown]
    * SSL connection using AES256-SHA
    * Server certificate:
    *        subject: O=*.github.com; OU=Domain Control Validated;
CN=*.github.com
    *        start date: 2009-12-11 05:02:36 GMT
    *        expire date: 2014-12-11 05:02:36 GMT
    *        subjectAltName: github.com matched
    *        issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com,
Inc.; OU=http://certificates.godaddy.com/repository; CN=Go Daddy
Secure Certification Authority; serialNumber=07969287
    *        SSL certificate verify ok.
    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1
    > User-Agent: curl/7.21.6 (i486-pc-linux-gnu) libcurl/7.21.6
OpenSSL/1.0.0d zlib/1.2.3.4 libidn/1.20 libssh2/1.2.8 librtmp/2.3
    > Host: github.com
    > Accept: */*
    >
      0     0    0     0    0     0      0      0 --:--:-- --:--:--
--:--:--     0< HTTP/1.1 200 OK
    < Server: nginx/0.7.67
    < Date: Mon, 23 May 2011 08:19:19 GMT
    < Content-Type: application/x-git-upload-pack-advertisement
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Expires: Fri, 01 Jan 1980 00:00:00 GMT
    < Pragma: no-cache
    < Cache-Control: no-cache, max-age=0, must-revalidate
    <
    { [data not shown]
    100  1531    0  1531    0     0   2773      0 --:--:-- --:--:--
--:--:--  2877
    * Connection #0 to host proxy-BROKEN.example.net left intact
    * Closing connection #0
    * SSLv3, TLS alert, Client hello (1):
    } [data not shown]
    5b7eb0b5c25a8700bfc8376a5a38da78724dc1dd  -

    $ curl -v https://github.com/magit/magit.git/info/refs?service=git-upload-pack
| sha1sum
    * About to connect() to proxy proxy-OK.example.net port 3128
    *   Trying 10.147.82.5... connected
    * Connected to proxy-OK.example.net (10.147.82.5) port 3128
    * Establish HTTP proxy tunnel to github.com:443
    > CONNECT github.com:443 HTTP/1.0
    > Host: github.com:443
    > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5
OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
    > Proxy-Connection: Keep-Alive
    >
    < HTTP/1.0 200 Connection established
    <
    * Proxy replied OK to CONNECT request
    * successfully set certificate verify locations:
    *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
      CApath: none
    * SSLv2, Client hello (1):
    SSLv3, TLS handshake, Server hello (2):
    SSLv3, TLS handshake, CERT (11):
    SSLv3, TLS handshake, Server finished (14):
    SSLv3, TLS handshake, Client key exchange (16):
    SSLv3, TLS change cipher, Client hello (1):
    SSLv3, TLS handshake, Finished (20):
    SSLv3, TLS change cipher, Client hello (1):
    SSLv3, TLS handshake, Finished (20):
    SSL connection using AES256-SHA
    * Server certificate:
    *        subject: /O=*.github.com/OU=Domain Control
Validated/CN=*.github.com
    *        start date: 2009-12-11 05:02:36 GMT
    *        expire date: 2014-12-11 05:02:36 GMT
    *        subjectAltName: github.com matched
    *        issuer: /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com,
Inc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure
Certification Authority/serialNumber=07969287
    * SSL certificate verify ok.
    > GET /magit/magit.git/info/refs?service=git-upload-pack HTTP/1.1
    > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5
OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
    > Host: github.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx/0.7.67
    < Date: Mon, 23 May 2011 08:18:47 GMT
    < Content-Type: application/x-git-upload-pack-advertisement
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Expires: Fri, 01 Jan 1980 00:00:00 GMT
    < Pragma: no-cache
    < Cache-Control: no-cache, max-age=0, must-revalidate
      % Total    % Received % Xferd  Average Speed   Time    Time
Time  Current
                                     Dload  Upload   Total   Spent
Left  Speed
    100  1531    0  1531    0     0   3173      0 --:--:-- --:--:--
--:--:--     0* Connection #0 to host proxy-OK.example.net left intact

    * Closing connection #0
    * SSLv3, TLS alert, Client hello (1):
    5b7eb0b5c25a8700bfc8376a5a38da78724dc1dd  -

The SSL part seems different though, but since curl(1) can get the
content perhaps there's some use of the curl library in Git that's
tripping me up?