git_inetd_server: run git-http-backend using inetd

git_inetd_server: run git-http-backend using inetd

[Kyle J. McKay]

===========
What is it?
===========
I have created a script (POSIX sh compatible) that allows running git- 
http-backend via inetd or similar service (it makes it like git-http- 
backend had a --inet option just like git-daemon does).

This allows the Git smart HTTP protocol to be served without needing  
to run a standard web server (assuming inetd, xinetd, launchd or  
similar is already available).

It is available at [1] with a GPL license.

====
Why?
====
I have several machines/virtual machines that need to be able to fetch  
from a central Git repository without authentication (several of them  
are test platforms and are therefore not trusted and typing a password  
on every fetch is a non-starter as some of the fetches are  
automated).  I also don't want to have to run a web server just to  
serve Git fetches.

Of course you say, "why not use git daemon for that?"  Well, I was (in  
--inetd mode), but it has a serious problem and until recently I  
didn't know the cause.

Talking to a git-daemon server works great for my FreeBSD, Darwin,  
cygwin, msysgit platforms, but it's terrible for my Linux platforms.

On the Linux platforms every git fetch would hang for at least 15  
seconds before starting the fetch.  I just wrote that off as some odd  
configuration issue I was going to have to live with (although it only  
happened with Git and not web browsing or curl fetches).  It's killer  
though when fetching a project with lots of submodules as each one has  
the delay.

Recently though, I discovered the cause.  While I don't actually run  
Debian itself, all my Linux platforms are Debian offspring (Ubuntu,  
Raspbian, Linux Mint, etc.).  The problem is in the Debian git package  
[2] specifically the "0010-transport-optionally-honor-DNS-SRV- 
records.diff" patch located in the Debian diffs file [3].

The 0010 patch makes "git://host/..." fetches first try to fetch a git  
SRV record from DNS.  This patch has a compile time option to enable  
it, but once baked in it cannot be turned off at runtime and the  
binary packages for my distributions all have it turned on and baked  
in.  And none of these distributions have a visible "WARNING: Git has  
patches that may cause it to behave oddly, violate RFC standards and/ 
or experience unexpected delays."

It so happens that my internal network is setup using mDNS [4] (on  
Linux/FreeBSD by installing Avahi and nss-mdns packages).

When I then try to fetch using a "git://host/..." URL where "host" is  
an mDNS host name, the 0010 patch causes git to attempt to lookup a  
DNS SRV record on the non-mDNS regular DNS service (a violation of RFC  
6762 [4] section 22) and this is what has to time out before the real  
fetch can start.

Since the patch does not try to lookup an SRV record for "http:// 
host/..." URLs, switching from git: to http: for fetches eliminates  
the problem.  However, git-http-backend does not have any of the -- 
daemon or --inetd related options that git-daemon does.  Hence the  
git_inetd_server.sh script [1] that provides the equivalent of a -- 
inetd option for git-http-backend.

I have since discovered that the 0010 patch behavior can be bypassed  
by explicitly including a port number in the git: URLs.  So instead of  
"git://example.local/repo.git" using "git://example.local:9418/ 
repo.git" avoids the delaying SRV lookup.  This is highly unintuitive  
since omitting the default port for a scheme should not change its  
behavior (according to RFC 3986 [5] section 3.2.3 the port should be  
omitted by producers and normalizers if it's the same as the scheme's  
default port).  In any case, unless git_inetd_server.sh can be run on  
the default http port (80), the resulting http: URLs are no more  
convenient than the git: URLs with an explicit port.

Nonetheless, others may find this script helpful as it allows the Git  
smart (and non-smart) HTTP protocol to be served without needing to  
run a standard web server so I decided to share it.

--Kyle

=====
Links
=====
[1] https://github.com/mackyle/git_inetd_server
[2] https://packages.debian.org/wheezy/git
[3] http://ftp.de.debian.org/debian/pool/main/g/git/git_1.7.10.4-1+wheezy1.diff.gz
[4] RFC 6762 http://tools.ietf.org/html/rfc6762
[5] RFC 3986 http://tools.ietf.org/html/rfc3986

Re: git_inetd_server: run git-http-backend using inetd

[Jonathan Nieder]

Hi,

Kyle J. McKay wrote:

> When I then try to fetch using a "git://host/..." URL where "host"
> is an mDNS host name, the 0010 patch causes git to attempt to lookup
> a DNS SRV record on the non-mDNS regular DNS service (a violation of
> RFC 6762 [4] section 22) and this is what has to time out before the
> real fetch can start.

That patch uses res_query from libresolv to do the lookup.  It doesn't
know what DNS server to use and relies on system libraries to get it
right.

It was added to respond to a feature request within Debian but it is
intended to eventually go upstream.  I'm glad you found this issue
before that could happen. :)

Should git automatically disable the SRV lookups when it sees one of
the six domains named in RFC6762, or is there some system library call
that can use mDNS when appropriate automatically (or get partway there
without having to hard-code the domains)?

Thanks,
Jonathan

Re: git_inetd_server: run git-http-backend using inetd

[Kyle J. McKay]

On Jul 17, 2014, at 15:10, Jonathan Nieder wrote:
> Hi,
>
> Kyle J. McKay wrote:
>
>> When I then try to fetch using a "git://host/..." URL where "host"
>> is an mDNS host name, the 0010 patch causes git to attempt to lookup
>> a DNS SRV record on the non-mDNS regular DNS service (a violation of
>> RFC 6762 [4] section 22) and this is what has to time out before the
>> real fetch can start.
>
> That patch uses res_query from libresolv to do the lookup.  It doesn't
> know what DNS server to use and relies on system libraries to get it
> right.

If I understand the libresolv implementation correctly, it only  
supports DNS and not mDNS.  And as far as I know, the nss plugins only  
support A and AAAA lookups, not SRV lookups.  However, it looks like  
there are some attempts to add mDNS support directly to res_query and  
friends [4].  Maybe that code could serve as a model.

> It was added to respond to a feature request within Debian but it is
> intended to eventually go upstream.  I'm glad you found this issue
> before that could happen. :)

Is there some reason the patch is not opt-in at runtime?  In other  
words the code is there, but not enabled unless you do something like  
'git config --system --bool git.srvlookup true'?  If it's off by  
default it doesn't matter so much that it's standards violating as it  
won't bite you unless you turn it on and then presumably if you do you  
know what you're doing and understand the possible downside.

> Should git automatically disable the SRV lookups when it sees one of
> the six domains named in RFC6762, or is there some system library call
> that can use mDNS when appropriate automatically (or get partway there
> without having to hard-code the domains)?

Sadly I think mDNS support is relegated to an add-on package on  
Linux.  And Avahi [1] is the package that generally provides it  
there.  The recommended interface for C is the avahi-client API (see  
[2]).  However, that is Avahi-only.

If the patch is to go upstream it probably needs to use the dns-sd.h  
header API for maximum compatibility (Avahi provides an avahi-compat- 
libdns_sd interface via the libavahi-compat-libdnssd-dev package).  I  
believe the correct function would be DNSServiceQueryRecord in this  
case.

That said, for the code to work properly it would need to:

1) Get the canonical name of the host.  If "local" is one of the  
search domains or the default domain you could have a mDNS name  
without an explicit .local suffix.  I'm not sure best how to do this.  
The getaddrinfo function has an AI_CANONNAME flag but the call will  
fail if the host does not have an A or AAAA record and in the case of  
redirection-by-SRV it may have neither.  You probably just have to  
loop through the search domains keeping in mind the Security  
Considerations of section 21 of RFC 6762 [3] (a host MUST NOT append  
the search suffix ".local.", if present, to any relative (partially  
qualified) host name containing two or more labels. Appending  
".local." to single-label relative host names is acceptable).  So if a  
host name does not end in .local (or .local.) it can only be an mDNS  
name if it contains no dots ('.') AND "local" is one of the search  
domains or is the default domain.

2) For mDNS either use the #include <dns_sd.h> DNSServiceQueryRecord  
functionality if available or just skip it if not available.

3) For non-mDNS use the same query as it does now.

Even if the choice is to just disable SRV lookups for mDNS hosts at a  
minimum the code will have to determine whether or not the given host  
name is a mDNS name and it can't reliably do that for a host name  
without any dots in it without at least looking at the default domain  
name and possibly the search domain(s) as well.

I think it would be much simpler just to make this opt-in via a config  
option rather than baked in as it's probably going to be rather messy  
without direct mDNS support in the res_query interface.

--Kyle

[1] http://avahi.org/
[2] http://avahi.org/download/doxygen/index.html
[3] http://tools.ietf.org/html/rfc6762#page-52
[4] https://www.mail-archive.com/tech@openbsd.org/msg06100.html

Re: git_inetd_server: run git-http-backend using inetd

[Jonathan Nieder]

Kyle J. McKay wrote:
> On Jul 17, 2014, at 15:10, Jonathan Nieder wrote:

>> It was added to respond to a feature request within Debian but it is
>> intended to eventually go upstream.  I'm glad you found this issue
>> before that could happen. :)
>
> Is there some reason the patch is not opt-in at runtime?

Yes, it's a feature for server admins to control where clients
connect, to control where the load goes.  If the client has to enable
it explicitly, that defeats the purpose.

I'd much rather fix it than turn it off completely.

[...]
>> Should git automatically disable the SRV lookups when it sees one of
>> the six domains named in RFC6762, or is there some system library call
>> that can use mDNS when appropriate automatically (or get partway there
>> without having to hard-code the domains)?
>
> Sadly I think mDNS support is relegated to an add-on package on
> Linux.  And Avahi [1] is the package that generally provides it
> there.  The recommended interface for C is the avahi-client API (see
> [2]).  However, that is Avahi-only.
[...]
> Even if the choice is to just disable SRV lookups for mDNS hosts at
> a minimum the code will have to determine whether or not the given
> host name is a mDNS name and it can't reliably do that for a host
> name without any dots in it without at least looking at the default
> domain name and possibly the search domain(s) as well.
>
> I think it would be much simpler just to make this opt-in via a
> config option

Thanks for these details.  I'll file a bug and mull it over some more.

RFC 6762 makes it clear that what the package is currently doing is
wrong.  Given that Debian's libc knows nothing about mdns on its own,
I think I'll need to parse resolv.conf (that's what libc-ares does).

Jonathan

Re: git_inetd_server: run git-http-backend using inetd

[Kyle J. McKay]

On Jul 17, 2014, at 19:22, Jonathan Nieder wrote:

> Thanks for these details.  I'll file a bug and mull it over some more.
>
> RFC 6762 makes it clear that what the package is currently doing is
> wrong.  Given that Debian's libc knows nothing about mdns on its own,
> I think I'll need to parse resolv.conf (that's what libc-ares does).

You might also want to take a look at [1] which suggests that when  
doing SRV lookups for URLs they should be done regardless of whether  
or not a port number is present (which then eliminates the RFC 3986  
issue the current SRV lookup code has).  It's only a draft (and  
expired now and looks like it received a rather chilly reception from  
some), but I haven't noticed anything else that addresses what the  
patch code is trying to do.  The thread at [2] is related to what the  
patch is trying to do.

[1] http://tools.ietf.org/html/draft-ohta-urlsrv
[2] http://www.ietf.org/mail-archive/web/pcp/current/msg01727.html

Re: git_inetd_server: run git-http-backend using inetd

[Jonathan Nieder]

Kyle J. McKay wrote:

> You might also want to take a look at [1] which suggests that when
> doing SRV lookups for URLs they should be done regardless of whether
> or not a port number is present (which then eliminates the RFC 3986
> issue the current SRV lookup code has).

"Git URLs" as described e.g. in git-clone(1) weren't intended to be
actual URIs.  What would be the interoperability advantage of making
them URIs?

This has come up before, with e.g. people asking to introduce a
git+ssh:// and git+http:// syntax to refer to repositories accessed
using those transports without conflicting with the usual meanings of
ssh:// and http://.  If there's a good use case for that, maybe it
would be worth doing some day.

Curious,
Jonathan

Re: git_inetd_server: run git-http-backend using inetd

[Kyle J. McKay]

On Jul 18, 2014, at 10:16, Jonathan Nieder wrote:

> Kyle J. McKay wrote:
>
>> You might also want to take a look at [1] which suggests that when
>> doing SRV lookups for URLs they should be done regardless of whether
>> or not a port number is present (which then eliminates the RFC 3986
>> issue the current SRV lookup code has).
>
> "Git URLs" as described e.g. in git-clone(1) weren't intended to be
> actual URIs.

According to RFC 3968 section 1.1.3:
"A URI can be further classified as a locator, a name, or both. The  
term "Uniform Resource Locator" (URL) refers to the subset of  
URIs" [...]

So actually they are URIs.

> What would be the interoperability advantage of making
> them URIs?

According to RFC 3968 they are already considered URIs.


> This has come up before, with e.g. people asking to introduce a
> git+ssh:// and git+http://

How is a discussion about changing the scheme name relevant to a  
discussion about treating a URL with an explicit default port the same  
as one without (which Git already does but stops doing with the 0010  
git SRV patch)?  That would seem to be an orthogonal discussion to  
whether or not to change the scheme name(s) used by Git more than 9  
years after it first came out.

Re: git_inetd_server: run git-http-backend using inetd

[Jonathan Nieder]

Kyle J. McKay wrote:
> On Jul 18, 2014, at 10:16, Jonathan Nieder wrote:
>> Kyle J. McKay wrote:

>>> You might also want to take a look at [1] which suggests that when
>>> doing SRV lookups for URLs they should be done regardless of whether
>>> or not a port number is present (which then eliminates the RFC 3986
>>> issue the current SRV lookup code has).
>>
>> "Git URLs" as described e.g. in git-clone(1) weren't intended to be
>> actual URIs.
>
> According to RFC 3968 section 1.1.3:
> "A URI can be further classified as a locator, a name, or both. The
> term "Uniform Resource Locator" (URL) refers to the subset of URIs"
> [...]
>
> So actually they are URIs.

You mean abusing the word URL when we meant URL-shaped thing makes it
into a URL?

>> What would be the interoperability advantage of making
>> them URIs?
>
> According to RFC 3968 they are already considered URIs.

I don't think you understood my question.

>> This has come up before, with e.g. people asking to introduce a
>> git+ssh:// and git+http://
>
> How is a discussion about changing the scheme name relevant to a
> discussion about treating a URL with an explicit default port the
> same as one without (which Git already does but stops doing with the
> 0010 git SRV patch)?

It's where the question of whether the things you pass to 'git clone'
are URIs came up before.

I don't understand what you want in this side-conversation.  Do you
mean that I should read that RFC and be convinced that what you are
saying about ports is the right thing to do?  I can easily be
convinced some other way, but "It's in a standard that you never
intended to follow" is not particularly convincing or relevant.

The same philosophy as the git project applies to POSIX conformance
issues applies here, too.  We live in the real world.

Hoping that clarifies,
Jonathan

Re: git_inetd_server: run git-http-backend using inetd

[Kyle J. McKay]

On Jul 18, 2014, at 17:19, Jonathan Nieder wrote:

> Kyle J. McKay wrote:
>> On Jul 18, 2014, at 10:16, Jonathan Nieder wrote:
>>>
>>> "Git URLs" as described e.g. in git-clone(1) weren't intended to be
>>> actual URIs.
>>
>> According to RFC 3968 section 1.1.3:
>> "A URI can be further classified as a locator, a name, or both. The
>> term "Uniform Resource Locator" (URL) refers to the subset of URIs"
>> [...]
>>
>> So actually they are URIs.
>
> You mean abusing the word URL when we meant URL-shaped thing makes it
> into a URL?

That is your contention.

> Do you
> mean that I should read that RFC and be convinced that what you are
> saying about ports is the right thing to do?

It's the right thing to do because it's the standard for how URLs are  
expected to behave.

> "It's in a standard that you never
> intended to follow" is not particularly convincing or relevant.

That is your contention.  If it is truly the case that where the Git  
documentation uses the "URL" acronym that Git does not actually intend  
for "URL" to be interpreted as a "URL" as defined by the various  
standards covering such URLs then explicit text needs to be added to  
the documentation saying so to avoid confusion.  In the absence of  
such text, expecting a reader of Git documentation to interpret the  
term "URL" any other way is irrational.

Re: git_inetd_server: run git-http-backend using inetd

[Torsten Bögershausen]

On 07/18/2014 12:10 AM, Jonathan Nieder wrote:
> Hi,
>
> Kyle J. McKay wrote:
>
>> When I then try to fetch using a "git://host/..." URL where "host"
>> is an mDNS host name, the 0010 patch causes git to attempt to lookup
>> a DNS SRV record on the non-mDNS regular DNS service (a violation of
>> RFC 6762 [4] section 22) and this is what has to time out before the
>> real fetch can start.
> That patch uses res_query from libresolv to do the lookup.  It doesn't
> know what DNS server to use and relies on system libraries to get it
> right.
>
> It was added to respond to a feature request within Debian but it is
> intended to eventually go upstream.  I'm glad you found this issue
> before that could happen. :)
>
> Should git automatically disable the SRV lookups when it sees one of
> the six domains named in RFC6762, or is there some system library call
> that can use mDNS when appropriate automatically (or get partway there
> without having to hard-code the domains)?
>
> Thanks,
> Jonathan
(My apologies, if this is spamish), but just to verify what is going on:

git fetch git://host.local/...
results in a DNS lookup ?

Kyle, did you verify the lookup with wireshark or a similar tool?

Jonathan, (I'm good in searching, but bad in finding)
could you point out where the source code for the git package for
debian is ?

I recently learned about mDNS, and will probably do some tests
and experiments later, and would like to test the lookup feature
of "0010".

Thanks.
/Torsten

Re: git_inetd_server: run git-http-backend using inetd

[Jonathan Nieder]

Torsten Bögershausen wrote:

> Jonathan, (I'm good in searching, but bad in finding)
> could you point out where the source code for the git package for
> debian is ?
>
> I recently learned about mDNS, and will probably do some tests
> and experiments later, and would like to test the lookup feature
> of "0010".

Thanks.  It's at git://git.debian.org/~jrnieder-guest/git branch
release+patches and mirrored at http://repo.or.cz/r/git/debian

Re: git_inetd_server: run git-http-backend using inetd

[Torsten Bögershausen]

On 07/19/2014 07:06 PM, Jonathan Nieder wrote:
> Torsten Bögershausen wrote:
>
>> Jonathan, (I'm good in searching, but bad in finding)
>> could you point out where the source code for the git package for
>> debian is ?
>>
>> I recently learned about mDNS, and will probably do some tests
>> and experiments later, and would like to test the lookup feature
>> of "0010".
> Thanks.  It's at git://git.debian.org/~jrnieder-guest/git branch
> release+patches and mirrored at http://repo.or.cz/r/git/debian
With my limited reading of the RFC and the code, and without having
test environment, my spontanous (and probably incomplete) change could 
look like this:
- Treat "host:9418" the same as "host"
- When the hosts ends with .local, do not use DNS.

static int git_tcp_connect_sock(char *host, int flags)
{
     struct strbuf error_message = STRBUF_INIT;
     int sockfd = -1, gai = 0;
     const char *port = NULL;
     struct host *hosts = NULL;
     int j, n = 0;

     get_host_and_port(&host, &port);
     if (!port)
         port = STR(DEFAULT_GIT_PORT);

     n = get_srv(host, &hosts);
     if ((n <= 0) && ends_with(host, ".local")) {
         /*
           host.local is really local, do not send it to a DNS resolver
           The user may try host without .local
          */
         die("Unable to look up %s", host);
     }
     if (!n) {
         hosts = xmalloc(sizeof(*hosts));
         hosts[0].hostname = xstrdup(host);
         hosts[0].port = xstrdup(port);
[snip]

Re: git_inetd_server: run git-http-backend using inetd

[Torsten Bögershausen]

On 07/20/2014 08:10 AM, Torsten Bögershausen wrote:
> On 07/19/2014 07:06 PM, Jonathan Nieder wrote:
>> Torsten Bögershausen wrote:
>>
>>> Jonathan, (I'm good in searching, but bad in finding)
>>> could you point out where the source code for the git package for
>>> debian is ?
>>>
>>> I recently learned about mDNS, and will probably do some tests
>>> and experiments later, and would like to test the lookup feature
>>> of "0010".
>> Thanks.  It's at git://git.debian.org/~jrnieder-guest/git branch
>> release+patches and mirrored at http://repo.or.cz/r/git/debian
I probably need to correct myself:
Donwloaded your branch, compiled and tested.
On my test system the lookup timed out after 1.9 sec for DNS,
and 5 seconds for MDNS (the lookup failed in both cases)

I'm not sure any more how to improve things here, and
the question remains why Kyle has 15 seconds timeout ?

Would it be possible to run wireshark, and give us an example
of the URL's you have been using ?