[PATCH] connect.c: add a way for git-daemon to pass an error back to client

[PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Tom Preston-Werner]

The current behavior of git-daemon is to simply close the connection on
any error condition. This leaves the client without any information as
to the cause of the failed fetch/push/etc.

This patch allows get_remote_heads to accept a line prefixed with "ERR"
that it can display to the user in an informative fashion. Once clients
can understand this ERR line, git-daemon can be made to properly report
"repository not found", "permission denied", or other errors.

Example

S: ERR No matching repository.
C: fatal: remote error: No matching repository.

Signed-off-by: Tom Preston-Werner <tom@github.com>
---
 connect.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/connect.c b/connect.c
index 0c50d0a..3af91d6 100644
--- a/connect.c
+++ b/connect.c
@@ -70,6 +70,9 @@ struct ref **get_remote_heads(int in, struct ref **list,
 		if (buffer[len-1] == '\n')
 			buffer[--len] = 0;

+		if (len > 4 && !memcmp("ERR", buffer, 3))
+			die("remote error: %s", buffer + 4);
+
 		if (len < 42 || get_sha1_hex(buffer, old_sha1) || buffer[40] != ' ')
 			die("protocol error: expected sha/ref, got '%s'", buffer);
 		name = buffer + 41;
-- 
1.6.0.2

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Tom Preston-Werner]

On Fri, Oct 31, 2008 at 7:19 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> The cute trick with this is that even older clients would be able to
> display the error, albeit with a sort of funny message:
>
>        protocol error: expected sha/ref, got 'No matching repository.'

This is exactly what GitHub does right now with our custom Erlang
git-daemon. Try doing:

$ git clone git://github.com/mojombo/foo

and you'll get:

fatal: protocol error: expected sha/ref, got '
*********'

No matching repositories found.

*********'

This has cut down tremendously on the number of support requests we
get from new users. It would be nice to cut out the clutter from the
error message though.

Tom

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Johannes Schindelin]

Hi,

On Fri, 31 Oct 2008, Tom Preston-Werner wrote:

> The current behavior of git-daemon is to simply close the connection on
> any error condition. This leaves the client without any information as
> to the cause of the failed fetch/push/etc.
> 
> This patch allows get_remote_heads to accept a line prefixed with "ERR"
> that it can display to the user in an informative fashion. Once clients
> can understand this ERR line, git-daemon can be made to properly report
> "repository not found", "permission denied", or other errors.
> 
> Example
> 
> S: ERR No matching repository.
> C: fatal: remote error: No matching repository.

Makes sense to me.

The cute trick with this is that even older clients would be able to 
display the error, albeit with a sort of funny message:

	protocol error: expected sha/ref, got 'No matching repository.'

Ciao,
Dscho

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Nicolas Pitre]

On Sat, 1 Nov 2008, Johannes Schindelin wrote:

> Hi,
> 
> On Fri, 31 Oct 2008, Tom Preston-Werner wrote:
> 
> > The current behavior of git-daemon is to simply close the connection on
> > any error condition. This leaves the client without any information as
> > to the cause of the failed fetch/push/etc.
> > 
> > This patch allows get_remote_heads to accept a line prefixed with "ERR"
> > that it can display to the user in an informative fashion. Once clients
> > can understand this ERR line, git-daemon can be made to properly report
> > "repository not found", "permission denied", or other errors.
> > 
> > Example
> > 
> > S: ERR No matching repository.
> > C: fatal: remote error: No matching repository.
> 
> Makes sense to me.

Note that this behavior of not returning any reason for failure was 
argued to be a security feature in the past, by Linus I think.


Nicolas

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Johannes Schindelin]

Hi,

On Fri, 31 Oct 2008, Nicolas Pitre wrote:

> On Sat, 1 Nov 2008, Johannes Schindelin wrote:
> 
> > On Fri, 31 Oct 2008, Tom Preston-Werner wrote:
> > 
> > > The current behavior of git-daemon is to simply close the connection 
> > > on any error condition. This leaves the client without any 
> > > information as to the cause of the failed fetch/push/etc.
> > > 
> > > This patch allows get_remote_heads to accept a line prefixed with 
> > > "ERR" that it can display to the user in an informative fashion. 
> > > Once clients can understand this ERR line, git-daemon can be made to 
> > > properly report "repository not found", "permission denied", or 
> > > other errors.
> > > 
> > > Example
> > > 
> > > S: ERR No matching repository.
> > > C: fatal: remote error: No matching repository.
> > 
> > Makes sense to me.
> 
> Note that this behavior of not returning any reason for failure was 
> argued to be a security feature in the past, by Linus I think.

Yes.  And it might still be considered one.  You do not need to patch 
git-daemon to use that facility (note that Tom's patch was only for the 
client side).

But for hosting sites such as repo.or.cz or GitHub, that security feature 
just does not make sense, but it makes for support requests that could be 
resolved better with a proper error message.

Ciao,
Dscho

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Tom Preston-Werner]

On Fri, Oct 31, 2008 at 7:35 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 31 Oct 2008, Nicolas Pitre wrote:
>
>> On Sat, 1 Nov 2008, Johannes Schindelin wrote:
>>
>> > On Fri, 31 Oct 2008, Tom Preston-Werner wrote:
>> >
>> > > The current behavior of git-daemon is to simply close the connection
>> > > on any error condition. This leaves the client without any
>> > > information as to the cause of the failed fetch/push/etc.
>> > >
>> > > This patch allows get_remote_heads to accept a line prefixed with
>> > > "ERR" that it can display to the user in an informative fashion.
>> > > Once clients can understand this ERR line, git-daemon can be made to
>> > > properly report "repository not found", "permission denied", or
>> > > other errors.
>> > >
>> > > Example
>> > >
>> > > S: ERR No matching repository.
>> > > C: fatal: remote error: No matching repository.
>> >
>> > Makes sense to me.
>>
>> Note that this behavior of not returning any reason for failure was
>> argued to be a security feature in the past, by Linus I think.
>
> Yes.  And it might still be considered one.  You do not need to patch
> git-daemon to use that facility (note that Tom's patch was only for the
> client side).
>
> But for hosting sites such as repo.or.cz or GitHub, that security feature
> just does not make sense, but it makes for support requests that could be
> resolved better with a proper error message.

We could also have the error messages sent back conditionally based on
a command line switch. I've begun porting the changes I made in our
Erlang git-daemon back to the C code, so maybe I'll give that a try.
We *definitely* need good error messages for GitHub and I see no
security risk in doing so.

Maybe this is worth asking the question: does anybody use git-daemon
for private code? If so, why are they not using SSH instead? And in
that case, how are informative error messages a security risk?

Tom

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Junio C Hamano]

"Tom Preston-Werner" <tom@github.com> writes:

> Example
>
> S: ERR No matching repository.
> C: fatal: remote error: No matching repository.

I like what this tries to do.

I briefly wondered if this should be restricted to the very first message
from the other end, but I think it is not necessary.  If the remote throws
a few valid looking "SHA-1 SP refname" lines and then said "ERR" (which
cannot be the beginning of a valid SHA-1), we can safely and unambiguously
declare that this is an error message from the remote end.

> diff --git a/connect.c b/connect.c
> index 0c50d0a..3af91d6 100644
> --- a/connect.c
> +++ b/connect.c
> @@ -70,6 +70,9 @@ struct ref **get_remote_heads(int in, struct ref **list,
>  		if (buffer[len-1] == '\n')
>  			buffer[--len] = 0;
>
> +		if (len > 4 && !memcmp("ERR", buffer, 3))

Would matching 4 bytes "ERR " here an improvement?  You are expecting
buffer+4 is where the message begins in die() anyway, and otherwise you
would show the message without "N" if you got "ERRNo matching repo".

> +			die("remote error: %s", buffer + 4);
> +

It was very considerate that you did not say "server error" in the error
message.  This code is shared between both the fetch side and the push
side.

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Tom Preston-Werner]

On Fri, Oct 31, 2008 at 10:39 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Tom Preston-Werner" <tom@github.com> writes:
>
>> Example
>>
>> S: ERR No matching repository.
>> C: fatal: remote error: No matching repository.
>
> I like what this tries to do.
>
> I briefly wondered if this should be restricted to the very first message
> from the other end, but I think it is not necessary.  If the remote throws
> a few valid looking "SHA-1 SP refname" lines and then said "ERR" (which
> cannot be the beginning of a valid SHA-1), we can safely and unambiguously
> declare that this is an error message from the remote end.
>
>> diff --git a/connect.c b/connect.c
>> index 0c50d0a..3af91d6 100644
>> --- a/connect.c
>> +++ b/connect.c
>> @@ -70,6 +70,9 @@ struct ref **get_remote_heads(int in, struct ref **list,
>>               if (buffer[len-1] == '\n')
>>                       buffer[--len] = 0;
>>
>> +             if (len > 4 && !memcmp("ERR", buffer, 3))
>
> Would matching 4 bytes "ERR " here an improvement?  You are expecting
> buffer+4 is where the message begins in die() anyway, and otherwise you
> would show the message without "N" if you got "ERRNo matching repo".

I saw several methods of testing for a specific prefix in connect.c.
Looking more closely at the source, the closest similar call is
actually the test for ACK:

	if (!prefixcmp(line, "ACK ")) {
		if (!get_sha1_hex(line+4, result_sha1)) {
			if (strstr(line+45, "continue"))
				return 2;
			return 1;
		}
	}

Explicitly testing for "ERR " (including the space) does seem like the
more correct thing to do. Would you like me to resubmit a modified
patch that uses prefixcmp()?

Tom

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Andreas Ericsson]

Nicolas Pitre wrote:
> On Sat, 1 Nov 2008, Johannes Schindelin wrote:
> 
>> Hi,
>>
>> On Fri, 31 Oct 2008, Tom Preston-Werner wrote:
>>
>>> The current behavior of git-daemon is to simply close the connection on
>>> any error condition. This leaves the client without any information as
>>> to the cause of the failed fetch/push/etc.
>>>
>>> This patch allows get_remote_heads to accept a line prefixed with "ERR"
>>> that it can display to the user in an informative fashion. Once clients
>>> can understand this ERR line, git-daemon can be made to properly report
>>> "repository not found", "permission denied", or other errors.
>>>
>>> Example
>>>
>>> S: ERR No matching repository.
>>> C: fatal: remote error: No matching repository.
>> Makes sense to me.
> 
> Note that this behavior of not returning any reason for failure was 
> argued to be a security feature in the past, by Linus I think.
> 

By me actually. I wrote the patch for it. Showing "no matching repository"
means git-daemon can be used to disclose information about the remote
server's filesystem layout. While I understand that it's sometimes a useful
feature, please don't ever enable it by default.

That said, I like this patch, as it only works client-side and just enables
others to write code that let's the daemon (if configured to do so) ship a
more informative error message.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Andreas Ericsson]

Tom Preston-Werner wrote:
> On Fri, Oct 31, 2008 at 7:35 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Hi,
>>
>> On Fri, 31 Oct 2008, Nicolas Pitre wrote:
>>
>>> On Sat, 1 Nov 2008, Johannes Schindelin wrote:
>>>
>>>> On Fri, 31 Oct 2008, Tom Preston-Werner wrote:
>>>>
>>>>> The current behavior of git-daemon is to simply close the connection
>>>>> on any error condition. This leaves the client without any
>>>>> information as to the cause of the failed fetch/push/etc.
>>>>>
>>>>> This patch allows get_remote_heads to accept a line prefixed with
>>>>> "ERR" that it can display to the user in an informative fashion.
>>>>> Once clients can understand this ERR line, git-daemon can be made to
>>>>> properly report "repository not found", "permission denied", or
>>>>> other errors.
>>>>>
>>>>> Example
>>>>>
>>>>> S: ERR No matching repository.
>>>>> C: fatal: remote error: No matching repository.
>>>> Makes sense to me.
>>> Note that this behavior of not returning any reason for failure was
>>> argued to be a security feature in the past, by Linus I think.
>> Yes.  And it might still be considered one.  You do not need to patch
>> git-daemon to use that facility (note that Tom's patch was only for the
>> client side).
>>
>> But for hosting sites such as repo.or.cz or GitHub, that security feature
>> just does not make sense, but it makes for support requests that could be
>> resolved better with a proper error message.
> 
> We could also have the error messages sent back conditionally based on
> a command line switch. I've begun porting the changes I made in our
> Erlang git-daemon back to the C code, so maybe I'll give that a try.
> We *definitely* need good error messages for GitHub and I see no
> security risk in doing so.
> 
> Maybe this is worth asking the question: does anybody use git-daemon
> for private code? If so, why are they not using SSH instead? And in
> that case, how are informative error messages a security risk?
> 

Because it can potentially allow attackers to gain a lot of information
about your system. For instance, if you have /var/lib/rpm on your system,
you're likely running an RPM-based installation. Otoh, if you have
/usr/bin/apt-get, you're most likely running a dpkg-based one. Such info
is vital for the attacker to know what version of a certain server-program
you're using, and can then be used to scan the very helpful world wide web
for security issues concerning your exact distribution.

I'm not saying that's possible with git-daemon now (although I haven't tried),
but if, one day, a git-daemon were to accept a path such as ../../../, we'd
be in real trouble, and an attacker would have no problems what so ever
doing educated guesses on exactly what kind of software is running on your
server.

So, please don't enable any such feature by default. Bury it somewhere deep
in documentation so that users do not enable it by default, or attach a big
fat warning to the docs mentioning it.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Alex Riesen]

Tom Preston-Werner, Sat, Nov 01, 2008 04:35:20 +0100:
> Maybe this is worth asking the question: does anybody use git-daemon
> for private code? If so, why are they not using SSH instead? And in
> that case, how are informative error messages a security risk?

Yes. I use both in my private network, with only ssh open to the
internet. git-daemon is smaller and faster (started from inetd). And
I'm absolutely sure wont ever accidentally push something in the
mirrored repos.

I never had the error reporting problem in this setup, though. It is a
fully controled environment and I can just look in syslog.

I support the original reason for not doing the errors, BTW. It cannot
be on by default.

Heh, try the patch for your private repos and private repos of your
employer, who can sack you for exposing confidential information, and
open them to internet. Than come back and tell us how safe you feel :)

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Shawn O. Pearce]

Tom Preston-Werner <tom@github.com> wrote:
> 
> I saw several methods of testing for a specific prefix in connect.c.
> Looking more closely at the source, the closest similar call is
> actually the test for ACK:
> 
> 	if (!prefixcmp(line, "ACK ")) {
> 		if (!get_sha1_hex(line+4, result_sha1)) {
> 			if (strstr(line+45, "continue"))
> 				return 2;
> 			return 1;
> 		}
> 	}
> 
> Explicitly testing for "ERR " (including the space) does seem like the
> more correct thing to do. Would you like me to resubmit a modified
> patch that uses prefixcmp()?

Yes, I think that is what Junio was hinting at.  The pattern above
is much more typical in Git sources, so keeping the new "ERR "
check consistent would be appreciated.

-- 
Shawn.

Re: [PATCH] connect.c: add a way for git-daemon to pass an error back to client

[Junio C Hamano]

"Tom Preston-Werner" <tom@github.com> writes:

> Explicitly testing for "ERR " (including the space) does seem like the
> more correct thing to do. Would you like me to resubmit a modified
> patch that uses prefixcmp()?

No need for resubmission to change something minor like that.  Will queue
with a fixup.

Thanks.