The git protocol and DoS

The git protocol and DoS

[H. Peter Anvin]

I've been concerned for a while that the git protocol may be inherently 
vulnerable to a "SYNful DoS" attack (spraying raw TCP SYN packets with 
enough data to start substantial server activity.)  Although SYN cookies 
protect against this to some degree, it makes me wonder if something 
should be added to the protocol itself.

One way to do this would be to start the transaction by having the 
server transmit a cookie to the client, and to require the client to 
send a SHA1 of the (cookie + request) together with the request.  This 
would be done with a fairly short timeout.

It would, however, require a protocol change; I would like to hear what 
people think about this at this stac=ge.

	-hpa

Re: The git protocol and DoS

[Junio C Hamano]

"H. Peter Anvin" <hpa@zytor.com> writes:

> It would, however, require a protocol change; I would like to hear what 
> people think about this at this stac=ge.

Well, it is full two days since a majorly visible git protocol
enabled server has been announced, and you probably know what
kind of hits you are getting (and please let us know if you have
numbers, I am curious).  If we do a protocol change, earlier the
better.  You already said that the kernel.org git is
experimental.  Does anybody run git daemons and rely on the
current protocol?

I suspect it would not make *any* sense to have a backward
compatible server that optionally allows this cookie exchange --
attackers can just say "I am an older client".  OTOH, it
probably makes sense to have an option on the client side to
skip the cookie exchange stage.  I do not think autodetecting
new/old server on the client side in connect.c is possible.

Re: The git protocol and DoS

[H. Peter Anvin]

Junio C Hamano wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
>>It would, however, require a protocol change; I would like to hear what 
>>people think about this at this stac=ge.
> 
> Well, it is full two days since a majorly visible git protocol
> enabled server has been announced, and you probably know what
> kind of hits you are getting (and please let us know if you have
> numbers, I am curious).

About 350 hits so far, total.  Utter peanuts.

> If we do a protocol change, earlier the
> better.  You already said that the kernel.org git is
> experimental.  Does anybody run git daemons and rely on the
> current protocol? 
 >
> I suspect it would not make *any* sense to have a backward
> compatible server that optionally allows this cookie exchange --
> attackers can just say "I am an older client".  OTOH, it
> probably makes sense to have an option on the client side to
> skip the cookie exchange stage.  I do not think autodetecting
> new/old server on the client side in connect.c is possible.
> 

You mean an option on the *server* to skip the cookie exchange?  If so, 
how would you expect the client to handle it?

	-hpa

Re: The git protocol and DoS

[Junio C Hamano]

"H. Peter Anvin" <hpa@zytor.com> writes:

> You mean an option on the *server* to skip the cookie exchange?  If so, 
> how would you expect the client to handle it?

No, what I was thinking was to tell the client "you will be
talking to an old server, so do not try to read that cookie and
get stuck".

Re: The git protocol and DoS

[H. Peter Anvin]

Junio C Hamano wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
>>You mean an option on the *server* to skip the cookie exchange?  If so, 
>>how would you expect the client to handle it?
> 
> No, what I was thinking was to tell the client "you will be
> talking to an old server, so do not try to read that cookie and
> get stuck".
> 

Oh, right.

	-hpa

Re: The git protocol and DoS

[Linus Torvalds]

On Wed, 19 Oct 2005, H. Peter Anvin wrote:
> 
> You mean an option on the *server* to skip the cookie exchange?  If so, how
> would you expect the client to handle it?

Hey guys, I actually planned for the protocol to be extensible.

The client always starts out by sending the "command" first, and if you 
want to add a challenge-response thing, I really think you should make it 
a nice compatible upgrade (and then later on, you can have a server option 
that says "if the client doesn't do the challenge-response version, I 
won't talk to him").

Basically, right now the client sends a

	"git-upload-pack /absolute/pathname/to/repo"

over the protocol, and the whole point of this was that (a) it's 
extensible and (b) the server knows what to expect, and can close the 
socket if it doesn't get a valid packet.

So if you add some extra challenge-response thing, please just do so by 
changing the string. Teach the server to also accept

	"git-upload-pack --challenge /absolute/pathname/to/repo"

for example. Then later, add a "secure server" mode that refuses to do the 
old non-challenge response.

HOWEVER. The server _already_ has some of this logic: if you start it 
outside of inetd, it will start killing its own children when there are 
too many of them, but it will start by sending them a SIGTERM. And the 
git-daemon code is set up so that a SIGTERM will kill any deamon that 
hasn't seen the proper handshake yet.

Once it's seen the proper handshake, the deamon will block SIGTERM. 
Exactly so that if there is a SYN attack, people who use a non-git-aware 
SYN generator will be second-class citizens. So there's not a real 
challenge-response thing, but at least it's set up so that real git 
clients (or something that looks like one) can be recognized, and get 
preferred treatment over people who just open a connection.

Of course, this part doesn't work with the kernel.org setup, since that 
uses inetd, but we could easily add a timeout too, and do the same exact 
thing for SIGALRM (and just do an "alarm(timeout)" at the head of 
"execute()" before we start really trying to read from the socket).

In other words, git-daemon _already_ has support to help fight SYN 
attacks, although it currently only works when stand-alone. It could be 
extended to work with inetd, though.

NOTE! Right now, a git-aware SYN-flooder could send a SYN + 
"git-upload-pack /valid/directory" thing in the proper packed-line format, 
and _then_ just go away. But once you're talking to a git-aware 
SYN-flooder, I don't think a challenge-response makes it any better, since 
a git-aware SYN-flooder would just be written to give the right response.

So unless you actually have _passwords_, and make the response something 
that the other end has to figure out some other way, I don't see what else 
we could do..

			Linus

Re: The git protocol and DoS

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> But once you're talking to a git-aware 
> SYN-flooder, I don't think a challenge-response makes it any better, since 
> a git-aware SYN-flooder would just be written to give the right response.

I think Peter's point is that the one that can give the right
response needs to read from the server to compute it, and at
that point it is not a "SYN-flooder" anymore.

Re: The git protocol and DoS

[H. Peter Anvin]

Junio C Hamano wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
> 
>>But once you're talking to a git-aware 
>>SYN-flooder, I don't think a challenge-response makes it any better, since 
>>a git-aware SYN-flooder would just be written to give the right response.
> 
> I think Peter's point is that the one that can give the right
> response needs to read from the server to compute it, and at
> that point it is not a "SYN-flooder" anymore.
> 

Right.  It has been shown that requiring some effort on the part of the 
client before the server spends work on it can greatly reduce the 
capabilities of a limited-resource client to execute a DoS.

	-hpa

Re: The git protocol and DoS

[Petr Baudis]

Dear diary, on Wed, Oct 19, 2005 at 10:00:05PM CEST, I got a letter
where "H. Peter Anvin" <hpa@zytor.com> told me that...
> One way to do this would be to start the transaction by having the 
> server transmit a cookie to the client, and to require the client to 
> send a SHA1 of the (cookie + request) together with the request.  This 
> would be done with a fairly short timeout.

  If (well, it sounds like a good idea, so rather "when") you do this,
it would be a good idea to do in a way that makes it easy to later add
support for some kind of authentication (really, not everyone wants to
give away ssh accounts). Let's say it works like:

[client]	git-upload-pack <path>
[server]	challenge somethingnonsensical
[client]	challenge-response <username>:sha1(somethingnonsensical<password>)
[server]	All right, the pack goes like this...

  Suddenly you have support for hopefully secure authentication, and at
the same time you have the cookie implemented in backwards-compatible
fashion (in the sense that new client will be able to talk to old
server) - just assume the username and password empty. This might be
even hardcoded for now, just leave a room for its addition (in an
elegant and compatible way) in the protocol, please.

  Thanks,

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: The git protocol and DoS

[Tony Luck]

On 10/19/05, Petr Baudis <pasky@suse.cz> wrote:
> [client]        git-upload-pack <path>
> [server]        challenge somethingnonsensical
> [client]        challenge-response <username>:sha1(somethingnonsensical<password>)
> [server]        All right, the pack goes like this...

I think this requires that the server store the cleartext version of
the password so
that it can validate sha1(somethingnonsensical<password>) ... which is generally
thought to be a bad idea.

-Tony

Re: The git protocol and DoS

[David Brown]

On Thu, Oct 20, 2005 at 12:20:44AM +0200, Petr Baudis wrote:

>   If (well, it sounds like a good idea, so rather "when") you do this,
> it would be a good idea to do in a way that makes it easy to later add
> support for some kind of authentication (really, not everyone wants to
> give away ssh accounts). Let's say it works like:
> 
> [client]	git-upload-pack <path>
> [server]	challenge somethingnonsensical
> [client]	challenge-response <username>:sha1(somethingnonsensical<password>)
> [server]	All right, the pack goes like this...
> 
>   Suddenly you have support for hopefully secure authentication, and at
> the same time you have the cookie implemented in backwards-compatible
> fashion (in the sense that new client will be able to talk to old
> server) - just assume the username and password empty. This might be
> even hardcoded for now, just leave a room for its addition (in an
> elegant and compatible way) in the protocol, please.

This kind of password authentication has several problems that make it
fairly unpractical.  It is prone to easy dictionary attacks for one thing.
It also for a spoofed server to do replays, and the likes.  It also
requires the server to store plaintext passwords.

There are other, much better, authentication algorithms, but short of doing
signatures, none are really much more secure.  The closest you'll get to
secure remote passwords is SRP <http://srp.stanford.edu/>, which is quite
good, and doesn't even require plaintext passwords to be stored.  It might
just be easier at that point to use signatures, though.

Dave

Re: The git protocol and DoS

[Andreas Ericsson]

Petr Baudis wrote:
> Dear diary, on Wed, Oct 19, 2005 at 10:00:05PM CEST, I got a letter
> where "H. Peter Anvin" <hpa@zytor.com> told me that...
> 
>>One way to do this would be to start the transaction by having the 
>>server transmit a cookie to the client, and to require the client to 
>>send a SHA1 of the (cookie + request) together with the request.  This 
>>would be done with a fairly short timeout.
> 
> 
>   If (well, it sounds like a good idea, so rather "when") you do this,
> it would be a good idea to do in a way that makes it easy to later add
> support for some kind of authentication (really, not everyone wants to
> give away ssh accounts). Let's say it works like:
> 
> [client]	git-upload-pack <path>
> [server]	challenge somethingnonsensical
> [client]	challenge-response <username>:sha1(somethingnonsensical<password>)
> [server]	All right, the pack goes like this...
> 
>   Suddenly you have support for hopefully secure authentication, and at
> the same time you have the cookie implemented in backwards-compatible
> fashion (in the sense that new client will be able to talk to old
> server) - just assume the username and password empty. This might be
> even hardcoded for now, just leave a room for its addition (in an
> elegant and compatible way) in the protocol, please.
> 

I think git-daemon would be better off without this, since
* A project rarely grants write access to the central repo (or whatever 
git has, I'm still fairly new to it) without being willing to give out 
ssh access, often limited by the ssh command whitelist.
* It's hard to do right.
* Passwords are never as secure or as convenient as public key 
authentication and there's no point in spending a lot of time 
re-inventing ssh.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231