remote_get_heads: reference length limit

remote_get_heads: reference length limit

[Andy Whitcroft]

I've been having trouble with git push apparently resending the entire
commit trace for the branch each and every time I push.  Poking at the
source it seems this is due to a length limit on reference names as
pulled from the remote repository.

When we are building the pack to send we are sent a list of remote
heads.  get_remote_heads() reads these in, validates them and finally
adds them to the remote_refs list.  Part of the validation is a simple
check for size and form; check_ref().

static int check_ref(const char *name, int len, unsigned int flags)
{
        if (!flags)
                return 1;

        if (len > 45 || memcmp(name, "refs/", 5))
                return 0;
[...]
}

With the refs/heads/ prefix included this limits the head names to 34
characters.  From what I can see there is no good reason for this limit
to be so low.  I can see we don't want the remote end bloating us out of
control, but we are already limiting the lines which contain these
references to 1000 bytes and making no attempt to limit the number of
them the remote server can send us.  There seems to be no limits imposed
on the name length other than MAX_PATHLEN.

Can anyone see a reason to keep this (len > 45) check?

-apw

Re: remote_get_heads: reference length limit

[Junio C Hamano]

Andy Whitcroft <apw@shadowen.org> writes:

> I've been having trouble with git push apparently resending the entire
> commit trace for the branch each and every time I push.  Poking at the
> source it seems this is due to a length limit on reference names as
> pulled from the remote repository.
>
> When we are building the pack to send we are sent a list of remote
> heads.  get_remote_heads() reads these in, validates them and finally
> adds them to the remote_refs list.  Part of the validation is a simple
> check for size and form; check_ref().
>
> static int check_ref(const char *name, int len, unsigned int flags)
> {
>         if (!flags)
>                 return 1;
>
>         if (len > 45 || memcmp(name, "refs/", 5))
>                 return 0;
> [...]
> }
>
> With the refs/heads/ prefix included this limits the head names to 34
> characters.  From what I can see there is no good reason for this limit
> to be so low.  I can see we don't want the remote end bloating us out of
> control, but we are already limiting the lines which contain these
> references to 1000 bytes and making no attempt to limit the number of
> them the remote server can send us.  There seems to be no limits imposed
> on the name length other than MAX_PATHLEN.
>
> Can anyone see a reason to keep this (len > 45) check?

Yes, it was a mis-conversion done with commit 2718ff0.
The data this part (and the caller) is dealing with are
peek-remote output (40-byte hex object name, tab and
"refs/...").  Before that commit we had a check like this:

+		if (ignore_funny && 45 < len && !memcmp(name, "refs/", 5) &&
+		    check_ref_format(name + 5))
+			continue;

That is, "if we are ignoring funny fake refs, continue when the
name begins with refs/ and is magic (e.g. refs/heads/master^{});
by the way that can only happen if the entire line length is
more than 45 bytes (because of 40-byte hex plus tab, anything
shorter than that cannot even fit "refs/" at the beginning of
the name).

But when the check_ref() function was written it was made to
take the name length as "len", so I do not think there is a
reason to check the name against 45 at all.  Also the direction
of the comparison is wrong in the new code.  It was meant to be
"line must be at least this long otherwise it is not right".

I think you can just drop "len > 45 || " part.

[PATCH] send pack remove remote reference limit

[Andy Whitcroft]

send-pack: remove remote reference limit

When build a pack for a push we query the remote copy for existant
heads.  These are used to prune unnecessary objects from the pack.
As we receive the remote references in get_remote_heads() we validate
the reference names via check_ref() which includes a length check;
rejecting those >45 characters in size.

This is a miss converted change, it was originally designed to reject
messages which were less than 45 characters in length (a 40 character
sha1 and refs/) to prevent comparing unitialised memory.  check_ref()
now gets the raw length so check for at least 5 characters.

Signed-off-by: Andy Whitcroft <apw@shadowen.org>
---
diff --git a/connect.c b/connect.c
index e501ccc..371aea1 100644
--- a/connect.c
+++ b/connect.c
@@ -17,7 +17,7 @@ static int check_ref(const char *name, i
 	if (!flags)
 		return 1;
 
-	if (len > 45 || memcmp(name, "refs/", 5))
+	if (len < 5 || memcmp(name, "refs/", 5))
 		return 0;
 
 	/* Skip the "refs/" part */

remote_get_heads: reference length limit

[Andy Whitcroft]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

I've been having trouble with git push apparently resending the entire
commit trace for the branch each and every time I push.  Poking at the
source it seems this is due to a length limit on reference names as
pulled from the remote repository.

When we are building the pack to send we are sent a list of remote
heads.  get_remote_heads() reads these in, validates them and finally
adds them to the remote_refs list.  Part of the validation is a simple
check for size and form; check_ref().

static int check_ref(const char *name, int len, unsigned int flags)
{
        if (!flags)
                return 1;

        if (len > 45 || memcmp(name, "refs/", 5))
                return 0;
[...]
}

With the refs/heads/ prefix included this limits the head names to 34
characters.  From what I can see there is no good reason for this limit
to be so low.  I can see we don't want the remote end bloating us out of
control, but we are already limiting the lines which contain these
references to 1000 bytes and making no attempt to limit the number of
them the remote server can send us.  There seems to be no limits imposed
on the name length other than MAX_PATHLEN.

Can anyone see a reason to keep this (len > 45) check?

-apw

[-- Attachment #2: send-pack-remove-remote-reference-limit --]
[-- Type: text/plain, Size: 841 bytes --]

send-pack: remove remote reference limit

When build a pack for a push we query the remote copy for existant
heads.  These are used to prune unnecessary objects from the pack.
As we receive the remote references in get_remote_heads() we validate
the reference names via check_ref() which includes a length check;
rejecting those >45 characters in size.

We appear to be able to handle reference names upto MAXPATHLEN in
size.  Relax the limit out to that size.

Signed-off-by: Andy Whitcroft <apw@shadowen.org>
---
diff --git a/connect.c b/connect.c
index 4422a0d..0a51e78 100644
--- a/connect.c
+++ b/connect.c
@@ -17,7 +17,7 @@ static int check_ref(const char *name, i
 	if (!flags)
 		return 1;
 
-	if (len > 45 || memcmp(name, "refs/", 5))
+	if (len > MAXPATHLEN || memcmp(name, "refs/", 5))
 		return 0;
 
 	/* Skip the "refs/" part */