[PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[Jeff King]

When a caller of sha1_object_info_extended() sets the
"contentp" field in object_info, we call unpack_sha1_rest()
but do not check whether it signaled an error.

This causes two problems:

  1. We pass back NULL to the caller via the contentp field,
     but the function returns "0" for success. A caller
     might reasonably expect after a successful return that
     it can access contentp without a NULL check and
     segfault.

     As it happens, this is impossible to trigger in the
     current code. There is exactly one caller which uses
     contentp, read_object(). And the only thing it does
     after a successful call is to return the content
     pointer to its caller, using NULL as a sentinel for
     errors. So in effect it converts the success code from
     sha1_object_info_extended() back into an error!

     But this is still worth addressing avoid problems for
     future users of "contentp".

  2. Callers of unpack_sha1_rest() are expected to close the
     zlib stream themselves on error. Which means that we're
     leaking the stream.

The problem in (1) comes from from c84a1f3ed4 (sha1_file:
refactor read_object, 2017-06-21), which added the contentp
field.  Before that, we called unpack_sha1_rest() via
unpack_sha1_file(), which directly used the NULL to signal
an error.

But note that the leak in (2) is actually older than that.
The original unpack_sha1_file() directly returned the result
of unpack_sha1_rest() to its caller, when it should have
been closing the zlib stream itself on error.

Signed-off-by: Jeff King <peff@peff.net>
---
 sha1_file.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 09ad64ce55..10c3a0083d 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1124,10 +1124,14 @@ static int sha1_loose_object_info(const unsigned char *sha1,
 	} else if ((status = parse_sha1_header_extended(hdr, oi, flags)) < 0)
 		status = error("unable to parse %s header", sha1_to_hex(sha1));
 
-	if (status >= 0 && oi->contentp)
+	if (status >= 0 && oi->contentp) {
 		*oi->contentp = unpack_sha1_rest(&stream, hdr,
 						 *oi->sizep, sha1);
-	else
+		if (!*oi->contentp) {
+			git_inflate_end(&stream);
+			status = -1;
+		}
+	} else
 		git_inflate_end(&stream);
 
 	munmap(map, mapsize);
-- 
2.14.2.1117.g65a3442612

Re: [PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> When a caller of sha1_object_info_extended() sets the
> "contentp" field in object_info, we call unpack_sha1_rest()
> but do not check whether it signaled an error.
>
> This causes two problems:
>
>   1. We pass back NULL to the caller via the contentp field,
>      but the function returns "0" for success. A caller
>      might reasonably expect after a successful return that
>      it can access contentp without a NULL check and
>      segfault.
>
>      As it happens, this is impossible to trigger in the
>      current code. There is exactly one caller which uses
>      contentp, read_object(). And the only thing it does
>      after a successful call is to return the content
>      pointer to its caller, using NULL as a sentinel for
>      errors. So in effect it converts the success code from
>      sha1_object_info_extended() back into an error!
>
>      But this is still worth addressing avoid problems for
>      future users of "contentp".
>
>   2. Callers of unpack_sha1_rest() are expected to close the
>      zlib stream themselves on error. Which means that we're
>      leaking the stream.
>
> The problem in (1) comes from from c84a1f3ed4 (sha1_file:
> refactor read_object, 2017-06-21), which added the contentp
> field.  Before that, we called unpack_sha1_rest() via
> unpack_sha1_file(), which directly used the NULL to signal
> an error.
>
> But note that the leak in (2) is actually older than that.
> The original unpack_sha1_file() directly returned the result
> of unpack_sha1_rest() to its caller, when it should have
> been closing the zlib stream itself on error.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---

Obviously correct.  (2) is as old as Git itself; it eventually
blames down to e83c5163 ("Initial revision of "git", the information
manager from hell", 2005-04-07), where read-cache.c::unpack_sha1_file()
liberally returns NULL without cleaning up the zstream.

Thanks.

>  sha1_file.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/sha1_file.c b/sha1_file.c
> index 09ad64ce55..10c3a0083d 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -1124,10 +1124,14 @@ static int sha1_loose_object_info(const unsigned char *sha1,
>  	} else if ((status = parse_sha1_header_extended(hdr, oi, flags)) < 0)
>  		status = error("unable to parse %s header", sha1_to_hex(sha1));
>  
> -	if (status >= 0 && oi->contentp)
> +	if (status >= 0 && oi->contentp) {
>  		*oi->contentp = unpack_sha1_rest(&stream, hdr,
>  						 *oi->sizep, sha1);
> -	else
> +		if (!*oi->contentp) {
> +			git_inflate_end(&stream);
> +			status = -1;
> +		}
> +	} else
>  		git_inflate_end(&stream);
>  
>  	munmap(map, mapsize);

Re: [PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[Jeff King]

On Fri, Oct 06, 2017 at 01:19:21PM +0900, Junio C Hamano wrote:

> > But note that the leak in (2) is actually older than that.
> > The original unpack_sha1_file() directly returned the result
> > of unpack_sha1_rest() to its caller, when it should have
> > been closing the zlib stream itself on error.
> >
> > Signed-off-by: Jeff King <peff@peff.net>
> > ---
> 
> Obviously correct.  (2) is as old as Git itself; it eventually
> blames down to e83c5163 ("Initial revision of "git", the information
> manager from hell", 2005-04-07), where read-cache.c::unpack_sha1_file()
> liberally returns NULL without cleaning up the zstream.

Thanks, I as too lazy to dig down further, but I'm always interested to
see the roots of these things (especially "bug in the original" versus
"introduced by a careless refactor").

I have a feeling that the world would be a better place if
unpack_sha1_rest() just always promised to close the zstream, since no
callers seem to want to look at it in the error case. But I wanted to go
for the minimal fix first.

-Peff

Re: [PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[Jeff King]

On Fri, Oct 06, 2017 at 12:30:08AM -0400, Jeff King wrote:

> On Fri, Oct 06, 2017 at 01:19:21PM +0900, Junio C Hamano wrote:
> 
> > > But note that the leak in (2) is actually older than that.
> > > The original unpack_sha1_file() directly returned the result
> > > of unpack_sha1_rest() to its caller, when it should have
> > > been closing the zlib stream itself on error.
> > >
> > > Signed-off-by: Jeff King <peff@peff.net>
> > > ---
> > 
> > Obviously correct.  (2) is as old as Git itself; it eventually
> > blames down to e83c5163 ("Initial revision of "git", the information
> > manager from hell", 2005-04-07), where read-cache.c::unpack_sha1_file()
> > liberally returns NULL without cleaning up the zstream.
> 
> Thanks, I as too lazy to dig down further, but I'm always interested to
> see the roots of these things (especially "bug in the original" versus
> "introduced by a careless refactor").
> 
> I have a feeling that the world would be a better place if
> unpack_sha1_rest() just always promised to close the zstream, since no
> callers seem to want to look at it in the error case. But I wanted to go
> for the minimal fix first.

Actually, there are only two callers left these days. One of them leaks,
and the other immediately closes the zstream. So something like:

diff --git a/sha1_file.c b/sha1_file.c
index 09ad64ce55..cea003d182 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -978,10 +978,10 @@ static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long s
 		while (status == Z_OK)
 			status = git_inflate(stream, Z_FINISH);
 	}
-	if (status == Z_STREAM_END && !stream->avail_in) {
-		git_inflate_end(stream);
+	git_inflate_end(stream);
+
+	if (status == Z_STREAM_END && !stream->avail_in)
 		return buf;
-	}
 
 	if (status < 0)
 		error("corrupt loose object '%s'", sha1_to_hex(sha1));
@@ -2107,7 +2107,6 @@ int read_loose_object(const char *path,
 		*contents = unpack_sha1_rest(&stream, hdr, *size, expected_sha1);
 		if (!*contents) {
 			error("unable to unpack contents of %s", path);
-			git_inflate_end(&stream);
 			goto out;
 		}
 		if (check_sha1_signature(expected_sha1, *contents,

seems reasonable. Doing it that (with my other patch on top) splits the
leak-fix and the not-yet-a-bug-but-confusing-error-return problems into
two separate patches.

I dunno. There aren't that many callers of unpack_sha1_rest(), so it may
not matter that much, but while we're here...

-Peff

Re: [PATCH] sha1_loose_object_info: handle errors from unpack_sha1_rest

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Actually, there are only two callers left these days. One of them leaks,
> and the other immediately closes the zstream. So something like:
>
> diff --git a/sha1_file.c b/sha1_file.c
> index 09ad64ce55..cea003d182 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -978,10 +978,10 @@ static void *unpack_sha1_rest(git_zstream *stream, void *buffer, unsigned long s
>  		while (status == Z_OK)
>  			status = git_inflate(stream, Z_FINISH);
>  	}
> -	if (status == Z_STREAM_END && !stream->avail_in) {
> -		git_inflate_end(stream);
> +	git_inflate_end(stream);
> +
> +	if (status == Z_STREAM_END && !stream->avail_in)
>  		return buf;
> -	}
>  
>  	if (status < 0)
>  		error("corrupt loose object '%s'", sha1_to_hex(sha1));
> @@ -2107,7 +2107,6 @@ int read_loose_object(const char *path,
>  		*contents = unpack_sha1_rest(&stream, hdr, *size, expected_sha1);
>  		if (!*contents) {
>  			error("unable to unpack contents of %s", path);
> -			git_inflate_end(&stream);
>  			goto out;
>  		}
>  		if (check_sha1_signature(expected_sha1, *contents,
>
> seems reasonable. Doing it that (with my other patch on top) splits the
> leak-fix and the not-yet-a-bug-but-confusing-error-return problems into
> two separate patches.
>
> I dunno. There aren't that many callers of unpack_sha1_rest(), so it may
> not matter that much, but while we're here...

Yeah, I agree that it is a reasonable thing to do.