Re: [PATCH v2 3/7] url: add str wrapper for end_url_with_slash()

Re: [PATCH v2 3/7] url: add str wrapper for end_url_with_slash()

[Jonathan Nieder]

Tay Ray Chuan wrote:

> --- a/url.c
> +++ b/url.c
> @@ -132,3 +132,11 @@ void end_url_with_slash(struct strbuf *buf, const char *url)
>  	if (buf->len && buf->buf[buf->len - 1] != '/')
>  		strbuf_addstr(buf, "/");
>  }
> +
> +void str_end_url_with_slash(const char *url, char **dest) {
> +	struct strbuf buf = STRBUF_INIT;
> +	end_url_with_slash(&buf, url);
> +	if (dest)
> +		free(*dest);
> +	*dest = strbuf_detach(&buf, NULL);

I think this should be

	end_url_with_slash(&buf, url);
	free(*dest);
	*dest = strbuf_detach(...

If dest were NULL, the assignment would segfault anyway.  If dest were
non-NULL and *dest NULL, the free() would be a no-op.

Re: [PATCH v2 3/7] url: add str wrapper for end_url_with_slash()

[Tay Ray Chuan]

Hi,

On Tue, Nov 23, 2010 at 1:14 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Tay Ray Chuan wrote:
>
>> --- a/url.c
>> +++ b/url.c
>> @@ -132,3 +132,11 @@ void end_url_with_slash(struct strbuf *buf, const char *url)
>>       if (buf->len && buf->buf[buf->len - 1] != '/')
>>               strbuf_addstr(buf, "/");
>>  }
>> +
>> +void str_end_url_with_slash(const char *url, char **dest) {
>> +     struct strbuf buf = STRBUF_INIT;
>> +     end_url_with_slash(&buf, url);
>> +     if (dest)
>> +             free(*dest);
>> +     *dest = strbuf_detach(&buf, NULL);
>
> I think this should be
>
>        end_url_with_slash(&buf, url);
>        free(*dest);
>        *dest = strbuf_detach(...
>
> If dest were NULL, the assignment would segfault anyway.  If dest were
> non-NULL and *dest NULL, the free() would be a no-op.

Thanks for the tip.

-- 
Cheers,
Ray Chuan

[PATCH v3 0/9] http: end urls with slash

[Tay Ray Chuan]

This patch series contains Gabriel's fix, as well as some related work I did in
the area while looking through the code paths.

Changed from v2:
 - new patches (#2 and #9) that reworks http-fetch url slashing code, and test
   that this doesn't break anything.
 - patch #4: remove pointer check before running free(), as pointed out by
   Johnathan.

All other patches remain unchanged.

Contents:
[1/9] t5550-http-fetch: add missing '&&'
[2/9] t5550-http-fetch: add test for http-fetch
[3/9] shift end_url_with_slash() from http.[ch] to url.[ch]
[4/9] url: add str wrapper for end_url_with_slash()
[5/9] http-backend: use end_url_with_slash()
[6/9] http-push: Normalise directory names when pushing to some WebDAV servers
[7/9] http-push: check path length before using it
[8/9] http-push: add trailing slash at arg-parse time, instead of later on
[9/9] http-fetch: rework url handling

 Makefile              |    2 +-
 http-backend.c        |    4 +---
 http-fetch.c          |   16 +++++-----------
 http-push.c           |   31 ++++++++++++++++---------------
 http.c                |    8 +-------
 http.h                |    2 +-
 t/t5550-http-fetch.sh |   16 ++++++++++++++--
 url.c                 |   14 ++++++++++++++
 url.h                 |    3 +++
 9 files changed, 56 insertions(+), 40 deletions(-)

-- 
1.7.3.2.495.gc7b3f

[PATCH v3 1/9] t5550-http-fetch: add missing '&&'

[Tay Ray Chuan]

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 t/t5550-http-fetch.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh
index 2fb48d0..39f7b7c 100755
--- a/t/t5550-http-fetch.sh
+++ b/t/t5550-http-fetch.sh
@@ -37,7 +37,7 @@ test_expect_success 'clone http repository' '
 test_expect_success 'fetch changes via http' '
 	echo content >>file &&
 	git commit -a -m two &&
-	git push public
+	git push public &&
 	(cd clone && git pull) &&
 	test_cmp file clone/file
 '
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 2/9] t5550-http-fetch: add test for http-fetch

[Tay Ray Chuan]

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---

New in v3.

 t/t5550-http-fetch.sh |   14 +++++++++++++-
 1 files changed, 13 insertions(+), 1 deletions(-)

diff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh
index 39f7b7c..d3bf5ce 100755
--- a/t/t5550-http-fetch.sh
+++ b/t/t5550-http-fetch.sh
@@ -30,7 +30,8 @@ test_expect_success 'create http-accessible bare repository' '
 '
 
 test_expect_success 'clone http repository' '
-	git clone $HTTPD_URL/dumb/repo.git clone &&
+	git clone $HTTPD_URL/dumb/repo.git clone-tmpl &&
+	cp -R clone-tmpl clone &&
 	test_cmp file clone/file
 '
 
@@ -42,6 +43,17 @@ test_expect_success 'fetch changes via http' '
 	test_cmp file clone/file
 '
 
+test_expect_success 'fetch changes via manual http-fetch' '
+	cp -R clone-tmpl clone2 &&
+
+	HEAD=$(git rev-parse --verify HEAD) &&
+	(cd clone2 &&
+	 git http-fetch -a -w heads/master-new $HEAD $(git config remote.origin.url) &&
+	 git checkout master-new &&
+	 test $HEAD = $(git rev-parse --verify HEAD)) &&
+	test_cmp file clone2/file
+'
+
 test_expect_success 'http remote detects correct HEAD' '
 	git push public master:other &&
 	(cd clone &&
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 3/9] shift end_url_with_slash() from http.[ch] to url.[ch]

[Tay Ray Chuan]

This allows non-http/curl users to access it too (eg. http-backend.c).

Update include headers in end_url_with_slash() users too.

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 Makefile |    2 +-
 http.c   |    8 +-------
 http.h   |    2 +-
 url.c    |    7 +++++++
 url.h    |    2 ++
 5 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/Makefile b/Makefile
index ca1d1fe..e02b0f4 100644
--- a/Makefile
+++ b/Makefile
@@ -1922,7 +1922,7 @@ builtin/tar-tree.o archive-tar.o: tar.h
 builtin/pack-objects.o: thread-utils.h
 connect.o transport.o http-backend.o: url.h
 http-fetch.o http-walker.o remote-curl.o transport.o walker.o: walker.h
-http.o http-walker.o http-push.o http-fetch.o remote-curl.o: http.h
+http.o http-walker.o http-push.o http-fetch.o remote-curl.o: http.h url.h
 
 xdiff-interface.o $(XDIFF_OBJS): \
 	xdiff/xinclude.h xdiff/xmacros.h xdiff/xdiff.h xdiff/xtypes.h \
diff --git a/http.c b/http.c
index 17bcf19..310dd86 100644
--- a/http.c
+++ b/http.c
@@ -2,6 +2,7 @@
 #include "pack.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "url.h"
 
 int data_received;
 int active_requests;
@@ -733,13 +734,6 @@ static inline int hex(int v)
 		return 'A' + v - 10;
 }
 
-void end_url_with_slash(struct strbuf *buf, const char *url)
-{
-	strbuf_addstr(buf, url);
-	if (buf->len && buf->buf[buf->len - 1] != '/')
-		strbuf_addstr(buf, "/");
-}
-
 static char *quote_ref_url(const char *base, const char *ref)
 {
 	struct strbuf buf = STRBUF_INIT;
diff --git a/http.h b/http.h
index 173f74c..5c6e243 100644
--- a/http.h
+++ b/http.h
@@ -8,6 +8,7 @@
 
 #include "strbuf.h"
 #include "remote.h"
+#include "url.h"
 
 /*
  * We detect based on the cURL version if multi-transfer is
@@ -117,7 +118,6 @@ extern void append_remote_object_url(struct strbuf *buf, const char *url,
 				     int only_two_digit_prefix);
 extern char *get_remote_object_url(const char *url, const char *hex,
 				   int only_two_digit_prefix);
-extern void end_url_with_slash(struct strbuf *buf, const char *url);
 
 /* Options for http_request_*() */
 #define HTTP_NO_CACHE		1
diff --git a/url.c b/url.c
index cd8f74f..7cebc64 100644
--- a/url.c
+++ b/url.c
@@ -125,3 +125,10 @@ char *url_decode_parameter_value(const char **query)
 	struct strbuf out = STRBUF_INIT;
 	return url_decode_internal(query, "&", &out, 1);
 }
+
+void end_url_with_slash(struct strbuf *buf, const char *url)
+{
+	strbuf_addstr(buf, url);
+	if (buf->len && buf->buf[buf->len - 1] != '/')
+		strbuf_addstr(buf, "/");
+}
diff --git a/url.h b/url.h
index 15817f8..8cb74d4 100644
--- a/url.h
+++ b/url.h
@@ -7,4 +7,6 @@ extern char *url_decode(const char *url);
 extern char *url_decode_parameter_name(const char **query);
 extern char *url_decode_parameter_value(const char **query);
 
+extern void end_url_with_slash(struct strbuf *buf, const char *url);
+
 #endif /* URL_H */
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 4/9] url: add str wrapper for end_url_with_slash()

[Tay Ray Chuan]

Helped-by: Johnathan Nieder <jrnieder@gmail.com>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---

Changed from v2: removed pointer check before free(), as suggested by Johnathan.

 url.c |    7 +++++++
 url.h |    1 +
 2 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/url.c b/url.c
index 7cebc64..6a54959 100644
--- a/url.c
+++ b/url.c
@@ -132,3 +132,10 @@ void end_url_with_slash(struct strbuf *buf, const char *url)
 	if (buf->len && buf->buf[buf->len - 1] != '/')
 		strbuf_addstr(buf, "/");
 }
+
+void str_end_url_with_slash(const char *url, char **dest) {
+	struct strbuf buf = STRBUF_INIT;
+	end_url_with_slash(&buf, url);
+	free(*dest);
+	*dest = strbuf_detach(&buf, NULL);
+}
diff --git a/url.h b/url.h
index 8cb74d4..7100e32 100644
--- a/url.h
+++ b/url.h
@@ -8,5 +8,6 @@ extern char *url_decode_parameter_name(const char **query);
 extern char *url_decode_parameter_value(const char **query);
 
 extern void end_url_with_slash(struct strbuf *buf, const char *url);
+extern void str_end_url_with_slash(const char *url, char **dest);
 
 #endif /* URL_H */
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 5/9] http-backend: use end_url_with_slash()

[Tay Ray Chuan]

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 http-backend.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/http-backend.c b/http-backend.c
index 14c90c2..8501504 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -510,9 +510,7 @@ static char* getdir(void)
 			die("GIT_PROJECT_ROOT is set but PATH_INFO is not");
 		if (daemon_avoid_alias(pathinfo))
 			die("'%s': aliased", pathinfo);
-		strbuf_addstr(&buf, root);
-		if (buf.buf[buf.len - 1] != '/')
-			strbuf_addch(&buf, '/');
+		end_url_with_slash(&buf, root);
 		if (pathinfo[0] == '/')
 			pathinfo++;
 		strbuf_addstr(&buf, pathinfo);
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 6/9] http-push: Normalise directory names when pushing to some WebDAV servers

[Tay Ray Chuan]

Fix a bug when pushing to WebDAV servers which do not use a trailing
slash for collection names. The previous implementation fails to see
that the requested resource "refs/" is the same resource as "refs"
and loads every reference twice (once for refs/ and once for refs).

This implementation normalises every collection name by appending a
trailing slash if necessary.

This can be tested with old versions of Apache (such as the WebDAV
server of GMX, Apache 2.0.63).

Based-on-patch-by: Gabriel Corona <gabriel.corona@enst-bretagne.fr>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 http-push.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/http-push.c b/http-push.c
index c9bcd11..565e580 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1090,6 +1090,10 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 	if (tag_closed) {
 		if (!strcmp(ctx->name, DAV_PROPFIND_RESP) && ls->dentry_name) {
 			if (ls->dentry_flags & IS_DIR) {
+
+				/* ensure collection names end with slash */
+				str_end_url_with_slash(ls->dentry_name, &ls->dentry_name);
+
 				if (ls->flags & PROCESS_DIRS) {
 					ls->userFunc(ls);
 				}
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 7/9] http-push: check path length before using it

[Tay Ray Chuan]

We use path_len to skip the base url/path, but we do not know for sure
if path does indeed contain the base url/path. Check if this is so.

Helped-by: Johnathan Nieder <jrnieder@gmail.com>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 http-push.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/http-push.c b/http-push.c
index 565e580..bfa1fe7 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1116,8 +1116,16 @@ static void handle_remote_ls_ctx(struct xml_ctx *ctx, int tag_closed)
 				}
 			}
 			if (path) {
-				path += repo->path_len;
-				ls->dentry_name = xstrdup(path);
+				const char *url = repo->url;
+				if (repo->path)
+					url = repo->path;
+				if (strncmp(path, url, repo->path_len))
+					error("Parsed path '%s' does not match url: '%s'\n",
+					      path, url);
+				else {
+					path += repo->path_len;
+					ls->dentry_name = xstrdup(path);
+				}
 			}
 		} else if (!strcmp(ctx->name, DAV_PROPFIND_COLLECTION)) {
 			ls->dentry_flags |= IS_DIR;
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 8/9] http-push: add trailing slash at arg-parse time, instead of later on

[Tay Ray Chuan]

That way, we don't have to update repo->path and repo->path_len again
after adding the trailing slash.

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---
 http-push.c |   15 ++-------------
 1 files changed, 2 insertions(+), 13 deletions(-)

diff --git a/http-push.c b/http-push.c
index bfa1fe7..ff41a0e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1801,7 +1801,6 @@ int main(int argc, char **argv)
 	int new_refs;
 	struct ref *ref, *local_refs;
 	struct remote *remote;
-	char *rewritten_url = NULL;
 
 	git_extract_argv0_path(argv[0]);
 
@@ -1847,8 +1846,8 @@ int main(int argc, char **argv)
 		}
 		if (!repo->url) {
 			char *path = strstr(arg, "//");
-			repo->url = arg;
-			repo->path_len = strlen(arg);
+			str_end_url_with_slash(arg, &repo->url);
+			repo->path_len = strlen(repo->url);
 			if (path) {
 				repo->path = strchr(path+2, '/');
 				if (repo->path)
@@ -1884,15 +1883,6 @@ int main(int argc, char **argv)
 	remote->url[remote->url_nr++] = repo->url;
 	http_init(remote);
 
-	if (repo->url && repo->url[strlen(repo->url)-1] != '/') {
-		rewritten_url = xmalloc(strlen(repo->url)+2);
-		strcpy(rewritten_url, repo->url);
-		strcat(rewritten_url, "/");
-		repo->path = rewritten_url + (repo->path - repo->url);
-		repo->path_len++;
-		repo->url = rewritten_url;
-	}
-
 #ifdef USE_CURL_MULTI
 	is_running_queue = 0;
 #endif
@@ -2100,7 +2090,6 @@ int main(int argc, char **argv)
 	}
 
  cleanup:
-	free(rewritten_url);
 	if (info_ref_lock)
 		unlock_remote(info_ref_lock);
 	free(repo);
-- 
1.7.3.2.495.gc7b3f

[PATCH v3 9/9] http-fetch: rework url handling

[Tay Ray Chuan]

Do away with a second url variable, rewritten_url, and make url
non-const. This is safe because the functions called with url (ie.
get_http_walker() and walker_fetch()) do not modify it (ie. marked with
const char *).

Also, replace code that adds a trailing slash with a call to
str_end_url_with_slash().

Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
---

New in v3.

 http-fetch.c |   16 +++++-----------
 1 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/http-fetch.c b/http-fetch.c
index 762c750..923904f 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -14,8 +14,7 @@ int main(int argc, const char **argv)
 	int commits;
 	const char **write_ref = NULL;
 	char **commit_id;
-	const char *url;
-	char *rewritten_url = NULL;
+	char *url = NULL;
 	int arg = 1;
 	int rc = 0;
 	int get_tree = 0;
@@ -57,19 +56,14 @@ int main(int argc, const char **argv)
 		commit_id = (char **) &argv[arg++];
 		commits = 1;
 	}
-	url = argv[arg];
+
+	if (argv[arg])
+		str_end_url_with_slash(argv[arg], &url);
 
 	prefix = setup_git_directory();
 
 	git_config(git_default_config, NULL);
 
-	if (url && url[strlen(url)-1] != '/') {
-		rewritten_url = xmalloc(strlen(url)+2);
-		strcpy(rewritten_url, url);
-		strcat(rewritten_url, "/");
-		url = rewritten_url;
-	}
-
 	http_init(NULL);
 	walker = get_http_walker(url);
 	walker->get_tree = get_tree;
@@ -93,7 +87,7 @@ int main(int argc, const char **argv)
 	walker_free(walker);
 	http_cleanup();
 
-	free(rewritten_url);
+	free(url);
 
 	return rc;
 }
-- 
1.7.3.2.495.gc7b3f