* [PATCH v3 0/4] http cleanups
@ 2011-05-03 15:47 Tay Ray Chuan
2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Tay Ray Chuan @ 2011-05-03 15:47 UTC (permalink / raw)
To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee
This is a re-roll of dm/http-cleanup. I've also included a test for
chunked http pushes that has been sitting on my side for some time.
Note to Junio: these patches should apply safely on
sp/maint-clear-postfields. Actually, the first patch depends on it.
[1/4] t5541-http-push: add test for chunked
[2/4] http: make curl callbacks match contracts from curl header
[3/4] http-push: use const for strings in signatures
[4/4] http-push: refactor curl_easy_setup madness
http-push.c | 152 +++++++++++++++++---------------------------------
http-walker.c | 4 +-
http.c | 12 ++--
http.h | 6 +-
remote-curl.c | 2 +-
t/t5541-http-push.sh | 28 +++++++++-
6 files changed, 90 insertions(+), 114 deletions(-)
--
1.7.3.3.585.g74f6e
^ permalink raw reply [flat|nested] 20+ messages in thread* [PATCH v3 1/4] t5541-http-push: add test for chunked 2011-05-03 15:47 [PATCH v3 0/4] http cleanups Tay Ray Chuan @ 2011-05-03 15:47 ` Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan ` (2 more replies) 2011-05-04 4:09 ` [PATCH v3 0/4] http cleanups Junio C Hamano 2011-05-04 10:11 ` [PATCH v4 " Tay Ray Chuan 2 siblings, 3 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-03 15:47 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee Trigger the chunked type of pushing for smart HTTP. This can serve as a regression test for the issue fixed in 1e41827 (http: clear POSTFIELDS when initializing a slot). Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- New. t/t5541-http-push.sh | 28 ++++++++++++++++++++++++++-- 1 files changed, 26 insertions(+), 2 deletions(-) diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh index 0492877..f589221 100755 --- a/t/t5541-http-push.sh +++ b/t/t5541-http-push.sh @@ -65,14 +65,16 @@ test_expect_success 'clone remote repository' ' git clone $HTTPD_URL/smart/test_repo.git test_repo_clone ' -test_expect_success 'push to remote repository' ' +test_expect_success 'push to remote repository (standard)' ' cd "$ROOT_PATH"/test_repo_clone && : >path2 && git add path2 && test_tick && git commit -m path2 && HEAD=$(git rev-parse --verify HEAD) && - git push && + GIT_CURL_VERBOSE=1 git push -v -v 2>err && + ! grep "Expect: 100-continue" err && + grep "POST git-receive-pack (376 bytes)" err && (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && test $HEAD = $(git rev-parse --verify HEAD)) ' @@ -140,5 +142,27 @@ test_expect_success C_LOCALE_OUTPUT 'push fails for non-fast-forward refs unmatc output ' +test_expect_success 'push (chunked)' ' + BRANCH=master-chunked && + REPO=test_repo_chunked && + + (cd "$HTTPD_DOCUMENT_ROOT_PATH" && + cp -R test_repo.git $REPO) && + git remote set-url origin $HTTPD_URL/smart/$REPO && + + # to trigger chunked pushing, we need a sufficiently large pack - use + # git v0.99 + GIT_REPO=$TEST_DIRECTORY/../.git && + test -d $GIT_REPO && + echo $GIT_REPO/objects > .git/objects/info/alternates && + git fetch $GIT_REPO refs/tags/v0.99 && + git branch $BRANCH FETCH_HEAD && + BRANCH_REF=$(git rev-parse --verify refs/heads/$BRANCH) && + git push -v -v origin $BRANCH 2>err && + grep "POST git-receive-pack (chunked)" err && + (cd "$HTTPD_DOCUMENT_ROOT_PATH"/$REPO && + test $BRANCH_REF = $(git rev-parse --verify refs/heads/$BRANCH)) +' + stop_httpd test_done -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 2/4] http: make curl callbacks match contracts from curl header 2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan @ 2011-05-03 15:47 ` Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 3/4] http-push: use const for strings in signatures Tay Ray Chuan 2011-05-03 16:32 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Shawn Pearce 2011-05-04 4:07 ` Junio C Hamano 2 siblings, 1 reply; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-03 15:47 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> Yes, these don't match perfectly with the void* first parameter of the fread/fwrite in the standard library, but they do match the curl expected method signature. This is needed when a refactor passes a curl_write_callback around, which would otherwise give incorrect parameter warnings. Signed-off-by: Dan McGee <dpmcgee@gmail.com> --- Unchanged. http-walker.c | 4 ++-- http.c | 12 ++++++------ http.h | 6 +++--- remote-curl.c | 2 +- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/http-walker.c b/http-walker.c index 9bc8114..51a906e 100644 --- a/http-walker.c +++ b/http-walker.c @@ -185,7 +185,7 @@ static void process_alternates_response(void *callback_data) struct active_request_slot *slot = alt_req->slot; struct alt_base *tail = cdata->alt; const char *base = alt_req->base; - static const char null_byte = '\0'; + const char null_byte = '\0'; char *data; int i = 0; @@ -218,7 +218,7 @@ static void process_alternates_response(void *callback_data) } } - fwrite_buffer(&null_byte, 1, 1, alt_req->buffer); + fwrite_buffer((char *)&null_byte, 1, 1, alt_req->buffer); alt_req->buffer->len--; data = alt_req->buffer->buf; diff --git a/http.c b/http.c index b27bb57..b2ae8de 100644 --- a/http.c +++ b/http.c @@ -60,7 +60,7 @@ static struct curl_slist *no_pragma_header; static struct active_request_slot *active_queue_head; -size_t fread_buffer(void *ptr, size_t eltsize, size_t nmemb, void *buffer_) +size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; struct buffer *buffer = buffer_; @@ -92,7 +92,7 @@ curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp) } #endif -size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *buffer_) +size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; struct strbuf *buffer = buffer_; @@ -102,7 +102,7 @@ size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *buffer return size; } -size_t fwrite_null(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf) +size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf) { data_received++; return eltsize * nmemb; @@ -1167,7 +1167,7 @@ abort: } /* Helpers for fetching objects (loose) */ -static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb, +static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb, void *data) { unsigned char expn[4096]; @@ -1184,7 +1184,7 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb, } while (posn < size); freq->stream.avail_in = size; - freq->stream.next_in = ptr; + freq->stream.next_in = (void *)ptr; do { freq->stream.next_out = expn; freq->stream.avail_out = sizeof(expn); @@ -1203,7 +1203,7 @@ struct http_object_request *new_http_object_request(const char *base_url, char *filename; char prevfile[PATH_MAX]; int prevlocal; - unsigned char prev_buf[PREV_BUF_SIZE]; + char prev_buf[PREV_BUF_SIZE]; ssize_t prev_read = 0; long prev_posn = 0; char range[RANGE_HEADER_SIZE]; diff --git a/http.h b/http.h index e9ed3c2..19b7134 100644 --- a/http.h +++ b/http.h @@ -66,9 +66,9 @@ struct buffer { }; /* Curl request read/write callbacks */ -extern size_t fread_buffer(void *ptr, size_t eltsize, size_t nmemb, void *strbuf); -extern size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf); -extern size_t fwrite_null(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); #ifndef NO_CURL_IOCTL extern curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp); #endif diff --git a/remote-curl.c b/remote-curl.c index 775d614..17d8a9b 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -347,7 +347,7 @@ static curlioerr rpc_ioctl(CURL *handle, int cmd, void *clientp) } #endif -static size_t rpc_in(const void *ptr, size_t eltsize, +static size_t rpc_in(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 3/4] http-push: use const for strings in signatures 2011-05-03 15:47 ` [PATCH v3 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan @ 2011-05-03 15:47 ` Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 4/4] http-push: refactor curl_easy_setup madness Tay Ray Chuan 0 siblings, 1 reply; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-03 15:47 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- Split off from patch #4. http-push.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/http-push.c b/http-push.c index d18346c..0fce38d 100644 --- a/http-push.c +++ b/http-push.c @@ -169,7 +169,7 @@ enum dav_header_flag { DAV_HEADER_TIMEOUT = (1u << 2) }; -static char *xml_entities(char *s) +static char *xml_entities(const char *s) { struct strbuf buf = STRBUF_INIT; while (*s) { @@ -797,7 +797,7 @@ static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed) } } -static void one_remote_ref(char *refname); +static void one_remote_ref(const char *refname); static void xml_start_tag(void *userData, const char *name, const char **atts) @@ -1471,7 +1471,7 @@ static int update_remote(unsigned char *sha1, struct remote_lock *lock) static struct ref *remote_refs; -static void one_remote_ref(char *refname) +static void one_remote_ref(const char *refname) { struct ref *ref; struct object *obj; @@ -1660,7 +1660,7 @@ static int verify_merge_base(unsigned char *head_sha1, unsigned char *branch_sha return (merge_bases && !merge_bases->next && merge_bases->item == branch); } -static int delete_remote_branch(char *pattern, int force) +static int delete_remote_branch(const char *pattern, int force) { struct ref *refs = remote_refs; struct ref *remote_ref = NULL; -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v3 4/4] http-push: refactor curl_easy_setup madness 2011-05-03 15:47 ` [PATCH v3 3/4] http-push: use const for strings in signatures Tay Ray Chuan @ 2011-05-03 15:47 ` Tay Ray Chuan 0 siblings, 0 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-03 15:47 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> We were doing (nearly) the same thing all over the place, in slightly different orders, different variable names, etc. Refactor most calls into two helper functions, one for GET and one for everything else, that do the heavy lifting leaving most callsites a lot cleaner in the process. Note that the setting of CURLOPT_PUT at the callsites of curl_setup_http() which previously didn't do it (eg. locking_available(), remote_ls()) is safe, since that option is deprecated in libcurl in place of, and has the same effect as, CURLOPT_UPLOAD. Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- Inserted the second paragraph on CURLOPT_PUT and CURLOPT_UPLOAD. http-push.c | 144 ++++++++++++++++++++--------------------------------------- 1 files changed, 48 insertions(+), 96 deletions(-) diff --git a/http-push.c b/http-push.c index 0fce38d..28bfe76 100644 --- a/http-push.c +++ b/http-push.c @@ -197,6 +197,34 @@ static char *xml_entities(const char *s) return strbuf_detach(&buf, NULL); } +static void curl_setup_http_get(CURL *curl, const char *url, + const char *custom_req) +{ + curl_easy_setopt(curl, CURLOPT_HTTPGET, 1); + curl_easy_setopt(curl, CURLOPT_URL, url); + curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, custom_req); + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, fwrite_null); +} + +static void curl_setup_http(CURL *curl, const char *url, + const char *custom_req, struct buffer *buffer, + curl_write_callback write_fn) +{ + curl_easy_setopt(curl, CURLOPT_PUT, 1); + curl_easy_setopt(curl, CURLOPT_URL, url); + curl_easy_setopt(curl, CURLOPT_INFILE, buffer); + curl_easy_setopt(curl, CURLOPT_INFILESIZE, buffer->buf.len); + curl_easy_setopt(curl, CURLOPT_READFUNCTION, fread_buffer); +#ifndef NO_CURL_IOCTL + curl_easy_setopt(curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); + curl_easy_setopt(curl, CURLOPT_IOCTLDATA, &buffer); +#endif + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_fn); + curl_easy_setopt(curl, CURLOPT_NOBODY, 0); + curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, custom_req); + curl_easy_setopt(curl, CURLOPT_UPLOAD, 1); +} + static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options) { struct strbuf buf = STRBUF_INIT; @@ -272,11 +300,8 @@ static void start_mkcol(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); /* undo PUT setup */ - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); + curl_setup_http_get(slot->curl, request->url, DAV_MKCOL); curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, request->errorstr); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MKCOL); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); if (start_active_slot(slot)) { request->slot = slot; @@ -395,19 +420,8 @@ static void start_put(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &request->buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, request->buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &request->buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); + curl_setup_http(slot->curl, request->url, DAV_PUT, + &request->buffer, fwrite_null); if (start_active_slot(slot)) { request->slot = slot; @@ -427,13 +441,10 @@ static void start_move(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); /* undo PUT setup */ - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MOVE); + curl_setup_http_get(slot->curl, request->url, DAV_MOVE); dav_headers = curl_slist_append(dav_headers, request->dest); dav_headers = curl_slist_append(dav_headers, "Overwrite: T"); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); if (start_active_slot(slot)) { request->slot = slot; @@ -458,10 +469,7 @@ static int refresh_lock(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK); + curl_setup_http_get(slot->curl, lock->url, DAV_LOCK); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { @@ -876,10 +884,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout) ep[1] = '\0'; slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MKCOL); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); + curl_setup_http_get(slot->curl, url, DAV_MKCOL); if (start_active_slot(slot)) { run_active_slot(slot); if (results.curl_result != CURLE_OK && @@ -909,19 +914,9 @@ static struct remote_lock *lock_remote(const char *path, long timeout) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK); + curl_setup_http(slot->curl, url, DAV_LOCK, &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); lock = xcalloc(1, sizeof(*lock)); lock->timeout = -1; @@ -987,9 +982,7 @@ static int unlock_remote(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_UNLOCK); + curl_setup_http_get(slot->curl, lock->url, DAV_UNLOCK); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { @@ -1167,19 +1160,10 @@ static void remote_ls(const char *path, int flags, slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND); + curl_setup_http(slot->curl, url, DAV_PROPFIND, + &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1250,19 +1234,10 @@ static int locking_available(void) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, repo->url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND); + curl_setup_http(slot->curl, repo->url, DAV_PROPFIND, + &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1436,19 +1411,9 @@ static int update_remote(unsigned char *sha1, struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); + curl_setup_http(slot->curl, lock->url, DAV_PUT, + &out_buffer, fwrite_null); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1572,19 +1537,9 @@ static void update_remote_info_refs(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); + curl_setup_http(slot->curl, lock->url, DAV_PUT, + &buffer, fwrite_null); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1742,10 +1697,7 @@ static int delete_remote_branch(const char *pattern, int force) sprintf(url, "%s%s", repo->url, remote_ref->name); slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_DELETE); + curl_setup_http_get(slot->curl, url, DAV_DELETE); if (start_active_slot(slot)) { run_active_slot(slot); free(url); -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v3 1/4] t5541-http-push: add test for chunked 2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan @ 2011-05-03 16:32 ` Shawn Pearce 2011-05-03 17:18 ` Tay Ray Chuan 2011-05-04 4:07 ` Junio C Hamano 2 siblings, 1 reply; 20+ messages in thread From: Shawn Pearce @ 2011-05-03 16:32 UTC (permalink / raw) To: Tay Ray Chuan; +Cc: Git Mailing List, Junio C Hamano, Dan McGee On Tue, May 3, 2011 at 08:47, Tay Ray Chuan <rctay89@gmail.com> wrote: > Trigger the chunked type of pushing for smart HTTP. This can serve as a > regression test for the issue fixed in 1e41827 (http: clear POSTFIELDS > when initializing a slot). ... > + # to trigger chunked pushing, we need a sufficiently large pack - use > + # git v0.99 The better way to trigger this is to set http.postBuffer to something very small, e.g. "4". -- Shawn. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v3 1/4] t5541-http-push: add test for chunked 2011-05-03 16:32 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Shawn Pearce @ 2011-05-03 17:18 ` Tay Ray Chuan 0 siblings, 0 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-03 17:18 UTC (permalink / raw) To: Shawn Pearce; +Cc: Git Mailing List, Junio C Hamano, Dan McGee On Wed, May 4, 2011 at 12:32 AM, Shawn Pearce <spearce@spearce.org> wrote: > On Tue, May 3, 2011 at 08:47, Tay Ray Chuan <rctay89@gmail.com> wrote: >> Trigger the chunked type of pushing for smart HTTP. This can serve as a >> regression test for the issue fixed in 1e41827 (http: clear POSTFIELDS >> when initializing a slot). > ... >> + # to trigger chunked pushing, we need a sufficiently large pack - use >> + # git v0.99 > > The better way to trigger this is to set http.postBuffer to something > very small, e.g. "4". Ah, thanks for the tip. -- Cheers, Ray Chuan ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v3 1/4] t5541-http-push: add test for chunked 2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan 2011-05-03 16:32 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Shawn Pearce @ 2011-05-04 4:07 ` Junio C Hamano 2 siblings, 0 replies; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 4:07 UTC (permalink / raw) To: Tay Ray Chuan; +Cc: Git Mailing List, Shawn O. Pearce, Dan McGee Tay Ray Chuan <rctay89@gmail.com> writes: > + GIT_REPO=$TEST_DIRECTORY/../.git && > + test -d $GIT_REPO && Shouldn't a test be runnable without failing in a tarball extract? ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v3 0/4] http cleanups 2011-05-03 15:47 [PATCH v3 0/4] http cleanups Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan @ 2011-05-04 4:09 ` Junio C Hamano 2011-05-04 9:50 ` Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 " Tay Ray Chuan 2 siblings, 1 reply; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 4:09 UTC (permalink / raw) To: Tay Ray Chuan; +Cc: Git Mailing List, Shawn O. Pearce, Dan McGee Tay Ray Chuan <rctay89@gmail.com> writes: > This is a re-roll of dm/http-cleanup. I've also included a test for > chunked http pushes that has been sitting on my side for some time. > > Note to Junio: these patches should apply safely on > sp/maint-clear-postfields. Actually, the first patch depends on it. It doesn't apply for me. It does apply to 'master' after that topic is merged to it, which is good enough for our purpose, though. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v3 0/4] http cleanups 2011-05-04 4:09 ` [PATCH v3 0/4] http cleanups Junio C Hamano @ 2011-05-04 9:50 ` Tay Ray Chuan 0 siblings, 0 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 9:50 UTC (permalink / raw) To: Junio C Hamano; +Cc: Git Mailing List, Shawn O. Pearce, Dan McGee On Wed, May 4, 2011 at 12:09 PM, Junio C Hamano <gitster@pobox.com> wrote: > It doesn't apply for me. It does apply to 'master' after that topic is > merged to it, which is good enough for our purpose, though. Sorry about the inaccurate note. I think this is due to sp/main-clear-postfields not containing t/t5541-http-push.sh. -- Cheers, Ray Chuan ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 0/4] http cleanups 2011-05-03 15:47 [PATCH v3 0/4] http cleanups Tay Ray Chuan 2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan 2011-05-04 4:09 ` [PATCH v3 0/4] http cleanups Junio C Hamano @ 2011-05-04 10:11 ` Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Tay Ray Chuan [not found] ` <1304529590-1032-1-git-send-email-rctay89@gmail.com> 2 siblings, 2 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 10:11 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee Only patch #1 has been changed. [1/4] t5541-http-push: add test for chunked [2/4] http: make curl callbacks match contracts from curl header [3/4] http-push: use const for strings in signatures [4/4] http-push: refactor curl_easy_setup madness -- 1.7.3.3.585.g74f6e ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 1/4] t5541-http-push: add test for chunked 2011-05-04 10:11 ` [PATCH v4 " Tay Ray Chuan @ 2011-05-04 10:11 ` Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan 2011-05-04 11:51 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Jakub Narebski [not found] ` <1304529590-1032-1-git-send-email-rctay89@gmail.com> 1 sibling, 2 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 10:11 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee Trigger the chunked type of pushing for smart HTTP. This can serve as a regression test for the issue fixed in 1e41827 (http: clear POSTFIELDS when initializing a slot). Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- Changed in v4: simplified the way chunked pushing is triggered. The test now runs in a tarball extract. t/t5541-http-push.sh | 18 ++++++++++++++++-- 1 files changed, 16 insertions(+), 2 deletions(-) diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh index 0492877..0c0917a 100755 --- a/t/t5541-http-push.sh +++ b/t/t5541-http-push.sh @@ -65,14 +65,16 @@ test_expect_success 'clone remote repository' ' git clone $HTTPD_URL/smart/test_repo.git test_repo_clone ' -test_expect_success 'push to remote repository' ' +test_expect_success 'push to remote repository (standard)' ' cd "$ROOT_PATH"/test_repo_clone && : >path2 && git add path2 && test_tick && git commit -m path2 && HEAD=$(git rev-parse --verify HEAD) && - git push && + GIT_CURL_VERBOSE=1 git push -v -v 2>err && + ! grep "Expect: 100-continue" err && + grep "POST git-receive-pack (376 bytes)" err && (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && test $HEAD = $(git rev-parse --verify HEAD)) ' @@ -140,5 +142,17 @@ test_expect_success C_LOCALE_OUTPUT 'push fails for non-fast-forward refs unmatc output ' +test_expect_success 'push (chunked)' ' + git checkout master && + test_commit commit path3 && + HEAD=$(git rev-parse --verify HEAD) && + git config http.postbuffer 4 && + git push -v -v origin $BRANCH 2>err && + grep "POST git-receive-pack (chunked)" err && + (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && + test $HEAD = $(git rev-parse --verify HEAD)) && + git config --unset http.postbuffer +' + stop_httpd test_done -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v4 2/4] http: make curl callbacks match contracts from curl header 2011-05-04 10:11 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Tay Ray Chuan @ 2011-05-04 10:11 ` Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 3/4] http-push: use const for strings in signatures Tay Ray Chuan 2011-05-04 11:51 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Jakub Narebski 1 sibling, 1 reply; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 10:11 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> Yes, these don't match perfectly with the void* first parameter of the fread/fwrite in the standard library, but they do match the curl expected method signature. This is needed when a refactor passes a curl_write_callback around, which would otherwise give incorrect parameter warnings. Signed-off-by: Dan McGee <dpmcgee@gmail.com> --- http-walker.c | 4 ++-- http.c | 12 ++++++------ http.h | 6 +++--- remote-curl.c | 2 +- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/http-walker.c b/http-walker.c index 9bc8114..51a906e 100644 --- a/http-walker.c +++ b/http-walker.c @@ -185,7 +185,7 @@ static void process_alternates_response(void *callback_data) struct active_request_slot *slot = alt_req->slot; struct alt_base *tail = cdata->alt; const char *base = alt_req->base; - static const char null_byte = '\0'; + const char null_byte = '\0'; char *data; int i = 0; @@ -218,7 +218,7 @@ static void process_alternates_response(void *callback_data) } } - fwrite_buffer(&null_byte, 1, 1, alt_req->buffer); + fwrite_buffer((char *)&null_byte, 1, 1, alt_req->buffer); alt_req->buffer->len--; data = alt_req->buffer->buf; diff --git a/http.c b/http.c index b27bb57..b2ae8de 100644 --- a/http.c +++ b/http.c @@ -60,7 +60,7 @@ static struct curl_slist *no_pragma_header; static struct active_request_slot *active_queue_head; -size_t fread_buffer(void *ptr, size_t eltsize, size_t nmemb, void *buffer_) +size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; struct buffer *buffer = buffer_; @@ -92,7 +92,7 @@ curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp) } #endif -size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *buffer_) +size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; struct strbuf *buffer = buffer_; @@ -102,7 +102,7 @@ size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *buffer return size; } -size_t fwrite_null(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf) +size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf) { data_received++; return eltsize * nmemb; @@ -1167,7 +1167,7 @@ abort: } /* Helpers for fetching objects (loose) */ -static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb, +static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb, void *data) { unsigned char expn[4096]; @@ -1184,7 +1184,7 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb, } while (posn < size); freq->stream.avail_in = size; - freq->stream.next_in = ptr; + freq->stream.next_in = (void *)ptr; do { freq->stream.next_out = expn; freq->stream.avail_out = sizeof(expn); @@ -1203,7 +1203,7 @@ struct http_object_request *new_http_object_request(const char *base_url, char *filename; char prevfile[PATH_MAX]; int prevlocal; - unsigned char prev_buf[PREV_BUF_SIZE]; + char prev_buf[PREV_BUF_SIZE]; ssize_t prev_read = 0; long prev_posn = 0; char range[RANGE_HEADER_SIZE]; diff --git a/http.h b/http.h index e9ed3c2..19b7134 100644 --- a/http.h +++ b/http.h @@ -66,9 +66,9 @@ struct buffer { }; /* Curl request read/write callbacks */ -extern size_t fread_buffer(void *ptr, size_t eltsize, size_t nmemb, void *strbuf); -extern size_t fwrite_buffer(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf); -extern size_t fwrite_null(const void *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); +extern size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); #ifndef NO_CURL_IOCTL extern curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp); #endif diff --git a/remote-curl.c b/remote-curl.c index 775d614..17d8a9b 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -347,7 +347,7 @@ static curlioerr rpc_ioctl(CURL *handle, int cmd, void *clientp) } #endif -static size_t rpc_in(const void *ptr, size_t eltsize, +static size_t rpc_in(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { size_t size = eltsize * nmemb; -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v4 3/4] http-push: use const for strings in signatures 2011-05-04 10:11 ` [PATCH v4 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan @ 2011-05-04 10:11 ` Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 4/4] http-push: refactor curl_easy_setup madness Tay Ray Chuan 0 siblings, 1 reply; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 10:11 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- http-push.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/http-push.c b/http-push.c index d18346c..0fce38d 100644 --- a/http-push.c +++ b/http-push.c @@ -169,7 +169,7 @@ enum dav_header_flag { DAV_HEADER_TIMEOUT = (1u << 2) }; -static char *xml_entities(char *s) +static char *xml_entities(const char *s) { struct strbuf buf = STRBUF_INIT; while (*s) { @@ -797,7 +797,7 @@ static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed) } } -static void one_remote_ref(char *refname); +static void one_remote_ref(const char *refname); static void xml_start_tag(void *userData, const char *name, const char **atts) @@ -1471,7 +1471,7 @@ static int update_remote(unsigned char *sha1, struct remote_lock *lock) static struct ref *remote_refs; -static void one_remote_ref(char *refname) +static void one_remote_ref(const char *refname) { struct ref *ref; struct object *obj; @@ -1660,7 +1660,7 @@ static int verify_merge_base(unsigned char *head_sha1, unsigned char *branch_sha return (merge_bases && !merge_bases->next && merge_bases->item == branch); } -static int delete_remote_branch(char *pattern, int force) +static int delete_remote_branch(const char *pattern, int force) { struct ref *refs = remote_refs; struct ref *remote_ref = NULL; -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH v4 4/4] http-push: refactor curl_easy_setup madness 2011-05-04 10:11 ` [PATCH v4 3/4] http-push: use const for strings in signatures Tay Ray Chuan @ 2011-05-04 10:11 ` Tay Ray Chuan 0 siblings, 0 replies; 20+ messages in thread From: Tay Ray Chuan @ 2011-05-04 10:11 UTC (permalink / raw) To: Git Mailing List; +Cc: Junio C Hamano, Shawn O. Pearce, Dan McGee From: Dan McGee <dpmcgee@gmail.com> We were doing (nearly) the same thing all over the place, in slightly different orders, different variable names, etc. Refactor most calls into two helper functions, one for GET and one for everything else, that do the heavy lifting leaving most callsites a lot cleaner in the process. Note that the setting of CURLOPT_PUT at the callsites of curl_setup_http() which previously didn't do it (eg. locking_available(), remote_ls()) is safe, since that option is deprecated in libcurl in place of, and has the same effect as, CURLOPT_UPLOAD. Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> --- http-push.c | 144 ++++++++++++++++++++--------------------------------------- 1 files changed, 48 insertions(+), 96 deletions(-) diff --git a/http-push.c b/http-push.c index 0fce38d..28bfe76 100644 --- a/http-push.c +++ b/http-push.c @@ -197,6 +197,34 @@ static char *xml_entities(const char *s) return strbuf_detach(&buf, NULL); } +static void curl_setup_http_get(CURL *curl, const char *url, + const char *custom_req) +{ + curl_easy_setopt(curl, CURLOPT_HTTPGET, 1); + curl_easy_setopt(curl, CURLOPT_URL, url); + curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, custom_req); + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, fwrite_null); +} + +static void curl_setup_http(CURL *curl, const char *url, + const char *custom_req, struct buffer *buffer, + curl_write_callback write_fn) +{ + curl_easy_setopt(curl, CURLOPT_PUT, 1); + curl_easy_setopt(curl, CURLOPT_URL, url); + curl_easy_setopt(curl, CURLOPT_INFILE, buffer); + curl_easy_setopt(curl, CURLOPT_INFILESIZE, buffer->buf.len); + curl_easy_setopt(curl, CURLOPT_READFUNCTION, fread_buffer); +#ifndef NO_CURL_IOCTL + curl_easy_setopt(curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); + curl_easy_setopt(curl, CURLOPT_IOCTLDATA, &buffer); +#endif + curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_fn); + curl_easy_setopt(curl, CURLOPT_NOBODY, 0); + curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, custom_req); + curl_easy_setopt(curl, CURLOPT_UPLOAD, 1); +} + static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options) { struct strbuf buf = STRBUF_INIT; @@ -272,11 +300,8 @@ static void start_mkcol(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); /* undo PUT setup */ - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); + curl_setup_http_get(slot->curl, request->url, DAV_MKCOL); curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, request->errorstr); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MKCOL); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); if (start_active_slot(slot)) { request->slot = slot; @@ -395,19 +420,8 @@ static void start_put(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &request->buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, request->buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &request->buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); + curl_setup_http(slot->curl, request->url, DAV_PUT, + &request->buffer, fwrite_null); if (start_active_slot(slot)) { request->slot = slot; @@ -427,13 +441,10 @@ static void start_move(struct transfer_request *request) slot = get_active_slot(); slot->callback_func = process_response; slot->callback_data = request; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); /* undo PUT setup */ - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MOVE); + curl_setup_http_get(slot->curl, request->url, DAV_MOVE); dav_headers = curl_slist_append(dav_headers, request->dest); dav_headers = curl_slist_append(dav_headers, "Overwrite: T"); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, request->url); if (start_active_slot(slot)) { request->slot = slot; @@ -458,10 +469,7 @@ static int refresh_lock(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK); + curl_setup_http_get(slot->curl, lock->url, DAV_LOCK); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { @@ -876,10 +884,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout) ep[1] = '\0'; slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_MKCOL); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); + curl_setup_http_get(slot->curl, url, DAV_MKCOL); if (start_active_slot(slot)) { run_active_slot(slot); if (results.curl_result != CURLE_OK && @@ -909,19 +914,9 @@ static struct remote_lock *lock_remote(const char *path, long timeout) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK); + curl_setup_http(slot->curl, url, DAV_LOCK, &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); lock = xcalloc(1, sizeof(*lock)); lock->timeout = -1; @@ -987,9 +982,7 @@ static int unlock_remote(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_UNLOCK); + curl_setup_http_get(slot->curl, lock->url, DAV_UNLOCK); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { @@ -1167,19 +1160,10 @@ static void remote_ls(const char *path, int flags, slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND); + curl_setup_http(slot->curl, url, DAV_PROPFIND, + &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1250,19 +1234,10 @@ static int locking_available(void) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_URL, repo->url); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND); + curl_setup_http(slot->curl, repo->url, DAV_PROPFIND, + &out_buffer, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1436,19 +1411,9 @@ static int update_remote(unsigned char *sha1, struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &out_buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); + curl_setup_http(slot->curl, lock->url, DAV_PUT, + &out_buffer, fwrite_null); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1572,19 +1537,9 @@ static void update_remote_info_refs(struct remote_lock *lock) slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_INFILE, &buffer); - curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, buffer.buf.len); - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer); -#ifndef NO_CURL_IOCTL - curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer); - curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, &buffer); -#endif - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PUT); + curl_setup_http(slot->curl, lock->url, DAV_PUT, + &buffer, fwrite_null); curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); - curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1); - curl_easy_setopt(slot->curl, CURLOPT_PUT, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1742,10 +1697,7 @@ static int delete_remote_branch(const char *pattern, int force) sprintf(url, "%s%s", repo->url, remote_ref->name); slot = get_active_slot(); slot->results = &results; - curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_DELETE); + curl_setup_http_get(slot->curl, url, DAV_DELETE); if (start_active_slot(slot)) { run_active_slot(slot); free(url); -- 1.7.3.3.585.g74f6e ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v4 1/4] t5541-http-push: add test for chunked 2011-05-04 10:11 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Tay Ray Chuan 2011-05-04 10:11 ` [PATCH v4 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan @ 2011-05-04 11:51 ` Jakub Narebski 2011-05-04 18:23 ` Junio C Hamano 1 sibling, 1 reply; 20+ messages in thread From: Jakub Narebski @ 2011-05-04 11:51 UTC (permalink / raw) To: Tay Ray Chuan Cc: Git Mailing List, Junio C Hamano, Shawn O. Pearce, Dan McGee Tay Ray Chuan <rctay89@gmail.com> writes: > +test_expect_success 'push (chunked)' ' > + git checkout master && > + test_commit commit path3 && > + HEAD=$(git rev-parse --verify HEAD) && > + git config http.postbuffer 4 && > + git push -v -v origin $BRANCH 2>err && > + grep "POST git-receive-pack (chunked)" err && > + (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && > + test $HEAD = $(git rev-parse --verify HEAD)) && > + git config --unset http.postbuffer > +' It is a very minor issue, a nitpick really; but wouldn't it be better to use + git config http.postbuffer 4 && + test_when_finished "git config --unset http.postbuffer" && This way you would have `http.postBuffer` unset even if any of intermediate commands fail, preparing clean state for next test. Though in this case that doesn't matter, as it is the very last test. -- Jakub Narebski Poland ShadeHawk on #git ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v4 1/4] t5541-http-push: add test for chunked 2011-05-04 11:51 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Jakub Narebski @ 2011-05-04 18:23 ` Junio C Hamano 0 siblings, 0 replies; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 18:23 UTC (permalink / raw) To: Jakub Narebski Cc: Tay Ray Chuan, Git Mailing List, Junio C Hamano, Shawn O. Pearce, Dan McGee Jakub Narebski <jnareb@gmail.com> writes: > It is a very minor issue, a nitpick really; but wouldn't it be better > to use > > + git config http.postbuffer 4 && > + test_when_finished "git config --unset http.postbuffer" && Not a minor issue it is. It needs to be fixed before queued. Thanks for pointing it out. It is a different matter if it needs to be rerolled. Something trivial like this I could remember to amend/tweak before queuing. Thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <1304529590-1032-1-git-send-email-rctay89@gmail.com>]
* Re: [PATCH v5 1/4] t5541-http-push: add test for chunked [not found] ` <1304529590-1032-1-git-send-email-rctay89@gmail.com> @ 2011-05-04 18:23 ` Junio C Hamano 2011-05-04 18:44 ` Junio C Hamano 1 sibling, 0 replies; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 18:23 UTC (permalink / raw) To: Tay Ray Chuan Cc: Git Mailing List, Shawn O. Pearce, Dan McGee, Jakub, Narebski, jnareb Tay Ray Chuan <rctay89@gmail.com> writes: > Trigger the chunked type of pushing for smart HTTP. This can serve as a > regression test for the issue fixed in 1e41827 (http: clear POSTFIELDS > when initializing a slot). > > Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> > --- > > Junio: only the first patch (this one) was changed; all the other three > remain as they are. > > Changed in v5: use test_when_finished to unset config changes. Thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v5 1/4] t5541-http-push: add test for chunked [not found] ` <1304529590-1032-1-git-send-email-rctay89@gmail.com> 2011-05-04 18:23 ` [PATCH v5 " Junio C Hamano @ 2011-05-04 18:44 ` Junio C Hamano 2011-05-04 20:28 ` Junio C Hamano 1 sibling, 1 reply; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 18:44 UTC (permalink / raw) To: Tay Ray Chuan; +Cc: Git Mailing List, Shawn O. Pearce, Dan McGee Tay Ray Chuan <rctay89@gmail.com> writes: > + GIT_CURL_VERBOSE=1 git push -v -v 2>err && > + ! grep "Expect: 100-continue" err && > + grep "POST git-receive-pack (376 bytes)" err && This "376 bytes" bothers me. Can our packing algorithm never improve? > (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && > test $HEAD = $(git rev-parse --verify HEAD)) > ' > @@ -140,5 +142,17 @@ test_expect_success C_LOCALE_OUTPUT 'push fails for non-fast-forward refs unmatc > output > ' > > +test_expect_success 'push (chunked)' ' > + git checkout master && > + test_commit commit path3 && > + HEAD=$(git rev-parse --verify HEAD) && > + git config http.postbuffer 4 && > + test_when_finished git config --unset http.postbuffer && It probably is a good style to send the command line as a single argument. There are a few violations in the existing code but that is not an excuse to add more violations. > + git push -v -v origin $BRANCH 2>err && > + grep "POST git-receive-pack (chunked)" err && > + (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && > + test $HEAD = $(git rev-parse --verify HEAD)) > +' > + > stop_httpd > test_done I'll queue this with the following trivial fix-up. By the way, the "(chunked) test" is failing for me, with Pushing to http://127.0.0.1:5541/smart/test_repo.git POST git-receive-pack (chunked) POST git-receive-pack (chunked) POST git-receive-pack (chunked) fatal: The remote end hung up unexpectedly fatal: The remote end hung up unexpectedly in trash/test_repo_clone/err; sometimes with two POSTs and sometimes with three POSTs before we get the "fatal:". --- t/t5541-http-push.sh | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh index fec552c..4b93fa1 100755 --- a/t/t5541-http-push.sh +++ b/t/t5541-http-push.sh @@ -74,7 +74,7 @@ test_expect_success 'push to remote repository (standard)' ' HEAD=$(git rev-parse --verify HEAD) && GIT_CURL_VERBOSE=1 git push -v -v 2>err && ! grep "Expect: 100-continue" err && - grep "POST git-receive-pack (376 bytes)" err && + grep "POST git-receive-pack ([0-9]* bytes)" err && (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && test $HEAD = $(git rev-parse --verify HEAD)) ' @@ -147,7 +147,7 @@ test_expect_success 'push (chunked)' ' test_commit commit path3 && HEAD=$(git rev-parse --verify HEAD) && git config http.postbuffer 4 && - test_when_finished git config --unset http.postbuffer && + test_when_finished "git config --unset http.postbuffer" && git push -v -v origin $BRANCH 2>err && grep "POST git-receive-pack (chunked)" err && (cd "$HTTPD_DOCUMENT_ROOT_PATH"/test_repo.git && ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH v5 1/4] t5541-http-push: add test for chunked 2011-05-04 18:44 ` Junio C Hamano @ 2011-05-04 20:28 ` Junio C Hamano 0 siblings, 0 replies; 20+ messages in thread From: Junio C Hamano @ 2011-05-04 20:28 UTC (permalink / raw) To: Junio C Hamano Cc: Tay Ray Chuan, Git Mailing List, Shawn O. Pearce, Dan McGee Junio C Hamano <gitster@pobox.com> writes: > By the way, the "(chunked) test" is failing for me, with > > Pushing to http://127.0.0.1:5541/smart/test_repo.git > POST git-receive-pack (chunked) > POST git-receive-pack (chunked) > POST git-receive-pack (chunked) > fatal: The remote end hung up unexpectedly > fatal: The remote end hung up unexpectedly > > in trash/test_repo_clone/err; sometimes with two POSTs and sometimes with > three POSTs before we get the "fatal:". Sorry. False alarm. ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2011-05-04 20:28 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-05-03 15:47 [PATCH v3 0/4] http cleanups Tay Ray Chuan
2011-05-03 15:47 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Tay Ray Chuan
2011-05-03 15:47 ` [PATCH v3 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan
2011-05-03 15:47 ` [PATCH v3 3/4] http-push: use const for strings in signatures Tay Ray Chuan
2011-05-03 15:47 ` [PATCH v3 4/4] http-push: refactor curl_easy_setup madness Tay Ray Chuan
2011-05-03 16:32 ` [PATCH v3 1/4] t5541-http-push: add test for chunked Shawn Pearce
2011-05-03 17:18 ` Tay Ray Chuan
2011-05-04 4:07 ` Junio C Hamano
2011-05-04 4:09 ` [PATCH v3 0/4] http cleanups Junio C Hamano
2011-05-04 9:50 ` Tay Ray Chuan
2011-05-04 10:11 ` [PATCH v4 " Tay Ray Chuan
2011-05-04 10:11 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Tay Ray Chuan
2011-05-04 10:11 ` [PATCH v4 2/4] http: make curl callbacks match contracts from curl header Tay Ray Chuan
2011-05-04 10:11 ` [PATCH v4 3/4] http-push: use const for strings in signatures Tay Ray Chuan
2011-05-04 10:11 ` [PATCH v4 4/4] http-push: refactor curl_easy_setup madness Tay Ray Chuan
2011-05-04 11:51 ` [PATCH v4 1/4] t5541-http-push: add test for chunked Jakub Narebski
2011-05-04 18:23 ` Junio C Hamano
[not found] ` <1304529590-1032-1-git-send-email-rctay89@gmail.com>
2011-05-04 18:23 ` [PATCH v5 " Junio C Hamano
2011-05-04 18:44 ` Junio C Hamano
2011-05-04 20:28 ` Junio C Hamano
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).